Hyper Parameter Optimization, Neural Architecture Search and Foundation Model Benchmarking using TerraTorch for (geospatial) computer vision
TerraTorch is a fine-tuning and benchmarking toolkit for Geospatial Foundation Models built on PyTorch Lightning and tailored for satellite, weather, and climate data. It integrates domain-specific data modules, pre-defined tasks, and a modular model factory that pairs any backbone with diverse decoder heads. These components allow researchers and practitioners to fine-tune supported models in a no-code fashion by simply editing a training or inference configuration. By consolidating best practices for model development and incorporating the automated hyperparameter optimization extension Iterate, TerraTorch reduces the expertise and time required to fine-tune or benchmark models on new Earth Observation use cases. Furthermore, TerraTorch directly integrates with GEO-Bench, allowing for systematic and reproducible benchmarking of Geospatial Foundation Models. TerraTorch Iterate is not only driving GeoBench, but also Hyperparameter Optimizations (HPO) and Neural Architecture Search (NAS). TerraTorch is open sourced under Apache 2.0, and can be installed via pip install terratorch from our Github repostitory.
Highlights
- Growing list of IBM and community Out-of-the-box implementations of foundation models (e.g.Prithvi, Satlas, Clay, timm models) and a large selection of decoders
- Model fine-tuning fully accessible through config files – no need to write code for segmentation, pixel-wise regression, object detection or classification tasks
- The functionalities of TerraTorch are a superset of Lightning and TorchGeo. All the goodies from these libraries come for free
Agenda
- Welcome & introductions
- Short recap on cuda, torch and lightning
- What are (geospatial) computer vision foundation models
- Finetuning and Inference
- Hyperparameter Optimization with terratorch iterate using optuna
- Benchmarking foundation models
- Q&A
- Closing
About the speaker
Romeo is an AI Research Engineer at IBM Research Europe in Zurich interested in understanding neural representations in artificial and biological neural networks.
About the AI Alliance
The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.
