Cryo-Bench 🧊

A Benchmark for Evaluating Geospatial Foundation Models on Cryosphere Applications

Cryo-Bench is a community benchmark that evaluates geospatial foundation models (GFMs) on six cryosphere segmentation datasets spanning five components: supraglacial debris, glacial lakes (two sensing configurations), sea ice, calving fronts, and Antarctic ice-shelf extent. It is built on top of the PANGAEA evaluation protocol using multi-sensor satellite imagery from Sentinel-1/2, Landsat-8, WorldView-2, and historical SAR missions.

---

🧰 Code

The full benchmarking code is available in the Cryo-Bench Github/ directory, which contains dataset configs, encoder definitions, and decoder heads built on top of PANGAEA.

📋 Tasks & Datasets

Cryo-Bench includes six benchmark tasks covering five components of the cryosphere:

GLB's own paper is still in review; a citation will be added once it's public.

---

The dataset contains the exact training, validation, and test splits used in Cryo-Bench, covering all six datasets above.

📥 Download Data

1. Install dependency

pip install huggingface_hub

2. Download all datasets

python download_data.py

3. Download specific datasets only

python download_data.py --datasets GLID GSDD SICD GLB Shelf-Bench

---

🏆 Benchmark Results

Table below reports mIoU (↑) for all models evaluated with frozen encoders and 100% training data using the UPerNet decoder, across all six datasets. Rows are ordered by six-dataset average mIoU. Baseline models (U-Net, ViT) are trained from scratch. Every value below is parsed directly from run logs (tables_v2.json). Avg Rank is the mean of a model's rank (1-15) on each of the six datasets individually, and can read differently from the ordering by Avg mIoU (e.g. DOFA has a better Avg Rank than RemoteCLIP despite a lower Avg mIoU).

Bold = best performance · Italic = second best

Encoders are kept frozen for all GFMs; U-Net and ViT are trained from scratch.

---

🧠 Pretrained Checkpoints

Frozen-encoder checkpoints (UPerNet decoder trained, backbone frozen) are included for all 15 models across all 6 datasets — 90 checkpoints in total, corresponding exactly to the cells in the results table above. Each is a PyTorch state_dict (checkpoint__best.pth) saved at the best-validation-mIoU epoch under the standard PANGAEA training protocol.

checkpoints/
├── <DATASET>/
│   └── frozen/
│       └── <encoder>/
│           └── checkpoint__best.pth

where <DATASET> is one of GSDD, GLID, GLB, SICD, CaFFe, Shelf-Bench and <encoder> is the PANGAEA encoder config name (e.g. croma, dofa, terramind_large, unet_encoder, vit_scratch, ...).

Download a single checkpoint

from huggingface_hub import hf_hub_download

ckpt = hf_hub_download(
    repo_id="Sk-21/Cryo-Bench",
    filename="checkpoints/GLID/frozen/dofa/checkpoint__best.pth",
)

Download all checkpoints

from huggingface_hub import snapshot_download

snapshot_download(repo_id="Sk-21/Cryo-Bench", allow_patterns="checkpoints/*")

U-Net and ViT are trained from scratch (no pretrained backbone), so their "frozen" checkpoint is the same end-to-end trained model reported elsewhere as their baseline result.

📜 License

This project is licensed under the MIT License.

---

🙏 Acknowledgements

Cryo-Bench builds on the PANGAEA benchmark. We thank the developers of DOFA, TerraMind, Prithvi, SatlasNet, and all other foundation models included in this benchmark. We also thank the dataset authors of GSDD, GLID, GLB, SICD, CaFFe, and Shelf-Bench for making their data publicly available.