GeoLeaf PoC — Experiments

git clone <this-repo>
cd experiments
make setup

This folder is a self-contained scaffold for the GeoLeaf proof-of-concept: vegetation classification (Woodland / Shrubland / Grassland) over Hong Kong, benchmarked against the AFCD THMHK 2021 habitat map. The current Run 3 baseline is territory-wide; pilot-zone field validation is planned for Tai Mo Shan and Sai Kung West.

It is designed to be developed locally on a laptop and then executed on a cloud GPU (vast.ai, RunPod, or a PolyU cluster node). Nothing here is trained or downloaded in the local sandbox.

---

1. Project overview

GeoLeaf classifies natural land into three height-defined vegetation classes that the Planning Department needs but currently cannot get at scale:

Three tiers, increasing in capability:

• Tier 1 — CHM threshold baseline. Lang-2023 global 10 m canopy height map + Sentinel-2 NDVI mask → W/S/G via fixed thresholds. ~5 min runtime. Establishes the lower bound any learned model must beat.
• Tier 2 — Weak-supervision ensemble. Majority vote across Lang 2023, ESA WorldCover, and Dynamic World. Run 3 shows this barely improves over the threshold baseline, which is useful negative evidence.
• Tier 3 — Prithvi-EO-2.0 LoRA fine-tune. PEFT-LoRA (rank 16) on attention layers of Prithvi-EO-2.0-600M, currently using the six Prithvi-compatible Sentinel-2 bands. Sentinel-1 and CHM channels are preprocessed and passed through the dataset interface, but real fusion is a planned follow-up. Run 3 measured active-class mIoU = 0.510, macro F1 = 0.666, and overall accuracy = 0.728 on held-out spatial blocks.

Reference: see GeoLeaf_PoC_Proposal_v15.pdf and GeoLeaf_Business_Plan_v15.pdf in the parent directory.

Run 3 metrics and the submission figure are tracked under results/run3/.

---

2. Setup

# clone + cd
git clone <your-repo-url>
cd experiments

# python venv
python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip wheel
pip install -r requirements.txt
# or:
make setup

Authentication (one-time per machine):

# Earth Engine (Lang CHM, GEDI fallback)
earthengine authenticate

# NASA Earthdata (GEDI L2A direct download)
#   Create an account at https://urs.earthdata.nasa.gov
#   Then put credentials in ~/.netrc :
echo "machine urs.earthdata.nasa.gov login <user> password <pass>" >> ~/.netrc
chmod 600 ~/.netrc

# HuggingFace (Prithvi-EO-2.0 backbone weights)
huggingface-cli login

Optional:

# Weights & Biases (auto-detected if WANDB_API_KEY is set; otherwise local TB)
export WANDB_API_KEY=xxxxxxxx

---

3. Data sources

The HK government layers do not currently expose a stable programmatic download URL — scripts/download_data.py prints a clear MANUAL DOWNLOAD banner with the source URL and target folder for those.

Run everything (skips already-downloaded sources):

make download

---

4. Tier 1 quick start

Three commands to get an mIoU from scratch:

make download      # ~20-60 min one-time data fetch
make preprocess    # ~5-15 min: co-register, tile, spatial-block split
make tier1         # ~5 min: CHM thresholds -> metrics + confusion matrix

Expected runtime:

• CPU laptop: download dominated; tier1 itself ~5 min for a 50 km² AOI.
• A100: same, since Tier 1 is not GPU-bound.

Outputs:

• outputs/tier1/metrics.json
• outputs/tier1/confusion.npy
• outputs/tier1/pred.tif

---

5. Tier 3 LoRA workflow

make preprocess                                # if not already done
make tier3                                     # default = A100 config
make tier3 GPU_CONFIG=config/t4.yaml           # fallback for T4 16GB

Expected runtime:

• RTX A5000 / 24 GB, batch 4 + grad-accum 4, Prithvi-600M, 40 epochs: ~1 hour observed for Run 3 after data were prepared.
• A100 40 GB, batch 16, Prithvi-600M, 40 epochs: expected faster than the A5000 path if memory allows the larger batch.
• T4 16 GB, batch 4 + grad-accum 4, 20 epochs: ~24–36 hours.

T4 notes:

• config/t4.yaml forces fp16, batch 4, accum 4, and the smaller Prithvi-300M override.
• Boundary-loss term is disabled by default on T4 to save memory.
• If OOM, drop patch.size from 256 to 192 in config/base.yaml.

Outputs:

• outputs/tier3/checkpoints/{epoch_N.pt, best.pt}
• outputs/tier3/metrics.json
• outputs/tier3/confusion.npy
• TensorBoard logs in outputs/logs/tb/ (or wandb)

Resume from a checkpoint:

python scripts/tier3_lora_finetune.py \
  --resume outputs/tier3/checkpoints/epoch_05.pt

Smoke-test the full graph (one batch only):

python scripts/tier3_lora_finetune.py --dry-run

---

6. Cloud GPU deployment

Concrete cloud-GPU workflow (vast.ai shown — RunPod and PolyU clusters are analogous):

6.1 Provision

1. On vast.ai, filter for RTX A5000 24GB or A100 40GB, image pytorch/pytorch:2.4.0-cuda12.1-cudnn9-runtime, >=60 GB disk, on-demand.
2. SSH into the instance:

   ssh -p <port> root@<host>
   

6.2 Bring Claude Code along (optional but recommended)

curl -fsSL https://claude.ai/install.sh | sh   # or your preferred install
claude login

6.3 Transfer this folder

Pick one:

# Option A: git
git clone <your-repo-url> /workspace/geoleaf && cd /workspace/geoleaf/experiments

# Option B: scp from your laptop
scp -P <port> -r ./experiments root@<host>:/workspace/geoleaf/

6.4 Setup and run

cd /workspace/geoleaf/experiments
make setup                # installs requirements.txt
source .venv/bin/activate

earthengine authenticate  # paste the token
huggingface-cli login

make download             # one-time
make preprocess
make tier1                # smoke-check baseline
make tier3 GPU_CONFIG=config/rtx24g.yaml  # 24GB GPU path used by Run 3
make figures              # regenerate Figure 2 from real metrics

6.5 Retrieve results

# from your laptop:
scp -P <port> -r root@<host>:/workspace/geoleaf/experiments/outputs ./outputs_a100/

---

7. Reproducibility

• Master seed lives in config/base.yaml::seed (default 42). Every script calls utils.set_seed() which seeds Python, NumPy, and PyTorch (incl. CUDA), and forces cudnn.deterministic = True.
• All scripts log the full resolved config at start.
• Dependencies are pinned by major version range in requirements.txt; for exact reproducibility on the cloud GPU, pip freeze > pip-lock.txt and commit it after first successful run.
• Spatial-block CV split is hash-based on the patch's 1 km tile coordinate, so the split is reproducible from the same seed without storing it.

---

8. What to do with results

After running Tier 1 and Tier 3:

make figures

Reads outputs/tier{1,2,3}/metrics.json and writes:

outputs/figures/figure_2_performance_real.png

Drop this PNG into the proposal in place of figure_2_performance_projection.png, update the caption to say "actual measured values on the held-out territory-wide Hong Kong spatial-block test fold", and update any in-text numbers from the metrics JSON directly:

python -c "import json; print(json.load(open('outputs/tier3/metrics.json'))['miou'])"

---

9. Folder layout

experiments/
├── README.md
├── requirements.txt
├── Makefile
├── .gitignore
├── config/
│   ├── base.yaml         # paths, AOI, classes, model, training
│   ├── a100.yaml         # A100 batch/precision overrides
│   ├── rtx24g.yaml       # RTX A5000 / 3090 / 4090 24GB overrides
│   └── t4.yaml           # T4 16GB overrides
├── data/                 # gitignored — populated by `make download`
├── outputs/              # gitignored — populated by experiments
├── notebooks/
│   ├── 01_data_exploration.ipynb
│   ├── 02_tier1_baseline.ipynb
│   └── 03_tier3_lora.ipynb
└── scripts/
    ├── download_data.py
    ├── preprocess.py
    ├── tier1_baseline.py
    ├── tier2_ensemble.py
    ├── tier3_lora_finetune.py
    ├── eval_metrics.py
    ├── make_figures.py
    └── utils.py