GeoReset Wiki Land-Cover

GeoReset Wiki Land-Cover experiments with whether geolocated Wikipedia text can help an LLM infer land-cover labels, evaluated against CORINE level-2 classes and project-scoped OSM land-cover tags.

• Project site: https://geo-reset.sylvainlobry.com/
• Code repository: https://github.com/NoeFlandre/georeset-wiki-landcover
• Data bucket: https://huggingface.co/buckets/NoeFlandre/georeset-wiki-landcover
• Slide deck: https://docs.google.com/presentation/d/19K7qVy4NUPheNVJTwGpItdQIpGlxbYzWq5XjyDynW5U/edit?usp=sharing

Code And Data Split

GitHub stores source code, tests, Docker configuration, and documentation. Generated and downloaded project data lives in the Hugging Face bucket. Keep data/ out of Git.

Download or refresh data from Hugging Face:

hf sync hf://buckets/NoeFlandre/georeset-wiki-landcover ./data

Upload local data changes back to Hugging Face:

hf sync ./data hf://buckets/NoeFlandre/georeset-wiki-landcover --delete --exclude '**/.DS_Store' --exclude '.DS_Store'

Before committing code, check that no data files are staged:

git status --short
git ls-files data build

git ls-files data build should print nothing. If data was staged by mistake:

git rm -r --cached data build
git add .gitignore README.md docs src scripts tests pyproject.toml uv.lock LICENSE Dockerfile .dockerignore .github

Repository Layout

• src/georeset_wiki_landcover/: installable Python package. The wheel packages only this tree.
• src/georeset_wiki_landcover/fetchers/data_fetcher.py: loads CORINE shapefiles and exposes bounds, class labels, centroids, and samples.
• src/georeset_wiki_landcover/fetchers/wiki_fetcher.py: fetches French Wikipedia geosearch metadata inside the CORINE bounds and project polygon filters.
• src/georeset_wiki_landcover/fetchers/wiki_content_fetcher.py: fetches full Wikipedia extracts from page IDs. It sanitizes existing output, skips already-fetched entries, writes checkpoints after each batch, and can be stopped/resumed at any time.
• src/georeset_wiki_landcover/fetchers/osm_fetcher.py: fetches project-relevant OSM land-cover polygons from Overpass.
• src/georeset_wiki_landcover/analysis/corine_polygon_stats.py: computes CORINE class area/share distributions inside OSM polygons.
• src/georeset_wiki_landcover/analysis/distribution_summary.py: summarizes distribution outputs.
• src/georeset_wiki_landcover/visualization/map_visualizer.py: writes Folium map visualizations.
• src/georeset_wiki_landcover/cli/: packaged CLI implementations exposed through [project.scripts] entry points such as georeset-wiki-landcover-classify-articles, georeset-wiki-landcover-summarize-articles, georeset-wiki-landcover-summarize-landuse-evidence, and georeset-wiki-landcover-run-corine-analysis.
• scripts/: thin repository-compatible wrappers around georeset_wiki_landcover.cli.* plus Grid5000 shell launchers. Prefer the georeset-wiki-landcover-* entry points in new docs and automation.
• src/georeset_wiki_landcover/classification/: label utilities, ground-truth builders, LLM classifier, and metrics for CORINE level-2 and OSM tag classification.
• src/georeset_wiki_landcover/spatial/corine_confidence.py: CORINE buffer-purity diagnostics in EPSG:2154 for spatial-confidence experiments.
• scripts/cluster/submit_summarization.sh: syncs the minimal repository state to Grid5000/Nancy and submits summarization OAR jobs. Auto-sync is disabled by default; use one-shot syncs to avoid repeated SSH polling.
• scripts/cluster/submit_landuse_evidence_summarization.sh: submits land-use evidence summary OAR jobs with no-place policy and configurable seed/temperature.

Data Artifacts

These files are expected under data/ after syncing the bucket:

• data/corine/: CORINE shapefile and bounds.
• data/wiki/wiki_articles.json: Wikipedia geosearch metadata.
• data/wiki/article_contents.json: resumable Wikipedia article content.
• data/osm/osm_project_polygons.geojson: project-relevant OSM polygons.
• data/distribution/osm_corine_distribution.csv: CORINE class area/share distribution inside OSM polygons.
• data/maps/: generated HTML visualizations.
• data/classification/: resumable working classification outputs, organized by run under data/classification/runs/.
• data/experiments/: frozen experiment folders and derived analysis tables.

The source CORINE data was downloaded from: https://www.datagrandest.fr/geonetwork/srv/api/records/c0ccbf45-2620-4bde-93f8-869558e51d7e?language=fre

Local Setup

Install dependencies with uv:

uv sync --frozen --group dev
hf sync hf://buckets/NoeFlandre/georeset-wiki-landcover ./data

The default development and CI environment uses the dev group only. LLM/GPU workflows additionally need the optional llm group:

uv sync --group dev --group llm

Run tests:

PYTHONDONTWRITEBYTECODE=1 uv run pytest

Run the synthetic reproducibility smoke path without bucket data or an LLM:

PYTHONDONTWRITEBYTECODE=1 uv run python scripts/reproduce_small.py \
  --output-dir build/reproducibility/small \
  --clean
PYTHONDONTWRITEBYTECODE=1 uv run python scripts/validate_artifacts.py \
  --root build/reproducibility/small \
  --profile small

See docs/reproducibility.md, docs/architecture.md, docs/data_flow.md, docs/cli.md, docs/artifacts.md, docs/configuration.md, and docs/troubleshooting.md for clean-clone reproduction, module boundaries, data flow, command surfaces, artifact validation, environment variables, and common failure modes.

Run the full local quality gate:

uv lock --check
PYTHONDONTWRITEBYTECODE=1 uv run python scripts/dev/check_repository_hygiene.py
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles --help
PYTHONDONTWRITEBYTECODE=1 uv run python scripts/reproduce_small.py \
  --output-dir build/reproducibility/small \
  --clean
PYTHONDONTWRITEBYTECODE=1 uv run python scripts/validate_artifacts.py \
  --root build/reproducibility/small \
  --profile small
PYTHONDONTWRITEBYTECODE=1 uv run ruff check .
PYTHONDONTWRITEBYTECODE=1 uv run ruff format --check .
PYTHONDONTWRITEBYTECODE=1 uv run mypy src scripts
PYTHONDONTWRITEBYTECODE=1 uv run pytest -q

Run only the resumable Wikipedia content fetcher tests:

PYTHONDONTWRITEBYTECODE=1 uv run pytest tests/fetchers/test_wiki_content_fetcher.py -q

Pipeline Commands

Installable commands are exposed as georeset-wiki-landcover-* entry points. The top-level scripts/ modules are kept as thin repository wrappers for backwards compatibility, but they are not part of the installed wheel.

Print a quick dataset snapshot:

PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-snapshot

Fetch Wikipedia article metadata:

PYTHONDONTWRITEBYTECODE=1 uv run python -m georeset_wiki_landcover.fetchers.wiki_fetcher

Fetch full Wikipedia article content. This command is resumable: stop it with Ctrl-C, then run it again and it will skip sane entries already saved in data/wiki/article_contents.json.

PYTHONDONTWRITEBYTECODE=1 uv run python -m georeset_wiki_landcover.fetchers.wiki_content_fetcher
hf sync ./data hf://buckets/NoeFlandre/georeset-wiki-landcover --delete --exclude '**/.DS_Store' --exclude '.DS_Store'

Regenerate the CORINE + Wikipedia article map:

PYTHONDONTWRITEBYTECODE=1 uv run python -m georeset_wiki_landcover.visualization.map_visualizer

Fetch/use OSM polygons, compute CORINE distributions, and generate the separate CORINE + OSM map:

PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-run-corine-analysis
hf sync ./data hf://buckets/NoeFlandre/georeset-wiki-landcover --delete --exclude '**/.DS_Store' --exclude '.DS_Store'

Run the full filter pipeline with existing local OSM/Wikipedia inputs:

PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-filter-pipeline --dry-run
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-filter-pipeline

Use --refetch-osm or --refetch-wiki only when you explicitly want fresh network fetches. The pipeline validates required inputs before write/prune steps, prunes both summary variants, and writes JSON/CSV/GeoJSON/HTML/parquet artifacts through atomic temp-file replacement helpers.

Grid5000 Article Summarization

Two summary variants are supported and must be generated with distinct --summary-mode values:

• place: normal one-sentence summary; place names may appear.
• no_place: one-sentence summary that asks the model not to mention the described place name.

The standard summary job writes data/wiki/article_summaries.json:

bash scripts/cluster/submit_summarization.sh

The remote job installs uv if needed, syncs the project with the dev and llm dependency groups (uv sync --group dev --group llm) for CUDA llama-cpp-python, and runs:

uv run georeset-wiki-landcover-summarize-articles \
  --input-path data/wiki/article_contents.json \
  --output-path data/wiki/article_summaries.json \
  --summary-mode place

The no-place job writes data/wiki/article_summaries_no_place.json:

uv run georeset-wiki-landcover-summarize-articles \
  --input-path data/wiki/article_contents.json \
  --output-path data/wiki/article_summaries_no_place.json \
  --summary-mode no_place

Land-use evidence summaries are generated through a dedicated extractor and write data/wiki/article_landuse_evidence_summaries.json:

bash scripts/cluster/submit_landuse_evidence_summarization.sh

or locally:

PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-summarize-landuse-evidence \
  --input-path data/wiki/article_contents.json \
  --output-path data/wiki/article_landuse_evidence_summaries.json \
  --seed 42 \
  --temperature 0.0

Optional environment overrides:

G5K_SITE=nancy G5K_REMOTE_DIR=georeset-wiki-landcover G5K_REMOTE_HOME=/home/nflandre \
GEORESET_WIKI_LANDCOVER_MODEL_PATH=Qwen3.6-27B-Q4_0.gguf \
  bash scripts/cluster/submit_summarization.sh

G5K_SITE=nancy G5K_REMOTE_DIR=georeset-wiki-landcover G5K_REMOTE_HOME=/home/nflandre \
GEORESET_WIKI_LANDCOVER_MODEL_PATH=Qwen3.6-27B-Q4_0.gguf \
GEORESET_WIKI_LANDCOVER_LANDUSE_EVIDENCE_OUTPUT_PATH=data/wiki/article_landuse_evidence_summaries.json \
  bash scripts/cluster/submit_landuse_evidence_summarization.sh

Sync a finished summary job with one SSH polling pass:

GEORESET_WIKI_LANDCOVER_SUMMARY_OUTPUT=data/wiki/article_summaries.json \
SYNC_ONCE=1 bash scripts/cluster/sync_summaries.sh

Article-Text Land-Cover Classification

Six primary classification runs are supported: 2 tasks (CORINE level-2 single-label, OSM multi-label) × 3 text sources (normal summary, no-place summary, raw article content). The same three text sources also have deterministic shuffled controls: summary_shuffled, summary_no_place_shuffled, and content_shuffled. Shuffled controls preserve the task, targets, and eligible article set, but reassign texts across eligible articles with the run seed and store shuffled_from_pageid in prediction metadata.

Local runs:

PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task corine_level2 --text-source summary
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task corine_level2 --text-source summary_no_place
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task corine_level2 --text-source content
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task osm --text-source summary
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task osm --text-source summary_no_place
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task osm --text-source content

Shuffled-control local runs:

PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task corine_level2 --text-source summary_shuffled
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task corine_level2 --text-source summary_no_place_shuffled
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task corine_level2 --text-source content_shuffled
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task osm --text-source summary_shuffled
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task osm --text-source summary_no_place_shuffled
PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-classify-articles \
  --task osm --text-source content_shuffled

Outputs default to data/classification/runs/default/:

• data/classification/runs/default/{task}_{text_source}_predictions.json: per-article predictions with raw LLM response, parsed prediction, a normalized prediction_labels list, and full metadata including fingerprint.
• data/classification/runs/default/{task}_{text_source}_metrics.json: aggregate metrics (n_eligible, n_predicted_ok, n_parse_error, coverage, accuracy/F1 scores, task, text_source, allowed_labels, labels_evaluated).

Use --output-dir or GEORESET_WIKI_LANDCOVER_CLASSIFICATION_OUTPUT_DIR to give important runs stable names, for example data/classification/runs/qwen3_6_27b_q4_0/ or data/classification/runs/gemma4_31b_it_q4_0/.

Resumability: articles with matching fingerprint and parse_status=="ok" are skipped; parse errors and ambiguous predictions are re-run. A classification policy version bump in the fingerprint invalidates old caches automatically. Use --limit N for smoke testing.

Label Policies:

• OSM is fully multi-label. Ground truth and predictions support multiple valid tags (e.g., both landuse and natural from the same polygon, or from overlapping polygons).
• CORINE remains single-label. If the LLM generates multiple valid labels for a single CORINE prediction, it is flagged as parse_status="ambiguous". Ambiguous records are excluded from metrics (lowering coverage) but are preserved for audit.

Grid5000 runs:

GEORESET_WIKI_LANDCOVER_CLASSIFICATION_TASK=corine_level2 \
GEORESET_WIKI_LANDCOVER_CLASSIFICATION_TEXT_SOURCE=summary \
bash scripts/cluster/submit_classification.sh

Grid5000 shuffled-control runs use the same launcher with a shuffled text source. Classification jobs request one GPU for 20 hours by default in scripts/cluster/run_classification_job.sh. Auto-sync is disabled by default to avoid repeated SSH polling; sync finished jobs with one-shot syncs only:

GEORESET_WIKI_LANDCOVER_CLASSIFICATION_TASK=corine_level2 \
GEORESET_WIKI_LANDCOVER_CLASSIFICATION_TEXT_SOURCE=summary_shuffled \
bash scripts/cluster/submit_classification.sh

GEORESET_WIKI_LANDCOVER_CLASSIFICATION_TASK=corine_level2 \
GEORESET_WIKI_LANDCOVER_CLASSIFICATION_TEXT_SOURCE=summary_shuffled \
SYNC_ONCE=1 bash scripts/cluster/sync_classification.sh

To freeze the shuffled-control batch after all six shuffled outputs are synced locally, or to regenerate overview tables for any experiment directory:

PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-summarize-classification-experiment \
  --experiment-dir data/experiments/001_qwen_e2e_shuffled_control/article_text_classification_e2e_with_shuffled_control_v1 \
  --title "Article-Text Classification E2E with Shuffled Control v1"

CORINE Spatial Confidence And Spatial-Subset Evaluation

The spatial-confidence experiment does not rerun the LLM. It derives CORINE level-2 point labels from the full CORINE dataset, validates them against frozen CORINE prediction targets where available, computes area-weighted buffer purity at 100 m, 250 m, 500 m, and 1000 m in EPSG:2154, and keeps artificial classes as ambiguity evidence.

PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-compute-corine-spatial-confidence \
  --parent-experiment-dir data/experiments/001_qwen_e2e_shuffled_control/article_text_classification_e2e_with_shuffled_control_v1 \
  --output-dir data/experiments/002_corine_spatial_confidence/corine_spatial_confidence_v1

Reevaluate the frozen parent predictions on spatially reliable subsets without changing prompts, summaries, model outputs, or temperature:

PYTHONDONTWRITEBYTECODE=1 uv run georeset-wiki-landcover-evaluate-spatial-confidence \
  --parent-experiment-dir data/experiments/001_qwen_e2e_shuffled_control/article_text_classification_e2e_with_shuffled_control_v1 \
  --spatial-confidence-path data/experiments/002_corine_spatial_confidence/corine_spatial_confidence_v1/spatial_confidence.csv \
  --output-dir data/experiments/003_qwen_spatial_confidence_reevaluation/article_text_classification_spatial_confidence_v1

Docker

The Docker image contains the installable georeset_wiki_landcover package, packaged CLI entry points, thin top-level scripts/ wrappers for repository compatibility, tests, and Python dependencies. It intentionally does not bake in data/; mount local synced data at /app/data.

Build the image:

docker build -t georeset-wiki-landcover .

Run a quick container smoke test:

docker run --rm georeset-wiki-landcover

Run tests in Docker:

docker run --rm -v "$PWD/data:/app/data" georeset-wiki-landcover uv run pytest tests/fetchers/test_wiki_content_fetcher.py -q

Run a packaged CLI smoke test in Docker:

docker run --rm -v "$PWD/data:/app/data" georeset-wiki-landcover uv run georeset-wiki-landcover-snapshot

Run a pipeline command in Docker:

hf sync hf://buckets/NoeFlandre/georeset-wiki-landcover ./data
docker run --rm -v "$PWD/data:/app/data" georeset-wiki-landcover uv run python -m georeset_wiki_landcover.fetchers.wiki_content_fetcher
hf sync ./data hf://buckets/NoeFlandre/georeset-wiki-landcover --delete --exclude '**/.DS_Store' --exclude '.DS_Store'

The regular Docker image syncs the dev dependency group, not the optional llm group, so it is suitable for tests and non-LLM pipeline commands. GPU LLM workloads should use Grid5000 scripts or an environment created with uv sync --group dev --group llm.

OSM Scope

OSM fetching is intentionally restricted to project-relevant land-cover tags. It excludes dense or unrelated tags such as buildings, amenities, commercial, industrial, residential, and leisure features.

Included landuse values:

farmland, farmyard, meadow, orchard, vineyard, forest, allotments,
plant_nursery, greenhouse_horticulture, grass

Included natural values:

wood, scrub, grassland, wetland, heath, water, bare_rock, sand, scree,
shingle, beach, mud

Publishing Workflow

Use this split every time:

1. Code/docs/tests go to GitHub.
2. Generated/downloaded artifacts go to the Hugging Face bucket.
3. Do not commit data/, build/, caches, or local environment files.

Code push:

git status --short
git add .gitignore README.md docs Dockerfile .dockerignore src scripts tests pyproject.toml uv.lock LICENSE .github
git commit -m "Describe code change"
git push origin main

Data push:

hf sync ./data hf://buckets/NoeFlandre/georeset-wiki-landcover --delete --exclude '**/.DS_Store' --exclude '.DS_Store'