Model Card — SatQuery AI

SatQuery AI answers natural-language questions about satellite imagery using a router + specialists design. This card documents the six trained artifacts released by the project. It is deliberately explicit about what is measured, what is not, and what was rejected.

The six trained artifacts are small modules on top of frozen, publicly-pinned backbones. No backbone weights are redistributed by this release — they are fetched from the Hugging Face Hub at run time, pinned by revision.

Machine-readable identities (byte counts and sha256) are in models/manifest.json and models/checksums.sha256, generated by reading the files (release/tools/generate_model_manifest.py). Where this card and the generated manifest disagree, the manifest wins — it is computed from disk, this card is written by hand.

Companion documents (same depth, same no-fabrication rule): docs/MODELS.md · docs/BENCHMARKS.md · docs/TRAINING.md · docs/EVALUATION.md · docs/LIMITATIONS.md · docs/RESEARCH_NOTES.md · docs/architecture/05-specialists.md.

---

Table of contents

1. How to read this card
2. Overview
3. The six artifacts in this release
4. Backbone dependencies — frozen, pinned by revision
5. Intended use
6. Out-of-scope use
7. Per-artifact reference
8. Full measured-performance table
9. Calibration — a measured negative result
10. Acceptance status
11. Evaluation gaps
12. Limitations
13. Training summary
14. Provenance and verification
15. Licence
16. Citation

---

1. How to read this card

The single most important rule in this document: do not fabricate. Every byte count, sha256, hyperparameter and metric below comes from a file that was read, and each one names its source. Where a fact is not established, this card writes UNKNOWN — not established from the available evidence rather than estimating.

Status vocabulary. Every substantive claim carries one of: IMPLEMENTED · VERIFIED · MEASURED · ATTEMPTED · NOT RUN · BLOCKED · DEFERRED · REJECTED · OPEN · RESOLVED · CLOSED.

The facts most easily stated wrongly, and therefore stated repeatedly:

---

2. Overview

SatQuery AI is a router-and-specialists system: a frozen sentence encoder plus a small trained adapter classify a query into one of six tasks; a deterministic planner dispatches it to the appropriate specialist; each specialist returns a structured ResultEnvelope carrying evidence and a confidence value. The design is CPU-first and frozen-backbone — small modules are trained on top of pretrained encoders, and no encoder is fine-tuned end to end.

The six tasks (configs/base.yaml → router.tasks): vqa · caption · grounding · change · optical_sar · unsupported.

The six trained artifacts (details in §3):

Total released weight payload: 131,130,325 bytes (~125 MiB) (HF_RELEASE_VERIFICATION.md §4).

What is not trained here. MiniLM, SmolVLM-500M, RemoteCLIP ViT-B/32 and CROMA-base are frozen and not redistributed. The one nuance is the change detector's ResNet-18, which is loaded pretrained (pretrained_used: true) and trained in-project as part of the change head — so its weights are part of the released change/head.pt, not a separately-distributed backbone (artifacts/change/eval_test/eval_result.json → checkpoint_embedded_config).

No end-to-end accuracy is claimed anywhere. The router → specialist → envelope pipeline has never been scored end to end. What exists is per-specialist metrics on their own training-family splits (§8) and a behavioural live-validation record that proves the pipeline runs and routes — 3 passes × 8 cases, 8/8 each, 24 live runs, 0 mock nodes, trace fill 94.4444 % (docs/BENCHMARKS.md §5, docs/RESEARCH_NOTES.md §3.3).

---

3. The six artifacts in this release

Reproduced from models/manifest.json → artifacts[*], cross-checked against models/checksums.sha256. Every artifact carries status: "PRESENT" and config_hash: "78f1e3700da15aa1".

The manifest also records each artifact's original repository path and its source metric artifact:

kind semantics. trained_head = a module trained in-project on a frozen encoder, loaded via torch.load of a state_dict (or the module's own loader). trained_adapter = a small classifier over a frozen sentence encoder's cached embeddings, loaded via IntentAdapter.from_config_dict + load_state_dict (router/adapter.py). lora_adapter = a PEFT LoRA delta attached at load time via peft.PeftModel.from_pretrained(model, dir) (specialists/vqa/model.py).

Two independent cross-checks (not self-consistency). The manifest is generated by hashing the files on disk; for two artifacts the computed digest can be compared against a value recorded independently, at a different time, by a different process:

• change_vqa = cfae5e43…d63a82a. Equals artifacts/change_vqa/run/PROMOTION.json → artifact.sha256, the digest recorded in the Kaggle run record before promotion (source.checkpoint_sha256_in_run_record), and artifacts/calibration_v001.json → provenance.checkpoint_sha256 (recorded when the temperature was fitted — a separate step). PROMOTION.json → source.hash_agrees_across records the digest agreeing across model_metadata.json, run_record.json and hashes.json, with byte_identical_to_source: true and artifact.weights_modified: false.
• vlm = 07c76a75…a5adf5e. Equals artifacts/vlm/phase6_closure.json → why_usable_verified.adapter_provenance → adapter_verification.json → weights_file_sha256, and the adapter's own ARTIFACT_SHA256SUMS.json, against which the 14-file directory was verified (manifest_check.clean: true, 14/14 present).

A third, independent re-download check: release/tools/hf_verify.py re-downloads each artifact over direct HTTPS and hashes the received bytes — 6/6 MATCH, 0 failed (HF_RELEASE_VERIFICATION.md §5).

On parameters: null. Four of six artifacts record parameters: null deliberately — the generator does not open checkpoints (that would make generation depend on the model code and torch). Counts measured elsewhere appear in §7 with their source; where a count is not established this card writes UNKNOWN — not established from the available evidence. Training checkpoints are not released artifacts: the VLM adapter's checkpoint-1500//checkpoint-2000/ are provenance only, and the promoted adapter is the top-level end-of-training save, not checkpoint-2000 (§7.6).

---

4. Backbone dependencies — frozen, pinned by revision

Backbones are resolved from the Hugging Face Hub on first use, pinned by revision — a moving main would make every benchmark number unreproducible.

All pins are declared in configs/base.yaml under the router:, vlm:, grounding: and croma: blocks and are validated at load time by core/config.py. Backbone licence terms are each repository's own — see §15. Backbones are not redistributed here.

---

5. Intended use

• Research and demonstration of a modular, CPU-first remote-sensing question-answering system.
• Routing and dispatch of natural-language queries to the appropriate specialist, using the router adapter over frozen MiniLM embeddings.
• Reproducible evaluation of each specialist on its own documented split, using the released artifacts and the frozen config hash 78f1e3700da15aa1.
• Teaching and ablation: the six artifacts are small and individually inspectable; the frozen-backbone design makes each head a self-contained experiment.

The artifacts are intended to be used with their pinned backbones (§4), which the consumer must fetch separately.

---

6. Out-of-scope use

• Safety-, legal- or life-critical decisions. No accuracy, calibration or robustness guarantee is offered. Grounding boxes are image-relative, not geodetic — no geolocation accuracy (docs/LIMITATIONS.md §7).
• Operational geospatial production without independent validation.
• Any use of the VLM adapter as a production model — it is ACCEPTANCE-REJECTED (§10); the deployed caption/VQA path uses the unadapted model.
• Treating per-specialist metrics as system-level accuracy. No end-to-end benchmark exists (§11).
• Any claim that these artifacts generalise beyond their training-family test splits — cross-dataset generalisation is NOT RUN.
• Redistribution of the backbones. This release contains no backbone weights.

---

7. Per-artifact reference

Each subsection gives architecture, hyperparameters (from configs/base.yaml unless noted), training data, evaluation protocol, measured numbers (with source artifact and key path), acceptance status, and limitations.

7.0 Enforced configuration invariants, with arithmetic

The frozen registry is configs/base.yaml; the project rule is "no magic numbers anywhere in Python; everything tunable lives here", and core/config.py loads, validates and hashes every value. Several values are enforced — a mismatch is a load-time error, not a comment:

The frozen hash is verified untouched by test_the_frozen_config_hash_has_not_moved (docs/OWNER_DECISIONS_2026-09-23.md, cross-cutting rule 4). New defaults live in code, not in the registry — which is why the grounding head's default path is DEFAULT_HEAD_PATH in code rather than a base.yaml key (owner decision D-4).

---

7.1 change — STANet-style Siamese change detector

Kind: trained head · File: change/head.pt · Bytes: 63,231,009 · sha256: c5ef31277b67aa01a593aec0eac503eeaccc6d674349fda20ca44c9cc6f8e9fa

Architecture. A STANet-style Siamese detector: a shared ResNet-18 encoder (SharedResNetEncoder), a DifferenceFusion module per stage, PAM spatial self-attention (SpatialAttention, sa_mode: PAM; BAM is the alternative), a three-stage decoder (dec3 → dec2 → dec1) with a final upsample and a 1×1 convolution head to one change logit. Source: specialists/change/stanet.py. Manifest architecture string: "STANet-style Siamese change detector (ResNet-18 + PAM)".

Hyperparameters (base.yaml → change:; artifact checkpoint_embedded_config):

Training data. LEVIR-CD-256, split train 7,120 / val 1,024 / test 2,048 (base.yaml → change.levir_split), matching the published LEVIR-CD counts exactly (docs/OWNER_DECISIONS_2026-09-23.md D-11). Trained on GPU (eval artifact: device: cuda, torch 2.10.0+cu128, python 3.12.13).

Evaluation protocol. Held-out test split, n = 2048, threshold 0.50, tile 256, no overlap. Source: artifacts/change/eval_test/eval_result.json. The artifact checks the checkpoint's embedded config against the frozen hash (checkpoint_config_hash_checked: true, config_drift: false).

Measured numbers:

Acceptance status: VERIFIED and accepted (shipped). This is the only headline metric in the project carrying the VERIFIED tag — measured against a single, immutable public test split with a frozen threshold (docs/BENCHMARKS.md §1.2).

Limitations. Pooled and macro figures diverge (IoU 0.8122 vs 0.8457; F1 0.8964 vs 0.7962), and the corpus is heavily zero-change (p50 change fraction 0.0; only 935 of 2,048 images contain change). No cross-dataset evaluation was run.

---

7.2 change_vqa — change question answering head

Kind: trained head · File: change_vqa/head.pt · Bytes: 5,822,809 · sha256: cfae5e43b97ca930f568dc5b8ae4f36b24e9ff717af226159802206ffd63a82a · Parameters: 1,453,912

Architecture. change_vqa_head_v1 — a two-stage reasoning head, not a generative decoder (training/change_vqa/model.py). Stage 1 maps the change representation to a class-wise change estimate — 6 magnitudes + 6 signed deltas + 1 global fraction = 13 outputs (N_ESTIMATOR_OUTPUTS = 2 × N_CHANGE_CLASSES + 1), supervised by label1/label2. Stage 2 concatenates that estimate with a question encoding and predicts one of 19 answers (N_ANSWERS). The estimator's outputs are also emitted as evidence, so an answer arrives with its own audit trail. Modules: an estimator MLP (change_feature_dim → 256 → 13), a change_trunk, a question_trunk (text feature + question-type embedding + temporal embedding) and an answer_head (fused → 512 → 256 → 19).

Hyperparameters (module constants; base.yaml → training: where applicable):

Training data. CDVQA (dataset_id: cdvqa), feature specs change_feat_v1, change_cache_spec: c801326f85a185f8, text_cache_spec: d2801ea1a314354a, preprocessing_version: change_vqa_preproc_v1. The head's change features are backed by the frozen STANet change detector — frozen_dependency.path: artifacts/change/levir_change_v001/head.pt, sha256 c5ef31277b67aa01…, verified_byte_exact_vs_local: true (the same artifact as §7.1). Trained on an external GPU (Kaggle) — see §13.

Evaluation protocol. Two held-out test sets, test (n = 39,686) and test2 (n = 31,036). epoch_selected was chosen on Val answer accuracy = 0.700018; PROMOTION.json records 93 checks passed, 0 failed, 0 unverified. Source: artifacts/change_vqa/run/PROMOTION.json → verification.

Measured numbers:

mask_gain: 0.0. metric_ruling: "OPEN — the plan leaves the accuracy/macro-F1 interpretation owner-gated. No official aggregate metric is asserted here."

Acceptance status: MEASURED on two test sets; ruling OPEN. PROMOTION.json is explicit that promotion "records provenance and wires the serving path. It does not itself confer VERIFIED status."

Limitations. The wide accuracy–macro-F1 gap (0.697626 vs 0.378373) is the signature of class imbalance: accuracy is dominated by frequent answers while macro-F1 exposes weak rare-class performance (docs/LIMITATIONS.md §1.4). Confidence at this head is raw, not calibrated (method reads "uncalibrated"). The two test sets disagree (0.697626 vs 0.651469), so quoting one alone is selective.

---

7.3 optical_sar — CROMA-base fusion head

Kind: trained head (production) · File: optical_sar/head.pt · Bytes: 14,427,457 · sha256: 785815729a3a39fc34dc41894efaf00d8739365d970a3f830a326e68ae888dab

Architecture. A fusion head over frozen CROMA-base features. CROMA emits three 768-d GAP vectors per sample (optical_GAP, SAR_GAP, joint_GAP); the head concatenates them with the availability masks — optical_mask (B,12) and sar_mask (B,2) — into a (B, 2318) tensor (3×768 + 12 + 2), then LayerNorm → Linear(2318 → 512) → GELU → Dropout(0.2) → Linear(512 → 19). Source: specialists/optical_sar/fusion_head.py. The availability mask is consumed by the head, not by CROMA (finding C-1): handing CROMA the mask would invite it to reconstruct missing channels — the fabrication the sensor adapter exists to prevent.

Hyperparameters (base.yaml → croma: and fusion:):

Training data. reBEN / BigEarthNet-S1 (data/bigearthnet_v2/, 480,038 rows in metadata.parquet; docs/OWNER_DECISIONS_2026-09-23.md D-11). The extraction used the require_single_label policy (n_skipped_by_policy: 0), which preserves the frozen single-label 19-class softmax but changes the evaluation population (see limitations). The A/B arm decision was made separately on best_val_accuracy — A 0.837100 vs B 0.839100, floor 0.0285 → Arm A retained (owner ruling R-14; docs/PHASE12_115_METRIC_COMPUTED.md §5).

Evaluation protocol. The pre-registered 11.5 metric: fusion-head accuracy and macro-F1 over the 19-class label space on the held-out test split, n = 4,000, cache arm A (docs/PHASE14_CROMA_NORMALISATION_CHANGE.md §4). Computed by a separate, later, read-only tool (scripts/eval_fusion_115.py); the trainer deliberately never opens the test split (pre_registered_metric_computed = false in every run record). Source: artifacts/optical_sar/fusion_head_production_v001/pre_registered_115_metric.json.

Measured numbers:

The majority class holds 2,264 / 4,000 = 0.566, so 0.931 is not a constant predictor. Per-class F1 (_per_class_f1) shows a wide spread: one class is perfect (1.000), while classes 5 and 6 are present but score 0.000 — genuine per-class failures, not absent-class artifacts. The median of the 14 present classes is 0.6857 against an accuracy of 0.931 — the signature of prediction dominated by frequent classes (docs/PHASE12_115_METRIC_COMPUTED.md §3.4–§3.5).

Acceptance status: MEASURED; ruling OPEN. Whether 0.931/0.434 constitutes a Phase 12 pass is the metric-of-record ruling, which "has not been made, and it is not engineering's to make." The artifact's own is_deciding_statistic: false and advisory text state that it "selects no head, ranks nothing and compares no arms."

Limitations. (1) Never quote accuracy alone — 0.931 travels with macro-F1 0.434161. (2) The metric describes a single-label subset, not multi-label reBEN: single-label patches are 17.57 % of the corpus (96,537 / 549,488), and under this policy the rarest class survives as 1 patch (a 59,204 : 1 imbalance). It may not be presented as a multi-label BigEarthNet/reBEN result, nor as comparable to published BigEarthNet numbers, nor as a statement about all 19 classes — 5 have no test samples here. (3) The live service returns a bare class index (class_18), not a CLC label (docs/LIMITATIONS.md §1.6).

---

7.4 grounding — RemoteCLIP grounding head

Kind: trained head · File: grounding/head.pt · Bytes: 12,639,041 · sha256: 93432f7034be91a8ffd9c1a84e3eeec00bed7832c043fe7f83d2be230284c6bb · Parameters: 1,052,677 (docs/OWNER_DECISIONS_2026-09-23.md D-4, measured against the real checkpoint).

Architecture. A text-conditioned per-cell box regressor over frozen RemoteCLIP ViT-B/32 tokens. At 224 px the patch grid is 7×7 = 49 tokens of projected dim 512; the text embedding (512) is broadcast to every cell, and each cell's feature is concat([patch, text, patch*text, global_pool]) = 4 × 512 = 2048. The head is Linear(2048 → 512) → LayerNorm → Dropout(0.10) → Linear(512 → 5), emitting [tx, ty, tw, th, obj] per cell. Boxes are cell-relative (YOLO-style), and exactly one cell per target is positive — the one containing the ground-truth box centre. Source: specialists/grounding/head.py.

Hyperparameters (base.yaml → grounding:, grounding_head:, grounding_training:):

Training data. VRSBench (training/data/vrsbench/), 16,159 eval records, all images present (docs/OWNER_DECISIONS_2026-09-23.md D-11). Resolution frozen at 224 by a pre-registered decision (see protocol).

Evaluation protocol. Full VRSBench eval split, 16,159 / 16,159 records, resolution 224, CPU (canonical and matched6 artifacts record device: cpu, torch 2.14.0+cpu). Grounding is reported under two protocols — canonical (config default top_k = 20) and matched6 (top_k = 6, matching the zero-shot baseline's mean 5.99 candidates) — and two decode variants — head_threshold (score threshold 0.40) and head_argmax. head_decode: nms_iou 0.5, score_threshold 0.4.

Measured numbers (canonical: …/eval_result_canonical.json; matched6: …/eval_result_matched6.json):

Latency: head_threshold 2.205 ms/image (canonical) / 2.158 (matched6); head_argmax 0.655 / 0.652; zero_shot_matched 17.9 s / 15.5 s total. The zero-shot decode is threshold_box_plus_local_maxima, delta 0.02, top_k 5, mean 5.99 candidates/image. The artifact's phase7_reference records the zero-shot floor mean_best_iou 0.0972, recall_at_0.50 0.0234 (docs/PHASE7_RESOLUTION_DECISION.md).

The 224-vs-448 decision (pre-registered, then confirmed). The rule was fixed before the result was seen: 448 wins if [email protected] improves by ≥ 0.05 absolute OR mean best IoU improves by ≥ 0.05 absolute; 224 wins otherwise. Result: 224 WINS — mean best IoU gain −0.0147, [email protected] gain −0.0022, at 1.59× the latency. Paired over the identical 16,159 samples: mean paired diff −0.0147, 95 % CI [−0.0160, −0.0134], t = −22.63; 448 better on 8.5 %, worse on 20.9 %. The artifact records rule_changed_since_preregistration: false (docs/PHASE7_RESOLUTION_DECISION.md).

Acceptance status: MEASURED under two protocols; shipped. The trained head is the production default (owner decision D-4); zero-shot is an explicit, labelled fallback, and the system must never silently claim trained while running zero-shot.

Limitations. Absolute IoU is low (0.2838 canonical / 0.2566 matched6) — the head clearly beats the zero-shot baseline (0.0972) but 0.28 is not "solved". The number is protocol-sensitive: an absolute value is meaningless without its protocol and decode variant. head_argmax (0.1215) is not apples-to-apples with the multi-box baseline (mean best IoU is a max over predictions, so 1 box vs ~6 flatters the head). Boxes are image-relative, not geodetic. 448 was rejected at the zero-shot level; whether a learned head has the same resolution sensitivity is UNKNOWN — not established from the available evidence (docs/PHASE7_RESOLUTION_DECISION.md).

---

7.5 router — intent adapter over frozen MiniLM

Kind: trained adapter · File: router/adapter.pt · Bytes: 211,961 · sha256: 8527c3ed28a293e13293d48601d48e3ceafa137b9acabddaf5de31a58a509b5c · Parameters: ~50,822

Architecture. The only trainable part of the router (router/adapter.py, IntentAdapter):

embedding (384) → LayerNorm → Linear(384 → 128) → GELU → Dropout(0.10)
  ├── task_head      Linear(128 → 6)   # vqa/caption/grounding/change/optical_sar/unsupported
  ├── modality_head  Linear(128 → 4)
  ├── temporal_head  Linear(128 → 1)   # logit; P(yes) = sigmoid(logit)
  ├── spatial_head   Linear(128 → 1)
  └── language_head  Linear(128 → 1)

Heads are initialised with small-std weights (std 0.02, zero bias) so the initial sigmoid sits near 0.5 and the binary heads do not start saturated. The adapter does not back-propagate into MiniLM.

Hyperparameters (base.yaml → router: and router.training:):

Training data. A synthetic query corpus: 576 queries in 54 groups (artifacts/router/threshold_sweep_val.json → corpus_total, corpus_groups), split train 410 / val 86 / test 80. Splits are by group (template / hard-negative family), never by example, so template variants cannot leak across the boundary (F4-3). Hard-negative families are placed in the test split so their accuracy measures generalisation, not memorisation. The encoder is frozen, so embeddings are cached and the adapter trains on cached vectors — measured on CPU: 20 epochs over 4,096 × 384 in 0.28 s (F4-2).

Evaluation protocol. A validation-only threshold sweep over 50 thresholds 0.50 … 0.99, on val n = 86, select_by: covered_accuracy. Source: artifacts/router/threshold_sweep_val.json. The test split was NOT touched (test_split_touched: false, n_test_examples_scored: 0).

Measured numbers:

Acceptance status: MEASURED (val only); shipped; test split NOT RUN.

Limitations. The artifact is explicit that this is not a calibration and not a test result: "corpus-limited: val n=86 vs plan >=500. This is NOT a calibration — the corpus is synthetic and too small (min per-class support 8, caption) and val carries 0 hard negatives (hn_ families are held out to TEST by design). Selecting a threshold here yields a justified default, not a calibrated value."* plan_min_val_queries: 500 and plan_min_hard_negatives: 100 are both unmet. The number is ungated accuracy, and the router has known residuals — e.g. "What is the new runway?" reads change, not vqa (docs/LIMITATIONS.md §2).

---

7.6 vlm — SmolVLM LoRA adapter (USABLE_VERIFIED, ACCEPTANCE-REJECTED)

Kind: LoRA adapter (PEFT) · File: vlm/adapter_model.safetensors · Bytes: 34,798,048 · sha256: 07c76a75fa04624880ed7730590f5fdd7b145a8232e3c0af411c3c545a5adf5e · Trainable params: 8,683,520 (1.6823 % of the 516,165,824-param base)

Architecture. A PEFT LoRA adapter (r = 16, alpha = 32, dropout = 0.05) on the text-model projections of frozen HuggingFaceTB/SmolVLM-500M-Instruct (rev a7da5b986cb5). Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj across 224 modules. trainable_subtrees is exactly {"model.text_model": 8683520} — the vision tower was untouched (86,433,024 frozen) and the connector (11,796,480) is frozen too. Precision: fp16.

Hyperparameters (base.yaml → training:, vlm:; phase6_closure.json):

Training data. BigEarthNet-derived presence questions (kind: bigearthnet_smolvlm_lora). Trained on an external GPU (Kaggle, T4) — see §13. The adapter directory holds 14 files verified against its own ARTIFACT_SHA256SUMS.json.

Evaluation protocol. A frozen 1,000-question test subset (available_per_split {val: 6750, test: 7772}, subset n = 1000, 19 classes summing to 1000). The pre-registered acceptance rule is v002: V1 requires aggregate test delta ≥ +5.00 pp; V2 (a per-class guardrail) fails a class with n ≥ 20 questions iff it both lost ≥ 4 questions and has z ≥ 1.96. Source: artifacts/vlm/phase6_closure.json → why_acceptance_rejected, why_usable_verified.

Measured numbers:

Why it is usable and verified. Gate D reproduced Run 1's adapted-test control exactly (exact_match 0.963, f1 0.9643201542912246, identical confusion), proving the local artifact is Run 1's adapter and that CPU/fp32 reproduces the Kaggle T4 endpoint. Gate A″ proved subset identity without a model. The 14-file manifest check is clean (0 missing, 0 mismatched, 0 extra) and the adapter loads through the production path (PeftModel.from_pretrained).

Why it is acceptance-rejected. V1 passes (+49.50 pp ≥ +5.00), but V2 fails: class Mixed forest (n = 33) goes 100.00 → 87.8788 pp, a drop of 12.1212 pp, lost_questions 4, z 2.1335 — failing both halves of v002. Per item V, a complete run that fails V2 is REJECTED. The rejection is narrow (1 of 19 classes fails; 11 improved, 5 held) and is not a split artefact — the same class also degraded on val in Run 1 (drop 6.4516 pp, n = 31). Residual risk, reported not resolved: the verdict rests on 4 questions in one class of 33, the unfloored minimum-size exposure recorded at PHASE6_AUDIT_AND_CONTRACT.md §8.6.

Acceptance status: USABLE_VERIFIED and ACCEPTANCE-REJECTED — both true, answering different questions. USABLE_VERIFIED ≠ ACCEPTANCE-ACCEPTED. The deployed caption/VQA path uses the unadapted model; the adapter is enabled only via the SATQUERY_VLM_ADAPTER environment variable (specialists/vqa/model.py → ADAPTER_ENV_VAR).

Limitations and traps. (1) The adapter is not accepted for production use. (2) The Mixed forest regression is not resolved. (3) adapter_sha256 names two different values and they are not interchangeable — a tree hash over the weight map (5c6b8631…, from training/vlm/artifact.py) versus the file sha256 of adapter_model.safetensors (07c76a75…, from specialists/vqa/model.py::_adapter_sha256); comparing one against the other produces a false "artifact was altered" conclusion. (4) The promoted adapter is not checkpoint-2000 — the three weight files have three distinct digests (top-level 07c76a75…, checkpoint-1500 7273588e…, checkpoint-2000 bf249943…). (5) The adapter's canonical path is under .scratch/; it is reconstructible from phase6_realbundle.zip and verified against the two digests above.

---

8. Full measured-performance table

Every row names its source artifact and the exact key path. The n and split columns are part of the claim, not decoration: a metric without its population is not a result. All 20 numeric claims are checked against these files by tools/verify_readme_metrics.py; its output (ALL CLAIMS VERIFIED) is committed as tools/readme_metrics_report.txt.

---

9. Calibration — a measured negative result

Temperature scaling is enabled in the frozen configuration (confidence.temperature_scaling: true, confidence.calibration_file: calibration_v001.json) and applied by evidence.confidence.TemperatureCalibration as sigmoid(logit(z)/T) for a scalar z and softmax(logits/T) for a distribution. Source: artifacts/calibration_v001.json.

The honest reading: the ECE got worse. Temperature scaling reduced the NLL very slightly (0.00011) but increased the expected calibration error from 0.013755 to 0.014929. It is retained only because it is part of the frozen configuration — not because it helped. This is a measured negative result and is reported as one (docs/BENCHMARKS.md §4.7, docs/MODELS.md §5).

Two caveats on the number. The artifact notes that "ECE is bin-count sensitive and is not an aggregate score", and the reliability_diagram it carries is the pre-scaling curve (ece 0.013755), labelled as such — the calibrated curve is NOT plotted (docs/LIMITATIONS.md §3.22).

---

10. Acceptance status

USABLE_VERIFIED ≠ ACCEPTANCE-ACCEPTED. The VLM adapter works and is not promoted. The two questions — is this the artifact we trained, and does it work? versus did it clear the bar we predeclared before looking? — are kept separate on purpose (docs/PHASE6_CLOSURE.md §1).

---

11. Evaluation gaps (stated, not hidden)

---

12. Limitations

A condensed catalogue; the full version is docs/LIMITATIONS.md. Per-artifact limitations are in §7; evaluation gaps in §11.

Model quality. Grounding absolute IoU is low (0.2838 / 0.2566) and protocol-sensitive. Optical-SAR accuracy is carried by common classes — 0.931 with macro-F1 0.434161. Change-VQA is weak on rare classes (0.697626/0.378373 and 0.651469/0.372309). The live optical-SAR service returns a bare class index (class_18), not a CLC label. The VLM adapter is not accepted. Calibration made ECE worse.

Router. 0.965116 is validation, ungated, n = 86, corpus-limited; the test split was NOT RUN. Known residuals: "What is the new runway?" reads change; "How much built-up area was added?" under-triggers vqa; with one asset the console reads change while dispatch correctly falls back to change_vqa (intentional, but visually surprising).

Evaluation. No end-to-end benchmark; no cross-dataset, human or robustness evaluation; most metrics are point estimates without confidence intervals. The BigEarthNet local subset is 100 % single-label, so its metrics are not comparable to published multi-label numbers.

Operational. Transient tunnel gaps (B-07) — a request can hang or return 504; patch prepared but NOT deployed. OPEN. /api/health codespace_name carries a trailing newline (B-02) — cosmetic. OPEN. Cold start is tens of seconds; single-region, no HA; no database, auth or queue (stateless by design).

Packaging and licensing. No LICENSE file exists in the source repository. OPEN. The six artifacts require their pinned backbones, which are not redistributed.

Documentation. docs/FINAL_DELIVERY_REPORT.md §6 is stale (it lists the bundled EO change pair as DEGRADED and B-01 as BLOCKED; both were resolved on 2026-09-25). The original master plan describes a superseded deployment (Gradio GUI + HF Space + ZeroGPU + Railway); the shipped system is a static frontend + Render + Codespace tunnel serving JSON.

Explicit non-claims. No state-of-the-art claim; no production-readiness claim for model quality; no claim that the trained heads generalise beyond their training-family splits; no claim that calibration improves confidence; no claim that the VLM adapter is accepted; no end-to-end accuracy claim; no robustness claim; no geolocation-accuracy claim; not a safety-, legal- or life-critical tool.

---

13. Training summary

All six artifacts are small modules on frozen backbones, trained with seed 42 and recording the frozen config hash 78f1e3700da15aa1.

Precision. training.precision: fp16 because the target GPU (T4) is compute capability 7.5 — bf16 is unavailable there (finding C-6). The loader validates the value is one of fp16|bf16|fp32.

Provenance discipline. change_vqa was promoted from a Kaggle export with 93/0/0 verification checks and a byte-identical source copy. The VLM adapter's Phase 6 closure was reached without retraining or modifying the adapter — the promoted weights are the end-of-training top-level save, and the closure record is generated from the evidence rather than restated. Full procedures: docs/TRAINING.md; dataset provenance: docs/DATASETS.md.

---

14. Provenance and verification

Verification chain. (1) models/manifest.json and models/checksums.sha256 are generated by reading the files (release/tools/generate_model_manifest.py); no byte count or hash is typed by hand. (2) tools/verify_readme_metrics.py walks every quoted metric to its source artifact; result ALL CLAIMS VERIFIED (20/20), with status assertions (VLM ACCEPTANCE-REJECTED, router corpus_limited n = 86, calibration ece_improvement negative) confirmed. (3) release/tools/hf_verify.py re-downloads each artifact over direct HTTPS and hashes the received bytes; 6/6 MATCH, 0 failed. (4) Two artifact digests agree with values recorded independently at promotion/fit time (§3).

Hugging Face release. thundercode/SatQuery (public), HEAD bf2779e18fcaa7476b93a48a978f08c108dfdfb7, lastModified 2026-09-25T21:46:52Z, 42 files on the Hub. No secret was uploaded; the token used is not written into any released file (HF_RELEASE_VERIFICATION.md §8).

GitHub release. The curated public repository target is Anish-lab-blip/SatQuery-AI; the release tree is staged and its links verified. At the time of release/RELEASE_EXECUTION_CHECKLIST.md Phase 5, the push was BLOCKED because the fine-grained token was read-only for repository contents (403 Resource not accessible by personal access token) — an owner action, not a defect in this release. Where this card and a live repository disagree, treat the live repository as authoritative for publication state and this card as authoritative for artifact identity.

---

15. Licence

The project ships no licence file; a licence must be selected by the owner before public release of the code (docs/LIMITATIONS.md §5, RELEASE_MANIFEST.md). This is an OPEN item. The Hugging Face card declares license: other because the correct licence has not yet been chosen.

Model weights carry the terms of their backbone licences. The six artifacts are small modules, but they depend on and are intended to be used with: sentence-transformers/all-MiniLM-L6-v2; HuggingFaceTB/SmolVLM-500M-Instruct; chendelong/RemoteCLIP (RemoteCLIP-ViT-B-32.pt); antofuller/CROMA (CROMA_base.pt); and torchvision ResNet-18 (IMAGENET1K_V1). Backbones are not redistributed here. Consult each backbone's Hugging Face page for the authoritative licence — the licence labels above are recorded for convenience and should be verified at the source before reuse.

---

16. Citation

If you use this work, cite the project repository:

@misc{satquery_ai_2026,
  title  = {SatQuery AI: A Modular Router-and-Specialists System for Satellite Imagery Question Answering},
  author = {SatQuery AI},
  year   = {2026},
  note   = {Public release: https://github.com/Anish-lab-blip/SatQuery-AI}
}

Model card version. This card documents release 1.0.0 (2026-09-25), frozen config hash 78f1e3700da15aa1. The changelog is docs/CHANGELOG.md.