K2-Horizon MoVA tiny random CPU fixture

Complete untrained K2HorizonForCausalLM, not Moonshot Kimi despite the K2 name. Architecture source: IFM/K2-Horizon-MoVA-36B-A4B at 05cab0a4d7150c1c460a000b37ff40cc1af2feaa. No pretrained weights, original tokenizer, training data, paid GPU or cloud compute used.

Complete text-only K2HorizonForCausalLM, not Kimi: three-layer dense prefix followed by two real MoVA+MoE layers, grouped RMSNorm, sigmoid top-k routing with selection-only bias, shared expert, softplus attention gates, RoPE and dynamic KV cache. No MTP. Random BF16 fixture does not establish full 37.4B-weight model quality, GPU runtime, long-context accuracy, or quantization fidelity.

Complete architecture

Hidden64, five decoder layers: original three-layer dense prefix then two MoVA+MoE layers. Q4/KV2 heads of dimension16, full16-dimensional rotary embedding, original theta10000000, grouped RMSNorm with two groups. Four value experts/top2 with SiLU values; four MLP experts/top2, 32-dimensional routed expert MLP, one shared expert. Dense intermediate128. Original sigmoid router, selection-only router bias, normalized top-k weights scaled2.5, and softplus gate beta log(2). Original query_key_norm=false and untied embedding/head policy retained. Full272-token head; real256-byte BPE tokenizer plus16 special/reserved IDs. Context256. No MTP.

Measured results and scope

• Parameters: 265,936; semantic tensors: 91; seed 20260907.
• Two independent generator subprocesses compared byte-for-byte; clean native reload and QFS semantic tensor values/dtypes verified, without allow-missing/mismatched overrides.
• Native report status: passed. Actual MoVA routed cache values, selection-only bias, ordinary MoE/shared-expert equation, grouped norm and softplus gate observations are in native-verification.json. Dynamic cache continuation at63/64/65 boundaries is repeated exactly; cached-vs-cold differences are measured, not asserted zero.
• Two fresh CPU BF16 QFS capture processes, two PyTorch threads each, honest lane other. Four sealed64-token synthetic windows,252 next-token positions, not benchmark data.
• Strict forced replay metric: 0.0; KL {'max': 0.0, 'mean': 0.0, 'median': 0.0, 'p95': 0.0, 'p99': 0.0, 'p99_9': 0.0}; top1 1.0. Actual252 positive float64 zero values checked in the NumPy artifact.
• Captured weights and reviewed code pin: malaiwah/k2-horizon-tiny-random-bf16@953fbee42d1503202daf18328aa08d8e4e29c9de. QFS source HEAD: d95b4e96d19e37c46e84e5fe54cc3e66c5316782; executed file hashes accompany the measured evidence.

The upstream public checkpoint is BF16, not FP8/MXFP4: pinned metadata reports37,444,792,020 BF16 parameters; no quantization_config. We did not download or run those weights. This tiny fixture proves neither full-model quality nor original GPU performance,512K context, quantization or cross-hardware equality.

Trust, licensing and evidence

The two retained runtime Python files are source-audited and immutable-pinned, not native Transformers classes. source-audit.json records their SHA256/import/effect closure; patch-manifest.json records compatibility source changes. Both runtime files retain Apache2.0 copyright headers. Independent random weights/tokenizer are Apache2.0; adapted workflow scripts retain MIT terms in WORKFLOW-LICENSE. No remote-code execution is enabled by default in QFS. Captures explicitly opt into audited code and allow HF network for immutable metadata/code verification only; model weights load locally. Comparison is offline/code-independent.

Evidence bundle is an artifact receipt bundle, not itself a single QFS root dataset. It contains first/,repeat/,comparison/, strict outputs, complete scripts, sealed panel, runtime/resource records, source/license records and byte-exact Hub proofs. No production registry row or paid admission profile is submitted.

Reproduce

Use Python3.12 with requirements-cpu.txt and a QFS checkout at $QFS. Download scripts and upstream/ source records from the evidence repository together. Outputs must not exist; substitute your explicit author identity:

python build_fixture.py --out /tmp/horizon-rebuilt
python verify_native.py --model-dir /tmp/horizon-rebuilt --model-repository malaiwah/k2-horizon-tiny-random-bf16 --model-revision 953fbee42d1503202daf18328aa08d8e4e29c9de --qfs-root "$QFS" --out /tmp/horizon-native.json
python build_panel.py --model-dir /tmp/horizon-rebuilt --model-repository malaiwah/k2-horizon-tiny-random-bf16 --model-revision 953fbee42d1503202daf18328aa08d8e4e29c9de --qfs-root "$QFS" --out /tmp/horizon-panel
python verify_cpu.py --model-dir /tmp/horizon-rebuilt --model-repository malaiwah/k2-horizon-tiny-random-bf16 --model-revision 953fbee42d1503202daf18328aa08d8e4e29c9de --panel-dir /tmp/horizon-panel --qfs-root "$QFS" --out /tmp/horizon-repro --author YOUR_HF_USERNAME

Publisher-only: release.py --stage-only --out NEW_DIR --qfs-root "$QFS" stages both fixed repositories privately, refuses collisions, and must pass every gate. Explicit --publish --out SAME_DIR releases only an unchanged successful stage and verifies anonymous downloads, including the initial immutable capture/code pin. No overwrite route.