Kimi K3 complete tiny random BF16 CPU fixture
0
2 commits
2 linked in READMEs
updated Sep 7, 2026
Untrained independently seeded random weights; no upstream weights or training data. This is a reproducibility fixture, not useful language modeling or production quality evidence.
Upstream moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06.
Actual loaded class: KimiK3ForConditionalGeneration. Complete untied head and real small vision tower/projector.
Parameters: 269688; vision parameters: 9184; semantic tensors: 154.
Complete upstream KimiK3ForConditionalGeneration wrapper with real 1-layer MoonViT-V2 and patchmergerv2; 4-layer KimiLinear text, 3 KDA + 1 gated full MLA, attention-residual block size2 (crosses block boundary), SITU(beta4,linear_beta25), latent MoE dimension32 with normalization/down/up projections, 4 routed/top2 and 2 shared experts, complete untied head. No GLM substitution.
Modified reviewed upstream runtime, not original serving stack: original eager MLA/vision attention and true Torch SDPA for packed vision clips when selected by Transformers, faithful Torch sequential KDA gate/decay/delta recurrence and cached short convolution, FLA fused RMSNorm+sigmoid replaced with explicit math. Transformers5.16.1 API compatibility patches recorded. Custom capture uses local weights but may use network for code-pin verification; not entirely offline. Exact floating-point kernel equivalence to GPU fused kernels is not asserted.
No MTP tensors or layers. Byte-level BPE tokenizer: 272 tokens, IDs 0..271.
Two independent builds have identical complete file inventories. Exact semantic reload checks values and dtypes.
Two fresh CPU capture processes, two threads each; honest lane other; sealed 4x64 synthetic panel, 252 scored positions.
Forced replay KL: 0.0; top-1 agreement: 1.0.
The actual NumPy float64 tokenwise artifact is checked for 252 positive zeros, not only a summary metric.
Cache boundary probes and real synthetic vision-input sensitivity are in native-verification.json.
Original packed formats are NOT validated by this BF16 fixture: Published source uses compressed-tensors mxfp4-pack-quantized weight format, floating4 group32 with uint8 scales; advertised serving includes MXFP8 activations. Original runtime needs a compressed-tensors version with MXFP4 pack support and matching supported accelerator kernels/serving engine; neither installed CPU Torch nor this BF16 reference fork implements packed MXFP4 or MXFP8. Fork preserves config/weight names for ordinary unquantized checkpoints only. Do not delete quantization metadata or reinterpret packed arrays as BF16; obtain an explicitly complete standard BF16/FP32 checkpoint for CPU capture.
Captured model/code immutable pin: malaiwah/kimi-k3-tiny-random-bf16@3a12212e4692e70b531f541e20f6237efb79637e.
Use Python 3.12 and requirements-cpu.txt; $QFS is the recorded QFS checkout.
python release.py --stage-only --out NEW_DIR --qfs-root "$QFS" stages privately and refuses repository collisions.
python release.py --resume FAILED_DIR --out FAILED_DIR/recovery-001 --qfs-root "$QFS" rebuilds corrected code twice and creates a guarded new private model/code revision, retaining the old revision and failed artifacts before rerunning every check.
python release.py --publish --out SAME_DIR publishes only unchanged successful stages and verifies anonymous downloads.
For third-party capture use the exact commands in support.json with an explicit --author.
Custom model code, if applicable, is never implicitly trusted; pass the exact immutable reviewed code pin.
Evidence: https://huggingface.co/datasets/malaiwah/kimi-k3-tiny-cpu-repro-v1
The evidence repository is a receipt bundle, not itself a single QFS root dataset; first/ and repeat/ are roots.
Reproduction is same-machine/same-stack repeatability, not cross-hardware equivalence.
Retain Kimi K3 License (commercial Model-as-a-Service >$20M aggregate12-month revenue requires separate agreement; UI attribution thresholds apply), Apache-2.0 adapted DeepSeek/Transformers portions, MIT FLA reference code. Random independent weights do not relicense copied runtime code. See LICENSE, upstream-code-audit.json and retained source/license evidence. No paid GPU/cloud use.
Recovery of a failed private native vision attempt after correcting the reviewed runtime SDPA backend. Original logs and receipts are retained unchanged under failed-attempt/ in the evidence repository and at the original local stage. This stage reruns both builds, native vision/cache/tensor verification, panel generation and both captures against a new immutable model/code revision; the old revision is retained in Hub history.
See recovery.json for the old and recovered immutable pins and retained file hashes.
Kimi K3 complete tiny random BF16 CPU fixture
0
2 commits
2 linked in READMEs
updated Sep 7, 2026
Untrained independently seeded random weights; no upstream weights or training data. This is a reproducibility fixture, not useful language modeling or production quality evidence.
Upstream moonshotai/Kimi-K3@f831ab66814297da540d832a5235f8e904f29d06.
Actual loaded class: KimiK3ForConditionalGeneration. Complete untied head and real small vision tower/projector.
Parameters: 269688; vision parameters: 9184; semantic tensors: 154.
Complete upstream KimiK3ForConditionalGeneration wrapper with real 1-layer MoonViT-V2 and patchmergerv2; 4-layer KimiLinear text, 3 KDA + 1 gated full MLA, attention-residual block size2 (crosses block boundary), SITU(beta4,linear_beta25), latent MoE dimension32 with normalization/down/up projections, 4 routed/top2 and 2 shared experts, complete untied head. No GLM substitution.
Modified reviewed upstream runtime, not original serving stack: original eager MLA/vision attention and true Torch SDPA for packed vision clips when selected by Transformers, faithful Torch sequential KDA gate/decay/delta recurrence and cached short convolution, FLA fused RMSNorm+sigmoid replaced with explicit math. Transformers5.16.1 API compatibility patches recorded. Custom capture uses local weights but may use network for code-pin verification; not entirely offline. Exact floating-point kernel equivalence to GPU fused kernels is not asserted.
No MTP tensors or layers. Byte-level BPE tokenizer: 272 tokens, IDs 0..271.
Two independent builds have identical complete file inventories. Exact semantic reload checks values and dtypes.
Two fresh CPU capture processes, two threads each; honest lane other; sealed 4x64 synthetic panel, 252 scored positions.
Forced replay KL: 0.0; top-1 agreement: 1.0.
The actual NumPy float64 tokenwise artifact is checked for 252 positive zeros, not only a summary metric.
Cache boundary probes and real synthetic vision-input sensitivity are in native-verification.json.
Original packed formats are NOT validated by this BF16 fixture: Published source uses compressed-tensors mxfp4-pack-quantized weight format, floating4 group32 with uint8 scales; advertised serving includes MXFP8 activations. Original runtime needs a compressed-tensors version with MXFP4 pack support and matching supported accelerator kernels/serving engine; neither installed CPU Torch nor this BF16 reference fork implements packed MXFP4 or MXFP8. Fork preserves config/weight names for ordinary unquantized checkpoints only. Do not delete quantization metadata or reinterpret packed arrays as BF16; obtain an explicitly complete standard BF16/FP32 checkpoint for CPU capture.
Captured model/code immutable pin: malaiwah/kimi-k3-tiny-random-bf16@3a12212e4692e70b531f541e20f6237efb79637e.
Use Python 3.12 and requirements-cpu.txt; $QFS is the recorded QFS checkout.
python release.py --stage-only --out NEW_DIR --qfs-root "$QFS" stages privately and refuses repository collisions.
python release.py --resume FAILED_DIR --out FAILED_DIR/recovery-001 --qfs-root "$QFS" rebuilds corrected code twice and creates a guarded new private model/code revision, retaining the old revision and failed artifacts before rerunning every check.
python release.py --publish --out SAME_DIR publishes only unchanged successful stages and verifies anonymous downloads.
For third-party capture use the exact commands in support.json with an explicit --author.
Custom model code, if applicable, is never implicitly trusted; pass the exact immutable reviewed code pin.
Evidence: https://huggingface.co/datasets/malaiwah/kimi-k3-tiny-cpu-repro-v1
The evidence repository is a receipt bundle, not itself a single QFS root dataset; first/ and repeat/ are roots.
Reproduction is same-machine/same-stack repeatability, not cross-hardware equivalence.
Retain Kimi K3 License (commercial Model-as-a-Service >$20M aggregate12-month revenue requires separate agreement; UI attribution thresholds apply), Apache-2.0 adapted DeepSeek/Transformers portions, MIT FLA reference code. Random independent weights do not relicense copied runtime code. See LICENSE, upstream-code-audit.json and retained source/license evidence. No paid GPU/cloud use.
Recovery of a failed private native vision attempt after correcting the reviewed runtime SDPA backend. Original logs and receipts are retained unchanged under failed-attempt/ in the evidence repository and at the original local stage. This stage reruns both builds, native vision/cache/tensor verification, panel generation and both captures against a new immutable model/code revision; the old revision is retained in Hub history.
See recovery.json for the old and recovered immutable pins and retained file hashes.
