Occamy-1.0 GGUF

GGUF quantizations of Accio-Lab/occamy-1.0. The existing Q4_K_M, Q8_0 and vision projector are unchanged. New quantizations use a calibration importance matrix; labels describe mixed-precision GGUF recipes, not uniform bits for every tensor.

Downloads

File size is not peak RAM/VRAM. KV cache, context, concurrency and images require additional memory. The projector is optional for text-only use. New quantizations were tested for text/code/JSON/tool use; vision was not retested for them.

Tokenizer and runtime

Read tokenizer compatibility before running. All released files store qwen2. New files were corrected from the converter-inferred qwen35 without changing tensor payloads. These rules differ on some Unicode text. The pinned source tokenizer uses NFC normalization and a qwen2-style rule. The legacy NFC and 14-case findings are preserved in the original README.

The reported benchmark used qwen35 throughout, including --override-kv tokenizer.ggml.pre=str:qwen35 on the legacy Q4_K_M/Q8_0 files to match its BF16 GGUF reference. To reproduce those measurements on the corrected new copies, the same qwen35 override is also required. It is not an out-of-the-box score for those two files or proof of tokenizer equivalence to the Transformers source. See the bounded source comparison and precise reproduction notes in TOKENIZER.md.

For normal source-compatible text input, normalize prompt content with unicodedata.normalize("NFC", text) and use the native qwen2 metadata. For example:

llama-server -m occamy-1.0-Q4_K_M.gguf -ngl 999 -c 8192 -np 1 -fa on --jinja

The new release copies each passed two short GPU load/generation checks with qwen2+NFC. This is not a full quality rerun. Published hashes and release checks are separate from the original benchmark hashes.

Measured validation

Validation report and machine-readable results cover a local BF16 reference and all twelve quantizations. Each has 24 GSM8K questions, 24 HumanEval+ code outputs actually executed in a sandbox, and six strict JSON plus six tool cases. Per-question outputs and results are separate. These small subsets do not reproduce full published benchmark scores. Lower-bit PPL degradation is reported, not treated as file corruption.

Runtime: CUDA llama.cpp 972d2313bc0bf0a45f634f77d95c9fb03aeab12c on NVIDIA B200; eight CPU threads per timed job, full GPU offload, Flash Attention and F16 KV cache. Engine tests use five repetitions at 512/2048-token contexts. The additional three variants ran independent jobs across four B200 cards; cross-card and shared CPU/I/O effects limit comparisons. CPU-only and Apple Silicon runtime compatibility were not established here.

Historical Q4_K_M/Q8_0 conversion and vision smoke evidence remains in TECHNICAL-DETAILS.md, VALIDATION.json and the preserved README.

MTP and license

The Occamy MTP head is separate. These language-model GGUF files do not contain an MTP head; this update does not establish GGUF draft-head compatibility. Apache 2.0.