AX-gemma-4-31b-MLX-AXQ-6bit-MTP — 6.00 BPW measured main

An AXQuant (AXQ) mixed-precision MLX checkpoint for Apple Silicon, converted directly from the BF16 source model. The language path is quantized while the external assistant MTP drafter and vision tower are preserved at BF16 in the checkpoint (or a bound sidecar when present).

Development evidence — not a certified AXQuant release. This package has conversion and artifact-integrity records, but it does not publish measured quality, long-context, kernel-speed, or MTP-speed evidence. Do not interpret the AXQ product label as a benchmark claim.

Model details

This repository contains MLX Safetensors. It does not contain PyTorch or GGUF weights.

Choosing an AXQ pack

AXQ names describe a storage-budget product class, not one uniform precision applied to every tensor. Protected tensors remain at higher precision, so the exact measured BPW is authoritative. In particular, a 6bit-named mixed plan may retain 4bit as its base precision while selecting 6-bit, 8-bit, or BF16 for other tensors to meet an approximately 6-BPW total budget. Protection floors can also raise a 4bit-named pack close to (or above) a 6bit budget on small or heavily protected models. When that collapse happens, AutomatosX does not publish a separate misleading 4bit sibling for that base.

See the AutomatosX collections for the family catalog, or the complete index.

Download

python -m pip install -U huggingface_hub
hf download AutomatosX/AX-gemma-4-31b-MLX-AXQ-6bit-MTP --local-dir ./AX-gemma-4-31b-MLX-AXQ-6bit-MTP

Allow at least 23.49 GB of free disk space. Pin the resulting Hub commit in reproducible deployments rather than relying indefinitely on main.

Run with MLX-VLM

python -m pip install -U mlx-vlm
python -m mlx_vlm.generate \
  --model AutomatosX/AX-gemma-4-31b-MLX-AXQ-6bit-MTP \
  --image ./image.png \
  --prompt "Describe this image." \
  --max-tokens 128 \
  --temperature 0.0

The protected vision tower and AXQ language decoder are loaded together by MLX-VLM. The artifact records MLX 0.32.2; runtime QA is reported separately from model-quality claims.

Serve with AX Engine and MTP

After installing AX Engine, download the complete repository (see AXQuant for conversion, certificates, and model-card tooling) and serve the local directory:

ax-engine serve ./AX-gemma-4-31b-MLX-AXQ-6bit-MTP --port 31418

AX Engine is the authority for the AXQ runtime contract and paired assistant-MTP bundle. This development package does not claim runtime speedups until identical-checkpoint benchmarks are published. The artifact records AX Engine version not recorded. Native model-manifest.json status: included as model-manifest.json.

Use the packaged Gemma assistant with oMLX VLM MTP

This repository uses an external gemma4_assistant drafter under assistant/. It is not an embedded-head checkpoint, so do not enable Lightning MTP or import it as a Qwen sidecar. Download the complete repository, add both ./AX-gemma-4-31b-MLX-AXQ-6bit-MTP and ./AX-gemma-4-31b-MLX-AXQ-6bit-MTP/assistant as local oMLX models, then configure the target model with VLM MTP enabled and select the assistant model. The equivalent model-setting fields are:

vlm_mtp_enabled: true
vlm_mtp_draft_model: ./AX-gemma-4-31b-MLX-AXQ-6bit-MTP/assistant
vlm_mtp_draft_block_size: 2

The normalized and indexed vision.safetensors layout is loadable by MLX-VLM 0.6.17 or newer. For gemma4_unified targets, the same sidecar includes the protected vision_embedder, embed_vision, and embed_audio modules and the output restores the upstream unified config. Runtime discovery does not establish identical-output, acceptance-rate, or speed certification for oMLX. Follow the exact checkpoint revision's Tier 2 status.

Quantization layout

• Quantization methods: affine, bf16.
• Group sizes used by quantized assignments: 32, 64.
• MTP sidecar: external assistant under assistant/ (checksum-bound composite).
• Vision sidecar: 356 tensors, 575.74M parameters, 1.15 GB, BF16.
• Vision weights: protected BF16 sidecar.
• Optimization scope: text-path.
• Support tier: convertible.

BF16 sidecars, when present, are included in total download size. Their presence does not by itself establish MTP acceleration or vision-language quality.

Evidence and validation status

Intended use and limitations

• Intended for local development and evaluation on Apple Silicon with MLX-compatible runtimes.

• No minimum unified-memory figure is claimed; loadability depends on model size, context length, KV-cache policy, runtime buffers, and other processes using unified memory.

• Architecture-prior allocation is not measured sensitivity. It must not be presented as measured model quality.

• Gemma assistant MTP requires an external-drafter runtime. oMLX VLM MTP discovery does not establish exactness or speed certification.

• Vision weights are preserved at BF16, but this release does not claim validated VLM quality.

• The configured context window can require substantially more memory as the KV cache grows.

• Upstream capabilities, limitations, biases, and responsible-use guidance still apply.

Provenance and audit files

• axquant_manifest.json: package identity, byte accounting, runtime contract, software versions, and file checksums.
• axquant_plan.json: per-tensor precision decisions and planning evidence.
• axquant_quantizer_execution.json: conversion coverage and fallback records.
• axquant_runtime.json: declared AX Engine and MLX compatibility metadata; runtime checks remain separate evidence.
• axquant_mtp_sidecar_manifest.json: MTP tensor provenance.
• axquant_vision_sidecar_manifest.json: protected vision tensor provenance.
• model-manifest.json: AX Engine native tensor manifest.

All published provenance uses repository-relative paths. Local source paths are stripped before publication. The checkpoint was converted from BF16 rather than re-quantized from an OptiQ artifact. If an OptiQ repository is published separately, it uses a different quantizer and should not be assumed to have identical BPW or quality.

License

The checkpoint follows the upstream model license where applicable (often Apache License 2.0). See the google/gemma-4-31B-it model card for license terms, model limitations, and responsible-use guidance.