AX Qwen3.5 9B MLX 4-bit MTP

Parameter count: approximately 9.41B logical target parameters (9B class). 4-bit is the target quantization precision, not a 4B model-size claim. The separately packaged MTP sidecar is not included in the target count.

This is an MLX model package for Apple Silicon. It combines the 4-bit MLX quantization of Qwen3.5-9B with an AX Engine-compatible multi-token-prediction (MTP) sidecar.

The target model verifies drafted tokens during speculative decoding. This repository does not contain PyTorch or GGUF weights.

Model details

• Base model: Qwen/Qwen3.5-9B
• MLX target: mlx-community/Qwen3.5-9B-MLX-4bit, revision 938d8919941c6e7efd3c7150eff7fe9d12afa631
• Format: MLX Safetensors
• Quantization: 4-bit affine, group size 64
• Task: image-text-to-text and conversational generation
• Configured context length: 262,144 tokens
• MTP sidecar: 15 tensors, recurrent draft depth 1
• Intended hardware: Apple Silicon

The MLX target weights are byte-identical to the pinned source revision. The MTP tensors were extracted from Qwen/Qwen3.5-9B at revision c202236235762e1c871ad0ccb60c8ee5ba337b9a and normalized with AX Engine's prepare_mtp_sidecar.py flow.

Download

hf download AutomatosX/AX-Qwen3.5-9B-MLX-4bit-MTP \
  --local-dir ./AX-Qwen3.5-9B-MLX-4bit-MTP

The repository is approximately 6.5 GB. Ensure that the destination has enough free space before downloading.

Use as a standard MLX model

Standard MLX-VLM inference loads the two model-*.safetensors target shards and ignores the AX-specific MTP sidecar:

pip install -U mlx-vlm

python -m mlx_vlm.generate \
  --model AutomatosX/AX-Qwen3.5-9B-MLX-4bit-MTP \
  --max-tokens 100 \
  --temperature 0.0 \
  --prompt "Describe this image." \
  --image <path-to-image>

Use with AX Engine MTP

Install AX Engine, download the repository, and serve the local directory:

ax-engine serve ./AX-Qwen3.5-9B-MLX-4bit-MTP --port 31418

The MTP package includes:

• mtp.safetensors: normalized 15-tensor MTP head
• mtplx_runtime.json: draft-depth and sampler guidance
• ax_mtp_sidecar_manifest.json: pinned source hashes and transformation provenance
• model-manifest.json: AX Engine native target manifest
• config.json: MLX target configuration with MTP sidecar fields

Validation and provenance

Validated on macOS arm64 with AX Engine 6.9.0 on 2026-07-20:

• AX native artifact validation: ready, with no issues
• All target weight shards: byte-exact against the pinned MLX source; the shard index keeps the upstream weight map and adds only the logical parameter count
• MTP tensor count: 15
• MTP draft depth: 1
• Norm transform: add 1 to MTP RMSNorm delta weights
• Exactness baseline: maximum absolute difference 0.0 at context length 2,048
• MTP provenance strict local re-hash: passed

The public provenance files use repository-relative paths and immutable Hugging Face URIs. They intentionally contain no local filesystem paths.

License

Apache License 2.0. See LICENSE and the original Qwen model card for model limitations and responsible-use guidance.