AX Qwen3 Embedding 8B MLX 4-bit DWQ

Parameter count: approximately 7.57B logical parameters (8B class). 4-bit is the DWQ quantization precision, not a 4B model-size claim.

This is an MLX text-embedding model for Apple Silicon. The weight tensor payload, configuration, and tokenizer are byte-identical to mlx-community/Qwen3-Embedding-8B-4bit-DWQ at revision 885642d6b98742ea03b77a1673579c92ca961efd.

AutomatosX adds an AX Engine native manifest, pinned provenance, tested serving instructions, and this model card. Only Safetensors header metadata was added to report the logical parameter count to the Hub; tensor payload bytes were not changed or re-quantized. This is an embedding-only release and does not use MTP.

Model details

• Base model: Qwen/Qwen3-Embedding-8B
• Format: MLX Safetensors
• Quantization: 4-bit DWQ, group size 64
• Embedding dimension: 4,096
• Transformer layers: 36
• Configured context limit: 40,960 tokens
• Intended hardware: Apple Silicon

Download

hf download AutomatosX/AX-Qwen3-Embedding-8B-MLX-4bit-DWQ \
  --local-dir ./AX-Qwen3-Embedding-8B-MLX-4bit-DWQ

Serve with AX Engine

Install AX Engine, then serve the downloaded directory:

ax-engine serve ./AX-Qwen3-Embedding-8B-MLX-4bit-DWQ \
  --port 31418 -- --model-id qwen3

Create full-size normalized embeddings:

curl http://127.0.0.1:31418/v1/embeddings \
  -H 'Content-Type: application/json' \
  --data '{
    "model": "qwen3",
    "input": [
      "Instruct: Find passages that answer this query.\nQuery: What is the capital of Canada?",
      "Ottawa is the capital city of Canada."
    ],
    "encoding_format": "float",
    "pooling": "last",
    "normalize": true
  }'

For text inputs, AX Engine appends the configured EOS token before last-token pooling. Add an English task instruction to retrieval queries when useful; documents normally remain unprefixed. The endpoint returns 4,096-dimensional vectors. Do not pass the OpenAI dimensions field to this AX Engine version.

Validation and provenance

The release was validated on macOS arm64 with AX Engine 6.9.0:

• AX native artifact validation: ready, with no issues
• Live POST /v1/embeddings batch: passed
• Returned dimensions: 4,096
• L2-normalized vector norms: approximately 1.0
• Upstream tensor payload, configuration, and tokenizer: byte-exact; only the Safetensors header adds the corrected logical parameter count

See ax_provenance.json for pinned source and SHA-256 values.

License

Apache License 2.0. See LICENSE and the upstream Qwen model card for model limitations and responsible-use guidance.