C-RADIOv4 Quantized MLX Bundles

This repository contains self-contained MLX bundles for NVIDIA C-RADIOv4 image embedding models on Apple Silicon.

Implementation repository:

https://github.com/stephansturges/c-radio_v4_MLX

Repository Structure

Each subdirectory contains:

• model.safetensors
• manifest.json
• upstream config and preprocessor metadata
• a subdirectory README.md with model-specific provenance, measurements, and usage

Source Models

• nvidia/C-RADIOv4-SO400M: https://huggingface.co/nvidia/C-RADIOv4-SO400M
  • Revision: c0457f5dc26ca145f954cd4fc5bb6114e5705ad8
• nvidia/C-RADIOv4-H: https://huggingface.co/nvidia/C-RADIOv4-H
  • Revision: 0057b339059c0b9e1b4ba996f975410ebbfdfcc8

Accuracy Summary

Measured against local bf16 MLX bundles at 512x512 on 12 WALDO crop images.

The 8-bit affine bundles are the recommended compact/high-precision artifacts, not the throughput tier. They are packed weight-only MLX artifacts and are not dequantized back to dense bf16 at load time. Cider W8A8 is the real weight/activation low-bit runtime path for Apple M5+ machines and trades a little more embedding drift for lower memory and modest speedups in some cells. The mxfp8 bundles are included for experimentation and are lower precision in these checks.

Smoke-image Cider W8A8 precision versus local bf16 MLX at 512x512:

WALDO 12-image Cider W8A8 precision versus local bf16 MLX at 512x512:

Speed Summary

MLX measurements on Apple M5 Max at 512x512, batch 1:

There are no supported dequantize-at-load artifacts in this repository. The MLX affine and mxfp8 bundles keep weights packed but are weight-only, so they prioritize compact storage and lower runtime weight memory over throughput. Cider W8A8 is the faster weight/activation low-bit runtime path found so far, but it requires Apple M5+ hardware and the optional Cider package.

Usage

Install the implementation package from the GitHub repository, then point --checkpoint at one of the downloaded subdirectories:

cradio-mlx embed \
  --backend mlx-so400m \
  --checkpoint so400m/8bit-affine \
  --image image.jpg \
  --image-size 512 \
  --dtype bfloat16 \
  --save-npz embedding.npz

The g128 variants are the balanced Cider choice. The p99.99 variants are faster and slightly lower precision; validate them against downstream task metrics before replacing bf16 or g128.

Use --backend mlx-h for the H model:

cradio-mlx embed \
  --backend mlx-h \
  --checkpoint h/8bit-affine \
  --image image.jpg \
  --image-size 512 \
  --dtype bfloat16 \
  --save-npz embedding.npz

Cider W8A8 bundles require Python >=3.12, Apple M5+ hardware, and Cider:

python -m pip install "cider @ git+https://github.com/Mininglamp-AI/cider.git"
cradio-mlx embed \
  --backend mlx-h \
  --checkpoint h/cider-w8a8 \
  --image image.jpg \
  --image-size 512 \
  --dtype bfloat16 \
  --save-npz embedding.npz

License

The implementation code in c-radio_v4_MLX is MIT licensed. The model weights and these converted bundles are governed by NVIDIA's Open Model License Agreement, not by the MIT license. Preserve NVIDIA provenance and license terms when redistributing these bundles.

NVIDIA Open Model License Agreement:

https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf