MLX Studio — native JANG support with reasoning, VLM, and full mixed-precision inference

---

JANG Studio — native macOS wizard for converting HuggingFace models to JANG and JANGTQ formats. Five-step flow with live logs, 10-row preflight gate, 12-row post-convert verifier, in-app chat preview, and one-click HuggingFace publishing. Ships as a signed DMG.

See the full user guide for install + wizard walkthrough + CLI usage + Swift integration.

---

App Support: MLX Studio has full native JANG support. oMLX has added JANG integration (PR #364). LM Studio, Ollama, and Inferencer do not support JANG yet — ask your favorite app's creators to add support, or use pip install "jang[mlx]".

---

Jang Adaptive N-bit Grading

Mixed-Precision Quantization for Apple Silicon

The GGUF equivalent for MLX — models stay quantized in GPU memory at full Metal speed.

Website • Models • PyPI • Format Spec

Highlights

• 397B on 128 GB Mac — JANG_1L: 112 GB, 36 tok/s, 86.5% MMLU with reasoning
• DeepSeek-V4-Flash runtime — first MLX runtime for DSV4's hybrid SWA + CSA + HSA attention (sliding-window + compressor-pool + hash-sparse) with mHC residuals and 1M-context pool quant cache
• JangPress for routed-MoE — load 167 GB Kimi-K2.6 on a 128 GB Mac via mmap + madvise cold-tier eviction (see docs/JANGPRESS.md)
• Nemotron-Cascade-2 in 10 GB — IMO Gold Medal reasoning model at 130 tok/s on 16 GB MacBooks
• MiniMax: only JANG works — MLX scores 25% (random), JANG scores 74%
• Nemotron-3-Super-120B in 43 GB — first working Nemotron-H quantization for Apple Silicon
• Mistral Small 4 (119B) — first on Apple Silicon — 82 tok/s, 5x faster prefill than MLX Community 4-bit, code/math/reasoning/VLM all working
• bfloat16 auto-detection — fixes float16 overflow on 512-expert models
• Reasoning mode — <think>...</think> / [THINK]...[/THINK] with configurable thinking on/off

Results (200-question MMLU)

Qwen3.5-397B-A17B — JANG runs where MLX can't

MLX cannot quantize 397B below 4-bit (float16 overflow). JANG solves this with bfloat16.

Nemotron-Cascade-2-30B — IMO Gold Medal in 10 GB

JANG_4M beats MLX 4-bit (93.0% vs 92.5%) at the same size.

Mistral Small 4 (119B-A6B) — First on Apple Silicon, fastest inference

94.0% MMLU at 2-bit (JANG_2L) — five subjects at 100%. 5x faster prefill than MLX Community. JANG_2L fits on 48 GB Macs at half the size.

119B total parameters, 6B active per token. MLA attention + 128 MoE experts + Pixtral vision. First model to support MLA + MoE quantization on Apple Silicon.

JANG_2L MMLU Results (200 Questions, Reasoning Mode)

Nemotron-3-Super-120B — Only JANG can go below 4-bit

MLX mlx_lm.convert crashes on Nemotron's mtp.* weights. Only JANG can produce sub-4-bit.

MiniMax-M2.5 — JANG is the ONLY working option

MLX is broken on MiniMax at ALL bit levels (~25% = random). MiniMax has 256 experts — MLX compresses attention to the same bits as expert MLP, destroying coherence.

Qwen3.5 MoE (122B, 35B)

The Full Picture: JANG vs MLX Across All Models

JANG wins at every size point. At equivalent sizes, JANG matches or beats MLX. At smaller sizes, JANG runs where MLX literally cannot (NaN, crashes, or random output).

Why MLX Fails on MoE Models

On MoE models, attention is only 1-5% of total parameters but controls 100% of coherence. MLX compresses everything equally:

MLX 4-bit: attention at 4-bit, experts at 4-bit → works but wastes bits on experts
MLX 2-bit: attention at 2-bit, experts at 2-bit → attention breaks → model breaks

JANG 2-bit: attention at 8-bit, experts at 2-bit → attention preserved → model works

The more experts a model has, the worse MLX performs at low bits:

• 128 experts (Cascade-2): MLX 4-bit still works, JANG slightly better
• 256 experts (122B, MiniMax): MLX 2-bit breaks badly, JANG dominates
• 512 experts (397B, Super-120B): MLX NaN/crash below 4-bit, only JANG works

DeepSeek-V4-Flash runtime — Hybrid SWA + CSA + HSA attention

DSV4-Flash is the first DeepSeek-class model whose attention is not plain MLA. JANG ships the only Apple-Silicon runtime that handles all three layer types correctly:

End-to-end verified: 1M-context pool accumulation, multi-turn /v1/chat/completions with prefix-cache reuse, dual-mode reasoning, 20.5 tok/s decode on M3 Ultra @ JANGTQ2 (79.5 GB), 24.5 tok/s on Swift.

JangPress — fit routed-MoE bundles bigger than RAM

JangPress is the load-time memory policy in osaurus-ai/vmlx-swift-lm that combines mmap-backed safetensors, per-token router-aware MADV_DONTNEED over canonical routed-expert pages, and an optional prestack overlay. It lets routed-MoE bundles bigger than RAM serve from a single Mac:

Python serve + bench scripts: scripts/jangpress/. Full guide: docs/JANGPRESS.md.

Install

pip install "jang[mlx]>=2.5.18"

For Vision-Language models:

pip install "jang[vlm]>=2.5.18"

Quick Start

Convert any model

# K-quant 4-bit (same size as MLX, smarter allocation)
jang convert Qwen/Qwen3.5-35B-A3B -p 4

# 2-bit for extreme compression
jang convert Qwen/Qwen3.5-122B-A10B -p 2

# Specific profile
jang convert model -p JANG_2L

Run inference

from jang_tools.loader import load_jang_model
from mlx_lm import generate

model, tokenizer = load_jang_model("JANGQ-AI/Qwen3.5-397B-A17B-JANG_1L")

# With reasoning (recommended for hard questions)
messages = [{"role": "user", "content": "Prove that sqrt(2) is irrational."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False,
    add_generation_prompt=True, enable_thinking=True)
result = generate(model, tokenizer, prompt=prompt, max_tokens=2048)

# Without reasoning (faster)
prompt = tokenizer.apply_chat_template(messages, tokenize=False,
    add_generation_prompt=True, enable_thinking=False)
result = generate(model, tokenizer, prompt=prompt, max_tokens=100)

VLM (Vision-Language) inference

from jang_tools.loader import load_jang_vlm_model
from mlx_vlm import generate as vlm_generate

model, processor = load_jang_vlm_model("JANGQ-AI/Qwen3.5-397B-A17B-JANG_2L")
messages = [{"role": "user", "content": [
    {"type": "image"},
    {"type": "text", "text": "Describe this image."},
]}]
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
result = vlm_generate(model, processor, prompt=prompt, image=["photo.jpg"], max_tokens=200)

MMLU Benchmark

python -m jang_tools.benchmark /path/to/model --max-thinking 1024

Smart two-pass: no-thinking first, then reasoning retry on wrong answers. Checkpointing, forced answers, full output logging.

Pre-quantized Models

* with reasoning mode

Full collection

Profiles

App Developers: Add JANG Support

JANG models are standard MLX safetensors. If your app loads MLX quantized models, adding JANG is minimal work.

Quickest Integration (5 lines)

# Detect JANG model
from pathlib import Path
is_jang = (Path(model_path) / "jang_config.json").exists()

# Load with jang-tools
if is_jang:
    from jang_tools.loader import load_jang_model
    model, tokenizer = load_jang_model(model_path)
    # model is a standard mlx_lm model — use like any MLX model

What's Different from Standard MLX

1. Mixed bit widths — different tensors have different bits (attention at 8-bit, experts at 2-bit). Each QuantizedLinear needs its bits and group_size set from tensor shapes.
2. bfloat16 for large models — 512+ expert models need model.set_dtype(mx.bfloat16) to prevent float16 overflow.
3. Nemotron-H weight renaming — switch_mlp.up_proj→fc1, down_proj→fc2, gate dequantization.

Full Integration Guide

See INTEGRATION.md for complete step-by-step with code for:

• Loading without jang-tools dependency
• Per-tensor bit inference from shapes
• bfloat16 auto-detection
• Nemotron-H special handling
• Chat template with thinking on/off
• VLM support
• Edge cases and gotchas

Supported Architectures

• Qwen3.5 (hybrid SSM + MoE + VLM) — 4B, 9B, 27B, 35B, 122B, 397B
• Nemotron-H (Mamba-2 + Latent MoE + Attention) — Cascade-2 30B, Super-120B
• MiniMax-M2.5 (256-expert MoE, FP8 source)
• DeepSeek-V2/V3 (MLA + MoE)
• Mixtral / Qwen2-MoE (standard MoE)
• Dense Transformers (Llama, Mistral, Gemma, Phi)
• Vision-Language (Qwen3.5-VL, Pixtral)
• Mamba / Hybrid SSM (Jamba, Nemotron-H)
• FP8 source models (auto-dequantization)
• Mistral Small 4 (119B MoE + MLA + Pixtral VL) — 94% MMLU, 82 tok/s, 30-84 GB

Changelog

v2.1.5 (2026-03-21)

• Nemotron-H loader: fc1/fc2 rename, gate weight dequantization, mtp.* key filtering
• bfloat16 auto-detection for 512+ expert models (prevents float16 overflow)
• MLP asymmetry floors: gate_proj=4-bit, down_proj=3-bit for 512+ expert models
• Benchmark script: smart two-pass MMLU with reasoning, checkpointing, forced answers
• eos_token_id auto-fix for Qwen3.5 (248044→248046)
• Auto-copy all .py files for trust_remote_code models
• Nemotron-3-Super-120B: 86% MMLU at 43 GB
• Qwen3.5-397B: 92% MMLU at 187 GB, 86.5% at 112 GB

v2.1.4 (2026-03-19)

• MLP asymmetry fix for 512-expert models
• eos_token_id auto-fix for Qwen3.5
• Auto-copy custom .py files

v2.1.3 (2026-03-18)

• Per-tensor group_size (router=64, experts=128 for 150+ expert models)
• Precision floor rules for shared expert
• VLM support for all Qwen3.5 models

How It Works

JANG redistributes bits based on tensor sensitivity — same total size, smarter allocation:

CRITICAL  (attention, MoE routers, MLA latent)  →  6-8 bit  →  Controls coherence
IMPORTANT (embeddings, linear attention)         →  4-6 bit  →  Moderate sensitivity
COMPRESS  (MLP, MoE experts)                     →  2-4 bit  →  95%+ of parameters

On MoE models, attention is only 1-5% of parameters. Boosting it to 8-bit costs ~2% overhead but dramatically improves quality. MLX compresses everything equally — that's why it breaks on MoE models at low bits.

Technical Features

• bfloat16 compute: Auto-detected for 512+ expert models. Prevents float16 overflow at shared expert down_proj.
• MLP asymmetry: gate_proj gets 4-bit floor (SiLU amplifier), down_proj gets 3-bit floor for 512+ expert models.
• FP8 dequantization: Handles FP8 source models (MiniMax, Nemotron) automatically.
• Latent MoE: Supports Nemotron-H's fc1/fc2_latent_proj compression.
• v2 format: MLX-native safetensors, instant mmap loading, no repack needed.

Requirements

• Python: 3.11+
• Conversion: any platform (numpy + safetensors)
• Inference: Apple Silicon Mac (M1/M2/M3/M4) with MLX
• Dependencies: safetensors>=0.4, numpy>=1.24, tqdm>=4.60, huggingface_hub>=0.20
• Optional: mlx>=0.22, mlx-lm>=0.20 (inference), mlx-vlm>=0.1 (VLM)

Links

• GitHub | HuggingFace | MLX Studio | PyPI | Format Spec

---

한국어

JANG은 Apple Silicon을 위한 혼합정밀도 양자화 포맷입니다. MLX를 위한 GGUF.

* 추론 모드 사용

pip install "jang[mlx]>=2.1.5"

GitHub · HuggingFace · MLX Studio · PyPI

---

장진호 제작 · Created by Jinho Jang — jangq.ai