MLX-RFSN Fusion Alpha 8.4

Status: MLX-RFSN Fusion Alpha 8.4 — Direct-Packed Correctness Release

Research alpha. MLX-RFSN is a KV-cache compression research and benchmarking system for Apple Silicon. It is not production-ready, not a general AI server, and not a full vector database.

Official promoted candidate: NONE
Best practical baseline: rfsn_direct_packed_k8v8
Promotion allowed: false (pending runtime-instrumented cache trace and token-sequence-hash provenance)

Release ID: alpha-8.4 (from release.toml)
Package Version: 10.2.0a84 (from release.toml)
Artifact Schema: 3.0 (from release.toml)

To verify the current state locally:

python -m compileall -q rfsn_v10 rfsn_v11 tests benchmarks scripts
bash scripts/release_gate.sh

---

Platform Support

---

Quick Start

# Three install modes — do not install everything at once

# Basic: core only, no MLX, no memory (any platform)
pip install -e ".[basic]"

# Fusion: MLX + KV compression benchmarks (macOS Apple Silicon)
pip install -e ".[fusion]"

# Memory: external vector memory (any platform)
pip install -e ".[memory]"

# Dev: everything for development
pip install -e ".[fusion,memory,dev]"

# Verify install
python -m rfsn_v10 version

# Health check
python -m rfsn_v10 healthcheck

# Validate config
python -m rfsn_v10 validate-config --config configs/default_runtime.yaml

# CPU-safe tests (no MLX required)
pytest tests/test_config.py tests/test_config_strict.py \
       tests/test_kernels_validation.py \
       tests/test_quantization_lazy_imports.py \
       tests/test_experimental_flags.py \
       tests/test_clickhouse_security.py \
       tests/test_no_runtime_raw_sdpa.py -q

# MLX-dependent tests (Apple Silicon required)
pytest tests/test_attention.py tests/test_bitpack.py \
       tests/test_bitpack_fuzz.py tests/test_kv_manager.py \
       tests/test_drift.py tests/test_attention_causal_mask.py \
       tests/test_short_prompt_decode_drift.py \
       tests/test_prefill_decode_split.py -q

# Central benchmark command
python benchmarks/kv_shootout.py --promotion-report

Inference Server

Run the OpenAI-compatible FastAPI server locally:

export RFSN_MODEL_ID=mlx-community/Llama-3-8B-Instruct-4bit
python -m rfsn_v10.server
# Or with uvicorn directly:
# uvicorn rfsn_v10.server.app:app --host 0.0.0.0 --port 8000

Test the endpoint:

curl http://localhost:8000/health

curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello"}],"stream":true}'

Or with Docker Compose (container validation + telemetry):

export CLICKHOUSE_PASSWORD=your-password
docker compose up -d
# Runs healthcheck and exits — inference server requires mlx/torch backend natively

---

Architecture

Stable Modules

• rfsn_v10/bitpack.py — Bit-packed quantizer (2–8 bit widths) with exact roundtrip guarantees
• rfsn_v10/kv_manager.py — TurboQuant KV manager with grouped symmetric quantization and WHT preconditioning
• rfsn_v10/attention.py — Adaptive block-sparse attention; all dense fallbacks route through attention_reference.py
• rfsn_v10/attention_reference.py — Canonical causal attention reference (always applies causal mask for T_q > 1)
• rfsn_v10/runtime/engine.py — Orchestrator integrating KV cache, sparse attention, audit mode, and telemetry
• rfsn_v10/runtime/__init__.py — Re-exports RFSNRuntime from engine.py
• rfsn_v10/runtime/generation.py — RFSNGenerator with prefill, decode, sampling, and telemetry
• rfsn_v10/config.py — Strict Pydantic config (extra='forbid' on all models)
• rfsn_v10/health.py — Health check system; returns UNHEALTHY until checks have been run
• rfsn_v10/clickhouse_client.py — HMAC-SHA256 prompt sanitization, recursive sanitizer, retry queue
• rfsn_v10/model_loader.py — Unified model/tokenizer loading (mlx-lm / transformers)
• rfsn_v10/server/app.py — FastAPI OpenAI-compatible server with SSE streaming

Experimental Modules (disabled by default)

All experimental paths require explicit opt-in. The runtime will not activate them silently — attempting to use an experimental feature without enabling it raises a RuntimeError.

# config.yaml
experimental:
  enable_qjl: false    # QJL score correction
  enable_polar: false  # Polar / hybrid quantization
  enable_adaptive: false  # Adaptive sparse controller

Or via environment:

RFSN_EXPERIMENTAL_QJL=true
RFSN_EXPERIMENTAL_POLAR=true
RFSN_EXPERIMENTAL_ADAPTIVE=true

Warning: Experimental features are not validated for production or quality-critical generation.

• rfsn_v10/quantization/polar_quant.py — Iterative hierarchical polar quantization
• rfsn_v10/quantization/hybrid_polar_cartesian.py — Hybrid polar-cartesian quantizer
• rfsn_v10/quantization/qjl_score_correction.py — QJL sketch-based score correction
• rfsn_v10/quantization/isoquant_precondition.py — IsoQuant quaternion preconditioner

---

Validated Stable Configs

Validated at the beta level on Apple Silicon. Quality thresholds (cosine ≥ 0.998 vs. FP32 reference) measured with tests/test_short_prompt_decode_drift.py.

---

Known Limitations

1. Sparse path is decode-focused. Prefill always uses dense attention with causal masking.
2. QJL correction is disabled by default. Experimental — not validated.
3. Experimental configs may degrade logits. Short-prompt drift under active investigation.
4. Full runtime requires MLX. Apple Silicon is mandatory for the core runtime.
5. Docker service mode is a research/local test server only. Not production-hardened. CLI health check only; no HTTP service exposed in default image.
6. CUDA not implemented. Do not depend on it.
7. End-to-end speedup not proven. Decode TPS is comparable; compression overhead makes total time slower at short contexts.
8. macOS x86 Metal not supported. MLX is ARM-only; Intel Mac users get NumPy backend only.

---

Security

• Telemetry prompt text is HMAC-SHA256 hashed before leaving the process boundary.
• Sanitization is recursive — nested dicts and message lists are also cleaned.
• RFSN_TELEMETRY_HMAC_KEY is required when events with sensitive fields are written. An absent key raises RuntimeError rather than silently hashing with an empty key.
• HTTPS is required for remote ClickHouse hosts; HTTP is only allowed for localhost.
• Retry queue stores (table, event) tuples — routing metadata is never mixed into event payloads.

---

Running Tests

# CPU-safe tests (no MLX required, runs on Linux CI)
RFSN_BACKEND=numpy pytest \
    tests/test_config.py \
    tests/test_config_strict.py \
    tests/test_kernels_validation.py \
    tests/test_quantization_lazy_imports.py \
    tests/test_experimental_flags.py \
    tests/test_clickhouse_security.py \
    tests/test_no_runtime_raw_sdpa.py -q

# MLX-dependent tests (Apple Silicon required)
pytest \
    tests/test_attention.py \
    tests/test_bitpack.py \
    tests/test_bitpack_fuzz.py \
    tests/test_kv_manager.py \
    tests/test_drift.py \
    tests/test_attention_causal_mask.py \
    tests/test_short_prompt_decode_drift.py \
    tests/test_prefill_decode_split.py -q

# Security tests
pytest tests/test_clickhouse_security.py \
       tests/test_clickhouse_routing.py \
       tests/test_tool_runner_security.py -q

# Full CI (mirrors .github/workflows/ci.yml linux-cpu job)
RFSN_BACKEND=numpy pytest --collect-only -q

---

Benchmarks

# Fast run (attention + bitpack)
python benchmarks/run_all.py --fast

# Full benchmark suite
python benchmarks/run_all.py

# Regression check against production_baseline.json
python benchmarks/run_all.py --check

Results are written to benchmarks/results/run_all_<timestamp>.json and benchmarks/results/latest.json.

---

Docker

docker build -t rfsn-qjl .

# Health check (Dockerfile default CMD now runs healthcheck)
docker run --rm rfsn-qjl

# With ClickHouse (production compose)
cp .env.example .env   # fill CLICKHOUSE_PASSWORD
docker-compose up

# With ClickHouse ports exposed (local dev only)
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

Requires Python 3.11. Core runtime does not function in Docker without Apple Silicon + MLX.

---

Environment Variables