MLX Direct Decider — MLX decision engine without fine-tuning

Your existing compatible MLX model → a local decision engine.
No fine-tuning required. No extra decision-model weights.

MLX Direct Decider is an open-source Python library for training-free decisions from existing MLX models on Apple Silicon Macs. Use it for local LLM action selection, classification, semantic routing, and enum / boolean decisions.

Reuse a compatible MLX-LM checkpoint for text or an MLX-VLM checkpoint for experimental text + image decisions. Supply application state, a natural-language request and allowed choices; receive a stable candidate ID, no_match or abstain. Integrate through a Python API, CLI or local text HTTP service.

No additional fine-tuning, trained decision head or separate decision-model weights are required. The full model processes the prompt normally; the library reads candidate logits from its unchanged original output head without generating a continuation. Compatible base-model weights are still needed.

Quick start · Tested models · Game demo · Measured results · AnyJev & related projects · FAQ · Methodology · Connect & support

English · 简体中文

MLX action selection and semantic routing

MLX Direct Decider uses the current application state, a user utterance, and dynamic allowed choices to select a stable business ID with a local model. No-match and abstention are explicit outcomes.

Embed it in an existing tool: open a feature, select a visible item, pause a player, or answer a boolean / enum question. Inference runs on your Mac with an existing checkpoint; no training is required.

Project and package: The project is MLX Direct Decider. The distribution and CLI are mlx-direct-decider, with Python imports from mlx_direct_decider. Version 0.1 is experimental and has not been published to PyPI.

Models can make mistakes. Candidate scores rank choices and are not calibrated probabilities of correctness. State guards do not establish semantic correctness.

Quick start

Requires Apple Silicon, native ARM Python 3.11+, and a local MLX-LM checkpoint. Start with the verified Qwen checkpoint in the model table.

Clone the repository and install:

git clone https://github.com/ItsOdeLeo/MLX-Direct-Decider.git
cd MLX-Direct-Decider
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[mlx]'
export MLX_DIRECT_MODEL=/absolute/path/to/your/local/mlx-model

import os
from mlx_direct_decider import Candidate, DecisionRequest, MLXDecisionEngine

engine = MLXDecisionEngine(os.environ["MLX_DIRECT_MODEL"])
result = engine.decide(DecisionRequest(
    state={"focused_window": "notes"},
    utterance="Close it",
    candidates=(Candidate("close.notes", "Close the open notes window"),),
    state_version=1,
))

print(result.status, result.candidate_id)
print(result.margin, result.timing)

The CLI uses the same request contract:

mlx-direct-decider decide --request examples/decision.json

Results include the candidate ID, raw logits, candidate scores, margin, state version, model identity, actual timing, and cache details. See the Python API for booleans, state updates, and versioned execution.

MLX, MLX-LM and MLX-VLM support

• MLX: shared compute foundation; no separate Metal backend is needed.
• MLX-LM: text decisions through MLXDecisionEngine, with bounded prefix reuse.
• MLX-VLM: text + image decisions through MLXVLMDecisionEngine or mlx-direct-decider decide --backend mlx-vlm --image ..., without fine-tuning.

Install image support with python -m pip install -e '.[mlx,vlm]'. Image quick start and real-model checks include Qwen3-VL, SmolVLM's mistakes, and the Gemma checkpoint loading limitation. VLM requests currently recompute the full context; the HTTP/game demos still use text state. See the model guide for checkpoint-specific text coverage.

A local model plays Blocks

Qwen3.5-9B · 20 model-selected placements · 6 cleared rows · 600 points. Freshly recorded with the MLX Direct Decider name on October 14, 2026. The GIF shows the complete run at 4× playback, including inference waits. The model chooses from every legal vertical-drop placement using structured board data and rule-computed outcomes; there is no best-move fallback. This is turn-based play, not screenshot understanding or frame-by-frame control.

Run the game locally · Full-speed video · All decisions

This single illustrative development run reached its planned 20-piece limit without retries, browser errors or recording errors. Earlier attempts, including the initial refusal, remain in the historical game report. This is not a held-out game benchmark or a speed claim. Manual play can run on a static server; new AI decisions require the local MLX backend.

All locally tested model checkpoints

11 distinct checkpoints tested locally as of 2026-10-13 · Apple M2 Max · 64 GiB. All use existing model weights, with no additional fine-tuning or decision-specific weights. The table records direct candidate scoring; completed runtime checks and decision accuracy are separate facts.

The 36-case frozen evaluation, 3-case development checks and 7-case synthetic vision checks use different inputs and should be read within their own scope. The fully evaluated models also returned wrong actions; small smoke checks do not establish general reliability or compatibility with every model in a family.

Text model details and evidence · Image model details and evidence · Full 36-case results

Measurements, with their limits

New six-domain evaluation · Apple M2 Max · 64 GiB · 36 fictional English cases.

Documents, calendar drafts, files, music queues, product comparisons, and settings. Three models receive the same frozen inputs, semantic prompt, and threshold, with four output methods each. These are the direct-scoring results:

The 36 cases contain 30 enum requests and six boolean decisions; action errors count enum requests only. Timings require loaded weights and a reusable page prefix. They do not describe startup or arbitrary new pages. See the extended report for every method, rejection and coverage rates, boolean quality, memory, and failures.

Gemma selected the wrong queue item; Qwen selected the wrong longest-battery product. GLM made more action errors. The earlier small sample's zero-error observation did not carry over to new cases. These results do not establish reliable unattended actions, arbitrary-model compatibility, or fixed 100 ms performance.

• 161 core tests passed: contracts, state updates, stale decisions, single-use authorization, HTTP, and related behavior. These do not measure model semantics.
• 168 / 168 original-suite cache comparisons passed: three models × 28 cases × two reuse conditions, each compared with fresh computation.
• 72 / 72 extended-suite cache comparisons passed: Gemma's 36 new cases × two reuse conditions, with observed maximum logit and score differences of 0. Numerical agreement does not establish semantic correctness.
• 16 recorded browser scenarios checked: actual DOM clicks and execution receipts. Decline cases accept either rejection outcome, unlike the strict quality set above.

How a choice is made

Application state + utterance + allowed choices
                       │
               Official MLX-LM model
                       │
           Final-position candidate logits
                       │
          selected(id) / no_match / abstain
                       │
       Application version check → execution → receipt

Choices map to tokenizer-verified single-token codes, then back to business IDs. Direct scoring reads a causal model's next-token logits without generating a JSON continuation. It retains the official quantized output head. Hybrid caches reuse only complete, valid prefix boundaries.

See the architecture and framework audit. Version 0.1 focuses on English, single-turn, single-step choices. Experimental text + image decisions use the separate MLX-VLM API. General chat, multi-step planning, arbitrary arguments, training, and GPU batching are outside its scope. Chinese documentation does not mean Chinese model behavior has been validated.

Related decision projects: AnyJev, OpenDecider, and Laya

Developers evaluating AnyJev, OpenDecider, Laya, or Laya-MLX may also be looking for local action selection, classification, or semantic routing. MLX Direct Decider focuses on existing compatible causal MLX-LM checkpoints on Apple Silicon, with no fine-tuning or separate decision-model weights. Application-defined choices are scored through the original model output head. AnyJev also offers a path without training; the implementations and score semantics differ.

Read the AnyJev, OpenDecider, and Laya comparison for source snapshots, integration boundaries, and evaluation considerations. These are related projects, not bundled backends or vendor partnerships.

Frequently asked questions

How does MLX Direct Decider use MLX, MLX-LM and MLX-VLM?

MLX provides the array and compute framework; MLX-LM runs text models, and the optional MLX-VLM backend runs text/image models. Metal execution is handled by MLX. Image decisions use fresh context; the existing text backend retains prefix reuse. MLX Direct Decider adds application-supplied choices, candidate scoring, rejection outcomes, and versioned execution. Direct decisions read next-token logits for verified single-token option codes without generating a JSON continuation. See the architecture.

The full model still processes the input. Reading final-position logits does not mean running only the final layer or replacing the output head.

Can I use MLX Direct Decider as a local semantic router?

Yes. Describe the currently allowed routes or actions as candidates with stable business IDs, supply the application state and user request, and inspect the selected ID. The host application defines and executes the actions. no_match and abstain handle unsuitable or ambiguous requests. See the Python API.

How is MLX Direct Decider different from AnyJev?

Both reuse existing models without additional training. MLX Direct Decider focuses on native MLX-LM and experimental MLX-VLM inference on Apple Silicon, with dynamic action IDs, prefix reuse for text, and versioned execution. AnyJev's Decider adds option-rotation and label-prior corrections. MLX Direct Decider uses direct candidate scoring; its scores are not calibrated correctness probabilities. See the MLX Direct Decider vs AnyJev comparison.

How does MLX Direct Decider compare with AnyJev, OpenDecider, and Laya-MLX?

MLX Direct Decider adds bounded action selection to existing compatible causal MLX-LM checkpoints without fine-tuning or decision-specific weights. AnyJev's Decider also reuses existing LLMs without training and adds corrections for option ordering and label priors. OpenDecider and Laya publish trained decision-model families, and Laya-MLX runs Laya weights. Their users can load published checkpoints without training them themselves. See the decision-model comparison.

Does installing MLX Direct Decider make every MacBook model compatible?

Point MLX Direct Decider at a compatible local MLX-LM checkpoint using --model or MLX_DIRECT_MODEL. Its model loader is shared across architectures; each checkpoint still needs a supported model implementation, compatible tokenizer and chat template, accessible logits, and enough memory. Installation does not attach it to other apps or load GGUF models from Ollama. The compatibility guide separates full evaluations, small runtime checks, and untested families.

Which MLX models have been evaluated?

The homepage model table lists all 11 locally tested checkpoints across Qwen, GLM, Gemma, NVIDIA Nemotron, Llama, Mistral and SmolVLM, including text/image paths and loading failures. The text compatibility guide and multimodal guide retain exact versions, complete results and limits. Full evaluations and small smoke checks are identified separately.

Do I need fine-tuning, separate decision-model weights or a cloud API?

No. Reuse a compatible local MLX-LM checkpoint, or an MLX-VLM checkpoint for experimental image decisions, on an Apple Silicon Mac. No cloud API, extra training, fine-tuning, new decision head, or decision-specific checkpoint is required. If you already have compatible base weights, reuse them. Otherwise obtain those base weights separately and pass their local directory to the engine; the package does not download weights.

How do I install MLX Direct Decider?

Clone ItsOdeLeo/MLX-Direct-Decider, create a native ARM Python 3.11+ environment, and run python -m pip install -e '.[mlx]' from the checkout. Set MLX_DIRECT_MODEL to your local checkpoint directory. Follow the quick start for a runnable example. Version 0.1 is experimental and has not been published to PyPI.

How fast is local MLX action selection?

Latency depends on the checkpoint, hardware, input, and reusable prefix. The measured results distinguish loaded weights with a reusable page prefix from startup, cold KV, and page updates. Warm measurements are not a fixed latency guarantee or evidence of reliable unattended execution.

Is MLX Direct Decider an official JEV implementation or a general chat assistant?

MLX Direct Decider is an independent JEV-inspired project at ItsOdeLeo/MLX-Direct-Decider. It uses existing MLX-LM models, not JEV weights, and does not reproduce unpublished JEV training. Version 0.1 supports English, single-turn, single-step choices; general chat and multi-step planning are outside its scope. Experimental text/image choices are documented in the multimodal guide.

Documentation

python -m pip install -e '.[dev]'
python scripts/check_docs.py
python -m pytest -m 'not model'
python -m ruff check src tests benchmarks scripts examples

Connect & support

Share use cases, reproduction results, and suggestions. Issues, documentation fixes, and new public test cases also support the project.

欧的Leo

X / Twitter · @ItsOdeLeo ↗ · Xiaohongshu · 欧的Leo ↗

Xiaohongshu ID: ItsOdeLeo

_{Support the project with a star, a share, or a pull request.}

License & inspiration

MIT. Independently maintained, inspired by TypeSafe AI's JEV, and built on official MLX-LM. No affiliation or endorsement, no JEV weights, and no claim to reproduce unpublished RLCD.

Models and dependencies retain their own licenses; see NOTICE. Recorded evidence retains former project names, source paths, and hashes as explained in results. See the naming research for the candidate name.

_{Local decisions. Visible evidence. Open source.}