jev-serve

Typed probabilistic decisions from any LLM. No text generation.

jev-serve exposes a /v1/systemone endpoint that scores structured decisions via first-token logit readout — the same approach openjev.com uses in the browser, running server-side on any MLX model or OpenAI-compatible API.

• 34× faster than structured JSON generation (0.23s vs 7.80s per decision)
• Full probability distributions — not a point estimate, a calibrated p per option
• Three question types: choice (pick one), noul (0–1 probability), score (ordinal level)
• MLX by default: direct Apple Silicon inference; swap to any OpenAI-compatible API with --api
• Apache 2.0 — derived from kev by Jared Palmer

---

Install

uv sync --no-dev

Start

# MLX — direct inference on any local model (default backend)
jev-serve lmstudio-community/Qwen3.8-27B-MLX-6bit

# API — logit readout via ollama, LM Studio, OpenAI, etc.
jev-serve qwen/qwen3.8-27b --api http://localhost:11434/v1

Usage

curl -X POST http://localhost:8008/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": "2010 Infiniti G37 S, 6MT, daily driver, performance-oriented owner.",
    "questions": {
      "top_mod": {
        "type": "choice",
        "instructions": "What is the most popular upgrade category?",
        "criteria": {
          "exhaust": "Exhaust",
          "intake":  "Intake",
          "wheels":  "Wheels",
          "audio":   "Audio"
        }
      },
      "wants_power": {
        "type": "noul",
        "instructions": "Is this owner primarily interested in power gains?"
      }
    }
  }'

{
  "answers": {
    "top_mod": {
      "type": "choice",
      "choice": "exhaust",
      "confidence": 0.51,
      "probabilities": { "exhaust": 0.64, "intake": 0.03, "audio": 0.05, "wheels": 0.28 }
    },
    "wants_power": { "type": "noul", "noul": 0.87 }
  },
  "latency_ms": 190
}

Question types

Benchmark — Qwen3.8-27B · MLX · Apple M-series

20 products, same 4-option choice question, sequential, no batching.

Scratch mode runs one forward pass and reads a single logit position. The decoder generates tokens one by one until EOS, then parses JSON. The gap widens with option count and output length.

How it works

Instead of generating text, jev-serve reads the model's next-token logit distribution at the boundary position (after the prompt), picks out each option's first token, and normalizes with softmax. One forward pass per question; no sampling, no JSON parsing, no hallucinated fields.

This is identical to what openjev.com does in the browser with wllama.

Attribution

• openjev.com
• kev, Copyright 2026 Jared Palmer, Apache 2.0.