jev-compatible-server

Run open decision models behind a Jev-compatible API.

Documentation · API · Models · Contributing · Issues

Quickstart

Install uv, then start the server with the Transformers backend:

uvx --from 'jev-compatible-server[transformers]' \
  jev-compatible-server --model bosun-v3.1-0.6b

Send a decision request to POST /v1/systemone:

curl http://localhost:8000/v1/systemone \
  --header 'content-type: application/json' \
  --data '{
    "state": "A customer says they were charged twice.",
    "questions": {
      "route": {
        "type": "choice",
        "instructions": "Which team should own this ticket?",
        "criteria": {
          "billing": "Payment, invoice, or refund problems",
          "technical": "Product bugs and technical failures"
        }
      }
    }
  }'

--model takes a bundled registry alias, such as bosun-v3.1-0.6b for Hanno-Labs/bosun-v3.1-0.6b. It downloads the model from Hugging Face when needed and loads it before the server accepts requests. Requests may omit model; the server rejects a request that names a different model. See the API reference for question and response types.

Installation

uvx installs the server into an isolated environment and runs it directly. Select the extra for the inference backend you need:

# Hugging Face Transformers models
uvx --from 'jev-compatible-server[transformers]' \
  jev-compatible-server --model bosun-v3.1-0.6b

# llama.cpp/GGUF models
DECISION_BACKEND=llama \
DECISION_MODEL_PATH=/path/to/model.gguf \
uvx --from 'jev-compatible-server[llama]' jev-compatible-server

Omit --model to run the bundled registry in multi-model mode. In that mode, each request may select a registry alias with its model field, and requests without one use the registry default (kev-4b). Custom GGUF models and registries require the environment described in the model recipe guide.

Description

jev-compatible-server is an open inference runtime for decision models. It accepts one shared state with one or more typed questions, runs the selected model through llama.cpp or Hugging Face Transformers, and returns normalized choice, score, and noul answers. Models that implement only part of that contract return an explicit unsupported result for each incompatible question without discarding compatible answers in the same request.

The HTTP interface implements Jev's POST /v1/systemone request and response shape so applications can move between hosted Jev and self-hosted models without replacing their decision API.

Goals

• Provide a common runtime for open decision models, as llama.cpp does for language models.
• Preserve the Jev API contract for straightforward application cutover.
• Keep model behavior declarative when an existing backend and readout can run it.
• Support multiple execution engines without coupling applications to model architecture.
• Make batching, model coverage, and compatibility behavior explicit and testable.

Supported backends

Backends execute the neural network; readouts convert model outputs into typed decision probabilities. See models and backends for supported checkpoints and the exact distinction.

Documentation

• API reference
• Supported models and backends
• Model recipes and registries
• Batching and performance

Contributing

Contributions for new backends, reusable readouts, model recipes, tests, and documentation are welcome. See CONTRIBUTING.md before making a change. Report bugs and request features in the central DecisionBench issue tracker.

Acknowledgements

jev-compatible-server builds on llama.cpp, llama-cpp-python, Transformers, FastAPI, and uv. It also depends on the authors who publish open decision-model checkpoints and document their readout contracts.