Laya MPS

Run Jev-style typed decisions locally on your Mac with low RAM usage and fast responses.

Laya delivers typed decisions with ~32 ms median latency using ~2.1 GiB RAM on M5 Pro, with a slower ~0.74 GiB mode for lower memory use.

MPS stands for Metal Performance Shaders, which PyTorch uses to run Laya on your Mac's GPU.

Laya typed-decisions chooses options, scores inputs, and estimates whether statements are true. The English model specializes in customer service, invoices, security incidents, and agent traces. It is not a general-purpose language model. Model comparison and benchmarks.

Requirements

• Apple Silicon Mac (M1 or newer), macOS 14+.
• Git and uv. uv installs Python 3.12 if needed.
• Allow 4 GB of free disk space for the runtime, download cache, and ~843 MB model.

Get started

Clone the repository and start the server:

git clone https://github.com/afshinm/laya-mps.git
cd laya-mps
./scripts/serve.sh

The first run installs dependencies and downloads the model. Open the demo and click Play or Run Benchmark. The benchmark runs for 60 seconds and plots response latency.

The server runs on 127.0.0.1:8000. Later runs use local files and inference works offline. Stop it with Ctrl+C. To update a clone, stop the server, run git pull, then run the script again.

Memory settings

All settings run the same complete model in FP32. Measured on an M5 Pro, 24 GiB RAM, macOS 26.4:

RAM is the peak across the benchmark suite, excluding macOS, the browser, and other apps. Latency uses fixed Pong inputs and excludes HTTP. All settings matched exactly on 260/260 decisions, including probabilities. Full measurements.

The default keeps transformer layers in RAM and reads embeddings from disk. minimal also streams layers from disk; full keeps all weights in RAM. To change the setting, stop the server and restart:

./scripts/serve.sh --memory minimal

Use the API

With the server running, choose a team for a support ticket:

curl --fail-with-body -sS http://127.0.0.1:8000/v1/decisions \
  -H 'Content-Type: application/json' \
  --data-raw '{
    "state": "The export page returns HTTP 500. Our team cannot finish its work.",
    "questions": {
      "owner": {
        "type": "choice",
        "instructions": "Which team should investigate this report?",
        "criteria": {
          "billing": "Payments and invoices",
          "engineering": "Software failures",
          "unknown": "Insufficient information"
        }
      }
    }
  }'

Example response:

{
  "model": "convaiinnovations/laya-typed-decisions",
  "answers": {
    "owner": {
      "type": "choice",
      "choice": "engineering",
      "probabilities": {
        "billing": 0.06904838234186172,
        "engineering": 0.6391704082489014,
        "unknown": 0.29178112745285034
      },
      "confidence": 0.24445859127951464
    }
  }
}

Check whether work is blocked and score the impact in one request:

curl --fail-with-body -sS http://127.0.0.1:8000/v1/decisions \
  -H 'Content-Type: application/json' \
  --data-raw '{
    "state": "The export page returns HTTP 500. Our team cannot finish its work.",
    "questions": {
      "blocked": {
        "type": "noul",
        "instructions": "Does the report say that work cannot proceed?"
      },
      "severity": {
        "type": "score",
        "instructions": "Rate the operational impact described in the report.",
        "levels": [
          "Cosmetic issue with no work affected",
          "Some inconvenience but work can proceed",
          "Work is blocked for the whole team"
        ]
      }
    }
  }'

Example response:

{
  "model": "convaiinnovations/laya-typed-decisions",
  "answers": {
    "blocked": {
      "type": "noul",
      "noul": 0.7066293954849243
    },
    "severity": {
      "type": "score",
      "score": 1.786898910999298,
      "probabilities": [
        0.019407516345381737,
        0.17428618669509888,
        0.8063063621520996
      ],
      "legend": {
        "0": "Cosmetic issue with no work affected",
        "1": "Some inconvenience but work can proceed",
        "2": "Work is blocked for the whole team"
      },
      "confidence": 0.4951949684505006
    }
  }
}

Responses contain model and typed answers. choice selects a label, noul is the probability of true (0–1), and score is a probability-weighted level number (0–2 in this example). Confidence describes how concentrated the probabilities are; it does not establish correctness. Evaluate the model on your own inputs before relying on it.

curl --fail-with-body -sS 'http://127.0.0.1:8000/v1/decisions?metrics=true' \
  -H 'Content-Type: application/json' \
  --data-raw '{
    "state": "The export page crashes. Our team cannot finish its work.",
    "questions": {
      "blocked": {
        "type": "noul",
        "instructions": "Does the report say that work cannot proceed?"
      }
    }
  }'

{
  "model": "convaiinnovations/laya-typed-decisions",
  "answers": {
    "blocked": {
      "type": "noul",
      "noul": 0.6566364169120789
    }
  },
  "metrics": {
    "request_ms": 20.240125013515353,
    "memory_bytes": 2068972792
  }
}

CLI and setup

Save the JSON payload from a curl example as request.json to evaluate it without starting a server:

uv run --locked laya-mps decide request.json --download
uv run --locked laya-mps decide request.json --metrics

Pass server options to scripts/serve.sh. Use --port 8001 if port 8000 is busy, --checkpoint english to evaluate the original English model, or --model-dir PATH to use another model directory. --device cpu is an explicit fallback; the published performance numbers use the Mac GPU. The default --question-batch-size 1 limits peak RAM. List all options with uv run --locked laya-mps serve --help.

uv run --locked laya-mps doctor
uv run --locked laya-mps setup
UV_OFFLINE=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 ./scripts/serve.sh

Development

Use Python 3.12 and Node.js 22+. From the repository root:

uv sync --locked
uv run --locked ruff check src tests benchmarks scripts
uv run --locked ruff format --check src tests benchmarks scripts
uv run --locked pytest -q
node --test tests/test_demo.mjs
uv build
uv run --no-sync python scripts/check_package.py

Tests use CPU fixtures and need no model download. Tokenization tests use the local tokenizer when available, otherwise a small test tokenizer. CI runs on macOS and tests the installed wheel and bundled demo assets. For GPU measurements, see benchmark reproduction.

uv.lock pins dependencies; laya_setup.py pins Laya source and model revisions. --locked fails if metadata and the lockfile disagree instead of updating dependencies. Format Python changes with ruff format. Inference changes must pass the full memory comparison; demo changes must update the shared asset revision so browsers cannot mix versions.

uv pip install --python .venv/bin/python --no-deps --reinstall-package laya-mps dist/laya_mps-0.1.0-py3-none-any.whl
uv run --no-sync laya-mps serve --help
uv run --no-sync pytest -q
uv sync --locked

License

MIT. Laya model weights and the bundled workflow benchmark data retain their Apache 2.0 license. Third-party notices.