Lamina

Lamina runs an AI model on your own Windows PC. You can chat in a terminal or connect a coding app such as pi. After the initial downloads, inference runs locally.

It supports Qwen3.6-35B-A3B and Ornith-1.5-35B-A3B. Each model is about 22 GB. Lamina keeps frequently used weights on the graphics card and computes the rest from system RAM, allowing these models to run with 8 GB of VRAM. It is a native C++/CUDA port of Strata.

What you need

The portable release includes Python, the engine and its runtime libraries. You need Windows 10/11 x64, an AVX2-capable CPU (with FMA/F16C/BMI2), an NVIDIA RTX 30/40/50 GPU with at least 8 GB VRAM and a CUDA 13.3-compatible driver, 64 GB RAM recommended, and about 40 GB of free disk space. Internet is needed for the initial model download. No Visual Studio, Git, Python installation or CUDA toolkit is needed to use the portable ZIP.

Quick start

Download lamina-windows-x64-portable.zip from GitHub Releases, extract it into a writable folder, then double-click START-HERE.bat. Extract the whole ZIP; do not run the launcher inside it.

On the first run:

1. Choose a model: Qwen3.6 or Ornith. Both work in the portable ZIP.
2. Choose how much conversation history to support: 8K, 32K, 64K or 128K.
3. Choose whether to enable MTP, an optional generation speedup.
4. Let setup download the model and load it. The first download is large; wait until you see Lamina is ready.

Your choices and downloads are saved in the neighboring Lamina-data folder. Later launches reuse them and start the server without asking again. The portable ZIP does not require compiling anything.

 Lamina is ready.

   Base URL : http://127.0.0.1:8000/v1
   Model    : Qwen3.6-35B-A3B-UD-Q4_K_M
   API key  : anything (it is not checked)

Leave the window open while using the model; close it or press Ctrl+C to stop. Connect a coding agent such as pi or another OpenAI-compatible app. If you simply want to chat, open a terminal in the extracted folder and run:

.\START-HERE.bat chat

Type your message at You:. Use /new to clear the conversation and /exit to quit. To open a terminal in the folder, right-click its empty space in File Explorer and choose Open in Terminal, or open PowerShell and cd there. The portable release includes text, tools, reasoning and image input for both models. Setup automatically downloads the selected model's image projector (about 900 MB). Apps can send screenshots or photos through the API using OpenAI-compatible image_url content, including base64 data URLs. The image encoder runs on the CPU, so large images can take longer to process. On older CPUs, an image response can take a minute or more. Text-only requests do not start the image encoder.

If using a source checkout, install Python 3.11 or newer and run START-HERE.bat. Setup installs runtime Python packages and downloads the pinned prebuilt engine. To compile locally, use START-HERE.bat setup --build-source; that requires the C++ build tools described in the manual setup. See Windows releases for packaging and validation.

To switch to Ornith, run:

.\START-HERE.bat --model ornith

Setup downloads Ornith, its tokenizer and image projector, and prepares its MTP head automatically. No compiler is needed.

Setup saves Ornith as your model choice. You can switch back later without deleting either model's downloaded files.

Choosing and changing your settings

Lamina saves model, context length and MTP in ..\Lamina-data\quickstart.json. You do not need to edit that file. To answer all three questions again, stop the running server and run:

.\START-HERE.bat --reconfigure

Press Enter at a question to keep its current setting. Your new choices are saved for future launches; setup downloads anything missing, then starts the server. To change settings without starting it, use .\START-HERE.bat setup --reconfigure.

Model: choose Qwen3.6 or Ornith. You can also switch directly with .\START-HERE.bat --model ornith or .\START-HERE.bat --model qwen3.6. An explicit --model skips the model question, even with --reconfigure.

Context length is how much text one conversation can hold: your messages, the model's answers and any pasted documents together. A token is roughly three quarters of an English word.

The context budget includes the answer you ask for. Choosing 128K does not make every small request ingest 128K tokens: memory and work grow with the actual conversation. The server reuses unchanged history on follow-up turns, so it usually needs to read only the added messages and tool results.

MTP speculative decoding uses a small extra part of the model (its multi-token-prediction head) to guess the next token in advance. The model checks the guesses before accepting them. This can make generation faster; the gain varies by model, hardware and context length. It applies to requests with temperature 0, the default. Setup prepares the model's prediction head, which uses some extra GPU memory. Saying yes is a reasonable starting choice; you can turn it off with --reconfigure.

Everyday commands

Using the server

.\START-HERE.bat starts an OpenAI-compatible HTTP server. Any client that supports a custom OpenAI base URL can use it, with base URL http://127.0.0.1:8000/v1, any API key, and the model name Qwen3.6-35B-A3B-UD-Q4_K_M. Copy the connection details printed under Lamina is ready into your app. The current API uses that same model ID when serving Ornith; your saved model choice determines the loaded weights. A quick test from a second PowerShell window:

$body = @{ model = "Qwen3.6-35B-A3B-UD-Q4_K_M"; messages = @(@{ role = "user"; content = "Name three primary colors." }) } | ConvertTo-Json -Depth 4
(Invoke-RestMethod http://127.0.0.1:8000/v1/chat/completions -Method Post -ContentType "application/json" -Body $body).choices[0].message.content

The server supports streaming, temperature, top_p, top_k, seed, stop, max_tokens, reasoning ("thinking") controls, function tools and JSON output. It handles one request at a time. Speculative decoding speeds up requests with temperature 0; other requests run at the normal speed.

Connecting pi and other apps

Start the server with .\START-HERE.bat and leave its window open. Then add Lamina to the app as an OpenAI-compatible provider. For the pi coding agent (needs Node.js 22.19 or newer and Git for Windows), put this in ~\.pi\agent\models.json, creating the file if needed. The example is for the 131072-token (128K) context and lets one answer use up to 32768 tokens:

{
  "providers": {
    "lamina": {
      "baseUrl": "http://127.0.0.1:8000/v1",
      "api": "openai-completions",
      "apiKey": "lamina",
      "models": [{
        "id": "Qwen3.6-35B-A3B-UD-Q4_K_M",
        "name": "Qwen3.6-35B-A3B (Lamina)",
        "reasoning": true,
        "input": ["text", "image"],
        "contextWindow": 131072,
        "maxTokens": 32768,
        "compat": {
          "supportsReasoningEffort": false,
          "supportsUsageInStreaming": true,
          "maxTokensField": "max_tokens",
          "thinkingFormat": "qwen"
        }
      }]
    }
  }
}

contextWindow must match the context length chosen at setup. If you chose 32K, use "contextWindow": 32768 and "maxTokens": 8192; run .\START-HERE.bat --reconfigure to switch the server to 128K. maxTokens is the longest answer pi asks for. The server rejects a request whose prompt and maxTokens together exceed the context, so keep it well below contextWindow. A whole file is written in one answer, so a small value cuts big files off. The id stays Qwen3.6-35B-A3B-UD-Q4_K_M when the server runs Ornith, because that is the only model name the server reports and accepts; name is only the label pi shows.

Then run pi --model lamina/Qwen3.6-35B-A3B-UD-Q4_K_M in your project folder and pick the model with /model if needed. With "reasoning": true and "thinkingFormat": "qwen" pi sends enable_thinking, so the model's reasoning appears in pi while it is generated; switch it off with /thinking for faster answers. If pi's bash tool reports that no shell is available, set "shellPath" in ~\.pi\agent\settings.json to your Git Bash, for example "C:\\Git\\bin\\bash.exe".

While the model works, pi shows its reasoning, its text and the file it is writing as they are generated, and the server window logs every request with its speed, for example done: 556 answer tokens in 12.6 s = 43.9 tokens/s. If an answer reaches maxTokens in the middle of a tool call, pi shows the error "The answer reached the max_tokens limit while the model was writing a tool call": raise maxTokens, or ask for a smaller step (a big page in several files, or one part at a time).

The first request in a long conversation takes longer because Lamina must read its history. Later requests reuse the unchanged conversation prefix and process only the new text. Restarting the server, editing earlier messages or compacting the conversation can require reading the history again. Check the server log's reused token count to see whether reuse is working.

Review generated code and changes before using them.

Troubleshooting

How fast is it

Measured on an RTX 3050 8 GB, Ryzen 7 5700X and 64 GB of DDR4, generating 256 tokens for three test prompts (prose, code and math):

The setup uses Lamina's fast mode: GPU computation with 8-bit activations and an FP16 context cache, which passed the quality check against full precision (mean KL divergence 0.003 nats on 1,024 tokens). Exact commands and results are in the hybrid prefill, speculation and tuning records. A GPU with more VRAM keeps more of the model on the GPU and should be able to reduce CPU work. Actual speed also depends on CPU, RAM, context length and model; the numbers above describe that specific machine.

On RTX 4060 8 GB / Ryzen 7 1700X, the latest source build measures 39.17 tokens/s for Ornith, or 41.27 with MTP, on short prompts with 128K capacity enabled. Full-history rates are lower. See the decode and context measurements for exact commands, memory use, full-context limits and cached 80K conversation timings, and the initial Ornith benchmark for setup and output-comparison limitations.

More

• docs/ADVANCED.md: manual setup, all lamina.py options (image input, full-precision mode, 128K options, environment variables) and the measurement history.
• docs/DEVELOPER_HANDOFF.md: how the engine works, checks to run after changes, and the performance work so far.
• docs/LAMINA_PORT.md: port status and validation limits.

Lamina is pinned to Strata commit 6f32ec070f23ced9f50e704d854d775da52591ab; upstream MIT licenses and attribution are preserved. The model files and downloads are pinned in tools/lamina_model.py and tools/lamina_assets.py.