BRaiN

Built by Swedish Embedded AB - we put AI on hardware that ships. Hire us.

Modern AI infrastructure is fragmented. Knowledge is spread across many frameworks. Nothing can be optimized all in one place.

Training usually means Python and PyTorch. Production inference means another runtime. Edge deployment means another toolchain. Browser inference means WebGPU. Intel NPUs mean OpenVINO. Distributed execution adds another layer again. Every new model arrives with its own assumptions, dependencies, code, and serving path.

BRaiN replaces that stack with one engine.

It trains and runs models locally, in pure Rust, across GPUs, CPUs, Intel NPUs, and WebGPU - without embedding Python, PyTorch, CUDA, or another deep-learning framework into the runtime.

The same primitives, model definitions, execution graph, and interfaces are used from training through deployment. GPU kernels are implemented directly, backpropagation is verified independently with finite-difference gradient checking, and models are exposed uniformly through CLI, HTTP, and D-Bus.

What brain solves today

• One runtime from training to serving - train, fine-tune, evaluate, and serve without moving the model between unrelated frameworks.
• One hardware abstraction - run the same model code on GPUs, CPUs, Intel NPUs, or in the browser through WebGPU.
• One serving stack - local models behind OpenAI-, Anthropic-, and OpenRouter-compatible APIs, with paged KV cache and continuous batching.
• Training without PyTorch - forward pass, backward pass, optimizers, and GPU kernels implemented directly in brain.
• One interface across model types - language, vision, speech, image generation, forecasting, and other models.
• Fine-tuning on your own data - including LoRA (support pending for more models)
• Multimodal workloads without separate runtimes - detection, depth, segmentation, recognition, restoration, upscaling, image generation, speech recognition, TTS, and voice cloning.
• Distributed execution built into the engine - data parallelism, pipeline parallelism, and tensor-parallel primitives over the same transport abstraction whether workers are threads, local processes, or machines on a network.
• Weight streaming - brain implements a residency engine that can efficiently stream weights and allows concurrent on demand model serving of multiple models at the same time with eviction control.

The goal of brain is to make model training and inference easily accessible, to make it run on absolutely anything, and to keep all knowledge in one place and to keep optimizing it relentless so that it becomes an extremely efficient AI workload runtime that scales across both accelerators and nodes.

Quick start

make build/release                          # build the optimized ./target/release/brain
make test                             # full test suite
make gradcheck                        # backprop correctness gate (finite differences)

Every architecture is reachable through one grammar: brain <verb> <architecture> and brain <architecture> <verb> are the same command. Every line below is a real command, run for real, with its real output - a chained demo where each model's output feeds the next one's input, so a passing chain is also a cross-model correctness check: if the caption names the same object the detector found, three independently-trained models just agreed on what's actually in the image.

Weights arrive on their own the first time a model is needed. To fetch them up front instead - on a connection you are watching, with a progress bar - brain pull takes the canonical id or the HuggingFace URL, and puts the files wherever --brain-data-dir says:

brain pull Qwen/Qwen3-0.6B
brain pull https://huggingface.co/Qwen/Qwen3-0.6B --brain-data-dir /mnt/models

Text generation - no weights on disk yet, no flags beyond the prompt:

$ brain infer qwen3 --prompt "The capital of France is" --max-new 12   # auto-fetches Qwen/Qwen3-0.6B
The capital of France is Paris. The capital of Italy is Rome. The capital of

Bidirectional text embedding - a different architecture family (a non-causal encoder, not a decoder):

$ brain lfm2 embed --text "Brain trains and runs neural networks from scratch, in Rust."   # auto-fetches LiquidAI/LFM2.5-350M
embedding[1024] mean-pool head: [1.97, -1.367, 3.628, 0.754, 2.136, -1.704, 1.704, -1.871] …

Image generation - the one seed image every step below reads:

$ brain --device gpu s3dit text2image \
    --prompt "a golden retriever dog and a red apple on a wooden table, photorealistic, natural lighting" \
    --width 512 --height 512 --seed 7 --steps 8 --precision int8 \
    --out image=seed.png                                    # auto-fetches Tongyi-MAI/Z-Image-Turbo (~33 GB)

Super-resolution, then object detection on the upscaled result:

$ brain rrdbnet upscale --in image=seed.png --tile 128 --out image=upscaled.png   # auto-fetches schwgHao/RealESRGAN_x4plus
$ brain yolov8 detect --weights <ckpt> --image upscaled.png                       # auto-fetches Ultralytics/YOLOv8
[737.14,1328.68,1330.23,1907.12,0.9882,47]
[251.78,48.02,1938.25,1630.42,0.9171,16]
[4.37,1526.75,2025.50,2046.93,0.6109,60]

Three independent boxes, three correct classes: 47 (apple, 0.99), 16 (dog, 0.92), 60 (dining table, 0.61) - YOLOv8n finding exactly what the generation prompt asked for, at 4x the seed image's resolution.

Promptable segmentation:

$ brain sam2 segment --in image=seed.png --points "220,180" --labels "1" --out mask=dog-mask.png --json   # auto-fetches facebook/sam2.1-hiera-tiny
{"area":113765,"iou":0.9763,"object_score":21.10,...}

dog-mask.png is kept - it drives the inpainting step below.

Monocular depth, false-colored (near = red, far = blue):

$ brain zipdepth --image seed.ppm --weights zipdepth_base.pth --headless --view depth --colormap turbo --out depth.ppm
depth: 512x512, inference 137.6 ms (engine)

Image + text → text, two ways - a dedicated fast captioner and a general VQA model, on the same image:

$ brain fastvlm caption --in image=seed.png --prompt "What is in this image? Answer in one sentence." --max_new 40   # auto-fetches apple/FastVLM-0.5B
A golden retriever with a red apple in front of it, both in sharp focus, with a blurred background.

$ brain qwen3vl generate --in image=seed.png --prompt "What is in this image? Answer in one sentence." --max_new 40   # auto-fetches Qwen/Qwen3-VL-4B-Instruct
A golden retriever is looking at a red apple on a wooden table.

Masked inpainting - invert sam2's own segmentation mask (above) so it reads "regenerate everything but the dog", then let the dog anchor the composition while the wood table, the apple, and the background all change together:

$ python3 -c "from PIL import Image, ImageOps; ImageOps.invert(Image.open('dog-mask.png').convert('L')).save('bg-mask.png')"
$ brain --device gpu s3dit inpaint --in image=seed.png --in mask=bg-mask.png \
    --prompt "a golden retriever dog sitting behind a slice of chocolate cake on a white marble kitchen countertop, bright natural daylight, blurred modern kitchen background, photorealistic" \
    --strength 1.0 --feather 0 --steps 8 --precision int8 \
    --out image=inpainted.png

A real segmentation mask driving inpainting, not a hand-picked rectangle: the dog is the one thing sam2 marked, so it's the one thing this leaves alone. feather=0 is deliberate here, not the default 2-3: any positive feather blends a few percent of the original pixels back in near the mask boundary at every sampling step, and the apple used to sit right against the dog's chin - so it kept reinforcing an apple-shaped ghost into the regenerated cake regardless of step count. sam2's mask already follows the dog's real silhouette, so a hard edge doesn't need softening to hide a straight line.

Virtual staging with a third-party LoRA - two independent commands, the first generating a room and the second furnishing it. Nothing here is a photograph; the source image is generated by the same engine one step earlier, so the whole example reproduces from a prompt.

FLUX.2 takes its weights from BRAIN_FLUX2_* rather than a --weights flag (--dit, --vae, --te, --tokenizer paths), and --adapter folds a .safetensors LoRA trained by ai-toolkit or ComfyUI straight into the fused matrices at load time, so it costs nothing per step.

$ brain --device gpu flux2 generate --variant klein-9b --precision int8 \
    --prompt "Photorealistic professional real estate photograph of an empty unfurnished bedroom. Bare white walls, plain light oak floorboards, a single large window, a radiator under the window, a white panelled door. Completely empty room, no furniture, no rugs, no curtains." \
    --width 1024 --height 768 --steps 12 --seed 20260827 \
    --out empty-room.png

$ brain --device gpu flux2 generate --variant klein-9b --precision int8 \
    --ref empty-room.png --strength 0.99 \
    --adapter my_staging_lora.safetensors --lora-scale 1.0 \
    --prompt "TOK, CRITICAL CAMERA LOCK: lock camera position to source image, preserve original field of view, preservation rules: keep original walls and windows, preserve architecture, bedroom containing bed, nightstand, lamp, in boheme style featuring colorful textiles, plants, eclectic mix, warm tones, layered textures, lighting: warm low-angle sunlight, golden tones, long directional shadows, photorealistic, professional real estate photography" \
    --width 1024 --height 768 --steps 12 --seed 7 \
    --out staged-room.png

--strength is the dial that matters, and it is a volume knob: a supplied reference always conditions the model, at every value. Below 1.0 the first reference does double duty - it is the initial latent and it is attended to - so low values repaint the existing pixels and leave furniture essentially where it was, and high values redraw the room while still seeing it. At exactly 1.0 nothing is consumed as an init latent and the denoise starts from pure noise, conditioned on the reference alone. The 0.99 above is the far end of that ramp: enough freedom to furnish an empty room.

Reference tokens are not free - they enter the same joint attention as the generated ones. The init reference is pinned to the output size by its init-latent role, so its conditioning copy is downscaled by default rather than doubling the image half of the sequence; --ref-cond-scale is that dial, with 1.0 conditioning at full size (the same cost as --strength 1.0) and 0 switching the conditioning copy off for the cheapest possible edit.

That freedom is spatially indiscriminate - at this setting the window and door are regenerated along with everything else, which is fine for a room that was generated in the first place and wrong for a photograph of a real property. --mask is the spatial dial for that case: a greyscale image where black preserves and white regenerates, blended into the latent at every step, so architecture is carried through the denoise rather than merely encouraged. See docs/models/flux2.md.

Text → video - the same one-command shape, a different modality: prompt in, playable .mp4 out, no second step to turn frames into a file:

$ brain --device gpu wan t2v \
    --prompt "a golden retriever running along a sandy beach at sunset, waves in the background, cinematic" \
    --frames 9 --width 416 --height 240 --steps 20 --seed 7 \
    --output-path wan.mp4                                   # auto-fetches Wan-AI/Wan2.1-T2V-1.3B (~17.6 GB)
wan: wrote wan.mp4 (416x240, 9 frames at 16 fps)

Every second frame of the clip, side by side. The subject moves across the frame and the sea and sky stay put, which is the thing a video model has to do and an image model cannot.

Those are not Wan's own defaults, and the difference is the point: 81 frames at 832x480 over 50 steps is what upstream ships and it occupies a Tesla P40 for the better part of an hour, so this page runs a half-scale 9-frame clip that finishes in minutes. Most of even that is the umT5-XXL text encoder, which runs on the CPU because 22.72 GB of fp32 weights do not fit a 24 GB card. docs/models/wan.md has the measured breakdown.

Speech: text → speech → text → text, a full TTS → ASR → LLM round trip, through two independently-trained ASR models on the exact same audio:

$ brain qwen3tts synth --text "Brain trains and runs neural networks from scratch, in Rust." --out spoken.wav   # auto-fetches Qwen/Qwen3-TTS-12Hz-0.6B-Base
$ brain nemotronasr transcribe --in audio=spoken.wav --json                                                     # auto-fetches nvidia/nemotron-3.5-asr-streaming-0.6b
{"text":"Brain trains and runs neural networks from scratch in rust. ",...}
$ brain qwen3asr transcribe --in audio=spoken.wav --json                                                        # auto-fetches Qwen/Qwen3-ASR-1.7B
{"text":"Brain training.",...}
$ brain infer qwen3 --prompt "Brain trains and runs neural networks from scratch in rust." --max-new 24
Brain trains and runs neural networks from scratch in rust. This is a problem that is very common in the field of AI, and it is also a problem that is very difficult

Document OCR - a second round trip through a different modality (text → rendered document image → text), the same self-verifying shape as the speech round trip above:

$ brain deepseek2ocr generate --in image=doc.png --prompt "<|grounding|>Convert the document to markdown."   # auto-fetches ggml-org/DeepSeek-OCR-GGUF
Brain trains and runs neural networks from scratch, in Rust.

A word-for-word match of the sentence rendered into doc.png - real OCR on a real (synthetically rendered) document image, not a toy string echo.

Time-series forecasting - a CSV in, a scored forecast and a chart out. The last 6 rows of each window are held back from the model and used as the answer key, at 16 disjoint origins:

$ brain forecast predict --csv examples/forecast/synthetic_hourly.csv \
    --horizon 6 --samples 16 --origins 16 --gnuplot kronos-forecast.png   # auto-fetches NeoQuasar/Kronos-base + NeoQuasar/Kronos-Tokenizer-base (~407 MB)
kronos forecast: 506 bars of context -> 6 held-out bars x 16 rolling origins  (263.7s, 16 samples)
  close, vs held-out truth       mean MAE       CRPS    pinball
  kronos                           0.5331     0.4139     0.1900
  persistence (last close)         0.5062     0.5062     0.2531
  drift (context mean return)      0.5062     0.5062     0.2531
  seasonal naive (24 bars)         1.1005     1.1005     0.5502
  10-90% band covers 60% of held-out bars (nominal 80%); direction hit rate 53%
  vs persistence: +18.2% CRPS reduction, better at 10/16 origins

Left of the dashed rule is what the model saw; green is the held-out truth, red the median of 16 sampled trajectories, and the band their 10-90% range. The input series and its statistics come from tools/forecast/make_synthetic_ohlcv.py, which prints the fingerprint of what it generated. Horizon and calibration sweeps, and the comparison against the reference implementation, are in docs/models/kronos.md.

LoRA fine-tuning, on a synthetic dataset generated in-process (no external data, ~200 steps):

$ brain data gen tts --out data/tts --n 4000 --seed 1337
$ brain qwen3tts finetune --base <ckpt>/talker.safetensors --data data/tts --out talker_lora.safetensors --steps 200
finetune done: loss 17.6053 -> 0.3237  saved -> talker_lora.safetensors

A real, measured loss descent from a rank-8 adapter, not a claim.

Serving - the same weights, behind a local OpenAI-compatible API:

$ brain serve --openai 8799 &
APIKEY openai sk-brain-...
$ curl http://127.0.0.1:8799/v1/chat/completions -H "Authorization: Bearer $APIKEY" \
    -H 'Content-Type: application/json' \
    -d '{"model":"Qwen/Qwen3-0.6B","messages":[{"role":"user","content":"Say hello in exactly five words."}]}'
Hi, how are you?

Every one of these is reachable the same way once you point it at weights (brain caps <arch> prints exactly what each action takes):

brain caps                                                  # every architecture + its actions
brain infer scrfd --in image=photo.ppm --json                # face detection (needs BRAIN_SCRFD_DIR)
brain infer glmdsa --weights F --prompt "..."                 # GLM-5.2 MoE decoder
brain zipdepth --image photo.ppm --weights zipdepth.pth       # monocular depth

Model support

Every model brain caps reports, with what it does and where its full page is. Toy architectures (toymoe, toypid, toyseq2seq, toyautoencoder -- brain's own tasks, no upstream reference) are excluded here; see docs/models/index.md for the complete catalog including those.

Where to go next

Who builds brain

brain is built by Swedish Embedded AB.

We build AI that runs on hardware that ships: on the GPU you already have, on a CPU with no GPU at all, on an Intel NPU, on a board in the field, or in a browser tab. Everything in this repository is that work done in the open - the WGSL kernels, the finite-difference gradient checker that gates every backward pass, the residency engine that keeps models inside a fixed memory budget, and the serving stack that puts them behind an API.

Every capability below is implemented in this repository, in the open, and held to tests you can run yourself. Read the code before you talk to us. If your team needs one of these, you can hire us to do it:

• Running models on the hardware you have - GPUs, CPUs with no accelerator, Intel NPUs, embedded Linux boards, WebGPU in the browser. One model, one implementation, every target.
• Getting a large model to fit - quantization, weight streaming, tiled and memory-bounded inference, multi-GPU sharding. The difference between "needs a datacenter" and "runs on the card in the machine".
• Porting a model from a paper or a PyTorch checkpoint to a dependency-free runtime, gated by real numerical parity against the reference rather than by hope.
• Writing and optimizing GPU compute kernels - and proving the result is still correct afterwards.
• Production inference systems - concurrent serving, paged KV cache, continuous batching, model residency and scheduling across accelerators.
• Embedded and real-time firmware alongside the AI, which is where this company started and still spends much of its time.

Send an email to info@swedishembedded.com and tell us what you are trying to ship.

License

Apache-2.0 - see LICENSE.

Copyright (c) 2026 Swedish Embedded AB.