→ Live demo

Quick start  ·  How it works  ·  Parity  ·  Upstream model

_{Real output. For the browser version, try the live demo — it runs all three generations.}

A pure-Rust + WebAssembly runtime for Needle by Cactus Compute — small transformers that map (query, tool list) to a JSON function call. Deploys to browsers, edge workers, CLIs, Python, and no_std embedded targets. No server, no API key, no data leaving the device.

All three model generations are supported in parallel: Needle 3 (121M, one 35.3 MB .cact file), Needle 2 (45M, 13.7 MB .cact) and Needle 1 (26M, SafeTensors + vocabulary). Same runtime, same API shape, one binary — the generation comes from the container, not the file name.

AI tool calling usually means a paid API round-trip or hundreds of megabytes on disk. This ships the whole agent in 14 MB with Needle 2, or 36 MB with Needle 3, and runs it in a browser tab.

The runtime is under 600 KB of WebAssembly (about 200 KB over the wire, brotli) with one runtime dependency, and carries all three model generations. A Needle 2 session needs about 23 MB of working memory; a Needle 3 session adds 8.8 MB of key/value cache at 512 tokens, or 2.3 MB with --kv-int8 — see choosing a generation.

Choosing a generation

All three run from the same binary, the same WASM module and the same Python package. The runtime picks the right engine from the container itself, so nothing depends on a file name.

Pick Needle 3 when quality matters most and you can afford a 35 MB download and roughly 9 MB of cache — or 2.3 MB with the int8 cache, which is the width the container declares and costs nothing in answer quality on our tests. It reasons before answering, which shows up on ambiguous queries and larger tool catalogues.

Pick Needle 2 for the smallest viable browser deployment, or when you need tool retrieval — the published Needle 3 weights export only a confidence head, so retrieve_tools and encode_contrastive are absent on the v3 API rather than present and always empty. Needle 3's architecture defines an embedding head; it is not in this checkpoint, and would load unchanged if a future one carried it.

Needle 1 remains supported for compatibility. It abstains readily as a tool catalogue grows; prefer a newer generation for new work.

Needle 3's weights are Apache-2.0, where v1 and v2 are MIT. needle-rs itself is MIT in every case; the difference applies to the model.

Get a model. v3 and v2 are each a single self-describing file; v1 needs a vocabulary alongside its weights.

# Needle v3 — weights, geometry and tokenizer in one container
hf download Cactus-Compute/needle3 needle3.cact --local-dir weights/

# Needle v2 — smaller and faster, same one-file story
hf download Cactus-Compute/needle2 needle2.cact --local-dir weights/

# Needle v1
hf download Abdalrahman/needle-rs-safetensors needle.safetensors vocab.txt --local-dir weights/

# v3 — the container states its own generation, so there is no flag to get wrong
needle-rs --json --constrain weights/needle3.cact \
  "What's the weather in Paris?" \
  '[{"name":"get_weather","parameters":{"type":"object",
     "properties":{"city":{"type":"string"}},"required":["city"]}}]'
# → [{"name":"get_weather","arguments":{"city":"Paris"}}]

# Drop --json to see the <think> block v3 reasons with first.
# --kv-int8 stores the cache at 8 bits: 2.3 MB instead of 8.8 MB at 512 tokens.
needle-rs --kv-int8 weights/needle3.cact "$QUERY" "$TOOLS"

# --layers runs a shallower rung of the same file: Needle 3 is trained so every
# depth from 2 to 20 blocks is a usable model. 8 blocks needs 3.5 MB of cache
# instead of 8.8 MB. Below 6 the answers stop being usable.
needle-rs --layers 8 weights/needle3.cact "$QUERY" "$TOOLS"

# v2 — same binary, same invocation
needle-rs --json weights/needle2.cact "$QUERY" "$TOOLS"

# v1 — same binary, two files
needle-rs weights/needle.safetensors weights/vocab.txt "$QUERY" "$TOOLS"

use needle_infer::v3_engine::V3Engine;          // Needle v3
let engine = V3Engine::load("weights/needle3.cact")?;
let text = engine.run(query, tools_json);       // reasoning + call
println!("{}", engine.run_json(query, tools_json).unwrap_or_default());
println!("{:?}", V3Engine::reasoning(&text));   // the <think> block, if any
// Pass the completion, never the bare query — the head scores a finished answer.
println!("{:?}", engine.confidence_for(query, tools_json, &text));

use needle_infer::v2_engine::V2Engine;          // Needle v2
let engine = V2Engine::load("weights/needle2.cact")?;
let out = engine.run(query, tools_json);
println!("{}", out.tool_call.unwrap_or(out.text));

use needle_infer::NeedleEngine;                 // Needle v1
let engine = NeedleEngine::load("weights/needle.safetensors", "weights/vocab.txt")?;
println!("{}", engine.run(query, tools_json).text);

import init, { NeedleV3Wasm, NeedleV2Wasm, NeedleWasm } from "needle-rs";
await init();

const v3 = NeedleV3Wasm.load(new Uint8Array(cactBytes));
const out = v3.run(query, toolsJson);
v3.run_json(query, toolsJson);                 // just the payload
v3.reasoning(out);                             // the <think> block, or undefined
v3.confidence_for(query, toolsJson, out);      // pass the completion, not the query
v3.kv_bytes(512);                              // 8.8 MB — budget a tab before loading
v3.kv_bytes_int8(512);                         // 2.3 MB at 8 bits
// No retrieve_tools on v3: these weights export only a confidence head.

const v2 = NeedleV2Wasm.load(new Uint8Array(cactBytes));
v2.retrieve_tools(query, descriptions, 3);     // rank tools by relevance

const v1 = NeedleWasm.load(weightsBytes, vocabText);
v1.run(query, toolsJson);

from needle_rs import V3Engine, V2Engine, NeedleEngine

engine = V3Engine.load("weights/needle3.cact")
engine.run_json(query, tools_json)                     # tool-call payload
engine.generate(query, tools_json, constrain=True)     # dict: text, tool_call,
                                                       # reasoning, stop_reason
engine.generate(query, tools_json, kv_int8=True)       # 8-bit key/value cache
engine.confidence_for(query, tools_json, completion)   # the completion, not the query
engine.kv_bytes(512, kv_int8=True)                     # what a session will cost
# V3Engine has no retrieve_tools — these weights carry only a confidence head.

V2Engine.load("weights/needle2.cact").retrieve_tools(query, descriptions, top_k=3)
NeedleEngine.load("weights/needle.safetensors", "weights/vocab.txt")

Upstream replaced v1's encoder–decoder with a decoder-only architecture in a new container, then rebuilt that again for v3 — hybrid local/global attention, a causal convolution over Q/K/V, five Engram sites and a reasoning step. No two generations share weights, loader or quantisation scheme, so needle-rs implements all three rather than migrating.

Every example in examples/ runs on all three.

Cactus's own engine targets mobile and NPUs with hand-tuned ARM SIMD. needle-rs targets everywhere else. MSRV is 1.87.

Architecture deep-dive: ARCHITECTURE.md · v2 port record: docs/v2-port-record.md.

The failure mode for a from-scratch reimplementation is silent drift: output that looks right but diverges in the third decimal, producing rare and untraceable bugs. All three engines are held to the reference implementation's exact output.

Needle v3 — verified against upstream's own model, with the tensor canon pinned by inverting export._tensors, since the container's directory is nameless and positional and per-tensor checks alone cannot catch a reordering:

Needle v2 — verified against upstream's own decode.forward_cached running the same weights, reconstructed from the shipped container by tools/cact_params.py:

Needle v1 — 560 generated examples across five tool-name conventions, 0–8 parameters, 1–20 tools: 560/560 token-exact.

Fixtures are committed, so the contract is version-pinned and reproducible without re-running Python. 369 Rust tests and 100 WASM binding assertions run in CI, in both the default and parallel feature configurations, alongside the C ABI and Python wheel.

A measured caution, because it cost two debugging sessions: the reference config ships dtype="bfloat16", and measuring against a bfloat16 oracle makes a correct implementation look catastrophically wrong — a relative error of 9.7, i.e. about 970%, not a small number with a missing exponent. Every figure above is measured against an f32 reference. See docs/v3-port-record.md.

Apple M5 Max, steady state, threaded — which is what the CLI, Python and C crates build. Needle v2 against the Python/JAX reference on the same machine and the same weights:

Decode is 1.35× faster and cold start about 50× faster; prefill is slower, because the reference multiplies dense f32 weights while this runs from 2-bit packed ones. On a single query the two cross at 48 generated tokens — faster above, slower below, and faster at any length on a cold process.

Prefill improved 3.6× during the v2 port (5.74 → 1.60 ms/token) via batching, threading and batched Engram projections. The packed dot product is 2.9× faster than a single-accumulator version and 9.1× faster than a naive one.

Needle v3, same machine and flags, carries 121M parameters against v2's 45M and costs roughly 2–3× per token: 10 ms to load, 4.20 ms/token prefill and 8.65 ms/token decode. A 100-token prompt answers in about 700 ms, 424 ms of it to first token. Its --kv-int8 cache costs about 2% of that speed and roughly a quarter of the memory. That is the trade v3 asks for, and the reason v2 is not deprecated.

Full methodology — including the optimisations that were measured and rejected, such as hand-written NEON losing to LLVM's autovectoriser — is in BENCHMARKS.md.

• In-browser agents. Route a user's sentence to one of your app's functions with no backend. See examples/browser-demo and the live demo.
• Dynamic tool sets. Generate tools from live state each turn and let the model pick — examples/dom-editor rewrites a page from plain English.
• Edge workers. The whole runtime fits inside a Cloudflare Worker.
• Large tool catalogues. Narrow hundreds of tools with the retrieval head before the call — Needle 2 only, which is the one generation that ships a contrastive head.
• Uncertainty-aware routing. Use the confidence head to escalate to a larger model only when needed.
• Offline and embedded. no_std kernels, one dependency, no allocator assumptions beyond alloc.

Not the right tool for open-ended chat, long-form generation, or reasoning beyond tool selection. It does one thing.

Needle is designed and trained by Henry Ndubuaku and the Cactus Compute team. The model architecture, training code, dataset, and weights are entirely their work, released openly — the upstream repository under Apache-2.0, the Needle 3 weights under Apache-2.0, and the Needle 2 and Needle 1 weights under MIT. needle-rs is an independent Rust runtime — no upstream code is copied, only the published architecture is implemented. See NOTICE.

If you find this useful, please star the upstream Needle repo as well.

The model is Cactus Compute's work. Cite it as they ask — these are their entries, reproduced verbatim from the upstream README. The design and ablations are in the paper, arXiv:2607.18363.

Current, and what to cite unless you mean an older generation specifically:

@misc{needle3_2026,
  title        = {Needle: Automation Foundation Model for Tiny Devices},
  author       = {Ndubuaku, Henry and Mosoyan, Karen and Mroz, Jakub and Cylich, Noah and
                  Kumar, Satyajit and Sandhu, Parkirat and Shemet, Roman and Lee, Justin H.},
  year         = {2026},
  organization = {Cactus Compute, Inc.},
  howpublished = {\url{https://github.com/cactus-compute/needle}}
}

If your work uses the v2 weights specifically:

@misc{needle2_2026,
  title        = {Needle 2: A 45M-Parameter Foundation Tool-Calling Model for Tiny Devices},
  author       = {Ndubuaku, Henry and Mosoyan, Karen and Mroz, Jakub and Cylich, Noah and
                  Kumar, Satyajit and Sandhu, Parkirat and Shemet, Roman and Lee, Justin H.},
  year         = {2026},
  organization = {Cactus Compute, Inc.},
  howpublished = {\url{https://github.com/cactus-compute/needle}}
}

If your work uses the v1 weights specifically, cite the v1 model instead:

@software{needle2026,
  author  = {Ndubuaku, Henry and {Cactus Compute}},
  title   = {Needle: A 26M-Parameter Tool-Calling Transformer},
  year    = {2026},
  url     = {https://github.com/cactus-compute/needle},
  license = {MIT}
}

And this runtime, if it is relevant to what you are reporting:

@software{needlers2026,
  author  = {Ibrahim, Abdalrahman},
  title   = {needle-rs: Pure-Rust WASM Runtime for Needle},
  year    = {2026},
  url     = {https://github.com/geekgineer/needle-rs},
  license = {MIT}
}