Local-first code context engine in Rust + DuckDB: incremental semantic indexing, hybrid search, MCP server, HTTP API and TUI. Rust rewrite of devai-context-engine.
Rust
0
311 commits
updated Aug 26, 2026
A local-first code context engine in Rust + DuckDB: incremental semantic
indexing, hybrid search, an MCP server, an HTTP API and a TUI. See
docs/architecture-spec.md for the architecture.
This is the Rust rewrite of snaven10/devai-context-engine — the original Go + Python implementation. See Lineage for what changed and why.
devctx init --name myproj
devctx index # git diff → parse → chunk → embed → store
devctx search "connect to a database" --limit 5
devctx remember "We chose Postgres for JSONB" --type decision --topic db-engine
devctx recall "which database did we pick"
devctx mcp # MCP server over stdio (for AI agents/editors)
devctx mcp configure --client cursor --scope project # register in an AI client
devctx tui # terminal UI: search, call-graph, memories, projects
devctx web # web dashboard: interactive call-graph + memories
macOS and Linux:
curl -fsSL https://raw.githubusercontent.com/snaven10/DevCtxEngine/main/install.sh | sh
Windows, in PowerShell:
irm https://raw.githubusercontent.com/snaven10/DevCtxEngine/main/install.ps1 | iex
Linux x86_64, macOS arm64 and Windows x64; anywhere else, cargo build --release
(it compiles DuckDB from source, so allow 20–25 minutes). The script installs the
binary — ~/.local/bin on Unix, %LOCALAPPDATA%\devctx\bin on Windows — and
stops there. It writes no configuration and downloads no model, because those
are decisions, and the embedding model in particular cannot be changed after
indexing without redoing it.
The steps that follow — choosing a model, registering repositories, migrating
memories from an older DevAI install, indexing — are written out in
AGENTS.md for a coding agent to carry out. Paste this to yours:
Set up DevCtxEngine on this machine for me. Read https://raw.githubusercontent.com/snaven10/DevCtxEngine/main/AGENTS.md and follow it in order, verifying each step the way it says to.
Before you start, ask me: what language are the code and comments mostly in (that decides the embedding model, and changing it later means re-indexing everything); which repositories to register, and whether they are one product that should share a memory group; and whether there is an existing
devaiinstall whose memories should be migrated.Several failures in this system are silent — a model whose name matches but whose vectors do not, an index that is never built, a reranker pointed at the wrong model. Do not report a step as done because a command exited zero; run the verification the document gives and show me its output.
devctx mcp configure writes the MCP server entry for you — Claude Code, Cursor,
Codex and Windsurf all read a mcpServers block, and the command knows where
each one keeps it. Paste this to your agent and it will do the whole thing:
Register DevCtxEngine as an MCP server for me.
Run
devctx mcp configure --helpfirst and use the flags it actually has; do not hand-write JSON into a config file unless that command cannot do it. Pick the client I am running, and project scope unless I say otherwise.Then verify it: restart the client, list the available tools, and confirm
search,recall,remember,memories_by_symbolandbuild_contextare present. A configure command that exits zero proves the file was written, not that the server starts — so call one read-only tool (list_projects) and show me the result.If no project is bound, that is expected for a globally-registered server: call
list_projectsto see what is registered anduse_projectto bind one.
Having the tools is not the same as knowing when to use them. An agent with
remember available and no policy either saves nothing, or saves a diary nobody
can search — and the next session re-derives what the last one already knew.
MEMORY-PROTOCOL.md is that policy, written to be handed
to an agent verbatim. Paste this:
Read https://raw.githubusercontent.com/snaven10/DevCtxEngine/main/MEMORY-PROTOCOL.md and follow it for the rest of our work together. It covers when to search before starting, when to save afterwards, and what to put in each field.
The part that is easiest to get wrong:
remembertakes afilesparameter, and it is what links the memory to the code, so a memory saved without it can only ever be found by text search. Fill it in, and name the symbols in the prose too — that is what the linker matches against.
If your setup supports a persistent instruction file — CLAUDE.md, AGENTS.md,
a system prompt — put the pointer there instead, so it survives past this
conversation.
Indexing a large repository takes minutes, and a status line is the one place
that can say so without being asked. contrib/statusline-devctx.sh
prints a bar for every project currently indexing — and nothing at all the rest
of the time, so it costs no room when there is nothing to report:
⚙ indexing web-app ████████░░ 82% (1561/1895)
Paste this to your agent:
Set up the DevCtxEngine indexing progress indicator in my Claude Code status line, using
contrib/statusline-devctx.shfrom https://github.com/snaven10/DevCtxEngine.I may already have a status line configured — check
statusLinein~/.claude/settings.jsonfirst. If I do, add to it, do not replace it, and back the original up before editing. Give the progress its own row rather than appending to an existing one: those rows already carry a model, a directory and a branch name, and a bar tacked on the end is the first thing a terminal truncates — which is the only part that changes second to second. The row must appear only while something is indexing.Then verify it by actually indexing something (
devctx index --fullin a registered project) and running the status line command by hand while that runs. A script that exits zero proves it ran, not that it printed anything.
The script asks each project's server for its progress over HTTP, caches the project list so it does not wake the central daemon on every render, and gives each request a quarter-second timeout — an unreachable server costs a moment, not a hung prompt.
Each repository keeps its own index. What is shared lives in a central store: a registry of every project, and the memory worth carrying between them — so an agent working in one repository knows the others exist and can recall what was learned there.
devctx projects add ~/code/api # register a repository (init does this too)
devctx projects list # name · model · index freshness · path
devctx remember "always verify webhook signatures" --scope global
devctx recall "how do I validate a webhook" # this project + the shared ones
devctx recall "..." --scope global --repo api # only what `api` contributed
A global memory saved from one repository is recalled from any other, and the
same lesson saved twice converges on one memory rather than two. Anything left
local — the default — never leaves its project.
Memories move between machines as JSONL, which any version can read:
devctx memories export --scope group > product.jsonl # one product's memories
devctx memories import product.jsonl # only ever adds
Import never overwrites: content already present is skipped, and a memory whose topic key belongs to a different local one is kept beside it rather than replacing it.
Over MCP this is list_projects, search_project, and scope on
recall/remember. See The Central Store.
devctx hooks install # re-index after each commit
devctx watch # re-index files as they are saved
devctx reindex --all # every registered project
The index mirrors the work tree, not the last commit: a file you have written but not committed is indexed like any other, and a full re-index does not throw it away. See Keeping the index fresh.
DuckDB allows a single read-write process. Run devctx serve
and it becomes the sole owner of the database; every other devctx command
discovers it (via .devctx/state/serve.json) and routes over HTTP instead of
opening the file, so concurrent CLI/editor/web use never hits a lock. When no
server is running, commands open the store directly as usual.
Every DB command (search, recall, remember, summarize, index,
impact, status, memory-stats, routes), the TUI, the web dashboard and
the MCP server route through one shared server, auto-spawning it on first
use. The server is the single owner of the DB, so nothing ever fights the lock —
you can run several Claude Code sessions (each an MCP client), the web dashboard,
the TUI and CLI commands against the same project at once, and query while an
index runs (readers see a consistent snapshot). The embedding model stays warm,
so repeated commands return in milliseconds. The daemon idles out after 15
minutes; stop it explicitly with devctx serve --stop, or disable auto-spawn
with DEVCTX_NO_AUTOSERVE=1.
The central store is a singleton and follows the same pattern: devctx serve --central owns it, auto-spawned on demand.
devctx web serves a self-contained dashboard (call-graph via a vendored,
offline cytoscape build + a memories browser) and opens it in your browser.
devctx tui is the terminal equivalent, with four views on F1–F4: search, graph,
memories (with a scope selector) and projects — where you can register and index
a repository without leaving the UI.
Because the server holds the loaded code, a rebuilt binary does not take effect
until the running server is restarted (devctx serve --stop).
| crate | responsibility | phase |
|---|---|---|
devctx-core | shared types, .devctx/config.yaml schema, errors | F0 |
devctx-cli | the devctx binary (clap) | F0 |
devctx-store | DuckDB store: vectors (brute-force cosine) + relational schema | F1 |
devctx-embed | embeddings: local (fastembed/ort) + OpenAI/Voyage/custom | F2 |
devctx-parse | tree-sitter symbols/imports/call-edges + framework route extractors | F3 |
devctx-chunk | semantic multi-level chunker (file/class/function/block) | F3 |
devctx-index | pipeline: git diff → parse → chunk → embed → store (incremental) | F4 |
devctx-rerank | cross-encoder reranking (fastembed BGE) + no-op fallback | F5 |
devctx-search | search orchestration: vector / keyword / hybrid (RRF) + rerank | F8 |
devctx-mcp | MCP server (rmcp, stdio): search / read_file / index_repo / index_status | F6 |
devctx-memory | memory engine: remember (dedup) + recall (intro/chunk blend) | F7 |
devctx-summarize | summarization: extractive (default) + OpenAI + local flan-t5 | F9 |
devctx-api | HTTP REST API (axum) reusing the MCP engine, Bearer-token auth | F9 |
devctx-tui | interactive terminal UI (ratatui): search, graph, memories, projects | F9 |
devctx-central | central store: project registry, global memories, daemon client | — |
The flan-t5 feature (off by default) adds a local abstractive summarizer via
candle — build with --features flan-t5 (heavy; downloads the model on first use).
The devctx-embed local feature (default) pulls in fastembed/ort; build
with --no-default-features for an API-only build where the ONNX Runtime binary
can't be fetched.
cargo build
cargo test
cargo run -p devctx-cli -- status
DevCtxEngine is a ground-up rewrite of snaven10/devai-context-engine (DevAI), which remains the reference implementation. The commit history here carries over from that project, so the migration is visible in the log rather than squashed away.
What DevAI was — a hybrid, ~20k LOC across two runtimes:
| Layer | Size | Responsibility |
|---|---|---|
| Go | ~10.4k LOC | Thin orchestrator: CLI (cobra), MCP server (21 tools), HTTP API, TUI (Bubble Tea), config/storage routing |
Python (devai_ml) | ~9.7k LOC | The actual work: embeddings, chunking, tree-sitter parsers, retrieval/reranking, summarization, stores |
The real contract between them was a JSON-RPC 2.0 bridge over stdio — about 27 methods — with Python running as a sidecar process.
What the rewrite changes:
fastembed-rs + ort (ONNX Runtime) for local models;
OpenAI/Voyage/custom over HTTP. Embedding dimension is parameterized rather
than pinned at 384.The full reasoning, with the cost of each decision, is in
docs/08-design-decisions.md; the system it
describes is specified in
docs/architecture-spec.md.
MIT — see LICENSE.
311 commits
Rust
97.4%
HTML
1.2%
Shell
1.1%
Local-first code context engine in Rust + DuckDB: incremental semantic indexing, hybrid search, MCP server, HTTP API and TUI. Rust rewrite of devai-context-engine.
Rust
0
311 commits
updated Aug 26, 2026
A local-first code context engine in Rust + DuckDB: incremental semantic
indexing, hybrid search, an MCP server, an HTTP API and a TUI. See
docs/architecture-spec.md for the architecture.
This is the Rust rewrite of snaven10/devai-context-engine — the original Go + Python implementation. See Lineage for what changed and why.
devctx init --name myproj
devctx index # git diff → parse → chunk → embed → store
devctx search "connect to a database" --limit 5
devctx remember "We chose Postgres for JSONB" --type decision --topic db-engine
devctx recall "which database did we pick"
devctx mcp # MCP server over stdio (for AI agents/editors)
devctx mcp configure --client cursor --scope project # register in an AI client
devctx tui # terminal UI: search, call-graph, memories, projects
devctx web # web dashboard: interactive call-graph + memories
macOS and Linux:
curl -fsSL https://raw.githubusercontent.com/snaven10/DevCtxEngine/main/install.sh | sh
Windows, in PowerShell:
irm https://raw.githubusercontent.com/snaven10/DevCtxEngine/main/install.ps1 | iex
Linux x86_64, macOS arm64 and Windows x64; anywhere else, cargo build --release
(it compiles DuckDB from source, so allow 20–25 minutes). The script installs the
binary — ~/.local/bin on Unix, %LOCALAPPDATA%\devctx\bin on Windows — and
stops there. It writes no configuration and downloads no model, because those
are decisions, and the embedding model in particular cannot be changed after
indexing without redoing it.
The steps that follow — choosing a model, registering repositories, migrating
memories from an older DevAI install, indexing — are written out in
AGENTS.md for a coding agent to carry out. Paste this to yours:
Set up DevCtxEngine on this machine for me. Read https://raw.githubusercontent.com/snaven10/DevCtxEngine/main/AGENTS.md and follow it in order, verifying each step the way it says to.
Before you start, ask me: what language are the code and comments mostly in (that decides the embedding model, and changing it later means re-indexing everything); which repositories to register, and whether they are one product that should share a memory group; and whether there is an existing
devaiinstall whose memories should be migrated.Several failures in this system are silent — a model whose name matches but whose vectors do not, an index that is never built, a reranker pointed at the wrong model. Do not report a step as done because a command exited zero; run the verification the document gives and show me its output.
devctx mcp configure writes the MCP server entry for you — Claude Code, Cursor,
Codex and Windsurf all read a mcpServers block, and the command knows where
each one keeps it. Paste this to your agent and it will do the whole thing:
Register DevCtxEngine as an MCP server for me.
Run
devctx mcp configure --helpfirst and use the flags it actually has; do not hand-write JSON into a config file unless that command cannot do it. Pick the client I am running, and project scope unless I say otherwise.Then verify it: restart the client, list the available tools, and confirm
search,recall,remember,memories_by_symbolandbuild_contextare present. A configure command that exits zero proves the file was written, not that the server starts — so call one read-only tool (list_projects) and show me the result.If no project is bound, that is expected for a globally-registered server: call
list_projectsto see what is registered anduse_projectto bind one.
Having the tools is not the same as knowing when to use them. An agent with
remember available and no policy either saves nothing, or saves a diary nobody
can search — and the next session re-derives what the last one already knew.
MEMORY-PROTOCOL.md is that policy, written to be handed
to an agent verbatim. Paste this:
Read https://raw.githubusercontent.com/snaven10/DevCtxEngine/main/MEMORY-PROTOCOL.md and follow it for the rest of our work together. It covers when to search before starting, when to save afterwards, and what to put in each field.
The part that is easiest to get wrong:
remembertakes afilesparameter, and it is what links the memory to the code, so a memory saved without it can only ever be found by text search. Fill it in, and name the symbols in the prose too — that is what the linker matches against.
If your setup supports a persistent instruction file — CLAUDE.md, AGENTS.md,
a system prompt — put the pointer there instead, so it survives past this
conversation.
Indexing a large repository takes minutes, and a status line is the one place
that can say so without being asked. contrib/statusline-devctx.sh
prints a bar for every project currently indexing — and nothing at all the rest
of the time, so it costs no room when there is nothing to report:
⚙ indexing web-app ████████░░ 82% (1561/1895)
Paste this to your agent:
Set up the DevCtxEngine indexing progress indicator in my Claude Code status line, using
contrib/statusline-devctx.shfrom https://github.com/snaven10/DevCtxEngine.I may already have a status line configured — check
statusLinein~/.claude/settings.jsonfirst. If I do, add to it, do not replace it, and back the original up before editing. Give the progress its own row rather than appending to an existing one: those rows already carry a model, a directory and a branch name, and a bar tacked on the end is the first thing a terminal truncates — which is the only part that changes second to second. The row must appear only while something is indexing.Then verify it by actually indexing something (
devctx index --fullin a registered project) and running the status line command by hand while that runs. A script that exits zero proves it ran, not that it printed anything.
The script asks each project's server for its progress over HTTP, caches the project list so it does not wake the central daemon on every render, and gives each request a quarter-second timeout — an unreachable server costs a moment, not a hung prompt.
Each repository keeps its own index. What is shared lives in a central store: a registry of every project, and the memory worth carrying between them — so an agent working in one repository knows the others exist and can recall what was learned there.
devctx projects add ~/code/api # register a repository (init does this too)
devctx projects list # name · model · index freshness · path
devctx remember "always verify webhook signatures" --scope global
devctx recall "how do I validate a webhook" # this project + the shared ones
devctx recall "..." --scope global --repo api # only what `api` contributed
A global memory saved from one repository is recalled from any other, and the
same lesson saved twice converges on one memory rather than two. Anything left
local — the default — never leaves its project.
Memories move between machines as JSONL, which any version can read:
devctx memories export --scope group > product.jsonl # one product's memories
devctx memories import product.jsonl # only ever adds
Import never overwrites: content already present is skipped, and a memory whose topic key belongs to a different local one is kept beside it rather than replacing it.
Over MCP this is list_projects, search_project, and scope on
recall/remember. See The Central Store.
devctx hooks install # re-index after each commit
devctx watch # re-index files as they are saved
devctx reindex --all # every registered project
The index mirrors the work tree, not the last commit: a file you have written but not committed is indexed like any other, and a full re-index does not throw it away. See Keeping the index fresh.
DuckDB allows a single read-write process. Run devctx serve
and it becomes the sole owner of the database; every other devctx command
discovers it (via .devctx/state/serve.json) and routes over HTTP instead of
opening the file, so concurrent CLI/editor/web use never hits a lock. When no
server is running, commands open the store directly as usual.
Every DB command (search, recall, remember, summarize, index,
impact, status, memory-stats, routes), the TUI, the web dashboard and
the MCP server route through one shared server, auto-spawning it on first
use. The server is the single owner of the DB, so nothing ever fights the lock —
you can run several Claude Code sessions (each an MCP client), the web dashboard,
the TUI and CLI commands against the same project at once, and query while an
index runs (readers see a consistent snapshot). The embedding model stays warm,
so repeated commands return in milliseconds. The daemon idles out after 15
minutes; stop it explicitly with devctx serve --stop, or disable auto-spawn
with DEVCTX_NO_AUTOSERVE=1.
The central store is a singleton and follows the same pattern: devctx serve --central owns it, auto-spawned on demand.
devctx web serves a self-contained dashboard (call-graph via a vendored,
offline cytoscape build + a memories browser) and opens it in your browser.
devctx tui is the terminal equivalent, with four views on F1–F4: search, graph,
memories (with a scope selector) and projects — where you can register and index
a repository without leaving the UI.
Because the server holds the loaded code, a rebuilt binary does not take effect
until the running server is restarted (devctx serve --stop).
| crate | responsibility | phase |
|---|---|---|
devctx-core | shared types, .devctx/config.yaml schema, errors | F0 |
devctx-cli | the devctx binary (clap) | F0 |
devctx-store | DuckDB store: vectors (brute-force cosine) + relational schema | F1 |
devctx-embed | embeddings: local (fastembed/ort) + OpenAI/Voyage/custom | F2 |
devctx-parse | tree-sitter symbols/imports/call-edges + framework route extractors | F3 |
devctx-chunk | semantic multi-level chunker (file/class/function/block) | F3 |
devctx-index | pipeline: git diff → parse → chunk → embed → store (incremental) | F4 |
devctx-rerank | cross-encoder reranking (fastembed BGE) + no-op fallback | F5 |
devctx-search | search orchestration: vector / keyword / hybrid (RRF) + rerank | F8 |
devctx-mcp | MCP server (rmcp, stdio): search / read_file / index_repo / index_status | F6 |
devctx-memory | memory engine: remember (dedup) + recall (intro/chunk blend) | F7 |
devctx-summarize | summarization: extractive (default) + OpenAI + local flan-t5 | F9 |
devctx-api | HTTP REST API (axum) reusing the MCP engine, Bearer-token auth | F9 |
devctx-tui | interactive terminal UI (ratatui): search, graph, memories, projects | F9 |
devctx-central | central store: project registry, global memories, daemon client | — |
The flan-t5 feature (off by default) adds a local abstractive summarizer via
candle — build with --features flan-t5 (heavy; downloads the model on first use).
The devctx-embed local feature (default) pulls in fastembed/ort; build
with --no-default-features for an API-only build where the ONNX Runtime binary
can't be fetched.
cargo build
cargo test
cargo run -p devctx-cli -- status
DevCtxEngine is a ground-up rewrite of snaven10/devai-context-engine (DevAI), which remains the reference implementation. The commit history here carries over from that project, so the migration is visible in the log rather than squashed away.
What DevAI was — a hybrid, ~20k LOC across two runtimes:
| Layer | Size | Responsibility |
|---|---|---|
| Go | ~10.4k LOC | Thin orchestrator: CLI (cobra), MCP server (21 tools), HTTP API, TUI (Bubble Tea), config/storage routing |
Python (devai_ml) | ~9.7k LOC | The actual work: embeddings, chunking, tree-sitter parsers, retrieval/reranking, summarization, stores |
The real contract between them was a JSON-RPC 2.0 bridge over stdio — about 27 methods — with Python running as a sidecar process.
What the rewrite changes:
fastembed-rs + ort (ONNX Runtime) for local models;
OpenAI/Voyage/custom over HTTP. Embedding dimension is parameterized rather
than pinned at 384.The full reasoning, with the cost of each decision, is in
docs/08-design-decisions.md; the system it
describes is specified in
docs/architecture-spec.md.
MIT — see LICENSE.
311 commits
Rust
97.4%
HTML
1.2%
Shell
1.1%