Local-first persistent memory CLI for LLM agents
Rust
0
214 commits
updated May 15, 2026
Local-first persistent memory for LLM agents.
voidm is a single-binary CLI that gives AI agents a durable, searchable, knowledge graph-backed memory store. Add typed memories, search them with hybrid chunk-semantic + BM25/title retrieval enhanced by optional query expansion and reranking, link memories in a graph, persist tags/scopes/types as first-class graph entities, and run practical read-only Cypher queries on Neo4j — all offline, no API keys required.
Status: Production-ready. Quality score 0.9392 (SOTA-competitive). 94% feature parity with state-of-the-art memory engines.
# Install
git clone https://github.com/autonomous-toaster/voidm
cd voidm && cargo install --path crates/voidm-cli
# Create a fresh default config (safe first-run bootstrap)
voidm config init
# Initialize models (pre-download models for offline use)
voidm init
# Add memories
voidm add "Docker chosen for deployment" --type conceptual --tags "containers,devops"
voidm add "Kubernetes orchestrates 100+ containers" --type semantic
voidm add "Deploy: apply manifests, verify rollout" --type procedural
# Add with metadata ranking signals
voidm add "Academic paper on container orchestration" --type semantic --source academic
voidm add "AI-generated summary" --type semantic --author assistant
# Search (automatic query expansion + reranking)
voidm search "deployment strategy" --verbose
# Explore graph
voidm graph neighbors <id> --depth 2
voidm graph cypher "MATCH (a:Memory)-[:SUPPORTS]->(b:Memory) RETURN a.id as from_id, b.id as to_id LIMIT 10"
Multiple retrieval methods with automatic signal fusion:
# Default: hybrid with all signals
voidm search "docker" --verbose
# Semantic only (vector similarity)
voidm search "docker" --mode semantic
# BM25 only (keyword matching)
voidm search "docker" --mode bm25
# With query expansion and reranking
voidm search "deployment" --query-expand true --reranker true
# Filter by scope, type, or tag
voidm search "auth" --scope work/projectx --type semantic --tag oauth,jwt
Performance: Semantic 200ms, BM25 50ms, fuzzy 30ms, reranking +1000ms (optional). Typical total: 300-500ms.
Strict content→tags auto-generation is now implemented behind the tinyllama feature.
What is implemented and proved:
metadata.auto_generated_tagsMemory -> HAS_TAG -> Tag persistence is proved in SQLiteWhat remains caveated:
Usage notes:
tinyllama feature to turn on strict local generation--tags remain the safe/default pathWhen you add a memory, the system automatically links it to related memories that share tags.
# Add memory 1
voidm add "REST API design" --tags "api,http,rest"
# Add memory 2 (automatically linked to memory 1)
voidm add "SOAP for APIs" --tags "api,soap,xml"
# Result: RELATES_TO edge created with note "Shares tags: api"
Features:
Configuration:
[insert]
auto_link = true
auto_link_limit = 5 # Max links per memory
Detects and masks sensitive secrets (API keys, DB credentials, JWT tokens) before storage to prevent leakage into vector DB.
$ voidm add "OpenAI key is sk-1a2b3c4d5e6f7g8h9i0j for API calls"
# ⚠️ Redacted 1 secret: 1 API key
$ voidm search "openai"
# Result: "OpenAI key is sk-...0j for API calls" (masked)
Detects:
sk-...)AKIA...)user:pass@host/db)eyJ...)Preserves context: First/last 3 chars visible (e.g., sk-...0j) to maintain readability.
Configuration:
[redaction]
enabled = true
[redaction.api_keys]
enabled = true
strategy = "mask" # Show start/end
prefix_length = 3
suffix_length = 2
Text is automatically chunked before embedding to ensure consistent quality for all memory sizes.
Benefits:
# All memories chunked automatically during insertion
voidm add "Very long technical documentation..." # Chunked, embedded, stored
voidm search "specific detail" # Finds it despite being in long text
Performance: +0ms for short texts, +50-100ms for large texts (negligible).
Every memory receives an automatic quality score (0.0-1.0) based on:
Current baseline: 0.9392 (SOTA-competitive after optimization)
# Filter by quality
voidm search "pattern" --min-quality 0.8 --limit 10 # Only high-quality
voidm list --min-quality 0.7 --scope work
Search results ranked by multiple signals beyond content matching: recency, author trust, source reliability, and citation counts.
Ranking Signals:
Usage:
# Add user-created memory (default)
voidm add "My important knowledge" --type semantic
# Tag as academic (higher ranking)
voidm add "Research findings" --type semantic --source academic
# Mark as AI-generated (lower confidence)
voidm add "Summary generated by assistant" --author assistant
# Explicit author + source combo
voidm add "Verified research" --author user --source verified
Scoring Formula:
final_score = rrf_score
+ 0.15 * importance_signal
+ 0.1 * quality_signal
+ 0.05 * recency_signal
+ 0.08 * author_trust
+ 0.05 * source_reliability
+ 0.0 * citation_boost (disabled by default)
Configuration (in ~/.config/voidm/config.toml):
[search.metadata_ranking]
weight_importance = 0.15 # Explicit importance (1-10)
weight_quality = 0.1 # Automatic quality score
weight_recency = 0.05 # 30-day decay
weight_author = 0.08 # Author trust tier
weight_source = 0.05 # Source reliability
weight_citations = 0.0 # Disabled by default
recency_half_life_days = 30
[search.metadata_ranking.source_reliability_boost]
academic = 1.0
verified = 0.7
user = 0.4
unknown = 0.0
Link memories with typed, directed edges. No external graph database — pure SQLx with transactional guarantees.
Edge types:
SUPPORTS — A supports BCONTRADICTS — A contradicts BDERIVED_FROM — A derived from BINVALIDATES — A supersedes/invalidates BPART_OF — A part of BRELATES_TO — A relates to B (with optional note)voidm link <memory1> SUPPORTS <memory2>
voidm link <memory1> CONTRADICTS <memory2>
voidm link <memory1> RELATES_TO <memory2> --note "both affect X"
Performance: Sub-millisecond edge traversal, pagerank <100ms for 100K memories.
Define architectural concepts, class hierarchies, and link memories as instances.
# Define concepts
voidm ontology concept add "AuthService" --description "JWT + OAuth2"
voidm ontology concept add "OAuth2" --description "Industry standard"
# IS-A hierarchy
voidm ontology link <oauth2-id> IS_A <auth-service-id>
# Link memory as instance
voidm ontology link <memory-id> INSTANCE_OF <concept-id>
# Query returns all instances + subclass instances (transitive)
voidm ontology concept get <auth-service-id>
# Returns instances of AuthService + OAuth2 (subclass) + JWT (subclass)
Features:
Extract people, organizations, locations from memories using Xenova/bert-base-NER (ONNX, 103MB, downloaded once).
# Batch enrich all memories with NER
voidm ontology enrich-memories --add --scope work
# Extract from single memory
voidm ontology extract <memory-id> --min-score 0.8
# Auto-link to concepts
voidm ontology enrich-memories --add --min-score 0.8
Performance: 150-170ms NER + 80-100ms concept linking = 230-270ms per memory (parallelizable).
Use cross-encoder/nli-deberta-v3-small to classify relations and detect contradictions between texts.
voidm ontology enrich <text1> <text2>
# Output: relation classification + confidence
voidm conflicts list
# Lists all CONTRADICTS edges found
Contradiction threshold: 0.80 (configurable).
Neo4j Backend: Full-featured graph database for large-scale deployments with transactional support, advanced Cypher queries, and enterprise scaling.
# ~/.config/voidm/config.toml
[database]
backend = "neo4j"
host = "neo4j+s://[instance-id].databases.neo4j.io"
username = "[username]"
password = "[password]"
database = "[database-name]" # Usually the instance ID (e.g., "15b4e645")
Nodes:
:Memory — Knowledge item with full metadata:Concept — Ontology concept defining domain entitiesRelationships:
INSTANCE_OF — Memory is instance of ConceptSUPPORTS — Memory A supports conclusion in BCONTRADICTS — Memory A contradicts BDERIVED_FROM — Memory A derived from BINVALIDATES — Memory A supersedes BPART_OF — Memory A is part of BRELATES_TO — Memory A relates to BIS_A — Concept inheritance hierarchyMemory Node:
id: String (UNIQUE)
content: String
type: String (episodic|semantic|procedural|conceptual|contextual)
scopes: [String]
tags: [String]
importance: Integer (1-10)
quality_score: Float (0.0-1.0)
author: String
source: String
created_at: ISO8601 timestamp
metadata: JSON
Concept Node:
id: String (UNIQUE)
name: String (UNIQUE)
description: String
scope: String
created_at: ISO8601 timestamp
1. Find All Memories Linked to a Concept
voidm graph cypher "
MATCH (m:Memory)-[:INSTANCE_OF]->(c:Concept {name: 'REST API'})
RETURN m.id, m.content, m.quality_score, m.importance
ORDER BY m.quality_score DESC
"
2. Find Directly Linked Memories
voidm graph cypher "
MATCH (m:Memory {id: 'a1b2c3d4'})-[r]-(related:Memory)
RETURN type(r) as relationship_type, related.id, related.content
"
3. Transitive Closure (Multi-Hop Relationships)
voidm graph cypher "
MATCH (m1:Memory)-[r*1..3]->(m2:Memory)
RETURN m1.id, m2.id, LENGTH(r) as hops
LIMIT 50
"
4. Concept Hierarchy (IS-A Relationships)
voidm graph cypher "
MATCH (parent:Concept {name: 'Authentication'})<-[:IS_A*1..3]-(child:Concept)
RETURN child.name, child.description
"
5. Find All Instances of Concept + Subclasses
voidm graph cypher "
MATCH (parent:Concept {name: 'Authentication'})<-[:IS_A*0..3]-(subclass:Concept)
MATCH (m:Memory)-[:INSTANCE_OF]->(subclass)
RETURN parent.name, subclass.name, COUNT(m) as memory_count
ORDER BY memory_count DESC
"
6. High-Quality Memories by Concept
voidm graph cypher "
MATCH (m:Memory)-[:INSTANCE_OF]->(c:Concept)
WHERE m.quality_score > 0.85
RETURN c.name, m.id, m.quality_score
ORDER BY c.name, m.quality_score DESC
"
7. Find Contradictions
voidm graph cypher "
MATCH (m1:Memory)-[:CONTRADICTS]->(m2:Memory)
RETURN m1.id, m1.content, m2.id, m2.content
ORDER BY m1.created_at DESC
"
8. Knowledge Graph Hubs (Most Connected)
voidm graph cypher "
MATCH (m:Memory)-[r]-(connected)
RETURN m.id, m.content, COUNT(r) as connections, COUNT(DISTINCT connected) as neighbors
ORDER BY connections DESC
LIMIT 20
"
9. Memories Supporting Others
voidm graph cypher "
MATCH (m:Memory)-[:SUPPORTS]->(other:Memory)
RETURN m.id, m.content, COUNT(other) as supports_count
ORDER BY supports_count DESC
LIMIT 10
"
10. Recent High-Importance Memories
voidm graph cypher "
MATCH (m:Memory)
WHERE m.importance >= 7
RETURN m.id, m.content, m.created_at, m.quality_score
ORDER BY m.created_at DESC
LIMIT 20
"
11. Concept Frequency by Scope
voidm graph cypher "
MATCH (m:Memory)-[:INSTANCE_OF]->(c:Concept)
UNWIND m.scopes as scope
RETURN c.name, scope, COUNT(DISTINCT m) as memory_count
ORDER BY memory_count DESC
"
12. Knowledge Derivation Chain
voidm graph cypher "
MATCH (m1:Memory)-[:DERIVED_FROM*1..5]->(m2:Memory)
RETURN m1.id, m2.id, m1.created_at, m2.created_at
ORDER BY m2.created_at, m1.created_at
"
| Query Type | Latency | Notes |
|---|---|---|
| Single lookup | <10ms | {id: '...'} |
| 1-hop neighbors | 50-100ms | Direct connections |
| Concept hierarchy | 100-300ms | With IS_A traversal |
| Complex multi-hop | 500-2000ms | Transitive closure |
| Aggregation | 100-1000ms | GROUP BY operations |
For Aura instances, open the Browser at https://console.neo4j.io and run:
MATCH (m:Memory)-[r:INSTANCE_OF]->(c:Concept)
RETURN m, r, c LIMIT 100
Supported Cypher: MATCH, WHERE, RETURN, ORDER BY, LIMIT, WITH, UNWIND, relationship traversal, aggregations. Write operations rejected.
Expose voidm as an MCP server over stdio for integration with Claude, other AI assistants, and agents.
# Start MCP server
voidm mcp --transport stdio
# Use with mcporter or other MCP clients
npx -y mcporter call \
--stdio ./voidm \
--stdio-arg mcp \
--stdio-arg --transport \
--stdio-arg stdio \
search_memories query=docker mode=semantic limit=5
Tools exposed:
search_memories — Hybrid search with intent/scope/type filtersadd_memory — Store memory with quality_score and warningsdelete_memory, link_memories, unlink_memoriesget_concepts, add_concept, link_memory_to_conceptsearch_concepts — Search and list conceptsChoose which features to enable based on your use case:
| Feature | Enabled | Latency | Storage | Notes |
|---|---|---|---|---|
| Core | ||||
| Memory CRUD | ✅ Always | <10ms | Minimal | Required foundation |
| Hybrid Search (BM25 + Semantic) | ✅ Default | 250ms | +50MB FTS index | Most searches |
| Vector Embeddings (7 models) | ✅ Default | 200ms | 50-200MB models | Downloaded once |
| Quality Scoring | ✅ Default | <1ms | Minimal | Automatic per memory |
| Search Enhancement | ||||
| Query Expansion (HyDE) | ✅ Optional | +300-500ms | 1.5-2.7GB models | Better recall, slower |
| Reranking (Cross-encoder) | ❌ Disabled | +1000ms | 100-250MB model | High precision, slow |
| Graph Retrieval (Tags + Concepts) | ✅ Default | +200-500ms | Minimal | More recall |
| Knowledge Organization | ||||
| Knowledge Graph | ✅ Always | <1ms edges | <1MB per 1K edges | Typed relationships |
| Ontology (Concepts + IS-A) | ✅ Default | <10ms | <1MB per 100 concepts | Hierarchical classes |
| Auto-Tagging | ✅ Feature-gated (tinyllama) | model-dependent | local LLM model/runtime | Strict generated tags in normal add flow |
| Auto-Linking | ✅ Optional | +50-100ms | Minimal | Discoverable graph |
| Advanced Features | ||||
| NER (Named Entity Recognition) | ✅ Optional | 150-170ms | 103MB model | Entity extraction |
| NLI (Relation Classification) | ✅ Optional | 100-200ms | 200MB model | Contradiction detection |
| Secrets Redaction | ✅ Optional | <100ms | Minimal | Prevent leakage |
| Text Chunking (Long Content) | ✅ Default | +50-100ms large | Minimal | Consistent embeddings |
| Batch NER Enrichment | ❌ Manual | 230-270ms/mem | 103MB model | On-demand concept linking |
| Storage | ||||
| SQLite (Embedded) | ✅ Default | <10ms | Depends on size | Transactional |
| PostgreSQL (Adapter) | ⚠️ Experimental | Network latency | Depends on size | For multi-user |
| Export | ||||
| HTML Visualization | ✅ Optional | <5s | 1-50MB | Interactive force-directed |
| Cypher Queries | ✅ Optional | <1s | Minimal | Read-only traversal |
| CSV Export | ✅ Optional | <1s | 1-50MB | Spreadsheet compatible |
| JSON Export | ✅ Optional | <1s | 1-50MB | Machine-readable |
| Integration | ||||
| MCP Server | ✅ Optional | Stdio | Minimal | AI assistant integration |
| CLI (Single Binary) | ✅ Always | N/A | ~30MB binary | No dependencies |
[search]
mode = "hybrid" # BM25 + Semantic only
min_quality = 0.7
[search.query_expansion]
enabled = false # Skip expansion (faster but less recall)
[search.reranker]
enabled = false # Skip reranking
[search.graph_retrieval]
enabled = true # Keep graph (cheap, helps recall)
[tagging]
enabled = true # Auto-tags (75ms, worth it)
[insert]
auto_link = true # Cheap linking
auto_link_limit = 3
Performance: ~300ms average search, <50ms memory add.
[search]
mode = "hybrid" # All signals (BM25 + Semantic + Graph)
min_quality = 0.75
[search.query_expansion]
enabled = true
model = "tinyllama" # 1.1B, balanced speed/quality
timeout_ms = 300
[search.reranker]
enabled = false # Reranking usually unnecessary with RRF
[search.graph_retrieval]
enabled = true
max_concept_hops = 2
[tagging]
enabled = true
ner_enabled = true
tf_enabled = true
[insert]
auto_link = true
auto_link_limit = 5
Performance: ~500-700ms search, +100ms memory add. Best recall/speed tradeoff.
[search]
mode = "hybrid"
min_quality = 0.6 # Include borderline memories
[search.query_expansion]
enabled = true
model = "phi-2" # 2.7B, highest quality
timeout_ms = 500
[search.reranker]
enabled = true
model = "ms-marco-MiniLM-L-6-v2" # 100MB, ~1s latency
apply_to_top_k = 20
[search.graph_retrieval]
enabled = true
max_concept_hops = 3 # More aggressive concept expansion
[tagging]
enabled = true
ner_enabled = true
tf_enabled = true
min_tf_score = 0.2 # Lower threshold = more tags
[insert]
auto_link = true
auto_link_limit = 10 # Link to more neighbors
[redaction]
enabled = true # Protect secrets
Performance: ~1.5-2s search, +150ms memory add. Highest recall, slower.
[search]
mode = "hybrid"
min_quality = 0.75
[search.query_expansion]
enabled = true
model = "tinyllama"
timeout_ms = 300
[search.reranker]
enabled = false
[search.graph_retrieval]
enabled = true
max_concept_hops = 2
[tagging]
enabled = true
ner_enabled = true
[ontology]
enabled = true
auto_link_concepts = true
[redaction]
enabled = true
[mcp]
enabled = true # Expose as MCP server
Best for: Claude integration, agent consumption, search_memories tool calls.
voidm/
├── crates/
│ ├── voidm-core/ # CRUD, hybrid search, quality scoring, NER/NLI
│ ├── voidm-sqlite/ # SQLite backend (default)
│ ├── voidm-postgres/ # PostgreSQL backend (experimental)
│ ├── voidm-neo4j/ # Neo4j backend (production-ready)
│ ├── voidm-embeddings/ # Fastembed + text chunking
│ ├── voidm-query-expansion/ # HyDE template + LLM inference
│ ├── voidm-reranker/ # Cross-encoder ranking
│ ├── voidm-graph/ # EAV schema + Cypher translator
│ ├── voidm-tagging/ # NER + TF tagging
│ ├── voidm-ner/ # Entity extraction (ONNX)
│ ├── voidm-nli/ # Relation classification (ONNX)
│ ├── voidm-redactor/ # Secrets detection + masking
│ ├── voidm-scoring/ # Quality score computation
│ ├── voidm-models/ # Model management + download
│ ├── voidm-mcp/ # MCP server implementation
│ └── voidm-cli/ # CLI + JSON output
└── migrations/ # SQLite schema (sqlx) + Migration scripts
Storage:
~/.local/share/voidm/memories.db (embedded, transactional, 100MB+ for large bases)~/.config/voidm/config.toml~/.cache/voidm/ (embeddings, NER, NLI, query expansion)Backend Selection (in config.toml):
[database]
backend = "sqlite" # Options: sqlite, neo4j
Environment Variables:
VOIDM_CONFIG — Override config location (useful for testing multiple backends)VOIDM_DB — Override SQLite database path (SQLite backend only)Search Pipeline:
Query
├→ Query Expansion (optional) → expanded query
├→ Semantic Search (embeddings + ANN) → results₁
├→ BM25 Search (FTS5) → results₂
├→ Fuzzy Search (Levenshtein) → results₃
├→ Graph Retrieval (tag overlap + concepts) → results₄
├→ RRF Fusion (merge 1-4) → ranked results
└→ Reranking (optional, cross-encoder) → final ranking
| Operation | Latency | Dataset |
|---|---|---|
| Add memory | 100-150ms | N/A |
| Add + Auto-tagging | 150-200ms | N/A |
| Add + Auto-linking | 200-300ms | 10K memories |
| Semantic search | 150-250ms | 100K memories |
| BM25 search | 30-100ms | 100K memories |
| Hybrid search | 300-500ms | 100K memories |
| With query expansion | 600-1000ms | 100K memories |
| With reranking | 1000-1500ms | 100K memories |
| Graph neighbors (depth 2) | 50-200ms | 100K memories |
| Cypher query | 100-1000ms | 100K memories, complex queries |
| Pagerank | 50-150ms | 100K memories |
quality_score = base_score * type_boost * temporal_factor * substance_factor * abstraction_factor
where:
base_score = 0.5 (foundation)
type_boost:
episodic = +0.035 (reliable, specific)
semantic = +0.025 (factual, general)
conceptual/procedural/contextual = +0.015 (variable)
temporal_factor = e^(-λ*days_old) with 30-day half-life
substance_factor = signal-to-noise (dense vs. generic)
abstraction_factor = well-defined vs. vague
Current production baseline: 0.9392 (optimized across 26 variations).
✅ Text Chunking: Consistent smart chunks (target 600 chars, 100-char overlap) for large memories
✅ HyDE Query Expansion: Hypothetical document generation for better semantic search
✅ Graph Retrieval in RRF: Tag and concept-based result expansion integrated into search
✅ NER Feature Gating: Optional dependency handling with clean builds
✅ Quality Score Optimization: 26 iterations reaching 0.9392 (SOTA-level)
✅ Neo4j Backend: Production-ready enterprise graph database support with full Cypher query language
✅ Backend Independence: CLI backend routing with VOIDM_CONFIG environment variable support
✅ Ontology Export Complete: 92 concepts + 131 memory-concept relationships (INSTANCE_OF edges) migrated
✅ MERGE Upsert Pattern: Graceful duplicate handling - duplicate IDs update existing records instead of failing
✅ Metadata Tracking: Author and source fields added to all memory operations for audit trails
Complete Neo4j Export Status (Session 2026-03-23):
Cypher Verification Queries:
# Verify total counts
voidm graph cypher "MATCH (m:Memory) RETURN COUNT(m) as memories"
voidm graph cypher "MATCH (c:Concept) RETURN COUNT(c) as concepts"
voidm graph cypher "MATCH ()-[r:INSTANCE_OF]->() RETURN COUNT(r) as edges"
# Check data integrity (no duplicates)
voidm graph cypher "MATCH (m:Memory) WITH m.id as id, COUNT(*) as cnt WHERE cnt > 1 RETURN COUNT(*) as duplicates"
# Sample memory-concept mapping
voidm graph cypher "MATCH (m:Memory)-[:INSTANCE_OF]->(c:Concept) RETURN c.name, COUNT(m) as instance_count ORDER BY instance_count DESC LIMIT 10"
Short-term (2 weeks):
Medium-term (Month 2): 4. Smart Memory Eviction (LRU + semantic similarity) → 10x+ scale 5. Active Learning for Importance Scoring → self-improving 6. Temporal Decay Ranking → knowledge freshness
Long-term (Q2+): 7. RDF Triple Storage → structured reasoning 8. Hierarchical Memory Compression → token efficiency 9. Metadata-Driven Ranking → semantic signals
# Clone and build
git clone https://github.com/autonomous-toaster/voidm
cd voidm
cargo install --path crates/voidm-cli
# Or build manually
cargo build --release
cp target/release/voidm ~/.local/bin/
# Initialize (download models for offline use)
voidm init
Requirements: Rust 1.94.0+, SQLite (bundled).
Models: Automatically downloaded to ~/.cache/voidm/ on first use (~300-400MB). Idempotent.
Use one of the three supported CLI profiles:
| Profile | Summary | Build Command |
|---|---|---|
| minimal | Lean local CLI with SQLite | cargo build --release --no-default-features --features minimal |
| standard | Recommended default profile | cargo build --release |
| full | Standard + Neo4j + MCP + experimental llama.cpp | cargo build --release --no-default-features --features full |
Public docs should prefer these profiles. Lower-level feature toggles still exist for local/dev composition.
# Default build (STANDARD profile, all recommended features)
cargo build --release
cp target/release/voidm ~/.local/bin/
# Or install directly
cargo install --path crates/voidm-cli
# For minimal deployments
cargo build --release --no-default-features --features minimal
# For the maximal curated CLI profile
cargo build --release --no-default-features --features full
Build custom combinations:
# Just embeddings + search (no NER/NLI/reranking)
cargo build --release --no-default-features --features "database-sqlite,database-postgres,embeddings,vector-search,tinyllama"
# Minimal + reranking only
cargo build --release --no-default-features --features "minimal,reranker"
# See all available features
cargo build --release --no-default-features --features "" 2>&1 | grep "unknown feature"
Available individual features:
database-sqlite, database-postgres, database-neo4jembeddings, vector-search, query-expansionnli, ner, rerankertinyllama, mcp, redactor| Command | Description |
|---|---|
voidm add <text> | Add memory. Returns suggested_links, duplicate_warning. |
voidm get <id> | Retrieve by ID or 4+ char prefix. |
voidm list | List all, filterable by scope/type/quality. |
voidm search <query> | Hybrid search. Modes: hybrid/semantic/bm25/fuzzy/keyword/vector. |
voidm delete <id> | Delete memory. |
voidm link <from> <EDGE> <to> | Create graph edge. RELATES_TO needs --note. |
voidm export | Export memories as JSON. |
| Command | Description |
|---|---|
voidm graph neighbors <id> | N-hop neighbors (--depth, default 1). |
voidm graph pagerank --top 10 | Rank by centrality. |
voidm graph cypher "<query>" | Read-only Cypher traversal. |
voidm graph export --format html | Interactive visualization (html/dot/json/csv). |
| Command | Description |
|---|---|
voidm init | Pre-download configured local models. |
voidm config init/show/set | Create and manage configuration. |
voidm info | Show backend, config and runtime settings. |
voidm stats | Memory and graph statistics. |
voidm migrate --from <backend> --to <backend> | Migrate data between sqlite and neo4j. |
Use --json for machine-readable output. Use --help for full flag details.
voidm add Flags| Flag | Values | Default | Description |
|---|---|---|---|
--type (required) | episodic, semantic, procedural, conceptual, contextual | — | Memory type affects quality scoring |
--scope | any string | — | Organizational context (repeatable). E.g., --scope work/project/backend |
--tags | comma-separated | — | Custom tags for filtering/linking (no max, overwrites auto-tags) |
--importance | 1-10 | 5 | Manual importance level (boosts ranking) |
--author | user, assistant, unknown | user | Author trust tier (affects ranking) |
--source | academic, verified, user, unknown | unknown | Source reliability (affects ranking) |
--link | <id>:<TYPE> or <id>:<TYPE>:<note> | — | Auto-link to existing memory. RELATES_TO requires --note |
--db | path | ~/.local/share/voidm/memories.db | Override database location |
--json | — | — | Machine-readable JSON output |
--quiet | — | — | Suppress decorative output |
Examples:
# Basic
voidm add "My knowledge" --type semantic
# With metadata
voidm add "Research" --type semantic --author user --source academic --importance 9
# Multiple scopes and tags
voidm add "Info" --scope work/proj/backend --scope personal --tags rust,performance,api
# With auto-linking
voidm add "New fact" --type semantic --link a1b2c3d4:SUPPORTS:"Explains core concept"
# Add foundational knowledge
voidm add "Project X uses Postgres for ACID" --type conceptual --scope work/projectx
voidm add "Deployment via GitHub Actions + Docker" --type procedural --scope work/projectx
voidm add "Auth uses OAuth2 with JWT tokens" --type semantic --scope work/projectx
# Define ontology
voidm ontology concept add "ProjectX" --description "Internal web platform"
# Search with bounded context and stricter filtering
voidm search "how do we authenticate" --scope work/projectx --tag oauth,jwt --min-score 0.7
# Visualize
voidm graph export --format html > projectx-graph.html
open projectx-graph.html
# Find all decisions that were invalidated by later decisions
voidm graph cypher "
MATCH (old:Memory)-[:INVALIDATES]->(new:Memory)
RETURN old.memory_id as invalidated, new.memory_id as invalidates
ORDER BY old.created_at DESC
LIMIT 20
"
# Find most central memories (hubs)
voidm graph pagerank --top 10
# Explore concept hierarchy
voidm graph cypher "
MATCH (child:Concept)-[:IS_A*1..2]->(parent:Concept)
WHERE child.name CONTAINS 'Service'
RETURN child.name, parent.name
"
Minimal (Lightweight):
[search]
mode = "hybrid"
[search.query_expansion]
enabled = false
[tagging]
enabled = true
Production (Balanced):
[search]
mode = "hybrid"
min_quality = 0.75
[search.query_expansion]
enabled = true
model = "tinyllama"
timeout_ms = 300
[search.graph_retrieval]
enabled = true
max_concept_hops = 2
[tagging]
enabled = true
ner_enabled = true
[redaction]
enabled = true
[insert]
auto_link = true
auto_link_limit = 5
High-Recall (Quality):
[search]
mode = "hybrid"
min_quality = 0.6
[search.query_expansion]
enabled = true
model = "phi-2"
timeout_ms = 500
[search.reranker]
enabled = true
model = "ms-marco-MiniLM-L-6-v2"
[search.graph_retrieval]
enabled = true
max_concept_hops = 3
[tagging]
enabled = true
ner_enabled = true
[insert]
auto_link = true
auto_link_limit = 10
| Issue | Solution |
|---|---|
| Slow first search | Normal (model download + embedding cache warmup). Subsequent searches <500ms. |
| "Model not found" | Run voidm init to download. Models cached in ~/.cache/voidm/. |
| High memory usage | Large datasets in SQLite. Consider PostgreSQL adapter or archiving old memories. |
| Search returns nothing | Enable query expansion (--query-expand true) or lower min_quality threshold. |
| Duplicate-like results | Enable reranking (--reranker true) to improve ordering. |
| Secrets not redacted | Check config [redaction] enabled. Run voidm config show to verify. |
| Code | Meaning |
|---|---|
0 | Success |
1 | Not found |
2 | Error (bad args, write Cypher rejected, etc.) |
Inspired by byteowlz/mmry and colliery-io/graphqlite.
Built with ❤️ using fastembed-rs, sqlx, ort (ONNX Runtime), and pi-coding-agent.
MIT — see LICENSE.
214 commits
Rust
99.7%
Local-first persistent memory CLI for LLM agents
Rust
0
214 commits
updated May 15, 2026
Local-first persistent memory for LLM agents.
voidm is a single-binary CLI that gives AI agents a durable, searchable, knowledge graph-backed memory store. Add typed memories, search them with hybrid chunk-semantic + BM25/title retrieval enhanced by optional query expansion and reranking, link memories in a graph, persist tags/scopes/types as first-class graph entities, and run practical read-only Cypher queries on Neo4j — all offline, no API keys required.
Status: Production-ready. Quality score 0.9392 (SOTA-competitive). 94% feature parity with state-of-the-art memory engines.
# Install
git clone https://github.com/autonomous-toaster/voidm
cd voidm && cargo install --path crates/voidm-cli
# Create a fresh default config (safe first-run bootstrap)
voidm config init
# Initialize models (pre-download models for offline use)
voidm init
# Add memories
voidm add "Docker chosen for deployment" --type conceptual --tags "containers,devops"
voidm add "Kubernetes orchestrates 100+ containers" --type semantic
voidm add "Deploy: apply manifests, verify rollout" --type procedural
# Add with metadata ranking signals
voidm add "Academic paper on container orchestration" --type semantic --source academic
voidm add "AI-generated summary" --type semantic --author assistant
# Search (automatic query expansion + reranking)
voidm search "deployment strategy" --verbose
# Explore graph
voidm graph neighbors <id> --depth 2
voidm graph cypher "MATCH (a:Memory)-[:SUPPORTS]->(b:Memory) RETURN a.id as from_id, b.id as to_id LIMIT 10"
Multiple retrieval methods with automatic signal fusion:
# Default: hybrid with all signals
voidm search "docker" --verbose
# Semantic only (vector similarity)
voidm search "docker" --mode semantic
# BM25 only (keyword matching)
voidm search "docker" --mode bm25
# With query expansion and reranking
voidm search "deployment" --query-expand true --reranker true
# Filter by scope, type, or tag
voidm search "auth" --scope work/projectx --type semantic --tag oauth,jwt
Performance: Semantic 200ms, BM25 50ms, fuzzy 30ms, reranking +1000ms (optional). Typical total: 300-500ms.
Strict content→tags auto-generation is now implemented behind the tinyllama feature.
What is implemented and proved:
metadata.auto_generated_tagsMemory -> HAS_TAG -> Tag persistence is proved in SQLiteWhat remains caveated:
Usage notes:
tinyllama feature to turn on strict local generation--tags remain the safe/default pathWhen you add a memory, the system automatically links it to related memories that share tags.
# Add memory 1
voidm add "REST API design" --tags "api,http,rest"
# Add memory 2 (automatically linked to memory 1)
voidm add "SOAP for APIs" --tags "api,soap,xml"
# Result: RELATES_TO edge created with note "Shares tags: api"
Features:
Configuration:
[insert]
auto_link = true
auto_link_limit = 5 # Max links per memory
Detects and masks sensitive secrets (API keys, DB credentials, JWT tokens) before storage to prevent leakage into vector DB.
$ voidm add "OpenAI key is sk-1a2b3c4d5e6f7g8h9i0j for API calls"
# ⚠️ Redacted 1 secret: 1 API key
$ voidm search "openai"
# Result: "OpenAI key is sk-...0j for API calls" (masked)
Detects:
sk-...)AKIA...)user:pass@host/db)eyJ...)Preserves context: First/last 3 chars visible (e.g., sk-...0j) to maintain readability.
Configuration:
[redaction]
enabled = true
[redaction.api_keys]
enabled = true
strategy = "mask" # Show start/end
prefix_length = 3
suffix_length = 2
Text is automatically chunked before embedding to ensure consistent quality for all memory sizes.
Benefits:
# All memories chunked automatically during insertion
voidm add "Very long technical documentation..." # Chunked, embedded, stored
voidm search "specific detail" # Finds it despite being in long text
Performance: +0ms for short texts, +50-100ms for large texts (negligible).
Every memory receives an automatic quality score (0.0-1.0) based on:
Current baseline: 0.9392 (SOTA-competitive after optimization)
# Filter by quality
voidm search "pattern" --min-quality 0.8 --limit 10 # Only high-quality
voidm list --min-quality 0.7 --scope work
Search results ranked by multiple signals beyond content matching: recency, author trust, source reliability, and citation counts.
Ranking Signals:
Usage:
# Add user-created memory (default)
voidm add "My important knowledge" --type semantic
# Tag as academic (higher ranking)
voidm add "Research findings" --type semantic --source academic
# Mark as AI-generated (lower confidence)
voidm add "Summary generated by assistant" --author assistant
# Explicit author + source combo
voidm add "Verified research" --author user --source verified
Scoring Formula:
final_score = rrf_score
+ 0.15 * importance_signal
+ 0.1 * quality_signal
+ 0.05 * recency_signal
+ 0.08 * author_trust
+ 0.05 * source_reliability
+ 0.0 * citation_boost (disabled by default)
Configuration (in ~/.config/voidm/config.toml):
[search.metadata_ranking]
weight_importance = 0.15 # Explicit importance (1-10)
weight_quality = 0.1 # Automatic quality score
weight_recency = 0.05 # 30-day decay
weight_author = 0.08 # Author trust tier
weight_source = 0.05 # Source reliability
weight_citations = 0.0 # Disabled by default
recency_half_life_days = 30
[search.metadata_ranking.source_reliability_boost]
academic = 1.0
verified = 0.7
user = 0.4
unknown = 0.0
Link memories with typed, directed edges. No external graph database — pure SQLx with transactional guarantees.
Edge types:
SUPPORTS — A supports BCONTRADICTS — A contradicts BDERIVED_FROM — A derived from BINVALIDATES — A supersedes/invalidates BPART_OF — A part of BRELATES_TO — A relates to B (with optional note)voidm link <memory1> SUPPORTS <memory2>
voidm link <memory1> CONTRADICTS <memory2>
voidm link <memory1> RELATES_TO <memory2> --note "both affect X"
Performance: Sub-millisecond edge traversal, pagerank <100ms for 100K memories.
Define architectural concepts, class hierarchies, and link memories as instances.
# Define concepts
voidm ontology concept add "AuthService" --description "JWT + OAuth2"
voidm ontology concept add "OAuth2" --description "Industry standard"
# IS-A hierarchy
voidm ontology link <oauth2-id> IS_A <auth-service-id>
# Link memory as instance
voidm ontology link <memory-id> INSTANCE_OF <concept-id>
# Query returns all instances + subclass instances (transitive)
voidm ontology concept get <auth-service-id>
# Returns instances of AuthService + OAuth2 (subclass) + JWT (subclass)
Features:
Extract people, organizations, locations from memories using Xenova/bert-base-NER (ONNX, 103MB, downloaded once).
# Batch enrich all memories with NER
voidm ontology enrich-memories --add --scope work
# Extract from single memory
voidm ontology extract <memory-id> --min-score 0.8
# Auto-link to concepts
voidm ontology enrich-memories --add --min-score 0.8
Performance: 150-170ms NER + 80-100ms concept linking = 230-270ms per memory (parallelizable).
Use cross-encoder/nli-deberta-v3-small to classify relations and detect contradictions between texts.
voidm ontology enrich <text1> <text2>
# Output: relation classification + confidence
voidm conflicts list
# Lists all CONTRADICTS edges found
Contradiction threshold: 0.80 (configurable).
Neo4j Backend: Full-featured graph database for large-scale deployments with transactional support, advanced Cypher queries, and enterprise scaling.
# ~/.config/voidm/config.toml
[database]
backend = "neo4j"
host = "neo4j+s://[instance-id].databases.neo4j.io"
username = "[username]"
password = "[password]"
database = "[database-name]" # Usually the instance ID (e.g., "15b4e645")
Nodes:
:Memory — Knowledge item with full metadata:Concept — Ontology concept defining domain entitiesRelationships:
INSTANCE_OF — Memory is instance of ConceptSUPPORTS — Memory A supports conclusion in BCONTRADICTS — Memory A contradicts BDERIVED_FROM — Memory A derived from BINVALIDATES — Memory A supersedes BPART_OF — Memory A is part of BRELATES_TO — Memory A relates to BIS_A — Concept inheritance hierarchyMemory Node:
id: String (UNIQUE)
content: String
type: String (episodic|semantic|procedural|conceptual|contextual)
scopes: [String]
tags: [String]
importance: Integer (1-10)
quality_score: Float (0.0-1.0)
author: String
source: String
created_at: ISO8601 timestamp
metadata: JSON
Concept Node:
id: String (UNIQUE)
name: String (UNIQUE)
description: String
scope: String
created_at: ISO8601 timestamp
1. Find All Memories Linked to a Concept
voidm graph cypher "
MATCH (m:Memory)-[:INSTANCE_OF]->(c:Concept {name: 'REST API'})
RETURN m.id, m.content, m.quality_score, m.importance
ORDER BY m.quality_score DESC
"
2. Find Directly Linked Memories
voidm graph cypher "
MATCH (m:Memory {id: 'a1b2c3d4'})-[r]-(related:Memory)
RETURN type(r) as relationship_type, related.id, related.content
"
3. Transitive Closure (Multi-Hop Relationships)
voidm graph cypher "
MATCH (m1:Memory)-[r*1..3]->(m2:Memory)
RETURN m1.id, m2.id, LENGTH(r) as hops
LIMIT 50
"
4. Concept Hierarchy (IS-A Relationships)
voidm graph cypher "
MATCH (parent:Concept {name: 'Authentication'})<-[:IS_A*1..3]-(child:Concept)
RETURN child.name, child.description
"
5. Find All Instances of Concept + Subclasses
voidm graph cypher "
MATCH (parent:Concept {name: 'Authentication'})<-[:IS_A*0..3]-(subclass:Concept)
MATCH (m:Memory)-[:INSTANCE_OF]->(subclass)
RETURN parent.name, subclass.name, COUNT(m) as memory_count
ORDER BY memory_count DESC
"
6. High-Quality Memories by Concept
voidm graph cypher "
MATCH (m:Memory)-[:INSTANCE_OF]->(c:Concept)
WHERE m.quality_score > 0.85
RETURN c.name, m.id, m.quality_score
ORDER BY c.name, m.quality_score DESC
"
7. Find Contradictions
voidm graph cypher "
MATCH (m1:Memory)-[:CONTRADICTS]->(m2:Memory)
RETURN m1.id, m1.content, m2.id, m2.content
ORDER BY m1.created_at DESC
"
8. Knowledge Graph Hubs (Most Connected)
voidm graph cypher "
MATCH (m:Memory)-[r]-(connected)
RETURN m.id, m.content, COUNT(r) as connections, COUNT(DISTINCT connected) as neighbors
ORDER BY connections DESC
LIMIT 20
"
9. Memories Supporting Others
voidm graph cypher "
MATCH (m:Memory)-[:SUPPORTS]->(other:Memory)
RETURN m.id, m.content, COUNT(other) as supports_count
ORDER BY supports_count DESC
LIMIT 10
"
10. Recent High-Importance Memories
voidm graph cypher "
MATCH (m:Memory)
WHERE m.importance >= 7
RETURN m.id, m.content, m.created_at, m.quality_score
ORDER BY m.created_at DESC
LIMIT 20
"
11. Concept Frequency by Scope
voidm graph cypher "
MATCH (m:Memory)-[:INSTANCE_OF]->(c:Concept)
UNWIND m.scopes as scope
RETURN c.name, scope, COUNT(DISTINCT m) as memory_count
ORDER BY memory_count DESC
"
12. Knowledge Derivation Chain
voidm graph cypher "
MATCH (m1:Memory)-[:DERIVED_FROM*1..5]->(m2:Memory)
RETURN m1.id, m2.id, m1.created_at, m2.created_at
ORDER BY m2.created_at, m1.created_at
"
| Query Type | Latency | Notes |
|---|---|---|
| Single lookup | <10ms | {id: '...'} |
| 1-hop neighbors | 50-100ms | Direct connections |
| Concept hierarchy | 100-300ms | With IS_A traversal |
| Complex multi-hop | 500-2000ms | Transitive closure |
| Aggregation | 100-1000ms | GROUP BY operations |
For Aura instances, open the Browser at https://console.neo4j.io and run:
MATCH (m:Memory)-[r:INSTANCE_OF]->(c:Concept)
RETURN m, r, c LIMIT 100
Supported Cypher: MATCH, WHERE, RETURN, ORDER BY, LIMIT, WITH, UNWIND, relationship traversal, aggregations. Write operations rejected.
Expose voidm as an MCP server over stdio for integration with Claude, other AI assistants, and agents.
# Start MCP server
voidm mcp --transport stdio
# Use with mcporter or other MCP clients
npx -y mcporter call \
--stdio ./voidm \
--stdio-arg mcp \
--stdio-arg --transport \
--stdio-arg stdio \
search_memories query=docker mode=semantic limit=5
Tools exposed:
search_memories — Hybrid search with intent/scope/type filtersadd_memory — Store memory with quality_score and warningsdelete_memory, link_memories, unlink_memoriesget_concepts, add_concept, link_memory_to_conceptsearch_concepts — Search and list conceptsChoose which features to enable based on your use case:
| Feature | Enabled | Latency | Storage | Notes |
|---|---|---|---|---|
| Core | ||||
| Memory CRUD | ✅ Always | <10ms | Minimal | Required foundation |
| Hybrid Search (BM25 + Semantic) | ✅ Default | 250ms | +50MB FTS index | Most searches |
| Vector Embeddings (7 models) | ✅ Default | 200ms | 50-200MB models | Downloaded once |
| Quality Scoring | ✅ Default | <1ms | Minimal | Automatic per memory |
| Search Enhancement | ||||
| Query Expansion (HyDE) | ✅ Optional | +300-500ms | 1.5-2.7GB models | Better recall, slower |
| Reranking (Cross-encoder) | ❌ Disabled | +1000ms | 100-250MB model | High precision, slow |
| Graph Retrieval (Tags + Concepts) | ✅ Default | +200-500ms | Minimal | More recall |
| Knowledge Organization | ||||
| Knowledge Graph | ✅ Always | <1ms edges | <1MB per 1K edges | Typed relationships |
| Ontology (Concepts + IS-A) | ✅ Default | <10ms | <1MB per 100 concepts | Hierarchical classes |
| Auto-Tagging | ✅ Feature-gated (tinyllama) | model-dependent | local LLM model/runtime | Strict generated tags in normal add flow |
| Auto-Linking | ✅ Optional | +50-100ms | Minimal | Discoverable graph |
| Advanced Features | ||||
| NER (Named Entity Recognition) | ✅ Optional | 150-170ms | 103MB model | Entity extraction |
| NLI (Relation Classification) | ✅ Optional | 100-200ms | 200MB model | Contradiction detection |
| Secrets Redaction | ✅ Optional | <100ms | Minimal | Prevent leakage |
| Text Chunking (Long Content) | ✅ Default | +50-100ms large | Minimal | Consistent embeddings |
| Batch NER Enrichment | ❌ Manual | 230-270ms/mem | 103MB model | On-demand concept linking |
| Storage | ||||
| SQLite (Embedded) | ✅ Default | <10ms | Depends on size | Transactional |
| PostgreSQL (Adapter) | ⚠️ Experimental | Network latency | Depends on size | For multi-user |
| Export | ||||
| HTML Visualization | ✅ Optional | <5s | 1-50MB | Interactive force-directed |
| Cypher Queries | ✅ Optional | <1s | Minimal | Read-only traversal |
| CSV Export | ✅ Optional | <1s | 1-50MB | Spreadsheet compatible |
| JSON Export | ✅ Optional | <1s | 1-50MB | Machine-readable |
| Integration | ||||
| MCP Server | ✅ Optional | Stdio | Minimal | AI assistant integration |
| CLI (Single Binary) | ✅ Always | N/A | ~30MB binary | No dependencies |
[search]
mode = "hybrid" # BM25 + Semantic only
min_quality = 0.7
[search.query_expansion]
enabled = false # Skip expansion (faster but less recall)
[search.reranker]
enabled = false # Skip reranking
[search.graph_retrieval]
enabled = true # Keep graph (cheap, helps recall)
[tagging]
enabled = true # Auto-tags (75ms, worth it)
[insert]
auto_link = true # Cheap linking
auto_link_limit = 3
Performance: ~300ms average search, <50ms memory add.
[search]
mode = "hybrid" # All signals (BM25 + Semantic + Graph)
min_quality = 0.75
[search.query_expansion]
enabled = true
model = "tinyllama" # 1.1B, balanced speed/quality
timeout_ms = 300
[search.reranker]
enabled = false # Reranking usually unnecessary with RRF
[search.graph_retrieval]
enabled = true
max_concept_hops = 2
[tagging]
enabled = true
ner_enabled = true
tf_enabled = true
[insert]
auto_link = true
auto_link_limit = 5
Performance: ~500-700ms search, +100ms memory add. Best recall/speed tradeoff.
[search]
mode = "hybrid"
min_quality = 0.6 # Include borderline memories
[search.query_expansion]
enabled = true
model = "phi-2" # 2.7B, highest quality
timeout_ms = 500
[search.reranker]
enabled = true
model = "ms-marco-MiniLM-L-6-v2" # 100MB, ~1s latency
apply_to_top_k = 20
[search.graph_retrieval]
enabled = true
max_concept_hops = 3 # More aggressive concept expansion
[tagging]
enabled = true
ner_enabled = true
tf_enabled = true
min_tf_score = 0.2 # Lower threshold = more tags
[insert]
auto_link = true
auto_link_limit = 10 # Link to more neighbors
[redaction]
enabled = true # Protect secrets
Performance: ~1.5-2s search, +150ms memory add. Highest recall, slower.
[search]
mode = "hybrid"
min_quality = 0.75
[search.query_expansion]
enabled = true
model = "tinyllama"
timeout_ms = 300
[search.reranker]
enabled = false
[search.graph_retrieval]
enabled = true
max_concept_hops = 2
[tagging]
enabled = true
ner_enabled = true
[ontology]
enabled = true
auto_link_concepts = true
[redaction]
enabled = true
[mcp]
enabled = true # Expose as MCP server
Best for: Claude integration, agent consumption, search_memories tool calls.
voidm/
├── crates/
│ ├── voidm-core/ # CRUD, hybrid search, quality scoring, NER/NLI
│ ├── voidm-sqlite/ # SQLite backend (default)
│ ├── voidm-postgres/ # PostgreSQL backend (experimental)
│ ├── voidm-neo4j/ # Neo4j backend (production-ready)
│ ├── voidm-embeddings/ # Fastembed + text chunking
│ ├── voidm-query-expansion/ # HyDE template + LLM inference
│ ├── voidm-reranker/ # Cross-encoder ranking
│ ├── voidm-graph/ # EAV schema + Cypher translator
│ ├── voidm-tagging/ # NER + TF tagging
│ ├── voidm-ner/ # Entity extraction (ONNX)
│ ├── voidm-nli/ # Relation classification (ONNX)
│ ├── voidm-redactor/ # Secrets detection + masking
│ ├── voidm-scoring/ # Quality score computation
│ ├── voidm-models/ # Model management + download
│ ├── voidm-mcp/ # MCP server implementation
│ └── voidm-cli/ # CLI + JSON output
└── migrations/ # SQLite schema (sqlx) + Migration scripts
Storage:
~/.local/share/voidm/memories.db (embedded, transactional, 100MB+ for large bases)~/.config/voidm/config.toml~/.cache/voidm/ (embeddings, NER, NLI, query expansion)Backend Selection (in config.toml):
[database]
backend = "sqlite" # Options: sqlite, neo4j
Environment Variables:
VOIDM_CONFIG — Override config location (useful for testing multiple backends)VOIDM_DB — Override SQLite database path (SQLite backend only)Search Pipeline:
Query
├→ Query Expansion (optional) → expanded query
├→ Semantic Search (embeddings + ANN) → results₁
├→ BM25 Search (FTS5) → results₂
├→ Fuzzy Search (Levenshtein) → results₃
├→ Graph Retrieval (tag overlap + concepts) → results₄
├→ RRF Fusion (merge 1-4) → ranked results
└→ Reranking (optional, cross-encoder) → final ranking
| Operation | Latency | Dataset |
|---|---|---|
| Add memory | 100-150ms | N/A |
| Add + Auto-tagging | 150-200ms | N/A |
| Add + Auto-linking | 200-300ms | 10K memories |
| Semantic search | 150-250ms | 100K memories |
| BM25 search | 30-100ms | 100K memories |
| Hybrid search | 300-500ms | 100K memories |
| With query expansion | 600-1000ms | 100K memories |
| With reranking | 1000-1500ms | 100K memories |
| Graph neighbors (depth 2) | 50-200ms | 100K memories |
| Cypher query | 100-1000ms | 100K memories, complex queries |
| Pagerank | 50-150ms | 100K memories |
quality_score = base_score * type_boost * temporal_factor * substance_factor * abstraction_factor
where:
base_score = 0.5 (foundation)
type_boost:
episodic = +0.035 (reliable, specific)
semantic = +0.025 (factual, general)
conceptual/procedural/contextual = +0.015 (variable)
temporal_factor = e^(-λ*days_old) with 30-day half-life
substance_factor = signal-to-noise (dense vs. generic)
abstraction_factor = well-defined vs. vague
Current production baseline: 0.9392 (optimized across 26 variations).
✅ Text Chunking: Consistent smart chunks (target 600 chars, 100-char overlap) for large memories
✅ HyDE Query Expansion: Hypothetical document generation for better semantic search
✅ Graph Retrieval in RRF: Tag and concept-based result expansion integrated into search
✅ NER Feature Gating: Optional dependency handling with clean builds
✅ Quality Score Optimization: 26 iterations reaching 0.9392 (SOTA-level)
✅ Neo4j Backend: Production-ready enterprise graph database support with full Cypher query language
✅ Backend Independence: CLI backend routing with VOIDM_CONFIG environment variable support
✅ Ontology Export Complete: 92 concepts + 131 memory-concept relationships (INSTANCE_OF edges) migrated
✅ MERGE Upsert Pattern: Graceful duplicate handling - duplicate IDs update existing records instead of failing
✅ Metadata Tracking: Author and source fields added to all memory operations for audit trails
Complete Neo4j Export Status (Session 2026-03-23):
Cypher Verification Queries:
# Verify total counts
voidm graph cypher "MATCH (m:Memory) RETURN COUNT(m) as memories"
voidm graph cypher "MATCH (c:Concept) RETURN COUNT(c) as concepts"
voidm graph cypher "MATCH ()-[r:INSTANCE_OF]->() RETURN COUNT(r) as edges"
# Check data integrity (no duplicates)
voidm graph cypher "MATCH (m:Memory) WITH m.id as id, COUNT(*) as cnt WHERE cnt > 1 RETURN COUNT(*) as duplicates"
# Sample memory-concept mapping
voidm graph cypher "MATCH (m:Memory)-[:INSTANCE_OF]->(c:Concept) RETURN c.name, COUNT(m) as instance_count ORDER BY instance_count DESC LIMIT 10"
Short-term (2 weeks):
Medium-term (Month 2): 4. Smart Memory Eviction (LRU + semantic similarity) → 10x+ scale 5. Active Learning for Importance Scoring → self-improving 6. Temporal Decay Ranking → knowledge freshness
Long-term (Q2+): 7. RDF Triple Storage → structured reasoning 8. Hierarchical Memory Compression → token efficiency 9. Metadata-Driven Ranking → semantic signals
# Clone and build
git clone https://github.com/autonomous-toaster/voidm
cd voidm
cargo install --path crates/voidm-cli
# Or build manually
cargo build --release
cp target/release/voidm ~/.local/bin/
# Initialize (download models for offline use)
voidm init
Requirements: Rust 1.94.0+, SQLite (bundled).
Models: Automatically downloaded to ~/.cache/voidm/ on first use (~300-400MB). Idempotent.
Use one of the three supported CLI profiles:
| Profile | Summary | Build Command |
|---|---|---|
| minimal | Lean local CLI with SQLite | cargo build --release --no-default-features --features minimal |
| standard | Recommended default profile | cargo build --release |
| full | Standard + Neo4j + MCP + experimental llama.cpp | cargo build --release --no-default-features --features full |
Public docs should prefer these profiles. Lower-level feature toggles still exist for local/dev composition.
# Default build (STANDARD profile, all recommended features)
cargo build --release
cp target/release/voidm ~/.local/bin/
# Or install directly
cargo install --path crates/voidm-cli
# For minimal deployments
cargo build --release --no-default-features --features minimal
# For the maximal curated CLI profile
cargo build --release --no-default-features --features full
Build custom combinations:
# Just embeddings + search (no NER/NLI/reranking)
cargo build --release --no-default-features --features "database-sqlite,database-postgres,embeddings,vector-search,tinyllama"
# Minimal + reranking only
cargo build --release --no-default-features --features "minimal,reranker"
# See all available features
cargo build --release --no-default-features --features "" 2>&1 | grep "unknown feature"
Available individual features:
database-sqlite, database-postgres, database-neo4jembeddings, vector-search, query-expansionnli, ner, rerankertinyllama, mcp, redactor| Command | Description |
|---|---|
voidm add <text> | Add memory. Returns suggested_links, duplicate_warning. |
voidm get <id> | Retrieve by ID or 4+ char prefix. |
voidm list | List all, filterable by scope/type/quality. |
voidm search <query> | Hybrid search. Modes: hybrid/semantic/bm25/fuzzy/keyword/vector. |
voidm delete <id> | Delete memory. |
voidm link <from> <EDGE> <to> | Create graph edge. RELATES_TO needs --note. |
voidm export | Export memories as JSON. |
| Command | Description |
|---|---|
voidm graph neighbors <id> | N-hop neighbors (--depth, default 1). |
voidm graph pagerank --top 10 | Rank by centrality. |
voidm graph cypher "<query>" | Read-only Cypher traversal. |
voidm graph export --format html | Interactive visualization (html/dot/json/csv). |
| Command | Description |
|---|---|
voidm init | Pre-download configured local models. |
voidm config init/show/set | Create and manage configuration. |
voidm info | Show backend, config and runtime settings. |
voidm stats | Memory and graph statistics. |
voidm migrate --from <backend> --to <backend> | Migrate data between sqlite and neo4j. |
Use --json for machine-readable output. Use --help for full flag details.
voidm add Flags| Flag | Values | Default | Description |
|---|---|---|---|
--type (required) | episodic, semantic, procedural, conceptual, contextual | — | Memory type affects quality scoring |
--scope | any string | — | Organizational context (repeatable). E.g., --scope work/project/backend |
--tags | comma-separated | — | Custom tags for filtering/linking (no max, overwrites auto-tags) |
--importance | 1-10 | 5 | Manual importance level (boosts ranking) |
--author | user, assistant, unknown | user | Author trust tier (affects ranking) |
--source | academic, verified, user, unknown | unknown | Source reliability (affects ranking) |
--link | <id>:<TYPE> or <id>:<TYPE>:<note> | — | Auto-link to existing memory. RELATES_TO requires --note |
--db | path | ~/.local/share/voidm/memories.db | Override database location |
--json | — | — | Machine-readable JSON output |
--quiet | — | — | Suppress decorative output |
Examples:
# Basic
voidm add "My knowledge" --type semantic
# With metadata
voidm add "Research" --type semantic --author user --source academic --importance 9
# Multiple scopes and tags
voidm add "Info" --scope work/proj/backend --scope personal --tags rust,performance,api
# With auto-linking
voidm add "New fact" --type semantic --link a1b2c3d4:SUPPORTS:"Explains core concept"
# Add foundational knowledge
voidm add "Project X uses Postgres for ACID" --type conceptual --scope work/projectx
voidm add "Deployment via GitHub Actions + Docker" --type procedural --scope work/projectx
voidm add "Auth uses OAuth2 with JWT tokens" --type semantic --scope work/projectx
# Define ontology
voidm ontology concept add "ProjectX" --description "Internal web platform"
# Search with bounded context and stricter filtering
voidm search "how do we authenticate" --scope work/projectx --tag oauth,jwt --min-score 0.7
# Visualize
voidm graph export --format html > projectx-graph.html
open projectx-graph.html
# Find all decisions that were invalidated by later decisions
voidm graph cypher "
MATCH (old:Memory)-[:INVALIDATES]->(new:Memory)
RETURN old.memory_id as invalidated, new.memory_id as invalidates
ORDER BY old.created_at DESC
LIMIT 20
"
# Find most central memories (hubs)
voidm graph pagerank --top 10
# Explore concept hierarchy
voidm graph cypher "
MATCH (child:Concept)-[:IS_A*1..2]->(parent:Concept)
WHERE child.name CONTAINS 'Service'
RETURN child.name, parent.name
"
Minimal (Lightweight):
[search]
mode = "hybrid"
[search.query_expansion]
enabled = false
[tagging]
enabled = true
Production (Balanced):
[search]
mode = "hybrid"
min_quality = 0.75
[search.query_expansion]
enabled = true
model = "tinyllama"
timeout_ms = 300
[search.graph_retrieval]
enabled = true
max_concept_hops = 2
[tagging]
enabled = true
ner_enabled = true
[redaction]
enabled = true
[insert]
auto_link = true
auto_link_limit = 5
High-Recall (Quality):
[search]
mode = "hybrid"
min_quality = 0.6
[search.query_expansion]
enabled = true
model = "phi-2"
timeout_ms = 500
[search.reranker]
enabled = true
model = "ms-marco-MiniLM-L-6-v2"
[search.graph_retrieval]
enabled = true
max_concept_hops = 3
[tagging]
enabled = true
ner_enabled = true
[insert]
auto_link = true
auto_link_limit = 10
| Issue | Solution |
|---|---|
| Slow first search | Normal (model download + embedding cache warmup). Subsequent searches <500ms. |
| "Model not found" | Run voidm init to download. Models cached in ~/.cache/voidm/. |
| High memory usage | Large datasets in SQLite. Consider PostgreSQL adapter or archiving old memories. |
| Search returns nothing | Enable query expansion (--query-expand true) or lower min_quality threshold. |
| Duplicate-like results | Enable reranking (--reranker true) to improve ordering. |
| Secrets not redacted | Check config [redaction] enabled. Run voidm config show to verify. |
| Code | Meaning |
|---|---|
0 | Success |
1 | Not found |
2 | Error (bad args, write Cypher rejected, etc.) |
Inspired by byteowlz/mmry and colliery-io/graphqlite.
Built with ❤️ using fastembed-rs, sqlx, ort (ONNX Runtime), and pi-coding-agent.
MIT — see LICENSE.
214 commits
Rust
99.7%