SparrowDB

The SQLite of graph databases. Embedded, Cypher-native, zero infrastructure.

---

⚠️ Only one process may hold a database open at a time

GraphDb::open takes an exclusive lock on the database root, process-wide. A second process that tries to open the same root while another process already has it open gets an immediate, clean error instead of being let in:

Error: database locked: another process already has '<path>' open for writing. SparrowDB allows
only one open handle per database root at a time — close the other process's connection (or wait
for it to exit) and retry.

This closes #524: on sparrowdb@0.1.26 and earlier, there was no lock file and no guard at open() at all, and two processes writing concurrently to the same root could corrupt catalog.tlv beyond recovery — measured at 4 of 5 concurrent runs leaving the database permanently unopenable (Error: corruption: duplicate label_id 0 in catalog file). If you hit that error on an older version, upgrading does not repair an already-corrupted catalog.tlv — it only prevents new corruption from this cause.

The lock is released automatically when the holding process's handle is dropped or the process exits, including a crash or SIGKILL — there is nothing to clean up by hand. Sequential access (one process exits, the next opens) has always been safe and still is.

The bindings still expose only open() with no read-only mode, so this remains a single-writer constraint rather than true multi-reader support: a second process must wait for the first to close its handle (or exit) before it can attach at all, even just to read.

---

SparrowDB is an embedded graph database. It links directly into your process — Rust, Python, Node.js, or Ruby — and gives you a real Cypher query interface backed by a WAL-durable store on disk. No server. No JVM. No cloud subscription. No daemon to babysit.

If your data is fundamentally relational — recommendations, social graphs, dependency trees, fraud rings, knowledge graphs — and you want to query it with multi-hop traversals instead of JOIN chains, SparrowDB is the drop-in answer.

---

Quick Start

use sparrowdb::GraphDb;

fn main() -> sparrowdb::Result<()> {
    let db = GraphDb::open(std::path::Path::new("social.db"))?;

    db.execute("CREATE (alice:Person {name: 'Alice', age: 30})")?;
    db.execute("CREATE (bob:Person   {name: 'Bob',   age: 25})")?;
    db.execute("MATCH (a:Person {name:'Alice'}), (b:Person {name:'Bob'}) CREATE (a)-[:KNOWS]->(b)")?;

    // Who does Alice know? Who do *they* know?
    let fof = db.execute("MATCH (a:Person {name:'Alice'})-[:KNOWS*1..2]->(f) RETURN DISTINCT f.name")?;
    // -> [["Bob"], ["Carol"]]  (Carol is a friend-of-friend)
    let _ = fof;
    Ok(())
}

That's it. The database is a directory on disk. Ship it.

---

Performance: Faster Than Neo4j Where It Counts

Benchmarked against Neo4j 5.x on the SNAP Facebook dataset (4,039 nodes, 88,234 edges). All figures are p50 latency, v0.1.15.

Point lookups, aggregations, and mutual-neighbor queries beat a running Neo4j server — with no JVM, no server process, no network hop.

Q8 dropped from 153ms → 0.67ms (−99.6%) in v0.1.15. Deep traversal (Q3/Q4/Q5) is slower than a warmed Neo4j server — that's expected for an embedded engine without parallel execution. The target workload is agents, CLIs, and apps that need a graph database without operating one.

Cold start: ~27ms — viable for serverless and short-lived processes where Neo4j's server startup is disqualifying.

---

Built for AI Agents and MCP

SparrowDB ships with a first-class MCP server (sparrowdb-mcp) — the only embedded graph database with native MCP support. It speaks JSON-RPC 2.0 over stdio and plugs directly into Claude Desktop and any MCP-compatible AI client.

cargo install sparrowdb --bin sparrowdb-mcp --locked

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "sparrowdb": {
      "command": "/absolute/path/to/sparrowdb-mcp",
      "args": []
    }
  }
}

Your AI assistant can now query and write to your graph database using natural tool calls:

Full setup: docs/mcp-setup.md

Why this matters for agent builders: Multi-agent systems need shared, persistent graph state. SparrowDB gives your agents a knowledge graph they can read and write without spinning up a server. Pair it with SparrowOntology for schema-enforced agent memory and governance.

---

Why SparrowDB

The graph database landscape has a gap.

Neo4j is powerful, but it requires a running server, a JVM, and a license the moment you need production features. DGraph is horizontally scalable, but you don't need horizontal scale — you need to ship your app. Every existing option assumes you want to operate a database cluster, not embed a graph engine.

SparrowDB fills the same role SQLite fills for relational data: zero infrastructure, full capability, open source, MIT licensed.

---

When to Use SparrowDB

SparrowDB is the right choice when:

• Your data has structure that's hard to flatten. Social follows, product recommendations, dependency graphs, org charts, bill-of-materials, knowledge graphs — these are terrible in SQL and natural in graphs.
• You're building an application, not operating a database. You want to cargo add sparrowdb and ship, not provision instances.
• You need multi-hop queries. MATCH (a)-[:FOLLOWS*1..3]->(b) is one query. In SQL it's recursive CTEs all the way down.
• You're embedding into a CLI, desktop app, agent, or edge service. SparrowDB opens in milliseconds and has no runtime overhead when idle.

SparrowDB is not the right choice when:

• Deep multi-hop traversal on large high-fanout graphs is your primary workload. If you're running 5-hop queries across a billion-edge social graph, use Neo4j. SparrowDB is a single-process embedded engine — it's not trying to win that race.
• You need distributed writes across many nodes, or your graph has billions of edges and requires horizontal sharding. Use Neo4j Aura or DGraph for that.

---

Install

Node.js

npm install sparrowdb

The sparrowdb npm package bundles prebuilt native binaries directly in the tarball — there is no separate @sparrowdb/<platform> package to resolve.

On an unsupported platform, require('sparrowdb') throws a clear error naming the platform and the supported list, rather than a raw module-resolution failure. To build from source on those platforms:

cargo build --release -p sparrowdb-node
cp target/release/libsparrowdb_node.<so|dylib|dll> npm/sparrowdb/sparrowdb.node

Rust

[dependencies]
sparrowdb = "0.1"

Python

# Build from source (requires Rust toolchain):
cd crates/sparrowdb-python && maturin develop

PyPI package coming soon. Pre-built wheels are on the roadmap.

Ruby

# Build from source (requires Rust toolchain):
cd crates/sparrowdb-ruby && bundle install && rake compile

RubyGems package coming soon.

CLI

cargo install sparrowdb --bin sparrowdb

MCP Server (Claude Desktop integration)

cargo install sparrowdb --bin sparrowdb-mcp --locked

---

Features

Cypher Support

Engine & Storage

• WAL durability — write-ahead log with crash recovery; survives hard kills
• SWMR concurrency — single-writer, multiple-reader; readers never block writers
• Chunked vectorized pipeline — 4-phase chunked execution engine for multi-hop traversals; FrontierScratch arena eliminates per-hop allocation; SlotIntersect for mutual-neighbor queries
• Factorized execution — multi-hop traversals avoid materializing O(N²) intermediate rows
• B-tree property index — equality lookups in O(log n), not full label scans; persisted to disk
• Inverted text index — CONTAINS / STARTS WITH routed through an index
• Full-text search — relevance-scored queryNodes without Elasticsearch
• External merge sort — ORDER BY on large results spills to disk; no unbounded heap
• At-rest encryption — optional XChaCha20-Poly1305 per WAL entry; wrong key errors immediately, never silently decrypts garbage
• execute_batch() — multiple writes in one fsync for bulk-load throughput
• Bulk CSV loader — sparrowdb bulk-import ingests node and edge CSVs with batched WriteTx for high-throughput imports
• execute_with_timeout() — cancel runaway traversals without killing the process
• export_dot() — export any graph to Graphviz DOT for visualization
• APOC CSV import — migrate existing Neo4j graphs in one command
• MVCC write-write conflict detection — two writers on the same node: the second is aborted

Language Bindings

All bindings open the same on-disk format. A graph written from Python can be read by Node.js.

---

Language Examples

Rust

use sparrowdb::GraphDb;
use std::time::Duration;

fn main() -> sparrowdb::Result<()> {
    let db = GraphDb::open(std::path::Path::new("my.db"))?;

    // Bulk load — one fsync for all five writes
    db.execute_batch(&[
        "CREATE (alice:Person {name:'Alice', role:'engineer', score:9.1})",
        "CREATE (bob:Person   {name:'Bob',   role:'designer', score:7.5})",
        "CREATE (carol:Person {name:'Carol', role:'engineer', score:8.8})",
        "MATCH (a:Person {name:'Alice'}),(b:Person {name:'Bob'})   CREATE (a)-[:KNOWS]->(b)",
        "MATCH (a:Person {name:'Bob'}),  (b:Person {name:'Carol'}) CREATE (a)-[:KNOWS]->(b)",
    ])?;

    // Multi-hop: friend-of-friend
    let fof = db.execute(
        "MATCH (a:Person {name:'Alice'})-[:KNOWS*2]->(f) RETURN f.name"
    )?;
    println!("{:?}", fof.rows); // [["Carol"]]

    // WITH pipeline: count edges, filter, continue
    let _prolific = db.execute(
        "MATCH (p:Person)-[:KNOWS]->(f)
         WITH p, COUNT(f) AS connections
         WHERE connections >= 1
         RETURN p.name, connections
         ORDER BY connections DESC"
    )?;

    // Upsert — creates on first call, updates on subsequent calls
    db.execute("MERGE (u:User {email:'alice@example.com'})
                ON CREATE SET u.created = '2024-01-01', u.logins = 0
                ON MATCH  SET u.logins  = u.logins + 1")?;

    // Cancel runaway traversal after 5 seconds
    let _ = db.execute_with_timeout(
        "MATCH (a)-[:FOLLOWS*1..10]->(b) RETURN b.name, count(*)",
        Duration::from_secs(5),
    );

    Ok(())
}

Python

import sparrowdb

# Context manager — database closes cleanly on exit; execute() releases the GIL
with sparrowdb.GraphDb("/path/to/my.db") as db:
    db.execute("CREATE (n:Product {id: 1, name: 'Widget',   price: 9.99})")
    db.execute("CREATE (n:Product {id: 2, name: 'Gadget',   price: 24.99})")
    db.execute("CREATE (n:Product {id: 3, name: 'Doohickey',price: 4.99})")
    db.execute(
        "MATCH (a:Product {id:1}),(b:Product {id:2}) CREATE (a)-[:RELATED]->(b)"
    )

    rows = db.execute(
        "MATCH (p:Product {name:'Widget'})-[:RELATED*1..2]->(r) "
        "RETURN DISTINCT r.name, r.price ORDER BY r.price"
    )
    print(rows)

# Thread-safe: GIL is released inside execute()
import concurrent.futures

def query_worker(limit):
    with sparrowdb.GraphDb("/path/to/my.db") as db:
        return db.execute(f"MATCH (n:Product) RETURN n.name LIMIT {limit}")

with concurrent.futures.ThreadPoolExecutor(max_workers=4) as pool:
    results = list(pool.map(query_worker, [1, 2, 3]))

Node.js / TypeScript

import SparrowDB from 'sparrowdb';

const db = new SparrowDB('/path/to/my.db');

db.execute("CREATE (n:Article {id: 'a1', title: 'Graph Databases 101', tags: 'graphs,rust'})");
db.execute("CREATE (n:Article {id: 'a2', title: 'Cypher Query Language', tags: 'cypher,graphs'})");
db.execute("MATCH (a:Article {id:'a1'}),(b:Article {id:'a2'}) CREATE (a)-[:RELATED]->(b)");

// Full-text search
db.execute("CALL db.index.fulltext.createNodeIndex('articles', ['Article'], ['title', 'tags'])");
const results = db.execute(
  "CALL db.index.fulltext.queryNodes('articles', 'rust') " +
  "YIELD node, score RETURN node.title, score ORDER BY score DESC"
);

db.close();

Ruby

require 'sparrowdb'

db = SparrowDB::GraphDb.new('/path/to/my.db')

db.execute("CREATE (n:Dependency {name: 'tokio',  version: '1.35'})")
db.execute("CREATE (n:Dependency {name: 'serde',  version: '1.0'})")
db.execute("MATCH (a:Dependency {name:'tokio'}),(b:Dependency {name:'serde'}) CREATE (a)-[:DEPENDS_ON]->(b)")

rows = db.execute(
  "MATCH (a:Dependency {name:'tokio'})-[:DEPENDS_ON*1..5]->(dep) RETURN DISTINCT dep.name"
)
puts rows.inspect  # [["serde"]]

db.close

---

Real-World Use Cases

Recommendation Engine

MATCH (u:User {id: $user_id})-[:LIKED]->(item:Item)
WITH collect(item) AS liked_items
MATCH (other:User)-[:LIKED]->(item) WHERE item IN liked_items
WITH other, COUNT(item) AS overlap ORDER BY overlap DESC LIMIT 20
MATCH (other)-[:LIKED]->(candidate:Item)
WHERE NOT candidate IN liked_items
RETURN candidate.name, COUNT(other) AS score ORDER BY score DESC LIMIT 10

Fraud Detection

MATCH (flagged:Account {status:'fraudulent'})-[:USED]->(device:Device)
MATCH (device)<-[:USED]-(suspect:Account)
WHERE suspect.status <> 'fraudulent'
WITH suspect, COUNT(device) AS shared_devices
WHERE shared_devices >= 2
RETURN suspect.id, suspect.email, shared_devices
ORDER BY shared_devices DESC

Dependency Graph

-- What breaks if we remove this package?
MATCH (pkg:Package {name: $package_name})<-[:DEPENDS_ON*1..10]-(dependent)
RETURN DISTINCT dependent.name, dependent.version
ORDER BY dependent.name

Knowledge Graph

MATCH (a:Concept {name: 'machine learning'}), (b:Concept {name: 'linear algebra'})
MATCH path = shortestPath((a)-[:RELATED_TO|REQUIRES|FOUNDATION_OF*]->(b))
RETURN [n IN nodes(path) | n.name] AS connection_chain

Org Chart Reporting

MATCH (emp:Employee {name: $name})-[:REPORTS_TO*]->(mgr:Employee)
RETURN emp.name, [m IN collect(mgr) | m.name + ' (' + m.title + ')'] AS chain

---

Advanced Features

Encrypted Database

use sparrowdb::GraphDb;

fn main() -> sparrowdb::Result<()> {
    let mut key = [0u8; 32];
    key[..16].copy_from_slice(b"my-secret-phrase");

    let db = GraphDb::open_encrypted(std::path::Path::new("secure.db"), key)?;
    db.execute("CREATE (n:Secret {data: 'classified'})")?;
    // Every WAL entry is XChaCha20-Poly1305 encrypted before hitting disk
    Ok(())
}

Graph Visualization

sparrowdb visualize --db my.db | dot -Tsvg -o graph.svg

Full-Text Search

db.execute("CALL db.index.fulltext.createNodeIndex('docs', ['Document'], ['content', 'title'])")?;
db.execute("CREATE (n:Document {title: 'Rust graph databases', content: 'Embedded and fast'})")?;

let results = db.execute(
    "CALL db.index.fulltext.queryNodes('docs', 'embedded graph') \
     YIELD node, score \
     RETURN node.title, score ORDER BY score DESC"
)?;

Per-Query Timeout

match db.execute_with_timeout(
    "MATCH (a)-[:FOLLOWS*1..10]->(b) RETURN b.name",
    Duration::from_secs(5),
) {
    Ok(rows) => println!("{} rows", rows.rows.len()),
    Err(e) if e.to_string().contains("timeout") => eprintln!("Query cancelled"),
    Err(e) => return Err(e),
}

Bulk CSV Import

# High-throughput node + edge ingestion via CLI
sparrowdb bulk-import \
  --db my.db \
  --nodes nodes.csv --node-label Person \
  --edges edges.csv --rel-type KNOWS \
  --src-label Person --dst-label Person \
  --batch-size 10000

// Or from Rust
use sparrowdb::bulk::{BulkLoader, BulkOptions};
let loader = BulkLoader::new(&db, BulkOptions::default());
let n = loader.load_nodes(Path::new("nodes.csv"), "Person")?;
let e = loader.load_edges(Path::new("edges.csv"), "KNOWS", "Person", "Person")?;

Neo4j Migration

# Export from Neo4j using APOC, then:
sparrowdb import --neo4j-csv nodes.csv,relationships.csv --db my.db

---

Performance Characteristics

Benchmark Results: SNAP Facebook Dataset (v0.1.15)

Measured against Neo4j 5.x (server, JVM warmed) and Kùzu (Shi et al. VLDB 2023). All figures are p50 latency in microseconds. Dataset: SNAP Facebook social graph (4,039 nodes, 88,234 edges), 50 warmup + 200 iterations.

¹ Q8: 153,300µs → 670µs (−99.6%) via SlotIntersect + BfsArena + bitvector set-intersection (v0.1.15, #357–#359).

² Deep traversal on a large social graph is not the target workload for an embedded database. See When to Use SparrowDB.

Neo4j reference: measured locally, Neo4j Docker v5.x, Bolt TCP. Kùzu reference: Shi et al. VLDB 2023 Table 5, in-process.

Where SparrowDB wins:

• Q1 (point lookup): 103µs — 3x faster than Neo4j's Bolt round-trip + JVM overhead.
• Q6 (global COUNT): 2.2µs — 93x faster. Catalog-level metadata, no scan.
• Q7 (top-10 by degree): 401µs — 44x faster. Pre-computed degree cache vs Neo4j's full adjacency scan.
• Q8 (mutual friends): 0.67ms — ~2x faster. SlotIntersect + BfsArena eliminated the O(N × disk_reads) bottleneck.
• Cold start: ~27ms on macOS SSD — viable for serverless and short-lived processes.
• Total session latency for agents: 50 queries × no network hop = no accumulated RTT tax. A server database adds ~100ms in round-trips alone.

Where SparrowDB trails: Deep multi-hop traversal (Q3/Q4/Q5) on large high-fanout graphs, and range scans (Q2). These are server-database workloads — not the embedded use case.

What this means in practice:

• Use SparrowDB for: CLIs, AI agents, desktop apps, edge services, knowledge graphs, recommendation engines — anything where embedded, zero-infrastructure, and fast startup matter.
• Use Neo4j for: deep multi-hop traversal on billion-edge graphs as the primary workload, or when you need a multi-user server.

Engine Design

---

Comparison

TL;DR: If you need embedded + Cypher + zero infrastructure, there's nothing else. SparrowDB is the only option in that row.

---

CLI Reference

sparrowdb query --db my.db "MATCH (n:Person) RETURN n.name LIMIT 10"
sparrowdb checkpoint --db my.db
sparrowdb info --db my.db
sparrowdb visualize --db my.db | dot -Tsvg -o graph.svg
sparrowdb import --neo4j-csv nodes.csv,relationships.csv --db my.db
sparrowdb bulk-import --db my.db --nodes nodes.csv --edges edges.csv --node-label Person --rel-type KNOWS

# NDJSON server mode
sparrowdb serve --db my.db
# stdin:  {"id":"q1","cypher":"MATCH (n) RETURN n LIMIT 5"}
# stdout: {"id":"q1","columns":["n"],"rows":[...],"error":null}

---

Project Status

SparrowDB is pre-1.0. We are building in public.

The API is stable enough to build on, but the on-disk format may change before 1.0. Pin your version.

What's done: Full Cypher subset · Multi-label nodes (n:A:B) · WAL durability + crash recovery · MVCC · At-rest encryption · 4-phase chunked vectorized pipeline · Factorized multi-hop engine · B-tree + full-text indexes · External merge sort · Per-query timeouts · Bulk CSV loader · Python / Node.js / Ruby bindings · MCP server · CLI tools · Neo4j APOC import

---

Roadmap

Recently shipped:

• Q8: 153ms → 0.72ms (−99.5%, 200×) — SlotIntersect + BfsArena + bitvector set-intersection shipped in v0.1.15 (#357–#359); mutual-friend queries now 2x faster than Neo4j on SNAP Facebook
• Q8 anchor-node bulk read (#357) — find_slot_by_props replaced O(N × file_reads) with a single bulk column read; eliminates N disk reads for each inline-prop anchor lookup in mutual-neighbor queries
• 4-phase chunked vectorized pipeline (#299) — FrontierScratch arena + SlotIntersect; Q4 −7%, Q8 p99 −37% in v0.1.13
• Multi-label nodes (n:A:B) (#289) — standard Cypher multi-label semantics
• Bulk CSV loader (#296) — sparrowdb bulk-import for high-throughput node/edge ingestion
• Reverse-arrow pattern fix (#294) — (a)->()<-(c) now returns correct results
• B-tree property index persisted to disk (#286) — index survives restart; no re-build on open
• Delta log O(1) index (#283) — un-checkpointed writes no longer degrade traversal
• Degree cache — Q7 (Top-10 Degree): 1,279ms → 401µs, now 44x faster than Neo4j
• B-tree property index — Q1 (Point Lookup): ~444µs → 103µs, now 3x faster than Neo4j
• Global COUNT optimization — Q6: 24µs → 2.2µs, now 93x faster than Neo4j
• 2-hop aggregate fix (#282) — COUNT(*) on multi-hop queries no longer returns null

---

Architecture

+------------------------------------------------------------------------+
|  Language Bindings                                                     |
|  Rust - Python (PyO3) - Node.js (napi-rs) - Ruby (Magnus)            |
|  CLI (sparrowdb) - MCP Server (sparrowdb-mcp)                         |
+------------------------------------------------------------------------+
|  Cypher Frontend  (sparrowdb-cypher)                                   |
|  Lexer -> AST -> Binder (name resolution, type checking)              |
+------------------------------------------------------------------------+
|  Execution Engine  (sparrowdb-execution)                               |
|  Chunked vectorized pipeline - ChunkedPlan selector                   |
|  FrontierScratch arena - SlotIntersect - factorized aggregation       |
|  External merge sort - EXISTS evaluation - deadline checks             |
+------------------------------------------------------------------------+
|  Catalog  (sparrowdb-catalog)                                          |
|  Label registry - B-tree property index - Inverted text index         |
+------------------------------------------------------------------------+
|  Storage  (sparrowdb-storage)                                          |
|  Write-Ahead Log - CSR adjacency store - Delta log index              |
|  XChaCha20-Poly1305 encryption (optional) - Crash recovery - SWMR    |
+------------------------------------------------------------------------+

---

Documentation

---

Contributing

Open an issue before submitting a large PR so we can discuss the design first.

git clone https://github.com/ryaker/SparrowDB
cd SparrowDB
cargo build
cargo test
cargo test -p sparrowdb  # integration tests — the signal
cargo clippy
cargo fmt --check

---

License

MIT — see LICENSE.