Conflux

Your AI tools are only as good as the context they have.

Every RAG app, research pipeline, and LLM workflow hits the same wall: stale documents, redundant content, and no way to know what's useful and what's dead weight. Vector databases solve retrieval but not lifecycle. Manual curation doesn't scale. Web search has no memory.

Conflux is a local knowledge store that manages itself. Documents go in with metadata and expiration dates. Automated feeds keep pulling fresh data. Semantic search surfaces what matters. And the lifecycle engine quietly archives what's stale, expires what's old, and cleans up what's forgotten — so your context stays fresh without you gardening it.

Single binary. Runs offline. No cloud. No API keys for core features. Your data stays on your disk.

What it does

• Ingests documents (PDF, HTML, Markdown, plain text) with metadata, tags, and expiration dates
• Indexes semantically using All-MiniLM-L6-v2 embeddings (384 dimensions, runs locally via ONNX)
• Extracts named entities automatically — people, organizations, locations — using BERT-base-NER
• Retrieves via hybrid ranked search combining semantic similarity, keyword matching, recency, usage frequency, and relevance feedback
• Manages lifecycle — documents expire by type (news in 7 days, data in 30, reference never), stale content gets archived, old archives get purged
• Feeds from RSS, HTTP endpoints, or shell commands on a schedule
• Deduplicates — exact (BLAKE3 hash) and near-duplicate (cosine >= 0.95) detection at ingest
• Exposes a programmatic Rust API and an MCP server for AI assistant integration

Quick start

# Build
cargo build --release

# Initialize a store
conflux store init ~/.conflux

# Ingest a document
conflux ingest report.pdf --type data --tags "quarterly,finance" --collection "q1-2026"

# Query
conflux query "revenue growth drivers" --collection "q1-2026" --limit 10

# Check store health
conflux report health

# Search entities
conflux entity search "Goldman Sachs"

# Set up an automated feed
cat > ~/.conflux/feeds/market-data.toml << 'EOF'
name = "market-data"
enabled = true
schedule = "*/6h"

[source]
type = "command"
command = "curl -s https://api.example.com/data"

[ingest]
collection = "market"
document_type = "data"
tags = ["market", "daily"]
expires_after_days = 30
EOF

Architecture

conflux (workspace)
├── conflux-core     # Library — all business logic
├── conflux-cli      # CLI binary (clap)
└── conflux-mcp      # MCP server (JSON-RPC over stdio)

Storage is a plain directory:

~/.conflux/
├── config.toml          # Store configuration
├── store.db             # SQLite (WAL mode) — metadata, entities, usage, feedback
├── content/             # Extracted text (one file per document UUID)
├── archive/             # Archived documents
├── vectors/index.bin    # CONFLUXV binary vector index
├── feeds/               # Feed TOML configs
├── ner/                 # Cached NER model
└── logs/                # Feed execution logs

Portable, inspectable, backupable. You can query store.db with sqlite3 and read documents with a text editor.

Scoring formula

score = 0.70 * semantic + 0.15 * recency + 0.10 * usage + 0.05 * feedback

Documents that get queried more rank higher. Documents marked "useful" rank higher. Stale documents fade. The store learns what matters.

Using as a library

[dependencies]
conflux-core = { path = "conflux-core" }

use conflux_core::{Store, QueryOptions, IngestSource, IngestOptions, FeedbackRating, FeedbackOptions};

let mut store = Store::open(Path::new("~/.conflux"))?;

// Query
let results = store.query("market conditions Q1", QueryOptions::with_limit(10))?;
for r in &results {
    let content = store.read_content(r.document.id)?;
    println!("{}: {:.3}", r.document.title, r.score);
}

// Entity-based retrieval
let docs = store.documents_by_entity("Federal Reserve", 20)?;

// Record feedback
store.record_feedback(doc_id, FeedbackRating::Useful, FeedbackOptions::default())?;

See docs/FLOWFLUX_INTEGRATION.md for a complete API guide written for downstream tool integration.

Document types and lifecycle

Lifecycle runs automatically on store open. Expired documents are archived. Documents unused for 180 days are flagged stale. Archives older than 30 days are purged. All configurable in config.toml.

MCP server

# Run the MCP server for AI assistant integration
conflux-mcp --store ~/.conflux

Exposes tools over JSON-RPC 2.0 on stdio: query, ingest, get_document, list_documents, search_entities, store_health, record_feedback, set_examination_status, set_evidence_quality, add_citation, get_citations.

Documentation

• PRODUCT.md — Product requirements and user flows
• ARCHITECTURE.md — System design and implementation phases
• TRD.md — Technical reference (as-built specification)
• FLOWFLUX_INTEGRATION.md — Integration guide for downstream tools

Status

v0.2.0 — Evidence tracking and citation chains:

• New: Document-to-document citation tracking (add_citation, get_cited_documents, get_citing_documents)
• New: Examination status field — track how thoroughly a document has been reviewed (cited → retrieved → examined → challenged)
• New: Evidence quality field — freeform quality assessment per document
• New: Four MCP tools: set_examination_status, set_evidence_quality, add_citation, get_citations
• New: Automatic schema migration for existing v0.1 stores
• All v0.1 features preserved — backward compatible, no breaking changes

v0.1.0 — Core functionality:

• Store init/open/destroy
• Document ingestion (text, PDF, HTML, Markdown)
• Semantic indexing + FTS5 keyword search
• Named entity extraction (Person, Organization, Location)
• Hybrid ranked retrieval with all four scoring signals
• Collections, tags, document type filtering, entity pre-filtering
• Lifecycle management (expiration, staleness, archival)
• Automated feeds (RSS, HTTP, command)
• Usage tracking and relevance feedback
• CLI and MCP server
• Integration tests for NER pipeline and ranking

License

MIT