langchainrust

A LangChain-inspired Rust framework for building LLM applications.

What it solves: Build Agents, RAG, BM25 keyword search, Hybrid retrieval, LangGraph workflows, MCP tools, A2A agent-to-agent protocols, Guardrails, multi-agent Handoffs — all in pure Rust.

---

Design Principles

The framework is engineered around a few hard rules that come out of its own design reviews. These are what make it feel different from hand-rolled LLM glue code:

---

Core Features

LLM & Providers

Embeddings

Composition: Chains & LCEL

Agents & Multi-Agent

Retrieval & RAG

Memory & Sessions

Protocols: MCP & A2A

Quality: Guardrails, Evaluation, Observability

Tools

Vector Stores

---

Architecture

langchainrust is a 23-crate workspace with a single facade crate langchainrust (in crates/lc) that re-exports the public API. Layers depend downward — lc-shared / lc-schema sit at the bottom and are depended on by everyone, which is exactly how the circular-dependency problem is solved.

                       ┌──────────────────────────────────────┐
                       │  langchainrust (facade, crates/lc)   │
                       └──────────────────┬───────────────────┘
                                          │
       ┌──────────────┬───────────────────┼───────────────────┬──────────────┐
    Protocol       Quality            Intelligence        Composition     Providers
  ┌────────────────────┐ ┌────────────────────┐ ┌────────────────────┐ ┌────────────────────┐ ┌────────────────────┐
  │ lc-mcp             │ │ lc-guardrails      │ │ lc-agents          │ │ lc-chains          │ │ lc-providers       │
  │ lc-a2a             │ │ lc-evaluation      │ │ lc-rag             │ │ lc-langgraph       │ │ lc-embeddings      │
  │                    │ │ lc-callbacks       │ │ lc-vector-stores   │ │                    │ │ lc-prompts         │
  │                    │ │ lc-observability   │ │                    │ │                    │ │ lc-tools           │
  └─────────┬──────────┘ └──────────┬─────────┘ └──────────┬─────────┘ └──────────┬─────────┘ └──────────┬─────────┘
            └───────────────────────┴──────────────────────┴──────────────────────┴──────────────────────┘
                                          │
        ┌─────────────────────────────────┴──────────────────────────────────────┐
        │                           Core & Foundation                             │
        │   ┌──────────┐   ┌──────────┐   ┌────────────────────────┐              │
        │   │ lc-core  │   │ lc-schema│   │ lc-shared              │              │
        │   │ Runnable │   │ Message  │   │ Document / ToolCall /  │              │
        │   │ LCEL     │   │ types    │   │ TextSplitter           │              │
        │   └──────────┘   └──────────┘   └────────────────────────┘              │
        └─────────────────────────────────────────────────────────────────────────┘

(lc-memory, lc-sessions, lc-testkit and lc-tools-derive are omitted from the diagram for clarity; the crate table below lists all 23.)

---

Installation

[dependencies]
langchainrust = "0.24.0"
tokio = { version = "1.0", features = ["full"] }

# Optional features — vector stores & storage
langchainrust = { version = "0.24.0", features = ["qdrant-integration"] }    # Qdrant vector DB
langchainrust = { version = "0.24.0", features = ["mongodb-persistence"] }   # MongoDB storage (memory + vector store + checkpoints)
langchainrust = { version = "0.24.0", features = ["redis-storage"] }         # Redis vector store
langchainrust = { version = "0.24.0", features = ["sqlite-storage"] }        # SQLite vector store (+ SQLTool)
langchainrust = { version = "0.24.0", features = ["pgvector-storage"] }      # PGVector (requires user-configured sqlx/pgvector deps)

# Durable LangGraph checkpointers (v0.22.4)
langchainrust = { version = "0.24.0", features = ["checkpoint-sqlite"] }     # rusqlite bundled, WAL — zero extra infra
langchainrust = { version = "0.24.0", features = ["checkpoint-postgres"] }   # tokio-postgres
langchainrust = { version = "0.24.0", features = ["checkpoint-redis"] }      # Redis with Lua CAS

# Local embeddings (vision/Cohere/Qwen multimodal embeddings need no feature)
langchainrust = { version = "0.24.0", features = ["local-embeddings"] }      # Local ONNX embeddings (ort)
langchainrust = { version = "0.24.0", features = ["fastembed"] }             # FastEmbed ONNX models
langchainrust = { version = "0.24.0", features = ["local-candle"] }          # Pure-Rust Candle BERT embeddings

# Observability
langchainrust = { version = "0.24.0", features = ["opentelemetry"] }         # OpenTelemetry tracing spans
langchainrust = { version = "0.24.0", features = ["otlp"] }                  # OTLP HTTP/JSON exporter (v0.22.4)
langchainrust = { version = "0.24.0", features = ["observability"] }         # MetricsSink + JsonLinesSink (v0.22.4)
langchainrust = { version = "0.24.0", features = ["observability-mongodb"] } # MongoDB observation sink

# Tools & memory
langchainrust = { version = "0.24.0", features = ["browser-cdp"] }           # CDP browser tool (v0.22.4)
langchainrust = { version = "0.24.0", features = ["vectorstore-memory"] }    # VectorStoreRetrieverMemory (semantic memory)
langchainrust = { version = "0.24.0", features = ["experimental"] }          # Experimental features
# PineconeStore / FileVectorStore / hosted search (Tavily, Serper, Exa) require no feature flag.
# The axum SSE agent endpoint is gated by lc-agents' own `sse-server` feature (see examples/agent_sse_server.rs).

Note on MSRV: Rust 1.85+ required.

---

Quick Start

1. Basic chat

use langchainrust::{OpenAIChat, OpenAIConfig, BaseChatModel};
use langchainrust::schema::Message;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = OpenAIConfig {
        api_key: std::env::var("OPENAI_API_KEY")?,
        base_url: "https://api.openai.com/v1".to_string(),
        model: "gpt-4o-mini".to_string(),
        ..Default::default()
    };

    let llm = OpenAIChat::new(config);

    let response = llm.chat(vec![
        Message::system("You are a helpful assistant."),
        Message::human("What is Rust?"),
    ], None).await?;

    println!("{}", response.content);
    Ok(())
}

2. Multi-provider — same interface, swap config

use langchainrust::{
    DeepSeekChat, MoonshotChat, ZhipuChat, QwenChat,
    AnthropicChat, OllamaChat,
};

let deepseek = DeepSeekChat::from_env_result()?;
let moonshot = MoonshotChat::with_model("moonshot-v1-128k")?; // reads key from env, returns Result
let claude = AnthropicChat::from_env_result()?;
let ollama = OllamaChat::new("llama3.2");

3. LCEL — compose everything into one chain

use langchainrust::{
    ChatPromptTemplate, Message, OpenAIChat, OpenAIConfig, RunnableExt, StrOutputParser,
};
use std::collections::HashMap;

let llm = OpenAIChat::new(OpenAIConfig {
    api_key: std::env::var("OPENAI_API_KEY")?,
    base_url: "https://api.openai.com/v1".to_string(),
    model: "gpt-4o-mini".to_string(),
    ..Default::default()
});

// prompt.pipe(llm).pipe(parser) — everything is a Runnable (v0.15)
let prompt = ChatPromptTemplate::from_messages([
    Message::system("你是一个简洁的 Rust 助手。"),
    Message::human("{question}"),
]);
let chain = prompt.pipe(llm).pipe(StrOutputParser::new());

let mut vars = HashMap::new();
vars.insert("question".to_string(), "一句话说明什么是 Rust".to_string());
let answer = chain.invoke(vars, None).await?;
println!("{answer}");

For the full five-way composition — prompt + memory + LLM + parser + RAG in one program — see lcel_compose.

4. BM25 keyword search (no API key)

use langchainrust::{BM25Retriever, Document};

let mut retriever = BM25Retriever::new();

retriever.add_documents_sync(vec![
    Document::new("Rust is a systems programming language"),
    Document::new("Python is a scripting language"),
]);

let results = retriever.search("systems programming", 3);

for result in results {
    println!("Document: {}", result.document.content);
    println!("Score: {}", result.score);
}

More examples in 中文使用指南 or Usage Guide.

---

Examples

The crates/lc/examples/ directory provides 42 runnable examples covering core functionality:

Examples requiring API keys read from environment variables:

export OPENAI_API_KEY="your-key"
cargo run --example basic_chat

Examples without API keys (BM25 / LangGraph / Memory / Loader) can run directly — great for quick exploration.

---

Production Considerations

Some hard-won guidance from the framework's design reviews:

• Don't rely on silent keyword fallbacks. LocalEmbeddings without the local-embeddings feature is a Bag-of-Words fallback — don't use it for real semantic retrieval. Enable the feature or use an API provider.
• Embedding batch alignment errors are explicit. Missing or shifted vectors now return EmptyVectorInBatch / BatchMismatch instead of empty slots — treat them as failures, don't skip them.
• Chinese text: prefer Tiktoken-based token counting. The len/4 heuristic over-counts Chinese; ContextWindow uses TiktokenCounter when available.
• Summary memory: old summaries survive LLM failure. A failed summary leaves the previous summary and raw messages intact (last_summary_error() reports the failure); it never silently wipes history.
• MCP / A2A auth is at the JSON-RPC layer. Bearer-token auth returns HTTP 200 with {"error":{"code":401,...}} in the body — check the body, not just the status code. Agent-card discovery is intentionally public.
• MCP failure semantics differ by transport. The stateless HTTP track has no handshake or heartbeat — an unreachable server fails the request immediately, so there is no reconnect to wait on. The stdio transport has no reconnect at all: a child exit / pipe EOF surfaces as MCPError::connection_lost() on the next call; restarting the subprocess is the caller's job. MRTR round trips are bounded by max_round_trips (-32003), and method rate limits fail fast with -32002.
• Pinecone intentionally lacks fetch-by-ID. get_document / get_embedding / clear return explicit StorageErrors; count uses describe_index_stats. Don't design around those ops with Pinecone.
• SQLTool is read-only by default, use parameterized queries. The Python blacklist is noise-filtering, not a security boundary — isolate real code execution with LocalSandbox.
• Summary memory and sessions need no special wiring. EventSessionManager accepts a BaseMemory via with_memory (or per-session with_memory_factory) for persistent multi-turn apps. The legacy SessionManager is deprecated and removed in 0.23.0.
• Unknown model prices bill $0, they don't crash the run. CostTracker records usage for models missing from the PricingTable at zero cost — register the model price explicitly if the USD budget gate must actually constrain it; otherwise the gate silently never trips on that model.
• Checkpoint writes are optimistic. Concurrent stale writes against the same LangGraph checkpoint return GraphError::CheckpointVersionConflict — re-read and retry, never last-write-wins (SQLite/Postgres/Redis backends all use compare-and-swap).

---

Documentation

---

Testing

cargo test

---

Contributing

Contributions welcome! See CONTRIBUTING.md.

---

License

MIT or Apache-2.0, at your option.