Recall is a Retrieval-Augmented Generation (RAG) application built as an AI Engineering learning project. You upload .txt, .md, or .pdf documents, and Recall indexes them so you can search and ask questions about their content using natural language.

How it works

You ask:  "How does authentication work?"
              ↓
Recall searches your documents for relevant passages
              ↓
Gemini generates a grounded answer with citations
              ↓
You get:  "Authentication uses JWT tokens [1]. Configure
           the secret via JWT_SECRET [2]."

---

Table of Contents

Part 1 - Project & Setup

• 1. What Is Recall?
• 2. Quick Start
• 3. Project Structure
• 4. Technology Stack & Packages
• 5. Configuration
• 6. API Reference
• 7. Evaluation Harness

Part 2 - Learn AI Concepts (in docs/)

• 8. What Is RAG?
• 9. The Recall Architecture
• 10. Phase 1 - Document Ingestion
• 11. Phase 2 - Retrieval
• 12. Phase 3 - Reranking
• 13. Phase 4 - Generation
• 14. Streaming AI Responses
• 15. End-to-End Request Flow
• 16. Real Query Walkthrough
• 17. Important Trade-offs
• 18. RAG Failure Modes
• 19. Evaluating a RAG System
• 20. How to Study This Project
• 21. Experiment Lab
• 22. Final Mental Model

---

Part 1 - Project & Setup

1. What Is Recall?

Recall is a personal knowledge application. You upload documents, and Recall indexes them so you can search and ask questions about their content.

When you ask a question like:

"How does authentication work in this project?"

Recall:

1. Searches your indexed documents for relevant passages
2. Ranks the most relevant passages
3. Passes those passages to Gemini as context
4. Generates an answer based on that context
5. Cites its sources so you can verify

Design principle: The LLM should not be responsible for remembering your documents. Retrieval supplies the knowledge; the LLM performs reasoning over that knowledge.

---

2. Quick Start

Prerequisites

• Node.js 18+ (recommended: 20+)
• npm (or pnpm)
• A Gemini API key (free tier available)
• ~2GB free disk space (for model downloads on first run)

Install & Run

git clone <repo-url>
cd recall
npm install

Set up your environment:

cp .env.example .env

Edit .env and add your Gemini API key:

GEMINI_API_KEY=your_key_here

That's the only required key. Everything else runs locally.

Start the dev server:

npm run dev

Open http://localhost:3000, upload a document, and ask a question.

Docker

docker compose up -d --build   # build & start on http://localhost:3001
docker compose down            # stop

The compose file reads your .env, mounts ./data and ./.cache as volumes (indexes and models persist across rebuilds), and bakes the local embedding model into the image via the WARM_EMBEDDINGS build arg.

Deploy to Render (free tier, no credit card)

The repo includes a Dockerfile. Render's Docker runtime builds it automatically — docker-compose.yml is local-only and ignored.

1. Push the repo to GitHub, then in dashboard.render.com create a Web Service from the repo (language: Docker)
2. Add environment variables:
   • GEMINI_API_KEY
   • EMBED_PROVIDER=gemini (fast uploads on the free tier's 0.1 vCPU)
   • RERANK_PROVIDER=jina + JINA_RERANKING_API_KEY (avoids loading local models)
3. Deploy

Because the dashboard cannot pass build args, Render builds a slim image with no ML models baked in (WARM_EMBEDDINGS=false, WARM_RERANKER=false by default) — correct, since API providers need no model files.

Free-tier caveats:

Switching embedding providers or dimensions? Vectors are not compatible across providers/dimensions. Delete data/lancedb/ and re-index all documents after changing EMBED_PROVIDER or GEMINI_EMBED_DIMENSION.

First Run Notes

On the first upload or query, Recall downloads ML models to the .cache/ directory (~180MB total). This only happens once, models are cached for subsequent runs.

Troubleshooting

"GEMINI_API_KEY not found" - Check that .env exists and contains your key. Restart the dev server after editing.

Model download fails - The first query requires internet access to download models from HuggingFace. Check your connection and retry.

Search returns "I don't have that information in your documents." - This is expected when no relevant chunk is found. Upload a document with content matching your question, or ask something different.

PDF parsing returns no text - Some PDFs are image-based (scanned). Recall requires text-based PDFs. Try a .txt or .md file instead.

---

3. Project Structure

recall/
├── app/
│   ├── api/
│   │   ├── upload/route.ts        # POST /api/upload — document ingestion
│   │   ├── search/route.ts        # POST /api/search — query with SSE streaming
│   │   └── documents/route.ts     # GET/DELETE /api/documents — list & delete
│   │
│   ├── components/                # UI components
│   │   ├── FileUpload.tsx         # File picker with upload logic
│   │   ├── DocumentManager.tsx    # Lists indexed documents, delete capability
│   │   ├── SearchBar.tsx          # Query input form
│   │   ├── Results.tsx            # Displays retrieved chunks with scores
│   │   └── Answer.tsx             # Streamed markdown answer with citations
│   │
│   └── lib/                       # Core AI pipeline (the interesting part)
│       ├── config.ts              # Pipeline constants (chunk size, top-k, etc.)
│       ├── types.ts               # Shared TypeScript types
│       ├── parsers.ts             # Text extraction (.txt/.md/.pdf)
│       ├── chunker.ts             # Overlapping word-window chunking
│       ├── embeddings.ts          # On-device MiniLM embeddings
│       ├── vectordb.ts            # LanceDB vector store
│       ├── minisearch.ts          # MiniSearch BM25 index
│       ├── search.ts              # Hybrid search + Reciprocal Rank Fusion
│       ├── rerank.ts              # Cross-encoder reranking (local or Jina API)
│       └── generate.ts            # Gemini grounded prompt + streaming
│
├── data/                          # Persisted indexes (auto-created, gitignored)
│   ├── lancedb/                   # Vector database
│   └── minisearch.json            # BM25 index
│
├── eval/                          # Offline retrieval evaluation
│   ├── sample-docs/               # 13 benchmark documents
│   └── qa.jsonl                   # 46 gold-standard queries
│
├── docs/                          # Part 2 — Learn AI Concepts guide
│
├── scripts/
│   ├── evaluate.ts                # Benchmark script (npm run eval)
│   └── warm-models.mjs            # Pre-downloads ML models for Docker builds
│
├── Dockerfile                     # Production image (Render-ready)
├── docker-compose.yml             # Local containerized run
├── .env.example                   # Environment variables template
├── .env                           # Your local configuration (gitignored)
└── package.json

The app/lib/ Pipeline

Each file in app/lib/ maps to one stage of the AI pipeline, making it easy to study one concept at a time:

parsers.ts       →  Extract text from files
      ↓
chunker.ts       →  Split text into smaller pieces
      ↓
embeddings.ts    →  Convert text chunks into vectors
      ↓
vectordb.ts      →  Store vectors for semantic search
minisearch.ts    →  Store text for keyword search
      ↓
search.ts        →  Find relevant chunks using both methods
      ↓
rerank.ts        →  Reorder chunks by relevance
      ↓
generate.ts      →  Generate an answer using the best chunks

---

4. Technology Stack & Packages

Core Framework

AI & ML

Data & Search

UI & Styling

Dev Tools

---

5. Configuration

All options live in .env. Copy .env.example to .env and adjust as needed.

Warning: vectors are not compatible across embedding providers or dimensions. After changing EMBED_PROVIDER or GEMINI_EMBED_DIMENSION, delete data/lancedb/ and re-index all documents.

Choosing an Embedding Provider

Documents are embedded with RETRIEVAL_DOCUMENT and queries with RETRIEVAL_QUERY, so Gemini optimizes each vector for its role. All Gemini embeddings are L2-normalized (including truncated dimensions), so cosine similarity works out of the box.

Choosing a Reranker Provider

---

6. API Reference

POST /api/upload

Uploads and indexes a document.

Request: multipart/form-data with a file field.

curl -X POST http://localhost:3000/api/upload \
  -F "file=@my-notes.md"

Pipeline: Upload → Parse → Chunk → Embed → Store (LanceDB + MiniSearch)

Response:

{
  "success": true,
  "fileName": "my-notes.md",
  "chunksIndexed": 42
}

Supported formats: .txt, .md, .pdf (text-based only) Max file size: 10 MB Re-uploading the same filename replaces the old document

---

POST /api/search

Searches indexed documents and streams a grounded answer.

Request:

{
  "query": "How does authentication work?"
}

Response: Server-Sent Events (SSE) stream.

Pipeline: Query → Embed → Hybrid Search (RRF) → Rerank → Gemini → Grounded Answer

Example SSE stream:

event: sources
data: [{"id":"chunk-1","text":"...","docId":"a1b2c3","score":0.92}, ...]

event: token
data: The

event: token
data:  application

event: done

Retrieval happens entirely on-device. Only the final generation calls Gemini. If reranking fails, the system falls back to the RRF ranking.

---

GET /api/documents

Lists all indexed documents.

[
  {
    "docId": "a1b2c3d4",
    "source": "my-notes.md",
    "chunkCount": 42
  }
]

---

DELETE /api/documents?docId=...

Deletes a document and all its chunks from both indexes.

curl -X DELETE "http://localhost:3000/api/documents?docId=a1b2c3d4"

---

7. Evaluation Harness

Recall ships with an offline retrieval benchmark that measures whether the retrieval pipeline actually finds the right chunks, without calling Gemini.

Run It

npm run eval                 # Index sample docs, then evaluate
npm run eval -- --fresh      # Wipe indexes first for an isolated benchmark
npm run eval -- --no-ingest  # Evaluate against already-indexed documents

What It Measures

• Dataset: 13 sample documents + 46 gold-standard queries with expected text snippets
• Relevance: A retrieved chunk is "relevant" if it contains the expected snippet
• Metrics: Hit Rate@K, MRR, nDCG@K, reported for both pre-rerank and post-rerank results

Sample Output

Retrieval evaluation (higher is better)

metric     | RRF (pre-rerank) | Reranked
------------------------------------------------------
HR@1       | 65.2%            | 76.1%
HR@3       | 87.0%            | 91.3%
HR@5       | 91.3%            | 95.7%
MRR        | 0.768            | 0.842
nDCG@5     | 0.797            | 0.866

Why Evaluate Retrieval Separately?

An incorrect final answer doesn't tell you where the system failed:

The benchmark isolates retrieval, making it cheap, reproducible, and fast; no generation calls, just measurable retrieval quality.

Reproducibility note: the benchmark uses whatever embedding/reranking providers are configured. The sample numbers above were produced with the defaults (EMBED_PROVIDER=local, RERANK_PROVIDER=local). Switching providers changes the scores — which is exactly what makes the harness useful for comparing them.

---

---

Part 2 - Learn AI Concepts

The full concept guide now lives in docs/. Each document explains one AI engineering concept and maps it directly to the file in app/lib/ that implements it.

---