YouRAG — Advanced YouTube RAG with Knowledge Graph & Hybrid Retrieval

YouRAG is a production-grade Retrieval-Augmented Generation (RAG) system that turns any YouTube video into a queryable knowledge base. It combines Knowledge Graph, Hybrid Search, Cross-Encoder Reranking, and Citation Grounding — state-of-the-art techniques that eliminate hallucination at the architecture level, backed by a RAGAS benchmark suite used to A/B test every architectural decision (see Benchmark Results).

Not just a video chatbot. YouRAG builds a "digital brain" for each video: extracts entities, builds a knowledge graph, validates every citation, and forces the AI to cross-check facts before answering.

---

Architecture

flowchart TD
    URL([YouTube URL]) --> YL[YouTubeLoader\npytubefix + transcript-api]
    YL -->|no captions| WH[WhisperTranscriber\nfaster-whisper STT]
    YL -->|has captions| SC
    WH --> SC[SemanticChunker\npause-aware + vector valleys]
    SC -->|optional| CE[ContextualEnricher\nLLM prefix injection]
    SC -->|optional| LC[LateChunkingEmbedder\nJina jina-embeddings-v3]
    CE --> GE
    LC --> GE
    SC --> GE[GraphExtractor\nrule-based keyword/entity metadata]
    GE --> QD[(Qdrant VectorDB\nBAAI/bge-m3 · 1024-dim\ncosine HNSW)]
    QD -->|background thread| KGB[KnowledgeGraphBuilder\nLLM triple extraction]
    KGB --> KG[(Knowledge Graph\nNetworkX DiGraph)]

    Q([User Query]) --> HR[HybridRetriever\nDense + SPLADE + QueryExpansion + HyDE · RRF Fusion]
    Q --> GR[GraphRetriever\nmulti-hop traversal]
    QD --> HR
    KG --> GR
    HR --> CR[CrossEncoderReranker\nbge-reranker-v2-m3]
    CR --> CC[ContextualCompressor]
    CC --> AG[AnswerGenerator]
    GR --> AG
    AG -->|opt-in flag, off by default| SC2[Self-Correction\nLLM audits draft vs graph facts]
    AG -->|default path| CG[Citation Grounding\nremove fabricated timestamps]
    SC2 --> CG
    CG --> ANS([Final Answer\n+ mm:ss Citations])

---

Key Features

Anti-Hallucination Stack

• Strict system prompt — "If not in the document, say you don't know"
• Citation Grounding — Every [mm:ss] in the answer is validated against retrieved chunks; fabricated citations are removed
• No-context early return — Returns "not found" immediately when no chunks are retrieved, instead of letting the LLM hallucinate from general knowledge
• Prompt-leak guard — Strips internal prompt section headers if the LLM accidentally echoes them into the final answer
• Self-Correction (optional, off by default) — A second LLM call that audits the draft against Knowledge Graph facts, gated by ENABLE_SELF_CORRECTION. Implemented and tested, but a controlled A/B benchmark (33 questions, RAGAS) showed it made factual_correctness worse (0.570 ON vs 0.582 OFF) while adding a full extra LLM call per query — so it's disabled by default. Kept as a flag for future re-evaluation rather than deleted, since the finding was reached by measurement, not assumption. See Benchmark Results.

RAG Pipeline

• 2-Phase Semantic Chunking: Pause-aware atomic splitting + vector semantic valley detection (dynamic percentile threshold, adapts per video)
• Knowledge Graph: Extracts (subject, predicate, object) triples → NetworkX directed graph → multi-hop reasoning
• 4-Signal Hybrid Search + RRF: Dense (bge-m3) + SPLADE neural sparse + Query Expansion (2 variants) + HyDE — all fused via Reciprocal Rank Fusion
• Cross-Encoder Reranking: BAAI/bge-reranker-v2-m3 for precise relevance scoring
• Contextual Compression: Filters irrelevant sentences from each chunk before LLM generation — reduces noise, improves faithfulness
• Semantic Cache: Per-collection Qdrant-backed cache — same question on different videos never cross-contaminates

Production Hardening

• API key authentication — X-API-Key header protects all write endpoints
• Rate limiting — slowapi: 20 req/min for chat, 5 req/min for ingest
• YouTube URL validation — Backend validates URL before queuing job (HTTP 422 on invalid)
• Redis job store — Ingest jobs persist in Redis with 24h TTL; auto-fallback to in-memory in dev
• /health endpoint — Checks Qdrant, Redis, Generator, Reranker; returns 200 or 503
• Groq key rotation — Auto-rotate backup keys when primary hits rate limit (GROQ_API_KEYS)
• Whisper STT fallback — Auto-transcribes videos without captions via faster-whisper

Frontend (Next.js 14)

• Dark / Light mode toggle
• Mobile responsive — drawer sidebar + tab layout on small screens
• Dynamic welcome suggestions — 4 questions generated from actual video content
• Source chips with seek-to-timestamp + open YouTube at exact timestamp
• Library page — manage all ingested videos, delete, rebuild graph
• Learn page — Quiz (MCQ với giải thích) + Flashcard (3D flip, Anki-style) với "Xem trong video" modal
• Analytics page — Prometheus metrics dashboard
• Settings page — system config overview

---

Tech Stack

---

Quick Start

Requirements

• Docker + Docker Compose
• Groq API Key (free)

1. Clone & Configure

git clone https://github.com/td041/YouRAG.git
cd YouRAG
cp .env.example .env
# Edit .env and set GROQ_API_KEY

2. Start

docker compose up -d --build

3. Access

---

API Endpoints

Protected endpoints require X-API-Key header when API_KEY is set in .env.

---

Testing

# Unit tests (266 tests, all mocked)
poetry run pytest tests/unit/ -v --cov=src --cov-report=term-missing

# Lint
poetry run ruff check src/ tests/

# Security
poetry run bandit -r src/ -ll

# RAGAS benchmark
make benchmark
# or with specific collection:
make benchmark COLLECTION=your-collection-name

---

Benchmark Results

Evaluated on 33 questions (mixed difficulty: factual, reasoning, comparative, synthesis) using:

• Generation: llama-3.3-70b-versatile via Groq (multi-key round-robin)
• Evaluator: mistral-small-latest via Mistral AI
• Reranker: BAAI/bge-reranker-v2-m3
• Embeddings (eval): paraphrase-multilingual-MiniLM-L12-v2 (multilingual)

Current baseline (Advanced tier, production config)

Run: 2026-07-16, current codebase (ENABLE_SELF_CORRECTION=false, prompt-leak guard, GPU-lock + model-warmup fixes for concurrent local inference).

factual_correctness reads lower than the other four metrics by design of the metric, not the model: RAGAS decomposes answers into atomic claims and scores strict claim-level F1 against a terse reference, which penalizes correct-but-elaborated answers (our answers average ~4x longer than the reference text). A manual audit of the mid-scoring questions confirmed most of that gap is measurement strictness, not factual errors — see commit history / tests/benchmark/eval_dataset.json for the audited dataset.

Self-correction A/B test

ENABLE_SELF_CORRECTION gates a second LLM call that audits the draft answer against Knowledge Graph facts. It sounds like it should only help — in practice, a controlled A/B run (same 33 questions, same code, only the flag flipped) showed the opposite:

ON loses on 4 of 5 quality metrics and is ~44% slower (one extra LLM round-trip per query). Disabled by default as a result — see the Anti-Hallucination Stack section for how the flag is kept for future re-evaluation instead of deleted.

3-tier ablation study (historical)

Isolates the retrieval architecture's contribution: same 33 questions, same generation step, only the retrieval tier changes.

Advanced tier wins on all 5 RAGAS metrics — the cross-encoder reranker's context precision gain (+48% over naive) is the largest single driver. Run: 2026-07-07, predates the self-correction A/B fix and current baseline above — kept for the relative naive→hybrid→advanced trend, not as an absolute comparison to the current baseline table.

---

Environment Variables

See .env.example for full reference.

---

Roadmap

• Semantic Chunking (pause-aware + vector valleys)
• Knowledge Graph RAG (entity extraction + multi-hop)
• Hybrid Search + RRF Fusion
• Cross-Encoder Reranking
• Graph-based Self-Correction (implemented, off by default — A/B benchmark showed it hurts factual correctness)
• Citation Grounding (no fabricated timestamps)
• Streaming + [mm:ss] Citations
• Whisper STT fallback
• Persistent Chat History (Redis + PostgreSQL)
• Dark/Light mode + Mobile responsive UI
• API Key auth + Rate limiting
• Redis job store + /health endpoint
• Groq backup key rotation
• Docker Compose (5 services) + uv fast builds
• CI/CD (Ruff + Bandit + Pytest, 266 unit tests)
• Multi-video cross-referencing (multi-collection chat)
• Visual Frame RAG (Groq/OpenAI vision — slides, diagrams, code on screen)
• Quiz & Flashcard generation (/learn page)
• Whisper Vietnamese language detection
• Grafana alerting (4 rules: error rate, latency, memory, uptime)
• SPLADE neural sparse retrieval (replaces BM25, bilingual VI+EN)
• Query Expansion (multi-query, 2 LLM-generated variants per query)
• HyDE (Hypothetical Document Embeddings for better semantic match)
• Contextual Compression (filter irrelevant sentences before LLM)
• Playlist / Channel bulk ingest
• Deploy to Railway + Vercel (configs ready)

---

Built by td041