🚜 Enterprise RAG Diagnostic Node

A production-grade Retrieval-Augmented Generation (RAG) pipeline designed for heavy machinery diagnostic manuals. This system enforces strict Data Provenance, utilizes Hybrid Search (Dense + Sparse), and features an NLI Cross-Encoder Guardrail to programmatically block LLM hallucinations before they reach field technicians.

Designed with a high-contrast, neobrutalist frontend for industrial deployment visibility.

🏗️ Core Architecture & SDLC

1. Massive Multimodal Ingestion (v1_ingest_knowledge.py):

   • Ingests local proprietary PDFs (e.g., technical schematics placed in /manuals) and scales across internal documentation wikis.
   • Employs RecursiveCharacterTextSplitter for semantic paragraph-level chunking to maintain engineering context.
   • Hybrid Search Indexing: Compiles both a Dense Vector Store (FAISS via Hugging Face all-MiniLM-L6-v2) and a Sparse Keyword Store (BM25) to guarantee retrieval of exact part numbers and conceptual issues.

2. The Diagnostic Agent & API (api.py):

   • Structured JSON Outputs: Implements strict prompt engineering and Pydantic-style schemas to enforce JSON diagnostic reports for downstream dashboard consumption.
   • Data Provenance: Actively tracks and returns the exact source documentation metadata (topic_cluster) used to generate the response.

3. Responsible AI Guardrails (The NLI Judge):

   • The Problem: LLMs hallucinate unsafe physical repair instructions, posing a severe risk in heavy manufacturing.
   • The Solution: An independent Natural Language Inference (NLI) Cross-Encoder (nli-distilroberta-base) evaluates the generated text against the retrieved context.
   • The Logic: If the Contradiction logit exceeds the Entailment logit (measuring groundedness), the system actively intercepts the payload, blocks the output, and returns a safe fallback error to the technician.

🚀 Tech Stack

• Frameworks: FastAPI, React (Neobrutalist UI), TailwindCSS
• RAG & Retrieval: LangChain, FAISS (Dense), BM25 (Sparse)
• Machine Learning: PyTorch, Hugging Face Transformers
• Models: Flan-T5-Large (Generation), nli-distilroberta-base (Evaluation)
• Focus Areas: Responsible AI, Hallucination Mitigation, Groundedness Evaluation, Prompt Management

🛠️ Local Setup & Execution

1. Install Dependencies: pip install fastapi uvicorn langchain langchain-community langchain-huggingface faiss-cpu sentence-transformers rank_bm25 pypdf wikipedia

2. Build the Knowledge Base: Drop any proprietary PDFs into the /manuals directory, then compile the FAISS and BM25 indices: python v1_ingest_knowledge.py

3. Boot the diagnostic API: python api.py

4. Launch the UI: Open index.html in any modern web browser to access the Diagnostic Node frontend.