🔍 Offline RAG (Retrieval-Augmented Generation)

An open-source, fully offline Retrieval-Augmented Generation (RAG) system ## 📋 To-Do List

• Add more example physics documents
• Implement batch processing for large files
• Add support for more document formats
• Enhance error handling and user feedback
• Add unit tests and documentation
• Optimize embedding process for better performance
• Add configuration file for easy model switching
• Implement user session management

✨ Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Sentence-Transformers for embedding models
• FAISS for efficient similarity search
• llama-cpp-python for local LLM integration
• Flask for the web framework

---

Designed & Developed by VRS Foundation (Vijay R. Shinde)

For questions or support, please open an issue on GitHub.hout internet**, without APIs, and supports any subject — driven by uploaded documents or a master knowledge base.

Built with:

• 🧠 SentenceTransformers for semantic embeddings
• 📦 FAISS for vector indexing
• 🤖 Local LLMs like LLaMA 2 / Mistral via llama-cpp-python
• 🖥️ Flask + HTML/JS frontend with animations and admin control

---

🚀 Features

• 💾 100% Offline: Works in disconnected or air-gapped environments
• 📚 Custom + Default Knowledge: Upload your own PDFs or rely on preloaded knowledge
• ⚡ Fast Embedding & Retrieval: Uses MiniLM or BGE with FAISS
• 🔒 No API Keys or External Calls: All models run locally
• 🎛️ Admin Panel: Upload/delete master KB files
• 🧑‍💻 User Panel: Upload files, ask questions, view answers
• 🎨 Dark-Themed, Animated UI: Modern frontend with clear feedback

---

🧱 Architecture Overview

File Upload -> Text Chunking -> Sentence Embedding -> FAISS Vector Store -> Top-k Retrieval -> Prompt Builder -> llama.cpp (LLaMA / Mistral GGUF) -> Final Answer

                              ┌────────────────────────────┐
                              │         User / Admin       │
                              └────────────┬───────────────┘
                                           │
                                           ▼
                              ┌────────────────────────────┐
                              │    🌐 Frontend (Flask UI)  │
                              │  HTML/CSS/JS + AJAX + Jinja│
                              └────────────┬───────────────┘
                                           │
                      ┌────────────────────▼────────────────────┐
                      │            Flask Web Server             │
                      │           (routes.py + API)             │
                      └────┬──────────────────────────────┬─────┘
                           │                              │
          ┌────────────────▼──┐            ┌──────────────▼──────────────┐
          │  📄 embedder.py   │            │     rag.py (Query Engine)   │
          │ - Chunking logic  │            │ - Embeds question           │
          │ - Embeddings      │            │ - FAISS top-k retrieval     │
          └────┬──────────────┘            └──────────────┬──────────────┘
               │                                          │
 ┌─────────────▼─────────────┐              ┌────────────▼─────────────┐
 │  🔍 FAISS Vector Store   │              │  🧠 llm.py (LLM Prompting)│
 │  - User Index (session)   │              │  - prompt construction    │
 │  - Master KB (fallback)   │              │  - llama-cpp-python call  │
 └───────────────────────────┘              └───────────────────────────┘
                                           ▲
                                           │
                          ┌────────────────▼───────────────┐
                          │   🦙 Local LLM Runtime Engine  │
                          │     (llama-cpp-python / GGUF)   │
                          │   Quantized Mistral / LLaMA     │
                          └─────────────────────────────────┘

---

📂 Project Structure

offline-rag/
├── app/
│   ├── routes.py        # Flask route handling
│   ├── embedder.py      # Chunking + Embedding logic
│   ├── vector_store.py  # FAISS vector store manager
│   ├── rag.py          # Retrieval + LLM answering
│   ├── llm.py          # llama-cpp wrapper
│   └── file_utils.py    # Helper functions
│
├── frontend/
│   ├── templates/       # HTML templates (index, admin, intro)
│   └── static/
│       ├── css/        # Dark UI styling
│       ├── js/         # AJAX & animation logic
│       └── images/     # UI assets
│
├── data/               # Stored embeddings + user/master files
├── models/            # GGUF LLM model (LLaMA/Mistral)
├── README.md
├── requirements.txt
└── run.py            # Entry point

---

📥 Installation

⚠️ Works best on Linux or Windows with Python 3.10+

1. Clone the repo

git clone https://github.com/vijay-shinde/offline-rag.git
cd offline-rag
Create virtual env + install dependencies

```bash
# Create virtual environment
python -m venv envGenAI
source envGenAI/bin/activate  # Linux/Mac
# or
envGenAI\Scripts\activate     # Windows

# Install dependencies
pip install -r requirements.txt

2. Download LLM Model

   • Download a GGUF quantized model (e.g., Mistral 7B Q4)
   • Place it in the models/ directory

3. Run the Application

python run.py

Visit http://localhost:5000 in your browser

🧪 Example Usage

Upload newton_laws_summary.pdf via Admin panel. Ask: “What does Newton’s Second Law explain?”

➡️ The system retrieves from master KB and responds using the local LLM.

📘 Research Summary Inspired by: “Attention is All You Need”, Vaswani et al. (2017)

Embedding: Sentence-BERT (MiniLM-L6-v2 or BGE-Small)

Vector Search: FAISS (Facebook AI)

LLM: LLaMA 2 / Mistral via llama-cpp-python (GGUF format)

📄 Full whitepaper: Offline_RAG_IEEE_Paper.pdf

🛡️ Use Cases Education (offline Q&A)

Secure Enterprises

Remote research labs

Edge AI / Dev sandboxes

🤝 Contribution PRs welcome! Feel free to fork, suggest improvements, or extend it for other domains.

📜 License MIT License

Designed & Developed by [VRS Foundation (Vijay R. Shinde)]

markdown Copy Edit

---