---

📋 Table of Contents

• Overview
• Key Features
• System Architecture
  • Phase 1: Offline Database Ingestion
  • Phase 2: Online Real-Time Inference
• Repository Structure
• Installation & Setup
• Benchmarking
• Future Work

---

🔬 Overview

This system bridges the gap between static clinical documentation and dynamic, context-aware medical decision-making. It ingests the 430+ page AIIMS Rishikesh Standard Treatment Guidelines and converts them into an interactive diagnostic engine that a physician can query using a raw patient EHR.

The architecture is deliberately decoupled: a heavy-inference FastAPI backend handles all LLM computation on GPU, while a lightweight Streamlit frontend provides a clean, responsive physician interface with a mandatory Human-in-the-Loop (HITL) verification gate before any prescription is generated.

Design Philosophy: Each component is labeled as an Agent (1–4). This reflects a modular design where every stage is a replaceable, upgradeable unit. Swap out an LLM call for a fine-tuned specialist model or a fully autonomous agent without touching the surrounding pipeline.

---

✨ Key Features

---

🏗️ System Architecture

The pipeline runs in two distinct phases: an offline ingestion phase (run once) and an online inference phase (run per patient).

╔══════════════════════════════════════════════════════════════════╗
║                   PHASE 1 · OFFLINE INGESTION                   ║
╠══════════════════════════════════════════════════════════════════╣
║                                                                  ║
║  AIIMS PDF (430+ pages)                                          ║
║       │                                                          ║
║       ▼                                                          ║
║  llama-parse  ──────────────►  Markdown (preserves tables)       ║
║       │                                                          ║
║       ▼                                                          ║
║  fuzzy_extract.py  ─────────►  300+ Disease Markdown Files       ║
║       │                                                          ║
║       ├──► Agent 1 (MedGemma-27B)  ──►  IF-ELSE Logic .txt Files ║
║       │                                                          ║
║       └──► Agent 2 (MedGemma-27B)  ──►  Clinical Checklists JSON ║
║                                                                  ║
╠══════════════════════════════════════════════════════════════════╣
║                  PHASE 2 · ONLINE INFERENCE                      ║
╠══════════════════════════════════════════════════════════════════╣
║                                                                  ║
║  Physician inputs EHR text                                       ║
║       │                                                          ║
║       ▼                                                          ║
║  Dense Semantic Search  ───────►  Top 5 Relevant Guidelines      ║
║       │                                                          ║
║       ▼                                                          ║
║  Agent 3 ──► Fetch checklists + Extract variables from EHR       ║
║       │      (Pydantic schema validation)                        ║
║       ▼                                                          ║
║  🧑‍⚕️ PHYSICIAN VERIFICATION  ◄── Edit / Confirm / Fill gaps       ║
║       │                                                          ║
║       ▼                                                          ║
║  Agent 4 ──► Evaluate IF-ELSE logic ──► Stream Final Prescription║
║                                                                  ║
╚══════════════════════════════════════════════════════════════════╝

Phase 1: Offline Database Ingestion

Run once to build the knowledge base. No GPU required for fuzzy extraction.

1. Document Parsing — The AIIMS PDF is converted to Markdown via llama-parse (external service). This critical step preserves structured content like dosage tables and diagnostic flowcharts that plain-text extraction destroys.

2. Fuzzy Extraction — fuzzy_extract.py uses regex and fuzzy string matching to segment the master Markdown document into 300+ individual disease files, one per condition.

3. Logic Translation · Agent 1 — MedGemma-27B converts each disease Markdown file into a strict, machine-evaluable IF-ELSE logic text file. This is the rule engine that Agent 4 will query at runtime.

4. Checklist Pre-generation · Agent 2 — MedGemma-27B pre-generates the required clinical question checklists (JSON) for every disease. Pre-computing these offline is what makes online inference fast.

Phase 2: Online Real-Time Inference

Called live for every patient. Runs on GPU.

1. Semantic Search — The physician pastes a patient's EHR. Dense vector search returns the top 5 most relevant guidelines from the database.

2. Extraction · Agent 3 — Fetches pre-generated checklists for the selected diseases and extracts answers directly from the EHR text using dynamic Pydantic schema validation.

3. Human Verification — The Streamlit UI presents all extracted variables to the physician. This is the critical safety gate: doctors review, correct, or fill any missing information before proceeding.

4. Recommendation · Agent 4 — Evaluates the confirmed variables against the combined IF-ELSE logic to stream a final, safe prescription alongside step-by-step clinical reasoning.

---

📂 Repository Structure

ai-clinical-decision-support-system/
│
├── offline phase/
│   ├── data/
│   │   ├── clinical_checklists_db/      # Pre-computed JSON question files (one per disease)
│   │   ├── disease_algorithms_db/       # IF-ELSE logic text files (Agent 1 output)
│   │   ├── disease_markdown_files/      # Segmented Markdown per disease
│   │   └── standard-treatment-guidelines.pdf  # Source AIIMS document
│   │
│   └── scripts/
│       ├── fuzzy_extract.py             # Document segmentation into disease files
│       ├── generate_questions.py        # Clinical checklist generation (Agent 2)
│       └── process_guidelines.py        # Markdown → IF-ELSE logic processing (Agent 1)
│
├── online phase/
│   ├── backend/
│   │   └── backend.py                   # FastAPI server — loads embeddings + MedGemma into VRAM
│   │
│   ├── benchmark data/
│   │   ├── samples.csv                  # sample cases for evaluation
│   │   └── benchmark.py                 # Automated comparison: 27B vs 4B models
│   │
│   └── frontend/
│       └── frontend.py                  # Streamlit UI — HITL verification + streaming output
│
├── requirements.txt
└── .gitignore

---

🚀 Installation & Setup

Prerequisites

Note: The backend (GPU inference) and the frontend (Streamlit UI) are designed to run on separate nodes. The frontend communicates with the backend over HTTP — ideal for cloud GPU instances paired with a local client.

---

Step 1 · Clone the Repository

git clone https://github.com/harshcooljn-iit/AI-Clinical-Decision-Support-System.git
cd ai-clinical-decision-support-system

Step 2 · Install Dependencies

pip install -r requirements.txt

Step 3 · Start the Backend (GPU Node)

This loads the embedding model and MedGemma-27B weights into VRAM. Allow a few minutes for model loading on first run.

cd "online phase/backend"
uvicorn backend:app --host 0.0.0.0 --port 8000

Step 4 · Start the Frontend (Local / Client Node)

In a separate terminal, launch the Streamlit UI. By default it connects to localhost:8000 — update the backend URL in frontend.py if running on a remote GPU node.

cd "online phase/frontend"
streamlit run frontend.py

The UI will be accessible at http://localhost:8501

---

(Optional) Re-run Offline Ingestion

To rebuild the disease database from scratch (e.g., after updating the guidelines PDF):

# 1. Segment the master document into per-disease Markdown files
python "offline phase/scripts/fuzzy_extract.py"

# 2. Generate IF-ELSE logic for each disease (Agent 1)
python "offline phase/scripts/process_guidelines.py"

# 3. Pre-compute clinical checklists (Agent 2)
python "offline phase/scripts/generate_questions.py"

---

📊 Benchmarking (Data Collection Pipeline)

The benchmark.py script generates evaluation data by running the full pipeline on real-world clinical records (MIMIC-III).

⚠️ Note: Quantitative evaluation metrics (e.g., hallucination rate, clinical safety) are not yet implemented. This script currently serves as a data generation pipeline for downstream analysis.

What the script does:

• Samples patient records from MIMIC-III
• Runs end-to-end inference (Search → Extract → Recommend)
• Stores:
  • Retrieved guidelines
  • Extracted clinical variables
  • Final generated prescriptions

This output can be used for:

• Manual evaluation by clinicians
• Future automated metric computation
• Model comparison (e.g., 27B vs 4B)

# Run the pipeline to generate evaluation data
python "online phase/benchmark data/benchmark.py"

Output is saved as CSV files for downstream analysis and visualization.

---

🔮 Future Work

Transition to True Multi-Agent Systems

The current architecture uses a single LLM across all agents. Future iterations will specialize each agent independently:

• Agent 3 (Extraction) → Fine-tuned clinical NER model, optimized purely for entity extraction from EHRs.
• Agent 4 (Recommendation) → A Reasoning Agent with access to live medical knowledge graphs (e.g., SNOMED CT, RxNorm).
• Self-Critique Agent → A dedicated "reviewer" that checks Agent 4's prescription against the source guidelines before the physician ever sees it.
• Autonomous Tool Use → Agents that proactively query external databases (PubMed, UpToDate) when internal AIIMS guidelines are insufficient for complex or rare cases.
• Multi-Agent Collaboration → Specialized departmental agents (e.g., a Cardiology Agent and a Pharmacology Agent) that debate and co-refine treatment plans for multi-morbidity patients.

Prompt & Pipeline Optimization

• DSPy Integration — Transition Agent 4 from static f-string prompts to programmatic prompt compilation via DSPy, mathematically optimizing for instruction-following and clinical accuracy.
• Dynamic Model Routing — A router that dispatches simple extraction tasks to fast small models (e.g., Llama-3-8B) while reserving complex reasoning for MedGemma-27B.

Expanded Knowledge Base

• Additional Guidelines — Integrate WHO and Mayo Clinic protocols into the offline ingestion pipeline alongside AIIMS.
• Multilingual Support — Extend to regional-language guidelines for broader Indian hospital deployment.

---