AI Viva — Validation Module

The Validation Module is the semantic checking engine of the AI-Based Viva System (SN Bose Summer Internship, NIT Silchar). It receives a student's transcribed spoken answer, verifies it is a genuine attempt (not silence, stuttering, ASR noise, or gibberish), and scores it against the examiner's reference answer using transformer embeddings, zero-shot NLI classification, and fuzzy concept matching.

It has no database — it is a stateless API service. Every request contains everything it needs, and nothing is stored. (The small built-in "RAG knowledge base" is an in-memory Python list used only as a fallback when no reference answer is sent.)

---

Where it fits in the AI Viva System

Question Bank Module ──(question + expected answer)──┐
                                                     ▼
Speech Module ──(student's transcribed answer)──► VALIDATION MODULE ──► Evaluation Module ──► Result Storage
                                                (this repository)

1. Question Bank Module supplies the question and expected_answer (its GET /questions/set/{id}/answers endpoint returns exactly the answer map this module needs).
2. Speech Module records the student, transcribes speech to text, and supplies student_answer (plus optional speech_metadata such as ASR confidence).
3. This module returns a ValidationResponse: valid/invalid, relevance, semantic similarity, completeness, confidence, remarks.
4. Evaluation Module converts those scores into marks.
5. Result Storage Module persists the final results (this module stores nothing itself).

---

Requirements

• Python 3.12 (recommended and verified) — 3.13 also works (the code includes a TorchScript workaround specifically for 3.13).
• ~2 GB free disk space for Python packages (PyTorch, Transformers), plus space for the NLP models you choose (90 MB – 2.3 GB, see below).
• Internet connection on first run — models download automatically from Hugging Face and are then cached locally.

No database, no API keys, and no .env file are required to run it.

---

Setup — macOS

Open the Terminal app and run these commands one at a time:

# 1. Go into the project folder
cd path/to/Validation-Module_AI-Viva-main

# 2. Create a virtual environment (a private box for this project's packages)
python3.12 -m venv .venv

# 3. Activate it (your prompt will show ".venv" when active)
source .venv/bin/activate

# 4. Install all dependencies (takes a few minutes — PyTorch is large)
pip install -r requirements.txt

# 5. (Recommended for laptops) use small, fast models:
cp .env.example .env

# 6. Start the server
uvicorn app.main:app --reload --port 8000

Setup — Windows

Open PowerShell and run:

# 1. Go into the project folder
cd path\to\Validation-Module_AI-Viva-main

# 2. Create a virtual environment
py -3.12 -m venv .venv

# 3. Activate it
.venv\Scripts\Activate.ps1
#    If you get a "running scripts is disabled" error, first run:
#    Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

# 4. Install all dependencies
pip install -r requirements.txt

# 5. (Recommended for laptops) use small, fast models:
copy .env.example .env

# 6. Start the server
uvicorn app.main:app --reload --port 8000

What each command does

The server runs at http://127.0.0.1:8000 — Swagger docs at http://127.0.0.1:8000/docs, ReDoc at /redoc. Stop it with Ctrl+C. Reactivate the venv in any new terminal before running again.

⏳ The first /validate request is slow — the server downloads the NLP models on first use (with the .env.example settings: ~180 MB; with the built-in defaults: ~2.9 GB). This happens once; models are cached in ~/.cache/huggingface afterward. Later requests take well under a second.

---

Configuration (.env — optional)

Every setting has a working default in app/config/config.py; the .env file only overrides them. See .env.example for the full list. The important ones:

---

API Endpoints

GET / — service info

Returns app name, environment, and docs URL. Quick "is it up?" check.

GET /health — health check

Returns status, the configured embedding model, and score thresholds.

POST /api/v1/validate — the main endpoint

Request body:

{
  "question": "What is an operating system and what does it do?",
  "expected_answer": "An operating system is software that acts as an interface between computer hardware and the user. It manages files, memory, processes, and input and output devices.",
  "student_answer": "An operating system is system software that manages hardware and software resources and acts as a bridge between the user and the hardware.",
  "language": "English",
  "speech_metadata": {}
}

• question (required, ≥5 chars) — the viva question.
• expected_answer (optional) — the reference answer from the Question Bank Module. If omitted, the module retrieves the closest match from its built-in RAG knowledge base.
• student_answer (required) — the transcript from the Speech Module.
• language (default "English") — selects the embedding model (English vs multilingual).
• speech_metadata (optional) — flags: use_llm (Gemini), use_local_llm (local 8B QA model), generate_feedback (local 30B feedback model), translate_input (Indic→English translation), plus any ASR metadata.

Verified real response:

{
  "validation_status": "Valid",
  "relevance_score": 0.93,
  "semantic_similarity": 0.95,
  "completeness": "Complete",
  "confidence": 0.92,
  "remarks": "Answer is relevant and semantically correct. Concept coverage: 66%."
}

And for a gibberish/stutter transcript ("uh um zzzzzz aaaa"):

{
  "validation_status": "Invalid",
  "relevance_score": 0.0,
  "semantic_similarity": 0.0,
  "completeness": "Irrelevant",
  "confidence": 0.1,
  "remarks": "Invalid response attempt. Gibberish or repeating characters detected in word: 'zzzzzz'"
}

How the score is computed (default local pipeline)

1. Clean the transcript (strip punctuation, lowercase English, remove fillers like um, matlab, है ना — preserving Indic matras).
2. Quality gate — reject empty, too-short, highly repetitive, or gibberish transcripts immediately.
3. Embed question, expected answer, and student answer; compute cosine similarities.
4. Relevance = 0.65 × similarity-to-expected-answer + 0.35 × similarity-to-question.
5. Completeness — extract key concepts from the reference answer, fuzzy-match them (RapidFuzz ≥85) against the student's words: ≥60% coverage = Complete, ≥30% = Partially Complete, else Irrelevant.
6. Validity — zero-shot NLI entailment check between student and reference answer.
7. Confidence = 0.3 × transcript quality + 0.7 × semantic similarity.

---

Testing

With Swagger (easiest)

Open http://127.0.0.1:8000/docs → expand POST /api/v1/validate → Try it out → edit the example JSON → Execute.

With curl

curl -X POST http://127.0.0.1:8000/api/v1/validate \
  -H "Content-Type: application/json" \
  -d '{"question":"What is RAM?","expected_answer":"Random Access Memory","student_answer":"RAM stands for random access memory","language":"English"}'

(Windows PowerShell: use curl.exe instead of curl, or use Postman — POST to the URL with the JSON as Body → raw → JSON.)

Automated tests

python -m pytest tests/test_text_processor.py tests/test_api.py tests/test_validation_service.py -v

⚠️ Do not run the full suite (pytest) blindly: tests/test_model_features.py exercises the optional QA, feedback, and translation models, which will try to download tens of gigabytes. Run the three files above instead.

---

Project Structure

Validation-Module_AI-Viva-main/
├── app/
│   ├── main.py                      # FastAPI app, CORS, global error handlers, / and /health
│   ├── api/v1/validation.py         # POST /api/v1/validate route
│   ├── schemas/validation.py        # Request/response models (ValidationRequest/Response)
│   ├── services/
│   │   ├── validation_service.py    # Core engine: model loading, embeddings, NLI, scoring
│   │   └── rag_service.py           # In-memory fallback knowledge base of reference answers
│   ├── utils/
│   │   ├── text_processor.py        # Cleaning, filler removal, concept extraction, quality gate
│   │   └── exceptions.py            # Domain exceptions → clean JSON error responses
│   └── config/config.py             # Settings (.env) + logging
├── tests/                           # pytest suite
├── requirements.txt
├── .env.example
└── README.md

---

Step-by-Step Execution Guide

The complete path from a fresh clone to a tested API, assuming the Question Bank Module already runs on PostgreSQL at port 8000.

Step 1 — Get the code and enter the folder

git clone <your-repo-url>
cd Validation-Module_AI-Viva-main

Step 2 — Create and activate a virtual environment

# macOS:                          # Windows PowerShell:
python3.12 -m venv .venv          # py -3.12 -m venv .venv
source .venv/bin/activate         # .venv\Scripts\Activate.ps1

You know it worked when (.venv) appears at the start of your prompt.

Step 3 — Install dependencies (a few minutes; PyTorch is big)

pip install -r requirements.txt

Step 4 — Create your settings file

cp .env.example .env              # Windows: copy .env.example .env

No database settings are needed — this module has no database.

Step 5 — Start the server on port 8001 (8000 is taken by the Question Bank Module)

uvicorn app.main:app --reload --port 8001

Leave this terminal open; the server stops if you close it (or press Ctrl+C).

Step 6 — Smoke-test in a browser Open http://127.0.0.1:8001/health — you should see "status": "healthy".

Step 7 — Test a real validation in Swagger Open http://127.0.0.1:8001/docs → POST /api/v1/validate → Try it out → Execute with the example body. The first call downloads the NLP models (one time, be patient); after that responses are fast.

Step 8 — Test the full pipeline with the Question Bank Module In a second terminal, log in to the Question Bank API, fetch a question set's answers (GET /questions/set/{id}/answers on port 8000), then send one question + answer_text + a made-up student answer to POST /api/v1/validate on port 8001. A relevant student answer should come back Valid.

---

Troubleshooting

• ModuleNotFoundError: requests / rapidfuzz — you have an old requirements.txt; the current one includes them. Run pip install -r requirements.txt again.
• First request hangs for minutes — it's downloading models (one-time). Watch the server terminal: you'll see "Loading HF AutoModel/AutoTokenizer...".
• Out of memory / laptop very slow — you're on the default 1B embedding model. Create .env from .env.example to use the small verified models.
• Port already in use — another app (e.g. the Question Bank Module) is on 8000. Run with --port 8001.
• zsh: command not found: uvicorn — the venv isn't activated; run the activate command first.