🎙️ AI Interview Summarizer

An ultra-lightweight, real-time voice interview summarization app. Runs 100% client-side with neural AI inference via ONNX Runtime WebAssembly — achieving <30MB model size and <50ms inference latency.

🏆 Built for hackathon constraints: No server calls, offline-capable, real-time performance.

---

✨ Key Features

---

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                         CLIENT-SIDE ARCHITECTURE                            │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  ┌──────────────┐    ┌─────────────────┐    ┌─────────────────────────────┐│
│  │  MAIN THREAD │    │  AUDIO THREAD   │    │      INFERENCE WORKER       ││
│  │  (SvelteKit) │    │  (AudioWorklet) │    │    (ONNX/WASM + SIMD)       ││
│  └──────┬───────┘    └────────┬────────┘    └──────────────┬──────────────┘│
│         │                     │                            │               │
│         │  ┌──────────────────┴─────────────┐              │               │
│         │  │    VAD Processor               │              │               │
│         │  │  • RMS voice level detection   │              │               │
│         │  │  • Runs in audio render thread │              │               │
│         │  │  • No UI blocking              │              │               │
│         │  └────────────────────────────────┘              │               │
│         │                                                  │               │
│         ▼                                                  ▼               │
│  ┌─────────────────────────────────────────────────────────────────────┐  │
│  │                     INCREMENTAL SUMMARIZATION                        │  │
│  │  ┌─────────────────────────────────────────────────────────────┐    │  │
│  │  │  1. New sentence arrives                                     │    │  │
│  │  │  2. Embed ONLY new sentence (single forward pass)            │    │  │
│  │  │  3. Update running sum → O(1) centroid calculation           │    │  │
│  │  │  4. Cosine similarity ranking (cached embeddings, no model)  │    │  │
│  │  │  5. Return top-k sentences                                   │    │  │
│  │  └─────────────────────────────────────────────────────────────┘    │  │
│  │                                                                      │  │
│  │  Model: Xenova/all-MiniLM-L6-v2 (Quantized Int8)                    │  │
│  │  • 384-dimensional embeddings                                        │  │
│  │  • ~23MB ONNX model                                                  │  │
│  │  • WASM backend with SIMD acceleration                               │  │
│  └─────────────────────────────────────────────────────────────────────┘  │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Performance Optimizations

---

📊 Hackathon Compliance

*Web Speech API uses cloud STT. See Offline STT Options for true offline mode.

---

🚀 Quick Start

Prerequisites

• Node.js 18+
• Chrome/Edge (best Web Speech API support)

Installation

# Clone the repository
git clone https://github.com/KUNALSHAWW/AI_Interviewer_summarizer.git
cd AI_Interviewer_summarizer

# Install dependencies
npm install

# Start development server
npm run dev

First Run

1. Open http://localhost:5173 in Chrome/Edge
2. Wait for model download (~23MB, cached after first load)
3. Allow microphone access
4. Click "Start Recording" and speak
5. Watch real-time transcription and summaries!

---

🗂️ Project Structure

AI_Interviewer_summarizer/
├── src/
│   ├── lib/
│   │   ├── components/          # Svelte 5 UI components
│   │   │   ├── InferenceProvider.svelte   # ONNX worker orchestration
│   │   │   ├── TranscriptFeed.svelte      # Live transcript display
│   │   │   ├── SummarySidebar.svelte      # AI summary panel
│   │   │   └── ...
│   │   │
│   │   ├── workers/
│   │   │   ├── onnx-inference.worker.ts   # Incremental neural inference
│   │   │   └── vad-worklet.ts             # Off-thread VAD (AudioWorklet)
│   │   │
│   │   ├── voice/
│   │   │   └── engine.ts          # Speech recognition + TTS + VAD
│   │   │
│   │   ├── stores/                # Svelte reactive stores
│   │   ├── types/                 # TypeScript definitions
│   │   └── utils/                 # Performance utilities
│   │
│   └── routes/
│       └── +page.svelte           # Main application
│
├── scripts/
│   └── quantize_model.py          # ONNX model quantization script
│
├── static/                        # Static assets
├── CODE_REVIEW.md                 # Original technical review
├── REFACTORING_SUMMARY.md         # Applied fixes documentation
└── package.json

---

🧪 Testing

# Run unit tests
npm test

# Watch mode
npm run test:watch

# Coverage report
npm run test:coverage

---

⚙️ Configuration

Environment Variables (Optional)

# .env (for Supabase persistence)
PUBLIC_SUPABASE_URL=your_supabase_url
PUBLIC_SUPABASE_ANON_KEY=your_anon_key

Model Configuration

// src/lib/workers/onnx-inference.worker.ts
const MODEL_ID = 'Xenova/all-MiniLM-L6-v2';  // ~23MB, 384-dim

// Alternatives:
// 'Xenova/paraphrase-MiniLM-L3-v2'   // ~17MB (smaller, less accurate)

---

🔇 Offline STT Options

Web Speech API uses cloud servers. For true offline mode, choose:

⚠️ Cannot have Neural STT + Neural Summarization under 30MB.

---

🌐 Browser Support

---

📦 Build & Deploy

# Build for production
npm run build

# Preview production build
npm run preview

Deploy to Vercel

npx vercel

The app is configured for static deployment via @sveltejs/adapter-static.

---

🔧 Development

Key Commands

Keyboard Shortcuts

---

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing-feature)
5. Open a Pull Request

🐰 CodeRabbit is enabled for automatic PR reviews!

---

📜 License

MIT License - see LICENSE for details.

---

🙏 Acknowledgments

• Hugging Face — Model hosting
• Xenova/transformers.js — Browser ML runtime
• ONNX Runtime — WASM inference engine
• SvelteKit — UI framework

---

Built with ❤️ for real-time AI inference on the edge