WisprGemma

Local-first multilingual dictation powered by Gemma 4 E2B. Speak in any language, get polished text. Audio never leaves your device.

🏆 Winner of 2nd prize at the Build with Gemma hackathon (Kolkata): project writeup on Kaggle.

WisprGemma is a WisprFlow-style dictation tool that runs entirely in the browser on WebGPU. I built it for anyone who wants fast voice-to-text without sending audio to a cloud API: no subscription, no API key, works offline after a one-time model download (~3.5 GB, q4f16).

One Gemma 4 model handles the full pipeline in a single pass: speech recognition, cleanup, and rewriting. No Whisper, no backend.

Output modes

Pick a mode before you speak. Each one is a different prompt to the same model, not a separate pipeline.

Pick Custom style… in the dropdown to reveal a prompt box (500 characters max). Your instruction is saved locally and stays filled in the next time you open the panel; leave it empty to fall back to Clean dictation.

Key features

Quick start

Web app

npm run serve:web

Open http://localhost:3000 for the landing page, then Open the app (or go straight to /app.html). Click Load model, allow the microphone, hold Space and speak.

Requires Chrome or Edge 121+ with WebGPU enabled. First load downloads ~3.5 GB, then spends a few minutes uploading weights to the GPU.

Try it online — the web app is also deployed to GitHub Pages at https://arindam200.github.io/WisprGemma/ (a landing page with a link into the dictation tool), no install needed (same browser and download requirements apply). If you fork this repo, enable it once under Settings → Pages → Source: GitHub Actions and the deploy workflow publishes web/ on every push to main.

Chrome extension

1. Open chrome://extensions, enable Developer mode
2. Load unpacked → select the extension/ folder — a welcome tab opens automatically on first install
3. Click the WisprGemma icon (or press ⌘/Ctrl+Shift+Space), load the model, allow the microphone
4. If mic access fails in the side panel, follow the permission tab that opens automatically
5. Focus a text field on the active page and dictate. Reopen WisprGemma from its toolbar icon after switching to a different site so Chrome can grant access to that tab

Optional: local model server

npm run download-model

Downloads weights into web/models/ and serves them on port 8975. Leave it running while developing. Both the web app and extension auto-detect it.

How it works

Mic (MediaRecorder)
  → decode to 16 kHz mono Float32
  → worker.js (Transformers.js + Gemma 4 E2B on WebGPU)
  → streamed tokens back to UI
  → display / insert into page / log stats

Vanilla HTML/CSS/JS, no build step. A module Web Worker runs inference so the UI stays responsive while a 3.5 GB model loads.

Model: onnx-community/gemma-4-E2B-it-ONNX, q4f16, WebGPU via Transformers.js 4.2.0. Weights download from Hugging Face Hub on first load, then cache in the browser. Recordings stop at 30 seconds; automatic segmentation is not implemented yet.

The extension adds MV3-specific workarounds: vendored Transformers.js and ONNX Runtime WASM, plus a one-time mic permission page (Chrome cannot prompt inside the side panel).

Tech stack

Repository layout

wisprgemma/
├── web/                   Landing page (index.html), dictation app (app.html), stats dashboard
├── extension/             Chrome MV3 extension
├── .github/workflows/     GitHub Pages deploy workflow
├── package.json           Convenience scripts
├── PRIVACY.md             Privacy policy (linked from the Chrome Web Store listing)
└── serve.json             Static server config

See web/README.md and extension/README.md for package-specific notes.

🖥️ Desktop app (macOS, MVP)

System-wide dictation: hold a key, speak, and polished text lands at your cursor in any app — same local Gemma pipeline, plus optional Ollama cleanup and bring-your-own-key cloud engines (OpenAI / Groq / Gemini).

cd desktop && npm install && npm start

See desktop/README.md for setup (permissions, engines) and DESKTOP_PLAN.md for the architecture and roadmap.

Privacy

Audio is captured locally, processed locally, and never sent to a server. Stats and history live in browser storage on your machine only. See PRIVACY.md for the full policy.