Demo Overview

This Space demonstrates an Automatic Speech Recognition (ASR) model that combines:

• Granite Speech 5.0 470M encoder for audio feature extraction
• Qwen3.5-2B decoder with LoRA adapters for text generation

Features

• 🎙️ Record from microphone or upload audio files
• ⏱️ Word-level timestamps via forced alignment
• 🗣️ Speaker diarization to label who said what
• 🎯 English transcription with punctuation and casing

Model Architecture

The model bridges audio and text with a trained projector and LoRA adapters:

1. Audio Encoder: Granite Speech 5.0 470M TurboCTC encoder (frozen)
2. Projection Layer: 2-layer MLP mapping audio features into the decoder's embedding space (~12.6M params)
3. Text Decoder: Qwen3.5-2B (frozen) with rank-64 LoRA adapters (~67M params), trained jointly with the projector

Usage

1. Upload an audio file (WAV, MP3, etc.) or record directly using your microphone
2. Click "Transcribe" to convert speech to text
3. The transcription will appear in the output box
4. Optionally tick Word Timestamps or Speaker Diarization. If you know how many people are speaking, set Number of Speakers: it caps how many speakers are kept

Limitations

• Plain transcription works best on clips up to about 19 seconds (the training length); with timestamps or diarization on, longer audio is transcribed in chunks automatically
• Requests run on a GPU server on RunPod; while it is starting up or offline, requests fail with a "try again" message
• Optimized for English language
• Best performance with clear speech and minimal background noise

Links

• 📦 Model on Hugging Face
• 💻 GitHub Repository
• 📄 Technical Details

Citation

If you use this model in your research, please cite:

@software{kroman2024tinyaudio,
  author = {Kroman, Alex},
  title = {Tiny Audio: A minimal, hackable speech recognition codebase},
  year = {2024},
  publisher = {GitHub},
  url = {https://github.com/alexkroman/tiny-audio}
}