Audire

Audire is a model-independent, local speech recognition engine written in Rust. It provides one stable API for transcription models without making the project an implementation-specific wrapper around Whisper, Qwen, or another model.

Direction

• Pure Rust inference on CPU, CUDA, and Metal where the backend permits it.
• A small CLI and an OpenAI-compatible HTTP service.
• Offline files, long audio, and streaming through the same result types.
• Reproducible CER/WER, latency, memory, and real-time-factor benchmarks.
• Model selection based on measured quality rather than permanent defaults.
• Mandarin, English, and Chinese-English code-switching are mandatory for the active baseline and its eventual replacement.

Planned Backends

The repository now includes a native Candle Whisper baseline behind the same backend contract intended for Qwen3-ASR and later models. See Model selection for the implementation order and selection criteria.

Qwen3-ASR-0.6B is available as a quality candidate. Its Rust backend covers official language normalization, Chinese-English mixed-language output parsing, configuration validation, audio placeholder sizing, native log-Mel extraction, the complete Candle audio tower, Qwen3 decoding, audio embedding injection, and KV-cached generation through the existing AsrBackend contract.

Whisper baseline

The default command downloads openai/whisper-tiny into the standard Hugging Face cache. Audio decoding, resampling, feature extraction, and inference all run in Rust.

cargo run --release -- transcribe speech.wav --language zh
cargo run --release -- transcribe speech.wav --timestamps --json
cargo run --release --features cuda -- transcribe speech.wav \
  --backend qwen3-asr --device cuda --language zh

For offline use, place config.json, tokenizer.json, and model.safetensors in one directory:

cargo run --release -- transcribe speech.wav --model-dir ./models/whisper-tiny

WAV input is mixed to mono and resampled to 16 kHz. Build with --features cuda, --features metal, or --features mkl when the corresponding Candle backend is available, then select it with --device (except MKL, which accelerates CPU execution).

Quality evaluation

audire evaluate scores a labeled JSONL manifest with character error rate (CER) for Chinese, word error rate (WER) for English, and mixed error rate (MER) for Chinese-English code-switching. Audio paths are resolved relative to the manifest. Each non-empty line has this shape:

{"id":"zh-001","audio":"audio/zh-001.wav","reference":"你好，世界。","language":"zh","tags":["clean"]}
{"id":"mix-001","audio":"audio/mix-001.wav","reference":"请 review 这个 PR","tags":["code-switch"]}

The optional language field is the expected ISO-style language label and is used to score automatic language detection. Add force_language when a case should bypass detection and force a backend language; prompt and tags are also optional. Text normalization is Unicode NFKC plus lowercase. CER removes whitespace and punctuation, WER uses alphanumeric word runs, and MER counts each Han character and each contiguous Latin-alphanumeric word as one unit.

cargo run --release -- evaluate ./evaluation/manifest.jsonl \
  --backend whisper --device cpu --json > whisper-report.json
cargo run --release --features cuda -- evaluate ./evaluation/manifest.jsonl \
  --backend qwen3-asr --device cuda --json > qwen3-report.json

The JSON report includes corpus-level micro rates, mean per-sample macro rates, automatic language-detection accuracy, RTF, and every reference/hypothesis pair. Keep private evaluation audio and manifests outside the repository.

HTTP service

Build with http-api to load one backend at startup and expose a model-resident transcription service. Like the serving design in gpt-sovits-rs, inference is serialized behind a bounded-wait queue because the backend and its CUDA Graph state are mutable.

cargo run --release --features "cuda,http-api" -- serve \
  --backend qwen3-asr --device cuda --cuda-graph \
  --model-dir ./models/Qwen3-ASR-0.6B

The default address is 127.0.0.1:8080. Check readiness and the resident model:

curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/status
curl http://127.0.0.1:8080/v1/models

Send an OpenAI-style multipart transcription request. The model value must match the ID returned by /v1/models.

curl http://127.0.0.1:8080/v1/audio/transcriptions \
  -F file=@speech.wav \
  -F model=qwen3-asr-0.6b \
  -F language=zh \
  -F response_format=json

The first service version accepts WAV uploads and json or text responses. Unsupported streaming, timestamp, temperature, format, and model requests return structured OpenAI-style errors instead of being silently ignored. See HTTP API for endpoint details and operational limits.

Profiling and benchmarks

The benchmark command keeps one model resident, performs configurable warm-up iterations, and reports model loading, WAV decode, synchronized inference, end-to-end latency, RTF, generated tokens, and backend-specific stage timings. Use --json for regression tooling.

cargo run --release --features cuda -- benchmark speech.wav \
  --model-dir ./models/Qwen3-ASR-0.6B \
  --device cuda --language zh --warmup 1 --iterations 10

For a resident service or repeated fixed-size chunks, CUDA Graph capture is available as an opt-in path. Use two warm-up iterations: the first warms the decoder and its parameter cache, the second captures, and measured iterations then replay. Audio encoder graphs are cached by frame count and decoder graphs by absolute token offset.

target/release/audire benchmark speech.wav \
  --model-dir ./models/Qwen3-ASR-0.6B \
  --device cuda --language zh --cuda-graph \
  --cuda-graph-max-tokens 64 --warmup 2 --iterations 10

Decoder graphs retain substantial CUDA graph state. The default limit captures at most 64 generated-token offsets; later tokens use the eager path. Increasing the limit trades memory and capture time for replay coverage. A larger KV cache allocation invalidates decoder graphs whose captured buffer addresses are no longer valid. Graph mode is therefore intended for serialized, model-resident workers, matching the mutable AsrBackend execution contract.

The Qwen3 decoder combines the Q, K, and V projection weights at model load so single-token decoding launches one projection GEMV per layer instead of three. The projected tensor is split into views before the unchanged GQA computation.

Qwen3-ASR currently exposes feature_extraction_cpu, feature_upload, audio_encoder_gpu, prompt_prepare, decoder_prefill_gpu, decoder_autoregressive, and output_decode_cpu. CUDA is explicitly synchronized at stage boundaries so asynchronous kernel execution is charged to the stage that launched it. decoder_autoregressive includes GPU decoding, per-token synchronization, and transfer of the selected token ID to the host.

For kernel-level analysis on systems with Nsight Systems installed:

nsys profile --trace=cuda,osrt --sample=none --cpuctxsw=none \
  --output=/tmp/audire-qwen \
  target/release/audire benchmark speech.wav \
  --model-dir ./models/Qwen3-ASR-0.6B --device cuda \
  --warmup 0 --iterations 1
nsys stats --report cuda_gpu_kern_sum,cuda_api_sum /tmp/audire-qwen.nsys-rep

For a synchronized decoder breakdown without NVTX, set AUDIRE_DECODER_BREAKDOWN=1. Benchmark output then adds decoder_layers_gpu and lm_head_gpu. This mode synchronizes every generated token and is diagnostic only; do not compare its total latency with normal or CUDA Graph steady-state results.

Development

cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo test --locked
cargo test --locked --features http-api
cargo run -- models
cargo run --release -- transcribe --help

Model weights, evaluation datasets, and generated audio are intentionally not stored in the repository.