Privoice

A private, uncensored voice agent. No refusal layer, no logging, no retention.

Talk to it and it talks back in about a second and a half, full duplex — cut it off mid-sentence and it stops. It answers what other assistants refuse, and it keeps no record that you asked.

git clone https://github.com/sidharth-n/privoice && cd privoice
uv sync
cp .env.example .env                        # add a Venice key for the hosted slots
LLM_BACKEND=venice uv run python voice_agent.py

Everything below is the measurement behind those claims. Every figure came off one machine, is labelled as warm or cold, and is reproducible with the scripts in this repo — nothing here is quoted from a vendor blog.

---

A real-time voice agent for Apple Silicon whose every slot — speech-to-text, LLM, text-to-speech — swaps by env var between on-device and Venice AI's hosted API. Speak to it, it speaks back, interrupt it and it stops.

mic ─► AEC ─► VAD ─► STT ─► LLM ─► TTS ─► speakers
                      │      │      │
                   local or venice, per slot

Venice serves the same Parakeet and Kokoro checkpoints this project already ran locally. That is what makes the comparison below controlled: for those two slots the model is held constant and only the substrate changes.

The finding

Apple M5 MacBook Air (32 GB), client in India, 2026-08-12, 5 samples per slot, medians. Method, controls and limits: docs/BENCHMARK.md.

Those are per-slot measurements on a fixed short prompt. A real conversation is slower: 34 live turns on the hybrid config measured 1,699 ms median / 2,757 ms p90 to first audio. The ranking holds, the magnitude does not. Every live turn logs its own breakdown, so this is measured rather than assumed.

Then the conversation number was fixed, not just measured

Instrumenting real turns showed the wait was not where the per-slot benchmark said it was, and almost none of it was fixable by making a model faster. After the work in issues/0002, 0009, 0010 and speculative dispatch — 20 turns through the real turn-taking loop now measure 1,112 ms median / 1,463 ms p90:

Nothing here is a faster model. The LLM is a hard floor — nine Venice models from 30B-A3B to 405B all return their first token in 850–1,100 ms — so the wins came from changing the workload instead: ask the model for a short opening sentence (time-to-first-audio scales at ~7.9 ms per character), synthesize later sentences underneath audio already playing, and start transcribing and generating during the 500 ms the voice-activity detector spends confirming you have stopped talking, which was previously dead time nobody was counting.

Method, controls and limits: docs/BENCHMARK.md. The after-figures come from say-synthesized speech driven through the real loop — cleaner than a room, no echo path, one voice — so treat them as a floor.

Three results, and the first two were not what I expected.

1. All-hosted is 2.2× slower than a laptop — but not because the GPUs are slow. Venice decodes 6.9× faster than the M5 (115 vs 16.6 tok/s). It loses because roughly 400 ms of fixed server-side latency attaches to every API request. Measured on a static GET /models that runs no inference: it survives connection reuse (TTFB still 406 ms with TLS already established) and needs no auth. A voice turn is three sequential calls, so all-hosted pays ~1.2 s of overhead before generating a single token.

2. Cold start reverses the answer. Warm, local wins. But Ollama evicts the model on a timer, so the first thing you say after a gap costs a measured ~7.4 s to first sentence. Hosted has no cold start. For anything that isn't a benchmark loop, hybrid is 5.7× faster on the turn users notice most.

3. Inside Venice, model choice matters more than hosting does. venice-uncensored reaches its first sentence in 990 ms; qwen-3-8-max decodes 2.5× faster and takes 4,702 ms, because prefill dominates. Choosing on decode rate or TTFT picks the wrong model.

Practical answer: keep the ears and mouth on the device, put the brain on Venice.

Why time to first sentence, not time to first token

TTS is driven per sentence, so nothing can be spoken until a complete sentence exists. TTFT — the number almost everyone publishes — is not a proxy for it, and here they differ by up to 4×. Every LLM figure in this repo is time to first sentence, measured through the same splitter the live agent uses.

An earlier revision of the upstream README claimed "sub-second". That was never measured and never met; see issues/0003.

What it actually does

• You speak. Silero VAD detects start/end of your utterance.
• It hears you cleanly. WebRTC AEC (via LiveKit) cancels its own voice from the mic, so it doesn't feed back even with built-in mic + speaker.
• STT transcribes — Parakeet TDT v3 locally (~63 ms warm), or hosted.
• The LLM streams a reply — a local uncensored 26B via Ollama, or Venice.
• Tokens are split into sentences and each sentence goes to TTS immediately, so the first audio plays while the LLM is still writing sentence two.
• Barge-in: start talking and it shuts up. An energy gate plus a sustain-frame check filters residual echo.
• Memory persists across runs in .voice_history.json (rolling 8 turns).

Quickstart

# 1. Install uv, then deps
curl -LsSf https://astral.sh/uv/install.sh | sh
uv python install 3.12       # kokoro-mlx requires 3.10–3.12
uv sync

# 2. For the Venice slots: key from venice.ai/settings/api
cp .env.example .env

# 3. For the local LLM slot
brew install ollama
OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve &
ollama pull 0xIbra/supergemma4-26b-uncensored-gguf-v2:Q4_K_M   # ~16 GB

Run it:

# hybrid — local ears and mouth, Venice brain (fastest measured)
LLM_BACKEND=venice uv run python voice_agent.py

# fully local, no network, no API key
uv run python voice_agent.py

# fully hosted — no 16 GB download, no Ollama
LLM_BACKEND=venice STT_ENGINE=venice TTS_ENGINE=venice uv run python voice_agent.py

No mic handy? scripts/smoke_pipeline.py runs the whole path from a synthesized utterance and prints per-stage latency.

Configuration

Room tuning (unchanged from the local build):

Stack

Parakeet was chosen on measured real speech, not reputation: 13.3% WER / 69 ms against Moonshine's 17.8% / 351 ms on the same clips.

Cost

Every number in docs/BENCHMARK.md cost under $0.05 of Venice credit in total. venice-uncensored is $0.000048 per turn at this prompt size; hosted Parakeet is $0.0001 per audio-second.

Critical gotchas

1. think: false is non-negotiable on the local model. SuperGemma4 ships thinking-mode ON; with it on the response field stays empty for 5–30 s while reasoning streams to a separate field, which no voice agent can tolerate. The Modelfile cannot disable it (ollama#14809).
2. Venice injects its own system prompt by default — measured at ~1,568 cached prompt tokens. This repo sets venice_parameters.include_venice_system_prompt: false, both because the agent ships its own persona and because leaving it on quietly changes what you are benchmarking.
3. AEC frames must be exactly 10 ms at 16 kHz. WebRTC APM is strict and fails silently on the wrong frame size.
4. Built-in MacBook mic+speaker geometry is hard for software AEC. Expect to tune for your room; headphones remove the problem.
5. History is a latency multiplier. Every retained turn lengthens prefill. HISTORY_MAX_TURNS=8 is the cap that keeps latency in the measured range.

Files

Why the context files are in the repo

CLAUDE.md, state.md, learning.md and issues/ are checked in deliberately. This project is built with coding agents doing most of the typing, and those files are what makes that work: the agent reads state.md to know where things stand, appends to learning.md when something breaks, and files an issue instead of derailing.

They are also the honest record. issues/0003 is where the sub-second claim died, and learning.md says why. If you want to know what this codebase does and where it falls short, those files will tell you faster than the source will.

Honest limits

• One machine, one network, one city, one day. Latency from a datacentre next to Venice's edge would look completely different.
• STT and TTS are like-for-like. The LLM comparison is not — a local 26B Q4 GGUF and venice-uncensored are different models on different hardware. It reflects the choice a builder faces, not a controlled swap, and no conclusion here should rest on it being one.
• Latency only. No quality evaluation of any kind was run.
• Single client, sequential requests. Says nothing about behaviour under load, where a hosted API's advantages are real and invisible here.

Why uncensored

Hardcoded "I cannot…" disclaimers and safety preambles ruin natural voice conversation. A friend on a phone call doesn't say "as an AI I cannot" — they just talk. That is the design goal, on both the local and hosted paths.

Use responsibly. You own anything it says; don't deploy it public-facing.

Credit

Extended from uncensored-local-voice, the fully on-device build, with hosted slots added so the two could be measured against each other.

Built by Sidharth N. Not affiliated with Venice AI.

MIT — see LICENSE. Built on open-source components (Parakeet TDT, Silero VAD, Kokoro, LiveKit RTC, Ollama, SuperGemma4); see their licenses.