Nemotron Diarization

Live Mic and Multilingual Live Mic sessions automatically stop after 30 seconds. Audio File accepts recordings up to 2 minutes long; trim longer recordings before uploading. The relay enforces audio-duration limits and uses a separate /api/diarization/file/stream route for files.

The Multilingual Live Mic tab uses Nemotron 3.5 multilingual streaming ASR with Nemotron-3-Diarization. It shares the microphone controls, speaker activity lanes, and transcript display with Live Mic, while using the separate /api/diarization/multilingual/stream relay. Language is detected automatically by the deployed model. Start the microphone and wait for Live before speaking. The original Live Mic, Audio File, and conversation experiences continue to use the English multitalker function.

Set NVCF_MULTILINGUAL_FUNCTION_ID and NVCF_MULTILINGUAL_FUNCTION_VERSION_ID as Space variables to select the multilingual deployment (see .env.example). Both functions use the existing NVCF_API_KEY Space secret. The multilingual tab stays disabled when its function is unconfigured; it never falls back to the English model. Both microphone routes share the Space's total session limit. Function IDs and credentials stay server-side.

This CPU-only Docker Space supports live microphone diarization, a Conversation Lab that turns an English Wikipedia topic into an overlapping two-to-eight-speaker conversation, and a Scenario Lab for professional four-, six-, or eight-participant stress tests. The Eight Voices tab offers five bundled deterministic scenarios plus a profile-backed Nemotron launch stand-up. The launch script is deterministic, while its audio is prepared at runtime: the saved Jensen and Francesco profiles use Magpie zero-shot streaming TTS and the other six actors use xAI streaming TTS. Before playback, clean lines from the same generated cast are sent silently to the same diarization stream. The audible overlapping scenario then continues with its session-local speaker context intact. This is context-assisted priming, not persistent voice enrollment. All external credentials remain in the Space runtime and are never sent to the browser.

The Audio File tab accepts WAV, MPEG/MP3/MP2, M4A, OGG, WebM, and FLAC files. Audio is decoded locally whenever the browser supports its codec; MPEG files use a bounded in-Space FFmpeg fallback when browser decoding is unavailable. The user can inspect the waveform, drag-select a range, and preview it before diarization. Clicking inside a selection sets a local preview-start marker; Preview plays from that marker to the selected end without changing the streamed range. Drag either edge to resize the range, drag a new range to replace it, or click outside to clear the selection. Only the chosen range is downmixed and resampled to mono 16 kHz PCM, then streamed to the diarization relay in real time. This view exposes anonymous speaker lanes, timestamps, and the live speaker-attributed ASR transcript without scripted role mapping. The selected PCM is sent exactly from its first sample through its last, with no hidden primer, pre-roll, synthetic noise or tail padding.

Pause / Resume suspends local playback and audio transmission without ending or reopening the model session. No silence is injected; WebSocket keep-alives are sent while paused. Existing relay and upstream session deadlines still apply, and a paused session continues to occupy a concurrency slot. Clear view clears only the anonymous activity and transcript panes. Speaker identities, the model session, and the selected audio range are preserved; later cumulative results are filtered so cleared history does not reappear. The display clock, activity positions, progress cursor, and transcript timestamps restart at zero at each clear boundary; the underlying audio and model timeline are unchanged. Because results have segment-level rather than word-level timestamps, a transcript segment already in progress at Clear is hidden until a new segment starts, while its activity after the clear boundary remains visible.

Configure NVCF_API_KEY as a Space Secret. Configure NVCF_GRPC_ENDPOINT, NVCF_FUNCTION_ID, NVCF_FUNCTION_VERSION_ID, MAX_CONCURRENT_SESSIONS, MAX_STREAM_SECONDS, and PRIMED_SESSION_WAIT_SECONDS and PRIMER_PROCESSING_TIMEOUT_SECONDS as Space Variables using the values in .env.example.

For Conversation Lab, configure CONVERSATION_LLM_API_KEY and XAI_API_KEY as Space Secrets. Configure CONVERSATION_LLM_BASE_URL, CONVERSATION_LLM_MODEL, XAI_TTS_URL, XAI_TTS_LANGUAGE, and optionally XAI_TTS_VOICES as Space Variables. Preparation limits and the prepared-audio TTL are also documented in .env.example.

Scenario Lab offers incident-response, contact-center escalation, and podcast-roundtable presets. It uses the same prepared-audio lifecycle as Conversation Lab, mixes all participants into one PCM feed, and consumes one diarization session per browser regardless of participant count.

The five gallery scenarios do not invoke the conversation LLM or TTS service at runtime; their shared hidden primer and audible stories are bundled assets. The Nemotron launch scenario resolves saved voice profiles named Jensen and Francesco when the user prepares it and returns a clear error if either is missing. Primer transcript and timing are suppressed by the relay; public timestamps are rebased to audible playback. Role names are scripted application logic, while speaker lanes, timestamps, and transcript text come from live model responses. Regenerate the bundled assets by running PYTHONPATH=. python tests/generate_mission_asset.py from this directory with XAI_API_KEY available in the parent deployment .env.

Keep the Space private for the initial deployment. The NVCF service supports four isolated streams per H100 pod. For ten total users, configure the NVCF deployment for four requests per instance and up to three instances, then set the relay's MAX_CONCURRENT_SESSIONS variable to 10.

Backend container source

Build and deploy your own ASR + diarization backend using these standalone Docker build contexts:

• English multitalker backend
• Multilingual backend

Both containers expose streaming gRPC and can run on a compatible NVIDIA GPU host, a VM, or Kubernetes. Each directory also includes NVCF deployment instructions and a client supporting direct local connections or authenticated NVCF requests. Supply your own model access credentials and test audio.

These directories are reference source. They are excluded from this Space's Docker build context; the running Space continues to call its configured NVCF backends. See each directory's dependency notes for upstream components and model terms.

Standalone Gradio app

The gradio-app directory contains a separate, single-user multilingual microphone and file-upload demo that runs ASR and diarization directly on your GPU. Docker and native Linux setup instructions are included.

This is reference source only: gradio-app/ is excluded from this Space's Docker build context. Its app, dependencies, and Dockerfile are not used by the live demo.

GTC Berlin multilingual preset

The Conversations tab includes GTC Berlin · Five languages, replacing Bioluminescence. Eight fictional attendees (two German, two English, two French, one Italian, and one Spanish speaker) meet over conference coffee, share what they hope to discover, and take turns with clear pauses. The cast and dialogue are fictional; they do not announce conference sessions or products. Audio is generated once and bundled, so playback does not invoke an LLM or TTS service. This preset uses the multilingual NVCF function; the other conversation presets and newly generated English conversations continue to use the English function. Actor names are scripted; speaker labels and transcripts are inferred live. No hidden speaker primer is used in this preset.

To regenerate only this asset, run from the Space directory with an environment file containing XAI_API_KEY plus MAGPIE_PRESET_TTS_API_KEY (or NVCF_API_KEY) for the configured Magpie preset service:

PYTHONPATH=. python scripts/generate_gtc_berlin.py --env-file /path/to/credentials.env

The generator uses eight distinct synthetic voices, a silent lead-in, clean first introductions, and at least 800 ms of silence between every turn, with no overlapping speech. It caches individual clips locally for retries.

The GTC Berlin cast mixes two Magpie voices with six xAI voices: Diego uses Magpie-Multilingual.ES-US.Diego.Calm with Spanish text and es-US synthesis, Felix uses Magpie-Multilingual.DE-DE.Jason, and Julien uses the male xAI voice atlas. Camille, the first French speaker, uses the female xAI voice liora. Magpie synthesis is performed only when regenerating the bundled recording; preset playback still uses the same multilingual ASR/diarization function. No audio pitch shifting or scripted speaker-label correction is applied.

Newly generated English conversations use a consistent subset of the fixed cast for 2–8 speakers: Magpie Mia, Jason, Louise and Long, followed by xAI Eve, Leo, Rex and Zagan. All synthesis is English, without Jensen or cloned profiles. Timing uses actual clip lengths and up to 500 ms overlap between successive turns. Short lines reduce overlap; the same speaker cannot overlap themselves, and at most two voices overlap. The fixed cast does not guarantee eight detected identities.

Transcript and speaker activity remain visible together in a horizontal split. The transcript follows new speech when at the bottom; users can scroll back and use Jump to latest to resume following. Public actor cards use character names, without TTS identifiers.