Coming Soon: A More Approachable NeuralCompanion

[!IMPORTANT] In development - not released yet. The redesigned interface shown here is planned for release soon. We are still refining and testing it; these screenshots preview the upcoming release, not the version currently available to download. Release timing may change while testing continues.

We are actively improving the desktop interface so that getting started, finding settings, and spending time with your companion feel more natural. The upcoming native Qt Quick interface brings setup, conversation, personas, and visual tools into dedicated workspaces, with clearer navigation and guidance inside the app.

Development work focuses on:

• A more helpful first run: a welcome screen shows setup readiness and offers activities to try, with guided tutorials that open the relevant workspace and highlight its controls.
• Settings grouped by purpose: session setup, language models, speech, visual replies, and avatars have their own sections, with the active provider stack visible alongside them.
• A conversation view you can adjust: give chat the full workspace or keep the system console beside it. Portraits, text size, speech highlighting, and other display controls let you choose how the conversation looks.
• Companions that are easier to personalize: quick personas bring prompts, voices, portraits, and Visual Reply behavior together. The visual identity editor keeps appearance references and mood portraits in one place.
• More comfortable everyday navigation: dedicated workspaces, remembered navigation choices, theme presets, and a custom theme preview reduce the need to hunt through a long settings page.

This work is ongoing. The screenshots below show the development installation on September 30, 2026; details may differ from your installed version. Click any image to see it at full size.

A Conversation with Nixie

These are real replies from a short demo conversation in the running app, using the configured language model and speech pipeline. Here, Nixie helps write a tiny story about a robot running a midnight cafe on the Moon.

Chat with the console split: follow the conversation, speech activity, and runtime output side by side.

Chat without the console: give the conversation the full workspace. Toggle the split from the console header or the conversation's Controls menu.

Getting Started and Personalizing Your Companion

Community

The project is intended to grow through community feedback, addon development, and shared workflows. Join the setup/help Discord here:

Support

Support your friendly NeuralCompanion devs!

Watch the NeuralCompanion overview video

Neural Companion

Neural Companion is a local desktop AI companion for realtime chat, speech, avatars, visual replies, and addon-driven workflows. This release targets Windows, and a Linux version is available at Rakile/NeuralCompanion-Linux.

It is designed for users who want a configurable AI companion that can talk, listen, roleplay, use local or API models, drive avatars, and grow through addons.

Skip the feature tour and jump to Requirements and Install

What It Can Do

• Chat through local or API providers such as LM Studio, OpenAI, xAI/Grok, Claude, DeepSeek, Ollama, and addon providers.
• Speak through TTS backends such as Chatterbox, Gemini TTS Preview, PocketTTS, and addon backends.
• Drive avatars through MuseTalk, Scenic still-image packs, VSeeFace, VaM, or no-avatar mode.
• Use screen, webcam, clipboard, heart-rate, and visual-reply workflows.
• Save, reload, export, and replay chat sessions while keeping very large chat transcripts manageable through a windowed chat view.
• Use Continuity Memory summaries and a versioned Long-Term Memory archive with semantic retrieval, linked image recall, and optional manual image review.
• Import external companion continuity through the experimental Identity Relay workflow, with explicit connection and per-session enable controls.
• Run Multi Persona Roleplay Companion, Discord Voice Bridge and TinyMVP room conversations, tutorials, presets, hotkeys, and addon tools.
• Build visual stories through Audio Story Mode, which is functional but still under active construction.
• Use spellchecking in typed chat and optional dependency repair for supported features.

Current Highlights

• Identity Relay (experimental): import a source-native identity artifact, review and connect it explicitly, then decide from the Chat window when that continuity is active. The addon now includes the complete export protocol.
• Long-session memory: Continuity Memory summaries, versioned SQLite archives, semantic retrieval, image-linked recall, legacy cleanup, readable exports, and a windowed transcript for very large conversations.
• Shared conversations: Multi Persona Roleplay Companion and Discord Voice Bridge with TinyMVP local rooms, participant routing, moderator controls, buffered speech, and shared room context.
• Visual and avatar workflows: Visual Reply history, Scenic Avatar packs, MuseTalk preprocessing and playback, VSeeFace, VaM, Neural Face Presence, Companion Orb integration, and the in-progress Audio Story Mode.
• Provider flexibility: local LM Studio and Ollama models or API providers, with streaming speech, provider switching, and model-aware request handling.
• Optional companion applications: an Android remote and a separate Companion Orb build are available from the repository releases page.

For release history, see CHANGELOG.md.

Interface Screenshots and Page Guide

The detailed feature tour below shows the earlier interface. For a preview of the upcoming redesign, see Coming Soon. The screenshots below are clickable thumbnails. Click a thumbnail to open the full-size image.

1. Host Runtime and Visual Reply

This page controls the active runtime preset: avatar mode, input mode, streaming, and the selected visual-reply provider.

What to do on this page:

1. Select the avatar engine, input mode, input role, and streaming mode for the active preset.
2. Choose the visual-reply provider and set image size, workflow JSON, server URL, and cleanup behavior.
3. Use Save or Save As after changing runtime settings so the preset can be reused.

2. Hidden Sensory Feedback

This page enables the hidden PING/PONG loop that can feed screen, webcam, clipboard, orb target, heart-rate, and Spotify context into replies.

What to do on this page:

1. Enable the PING/PONG loop when the companion should react to hidden sensory context.
2. Choose which sources are included, such as Screen, Clipboard, Webcam, Spotify Sense, or Companion Orb Target.
3. Tune refresh seconds and retained PONGs so the model receives fresh context without flooding the chat.

3. Conversation Flow and Memory

This page manages proactive replies, context-window size, continuity summaries, and long-term archive retrieval.

What to do on this page:

1. Use proactive replies and delay settings to control when the assistant may speak first.
2. Adjust context-window and overflow settings to control how much recent chat is sent to the model.
3. Enable continuity summary or archive retrieval when longer sessions need memory beyond the visible chat.

4. Color Theme Presets

This page lets you switch the shell color palette between light, gray, forest, ocean, midnight, and other preview themes.

What to do on this page:

1. Pick a theme preset to quickly change the preview shell appearance.
2. Use lighter themes for readability and darker themes for long coding or testing sessions.
3. Treat these as visual presets only; they do not change chat, audio, or model behavior.

5. Story Visual Replies

This page controls visual-reply behavior for story mode, including image count, style selection, continuity, and master style anchors.

What to do on this page:

1. Enable Story Mode when visual replies should follow a story or roleplay flow.
2. Choose the active visual style, continuity level, and maximum number of generated pictures.
3. Use the master style anchor to keep character and scene style more consistent over time.

6. RAG Context

This page builds a local retrieval database from selected text or PDF files and injects matching chunks into chat requests.

What to do on this page:

1. Add source files, then build or reload the local RAG database.
2. Set max chunks, minimum score, and max context characters before enabling RAG during chat.
3. Use Test Retrieval to verify that the database returns useful chunks before relying on it in conversation.

7. Chat Window and System Console

This page shows live chat output, console events, buffer status, audio input/output selection, and runtime action buttons.

What to do on this page:

1. Watch the buffer race area for streaming, preview, and playback timing problems.
2. Use the console to debug sensory input, hidden PING/PONG events, TTS chunks, and generated image requests.
3. Use runtime buttons such as Initialize System, Terminate, Regenerate, Retry Input, or Skip Speech during testing.

8. Spotify Sense Addon

This addon page connects Spotify, reads current-track context, controls playback, and supports music-aware companion behavior.

What to do on this page:

1. Enter the Spotify Client ID and redirect URI, then log in to connect the account.
2. Enable the Spotify features you want, such as music awareness, playback control, ducking, and queue changes.
3. Set default and duck volumes so music lowers while Neural Companion speaks and restores afterward.

9. Addon Manager

This page enables or disables addon categories and shows which addons are loaded, initialized, or active on next launch.

What to do on this page:

1. Enable or disable parent addon categories such as Audio, Avatar, and other feature groups.
2. Review each addon card to see version, description, and whether it is initialized or active on next launch.
3. Restart Neural Companion after changing addon loading state so the new configuration is applied cleanly.

10. Pipeline Chunking and Performance

This page tunes system-wide response chunking for standard, MuseTalk, and streaming playback pipelines.

What to do on this page:

1. Tune target and maximum character counts for normal replies and MuseTalk playback.
2. Adjust streaming flush values when speech starts too late, cuts too early, or feels uneven.
3. Save known-good performance profiles so different hardware or TTS backends can be switched quickly.

11. Dry Run Performance Profiler

This page runs safe reply-generation tests and recommends machine-specific startup and chunking values without changing the live pipeline.

What to do on this page:

1. Arm Dry Run to collect reply samples without disturbing the live pipeline.
2. Let it generate enough samples to measure safe startup and chunking behavior on the current machine.
3. Apply or save the recommended profile only after the dry run results look stable.

12. Tutorial Browser

This page lists built-in walkthroughs loaded from JSON files and starts guided setup or feature tutorials.

What to do on this page:

1. Select a tutorial category such as First Run, Installer, Avatars, Visual Replies, or Audio Story Mode.
2. Read the summary and number of steps before starting the walkthrough.
3. Press Start Tutorial to launch the guided sequence inside the application.

13. VSeeFace Avatar Dynamics

This page adjusts VSeeFace body and face movement values such as eye activity, breathing speed, shoulder lift, and body sway.

What to do on this page:

1. Adjust eye activity, breathing, shoulder lift, and body sway to make the avatar feel more alive.
2. Lower movement values if the avatar looks unstable, jittery, or too animated for the current voice.
3. Use Hide NC Interface when VSeeFace should remain as the main visible avatar view.

14. Audio Story Mode

This page imports audiobook or story audio, transcribes it through Whisper, and generates visual story images from the transcript.

What to do on this page:

1. Import an audiobook or story audio file, then choose playback, precision, and transcription settings.
2. Set image timing, continuity, style prompts, and provider-specific image settings before generation.
3. Use story analysis when the transcript needs scene-aware prompts instead of simple fixed-second image prompts.

15. MuseTalk Avatar Preprocess

This page prepares source video, image, or frame folders into MuseTalk avatar packs with optional emotion-tag mapping.

What to do on this page:

1. Choose or create an avatar pack before selecting a source video, still image, or PNG frame folder.
2. Set the Avatar ID and optional emotion tags so runtime can map expressions to prepared variants.
3. Run Preprocess Avatar and create the startup cache for faster loading later.

16. AI Presence Mode

This page configures the full-screen or floating AI presence overlay, including breathing orb style, mood colors, particles, and audio sync.

What to do on this page:

1. Choose full-screen or floating display mode and the active presence style.
2. Enable overlay behavior such as always-on-top, click-through, transparent background, and space-to-exit.
3. Tune opacity, pulse, reactivity, particle density, glow, blur, and mood colors for the desired presence effect.

17. Orb Targeting During Preprocess

This page shows the Companion Orb overlay sitting on top of a work area, useful for confirming placement, click-through behavior, and target focus.

What to do on this page:

1. Use this view to confirm that the orb can stay visible above other Neural Companion pages.
2. Check that click-through and target focus do not block normal UI interaction when the orb is placed over the app.
3. Reset or move the orb if it covers important buttons, form fields, or scroll areas.

18. Neural Face Presence

This page controls the wireframe face overlay, including style, lip sync, blink, emotion reaction, glow, and reduced-animation options.

What to do on this page:

1. Enable Neural Face Presence and choose the face style used by the overlay.
2. Toggle wire lines, node glow, blink, eye movement, lip sync, and emotion reaction to shape the final look.
3. Use size, opacity, animation, and lip-sync sliders to balance visibility against distraction.

19. Companion Orb Overlay

This page controls the desktop Companion Orb: visibility, position, click-through mode, movement, effects, colors, and voice-reactive animation.

What to do on this page:

1. Enable the Companion Orb Overlay and choose display mode, style, reply style, and position.
2. Set window behavior such as always-on-top, click-through, remember position, and external animation runtime.
3. Tune movement, effects, state colors, opacity, frame rate, and voice-reactive animations until the orb feels smooth.

20. Multi Persona Story Play Mode

This page runs the Multi Persona Roleplay Companion play view, with story feed, scene state, cast, events, choices, director tools, and player input.

What to do on this page:

1. Press Start Play to begin or resume the isolated story runtime.
2. Use the story feed and scene-state panels to track location, mood, objective, active speaker, cast, events, and choices.
3. Write player actions in the input box or use Advance, Summarize, and Repair to steer or maintain the story.

21. Scenic Pack Editor

This page creates and edits Scenic Avatar packs by assigning images to tags that can be used as still-image avatar states.

What to do on this page:

1. Create or select a Scenic Pack before adding images.
2. Tag each image with states such as neutral, happy, or thinking so runtime can choose the correct visual.
3. Use Show In MuseTalk Preview to verify how the selected scenic image appears in the avatar preview window.

22. Spotify Sense Current Track

This page shows Spotify Sense while connected, including the current track, song-change monitor, ducking, queue options, and playback controls.

What to do on this page:

1. Use Login / Connect and Refresh Devices when Spotify control or device detection needs to be refreshed.
2. Enable music awareness, ducking, queue control, and song-change comments according to how much control the companion should have.
3. Use Current Track, Play / Pause, Next, and Volume controls to test Spotify integration directly from Neural Companion.

23. VaM Avatar Bridge

This page configures the Virt-A-Mate bridge path, target atom, plugin storables, VMC connection, audio bridge, and launch buttons.

What to do on this page:

1. Set the VaM root and bridge path so Neural Companion can find the VaM plugin bridge files.
2. Configure the target atom, storable ID, VMC host, and VMC port before enabling VMC or file bridge features.
3. Use Start VaM Desktop or Start VaM VR after the detected setup values look correct.

24. Hotkey Bindings

This page manages global and UI hotkeys for push-to-talk, regenerate, retry input, pause speech, skip speech, replay, reset memory, and launch actions.

What to do on this page:

1. Select an action from the left list to inspect or change its current binding.
2. Use Record Binding and Apply Binding to assign a new key combination.
3. Use Reset To Default or Reset All Defaults when bindings become confusing or conflict with other software.

25. Chat Replay

This page replays saved or loaded chat sessions through the active TTS and avatar pipeline without generating fresh LLM responses.

What to do on this page:

1. Load a chat context or quick-load the current session before starting replay.
2. Select a message and use Previous, Next, or Start Playing From Selected to replay from the right point.
3. Choose replay voices if the replay should use specific TTS voice references instead of the current live voice.

26. Visual Reply History

This page displays generated visual replies, lets you browse stored images, load story images, caption images, reuse image style, and clear old outputs.

What to do on this page:

1. Use Previous and Next to browse the generated visual-reply history.
2. Use Load Image, Load Current Story Image, or Use Current Image Style when a generated image should be inspected or reused.
3. Use Caption, Delete Image, Clear, or Delete All to manage visual outputs and storage size.

Requirements

Neural Companion currently requires:

• Windows
• Python 3.11
• FFmpeg, either installed on PATH or provided by the bundled installer tools
• A local or API chat provider
• An NVIDIA CUDA GPU for MuseTalk avatar generation and playback

Python 3.11 is required. Python 3.12+ is not currently supported by the full runtime stack, and older Python versions may fail during dependency installation.

Useful external tools:

• LM Studio for local LLMs
• VSeeFace for VRM-style avatar output
• VaM plus the Neural Companion bridge/plugin for VaM output

Install

For most users, use the graphical installer:

INSTALL_NEURAL_COMPANION.bat

The installer should be run with Python 3.11 available on the system. If Python 3.11 is not your default Python, choose or provide the Python 3.11 executable when installing.

For the detailed public install guide, see:

• docs/install.md
• docs/manual/installation.md

Advanced command-line install options are documented in the manual. They are intended for users who already know which runtime components they want to install.

Run

Start Neural Companion with:

run_neural_companion.bat

This is the recommended launch method because it uses the installed Neural Companion environment.

If you need to run the app manually from the project folder, use the installed environment explicitly:

.venv\Scripts\python.exe qt_app.py

Or activate the environment first:

.venv\Scripts\activate.bat
python qt_app.py

First Run

The simplest first run is to follow the tutorial displayed at first launch.

If you want to start manually:

1. Start LM Studio and load a chat model.
2. Start Neural Companion.
3. Select LM Studio as Chat Provider.
4. Select None as Avatar Engine.
5. Select a TTS backend.
6. Press Initialize System.
7. Use push-to-talk or typed input to verify chat and speech.

Once that works, enable MuseTalk, VSeeFace, VaM, visual replies, or sensory addons one at a time.

Voices

The public repo ships with two voice samples.

If you want Chatterbox or another backend to clone a reference voice, place your own .wav files under:

voices/

Only use voice files you have the right to use.

Avatar Packs

MuseTalk avatar packs belong in:

avatar_packs/<pack_id>/

Large avatar packs and frame caches are intentionally not stored in the main repository. Demo packs live in the separate NeuralCompanion-AvatarPacks repository.

Useful docs:

• docs/avatar_packs.md
• docs/release_asset_policy.md

Addons

Most runtime capabilities are implemented as addons under addons/.

Useful docs:

• docs/addon_quickstart.md
• docs/chat_provider_addons.md
• docs/vision_source_addons.md
• docs/visual_reply_addons.md
• docs/addon_state_and_presets.md

Discord Voice Bridge

Discord Voice Bridge runs one or more isolated voice companions through the selected Neural Companion STT, chat, and TTS providers. Its room context and audio playback remain separate from the normal Chat tab.

Two bridge modes are available:

• HTTP / Discord: connect NC-controlled bots to a real Discord voice channel through the bundled Node bridge.
• TinyMVP local room: test multi-participant conversations, routing, buffering, moderator behavior, and local microphone input without connecting to Discord.

If the addon is disabled in the Addons tab, enable it there and restart NC once so it can initialize. Then choose a bridge mode, configure the Discord channel or TinyMVP room, add any bot instances and personas, save the settings, and press Start Bridge. Enable Start bridge when NC launches only when you want the configured bridge to start automatically with NC. The Bots and Moderator tabs provide multi-bot voices, speaker routing, floor control, and dead-air recovery.

Keep Discord bot tokens private and prefer environment variables for normal use. See the complete Discord Voice Bridge guide for Discord permissions, Node dependency setup, TinyMVP instructions, multi-bot configuration, privacy guidance, and troubleshooting.

Identity Relay

Identity Relay can import continuity exported from an external LLM conversation.

1. Open the Identity Relay addon and use View Export Protocol or Copy Export Protocol.
2. Give the complete protocol to the external LLM session containing the identity or continuity you want to preserve.
3. Import the complete NC_IDENTITY_EXPORT JSON response without rewriting it, either by pasting it or selecting the saved file.
4. Connect the imported identity in the Persona tab, then enable Identity Relay in the Chat window when you want that continuity active.

The export reflects only what the source LLM can honestly derive from the conversation and memory available to it. A richer source history can therefore produce a richer artifact.

More Documentation

See the Neural Companion Manual for installation, first-run, avatar, TTS, PocketTTS, MuseTalk, addon, and troubleshooting guidance.

For development and repository details, see:

• CONTRIBUTING.md
• docs/release_checklist.md
• docs/third_party_and_assets.md
• docs/troubleshooting.md
• docs/known_limitations.md

Optional Companion Applications

• Companion Orb
• Android Remote
• View all downloads

Licensing

Neural Companion is released under the MIT License. See LICENSE.

Bundled third-party components may carry their own licenses. MuseTalk is included under its upstream MIT license in MuseTalk/LICENSE.

You are responsible for complying with the terms of any external model, provider, voice, avatar, or generated asset you use with the app.

Current Limitations

• This release package targets Windows. Linux users can use Rakile/NeuralCompanion-Linux.
• MuseTalk requires an NVIDIA CUDA GPU.
• Some integrations require external applications or plugins.
• Public demo assets are intentionally not bundled in the main repo.