Callisto - A WebGPU based in-browser AI assistant

CHECK IT OUT AT callisto-lyart.vercel.app!

Callisto is a fully in-browser conversational AI assistant. It does NOT have a backend API.

Your browser MUST support WebGPU to run this AI assistant. Please figure out if your browser is compatible before proceeding. As of writing this, as long as you are on a Mac running the latest Google Chrome browser, you should be able to run this out of the box. On Linux machines, however, you may need to use a special flag to enable WebGPU on your Chrome. Also, right now, WebGPU support on Firefox is experimental.

Callisto will load the following 4 different AI models. It'll then use WebGPU and the ONNX runtime through the Transformers.js library to run them all inside your browser.

• Large Language Model (HuggingFaceTB/SmolLM2-1.7B-Instruct)
• Speech to text (onnx-community/whisper-base)
• Text to speech (onnx-community/Kokoro-82M-v1.0-ONNX)
• Voice detection model (onnx-community/silero-vad)

Running Locally

Just clone this repo and run the following commands:

• npm install
• npm run dev

---

Refined and added features over webml-community/conversational-webgpu

The following features were added:

• Ability to see the conversation history of the current call
• Ability to interrupt the assistant whenever it is talking (by pressing the orb)
• Lots and lots of styling updates!