Realtime Captions

This is an example of how we can use Transformers.js and WebGPU to run Whisper (whisper-base) directly in the browser to generate realtime captions.

The RealtimeCaptions implementation was heavily influenced by the WebGPU Whisper example by Xenova but comes with some improvements so no history is lost while still being as performant as possible.

This repository contains a TypeScript Class that could be used with any framework as well as a React Hook abstraction used in the demo application.