Sal7one/Hearth

Offline-first live captions, translation and conversations on Android., STT, TTS, OCR, ONNX custom Audio and NDK pipelines

C++

9

140 commits

updated Oct 18, 2026

See the code

README

Hearth

Hearth: English and Arabic speech bubbles

Read speech. Translate your screen. Talk across languages.

An Android app for live captions, translation and conversations. Run models on this device, use your own cloud keys, or mix local and cloud stages. Translation and read-aloud are optional, with their own settings. Built to make speech and languages more accessible, including for deaf and hard-of-hearing users.

Download · Screenshots · Quick start · Models · Build

Download

Hearth 0.23.10 — release notes and SHA256SUMS

Choose Universal if you do not know your phone or TV's CPU.

CPUCloud + localOffline only
Universal · recommendedDownload APKDownload APK
ARM64Download APKDownload APK
ARMv7 · 32-bitDownload APKDownload APK
x86_64Download APKDownload APK
x86 · 32-bitDownload APKDownload APK

Android 9+; device-audio capture requires Android 10+. Cloud + local includes optional provider connections and model downloads. Offline only has no network permission and imports models from files. Model weights are separate from the APK.

These are test-signed QA builds. Both flavors replace each other. Update without uninstalling to retain models, keys, settings and history. CPU support does not remove a 32-bit device's memory limits. CPU and TV support.

Screenshots

Real use shared by the project owner: Quick Settings access, English live captions, and Arabic translation over another app. Tap an image to see it full size.

Start from Quick SettingsRead live captions
Android Quick Settings with Hearth Live captions and Screen translation tiles, both ready to startHearth displaying English captions over a cooking video, with history, pause, clear and settings controls
Translate with a transparent bubbleMove the bubble where you want
Arabic translated captions in a transparent Hearth overlay near the top of a videoArabic translated captions in a light Hearth overlay positioned near the bottom of a video

The video examples show @thehappypear in YouTube; third-party content belongs to its respective owners. Screenshot details.

Inside the app

Phone captures from an earlier release. Conversation and OCR examples use synthetic text.

Home cardsEasy setupLive captions
Illustrated floating Home card with a direct action and horizontal service navigationTwo large choices for local models or a cloud connectionCaption setup with device audio, Nemotron recognition and Arabic translation
ConversationFace to faceType to translate
Synthetic English station-directions question and Arabic translation, with language swap and speech controlsTwo-person split layout with one half rotated and large Speak buttonsFresh local Hy-MT2 translation of an English station-directions question into Arabic
Camera & OCRScreen & mangaSettings
Synthetic English image recognized locally and translated into ArabicScreen translation setup with manual and page-change controlsLocal and Cloud setup tabs with focused feature settings

Android TV

On a TV, the home screen centers on large Start Hearth and Stop Hearth buttons over local artwork. Speech setup opens compact Speech, Language and Voice tabs; setup cards do not remain on the everyday dashboard. Saved configuration is read from the current settings, including after phone pairing. The Play build uses OpenAI in this TV setup flow, with one key reused for its official speech and voice endpoints. FOSS keeps its offline model and device-voice setup. Other providers and full settings remain available through Full interface in setup; TV mode brings the TV home back. In Speech, choose Set up from phone · QR, scan with iPhone or Android, paste the key and choose the three settings on the phone, then approve them on the TV. Both devices need the same private home network. The temporary listener closes after five minutes or when you leave. The paired Google TV phone keyboard is also available as a fallback on supported TVs. The background and preview images are bundled offline; artwork prompts. Phone layouts keep their existing home and tabs, including in landscape.

For remote-only control, open Hearth controls from the TV app list, or choose Overlay & speech controls inside Hearth. Use the D-pad and OK to hide/show, pause/resume, stop captions, move/resize the overlay, adjust its appearance, or Stop & close Hearth. The editor has side categories for Position, Size, Text, Style and Session, with a scrolling options pane and a large fixed preview. The preview uses sample captions and follows the real overlay geometry and theme. Bottom is the default position. New/reset landscape layouts (TV and phones) use 80% width / 30% height; portrait uses 85% / 40%. Saved custom sizes take priority. Speech or provider errors preserve the saved layout. The overlay stays out of the way while controls are open and returns in its saved position when you leave. Hiding keeps speech and read-aloud running; Stop ends both.

For an emergency exit, press Home and open Stop Hearth (the red stop icon) in the TV app list. It immediately stops caption and screen-reading capture, removes their overlays, ends their speech, and closes Hearth windows. It needs no shortcut buttons or setup and is safe to select repeatedly while stopped.

On devices with custom Quick Settings tiles, Live captions now toggles capture on/off. Long-press it to open controls. TV system quick-panel support varies by manufacturer; the Hearth controls launcher entry works independently of it.

Captured directly on the owner's TCL Smart TV Pro, Android 14, running Hearth 0.23.10 in Arabic. These are unedited device screenshots of the installed app. The illustrations and editor's sample-caption preview are built into Hearth. Device and capture details.

Actual TCL screenshot of Hearth's Arabic TV dashboard with Start and Stop buttons Actual TCL screenshot of Hearth's Speech, Language and Voice setup tabs Actual TCL screenshot of Hearth's overlay editor with scrolling controls and fixed caption placement preview

What you can do

FeatureControls and options
Live captionsDevice audio or microphone, optional translation and read-aloud. Move/resize the overlay, adjust transparency, pause, clear, scroll or pass touches through.
Read aloudThe overlay's third Audio tab has male/female voice preferences, TTS loudness and device media volume. Choose Android, local, self-hosted or cloud caption voices.
Saved captionsOff by default. Save original CC, translations, or both when available. View/rename sessions, export TXT/JSON and delete them. History saves text, not audio. Guide.
Conversation / Face to faceSpeak or type in two languages; keep originals and translations, swap languages and hear replies. Face to face rotates one half for the other person.
Type / selected textType, copy or hear a translation in Hearth. Select text in another app → Hearth Translate, or Share → Hearth Translate, for a focused translation window without opening Home.
Camera & OCRTranslate camera or imported-photo text using local OCR and your chosen translator.
Screen & mangaTranslate a shared screen or selected region; refresh after a settled page change and scroll underneath with the lock handle. Guide.
Compare local modelsTry Speed first, Balanced and Quality first presets. Compare timings and outputs, test supported languages, use your own samples, and save/export results. Benchmark guide.

Home uses illustrated cards and remembers the last feature. Ink + Dark is the default; other themes and Classic tabs are available. The interface supports English, العربية and 简体中文, including Arabic RTL. Speech and translation languages depend on the selected engine, independently of the interface language.

Quick start

  1. Open Easy setup and choose On this phone or Cloud. The offline build uses model imports; cloud setup accepts OpenAI, OpenRouter or a compatible server.
  2. Open a feature, choose its languages and speech/translation setup, then start.
  3. Accept Android's microphone, overlay or capture permissions when requested. Add translation or read-aloud only if you want them.

Full model/provider choices are in Settings → Local / Cloud. Easy setup guide.

For quick access, add Live captions and Screen translation through Settings → Phone shortcuts or Android's Quick Settings editor. Tiles use your saved setup; notifications provide session controls. Android capture consent still applies, and some apps block capture. Quick access guide.

Models and connections

StageChoices
Local speechNemotron 3.5 ASR, Qwen3-ASR, Whisper, Vosk, English Moonshine and Phonon 2, experimental Omnilingual CTC.
Local translationHY-MT1.5 / Hy-MT2, TranslateGemma 4B, experimental MiLMMT-46 1B, Marian / OPUS-MT language pairs and ML Kit packs in the cloud-capable build.
Cloud speech / translationOpenAI, OpenRouter and compatible server connections; Soniox and ElevenLabs Scribe for speech. Separate text translators include Google Cloud, Azure, DeepL and LibreTranslate.
Local OCRPaddleOCR, Manga OCR and Meiki, with reader-specific language support.
VoicesAndroid voices, local Supertonic 3, self-hosted Chatterbox/Qwen3-TTS/Fish, and cloud caption voices through OpenAI, OpenRouter, Google Gemini or ElevenLabs.

Use Download & install for catalogued models, including Phonon 2, or import a supported package. Downloads keep a verified app-owned model copy; the offline build has no downloader. Model variants, language coverage and licenses are shown in their setup pages. A benchmark preset is a suggestion to test on your device, not a measured universal ranking.

Local speech and translation use separate models. Whisper's built-in translation targets English; other recognizers use the selected translation stage. Provider integrations can supply their own translation. Translator choices are remembered by feature group; Use everywhere explicitly applies one across Hearth.

Build

Use JDK 17 for compilation and JDK 21 for Android 16 unit tests, Android SDK 36, NDK 27.0.12077973 and CMake 3.22.1. Set ANDROID_HOME or an ignored local.properties. Pinned native runtimes and the Gradle wrapper are included.

./gradlew test :app:compilePlayQaKotlin :app:compileFossQaKotlin
./gradlew :common-jni:externalNativeBuildDebug :app:assemblePlayQa :app:assembleFossQa
python3 scripts/verify-release.py

Outputs: app/build/outputs/apk/{play,foss}/qa/ — four CPU-specific APKs and one universal APK per flavor. QA uses the debug signing key and application ID com.sal7one.transiber.qa. For production signing and native/host validation, see AGENTS.md, CPU/native build guide and release guide. Never commit keys or credentials.

The common-jni Android library owns native model sessions, audio utilities and JNI adapters. The app supplies UI and configuration. Shared utilities and pipeline ownership are documented in the utils bible and pipeline guide.

Contribute

Report a bug with the version, device, selected engines and actual error. Remove keys and private text from logs. Model adapters, language capabilities, translations and device testing are welcome. Changelog · Backlog.

Privacy and license

Local models run on the device. Cloud stages send their input to the provider you choose; credentials are stored encrypted. The offline flavor has no network permission. Privacy policy · Security reporting.

This is the standalone caption/translation app. Media editing, FFmpeg, yt-dlp and sign recognition are outside this repository's scope. Native package names remain unchanged for compatibility.

First-party code: Apache-2.0. Runtimes and model weights have separate terms: THIRD_PARTY_NOTICES.md. Screenshot content from other apps belongs to its respective owners.

android
camera
ffmpeg
ffmpeg-android
jni
jni-android
ndk
nemotron
ocr
offline-ai
offline-transcription
offline-translation
onnx
stt
translation
tts
voice
vosk
whisper
whisper-cpp

Sal7one/Hearth

Offline-first live captions, translation and conversations on Android., STT, TTS, OCR, ONNX custom Audio and NDK pipelines

C++

9

140 commits

updated Oct 18, 2026

See the code

README

Hearth

Hearth: English and Arabic speech bubbles

Read speech. Translate your screen. Talk across languages.

An Android app for live captions, translation and conversations. Run models on this device, use your own cloud keys, or mix local and cloud stages. Translation and read-aloud are optional, with their own settings. Built to make speech and languages more accessible, including for deaf and hard-of-hearing users.

Download · Screenshots · Quick start · Models · Build

Download

Hearth 0.23.10 — release notes and SHA256SUMS

Choose Universal if you do not know your phone or TV's CPU.

CPUCloud + localOffline only
Universal · recommendedDownload APKDownload APK
ARM64Download APKDownload APK
ARMv7 · 32-bitDownload APKDownload APK
x86_64Download APKDownload APK
x86 · 32-bitDownload APKDownload APK

Android 9+; device-audio capture requires Android 10+. Cloud + local includes optional provider connections and model downloads. Offline only has no network permission and imports models from files. Model weights are separate from the APK.

These are test-signed QA builds. Both flavors replace each other. Update without uninstalling to retain models, keys, settings and history. CPU support does not remove a 32-bit device's memory limits. CPU and TV support.

Screenshots

Real use shared by the project owner: Quick Settings access, English live captions, and Arabic translation over another app. Tap an image to see it full size.

Start from Quick SettingsRead live captions
Android Quick Settings with Hearth Live captions and Screen translation tiles, both ready to startHearth displaying English captions over a cooking video, with history, pause, clear and settings controls
Translate with a transparent bubbleMove the bubble where you want
Arabic translated captions in a transparent Hearth overlay near the top of a videoArabic translated captions in a light Hearth overlay positioned near the bottom of a video

The video examples show @thehappypear in YouTube; third-party content belongs to its respective owners. Screenshot details.

Inside the app

Phone captures from an earlier release. Conversation and OCR examples use synthetic text.

Home cardsEasy setupLive captions
Illustrated floating Home card with a direct action and horizontal service navigationTwo large choices for local models or a cloud connectionCaption setup with device audio, Nemotron recognition and Arabic translation
ConversationFace to faceType to translate
Synthetic English station-directions question and Arabic translation, with language swap and speech controlsTwo-person split layout with one half rotated and large Speak buttonsFresh local Hy-MT2 translation of an English station-directions question into Arabic
Camera & OCRScreen & mangaSettings
Synthetic English image recognized locally and translated into ArabicScreen translation setup with manual and page-change controlsLocal and Cloud setup tabs with focused feature settings

Android TV

On a TV, the home screen centers on large Start Hearth and Stop Hearth buttons over local artwork. Speech setup opens compact Speech, Language and Voice tabs; setup cards do not remain on the everyday dashboard. Saved configuration is read from the current settings, including after phone pairing. The Play build uses OpenAI in this TV setup flow, with one key reused for its official speech and voice endpoints. FOSS keeps its offline model and device-voice setup. Other providers and full settings remain available through Full interface in setup; TV mode brings the TV home back. In Speech, choose Set up from phone · QR, scan with iPhone or Android, paste the key and choose the three settings on the phone, then approve them on the TV. Both devices need the same private home network. The temporary listener closes after five minutes or when you leave. The paired Google TV phone keyboard is also available as a fallback on supported TVs. The background and preview images are bundled offline; artwork prompts. Phone layouts keep their existing home and tabs, including in landscape.

For remote-only control, open Hearth controls from the TV app list, or choose Overlay & speech controls inside Hearth. Use the D-pad and OK to hide/show, pause/resume, stop captions, move/resize the overlay, adjust its appearance, or Stop & close Hearth. The editor has side categories for Position, Size, Text, Style and Session, with a scrolling options pane and a large fixed preview. The preview uses sample captions and follows the real overlay geometry and theme. Bottom is the default position. New/reset landscape layouts (TV and phones) use 80% width / 30% height; portrait uses 85% / 40%. Saved custom sizes take priority. Speech or provider errors preserve the saved layout. The overlay stays out of the way while controls are open and returns in its saved position when you leave. Hiding keeps speech and read-aloud running; Stop ends both.

For an emergency exit, press Home and open Stop Hearth (the red stop icon) in the TV app list. It immediately stops caption and screen-reading capture, removes their overlays, ends their speech, and closes Hearth windows. It needs no shortcut buttons or setup and is safe to select repeatedly while stopped.

On devices with custom Quick Settings tiles, Live captions now toggles capture on/off. Long-press it to open controls. TV system quick-panel support varies by manufacturer; the Hearth controls launcher entry works independently of it.

Captured directly on the owner's TCL Smart TV Pro, Android 14, running Hearth 0.23.10 in Arabic. These are unedited device screenshots of the installed app. The illustrations and editor's sample-caption preview are built into Hearth. Device and capture details.

Actual TCL screenshot of Hearth's Arabic TV dashboard with Start and Stop buttons Actual TCL screenshot of Hearth's Speech, Language and Voice setup tabs Actual TCL screenshot of Hearth's overlay editor with scrolling controls and fixed caption placement preview

What you can do

FeatureControls and options
Live captionsDevice audio or microphone, optional translation and read-aloud. Move/resize the overlay, adjust transparency, pause, clear, scroll or pass touches through.
Read aloudThe overlay's third Audio tab has male/female voice preferences, TTS loudness and device media volume. Choose Android, local, self-hosted or cloud caption voices.
Saved captionsOff by default. Save original CC, translations, or both when available. View/rename sessions, export TXT/JSON and delete them. History saves text, not audio. Guide.
Conversation / Face to faceSpeak or type in two languages; keep originals and translations, swap languages and hear replies. Face to face rotates one half for the other person.
Type / selected textType, copy or hear a translation in Hearth. Select text in another app → Hearth Translate, or Share → Hearth Translate, for a focused translation window without opening Home.
Camera & OCRTranslate camera or imported-photo text using local OCR and your chosen translator.
Screen & mangaTranslate a shared screen or selected region; refresh after a settled page change and scroll underneath with the lock handle. Guide.
Compare local modelsTry Speed first, Balanced and Quality first presets. Compare timings and outputs, test supported languages, use your own samples, and save/export results. Benchmark guide.

Home uses illustrated cards and remembers the last feature. Ink + Dark is the default; other themes and Classic tabs are available. The interface supports English, العربية and 简体中文, including Arabic RTL. Speech and translation languages depend on the selected engine, independently of the interface language.

Quick start

  1. Open Easy setup and choose On this phone or Cloud. The offline build uses model imports; cloud setup accepts OpenAI, OpenRouter or a compatible server.
  2. Open a feature, choose its languages and speech/translation setup, then start.
  3. Accept Android's microphone, overlay or capture permissions when requested. Add translation or read-aloud only if you want them.

Full model/provider choices are in Settings → Local / Cloud. Easy setup guide.

For quick access, add Live captions and Screen translation through Settings → Phone shortcuts or Android's Quick Settings editor. Tiles use your saved setup; notifications provide session controls. Android capture consent still applies, and some apps block capture. Quick access guide.

Models and connections

StageChoices
Local speechNemotron 3.5 ASR, Qwen3-ASR, Whisper, Vosk, English Moonshine and Phonon 2, experimental Omnilingual CTC.
Local translationHY-MT1.5 / Hy-MT2, TranslateGemma 4B, experimental MiLMMT-46 1B, Marian / OPUS-MT language pairs and ML Kit packs in the cloud-capable build.
Cloud speech / translationOpenAI, OpenRouter and compatible server connections; Soniox and ElevenLabs Scribe for speech. Separate text translators include Google Cloud, Azure, DeepL and LibreTranslate.
Local OCRPaddleOCR, Manga OCR and Meiki, with reader-specific language support.
VoicesAndroid voices, local Supertonic 3, self-hosted Chatterbox/Qwen3-TTS/Fish, and cloud caption voices through OpenAI, OpenRouter, Google Gemini or ElevenLabs.

Use Download & install for catalogued models, including Phonon 2, or import a supported package. Downloads keep a verified app-owned model copy; the offline build has no downloader. Model variants, language coverage and licenses are shown in their setup pages. A benchmark preset is a suggestion to test on your device, not a measured universal ranking.

Local speech and translation use separate models. Whisper's built-in translation targets English; other recognizers use the selected translation stage. Provider integrations can supply their own translation. Translator choices are remembered by feature group; Use everywhere explicitly applies one across Hearth.

Build

Use JDK 17 for compilation and JDK 21 for Android 16 unit tests, Android SDK 36, NDK 27.0.12077973 and CMake 3.22.1. Set ANDROID_HOME or an ignored local.properties. Pinned native runtimes and the Gradle wrapper are included.

./gradlew test :app:compilePlayQaKotlin :app:compileFossQaKotlin
./gradlew :common-jni:externalNativeBuildDebug :app:assemblePlayQa :app:assembleFossQa
python3 scripts/verify-release.py

Outputs: app/build/outputs/apk/{play,foss}/qa/ — four CPU-specific APKs and one universal APK per flavor. QA uses the debug signing key and application ID com.sal7one.transiber.qa. For production signing and native/host validation, see AGENTS.md, CPU/native build guide and release guide. Never commit keys or credentials.

The common-jni Android library owns native model sessions, audio utilities and JNI adapters. The app supplies UI and configuration. Shared utilities and pipeline ownership are documented in the utils bible and pipeline guide.

Contribute

Report a bug with the version, device, selected engines and actual error. Remove keys and private text from logs. Model adapters, language capabilities, translations and device testing are welcome. Changelog · Backlog.

Privacy and license

Local models run on the device. Cloud stages send their input to the provider you choose; credentials are stored encrypted. The offline flavor has no network permission. Privacy policy · Security reporting.

This is the standalone caption/translation app. Media editing, FFmpeg, yt-dlp and sign recognition are outside this repository's scope. Native package names remain unchanged for compatibility.

First-party code: Apache-2.0. Runtimes and model weights have separate terms: THIRD_PARTY_NOTICES.md. Screenshot content from other apps belongs to its respective owners.

android
camera
ffmpeg
ffmpeg-android
jni
jni-android
ndk
nemotron
ocr
offline-ai
offline-transcription
offline-translation
onnx
stt
translation
tts
voice
vosk
whisper
whisper-cpp