TheStage Apple SDK

On-device speech, language, and audio inference for iOS and macOS on Apple Silicon. Engines ship as CoreML bundles from Hugging Face; the SDK picks ANE / GPU / CPU per device. After initialize, inference never leaves the device — there is no server in the hot path.

Not supported: iOS Simulator, Intel Macs, Android, server-side inference.

Known issues (1.5.0)

• Voice tool-call streaming / parsing: filler text and tool-call markup can appear interleaved on the same stream, so TTS may start speaking while a tool call is still being recognized or executed. Some edge cases in tool call parsing remain. Planned fix in the next release.
• iOS 27 memory accounting: iOS 27 charges Neural Engine memory to the app. Large multifunction packs (Gemma-4 E2B) run close to the per-process limit on 12 GB phones in speculative mode; the SDK releases the speculative-only engine views while decoding plain and reloads them for the next speculative call. Switching plain → speculative on a loaded Gemma-4 can still exceed the limit on an iPhone 18 Pro. Stay on one setting per session, or prefer the LFM2.5 packs on iOS 27 devices.

---

Built for battery — not just tok/s

On-device generation on the Apple Neural Engine draws a fraction of the power of GPU paths like MLX — phones stay cooler, laptops stay on battery, and sustained chat / voice agents remain practical.

Gemma3-1B, sustained load. MacBook M3 Max via macmon; iPhone 13 via battery drain. ~5.1× less power on M3 Max and ~3.5× on iPhone 13 vs MLX (GPU). Lower is better.

Speed and latency charts live under Performance.

---

Read this first

Humans — do the 60-second Mac quick start, then pick Swift or Flutter under Integrate. Dive into contracts only when something breaks.

Agents / coding assistants — treat this file as the index, then open docs/llms.txt and the linked page for the pipeline you touch. Hard rules:

1. Always initialize before any pipeline / start_model.
2. Pin the SDK to tag 1.5.0 (do not float from:).
3. Pass HF repo ids like "TheStageAI/Qwen3-0.6B" — omit revision unless you intentionally override (defaults track this SDK line).
4. Audio is mono Float / Float32List in [-1.0, 1.0] — never Float64 / Int16 without converting.
5. Do not invent device IDs, seat accounting, or crypto details — see licensing.

initialize (online) ──► start_model / Pipeline(...) ──► infer / infer_stream
        │                         │
        │                         ▼
        │              HF download + cache (first time)
        ▼
   process session OK     inference fully on-device

---

Table of contents

1. What's in this repo
2. Capabilities & model fleet
3. Performance
4. Quick start
5. Prerequisites
6. Integrate
7. Mental model
8. Contracts (audio · progress · Swift↔Flutter)
9. Documentation map
10. Troubleshooting
11. Secrets & license

---

What's in this repo

---

Capabilities & model fleet

Everything below is the production @v1.5 fleet. Pass the HF id as engines_path (or construct the typed pipeline with the same string).

Out of this release (do not pin in new apps): full neutts-multilingual, English espeak neutts / NeuTTSNanoPipeline, wake-word, VLM / YOLO.

Model cards (contracts + acknowledgments): huggingface.co/TheStageAI.

---

Performance

Release-build comparisons vs common on-device stacks. Device class and thermals matter — treat these as relative guidance, not an SLA.

Decode throughput vs MLX (GPU)

Gemma-3-1B-it tokens/sec across iPhones — TheStage on ANE vs MLX on GPU (higher is better):

Time to first token

LFM2.5-350M TTFT — TheStage stays in the low teens of ms while MLX stretches on older phones (lower is better):

ANE runtime bake-off (same chip)

Qwen3-0.6B on iPhone 17 Pro, all paths on the Neural Engine — TheStage SDK vs CoreML-LLM and CoreAI (higher is better):

Power / efficiency (intro highlight): Built for battery.

---

Quick start

60 seconds on Mac (no Xcode, no device)

cd examples/macos_swift_tts
export TS_API_TOKEN=th_…          # from app.thestage.ai
swift run

First run downloads NeuTTS engines and caches them; later runs start cold from disk. Playback-only — no mic permission. Details: examples/macos_swift_tts/README.md.

Physical iPhone (Flutter)

./scripts/setup.sh                # idempotent; espeak only if you need nano-EN
cp examples/voice_agent/secrets.example.json \
   examples/voice_agent/secrets.json
# edit secrets.json → TS_API_TOKEN

In Xcode (examples/voice_agent/ios/Runner.xcodeproj): set Team + unique Bundle Identifier, then:

cd examples/voice_agent
flutter pub get
flutter run --release \
    --dart-define-from-file=secrets.json \
    -d <YOUR_IPHONE_DEVICE_ID>

examples/voice_agent is the same recipe (+ OPENAI_API_KEY if you use the cloud LLM provider).

---

Prerequisites

# Flutter path only
brew install flutter
flutter config --enable-swift-package-manager

---

Integrate

Native Swift (iOS + macOS)

Xcode → File → Add Package Dependencies… → this repo URL → product TheStageSDK. Or:

.package(
    url: "https://github.com/TheStageAI/AppleSDK.git",
    exact: Version(1, 5, 0)
)

import TheStageSDK

let ai = TheStageAI.shared
try await ai.initialize(api_token: "th_…")

let llm = try await TheStageLLM(
    engines_path: "TheStageAI/Qwen3-0.6B",
    on_load_progress: { p in
        print("[\(p.model)] \(p.phase) \(Int(p.fraction * 100))%")
    }
)

let result = llm.infer(
    prompt: "Give me a one-line haiku about Swift.",
    max_new_tokens: 64
)
print(result.text)

Two ways to drive models (same cache, same progress events):

Flutter (iOS)

dependencies:
  thestage_apple_sdk:
    git:
      url: https://github.com/TheStageAI/AppleSDK.git
      path: plugin/thestage_apple_sdk
      ref: 1.5.0

flutter config --enable-swift-package-manager
# Xcode → Runner → Minimum Deployments → iOS 18.0

import 'package:thestage_apple_sdk/thestage_apple_sdk.dart';

await TheStageFlutterSDK.initialize(api_token: 'th_…');

await TheStageFlutterSDK.start_model(
  model_name: 'llm',
  engines_path: 'TheStageAI/Qwen3-0.6B',
);

final result = await TheStageFlutterSDK.infer(
  model_name: 'llm',
  input_json: {
    'prompt': 'Give me a one-line haiku about Swift.',
    'max_new_tokens': 64,
  },
);
print(result[0]['text']);

More: plugin README. Fastest path to a working UI: copy an examples/ app.

---

Mental model

Lifecycle

1. initialize(api_token:) — online token check + device seat registration. Fails offline / on network errors. Once per process when reachable; after success, inference is on-device for that process.
2. Load — Pipeline(engines_path:) or start_model(...). First hit downloads the HF revision for this SDK line into Application Support (not purged by iOS, excluded from iCloud backup).
3. Infer — batch infer or streaming infer_stream / TTS streamer.
4. Stop — stop_model / drop pipeline references to free memory.

Revisions

Omit revision: in normal apps. This build resolves HF tags via an internal map aligned with SDK 1.5.0 → fleet v1.5. Override only when you intentionally pin an older engine tag.

Init & seats (product)

• Seat = (apiToken, deviceId) — see licensing.md.
• Pricing / plans: Service Request at app.thestage.ai/contact.
• Do not document or depend on how deviceId is derived.

---

Contracts

Audio I/O

All public audio is PCM mono, samples in [-1.0, 1.0].

Prefer tts.sample_rate over hardcoding. Mic path for agents is 16 kHz; TTS playback is 24 kHz — resample at the edge if you mix them.

Swift: [Float] · Flutter: Float32List only.

Load progress

Optional on_load_progress (Swift) / TheStageFlutterSDK.on_progress (Flutter). Phases are monotonic 0...1:

Swift ↔ Flutter parity

Naming trap: Swift apiToken: vs Flutter api_token:.

---

Documentation map

---

Troubleshooting

---

Secrets & license

Flutter examples load tokens via --dart-define-from-file=secrets.json (from secrets.example.json). Keep real keys out of git. The macOS example uses TS_API_TOKEN in the environment.

License: LICENSE. Commercial terms: product_terms.md.