Use it from Swift

Add the package

Package.swift:

.package(url: "https://github.com/john-rocky/CoreML-LLM", branch: "main"),

// In your target:
.product(name: "CoreMLLLM", package: "CoreML-LLM"),

Platforms: iOS 18+ / macOS 15+.

Download + chat (one call)

import CoreMLLLM

let llm = try await CoreMLLLM.load(repo: "mlboydaisuke/qwen3-vl-8b-stateful-coreml")

let stream = try await llm.generate(
    [CoreMLLLM.Message(role: .user, content: "Hello!")],
    maxTokens: 256
)
for await chunk in stream { print(chunk, terminator: "") }

With an image

import CoreGraphics

let cgImage: CGImage = ...   // your CGImage

let stream = try await llm.generate(
    [CoreMLLLM.Message(role: .user,
                       content: "What's in this image?")],
    image: cgImage,
    maxTokens: 256
)
for await chunk in stream { print(chunk, terminator: "") }

Qwen3-VL 8B — Core ML stateful

Core ML port of Qwen/Qwen3-VL-8B-Instruct for iPhone / iPad / Mac Apple Neural Engine. Text + vision, 6.19 GB on disk.

The KV cache lives inside the ANE via MLState, so it does not spill to GPU memory as the context grows. Same layout as qwen3-vl-2b-stateful-coreml, scaled to 8B: 6 body chunks instead of 4.

Files

qwen3_vl_8b_stateful_chunks/
├── chunk_0.mlpackage … chunk_5.mlpackage   ← body, multifunction: infer (T=1) + prefill_b8 (T=8)
├── chunk_0_vision.mlpackage                ← chunk_0 + DeepStack injection
├── chunk_head.mlpackage                    ← final_norm + lm_head + in-graph argmax
└── embed_weight.bin                        ← raw fp16 embed table, Swift mmaps it

qwen3_vl_8b_vision/
└── vision.mlpackage                        ← image encoder + DeepStack taps

Each body chunk carries two Core ML functions sharing one state. Swift creates the MLState once from the prefill instance and reuses it across both — Core ML binds state by name and shape, not by MLModel instance.

What this repo does NOT ship

• No model_config.json — Core ML packs shapes into each .mlpackage, and coremltools reads them directly.
• No tokenizer / processor — pull them from the upstream model:

from transformers import AutoTokenizer, AutoProcessor
tok  = AutoTokenizer.from_pretrained("Qwen/Qwen3-VL-8B-Instruct")
proc = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-8B-Instruct")

Standalone usage (Python / Mac)

import coremltools as ct, numpy as np
from huggingface_hub import snapshot_download

local = snapshot_download("mlboydaisuke/qwen3-vl-8b-stateful-coreml")
root  = f"{local}/qwen3_vl_8b_stateful_chunks"

prefill_chunks = [ct.models.MLModel(
    f"{root}/chunk_{i}.mlpackage", function_name="prefill_b8"
) for i in range(6)]
decode_chunks  = [ct.models.MLModel(
    f"{root}/chunk_{i}.mlpackage", function_name="infer"
) for i in range(6)]
head = ct.models.MLModel(f"{root}/chunk_head.mlpackage")

# State is created once and shared across infer + prefill_b8.
state = prefill_chunks[0].make_state()

Conversion

github.com/john-rocky/CoreML-LLM

License

Apache 2.0 (inherits from the base model).

---

More models in this format: Core ML Model Zoo — 46 models, each with the recipe that produced it.

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.