litert-community/Qwen3-0.6B

Main model card: Qwen/Qwen3-0.6B

This repository contains LiteRT-LM variants of Qwen3-0.6B for Android and desktop deployment.

Available Artifacts

Conversion Notes

The mixed INT4 .litertlm artifact was produced with a TorchAO-based quantize-first recipe from the original Hugging Face checkpoint. This is a mixed quantization layout rather than a uniform all-INT4 model: eligible linear projection weights are stored as blockwise INT4 with group size 32 and floating-point scales, token embedding weights use weight-only INT8 quantization, and normalization/reduction paths plus KV cache tensors remain floating point.

The mixed INT4 bundle also uses LiteRT-LM StableHLO composite ops for attention/cache execution, including odml.runtime_bmm and odml.cache_update.

Qwen3-0.6B.litertlm is a separate dynamic INT8 artifact. It was converted through the LiteRT Torch (litert-torch) path and quantized with AI Edge Quantizer. This artifact is independent from qwen3_0_6b_mixed_int4.litertlm, which uses the TorchAO-based mixed INT4 recipe described above.

Qwen3-0.6B_dynamic_wi4b32_afp32.litertlm is a separate dynamic INT4 variant (block-32 weights, FP32 activations). It was converted through LiteRT Torch (litert-torch) path and quantized with AI Edge Quantizer. This artifact incorporates LiteRT-LM GPU graph optimizations, including composite ops for RoPE, fused QKV, and fused Gate/Up projections, and is configured with static prefill memory allocation.

Android Performance Examples

These are representative measurements from retail devices to give a rough sense of on-device runtime behavior, not a direct comparison between hardware platforms. All numbers were collected with LiteRT-LM's litert_lm_advanced_main launched from an adb command line on the connected device; they are not app-level measurements from an integrated Android application.

Hardware benchmark disclosure: Results were measured by us on retail devices purchased through normal channels. These results are not affiliated with, sponsored by, endorsed by, or verified by Samsung, vivo, Qualcomm, MediaTek, Google, MLCommons, or Hugging Face. Results depend on device SKU, OS build, thermal state, battery mode, backend, model quantization, runtime version, and benchmark settings.

qwen3_0_6b_mixed_int4.litertlm

Context: 2048. Shape: 256 prefill tokens / 256 decode tokens. Rows use LiteRT-LM v0.13.1. Values report the warmed iteration from a two-iteration run unless noted.

Qwen3-0.6B.litertlm

Context: 4096. Samsung and TECNO rows use 256 prefill tokens / 256 decode tokens with LiteRT-LM v0.13.1. The vivo rows are previously published 4096-context reference results; TTFT, peak footprint, and exact prompt/decode shape were not recorded in this update.

Qwen3-0.6B.mediatek.mt6993.litertlm

Context: 4096. This is a previously published MediaTek MT6993 NPU reference result; TTFT, peak footprint, and exact prompt/decode shape were not recorded in this update.

Desktop Smoke Benchmark

Benchmarked on AMD Radeon AI PRO R9700 via LiteRT-LM WebGPU with 256 prefill tokens and 32 decode tokens.

Use the model

Try It (Desktop/CLI)

Install uv and run:

uv tool install litert-lm
uvx litert-lm run --from-huggingface-repo=litert-community/Qwen3-0.6B qwen3_0_6b_mixed_int4.litertlm --prompt="What is the capital of France?"

Edge Gallery App

1. Get the App: Install the app from Google Play or download the latest APK from the GitHub releases page.
2. Importing the Model: Navigate to the Model manager within the app and click the "+" (plus) icon in the bottom-right corner. Two options will appear:
   • Import from HF (Recommended): Select this option, and a dialog box will appear showing an example Hugging Face model URL. Enter the HF link for the desired .litertlm model and click submit. The model will then appear in your list, and you can proceed to download it (a Hugging Face account login is required).
   • From local model file: First, download the .litertlm model directly to your Android device, OR download it to your computer and push it via ADB (e.g., adb push Qwen3-0.6B_dynamic_wi4b32_afp32.litertlm /sdcard/Download/). Then, select this option, choose the downloaded file from your storage, configure your preferred parameters, and tap "Import". For full details on importing models and other features, see the Edge Gallery App Wiki. To build the demo app from source, please follow the instructions from the GitHub repository.

Integration

Ready to integrate this into your product? Get started in the LiteRT-LM documentation.

Citation

@misc{qwen3technicalreport,
      title={Qwen3 Technical Report},
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.09388},
}