78 repos across 4 sub-areas
On-Device LLM Inference & Apple Silicon
39 repos
Running large language models efficiently on personal devices and Apple Silicon hardware, with emphasis on optimized inference, memory constraints, and native integration. The cluster centers on practical implementations of quantized and specialized model variants (including multimodal vision-language models) designed for macOS and iOS, along with Swift-native tooling to deploy and interact with these models locally without cloud dependencies.
On-device LLM deployment via CoreML
16 repos
Quantized and optimized versions of large language models (Qwen, Gemma, and other architectures) compiled to Apple's CoreML format for efficient inference on iOS devices and Apple Silicon hardware. These repositories focus on making modern language models run locally on mobile and edge devices through aggressive quantization (often 4-bit or lower), stateful execution for efficient token generation, and native Swift/iOS integration. Useful for anyone building privacy-preserving, latency-sensitive LLM applications on Apple platforms.
Cluster 462327
12 repos
On-Device AI Models & CoreAI
11 repos
Optimized machine learning models designed to run efficiently on mobile and edge devices, with a focus on Apple platforms and lightweight architectures. The cluster centers on the CoreAI framework and includes specialized models for speech recognition, language understanding, audio processing, and multimodal tasks—all built for minimal computational footprint while maintaining practical performance. These repos represent a cohesive effort to bring practical AI inference to resource-constrained environments.