78 repos across 4 sub-areas
Open-source and optimized implementations of large language models, with a strong focus on efficient deployment on consumer hardware including Apple Silicon and macOS. This cluster contains model implementations, fine-tuning frameworks, and inference optimization tools that enable running capable LLMs locally and on-device. The central repos represent various model architectures and sizes (from 1B to 35B parameters) optimized for resource-constrained environments, reflecting the broader trend toward making advanced AI accessible without cloud infrastructure dependencies.
On-Device LLM Inference & Apple Silicon
39 repos
Running large language models efficiently on personal devices and Apple Silicon hardware, with emphasis on optimized inference, memory constraints, and native integration. The cluster centers on practical implementations of quantized and specialized model variants (including multimodal vision-language models) designed for macOS and iOS, along with Swift-native tooling to deploy and interact with these models locally without cloud dependencies.
On-device LLM deployment via CoreML
16 repos
Quantized and optimized versions of large language models (Qwen, Gemma, and other architectures) compiled to Apple's CoreML format for efficient inference on iOS devices and Apple Silicon hardware. These repositories focus on making modern language models run locally on mobile and edge devices through aggressive quantization (often 4-bit or lower), stateful execution for efficient token generation, and native Swift/iOS integration. Useful for anyone building privacy-preserving, latency-sensitive LLM applications on Apple platforms.
On-Device ML & Apple Silicon Optimization
12 repos
Machine learning models optimized for edge deployment and Apple Silicon (CoreML/neural processing). The cluster centers on compact embeddings, image understanding, and reranking models designed for local inference—with strong emphasis on CoreML framework integration and Gaussian splatting for 3D vision tasks. Repos here focus on making modern LLMs and vision models practical for resource-constrained devices rather than cloud inference.
On-Device AI Models & CoreAI
11 repos
Optimized machine learning models designed to run efficiently on mobile and edge devices, with a focus on Apple platforms and lightweight architectures. The cluster centers on the CoreAI framework and includes specialized models for speech recognition, language understanding, audio processing, and multimodal tasks—all built for minimal computational footprint while maintaining practical performance. These repos represent a cohesive effort to bring practical AI inference to resource-constrained environments.