LLM inference on Apple Silicon

Quantized machine learning models and inference frameworks optimized for Apple Silicon using the MLX library and Metal acceleration. The cluster centers on efficient implementations of large language models and video generation systems in 4-bit, 8-bit, and bfloat16 precision, enabling on-device inference on Mac hardware. Repositories here demonstrate practical deployment strategies for models like MiniMax-H3 and LongCat video avatars using MLX's native Apple Silicon support.

Optimized implementations of large language models running on Apple's MLX framework with aggressive quantization techniques (AXQ/mixed-precision formats) to fit models like Qwen on resource-constrained devices. These repositories focus on model compression, inference optimization, and making state-of-the-art LLMs practically deployable on Mac and iOS hardware through low-bit quantization schemes.

Optimized inference implementations of large language models using MLX (Apple's machine learning framework) with AXQuant quantization techniques to reduce model size and memory requirements. The cluster centers on quantized versions of popular models like Qwen, Mistral, and Ministral, enabling efficient text generation and conversational AI on resource-constrained devices. Most repos are model artifact collections and deployment configurations rather than algorithmic frameworks, making this primarily a catalog of pre-optimized model checkpoints and inference recipes for Apple Silicon and edge inference scenarios.

Machine learning libraries and applications built on MLX, Apple's framework for efficient ML on Apple Silicon devices. The cluster spans Python-based ML tools (22 repos), Swift/native iOS applications (8 repos), and integrations with popular models like Qwen and LLMs. Common themes include local inference, LLM deployment on macOS/iOS, and OpenAI-compatible APIs, reflecting a focus on bringing capable AI workloads to consumer Apple hardware without cloud dependency.

Optimized and quantized versions of large language models, primarily focusing on 4-bit and 6-bit quantization formats for reduced memory footprint and faster inference. The cluster contains numerous variants of Qwen models (base, coding, and vision versions) packaged in AWQ and other quantization schemes, compatible with inference frameworks like SGLang and optimized for hardware like RDNA4 GPUs. These repositories represent the practical engineering of model compression and deployment artifacts rather than training or architecture research.

Libraries and applications for speech recognition, text-to-speech, and audio processing optimized for Apple Silicon using the MLX framework. Python dominates the cluster for model implementations and audio processing, while Swift appears in client applications leveraging these capabilities natively on macOS and iOS. This represents the emerging ecosystem of on-device speech and audio AI, with particular focus on efficient inference and multimodal integration.

Low-bit quantization (2-bit, 4-bit, 6-bit) techniques for compressing and optimizing large language and vision models, particularly for deployment on resource-constrained hardware like Apple Silicon via MLX. The cluster focuses on making state-of-the-art models (Qwen, DeepSeek, MiniMax, Muse) inference-efficient through aggressive quantization while maintaining usable accuracy, with AXQ emerging as a common quantization standard across implementations.

Techniques and implementations for quantizing large language models to run efficiently on Apple Silicon using the MLX framework. The cluster focuses on reducing model size through various quantization schemes (4-bit, NVFP4, DWQ) applied to models like Lance-3B, enabling inference on resource-constrained devices. Most repositories appear to be model variants or configuration implementations rather than foundational libraries, but collectively demonstrate practical approaches to deploying conversational AI on Apple hardware with quantized weights distributed via safetensors format.

Quantized language model variants optimized for Apple Silicon and resource-constrained devices. This cluster contains multiple compressed versions of open-source language models (GPT, DeepSeek, MiniCPM, Devstral) post-training quantized to 2-bit, 4-bit, and 6-bit precision using the MLX framework, enabling efficient inference on edge hardware. Repos focus on reducing model size and computational requirements while maintaining usable performance for on-device deployment.