PMetal

Powdered Metal — An ML SDK, framework, and application suite for Apple Silicon, written in Rust.

PMetal is a complete machine learning platform for Apple Silicon — from low-level Metal GPU kernels and Apple Neural Engine integration to high-level training APIs, a terminal TUI, and a full desktop GUI. Ship fine-tuned models without leaving the Apple ecosystem.

Use PMetal Your Way

Desktop GUI

A full Tauri + Svelte desktop application for visual model management, training, and inference.

cd crates/pmetal-gui
bun install && bun tauri dev

19 pages: Dashboard, Training, GRPO, Distillation, Pretrain, Inference, DFlash, Models, Datasets, Merging, Quantize, Embed Train, RLKD, Ollama, Serve, Bench, Eval, Jobs, and Settings. Download models from HuggingFace, configure LoRA training with live loss metrics, chat with models, merge weights, and quantize — all from the GUI. Training, inference, distillation and GRPO run in-process with real-time progress updates; the remaining pages drive the pmetal CLI as a subprocess, which the app bundles.

Terminal TUI

A full-featured terminal control center with 20 tabs.

pmetal tui

Keybindings: Ctrl+P to jump to any tab (type to filter), Tab/Shift+Tab to cycle, Alt+1-9 (or Ctrl+1-9) for the first nine, L to adjust learning rate mid-run, ? for contextual help, q to quit.

CLI

# LoRA fine-tuning with sequence packing (default)
pmetal train \
  --model Qwen/Qwen3-0.6B \
  --dataset train.jsonl \
  --output ./output \
  --lora-r 16 --batch-size 4 --learning-rate 2e-4

# Inference with LoRA adapter
pmetal infer \
  --model Qwen/Qwen3-0.6B \
  --lora ./output/lora_weights.safetensors \
  --prompt "Explain quantum entanglement" \
  --chat

# Train and use a Qwen3Next/Qwen3.6 MTP predictor
pmetal tokenize --input train.jsonl --output ./tok --tokenizer Qwen/Qwen3.6-30B-A3B-Instruct
pmetal train-mtp \
  --model Qwen/Qwen3.6-30B-A3B-Instruct \
  --family qwen3-next \
  --shards ./tok/shard_00000.bin \
  --output ./qwen-mtp
pmetal infer \
  --model Qwen/Qwen3.6-30B-A3B-Instruct \
  --mtp --mtp-model ./qwen-mtp \
  --prompt "Explain monotonic queues."

# Knowledge distillation
pmetal distill \
  --teacher Qwen/Qwen3-4B \
  --student Qwen/Qwen3.5-0.8B-Base \
  --dataset train.jsonl

# GRPO reasoning training
pmetal grpo \
  --model Qwen/Qwen3-0.6B \
  --dataset reasoning.jsonl \
  --reasoning-rewards

# HuggingFace model search with memory fit
pmetal search "qwen 0.6b" --detailed

# Merge models with SLERP
pmetal merge \
  --model-a model-a --model-b model-b \
  --method slerp --t 0.5

# Quantize to GGUF
pmetal quantize \
  --model ./output \
  --output model.gguf --method q4_k_m

# Fuse LoRA into base model
pmetal fuse \
  --model Qwen/Qwen3-0.6B \
  --lora ./output/lora_weights.safetensors

# Evaluate perplexity
pmetal eval \
  --model Qwen/Qwen3-0.6B \
  --dataset eval.jsonl

# Start OpenAI-compatible server
# (in the prebuilt binary and the brew formula; add --features serve if you build it yourself)
pmetal serve --model Qwen/Qwen3-0.6B --port 8080

All CLI Commands

Multi-Mac Cluster (Thunderbolt-aware)

Connect two or more Apple Silicon Macs into a "home cluster" for distributed training and inference. PMetal auto-detects every NIC on the box (Thunderbolt-Bridge, Ethernet, Wi-Fi), advertises them via mDNS, and forms a ring biased toward the fastest fabric — Thunderbolt cables are picked over Ethernet, Ethernet over Wi-Fi, all without configuration.

# 1. Connect the Macs (Thunderbolt-4/5 cable recommended; Ethernet works too).
# 2. On every Mac:
pmetal cluster status        # Show local NICs + any peers already announcing.
pmetal cluster up            # Join the cluster, hold connection open.

# 3. On every Mac at the same time:
pmetal cluster bench --mb 64 --iters 10                  # All-reduce throughput per fabric.
pmetal cluster pipeline-bench --tokens 16 --layers 32    # Pipeline activation transport bench.

# 4. Distributed training (each Mac runs the same command simultaneously):
pmetal train --model Qwen/Qwen3-0.6B \
             --dataset train.jsonl   \
             --distributed-auto        # mDNS-discovers peers, all-reduce gradients.

pmetal cluster status example output:

Local peer: 12D3KooW…XyZ  (rank 0/2)
Thunderbolt ring: yes

Local interfaces:
  bridge0    thunderbolt   169.254.42.1
  en0        ethernet      192.168.1.10
  lo0        loopback      127.0.0.1, ::1

Cluster peers:
  peer-id                                   local  primary-addr           fabric        paths
  12D3KooW…XyZ                              yes    169.254.42.1:52416     thunderbolt   2
  12D3KooW…AbC                              no     169.254.42.2:52416     thunderbolt   2

What's wired today: gradient all-reduce (multi-machine training, real), fabric-aware ring formation with Thunderbolt > Ethernet > Wi-Fi priority, automatic fabric fallback when a cable is unplugged mid-job, gradient compression (TopK, FP16/BF16/INT8 quantization, error feedback), and a transport-tested pipeline harness with multi-process integration tests. Per-architecture partial-layer execution (the prerequisite for serving a model that doesn't fit on one Mac) is the next step — the harness is ready, the model API is the bottleneck.

SDK

PMetal is an embeddable SDK — integrate training, inference, and model operations into your own Rust applications. pmetal re-exports every sub-crate, so one dependency gets you the whole framework.

orchestrator::run_training is the one-call training entry point the CLI itself uses:

use pmetal::trainer::orchestrator::{TrainingJobConfig, run_training};

let config = TrainingJobConfig {
    model_id: "Qwen/Qwen3-0.6B".to_string(),
    dataset: "train.jsonl".to_string(),
    output_dir: "./output".to_string(),
    ..Default::default()
};

let result = run_training(config, None, Vec::new()).await?;
println!("final loss {:.4} over {} steps", result.final_loss, result.total_steps);

Inference goes through the model dispatcher:

use pmetal::models::generate;
use pmetal::prelude::*;

let model_dir = pmetal::hub::resolve_model_path("Qwen/Qwen3-0.6B", None, None).await?;
let mut model = DynamicModel::load(&model_dir)?;
let tokenizer = Tokenizer::from_model_dir(&model_dir)?;

let input_ids = tokenizer.encode_with_special_tokens("What is 2+2?")?;
let output = generate(
    |input| model.forward(input, None),
    &input_ids,
    GenerationConfig::sampling(256, 0.7),
)?;
println!("{}", tokenizer.decode(&output.token_ids[input_ids.len()..])?);

Preference optimization (DpoTrainer, SimpoTrainer, OrpoTrainer, KtoTrainer) and TAID distillation are library-only for now — there is no CLI subcommand for them yet.

For step-by-step control, use the crates directly: pmetal_trainer::TrainingLoop, pmetal_models::DynamicModel, pmetal_lora::DynamicLoraModel, pmetal_distill::Distiller. Every code block in each crate's README is compiled as a doctest, so they stay honest. See examples/ for complete working programs, including manual training-loop orchestration and ANE-specific workflows.

Python SDK

PMetal exposes a Python extension module via PyO3. Install with maturin develop from crates/pmetal-py.

Quick Start (Easy API)

import pmetal

# Fine-tune with sensible defaults
result = pmetal.finetune(
    "Qwen/Qwen3-0.6B",
    "train.jsonl",
    lora_r=16,
    learning_rate=2e-4,
    epochs=3,
)
print(f"Loss: {result['final_loss']}, Steps: {result['total_steps']}")

# Inference
text = pmetal.infer("Qwen/Qwen3-0.6B", "What is 2+2?")
print(text)

# Inference with LoRA adapter
text = pmetal.infer(
    "Qwen/Qwen3-0.6B",
    "Explain quantum entanglement",
    lora="./output/lora_weights.safetensors",
)

Full Control

import pmetal

# Configure training components
lora_config = pmetal.LoraConfig(r=16, alpha=32.0)
training_config = pmetal.TrainingConfig(
    learning_rate=2e-4,
    num_epochs=3,
    batch_size=4,
    max_seq_len=2048,
)

# Create trainer
trainer = pmetal.Trainer(
    model_id="Qwen/Qwen3-0.6B",
    lora_config=lora_config,
    training_config=training_config,
    dataset_path="train.jsonl",
)
trainer.add_callback(pmetal.ProgressCallback())
result = trainer.train()

# Load model for inference
model = pmetal.Model.load("Qwen/Qwen3-0.6B")
print(model.generate("Hello world", temperature=0.7))

Installation

Prebuilt signed binaries are available on the Releases page.

Crates are available on crates.io.

Build from source:

git clone https://github.com/epistates/pmetal.git && cd pmetal
cargo build --release          # CLI + TUI
cd crates/pmetal-gui && bun install && bun tauri build  # GUI (optional)

Hardware Support

PMetal automatically detects Apple Silicon capabilities at startup and tunes kernel parameters accordingly.

Auto-detected features: GPU family, device tier, core counts, memory bandwidth, dynamic caching, mesh shaders, NAX (M5+), UltraFusion topology (via sysctl hw.packages), ANE availability.

Tier-based kernel tuning: Matrix tile sizes, FlashAttention block sizes, fused kernel threadgroup sizes, and batch multipliers are automatically selected based on device tier (Base/Pro/Max/Ultra) and GPU family. See docs/hardware-support.md for the full tuning matrix.

Architecture

PMetal is organized as a Rust workspace with 20 specialized crates:

pmetal/
├── pmetal-bridge       # Zero-allocation MLX C++ bridge (inline array FFI)
├── pmetal-core         # Foundation: configs, traits, types, error handling
├── pmetal-core-derive  # Derive macros for the core traits
├── pmetal-metal        # Custom Metal GPU kernels + ANE runtime
├── pmetal-mlx          # MLX backend integration (KV cache, RoPE, etc.)
├── pmetal-models       # LLM architectures (Llama, Qwen, DeepSeek, etc.)
├── pmetal-lora         # LoRA/QLoRA training implementations
├── pmetal-trainer      # Training loops (SFT, DPO, SimPO, ORPO, KTO, GRPO, etc.)
├── pmetal-data         # Dataset loading, chat templates, tokenization
├── pmetal-hub          # HuggingFace Hub integration + model fit estimation
├── pmetal-distill      # Knowledge distillation losses, offline caches, and TAID
├── pmetal-merge        # Model merging (15 strategies)
├── pmetal-gguf         # GGUF format with imatrix quantization
├── pmetal-mhc          # Manifold-Constrained Hyper-Connections
├── pmetal-distributed  # Distributed training (mDNS, Ring All-Reduce)
├── pmetal-vocoder      # BigVGAN neural vocoder
├── pmetal-serve        # OpenAI- and Anthropic-compatible inference server
├── pmetal-mcp          # MCP server (51 tools for Claude Desktop)
├── pmetal-py           # Python bindings (maturin/PyO3)
├── pmetal              # Umbrella crate: CLI binary + TUI control center
└── pmetal-gui          # Desktop GUI (Tauri + Svelte + TailwindCSS)

The pmetal crate is both the umbrella library — it re-exports every sub-crate behind a feature flag — and the binary that ships the CLI and TUI.

Supported Models

Inference (via DynamicModel dispatcher)

All causal language models below can be loaded from HuggingFace Hub or local safetensors and used for generation via the CLI, TUI, GUI, or SDK.

Gemma 4 MTP assistant checkpoints (model_type = "gemma4_assistant") are supported as draft assistants via pmetal infer --draft-model <assistant>. The standard sampling controls (--temperature, --top-k, --top-p, --min-p, and penalties) are preserved through speculative verification, so MTP does not change generation quality.

Qwen3Next/Qwen3.6 checkpoints with bundled mtp.* weights can use exact speculative decoding via pmetal infer --mtp --mtp-draft-tokens 3. This path preserves the same sampling controls, supports bundled multi-predictor MTP heads, works with FP8 target/MTP weights, packed expert offload, and LoRA-merged targets, and verifies every drafted token against the target model. LoRA and packed expert offload are separate modes; fuse the adapter first if you need both together. MTP inference prints draft acceptance metrics, including accepted/attempted draft tokens, accepted tokens per verify step, and target bonus/correction tokens.

Custom draft checkpoint creation is wired through pmetal train-mtp and pmetal train-draft. train-mtp exports HF-compatible Gemma 4 assistant checkpoints and Qwen mtp.* predictor checkpoints; load the latter with pmetal infer --mtp --mtp-model ./qwen-mtp. train-draft exports DFlash draft checkpoints for the dedicated pmetal dflash runtime.

Embedding / Encoder Models

LoRA/QLoRA Training Support

LoRA training is supported for models that have implementations in DynamicLoraModel. Architecture detection is automatic — just point pmetal train at a model directory or HuggingFace ID.

Architecture Modules (Not Yet in Dispatcher)

The following architectures have implementations in pmetal-models but are not wired into the DynamicModel dispatcher and cannot be loaded via the CLI or DynamicModel::load():

These modules can be used directly via their Rust types (e.g., pmetal_models::architectures::pixtral::Pixtral) but require manual weight loading.

Diffusion Models

Training Methods

All training methods support callback-based cancellation (should_stop()), metrics JSONL logging, and adaptive learning rate control.

Additional methods available via the library only: GSPO (GspoTrainer), PPO (PpoTrainer), Online DPO (OnlineDpoTrainer).

Key Features

Metal GPU Optimizations

Custom Metal shaders provide significant speedups:

• FlashAttention: O(n) memory attention with fused softmax, tier-aware block sizes
• Fused GDN: Gated Delta Network recurrence kernel (ported from FLA Triton) — single-pass state update with SIMD reductions
• Fused LoRA: Combined forward pass for adapter layers (~2x speedup with lora-metal-fused feature)
• Fused Cross-Entropy: Chunked vocabulary loss computation
• Fused Linear Cross-Entropy: Skips logits materialization entirely
• Fused RoPE: Rotary position embeddings in-kernel
• Fused SwiGLU: Fused gate + activation with tier-tuned threadgroups
• Fused RMSNorm + LoRA: Combined normalization and adapter projection
• Fused Sampler: JIT-compiled token sampling
• Fused MLP: Combined gate/up/down projections
• Async Scheduler: Double/triple-buffered GPU command scheduling

ANE (Neural Engine) Pipeline

Native ANE integration for power-efficient training and inference:

• Dynamic Weight Pipeline: 9 MIL kernels compiled once at startup; weights packed alongside activations in IOSurface spatial dimension
• Hybrid Inference: ANE prefill + CPU decode with KV cache. Power-of-2 sequence bucketing for optimal kernel compilation
• CPU RMSNorm: RMSNorm computed in f32 on CPU to avoid fp16 overflow on ANE (saturation arithmetic)
• IOSurface Zero-Copy: fp32 shared memory surfaces for CPU-ANE data transfer with no serialization overhead
• M1-M5 Compatibility: Per-matrix weight blobs for M1, single-blob for M3+. CPU FFN fallback for 4B+ models

TurboQuant KV Cache

Near-optimal KV cache compression for long-context inference:

• Random rotation + Lloyd-Max quantization: 4-6x cache compression with near-zero quality loss
• Mixed-precision presets: q3_5 (near-lossless), q2_5 (6.4x compression)
• QJL residual correction: Unbiased inner product estimates via Johnson-Lindenstrauss random projection
• Direct attention path: Single-token decode avoids full cache dequantization
• Data-oblivious: No calibration data required — quantizes online as KV entries are generated

Training Infrastructure

• Sequence Packing: Efficiently pack multiple sequences into single batches for 2-5x throughput. Enabled by default
• Gradient Checkpointing: Trade compute for memory on large models with configurable layer grouping
• Adaptive LR: EMA-based anomaly detection with spike recovery, plateau reduction, and divergence detection
• Callback System: TrainingCallback trait with lifecycle hooks (on_step_start, on_step_end, should_stop) for metrics logging, progress reporting, and clean cancellation
• Checkpoint Management: Save and resume training from checkpoints with best-loss rollback
• Tool/Function Calling: Chat templates with native tool definitions for Qwen, Gemma 4, Llama 3.1+, Mistral v3+, and DeepSeek
• Schedule-Free Optimizer: Memory-efficient optimizer without learning rate schedules
• Metal Fused Optimizer: GPU-accelerated AdamW parameter updates
• 8-bit Adam: Memory-efficient optimizer for large models
• LoRA+: Differentiated learning rates for LoRA A and B matrices
• NEFTune: Noise-augmented fine-tuning for improved generation quality
• Distributed Training: mDNS auto-discovery, Ring All-Reduce with gradient compression

Dataset Formats

Auto-detected training data formats:

• ShareGPT: {"conversations": [{"from": "human", "value": "..."}, ...]}
• Alpaca: {"instruction": "...", "input": "...", "output": "..."}
• OpenAI/Messages: {"messages": [{"role": "user", "content": "..."}, ...]}
• Reasoning: {"problem": "...", "thinking": "...", "solution": "..."}
• Simple: {"text": "..."}
• Parquet: Supports both standard text columns and reasoning formats

Custom columns: Use --text-column for arbitrary field names, --text-columns col1,col2 to concatenate multiple columns, and --prompt-column/--response-column for SFT loss masking. All training commands (train, distill, grpo, rlkd) support column flags uniformly.

The pmetal dataset subcommand provides utilities for analysis, download from HuggingFace, and format conversion (Parquet, JSON, JSONL, CSV, ShareGPT, Alpaca).

Model Operations

• HuggingFace Hub Search: pmetal search with memory fit estimation and download

• Model Merging (15 strategies via MergeConfig, 12 via CLI):

  CLI	Library	Description
  linear	LinearMerge	Simple weighted averaging
  slerp	SlerpMerge	Spherical linear interpolation
  ties	TiesMerge	Task arithmetic with sparsification and sign consensus
  dare_ties	DareMerge	Random pruning with rescaling (TIES variant)
  dare_linear	DareMerge	Random pruning with rescaling (linear variant)
  task_arithmetic	TaskArithmeticMerge	Task vector arithmetic
  della	DellaMerge	Adaptive magnitude-based pruning
  della_linear	DellaMerge	Adaptive magnitude pruning (linear variant)
  breadcrumbs	BreadcrumbsMerge	Breadcrumbs merge strategy
  model_stock	ModelStockMerge	Geometric interpolation based on task vector similarity
  nearswap	NearswapMerge	Near-swap merge strategy
  passthrough	PassthroughMerge	Layer passthrough composition
  —	RamMerge	RAM merge strategy
  —	SouperMerge	Souper merge strategy
  —	MultiSlerpMerge	Multi-model SLERP

• GPU-Accelerated Merging: Metal-based merge operations for large models

• FP8-Aware Merging: Merge with FP8 quantization for memory efficiency

• Async Merge Pipeline: Double-buffered streaming merge for large models

• LoRA Fusing: Merge LoRA adapters into base weights (standard and accurate modes)

• GGUF Quantization (13 format options):

  Format	Description
  dynamic	Auto-select per layer
  q8_0	8-bit quantization
  q6k	6-bit k-quant
  q5km	5-bit k-quant (medium)
  q5ks	5-bit k-quant (small)
  q4km	4-bit k-quant (medium)
  q4ks	4-bit k-quant (small)
  q3km	3-bit k-quant (medium)
  q3ks	3-bit k-quant (small)
  q3kl	3-bit k-quant (large)
  q2k	2-bit k-quant
  f16	Float16
  f32	Float32

  Supports importance matrix (--imatrix) for improved quantization quality. KL-calibrated quantization (--kl-calibrate) selects per-tensor quantization types via NRMSE + cosine distance, with optional --target-bpw for budget-constrained quantization.

• FP8 Runtime Quantization: Convert to FP8 (E4M3) at inference time for ~2x memory reduction

Knowledge Distillation

Multiple distillation methods and loss functions:

• Methods: Online (live teacher inference), Offline (cached logits with compression), Progressive
• TAID: Temporally Adaptive Interpolated Distillation (ICLR 2025 SOTA) — TaidDistiller
• Token-Level Losses: KL Divergence, Jensen-Shannon, Soft Cross-Entropy, TVD, Hinge Ranking, Logistic Ranking
• Hidden State Losses: MSE, Cosine similarity, L1
• Reasoning-Aware: Rationale distillation for reasoning models
• Cross-Vocabulary: Distill between models with different tokenizers
• Offline Logit Caching: Compressed logit storage for memory-efficient offline distillation

Configuration

pmetal train Parameters

pmetal infer Parameters

Feature Flags

Defaults are cli, dashboard, trainer, lora, merge, ane and distributed; the rest are pulled in transitively.

serve and mcp stay out of the default set so library consumers don't inherit axum and rmcp. The prebuilt binary and the Homebrew formula both build with --features serve,mcp, so pmetal serve and pmetal mcp are there if you installed either way. Building yourself, add the flag: cargo install pmetal --features serve,mcp.

Development

Building

# Release build (default features: ANE + Dashboard)
cargo build --release

# Build without ANE
cargo build --release --no-default-features --features dashboard

# Run tests (single-threaded for Metal compatibility)
just test

# Build GUI
cd crates/pmetal-gui && bun install && bun tauri build

Formal Verification

# cargo-kani proofs for ring all-reduce and topology
just kani-verify

License

Licensed under either of MIT or Apache-2.0.

Acknowledgments

• MLX - Apple's machine learning framework
• pmetal-bridge - PMetal's Rust bridge for MLX, Metal, and runtime dispatch
• Fused kernel techniques — see THIRD_PARTY_NOTICES for attributions
• Tauri - Desktop application framework