alloy

A fast CLI for LLM model merging, format conversion, and diffing. Written in Rust. Reads safetensors natively. Compatible with mergekit configs.

Website · GitHub

Install

cargo install --path .

Requires Rust (any recent stable version).

Usage

# Merge two models
alloy merge config.yaml --output ./merged

# Compare two models tensor-by-tensor
alloy diff ./model_a ./model_b

# Convert dtype (e.g. FP32 to FP16)
alloy convert ./model --output ./model_f16 --dtype f16

# Inspect model metadata
alloy info ./model

Merge methods

Config format

alloy reads mergekit-compatible YAML configs:

merge_method: slerp
base_model: mistralai/Mistral-7B-v0.1
models:
  - model: mistralai/Mistral-7B-v0.1
  - model: teknium/OpenHermes-2.5-Mistral-7B
parameters:
  t: 0.5
dtype: float16

HuggingFace model IDs work directly. Models download to ~/.cache/huggingface/hub/. Set HF_TOKEN for gated models (Llama, Gemma, etc.). PyTorch .bin files are auto-converted on first use.

Benchmarks

Real models on Azure Standard_E48s_v5 (48 cores, 384 GB RAM). All times are wall-clock (hyperfine, 3 runs with warmup).

7B - Mistral-7B-v0.1 (BF16, 14.48 GB)

14B - Qwen2.5-14B (BF16, 29.54 GB)

alloy uses fused SIMD kernels (AVX2/NEON) that read BF16 directly from memory-mapped safetensors and compute in f32 registers, avoiding intermediate allocations. IO-bound methods (linear, slerp) are consistently faster. Compute-heavy methods (ties, dare_ties) are still slower due to PyTorch's optimized C tensor kernels.

See alloy.how for the full technical writeup including architecture, algorithm breakdowns, and future work.

License

MIT