Stems

High-quality audio stem separation using Demucs v4 via ONNX Runtime. Separate audio tracks into vocals, drums, bass, and other instruments with true multi-core parallelisation.

Features

• High Quality: State-of-the-art Demucs v4 (htdemucs) model with 9.0+ dB SDR
• Fast Processing: Multi-threaded C++ implementation, no Python GIL limitations
• Multiple Interfaces:
  • CLI for developers and automation
  • Web UI for general use (local or remote)
  • Desktop GUI (planned)
• Flexible Deployment:
  • Run locally for privacy and offline use
  • Deploy as web service on VPS
  • Queue system for batch processing
• 4-Stem Separation: Vocals, drums, bass, other

Architecture

┌─────────────┐
│   Web UI    │  HTML/CSS/JS (works local and remote)
└──────┬──────┘
       │ HTTP/WebSocket
┌──────┴──────┐
│  API Server │  Crow C++ HTTP server
└──────┬──────┘
       │
┌──────┴──────┐
│ C++ Backend │  ONNX Runtime + Demucs model
└─────────────┘  Multi-threaded processing

Library Comparison

Selection Rationale: Demucs v4 (htdemucs) chosen for:

• Highest quality separation (9.2+ dB SDR on MUSDB18)
• True multi-core parallelisation via ONNX Runtime C++ (no Python GIL)
• Hybrid architecture (time-domain + transformer) handles phase information better
• Active ONNX conversion support from Mixxx project

Technology Stack

• Backend: C++23 (tracking latest standard) with ONNX Runtime
• Platforms: macOS and latest Linux (no Windows support)
• Build System: CMake with top-level Makefile wrapper
• Models: Demucs v4 (htdemucs) exported to ONNX
• Audio I/O: libsndfile (WAV, FLAC, AIFF - lossless only)
• GPU Support: CUDA (optional), Metal/CoreML on macOS
• CPU Optimisation: Thread pool, parallel batch processing

Quick Start

CLI Usage

# Single file
stems input.wav --output ./stems

# Batch processing
stems *.wav --batch --jobs 4

# Quality/speed trade-off
stems input.wav --model htdemucs      # best quality (slower)
stems input.wav --model mdx_extra     # faster, good quality

Web Service (planned)

# Start local server
stems-server --port 8080

# Visit http://localhost:8080
# Drag and drop audio files, download stems

Deploy to VPS (planned)

# Docker deployment
docker pull deanturpin/stems
docker run -p 8080:8080 deanturpin/stems

# Or systemd service
make install-service

Building from Source

Prerequisites

Requires latest C++ compiler with C++23 support.

# macOS
brew install cmake onnxruntime libsndfile fftw

# Latest Ubuntu/Debian
apt install cmake libonnxruntime-dev libsndfile1-dev libfftw3-dev g++-13

# Arch Linux (rolling release - always latest)
pacman -S cmake onnxruntime libsndfile fftw

Build

git clone https://github.com/deanturpin/stems.git
cd stems
make

Get the Model

The Demucs model must be converted from PyTorch to ONNX format (not included in git due to size):

# Automated conversion (recommended)
./scripts/download_model.sh

# Or follow manual steps in models/README.md

This downloads and converts the model (~300MB, takes 5-10 minutes).

Project Structure

stems/
├── src/              # Core C++ stem separation logic and CLI
│   ├── main.cxx
│   ├── stem_processor.cxx
│   └── onnx_wrapper.cxx
├── include/          # Public headers
│   ├── stem_processor.h
│   └── onnx_wrapper.h
├── models/           # ONNX model files (download separately)
│   └── htdemucs.onnx
├── tests/            # Unit tests
├── CMakeLists.txt    # CMake build configuration
└── Makefile          # Top-level build wrapper

Performance

CPU-Only (8-core VPS)

• 4-minute song: ~5-8 minutes (htdemucs)
• 4-minute song: ~3-5 minutes (mdx_extra)
• Parallel batch: 3 jobs concurrently

With GPU (CUDA)

• 4-minute song: ~30-60 seconds
• 10-50x speedup over CPU

Roadmap

• Research and select best model (Demucs v4)
• C++ backend with ONNX Runtime integration
• CLI interface
• HTTP API server with job queue
• Web UI with drag-and-drop
• Docker deployment
• Model download and management
• Progress tracking via WebSocket
• Email notifications for completed jobs
• File size and rate limiting
• Desktop GUI (Qt)
• Extended stem separation (piano, guitar - via UVR models)
• GPU acceleration support

Contributing

Contributions welcome! Please read CONTRIBUTING.md first.

Licence

MIT Licence - see LICENCE for details.

Documentation

• Signal Processing Options - FFT library comparison and STFT implementation strategies
• Model Setup - How to obtain and convert ONNX models

Acknowledgements

• Demucs by Meta Research for the outstanding separation model
• ONNX Runtime for cross-platform ML inference
• Mixxx GSoC 2025 for Demucs ONNX conversion work
• sevagh/demucs.onnx and sevagh/demucs.cpp for ONNX and C++ reference implementations
• Ultimate Vocal Remover community for model research

References

• Demucs Paper - Hybrid Spectrogram and Waveform Source Separation
• ONNX Runtime Documentation
• Audio Source Separation Benchmarks