QuarkAudio-HCodec: A Unified Discrete Audio Tokenizer for High-Fidelity, Multitask Audio Generation

🔊 H-Codec: A Unified, Dual-Stream Neural Audio Codec with Adaptive Frame Rate and 48kHz Support
Enabling high-fidelity, efficient, and semantically rich audio tokenization for next-generation LLM-based audio generation.

🚀 Key Highlights:

• ✅ Dual-Stream Tokenization: Separately quantizes acoustic and semantic features into independent codebooks — preserving both signal fidelity and linguistic content.
• 🔄 Dynamic Frame Rate (H-Codec-1.5): Introduces an adaptive temporal resolution mechanism built upon H-Codec-1.0, enabling variable frame rates based on content complexity.
• ⚙️ Multi-Sampling Rate (H-Codec-2.0): Extends the sampling rate from 16kHz to 48kHz under a fixed frame rate, significantly improving audio fidelity and high-frequency detail preservation.
• 🌍 Unified Foundation: Designed as a core component for multimodal LLMs, supporting diverse downstream tasks: TTS, VC, Editing, TTA, SE, and more.

📄 Paper: arXiv:2510.26372 | 🤗 Model: Hugging Face Spaces

---

📦 Overview

This project introduces H-Codec, a unified discrete audio tokenizer that integrates self-supervised learning (SSL) representations into the codec architecture to enable dual-stream (acoustic + semantic) tokenization. Unlike prior work that fuses modalities before quantization (e.g., X-Codec), H-Codec employs separate codebooks for acoustic and semantic streams, allowing independent optimization and better reconstruction quality.

We extend the original H-Codec (aka H-Codec-1.0) in UniTok-Audio (Liu et al., 2025) into two advanced variants:

These improvements significantly enhance audio fidelity, temporal efficiency, and applicability across speech, music, and general audio.

🔧 Architecture Core Components:

1. Encoder: Extracts continuous representations from waveform and SSL model (e.g., WavLM).
2. Quantizer Module: Two independent codebooks — one for acoustic details, one for semantic meaning.
3. Decoder: Reconstructs high-quality audio from discrete token sequences.

💡 H-Codec is designed as a foundational module for LLM-based audio generation, seamlessly integrating with autoregressive language models for end-to-end training and inference.

---

🎯 Quick Start: Run Inference in 3 Minutes

1. Clone Repository

git clone https://github.com/alibaba/unified-audio.git
cd QuarkAudio-HCodec

2. Create a Conda environment and install dependencies

conda create -n unise python=3.10
conda activate unise
pip install -r requirements.txt

3. Tokenizer

#!/bin/bash
python audio_tokenizer.py