A Semantically Consistent Dataset for Data-Efficient Query-Based Universal Sound Separation

Kai Li^*, Jintao Cheng^*, Chang Zeng, Zijun Yan, Helin Wang, Zixiong Su, Bo Zheng, Xiaolin Hu
Tsinghua University, Shanda AI, Johns Hopkins University
^*Equal contribution
📜 Arxiv 2026 | 💻 Code | 🎶 Demo

Usage

from datasets import load_dataset

# Load full dataset
dataset = load_dataset("ShandaAI/Hive")

# Load specific split
train_data = load_dataset("ShandaAI/Hive", split="train")

# Streaming mode (recommended for large datasets)
dataset = load_dataset("ShandaAI/Hive", streaming=True)

📄 Dataset Description

Hive is a high-quality synthetic dataset designed for Universal Sound Separation (USS). Unlike traditional methods relying on weakly-labeled in-the-wild data, Hive leverages an automated data collection pipeline to mine high-purity single-event segments from complex acoustic environments and synthesizes mixtures with semantically consistent constraints.

Key Features

• Purity over Scale: 2.4k hours achieving competitive performance with million-hour baselines (~0.2% data scale)
• Single-label Clean Supervision: Rigorous semantic-acoustic alignment eliminating co-occurrence noise
• Semantically Consistent Mixing: Logic-based co-occurrence matrix ensuring realistic acoustic scenes
• High Fidelity: 44.1kHz sample rate for high-quality audio

Dataset Scale

Dataset Splits

---

📂 Dataset Structure

Directory Organization

hive-datasets-parquet/
├── README.md
├── train/
│   └── data.parquet
├── validation/
│   └── data.parquet
└── test/
    └── data.parquet

Each split contains a single Parquet file with all mixture metadata. The num_sources field indicates the number of sources (2-5) for each mixture.

---

📋 Data Fields

JSON Schema

Each JSON object contains complete generation parameters for reproducing a mixture sample:

{
    "mix_id": "sample_00000003",
    "split": "train",
    "sample_rate": 44100,
    "target_duration": 4.0,
    "num_sources": 2,
    "sources": {
        "source_id": ["s1", "s2"],
        "path": ["relative/path/to/audio1", "relative/path/to/audio2"],
        "label": ["Ocean", "Rain"],
        "crop_start_second": [1.396, 2.5],
        "crop_end_second": [5.396, 6.5],
        "chunk_start_second": [35.0, 20.0],
        "chunk_end_second": [45.0, 30.0],
        "rms_gain": [3.546, 2.1],
        "snr_db": [0.0, -3.0],
        "applied_weight": [3.546, 1.487]
    },
    "global_normalization_factor": 0.786,
    "final_max_amplitude": 0.95
}

Field Descriptions

1. Basic Info Fields

2. Source Information (sources)

Metadata required to reproduce the mixing process for each audio source. Stored in columnar format (dict of lists) for efficient Parquet storage:

3. Mixing Parameters

Global processing parameters after combining multiple audio sources:

Detailed Field Explanations

Cropping Logic

• chunk_start/end_second: Defines the reading interval from the original audio file
• crop_start/end_second: Records the precise random cropping position, ensuring exact reproducibility across runs

Energy Normalization (rms_gain)

Adjusts different audio sources to the same energy level: $$\text{rms_gain} = \frac{\text{target_rms}}{\text{current_rms}}$$

Signal-to-Noise Ratio (snr_db)

The SNR value assigned to each source, sampled from a predefined range using random.uniform(snr_range[0], snr_range[1]).

Applied Weight

The comprehensive scaling weight combining energy normalization and SNR adjustment: $$\text{applied_weight} = \text{rms_gain} \times 10^{(\text{snr_db} / 20)}$$

This is the final coefficient applied to the original waveform.

Global Normalization Factor

Prevents audio clipping after mixing: $$\text{global_normalization_factor} = \frac{0.95}{\text{max_val}}$$

Where max_val is the peak amplitude (absolute value) of the mixed signal.

---

🔧 Usage

Download Metadata

from datasets import load_dataset

# Load specific split and mixture type
dataset = load_dataset("ShandaAI/Hive", split="train")

Generate Mixed Audio

Please refer to the official GitHub repository for the complete audio generation pipeline.

# Clone the repository
git clone https://github.com/ShandaAI/Hive.git
cd Hive/hive_dataset

# Generate mixtures from metadata
python mix_from_metadata/mix_from_metadata.py \
    --metadata_dir /path/to/downloaded/metadata \
    --output_dir ./hive_dataset \
    --dataset_paths dataset_paths.json \
    --num_processes 16

---

📚 Source Datasets

Hive integrates 12 public datasets to construct a long-tailed acoustic space:

Important Note: This repository releases only metadata (JSON files containing mixing parameters and source references) for reproducibility. Users must independently download and prepare the source datasets according to their respective licenses.

---

📖 Citation

If you use this dataset, please cite:

@article{li2026hive,
  title={A Semantically Consistent Dataset for Data-Efficient Query-Based Universal Sound Separation},
  author={Li, Kai and Cheng, Jintao and Zeng, Chang and Yan, Zijun and Wang, Helin and Su, Zixiong and Zheng, Bo and Hu, Xiaolin},
  journal={arXiv preprint arXiv:2601.22599},
  year={2026}
}

---

⚖️ License

This dataset metadata is released under the Apache License 2.0.

Please note that the source audio files are subject to their original licenses. Users must comply with the respective licenses when using the source datasets.

---

🙏 Acknowledgments

We extend our gratitude to the researchers and organizations who curated the foundational datasets that made Hive possible:

• BBC Sound Effects - Professional-grade recordings with broadcast-level fidelity
• AudioSet (Google) - Large-scale audio benchmark
• VGGSound (University of Oxford) - Real-world acoustic diversity
• FreeSound (MTG-UPF) - Rich crowdsourced soundscapes
• And all other contributing datasets

---

📬 Contact

For questions or issues, please open an issue on the GitHub repository or contact the authors.