TrueMuse: A Benchmark for Data Attribution in Text-to-Music Models

Paper (arXiv) | Code (GitHub)

TrueMuse evaluates training-data attribution for text-to-music generation. Each attribute (a musician, an instrument, a genre or a melody) is injected into a generator through DreamBooth fine-tuning on a small set of source clips, so the source of every generated sample is known. Attribution methods are asked to retrieve those source clips from the generated audio, among a large pool of distractors.

This repository contains the data. Fine-tuning, generation and evaluation code is on GitHub.

---

Download

The data is split into 8 zip archives (about 150 GB in total).

hf download AnonymousAuthorsssss/TrueMuse --repo-type dataset --local-dir <dataset_root>
cd <dataset_root>
unzip -q truemuse_concepts.zip -d data                                                # no data/ prefix
unzip -q truemuse_generated_stable_audio_musician.zip -d data/generated/stable-audio    # no data/ prefix
for f in truemuse_generated_audioldm2.zip truemuse_generated_mustango.zip \
         truemuse_generated_stable_audio_{genre,instrument,melody}.zip truemuse_embeddings_jamendo.zip; do
    unzip -q "$f"
done

The instrument source audio is not included; rebuild it from YouTube with scripts/download_instruments.py from the code repository.

---

Structure

<dataset_root>/data/
├── concepts/                       # source clips used for fine-tuning
│   ├── musician_6/                 #   50 musicians: composer_XXX/song{1,2}_{1,2,3}.wav
│   ├── musician_3/                 #   same, song1_{1,2,3}.wav only
│   ├── genre_3/     genre_6/       #   8 genres: <genre>/clip_0X.wav
│   ├── melody_1/                   #   50 melodies: melody_XXX/clip_01.wav (+ class_name.json)
│   └── metadata_instruments.csv    #   25 instruments: YouTube ID and start time (no audio)
├── generated/                      # generated audio (the attribution queries)
│   ├── audioldm2/
│   ├── mustango/
│   └── stable-audio/
│       └── <task>/<concept>_{ldm2,mustango,sa}_db/
│           ├── contextual/*.wav
│           ├── style/*.wav
│           └── metadata.csv        #   prompt_type, prompt_id, prompt, file
└── embeddings/jamendo/             # distractor embeddings
    └── {mert-v0, mert-v0-public, music2vec-v1, dac-16k}/pool.npz

<task> is one of musician_3, musician_6, instrument_3, instrument_6, genre_3, genre_6, melody_1. The 3-clip sets use the first 3 clips of the corresponding 6-clip sets.

Source clips (data/concepts/)

• Musician: 50 artists from the Free Music Archive (FMA) small split; two tracks per artist, three consecutive 10 s clips per track, vocals removed with Demucs.
• Genre: 8 FMA genres (electronic, experimental, folk, hip_hop, jazz, pop, punk, rock); each clip from a different track of the genre, vocals removed with Demucs.
• Melody: 50 FMA tracks, one 10 s clip each.
• Instrument: 25 instruments. metadata_instruments.csv gives the YouTube ID and start_sec; the clips are 6 consecutive 10 s segments starting at start_sec, and the 3-clip set uses the first 3.

concept_licenses.csv (in the root of this repository) lists, for every source clip, its FMA track ID, artist, title and license (or its YouTube source for instruments).

Generated audio (data/generated/)

Each fine-tuned model generates 4 waveforms per prompt from two prompt types: contextual prompts place the attribute in a richer musical scene, style prompts request it in a given genre (15 contextual + 28 style prompts; Stable Audio uses 15 + 8, and 15 + 28 for genre). Clips are about 10 s long (10.24 s for Mustango): 16 kHz mono for AudioLDM2 and Mustango, 44.1 kHz stereo for Stable Audio Open. The metadata.csv in each concept folder records the prompt of every clip.

Distractor embeddings (data/embeddings/jamendo/)

pool.npz holds emb (228,598 x D, float32, L2-normalised; D = 9,984 for MERT-v0, MERT-v0-public and Music2Vec-v1, 12,288 for DAC-16k) and files, the source of each row as <track>.mp3@<offset>s, from JamendoMaxCaps (non-overlapping 10 s segments).

---

Evaluation protocol

Attributes are split 70 / 15 / 15 into train / validation / test (seed 42, shared across encoders and generators). For each test query, the gallery holds the source clips of all test attributes of the same task (for example musician_3) plus the 228,598 distractors. R@k is the fraction of the query's own source clips ranked in the top k; mAP is also reported. See the code repository for scripts.

---

Licensing

TrueMuse is meant for non-commercial research. Following FMA, we do not hold the copyright on the source audio and distribute it under the license chosen by the artist.

Using the dataset does not grant any rights to the underlying recordings, compositions or performances.

Intended use

The data is intended for research on data attribution and credit assignment in music generation. Please do not use it to imitate or impersonate specific artists, for commercial style cloning, or to train models that reproduce individual artists.

Rights holders who would like an item removed can open a discussion on this page or an issue on GitHub.

Citation

@article{yu2026truemuse,
  title={TrueMuse: A Benchmark for Data Attribution in Text-to-Music Models},
  author={Yu, Jiawei and Liu, Jian},
  journal={arXiv preprint arXiv:2610.00835},
  year={2026}
}