Activation-patching prompt pairs (MusicCaps-derived)

Paper

TADA! Tuning Audio Diffusion Models through Activation Steering — https://huggingface.co/papers/2602.11910

3246 rows of (clean, corrupted) prompt pairs derived from the MusicCaps captions by swapping feature-bearing words (e.g. violin↔trumpet, female↔male, fast↔slow) using the mapping in src/preprocess/features.py.

Features covered (21): bongos, cello, drums, fast, female, flute, happy, harmonica, jazz, male, maracas, metal, opera, reggae, sad, saxophone, slow, trombone, trumpet, violin, xylophone.

Schema

• original_feature: feature being localised (filter on this for a per-feature run).
• clean_prompt: original MusicCaps caption mentioning the feature.
• corrupted_prompt: caption with feature-bearing words swapped for the counterfactual.

Use the clean prompt to collect the un-patched activations and the corrupted prompt to source the activations that get patched in. Counterfactuals come from MUSICCAPS_SWAPS_FEATURES in src/preprocess/features.py.

Quickstart

from datasets import load_dataset
ds = load_dataset('lukasz-staniszewski/patching-music-musiccaps-prompts')
violin_rows = ds['train'].filter(lambda r: r['original_feature'] == 'violin')

Regenerating from source

python src/preprocess/prepare_prompts.py \
    --input_file data/music_caps.csv \
    --limit 256 \
    --output_file data/generated_prompts.csv