TalkPlayData 2: An Agentic Synthetic Data Pipeline for Multimodal Conversational Music Recommendation

By Keunwoo Choi and Seungheon Doh: Team talkpl.ai; and Juhan Nam

This dataset serves as training and evaluation data for the conversational music recommendation system presented in:

• Application Paper: TalkPlay-Tools: Conversational Music Recommendation with LLM Tool Calling
• Application Project page: https://talkpl.ai/p/talkplay_tools/index.html
• Application Code: https://github.com/talkpl-ai/talkplay-tools

---

The creation of this dataset, TalkPlayData 2, is described in its own paper:

• Dataset Creation Paper: TalkPlayData 2: An Agentic Synthetic Data Pipeline for Multimodal Conversational Music Recommendation
• Dataset Project page: https://talkpl.ai/p/talkplaydata2/index.html
• Dataset Code: https://github.com/talkpl-ai/talkplaydata-2

TalkPlayData 2 is a synthetic dataset for conversational music recommendation, that is multimodal (audio, album art, lyrics, metadata) and made by an agentic framework.

Sample Usage (with TalkPlay-Tools)

This dataset is utilized by the TalkPlay-Tools system. Below is a quick start example from the TalkPlay-Tools GitHub repository demonstrating how to run an example query using the system with test data. Due to licensing constraints, the system returns Spotify links instead of direct audio files.

python run.py --user_query "I'm looking for calm and slow tempo piano music."

Example output:

----------------------------------------------------------------------------------------------------
🎵 Music: https://open.spotify.com/track/00CXUMREit80f2McJsjcIz
🤖 Assistant Response:
I’ve found a perfect match for your request!
**"Lieder ohne Worte (Songs without Words), Book 2, Op. 30: No. 7 in E-flat major"**
by **Felix Mendelssohn**, performed by **Péter Nagy**.
This classical piano piece features a **slow tempo (65.79 BPM)**, **mellow melodies**,
and a **romantic, emotional tone** that exudes calm and introspection.

The track’s **F# major key** and **instrumental, melancholic style**
make it ideal for a relaxed, reflective mood. It’s a beautifully
crafted piece that aligns perfectly with your request for calm and slow piano music.

Would you like to explore similar tracks, or need recommendations for different moods?
I’m here to help! 🎹
----------------------------------------------------------------------------------------------------
More detail results (Chain of Thought / Tool Calling / Response) are saved in ./demo/static

Overall Statistics

This table shows the distribution of unique tracks across training and test sets, including the breakdown of warm-start (tracks seen during training) and cold-start (unseen tracks) scenarios in the test set.

Conversation Goal Distribution

These tables show the distribution of conversation goals across different categories and specificity levels. The first table categorizes conversations by their primary discovery mechanism (e.g., artist-based, lyric-based), while the second table categorizes them by the specificity of both the user's query and the target recommendation.

Category Examples

• A - Audio-Based Discovery: "Discover songs with immersive soundscapes"
• B - Lyrical Discovery: "Songs about love"
• C - Visual-Musical Connections: "Music that looks colorful and vibrant"
• D - Contextual & Situational: "Music for working and studying"
• E - Interactive Refinement: "Let's play hard rock and transit to modern rock"
• F - Metadata-Rich Exploration: "Find multiple songs from the Hamilton musical"
• G - Mood & Emotion-Based: "I need something to cheer me up"
• H - Artist & Discography Discovery: "Tell me about this artist's other works"
• I - Cultural & Geographic: "Music from Alaska"
• J - Social & Popularity Context: "What's trending right now?"
• K - Temporal & Era Discovery: "Music from the 80s please"

Specificity Categories

The specificity dimension captures how precise both the user's query and the target recommendation are:

• LL (Low query, Low target): User asks vague questions and receives broad recommendations
  • Example: "Play some chill music" (Many tracks are possible as a successful recommendation)
• HL (High query, Low target): User provides specific details but receives general recommendations
  • Example: "Find bebop jazz with saxophone, 1950s-60s" (Many tracks are possible, the query is somewhat specific)
• LH (Low query, High target): User asks general questions but receives very specific recommendations
  • Example: "What was the popular song from a recent musical movie?" (One or few tracks are possible, the query is not specific)
• HH (High query, High target): Both the user's query and the recommendation are highly specific
  • Example: "Windup by Hayoung Lyou, the jazz composer and pianist" (One track is possible, the query is highly specific)

User Demographics Distribution

These tables show the distribution of synthetic user profiles across gender, age groups, and countries. Each conversation is associated with a unique user profile that influences the conversation style and preferences.

Gender Distribution

The training set shows a notable gender imbalance with approximately 72% male and 28% female users, which may reflect historical biases in music streaming platform demographics or data collection. However, the test set is nearly balanced (50.6% male, 49.4% female), providing a more equitable evaluation benchmark. Researchers should be aware that models trained on this data may exhibit some bias towards male user interaction patterns, though the balanced test set allows for fair performance assessment across genders.

Age Group Distribution

The age distribution in the dataset shows a notable bias towards younger users, particularly those in their 20s, which reflects typical music streaming platform demographics. Key observations:

• 10s age group has substantial representation (~22% train, ~36% test), capturing younger users' music discovery patterns
• 20s age group represents the majority (~64% train, ~40% test), aligning with the primary demographic of active music streaming users
• 30s and older are progressively underrepresented, with 40s+ comprising less than 4% of the dataset

This distribution mirrors real-world music streaming platform usage patterns, where younger users tend to be more active in conversational music discovery. However, researchers should be aware that model performance may be optimized for younger user interactions and may require additional data or fine-tuning for older demographic groups.

Country Distribution

The dataset includes conversations from users across 20 countries, representing diverse musical cultures and preferences. The distribution reflects the global nature of music streaming platforms, with the United States and Brazil having the largest representation, followed by European countries like Poland, United Kingdom, and Germany. This geographic diversity ensures the dataset captures varied musical tastes, cultural contexts, and language preferences in conversational music recommendation scenarios.

Total: 15,199 training conversations, 1000 test conversations

Dataset Updates logs

• 2.0.0-rc.1: First release candidate after fixing some issues
  • 2024-10-07:
    • Removed conversations with unclear image or audio upload queries.
      • This resulted in reducing the train split: 15,500 → 15,199 conversations
      • Test split remains as the same size (1,000 conversations) as the 33 affected items are re-generated with a slightly modified prompt that prevents the issue.
• 2.0.0-beta: Our ICLR2026 submission version
  • 2024-10-06: Train split release (15,500 conversations)
  • 2024-09-28: Initial test split release (1,000 conversations)

Citation

If this dataset is used with the TalkPlay-Tools framework or for research related to conversational music recommendation using LLM tool calling, please cite the associated paper:

@misc{choi2025talkplaytoolsconversational,
      title={TalkPlay-Tools: Conversational Music Recommendation with LLM Tool Calling},
      author={Keunwoo Choi and Seungheon Doh and Juhan Nam},
      year={2025},
      eprint={2510.01698},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2510.01698},
}

If you are primarily interested in the dataset generation methodology or using the raw dataset, please cite the TalkPlayData 2 paper:

@misc{choi2025talkplaydata2agenticsynthetic,
      title={TalkPlayData 2: An Agentic Synthetic Data Pipeline for Multimodal Conversational Music Recommendation},
      author={Keunwoo Choi and Seungheon Doh and Juhan Nam},
      year={2025},
      eprint={2509.09685},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2509.09685},
}