Melody Matcher

Music recommendations should help us discover songs we love for reasons we can't quite explain. Melody Matcher is an attempt to learn those reasons from the music itself and people's listening behavior.

I love both HARDY's "Favorite Country Song" and Aretha Franklin's "Don't Play That Song," without being able to explain what connects them. That's the music's DNA: something the brain hears that a genre label or a "danceability" score fails to describe.

A friend who knows your taste can hear a song and immediately know you need to hear it. Train a neural network to learn those connections from the music itself and people's listening behavior, without first teaching it our vocabulary for describing songs. That's the bet behind Melody Matcher.

The ambition goes beyond another playlist. When you're obsessed with a song, find something that gives you the same feeling before you play it to death. Or reach further out and discover music you'd never have thought to look for. The thesis and the reasoning explain the idea, what we tried, and where we got stuck.

If this works, the same principle can extend to books, movies, maybe even clothes: represent the thing richly, then train a large neural network to learn how people's preferences relate to it. Music is the first test.

The project is paused, and the recommender doesn't work well. The current hypothesis is that we lack the right listening data, or enough of it, to train the model properly. Scaling the network without scaling that evidence doesn't get us anywhere. We're sharing the source and thinking so someone else can work from them.

The source uses audio embeddings and candidate-conditioned attention over a listener's library. The docs cover the historical trials, limitations, and possible next experiments. This is unfinished research; the experiments weren't rerun for this release.

In these docs, ‘I’ is Moritz Linn; ‘we’ means Moritz and his AI collaborators. Marty is Moritz's custom AI agent built on Hermes, one of those collaborators.

Read the research

• The thesis: music DNA, what better recommendations would make possible, and why the project is paused.
• The research history: what we believed → what we built and tried → what happened → what we realized → what changed next. Start here to learn from the attempts.
• Experiment index: the chronological route into detailed sampling, training, and future-test explanations.
• How the snowball works: why the collection plan changed, exactly what overlap means, and what each cohort was meant to test.
• Training experiments: the original regression task, all dataset-variant families, negative sampling, hurdle heads, and evaluation limitations.
• A possible next proof: the later exposure/return proposal, not an implemented experiment.
• Components: source navigation, expected schemas and artifacts, preexisting defects, and reader hazards.
• Data and rights: privacy and external model, music, and service restrictions.
• Provenance: what this curated snapshot includes and excludes.

Source map

No listening data, personal library exports, audio, embeddings, weights, checkpoints, databases, or experiment logs are supplied. The harness requires absent artifacts and is not a hosted demo. Its source contains unsafe artifact loaders, storage writes, and external browser requests. Read the component guide before considering execution; source examples are not tested setup instructions.

Owned source and documentation are licensed under MIT, copyright Linn Autoracing Excellence LLC. That grant does not cover external models, music, service data, or derived artifacts. In particular, the selected MERT model has a noncommercial license; see third-party notices.

Forks and research contributions are welcome. Active maintenance, timely review, and operational support are not promised. See Contributing.