What this is
Shazam answers "what song is this, exactly?" This answers the question next door: "what's close to this?" — with closeness measured the way a musician hears it, tolerant of different keys, substituted chords, and rewritten melodies. It's a search engine where the query is music and the ranking function is "sounds like."
It has three parts, and you just used all of them: a chord encoder that turns a progression into a vector, a melody encoder that does the same for a tune, and a shared space that lets the two talk to each other. Everything below is built and trained from a cold start — no pretrained backbone anywhere — in plain PyTorch on a Mac.
The gap it fills
Before building anything I ran a systematic literature and product sweep. The neighboring cells are all occupied: exact chord-tab search exists at scale (Chordify's 36M crowd-tabs, Chord Genome's 500k), audio cover-song embeddings are strong (Sony's CLEWS, ICML 2025), melody-plagiarism matching is a commercial product (MIPPIA). But a similarity embedder over symbolic chord progressions — a model that maps a progression to a vector so nearby vectors are musically related progressions — didn't turn up anywhere: no paper, no repo, no product. And Chordonomicon, a public corpus of 679,807 progressions, had been out since 2024 with no retrieval model trained on it. The corpus existed; the model didn't.
How it works
The chord encoder
Chords become root motion — the interval each root moves, not its absolute letter — plus factored features (quality, bass interval, a 12-d pitch-class set). A progression in C and the same progression in F♯ produce identical tokens; substituting an extension perturbs the input only slightly, by construction. A 2.3M-parameter, 4-layer transformer reads these and mean-pools to one L2-normed 256-d vector. It's trained with contrastive InfoNCE: asymmetric crops of the same song must embed near each other, other songs in the batch must not, plus music-theory augmentations (extension jitter, relative substitution, passing chords) and all-12 transposition. Underneath it sits a hand-written parser for the corpus's notation dialect (s means sharp — except before us, so Fs7 but Csus4) covering 99.999998% of 52M chord tokens.
The melody encoder
Same idea, different alphabet. A tune becomes seven parallel channels — signed interval from the previous note (transposition-invariant), scale degree, inter-onset interval, duration, rest, register (relative to the tune's own range, not absolute pitch — otherwise it silently becomes a corpus label), metric position — summed into a token per note and read by a 7.5M transformer. It's first mask-pretrained on 1.6M melodies drawn from four corpora (GigaMIDI, MelodyHub, PDMX, The Session) — predicting deliberately hidden notes teaches the shape of melody before any similarity objective is applied — then contrastively fine-tuned on Hooktheory. It learns melodic style and contour: querying it clusters punk with punk and game/anime themes with each other, and pulls other sections of the same song toward the top, all with no genre labels anywhere.
The shared space
Every Hooktheory section carries both a melody and its chords — naturally paired data. Freezing both encoders and training two small projection heads (CLIP-style) pulls a section's melody and chords toward the same point, so the two vocabularies become mutually searchable. (This lane is the newest and the most experimental — see the honest note in Results.)
An eval you can trust
Retrieval evals live or die on leakage, and these corpora offer natural ground truth. For chords: 8,142 songs were transcribed more than once (one has 48 tabs). Different transcribers disagree on key, skip sections, and make errors — exactly the noise a fuzzy engine must survive. Each multi-tab song is a clique: query with one transcription, rank the others at the top of a 69,319-doc index. Splits are artist-disjoint, and cliques whose progression also appears in training (in transposition-invariant space) are dropped. For melody, the ground truth is different sections of the same song — a looser target (a verse and its chorus are genuinely different tunes), but exactly the "songs like this" objective. Scoring is mAP and Recall@10 throughout.
Results so far
Classical baselines were measured first and set the bar — TF-IDF over root-motion n-grams is the same math as text search. A neural model that can't beat grep shouldn't exist.
| Chord retrieval — method | mAP | R@10 |
|---|---|---|
| Bag of absolute chords (TF-IDF) | 0.078 | 0.102 |
| Delta n-gram TF-IDF | 0.132 | 0.163 |
| Edit distance | 0.139 | 0.169 |
| Chord encoder (v2) | 0.146 | 0.185 |
| Hybrid (0.6·encoder + 0.4·TF-IDF) | 0.154 | 0.191 |
The encoder beats every classical baseline solo, and the hybrid adds another +16.5% over the best single method. Absolute numbers look small because the task is brutal — real transcriber noise against tens of thousands of near-identical pop progressions — but every method runs the same gauntlet.
The melody encoder is best judged on a benchmark it had no hand in making: MTC-ANN, 360 folk melodies in 26 tune families. Asked to retrieve same-family tunes, it scores 0.612 mAP and 0.875 nearest-neighbor accuracy — and on this task it beats two general-purpose symbolic foundation models many times its size: CLaMP 3 (457M params, 0.343 mAP) and Aria (650M, 0.347). Masked pretraining is what buys the edge — the same 7.5M architecture trained from scratch scores 0.530 — a small, task-matched model outrunning models 60–90× larger on melodic similarity. Its nearest-neighbor lists are musically coherent across different songs, too: Mr. Brightside's melodic neighbors are Message in a Bottle and Dirty Little Secret, not just anything in the same key.
The v2 recipe came out of a literature sweep plus a day of matched-pair ablations on two GPUs, and the losers were as informative as the winner. The 2025 cover-song SOTA loss (CLEWS) underperformed here — its best-pair positive term optimizes a different metric than per-section retrieval — while three supporting ideas from the same literature stacked into a new record: multi-positive batches (all of a song's sections pull on each other), false-negative masking (an n-gram oracle stops training from pushing apart melodically-identical "negatives" — pop tunes recycle material, which also explains why classic hard-negative mining always hurt in our sweeps), and asymmetric views (one clean anchor vs one heavily-augmented view). Each component alone scored below the old recipe; only the full stack beat it — matching v1's best number in a third of the training steps, with EMA weight-averaging adding a free edge at nearly every checkpoint.
The finding I didn't expect
A fully external chord eval: Wikipedia's lists of songs built on named progressions (the Axis of Awesome I–V–vi–IV, the '50s doo-wop changes) give a cross-song family retrieval task with ground truth this project had no hand in making. On families, plain TF-IDF beats the neural model; on same-song cliques, the model wins; and the hybrid wins both — with the blend weight flipping by task. Named families are exact motifs (n-gram turf); the embedder's edge is tolerance. Embeddings and string methods capture different similarity, and a real system ships both. My separate text-retrieval lab reached the same conclusion on a different corpus — twice, now.
From scratch, and why that's the point
There's no pretrained backbone in any of this — two encoders, a parser, a key detector, the evals, all cold-started on a desktop. 2.3M parameters is nobody's idea of scale, but when the model category doesn't exist yet, scale isn't the interesting axis; the eval design is. This is a repeatable move: find a public corpus nobody has trained a retrieval model on, verify the whitespace, build the honest eval before the model, make classical baselines the bar, and let a small from-scratch encoder earn its place.