Can state-of-the-art audio language models dissect my composition?
I asked five audio language models to “identify all audible sound sources in each section, including background and overlapping sources” in a piece I composed. The best score was 5.5 out of 16.
| Model | Score / 16 | Percentage |
|---|---|---|
| Gemini 4 Argon | 5.5 | 34.4% |
| Meta Muse Wonder | 4.5 | 28.1% |
| Qwen3.8 Omni Flash | 3.5 | 21.9% |
| Xiaomi MiMo-V2.6-Pro | 3 | 18.8% |
| Perceptron Mk1.5 | 3 | 18.8% |
Results for each sound
| Sound | Qwen3.8 Omni Flash | Xiaomi MiMo-V2.6-Pro | Meta Muse Wonder | Perceptron Mk1.5 | Gemini 4 Argon |
|---|---|---|---|---|---|
| Swiss cowbells | ½ | 1 | 1 | 0 | ½ |
| Rocket engine roar | 0 | 0 | 0 | 0 | 0 |
| Explosion | 0 | 0 | ½ | 0 | ½ |
| Airy, magical musical flourish | 0 | 0 | 0 | 0 | 0 |
| Rhythmic bubbling made with the mouth | 0 | 0 | 0 | 0 | 0 |
| Playing cards being shuffled | 0 | 0 | 0 | 0 | 0 |
| Toy laser gun | 0 | 0 | 0 | 0 | ½ |
| Kitchen knife being polished | 0 | 0 | 0 | 0 | 0 |
| TV switching on | 0 | 0 | 0 | 0 | 0 |
| Baby sounds | 1 | 1 | 1 | 1 | 1 |
| Frogs and toads calling | 0 | 0 | 0 | 0 | 0 |
| Human voice saying “I am God” | 1 | 1 | 1 | 1 | 1 |
| Toy accordion | 0 | 0 | 0 | 0 | 1 |
| Splash | 1 | 0 | 1 | 1 | 1 |
| Donkey Balthazar braying | 0 | 0 | 0 | 0 | 0 |
| My piano waltz | 0 | 0 | 0 | 0 | 0 |
None of them heard the piano waltz I composed in F minor/major. All that work, only for audio language models to completely not hear it!
They also missed the frogs, the donkey Balthazar, the rocket engine, the playing cards being shuffled, and the magical musical flourish. All, except Gemini 4 Argon, also missed the accordion.
Balthazar was variously described as human screaming, a siren, or other animals. He too, he told me, felt let down by the technology. Imagine he had been calling 911 and an audio-language model picked up the phone!
Read all model responses (PDF)
The results made me wonder whether my composition style was maybe out of distribution for the models. I therefore decided to ask a simpler question: "What time signature is my improvisation in?"
I conclude that no model was able to feel the beat of my music: Meta Muse Wonder was close: 7/8 indeed, but grouped 3+2+2, not 2+2+3. The other models were vibing on different music (4/4, 3/4) I believe...
Read all model responses: time signature
Gemini 4 Argon
The time signature of this piece changes at 01:07 from 4/4 to 3/4, and again at 01:36 from 3/4 to 4/4.
Meta Muse Wonder
7/8
The piece is driven throughout by a relentless quaver ostinato grouped 2+2+3, giving seven quaver beats to the bar.
Qwen3.8 Omni Flash
The piece is in 3/4 time – a triple meter. Throughout the work the figuration (the opening motor rhythm, the chordal "ta‑ta‑TA" patterns, the octave ostinato, and the final fast section) consistently groups in threes, giving the piece its driving 3/4 pulse, even in the slower, more rubato middle passage.
Xiaomi MiMo-V2.6-Pro
Based on the rhythmic structure and the grouping of beats heard throughout the piece, the time signature is 4/4 (common time). This can be heard clearly in both the fast, driving sections (where the beats group into four distinct pulses per measure) and the slower, lyrical middle section.
Perceptron Mk1.5
4/4