Music

I am deeply interested in how music forms a communicative language (don't know what I'm talking about? Read Susanne Langer's book!). I have explored this throughout my career as a musician and more recently as a researcher.

Music AI Research

I am very excited for the Aria project, a 3-year collaboration with Louis Bradshaw, funded by EleutherAI, to bear fruit! The goal of Aria was to train state-of-the-art generative music models. As a highlight, a Disklavier demo video is shown:

On Oct. 23, 2025, I gave an overview of the Aria project and its relation to linguistics in a talk at the Stanford NLP Lunch [slides].

We pretrain autoregressive transformers on roughly 60,000 hours of transcribed solo piano, then finetune on a small, high-quality subset. The generative model produces continuations that beat prior symbolic systems and compete with proprietary audio models; a contrastive variant yields MIDI embeddings that set the state of the art on MIR classification, often from a few hundred labeled examples.

These are the models behind the Disklavier demo above.

None of this works without transcription: the pretraining corpus is audio recordings turned into MIDI. Aria-MIDI, Louis's ICLR 2025 dataset, used a language model to crawl and score piano recordings and an audio classifier to prune them, transcribing about 100,000 hours into over a million MIDI files.

Transcription models trained only on clean studio audio, though, fall apart on the noise and reverb of real recordings. Our transcription work folds unlabeled audio into training to make them robust, and releases the models, re-transcriptions of the standard expressive MIDI datasets and 5,000+ new piano transcriptions.

Musicians have been slow to adopt generative composition tools, and I don't think it is because they object to sharing creative control: text-prompting is simply nothing like playing. Aria-Duet puts the model on a Yamaha Disklavier for a live, turn-taking duet. You play, signal a handover, and the model's continuation is performed acoustically on the same piano.

Musicologically, the model holds stylistic semantics across genres and develops coherent phrasal ideas rather than merely continuing. The same questions run through the ICML 2026 workshop on human-AI co-creativity that I co-organized.

All Publications expand all | collapse all

The Ghost in the Keys: A Disklavier Demo for Human-AI Musical Co-Creativity
NeurIPS 2025 Creative AI [ paper ]
Louis Bradshaw, Alexander Spangher, Stella Biderman, Simon Colton.
Aria-Duet puts our piano model on a Disklavier for a live, turn-taking duet with a human pianist; it holds the style and develops phrases rather than merely continuing them.
Scaling Self-Supervised Representation Learning for Symbolic Piano Performance
ISMIR 2025 [ paper ]
Louis Bradshaw, Honglu Fan, Alexander Spangher, Stella Biderman, Simon Colton.
We trained autoregressive transformers on 60,000 hours of transcribed solo piano; they generate coherent continuations and yield MIDI embeddings that set the state of the art on MIR classification.
Musically Aware Automatic Piano Transcription Using Synthetic Pretraining
In submission [ paper ]
Louis Bradshaw*, Alexander Spangher*, Stella Biderman, Simon Colton.
Transcription models trained only on clean studio audio break on real recordings; folding in unlabeled audio makes them robust, and we release the models, re-transcribed MIDI datasets and 5,000+ new piano transcriptions.
Aria-MIDI: A Dataset of Piano MIDI Files for Symbolic Music Modeling
ICLR 2025 [ paper ]
Louis Bradshaw, Simon Colton. (Acknowledged: Alexander Spangher.)
The dataset behind Aria: a language model crawls and scores piano recordings, an audio classifier prunes them, and roughly 100,000 hours are transcribed into over a million MIDI files.

We are continuing to work on the Aria project, if you are interested in working on music AI, please reach out!

Videos of Musical Performances

I have been playing the piano since I was 3 years old. Here are some videos of my performances.