Polyphonic music transcription using note event modeling, transcription examples

Author: Matti Ryynänen, homepage
Last modified: April 19, 2005

Transcription method

M. P. Ryynänen and A. Klapuri, "Polyphonic music transcription using note event modeling," in Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA'05), October 2005. pdf

We propose a method for the automatic transcription of real-world polyphonic music signals, including a variety of musical genres. The method transcribes notes played with pitched musical instruments. Percussive sounds, such as drums, may be present but they are not transcribed.

The method produces musical notations (i.e., MIDI files) from acoustic stereo input files using probabilistic note event modeling. Note events are described with a hidden Markov model (HMM). The model uses three features, which are extracted with a multiple fundamental frequency (F0) estimator, to calculate the likelihoods of different notes and performs temporal segmentation of notes. The transitions between notes are controlled with a musicological model involving musical key estimation and bigram models. The final transcription is obtained by searching for several note paths through the models. Evaluation was carried out with a real-world music database.

Using strict evaluation criteria, 39% of all the notes are found (recall) and 41% of the transcribed notes are correct (precision). Considering a realistic polyphonic transcription task, the results are encouraging.

The note-event-modeling approach was previously applied to singing transcription [1]. See also the singing transcription demopage.

RWC music database

All the following transcription examples are from the RWC (Real World Computing) music genre database which consists of stereophonic acoustic recordings sampled at 44.1 kHz from several musical genres, including popular, rock, dance, jazz, classical, and world music [2]. The same database was used in the development and the evaluation of the transcription method. See RWC music database homepage for additional information on the music performance examples. The authors of the RWC database kindly granted a permission to set the following RWC input music files available.


[1] M. P. Ryynänen and A. Klapuri, "Modelling of Note Events for Singing Transcription," in Proc. of ISCA Tutorial and Research Workshop on Statistical and Perceptual Audio Processing, Oct. 2004. Available online.

[2] M. Goto, H. Hashiguchi, T. Nishimura, and R. Oka, "RWC music database: Music genre database and musical instrument sound database," in Proc. of the 4th International Conference on Music Information Retrieval, Oct. 2003.

Transcription examples

Three music files are provided for each transcription example:

Song name RWC ID Genre Acoustic input Transcription output Mix Comments
Forget about It RWC-MDB-G-2001 No. 2 Popular mp3 mp3 mp3 Overdrive guitar and singing are difficult to transcribe
Everyday Lovin' RWC-MDB-G-2001 No. 7 Rock mp3 mp3 mp3 Some additional notes
21st Century RWC-MDB-G-2001 No. 11 Heavy Metal mp3 mp3 mp3 Powerful drums cause additional notes
Life Gage RWC-MDB-G-2001 No. 16 House mp3 mp3 mp3 Short guitar notes hard to transcribe, drum beat results in additional note boundaries
Tonight's the Night for Love RWC-MDB-G-2001 No. 17 House mp3 mp3 mp3 Loud bass drum produces lots of additional notes
Bubble RWC-MDB-G-2001 No. 29 Big Band mp3 mp3 mp3 Many instruments
Chuggin' RWC-MDB-G-2001 No. 32 Modern Jazz mp3 mp3 mp3 OK transcription
The Ramp RWC-MDB-G-2001 No. 33 Modern Jazz mp3 mp3 mp3 Some additional short notes
Tea Break RWC-MDB-G-2001 No. 34 Fusion mp3 mp3 mp3 Lots of notes, basic chords found
Misty Rouge RWC-MDB-G-2001 No. 38 Bossa Nova mp3 mp3 mp3 Quite OK
Lovely Women RWC-MDB-G-2001 No. 40 Samba mp3 mp3 mp3 Lots of percussions, difficult guitar theme
Eric's Reggae RWC-MDB-G-2001 No. 44 Reggae mp3 mp3 mp3 Additional notes according to drums
Where Are You? RWC-MDB-G-2001 No. 45 Reggae mp3 mp3 mp3 Some additional short notes
Water Music, Suite no.2 in D major, HWV.349. 2. `Alla Hornpipe' RWC-MDB-G-2001 No. 49 Baroque (Orchestral) mp3 mp3 mp3 Correct notes with varying rhythm
`Radetzky March', op.228 RWC-MDB-G-2001 No. 55 Brass Band mp3 mp3 mp3 Large dynamic variations during the song
Two-Part Inventions no.4 in D minor, BWV.775 RWC-MDB-G-2001 No. 58. Tr. 03 Baroque (Solo) mp3 mp3 mp3 Quite nice transcription, not all notes found though
Rondo in D major, K.485 RWC-MDB-G-2001 No. 59 Classic (Solo) mp3 mp3 mp3 Quiet notes are missed
String Quartet no.77 in C major, op.76-3 (Hob.III-77) `Kaiser'. 2nd mvmt. RWC-MDB-G-2001 No. 60 Classic (Chamber) mp3 mp3 mp3 Note beginnings have varying rhythm
Blue Print RWC-MDB-G-2001 No. 64 Blues mp3 mp3 mp3 Additional notes, groovy
What Should I Tell Them? RWC-MDB-G-2001 No. 69 Folk mp3 mp3 mp3 Nice transcription, singing causes some problems
Bjc Fiddle Rag RWC-MDB-G-2001 No. 70 Country mp3 mp3 mp3 Happy transcription
With My Jalopy RWC-MDB-G-2001 No. 74 Gospel mp3 mp3 mp3 Choir singing is hard to transcribe
Tangos RWC-MDB-G-2001 No. 83 Flamenco mp3 mp3 mp3 Very short notes
Le Temps Des Cerises RWC-MDB-G-2001 No. 86 Chanson mp3 mp3 mp3 Expressive singing
Ai Hitohira RWC-MDB-G-2001 No. 93 Popular (Enka) mp3 mp3 mp3 Additional notes
Sado Okesa RWC-MDB-G-2001 No. 94 Folk (Min'you) mp3 mp3 mp3 OK transcription