Author: Matti Ryynänen,
Last modified: April 19, 2005
M. P. Ryynänen and A. Klapuri, "Polyphonic music transcription using note event modeling," in Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA'05), October 2005. pdf
We propose a method for the automatic transcription of real-world polyphonic music signals, including a variety of musical genres. The method transcribes notes played with pitched musical instruments. Percussive sounds, such as drums, may be present but they are not transcribed.
The method produces musical notations (i.e., MIDI files) from acoustic stereo input files using probabilistic note event modeling. Note events are described with a hidden Markov model (HMM). The model uses three features, which are extracted with a multiple fundamental frequency (F0) estimator, to calculate the likelihoods of different notes and performs temporal segmentation of notes. The transitions between notes are controlled with a musicological model involving musical key estimation and bigram models. The final transcription is obtained by searching for several note paths through the models. Evaluation was carried out with a real-world music database.
Using strict evaluation criteria, 39% of all the notes are found (recall) and 41% of the transcribed notes are correct (precision). Considering a realistic polyphonic transcription task, the results are encouraging.
The note-event-modeling approach was previously applied to singing transcription . See also the singing transcription demopage.
All the following transcription examples are from the RWC (Real World Computing) music genre database which consists of stereophonic acoustic recordings sampled at 44.1 kHz from several musical genres, including popular, rock, dance, jazz, classical, and world music . The same database was used in the development and the evaluation of the transcription method. See RWC music database homepage for additional information on the music performance examples. The authors of the RWC database kindly granted a permission to set the following RWC input music files available.
 M. P. Ryynänen and A. Klapuri, "Modelling of Note Events for Singing Transcription," in Proc. of ISCA Tutorial and Research Workshop on Statistical and Perceptual Audio Processing, Oct. 2004. Available online.
 M. Goto, H. Hashiguchi, T. Nishimura, and R. Oka, "RWC music database: Music genre database and musical instrument sound database," in Proc. of the 4th International Conference on Music Information Retrieval, Oct. 2003.
Three music files are provided for each transcription example:
|Song name||RWC ID||Genre||Acoustic input||Transcription output||Mix||Comments|
|Forget about It||RWC-MDB-G-2001 No. 2||Popular||mp3||mp3||mp3||Overdrive guitar and singing are difficult to transcribe|
|Everyday Lovin'||RWC-MDB-G-2001 No. 7||Rock||mp3||mp3||mp3||Some additional notes|
|21st Century||RWC-MDB-G-2001 No. 11||Heavy Metal||mp3||mp3||mp3||Powerful drums cause additional notes|
|Life Gage||RWC-MDB-G-2001 No. 16||House||mp3||mp3||mp3||Short guitar notes hard to transcribe, drum beat results in additional note boundaries|
|Tonight's the Night for Love||RWC-MDB-G-2001 No. 17||House||mp3||mp3||mp3||Loud bass drum produces lots of additional notes|
|Bubble||RWC-MDB-G-2001 No. 29||Big Band||mp3||mp3||mp3||Many instruments|
|Chuggin'||RWC-MDB-G-2001 No. 32||Modern Jazz||mp3||mp3||mp3||OK transcription|
|The Ramp||RWC-MDB-G-2001 No. 33||Modern Jazz||mp3||mp3||mp3||Some additional short notes|
|Tea Break||RWC-MDB-G-2001 No. 34||Fusion||mp3||mp3||mp3||Lots of notes, basic chords found|
|Misty Rouge||RWC-MDB-G-2001 No. 38||Bossa Nova||mp3||mp3||mp3||Quite OK|
|Lovely Women||RWC-MDB-G-2001 No. 40||Samba||mp3||mp3||mp3||Lots of percussions, difficult guitar theme|
|Eric's Reggae||RWC-MDB-G-2001 No. 44||Reggae||mp3||mp3||mp3||Additional notes according to drums|
|Where Are You?||RWC-MDB-G-2001 No. 45||Reggae||mp3||mp3||mp3||Some additional short notes|
|Water Music, Suite no.2 in D major, HWV.349. 2. `Alla Hornpipe'||RWC-MDB-G-2001 No. 49||Baroque (Orchestral)||mp3||mp3||mp3||Correct notes with varying rhythm|
|`Radetzky March', op.228||RWC-MDB-G-2001 No. 55||Brass Band||mp3||mp3||mp3||Large dynamic variations during the song|
|Two-Part Inventions no.4 in D minor, BWV.775||RWC-MDB-G-2001 No. 58. Tr. 03||Baroque (Solo)||mp3||mp3||mp3||Quite nice transcription, not all notes found though|
|Rondo in D major, K.485||RWC-MDB-G-2001 No. 59||Classic (Solo)||mp3||mp3||mp3||Quiet notes are missed|
|String Quartet no.77 in C major, op.76-3 (Hob.III-77) `Kaiser'. 2nd mvmt.||RWC-MDB-G-2001 No. 60||Classic (Chamber)||mp3||mp3||mp3||Note beginnings have varying rhythm|
|Blue Print||RWC-MDB-G-2001 No. 64||Blues||mp3||mp3||mp3||Additional notes, groovy|
|What Should I Tell Them?||RWC-MDB-G-2001 No. 69||Folk||mp3||mp3||mp3||Nice transcription, singing causes some problems|
|Bjc Fiddle Rag||RWC-MDB-G-2001 No. 70||Country||mp3||mp3||mp3||Happy transcription|
|With My Jalopy||RWC-MDB-G-2001 No. 74||Gospel||mp3||mp3||mp3||Choir singing is hard to transcribe|
|Tangos||RWC-MDB-G-2001 No. 83||Flamenco||mp3||mp3||mp3||Very short notes|
|Le Temps Des Cerises||RWC-MDB-G-2001 No. 86||Chanson||mp3||mp3||mp3||Expressive singing|
|Ai Hitohira||RWC-MDB-G-2001 No. 93||Popular (Enka)||mp3||mp3||mp3||Additional notes|
|Sado Okesa||RWC-MDB-G-2001 No. 94||Folk (Min'you)||mp3||mp3||mp3||OK transcription|