Polyphonic music transcription using note event modeling, transcription examples

Author: Matti Ryynänen, homepage
Last modified: April 19, 2005

Transcription method

M. P. Ryynänen and A. Klapuri, "Polyphonic music transcription using note event modeling," in Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA'05), October 2005. pdf

We propose a method for the automatic transcription of real-world polyphonic music signals, including a variety of musical genres. The method transcribes notes played with pitched musical instruments. Percussive sounds, such as drums, may be present but they are not transcribed.

The method produces musical notations (i.e., MIDI files) from acoustic stereo input files using probabilistic note event modeling. Note events are described with a hidden Markov model (HMM). The model uses three features, which are extracted with a multiple fundamental frequency (F0) estimator, to calculate the likelihoods of different notes and performs temporal segmentation of notes. The transitions between notes are controlled with a musicological model involving musical key estimation and bigram models. The final transcription is obtained by searching for several note paths through the models. Evaluation was carried out with a real-world music database.

Using strict evaluation criteria, 39% of all the notes are found (recall) and 41% of the transcribed notes are correct (precision). Considering a realistic polyphonic transcription task, the results are encouraging.

The note-event-modeling approach was previously applied to singing transcription [1]. See also the singing transcription demopage.

RWC music database

All the following transcription examples are from the RWC (Real World Computing) music genre database which consists of stereophonic acoustic recordings sampled at 44.1 kHz from several musical genres, including popular, rock, dance, jazz, classical, and world music [2]. The same database was used in the development and the evaluation of the transcription method. See RWC music database homepage for additional information on the music performance examples. The authors of the RWC database kindly granted a permission to set the following RWC input music files available.

References

[1] M. P. Ryynänen and A. Klapuri, "Modelling of Note Events for Singing Transcription," in Proc. of ISCA Tutorial and Research Workshop on Statistical and Perceptual Audio Processing, Oct. 2004. Available online.

[2] M. Goto, H. Hashiguchi, T. Nishimura, and R. Oka, "RWC music database: Music genre database and musical instrument sound database," in Proc. of the 4th International Conference on Music Information Retrieval, Oct. 2003.

Transcription examples

Three music files are provided for each transcription example:

• Acoustic input, a 30 second excerpt of an original polyphonic music performance which is the input for the transcription method.
• Transcription output, a MIDI transcription of the acoustic input produced with the transcription method. The MIDI file is directly synthesized to a sound file, i.e., no additional post-processing is performed.
• Mix, the acoustic input (in the left audio channel) and the synthesized transcription (in the right audio channel) mixed together.