Combined Audio and Video Analysis for Guitar Chord Identification

Overview


This research presents a multi-modal approach to automatically identifying guitar chords using audio and video of the performer. Chord identification is for stringed instruments adds extra ambiguity as a single chord or melody may be played in different positions on the fretboard. Preserving this information is important, because it signifies the original fingering, and implied “easiest” way to perform the selection. This chord identification system combines analysis of audio to determine the general chord scale (i.e. A major, G minor), and video of the guitarist to determine chord voicing (i.e. open, barred, inversion), to accurately identify the guitar chord.

Audio Analysis


When playing a single note, the guitar, and many other instruments produce natural harmonics (overtones) in addition to the note’s fundamental frequency. When playing multiple notes, the frequency spectrum of the audio appears cluttered, making detecting the fundamental frequencies (the actual notes) hard to locate. Using a technique known as Specmurt analysis [1], the notes of the guitar chord can be extracted from the audio signal.