Identifying Musical Chords

Using an RP2040

If you want to learn a new instrument or to learn about music theory, this is the project for you! These two Cornell University seniors built a musical chord identifier, using a Raspberry Pi Pico RP2040 microcontroller (MCU), electret microphone amplifier, and a UART serial interface. The device takes in musical audio input, determines the name and numb er of the chord being played, and displays it on a VGA display, and other characteristics are identified on a serial monitor. An introductory discussion of music theory is included.

MATH AND MUSIC THEORY BACKGROUND

The basic building blocks of music are notes. A musical note maps to a unique frequency, and is identified by a note name (a letter) and an octave number. There are 12 note letter names, which are shown on a standard piano keyboard in Figure 1. Each collection of 12 notes makes up an octave, and all notes in a given octave are given the same octave number. Then, the next 12 higher notes are given the next octave number. In Figure 1, the octave shown is octave 4, with the highest C starting a new octave, 5. We want to enumerate notes in order of increasing frequency, and can refer back to the piano keyboard. A0 (the A note in octave 0) is the bottom note of a keyboard. Calling this “note 1,” we can now number each note relative to A0.

To associate note names with frequencies, musicians use a mapping method called a “tuning system.” The most common tuning system by far in Western music theory is “12-tone equal temperament,” in which any note’s frequency is higher than that of the one directly below it by a factor of 2^1/12. This has a convenient interpretation. Since there are 12 notes in an octave, for any note in a given octave, the same note an octave up (that is, the note with the same name in the next consecutive octave, such as A3 to A4) is the note with double the frequency. In this case, A3 is 220Hz, and A4 is 440Hz.

Notes with the same name harmonize with each other closely, as seen by the relatively small period of two added signals with a 2:1 frequency ratio. In any case, the standard for Western music theory is to define the note A4, the 49th note on a keyboard, with the frequency 440Hz. This gives us the simple mapping shown in Equation 1,

where n is the note number on the keyboard. In this project, we want to compute the note from an input frequency, so we can invert this equation to give us Equation 2:

Note that since n should be an integer, we round the result of this equation to the nearest integer, because it is generally unlikely that an arbitrary instrument will produce the exact, precise frequency associated with a note.

For the full name of a note, we also compute its octave number, which is found by using Equation 3:

This simply finds which group of 12 notes the note falls into. The shift of 8 accounts for the fact that the first 4 notes on a standard keyboard are defined to be in octave 0.

Chords: Moving up a level of abstraction, we are interested in sounds with multiple frequencies—collections of notes known as “chords,” which are more complex musical structures responsible for the wide array of sounds possible in music. For this project we focus on a subset of chords called “triads,” which are made up of three distinct notes in some defined pattern. We can denote such a triad as shown in Equation 4,

where the n_i refer to the numbers of the notes in the chord, in increasing order. Therefore, n₁ is the lowest note in the chord, n₂ is the middle, and n₃ is the highest.

A chord is given a certain classification based on its harmonic content, which can be determined by the differences between its notes; thus, looking at n₂ − n₁ and n₃ − n₂, we can compute its general type.

We consider three basic types of triads: Major (which are often described as “happy-sounding” chords), Minor (often described as “sad-sounding”) and Diminished (often described as “scary-sounding”). In addition, a chord is also characterized by a “root note,” which is a note in the chord (named in the format we discussed earlier, such as A4) that serves as the “base” of the chord. It turns out that identifying the root note is not as simple as relaying n₁. In practice, a chord can exist in one of three “inversions,” meaning three different patterns that produce the same classification, but have the root note in any one of the three note positions (bottom, middle, or top).

Skipping over an hour or so of drawing and enumerating chord diagrams, Table 1 is the final chord table, and also is the lookup table used later in the project’s software. As discussed above, we identify the chords and root by identifying each individual note and looking at the corresponding differences. As an example, suppose we have a chord, and we find that n₂ − n₁ = 4 and n₃ − n₂ = 3, and that n₁ is A4. We see from the differences that this is a major triad, and that its root is n₁, so, we call this an A4 Major Triad. (In many places, musicians omit the octave number in chord names, so we could also call this an A Major Triad).

For someone learning music theory, it’s useful to look at the display of the three musical notes identified to see if you can guess the chord, before checking your work with our identifier!

HIGH-LEVEL OVERVIEW

The chord identifier works as follows. When the user plays a chord, a microphone picks up the audio sample. A low-pass filter filters out frequencies above 1,200Hz from the sample before digital processing. The software calculates the chord by first performing a Fast Fourier Transform (FFT) to map the audio samples to the frequency domain, and then identifying the three frequencies with the highest amplitudes. From here, the equations described in the previous section are used to calculate the notes from the frequencies. Once the note name and number are calculated, we look at the gap between notes to identify the chord.

Finally, we output the chord identified on a VGA display and output the maximum frequencies, note numbers, note names, and the chord identified on a serial monitor.

HARDWARE OVERVIEW

This project uses the general purpose input/output (GPIO) pins on the Raspberry Pi Pico 2040 microcontroller in combination with some of its internal peripherals. There are two main parts to the hardware design of the chord identifier: audio detection from the environment, and two displays providing information about the notes and chords identified. Figure 2 shows a diagram of the full hardware design.

Audio input is gathered from an Electret Microphone Amplifier, and then passed through an analog low-pass filter. The resulting audio signal connects to a GPIO pin on the RP2040 assigned to analog-to-digital conversion (ADC). This pin is controlled by a DMA channel, which samples the converted audio input and stores it into an array in memory for use in the Fast Fourier Transform (FFT). (See the ”Software Overview” section for a discussion of our FFT implementation.) When the sampling DMA channel is finished sampling, it triggers another DMA channel to send a signal to start the sampling channel again, allowing for continuous sampling.

As we’ll discuss, the sample rate of the FFT is 5kHz, and therefore, input frequencies must remain below 2.5kHz to avoid aliasing. It turns out, though, that you can eliminate frequencies well below this (down to ~1kHz), and still be able to recognize most notes on a standard piano reasonably well. We avoid aliasing using an analog low-pass filter, with a second-order pole at 1.2kHz.

Two displays show the notes and the identified chord generated by the notes. A VGA screen shows the chord identified, and a UART serial interface displays the three maximum amplitude frequencies detected by the FFT, the note names, and the chord identified.

Audio Sampling and Processing: Initially, we tested the chord identification with only the FFT in place, and without any filtering. We found that this led to imprecise identification of the notes and therefore the chords. As a result, we chose to add an analog low-pass filter to limit the presence of higher frequencies. This allowed us to decrease the sampling rate of the FFT while avoiding aliasing.

The low-pass filter uses a Sallen-Key topology (Figure 3), with values set to achieve a cut-off frequency of 1.2kHz. Resistor and capacitor values were chosen to achieve this frequency according to Equation 5:

An op-amp (MCP6242) is used both in the filter, itself, and in an AC coupling stage on the filter’s input, since the filter requires a strictly positive voltage input. The final filter design with the AC coupling stage is shown in Figure 4. A frequency sweep of the filter demonstrates the frequency roll-off starting at 1.2kHz (Figure 5).

Monitor Display: For quick display of the determined chord, the currently-detected chord name is displayed on a VGA screen, which updates every time a new chord is detected. If the chord cannot be classified by the identifier, which also occurs when no chord is being played, “Unknown” is shown on the display. During testing phases, the VGA also displayed the output of the FFT, which was useful in making sure each individual part of the program—sampling, filtering, FFT, and chord computation—worked as expected.

For more detailed information (especially used during testing), a UART connection from the RP2040 to a serial monitor displays the detected frequencies, associated note numbers and names, and a determined chord on the monitor every second. The VGA screen and serial monitor displays are shown in Figure 6.

SOFTWARE OVERVIEW

Source code for our project is available on the Circuit Cellar Code and Files webpage. We have three main components in the software design of our chord identifier. First, is the audio sampling using the microphone and low-pass filter input, and subsequent processing using an FFT, which identifies the notes played. Second, the chord names are calculated according to a lookup table encoding (Table 1). Third, the note and chord names are displayed on the serial monitor, and the chord on the VGA screen.

We separate the processes across the two cores on the RP2040 (Figure 7), to allow for concurrent operation with Protothreads, a light-weight, stackless, threading library that is implemented in C [1]. On Core 0, a looping thread runs the ADC input sampling, performs the FFT, and then determines notes and chords from the frequency information given by the FFT. Core 1 controls the display; information is communicated via the VGA and serial monitor, each of which uses a different thread.

Audio Sampling and Processing: After the analog-to-digital conversion is performed to obtain an array of audio samples, the audio samples are sent as an input to the FFT algorithm, which maps the time-domain audio content into the frequency domain. The FFT implementation used is the common Cooley-Tukey FFT, which recursively splits the transform into sums of smaller transforms, eventually reaching a base case of length 1 and then rebuilding the full Fourier Transform. By taking the magnitude of this output at each frequency, we have a power spectrum, denoting the strength of the audio signal at each frequency. We take from it the three strongest frequencies as our chord tones, and compute the corresponding note numbers using Equation 2.

Our sampling and FFT had a sample size of 2,048 samples, and a sample rate of 5kHz. These values were chosen as a result of testing, attempting to keep reasonable accuracy while having computations and identification happen relatively quickly. We found that a chord being played for 0.6s was the cut-off for accurate identification with our design.

Once we have a note number, the following functions are used for translation into more well-known names:

• getLetter takes as a parameter a note number, and returns its letter name (for example, A, A#, B) by looking it up on a table of note names. The table is 12 entries long, for the 12 possible note letter names.
• getName is given a note number and returns the full name of a note, which is made up of its letter (found as in getLetter) and its octave number (as in Equation 3).
• getChord takes a list of note numbers (sorted in increasing order) and determines the corresponding chord. This is done by finding the differences between notes in the chord—so, finding the gap between the lowest and middle notes, and the gap between the middle and highest notes. This function encodes the information shown in Table 1.

Monitor Display: The VGA display uses Hunter Adams’ VGA driver code [2] to display text showing the chord identifier, and updates as new chords are identified. During testing, we used the VGA to display a visualization of the FFT output, but did not include this in the final VGA display so that the frame rate could be faster to display updates in the detected chord. A sample VGA display is shown in Figure 8.

The serial monitor thread prints the top three detected frequencies from the FFT, their associated note numbers and note names, and the detected chord. This thread is only queued every second, which makes the output on the monitor more readable for the user. A sample serial monitor output is shown in Figure 9.

TESTING

To test basic functionality, we played chords made of pure sine-wave notes using Helm synthesizer software, and observed the determined chord on the VGA monitor. Since these notes are pure frequencies, the identifier is able to determine each note easily, and then to determine the chord from the notes.

The next tests employed different instrument sounds, while playing the same chords as those used in our initial tests. We chose to test our chord identifier against different instruments, because instrument sounds inherently have additional harmonic frequencies, which contribute to the “timbre,” or characteristic sound of the instrument. These frequencies tend to mess with the identification process, since prominent harmonics may register as distinct notes and overpower the fundamental notes.

Additionally many instruments have different amplitude profiles. For example, a plucked guitar has a strong initial attack but quickly dies down. As a result, there is only a small window for our program to get good samples and make calculations. The identifier worked well with some instruments, such as electric piano and violin, but less so with others, including the guitar.

Finally, we performed a basic “speed test.” Using the Ableton Digital Audio Workstation program, we set up a few chord progressions to play each beat at increasing beats per minute (bpm), again using the sine-wave sound. The system was reasonably able to catch every chord in a progression up to about 100bpm, which corresponds to a played chord being identified in 0.6s at most.

CONCLUSION AND FUTURE STEPS

The program works quite well at identifying pure-tone sound chords, and does a decent job with real instruments. For a person learning music theory, our program is able to identify chords with high accuracy, as long as the notes are played for a sufficient length of time. Therefore, as long as the user is not constrained by playing the chords for a short time (for instance, chord identification of each chord in a pop song, where chords are quickly changing), the program is able to identify chords quickly and accurately.

First, a simple further step would be to encode more types of chords, including adding 4-note chord functionality. There is a much wider variety of 4-note chords in music, but this step simply comes down to encoding all these types of chords (in a similar way to the 3-note chords) in the lookup table.

Second, we might also consider looking into signal processing techniques, to better identify notes played on actual instruments. One possible avenue for this is to look into the harmonic profile of different instruments, and then eliminate from consideration frequencies that are most likely harmonics, thus making it more likely that the program will pick up the fundamental notes. A better microphone might also be helpful with this, since it would allow the program to pick up audio at low levels. This would be good for instruments with sounds that decay (fade to silence) quickly.

Third, we could look into increasing speed by optimizing code or overclocking the RP2040, thereby allowing faster chord recognition and possibly more accurate results at higher tempos.

Finally, we’ve considered taking some note of the timing and length of played chords, so that we may be able to tell the musical length (that is, eighth, quarter, half notes) of chords. This feature would better serve the goal of teaching music theory or telling a player how long to play a given chord in a song. 

Acknowledgements: We thank Ariana Haghighi for her contributions to our project, and Professors Hunter Adams and Bruce Land for all their help!

REFERENCES
[1] Protothreads on Raspberry Pi Pico RP2040.
https://people.ece.cornell.edu/land/courses/ece4760/RP2040/C_SDK_protothreads/index_Protothreads.html
[2] Hunter Adams, Demo Code for VGA Driver.
https://github.com/vha3/Hunter-Adams-RP2040-Demos/tree/master/VGA_Graphics

SOURCES
RP2040 Pinout Diagram. https://datasheets.raspberrypi.com/pico-R3-A4-Pinout.pdf
RP2040 Data Sheet. https://datasheets.raspberrypi.com/rp240/rp240-datasheet.pdf
Op-Amp Data Sheet. https://www.mouser.com/datasheet/2/268/21882c-8113.pdf
Microphone Data Sheet (Electret Microphone Amplifier – MAX4466 with Adjustable Gain).
https://www.adafruit.com /product/1063?gclid=CjwKCAiA9aKQBhBREiwAyGP5ldcS2LZrDPHO5TP5YRw7A3rMfDp1zmeDc96i9LIHtZKfg1KEYXaRaBoCTlkQAvD_BwE
Sallen-Key Topology. https://www.ti.com/lit/pdf/sloa024
Frequency-to-Note Name Conversions. https://pages.mtu.edu/~suits/notefreqs.html
RP2040 Demo Code from Hunter Adams: https://github.com/vha3/Hunter-Adams-RP2040-Demos/blob/master/Audio/g_Audio_FFT/fft.c

Code and Supporting Files

PUBLISHED IN CIRCUIT CELLAR MAGAZINE • January 2024 #402 – Get a PDF of the issue

Sponsor this Article