Sensors and Synthesis
A stringless bass guitar isn’t something you’d likely see at your next rock concert. But these three Cornell students used a Microchip PIC32 MCU, together with several break-beam sensors, a distance sensor, a plank of wood and an algorithm for additive synthesis to create just that.
Picking up a new instrument is always a fun endeavor, but it can be expensive and sometimes even physically cumbersome. Realizing the difficulties associated with learning to play the electric bass guitar, we built a device we lovingly deemed AirBass. It’s a sensor-based emulation of an electric bass guitar that allows the user to play distinct notes without the added weight and cost of a real guitar.
At a high level, the functionality of AirBass can be broken into four major components: sensing “strings,” sensing “frets,” plucking “strings” and producing sound. The strings are composed of four pairs of break-beam sensors mounted onto a plank of wood, which models the neck of the guitar (Figure 1). An ultrasonic distance sensor, also attached to the neck, determines the fret position.
The user is able to play notes by bending a finger on the plucking glove retrofitted with flex sensors along four fingers (Figure 2). To read the sensors and perform logic to produce the corresponding bass note, we use a PIC32 microcontroller (MCU) with SPI control of an external MCP4822 DAC and MCP23S17 port expander—all three from Microchip Technology.
The experience of playing AirBass is meant to be similar to playing a real bass guitar. The user needs to place a finger on the desired string-fret combination to select a note. To indicate an open string, no finger should be placed on the fretboard. To produce sound for the note, the user bends the plucking glove finger that corresponds to the string. A full circuit schematic is shown in Figure 3a and Figure 3b.
(Click to enlarge)
{Click to enlarge)
THE “STRINGS”
To synthesize the four individual strings, we aligned four break-beam sensors parallel to the length of the wooden neck. If the user places a finger on one of the strings, the break-beam sensor registers this event, and it narrows down the range of potential sound frequencies to the possible frequencies for the string. Each pair of sensors (Adafruit PN A2167) has both an emitter and a receiver. The emitter sends an IR signal to the receiver, and the receiver sends a signal to the PIC32 MCU, depending on whether or not it has received the signal from the emitter.
When the emitter and receiver are in line, the receiver sends a low signal to the MCU. If the pair is not in line or if something is blocking the signal, then the receiver sends a high signal to the MCU. In the system schematic (Figure 3a), we use a 10kΩ resistor to pull the open drain signal up to 3.3V, and a 1 kΩ resistor in series to protect the MCU pins from any possible spikes in the receiver’s output signal.
— ADVERTISMENT—
—Advertise Here—
The cone of emission and cone of detection for the break-beam sensors is about 10 degrees. This means that if we positioned all receivers on one end of the guitar and emitters in straight-line paths (strings) on the other end, there would be interference from nearby emitter-receiver pairs. That is, if we placed an object in one of the emitter-detector paths, the blocked detector could still read a signal from another emitter’s cone, which would result in the MCU not registering when the user is trying to play a note.
To overcome this issue, we alternated the emitters and receivers on each side of the guitar neck. Furthermore, we angled all the break-beam sensors to capitalize on the cone of their signal transmission and reception, so the edge of each emitter’s cone would be at the edge of the detection range of its receiver’s cone (Figure 4). From these two adjustments, the beam breaks did not interfere with each other, and we could depress each string individually.
A thread runs every 5ms, to keep track of which break-beam pair(s) is broken. This thread runs quickly enough, since no human finger can move or shift faster than a few dozen Hertz, let alone 200Hz. We wired the sensors to separate pins on the port expander, one for each of the four emitter-receiver pairs, which we were able to read using the SPI bus. Later in our program, we use the information about which strings are pressed, to determine if we need to calculate the user’s fret placement as well. If no string is pressed, the desired note is an open string.
THE FRETS
Break-beam sensors can only detect if an object has passed between the pair, not the location of where the object broke the line. Since the positioning of one’s fingers on each string of a guitar is crucial to playing a certain note, we needed to analyze what finger was being placed on the string and at what fret.
To determine which fret the user is playing, we used an ultrasonic distance sensor (Sparkfun SEN-15569). At the end of the 1ms trigger signal, the sensor sends out an eight-cycle burst of ultrasound at 40kHz to potentially reflect off of any objects that are in front of it. If the sensor receives a pulse back, it sets the output signal (the echo) high for a duration proportional to the delay between sending and receiving the pulse. The ultrasonic distance sensor was used to determine the distance from the top of the neck of our bass guitar to the user’s hand.
We attached the sensor to the side of the fretboard. Then we used these distance data to determine at which fret the user was positioned, based on the length of a guitar neck and typical placement for each fret—therefore, in part, to determine which note the user was attempting to play. Note that in our implementation, we assume that the user is playing a single note, because that is most common for an actual bass guitar player.
To get a reading from the ultrasonic distance sensor, we set the trigger signal high for 1ms and then clear it. The minimum length of time the trigger signal had to be high was on the order of microseconds, so the 1ms delay was reasonable.
To read the duration of time the echo signal was high, we used an input capture on the PIC32. An input capture is a hardware timer that measures the time when a specified edge occurs for a given input signal on an input capture pin. This provides an extremely accurate timing of events, up to within a cycle of the signal edge occurrence.
The input capture requires a timer to keep track of the edge of the inputted signal. For our program, we initialized and pre-scaled a timer to take continuous readings without the result overflowing. We connected an interrupt service routine (ISR) to the input capture to read its value on the falling edge of a signal. Using an ISR gives a timely reading of the distance sensor, so that we get the most accurate readings possible from the input capture. Taking values from the falling edge of the signal only provides a distance once the sensor receives the entirety of a returned pulse.
— ADVERTISMENT—
—Advertise Here—
Since the ISR preempts any other threads from running at that time, we took care to make the ISR as brief as possible, to avoid occupying the processor for too long. The ISR only ran two lines of code—one to save the value off the capture register when the sensor received the reflected pulse, and another to clear the input capture interrupt flag. After sending the trigger pulse, the timer was reset to zero so that no additional arithmetic had to be performed to find the length of time between the 40kHz pulses and the reflected pulse.
We noticed a large amount of nondeterministic fluctuation in the measured distance. To ameliorate this problem, we applied a simple four-sample averaging filter with response:
When reading the echo pin from the sensor on the oscilloscope, we noticed some noise. The theory behind this is that it’s caused by the switching within the sensor, itself, most likely due to the megahertz clock. We built a low-pass voltage divider to filter the signal and to step down the voltage to 3.3V, as shown in the schematic (Figure 3b). We used a ceramic capacitor with a time constant of around 1ms to filter out anything above 1MHz.
After determining which string was plucked, we needed to figure out the dispersion of the distance sensor at each fret. After working with the distance sensor, we noticed that the distance values changed based on the positioning of the object at which it was looking. In the case of our system, this object was the side of a hand. A human hand is extremely hard to maintain in a static position while presenting the same amount of surface area, despite the string and fret at which the hand is positioned. To remedy this, we measured the distance sensor’s output across various combinations of depressed strings and frets. The calibration data for this are shown in Table 1.
PLUCKING STRINGS
When analyzing the output of the flex sensors, the only relevant information was binary: Is a string being plucked or not? Accordingly, we designed a circuit to convert this analog output voltage to a digital high or low signal. The constraints were that: (1) it had a relatively fast response, and (2) it had some hysteresis, so that a user has to un-flex a finger intentionally to stop plucking the string, rather than mechanical effects of the sensor causing this accidentally. To achieve these objectives, we used the inverting Schmitt trigger, shown in the schematic (Figure 3b).
The output voltage of the voltage divider from the flex sensor fed into the inverting Schmitt trigger on the inverting terminal of the amplifier. The resulting circuit, with resistance values as stated above, has a high threshold of 2.1V and a low threshold of 1.5V. That is, when the output is currently low (sitting at 0V) and the input (V-) to the inverting terminal goes above 1.5V, the circuit triggers and brings the output to 3.3V. When the output is currently high and the input goes below 2.1V, the circuit triggers and brings the output to 0V.
Because our voltage divider ranges over values from about 1.2 to 2.2V, these thresholds are within our capabilities. We thus effectively have an integrated circuit, whose output will be digital high when the sensor is flexed and low when it is not flexed. Each flex sensor has its own Schmitt trigger circuit feeding into an individual pin on the PIC32 (Figure 3a). Although we saw this worked reasonably well, for some of the sensors, un-flexing to go back below the high threshold was a little difficult. All the sensors don’t have the same range of resistances, and for one sensor in particular, it was difficult to make the digital high output go low again, though it worked easily for the others.
PRODUCING SOUND
When the user bent a finger on the plucking glove, we determined which finger was bent, which of the break-beam sensors was broken, and how far down the neck the user’s hand was, to determine which note to play. The resulting sound was then calculated using conditional logic, and outputted via the on-board MCP4822 DAC through additive synthesis. The timer controlling the synthesis was configured to interrupt 909 times a second (equal to 40MHz divided by our hard-coded sampling frequency, 44kHz), a standard frequency for musical synthesis. We used a low-pass filter to remove possible high-frequency noise in the DAC output to the speakers, with a roll-off of 1,600Hz. The final result had limited noise artifacts and matched the expected frequency, amplitude and duration, no matter the note in question.
To have all the frequencies of possible sounds at the user’s fingertips, a sine look-up table over all the values in a complete sine wave was calculated. Using additive synthesis, we produced five harmonics with hard-coded weights, which we added to the values in the sine look-up table to produce a more string-like sound. A DDS phase increment array was filled with frequency values to match the specific string/fret combinations of a bass guitar. This array contained 23 values, since each string had an open frequency and six frets. This led to approximately bass-like sounds from all this information.
Playing the fifth fret on a string produced a sound equivalent to the next open string. For example, if E were plucked, the increment was five less than if A were plucked. The distance ranges from the ultrasonic distance sensor were used in a series of conditional statements to set the fret for each individual string being depressed. If none of the four strings was depressed—meaning that zero of them had their break-beam broken—then an open string was the only possible note. If at least one of the four strings was being depressed, then the fret variables were set if: 1) that specific string was depressed, and 2) the distance sensor reading was within the range for that fret. We tried to allow playing several notes simultaneously. Since we pre-computed the values in the look-up table, they were phase locked, leading to an output that was more noise than actual sound.
Now that the frequency of the sound had been set, it was time to set the amplitude envelope of the sound wave, itself. This was also based on the flex sensor glove output. Boolean state variables indicated whether or not one of the four strings was currently being plucked or had been plucked in the previous read. If we transitioned from nothing plucked to some string plucked, we started timing a new note/amplitude envelope. Similarly, if we transitioned from something plucked to nothing plucked, we started a new envelope decay.
In the audio ISR, if the amplitude envelope was in the amplifying state and wasn’t at its maximum amplitude, or in the decaying state and not at an amplitude of zero, the amplitude was increased or decreased, respectively. The attack portion of the sine wave increased linearly, whereas the decay part of the sine wave decreased linearly. The result of this logic gives the ability to play sustained notes with the attack and decay part of the envelope, so that it sounds less harsh and ideally more like an actual bass guitar.
PLAYABILITY AND LIMITATIONS
Our final system accomplished our goal of building a playable synthetic bass guitar. A demo video of AirBass is shown below. There are ways, though, that it can be improved for the future. First, the distance sensor was hard to calibrate. If the user’s hand was tilted differently than how it had been when calibrating—reducing or increasing the amount and angle of the surface area of the side of the hand—the distance measured would change slightly. This slight change caused the program to detect an incorrect note. Possible fixes include adding multiple distance sensors—one for each string (to accommodate for the different readings at each string position), or replacing our averaging filter with a software median filter, to better respond to incorrect spikes or valleys in the signal.
Second, the flex sensor glove exhibited hysteresis when flexing or un-flexing each finger, meaning that the user had to fully straighten a finger to un-pluck a string. This could be remedied by adjusting the thresholds of the Schmitt trigger, so that a partially un-flexed finger would be enough to drive the output high. Not only would this prevent the user from having to strain individual fingers, but it would also be more authentic to the plucking motion of the glove. Replacing the direct connection between the glove and the board with a wireless connection could potentially increase the user’s range of motion, making flexing fingers a simpler task. It could also close the gap between AirBass and an actual bass guitar, by keeping the guitar and human relatively separate.
Third, the physical layout of AirBass was not as true to that of an electric bass guitar as originally intended. In the future, we could nestle the break-beam sensors inside the wood, so they could be closer together on the fretboard, allowing more room for the user’s hand to reach all the frets comfortably. In our design, the width of the neck was necessary to prevent the break-beam receivers/emitters from overlapping one another and detecting a broken beam incorrectly—but it made playing somewhat awkward.
— ADVERTISMENT—
—Advertise Here—
Finally, although the use of five harmonics produced a string-like sound, we could slim-down the difference between a real bass guitar and AirBass even more. Implementing the Karplus-Strong algorithm when producing our DAC output might have made a sound more akin to plucking a physical string. This would have turned our synthetic bass guitar sounds into notes with more richness and cadence, like those in a true bass guitar string. However, to use this method, the sampling frequency would need to be reduced from 44kHz to around 20kHz.
Even without these proposed improvements, the finished prototype for AirBass exhibits the qualities we set out to implement. It uses a collection of various sensors to replicate the motions and decisions a user must make to play an electric bass guitar, while keeping the cost relatively low in comparison, and requiring less physical effort to play. Users are able to play notes on all four strings and pluck their desired melody without dealing with physical strings or frets. For those looking for an inexpensive alternative for a beginner bass guitar, AirBass is the solution. 
RESOURCES
“Inverting Schmitt Trigger Calculator.” Random Science Tools, 2018,
www.random-science-tools.com/electronics/inverting-schmitt-trigger-calculator.htm
“Schmitt Trigger.” Wikipedia, Wikimedia Foundation, 8 Feb. 2020
www.en.wikipedia.org/wiki/Schmitt_trigger
Instructables. “Simple Arduino and HC-SR04 Example.” Instructables, 26 Oct. 2017
www.instructables.com/id/Simple-Arduino-and-HC-SR04-Example
Bruce Land “Time Measurement and Control Plus Clock Generation. PIC32MX250F128B.” ECE4760 Timers PIC32, 2019
http://people.ece.cornell.edu/land/courses/ece4760/PIC32/index_Timers.html
Bruce Land “Fixed Point DSP Functions in GCC.” DSP for GCC, 2013, EE476
http://people.ece.cornell.edu/land/courses/ece4760/Math/avrDSP.htm
Bruce Land “PIC32MX250F128B I/O Pin Details including Peripheral Pin Select and Configuration Bit Details.” PIC32 Details
http://people.ece.cornell.edu/land/courses/ece4760/PIC32_details.html
Bruce Land “Development Boards, PIC32MX250F128B.” Dev Boards ECE4760, 2020
http://people.ece.cornell.edu/land/courses/ece4760/PIC32/target_board.html
Julius O. Smith “The Karplus-Strong Algorithm.” The Karplus-Strong Algorithm, 2010
https://ccrma.stanford.edu/~jos/pasp/Karplus_Strong_Algorithm.html
Adafruit | www.adafruit.com
Microchip Technology | www.microchip.com
Sparkfun | www.sparkfun.com
PUBLISHED IN CIRCUIT CELLAR MAGAZINE • NOVEMBER 2020 #364 – Get a PDF of the issue
Sponsor this Article
Peter Cook (pac256@cornell.edu). Peter is currently a senior at Cornell University, working toward finishing his Bachelor of Science in Electrical and Computer Engineering. He’s also planning on completing a Masters of Engineering in the same field by December 2020. His main interests include digital signal and image processing, machine learning, digital hardware and playing the trombone.
Jackson Kopitz (jsk363@cornell.edu). Jackson is currently a senior at Cornell University. He will be receiving a Bachelor of Science in Electrical and Computer Engineering in May 2020 and a Master of Engineering in Systems Engineering in December 2020. Jackson is most interested in the intersection of computer science and sustainability.
Caitlin Stanton (cs968@cornell.edu). Caitlin is currently a senior at Cornell University and will be finishing her Bachelor of Science in Electrical and Computer Engineering in May 2020. She’ll be pursuing a Master of Engineering in Electrical and Computer Engineering, with an expected graduation date of May 2021. She’s passionate about embedded systems and diversity in technology.
