Guitar Game Uses PIC32 MCU

Shred Master

Guitar Hero has been an extremely popular game for decades. Many college students who played it when they were kids still enjoy playing it today. These three Cornell undergrads are just such fans. Learn how they used Microchip’s microcontroller and 12-bit DAC to craft their own version, called Shred Master, that lets them play any song they wish by using MIDI files.

Although Guitar Hero was initially released in our elementary-school years, we still find ourselves playing it today. Yet, we always found the setlist limiting. As a group of music-loving Electrical and Computer Engineering (ECE) students, we set out to create our own version that could play any song we wish. Our solution is called Shred Master. Shred Master achieves this goal by utilizing MIDI files, which store information regarding the notes and timing. We parsed the MIDI files for the MIDI messages containing the notes to play, so that we could play the songs back using direct digital synthesis (DDS) on the PIC32. Many songs have free MIDI files online, so when we pre-process the MIDI files and input them into our program, Shred Master has unparalleled flexibility in gameplay (Figure 1).

This article describes our project in three major sections: Music Processing, User Input and Graphics. Music Processing includes an explanation of the MIDI pre-processing in Python and the digital sound generation via DDS. User Input covers how we built our guitar controller, with a custom strummer and button frets. Graphics contains information about our display for the main menu, gameplay and the timing of the notes falling on the screen.

This game utilizes the PIC32MX250F128B microcontroller (MCU) and MCP4822 12-bit DAC, both made by Microchip Technology and both available from Digi-Key. Our program runs on the PIC32 MCU, and the 12-bit DAC is used for sound generation. This program is written in C99 using Protothreads [4], a threading library that has no stack, making it ideal for a system where memory is severely limited and real-time performance is required. The controller makes use of different hardware components that are easily found online. A list of the components we used and their costs is available at the end of this article. Refer to Figure 2 for the hardware schematic of our project.

MUSIC PROCESSING
Music processing was the first part of the project that we worked on. That’s because it was initially our main goal to ensure that the program could play any song with a MIDI file. This required a lot of research into MIDI files, processing them and then achieving digital sound generation via DDS and the DAC through an amplified speaker attached to the PIC32.

We used the freely available Mido Python library [6] to accurately parse the MIDI files—which are binary encoded—into more human-readable messages. Each message can have many different purposes, from signaling that a note should be turned on or off, to indicating the tempo at which the notes should be played. Using a stand-alone Python script, we isolated the note-on messages (which also store the note number indicating the frequency of the note) from all the other messages.

Then, we stored each note’s note number together with a number ranging from 1 to 5, inclusive, indicating in which column (red, green, yellow, blue or orange, respectively) the note should appear on the TFT LCD. The column number was calculated by modding the note number by 12 (the number of notes per octave), and then separating the result into “bins.” We chose the bins empirically, ensuring an even distribution of notes in each bin for our initial song, ”Sweet Child O’ Mine.” By cutting away all other information from within the MIDI files, we maximized the number of songs that could be stored in flash memory on the PIC32 itself.

In every version of Guitar Hero, the setlist quickly becomes the most iconic aspect of the game. With that in mind, we had to ensure the songs the user heard were recognizable and engaging. To accomplish this, we used the note number stored in our song list (as described earlier) as an index into an array of 128 frequencies. These frequencies correspond to the MIDI note frequencies, ranging from about 8 Hz to 12,544 Hz [7]. Using these frequencies, we update a phase accumulator, as shown in Figure 3, used to index into a sine table, represented as a red circle in Figure 3. An explanation for the mathematical details of the DDS is given in the sidebar “Understanding the Phase Accumulator” on page 26.

Subsequently, we scaled this to have a maximum of 4,095 and a minimum of 0 (because we’re limited to the capacity of our 12-bit DAC), sending the new value over SPI to the DAC. We output the DAC to the headphone jack, which plugs into external speakers. This allows us to create any output frequency we desire.

USER INPUT
An evolved form of the air guitar tradition, Shred Master provides a physical guitar-like controller for capturing user input and supplementing the realism of the game (Figure 1). This controller takes the shape of a classic triangular electric guitar, and is made wholly of layered, laser-cut scrap cardboard. This is extremely cost-effective, and is also eco-friendly—consistent in reducing unnecessary waste. The guitar has two primary user interface methods: button frets and the strummer.

The button frets comprise five push-button switches, sourced from Adafruit [8], as shown in Figure 4. Each button is a different color: green, red, yellow, blue and orange. The normally open switches are in series with 330 Ω resistors, which are tied to ground. The inputs are internally pulled up when the switches are open (not pushed) and pulled down when closed (pushed). The five buttons were soldered onto a traditional protoboard, and set about 0.75″ apart in an array. This allows adequate spacing between keys for user ergonomics. The fretboard was then mounted on the guitar base and glued, for adequate support and placement. For both user input types, we use a basic debouncing state machine, shown in Figure 5.

The strummer hardware (Figure 6 and Figure 7) needed to be simple and intuitive, allowing any user to play our game. At a high level, we knew that the strummer had to incorporate a user-satisfying amount of audible and tactile feedback when pressed, alongside precise binary states. We opted for a set of two limit switches, paired in opposite directions from each other and individually wired to different digital input pins on the PIC32 (Figure 6 and Figure 7). This ensures a precise “no-strum” state, where each switch is not pressed and the rocker rests in the middle, tensioned on either side by the limit switches.

Because each limit switch was placed opposite the other, the user can strum up or down to hit a note, thus allowing a choice of ergonomic solutions. The independent wiring of each switch enables us to distinguish up and down strumming motion, allowing the main menu to be especially intuitive. The user simply selects different tracks and difficulty by strumming up or down and then selecting with the green fret. The strummer is debounced in the same way as the button frets (Figure 5).

GRAPHICS
The LCD is the primary human interface device beyond the controller buttons. It shows our game in its entirety—from menus to actual gameplay (Figure 8). Our biggest concern was to ensure that lag and graphics tearing on the screen were not terribly noticeable. We explain ways to minimize this in the section on Note Timing. Careful attention to detail was required at the end of the song, to reset the state and ensure a proper restart, and to provide transitions between the end of game and main menu.

We use state machines not only to button debounce but also to store information for the menu. Our menu contains a welcome screen, song selection screen and difficulty level screen (Figure 8). Each screen is its own state in the main menu state machine. The user can toggle between the screens using the red and green buttons, and can use the strummer for scrolling between choices on each screen (Figure 9).

Based on the user selection, different variables within the program are set. When selecting a difficulty level, the user is really selecting how many notes will be displayed on the screen during the game. We mod the note index by either 1, 4 or 8, and check whether the result is 0 to determine whether or not to display the note. 1 is for the difficult level (because any number modulo 1 is 0, so each note that is heard is also displayed on the screen), and 8 is for easy, because it displays every eighth note. Regardless of the difficulty, each note is played through the DAC, so that the entire song is played. However, the user only has to play the notes according to the difficulty value.

Precisely controlling note timing is a challenging graphics component that enables notes to fall from the screen in time with the music. We wanted to make sure this occurred at a rate that did not create visible lag. The gameplay screen is set up such that the notes fall from the top of the screen to static target circles at the bottom of the screen. The object of the game is to strum and press the correct button fret when the falling note is within the region of its corresponding target. Each note is a different color and, for contrast, has a small white circle in its center.

NOTE TIMING
When the note is played correctly, it disappears. Otherwise, it continues to fall off the screen. The points are displayed at the top of the screen and change during gameplay. The pixel drawing time is about 1.3 μs per pixel. We run our GUI at 15 frames per second (fps), which limits flickering. With a TFT screen of 320×240 resolution, this is close to the maximum refresh rate. This is the reason that we drew our circles relatively small on the screen, rather than scaling them up. At first our circles for notes were large, but this caused some significant flickering. A photo of the notes during gameplay is shown in Figure 10.

Each note is separated from the next by 200 ms in our implementation. This is controlled by using a yield time method that yields after a period determined by the following:

This allows the notes to be nearly 200 ms apart. Because the threader is cooperative, threads are not perfectly periodic, so we must yield as a function of the execution time of the thread. We need to know when a thread starts and how long it takes, which allows us to change the thread timing dynamically. We use 200 ms, because it produces music at approximately the correct BPM (beats per minute) rate for ”Sweet Child O’ Mine” (the first song that we programmed) to sound like the original song. To achieve the correct timing, we would have to parse the information from the MIDI files and replace the 200 ms in the yield time expression with the correct length of time for which each note should be played. We used a struct to keep track of the notes falling on the screen. See Listing 1 for the note struct.

note struct

//position
Int x;
Int y;

//1 if the note should be displayed on the screen, 0 if it is currently being displayed int to_display;

//color of note
unsigned short color;

LISTING 1- The note struct includes fields for the (X,Y) position of the note on the screen, the display variable of the note and the color of the note.

We create an array of notes at the beginning of the game with a 20-note display queue for each of the five color columns. As we go through the song, we determine which column each note should be placed in, based on the display note number in our song array. Depending on the note color, we find its index and values in the notes array in the corresponding column, and initialize it to be displayed. To initialize the note, we set its X value and display value so that it appears on the correct side of the screen. We then iterate through the active note array (for notes in which to_display is set), updating the X position to make each note move down the screen. If the user earns a point, the note will disappear. Otherwise, it will run the entire length of the screen until it goes off-screen and we erase it.

To earn points, the user must be holding down the correct colored note—that is, pressing the correct pushbutton—while strumming either up or down, all at the correct timing. The correct timing is when any part of the falling note is within the correct static note area at the bottom of the screen. To achieve this, we go through each row in our notes array to see if the correct button is being pressed while the falling note is within range of the circle at the bottom of the screen. The value of strumming up or strumming down must be true also. If the user is correct, the points increase by one and update at the top of the screen.

RESULTS & CONCLUSIONS
We created a convincing and quite effective game, in terms of the number of songs we could store, for a total project cost of $93. By representing each note as only two integers, we found that each song was about 4 KB, so we would be able to store around 32 songs with a memory capacity of 128 KB. In addition, we were able to realize a difficulty selection system that required only a single integer and a few clock cycles to calculate a modulo every 200 ms. By spending a significant amount of time working on the menu button interactions, we ended up with a menu system that was easy to navigate. Similarly, for the rocker we ensured that the cursor could always be moved with just a single click.

As for gameplay, we obtained a stable 15 fps throughout the game. Additionally, we combined the note display spawn thread with the sound generation thread, guaranteeing synchronization of the notes and sound. With our system of using frequency bins to determine which of the five colors each note should be, we generated an even distribution of notes among all song choices. Regarding the user inputs, we found that our method of polling the input ports for button inputs every 10 ms gave a responsive game. When it appeared on the screen that the user had pressed the correct note and strummed at the correct time, the note disappeared and the points incremented.

Future modifications would include correcting the note timing to generate more realistic songs and using the built-in MIDI timing specification for sound playback. This would greatly improve gameplay, because complicated timing adjustments could be made on-the-fly as the MIDI file signals. We would also like to make the controller more user friendly by changing the buttons to be larger, closer together, and require a softer press. It would also be nice to display the GUI on a larger monitor rather than the TFT.

To see the game in action, check out our YouTube video below:

Understanding the Phase Accumulator
A phase accumulator is a 32-bit unsigned integer, incremented at a set frequency (50 kHz) akin to sample rate in other forms of digital sound processing. The first element in the sine table is at 0 radian, and the last element is at 2π. We update the phase accumulator at different intervals, depending on the desired output frequency. By incrementing the value by varying amounts at a set frequency, we change how quickly the phase accumulator will reach the end of the sine table and overflow. Each time we overflow can be thought of as traversing a full circle. This is represented in Figure 3 by the phase accumulator arrow moving around the circle, with its tail end pinned to the origin of the axes, from the first element to the last.

On the right side of Figure 3, we show the phase accumulator traversing the sine table (array), simultaneously representing a sine wave as a function of the speed of traversal (phase increment value). With this periodic variable, we index into a sine table with 256 entries, ranging sinusoidally over 0 to 2π, using the top 8 bits of the accumulator (2⁸ = 256) and use the latter 24 bits as fractional samples. By indexing into the sine table at different intervals at the set sample rate of 50 kHz, the outputs from the sine table will form a new sine wave at the desired output frequency given by:

RESOURCES

Link to our Github repository, containing design files and source code:
https://github.com/liampatterson/Shred-Master

The same code and design files are also available on Circuit Cellar’s Code & Files download page:  

References:
[1] Guitar Hero, Guitar Hero II, Guitar Hero III are registered trademarks of Activision Publishing, Inc.[2] NPD 2006 Annual Report, Sales Data[3] NPD 2007 Annual Report, Sales Data
[4] http://dunkels.com/adam/pt
[5] http://people.ece.cornell.edu/land/courses/ece4760/PIC32/target_board.html
[6] https://mido.readthedocs.io/en/latest
[7] http://www.indiana.edu/~emusic/361/midi.htm
[8] https://www.adafruit.com

Adafruit | www.adafruit.com
Digi-Key | www.digi-key.com
Microchip Technology | www.microchip.com

PUBLISHED IN CIRCUIT CELLAR MAGAZINE • SEPTEMBER 2019 #350 – Get a PDF of the issue

Sponsor this Article