Using Wavelet Transform
Microcontrollers are perfect for systems that need to process analog signals such as audio and do real-time digital control in conjunction with those signals. Along just those lines, learn how these two Cornell students recreated the classic arcade game “Dance Dance Revolution” using a Microchip PIC32 MCU. Their version performs wavelet transforms to detect beats from an audio signal to synthesize dance move instructions in real time without preprocessing.
By Michael Solomentsev and Drew Dunne
We designed a version of the traditional arcade game, “Dance Dance Revolution,” that synthesizes dance instructions from any audio source using the PIC32 MCU. Unlike the original game, in which users must choose from a pre-selected list of songs, our system allows users to plug in their audio device and play songs of their choice. The dance move instructions are then generated in real time by buffering the audio and processing it, using the discrete wavelet transform.
We were inspired by a mutual desire to work on a music-related project, and both of us had fond memories of playing this kind of game. We also wanted to add some sort of novel, interesting component, so we brainstormed the idea of allowing the player to play whatever song he or she wanted. The game is much more fun when the song playing is your favorite tune. All versions of the commercial game have pre-programmed song libraries, so replay value is limited. Our version has no such limitation. The discrete wavelet transform was selected as a processing method because we needed both time and frequency resolution. We also needed a computationally efficient algorithm.
The system requires two kinds of user input: an audio source and button presses from the dance mat floor tiles. The audio input must be processed, so it needs to be delayed or buffered until the processing is complete. Another reason for the delay of the audio output is to give ample time for the user to react to the instructions created from processing the audio. In contrast, the user input needs to be in real time. We use two PIC32s to do the input processing—one detects beats and reads the dance mat input, while the other buffers audio. We use a macOS application to display the beats and handle scoring.
We built a custom dance mat for the game, consisting of five individual tiles that could each detect when players put their weight on it (Figure 1). They needed to be both durable and sensitive to pressure. To achieve this, we used force sensitive resistors wired in parallel. These were polled at approximately 20 Hz for changes in resistance. These resistors were placed directly between the tiles—which were made out of canvas covered boards—and the supports that raised them off the ground.
We used a PIC32 development board designed by Sean Carroll, with an DAC socket and GPIO pins brought out, to provide flexibility for development . We also used a second PIC32 on a smaller development board, with connections to the floor tiles and the Serial to USB cable. The floor tiles were wired underneath to a protoboard, and all-important signals were fed up to our main protoboard using a ribbon cable. Figure 2 shows the schematic of the system, incorporating Sean Carroll’s full-size and small PIC32 development boards.
We soldered our audio circuitry on a protoboard to make it easier to debug and to reduce noise. Our audio input jack fed into a 500 µF capacitor to cut out any DC component, then we fed it into an offset circuit, such that the ADCs could read it with no clipping. The DAC output was fed directly to an audio jack and speakers.
Another PIC32 MCU handled audio buffering, because SPI communication with both a 128 KB serial SRAM and DAC took too many cycles to perform the necessary signal processing simultaneously. We used our professor Bruce Land’s code for the SRAM chip for reading and writing to the SRAM and writing to the DAC . His code included some read/write methods, and handled the SPI setup and mode changes. We had to add code to read from the ADC in a timer interrupt at 40 kHz, write to a location in the SRAM, and finally read from a different location and write that value to the DAC. The locations written to and read from were incremented each time, to create a loop around the SRAM memory locations. To change how long we wanted to buffer the audio, we just needed to change the values of MAX_ADDR and MIN_ADDR. The closer together they were, the smaller the range of the SRAM we used. This was important, because using the entire SRAM gave us a buffer of about 3.3 seconds, and we wanted only about 2.5 seconds.
The major consideration that affected our tile construction was a desire for resiliency. Because users would probably stomp on each of the tiles fairly hard, we wanted to make sure that our press detection system could withstand a lot of force. We also wanted a simple, easy solution to mock-up and build.
Initially we looked into using strain gauges, but they would require mounting to a base plate and the tile to be pushed. Traditional buttons did not seem like a robust enough option. Instead, we decided to use force sensitive resistors (FSRs). Initial testing revealed that the unpressed FSRs had resistance of approximately 6 MΩ. When pressed, it was approximately 1 kΩ. This huge discrepancy made it easy to probe it for a press. We thank Interlink Electronics, who were gracious enough to donate 10 FSR402s for use in our project. ..
Read the full article in the November 340 issue of Circuit Cellar
Don’t miss out on upcoming issues of Circuit Cellar. Subscribe today!