PIC32 Tames Real-Time Stock Monitoring

Market Matador

With today’s technology, even simple microcontroller-based devices can fetch and display data from the Internet. Learn how these two Cornell students built a system that can track stock prices in real time and display them conveniently on an LCD screen. For the design, they used an Espressif Systems ESP8266 Wi-Fi module controlled by a Microchip PIC32 MCU.

We challenged ourselves to build a system using the PIC32 microcontroller (MCU) from Microchip Technology that could track stock prices in real time and display them. The goal was to create a PIC32 system that connects to the Internet and can work as a server/client to perform several functions and eventually serve as a central home hub. The system can be easily modified to fetch and display any kind of data from the Internet, as long as there is an API for it.

The rationale behind this project was that there are few libraries or applications of the PIC32 using the Espressif Systems ESP8266. Both the chips individually are highly capable, inexpensive and can be used for even large-scale manufacturing. We wanted to create a prototype PIC32 system that has Internet connectivity and can be easily extended to perform a multitude of things. We used “Protothreads”, a lightweight threading library created by Adam Dunkels [1], to make our system efficient and capable of handling a variety of tasks simultaneously.

SYSTEM DESIGN
The system works as a TCP server that connects to a Python Client and fetches real-time stock information for any company the user inputs. The input is a 12-digit keypad that works like a cell phone keyboard. The user inputs the stock symbol for a company, and the system displays the stock price on an LCD monitor, along with the corresponding arrows for increase or decrease in price relative to the last fetch. A high-level block diagram for our system design is given in Figure 1. In essence, we first wait for the Python Client to connect to the server. Until this happens, the server remains idle. The user can then input a stock symbol using the keyboard at any time, which triggers an API call. The display is then updated with the price of the stock, and refreshes automatically every 5 seconds.

We made use of Sean Carroll’s Development Board, which contains a TFT LCD, the PIC32 MCU, several peripheral pins and support for a port expander. We connected the ESP8266 transmit and receive pins to RA1 and RB10, respectively, on the PIC, as these pins support UART. We used UART channel 2 for communication with the ESP8266. The LCD communicates with the PIC via SPI channel 1 on pin SCK1.

To use the 12-digit keyboard, we made use of a port expander. This allowed us to wire the keyboard to a series of seven consecutive pins on the PIC, which helped us to write more concise code. Additionally, by using the port expander, we were able to refer to Bruce Land’s sample code [2], which helped serve as a template for writing a bitmask lookup table. The keyboard worked by reading bits on pins RY0-RY6, which were the seven pins used by the port expander. They were then compared against the entries in the bitmask lookup table. This is how readings were made by our keyboard thread, which we discuss further in a later section of this article. The hardware schematic is shown in Figure 2. A link to a more detailed diagram of Sean Carroll’s Development Board is available here. We did not use external resistors for pins RY4-RY6, because the port expander has internal pull up resistors that we enabled in software. The bit readings for these three pins were then active low, so we inverted our logic to correctly register key presses.

SYSTEM COMMUNICATION
You can program and communicate with the ESP8266 Wi-Fi Module using AT Commands. Numerous AT Commands for this module are provided in the ESP8266 datasheet. We sent AT commands from the PIC32 to the ESP module using UART Serial communication. We would send strings (or rather, character arrays) from the PIC32 to the ESP module containing the AT Command, and would await a response on the receive line of the PIC.

The ESP communicated with our Python Client (independent of OS) via a socket connection. The client connected to the module at its IP address on port 333 (the default when the ESP module is set up as a server). The ESP module sent this client our custom commands based on user input on the keypad. The client received these commands, retrieved the necessary information (described in the next section), and sent this information back over the socket connection. The socket communication client side was handled by the standard Python socket library.

Next, our Python Client communicated with the Intrinio Web API using HTTP GET Requests. Based on the command it received from the ESP, the client made the necessary GET request, formatted the response, and sent it back to the ESP module. The GET requests were formatted based on the API requirements, and contained our Intrinio API key. On an API call, Intrinio returned a JSON string with the necessary information. The client then parsed this JSON string to get the stock price. The overall communication of our project is shown in Figure 3.

SOFTWARE THREAD STRUCTURE
Our software for this project consisted of three distinct threads: a main thread, one to set up the ESP module and one to handle keyboard input. We also had a function to make an API call from the PIC MCU.

Main Thread: To make the software design more structured and the software workflow clearer, we developed a “main” protothread. This thread first spawned the ESP Init thread, and then waited for a connection from our client, blocking the rest of the system until this requirement was satisfied. Once it had detected the client had connected, it would then check to see if a stock symbol had been entered. If so, it made an API call which updated the corresponding stock price on the LCD in real time.

ESP Init Thread: To set up the ESP module, we had to send it various AT commands before we began to use it in our application. To check if the module worked, simply sending it “AT” via serial—which can easily be done using Putty or the Arduino Serial Monitor—and receiving the string “OK” verified its proper operation. Next, we reset the module, using “AT+RST” to ensure any previous settings would not interfere with our current setup.

Once we ensured the module worked and was reset, the next step was to connect the module to a Wi-Fi network. We used RedRover, a free Wi-Fi network available at Cornell University. To connect this module to RedRover, we first registered the device with Cornell IT, and then sent the AT command “AT+CWJAP_DEF=RedRover” to the module. It should be noted that RedRover does not require a password to join the network, but the AT Command can accept a password argument. After it was connected, we got the IP address of the module using “AT+CIFSR.” Next, we enabled the ESP module to have multiple connections using “AT+CIPMUX=1” and configured the ESP module as a server using “AT+CIPSERVER=1”. This thread sent this series of AT commands, and was spawned once at system start.

Keyboard Thread: Our keyboard thread works by reading bits from pins RY0-RY3 on the port expander (which map to horizontal rows of the keyboard) on the PIC, and then the bits from RY4-RY6 (which map to vertical columns). The bit readings from these two groups are then ORed together and the value compared against a lookup table, which we defined locally in our thread. We would have used a different set of pins RA0-RA3 and RB7-RB9 on the PIC32. However, due to a usage conflict with other pins on the ESP module, we used a port expander.

We could not use a series of seven consecutive pins on the MCU outright, due to these conflicts with the ESP. Merely swapping single pins where conflicts existed would mean different bit masks and subsequently produce different values when the row and column values were ORed together. These would not match any of the lookup table entries. One solution to this problem was to manually calculate the expected values from these OR operations and create a new bitmask lookup table. However, we felt a cleaner solution involved using the port expander. For this, we referred to the example code in [2], which initialized and set up the port expander and the main keypad scanning logic.

To translate keypad number presses to letters, we made use of a large case statement. We had 27 cases—one for each letter of the alphabet, plus an additional space character. In our logic, once the pound sign was pressed, which worked as our “enter“’ button, the current value in a running buffer was passed as the argument to this case statement. The letter corresponding to the numbers in the buffer was then stored in a second ticker symbol buffer. Once a total of four characters (which is the maximum length for an NYSE stock symbol) had been input, the buffers reset, and an API call was made using that ticker. The keypad thread continuously checked if any input had been received from the keypad.

Reading and Sending via UART: API Call: Reading and Sending messages via UART between the PIC32 chip and the ESP Wi-Fi Module was tricky. To do these things, we used the function DMA_PutSerialBuffer provided by [3] and a heavily modified version of GetSerialBuffer. The DMA_PutSerialBuffer function sent the string placed in PT_send_buffer through UART to the ESP module using DMA one byte at a time. Modifying the GetSerialBuffer function was tricky, because we could not find any documentation on how the ESP module responses were terminated. After experimenting, we concluded that most responses terminated with a ‘\n’ (new line) and ‘\r’ (carriage return) character in succession. We read up to 200 characters from the buffer (which was slightly more than the largest response we expected to receive), and stopped reading as soon as we saw that terminator. As each character was read, it was put into a character array that could be used by other functions. At the beginning of this function, we also cleared all UART2 errors.

Unfortunately, this method did not work for receiving stock price responses, which was the most important function of our project. To implement this task, we created a separate function, called APICall(), which was responsible for sending a custom command to the ESP via serial, based on the stock ticker entered on the keypad. The stock price was returned and displayed with its symbol and a triangle to indicate how the price instantaneously changed on the TFT LCD. To receive stock price responses, we read a preset number of characters from the buffer, instead of relying on a terminator. This was feasible because the response containing the stock price was always the same number of characters. We then parsed this response to get only the stock price, so we could display it and compare it with the last price.

RESULTS AND CONCLUSION
In conclusion, we finished with a reliable framework for making API calls to financial data servers. The latency for a call was roughly 1 s. The stock prices were updated every 10 s (manually set to avoid spamming the API services and reaching the daily call limit), provided we left the PIC running without requesting a different stock quote. Initial configuration of the chip took about 2 s, meaning our system had relatively small startup costs and could make quick and accurate updates in real time. Additionally, stock prices were reported to two decimal points accuracy.

Figure 4 shows some of the results. Overall, the results of the project met our expectations, despite various complexities along the way. In a broad sense, our project served as a proof of concept for lightweight wireless communication projects using Wi-Fi over the ESP module. Protothreads made our code more efficient, organized, and easier to understand.

It is important to note the extensibility of the PIC32 MCU to other projects and applications. In general, this project served as demonstration of the PIC32 and the ESP as a TCP server and client. This manner of functionality could be easily applied to other peripheral circuits or additional modules. Wireless communication makes many new functions possible, and allows us to increase the usability and relevance of our smart home merely by incorporating additional API calls and data requests to various external servers.

In a similar fashion to our stock quote requests, we could make API calls to servers of the national weather service and display temperature, environmental conditions, and general weather data on our TFT display. We could take this even further by then analyzing the weather data and, based on the conditions, stream music that fit the mood of the weather. Streaming would occur via Wi-Fi communication with the ESP module.

Aside from the incorporation of API calls and data requests to external servers, Wi-Fi communication with the PIC could be used in various other applications. It is extremely versatile, so any sort of light display, temperature control circuit, home security unit, or IoT application could be implemented using the ESP module and serial communication over the PIC. 

Authors’ Note: Special thanks to our team member, Shrinidhi Kulkarni, for his contribution to the development of this project. Shrinidhi is a second-year masters student studying Applied and Engineering Physics at Cornell University. He helped develop some of the code for this project, and provided some reference text for this article.

RESOURCES

References:
[1] A. Dunkels, “Protothreads” http://dunkels.com/adam/pt
[2] B. Land, “TFT_KEY_expander_shift_BRL4.c,” Cornell University.
http://people.ece.cornell.edu/land/courses/ece4760/PIC32/Target_board/TFT_KEY_expander_shift_BRL4.c
[3] B. Land, “Cornell University ECE4760 Uart Serial,” Cornell University.
http://people.ece.cornell.edu/land/courses/ece4760/PIC32/index_UART.html

Sources/BOM:

Sean Carroll’s PIC32 development Boards http://people.ece.cornell.edu/land/courses/ece4760/PIC32/target_board.html

“ESP8266 Module (WRL-13678) Datasheet,” Sparkfun Electronics.

Rudinskiy, “Wi-Fi Communication Using ESP8266 & PIC32,”
Cornell University, 2014.

“PIC32 Family Reference Manual – Section 21. UART,” Microchip.

“Socket Programming HOWTO,” Python Software Foundation, 2018.

Espressif Systems | www.espressif.com
Microchip Technology | www.microchip.com
SparkFun | www.sparkfun.com

PUBLISHED IN CIRCUIT CELLAR MAGAZINE • MAY 2019 #346 – Get a PDF of the issue

David Valley & Saelig Khattar

+ posts

A graduate of Cornell University, Saelig Khattar is currently a graduate student in Electrical Engineering at Stanford University and is a research assistant in the Stanford AI Lab. In his free time, Saelig enjoys playing tennis and eating soup dumplings.

David Valley graduated from Cornell last fall with a degree in Electrical and Computer Engineering. Some of his interests in electronics include embedded systems programming, PCB design and computer architecture. When he is not busy with classwork, he enjoys playing guitar, making playlists on Spotify and spending time outdoors. He will begin work as a software engineer this August.

This author does not have any more posts.