Eyes as Interface
There’s much be to learned about how electronics can interact with biological signals—not only to record, but also to see how they can be used as inputs for control applications. With ongoing research in fields such as virtual reality and prosthetics, new systems are being developed to interpret different types of signals for practical applications. Learn how these three Cornell graduates use electrooculography (EOG) to control a simple video game by measuring eye movements.
The human eye naturally acts as a dipole, in which the retina at the back of the eye is negatively charged, and the cornea at the front of the eye is positively charged. EOG is a recording technique that measures this potential difference, and can be used to quantify eye movement [1]. A typical electrode placement pattern for EOG is shown in Figure 1. Each of the electrodes A and B records a voltage related to eye movement, and an electrode at point C serves as a ground reference.
When a user looks left, the cornea is close to electrode B and it records a positive voltage, while the retina is closer to electrode A, yielding a negative voltage. Similarly, looking right produces a negative voltage at B and a positive voltage at A. The difference between VB and VA relative to ground at C changes monotonically with gaze direction, and can be reliably used to model horizontal eye movement.
SYSTEM OVERVIEW
The system we designed uses eye movements to play a video game on a display screen. Electrodes are placed on a player’s head to record only the horizontal EOG signal as shown in Figure 2. This signal is then filtered and amplified via an analog circuit and sent to an ADC on a Microchip Technology PIC32 microcontroller (MCU) (Figure 3). The PIC32 MCU stores the reading as a digital value and uses it to control a cursor on an LCD display screen. A program on the PIC32 continually displays obstacles that move across the screen, and the player moves his or her eyes to control the cursor and avoid obstacles.
This system is entirely powered without connection to an AC power source, instead using a 9 V battery to provide power for amplification and a chargeable power source to power the PIC32. This choice of a power source was important, because it enforces necessary safety considerations for biomedical recording. Connecting a high voltage source to a human user and accidentally completing a circuit path to AC ground could result in serious injury, so great care was taken to use battery power for this project.
A secondary oscilloscope program was also necessarily designed to satisfy a key safety need: The ability to view the recorded EOG signal and test the recording hardware while the circuit is isolated. A normal oscilloscope cannot be used for this purpose for the reasons stated earlier. Care was also taken to apply and fasten the electrodes properly before every session.
— ADVERTISMENT—
—Advertise Here—
RECORDING AND AMPLIFICATION
Three Ag-AgCl (silver-silver chloride) electrodes are placed around the eyes using a skin-safe adhesive gel—one beside each eye, and one on the forehead as a ground reference—at points A, B, and C respectively, in Figure 1. These electrodes provide the gateway between the biological signal and the digital world, detecting the voltage generated by ions at the skin surface and transducing it into an equivalent electron-based signal.
This voltage is generated directly at the eye, and has some attenuation through the skin surface. A typical magnitude of the raw EOG signal is several millivolts. The voltage readings from the two eye electrodes are sent to a Texas Instruments (TI) INA121 differential amplifier, which amplifies the difference between the two input signals. This yields a negative or positive voltage based on direction of eye movement. The INA121 provides low noise, a high common-mode rejection ratio, and is suitable for the high-input impedance requirement associated with recording biological signals. Figure 4 shows the full schematic of the implementation.
A second amplification stage using a TI LM358-based balanced subtractor configuration provides further amplification. This stage reduces the DC voltage component output from the differential amplifier, while further amplifying the difference to a range of 0 to 3.3 V—the scale allowed by the PIC32 MCU’s on-chip ADC. The resulting signal is a voltage centered at approximately 1.6 V when the user looks straight, with about a 1 V increase or decrease when the user looks left or right, respectively.
This circuit includes several RC components for filtering. It is most important for an EOG recording system to be able to detect the frequency of eye motion—primarily about 0-15 Hz, but with components up to 100 Hz [1]. A system should be able to filter lower-frequency noise present in biological systems (typically 0.5 Hz to 30 Hz) and noise from surrounding electrical systems at about 60 Hz, while preserving information of interest below this frequency range [1]. Therefore, low-pass, 2-pole filter elements with time constants of 0.01 seconds (for a cutoff frequency of 16 Hz) are used to reduce noise outside the intended range for recording, at both inputs to the circuit and at the differential amplifier’s feedback component.
A third op amp is used in a unity gain configuration to provide a power bias to our amplifier circuit. The op amp is used to split a 9 V battery into a +4.5 V line and a -4.5 V line for the amplifier. It provides both positive and negative power supply rails for our differential amplifier, and allows for detection of a large voltage swing above or below 0. The unity gain buffer configuration also allows us to use the op amp’s low output impedance to create a low-impedance ground.
balanced subtractor stage
The balanced subtractor stage is also needed in the design to reduce DC offset in amplification. The magnitude of the raw EOG voltage being recorded can differ between individuals, and this difference will manifest as a large, relatively unpredictable DC offset at the output of the differential amplifier. The electrodes used for recording also have an inherent DC offset that can slowly drift during long-term use. The purpose of the balanced subtractor stage is to eliminate this DC offset from our amplified signal, using a voltage reference that is provided as the subtractor’s second input using a potentiometer.
Manually tuning this potentiometer to provide the right voltage and negate the DC component allows the device to capture only the difference resulting from eye movement at a magnitude appropriate for the ADC. The ADC voltage range is 0 to 3.3 V. The subtractor allows us to center the EOG signal at 1.6 V and encode left and right eye movement as an increase or decrease from this baseline value. The flexibility of the potentiometer allows the game to be calibrated for different users, or even as the quality of the recorded electrode voltage changes over time for a single user. Using this configuration, the signal and calibration appeared stable for at least several minutes.
The difference mode gain of the first stage in this implementation was 17.7. The gain of the second stage was 50, yielding a total gain of 885. Two Schottky diodes were used to restrict the voltage to between 0 and 3.3 V as a safety measure for the ADC pin on the PIC32.
The EOG recording system described can be customized and tuned to change the range of the amplified voltage or tolerate higher precision. The calibration sensitivity is determined by a voltage divider, shown in the circuit in Figure 4, where the potentiometer is connected in series with two 5.1 kΩ resistors. If the two 5.1 kΩ resistors are replaced with higher-valued resistors, the DC voltage range determined by the potentiometer is smaller, and the offset can be controlled more precisely.
— ADVERTISMENT—
—Advertise Here—
SOFTWARE DESIGN
To use and test our EOG signal, we developed a video game and oscilloscope test program on the PIC32. The video game was shown on a TFT LCD screen. Its components include a bar-shaped cursor at the bottom of the screen that the user controls via eye movement, and randomly generated obstacles that start at the top of the screen and make their way to the player (Figure 5). The player’s goal is to use eye movement to avoid the oncoming obstacles, scoring points for each obstacle that passes, until the player errs or the game ends after a predetermined time.
The player control system is divided into three discrete regions by thresholds, so that looking center keeps the cursor in the center of the screen, looking left moves the cursor to the left side of the screen, and looking right moves the cursor to the right side of the screen. The obstacles also spawned in one of these three parts of the screen, moving toward the bottom at a constant velocity.
Before starting the game, a calibration stage also allows the user to tune the potentiometer and make sure the control system works properly. During this stage, the user looks at the center of the screen while slowly adjusting the potentiometer. At the correct DC offset, the cursor will appear at the center of the screen instead of one of the sides, and looking left and right will move the cursor properly as described above. This step typically takes only a few seconds, and should be repeated before every game to account for electrode drift.
The game software consists of three concurrently running threads, using the open source ProtoThreads library [2]. The first is the protothread_controller{} function, which periodically updates the position of the user’s cursor via the EOG signal provided by the ADC. This function simply places the cursor in one of three locations on the screen, using two thresholds for the ADC value to determine whether the user is looking right, left or center. Specifically, the EOG signal is centered at about 1.65 V, with a range of 0-3.3 V. This signal is digitally converted to a range of 0 to 1,024, centered at approximately 512.
We set two thresholds at 700 and 300, so that significant eye movement away from center moves the cursor to the left or right side of the screen. This threshold scheme appeared to perform consistently, and avoided any problems arising from finer noise or oscillations in the EOG recording. But these threshold values could still be adjusted as needed, to be closer to or farther from center for greater or lower sensitivity. Once calibrated properly, the game is begun by pressing a button—activating a Boolean variable in software using a pull-down resistor to launch the game.
The second thread, protothread_dynamics{}, randomly spawns obstacles every few frames and updates locations as they progress toward the user. It also determines whether a collision has occurred, and ends the game if the condition is met. The obstacles are stored as an array of structs with X and Y coordinates, using pixels on the screen as Y values and 0, 1 or 2 as X values to indicate the horizontal position. A valid bit indicates that the obstacle should be displayed on the screen.
Every 67 ms, this thread generates a new obstacle, using a pseudorandom number generator to choose the X position. It updates the Y position so that all active obstacles move towards the user’s cursor in Y. A seed for the pseudorandom number generator is set based on the time of the game-starting button press, essentially making the random sequence different on every play. On each iteration, the thread also checks if the user’s X and Y positions are the same as one of the obstacles, setting a “game over” variable if so, enabling the timer thread to display a “game over” message.
The third thread, protothread_timer{}, is a simple timer thread that updates a 1-second tick counter on the game screen, display the game score, and end the game (printing a “Game Over” message) once time is up or a game-ending collision has been detected.
A second program was developed to provide an oscilloscope function. It plots the waveform of the recorded signal over several seconds, which is useful for confirming that the recording is working properly. Information on the software resources we used, including details and resources on borrowed graphics libraries is available on the Circuit Cellar article materials webpage [3]
RESULTS
The final running system (Figure 5) successfully used EOG to control a video game in which the control was heavily separated into thresholds, rather than in a completely continuous control scheme. While use of a continuous control scheme was not specifically tested, the sequence described above appeared to display very stable behavior, and would likely have operated properly if the screen had been divided into smaller subsections.
At a small enough scale, it is difficult to determine whether noise interferes with the PIC32’s ability to determine fine changes in the EOG signal, or whether volatility in the control scheme is due to the user’s own inability to control eye movement precisely. The eyes constantly flicker and produce sporadic movement, introducing noise in and of themselves. However, the EOG clearly has potential for a greater level of precision than what was demonstrated in this project, as indicated by a typically clean oscilloscope signal, even when testing on a real player.
This project implemented only horizontal EOG recording, but electrodes can also be placed above and below the eye to record two-dimensional EOG. Such a control system could open many more possibilities for inclusion in different software systems, with only the added cost of implementing a second amplification circuit with the same configuration as the first. Figure 6 shows a signal shows an amplified sine wave test signal, generated via a function generator and recorded by our oscilloscope program.
The EOG recording method provides an inexpensive, accessible, and portable way of using biomedical signals to control an electronic device or simply record eye movement. One aspect of implementing this scheme is to calibrate it to accommodate different users. The calibration method for this implementation was effective, but still manual. We examined the user’s control of the cursor before the game began, and manually adjusted the potentiometer until it was as expected (the cursor moved left, right and center when the user looked in each respective direction. This adjustment corresponds to proper setting of the subtractor DC input, biasing the recorded signal to center at the middle of the ADC’s voltage range.
Over the course of a given session, the EOG signal displayed drifting, and it did not maintain a constant magnitude over time. This was necessarily due to the instability of the electrodes used, which dry out and lose the integrity of their connection. This can occur when the gel dries over time, shorting the path between the electrode and skin. Therefore, if accuracy is important, this method of implementing EOG may not be suitable for long-term usage without replacing and reaffixing electrodes.
The calibration process could be automated or improved in various ways to adjust for these complications. For example, a combination of a variable resistor controlled by software and sufficient programming to determine this value could result in a completely automated system that accommodates different users. If the EOG signal also doesn’t rail to a single voltage value, a program can be used to set threshold values automatically that correspond to differences in signals between individuals.
— ADVERTISMENT—
—Advertise Here—
This affordable and effective way of recording eye movement could be safely introduced in many applications. EOG could improve the development of virtual reality devices, greatly increasing ease of use and cost in comparison to other methods—such as use of cameras and computer vision techniques to interpret eye movement. EOG could also find effective use in supplementing motor control with prosthetics for any devices that heavily utilize a single degree of freedom in motion—wheelchair rotation, for example.
For detailed article references and additional resources go to:
www.circuitcellar.com/article-materials
References [1] through [3] as marked in the article can be found there
RESOURCES
Adafruit | www.adafruit.com
Microchip Technology | www.microchip.com
Texas Instruments | www.ti.com
PUBLISHED IN CIRCUIT CELLAR MAGAZINE • JULY 2019 #348 – Get a PDF of the issue
Sponsor this Article
Eric Cole is now a Bioengineering PhD student at Georgia Tech, having graduated from Cornell University in May 2018. His research in the Department of Neurosurgery at Emory University applies electrical engineering principles to evaluate and develop novel stimulation methods to treat epilepsy. He also enjoys playing piano and composing musical theatre and jazz.
Alex Huang graduated from Cornell University in May 2018 with a B.S. in Electrical and Computer Engineering, and is currently a Software Engineer at DHPC Technologies. His technical interests include programming in C/C++. During his free time, Alex enjoys going to the gym and DJing.
Evan Mok graduated from Cornell University in May 2018 with a B.S. in Electrical and Computer Engineering, and is currently a physical design engineer at IBM. His technical interests include VLSI design and computer architecture. During his free time, Evan enjoys playing volleyball, bouldering, and road tripping.
