CC278: Evolving Neural Networks in Robotics

ccpostrobotAre you curious about how an evolving neural network helps a robot learn about itself and its environment?


A neural network with two inputs, one output, and three hidden neurons.

In the September issue of Circuit Cellar, Walter O. Krawec begins a two-part series that describes an ENN he uses in robot development experiments, explains how short-term memory (STM) evaluates a network’s conditions and how to add data to STM, and discusses how an ENN uses a robot’s minimalistic “instincts” and “reflexes” to guide a robot’s evolution.

Krawec, who has been building robots since 1999, is a research assistant and PhD student in Computer Science at the Stevens Institute of Technology in Hoboken, N.J. The work presented in his two-part series is based on a paper published in the proceedings of the 13th International Artificial Life Conference in 2012.

The overall goal of the Krawec’s experiments in developmental robotics is to enable a robot to learn on its own without human intervention. “An ENN is used to accomplish this,” he says.  “This network will be capable of growing and learning in real time as the robot operates.”

In his series, Krawec presents an architecture he says “enables a robot to ‘grow’ from a naive individual with no knowledge of itself (i.e., no notion of what its sensors are reporting or what its outputs actually do) to one that can operate in an environment.”

“This architecture will consist of an evolving neural network (ENN), a short-term memory (STM), and simple instincts and reflexes.

“Despite a minimal set of instincts, which provide penalties and rewards for certain actions (e.g., crashing into a wall, the robots described in this article sometimes develop complicated and unexpected behaviors. Such behaviors range from following walls (despite the robots’ binary proximity sensors) to games of ‘follow the leader.’…

“This article explores basic artificial neural network (ANN) concepts and outlines the ENN I’m using in this project. This is a neural network that, over time, learns not only by adjusting synaptic weights but also by growing new neurons and new connections (generally resulting in a recurrent neural network). Finally, I’ll discuss the STM system and how it is used to evaluate a network’s fitness.”

The second article in Krawec’s series appears in Circuit Cellar’s October issue.

“In Part 2, I’ll examine the reflex and instinct system, which feeds reward information to an ENN and the ‘decision path’ system, which rewards or penalizes chains of actions,” Krawec says. “Finally, I’ll discuss experiments conducted to demonstrate this architecture in a simulated environment. In particular, I’ll describe some interesting behaviors that robots have developed in trial runs.”

For more, check out Krawec’s articles on “Experiments in Developmental Robotics” in the September and October issues. You will also find information and videos about his work with robots on his website.


AAR Arduino Autonomous Mobile Robot

The AAR Arduino Robot is a small autonomous mobile robot designed for those new to robotics and for experienced Arduino designers. The robot is well suited for hobbyists and school projects. Designed in the Arduino open-source prototyping platform, the robot is easy to program and run.

The AAR, which is delivered fully assembled, comes with a comprehensive CD that includes all the software needed to write, compile, and upload programs to your robot. It also includes a firmware and hardware self test. For wireless control, the robot features optional Bluetooth technology and a 433-MHz RF.

The AAR robot’s features include an Atmel ATmega328P 8-bit AVR-RISC processor with a 16-MHz clock, Arduino open-source software, two independently controlled 3-VDC motors, an I2C bus, 14 digital I/Os on the processor, eight analog input lines, USB interface programming, an on-board odometer sensor on both wheels, a line tracker sensor, and an ISP connector for bootloader programming.

The AAR’s many example programs help you get your robot up and running. With many expansion kits available, your creativity is unlimited.

Contact Global Specialties for pricing.

Global Specialties

Microcontroller-Based, Cube-Solving Robot

Cube Solver in ActionCanadian Nelson Epp has earned degrees in physics and electrical engineering. But as a child, he was stumped by the Rubik’s Cube puzzle. So, as an adult, he built a Rubik’s Cube-solving robot that uses a Parallax Propeller microcontroller and a 52-move algorithm to solve the 3-D puzzle.

Designing and completing the robot wasn’t easy. Epp says he originally used a “gripper”-type robot that was “a complete disaster.” Then he experimented with different algorithms–“human memorizable ones”—before settling on a solution method developed by mathematician Morwen Thistlethwaite. (The algorithm is based on the mathematical concepts of a group, a subgroup, and generator and coset representatives.)

Nelson also developed a version of his Rubik’s Cube solver that used neural networks to analyze the cube’s colors, but that worked only half the time.

So, considering the time he had to spend on project trial and error (and his obligations to work, family, and pets), it took about six years to complete the robot. He writes about the results in the September issue of Circuit Cellar magazine. 

Here, he describes some of the choices he made in hardware components.

“The cube solver hardware uses two external power supplies: 5 VDC for the servomotors and 12 VDC for the remaining circuits. The 12-VDC power supply feeds a Texas Instruments (TI) UA78M33 and a UA78M05 linear regulator. The UA78M05 regulator powers an Electronics123 C3088 camera board. The UA78M33 regulator powers a Maxim Integrated MAX3232 ECPE RS-232 transceiver, a Microchip Technology 24LC256 CMOS serial EEPROM, remote reset circuitry, the Propeller, a SD/MMC card, the camera board’s digital output circuitry, and an ECS ECS-300C-160 oscillator. The images at right show my cube solver and circuit board.
“The ECS-300C-160 is a self-contained dual-output oscillator that can produce clock signals that are binary fractions of the 16-MHz base signal. My application uses the 8- and 16-MHz clock taps. The Propeller is clocked with the 8-MHz signal and then internally multiplied up to 64 MHz. The 16-MHz signal is fed to the camera.

“I used a MAX3232 transceiver to communicate to the host’s RS-232 port. The Propeller’s serial input pin and serial output pin are only required at startup. After the Propeller starts up, these pins can be used to exchange commands with the host. The Propeller also has pins for serial communication to an EEPROM, which are used during power up when a host is not sending a program.

“The cube-solving algorithm uses the coset representative file stored on an SD card, which is read by the Propeller via a SparkFun Electronics Breakout Board for SD-MMC cards. The Propeller interface to the SD card consists of a chip select, data in, data out, data clock, and power. The chip select is fixed into the active state. The three lines associated with data are wired to the Propeller.

“The Propeller uses a camera to determine the cube’s starting permutation. The C3088 uses an Electronics123 OV6630 color sensor module. I chose the camera because its data format and clocking speed was within the range of the Propeller’s capabilities. The C3088 has jumpers for external or internal clocking.”

To read more about Epp’s design journey—and outcomes—check out Circuit Cellar’s September issue. And click here for a video of his robot at work.


CC 276: MCU-Based Prosthetic Arm with Kinect

In its July issue, Circuit Cellar presents a project that combines the technology behind Microsoft’s Kinect gaming device with a prototype prosthetic arm.

The project team and  authors of the article include Jung Soo Kim, an undergraduate student in Biomedical Engineering at Ryerson University in Toronto, Canada, Nika Zolfaghari, a master’s student at Ryerson, and Dr. James Andrew Smith, who specializes in Biomedical Engineering at Ryerson.

“We designed an inexpensive, adaptable platform for prototype prosthetics and their testing systems,” the team says. “These systems use Microsoft’s Kinect for Xbox, a motion sensing device, to track a healthy human arm’s instantaneous movement, replicate the exact movement, and test a prosthetic prototype’s response.”

“Kelvin James was one of the first to embed a microprocessor in a prosthetic limb in the mid-1980s…,” they add. “With the maker movement and advances in embedded electronics, mechanical T-slot systems, and consumer-grade sensor systems, these applications now have more intuitive designs. Integrating Xbox provides a platform to test prosthetic devices’ control algorithms. Xbox also enables prosthetic arm end users to naturally train their arms.”

They elaborate on their choices in building the four main hardware components of their design, which include actuators, electronics, sensors, and mechanical support:

“Robotis Dynamixel motors combine power-dense neodymium motors from Maxon Motors with local angle sensing and high gear ratio transmission, all in a compact case. Atmel’s on-board 8-bit ATmega8 microcontroller, which is similar to the standard Arduino, has high (17-to-50-ms) latency. Instead, we used a 16-bit Freescale Semiconductor MC9S12 microcontroller on an Arduino-form-factor board. It was bulkier, but it was ideal for prototyping. The Xbox system provided high-level sensing. Finally, we used Twintec’s MicroRAX 10-mm profile T-slot aluminum to speed the mechanical prototyping.”

The team’s goal was to design a  prosthetic arm that is markedly different from others currently available. “We began by building a working prototype of a smooth-moving prosthetic arm,” they say in their article.

“We developed four quadrant-capable H-bridge-driven motors and proportional-derivative (PD) controllers at the prosthetic’s joints to run on a MC9S12 microcontroller. Monitoring the prosthetic’s angular position provided us with an analytic comparison of the programmed and outputted results.”

A Technological Arts Esduino microcontroller board is at the heart of the prosthetic arm design.

The team concludes that its project illustrates how to combine off-the-shelf Arduino-compatible parts, aluminum T-slots, servomotors, and a Kinect into an adaptable prosthetic arm.

But more broadly, they say, it’s a project that supports the argument that  “more natural ways of training and tuning prostheses” can be achieved because the Kinect “enables potential end users to manipulate their prostheses without requiring complicated scripting or programming methods.”

For more on this interesting idea, check out the July issue of Circuit Cellar. And for a video from an earlier Circuit Cellar post about this project, click here.


Electrostatic Cleaning Robot Project

How do you clean a clean-energy generating system? With a microcontroller (and a few other parts, of course). An excellent example is US designer Scott Potter’s award-winning, Renesas RL78 microcontroller-based Electrostatic Cleaning Robot system that cleans heliostats (i.e., solar-tracking mirrors) used in solar energy-harvesting systems. Renesas and Circuit Cellar magazine announced this week at DevCon 2012 in Garden Grove, CA, that Potter’s design won First Prize in the RL78 Green Energy Challenge.

This image depicts two Electrostatic Cleaning Robots set up on two heliostats. (Source: S. Potter)

The nearby image depicts two Electrostatic Cleaning Robots set up vertically in order to clean the two heliostats in a horizontal left-to-right (and vice versa) fashion.

The Electrostatic Cleaning Robot in place to clean

Potter’s design can quickly clean heliostats in Concentrating Solar Power (CSP) plants. The heliostats must be clean in order to maximize steam production, which generates power.

The robot cleaner prototype

Built around an RL78 microcontroller, the Electrostatic Cleaning Robot provides a reliable cleaning solution that’s powered entirely by photovoltaic cells. The robot traverses the surface of the mirror and uses a high-voltage AC electric field to sweep away dust and debris.

Parts and circuitry inside the robot cleaner

Object oriented C++ software, developed with the IAR Embedded Workbench and the RL78 Demonstration Kit, controls the device.

IAR Embedded Workbench IDE

The RL78 microcontroller uses the following for system control:

• 20 Digital I/Os used as system control lines

• 1 ADC monitors solar cell voltage

• 1 Interval timer provides controller time tick

• Timer array unit: 4 timers capture the width of sensor pulses

• Watchdog timer for system reliability

• Low voltage detection for reliable operation in intermittent solar conditions

• RTC used in diagnostic logs

• 1 UART used for diagnostics

• Flash memory for storing diagnostic logs

The complete project (description, schematics, diagrams, and code) is now available on the Challenge website.