Debugging USB Firmware

You’ve written firmware for your USB device and are ready to test it. You attach the device to a PC and the hardware wizard announces: “The device didn’t start.” Or, the device installs but doesn’t send or receive data. Or, data is being dropped, the throughput is low, or some other problem presents itself. What do you do?

This article explores tools and techniques to debug the USB devices you design. The focus is on USB 2.0 devices, but much of the information also applies to developing USB 3.0 (SuperSpeed) devices and USB hosts for embedded systems.

VIEWING BUS TRAFFIC

If you do anything beyond a small amount of USB developing, a USB protocol analyzer will save you time and trouble. Analyzers cost less than they used to and are well worth the investment.

A hardware-based analyzer connects in a cable segment upstream from the device under test (see Photo 1).

Photo 1: The device under test connects to the analyzer, which
captures the data and passes it unchanged to the device’s host. The
cable on the back of the analyzer carries the captured data to the
analyzer’s host PC for display.

You can view the data down to each packet’s individual bytes and see exactly what the host and device did and didn’t send (see Photo 2).

Photo 2: This bus capture shows the host’s request for a configuration
descriptor and the bytes the device sent in response. Because the endpoint’s
maximum packet size is eight, the device sends the first 8 bytes in one
transaction and the final byte in a second transaction.

An analyzer can also decode data to show standard USB requests and class-specific data (see Photo 3).

Photo 3: This display decodes a received configuration descriptor and its subordinate descriptors.

To avoid corrupted data caused by the electrical effects of the analyzer’s connectors and circuits, use short cables (e.g., 3’ or less) to connect the analyzer to the device under test.

Software-only protocol analyzers, which run entirely on the device’s host PC, can also be useful. But, this kind of analyzer only shows data at the host-driver level, not the complete packets on the bus.

DEVELOPMENT STRATEGIES

The first rule for developing USB device firmware is to remember that the host computer controls the bus. Devices just need to respond to received data and events. Device firmware shouldn’t make assumptions about what the host will do next.

For example, some flash drives work under Windows but break when attached to a host with an OS that sends different USB requests or mass-storage commands, sends commands in a different order, or detects errors Windows ignores. This problem is so common that Linux has a file, unusual_devs.h, with fixes for dozens of misbehaving drives.

The first line of defense in writing USB firmware is the free USB-IF Test Suite from the USB Implementers Forum (USB-IF), the trade group that publishes the USB specifications. During testing, the suite replaces the host’s USB driver with a special test driver. The suite’s USB Command Verifier tool checks for errors (e.g., malformed descriptors, invalid responses to standard USB requests, responses to Suspend and Resume signaling, etc.). The suite also provides tests for devices in some USB classes, such as human interface devices (HID), mass storage, and video.

Running the tests will usually reveal issues that need attention. Passing the tests is a requirement for the right to display the USB-IF’s Certified USB logo.

LAYERED COMMUNICATIONS

Like networks, USB communications have layers that isolate different logical functions (see Table 1).

Table 1: USB communications use layers, which are each responsible for a
specific logical function.

The USB protocol layer manages USB transactions, which carry data packets to and from device endpoints. A device endpoint is a buffer that is a source or sink of data at the device. The host sends data to Out endpoints and receives data from In endpoints. (Even though endpoints are on devices, In and Out are defined from the host’s perspective.)

The device layer manages USB transfers, with each transfer moving a chunk of data consisting of one or more transactions. To meet the needs of different peripherals, the USB 2.0 specification defines four transfer types: control, interrupt, bulk, and isochronous.

The function layer manages protocols specific to a device’s function (e.g., mouse, printer, or drive). The function protocols may be a combination of USB class, industry, and vendor-defined protocols.

CONTROLLER ARCHITECTURES

The layers supported by device firmware vary with the device hardware. At one end of the spectrum, a Future Technology Devices International (FTDI) FT232R USB UART controller handles all the USB protocols in hardware. The chip has a USB device port that connects to a host computer and a UART port that connects to an asynchronous serial port on the device.

Device firmware reads and writes data on the serial port, and the FT232R converts it between the USB and UART protocols. The device firmware doesn’t have to know anything about USB. This feature has made the FT232R and similar chips popular!

An example of a chip that is more flexible but requires more firmware support is Microchip Technology’s PIC18F4550 microcontroller, which has an on-chip, full-speed USB device controller. In return for greater firmware complexity, the PIC18F4550 isn’t limited to a particular host driver and can support any USB class or function.

Each of the PIC18F4550’s USB endpoints has a series of registers—called a buffer descriptor table (BDT)—that store the endpoint buffer’s address, the number of bytes to send or receive, and the endpoint’s status. One of the BDT’s status bits determines the BDT’s ownership. When the CPU owns the BDT, firmware can write to the registers to prepare to send data or to retrieve received data. When the USB module owns the BDT, the endpoint can send or receive data on the bus.

To send a data packet from an In endpoint, firmware stores the bytes’ starting address to send and the number of bytes and sets a register bit to transfer ownership of the BDT to the USB module. The USB module sends the data in response to a received In token packet on the endpoint and returns BDT ownership to the CPU so firmware can set up the endpoint to send another packet.

To receive a packet on an Out endpoint, firmware stores the buffer’s starting address for received bytes and the maximum number of bytes to receive and transfers ownership of the BDT to the USB module. When data arrives, the USB module returns BDT ownership to the CPU so firmware can retrieve the data and transfer ownership of the BDT back to the USB module to enable the receipt of another packet.

Other USB controllers have different architectures and different ways of managing USB communications. Consult your controller chip’s datasheet and programming guide for details. Example code from the chip vendor or other sources can be helpful.

DEBUGGING TRANSACTIONS

A USB 2.0 transaction consists of a token packet and, as needed, a data packet and a handshake packet. The token packet identifies the packet’s type (e.g., In or Out), the destination device and endpoint, and the data packet direction.

The data packet, when present, contains data sent by the host or device. The handshake packet, when present, indicates the transaction’s success or failure.

The data and handshake packets must transmit quickly after the previous packet, with only a brief inter-packet delay and bus turnaround time, if needed. Thus, device hardware typically manages the receiving and sending of packets within a transaction.

For example, if an endpoint’s buffer has room to accept a data packet, the endpoint stores the received data and returns ACK in the handshake packet. Device firmware can then retrieve the data from the buffer. If the buffer is full because firmware didn’t retrieve previously received data, the endpoint returns NAK, requiring the host to try again. In a similar way, an In endpoint will NAK transactions until firmware has loaded the endpoint’s buffer with data to send.

Fine tuning the firmware to quickly write and retrieve data can improve data throughput by reducing or eliminating NAKs. Some device controllers support ping-pong buffers that enable an endpoint to store multiple packets, alternating between the buffers, as needed.

LOST DATA

In all but isochronous transfers, a data-toggle value in the data packet’s packet identification (PID) field guards against missed or duplicate data packets. If you’re debugging a device where data is transmitting on the bus and the receiver is returning ACK but ignoring or discarding the data, chances are good that the device isn’t sending or expecting the correct data-toggle value. Some device controllers handle the data toggles completely in hardware, while others require some firmware control.

Each endpoint maintains its own data toggle. The values are DATA0 (0011B) and DATA1 (1011B). Upon detecting an incoming data packet, the receiver compares its data toggle’s state with the received data toggle. If the values match, the receiver toggles its value and returns ACK, causing the sender to toggle its value for the next transaction.

The next received packet should contain the opposite data toggle, and again the receiver toggles its bit and returns ACK. Except for control transfers, the data toggle on each end continues to alternate in each transaction. (Control transfers always use DATA0 in the Setup stage, toggle the value for each transaction in the Data stage, and use DATA1 in the Status stage.)

If the receiver returns NAK or no response, the sender doesn’t toggle its bit and tries again with the same data and data toggle. If a receiver returns ACK, but for some reason the sender doesn’t see the ACK, the sender thinks the receiver didn’t receive the data and tries again using the same data and data toggle. In this case, the repeated data receiver ignores the data, doesn’t toggle the data toggle, and returns ACK, resynchronizing the data toggles. If the sender mistakenly sends two packets in a row with the same data-toggle value, upon receiving the second packet, the receiver ignores the data, doesn’t toggle its value, and returns ACK.

DEFINING A TRANSFER

All USB devices must support control transfers and may support other transfer types. Control transfers provide a structure for sending requests but have no guaranteed delivery time. Interrupt transfers have a guaranteed maximum latency (i.e., delay) between transactions, but the host permits less bandwidth for interrupt transfers compared to other transfer types. Bulk transfers are the fastest on an otherwise idle bus, but they have no guaranteed delivery time, and thus can be slow on a busy bus. Isochronous transfers have guaranteed delivery time but no built-in error correction.

A transfer’s amount of data depends in part on the higher-level protocol that determines the data packets’ contents. For example, a keyboard sends keystroke data in an interrupt transfer that consists of one transaction with 8 data bytes. To send a large file to a drive, the host typically uses one or more large transfers consisting of multiple transactions. For a high-speed drive, each transaction, except possibly the last one, has 512 data bytes, which is the maximum-allowed packet size for high-speed bulk endpoints.

What determines a transfer’s end varies with the USB class or vendor protocol. In many cases, a transfer ends with a short packet, which is a packet that contains less than the packet’s maximum-allowed data bytes. If the transfer has an even multiple of the packet’s maximum-allowed bytes, the sender may indicate the end of the transfer with a zero-length packet (ZLP), which is a data packet with a PID and error-checking bits but no data.

For example, USB virtual serial-port devices in the USB communications device class use short packets to indicate the transfer’s end. If a device has sent data that is an exact multiple of the endpoint’s maximum packet size and the host sends another In token packet, the endpoint should return a ZLP to indicate the data’s end.

DEBUGGING ENUMERATION

Upon device attachment, in a process called enumeration, the host learns about the device by requesting a series of data structures called descriptors. The host uses the descriptors’ information to assign a driver to the device.

If enumeration doesn’t complete, the device doesn’t have an assigned driver, and it can’t perform its function with the host. When Windows fails to find an appropriate driver, the setupapi.dev.log file in Windowsinf (for Windows 7) can offer clues about what went wrong. A protocol analyzer shows if the device returned all requested descriptors and reveals mistakes in the descriptors.

During device development, you may need to change the descriptors (e.g., add, remove, or edit an endpoint descriptor). Windows has the bad habit of remembering a device’s previous descriptors on the assumption that a device will never change its descriptors. To force Windows to use new descriptors, uninstall then physically remove and reattach the device from Windows Device Manager. Another option is to change the device descriptor’s product ID to make the device appear as a different device.

DEBUGGING TRANSFERS

Unlike the other transfer types, control transfers have multiple stages: setup, (optional) data, and status. Devices must accept all error-free data packets that follow a Setup token packet and return ACK. If the device is in the middle of another control transfer and the host sends a new Setup packet, the device must abandon the first transfer and begin the new one. The data packet in the Setup stage contains important information firmware should completely decode (see Table 2).

Table 2: Device firmware should fully decode the data received in a control transfer’s Setup stage. (Source: USB Implementers Forum, Inc.)

The wLength field specifies how many bytes the host wants to receive. A device shouldn’t assume how much data the host wants but should check wLength and send no more than the requested number of bytes.

For example, a request for a configuration descriptor is actually a request for the configuration descriptor and all of its subordinate descriptors. But, in the first request for a device’s configuration descriptor, the host typically sets the wLength field to 9 to request only the configuration descriptor. The descriptor contains a wTotalLength value that holds the number of bytes in the configuration descriptor and its subordinate descriptors. The host then resends the request setting wLength to wTotalLength or a larger value (e.g., FFh). The device returns the requested descriptor set up to wTotalLength. (Don’t assume the host will do it this way. Always check wLength!)

Each Setup packet also has a bmRequestType field. This field specifies the data transfer direction (if any), whether the recipient is the device or an interface or endpoint, and whether the request is a standard USB request, a USB class request, or a vendor-defined request. Firmware should completely decode this field to correctly identify received requests.

A composite device has multiple interfaces that function independently. For example, a printer might have a printer interface, a mass-storage interface for storing files, and a vendor-specific interface to support vendor-defined capabilities. For requests targeted to an interface, the wIndex field typically specifies which interface applies to the request.

INTERRUPT TRANSFER TIMING

For interrupt endpoints, the endpoint descriptor contains a bInterval value that specifies the endpoint’s maximum latency. This value is the longest delay a host should use between transaction attempts.

A host can use the bInterval delay time or a shorter period. For example, if a full-speed In endpoint has a bInterval value of 10, the host can poll the endpoint every 1 to 10 ms. Host controllers typically use predictable values, but a design shouldn’t rely on transactions occurring more frequently than the bInterval value.

Also, the host controller reserves bandwidth for interrupt endpoints, but the host can’t send data until a class or vendor driver provides something to send. When an application requests data to be sent or received, the transfer’s first transaction may be delayed due to passing the request to the driver and scheduling the transfer.

Once the host controller has scheduled the transfer, any additional transaction attempts within the transfer should occur on time, as defined by the endpoint’s maximum latency. For this reason, sending a large data block in a single transfer with multiple transactions can be more efficient than using multiple transfers with a portion of the data in each transfer.

DEVICE FUNCTIONS

Most devices’ functions fit a defined USB class (e.g., mass storage, printer, audio, etc.). The USB-IF’s class specifications define protocols for devices in the classes.

For example, devices in the HID class must send and receive all data in data structures called reports. The supported report’s length and the meaning of its data (e.g., keypresses, mouse movements, etc.) are defined in a class-specific report descriptor.

If your HID-class device is sending data but the host application isn’t seeing the data, verify the number of bytes the device is sending matches the number of bytes in a defined report. The device should prepend a report-ID byte to the data only if the HID supports report IDs other than the zero default value.

In many devices, class specifications define class-specific requests or other requirements. For example, a mass storage device that uses the bulk-only protocol must provide a unique serial number in a string descriptor. Carefully read and heed any class specifications that apply to your device!

Many devices also support industry protocols to perform higher-level functions. Printers typically support one or more printer-control languages (e.g., PCL and Postscript). Mass-storage devices support SCSI commands to transfer data blocks and a file system (e.g., FAT32) to define a directory structure.

The higher-level industry protocols don’t depend on a particular hardware interface, so there is little about debugging them that is USB-specific. But, because these protocols can be complicated, example code for your device can be helpful.

In the end, much about debugging USB firmware is like debugging any hardware or software. A good understanding of how the communications should work provides a head start on writing good firmware and finding the source of any problems that may appear.

Jan Axelson is the author of USB Embedded Hosts, USB Complete, and Serial Port Complete. Jan’s PORTS web forum is available at www.lvr.com.

RESOURCES

Jan Axelson’s Lakeview Research, “USB Development Tools: Protocol analyzers,” www.lvr.com/development_tools.htm#analyzers.

This article appears in Circuit Cellar 268 (November 2012).

DIY Green Energy Design Projects

Ready to start a low-power or energy-monitoring microcontroller-based design project? You’re in luck. We’re featuring eight award-winning, green energy-related designs that will help get your creative juices flowing.

The projects listed below placed at the top of Renesas’s RL78 Green Energy Challenge.

Electrostatic Cleaning Robot: Solar tracking mirrors, called heliostats, are an integral part of Concentrating Solar Power (CSP) plants. They must be kept clean to help maximize the production of steam, which generates power. Using an RL78, the innovative Electrostatic Cleaning Robot provides a reliable cleaning solution that’s powered entirely by photovoltaic cells. The robot traverses the surface of the mirror and uses a high voltage AC electric field to sweep away dust and debris.

Parts and circuitry inside the robot cleaner

Cloud Electrofusion Machine: Using approximately 400 times less energy than commercial electrofusion machines, the Cloud Electrofusion Machine is designed for welding 0.5″ to 2″ polyethylene fittings. The RL78-controlled machine is designed to read a barcode on the fitting which determines fusion parameters and traceability. Along with the barcode data, the system logs GPS location to an SD card, if present, and transmits the data for each fusion to a cloud database for tracking purposes and quality control.

Inside the electrofusion machine (Source: M. Hamilton)

The Sun Chaser: A GPS Reference Station: The Sun Chaser is a well-designed, solar-based energy harvesting system that automatically recalculates the direction of a solar panel to ensure it is always facing the sun. Mounted on a rotating disc, the solar panel’s orientation is calculated using the registered GPS position. With an external compass, the internal accelerometer, a DC motor and stepper motor, you can determine the solar panel’s exact position. The system uses the Renesas RDKRL78G13 evaluation board running the Micrium µC/OS-III real-time kernel.

[Video: ]

Water Heater by Solar Concentration: This solar water heater is powered by the RL78 evaluation board and designed to deflect concentrated amounts of sunlight onto a water pipe for continual heating. The deflector, armed with a counterweight for easy tilting, automatically adjusts the angle of reflection for maximum solar energy using the lowest power consumption possible.

RL78-based solar water heater (Source: P. Berquin)

Air Quality Mapper: Want to make sure the air along your daily walking path is clean? The Air Quality Mapper is a portable device designed to track levels of CO2 and CO gasses for constructing “Smog Maps” to determine the healthiest routes. Constructed with an RDKRL78G13, the Mapper receives location data from its GPS module, takes readings of the CO2 and CO concentrations along a specific route and stores the data in an SD card. Using a PC, you can parse the SD card data, plot it, and upload it automatically to an online MySQL database that presents the data in a Google map.

Air quality mapper design (Source: R. Alvarez Torrico)

Wireless Remote Solar-Powered “Meteo Sensor”: You can easily measure meteorological parameters with the “Meteo Sensor.” The RL78 MCU-based design takes cyclical measurements of temperature, humidity, atmospheric pressure, and supply voltage, and shares them using digital radio transceivers. Receivers are configured for listening of incoming data on the same radio channel. It simplifies the way weather data is gathered and eases construction of local measurement networks while being optimized for low energy usage and long battery life.

The design takes cyclical measurements of temperature, humidity, atmospheric pressure, and supply voltage, and shares them using digital radio transceivers. (Source: G. Kaczmarek)

Portable Power Quality Meter: Monitoring electrical usage is becoming increasingly popular in modern homes. The Portable Power Quality Meter uses an RL78 MCU to read power factor, total harmonic distortion, line frequency, voltage, and electrical consumption information and stores the data for analysis.

The portable power quality meter uses an RL78 MCU to read power factor, total harmonic distortion, line frequency, voltage, and electrical consumption information and stores the data for analysis. (Source: A. Barbosa)

High-Altitude Low-Cost Experimental Glider (HALO): The “HALO” experimental glider project consists of three main parts. A weather balloon is the carrier section. A glider (the payload of the balloon) is the return section. A ground base section is used for communication and display telemetry data (not part of the contest project). Using the REFLEX flight simulator for testing, the glider has its own micro-GPS receiver, sensors and low-power MCU unit. It can take off, climb to pre-programmed altitude and return to a given coordinate.

High-altitude low-cost experimental glider (Source: J. Altenburg)

Infrared Communications for Atmel Microcontrollers

Are you planning an IR communications project? Do you need to choose a microcontroller? Check out the information Cornell University Senior Lecturer Bruce Land sent us about inexpensive IR communication with Atmel ATmega microcontrollers. It’s another example of the sort of indispensable information covered in Cornell’s excellent ECE4760 course.

Land informed us:

I designed a basic packet communication scheme using cheap remote control IR receivers and LED transmitters. The scheme supports 4800 baud transmission,
with transmitter ID and checksum. Throughput is about twenty 20-character packets/sec. The range is at least 3 meters with 99.9% packet receive and moderate (<30 mA) IR LED drive current.

On the ECE4760 project page, Land writes:

I improved Remin’s protocol by setting up the link software so that timing constraints on the IR receiver AGC were guaranteed to be met. It turns out that there are several types of IR reciever, some of which are better at short data bursts, while others are better for sustained data. I chose a Vishay TSOP34156 for its good sustained data characteristics, minimal burst timing requirements, and reasonable data rate. The system I build works solidly at 4800 baud over IR with 5 characters of overhead/packet (start token, transmitter number, 2 char checksum , end token). It works with increasing packet loss up to 9000 baud.

Here is the receiver circuit.

The receiver circuit (Source: B. Land, Cornell University ECE4760 Infrared Communications
for Atmel Mega644/1284 Microcontrollers)

Land explains:

The RC circuit acts a low-pass filter on the power to surpress spike noise and improve receiver performance. The RC circuit should be close to the receiver. The range with a 100 ohm resistor is at least 3 meters with the transmitter roughly pointing at the receiver, and a packet loss of less then 0.1 percent. To manage burst length limitations there is a short pause between characters, and only 7-bit characters are sent, with two stop bits. The 7-bit limit means that you can send all of the printing characters on the US keyboard, but no extended ASCII. All data is therefore sent as printable strings, NOT as raw hexidecimal.

Land’s writeup also includes a list of programs and packet format information.

DIY Internet-Enabled Home Control System

Why shell out hundreds or thousands of dollars on various home control systems (HCS) when you have the skills and resources to build your own? You can design and implement sophisticated Internet-enabled systems with free tools and some careful planning.

John Breitenbach did just that. He used a microcontroller, free software, and a cloud-based data platform to construct a remote monitoring system for his home’s water heater. The innovative design can email or text status messages and emergency alerts to a smartphone. You can build a similar system to monitor any number of appliances, rooms, or buildings.

An abridged version of Breitenbach’s article, “Internet-Enabled Home Control” (Circuit Cellar 264, July 2012), appears below. (A link to the entire article and an access password are noted at the end of this post.) Breitenbach writes:

Moving from the Northeast to North Carolina, my wife and I were surprised to find that most homes don’t have basements. In the north, the frost line is 36˝–48 ˝ below the surface. To prevent frost heave, foundations must be dug at least that deep. So, digging down an extra few feet to create a basement makes sense. Because the frost line is only 15 ˝ in the Raleigh area, builders rarely excavate the additional 8’ to create basements.

The lack of basements means builders must find unique locations for a home’s mechanical systems including the furnace, AC unit, and water heater. I was shocked to find that my home’s water heater is located in the attic, right above one of the bedrooms (see Photo 1).

Photo 1: My home’s water heater is located in our attic. (Photo courtesy of Michael Thomas)

During my high school summers I worked for my uncle’s plumbing business (“Breitenbach Plumbing—We’re the Best, Don’t Call the Rest”) and saw firsthand the damage water can do to a home. Water heaters can cause some dramatic end-of-life plumbing failures, dumping 40 or more gallons of water at once followed by the steady flow of the supply line.

Having cleaned up the mess of a failed water heater in my own basement up north, I haven’t had a good night’s sleep since I discovered the water heater in my North Carolina attic. For peace of mind, especially when traveling, I instrumented my attic so I could be notified immediately if water started to leak. My goal was to use a microcontroller so I could receive push notifications via e-mails or text messages. In addition to emergency messages, status messages sent on a regular basis reassure me the system is running. I also wanted to use a web browser to check the current status at any time.

MCU & SENSOR

The attic monitor is based on Renesas Electronics’s YRDKRX62N demonstration kit, which features the RX62N 32-bit microcontroller (see Photo 2). Renesas has given away thousands of these boards to promote the RX, and the boards are also widely available through distributors. The YRDK board has a rich feature set including a graphics display, push buttons, and an SD-card slot, plus Ethernet, USB, and serial ports. An Analog Devices ADT7420 digital I2C temperature sensor also enables you to keep an eye on the attic temperature. I plan to use this for a future addition to the project that compares this temperature to the outside air temperature to control an attic fan.

Photo 2: The completed board, which is based on a Renesas Electronics YRDKRX62N demonstration kit. (Photo courtesy of Michael Thomas)

SENSING WATER

Commercial water-detection sensors are typically made from two exposed conductive surfaces in close proximity to each other on a nonconductive surface. Think of a single-sided PCB with no solder mask and tinned traces (see Photo 3).

Photo 3: A leak sensor (Photo courtesy of Michael Thomas)

These sensors rely on the water conductivity to close the circuit between the two conductors. I chose a sensor based on this type of design for its low cost. But, once I received the sensors, I realized I could have saved myself a few bucks by making my own sensor from a couple of wires or a piece of proto-board.

When standing water on the sensor shorts the two contacts, the resistance across the sensor drops to between 400 kΩ and 600 kΩ. The sensor is used as the bottom resistor in a voltage divider with a 1-MΩ resistor up top. The output of the divider is routed to the 12-bit analog inputs on the RX62N microcontroller. Figure 1 shows the sensor interface circuit. When the voltage read by the analog-to-digital converter (ADC) drops below 2 V, it’s time to start bailing. Two sensors are connected: one in the catch pan under the water heater, and a second one just outside the catch pan to detect failures in the small expansion tank.

Figure 1: The sensor interface to the YRDK RX62N board

COMMUNICATIONS CHOICES

One of my project goals was to push notifications to my cell phone because Murphy’s Law says water heaters are likely to fail while you’re away for the weekend. Because I wanted to keep the project costs low, I used my home’s broadband connection as the gateway for the attic monitor. The Renesas RX62N microcontroller includes a 100-Mbps Ethernet controller, so I simply plugged in the cable to connect the board to my home network. The open-source µIP stack supplied by Renesas with the YRDK provides the protocol engine needed to talk to the Internet.

There were a couple of complications with using my home network as the attic monitor’s gateway to the world. It is behind a firewall built into my router and, for security reasons, I don’t want to open up ports to the outside world.

My Internet service provider (ISP) occasionally changes the Internet protocol (IP) address associated with my cable modem. So I would never know what address to point my web browser. I needed a solution that would address both of these problems. Enter Exosite, a company that provides solutions for cloud-based, machine-to-machine (M2M) communications.

TALKING TO THE CLOUD

Exosite provides a number of software components and services that enable M2M communications via the cloud. This is a different philosophy from supervisory control and data acquisition (SCADA) systems I’ve used in the past. The control systems I’ve worked on over the years typically involve a local host polling the hundreds or thousands of connected sensors and actuators that make up a commercial SCADA system. These systems are generally designed to be monitored locally at a single location. In the case of the attic monitor, my goal was to access a limited number of data points from anywhere, and have the system notify me rather than having to continuously poll. Ideally, I’d only hear from the device when there was a problem.

Exosite is the perfect solution: the company publishes a set of simple application programming interfaces (APIs) using standard web protocols that enable smart devices to push data to their servers in the cloud in real time. Once the data is in the cloud, events, alerts, and scripts can be created to do different things with the data—in my case, to send me an e-mail and SMS text alert if there is anything wrong with my water heater. Connected devices can share data with each other or pull data from public data sources, such as public weather stations. Exosite has an industrial-strength platform for large-scale commercial applications. It provides free access to it for the open-source community. I can create a free account that enables me to connect one or two devices to the Exosite platform.

Embedded devices using Exosite are responsible for pushing data to the server and pulling data from it. Devices use simple HTTP requests to accomplish this. This works great in my home setup because the attic monitor can work through my firewall, even when my Internet provider occasionally changes the IP address of my cable modem. Figure 2 shows the network diagram.

Figure 2: The cloud-based network

VIRTUAL USER INTERFACE

Web-based dashboards hosted on Exosite’s servers can be built and configured to show real-time and historical data from connected devices. Controls, such as switches, can be added to the dashboards to push data back down to the device, enabling remote control of embedded devices. Because the user interface is “in the cloud,” there is no need to store all the user interface (UI) widgets and data in the embedded device, which greatly reduces the storage requirements. Photo 4 shows the dashboard for the attic monitor.

Photo 4: Exosite dashboard for the attic monitor

Events and alerts can be added to the dashboard. These are logical evaluations Exosite’s server performs on the incoming data. Events can be triggered based on simple comparisons (e.g., a data value is too high or too low) or complex combinations of a comparison plus a duration (e.g., a data value remains too high for a period of time). Setting up a leak event for one of the sensors is shown in Photo 5.

Photo 5: Creating an event in Exosite

In this case, the event is triggered when the reported ADC voltage is less than 2 V. An event can also be triggered if Exosite doesn’t receive an update from the device for a set period of time. This last feature can be used as a watchdog to ensure the device is still working.

When an event is triggered, an alert can optionally be sent via e-mail. This is the final link that enables an embedded device in my attic to contact me anywhere, anytime, to alert me to a problem. Though I have a smartphone that enables me to access my e-mail account, I can also route the alarm message to my wife’s simpler phone through her cellular provider’s e-mail-to-text-message gateway. Most cellular providers offer this service, which works by sending an e-mail to a special address containing the cell phone number. On the Verizon network, the e-mail address is <yourcellularnumber>@vtext.com. Other providers have similar gateways.

The attic monitor periodically sends heartbeat messages to Exosite to let me know it’s still working. It also sends the status of the water sensors and the current temperature in the attic. I can log in to Exosite at any time to see my attic’s real-time status. I have also configured events and alarms that will notify me if a leak is detected or if the temperature gets too hot…

The complete article includes details such about the Internet engine, reading the cloud, tips for updating the design, and more.  You can read the entire article by typing netenabledcontrol to open the password-protected PDF.

Build a Microcontroller-Based Mail Client

Does the sheer amount of junk mail that fills your Inbox make you hate everything about e-mail? If so, it’s time to have a little fun with electronic mail by building a compact microcontroller-based mail client system. Alexander Mann designed a system that uses an Atmel ATmega32 and a Microchip Technology ENC28J60 Ethernet controller to check continuously for e-mail. When a message arrives, he can immediately read it on the system’s LCD and respond with a standard keyboard.

Mann writes:

My MiniEmail system is a compact microcontroller-based mail client (see Photo 1). The silent, easy-to-use system doesn’t require a lot of power and it is immune to mail worms. Another advantage is the system’s short start-up time. If you want to write a quick e-mail but your PC is off, you can simply switch on the miniature e-mail client and start writing without having to wait for your PC to boot up and load the necessary applications. All you need is an Ethernet connection and the MiniEmail system.

Photo 1: The complete MiniEmail system includes an LCD, a keyboard, and several connections. (A. Mann, Circuit Cellar 204)

HARDWARE

The hardware for the MiniEmail system is inexpensive. It cost me about $50. The LCD is the most expensive part. To keep things simple, I left the system’s power supply, 5- to 3.3-V conversion crystals, and latch out of Figure 1.

Figure 1: This is a block diagram of MiniEmail’s hardware. The arrows indicate the directions of data flow between the devices. The rounded boxes indicate parts that do not sit on the circuit board.

The main components are an Atmel ATmega32 microcontroller and a Microchip Technology ENC28J60 Ethernet controller. Because a mail client is a piece of complex software, you need a fast microcontroller that has a considerable amount of program space. The MiniEmail system uses almost all of the ATmega32’s features, including the SPI, internal EEPROM and SRAM, counters, USART interface, sleep modes, all 32 I/O lines, and most of the 32 KB of program memory. The ENC28J60 is a stand-alone Ethernet controller that provides basic functionality for transmitting frames over an Ethernet connection. It has 8 KB of built-in SRAM, which can be divided into transmit and receive buffers as desired, and it provides several interrupt sources (e.g., when new packets have arrived). The ATmega32 also has 128 KB of external SRAM connected as well as an LCD, which is a standard module with a resolution of 128 × 64 pixels.

Take a look at the ATmega32’s pin connections in Figure 2. Ports A and C are used as 8-bit-wide general I/O ports, one of which is latched using an NXP Semiconductors 74HC573.

Figure 2: Here’s the complete schematic for the MiniEmail. The LF1S022 is the RJ-45 connector for the Ethernet connection.

The two ports provide data connections to the LCD and SRAM (U3). For the SRAM, you need three additional wires: write (*RAM_WR), read strobe (*RAM_RD), and the seventeenth bit of the address (ADDR16). The LCD connector (CON1) uses five additional wires (for the signals CS1, CS2, DI, EN, and RW). CS1 and CS2 are taken from the general I/O port A (DATA6 and DATA7) and determine which of the two halves of the LCD is selected (i.e., the two controllers on the LCD module you are talking to). RW (where you can use ADDR16 again) sets the direction of the LCD access (read or write). DI describes the type of instruction sent to the LCD. EN is the enable signal for read and write cycles. For the keyboard, you need only two pins: KEY_DATA and KEY_CLOCK. The clock signal must be connected to an external interrupt pin, INT1. One additional wire is needed to switch the latch (LE).

You are left with eight I/O pins on the ATmega32’s ports B and D. RXD and TXD are connected to a MAX232, an RS-232 level converter that also provides the negative supply voltage needed for the LCD (LCD_VOUT in Figure 2). The ATmega32’s USART functionality is used as a debugging interface. It isn’t needed for normal operation.

SOFTWARE

The firmware for this project is posted on the Circuit Cellar FTP site. I wrote the firmware in C language with a few small parts of inline assembler. I used the open-source software suite WinAVR, which includes the GNU GCC compiler with special libraries for AVR devices and avrdude, a tool for the in-system programming of AVR microcontrollers…

USER INTERFACE

The user interface consists of three control elements: menus, edit fields, and an elaborate text editor. A special screen (the Mail Menu) enables you to quickly browse through your mailbox. After power-up, the system displays a greeting message. After a short while, the Main menu appears (see Photo 2).

Photo 2: This is a screenshot of MiniEmail’s main menu. In the upper-right corner, a clock shows the current time, which is retrieved from the Internet. An arrow to the left of the menu items indicates the selected item. (A. Mann, Circuit Cellar 204)

The Compose Mail, Check Mailbox, and Configuration submenus form a hierarchical menu structure. When the other items listed beneath the respective menu titles in the diagram are activated (e.g., start the text editor), they enable you to input data, such as a username and password, or retrieve mail from the mail server. “Standby” is the only action that is accessible directly from the main menu. All other actions are grouped by function in the submenus.

WRITING MAIL

With respect to the firmware, sending mail is much easier than reading it, so let’s first focus on the Compose Mail menu. The first item in the menu starts the text editor so you can enter the body of your letter. You then enter the recipient’s mailing address and the subject of your e-mail, just like you would do when sending e-mail from your PC. Additional fields, such as CC or BCC are not included, but since this requires only one more line in the header of the mail, it is not difficult. Your e-mail also needs a reply address, so the recipient knows who sent the mail. The reply address is normally the same for all of the messages you write. The text you enter in this edit field is stored in the ATmega32’s EEPROM, so you don’t have to type it every time you write a letter. After you select the last menu item, “Send” initiates the dispatch of the mail and displays a message that indicates whether or not it was successful.

CHECKING FOR MAIL

What makes this part more sophisticated is the ability to handle not only one e-mail at a time, but also fetch mail from the server. The system can determine which messages are new and which messages have been read. It can also extract data such as the sender, subject, or sent date from the header of the mail and then display the information.

The amount of mail the firmware can handle is limited by the size of the external SRAM. The maximum number of e-mails is currently 1,024. (If you’ve got more mail, you will be so busy answering it that you won’t have time to build your own MiniEmail client—or you should delete some old mail). Note that 1,024 is the number of unique identifiers that the system can remember. The server assigns a unique identifier to each piece of mail. The system uses the identifiers to keep track of which letters are new on the server, which have already been read, and which have been marked for deletion.

All of the header data for all of the 1,024 messages cannot be held in SRAM at once; only the most recent (about 50) mail headers are held. When you want to browse through older e-mails, the firmware automatically reconnects to the server and fetches the headers of the next 50 e-mails.

When you select Check Mailbox in the main menu, you get to a submenu where you can retrieve and read mail. Before you can collect your mail, you must enter your username and password, which can be stored in EEPROM for your convenience. The firmware then retrieves the headers and displays the Mail Menu, where you can browse through your e-mail. Apart from the size and the date, the first 42 characters of the subject and the mail sender are shown. In the first row, additional icons indicate (from left to right) whether a message is new, has been marked for deletion, or has been read. You can view the content of the selected message by pressing Return. When the mail is fetched from the server, it is prepared for viewing. The header and HTML tags, as well as long runs of the same character, are stripped from the mail and base64 decoding (used to encode 8-bit characters) is performed, so the content of the message is as readable as plain text. Binary attachments (e.g., images) can’t be handled. Following this, the mail is viewed in the text editor (with editing disabled).

A similar action is performed when you press “r” in the Mail Menu. In that case, you can edit the text so you can add your reply. Leaving the text editor will bring you back to the Send Mail menu, where the reply address and subject will be filled in so your mail will be clear for take-off. To delete a message, simply press D to mark it for deletion….

OUTLOOK

I hadn’t imagined how many details would need to be considered when I started this project more than a year ago. It has been a very interesting and challenging project. It has also been a lot of fun.

The MiniEmail system provides all of the basics for communicating via email, but such a project is never really finished. There are still dozens of items on my to-do list. Fortunately, the ATmega32 can be replaced with a new member of the AVR family, the Atmel ATmega644, which is pin-compatible to the ATmega32 and has twice the flash memory (and internal SRAM). That will provide enough space for many of my new ideas. I want to get rid of the static IP address, add CC and BCC fields, use a bigger display or a smaller (variable-width) font, improve the filtering and display of mail content and attachments, and add an address book (it would be best in combination with an additional external EEPROM with an SPI, such as the AT25256).

This project proves, rather impressively, that the ATmega32 and the ENC28J60 are a powerful combination. They can be used for many useful Internet applications. My e-mail client system is surely one of the most exciting. I can think of many other interesting possibilities. At the moment, my MiniEmail assembly serves as an online thermometer so I can check my room’s temperature from anywhere in the world…

Mann’s entire article appears in Circuit Cellar 204, 2007. Type “miniemailopen”  to access the password-protected article.