Debugging Embedded Systems with Minimal Resources

Debugging an embedded system can be difficult when you’re dealing with either a simple system with few pins or a complex system with nearly every pin in use. Stuart Ball provides some tips to make debugging such systems a little easier.

Debugging a microcontroller system can be difficult. Things don’t work right and it often isn’t even clear why. Was something initialized wrong? Is it a timing issue? Is there conflicting use of shared resources?

Debugging is more complicated when there are limited resources. If all the processor pins are used, what do you connect to? How do you get debug information out of the firmware so you can see what is going on?

This article isn’t about debugging when you have Ethernet, USB, and Bluetooth interfaces available, or when you have a full-speed emulator. This is about debugging when there aren’t many resources available—simple systems with few pins, or more complex systems with nearly every pin already used for something.

This is the schematic for a serial port RS-232 driver. It's a standardized circuit that plugs into a header on the board to be debugged.

Figure 1: This is the schematic for a serial port RS-232 driver. It’s a standardized circuit that plugs into a header on the board to be debugged.

Postmortem vs. Real-Time Debugging

There are two general ways of debugging an embedded system. One is postmortem, looking at the state of the system after it has failed or after it has stopped at a breakpoint. The other is real-time, debugging while the system is running. Each has its own place and its own set of challenges.

Generally, the two methods use different debug techniques. A postmortem debug happens when the motor is stalled, the software can’t recover, and you have no idea why it happened. You want to know the system’s state and how it got there. Setting breakpoints is a method used in postmortem debugging; you stop the system and look at the static state after a particular point in the code is reached.

Real-time or active debugging is more appropriate for looking at timing issues, missed interrupts, cumulative latency issues, and cases where the system just does occasionally does something strange but doesn’t actually stop. Real-time debug can tell you how the system got into the state that you are trying to analyze using postmortem methods. If you can capture enough information while the system is running, you have a chance to turn a real-time problem into a simpler static post-mortem analysis.

This article appears in Circuit Cellar 312, July 2016. Download the complete article.

Universal Debug Solution

An asynchronous serial port may be the most common debug tool used in the embedded world. Most microcontrollers have at least one serial port built in. The serial port has limitations. Its speed is limited and it requires level translation to connect it to the RS-232 voltage levels of a PC.

In many cases, you might not want to put the RS-232 driver on your board. You don’t want to use the space required by either the IC or the RS-232 connector, especially for something that is only used while debugging. One way I’ve solved this problem is shown in Figure 1, which depicts just a Maxim Integrated (or Texas Instruments) MAX3232 RS-232 driver IC connected to a DE9 connector. The other side connects to a four-pin header. This is connected via a cable to the embedded system to be tested. This allows the embedded system to have just the four-pin connector wired to the microcontroller serial port pins, power, and ground.

You plug in the external circuit when you need to debug and unplug it when you are done. There is nothing special about this circuit, it is exactly the same as you might put on your microcontroller board. Except you don’t need the space on your board for this. The circuit takes power from the microcontroller board via pins 1 and 4 of the four-pin Molex connector. The connector indicated is polarized so you can’t plug it in backward. I’ve standardized on this in my embedded systems at least where the serial port is used for debug or download.

Although I used a connector with 0.1” centers on the interface board, there is nothing to prevent you from using a 2 mm or 0.05” connector, or even a row of pads at the edge of the board being debugged. You just have to make a cable that has the Molex connector at one end and whatever you need to match your embedded board at the other end. You can keep the driver board in your toolbox, put the connector on your embedded system boards, and you have it when you need it.

You can house the board into a plastic project box. In one case, I built one on a narrow piece of perforated project board, and covered the entire thing in heat shrink tubing. It has the right-angle Molex connector on one end, and a short cable with the RS-232 connector on the other end. I keep that one in my desk drawer at work.

Read the entire article (PDF)

8-Bit Microcontroller IP Core

DigitalCoreDesignThe DF6808 IP core is binary-compatible with the industry-standard Motorola 68HC08 8-bit microcontroller. The IP core uses sophisticated on-chip peripheral capabilities to perform 45 to 100 million instructions per second. FAST architecture implemented in DF6808 enables the 68HC08 microcontroller to run at least three times faster than the original solution.

The DF6808’s 16-bit, free-running timer system has two input-capture lines and two output-compare lines. The IP core is equipped with proprietary safety functions, including self-monitoring circuitry, which helps protect against system errors; the computer operating properly (COP) watchdog system, which protects against software failures; and an illegal opcode detection circuit, which provides a non-maskable interrupt if an illegal opcode occurs.

For power conservation, the IP core includes two software-controlled power-saving modes (Wait and Stop). These modes make the DF6808 IP core well suited for automotive and battery-driven applications.

The DF6808 includes the DoCDTM real-time hardware debugger, which provides built-in support for Digital Core Design’s hardware debug system and the debugging capability of an entire system-on-a-chip (SoC). The DoCDTM enables nonintrusive debugging of running applications. It can halt, run, step into, or skip an instruction and read/write any microcontroller contents, including all registers, user-defined peripherals, data, and program memories.

Contact Digital Core Design for pricing.

Digital Core Design

A Trace Tool for Embedded Systems

Tracing tools monitor what is going in a program’s execution by logging low-level and frequent events. Thus tracing can detect and help debug performance issues in embedded system applications.

In Circuit Cellar’s April issue, Thiadmer Riemersma describes his DIY tracing setup for small embedded systems. His system comprises three parts: a set of macros to include in the source files of a device under test (DUT), a PC workstation viewer that displays retrieved trace data, and a USB dongle that interfaces the DUT with the workstation.

Small embedded devices typically have limited-performance microcontrollers and scarce interfaces, so Riemersma’s tracing system uses only a single I/O pin on the microcontroller.

Designing a serial protocol that keeps data compact is also important. Riemersma, who develops embedded software for the products of his Netherlands-based company, CompuPhase, explains why:

Compactness of the information transferred from the embedded system to the workstation [which decodes and formats the trace information] is important because the I/O interface that is used for the transfer will probably be the bottleneck. Assuming you are transmitting trace messages bit by bit over a single pin using standard wire and 5- or 3.3-V logic levels, the transfer rate may be limited to roughly 100 Kbps.

My proposed trace protocol achieves compactness by sending numbers in binary, rather than as human-readable text. Common phrases can be sent as numeric message IDs. The trace viewer (or trace ‘listener’) can translate these message IDs back to the human-readable strings.

One important part of the system is the hardware interface—the trace dongle. Since many microcontrollers are designed with only those interfaces used for specific application needs, Riemersma says, typically the first step is finding a spare I/0 pin that can be used to implement the trace protocol.

In the following article excerpt, Riemersma describes his trace dongle and implementation requiring a single I/O pin on the microcontroller:

This is the trace dongle.

This is the trace dongle.

Photo 1 shows the trace dongle. To transmit serial data over a single pin, you need to use an asynchronous protocol. Instead of using a kind of (bit-banged) RS-232, I chose biphase encoding. Biphase encoding has the advantage of being a self-clocking and self-synchronizing protocol. This means that biphase encoding is simple to implement because timing is not critical. The RS-232 protocol requires timing to stay within a 3% error margin; however, biphase encoding is tolerant to timing errors of up to 20% per bit. And, since the protocol resynchronizes on each bit, error accumulation is not an issue.

Figure 1 shows the transitions to transmit an arbitrary binary value in biphase encoding—to be more specific, this variant is biphase mark coding. In the biphase encoding scheme, there is a transition at the start of each bit.

Figure 1: This is an example of a binary value transferred in biphase mark coding.

Figure 1: This is an example of a binary value transferred in biphase mark coding.

For a 1 bit there is also a transition halfway through the clock period. With a 0 bit, there is no extra transition. The absolute levels in biphase encoding are irrelevant, only the changes in the output line are important. In the previous example, the transmission starts with the idle state at a high logic level but ends in an idle state at a low logic level.

Listing 1 shows an example implementation to transmit a byte in biphase encoding over a single I/O pin. The listing refers to the trace_delay() and toggle_pin() functions (or macros). These system-dependent functions must be implemented on the DUT. The trace_delay() function should create a short delay, but not shorter than 5 µs (and not longer than 50 ms). The toggle_pin() function must toggle the output pin from low to high or from high to low.

For each bit, the function in Listing 1 inverts the pin and invokes trace_delay() twice. However, if the bit is a 1, it inverts the pin again between the two delay periods. Therefore, a bit’s clock cycle takes two basic “delay periods.”

Listing 1: Transmitting a byte in biphase encoding, based on a function to toggle an I/O pin, is shown.

Listing 1: Transmitting a byte in biphase encoding, based on a function to toggle an I/O pin, is shown.

The biphase encoding signal goes from the DUT to a trace dongle. The dongle decodes the signal and forwards it as serial data from a virtual RS-232 port to the workstation (see Photo 2 and the circuit in Figure 2).

Photo 2: The trace dongle is inserted into a laptop and connected to the DUT.

Photo 2: The trace dongle is inserted into a laptop and connected to the DUT.

This trace dongle interprets biphase encoding.

Figure 2: This trace dongle interprets biphase encoding.

The buffer is there to protect the microcontroller’s input pin from spikes and to translate the DUT’s logic levels to 5-V TTL levels. I wanted the trace dongle to work whether the DUT used 3-, 3.3-, or 5-V logic. I used a buffer with a Schmitt trigger to avoid the “output high” level of the DUT at 3-V logic, plus noise picked up by the test cable would fall in the undefined area of 5-V TTL input.

Regarding the inductor, the USB interface provides 5 V and the electronics run at 5 V. There isn’t room for a voltage regulator. Since the USB power comes from a PC, I assumed it might not be a “clean” voltage. I used the LC filter to reduce noise on the power line.

The trace dongle uses a Future Technology Devices International (FTDI) FT232RL USB-to-RS-232 converter and a Microchip Technology PIC16F1824 microcontroller. The main reason I chose the FT232RL converter is FTDI’s excellent drivers for multiple OSes. True, your favorite OS already comes with a driver for virtual serial ports, but it is adequate at best. The FTDI drivers offer lower latency and a flexible API. With these drivers, the timestamps displayed in the trace viewers are as accurate as what can be achieved with the USB protocol, typically within 2 ms.

I chose the PIC microcontroller for its low cost and low pin count. I selected the PIC16F1824 because I had used it in an earlier project and I had several on hand. The microcontroller runs on a 12-MHz clock that is provided by the FTDI chip.

The pins to connect to the DUT are a ground and a data line. The data line is terminated at 120 Ω to match the impedance of the wire between the dongle and the DUT.

The cable between the DUT and the trace dongle may be fairly long; therefore signal reflections in the cable should be considered even for relatively low transmission speeds of roughly 250 kHz. That cable is typically just loose wire. The impedance of loose wire varies, but 120 Ω is a good approximate value.

The data line can handle 3-to-5-V logic voltages. Voltages up to 9 V do not harm the dongle because it is protected with a Zener diode (the 9-V limit is due to the selected Zener diode’s maximum power dissipation). The data line has a 10-kΩ to 5-V pull-up, so you can use it on an open-collector output.

The last item of interest in the circuit is a bicolor LED that is simply an indicator for the trace dongle’s status and activity. The LED illuminates red when the dongle is “idle” (i.e., it has been enumerated by the OS). It illuminates green when biphase encoded data is being received.

After the dongle is built, it must be programmed. First, the FT232RL must be programmed (with FTDI’s “FT Prog” program) to provide a 12-MHz clock on Pin C0. The “Product Description” in the USB string descriptors should be set to “tracedongle” so the trace viewers can find the dongle among other FTDI devices (if any). To avoid the dongle being listed as a serial port, I also set the option to only load the FTDI D2XX drivers.

To upload the firmware in the PIC microcontroller, you need a programmer (e.g., Microchip Technology’s PICkit) and a Tag-Connect cable, which eliminates the need for a six-pin header on the PCB, so it saves board space and cost.

The rest of the article provides details of how to create the dongle firmware, how to add trace statements to the DUT software being monitored, and how to use the GUI version of the trace viewer.

The tracing system is complete, but it can be enhanced, Riemersma says. “Future improvements to the tracing system would include the ability to draw graphs (e.g., for task switches or queue capacity) or a way to get higher precision timestamps of received trace packets,” he says.

For Riemersma’s full article, refer to our April issue now available for membership download or single-issue purchase.

Dual-Channel 3G-SDI Video/Audio Capture Card


ADLINK PCIe-2602 Video/Audio Capture Card

The PCIe-2602 is an SDI video/audio capture card that supports all SD/HD/3G-SDI signals and operates at six times the resolution of regular VGA connections. The card also provides video quality with lossless full color YUV 4:4:4 images for sharp, clean images.

The PCIe-2602 is well suited for medical imaging and intelligent video surveillance and analytics. With up to 12-bit pixel depth, the card  provides extreme image clarity and smoother transitions from color-to-color enhance image detail to support critical medical imaging applications, including picture archiving and communication system (PACS) endoscopy and broadcasting.

The card’s features include low latency uncompressed video streaming, CPU offloading, and support for high-quality live viewing for video analytics of real-time image acquisition, as required in casino and defense environments. PCIe-2602 signals can be transmitted over 100 m when combined with a 75-Ω coaxial cable.

The PCIe-2602 is equipped with RS-485 and digital I/O. It accommodates external devices (e.g., PTZ cameras and sensors) and supports Windows 7/XP OSes. The card comes with ADLINK’s ViewCreator Pro utility to enable setup, configuration, testing, and system debugging without any software programming. All ADLINK drivers are compatible with Microsoft DirectShow.

Contact ADLINK for pricing.

ADLINK Technology, Inc.

A Well-Organized Workspace for Home Automation Systems

Organization and plenty of space to work on projects are the main elements of Dean Boman’s workspace (see Photo 1). Boman, a retired systems engineer, says most of his projects involve home automation. He described some of his workspace features via e-mail:

My test equipment suite consists of a Rigol digital oscilloscope, a triple-output power supply, various single-output power supplies, and several Microchip Technology in-circuit development tools.

I have also built a simple logic analyzer, an FPGA programmer, and an EPROM programmer. For PCB fabrication, I have a complete setup from MG Chemicals to expose, develop, etch, and plate boards up to about 6” × 9” in size.

Photo 1: Boman’s workbench includes overhead cabinets to help reduce clutter. The computer in the foreground is his web server and main home-automation system controller. (Source: D. Boman)

Photo 1: Boman’s workbench includes overhead cabinets to help reduce clutter. The computer in the foreground is his web server and main home-automation system controller. (Source: D. Boman)

Boman is currently troubleshooting a small 1-W ham radio transmitter (see Photo 2).

Photo 2: Boman is currently troubleshooting a small 1-W ham radio transmitter (Source: D. Boman)

Photo 2: Here is his workbench with the radio transmitter. (Source: D. Boman)

Boman says the 10’ long countertop surface (in the background in Photo 3) is:

Great for working on larger items (e.g., computers). It is also a great surface for debugging designs as you have plenty of room for test equipment, drawings, and datasheets.

Photo 3: Boman’s setup includes plenty of spacefor large projects. (Source: D. Boman)

Photo 3: Boman’s setup includes plenty of room for large projects. (Source: D. Boman)

Most of Boman’s projects involve in-home automation (see Photo 4).

My current system provides functions such as: security system monitoring, irrigation control, water leak detection, temperature monitoring, electrical usage monitoring, fire detection, access control, weather monitoring, water usage monitoring, solar hot water system control, and security video recording. I also have an Extra Class ham radio license (WE7J) and build some ham radio equipment.

Here is how he described his system setup:

The shelf on the top contains the network routers and the security system. The cabinets on the wall contain an irrigation system controller and a network monitor for network management. I was fortunate in that we built a custom home a few years ago so I was able to run about two miles of cabling in the walls during construction.

Photo 4: Boman has various elements of his home-automation control system mounted on the wall. (Source: D. Boman)

Photo 4: Boman has various elements of his home-automation control system mounted on the wall. (Source: D. Boman)

Boman uses small containers to hold an inventory of surface-mount components (see Photo 5).

Over the past 10 years or so I have migrated to doing surface-mount designs almost exclusively. I have found that once you get over the learning curve, the surface-mount designs are much simpler to design and troubleshoot then the through-hole type technology. The printed wiring boards are also much simpler to fabricate, which is important since I etch my own boards.

Photo 5: Surface-mount components are neatly corralled in containers. (Source: D. Boman)

Photo 5: Surface-mount components are neatly corralled in containers. (Source: D. Boman)

Overall, Boman’s setup is well suited to his interests. He keeps everything handy in well-organized containers and has plenty of testing space In addition, his custom-built home enabled him to run behind-the-scenes cabling, freeing up valuable workspace.

Do you want to share images of your workspace, hackspace, or “circuit cellar”? Send your images and space info to editor<at>