CC Blog Editor's Letter Insights

Benchmarks for the IoT

Written by Jeff Child

I remember quite vividly back in 1997 when Marcus Levy founded the Embedded Microprocessor Benchmark Consortium, better known as EEMBC. It was big deal at the time because, while benchmarks where common in the consumer computing world of desktop/laptop processors, no one had ever crafted any serious benchmarks for embedded processors. I was an editor covering embedded systems technology at the time, and Marcus, as an editor with EDN Magazine back then, traveled in the same circles as I did. On both the editorial side and on the processor vendor side, he had enormous respect in the industry—making him an ideal person to spin up an effort like EEMBC.

Creating benchmarks for embedded processors was more complicated than for general purpose processors, but EEMBC was up the challenge. Fast forward to today, and EEEBC now boasts a rich list of performance benchmarks for the hardware and software used in a variety of applications including autonomous driving, mobile imaging, mobile devices and many others. In recent years, the group has taken on the complex challenge of developing benchmarks for the Internet-of-Things (IoT).

I recently had the chance to talk with EEMBC’s current president, Peter Torelli, about the consortium’s latest effort: its IoTMark-BLE benchmark. It’s part of the EEMBC’s IoTMark benchmarking suite for measuring the combined energy consumption of an edge node’s sensor interface, processor and radio interface. IoTMark-BLE focuses on Bluetooth Low Energy (BLE) devices. In late September, EEMBC announced that the IoTMark-BLE benchmark is available for licensing.

The IoTMark-BLE benchmark profile models a real IoT edge node consisting of an I²C sensor and a BLE radio through sleep, advertise and connected-mode operation. The benchmark measures the energy required to power the edge node platform and to run the tests fed by the benchmark. At the center of the benchmark is the IoTConnect framework, a low-cost benchmarking harness used by multiple EEMBC benchmarks. The framework provides an external sensor emulator (the I/O Manager), a BLE gateway (the radio manager) and an Energy Monitor.

Benchmark users interact with the DUT via an interface with which they can set a number of tightly defined parameters, such as connection interval, I²C speed, BLE transmission power and more. Default values are provided to enable direct comparisons between DUTs, or users can change them to analyze a design’s sensitivity to each parameter. IoTMark-BLE’s IoTConnect framework supports microcontrollers (MCUs) and radio modules from any vendor, and it is compatible with any embedded OS, software stack or OEM hardware.

It makes sense that IoT benchmarks focus on power and energy use. IoT edge devices need to work in remote locations near the sensors they’re linked with. With that in mind, Peter Torelli says that the benchmark measures everything inside an IoT system-on-chip (SoC)—including the peripheral I/O reading from the I2C sensor, the transmit and receive amplifiers in the BLE radio—everything except the sensor itself. Torelli says it was important to not use intelligent sensors for the benchmark, the idea being that its important that the MCU’s role performing communication be part of the measurement. Interestingly, in developing the benchmark, it was found that even the software stacks on IoT SoCs have a big impact on performance. “Some are very efficient when they’re in advertise mode or in active mode, and then go to sleep,” says Torelli, “And there are others that remain active for much longer times and burn a lot of power.”

— ADVERTISMENT—

Advertise Here

Shifting gears, I want to take moment to praise long time columnist and member of the Circuit Cellar family, Ed Nisley. Over 30 years ago, Steve Ciarcia asked Ed to write a regular column for the brand-new Circuit Cellar INK magazine. After an even 200 articles, Ed decided to make his September column his last. Thank you, Ed, for your many years of insightful, quality work in the pages of this magazine. You’ll be missed. Readers can follow Ed’s continuing series of shop notes, projects and curiosities on his blog at softsolder.com.

Let me welcome Brian Millier as our newest Circuit Cellar columnist—his column Pickup Up Mixed Signals begins this issue. Brian is no stranger to the magazine, penning over 50 guest features in the magazine since the mid-90s on a variety of topics including guitar amplifier electronics, IoT system design, LCDs and many others. I’m thrilled to have Brian joining our team. With his help, we promise to continue fulfilling Circuit Cellar’s role as the leading media platform aimed at inspiring the evolution of embedded system design.

This appears in the November 340 issue of Circuit Cellar magazine

Not a Circuit Cellar subscriber?  Don’t be left out! Sign up today:


Don't miss out on upcoming issues of Circuit Cellar. Subscribe today!

 
 
Note: We’ve made the October 2017 issue of Circuit Cellar available as a free sample issue. In it, you’ll find a rich variety of the kinds of articles and information that exemplify a typical issue of the current magazine.


Would you like to write for Circuit Cellar? We are always accepting articles/posts from the technical community. Get in touch with us and let's discuss your ideas.

Editor-in-Chief at Circuit Cellar | Website | + posts

Jeff Child has more than 28 years of experience in the technology magazine business—including editing and writing technical content, and engaging in all aspects of magazine leadership and production. He joined the Circuit Cellar after serving as Editor-in-Chief of COTS Journal for over 10 years. Over his career Jeff held senior editorial positions at several of leading electronic engineering publications, including EE Times and Electronic Design and RTC Magazine. Before entering the world of technology journalism, Jeff worked as a design engineer in the data acquisition market.