Machine Learning Eases Power Requirements for Edge and IoT Devices
The Internet of Things (IoT) has become integrated into nearly every industry and consumer market, and the trend is expected to grow exponentially. It has also fueled increased demand for devices with greater intelligence, specifically machine learning (ML), at the network edge. Deploying ML models or algorithms on edge or IoT devices diminishes the need to rely on the cloud for processing, which can be inconvenient, expensive, or both. While utilizing the cloud has several advantages, including improved security and reduced latencies, it can be challenging when that processing power is shifted to the devices themselves.
One of the primary challenges of running ML on those resource-constrained edge devices is the amount of power that can be consumed depending on the application. Large or complex ML models have a habit of taxing hardware to its limits, inducing it to draw large amounts of power. Consuming large amounts of power can adversely affect those edge and IoT devices, reducing battery life, incurring higher operating costs, and increasing failure rates.
There are several ways to mitigate the amounts of power on resource-limited devices, including hardware acceleration using GPUs and optimized CPUs; federated learning, which trains the AI algorithm locally on edge devices; and sleep/wake scheduling, which powers down the device until it’s needed. A new method of lowering the amount of power required on edge and IoT devices involves using optimized ML models that reduce their size and complexities, thus using less power for the intended application.
TinyML  is one of those scaled-down platforms that’s optimized for use on limited hardware, including microcontrollers (MCUs), single board computers (SBC), and other edge devices. It can be traced back several years as it was intertwined with advancements in machine learning and embedded systems (Figure 1).
Several factors led to its emergence, with the first being the increased data growth from different sources that called for processing data at the edge. Cloud services at the time were limited in resources in terms of latency, privacy, and power consumption. Advances in hardware also contributed to the release of TinyML, as MCUs became more powerful with increased efficiency in smaller, more energy-efficient formats. Finally, the progress in ML techniques, especially the design and training of neural networks, have contributed to the development of TinyML.
Of course, the main contributing factors to the rise of TinyML were the introduction and widespread availability of IoT devices and hardware that hit the market over the last decade. The inclusion of low-cost hardware propelled the technology forward at an explosive rate, allowing engineers and makers to develop projects for remote applications. The only thing holding those projects back was the amount of power they consumed, which fell directly on hardware optimization and battery utilization. The idea of using ML or other AI platforms to mitigate power consumption was only a theory during IoT’s infancy.
That said, TinyML still functions in much the same manner as it had when it became available in 2020. In order to work with significant resource constraints, the TinyML algorithms must conform to several strategies, including being used as an inference of pre-trained models—meaning it is trained before being deployed on the host hardware, which is how space is saved for other applications. That footprint is further reduced by trimming or pruning the neural networks (or synapses) underlining the TinyML models.
Quantization is then employed to reduce the memory required to store numerical values, translating floating-point numbers (4 bytes each) to 8-bit integers (1 byte each), for example. A technique known as Knowledge Distillation is then used to help identify and retain only the most essential features of a model. Combining these strategies allows users to run additional applications without bogging the hardware down, and frees up additional resources such as power management.
TINYML PROPERTIES AND FEATURES
TinyML has multiple properties and features that make it ideal for edge and IoT applications. While there are features that can be utilized via project applications, TinyML provides a host of its own, including Model Optimization, which streamlines models for deployment on small devices. This uses compression techniques to make the models even smaller and more efficient, in addition to those methods mentioned earlier.
TinyML also features real-time inference for edge and IoT devices, allowing them to make decisions without relying on the cloud or other connectivity options. This is significant for applications that require rapid responses with low latencies. Privacy and security are also notable features of the platform, as data doesn’t need to be sent elsewhere for processing, reducing the risk of exposing sensitive information. It also reduces the attack surface for potential breaches, keeping data secure.
Additional features include development and frameworks that are continuously emerging with new tools supporting TinyML development. These include libraries, optimization tools, and deployment solutions designed for space-constrained devices and platforms. Many devices include multiple sensors, and TinyML’s Sensor Fusion allows models to process the data garnered from those sensors and make informed decisions based on the data streams. Finally, the power-saving techniques help extend the battery life of any edge or IoT device without compromising processing power.
Of course, TinyML can take advantage of a host of AI processing architectures, including TensorFlow and PyTorch, along with lightweight libraries such as Arm CMSIS-NN and TensorFlow Lite Micro for MCU optimization. It can also take advantage of integrated development environments (IDEs), including Arduino IDE and PlatformIO, and cloud-based platforms for modeling and deployment, including Edge Impulse and AWS IoT Greengrass.
The applications of TinyML are nearly endless as well. For example, it enables industrial predictive maintenance—a host of smart MCUs outfitted with TinyML models can monitor manufacturing systems autonomously. TinyML can be helpful in agriculture, too, by monitoring crops for diseases and water absorption rates, as well as soil pH levels, insect levels, and the amount of sunlight crops receive. Healthcare is another area where TinyML shines, used for everything from mosquito monitoring where transmitted diseases are a problem, to air quality monitoring for people with lung issues. TinyML has also been used to reduce fatal collisions with elephants on Indian railways and to reduce the risk of whale strikes in busy shipping lanes.
The use of TinyML in edge and IoT applications is a game-changer for machine learning on host systems. It’s already been proven through the number of applications developed by numerous academic institutions, engineers, and makers.
One barrier has been the amount of power needed to drive more demanding ventures, which has been mitigated using several techniques, including hardware optimization. Placing AI directly on the host processor significantly reduces power consumption, opening the door to many new projects and possibilities in various industries and commercial markets.
As hardware advances and new applications emerge, TinyML will likely continue to grow and adapt to take advantage of new technologies. It has the potential to enable a wide range of new applications that aren’t currently feasible. The future could bring adoption across a broader range of industries than what it’s used for today, such as aerospace and automotive industries, where continued monitoring is a vital asset.
It’s safe to say more powerful and energy-efficient models will be produced for increasingly complex tasks. Standardization will make the technology more accessible and accelerate development and deployment. Of course, there will be obstacles along the way, but the bright people at the TinyML Foundation are already looking at ways to overcome some of those challenges. It will be interesting to see how the ML platform will evolve.
PUBLISHED IN CIRCUIT CELLAR MAGAZINE • DECEMBER #401 – Get a PDF of the issueSponsor this Article
Brian Santo has been writing about advanced electronics and emerging technologies for over 30 years for a number of publications, including Electronic News, IEEE Spectrum, and CED, later serving as editor-in-chief of EE Times.