Using ESP-NOW Protocol – Part 1

A Low-Power Wi-Fi Alternative

ESP32-NOW protocol enables low-overhead, peer-to-peer wireless data transfers among members of Espressif’s ESP MCU family. In Part 1 of this series, Brian discusses the benefits of the protocol, and shares how it improved battery life in his ESP-based leak detector modules.

I’ve built several ESP8266/32 home monitoring and control projects over the past few years. In each case, the project connected via Wi-Fi to my home router/wireless access point. Sometimes I implemented web-server firmware in the project. In other cases, I used various cloud services to collect data.

This worked OK, but there were some challenges. I am lucky enough to have Internet service delivered via fiber optic cable right into my house. Therefore, I must use the integrated fiber optic modem/router/Wi-Fi Access Point (WAP) unit supplied by my service provider. Although I am getting 500Mbps speeds consistently, for some unknown reason, I occasionally find that one or more devices in the house lose their connection (IP lease) to the modem/router/WAP. This requires that the modem/router/WAP be unplugged or rebooted. I’m sure that it’s not a problem with my homebrew devices, since it’s just as likely to occur on my iPad or the TV set-top box itself. However, if home monitoring/control devices lose their wireless connection, it may not be evident immediately that rebooting is necessary.

CLOUD AND POWER ISSUES
The other issue I ran into concerned the cloud service I was using to handle three water leak detectors I had built. When I built the original units, my goal was to have them send me both an email and SMS text message if a leak was detected. Although the leak detector included a piezo speaker that provides a loud beep, there is no guarantee that you are going to be at home if a pipe springs a leak. Later in the article I’ll explain my earlier method of doing this, and why I decided to use another method.

Another issue that ties in with this is the power consumption of the ESP8266 or ESP32 when the units are battery operated—which is the case with the water leak detectors. While the ESP8266/32 MCUs have low-power sleep modes, when they “wake-up” and connect to your wireless access point, the amount of time it takes to perform the connection/authentication protocol is significant, and the power consumed by the Wi-Fi transmitter section is more than 350mA.

With all of this in mind, I became quite interested when Espressif introduced a peer-to-peer Wi-Fi communication protocol that promised a very short connection time. This protocol is called ESP-NOW and works equally well with both the ESP8266 and the ESP32 MCUs.

I figured that I would use this protocol with a dedicated central controller hub (using an ESP32) and remote sensor/activator units. I planned on abandoning the IFTTT external cloud service (plus a few other cloud schemes I had tried), and instead have my dedicated Controller unit use Google’s Gmail server. I felt that Google was unlikely to change the way it works any time soon, so the firmware for this project should hold up well.

This dedicated central controller hub contains a 4.3″ TFT touchscreen display, which allows for a fairly nice user interface. It would give me a visual indication that all the battery-operated leak detectors were still running. It would also allow me to control my air exchanger and irrigation pump on/off times, using a touch-controlled 24-hour table on the display. Finally, it would display temperature/humidity at the hub location and leak detector locations.

I used an ESP32 for the central controller hub because it contains more GPIO lines. Since it’s powered by the AC line, the Controller’s power consumption (3W) is not an important factor. Figure 1 shows the unit in operation. Later in the article, when I describe the battery-powered leak detectors, I’ll detail the ways in which I reduced their power consumption.

ESP32-NOW PROTOCOL
The ESP32-NOW protocol was developed by Espressif to enable low-overhead peer-to-peer wireless data transfers among members of the ESP MCU family. The ESP-NOW protocol transmits in the same 2.4GHz band as Wi-Fi, but no connection is made to your Wi-Fi WAP, and it doesn’t interfere with your home’s normal Wi-Fi operations.

Power savings are not realized by reducing the RF transmitter power, but rather, by simplifying the connection protocol, thus reducing the time it takes for one unit to connect to another and then transfer a data packet. The shorter the time that the complete transfer takes, the less battery power is needed.

The ESP-NOW protocol is tailored for small data packets of 250 bytes maximum. If you needed to transfer large blocks of data, the much shorter connection time of the ESP-NOW protocol would not be so advantageous, since you would be spending most of the time transferring the actual data packets anyway.

The usage of the terms Master and Slave is now being discouraged for some valid social reasons. We’ve all used these terms in electronics circuits, ICs and communication protocols for decades. Recently I’ve seen some forum threads where alternate terms are being suggested for the common SPI MISO and MOSI signals, for example.

Espressif’s literature on the ESP-NOW protocol calls it a connectionless communication protocol and defines devices as Controller or Slave. Those terms show up throughout the Espressif API documentation, and third-party-ESP-NOW examples also use the Master/Slave nomenclature. In this article, I’ll try to stick to the terms Controller and Remote, so keep that in mind if you are looking through the Espressif documentation or my program code, and find that the terms don’t match up exactly.

Espressif doesn’t provide comprehensive documentation of this protocol. On its website, they describe the Frame Format and Security, and outline the various functions in its ESP-NOW API. But it doesn’t give any example code, and since you must write MCU code for both ends of the data transfer, getting started is difficult.

ARDUINO IDE
I use the Arduino IDE to develop code for both the ESP8266 and ESP32, and there are now built-in ESP-NOW examples. To use the Arduino IDE, you’ll need to have the ESP8266/ESP32 board package(s) installed in the Arduino IDE. As long as you have current versions of the board packages, the ESP-NOW library will be included automatically. I found examples of ESP-NOW programs–but only for the ESP32. Such examples will show up in Arduino IDE under the File > Example menu in the “Examples for the ESP32 Dev. Module” section (or whatever ESP32 module you have chosen as the target board). The actual disk location of these examples is:

C:\Users\yourusername\Documents\Arduino\Hardware\espressif\esp32\libraries\ESP32\examples\ESPNow

The ESP-NOW examples are in two folders—Basic and Multi-Slave—and each folder has a folder for both the Controller and the Remote devices. The code that I wrote for the central controller hub is based on the Master.ino example program, and all the Remotes are based on the Slave.ino example.

In my project, I wanted the central controller hub to be able to automatically scan for any Remote devices present/powered-up, and automatically connect to those devices (a process called “pairing”). For this to work, you must configure the ESP8266 RF circuitry on all the Remote units to be in the Access Point mode, or they won’t be seen by the scan. As Access Points, they will broadcast an SSID, which I define in my firmware to describe the function of that Remote unit (including Air Exchanger, Pump Controller and Leak Detector1).

HARD-CODED MAC ADDRESSES
The Remote units don’t perform any RF scans. Instead, the MAC address of the central controller hub is hard-coded into their firmware. I felt this would be OK, because the central controller hub’s ESP32 MCU would not be swapped. Therefore its MAC address would be constant. The central controller hub (ESP32) is configured in the Station mode. Incidentally, both the ESP32 and ESP8266 MCUs contain two hard-coded MAC addresses—one for Station Mode and one for Access mode—so you must choose the applicable one.

The example programs were for the ESP32, which was fine for my ESP32-based central controller hub. Since the Remotes needed fewer GPIO lines and would benefit from lower power consumption, I used the ESP8266 modules for them. The code is virtually the same in both cases, but there is a difference in the way the libraries are included in each case.

For ESP8266:

#include <ESP8266WiFi.h>
extern "C" {
#include <espnow.h>
#include "user_interface.h"
}

For ESP32:

#include <WiFi.h>
#include <esp_now.h>

I believe the discrepancy arises from the fact that the ESP32 runs an RTOS as its operating firmware, whereas the ESP8266 does not. Therefore, the APIs for the two MCUs differ in some respects.

USING THE ESP-NOW API
An ESP-NOW device can operate under three roles:

1) ESP_NOW_ROLE_CONTROLLER
2) ESP_NOW_ROLE_SLAVE
3) ESP_NOW_ROLE_COMBO

The first two are self-explanatory. The third, Combo role, is a mode in which the device can act as both a Remote and a Controller. What I have found is that if a device is a Remote, it can receive messages, but not send them. If it is placed in the Combo mode, it can receive and send messages.

First, let’s look at what must be done to get the central controller unit’s ESP-NOW protocol running. For this project, the controller must be able to address multiple Remote units, so it is a bit more involved than a situation where there is only one Controller and one Remote unit.

We place the ESP32 in Station mode using the statement:

WiFi.mode(WIFI_STA);

After that, we do something counter-intuitive: we call the WiFi.disconnect() function. This disables the firmware stack that would normally be used for a standard Wi-Fi connection. This means that when we are using the ESP-NOW protocol, we are not connected to our home WAP, so we can’t do any normal Internet operations. If you are paying close attention, you may wonder how the central controller will be able to contact the Gmail server when it needs to send out an alarm email/SMS message. This involves a trick, which I’ll explain later in the article.

With the normal Wi-Fi protocol stack disabled, we next enable the ESP-NOW protocol by calling:

InitESPNow();

This routine will initialize the ESP-NOW stack. Nothing should prevent this from happening, but if the initialization is unsuccessful, the routine merely re-starts the ESP32. I have never seen a re-start occur when calling this routine.

Some functions of the ESP-NOW API are performed by callback routines. Specifically, there are two callbacks. OnDataSent gets called when an outgoing message has been sent. OnDataRecv gets called when some ESP-NOW data have been received.

Your program must define (register) routines that will be called for each of these two situations. That is done as follows:

esp_now_register_send_cb(OnDataSent);
esp_now_register_recv_cb(OnDataRecv);

You will have to provide the OnDataSent and OnDataRecv routines elsewhere in your program. The OnDataSent callback routine will be invoked shortly after you send a message. It will either return a “0” if the target received the message (and transmitted an acknowledgement), or a “1” if an acknowledgement is not received within the timeout interval.

PAIRING CONTROLLER AND REMOTES
At this point, you are ready to “pair” the Controller with all the available Remote devices. That is done in the DeviceListMenu() routine. This routine is called automatically when the Controller is first powered-up, and can also be invoked by the user from the main menu screen if desired. This routine invokes the function WiFi.scanNetworks(); repeatedly for 60 seconds. It should find all the powered-up Remote devices, since they are configured as Access Points. (It will also find your home Wi-Fi access point and maybe those belonging to your neighbors, but you don’t want to pair with those.) For each device found, the scanNetworks() routine will discover the device’s MAC address, its SSID and its RSSI (RF signal strength).

I compare this SSID with the SSIDs that I have assigned to the various Remotes. When I find a match, two things happen:

1) I make an entry in the ESP32’s Non-Volatile Memory (using the preferences class) with the SSID of the device and its MAC address.
2) I execute the ManageSlave() function, which checks to see if that MAC address has already been paired. If not, it adds that MAC address to the ESP32’s peer list.

While scanning, the DeviceListMenu() routine provides a touchscreen display of the MAC address for each of the devices found in the preference database—that is, all devices previously found or found during the current scan. Any devices that are powered-up during the 60s scan will be (re)discovered. The RSSI value (current RF signal strength) will also appear, indicating that the device is currently running/discovered. If the RSSI is less than -90, it means that this Remote is probably so far away that it is on the edge of working reliably.

The leak detectors, being battery operated, will only stay on for a brief measurement of time, and then remain powered off for 10 minutes (as the firmware is currently written). Because of this, when a leak detector board is newly introduced to the system, it should be powered-up with the leak electrodes bridged.

It will then repeatedly try to send an alarm message to the central control hub, which, at this time, should be placed in the Device List screen, so that it can be discovered and paired. This only has to happen once; after that, the central control hub will remember that it has already been paired, because the information will be stored in the preferences database. After the ESP-NOW protocol has been initialized, and the devices paired, sending and receiving data is straightforward.

TRANSMISSION AND RECEPTION
Data can be transmitted merely by calling the following routine:

int result= esp_now_send(remoteMac, bs, sizeof(sensorData));

The parameters are:

1) A pointer to a 6-byte array containing the MAC number of the receiving device
2) A pointer to the buffer containing the message data
3) The size of the packet to be transmitted (250 bytes maximum)

The routine will return a “0” if the message has been transmitted successfully, but not necessarily received by the target.

The ESP-NOW data transfers are completely data-type agnostic; you can send any values as long as the message length is less than 250 characters, and you must pass the proper message length in the third parameter. The OnDataSent callback routine mentioned earlier will be called automatically, and will provide a status byte indicating whether the message was received successfully at the target device (status = 0 if successful).

Whenever ESP-NOW messages are received, the callback routine registered by the following routine, will be called:

esp_now_register_recv_cb(OnDataRecv);

In this case, I have registered a routine labeled OnDataRecv to handle incoming data. This routine gets passed the following three parameters:

1) A pointer to a 6-byte array containing the MAC address of the device that transmitted the message.
2) A pointer to a data buffer containing the message. You should use the memcpy() routine to move these data to the buffer that you’re using to read/parse data, thus freeing-up the ESP-NOW’s buffer for further incoming messages.
3) The size of the received message in bytes. If you were receiving a string message, this size would include the string terminator character (usually a 0x00).

REMOTE OR COMBO MODE
Configuring an ESP8266/32 device as a Remote unit involves setting it up in the Combo role, as mentioned earlier. This is a bit different than the Controller role.

First, we must configure the ESP8266/32’s RF section in the Access Point mode, as follows:

WiFi.mode(WIFI_AP);

I found it somewhat counter-intuitive to set up the Controller as a Station, and all the Remotes as Access Points. However, for the Controller to be able to “discover” the presence of the Remote devices, it must do a Wi-Fi network scan to find them. This requires the Remotes to be configured as Access Points.

In the case of a single Controller and a single Remote, where both units know the MAC number of the other, it may be possible to set up both units in the Station mode and skip the Wi-Fi network scan completely. Espressif doesn’t explain any of this in their documentation, nor have I tried this myself.

After setting up the Access Point mode, you need to call the following routine to define the Access Point parameters:

bool result = WiFi.softAP(SSID, "Slave_2_Password", CHANNEL, 0);

The four parameters are:

1) SSID. I use a descriptive name like “PumpController” or “LeakDetector1” for the SSID, so that the Controller knows what the device does, and can handle messages accordingly.
2) Password. I specify a dummy password, since none the devices in this project is reachable via the Internet.
3) Channel. The Channel is specified as 1-13, which corresponds to standard Wi-Fi channels in the 2.4GHz band. The actual channel chosen doesn’t matter, but to communicate with each other, all ESP- NOW devices must be configured for the same channel.
4) Hidden. If this parameter is set to “1,” the SSID will not be revealed during a Wi-Fi network scan. This must be set to “0” in this project, because the Controller identifies the Remote devices by their SSID.

Like the Controller, the Remote units must next perform the WiFi.disconnect() function. With the normal Wi-Fi protocol stack disabled, one must next enable the ESP-NOW protocol, using the esp_now_init() function. The ESP-NOW role must be set using the following statement:

esp_now_set_self_role(ESP_NOW_ROLE_COMBO);

For the Remote module to be able to send and receive messages, the Combo role must be used. I used the Combo role for all my Remote units, even though some of them didn’t have to send any data back to the Controller.

I noted that this esp_now_set_self_role function is not defined when you are compiling the firmware for the ESP32. The ESP-NOW library for each device is obviously different. From the examples (which were for an ESP32), it appeared that the ESP-32-NOW role did not have to be set explicitly when using an ESP32. Or, it’s possible that the role gets set, by default, to Controller during the esp_now_init function, and that is what I need for the central controller (ESP32) anyway. Apart from those steps, registering the two callback routines and the ESP-NOW send and receive routines is the same as described earlier in my description of the Controller’s ESP-NOW configuration.

The Remote module’s pairing process is different. There is no network scan performed. Instead, the API’s pairing function is called with the MAC address of the central controller hub:

(esp_now_add_peer(remoteMac, ESP_NOW_ROLE_COMBO, CHANNEL, NULL, 0)

The fact that I specify the Controller’s MAC address by the variable remoteMac only means that it refers to the “other” device, not to the device’s local MAC address. This is a bit confusing, but the examples were coded this way. The other parameters are mostly self-explanatory, except for the NULL, which specifies that no encryption is used.

To allow for easier updates, for example, if the Controller’s ESP32 had to be swapped out, I defined the Controller’s MAC address as the byte array remoteMAC at the beginning of the firmware in all the Remotes:

uint8_t remoteMac[] = {0x84, 0x0D, 0x8E, 0x28, 0x9C, 0x74};

Note that the ESP-NOW communication protocol can be done using encryption. If encryption is used, only 10 devices can be linked together, otherwise the maximum is 20. There isn’t a lot of documentation on the encryption. Since my ESP-NOW devices would not be exposed to the anyone on the Internet through my home Wi-Fi router, I did not feel the need to implement encryption. I have a very large lot and the RF signals from the various ESP32/8266s in this project cannot be picked up by anyone that I don’t allow in/near my house.

LOW-POWER LEAK DETECTORS
One of the reasons I re-designed my home control system using ESP-NOW was to allow for lower power consumption on the battery-powered, ESP8266-based leak detectors. One such unit is shown in the lead image of this article. Let’s look at how they operate.

Table 1 shows the ESP8266’s power consumption in various operating modes. Some are expressed as ranges, since actual user measurements often don’t match Espressif’s published specifications. I measured the ESP8266 consuming about 80mA at 3.3V when receiving or otherwise executing code. In this mode, its RF circuitry (Wi-Fi/ Bluetooth) is powered-up and can receive whatever type of RF signal the unit has been configured for (Wi-Fi or Bluetooth, but not both).

When the ESP8266 is transmitting a signal, current consumption increases dramatically, with numerous short “transmit” power spikes of about 350-450mA. The average current consumption is nowhere near the peak current of these spikes. However, when connecting to your home’s WAP, these spikes will occur frequently during the 2-5 seconds needed to complete the connection. This lengthy connection time is a result of the complex Wi-Fi protocol that is being executed, and overhead due to security considerations.

Although I can see the transmit spikes on my oscilloscope, they are so narrow that it’s difficult to measure their contribution to the overall current consumption accurately. My digital multimeter can measure the average DC current, but it only performs 1-2 measurements per second, and its readings fluctuate markedly during the connection period.

From the oscilloscope trace, I would estimate that the transmit spikes would add about 15% to the 80mA average current that the ESP8266 draws when it is not transmitting a Wi-Fi signal. Figure 2 shows a scope capture of an ESP8266 connecting to my Wi-Fi Access Point. I am measuring current via the voltage drop across a 0.25Ω resistor in the negative power lead. The transmit spikes measure at about 408mA, and the non-transmitting execution power consumption is about 80mA.

I can tell from the serial output of the running program that the Wi-Fi connection is made in 3-4 seconds. However, from the scope capture, you can see that the transmissions persist beyond this time, though the program is in a loop printing a message every second. I don’t know why this is, but it results in significant power being consumed, even when the user’s program isn’t calling for any Wi-Fi activity.

It’s obvious from the above that 92mA (80 + 15%) average current will be drawn for several seconds just to make the initial connection to the WAP. Even if the ESP8266 could be completely powered-down for long periods of time between measurements/Wi-Fi connection, how long would a small LiPo cell last?

Figure 3 shows the results from a web-based battery-life calculator (IoT Battery-Life Calculator) [1], for a scenario in which the ESP8266 was powered-down completely during what is labeled the “sleep” interval. It only operates once every 15 minutes to take a reading. A battery life of 109 days isn’t too good. Note that I am also assuming that the ESP8266 is drawing no power at all when not actually running the program doing the measurement/data transfer.

DEEP SLEEP MODE
In real life, when the ESP8266 is placed in Deep-Sleep mode, it will draw between 10µA and 20µA. You must also add some current to this to allow for the quiescent current draw of the 3.3V voltage regulator. If you were to use the common Texas Instruments (TI) LM1117MPX-3.3, for example, its quiescent current is 5mA, which would totally overwhelm the 20µA ESP8266 Deep-Sleep current. However, you could choose a much lower quiescent current 3.3V LDO, such as the Microchip Technology MCP1702, which draws only 2.0µA quiescent current. If you run the calculations again with a Deep Sleep current of 20 + 2µA (MCP1702), you now get 94 days, which is actually not a lot less time.

The largest contributor to the power used is the ESP8266’s approximately 92mA current draw for the 3 seconds it takes to connect to your Wi-Fi WAP. By using the ESP-NOW protocol, you can shrink the time it takes to boot-up the ESP8266 and check for a leak to about 168ms.

Since a leak will rarely be present, you don’t have to make a connection to the central controller to report this. However, from a practical standpoint, once out of every “x” times that the ESP8266 is powered-up and checks for a leak, it is prudent to also measure the battery voltage and report that back to the central controller. For example, if you power-up the ESP8266 every 10 minutes for a leak check, and you check the battery voltage every 12 iterations, the controller will be getting a report every 2 hours about battery condition. Worst-case leak detection delay would be 10 minutes.

To further improve battery life, I decided to use a Microchip MIC5323-3.3 LDO regulator with an enable line. I control this enable line with the alarm output pin of a Microchip MCP79401 real-time clock chip. This way, the ESP8266 is powered-down completely during the Deep Sleep interval. The leak detector’s total quiescent current is now only about 4µA (1µA for the regulator, 1µA for the MCP79401 and 2µA for the enable line’s 2.2MΩ pull-up resistor). The “sleep” interval is now controlled by the alarm function in the RTC chip.

As the program is currently written, the ESP8266 operates for 170ms and draws 80mA during 11 of the 12 reboot cycles (assuming no leak is detected). On the twelfth reboot, it operates for 2,300ms since it must:

1) Wait 2 seconds for the AM2320 sensor (Aosong Electronics) to “wake-up.”
2) Transmit a NOW packet back to the Controller.

During this iteration, the ESP8266 draws an average of about 400mA during the 40ms that it is transmitting/receiving the NOW packet (Figure 4). For the remaining 2,260ms, it draws about 80mA. Therefore, the average current draw over the 2,300ms is:

The power consumption is thus:

1) Eleven iterations of 85.6mA for 170ms
(= 14.5mA-seconds)
2) One iteration of 85.6mA for 2,300ms (=198.9mA-seconds)
3) This makes the total current consumption equal to 213.4mA-seconds over the 120-minute interval, or 12 iterations, 10 minutes apart.

Plugging these numbers into the IoT Battery-Life Calculator [1], yields 991 days (Figure 5). In actuality, the LiPo cell will experience some self-discharge, but it is not unreasonable to expect to get 1 year of operation between battery recharges. This big improvement over the 94 days quoted earlier, is mostly due to the much shorter transmission times of the ESP-NOW protocol. The rest of the improvement comes from adding the MCP79401 RTC to turn off the power to the ESP8266 completely during “sleep” intervals.

In Part 2 of this series (in Circuit Cellar 370, May 2021), I’ll describe the circuitry for the Controller and various modules. I’ll explain how I can make the Controller ESP32 run in the ESP-NOW mode, and also be able to send an email message to the Google Mail server, which requires Internet access via my home WAP. 

Read Part 2 Here

RESOURCES

Reference:
[1]  IoT Battery Life Calculatorhttp://www.of-things.de/battery-life-calculator.php

Aosong Electronics | www.aosong.com
Espressif Systems | www.espressif.com
Microchip Technology | www.microchip.com
Texas Instruments | www.ti.com

PUBLISHED IN CIRCUIT CELLAR MAGAZINE • MARCH 2021 #368 – Get a PDF of the issue

Sponsor this Article