Using Amazon Alexa to Control Custom IoT Gadgets

Integrating Voice Recognition for Smart Home Projects

After purchasing an Echo smart speaker, Brian explores ways to control custom IoT gadgets by integrating Alexa voice recognition into an ESP32 board through an Arduino cloud service. In the first part of a two-part journey, he describes different methods, one through emulation and the second through custom integration. Both help him control his heat pump remotely.

Right from the start I was impressed at the ease of setup of the Echo device as well as how easy it was to pair them with the smart plugs/receptacle. I was also impressed with the Echo’s ability to understand my voice. It seemed to understand virtually any of the phrases that I spoke. It didn’t appear to make much difference how far I was away from the Echo, when issuing commands. It even seemed to ignore extraneous sounds, such as the TV being on, while I was speaking. I went through the Alexa voice “training” process initially, but Alexa was equally able to understand my wife, who has not done this “training” procedure.

My ulterior motive in purchasing the Echo smart speaker was to try to design some of my own IoT gadgets, that could be integrated with the Echo/Alexa voice recognition system. In the first part of a 2-part column I’ll describe three different approaches to integrating a custom IoT gadget with the Echo/Alexa smart home system.

SIMPLE EMULATION APPROACH

If your proposed Alexa-controlled application is simple on/off control that doesn’t lend itself to the commercially available Alexa compatible smart plugs, receptacles etc., the simplest way to accomplish this is through emulation. That is, you program your device in such a way that it emulates a common Alexa-compatible commercial product. Once you add that product’s Alexa Skill to your profile, your Echo device will “discover” your custom device as if it were one of those devices, and it will operate in the same way.

A popular way of doing this involves emulating the common Philips Hue smart lightbulb, which is Alexa-compatible. There is an Arduino library called Espalexa that works on both the ESP32 and ESP8266 MCUs. This is readily obtained using the Arduino Library Manager, and was written by Christian Schwinne. This library implements the Simple Service Discovery Protocol (SSDP) which allows an IoT device to advertise its presence on a WiFi network, allowing an Echo device to discover it. This process occurs without the need for either the DHCP or DNS services but co-exists with them, if they are present (almost always the case in a home WiFi network). In addition to this SSDP protocol, the ESP32/8266 MCU also implements a webserver.

To use this library in your program, you must add “#include <espalexa.h> at the head of your program, as well as <WiFi.h> (or ESP8266WiFi.h if you are using an ESP8266). Ahead of the setup() routine, you must instantiate the Espalexa class as follows:

Espalexa espalexa

As with any ESP32/8266 program that requires a WiFi network connection, you have to call WiFi.begin(ssid password); After that, you start up the Espalexa service with the following two lines of code:

espalexa.addDevice(“HP1”, HP1Changed); //simplest definition, default state off

espalexa.begin(

In the above, the name that Alexa will discover for this device is HP1 (my primary heat pump). HP1Changed is the label of the callback routine that will be invoked when an Echo device sends a command to device HP1. This callback is the following simple routine:

void HP2Changed(uint8_t brightness) {

if (brightness ==0) SendOffIRCode();

if (brightness == 255) SendOnIRCode();

}

In the loop() section of your program, you need to call the espalexa.loop() function periodically, to allow Espalexa to check for incoming messages and to call the appropriate call-back routine.

When the Echo sends a command to what it thinks is a Philips Hue lightbulb, it will send a brightness parameter of 0 for off and 255 for on. The brightness parameter is passed to this callback routine. I check its value and call one of two routines to send the proper IR code to the heat pump- turning it on or off as requested. While not applicable to this project, Alexa treats a Philips Hue lightbulb as dimmable, so you can send a brightness value from 0-100% to it, and the variable “brightness” will map this to 0-255 and pass that value on to the callback routine listed above.

For troubleshooting/debugging purposes, I recommend you add the following code, after the ESP32/8266 MCU has successfully connected to your home WiFi access point:

Serial.println(WiFi.localIP());

Take note of the IP# that your WiFi access point’s DHCP has assigned. With your IoT project up and running enter the following URL into your PC’s web browser:

http://DEVICE_IP_NUMBER/espalexa/

If everything is working properly you will get the short text message “Hello from Espalexa” and it will report the name and current state of that device.

CONTROL APPLIANCE REMOTELY

As I mentioned earlier, it’s easy to have an Echo device control most anything that plugs into a 120-volt outlet, using Alexa-compatible smart plugs and smart receptacles. However, if you have something that is operated by an IR remote control, then you can control it using a custom IoT device that responds to Echo commands (using Espalexa) and which calls routines that transmit the appropriate IR patterns. Let’s look at a device I designed, allowing my Echo devices to turn my heat pump on and off at programmed times.

If you read my column in Circuit Cellar #396, you’ll recall I designed a standalone controller that scheduled both my heat pump and a separate air exchanger. That unit worked fine, but I realized that an Echo device, with its amazing voice recognition and comprehensive scheduling abilities, could do the same tasks better. Therefore, I decided to build a new Alexa-controlled device to schedule the heat pump using IR commands. Since my air exchanger simply plugs into a wall receptacle, I was able to control it using a commercial TP-Link HS103 smart plug. Figure 1 shows the finished unit. While not visible, the LED which transmits the IR signal to the heat pump, protrudes out the side of the unit oriented towards the heat pump head unit mounted high on an adjacent wall.

The firmware for this project is written as an Arduino sketch, and the ESP32 board support package must first be added to the Arduino IDE. Basically, I re-used much of the code that I had written for the original heat pump controller to implement an IR “learning” remote. Please refer to Figure 2 for a schematic of this project. If SW3 is pressed at boot up, the ESP32 will enter it’s IR learning mode. Having already performed the IR “learning” process on my earlier controller, I knew that my Fujitsu heat pump used a 38 KHz IR carrier frequency. I am using a Vishay TSOP38238 IR receiver module, but you can get this same module for other common carrier frequencies as well. The ESP32 firmware for this project can be found on Circuit Cellar’s website. I’ve written a routine called analyseIR() that captures the timing of a Fujitsu IR pattern, for a single button press. I’ve used the ESP32’s specialized RMT function block (designed specifically for IR remote reception/transmission) in the past. Here, I just used the micros() function to capture the timing of the IR pulses/spaces. This timing information is stored in array buf[4000]. The Fujitsu IR remote uses a very long pattern, so I am allowing for up to 4 buttons with 250 discrete space intervals and 250 pulse intervals (both 16-bit integers). I call the analyseIR() twice, once for the ON command and once for the OFF command, and the two are stored sequentially in the buf[4000] array.

Storage of this buf[4000] array is done by saving it as the “/IRcode.bin” file in the SPIFFS file system that I have initialized in the ESP32’s flash memory. In the setup() routine, I call the SPIFFS.begin() function. The very first time that the firmware runs, this function will fail because the SPIFFS partition has yet to be defined/initialized. I use the SPIFFS.format() routine to take care of this, the first time around, but I’ve since read that the ESP32 will do this automatically if the SPIFFS.begin() function initially fails.

When this firmware runs, if SW3 is not pressed, it will enter the normal operational mode. Here, the Espalexa class is instantiated and the callback routine to handle the incoming commands from the ECHO device is defined. The loop() portion of the firmware calls the espalexa.loop() function periodically to service the incoming Echo commands, by invoking the callback routine “HP1Changed”. This, in turn, calls either SendOnIRcode() or SendOffIRcode(). Also, in the loop() section I check to see if the physical On, Off pushbuttons, located on the front panel, have been pressed. If so, I send out the proper IR codes. In the loop, I also flash LED2 which acts as a “heartbeat” to inform me that everything is working.

To transmit IR codes, I have to modulate the pulse train (stored in the buf[4000] array) with a 38 KHz carrier, before sending them out. This is done using the ESP32’s LEDC function, as follows:

ledcAttachPin(CarrierPin,0);

ledcSetup(0,38000,8); // set ledc PWM to 38 KHZ 8 bit resolution

ledcWrite(0,128);

// 50% duty cycle

CarrierPin is ESP32 GPIO4. I used a DF Robot FireBeetle ESP32-E module for this project. This board must be selected as the target board from among all the various ESP32 board options. This is necessary since I use the FireBeetle’s pin definitions in my code (which are labelled to match the old Arduino AVR pin definitions). For example, ESP32 GPIO4 is defined as D12 on the FireBeetle ESP32-E board.

Referring to Figure 2, NPN transistor T1 is biased on by R3 and D1/D2 act as the equivalent of a NOR gate. The IR pulse train comes out on ESP32 D3 and the 38 KHz carrier square wave signal comes out on D12. When both of these signals are High, T1 will be biased on, and the IR LED will light. This will provide correctly timed pulses, modulated at a 38 KHz rate.

To make it clear what is happening at any time, I included a 16×2 character LCD display. This comes in handy when running the IR “learning” routines.

Figure 3 shows the internal circuit board etc.

CUSTOM INTEGRATION

While the easiest way to handle on/off control using Alexa and a custom IoT device, is the emulation method described earlier, there are other methods. One other possibility involves using the Alexa Arduino Skill. The Arduino company runs the Arduino cloud service, which allows you to:

• Do Arduino code development on the cloud, instead of (or in addition to) running the Arduino IDE on your own PC computer. This includes, but is not limited to, writing an Arduino sketch which runs on a custom IoT device that can communicate with this cloud service.
• Define Things on this cloud server that are associated with a specific MCU and define variables (sensor or actuator values) that are defined/controlled by this MCU.
• Define Dashboards which contain text and graphic objects that display the values of the cloud variables that this MCU has sent/received.

The Arduino Cloud service is available in several tiers. You can get started with a free plan which allows you to define a maximum of 2 Things. That is enough to build 2 different custom IoT devices that can be linked to your Echo device using the Arduino Skill.

What the Alexa Arduino Skill does is link your Arduino Cloud account with your Amazon Alexa account. This allows your Echo device to access variables that are stored on the Arduino Cloud by IoT devices, powered by connectivity-enabled Arduino boards, as well as boards containing ESP32/8266 MCUs.

First, you must set up an Arduino Cloud account by navigating to the arduino.cc website and signing up for one. The limited, free tier doesn’t require giving a credit card number. Arduino will send you an email, at the email address you used during the sign-up. You must click on the confirm link in this email, to activate the account. You will be directed to a web page allowing you to add personal info to your profile or to set up 2-factor authentication (2FA), if desired.

Figure 4 shows the Arduino Cloud home page. the first thing to do is “Add a device”. You have the choice of Arduino boards, third-party devices, or manual. The manual choice allows you to pick virtually any board containing any internet enabled MCU/board. However, the manual option involves custom coding on your part, which the other 2 choices don’t.

In my case, I picked ESP32 from the third-party devices list. You must choose a board model from a drop-down list. There is a good chance your board may not be listed there, but you can just pick one that contains the same ESP32 model (original ESP32, C3, S2, S3, etc.). that your board uses.

After choosing your board, a Device ID and secret key will be auto-generated by the Arduino Cloud server and allow it to be used by the MCU running this specific IoT device. Download the provided PDF file, as this information will be needed later in your sketch. Next, a page will be displayed containing a bit of information about this MCU device. The information will indicate that it is offline, and that no history is yet available. Click on the square green icon at the top left of the screen to get back to the main menu.

Note that sometime during this procedure, the Arduino Cloud will launch a message box asking permission to add some drivers to your PC. This box can easily be hidden by other objects on your screen and is modal (persists until you respond to it with a Yes or No). When you answer Yes, there is no visual feedback as to what is happening, but the Arduino Create Agent is downloaded. This is a process that runs in the background and makes it possible to communicate with, and flash your target MCU over USB, from within your web-browser.

From the main menu, click on Things and create a Thing. There are three procedures involved here:

• Assign an associated device – pick the device you just added.
• Network – Add your WiFi access point credentials and enter the secret key generated for this specific device, using the PDF file that you just downloaded. Save these three credentials.
• Add cloud variable(s). For this example, I’m aiming to have a sensor value sent to the Arduino Cloud, regularly, and have it made available to Alexa, on a scheduled or polled basis. Note: this is not a randomly timed alarm Trigger (as described in Part 2). I chose the variable name “Humidity”, as an example, but used the “Temperature” variable type which is the closest Alexa-compatible type available. I chose the “Update” option and set the update interval at 600 seconds. Click the Add Variable button when done.

Go back to the Things object in the main menu and click on the Thing that you just defined. Check the box next to the variable that you just defined. Also, at the top of page, enter a meaningful Thing name by renaming it from the default “untitled-1”. I chose to name it Humidity1.

Again, go back to the main menu using the square green icon at the top left, select Dashboard and then “Create Dashboard”. First, give it a name at the top of page, and click the green ADD icon. Pick an appropriate option, like percentage (a good fit for humidity readings). I picked a chart. This chart will now show up on the screen, initially with some dummy data plotted. You can move the Chart around on the screen using the “movement” icon.

Click on the green button with a chain link symbol, which should be pulsating. Pick the desired Thing and this associated variable and click on the green “Link Variable” button and then the green “Done” button.

Return to the main menu and select Sketches. Click on the entry bearing the name of the Thing you just created (with today’s date appended). When selected, there will be an icon on the right, with three dots. You can rename this sketch, using this icon. There is a “Create sketch” button, but I found that by selecting the Sketch entry mentioned above, the app will sometimes jump immediately to a new page displaying a Sketch that has been auto generated for you. Besides the displayed .ino file tab, there are three more tabs:

• Readme.aDoc This is a summary of what you would do next to make this sketch available to the public.
• thingProperties.h. This contains the #include files needed to allow your sketch to connect to the Arduino cloud. It also contains entries for DEVICE_LOGIN_NAME (the Device ID that was earlier auto-generated and which you saved to a PDF file), your WiFi credentials and the auto-generated Device Key for this device (also saved to that PDF file). These are defined here as SECRETs (see the next bullet item).

This file also contains:

1. a prototype or function definition which is called when your linked variable has changed.
2. the routine initProperties() that sets the board ID, secretDeviceKey and variable properties
3. a WiFiConnectionHandler, to which is passed your WiFi credentials.

• Secret tab. Here you will see your WiFi credentials and SECRET_DEVICE_KEY listed. By placing “SECRET_” in front of a const definition in the thingProperties.h file, the corresponding string, also listed in this SECRET tab, will be substituted into that const declaration, and downloaded as part of your ESP32 firmware. That means that these credentials will be included in the ESP32 flash memory, without these credential strings having to be included in a distributed .ino file or its associated thingProperties.h file.

Getting back to the .ino file tab, you will see a simple sketch template that will connect the ESP32 to the Arduino Cloud. The thingProperties.h file contains the function:

ArduinoCloud.addProperty(humidity, READWRITE, 600 * SECONDS, onHumidityChange);

This function will connect to the Arduino cloud every 600 seconds (the update interval earlier defined for this variable). While I specified a polling interval rather than the “on-change” option, there is also a call to the prototype “void onHumidityChange()” also auto-generated in the thingProperties.h file. Basically, this synchronizes the ESP32’s humidity variable to the cloud variable of the same name. It works both ways – if you change this variable from the Dashboard (or Alexa does it, using the Arduino Skill), the local ESP32 variable will match that change. Similarly, if the ESP32 changes the variable’s value, that will be transmitted to the cloud server.

In the sketch’s loop() routine, the ArduinoCloud.update() routine must be called periodically, and you would need to add code here to read the sensor and store the humidity value in the humidity variable.

At this point you have a simple sketch ready to be compiled and uploaded to your ESP32 MCU. When you are using the Arduino Cloud, the actual compiling and device flashing is done from the cloud, by default. However, you could copy these files to your own computer, compile it using the Arduino IDE, and then flash the ESP32 device from there.

To check the program code, click the check mark icon near the top of the screen. After the animated Busy icon disappears, this sketch will have been compiled and saved to the cloud. Correct any errors as necessary. Also, change the baud rate in the Serial.begin() statement to 115,200, since that is the baud rate at which the ESP32 sends out its boot-up message, and normally you will have set this baud rate as the default in your terminal program.

Assuming you are using the Edge or Chrome web browser, from within the browser you can download this sketch to the target ESP32 via USB, by clicking on the → icon. Before doing so, go to the option box to the right and scroll through the available Com ports, to select the port to which your ESP32 board has been enumerated. Also, select the type of ESP32 board used. I see different/more board options here than I saw when initially added a Device. While the sketch is being uploaded, the Busy icon will be animated. At the bottom of the web browser window is a black status block. The status messages that you would normally see when programming an ESP32 from the Arduino IDE program on your PC, will appear here. Assuming this upload is successful, you can open a serial terminal program and monitor the serial output from the ESP32. You should see a message informing you whether the ESP32 has connected to your WiFi AP- it may take several tries before this connection succeeds. As in Figure 5, you should see the Device ID, which should match the DEVICE_LOGIN_NAME contained in the thingProperties.h tab for the sketch. At some later time, you should see the message “Connected to Arduino IoT cloud”, but this may not show up immediately.

If you leave the Arduino Cloud’s Sketch screen, and go back to the main menu, you can choose Dashboard. You should be able to select the Dashboard that you just created. My Dashboard contains a chart, and the data points will show up, in due time, as in Figure 6. These are just randomly generated values. I had no humidity sensor connected at this stage.

When discussing the DEVICE_LOGIN_NAME and Access KEY earlier, I said that these two ID/authentication strings were generated when you added a Device to the Arduino cloud. While this is true, that doesn’t mean that they are tied to any specific MCU device. When you download your sketch’s firmware to a specific MCU device, it will contain both the auto-generated ID and access KEY and will connect to the Arduino Cloud using those credentials. However, you could program another ESP32 (even a different model ESP32) with that identical firmware and it will connect to the Arduino Cloud properly. If you had an application which required many IoT devices, all doing the exact same thing, I believe that they could all co-exist, using identical firmware, and still interact with the Arduino Cloud server. However, from a practical standpoint, the values that you observed in your Arduino Dashboard would not be traceable to any IoT device that you had deployed in such a way.

Now that the Arduino IoT cloud is receiving/storing your ESP32 data, you must next link your Arduino IoT Cloud account with your Amazon Alexa account. Do so by searching for the Arduino Skill in the Amazon Alexa app. When you’ve selected this Skill, Enable it. This will re-direct you to the Arduino Cloud login page, where you’ll need to login again, using your Arduino Cloud username/password. Then you will be returned to the Amazon Alexa app. The app will do a scan looking for your shared variables on the Arduino Cloud. In my case I used the Alexa-compatible variable type “Temperature” because humidity is not a valid variable type. During the discovery process, Alexa should find and report a Thermostat (given the temperature variable type I chose) and will add it to your Alexa account (or profile as I sometimes refer to it in this article). You can then define the room it is in or a group, neither of which are mandatory.

If you now check your available devices in the Alexa app, you will see one called humidity, of type “thermostat”. It should be enabled, and you should see the most recent value sent from the ESP32. Since the variable type is “Thermostat” and I defined it as read/write, you can use the + and – icons in the Alexa app, when displaying this device, to change the value stored on the Arduino Cloud. This change will be reflected in the appropriate Dashboard. Without your having written any custom code, when Alexa changes the value of the variable “humidity”, that change will also be sent back to the ESP32 IoT device. In my code, the Serial.println() displays this variable in a serial terminal window, and I can see its value changing to match the value sent from the Alexa app.

During Alexa’s discovery process for an Arduino Cloud IoT device, the Alexa app will likely report that it has found nothing, but this isn’t correct. You will find the device listed among your existing devices when you look for it. Alexa will label this device using the same name as the shared variable that you linked with the Thing associated with this ESP32 device. You can, however, select that device in the Alexa app, click on the “Gear” icon, and rename it to something more descriptive. You could change it to “office humidity”, for example. Then say “Alexa, set the office humidity to 60” and it will respond that the thermostat has been set to 60, and update the ESP32 immediately. When I chose the humidity parameter for this example it didn’t occur to me that if I asked Alexa for the humidity, the answer reported would be the humidity in my locality, as reported by an online weather service. However, the method I’ve described for linking an Arduino Cloud-based IoT device to the Alexa service is still valid and useful.

CONCLUSION

In this initial article, I’ve described two different methods of integrating a custom designed IoT gadget with the Amazon Alexa smart home environment. The second method is definitely more work, but neither of them is too complicated. In Part 2, I’ll cover a few more methods that I tried out. One of them involves a design in which an IoT gadget can send a message to your Echo device which will trigger what is best described as an alarm. Your Echo device is not constantly polling such a device, but will react, in real-time to the alarm condition- performing some action that notifies you of the occurrence of that alarm. 

RESOURCES
Espressif | https://www.espressif.com

SOURCES
Espalexa Arduino library (github): https://github.com/Aircoookie/Espalexa
Arduino library reference document: https://www.arduino.cc/reference/en/libraries/espalexa
Alexa Skill- IFTTTrigger-Alexa-Actions by mkZense https://mkzense.com
DFRobot FireBeetle ESP32-E module (DFR0654): https://www.dfrobot.com/product-2195.html
Espressif ESP32-Wroom-02 module (datasheet): https://www.espressif.com/sites/default/files/documentation/0c-esp-wroom-02_datasheet_en.pdf

Code and Supporting Files

PUBLISHED IN CIRCUIT CELLAR MAGAZINE • NOVEMBER 2024 #412 – Get a PDF of the issue

Sponsor this Article