Protocols and Programming
Many embedded systems need to make use of synchronized time information. In this article, Jeff explores the history of time measurement and how it has led to NTP and other modern technologies for coordinating universal date and time. Using Arduino and the Espressif System’s ESP32, Jeff then goes through the steps needed to enable an embedded system to request, retrieve and display the synchronized date and time to a display.
It’s been said that ”If you are on time, you’re late,” or, “To be early is to be on time.” It’s all relative. If you go to a meeting and people are already there, you feel as if you are late. If you are the first to arrive, you wonder if you’ve got the schedule wrong, and then you check your watch or phone for the time. Time can be troublesome for us, because the present is an ever-changing instant where the past meets the future. We cruise through life when all players reference the same moment, but should we become out of sync, the ride gets bumpy.
We can imagine that in humanity’s early times the first concepts of time were cyclic periods—like day/night, seasons and life/death. Our fundamental measurement of a day directly relates to our life and history, and seems to tie all nature together. But what about those activities that occur within the confines of each day? Some way of defining the parts of a day were needed. At the time, we had one division—day/night—with most considering the start of a day to be daybreak or sunrise, and the start of night to be sunset.
Since daytime was directly related to the sun’s position, the day could be divided into two parts based on whether the sun was rising in the sky or falling back toward the horizon. Observing the sun’s shadow gave way to the first sundials, which provided a visual indication of time relative to sunrise and sunset without physical divisions. One such division of the day was religious in origin: canonical hours or periods of fixed prayer at regular intervals were defined in monastic communities. At that time, our understanding of the sky was astrological and not astronomical. The latter would eventually define the breakdown of a day into hours, minutes and seconds.
For the most part, the hour was a variable concept. Around the 14th century, 12 was chosen as a practical division of the day (and the night) into equal parts. It was the most convenient number for dividing into fractions because it’s divisible by 2, 3 and 4—thus giving us the 24-hour day we use today. Without the sun, sundials were worthless, so other means of recording the passage of time were invented, including water, candles and weights. These and early mechanical clocks of the 16th century were not accurate, because their mechanisms were essentially unregulated. It wasn’t until the next century that the pendulum gave the mechanical clock accuracy to within 1 minute a day. Today, we have access to extremely accurate clocks. Atomic clocks measure an atom’s fluctuating energy levels to produce an accuracy of ± 1 second in over a billion years.
TIME KEEPER
The International Bureau of Weights and Measures (called Bureau International des Poids et Mesures or BIPM in France) is an intergovernmental organization that was established to oversee measurement science and measurement standards. One important role for the BIPM is maintaining the accuracy of worldwide time of day. It combines, analyzes and averages the official atomic time standards of member nations around the world, to create a single, official Coordinated Universal Time (UTC). The Royal Observatory, Greenwich, England was chosen as the reference point to define the Universal day, counted from 0 hours at mean midnight, as used on the island since 1847. By 1884, the Greenwich Meridian was used for two-thirds of all charts and maps as their Prime Meridian. The world is divided into 24 time zones, each 15 degrees in width (24 hr/360 degrees). However, as shown in Figure 1, time zone boundaries are shifted to prevent a country from being needlessly split into separate zones.
— ADVERTISMENT—
—Advertise Here—
All time on earth is related to the official time in Greenwich, England by denoting a time zone offset. Current civil time can be determined by adding or subtracting the UTC offset (number of hours and minutes). This ranges from UTC−12:00 in the west to UTC+14:00 in the east. Table 1 lists those offsets that relate to the United States.
| UTC−04:00 (AST) | Puerto Rico and US Virgin Islands |
| UTC−05:00 (EST) | Atlantic coast |
| UTC−06:00 (CST) | Gulf Coast to the Great Lakes |
| UTC−07:00 (MST) | Mountain states |
| UTC−08:00 (PST) | Pacific coast states |
| UTC−09:00 (AKST) | Alaska |
| UTC−10:00 (HST) | Hawaii |
TABLE 1 – Time zone offsets are listed here for the US daylight saving times have an additional offset of 1 hour and must be accounted for locally.
The US spans seven time zones. When a time zone uses daylight saving time, the ST for Standard Time is replaced by DT indicating Daylight Saving Time. Daylight Saving Time increases the regional offset by 1:00, and was implemented to shift daylight activities during the longer summer hours. Daylight Saving Time is a local shift that must be handled locally, and as such does not affect the UTC in any way.
In my youth I recall the phone company providing a number you could to call to hear the current time. The first radio station, WWV in Colorado, morphed into National Institute of Standards and Technology (NIST), whose broadcast focused on developing frequency standards and eventually broadcasting time and frequency information on the 2.5-, 5-, 10-, 15- and 20-MHz shortwave bands. Today, the time is available almost everywhere, and that time is synchronized to the UTC, all thanks to the Internet.
NATIONAL STANDARD TIME
The Network Time Protocol (NTP) is used to synchronize our clocks via the Internet. The NTP architecture, protocol and algorithms provide a nominal accuracy of tens of milliseconds on WANs, sub-milliseconds on LANs, and sub-microseconds using a precision time source such as a cesium oscillator or GPS receiver. Reliability is assured by redundant tiered servers and diverse network paths. The “NTP pool” is a dynamic collection of networked computers that volunteer to provide highly accurate time via the NTP to clients (like us) worldwide. We can use one of the NTP pool servers to get UTC information. Although using the NTP protocol will assure the accuracies listed above, this is often unnecessary and overly complicated for those applications that are only interested in whole-second times for RTC (Real Time Clocks). SNTP, a simplified subset of the NTP protocol, generally is sufficient for our needs.
SNTP uses a UDP connection to send a datagram or packet, as opposed to a TCP connection. The basic transaction is simple. We send an SNTP data structure as a UDP packet using port 123 to the server. The time server (one of the NTP pool) then sends back an SNTP data structure as a UDP packet. That’s it! The structure of the datagram consists of four 32-bit words (4 × 32 bits = 128 bits or 16 bytes), followed by four 64-bit time stamps (4 × 64 bits = 256 b or 32 bytes) as shown in Figure 2. There can be optional data, but we won’t need it. In fact, we need only to worry about the first byte of the (16 bytes + 32 bytes = 48 bytes) datagram to make a request.
This is set according to RFC 4330:
LeapsecondInformation 2 bits = “00” disregarded
VersionNumber 3 bits = “100” 4
MODE 3 bits = “011” Client
Therefore:
— ADVERTISMENT—
—Advertise Here—
First Byte = “00100011” or 0x23
The returned datagram will be in the same format. The time stamps we sent as zeros could have been used to determine the actual propagation delay in the message trip, to calculate an accurate sub-second synchronized time. We are not concerned with that level of accuracy. However, we do want to get the time from the server, and that will be populated in the last of the four time stamps in the reply. So how does this time stamp relate to the present second, minute, hour, day, month and year? Sad to say, it does not specify any of those. This time stamp gives the number of seconds from 0:00 on 1 January 1900.
Once we have this “elapsed time,” we will need to figure out the rest. So, for now let’s just be content to get this number. To do that I’m going to use the Sparkfun’s ESP32 Thing board. This board shown in Figure 3 contains the Espressif’s ESP32 and all supporting hardware necessary to connect to your home wireless network and to be programmed using the Arduino IDE. This is the “next gen” ESP8266 and has 4 MB of external flash memory, compared to the ESP8266’s 4 Mb.
You can visit the ESP32 support for Arduino on Espressif’s github page [1]. In the Arduino IDE, begin installation by clicking on the File tab and selecting Preferences. Enter https://dl.espressif.com/dl/package_esp32_index.json into the “Additional Board Manager URLs” field and close Preferences. Then, click on the Tools tab and open Boards > Open Boards Manager, install the esp32 platform and close the manager. I connected my ESP32 Thing to a USB port, restarted the IDE so it can find the port, selected Sparkfun ESP32 thing from Tools>Boards, and the proper port from Tools>Port in preparation to begin writing an application.
Sparkfun’s ESP32 thing is just one of the ESP32 platforms that were just added to the IDE. Many times, along with a platform’s installation, is a number of example programs. We can begin with one of these to get started. As a bonus here, we’ll choose the BasicOTA (Over the Air) example from the File>Examples:>Examples for the Sparkfun ESP32 thing>ArduinoOTA>BasicOTA. You will need to modify two lines in the application:
const char* ssid = “..........”;
const char* password = “..........”;
Then, substitute your home wireless network’s SSID and password—and remove the slashes from these two REM’d statements—to enable the basic password protection:
// ArduinoOTA.setPassword(“admin”);
// ArduinoOTA.etPasswordHash
(“21232f297a57a5a743894a0e4a801fc3”);
Save the application as “FTB343,” so you won’t overwrite the example. Compile and download the program to the ESP32 Thing. Upon download success you can click on the spyglass icon to open the IDE’s serial monitor and you will see the information in Listing 1.
ets Jun 8 2016 00:22:57
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
ets Jun 8 2016 00:22:57
rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:808
load:0x40078000,len:6084
load:0x40080000,len:6696
entry 0x400802e4
Booting
Ready
IP address: 192.168.1.242
LISTING 1 – After you compile and download the program to the ESP32 Thing, you can click on the spyglass icon to open the IDE’s serial monitor and you will see the information shown here.
Note that debug information is dumped from the ESP32 at every reset. Following this dump, you’ll see the result of our print statements, Booting, Ready and IP address. This indicates the Wi-Fi connection is established at local port 192.168.1.242. If you close and reopen the Arduino IDE, you’ll find an addition in the Port>List: esp3232-[MAC] at 192.168.1.242 (Sparkfun ESP32 Thing). If you choose this Port selection you will be downloading any compiled application, using Wi-Fi instead of the USB. You can monitor this through the now unused USB port with a serial terminal program such as RealTerm. The Arduino’s IDE serial terminal can’t be used during a Wi-Fi download. You can always choose to continue to use the USB port to program, in a wired configuration. OTA is just a bonus example you may find helpful, and was a consequence of getting the UDP client installed.
SOME INITIALIZTING
With the UDP client, we can send a request to one of the servers in the NTP pool. Rather than adding another library— NTPClient, for example—I’ll add some raw code to send out a UDP datagram. But first we need to initialize a few things and add some code to keep track of the passing seconds. This is based on millis() the system variable that keeps track of the number of milliseconds that have elapsed since the application started. An alternative might be to have a hardware RTC attached, but I chose to do this all in software. Note that I’ve added some constants that define the time zones for the US.
— ADVERTISMENT—
—Advertise Here—
Since our time reference will be UTC, we need to adjust this for our specific time zone by adding a timeZoneOffset = zone × 3,600 seconds/zone (one hour of seconds for each time zone). It is suggested that you keep your NTP requests to no less than 10 requests/minute, or 6 seconds), I’ll be using 1 minute here. While epoch will become the total seconds count received from a request, myEpoch is initialized to the negative of timeZoneOffset. As a result, as the application begins, myEpoch is initialized to the negative of my EDT timeZoneOffset (in seconds) (Listing 2). With these additions—when the application is compiled, downloaded and run—we should get a display of time and date, which increments each second.
const int EEST = +3; // Eastern European Summer Time
const int EET = +2; // Eastern European Time
const int CEST = +2; // Central European Summer Time
const int CET = +1; // Central European Time
const int WES = +1; // Western European Summer Time
const int WET = +0; // Western European Time
const int GMT = +0; // Greenwich Mean Time
const int EGT = -1; // East Greenland Time
const int EDT = -4; // Eastern Daylight Time
const int EST = -5; // Eastern Standard Time
const int CDT = -5; // Central Daylight Time
const int CST = -6; // Central Standard Time
const int MDT = -6; // Mountain Daylight Time
const int MST = -7; // Mountain Standard Time
const int PDT = -7; // Mountain Daylight Time
const int PST = -8; // Mountain Standard Time
String timeZone = “Eastern Daylight Time”;
long timeZoneOffset = EDT * 3600; // Zone * sec/hr
int updateInterval = 60; //secs
unsigned long lastTime;
unsigned long presentTime;
unsigned long myEpoch=timeZoneOffset * -1;
int year = 0;
int month = 0;int day = 0;
int DOW = 0;
int hour = 0;
int minute = 0;
int second = 0;
int lastSecond = 0;
boolean newSecond = false;
void loop()
{
ArduinoOTA.handle(); // handles the OTA stuff
//
presentTime = millis(); // seconds since reboot
newSecond = false; // not a new second
if((presentTime – lastTime)>=1000)
// if seconds has advanced by at least 1000ms
{
myEpoch++;
// we have a new second so add one to the total
lastTime = lastTime + 1000;
// new advanced time to look for
newSecond = true; // we have a new second
}
if(newSecond) // if we have a new second
{
calculateTime(myEpoch+timeZoneOffset);
// calculate date and time based on the total + timeZoneOffset
displayTime(); // now display the calculated information
}
}
LISTING 2 – After making the additions shown here, when the application is compiled, downloaded and run, we should get a display of time and date, which increments each second.
You will note that while myEpoch is initialized to local zero (timeZoneOffset × -1), the actual date/time displayed is Thursday January 1, 1970 00:00:00. That is the standard Epoch that is used as the UNIX=0 reference. The number of seconds given by NTP will be referenced to this date/time. So naturally, zero seconds should display as this date/time. Now with all this background material in place, we can finally get in sync with NTP (Listing 3).
NTP initialization:
const char timeServer[] = “time.nist.gov”; // time.nist.gov NTP server
const int NTP_PACKET_SIZE = 48; // NTP time stamp is in the first 48 bytes of the message
byte packetBuffer[ NTP_PACKET_SIZE]; //buffer to hold incoming and outgoing
if(!(myEpoch % updateInterval) & newSecond) // limit how often we request an NTP packet (required)
{
sendNTPpacket(timeServer); // send an NTP packet to a time server
Serial.print(“requesting packet from “);
for(int i=0; i<sizeof(timeServer) - 1; i++)
{
Serial.write(timeServer[i]);
}
Serial.println();
}
if(Udp.parsePacket()) // NTP response packet could come at any time
{
Serial.println(“packet received”);
decodePacket(); // get information and sync it
}
LISTING 3 – Code for synchronizing with NTP
Remember, it is suggested that you do not make requests to an NTP server more often than every 6 seconds. Faster requests are considered flooding, and you may get rejected from further services. In this application, I set my updateInterval to once an hour. In reality, your system clock should be able to keep pretty good time without having to sync for many hours, or even just once a day. Please note here I would rather use an interrupt to ensure I don’t miss updating each second. However, if you don’t have any code that prevents loop() from executing at least once a second, all will be fine. The request to NTP is simple, especially when UDP has already been installed. Listing 4 is the support routine to do the request.
unsigned long sendNTPpacket(const char * address)
{
// set all bytes in the buffer to 0
memset(packetBuffer, 0, NTP_PACKET_SIZE);
// fill packet with ‘zero’ data
// Initialize values needed to form NTP request
// (see RFC 4330 for details on the packets)
packetBuffer[0] = 0b00100011; // LI, Version, Mode
packetBuffer[1] = 0; // Stratum, or type of clock
packetBuffer[2] = 0; // Polling Interval
packetBuffer[3] = 0; // Peer Clock Precision
Udp.beginPacket(address, 123);
// NTP requests are to port 123
Udp.write(packetBuffer, NTP_PACKET_SIZE);
Udp.endPacket();
}
LISTING 4 – The request to NTP is simple, especially when UDP has already been installed. Shown here is the support routine to do the request.
Note that once our packet has been filled with zero data, we only need to change the first byte in the packet before sending it. Since UDP communication has no handshaking, a response isn’t guaranteed. However, we can test for a response, then get data once it comes in. If a packet is received, we can find what we are looking for in packet bytes 40:43. Listing 5 is the support routine for decoding packets.
void decodePacket()
{
// read the data from the packet
Udp.read(packetBuffer, NTP_PACKET_SIZE); // read the packet into the buffer
//the timestamp starts at byte 40 of the received packet and is four bytes, // or two words, long. First, esxtract the two words:
unsigned long highWord = word(packetBuffer[40], packetBuffer[41]);
unsigned long lowWord = word(packetBuffer[42], packetBuffer[43]);
// combine the four bytes (two words) into a long integer
// this is NTP time (seconds since Jan 1 1900):
unsigned long secsSince1900 = highWord << 16 | lowWord;
Serial.print(“Seconds since Jan 1 1900 = “);
Serial.println(secsSince1900);
// now convert NTP time into everyday time:
Serial.print(“Unix time = “);
// Unix time starts on Jan 1 1970. In seconds, that’s 2208988800:
const unsigned long seventyYears = 2208988800UL;
// subtract seventy years:
unsigned long epoch = secsSince1900 - seventyYears;
// print Unix time:
Serial.println(epoch);
// print the hour, minute and second:
Serial.print(“The UTC time is “);
// UTC is the time at Greenwich Meridian (GMT)
calculateTime(epoch);
displayTime();
Serial.println(“My Time Zone is “ + String(timeZone));
Serial.println(“It’s time offset is “ + String(timeZoneOffset) + “ seconds”);
Serial.print(“The EDT time is “); // UTC is the time at Greenwich Meridian (GMT)
calculateTime(epoch + timeZoneOffset);
myEpoch = epoch; // update myEpoch so I can use the system clock until I sync again
}
LISTING 5 – Shown here is the support routine for decoding packets.
The four packet bytes of interest are the first four of the 8-byte Transmit Timestamp shown in Figure 2. The first four are seconds, and the last four are fractions of a second. We don’t need the sub-second accuracy of the last four bytes. The first four bytes show seconds, up to 4,294,967,295 seconds. Based on the Epoch of 1970, rollover will happen in 2038. Y2K anyone? While proposals have been made, presently there is no accepted solution. As in many proposals, backwards compatibility is the issue.
EPOCH TO EPOCH
When this application is first run, myEpoch begins from zero. All we need to do now is make myEpoch = “Epoch” (as received from NTP) and then the Date and Time will be displayed based on this synchronized epoch (Figure 4). That’s it. But let’s add one more item. To see the date and time, we are connected via USB, and it is displayed on our PC’s serial terminal application. so, let’s add a small display like the 0.96” 128×64 bit I2C OLED graphics display (<$10). This has support for 16×8 characters using the U8x8lib.h library connected via I2C. This allows the application to be stand-alone.
A four-wire cable—including power, ground, clock and data (Arduino dedicated I2C pins)—is all you need to add to the microcontroller. While 8×8 character size is approximately 0.05”, I like this display’s small size and the fact that it will operate at 3.3 V or 5 V. Because of the 16-character horizontal limit, I need to break the date into two pieces, one for the month and one for the day and year. This all fits nicely displaying on the even numbered lines. To make it even more pleasing, space characters are added to center the displayed data. Listing 6 is the code I added to use this display.
Initialization:#include <U8x8lib.h>
U8X8_SSD1306_128X64_NONAME_HW_I2C u8x8(/* reset=*/ U8X8_PIN_NONE);
Setup(): u8x8.begin();
u8x8.setPowerSave(0);
u8x8.setFont(u8x8_font_chroma48medium8_r);
Support functions in displayTime():
printString(0, 0, DOW[dayofWeek]);
printString(0, 2, MOY[month.toInt()]);
printString(0, 4, day + “ “ + year);
if(hour.toInt()>12)
{
//pm
hour=String(hour.toInt()-12);
printString(0,6,hour + “:” + minute + “:” + second + “ PM”);
}
else
{
//am
if(hour.toInt()==00) hour=”12”;
printString(0, 6, hour + “:” + minute + “:” + second + “ AM”);
}
DisplayOLED: void printString(int col, int row, String myString)
{
int pad = ((128/8) - myString.length())/2;
for(int z=0; z<pad; z++)
{
u8x8.drawGlyph(col+z,row,32);
}
for(int z=pad; z<myString.length()+pad; z++)
{
u8x8.drawGlyph(col+z,row,myString.charAt(z-pad));
}
pad=z;
for(int z=pad; z<16; z++)
{
u8x8.drawGlyph(col+z,row,32);
}
}
LISTING 6 – Shown here is the code I add to use the I2C OLED graphics display.
You can see the result in Figure 5. While the library offers a number of OLED functions, I used just one function for simplicity, drawing a glyph (character) at a column, row position. The padding at front and back of the data string clears out any old characters.
IT’S TIME
It’s probably clear that most of the effort required for this application is in preliminary work and not in the actual requesting, retrieving and even displaying the synchronized date and time to an OLED display. While I have shown the important tasks as user-written functions, there are libraries already written to handle most functions. Unless you dig into the library code, however, you don’t learn much, and it all seems like black magic. Once you understand how the tasks work, using the libraries can save you coding time.
I mentioned earlier one reason for choosing the Sparkfun ESP32 Thing was its inclusion of a Li-ion battery connector and the ability to recharge the battery via a USB connection. Since you can find small solar-cell 10 W USB chargers online for less than $20, you can create a totally solar-powered Date and Time display. You will need to add some code for choosing an open Wi-Fi connection, or search for one. But your application will no doubt want to do more with the date and time than just display it. So, expand on what’s offered here and let me know how you use this new-found tool!
Additional materials from the author are available at:
www.circuitcellar.com/article-materials
Reference [1] as marked in the article can be found there.
RESOURCES
Espressif Systems | www.espressif.com
SparkFun | www.sparkfun.com
PUBLISHED IN CIRCUIT CELLAR MAGAZINE • February 2019 #343 – Get a PDF of the issue
Sponsor this Article
Jeff Bachiochi (pronounced BAH-key-AH-key) has been writing for Circuit Cellar since 1988. His background includes product design and manufacturing. You can reach him at: jeff.bachiochi@imaginethatnow.com or at: www.imaginethatnow.com.
