Building an All-in-One Serial Terminal

With a Tiny Touchscreen

Many embedded systems require as least some sort of human interface. In this project article, Jeff explains his choice of a touchscreen that met his needs. He makes use of the display’s Espressif Systems ESP8266 processor and Arduino IDE support to turn the display module into a serial terminal with a serial TTL connection to other equipment.

Recently I was researching alternatives to mechanical keypads when I came across 4D Systems. 4D is an Australian company that specializes in intelligent graphics solutions integrating graphics processors with a host of display modules. Its display screens are sized from 0.9″ (yup, that’s not a typo!) up to 7″—many available with a resistive/capacitive touchscreen. The device I chose for this project was the Gen4-IOD-24T. At $29, this 2.4″ touchscreen display is supported by an Espressif Systems ESP8266 with
Wi-Fi capability, instead of the normal graphics processor used in other models. Its current draw measures just under 200 mA at 5 V with the back lighting enabled. While I chose this for its Wi-Fi capability, I am not implementing a Wi-Fi interface for this project. I will be programming the processor using the Arduino IDE. Before I get into the project, let’s take a quick tour of this product.

Figure 1 shows a PCB mounted to the back of the LCD screen. The two are contained within a plastic frame that provides convenient tabs for easy mounting. The LCD has a diagonal display dimension of 2.4″ with a resistive touchscreen applied. The micro SD card is used to hold various support files you may use in your application. These might be static text strings, fonts, images, animations/video files or general purpose storage for data logging applications. Below the SD card are the two processors, the touch controller and the ESP8266.

Those of you familiar with the ESP8266 know it’s usually accompanied by a serial EEPROM (for program storage). This module has an on-board antenna. It also has a U.FL connector in case you want to use an external antenna for extended range. Connection to the device is through a 10-pin 0.5-mm pitch FPC cable. A small dongle at the opposite end of this cable provides a USB interface used in programming the module (or TTL serial). Access to power, ground, reset, TX and RX are all available on this dongle.

4D Systems gives you its basic version of Workshop4, a Microsoft Windows application that provides an integrated software development platform for the entire 4D family of processors and modules. This IDE combines an Editor, Compiler, Linker and Downloader to aid in the development of 4DGL application code. All user application code can be developed within the Workshop4 IDE or, in this case, you can use the Arduino IDE and the GFX4d Library. Workshop has some great tools for designing your application, which include Buttons, Switches, LEDs, Meters, Knobs and drawing primitives. To really customize the look, the Pro upgrade ($70) will unlock the Smart Widget Editor, a comprehensive tool allowing you to create and animate Gauges, Sliders, Knobs and more.

ARDUINO IDE
While I could have used Workshop4 to write this entire application in 4DGL code, I wanted to show how this could be accomplished using just the Arduino IDE and the GFX4d Library. This library was developed by 4D Systems for its legacy line of Arduino products. These contained a graphics processor with a serial interface and an Arduino piggy-back interface board. The newer Gen4 modules are totally surface mount technology (SMT). The IoT modules, implemented with the ESP8266, can become a single board solution.

This project will turn the Gen4-IOD-24T module into a Serial terminal with a serial TTL connection to other equipment. Almost every project requires some kind of display interface, and most of my projects use serial I/O to collect or display data. I’ll split the LCD into an upper display area and a lower keyboard area that can be used to enter data. When designing a project, I like to use a 6-pin connector that I can hang a USB dongle on. That’s to get serial input in and out using my PC. This project will connect to that same interface and make the PC unnecessary. This connector matches the FTDI232 dongle—GND, CTS, VCC, TXD, RXD and RTS. We don’t need the handshaking lines for this project, so two of the connections aren’t necessary. The only thing you need to know when wiring your connector is whether you are connecting to Data Terminal Equipment (DTE) or Data Circuit-terminating Equipment (DCE).

For those of you who are unfamiliar with RS- 232 serial, it used male connectors (DB9M or DB25M) to indicate a DTE device (like a PC) and female connectors to indicate a DCE device (for example a modem). The cable for interconnecting the two was a straight connection between a male and female connector wired pin to pin (1-1, 2-2 and so on). The sex of the connector defined the direction of the signals flowing into or out of any particular pin. This ensures that the TX of the DTE device connects to the RX of a DCE device. Schematically, most devices will have their serial output lines labeled as the signal name TX. It’s good engineering practice to wire signal name TX to a TTL connector, based on its function—DTE or DCE. The connectors’ pins (not the signal names) should always be labeled from a DTE point of view.

If your TX signal is going to the pin labeled TX, you know the connector is for DTE. If your TX signal is going to the pin labeled RX, you know the connector is for DCE. When you make a DTE to DCE connection, the wiring is straight through (1-1, 2-2 and so on). When you wire together two DTE devices, you will need to swap RX and TX (and other handshaking lines if used). With most of my projects being DCE, connecting an FTDI232 (DTE) is a straight-through cable. This project is a DTE, so the connection is a straight-through cable. However, when connecting this project to a FTDI232 dongle, since that is DTE to DTE, the TX and RX would need to be swapped.

Initially, all that is moot. That’s because we only need to plug in a USB cable between the LCD module and a PC. The dongle on the LCD’s flat ribbon cable is a TTL to USB dongle, and gets installed as a serial port. So, note the COM port number because you’ll need that to connect the module with the Arduino IDE. If you need to install the GFX4d Library, you can find installation directions at
github.com/4dsystems/GFX4d. If you haven’t added the core board support for the ESP8266, installation instructions are included on the same page. With these installed into the Arduino IDE, you can begin an application by selecting 4D Systems gen4 IoD Range, from the Tools > Board menu, along with the appropriate COM port.

With the Gen4-IOD-24T module’s USB port connected to your PC, you can begin programming the application. This project will use the serial port (USB) as a user interface and divide the LCD screen into an upper portion (to display characters coming into the user interface) and a lower portion (to display a keyboard), allowing the user to send characters out to the user interface.

The first thing to understand is the LCD just displays graphics. The resistive touch matrix is applied to the surface of the LCD and provides the input. The coordinates of the touch matrix and the position of the graphics must be coordinated— in other words, the graphics and touch areas must coincide. This is handled by the ESP8266 and the touch controller, because there is no graphics coprocessor (as such) on this module. 4D Systems’ early Arduino support for these LCD/touch modules—along with Arduino ESP8266 support—are instrumental to this module. The low bill of materials (BOM) costs allow for a very inexpensive product. So, let’s begin with the most difficult function—displaying a keyboard on the LCD.

KEY TAPPING
We must begin the Arduino application by including the two libraries necessary to support this module, ESP8266 and graphics support.

#include “ESP8266WiFi.h”
#include “GFX4d.h”
GFX4d gfx = GFX4d();

The keyboard display I want to create consists of many individual buttons. The button function in the graphics library is important to this project, so let’s look at its requirements.

drawButton(state, x, y, w, h, colorb, btext, tfont, wm, hm, tcolor);

where:

state = the state of the button (boolean)
x = the upper left column position (integer)
y = the upper left row position (integer)
w = the width in pixels (integer)
h = the height in pixels (integer)
colorb = background color of the button (byte)
btext = the label for the button (String)
tfont = font style (array[byte])
wm = font height multiplier (byte)
hm= font width multiplier (byte)
tcolor = font color (integer)

As you can see, there is a lot going on here. The button can be drawn in one of two states—up or down. In our case no buttons latch, so the momentary down state is only shown when your finger or a stylus is pressing on the graphic (button). The next four values place and size the button. Also note wm and hm are used to increase the size of the button; multiply the h and w by a factor greater than 1 to magnify the button’s size. The remaining values choose colors, a text label and font style. To simplify things, the layout will have a standard-size button in a matrix of 10 by 4 rows. Some larger buttons use multiple matrix locations (such as the space key).

There will be two different layouts. However, the first layout is used twice—once for upper case and once for lower case. We can set up an array to hold information about each button. The first layout has 33 buttons, and the second layout has 34. For each button, we supply 6 bytes of data: the button (ASCII value), column and row matrix position, button width and height and font size multiplier. The two layout arrays are defined as shown in Listing 1.

const byte k1p[] = {
81, 0, 0, 24, 24, 2, // Q/q
87, 24, 0, 24, 24, 2, // W/w
69, 48, 0, 24, 24, 2, // E/e
82, 72, 0, 24, 24, 2, // R/r
84, 96, 0, 24, 24, 2, // T/t
89, 120, 0, 24, 24, 2, // Y/y
85, 144, 0, 24, 24, 2, // U/u
73, 168, 0, 24, 24, 2, // I/i
79, 192, 0, 24, 24, 2, // O/o
80, 216, 0, 24, 24, 2, // P/p
65, 0, 24, 24, 24, 2, // A/a
83, 24, 24, 24, 24, 2, // S/s
68, 48, 24, 24, 24, 2, // D/d
70, 72, 24, 24, 24, 2, // F/f
71, 96, 24, 24, 24, 2, // G/g
72, 120, 24, 24, 24, 2, // H/h
74, 144, 24, 24, 24, 2, // J/j
75, 168, 24, 24, 24, 2, // K/k
76, 192, 24, 24, 24, 2, // L/l
8, 216, 24, 24, 24, 1, // BS/BS (back space)
90, 0, 48, 24, 24, 2, // Z/z
88, 24, 48, 24, 24, 2, // X/x
67, 48, 48, 24, 24, 2, // C/c
86, 72, 48, 24, 24, 2, // V/v
66, 96, 48, 24, 24, 2, // B/b
78, 120, 48, 24, 24, 2, // N/n
77, 144, 48, 24, 24, 2, // M/m
13, 168, 48, 72, 24, 1, // Enter/Enter
1, 0, 72, 48, 24, 1, // abc/ABC
44, 48, 72, 24, 24, 2, // ,/,
32, 72, 72, 96, 24, 1, // Space/Space
46, 168, 72, 24, 24, 2, // ./.
3, 192, 72, 48, 24, 1 // sym/sym
};
const byte k2p[] =
{
49, 0, 0, 24, 24, 2, // 1
50, 24, 0, 24, 24, 2, // 2
51, 48, 0, 24, 24, 2, // 3
52, 72, 0, 24, 24, 2, // 4
53, 96, 0, 24, 24, 2, // 5
54, 120, 0, 24, 24, 2, // 6
55, 144, 0, 24, 24, 2, // 7
56, 168, 0, 24, 24, 2, // 8
57, 192, 0, 24, 24, 2, // 9
48, 216, 0, 24, 24, 2, // 0
33, 0, 24, 24, 24, 2, // !
34, 24, 24, 24, 24, 2, // “
35, 48, 24, 24, 24, 2, // #
36, 72, 24, 24, 24, 2, // $
37, 96, 24, 24, 24, 2, // &
38, 120, 24, 24, 24, 2, // `
39, 144, 24, 24, 24, 2, // (
40, 168, 24, 24, 24, 2, // )
41, 192, 24, 24, 24, 2, // BS (backspace)
8, 216, 24, 24, 24, 1, // *
42, 0, 48, 24, 24, 2, // +
43, 24, 48, 24, 24, 2, // -
45, 48, 48, 24, 24, 2, // /
47, 72, 48, 24, 24, 2, // :
58, 96, 48, 24, 24, 2, // ;
59, 120, 48, 24, 24, 2, // +
61, 144, 48, 24, 24, 2, // =
13, 168, 48, 72, 24, 1, // Enter
2, 0, 72, 48, 24, 1, // ABC
60, 48, 72, 24, 24, 2, // <
32, 72, 72, 96, 24, 1, // Space
62, 168, 72, 24, 24, 2, // >
95, 192, 72, 24, 24, 1, // _
64, 216, 72, 24, 24, 2 // @
};

LISTING 1 – For each button, we supply 6 bytes of data: the button (ASCII value), column and row matrix position, button width and height, and font size multiplier. This listing defines the two layout arrays.

We need to establish several variables in support of the keyboard. These are global variables before the setup() function of the Arduino application:

byte drawnbut[128]; // maximum number of buttons we could
have in the Keyboard
int but; // present key
int buttw; // key returned as pressed
int lastbut; // last key pressed
boolean skipchk; // is this a special function key
uint16_t bcolor; // background color
int modedelay; // counter for special function delays

The setup() function performs all the initialization. Here we start the serial port at 115,200 baud. Next, we set up the graphics, by clearing the LCD, turning on the backlight, setting the screen orientation to landscape mode and establishing the font and text size and button color. The last line in the setup() function will provide a display of the keyboard layout with the function call keypad1landscape(112, bcolor, BLACK). This function is for layout 1 and establishes the upper left corner of the keyboard, its button color with a BLACK background.

void setup()
{
Serial.begin(115200);
gfx.begin();
gfx.Cls();
gfx.BacklightOn(true);
gfx.Orientation(LANDSCAPE);
gfx.Font(2); gfx.TextSize(1);
bcolor = LIGHTGREY;
delay(100);
keypad1landscape(112, bcolor, BLACK);
}

Let’s take a look at what happened to accomplish this. This function calls two additional functions, delbuttons(bcolor) and kpabc(1, 2, ypos, butcol, tcol). The first checks the whole drawnbut[n] array for those buttons which are drawn (drawnbut[n]=1) and if they are flagged as drawn (from some previous request) it erases the graphic via the gfx.DeleteButton(n, bcolor) graphic function. Then it marks the button as not drawn (drawnbut[n]=0).

void keypad1landscape(int ypos ,uint16_t butcol, uint16_t tcol)
{
delbuttons(bcolor);
kpabc(1, 2, ypos, butcol, tcol);
}

Next, the alphabet matrix of buttons is drawn using the kpabc(1, 2, ypos, butcol, tcol) function. Note this function is passed the upper case or lower case request through its first variable =1 (lower case), along with the button size (2 times), yposition (row), button color (LIGHTGRAY), text color (BLACK). In this function, which is similar to the previous one, we will need to draw each key using the graphics function gfx.Buttonx(p[0], p[1], p[2] + ypos, p[3], p[4], butcol, bt, p[5], tcol), and mark it as drawn via drawnbut[p[0]] = 1. After a few local variables get defined, we want to fill array p[6] with the values needed for the gfx.Buttons() function, once for each button in array k1p[apos].

Do you remember the array that defined each of 33 buttons with 6 parameters p[0] = ASCII value, p[1] = matrix column, p[2] = marix row, p[3] = button width, p[4] = button height, and p[5] = font size multiplier? We need to adjust some of these parameters based on the five parameters passed in the kpabc() function.

When ulcase = 2 (lower case), 32 is added to each alpha ASCII character of p[0], so it will be displayed as lower case. When bsize=2 (2 times), the values of x, y, width, height are adjusted, so the text is located correctly. Finally, btext (button text) needs to be updated with the appropriate ASCII character or special function such as BS, Enter, space, symbol, upper- and lowercase alpha.

The code shown in Listing 2 will get the initial keyboard displayed on the LCD. Now we finish off the application with the loop() function. At this time we only need to loop on two functions, checkForKey() presses and yield() for the display communication to be updated.

void kpabc(byte ulcase, byte bsize, int ypos, uint16_t butcol, uint16_t tcol)
{
String bt;
int apos;
int convp;
int p[6];
for(int n = 0; n < 33; n ++)
{
for(int o = 0; o < 6; o ++)
{
apos = (n * 6) + o;
p[o] = k1p[apos];
}
if(bsize == 2)
{
p[1] = (p[1] / 3) * 4;
p[2] = (p[2] / 3) * 4;
p[3] = (p[3] / 3) * 4;
p[4] = (p[4] / 3) * 4;
}
if(p[0] > 64 && p[0] < 91 && ulcase == 2)
{
p[0] = p[0] + 32;
}
bt = char(p[0]);
if(p[0] == 13)
{
bt = “Enter”;
}
if(p[0] == 8)
{
bt = “BS”;
}
if(p[0] == 32)
{
bt = “Space”;
}
if(p[0] == 3)
{
bt = “sym”;
}
if(p[0] == 1 && ulcase == 2)
{
p[0] = 2;
}
if(p[0] == 2)
{
bt = “ABC”;
}
if(p[0] == 1)
{
bt = “abc”;
}
drawnbut[p[0]] = 1;
gfx.Buttonx(p[0], p[1], p[2] + ypos, p[3], p[4],
butcol, bt, p[5], tcol);
}
}

LISTING 2 – The code gets the initial keyboard displayed on the LCD.

void loop()
{
checkForKey();
yield();
}

Key presses are returned via the graphics function gfx.CheckButtons() (Listing 3). Note: the resistive touch areas are mapped for each key by the graphic library.

void checkForKey()
{
buttw = gfx.CheckButtons();
if(buttw != lastbut && buttw > 0)
{
but = buttw;
}
if(modedelay > 0)
{
modedelay = modedelay -1;
}
skipchk = false;
if(modedelay > 0 && but == 64)
{
skipchk = true;
}
if(modedelay > 0 && but == 95)
{
skipchk = true;
}
if(modedelay > 0 && but == 1)
{
skipchk = true;
}
if(modedelay > 0 && but == 2)
{
skipchk = true;
}
if(skipchk == false)
{
if(but > 31 && but < 127)
{
//gfx.TWwrite(but);
Serial.write(but);
}
if(but == 13)
{
//gfx.TWwrite(but);
Serial.write(but);
gfx.ButtonUp(13);
}
if(but == 8 || but == 127)
{
//gfx.TWwrite(8);
Serial.write(8);
}
if(but == 1)
{
keypad2landscape(112, bcolor, BLACK);
but = 2;
skipchk = true;
modedelay = 30;
}
if(but == 2 && skipchk == false)
{
keypad1landscape(112, bcolor, BLACK);
but = 1;
skipchk = true;
modedelay = 30;
}
if(but == 3 && skipchk == false)
{
keypad3landscape(112, bcolor, BLACK);
modedelay = 30;
}
}
lastbut = buttw;
}

LISTING 3 – Key presses are returned via the graphics function gfx.CheckButtons(). Note: the resistive touch areas are mapped for each key by the graphic library.

Once we have a new key, we must decide what to do about the key press. For keys that are for a special function—like the ABC, abc or sym keys—we will need to erase the keyboard and redraw a new one. For most keys we need to send the ASCII character out the serial port. And that is it—unless you want to display characters coming in through the serial port.

VIEWING TEXT
Receiving characters is a much easier process. The graphics function gfx.TextWindow() handles everything we need for a scrolling display of input text. We need to add the support variables for this:

String signOn = “Serial Terminal Gen4-IoT-24T”;
// The sign on message
byte inChar; // character from the UART
uint16_t twcol; // text color
String inputString; // string of text built from the input
characters

In the setup() function, options are set that define aspects of the text window before it is initialized. Finally, the text window can be drawn with the upper left corner at column=0, row=0, with a width=320 pixels and a height=112 pixels. The text displayed in the window will be twcol=PALEGREEN with BLACK as the text window background and LIGHTGRAY as the window’s border.

A Sign-On message will then be displayed at the top of the text window. With a window height of 112, the window can hold six lines of text characters, because the text font is 5×8 pixels and the font magnification is 2. If you remove the upper and lower borders from the window height, you now have an inside height of approximately 100 pixels, and since each character is 16 pixels high, 100/16 = 6 rows of characters.

gfx.ScrollEnable(true); // enable the scrolling function
gfx.SmoothScrollSpeed(0); // slow down scrolling
twcol = PALEGREEN; // text color
gfx.Font(2); // set the font used
gfx.TextSize(1); // set the text magnification
gfx.TextWindow(0, 0, 320, 112, twcol, BLACK, LIGHTGREY);
// draw the text window
gfx.TWcursorOn(false); // set cursor off
inputString.reserve(100); // set a bit of memory for strings
displayStringTW(); // display string in text window

Within the loop() function we add a check for serial characters.

checkForSerialIn(); // check for UART RX

Each character is displayed as received. The function displayTW() uses the graphic function gfx.TWwrite(inChar) to display the character. This is different from the displayStringTW() function, which uses the graphic function gfx.TWprintln(inputString). Alternately, we could have saved the string until a CR was received, and then used the graphic function gfx.TWprintln(inputString) to display it, but I want to show characters as they are received.

void checkForSerialIn()
{
inChar = 0;
if(Serial.available() > 0)
{
inChar = Serial.read();
displayTW();
}
}

void displayStringTW()
{
gfx.TWprintln(inputString);
}

void displayTW()
{
gfx.TWwrite(inChar);
}

Sometimes you want to print text on the display at a specific location. Presently, the graphic routines don’t allow for this. I wrote some code to fake out the display. In other words, it gathers positional text and substitutes data within six strings of data, which would then be printed to the display all at once. This won’t be necessary with the next release of the GFX4d graphics Library. It will contain text positioning functions.

One other note if you are using this library: a couple of lines were inadvertently left out of the present GFX4d.cpp file in the library. You can fix this by adding a couple of lines to reinitialize the x and y positions. Without these, your program will crash consistently if you use this function—which I found out the hard way!

void GFX4d::TWcls()
{
RectangleFilled(txtx-3,txty-3,(txtx-3)+(txtw+2)-1,
(txty-3)+(txth+2)-1,txtb);
twcurx = txtx;
twcury = txty;
twxpos = 0; // added 11 19 2018
twypos = 0; // added 11 19 2018
for(int n = 0; n < sizeof(txtbuf); n ++){
txtbuf[n] = 0;
}
}

DISPLAY DUTY
I can use my finger to tap keys on this small 2.4″ display, however it is much easier to use a stylus. If you find you want to use the total area of the display, you can easily modify this and get rid of the keyboard entirely. Figure 2 shows Gen4-IOD-24T module being used to display the results from a previous project. I added a one-shot routine to turn off the backlight 5 seconds after the last key is pressed. It also keeps track of the backlight state. So, when it sees a touch with the backlight off, it disregards that character, but turns on the backlight.

Although I didn’t show how the 4D System’s Workshop IDE could be used for developing projects, you may wish to investigate this free application for its complete line of LCD modules. A true graphic controller can add functions just not attainable with the ESP8266. Each can operate in a stand-alone mode for some simple applications, or be connected to an external system to handle all the display needs.

I look forward to using this display for some Wi-Fi projects in the future. My weather station, for example, could use a local display. This module has the dimensions that fall within the size of a single gang electrical box, which means it doesn’t have to stick out like a sore thumb when mounted on the wall. Hmm… I guess I could replace many of my mechanical light switches and control all my Wi-Fi outlets and lights from each module. Too much to do, so little time, and never a dull moment.

For detailed article references and additional resources go to:
www.circuitcellar.com/article-materials

RESOURCES
4D Systems | www.4dsystems.com.au
Espressif Systems | www.espressif.com

PUBLISHED IN CIRCUIT CELLAR MAGAZINE • APRIL 2019 #345 – Get a PDF of the issue

Sponsor this Article