Low-Cost Machine Vision Is Yours for the Hacking
Join Dr. Torq and explore several readily available machine vision sensors that can give your projects “sight” right now. He discusses practical features of the Pixy, JeVois and Huskylens2, then share how he’s used them in his Steampunk Eye Ball and Hedley, his robot skull. Highlights of the hardware, software, firmware and workflows will get you started down your own DIY AI-at-the-edge machine vision road.
Wouldn’t it be cool if your robot could actually “see” stuff? Maybe you’d want to unlock a door, using a series of secret hand signals. Perhaps you’d like your robot arm to recognize and pick up different parts.
Those are but a few examples of where machine vision (MV) might come in handy.
What’s even more exciting is that the technology is out there right now at very affordable prices. We’re talking sub-$100 territory! Any serious high-schooler, garage inventor, or everyday engineer can order a device online, hook it up to their laptop, and dive right into the software, firmware, and hardware. Then, they can put it in a project and do super-interesting things.
The key is dedicating time to learning and understanding the technology and its application. Information is freely available: Let your imagination and curiosity run wild. We’ll look at the Pixy, JeVois, and Huskylens2 shown in Figure 1 so you can jump-start your own machine vision journey.
Tthe Pixy, JeVois, and Huskylens2 sensors are all small, embedded vision options.
Machine Vision, Right Now
Early DIY AI and MV have come a long way. Years ago, they required a stout desktop, a webcam, a strong graphics processing unit (GPU), and fairly simple applications. Complex inference engines and machine learning (ML) weren’t yet widely understood or available.
Inference engines and ML living in the sensor firmware have become increasingly sophisticated. The typical workflow for one of these sensors starts with the camera grabbing a video frame, then running that image through its AI algorithms (the inference engine) to analyze what it “sees.”
The algorithm might output an object’s ID, object X-Y coordinates in the frame, the object’s identification name based on installed models, the statistical probability of correct identification, and so on. This process can happen at a few to hundreds of frames per second. Output data is typically streamed to another processor, such as an Arduino, an ESP32, a Raspberry Pi-style single-board computer (SBC), or other edge devices, for further processing. The sensors support numerous programming options, lots of canned examples, and in one case, a built-in color video touchscreen for viewing results.
Let’s look at the devices, their features, capabilities and how you can get them. I’ve referenced my Steampunk Eye Ball, shown in Figure 2, and Hedley the Robotic Skull as examples. I’ll also share a few insights on what you need to get started. I’ve used the Pixy and Jevois products in my projects. The Huskylens2 recently hit the scene and I’m in the process of coming up to speed on that one.
The Steampunk Eye Ball. The pan and tilt servos were long since salvaged for other duties.
Introducing The Pixy
Pixy, shown in Figure 3, was developed at Carnegie Mellon University in the mid 2010’s through Charmed Labs. It can easily track a hundred objects, based on colors, at 50 frames per second (fps) and shoot its output to several standard hardware interfaces. These include serial, SPI, I2C, USB, video, and digital/analog output. The back side of the board, shown in Figure 4, has a mini USBi socket, a two-pin power socket, and several other interface connectors. The front mounts the miniature camera module, the lens, and the large custom GPU chip that runs its vision algorithms. Two mounting holes are on the bottom edge, and a white mode/learn button is up on top.
Front view of the Pixy board.
Back view of the Pixy board.
I put the Pixy into my Steampunk Eye Ball, shown in Figure 2, along with a pan-tilt mechanism to have it track me as I walked around in front of audiences. I typically wore a bright blue shirt and it worked fairly well.
Mounting the Pixy inside the eyeball was a hack. I soldered brass flat bar stock and tubing to form the frame and brackets, as shown in Figure 5. This was long before 3D printing showed up on my desktop. I used a miniature servo inside the ball, behind the Pixy, to tilt the camera (and eyeball) up and down. A drive shaft connected the internal servo to a bracket on the frame via the horizontal ball-tubing bearing.
The Pixy wedged into the eyeball.
The servos were plugged right into the onboard servo electronics. I used the PixyMon application to set a few parameters, then stood in front of the Pixy camera and pushed the white button to let the Pixy “learn” the blue color of my shirt. After that, the eyeball would follow me left and right, up and down.
Since the Pixy has a built-in servo controller, I didn’t stream any location data out to an Arduino or Raspberry Pi. Consequently, no extra programming was needed, to make the Steampunk Eye Ball operational. With no additional software required, it’s pretty easy to get a basic object project up and running, right out of the box.
Pixy Tips
PixyMon: You’ll need to get the companion PixyMon application running on your laptop. There are versions for Windows, MacOS, and Linux. The program allows you to configure the Pixy, set parameters, and open a window on the screen showing what the Pixy sees. Once configured, the Pixy will run without PixyMon.
Lighting: Performance was very dependent on lighting. Low lighting frequently made color identification unreliable. Normal daylight or a well-lit scene gave the best results.
Board Size and Orientation: The board measures 2-1/8” wide x 2” tall, with the camera at the top pointing away from the board. While small, it was a bit of a tight fit in the brass lightning arrester ball I was using. Putting the Pixy on the front of a small robot would be pretty easy using its two mounting holes.
At around $70, the Pixy is a great starter sensor. The next device, the JeVois, is literally a research project on your desk.
Deep Into AI R&D With The JeVois
The JeVois machine vision sensor came out a few years after the Pixy. It’s the brainchild of machine vision expert Dr. Laurent Itti at USC, and is massively different than the Pixy.
The tiny 1” × 1” × 3/4” blue brick shown in front view in Figure 6 and rear view in Figure 7 runs a highly integrated Linux-based inference engine mated to a tiny color camera module. It can recognize about 1000 common objects, detect humans, distinguish colors, and is very customizable through Python and various ML algorithms. A small sample of the algorithms the JeVois can use is listed in Table 1.
Front view of the JeVois sensor.
Back view of the JeVois sensor.
The JeVois website provides machine vision modules that recognizes and track objects..
I put a JeVois sensor in the right eye socket of Hedley, my Steampunk robotic skull as shown in Figure 8. As with the earlier eyeball, I wanted Hedley to track me via his pan-and-tilt mechanism as I moved around in his field of view. The JeVois can easily identify human forms, as well as colors. You can also use the other canned models or build your own using Python and C++.
Hedley, showing the Jevois sensor in his right eye socket.
The JeVois is sophisticated enough to correctly recognize our little white dog as a Maltese. One time, a friend of my daughter exclaimed, “It likes Margaret,” when my daughter stepped into Hedley’s field of view and he started tracking her.
The hardware hookup is much like the Pixy, featuring serial and mini USB connectors conveniently located at the back. There’s a tiny fan on the top of the JeVois because its powerful quad-core processor and high-speed camera use quite a bit of electrical power.
While you have to use the PixyMon application with the Pixy in order to view real-time video, the JeVois video feed is easily displayed via standard USB, using guvcview, VLC, or the ffmpeg program. I use Linux almost exclusively, so guvcview is my choice for watching what Hedley sees. An example of what the JeVois camera outputs using the guvcview application, “sees” at its high resolution is shown in Figure 9.
The JeVois sensor coffee cup object recognition at 15fps using guvcview.
The JeVois is a miniature Linux computer and the programs, models and algorithms reside on a micro-SD card, much like a Raspberry Pi. You’ll typically burn a new version (including the Linux OS) on the micro SD card, reinsert it in the card slot, and reboot. You can also add models via copying files from the laptop to the JeVois. All the code for the JeVois is available on the JeVois website and is quite comprehensive. You can modify the thing to your heart’s content.
Interacting with the JeVois is a bit complex. Setting the display resolution controls the vision algorithm, both when viewing on video or under program control through the serial and USB connections.
During early development, Hedley had an Arduino Mini mounted inside his skull to control the pan servo. At the time, he could only turn his head left and right. I hadn’t yet figured out how to make him look up and down. The Arduino was connected directly to the back of the JeVois sensor by a small serial cable.
On power up, the JeVois connected to the Arduino (using the serial cable) and received commands to initialize the resolution. After that, the JeVois would send formatted X-Y location text, that was interpreted by a program on the serial-connected Arduino, and move the pan servo to center his gaze on detected objects. The Arduino program was available from the JeVois site and is included in Listing 1. Some tuning of the program was required for proper servo limits, servo gain, etc. I simply ignored the tilt servo hook up and actions.
Listing 1
JeVois developed an Arduino sketch to control pan and tilt.
----------------------// JeVois control steering or a pan/tilt head from the output of JeVois modules//// We handle messages “T2 <targetx> <targety>”, “T1 <targetx>”, “PANGAIN <gain>”, and “TILTGAIN <gain>”.// targetx and targety are assumed to be in the -1000 ... 1000 range as output by the JeVois Kalman filters.// Here we only do simple PD control under the assumption that target coordinates have already been filtered upstream.#include <Servo.h>// Pin for LED, blinks as we receive serial commands:#define LEDPIN 13// Serial port to use: on chips with USB (e.g., 32u4), that usually is Serial1. On chips without USB, use Serial:#define SERIAL Serial1// Pins for up to two servos:Servo panservo;#define PANPIN 3Servo tiltservo;#define TILTPIN 5// Initial servo values in degrees:#define PANZERO 90#define TILTZERO 90// With updates typically coming in at 60Hz or up to 120Hz, we will often need to move by a fraction of a degree. Hence we keep track of the pan and tilt values multiplied by SCALE. For the gains, a gain of 100// means we will update servo angle by the 0.1*(target value/SCALE) degrees on each update. Higher gains mean// larger angular updates.#define SCALE 100long pangain = 100;long tiltgain = 100;long panval = PANZERO * SCALE;long tiltval = TILTZERO * SCALE;// Buffer for received serial port bytes:#define INLEN 128char instr[INLEN + 1];void setup(){ SERIAL.begin(115200); SERIAL.setTimeout(1000000); pinMode(LEDPIN, OUTPUT); digitalWrite(LEDPIN, LOW); panservo.attach(PANPIN); panservo.write(panval / SCALE); tiltservo.attach(TILTPIN); tiltservo.write(tiltval / SCALE); // We are ready to rock, disable logs and turn on serial outputs on JeVois platform: SERIAL.println(“setpar serlog None”); SERIAL.println(“setpar serout Hard”);}void loop(){ digitalWrite(LEDPIN, LOW); byte len = SERIAL.readBytesUntil(‘\n’, instr, INLEN); instr[len] = 0; digitalWrite(LEDPIN, HIGH); char * tok = strtok(instr, “ \r\n”); int state = 0; int targx = 0, targy = 0; while (tok) { // State machine: // 0: start parsing // 1: T2 command, parse targx // 2: T2 command, parse targy // 3: T2 command complete // 4: T1 command, parse targx // 5: T1 command complete // 6: PANGAIN command, parse pangain // 7: PANGAIN command complete // 8: TILTGAIN command, parse tiltgain // 9: TILTGAIN command complete // 1000: unknown command switch (state) { case 0: if (strcmp(tok, “T2”) == 0) state = 1; else if (strcmp(tok, “T1”) == 0) state = 4; else if (strcmp(tok, “PANGAIN”) == 0) state = 6; else if (strcmp(tok, “TILTGAIN”) == 0) state = 8; else state = 1000; break; case 1: targx = atoi(tok); state = 2; break; case 2: targy = atoi(tok); state = 3; break; case 4: targx = atoi(tok); state = 5; break; case 6: pangain = atoi(tok); state = 7; break; case 8: tiltgain = atoi(tok); state = 9; break; default: break; // Skip any additional tokens } tok = strtok(0, “ \r\n”); } // Target coordinates are in range -1000 ... 1000. Servos want 0 ... 180. // We also need to negate as needed so that the servo turns to cancel any offset from center: if (state == 3 || state == 5) { panval -= (targx * pangain) / 1000; if (panval < 5 * SCALE) panval = 5 * SCALE; else if (panval > 175 * SCALE) panval = 175 * SCALE; panservo.write(panval / SCALE); } if (state == 3) { tiltval += (targy * tiltgain) / 1000; if (tiltval < 5 * SCALE) tiltval = 5 * SCALE; else if (tiltval > 175 * SCALE) tiltval = 175 * SCALE; tiltservo.write(tiltval / SCALE); }}---------------------Useful JeVois Tips
Power: JeVois uses quite a bit of power, so make sure the power supply and cable are up to the job. I advise getting the kit that includes the split cable that plugs into two USB ports, or you can use USB 3.0.
Interfacing: The application interfaces are complex and the learning curve is steep. Setting resolution (using guvcview or under program control) determines the AI/MV algorithm used. I usually watched the video feed (over USB) to check that Hedley was “seeing” the right things. I’d then use the command-line application program interface (API) connected to an Arduino over serial, to control how Hedley moved his servos. It will probably take some effort to understand the JeVois interface.
JeVois needs to send it’s data to another machine for anything to happen in the physical space. The other machine is usually an Arduino, Raspberry Pi, or ESP32. Those platforms can then forward the “what the JeVois interprets” data up to the cloud or use the results to make decisions at the local hardware level, as in a robot. For example, the JeVois might “see” a cat, send that result to the Pi, and the Pi will say “I see a cat” through its audio speech program and speaker.
The JeVois sensor as shown in Figure 6 and placed as in Figure 10 will set you back around $70. If you are an enthusiastic programming nerd and like developing, this is the sensor for you. For mashing hardware vision (as on your robot) together with other processors or cloud-based services, the Huskylens2 might be your golden ticket.
Top view of the JeVois sensor mounting inside Hedley’s skull.
New Ground With The Huskylens2
A few months ago, I ordered a Husklylens2. It’s brought to us by the hardware development folks at DFRobot and you might say it is a new direction for DIY MV.
The compact little 2.8” × 2” × 0.75” aluminum block shown in Figure 11 has a camera, lighting LEDs, a microphone, and push-button on the front. The bottom houses the USB-C connector and socket for the I2C/UART connection. A 2.4” color touchscreen, shown in Figure 12, takes up the back of the device. There’s a micro-SD card slot on the left side.
Front view of the Huskylens2 sensor.
A Husklens2 menu on the built-in touchscreen.
The Huskylens2 has a dual-core 1.6GHz processor with an integrated 6 trillion-operations-per-second (TOPS) AI acceleration unit, along with 1GB of RAM. It also has 8GB of internal eMMC storage for firmware, programs, and models.
Unlike the Pixy and the JeVois, this device has an optional internal Wi-Fi card. The module allows wireless video and data transmission, for access to cloud-based large language models (LLM) and services.
The Huskylens2 is also relatively power hungry. It worked just fine with a 45W power adapter and a thick USB-C cable. It also worked with a standard USB-to-USB-C cable. It kept rebooting when I tried to use the same cable from the accessory USB port in my vehicle.
DFRobot provides an image-burning program that you use over USB. I had to fiddle with permissions on my Linux notebook to get it to work. Windows burning should be straightforward, although you’ll also need to download a companion driver from the DFRobot website. Over-the-air (OTA) might be possible, although I haven’t tried that yet.
I’ve mostly updated the firmware and looked at the various machine vision “applications,” using the built-in screen. One application streams model results, such as those shown in Figure 13 and Figure 14, superimposed on the live video, over to my laptop. I’ve used ffmpeg to display what the Huskylens2 was interpreting.
The Huskylens2 sensor recognizing a coffee cup.
The Huskylens2 sensor showing pose recognition.
The DFRobot webpages have good directions on hooking up the Huskylens2 to an Arduino, using I2C (the default) so you can get started outputting data. I hooked up the Huskylens2 to the included external power board using the accompanying white I2C cable. I then plugged the included I2C/Serial cable into the Gravity socket on the power board. On the ESP32 board, I attached the red wire to the 3.3V pin, the black wire to GND, the green wire to D21 (I2C-SDA) and the blue wire to D22 (I2C-SCL). Both the ESP32 and the Huskylens2 were powered up with their respective USB cables.
Of course, the Huskylens2 libraries had to be downloaded and added to the Arduino IDE. The example code for “face recognition” was then uploaded to the ESP32. I next selected “Face Recognition” from the touchscreen on the Huskylens2. After all that, I could watch the “face ID,” “face coordinates,” and other data scroll by using the Arduino IDE serial terminal as shown in Figure 15. It’s not too hard to imagine reading the data and commanding some servos to pan and tilt a robot skull, with a bit of additional Arduino code. See the basic program in Listing 2.
Screenshot of face data on the Arduino IDE serial terminal.
Listing 2
DFRobot developed Huskylens2 to Arduino script to output vision recognition data to the terminal.
----------------------#include <DFRobot_HuskylensV2.h>#include <ProtocolV2.h>#include <Result.h> // Create objectHuskylensV2 huskylens; // Main program startsvoid setup() { Serial.begin(9600); Wire.begin(); while (!huskylens.begin(Wire)) { delay(100); } huskylens.switchAlgorithm(ALGORITHM_FACE_RECOGNITION);}void loop() { huskylens.getResult(ALGORITHM_FACE_RECOGNITION); if ((huskylens.available(ALGORITHM_FACE_RECOGNITION))) { Serial.println((String(“Face ID”) + String((RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_FACE_RECOGNITION), Result, ID))))); Serial.println((String(“Face name”) + String((RET_ITEM_STR(huskylens.getCachedCenterResult(ALGORITHM_FACE_RECOGNITION), Result, name))))); Serial.println((String(“Face center coordinates”) + String((String((RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_FACE_RECOGNITION), Result, xCenter))) + String((String(“, “) + String((RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_FACE_RECOGNITION), Result, yCenter))))))))); Serial.println((String(“Face width”) + String((RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_FACE_RECOGNITION), Result, width))))); Serial.println((String(“Face height”) + String((RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_FACE_RECOGNITION), Result, height))))); Serial.println(); delay(500); }} ----------------------Included in the Huskylens2 library package are numerous examples for the various recognition modes including those shown in Table 2. With the hardware hooked up you can run them just as you would regular Arduino example programs.
The Huskylens2 library package contains numerous examples of recognition algorithms.
Another cutting-edge feature of the Huskylens2 is its built-in model context protocol (MCP) service. MCP lays out a common AI data format to interface with external systems, using an open-source standard.
Huskylens2, at around $85 including the Wi-Fi card, is the current cutting-edge DIY level machine vision sensor. It’s a lot of potential in a small package.
People have put tons of work into developing MV hardware, firmware and software. I think applying the results, in the physical world, is where the real magic happens.
— ADVERTISMENT—
—Advertise Here—
It’s All About Analyzing Data—Where Will You Go?
Vision sensors analyze and identify what they capture and send the results to other machines, so actions and decisions can happen in the physical world. They are real AI-at-the-edge devices.
For example, the Pixy might watch a ping-pong table, use its AI algorithms to figure out if it is looking at a fast-moving ping-pong ball (actually a color blob), then direct it’s on-board controller to pan/tilt servos to keep the ball in it’s field of view.
Similarly, Hedley might watch my office chair, through his JeVois sensor, use AI algorithms to identify me, his master, as Dr. Torq. The JeVois could then send my rough size, X-Y coordinates, and statistical probability of correct identification to an ESP32 over serial. That microcontroller could then activate servos that keep Hedley pointed at me, as per its firmware program (Arduino sketch).
I might try using the Huskylens2 to capture hand gestures and initiate various tasks, like turning on a lamp if I give it a thumbs up. Maybe I can add sign language-like capabilities to Hedley. I’m curious about how to use the MCP service with ChatGPT, Grok, and other AI services. DFRobot says they are planning some kind of MQTT capability; Think of the possibilities!
All three sensors are useful. The Pixy (now Pixy2) has modest capabilities, is easy to learn and use. JeVois has extensive programming capabilities, a steep learning curve and a complex hardware/software interface. The Huskylens2 has advanced built-in models, a cool touchscreen display and respectable recognition performance.
Pick one up or get them all. For a very modest cost, you can get into the AI vision game right now. Take some time to learn and understand how to use these things, and I think you’ll build some truly awesome new projects.
RESOURCES
Charmed Lab | https://charmedlabs.com
DFRobot-Gravity | https://www.dfrobot.com
JeVois Smart Machine Vision Camera | http://jevois.org
PUBLISHED IN CIRCUIT CELLAR MAGAZINE • JUNE 2026 #431 – Get a PDF of the issue
Sponsor this Article
Dr. Torq (aka Rob Reilly) is a designer, fabricator, writer, and speaker. His tech articles cover microcontrollers, physical computing, Linux, and DIY projects and have appeared both in print and on the Web. He periodically gives talks about the one-off gadgets he builds, at large tech conferences. He’s available for consulting or gigs at 407-718-3274 or via email at doc@drtorq.com.
