Drone Camera Solutions Aim for New Heights

Eyes in the Skies Evolve

We’re now way past the era when an ordinary camera on a quadcopter style drone is a big deal. Today’s drone cameras are highly sophisticated, with designs evolved for drone use. The latest solutions support thermal imaging, multiple camera streams and more.

Drone cameras
Multi-sensor cameras
Thermal imaging drone cameras
Drone collision avoidance
HD cameras
GPU-based systems

Today’s drone cameras are highly sophisticated with designs evolved specifically for drone use. In fact, some cameras embed so much processing, the term camera-computer is gaining steam. Advanced drone camera subsystems support multiple camera video streams and even perform AI-based intelligence functions aboard drones.

Today’s drone camera solutions include not just visible light solutions. Thermal camera technology is now moving into the size, weight and power levels practical for commercial drones. Collision detection-and-avoidance solutions are likewise becoming practical for commercial drone designs.

Camera technology for drones spans a wide area of subjects including 4k HD video capture, image stabilization, complex board-level video processing, multi-camera drones, hybrid IR/video cameras and drone development platforms. Over the past 12 months, vendors at the camera, drone and system-level have been evolving their existing drone camera technologies while also developing new innovative products.

MULTI-SENSOR CAMERAS
Exemplifying these trends, in November, FLIR Systems announced its StormCaster family of drone payloads for its SkyRanger R70 and R80D SkyRaider drone airframes. The new line launched with two multi-sensor products: The StormCaster-T, which delivers continuous zoom and longwave infrared (LWIR) imaging, and the StormCaster-L, which provides ultra-low-light imaging, tracking and mapping.

The StormCaster family serves as FLIR Systems’ next generation payload family for the company’s own drone airframes gained through the acquisition of Aeryon Labs in February 2019. According to the company, the new cameras offer a 7x improvement in line-of-sight stabilization, enhanced range of motion and greater geolocation accuracy. Designed with a rugged package, the StormCaster family is interoperable so that users can quickly and easily change modules on the FLIR SkyRanger and SkyRaider drone systems.

The StormCaster-T features a FLIR Boson thermal camera to support detection, recognition, identification and target acquisition day or night, with maximum range and time-on-station (Figure 1). Its continuous zoom lens, with a 75mm maximum focal length, permits long distance ISR while providing clear, actionable imagery. The StormCaster-L ultra-low-light imaging camera offers high ISR and mapping performance during twilight and nighttime operations. Key features include 4K recorded video and full-color night vision in low-light conditions.

— ADVERTISMENT—

—Advertise Here—

FIGURE 1 – The StormCaster-T features a FLIR Boson thermal camera that supports detection, recognition, identification and target acquisition day or night, with maximum range and time-on-station. Its continuous zoom lens, with a 75mm maximum focal length, permits long distance ISR while providing clear, actionable imagery.

Developed for US defense and federal agencies, the R80D SkyRaider delivers a range of versatile Group 2 and Group 3 payload capabilities with the agility and single-operator deployment footprint of a proven Group 1 Vertical Take-Off and Landing (VTOL) aircraft. SkyRaider can carry and deliver multiple payloads up to 4.4 pounds and features an open architecture, plus one of the most powerful embedded AI computing devices available on a small drone. Similar in capabilities to the R80D but for non-US military customers, the SkyRanger R70 was designed for the most demanding drone operators within the global defense, security and public safety markets.

HIGH-FRAME RATE SOLUTION
For its part, leading drone vendor DJI also builds a variety of its own camera solutions. DJI’s latest camera innovations move the thermal drone camera technology into the mainstream civilian sector. Along such lines, in December DJI introduced a new thermal imaging camera, the Zenmuse XT S, for DJI’s Matrice 200 Series drones (Figure 2). The camera features an infrared thermal imaging sensor with high thermal sensitivity and resolutions for clear and detailed thermal imagery. The Zenmuse XT S is expected to enable firefighters, police officers, inspectors and more to gather intel beyond the capabilities of visual data, and act on them quickly.

FIGURE 2 – The Zenmuse XT S is designed for DJI’s Matrice 200 Series drone—DJI Matrice 210 V2 shown here. The camera features an infrared thermal imaging sensor with high thermal sensitivity and resolutions for clear and detailed thermal imagery. This is enabled through the 19mm lens on a 640 x 512, 25Hz refresh rate radiometric sensor, and improved further with 2x and 4x digital zoom capabilities.

DJI says it’s seen an increased demand for thermal imaging drone solutions since introducing the industry’s first integrated thermal drone camera through the original Zenmuse XT in late 2015. The company believes the Zenmuse XT S will make high- resolution and frame-rate thermal imaging cameras more affordable and accessible around the world.

The Zenmuse XT S was developed to handle even the toughest thermal imaging applications, such as long-range inspections, says DJI. This is enabled through the 19mm lens on a 640 × 512, 25Hz refresh rate radiometric sensor, and improved further with 2x and 4x digital zoom capabilities. A live view of thermal data is streamed directly to the operator on their mobile device through the DJI Pilot app, where they can access intelligent features that can quickly extract insights from the thermal data.

These features include a spot meter that enables users to tap a point for realtime temperature measurements. Area measurement is supported as well. Users can select an area to get the maximum, minimum and average temperatures. A temperature alarm provides notification when an object’s temperature exceeds a pre-set limit. Users can also adjust the colors applied to thermal data and select how colors are distributed across a specific temperature range.

The radiometric sensor on the Zenmuse XT S captures temperature data on each picture and, when stored in R-JPEG file formats, can be analyzed later using the DJI Thermal Analysis Tool software to adjust parameters such as emissivity and reflected temperature for a more detailed inspection report. The Zenmuse XT S is compatible with DJI’s leading Matrice 200 Series and Matrice 200 Series V2 drone platforms when used with the DJI Pilot flight control app.

DRONE COLLISION AVOIDANCE
As commercial drones move into more complex beyond-visual-line-of-sight (BVLOS) use cases, drone developers have been hungry for technology to automate collision avoidance. One such drone developer is ZM Interactive (ZMI). In January, Iris Automation announced that is has been selected as the detect and avoid (DAA) provider to ZMI’s drones, enabling ZMI customers to conduct BVLOS operations.

The Iris Automation Casia system is a turnkey solution that detects, tracks and classifies other aircraft and makes intelligent decisions about the threat they may pose to the drone. It then triggers automated maneuvers to avoid collisions, and alerts the pilot on the ground in command of the mission. The collision avoidance system helps drones see the world as pilots do.

ZMI manufactures the xFold drone—an industrial, military-grade UAV that comes in a variety of different sizes and configurations. xFold drones are designed for uses ranging from aerial cinematography, 3D mapping and inspections, cargo delivery and firefighting efforts. Because its frame can change between a x4 (Quad), x6 (Hexa), X8 (octo) (Figure 3) and X12 (Dodeca) configurations in minutes and offers heavy payload capability of over 300 pounds, the xFold is suitable for a wide range of commercial, industrial, military and emergency response applications. xFold drones are said to be the only FAA-approved airworthy drone at that payload level. ZMI now provides the option of Casia integrated onto all of its drone platforms.

— ADVERTISMENT—

—Advertise Here—

FIGURE 3 – Shown here is an xFold Dragon X8 U11 Drone with 3-axis gimbal for DSLR/cinema cameras.

Last year, Iris Automation conducted the first Part 107 FAA-approved BVLOS drone flights without a requirement for visual observers or ground-based radar. The FAA’s approval for the flight was granted based on the utilization of the Casia detect-and-avoid system. As part of the partnership, Iris also offers customers with Casia onboard regulatory support for Part 107 waiver writing and regulatory approval processes to secure the necessary permissions for their unique UAS BVLOS operations.

MORE CASIA DETAILS
Iris Automation launched Casia about a year ago (April 2019), touting it as the first commercially available computer vision detect-and-avoid solution to enable BVLOS operations for autonomous vehicles. According to the company, the system allows a drone to truly understand the aviation environment around it as if a pilot were on board. Casia detects other aircraft, uses machine learning to classify them, makes intelligent decisions about the threat they may pose to the vehicle and triggers automated maneuvers to avoid collisions.

Casia is a combination of both hardware and software that’s ultra-lightweight, low power and small in size (Figure 4). It comprises sophisticated artificial intelligence algorithms and software packaged in a self-contained supercomputer that works with a machine vision camera. The Casia technology has been extensively tested with over 7,000 real-world test flights and mid-air collision scenarios—flying various manned aircraft against drones and over 40,000 encounters in simulation. Casia also ran a successful early adopter program with more than 30 participating beta customers from five countries.

FIGURE 4 – A detect-and-avoid solution developed to enable BVLOS operations for drones, Casia is a combination of both hardware and software that’s ultra-lightweight, low power and small in size. It comprises sophisticated AI algorithms and software packaged in a self-contained supercomputer that works with a machine vision camera.

Casia uses industrial cameras onboard the drone to observe and interpret the world for full situational awareness of the operational environment. Finding moving aircraft is the first step in developing a safe and robust collision avoidance system. Casia then processes information about intruder aircraft at lightning speed and uses Iris Automation’s proprietary algorithms to make intelligent decisions in realtime. Using location, heading and speed information, Casia determines the safest course of action to avoid collisions.

The system is designed to be modular, expandable and upgradable, enabling easy integration and system maintenance. It includes configurable camera modules and has wide power input and protection. The company provides multiple ways for drone system developers to integrate the Casia solution. First, it’s available as a rugged, self-contained and seamless plug-and-play system. The second is a bare board option that offers flexibility and customization. And finally, a software-only Casia solution is available for deep integration and scalability. According to Iris Automation, the small modular Casia system is easily integrated into your drone design, whether custom made or off-the-shelf—including multi-rotors to fixed wings, and anything in between.

FOUR HD CAMERAS
For its part, E-con Systems makes a number of camera solutions for a variety of applications, including drones. Last summer, E-con Systems launched its SurveilsQUAD camera for the NVIDIA Jetson AGX Xavier Developer kit and NVIDIA Jetson TX2 Developer kit. SurveilsQUAD is four synchronized full HD cameras (Figure 5).

FIGURE 5 – SurveilsQUAD is comprised of four synchronized full HD cameras. SurveilsQUAD is built around E-con’s e-CAM21_CUMI290_MOD camera, which incorporates a Sony Starvis IMX290 color, 1/2.8″ CMOS image sensor.

The SurveilsQUAD is built around E-con’s e-CAM21_CUMI290_MOD camera, which incorporates a Sony Starvis IMX290 color, 1/2.8″ CMOS image sensor. The 2-megapixel (MP) camera supports HD (1280 × 720) and FHD (1920 × 1080) at 30fps in sync mode and either 120fps or 60fps in async mode, depending on whether you use 10-bit or 12 bit output. The system depends on the Xavier Image Signal Processor (ISP) to process Auto White Balance and Auto Exposure control functions. The S-Mount compatible system consists of the camera board and a baseboard to interface with the Xavier Developer Kit. The system supports -30°C to 85°C temperatures and ships with five 30cm micro-coaxial cables and an SD card with the Linux code.

NVIDIA’s 105mm × 87mm × 16mm Jetson AGX Xavier module has greater than 10x the energy efficiency and more than 20x the performance of the Jetson TX2, claims NVIDIA. The module features 8x ARMv8.2 cores and a 512-core NVIDIA Volta GPU with 64 tensor cores with 2x NVIDIA Deep Learning Accelerator (DLA) engines. There’s also a 7-way VLIW vision chip, as well as 16GB 256-bit LPDDR4 RAM and 32GB eMMC 5.1. The module is available via a Xavier Development Kit.

NVIDIA JETSON SUPPORT
In October, E-con Systems announced that its STEEReoCAM 2MP global shutter MIPI Stereo camera is available to evaluate with NVIDIA Jetson Nano developer kit. The camera is designed for robotics and autonomous guided vehicles, drones, surgical robotics, depth sensing, gesture recognition, 3D video recording, 3D measurements and embedded vision.

The 2MP STEEReoCAM can stream dual 1600 × 1300-pixel images at 30fps and can provide 3D depth maps for 1600 × 1300 resolution at 22fps without stressing the CPU, says E-Con. On the TX2, the camera supports three modes: ultra-accuracy with 5fps depth mapping, high-accuracy at 15fps and high frame rate at 22fps.

Measuring 135mm × 20mm × 23.65mm (without enclosure), the STEEReoCAM has a 2MP global shutter sensor and 10-bit monochrome output format (Figure 6. The camera is equipped with 2x OmniVision 1/2.9″ OV2311 CMOS sensors separated by an inter-ocular distance or baseline of 100mm. There’s also a pre-calibrated lens pair mounted on S-Mount (M12) holder.

FIGURE 6 – Measuring 135mm x 20mm x 23.65mm, the STEEReoCAM has a 2MP global shutter sensor and 10-bit monochrome output format. The camera is equipped with 2x OmniVision 1/2.9″ OV2311 CMOS sensors separated by an inter-ocular distance or baseline of 100mm.

The STEEReoCAM is primarily intended to connect to a Jetson TX2 Developer Kit via a supplied MIPI-CSI2 adapter baseboard. The MIPI interface enables the camera to stream uncompressed stereo, which is processed by TaraXL SDK on the Jetson module to generate a depth map. There’s a flexible 0.95-meter to 8-meter range and a 6-axis IMU to facilitate autonomous guided vehicle applications along with visual odometry and SLAM, says E-con. Kit contents include the camera, tripod, baseboard and cable. The system works at -30°C to 85°C.

While it’s true that commercial drones are used for a wide variety of uses and carry many kinds of payloads, there’s no denying that cameras are the most widely used kind of payload for drones. To keep pace with demands for higher-performance and greater functionality, drone camera manufacturers will continue to innovate with new capabilities and options.

PUBLISHED IN CIRCUIT CELLAR MAGAZINE • APRIL 2020 #357 – Get a PDF of the issue

— ADVERTISMENT—

—Advertise Here—

Keep up-to-date with our FREE Weekly Newsletter!	Don't miss out on upcoming issues of Circuit Cellar. Subscribe to Circuit Cellar Magazine Note: We’ve made the May 2020 issue of Circuit Cellar available as a free sample issue. In it, you’ll find a rich variety of the kinds of articles and information that exemplify a typical issue of the current magazine.
Would you like to write for Circuit Cellar? We are always accepting articles/posts from the technical community. Get in touch with us and let's discuss your ideas.

Sponsor this Article

Jeff Child

Former Editor-in-Chief at Circuit Cellar | Website | + posts

Jeff served as Editor-in-Chief for both LinuxGizmos.com and its sister publication, Circuit Cellar magazine 6/2017—3/2022. In nearly three decades of covering the embedded electronics and computing industry, Jeff has also held senior editorial positions at EE Times, Computer Design, Electronic Design, Embedded Systems Development, and COTS Journal. His knowledge spans a broad range of electronics and computing topics, including CPUs, MCUs, memory, storage, graphics, power supplies, software development, and real-time OSes.