Drone Video Technologies are Flying High

Cameras, Boards and Kits

The video technologies available for today’s drones continue to advance. New products and solutions are adding new intelligence, features and performance levels to enhance how video is captured and processed aboard both consumer and commercial drones.

While drones can have a variety of sensor types, clearly video ranks the most common capability of today’s consumer and commercial drones. Long gone are the days when placing an ordinary camera on a quadcopter style drone is a big deal. Today, drone cameras are highly sophisticated with designs evolved for drone use. In fact, some cameras embed so much processing, the term camera-computer is gaining steam.

Drone cameras are linked with board-level solutions that support multiple camera video streams and even perform AI-based intelligence functions aboard drones. Add those to the emergence of complete drone design platforms—that include camera and all—and it’s clear that we’re in a golden age for designing and developing powerful drones.

Video technology for drones spans a wide area of subjects including chip-level video processing, 4K HD video capture, image stabilization, complex board-level video processing, drone-mounted cameras, hybrid IR/video cameras and drone development platforms. Over the past 12 months, vendors at the camera-, board- and system-level have been evolving their existing drone video technologies while also creating new innovative solutions.

COMPLETE REFERENCE PLATFORM
Starting at the platform level, drone development has definitely become easier these days, with companies both large and small providing complete drone development kits. One of these on the large company side is Qualcomm. In December, Qualcomm’s partner Intrinsyc Technologies announced it will distribute the Qualcomm Flight Pro reference platform (Figure 1). The platform is Qualcomm’s latest optimized board and development kit targeted specifically for consumer drones.

The Qualcomm Flight Pro reference platform for consumer drones and robotics applications is a follow-on to the Qualcomm Flight platform, which was previously launched under the name Snapdragon Flight. The Qualcomm Flight Pro steps up from a 2.2 GHz Qualcomm Snapdragon 801 with 4x Cortex-A53 like Krait cores to a Snapdragon 820 (APQ8096SG) with 4x higher-end Kryo cores—2x at 2.15 GHz and 2x at 1.6 GHz. The Snapdragon 820 also integrates an Adreno 530 GPU and Hexagon 680 DSP.

The system runs on a Linux 3.18 and Yocto/OpenEmbedded based stack with SDK, a Docker container and support for the Robot Operating System (ROS). An optional Qualcomm Navigator SDK supports autonomous, vision-supported Wi-Fi-based flight controls with advanced flight modes, built-in sensor calibration and automatic flight logging.

The Qualcomm Flight Pro is slightly larger than the original at a still very compact 75  mm x 36 mm, making room for 4x cameras driven by MIPI-CSI interfaces. The kit includes a pair of forward-facing stereo-vision cameras using Omnivision OV7251 black and white VGA sensors by way of a Sunny GP161C module, as well as a forward-facing, 13-megapixel, 4K-at-30-fps camera with a Sony IMX214 color sensor in a KLT Module. There’s also a downward-facing camera with a black and white VGA OV7251 sensor via a Sunny MD102A module.

The Pro board includes 4 GB LPDDR4, a microSD slot and 32 GB UFS 2.0 (HS-G3 1-Lane) storage. Other features include a Qualcomm QCA6174 wireless module with 802.11ac 2×2 MIMO and Bluetooth 4.2 (with antenna mounts), as well as a Qualcomm WGR7640 GNSS location chip that supports an optional U-blox GPS module. The SBC is further equipped with an IMU with gyroscope, accelerometer, compass (Dual Invensense MPU9250) and a barometer/pressure sensor (Bosch BMP280).

More recently, in late February, Qualcomm and Thundercomm launched their Robotics RB3 Platform” that includes an octa-core Snapdragon 845 via a new “DragonBoard 845c” 96Boards SBC and tracking cameras. While that platform appears to be marketed toward terrestrial robots, Qualcomm did tell us that it can also be used for developing drones.

CAMERAS, CAMERAS, CAMERAS
Switching to the camera side of drone video, the latest crop of drone-based camera systems includes a wide range of solutions, some focusing on photo and video quality, others on new features and capabilities. For its part, in December, FLIR Systems announced three Neutrino midwave infrared (MWIR) camera cores. These include the small, lightweight FLIR Neutrino LC and two FLIR Neutrino Performance series cores, the SX12 and QX (Figure 2). The latest models expand the FLIR Neutrino cooled camera core family for commercial, industrial and defense OEMs and system integrators.

CAMERA MEETS SWaP+C NEEDS
The Neutrino LC is FLIR’s first High Operating Temperature (HOT) MWIR camera core and the first model in the SWaP+C (Size, Weight, Power and Cost) series. As the smallest, lightest weight and lowest power consuming Neutrino model available, the LC can be integrated with smaller drones and allow drone operators to fly longer. With HOT technology, Neutrino starts imaging two times faster than previous models, allowing optical gas imaging professionals to detect gases faster. Additionally, the Neutrino’s longer operational lifetime allows installation in security applications where maintenance access is restricted, difficult or costly.

The two new Neutrino Performance series products, the Neutrino SX12 and the Neutrino QX, offer the highest-resolution MWIR performance from FLIR. The Neutrino SX12 produces high-definition (HD) thermal imaging video, while Neutrino QX, with more than 3.1 megapixels, is FLIR’s highest resolution MWIR core. Both Neutrino Performance models provide crisp imagery at long distances while maintaining a wide field of view and are well suited for ground-based or airborne intelligence, surveillance, reconnaissance (ISR) and counter-drone solutions. The Neutrino SX12, QX and LC are dual-use camera cores for commercial, industrial and defense products and are classified under the U.S. Department of Commerce Export Administration Regulations as Export Control Classification Number 6A003.b.4.a.

INTELLIGENT CAMERA
One area of innovation in drone cameras is adding more intelligence to them. Exemplifying this trend, in August last year Aerialtronics launched a new version of its Pensar camera-computer driven by artificial intelligence. According to te company, Pensar is one of the world’s first platforms with dual spectrum digital vision that allows real-time analysis of images or data (Figure 3). Infinitely customizable, it can be mounted on a professional drone, mobile robot or used as an independent camera. The dual spectrum is provided by a built-in Sony 30x full HD optical zoom camera with 1920 x 1080 resolution and a 30 fps Boson FLIR integrated thermal camera. The Pensar is 112.5 mm x 98.5 mm x 67.5 mm in size and weighs 672 g.

Pensar does real-time data analysis using a miniaturized Nvidia embedded processor with 1.5 teraflops of power. Its computing power, accelerated by the Nvidia Jetson TX1 GPU processor in the Nvidia Jetson module, enables it to detect, recognize, analyze and classify objects or people in real time. Simultaneous data acquisition and processing allows for immediate decision making.

Pensar’s integrated camera with a 30x optical zoom makes it possible to spot very small details. Also embedded in Pensar is a FLIR thermal camera used to identify heat sources and determine their temperature. The streams from these two cameras, recorded simultaneously, help optimize image analysis in day and night time and bad weather conditions.

This camera-computer can be customized and adapted for multiple applications: surveillance, inspection, public security and anti-terrorist operations, search and rescue and so on. It’s equipped with a system for facial recognition, object recognition such as license plates, animal recognition and similar tasks. A digital “privacy mask” can be integrated into the images to guarantee confidentiality and anonymity. The intelligent platform comes with an Ubuntu Linux Open Source operating system that allows system integrators to customize it to suit their needs. Pensar is compatible with open source libraries such as Google’s Tensor Flow.

FANCY PHOTOGRAPHY
As today’s drone cameras have evolved, they’re now offering many very sophisticated features for high-end photography. Along those lines, Lucint Systems’ Lucint12 camera basically provides a complete aerial image collection system in a small, rugged, low-power box (Figure 4). Lucint12 is a 12-megapixel high-quality color or monochrome image sensor with all the controls, metadata, processing and storage needed for a complete system. The unit also features a powerful built-in GPU processor to handle real time georeferencing, image preprocessing, and custom user algorithms.

Designed for photogrammetry the camera features large pixels that result in excellent dynamic range. It has lightweight, high-quality Micro Four Thirds lenses and captures metadata and precise GPS timestamp with each frame. Lucint12 provides a number of automated functions including auto-exposure designed for aerial capture ensures consistent exposures, auto-focus optimized for aerial, automotive or ground installations, and auto-trigger at fixed rate, percent image overlap, or external trigger.

The Lucint12 is complete system with rugged and reliable design features. Its global electronic shutter has no moving parts and no rolling shutter distortion. Industrial components extend operating temperature range. The unit has a fully sealed housing for harsh operating environments. The Lucint12 integrates an internal GPS to record image capture location, on-board mSATA-based image storage up to 1TB. Users can configure settings over Wi-Fi by phone or tablet.

CAMERA WITH V-SLAM TECH
Qualcomm isn’t the only big chip vendor with a hand in drone technology. Intel’s latest drone video offering is its RealSense Tracking Camera T265. Announced in January, the T265 uses proprietary visual inertial odometry simultaneous localization and mapping (V-SLAM) technology and is suited for applications that require a highly accurate and low-latency tracking solution, including robotics, drones, augmented reality (AR) and virtual reality. V-SLAM uses a combination of cameras and Inertial Measurement Units (IMU) to navigate in a similar way, using visual features in the environment to track its way around even unknown spaces with accuracy.

At the heart of the T265 is the Intel Movidius Myriad 2 vision processing unit (VPU), which directly handles all the data processing necessary for tracking on the machine (Figure 5). According to Intel, the T265 is good for applications where tracking the location of a device is important, especially in locations without GPS service, such as warehouses or remote outdoor areas where the camera uses a combination of known and unknown data to accurately navigate to its destination. The T265 is also designed for flexible implementation and can be easily added to small-footprint mobile devices like lightweight robots and drones, as well as for connectivity with mobile phones or AR headsets.

The T265 uses inside-out tracking, which means the device does not rely on any external sensors to understand the environment. Unlike other inside-out tracking solutions, the T265 delivers 6-degrees-of-freedom (6DoF) inside-out tracking by gathering inputs from two onboard fish-eye cameras, each with an approximate 170-degree range of view. The V-SLAM systems construct and continually update maps of unknown environments and the location of a device within that environment.

Because all position calculations are performed directly on the device, tracking with the T265 is platform independent and allows the T265 to run on very low-compute devices. The only hardware requirements are sufficient non-volatile memory to boot the device and a USB 2.0 or 3.0 connection that provides 1.5 W of power. The camera measures 108 mm x 25 mm x 13 mm in size and weighs only 55 g.

MULTI-CAMERA SUPPORT
In August last year, Aetina launched its first carrier board for Nvidia’s Jetson TX1 and TX2 modules that supports up to 6x cameras and offers -40°C to 85°C support. The ACE-N310 enables “360-degree surrounded view application in vehicles, drones, robots, surveillance and automation and intelligent systems at the edge,” says Aetina. With the help of the Jetson modules’ AI-enabled Pascal GPU, the ACE-N310 lets you build multi-visual intelligent systems with advanced on-premises analytics and inference, according to the company (Figure 6).

The module integrates its iNAVI Linux distribution, which adds customizable security, system recovery and backup features. iNAVI is also available with the ACE-N310 and other Aetina Jetson carrier boards, which similarly support the TX1, TX2 and TX2i. The 87 mm x 70 mm board is most closely comparable to the ACE-N510 carrier, which has an 87 mm x 50 mm footprint that matches that of the TX2 and TX2i modules themselves. Aetina also offers the Nano-ITX (120 mm x 120 mm) form factor ACE-N261 and ACE-N622 boards.

The ACE-N310 can be configured with up to 12x lanes of MIPI-CSI connectors through CSI-II or FPD-LINK III extension modules. This enables the connection of 6x 2-lane, 2-megapixel cameras with 1080p/30fps resolution or 3x 4-lane 4K cameras. Aetina offers a variety of Sony IMX based, HD resolution MIPI-CSI camera modules to choose from, as well as an optional, FPC-connected ACE-CAM6C camera board with 6x CSI-2 cameras. There are also “certified” mini-PCIe based I/O modules including dual isolated GbE and PoE add-ons and a 4x USB 3.0 option, all with 0 to 70°C and -40°C to 85°C support.

Other ACE-N310 features include HDMI, GbE, micro-USB 2.0 and 2x USB 3.0 host ports. Onboard interfaces include RS-232, I²C and 5x GPIO, as well as 2x CAN Bus connections that work only with the Jetson TX2 and TX2i. A mini-PCIe slot supports PCIe and mSATA, and there’s a 9-19 VDC input. Other options include fan and heatsink add-ons, cable kits, and a 100-240 V, 60 W 12V/5A adapter.

BOARD FOR SMALL DRONES
Last summer Gumstix released a version of its Aerocore 2 drone control board that runs Linux on Nvidia’s Jetson TX2. The Aerocore 2 drone control board arrived in 2014 and was followed in 2016 by a more advanced version that swapped out the original’s Gumstix Overo module for a DragonBoard 410C SBC. This most recent 2018 board—dubbed Aerocore 2 for Nvidia Jetson—works with Nvidia’s Jetson TX1 and Jetson TX2 modules and can be customized in Gumstix’s Geppetto online design service (Figure 7).

The Jetson TX2 is equipped with dual high-end “Denver 2” Arm cores and 4x Arm Cortex-A57 cores. The 256-core Pascal GPU with CUDA libraries for running AI and machine learning algorithms offer the potential for improved image recognition applications in drones and robotics. The Aerocore 2 is best suited for small drones called micro-aerial vehicles (MAVs), but it can also be used for larger drones, robots and other image processing applications.

The Jetson TX2 module provides the Aerocore 2 for Nvidia Jetson with 8 GB of LPDDR4 RAM, 32GB of eMMC 5.1 and 802.11ac Wi-Fi and Bluetooth. The Aerocore 2 carrier board adds an STMicroelectronics STM32F427 Cortex-M4 chip clocked at 168 MHz. This MCU is pre-loaded with the open source NuttX RTOS and APM-based PX4 firmware for real-time drone autopilot operation. It should also work with PX4-compatible projects such as QGroundControl and MAVLink. Because the Jetson boards are modules rather than an SBC, the Aerocore 2 carrier board has added more ports and other features to compensate. The board ships with a microSD slot, as well as micro-HDMI, USB 3.0 host, micro-USB OTG and micro-USB device ports.

There are two separate 4-lane MIPI-CSI-2 interfaces that support Gumstix’s $30 Caspa 4K cameras, which are built on Sony’s 13-megapixel IMX214 AF Camera sensor and support 4208 x 3120-pixel stills and 4K video at 30 fps. In addition, you get a pair of 2-lane CSI-2 connectors for 5-megapixel cameras with 2592 x 1944 resolution. The Aerocore 2 board is capable of driving 4x cameras with HD or higher resolution simultaneously.

Like other Aerocore boards and most other Gumstix boards, the Aerocore 2 for Nvidia Jetson can be customized with the Gumstix Geppetto D2O online development platform. The Geppetto drag-and-drop GUI interface lets developers add network connections or I/O, as well as create multiple projects and compare alternative designs for features and costs. Geppetto supplies free automated documentation on demand with all saved designs. The service lets you develop custom BSPs and go straight from a design to an order in one session, with 15-day manufacturing turnaround.

We’re obviously far past the days when commercial drone video was just a straightforward proposition of mounting a camera on a drone. Today there are many technology options for building drones for a variety of applications and mission types. Camera, board and system vendors are keeping pace with these trends, feeding the demands of this dynamic and growing market. 

RESOURCES
Aerialtronics | www.aerialtronics.com
Aetina | www.aetina.com
FLIR Systems | www.flir.com
Gumstix | www.gumstix.com
Intel | www.intel.com
Intrinsyc Technologies | www.intrinsyc.com
Lucint Systems | www.lucintsystems.com
Nvidia | www.nvidia.com
Qualcomm | www.qualcomm.com

PUBLISHED IN CIRCUIT CELLAR MAGAZINE • APRIL 2019 #345 – Get a PDF of the issue

Sponsor this Article