AI-Based Vision System Prevents Contraband Drops in Prisons

• By John DeWaal
• Sep 05, 2023

Recreational drone popularity continues to rise as the devices become less expensive, easier to use, and safer to fly. Current Federal Aviation Administration figures show more than 500,000 recreational drones registered, and this does not include those that weigh 0.55 pounds or less. These figures, of course, do not show unregistered drones being used for nefarious and otherwise illegal activities.

Such activities include contraband drops in prison yards, a growing problem for corrections departments around the world. In such instances, operators use unregistered drones to drop drugs, money, cigarettes, and other contraband into prison yards at nighttime to avoid detection. But as bad actor drone operators become more intelligent and adapt, drone detection capabilities have advanced as well. Some systems offer autonomous and threat-agnostic capabilities.

Bad Actors Adapt
Fighting illegal drone activity has traditionally involved deploying a radio frequency–based system to detect communication between a pilot and a commonly used drone, such as one from DJI. But this approach has become less effective because over time, tactics for drone-based contraband drops have evolved, according to Kyle Meloney, co-founder and CEO of airspace awareness company Walaris. 

“Traditional drone detection systems rely on libraries to match signals and detect known drone platforms they’ve encountered before. Perpetrators are becoming more clever, oftentimes using lesser-known drones and sometimes repurposing drones, making them more difficult to detect, track, and identify using traditional means,” he said.

As perpetrators learn more about traditional detection systems and develop ways to get around them, the need for more threat-agnostic (or “library-less”) systems has emerged. Systems today must be capable of quickly detecting, tracking, and classifying all drone makes and models, even if they haven’t encountered the drone before or if the drone has been modified.

A Multisensor Approach
Designed to work alongside other detection modalities — such as radar, radio frequency, and acoustic sensors — Walaris’ AirScout Verify is a tracking and identification software solution with autonomous threat-verification capabilities. Depending on the individual deployment, the hardware-agnostic system may deploy an AXIS Communications Q6225-LE pan-tilt-zoom (PTZ) camera with near infrared illuminators to capture images at night or a dual daytime and thermal infrared camera from Bosch.

When the slew-to-cue system (a drone detection unit integrated with another sensor, such as radar, that tells the camera the initial location of the object) receives a cue, system software positions the camera to target the detected object’s airspace. It then analyzes the camera’s imagery to locate the suspected threat and capture its precise coordinates. A PTZ algorithm triggers the camera to zoom in on the coordinates to further investigate the object while artificial intelligence (AI) and range-to-focus control algorithms enable the system to acquire the target, maintain a clear image, and classify it. The system capably tracks only verified drone threats, ignoring objects not considered a threat, such as birds. 

Once the software classifies the detected object as a drone, it sends an alert to the system operator, along with full-motion video of the threat, so the operator can decide how to handle the situation. In a prison scenario, this might mean securing the yard and bringing inmates inside, but the response also involves identifying the location of the drone to intercept the contraband. In addition, finding the location of the pilot is a priority as well, according to Meloney.

“Beyond interdicting the contraband, prisons want to find the operator of the drone. A lot of the same people are doing this repeatedly,” he said. “If there are 300 incursions in a year, it’s unlikely that 300 different people are using drones to drop contraband. It might be just a handful of groups doing it, so the system can also be used to track the drone back to its origin and make an arrest.”

Enter Edge AI
While traditional techniques work in simpler scenarios, they are less effective when dealing with complex scenes, according to Meloney. “Aerial scenes often contain a lot of clutter, motion, and varying levels of contrast, making it difficult for traditional techniques to find and verify drone threats,” he said.

To handle such complexity, the system software features proprietary AI computer vision algorithms for detection, acquisition, classification, tracking, and more. “Our software processes sequences of images in real time and preclassifies every object to determine relevance and prioritize suspected threats. This approach can effectively manage dynamic and complex scenes, while reducing false positives and automating the detect–track–identify response chain,” Meloney said. Getting to that point, however, requires a lot of algorithm and data preparation.

Walaris’ data acquisition pipeline involves acquiring a large and diverse set of high-quality images to better train its AI models. “Our AI models continuously retrain on a growing preprocessed dataset, and the AI model development, fine tuning, and training process is perpetual,” said Meloney. “Walaris constantly experiments with new AI models, optimizes existing models, and adds new images to its dataset to improve AirScout’s performance in all spectral domains and ultimately make the system more effective.”

Powering the software and its complex algorithms is a custom-configured IP67-rated industrial PC from CoastIPC, which features an NVIDIA graphics processing unit (GPU).

“Applications that require real-time responses need reliable, powerful PCs,” said Meloney. “We partnered with CoastIPC because every PC they supply fits the exact requirements needed to execute on every mission without failure, and the units always arrive on time.”

A Hands-Off Detection Approach
While contraband drops remain an issue for prisons around the world, several methods for drone detection exist on the market today. Walaris’ hands-off, comprehensive optical approach to the problem allows prisons to use advanced technologies such as radar and AI computer vision for autonomous detecting, tracking, and identifying drones in dynamic environments.

The AirScout Verify system has been operationally deployed at many corrections facilities in the United States, with many more currently rolling out. In addition, the company’s AirScout Sentry end-to-end optical detection system has been operationally deployed at prisons in Europe.

“Using an integrated sensor set that includes EO/IR cameras, radar, and other military-grade technology at a prison is a new concept,” said Meloney. “These are complex technologies and being able to run these systems fully autonomously as a force multiplier at prisons and have them work reliably against the full spectrum of drone threats is a novel concept.”