All About the Data

How video and audio analytics deliver an intelligent advantage

• By Joe Morgan
• Mar 01, 2020

When you consider today’s smartphones, the phone part is almost ancillary. It’s really the apps and the advanced optics that drive the purchase. The same is true of today’s intelligent video surveillance cameras. Advanced optics are a given. The differentiator is what you can do with that image or data and what actionable information you can glean from processing it.

We Have Come a Long Way

We have come a long way from the early days of video analytics. Back then, overblown promises often fell short and delivered underwhelming results. Fast forward to today where advanced algorithms can reliably extract and process a wealth of usable data from video images. Some sophisticated algorithms can identify the molecular makeup of venting plumes at oil refineries or pinpoint an overheating transformer at a power substation. There are algorithms designed to trigger alerts based on anomalous behavior patterns like someone walking the wrong way on an exit ramp.

The enormous progress being achieved is largely due to advanced application development, greater processing power embedded in-camera and instantaneous connectivity to supercomputing power in the cloud to filter and analyze the collected data and direct a response in milliseconds.

What Does This Mean for Perimeter Security?

Perimeter security is all about situational awareness. Who or what is out there, and does that pose a threat? The more we know, the faster we know, the sooner we can mitigate the risk. Cameras generally capture a huge amount of footage, too much for a human to process in a timely manner. Video analytics can quickly whittle down this vast collection of pixels to just the relevant information we need to take appropriate action.

Applying artificial intelligence and machine learning, we can scan video data for specific images based on physical characteristics, movement, behavior, color and other criteria. These sophisticated algorithms can help security staff differentiate between benign events and menacing danger and automatically initiate a suitable response (e.g., auto-lock a gate, alert a guard, launch a drone, trigger an audio warning to trespassers, etc.) based on previous experience. Applying predictive analysis to the video metadata assists in projecting future events and behaviors – a decided asset for security personnel who strive to be more proactive in heading off trouble.

Creating Smarter Perimeter Protection

At its roots, video analytics falls into three categories: pixel-based, object-based and application-specific. Pixel-based analytics sends an alert when it detects changes in the pattern of pixels in the video frame. It’s used in applications like motion detection and camera tampering.

• Object-based analytics is far more sophisticated in that it can recognize and differentiate between objects such as cars, people, trees and buildings and track them.
• Application-specific analytics use pixeland object-based information to examine the video for specific criteria such as license plate identification, facial recognition or fire detection.

Innovative developers are continuing to add to that portfolio, designing ever more powerful, targeted analytics to address the evolving challenges to perimeter and subperimeter protection. Let’s look at some of the significant advances to date:

Characteristics analysis. We’ve come a long way from basing event alarms solely on detecting pixel change. Now we have analytics able to categorize and understand the behavioral characteristics of those moving pixels. How does this benefit perimeter security?

Security can use video analytics to automatically detect unusual behavior – such as a person walking by the perimeter of a restricted area and stopping for a short period of time. Before alarming, the analytics could direct the camera to zoom out to look for associated concerns like whether the person is wearing a backpack or carrying a weapon. The analytics can direct the camera to zoom in close to the face to categorize certain behavior such as eye movement or expressions that might indicate suspicious intent.

Minimal pixel detection. In the past, video analytics needed quite a few pixels for reliable detection and classification of a person or object. Today there are advanced analytics that only need one or two pixels to reliably detect a person or object over a mile away. This is a major breakthrough. Historically, for a camera to detect something long-range, it would have needed to be outfitted with a larger, more expensive lens. Unfortunately, the larger lens would narrow the field of view. This latest innovation takes long-range detection to a new level.

Not only does the analytics enable the network camera (visible light or thermal) to detect a distant object using only one or two pixels on target, it does so while maintaining the camera’s normal field of view. This means users no longer have to sacrifice width for distance. Coupling this long-distance, minimal-pixels analytics with advanced camera features like optical zoom provides security staff with greater situational awareness much earlier than previously possible.

Thermal gradient analysis. Thermal cameras have always excelled at detecting people, objects and incidents under adverse conditions such as complete darkness, smoke, haze, dust, light fog and even bright sunlight. But today’s thermal analytics have gone well beyond simply detecting and identifying the heat signature differences between people, cars and objects. Now we have thermal analytics that can monitor temperature variances and the speed of temperature change and trigger an alarm if they detect unacceptable variances. They translate data into an isothermal color palette to make it easier for operators to see which areas need their attention.

The implications for better safety and security are enormous. Thermal analytics can raise the alarm that self-igniting material – such as dust or oily rags – are about to combust. The thermal gradient algorithms can be used to predicting transformer and switch gear failures at power substations in time to forestall wide-scale outages. Factories can use thermal analytics to identify overheating machinery and leaky pipes. Refineries can use thermal analytics to monitor stacks for gas flares that might indicate pressure buildup that could damage critical equipment.

Specific application analytics. In recent years, we’ve seen a rise in applicationspecific analytics: everything from license plate recognition and facial recognition to perimeter defense, direction and speed detection, as well as smoke and fire recognition. We’re seeing facial recognition analytics moving beyond the realm of simply recognizing and comparing facial features into the realm of recognizing and identifying emotions. We’re also at the forefront of artificial intelligence and machine learning that will take video analytics into the realm of predictive analysis – giving security even greater situational awareness and earlier warning of potential threats.

Alerting on Sound Patterns

Two-way audio has been a staple of video cameras for decades. Adding audio analytics takes this feature to a whole new level. When you equip a surveillance system with intelligent ears, it can detect and identify sound waves that might indicate a problem or a threat and trigger an alert. Today we have audio analytics designed to detect and recognize gunshots, breaking glass, car alarms, even voice patterns that indicate anger, stress or fear.

Integrating analytics other physical security systems. Because video and audio analytics are embedded in the network camera, they can be programmed to automatically trigger specific responses from other systems on the network. For instance, video analytics might trigger a network speaker to warn trespassers that they’ve been spotted, which generally deters them from breaching the perimeter.

Gunshot detection could trigger an automatic lockdown and call to police over the property’s VoIP phone system. Aggression detection could alert security when a peaceful protest at the gate starts to sound like it’s turning into an angry mob.

At the end of the day, it’s all about the data: capturing it, managing it and using it to prevent events or, when that’s not possible, intercede before things escalate.

Analytics give you an important perimeter security advantage. It is important to remember that perimeter protection doesn’t just involve the outer limits of a facility like the parking lots and the roads abutting the property. It includes the sub-perimeters within a facility as well. That could be anything from a data center to a narcotics cabinet, from a high-asset equipment shed to critical process machinery.

From a security perspective, advanced video and audio analytics provide reliable ways to monitor and assess the actions of people, cars, objects and equipment and trigger alerts when potential threats are imminent. It’s a true force multiplier, especially for organizations with limited resources for around-the-clock security personnel.

If past experience has made you leery of analytics, the time has come to put aside your bias. Just like the early failures of the Wright brothers eventually led to commercial aviation, the early disappointments in intelligent video eventually led us to today’s smarter, more reliable technology.

Intelligent analytics not only detect unusual movement, sounds and behaviors – they’re able to quickly categorize events while they’re still on the perimeter. Because the data is more comprehensive and reliable than network sensors alone can provide, security is able to quickly verify the situation and avert potential threats before any harm is done. That’s what safe, effective perimeter security is really all about.

This article originally appeared in the March 2020 issue of Security Today.