Enhanced Situation Awareness

If a camera recognizes the sound signature, it simply issues an alert

  • By Rui Barbosa
  • Jul 24, 2024

Did someone break into the building? Maybe it is just an employee pulling an all-nighter. Or is it an actual perpetrator? Audio analytics, available in many AI-enabled cameras, can add context to what operators see on the screen, helping them validate assumptions. If a glass-break detection alert is received moments before seeing a person on camera, the added situational awareness makes the event more actionable.

In the world of security and video surveillance, the value of audio is often underestimated. While audio plays a pivotal role in intercom systems, its significance in broader security and event management contexts is frequently overlooked. This oversight occurs partly due to the privacy implications associated with audio surveillance, which is strictly regulated and varies significantly across jurisdictions. However, in-camera, or edge-processed, audio analytics that detects gunshots, yells, glass breaks and vehicle horns don’t require the audio to be recorded or captured in any way.

This avoids violating privacy laws because the audio is processed instantaneously at the edge and never leaves the camera. If the camera recognizes a known sound signature, it simply issues an alert. It is oblivious to all other sounds.

Audio analytics can also enhance situational awareness in areas where video is not allowed. For example, restrooms are a no-go area for cameras, but an analytic that detects glass breaks and yells can prevent such an area from being a complete blind spot.

The Power of Audio Analytics
Audio analytics, when processed directly within an AI-enabled camera, have emerged as a specialized niche’ for many security system installers and users. When a separate purpose-built audio system is beyond the budget, modern AI-enabled cameras can step up and do double duty, reducing the overall cost of installing purpose-built glass break sensors at every point of ingress.

Using deep learning algorithms, the cameras can provide a range of audio classification and detection at the edge, including glass breaks, gunshots, yells and even persistent vehicle horns.

Microphones
Modern IP-based surveillance cameras often come equipped with built-in microphones, though some models offer jacks for attaching external microphones. Indoor camera microphones are particularly effective due to their design, which allows sound waves to penetrate through small openings in the housing. Conversely, outdoor cameras, typically certified against water and dust ingress (IP66), may exhibit reduced sensitivity due to their sealed design.

In such cases, employing an external, strategically positioned microphone can greatly enhance the accuracy of audio analytics running outdoors. High-quality directional microphones, capable of mitigating wind noise, are recommended for critical audio data collection outdoors.

Any high-quality external microphone should easily outperform an internal microphone regarding analytic accuracy, so it is worth considering in areas where audio information gathering is crucial. AI sound classification is in the range of 200Hz to 8Khz, and the frequency distribution of a captured sound is an important characteristic during analysis. Therefore, a microphone must be able to pick up frequencies across this range with a flat or neutral characteristic.

AI SoCs Enhance Accuracy
Recent advancements have seen the introduction of surveillance cameras equipped with dedicated AI System on Chips (SoC), such as the Ambarella CV52. This chip can perform both video and audio analytics simultaneously.

Using an SoC allows for integrating advanced features, including a sound database against which audio from the scene is compared for real-time classification. Deep learning algorithms make these comparisons even more accurate. For example, when identifying a sound, an i-PRO camera compares the captured sound volume level with a preset threshold value. If it is greater than the threshold, AI is then used to determine what kind of sound it could be.

With the goal of creating an AI-derived similarity score, the system determines whether the captured sound corresponds to any of the four target sound categories: yell, glass break, vehicle horn, and gunshot.

This is done by dividing the sound into regular segments, performing signal processing, and extracting relevant features that can be used for analysis and comparison. An AI inference calculation uses machine learning algorithms to analyze the audio data and classify the audio data into distinct categories with a score based on similarity to the target sound. An alarm/notification is triggered when the similarity score exceeds a certain value.

Camera Configuration for Audio Analytics
Audio detection. Proper configuration begins with setting a camera to detect relevant sounds while ignoring irrelevant background noise. Since audio levels are typically louder in abnormal situations, cameras should be tailored to their specific environments, and sound level thresholds should be set only to flag audio levels suggestive of unusual activity.

AI-based audio analytics should be trained to identify target sounds under various conditions, such as situations with typical environmental noise or other non-target sounds and at different distances. This reduces the possibility of false positives caused by background noise.

Source classification. Ensuring a high signal-to-noise ratio is crucial for accurate sound classification. Installers need to consider the placement of cameras and microphones to avoid areas that may amplify background noise, which could skew the analytics. For example, while a corner might be an ideal location for video coverage, it can be a poor choice for audio due to an artificial amplification of background noise.

Making sense of alerts. Selecting a VMS that fully integrates with the camera’s API (application programming interface) is essential for capturing detailed audio analytic events. While standards like ONVIF also support audio analytics messages, advanced integration with VMS platforms can discern, categorize and search for audio-triggered events based on classification ID (i.e., glass break, car horn, gunshot, yell). It is important to ensure camera and VMS messaging handling methods are compatible.

Well-configured audio analytics can deliver an extra layer of situational awareness. They help validate what operators see on screen, allowing them to accelerate response times while providing detailed insights that go beyond traditional video surveillance.

When a separate purpose-built audio system is beyond the budget, modern AI-enabled cameras can step up and reduce the overall cost of installing purpose-built glass break sensors at every point of ingress.

By effectively addressing privacy concerns, audio analytics allow for the responsible utilization of audio capabilities in security cameras. i-PRO AI-enabled cameras, for example, feature customizable settings for audio classification type, sensitivity, and detection levels, ensuring superior performance across multiple installation environments. Pairing AI-enabled cameras with audio analytics with a compatible VMS is important to ensure success.

This article originally appeared in the July / August 2024 issue of Security Today.

Featured

  • Report: 47 Percent of Security Service Providers Are Not Yet Using AI or Automation Tools

    Trackforce, a provider of security workforce management platforms, today announced the launch of its 2025 Physical Security Operations Benchmark Report, an industry-first study that benchmarks both private security service providers and corporate security teams side by side. Based on a survey of over 300 security professionals across the globe, the report provides a comprehensive look at the state of physical security operations. Read Now

    • Guard Services

  • Identity Governance at the Crossroads of Complexity and Scale

    Modern enterprises are grappling with an increasing number of identities, both human and machine, across an ever-growing number of systems. They must also deal with increased operational demands, including faster onboarding, more scalable models, and tighter security enforcement. Navigating these ever-growing challenges with speed and accuracy requires a new approach to identity governance that is built for the future enterprise. Read Now

  • Eagle Eye Networks Launches AI Camera Gun Detection

    Eagle Eye Networks, a provider of cloud video surveillance, recently introduced Eagle Eye Gun Detection, a new layer of protection for schools and businesses that works with existing security cameras and infrastructure. Eagle Eye Networks is the first to build gun detection into its platform. Read Now

  • Report: AI is Supercharging Old-School Cybercriminal Tactics

    AI isn’t just transforming how we work. It’s reshaping how cybercriminals attack, with threat actors exploiting AI to mass produce malicious code loaders, steal browser credentials and accelerate cloud attacks, according to a new report from Elastic. Read Now

  • Pragmatism, Productivity, and the Push for Accountability in 2025-2026

    Every year, the security industry debates whether artificial intelligence is a disruption, an enabler, or a distraction. By 2025, that conversation matured, where AI became a working dimension in physical identity and access management (PIAM) programs. Observations from 2025 highlight this turning point in AI’s role in access control and define how security leaders are being distinguished based on how they apply it. Read Now

Most   Popular

New Products

  • PE80 Series

    PE80 Series by SARGENT / ED4000/PED5000 Series by Corbin Russwin

    ASSA ABLOY, a global leader in access solutions, has announced the launch of two next generation exit devices from long-standing leaders in the premium exit device market: the PE80 Series by SARGENT and the PED4000/PED5000 Series by Corbin Russwin. These new exit devices boast industry-first features that are specifically designed to provide enhanced safety, security and convenience, setting new standards for exit solutions. The SARGENT PE80 and Corbin Russwin PED4000/PED5000 Series exit devices are engineered to meet the ever-evolving needs of modern buildings. Featuring the high strength, security and durability that ASSA ABLOY is known for, the new exit devices deliver several innovative, industry-first features in addition to elegant design finishes for every opening.

  • Compact IP Video Intercom

    Viking’s X-205 Series of intercoms provide HD IP video and two-way voice communication - all wrapped up in an attractive compact chassis.

  • ResponderLink

    ResponderLink

    Shooter Detection Systems (SDS), an Alarm.com company and a global leader in gunshot detection solutions, has introduced ResponderLink, a groundbreaking new 911 notification service for gunshot events. ResponderLink completes the circle from detection to 911 notification to first responder awareness, giving law enforcement enhanced situational intelligence they urgently need to save lives. Integrating SDS’s proven gunshot detection system with Noonlight’s SendPolice platform, ResponderLink is the first solution to automatically deliver real-time gunshot detection data to 911 call centers and first responders. When shots are detected, the 911 dispatching center, also known as the Public Safety Answering Point or PSAP, is contacted based on the gunfire location, enabling faster initiation of life-saving emergency protocols.