Enhanced Situation Awareness

If a camera recognizes the sound signature, it simply issues an alert

Did someone break into the building? Maybe it is just an employee pulling an all-nighter. Or is it an actual perpetrator? Audio analytics, available in many AI-enabled cameras, can add context to what operators see on the screen, helping them validate assumptions. If a glass-break detection alert is received moments before seeing a person on camera, the added situational awareness makes the event more actionable.

In the world of security and video surveillance, the value of audio is often underestimated. While audio plays a pivotal role in intercom systems, its significance in broader security and event management contexts is frequently overlooked. This oversight occurs partly due to the privacy implications associated with audio surveillance, which is strictly regulated and varies significantly across jurisdictions. However, in-camera, or edge-processed, audio analytics that detects gunshots, yells, glass breaks and vehicle horns don’t require the audio to be recorded or captured in any way.

This avoids violating privacy laws because the audio is processed instantaneously at the edge and never leaves the camera. If the camera recognizes a known sound signature, it simply issues an alert. It is oblivious to all other sounds.

Audio analytics can also enhance situational awareness in areas where video is not allowed. For example, restrooms are a no-go area for cameras, but an analytic that detects glass breaks and yells can prevent such an area from being a complete blind spot.

The Power of Audio Analytics
Audio analytics, when processed directly within an AI-enabled camera, have emerged as a specialized niche’ for many security system installers and users. When a separate purpose-built audio system is beyond the budget, modern AI-enabled cameras can step up and do double duty, reducing the overall cost of installing purpose-built glass break sensors at every point of ingress.

Using deep learning algorithms, the cameras can provide a range of audio classification and detection at the edge, including glass breaks, gunshots, yells and even persistent vehicle horns.

Microphones
Modern IP-based surveillance cameras often come equipped with built-in microphones, though some models offer jacks for attaching external microphones. Indoor camera microphones are particularly effective due to their design, which allows sound waves to penetrate through small openings in the housing. Conversely, outdoor cameras, typically certified against water and dust ingress (IP66), may exhibit reduced sensitivity due to their sealed design.

In such cases, employing an external, strategically positioned microphone can greatly enhance the accuracy of audio analytics running outdoors. High-quality directional microphones, capable of mitigating wind noise, are recommended for critical audio data collection outdoors.

Any high-quality external microphone should easily outperform an internal microphone regarding analytic accuracy, so it is worth considering in areas where audio information gathering is crucial. AI sound classification is in the range of 200Hz to 8Khz, and the frequency distribution of a captured sound is an important characteristic during analysis. Therefore, a microphone must be able to pick up frequencies across this range with a flat or neutral characteristic.

AI SoCs Enhance Accuracy
Recent advancements have seen the introduction of surveillance cameras equipped with dedicated AI System on Chips (SoC), such as the Ambarella CV52. This chip can perform both video and audio analytics simultaneously.

Using an SoC allows for integrating advanced features, including a sound database against which audio from the scene is compared for real-time classification. Deep learning algorithms make these comparisons even more accurate. For example, when identifying a sound, an i-PRO camera compares the captured sound volume level with a preset threshold value. If it is greater than the threshold, AI is then used to determine what kind of sound it could be.

With the goal of creating an AI-derived similarity score, the system determines whether the captured sound corresponds to any of the four target sound categories: yell, glass break, vehicle horn, and gunshot.

This is done by dividing the sound into regular segments, performing signal processing, and extracting relevant features that can be used for analysis and comparison. An AI inference calculation uses machine learning algorithms to analyze the audio data and classify the audio data into distinct categories with a score based on similarity to the target sound. An alarm/notification is triggered when the similarity score exceeds a certain value.

Camera Configuration for Audio Analytics
Audio detection. Proper configuration begins with setting a camera to detect relevant sounds while ignoring irrelevant background noise. Since audio levels are typically louder in abnormal situations, cameras should be tailored to their specific environments, and sound level thresholds should be set only to flag audio levels suggestive of unusual activity.

AI-based audio analytics should be trained to identify target sounds under various conditions, such as situations with typical environmental noise or other non-target sounds and at different distances. This reduces the possibility of false positives caused by background noise.

Source classification. Ensuring a high signal-to-noise ratio is crucial for accurate sound classification. Installers need to consider the placement of cameras and microphones to avoid areas that may amplify background noise, which could skew the analytics. For example, while a corner might be an ideal location for video coverage, it can be a poor choice for audio due to an artificial amplification of background noise.

Making sense of alerts. Selecting a VMS that fully integrates with the camera’s API (application programming interface) is essential for capturing detailed audio analytic events. While standards like ONVIF also support audio analytics messages, advanced integration with VMS platforms can discern, categorize and search for audio-triggered events based on classification ID (i.e., glass break, car horn, gunshot, yell). It is important to ensure camera and VMS messaging handling methods are compatible.

Well-configured audio analytics can deliver an extra layer of situational awareness. They help validate what operators see on screen, allowing them to accelerate response times while providing detailed insights that go beyond traditional video surveillance.

When a separate purpose-built audio system is beyond the budget, modern AI-enabled cameras can step up and reduce the overall cost of installing purpose-built glass break sensors at every point of ingress.

By effectively addressing privacy concerns, audio analytics allow for the responsible utilization of audio capabilities in security cameras. i-PRO AI-enabled cameras, for example, feature customizable settings for audio classification type, sensitivity, and detection levels, ensuring superior performance across multiple installation environments. Pairing AI-enabled cameras with audio analytics with a compatible VMS is important to ensure success.

This article originally appeared in the July / August 2024 issue of Security Today.

Featured

  • Pragmatism, Productivity, and the Push for Accountability in 2025-2026

    Every year, the security industry debates whether artificial intelligence is a disruption, an enabler, or a distraction. By 2025, that conversation matured, where AI became a working dimension in physical identity and access management (PIAM) programs. Observations from 2025 highlight this turning point in AI’s role in access control and define how security leaders are being distinguished based on how they apply it. Read Now

  • Report: Cyber Attackers Continue to Turn to AI-Based Tools to Avoid Detection

    Comcast Business recently released its 2025 Cybersecurity Threat Report, a comprehensive analysis of 34.6 billion cybersecurity events detected between June 1,2024 and May 31, 2025. Now in its third year, the report offers business leaders a unique perspective into the evolving threat landscape and provides actionable insights to help organizations strengthen their defenses and align cybersecurity with business risk. Read Now

  • Axis Communications Creates AI-powered Video Surveillance Orchestra

    What if cameras could not only see the world, but interpret it—and respond like orchestra musicians reading sheet music: instantly, precisely, and in perfect harmony? That’s what global network technology leader Axis Communications set to find out. Read Now

  • Just as Expected

    GSX produced a wonderful tradeshow earlier this week. Monday was surprisingly strong in the morning, and the afternoon wasn’t bad at all. That’s Monday’s results and asking attendees to travel on Sunday. Just a quick hint, no one wants to give up their weekend to travel and set up an exhibit booth. I’m just saying. Read Now

    • Industry Events
    • GSX
  • NOLA: The Crescent City

    Twenty years later we finds ourselves in New Orleans. Twenty years ago the aftermath of Hurricane Katrina forced exhibitors and attendees to look elsewhere for tradeshow floor space. Read Now

    • Industry Events
    • GSX

New Products

  • AC Nio

    AC Nio

    Aiphone, a leading international manufacturer of intercom, access control, and emergency communication products, has introduced the AC Nio, its access control management software, an important addition to its new line of access control solutions.

  • Camden CV-7600 High Security Card Readers

    Camden CV-7600 High Security Card Readers

    Camden Door Controls has relaunched its CV-7600 card readers in response to growing market demand for a more secure alternative to standard proximity credentials that can be easily cloned. CV-7600 readers support MIFARE DESFire EV1 & EV2 encryption technology credentials, making them virtually clone-proof and highly secure.

  • HD2055 Modular Barricade

    Delta Scientific’s electric HD2055 modular shallow foundation barricade is tested to ASTM M50/P1 with negative penetration from the vehicle upon impact. With a shallow foundation of only 24 inches, the HD2055 can be installed without worrying about buried power lines and other below grade obstructions. The modular make-up of the barrier also allows you to cover wider roadways by adding additional modules to the system. The HD2055 boasts an Emergency Fast Operation of 1.5 seconds giving the guard ample time to deploy under a high threat situation.