Enhanced Situation Awareness

If a camera recognizes the sound signature, it simply issues an alert

Did someone break into the building? Maybe it is just an employee pulling an all-nighter. Or is it an actual perpetrator? Audio analytics, available in many AI-enabled cameras, can add context to what operators see on the screen, helping them validate assumptions. If a glass-break detection alert is received moments before seeing a person on camera, the added situational awareness makes the event more actionable.

In the world of security and video surveillance, the value of audio is often underestimated. While audio plays a pivotal role in intercom systems, its significance in broader security and event management contexts is frequently overlooked. This oversight occurs partly due to the privacy implications associated with audio surveillance, which is strictly regulated and varies significantly across jurisdictions. However, in-camera, or edge-processed, audio analytics that detects gunshots, yells, glass breaks and vehicle horns don’t require the audio to be recorded or captured in any way.

This avoids violating privacy laws because the audio is processed instantaneously at the edge and never leaves the camera. If the camera recognizes a known sound signature, it simply issues an alert. It is oblivious to all other sounds.

Audio analytics can also enhance situational awareness in areas where video is not allowed. For example, restrooms are a no-go area for cameras, but an analytic that detects glass breaks and yells can prevent such an area from being a complete blind spot.

The Power of Audio Analytics
Audio analytics, when processed directly within an AI-enabled camera, have emerged as a specialized niche’ for many security system installers and users. When a separate purpose-built audio system is beyond the budget, modern AI-enabled cameras can step up and do double duty, reducing the overall cost of installing purpose-built glass break sensors at every point of ingress.

Using deep learning algorithms, the cameras can provide a range of audio classification and detection at the edge, including glass breaks, gunshots, yells and even persistent vehicle horns.

Microphones
Modern IP-based surveillance cameras often come equipped with built-in microphones, though some models offer jacks for attaching external microphones. Indoor camera microphones are particularly effective due to their design, which allows sound waves to penetrate through small openings in the housing. Conversely, outdoor cameras, typically certified against water and dust ingress (IP66), may exhibit reduced sensitivity due to their sealed design.

In such cases, employing an external, strategically positioned microphone can greatly enhance the accuracy of audio analytics running outdoors. High-quality directional microphones, capable of mitigating wind noise, are recommended for critical audio data collection outdoors.

Any high-quality external microphone should easily outperform an internal microphone regarding analytic accuracy, so it is worth considering in areas where audio information gathering is crucial. AI sound classification is in the range of 200Hz to 8Khz, and the frequency distribution of a captured sound is an important characteristic during analysis. Therefore, a microphone must be able to pick up frequencies across this range with a flat or neutral characteristic.

AI SoCs Enhance Accuracy
Recent advancements have seen the introduction of surveillance cameras equipped with dedicated AI System on Chips (SoC), such as the Ambarella CV52. This chip can perform both video and audio analytics simultaneously.

Using an SoC allows for integrating advanced features, including a sound database against which audio from the scene is compared for real-time classification. Deep learning algorithms make these comparisons even more accurate. For example, when identifying a sound, an i-PRO camera compares the captured sound volume level with a preset threshold value. If it is greater than the threshold, AI is then used to determine what kind of sound it could be.

With the goal of creating an AI-derived similarity score, the system determines whether the captured sound corresponds to any of the four target sound categories: yell, glass break, vehicle horn, and gunshot.

This is done by dividing the sound into regular segments, performing signal processing, and extracting relevant features that can be used for analysis and comparison. An AI inference calculation uses machine learning algorithms to analyze the audio data and classify the audio data into distinct categories with a score based on similarity to the target sound. An alarm/notification is triggered when the similarity score exceeds a certain value.

Camera Configuration for Audio Analytics
Audio detection. Proper configuration begins with setting a camera to detect relevant sounds while ignoring irrelevant background noise. Since audio levels are typically louder in abnormal situations, cameras should be tailored to their specific environments, and sound level thresholds should be set only to flag audio levels suggestive of unusual activity.

AI-based audio analytics should be trained to identify target sounds under various conditions, such as situations with typical environmental noise or other non-target sounds and at different distances. This reduces the possibility of false positives caused by background noise.

Source classification. Ensuring a high signal-to-noise ratio is crucial for accurate sound classification. Installers need to consider the placement of cameras and microphones to avoid areas that may amplify background noise, which could skew the analytics. For example, while a corner might be an ideal location for video coverage, it can be a poor choice for audio due to an artificial amplification of background noise.

Making sense of alerts. Selecting a VMS that fully integrates with the camera’s API (application programming interface) is essential for capturing detailed audio analytic events. While standards like ONVIF also support audio analytics messages, advanced integration with VMS platforms can discern, categorize and search for audio-triggered events based on classification ID (i.e., glass break, car horn, gunshot, yell). It is important to ensure camera and VMS messaging handling methods are compatible.

Well-configured audio analytics can deliver an extra layer of situational awareness. They help validate what operators see on screen, allowing them to accelerate response times while providing detailed insights that go beyond traditional video surveillance.

When a separate purpose-built audio system is beyond the budget, modern AI-enabled cameras can step up and reduce the overall cost of installing purpose-built glass break sensors at every point of ingress.

By effectively addressing privacy concerns, audio analytics allow for the responsible utilization of audio capabilities in security cameras. i-PRO AI-enabled cameras, for example, feature customizable settings for audio classification type, sensitivity, and detection levels, ensuring superior performance across multiple installation environments. Pairing AI-enabled cameras with audio analytics with a compatible VMS is important to ensure success.

This article originally appeared in the July / August 2024 issue of Security Today.

Featured

  • New Report Reveals Top Trends Transforming Access Controller Technology

    Mercury Security, a provider in access control hardware and open platform solutions, has published its Trends in Access Controllers Report, based on a survey of over 450 security professionals across North America and Europe. The findings highlight the controller’s vital role in a physical access control system (PACS), where the device not only enforces access policies but also connects with readers to verify user credentials—ranging from ID badges to biometrics and mobile identities. With 72% of respondents identifying the controller as a critical or important factor in PACS design, the report underscores how the choice of controller platform has become a strategic decision for today’s security leaders. Read Now

  • Overwhelming Majority of CISOs Anticipate Surge in Cyber Attacks Over the Next Three Years

    An overwhelming 98% of chief information security officers (CISOs) expect a surge in cyber attacks over the next three years as organizations face an increasingly complex and artificial intelligence (AI)-driven digital threat landscape. This is according to new research conducted among 300 CISOs, chief information officers (CIOs), and senior IT professionals by CSC1, the leading provider of enterprise-class domain and domain name system (DNS) security. Read Now

  • ASIS International Introduces New ANSI-Approved Investigations Standard

    • Guard Services
  • Cloud Security Alliance Brings AI-Assisted Auditing to Cloud Computing

    The Cloud Security Alliance (CSA), the world’s leading organization dedicated to defining standards, certifications, and best practices to help ensure a secure cloud computing environment, today introduced an innovative addition to its suite of Security, Trust, Assurance and Risk (STAR) Registry assessments with the launch of Valid-AI-ted, an AI-powered, automated validation system. The new tool provides an automated quality check of assurance information of STAR Level 1 self-assessments using state-of-the-art LLM technology. Read Now

  • Report: Nearly 1 in 5 Healthcare Leaders Say Cyberattacks Have Impacted Patient Care

    Omega Systems, a provider of managed IT and security services, today released new research that reveals the growing impact of cybersecurity challenges on leading healthcare organizations and patient safety. According to the 2025 Healthcare IT Landscape Report, 19% of healthcare leaders say a cyberattack has already disrupted patient care, and more than half (52%) believe a fatal cyber-related incident is inevitable within the next five years. Read Now

New Products

  • FEP GameChanger

    FEP GameChanger

    Paige Datacom Solutions Introduces Important and Innovative Cabling Products GameChanger Cable, a proven and patented solution that significantly exceeds the reach of traditional category cable will now have a FEP/FEP construction.

  • A8V MIND

    A8V MIND

    Hexagon’s Geosystems presents a portable version of its Accur8vision detection system. A rugged all-in-one solution, the A8V MIND (Mobile Intrusion Detection) is designed to provide flexible protection of critical outdoor infrastructure and objects. Hexagon’s Accur8vision is a volumetric detection system that employs LiDAR technology to safeguard entire areas. Whenever it detects movement in a specified zone, it automatically differentiates a threat from a nonthreat, and immediately notifies security staff if necessary. Person detection is carried out within a radius of 80 meters from this device. Connected remotely via a portable computer device, it enables remote surveillance and does not depend on security staff patrolling the area.

  • AC Nio

    AC Nio

    Aiphone, a leading international manufacturer of intercom, access control, and emergency communication products, has introduced the AC Nio, its access control management software, an important addition to its new line of access control solutions.