Enhanced Situation Awareness

If a camera recognizes the sound signature, it simply issues an alert

  • By Rui Barbosa
  • Jul 24, 2024

Did someone break into the building? Maybe it is just an employee pulling an all-nighter. Or is it an actual perpetrator? Audio analytics, available in many AI-enabled cameras, can add context to what operators see on the screen, helping them validate assumptions. If a glass-break detection alert is received moments before seeing a person on camera, the added situational awareness makes the event more actionable.

In the world of security and video surveillance, the value of audio is often underestimated. While audio plays a pivotal role in intercom systems, its significance in broader security and event management contexts is frequently overlooked. This oversight occurs partly due to the privacy implications associated with audio surveillance, which is strictly regulated and varies significantly across jurisdictions. However, in-camera, or edge-processed, audio analytics that detects gunshots, yells, glass breaks and vehicle horns don’t require the audio to be recorded or captured in any way.

This avoids violating privacy laws because the audio is processed instantaneously at the edge and never leaves the camera. If the camera recognizes a known sound signature, it simply issues an alert. It is oblivious to all other sounds.

Audio analytics can also enhance situational awareness in areas where video is not allowed. For example, restrooms are a no-go area for cameras, but an analytic that detects glass breaks and yells can prevent such an area from being a complete blind spot.

The Power of Audio Analytics
Audio analytics, when processed directly within an AI-enabled camera, have emerged as a specialized niche’ for many security system installers and users. When a separate purpose-built audio system is beyond the budget, modern AI-enabled cameras can step up and do double duty, reducing the overall cost of installing purpose-built glass break sensors at every point of ingress.

Using deep learning algorithms, the cameras can provide a range of audio classification and detection at the edge, including glass breaks, gunshots, yells and even persistent vehicle horns.

Microphones
Modern IP-based surveillance cameras often come equipped with built-in microphones, though some models offer jacks for attaching external microphones. Indoor camera microphones are particularly effective due to their design, which allows sound waves to penetrate through small openings in the housing. Conversely, outdoor cameras, typically certified against water and dust ingress (IP66), may exhibit reduced sensitivity due to their sealed design.

In such cases, employing an external, strategically positioned microphone can greatly enhance the accuracy of audio analytics running outdoors. High-quality directional microphones, capable of mitigating wind noise, are recommended for critical audio data collection outdoors.

Any high-quality external microphone should easily outperform an internal microphone regarding analytic accuracy, so it is worth considering in areas where audio information gathering is crucial. AI sound classification is in the range of 200Hz to 8Khz, and the frequency distribution of a captured sound is an important characteristic during analysis. Therefore, a microphone must be able to pick up frequencies across this range with a flat or neutral characteristic.

AI SoCs Enhance Accuracy
Recent advancements have seen the introduction of surveillance cameras equipped with dedicated AI System on Chips (SoC), such as the Ambarella CV52. This chip can perform both video and audio analytics simultaneously.

Using an SoC allows for integrating advanced features, including a sound database against which audio from the scene is compared for real-time classification. Deep learning algorithms make these comparisons even more accurate. For example, when identifying a sound, an i-PRO camera compares the captured sound volume level with a preset threshold value. If it is greater than the threshold, AI is then used to determine what kind of sound it could be.

With the goal of creating an AI-derived similarity score, the system determines whether the captured sound corresponds to any of the four target sound categories: yell, glass break, vehicle horn, and gunshot.

This is done by dividing the sound into regular segments, performing signal processing, and extracting relevant features that can be used for analysis and comparison. An AI inference calculation uses machine learning algorithms to analyze the audio data and classify the audio data into distinct categories with a score based on similarity to the target sound. An alarm/notification is triggered when the similarity score exceeds a certain value.

Camera Configuration for Audio Analytics
Audio detection. Proper configuration begins with setting a camera to detect relevant sounds while ignoring irrelevant background noise. Since audio levels are typically louder in abnormal situations, cameras should be tailored to their specific environments, and sound level thresholds should be set only to flag audio levels suggestive of unusual activity.

AI-based audio analytics should be trained to identify target sounds under various conditions, such as situations with typical environmental noise or other non-target sounds and at different distances. This reduces the possibility of false positives caused by background noise.

Source classification. Ensuring a high signal-to-noise ratio is crucial for accurate sound classification. Installers need to consider the placement of cameras and microphones to avoid areas that may amplify background noise, which could skew the analytics. For example, while a corner might be an ideal location for video coverage, it can be a poor choice for audio due to an artificial amplification of background noise.

Making sense of alerts. Selecting a VMS that fully integrates with the camera’s API (application programming interface) is essential for capturing detailed audio analytic events. While standards like ONVIF also support audio analytics messages, advanced integration with VMS platforms can discern, categorize and search for audio-triggered events based on classification ID (i.e., glass break, car horn, gunshot, yell). It is important to ensure camera and VMS messaging handling methods are compatible.

Well-configured audio analytics can deliver an extra layer of situational awareness. They help validate what operators see on screen, allowing them to accelerate response times while providing detailed insights that go beyond traditional video surveillance.

When a separate purpose-built audio system is beyond the budget, modern AI-enabled cameras can step up and reduce the overall cost of installing purpose-built glass break sensors at every point of ingress.

By effectively addressing privacy concerns, audio analytics allow for the responsible utilization of audio capabilities in security cameras. i-PRO AI-enabled cameras, for example, feature customizable settings for audio classification type, sensitivity, and detection levels, ensuring superior performance across multiple installation environments. Pairing AI-enabled cameras with audio analytics with a compatible VMS is important to ensure success.

This article originally appeared in the July / August 2024 issue of Security Today.

Featured

  • Evolving Cybersecurity Strategies: Uniting Human Risk Management and Security Awareness Training

    Organizations are increasingly turning their attention to human-focused security approaches, as two out of three (68%) cybersecurity incidents involve people. Threat actors are shifting from targeting networks and systems to hacking humans via social engineering methods, living off human errors as their most prevalent attack vector. Whether manipulated or not, human cyber behavior is leveraged to gain backdoor access into systems. This mainly results from a lack of employee training and awareness about evolving attack techniques employed by malign actors. Read Now

  • Report: 1 in 3 Easily Exploitable Vulnerabilities Found on Cloud Assets

    CyCognito recently released new research highlighting critical security vulnerabilities across cloud-hosted assets, revealing that one in three easily exploitable vulnerabilities or misconfigurations are found on cloud assets. As organizations increasingly shift to multi-cloud strategies, the findings underscore significant security gaps that could provide attackers with potential footholds into networks. Read Now

  • Built for Today, Ready for Tomorrow

    Selecting the right VMS is critical for any organization that depends on video surveillance to ensure safety, security and operational efficiency. While many organizations focus on immediate needs such as budget and deployment size, let us review some of the long-term considerations that can significantly impact a VMS's utility and flexibility. Read Now

  • Paving the Way to Smart Buildings

    In today's rapidly evolving security landscape, the convergence of on-prem, edge and cloud technologies are critical. The physical security landscape is undergoing a profound transformation, driven by the rapid digitalization of buildings and the evolving needs of modern organizations. As the buildings sector pivots towards smart, AI and data-driven operations, the integration of both edge and cloud technology has become crucial. Read Now

  • The Cybersecurity Time Bomb

    If you work in physical security, you have probably seen it: a camera, access control system, or intrusion detection device installed years ago, humming along without a single update. It is a common scenario that security professionals have come to accept as "normal." But here is the reality: this mindset is actively putting organizations at risk. Read Now

Most   Popular

New Products

  • Unified VMS

    AxxonSoft introduces version 2.0 of the Axxon One VMS. The new release features integrations with various physical security systems, making Axxon One a unified VMS. Other enhancements include new AI video analytics and intelligent search functions, hardened cybersecurity, usability and performance improvements, and expanded cloud capabilities

  • A8V MIND

    A8V MIND

    Hexagon’s Geosystems presents a portable version of its Accur8vision detection system. A rugged all-in-one solution, the A8V MIND (Mobile Intrusion Detection) is designed to provide flexible protection of critical outdoor infrastructure and objects. Hexagon’s Accur8vision is a volumetric detection system that employs LiDAR technology to safeguard entire areas. Whenever it detects movement in a specified zone, it automatically differentiates a threat from a nonthreat, and immediately notifies security staff if necessary. Person detection is carried out within a radius of 80 meters from this device. Connected remotely via a portable computer device, it enables remote surveillance and does not depend on security staff patrolling the area.

  • AC Nio

    AC Nio

    Aiphone, a leading international manufacturer of intercom, access control, and emergency communication products, has introduced the AC Nio, its access control management software, an important addition to its new line of access control solutions.