Enhanced Situation Awareness

If a camera recognizes the sound signature, it simply issues an alert

Did someone break into the building? Maybe it is just an employee pulling an all-nighter. Or is it an actual perpetrator? Audio analytics, available in many AI-enabled cameras, can add context to what operators see on the screen, helping them validate assumptions. If a glass-break detection alert is received moments before seeing a person on camera, the added situational awareness makes the event more actionable.

In the world of security and video surveillance, the value of audio is often underestimated. While audio plays a pivotal role in intercom systems, its significance in broader security and event management contexts is frequently overlooked. This oversight occurs partly due to the privacy implications associated with audio surveillance, which is strictly regulated and varies significantly across jurisdictions. However, in-camera, or edge-processed, audio analytics that detects gunshots, yells, glass breaks and vehicle horns don’t require the audio to be recorded or captured in any way.

This avoids violating privacy laws because the audio is processed instantaneously at the edge and never leaves the camera. If the camera recognizes a known sound signature, it simply issues an alert. It is oblivious to all other sounds.

Audio analytics can also enhance situational awareness in areas where video is not allowed. For example, restrooms are a no-go area for cameras, but an analytic that detects glass breaks and yells can prevent such an area from being a complete blind spot.

The Power of Audio Analytics
Audio analytics, when processed directly within an AI-enabled camera, have emerged as a specialized niche’ for many security system installers and users. When a separate purpose-built audio system is beyond the budget, modern AI-enabled cameras can step up and do double duty, reducing the overall cost of installing purpose-built glass break sensors at every point of ingress.

Using deep learning algorithms, the cameras can provide a range of audio classification and detection at the edge, including glass breaks, gunshots, yells and even persistent vehicle horns.

Microphones
Modern IP-based surveillance cameras often come equipped with built-in microphones, though some models offer jacks for attaching external microphones. Indoor camera microphones are particularly effective due to their design, which allows sound waves to penetrate through small openings in the housing. Conversely, outdoor cameras, typically certified against water and dust ingress (IP66), may exhibit reduced sensitivity due to their sealed design.

In such cases, employing an external, strategically positioned microphone can greatly enhance the accuracy of audio analytics running outdoors. High-quality directional microphones, capable of mitigating wind noise, are recommended for critical audio data collection outdoors.

Any high-quality external microphone should easily outperform an internal microphone regarding analytic accuracy, so it is worth considering in areas where audio information gathering is crucial. AI sound classification is in the range of 200Hz to 8Khz, and the frequency distribution of a captured sound is an important characteristic during analysis. Therefore, a microphone must be able to pick up frequencies across this range with a flat or neutral characteristic.

AI SoCs Enhance Accuracy
Recent advancements have seen the introduction of surveillance cameras equipped with dedicated AI System on Chips (SoC), such as the Ambarella CV52. This chip can perform both video and audio analytics simultaneously.

Using an SoC allows for integrating advanced features, including a sound database against which audio from the scene is compared for real-time classification. Deep learning algorithms make these comparisons even more accurate. For example, when identifying a sound, an i-PRO camera compares the captured sound volume level with a preset threshold value. If it is greater than the threshold, AI is then used to determine what kind of sound it could be.

With the goal of creating an AI-derived similarity score, the system determines whether the captured sound corresponds to any of the four target sound categories: yell, glass break, vehicle horn, and gunshot.

This is done by dividing the sound into regular segments, performing signal processing, and extracting relevant features that can be used for analysis and comparison. An AI inference calculation uses machine learning algorithms to analyze the audio data and classify the audio data into distinct categories with a score based on similarity to the target sound. An alarm/notification is triggered when the similarity score exceeds a certain value.

Camera Configuration for Audio Analytics
Audio detection. Proper configuration begins with setting a camera to detect relevant sounds while ignoring irrelevant background noise. Since audio levels are typically louder in abnormal situations, cameras should be tailored to their specific environments, and sound level thresholds should be set only to flag audio levels suggestive of unusual activity.

AI-based audio analytics should be trained to identify target sounds under various conditions, such as situations with typical environmental noise or other non-target sounds and at different distances. This reduces the possibility of false positives caused by background noise.

Source classification. Ensuring a high signal-to-noise ratio is crucial for accurate sound classification. Installers need to consider the placement of cameras and microphones to avoid areas that may amplify background noise, which could skew the analytics. For example, while a corner might be an ideal location for video coverage, it can be a poor choice for audio due to an artificial amplification of background noise.

Making sense of alerts. Selecting a VMS that fully integrates with the camera’s API (application programming interface) is essential for capturing detailed audio analytic events. While standards like ONVIF also support audio analytics messages, advanced integration with VMS platforms can discern, categorize and search for audio-triggered events based on classification ID (i.e., glass break, car horn, gunshot, yell). It is important to ensure camera and VMS messaging handling methods are compatible.

Well-configured audio analytics can deliver an extra layer of situational awareness. They help validate what operators see on screen, allowing them to accelerate response times while providing detailed insights that go beyond traditional video surveillance.

When a separate purpose-built audio system is beyond the budget, modern AI-enabled cameras can step up and reduce the overall cost of installing purpose-built glass break sensors at every point of ingress.

By effectively addressing privacy concerns, audio analytics allow for the responsible utilization of audio capabilities in security cameras. i-PRO AI-enabled cameras, for example, feature customizable settings for audio classification type, sensitivity, and detection levels, ensuring superior performance across multiple installation environments. Pairing AI-enabled cameras with audio analytics with a compatible VMS is important to ensure success.

This article originally appeared in the July / August 2024 issue of Security Today.

Featured

  • The Impact of Convergence Between IT and Physical Security

    For years, the worlds of physical security and information technology (IT) remained separate. While they shared common goals and interests, they often worked in silos. Read Now

  • Unlocking Trustworthy AI: Building Transparency in Security Governance

    In situations where AI supports important security tasks like leading investigations and detecting threats and anomalies, transparency is essential. When an incident occurs, investigators must trace the logic behind each automated response to confirm its validity or spot errors. Demanding interpretable AI turns opaque “black boxes” into accountable partners that enhance, rather than compromise, organizational defense. Read Now

  • Seeking Innovative Solutions

    Denial, Anger, Bargaining, Depression and Acceptance. You may recognize these terms as the “5 Phases” of a grieving process, but they could easily describe the phases one goes through before adopting any new or emerging innovation or technology, especially in a highly risk-averse industry like security. However, the desire for convenience in all aspects of modern life is finally beginning to turn the tide from old school hardware as the go-to towards more user-friendly, yet still secure, door solutions. Read Now

  • Where AI Meets Human Judgment

    Artificial intelligence is everywhere these days. It is driving business growth, shaping consumer experiences, and showing up in places most of us never imagined just a few years ago. Read Now

  • Report: Only 44 Percent of Organizations are Fully Equipped to Support Secure AI

    Delinea recently published new research on the impact of artificial intelligence in reshaping identity security. According to the report, “AI in Identity Security Demands a New Playbook,” only 44% of organizations say their security architecture is fully equipped to support secure AI, despite widespread confidence in their current capabilities. Read Now

New Products

  • Mobile Safe Shield

    Mobile Safe Shield

    SafeWood Designs, Inc., a manufacturer of patented bullet resistant products, is excited to announce the launch of the Mobile Safe Shield. The Mobile Safe Shield is a moveable bullet resistant shield that provides protection in the event of an assailant and supplies cover in the event of an active shooter. With a heavy-duty steel frame, quality castor wheels, and bullet resistant core, the Mobile Safe Shield is a perfect addition to any guard station, security desks, courthouses, police stations, schools, office spaces and more. The Mobile Safe Shield is incredibly customizable. Bullet resistant materials are available in UL 752 Levels 1 through 8 and include glass, white board, tack board, veneer, and plastic laminate. Flexibility in bullet resistant materials allows for the Mobile Safe Shield to blend more with current interior décor for a seamless design aesthetic. Optional custom paint colors are also available for the steel frame.

  • HD2055 Modular Barricade

    Delta Scientific’s electric HD2055 modular shallow foundation barricade is tested to ASTM M50/P1 with negative penetration from the vehicle upon impact. With a shallow foundation of only 24 inches, the HD2055 can be installed without worrying about buried power lines and other below grade obstructions. The modular make-up of the barrier also allows you to cover wider roadways by adding additional modules to the system. The HD2055 boasts an Emergency Fast Operation of 1.5 seconds giving the guard ample time to deploy under a high threat situation.

  • 4K Video Decoder

    3xLOGIC’s VH-DECODER-4K is perfect for use in organizations of all sizes in diverse vertical sectors such as retail, leisure and hospitality, education and commercial premises.