Video Metadata: Describing the Details that Matter

  • By Joe Danielson
  • Dec 01, 2022

Modern surveillance systems generate an overwhelming (and mostly unused) amount of data. This is especially true when recording video in 24/7 operations, which is essential to capturing evidence, incidents and events. It is not only hard to pick out what really matters in a scene, but also extremely time consuming. Making data more identifiable and actionable is a key problem to solve. Applying metadata to describe key details in a scene allows data to be more identifiable and actionable.

This is why metadata is the foundation for gathering intelligence from surveillance video and/or audio streams. Metadata provides a fast way to find, evaluate, and act on the singular details that matter the most through one, hundreds or thousands of video and audio footage streams. Metadata is now an essential part of effective security and business operations.

What is Metadata?
Typically, Metadata is referred to as ‘data about other data.’ In the context of video surveillance, that translates to ‘data about video data’. Video metadata accurately describes the details that matter in a scene. For instance, attributes for metadata can describe all sort of details about moving objects of interest, e.g. location, time, colors, sizes, shapes, coordinates, volume decibels, speed, direction, etc.
Additionally, foundational details can be added, such as video stream description, codec, time stamps and device identity.

The aforementioned are ‘meta’ descriptions of details in, or related to, a scene. Based on AI machine and deep learning, Meta descriptions can be more (or less) granular. This allows for classifying a group of pixels as a person, animal, vehicle or other pre-defined object classes. Being more precise with more refined descriptions of people or objects e.g. vehicle type, make model, color, speed, direction, etc.

The Value of Metadata
Metadata not only provides details about people, objects and events in a scene. It also allows large amounts of video and recorded footage to quickly group, sort, search, recover and use. As a result, the overall use cases for metadata fit into three areas.

1. Real-time alarm triggering and notifications
2. Post event forensic searching
3. Statistical analysis and reporting

Adding Intelligence to Scenes
Metadata essentially assigns digital meaning to each video frame about the objects and events within it. In other words, it adds interpretation or intelligence about the scene rather than just the raw video footage, which needs to be processed manually by an operator.

Once software can interpret scenes in this way, it can understand the scene details and enable the scene to be acted upon in real-time via events, after events (post-event), via manual search or simply analyzed for statistical analysis. This enables the use of metadata to design baselines that define what is ‘normal’ for any scene feed from any individual camera. In turn, this allows software to recognize any degree of deviation, anomaly or specific behavior or activities, etc. as well as predict what will happen in that scene to a specific probability.

Harnessing the Full Potential of Metadata
Video metadata adds immense value to a video management system. In fact, its true potential is realized when applied to multiple inputs spanning visual, audio, activity, and process-related inputs. In the management of any site, things like RFID tracking, GPS coordinates, tampering alerts, noise detection, and point of sale transactional data, are all high value data sources. Unifying this metadata generated from many different sources means gaining much more insights than one can ever get from each system alone. Interoperability is key, and open-protocols and industry standards are essential to this effort. Ultimately seamless metadata integration will allow us to harness massive amounts of data from all kids of systems and gain a greater understanding of everything around us.

This article originally appeared in the November / December 2022 issue of Security Today.

About the Author

Joe Danielson, Global Enterprise Solutions, Axis Communications, AB.

Featured

  • The Future is Happening Outside the Cloud

    For years, the cloud has captivated the physical security industry. And for good reason. Remote access, elastic scalability and simplified maintenance reshaped how we think about deploying and managing systems. But as the number of cameras grows and resolutions push from HD to 4K and beyond, the cloud’s limits are becoming unavoidable. Bandwidth bottlenecks. Latency lags. Rising storage costs. These are not abstract concerns. Read Now

  • The Impact of Convergence Between IT and Physical Security

    For years, the worlds of physical security and information technology (IT) remained separate. While they shared common goals and interests, they often worked in silos. Read Now

  • Unlocking Trustworthy AI: Building Transparency in Security Governance

    In situations where AI supports important security tasks like leading investigations and detecting threats and anomalies, transparency is essential. When an incident occurs, investigators must trace the logic behind each automated response to confirm its validity or spot errors. Demanding interpretable AI turns opaque “black boxes” into accountable partners that enhance, rather than compromise, organizational defense. Read Now

  • Seeking Innovative Solutions

    Denial, Anger, Bargaining, Depression and Acceptance. You may recognize these terms as the “5 Phases” of a grieving process, but they could easily describe the phases one goes through before adopting any new or emerging innovation or technology, especially in a highly risk-averse industry like security. However, the desire for convenience in all aspects of modern life is finally beginning to turn the tide from old school hardware as the go-to towards more user-friendly, yet still secure, door solutions. Read Now

  • Where AI Meets Human Judgment

    Artificial intelligence is everywhere these days. It is driving business growth, shaping consumer experiences, and showing up in places most of us never imagined just a few years ago. Read Now

Most   Popular

New Products

  • QCS7230 System-on-Chip (SoC)

    QCS7230 System-on-Chip (SoC)

    The latest Qualcomm® Vision Intelligence Platform offers next-generation smart camera IoT solutions to improve safety and security across enterprises, cities and spaces. The Vision Intelligence Platform was expanded in March 2022 with the introduction of the QCS7230 System-on-Chip (SoC), which delivers superior artificial intelligence (AI) inferencing at the edge.

  • AC Nio

    AC Nio

    Aiphone, a leading international manufacturer of intercom, access control, and emergency communication products, has introduced the AC Nio, its access control management software, an important addition to its new line of access control solutions.

  • EasyGate SPT and SPD

    EasyGate SPT SPD

    Security solutions do not have to be ordinary, let alone unattractive. Having renewed their best-selling speed gates, Cominfo has once again demonstrated their Art of Security philosophy in practice — and confirmed their position as an industry-leading manufacturers of premium speed gates and turnstiles.