Everything in Between

Edge computing: the evolution of video content analytics

Today, artificial intelligence and machine learning are backing a wide range of technologies and applications, powering diverse solutions to a broad set of challenges. For video analytics, deep learning has accelerated the technology’s evolution, particularly when it comes to accurate detection. AI-backed video analytics enable object extraction, recognition, classification, and indexing—activities that can advance various business and security applications by making video searchable, quantifiable and actionable.

This article will review the factors driving the adoption of deep learning-based video content analytics, the creation of more sophisticated cameras and higher resolution video and, ultimately, the resulting need to identify efficient data processing and computing solutions to support these changes.

Video-Based Alerting: From Computer Vision to the AI Age

When video analytics first emerged, products were primary designed as alerting solutions. Through triggering calls to action, these early solutions were attempting to eliminate the need for active human video monitoring. However, these computer vision-based solutions did not fully achieve the aim of removing human involvement in video surveillance and oversight: For one, these video alerting technologies tended to produce false positives and inaccurate matches for video search criteria.

An alternative approach adopted by other solutions was to maintain and maximize human involvement in the video surveillance process: These interactive video solutions didn’t focus on entirely removing human operators from surveillance monitoring but strived instead to accelerate video review for users and make it easier to understand whole scenes captured by video. While alert-based video monitoring yielded imprecise results, solutions that streamlined users’ comprehension of entire video scenes enabled operators to overcome video-based alerting limitations and quickly identify critical information in captured video.

The Renaissance of Alerting: New Innovations and New Challenges

The introduction of deep learning-backed video analytics revolutionized the video content analytics industry, driving more accurate detection capabilities and precise alerting. The demand for higher quality video analytics, among other considerations, catalyzed the development of more sophisticated cameras, as well as end user adoption of more cameras to optimize real-time alerting. These developments have enabled capabilities such as people counting and face recognition- based alerting: Higher resolution video makes it possible to more accurately distinguish between people in crowds and capture individual faces, which could then be analyzed by state-of-the-art analytics to trigger real-time alerts when certain conditions are met. Furthermore, beyond alerting, deep learning-driven solutions make it possible to leverage the valuable and powerful video metadata to drive deeper insight in other ways, such as business intelligence and trend visualizations.

While driving the deployment of real-time, deep learning-based alerting solutions, the proliferation of cameras with higher resolution video also drove up the total cost of ownership for video surveillance integrations. Specifically, these conditions entail higher processing demands and hardware requirements. For real-time video analytic solutions such as face recognition alerting, better accuracy drives up operating costs—a new challenge that must be overcome.

Lowering the Increasing Cost of Computing

Current video analytics research and development is focused on lowering the cost of processing. Whereas today’s deep learning driven video content analytics are mostly based on GPU computing, looking forward, solution innovators must consider continual improvements to camera technology, increasing availability and volumes of high-resolution video, and powerful, deep-learning-driven video analytics, and determine which processing model could best keep costs down: edge or centralized computing?

Because there are advantages to both options, it’s important to understand what makes edge processing and centralized computing respectively effective. Today’s leading solutions rely on centralized computing for several compelling reasons.

Flexible resource allocation. Today, organizations rely on large video surveillance installations with multiple cameras. At different times, each camera will have varying levels of activity, and by distributing processing with centralized computing, lagging can be prevented. Centralized computing is flexible, enabling the sharing of processing resources between cameras so that unusually high activity can be processed without slowing down computation across cameras. Statistically, relying on more video streams increases the likelihood of maintaining a steady state.

By contrast, edge computing is rigid, requiring pre-defining computation resources and scenario specifications, such as activity and resolution, which are dynamic conditions. When working with edge devices, users or developers must decide up front whether to provision for normal situations—in which case there is a risk of lagging and missed alerts during high activity scenarios—or for extreme situations—driving up costs, because resources are often idle and do not require the allocation of high processing resources.

When one camera is driving up activity and time is of the essence, overloading processing requirements could cause alerts to be delayed or missed when they matter most. By distributing the computing, lags in alerting can be prevented, timely processing is ensured, and lower processing costs are maintained. The ability to flexibly distribute processing with centralized computing is more beneficial to deployments with more cameras.

Broader coverage of analytic capabilities. On-camera analytics require pre-configuring the specific analysis activities for edge processing. Edge devices are typically designated for dedicated purposes, and the range of analytic activities that can be completed per device is limited. Because of device memory constraints, at the onset of deployment, the user will need to manually configure the relevant analytics based on the camera location. If the camera points to an area where faces can be viewed in high resolution, the device will likely be dedicated for face recognition, but not for license plate recognition (LPR) for fear of overbearing the processing load.

With centralized processing, there is no need for manual calibration. There is sufficient memory to share different Deep Neural Networks between video streams and cameras, so that when a person of interest on a pre-defined watchlist passes a dedicated LPR camera— capturing a high-resolution image – a call to action can still be triggered based on face recognition or other analytics, even if that wasn’t the dedicated purpose of that specific camera.

Shorter development cycle. By developing software that can be deployed on general purpose hardware—instead of developing the edge hardware itself—the end product is more broadly applicable, essentially shortening the development cycle.

The Main Drivers of On-Camera Analytics

Today, due to memory and computation limitations, on-camera analytics tend to be used for point solutions. Initially, this was limited to motion detection, but on camera analytics have evolved and can now identify and classify objects such as people and vehicles enabling advanced activity such as intruder detection, license plate recognition and people counting. The big question in the VCA industry today is whether edge devices will become sophisticated enough to enable general purpose identification, extraction, tracking and classification of all objects in the video. There are three main considerations driving development towards the edge.

Higher Demands for Real-Time Processing. Today, there is a higher volume of realtime data processing from a higher number of cameras. Because of these increasing demands, technology providers can justify the large initial investment in creating, marketing and distributing smarter cameras to meet the demand.

Deep learning driving AI chip development. Now that deep learning is considered standard for video analytics enablement leading hardware providers are developing dedicated AI chips. Since these chips only support specific instructions required for deep learning inference, they feature high efficiency, low energy consumption and small form factor.

Due to their flexibility, deep learning hardware solutions are enabling broad applications. Autonomous cars, for instance, rely on this type of hardware, transplanting the deep learning enabling hardware in the car itself, instead of in a centralized server center.

Lowering costs for decoding high-resolution video. To run video analytics, captured video must be transmitted to recording archives, live monitors or centralized processing servers, requiring significant bandwidth. By encoding video, solutions reduce transmission costs, but then face another obstacle: the work intensive demands of decoding higher resolution video, such as 4K.

A byproduct of processing video on the edge, circumventing video decoding ultimately reduces the computational requirements for processing the overwhelming amounts of high-quality video data.

By the time the video captured by the edge device is transferred to the centralized location, it is already processed and can be decoded as needed. For post-event investigation, for instance, only the video for the relevant time and camera ranges need to be decoded. Thus, the extraction of evidence isn’t inhibited even though the demands of decoding have been reduced.

Balancing the Benefits of Edge and Centralized Computing

At the onset of 2019, VCA industry predictions focused on pain points driving the shift towards edge processing and cloud computing— changes that will play a critical role in accelerating the adoption of advanced video content analytics. On camera analytics technologies are focused on transforming from point solutions to offering a complete analytics suite, including object tracking, classification and recognition. However, for the foreseeable future, centralized computing will remain critical for deriving comprehensive intelligence from edge devices. To enable cross-camera analytics, there must be a centralized computing service, aggregating insights from across cameras and feeds.

By overcoming decoding challenges, edge providers can drive enhanced operational reliability and processing speeds; reduced privacy risks by transmitting only encoded metadata; and, ultimately, accelerated migration to cloud-based solutions. When computing activities are limited to data and applications and not video processing, centralized cloud platforms become a more affordable option for running intensive video analytics, such as alerting, business intelligence, and video indexing and search.

This article originally appeared in the May/June 2019 issue of Security Today.

Featured

  • Allegion, Comfort Technologies Implement Mobile Credentials at the Artisan Apartment Homes in Florida

    Artisan Apartment Homes, a luxury apartment complex in Dunedin, Florida, recently transitioned from mechanical keys to electronic locks and centralized system software with support from Allegion US, a leading provider of security solutions, technology and services, and Florida-based Comfort Technologies, which specializes in deploying multifamily access control, IoT devices and software management solutions. Read Now

  • Mall of America Deploys AI-Powered Analytics to Enhance Parking Intelligence

    Mall of America®, the largest shopping and entertainment complex in North America, announced an expansion of its ongoing partnership with Axis Communications to deploy cutting-edge car-counting video analytics across more than a dozen locations. With this expansion, Mall of America (MOA) has boosted operational efficiency, improved safety and security, and enabled more informed decision-making around employee scheduling and streamlining transportation for large events. Read Now

  • Security Industry Association Launches New “askSIA” AI Tool

    The Security Industry Association (SIA) has unveiled a brand-new SIA member benefit – askSIA, a conversational AI agent designed to help users get the most out of their SIA membership, easily access SIA resources and find the latest information on SIA’s training and courses, reports and publications, events, certification offerings and more. SIA members can easily find askSIA by visiting the SIA homepage or looking for the askSIA icon in the top left of webpages. Read Now

    • Industry Events
  • Industry Embraces Mobile Access, Biometrics and AI

    A combination of evolving workplace dynamics, technology innovation and new user expectations is changing how people enter and interact with physical spaces. Access control is at the heart of these changes. Combined with biometrics and AI, mobile access control has become increasingly crucial for deploying entry solutions that are seamless, secure and adaptive to user needs. Read Now

  • Sustainable Video Solution Delivered for Landmark City of London Office Development

    An advanced, end-to-end video solution from IDIS, with a focus on reducing waste and costs, has helped a major office development in the City of London align its security with sustainability objectives. Read Now

New Products

  • Luma x20

    Luma x20

    Snap One has announced its popular Luma x20 family of surveillance products now offers even greater security and privacy for home and business owners across the globe by giving them full control over integrators’ system access to view live and recorded video. According to Snap One Product Manager Derek Webb, the new “customer handoff” feature provides enhanced user control after initial installation, allowing the owners to have total privacy while also making it easy to reinstate integrator access when maintenance or assistance is required. This new feature is now available to all Luma x20 users globally. “The Luma x20 family of surveillance solutions provides excellent image and audio capture, and with the new customer handoff feature, it now offers absolute privacy for camera feeds and recordings,” Webb said. “With notifications and integrator access controlled through the powerful OvrC remote system management platform, it’s easy for integrators to give their clients full control of their footage and then to get temporary access from the client for any troubleshooting needs.”

  • Connect ONE’s powerful cloud-hosted management platform provides the means to tailor lockdowns and emergency mass notifications throughout a facility – while simultaneously alerting occupants to hazards or next steps, like evacuation.

    Connect ONE®

    Connect ONE’s powerful cloud-hosted management platform provides the means to tailor lockdowns and emergency mass notifications throughout a facility – while simultaneously alerting occupants to hazards or next steps, like evacuation.

  • ResponderLink

    ResponderLink

    Shooter Detection Systems (SDS), an Alarm.com company and a global leader in gunshot detection solutions, has introduced ResponderLink, a groundbreaking new 911 notification service for gunshot events. ResponderLink completes the circle from detection to 911 notification to first responder awareness, giving law enforcement enhanced situational intelligence they urgently need to save lives. Integrating SDS’s proven gunshot detection system with Noonlight’s SendPolice platform, ResponderLink is the first solution to automatically deliver real-time gunshot detection data to 911 call centers and first responders. When shots are detected, the 911 dispatching center, also known as the Public Safety Answering Point or PSAP, is contacted based on the gunfire location, enabling faster initiation of life-saving emergency protocols.