Keys to the Kingdom
Unlocking the secrets of video analytics means ability to leverage technology, improve productivity
- By Steve Goldberg
- Aug 02, 2007
EMERGING robust video analytics in a broad range of traditional physical security applications highlights the impact of technology in today's world. It's no longer enough to hire a few well-placed security guards. Today, it is all about having the ability to leverage technology to improve productivity. It's one thing to have more than 1,000 security cameras guarding a given location. It's quite another to guarantee that someone is watching, or will watch, the live or recorded video 24 hours a day, seven days a week.
In most cases, people's perception of security is just that: a perception, not a reality. Video analytics, also know as intelligent video, has burst onto the high-technology marketplace over the past few years to address and resolve this and other related challenges. Markets currently served by VA include three general video surveillance categories: infrastructure security, personal safety and retail sales and marketing research.
Escalating global video surveillance needs, both in government and the private sector, have raised the bar for robust performance and reliability of intelligent VA algorithms. Modern VA and the systems used to support it are now available to a much wider range of organizations. They are no longer benefits solely enjoyed by well-funded governments and multinational corporations.
What are Video Analytics?
Video analytics are often a component in a larger system, including cameras, DVRs and NVRs; encoding and video compression engines; and video management and networking equipment. The analytics themselves are deployed server-based in a central location and embedded in either cameras or DVRs/NVRs. Video analytics have historically been considered a software product. Recent product announcements and system designs now call for a broader definition, including embedded systems and standalone VA appliances.
Video analytics software is driven by advanced algorithms designed to expertly evaluate real-world situations, commonly known as behaviors. This may include the analysis of basic events such as unattended objects left in hotel lobbies and unattended vehicles in parking garages, all the way to sophisticated interactions between people, such as tailgating.
More than just behavior recognition, the software optimizes the performance of behaviors, manages addition and removal of behaviors, stores video clips, sends alerts and logs activities. Combining results from various algorithms into "the big picture" is like fitting pieces of a jigsaw puzzle. A single camera, or channel, will often have a number of behaviors being processed simultaneously to meet a given requirement.
Intelligent Video Analytics Limitations
Fundamentally, video analytics are designed to be an extension of a person's own vision. Not surprisingly, the technology is sometimes referred to as machine vision or robot vision. A key goal in the design of any vision system is to maximize the probability of detection,and minimizing the probability of false alarm. In most algorithms used in the VA industry, maximizing probability of detection also will raise the probability of false alarm. Successful VA algorithms provide enough flexibility to set an acceptable level of false alarms while providing a given threshold of detection performance. There is no single right answer to how well the system will perform.
Advantage of Video Analytics
There are several reasons why VA is primed to soon become widely accepted in a variety of fields:
Forensics in applications. To enhance the use of video files in legal proceedings, video analytics can provide added information in the form of reliable date and time stamps and other behavior-description metadata. Law enforcement agencies can benefit from rapid searches of video, using these "behavior tags" during investigations.
Technology improvements are here. Analyzing multiple VA behaviors in real time—not just the fast linear processing of one task after another—is critical to accurately interpreting video content. The steady improvement of processing power, driven by the semiconductor industry's tracking of Moore's Law, has brought sophisticated VA to the technology mainstream. New network processors and DSPs built on powerful multi-processor and parallelized architectures have opened new opportunities for growth in the VA industry.
The power of standardization. New approaches to standardized programming, such as metadata tagging in common XML, will allow security networks to connect any outside security network, appliance or device authorized to join a given network. Several large companies, including Cisco and IBM, are now involved with the retail industry to begin to set video analytics and content analysis metadata standards.
Distributed intelligence across networks. Intelligent video routers, which could become the next major security platform, are now coming to the market. The IVR, located at the edge of a security network, can interface with cameras, route live video and alerts to central servers and storage devices, and interface with access control systems. Perhaps more importantly, an IP-based IVR can provide a real-time interface between a physical security network and the now ever-present IT network. The convergence of these two currently disparate security networks will usher in a new and exciting security administration landscape.
Acknowledgment of value. Few will deny that intrinsic and economic values are created through saving human lives and building global economies free of terror. Benefits derived from developing VA algorithms into powerful software tools to achieve these noble objectives are often not factored into the value of VA software. Mass deployment of VA technologies will certainly drive market valuations of companies that manufacture, market and deploy VA products. The ability to thwart a single act of terrorism could save a multitude of business and civic interests on local, national and even international levels. |
The wide adoption of video analytics will be delayed until users acknowledge and accept this. The fundamental complexity of the algorithms, the historical hardware cost of the platforms to run the algorithms and the underlying camera/video storage system also have delayed the adoption of VA. The expectation for these complex algorithms lies squarely on the border of science fiction and science fact. On one hand, video analytics provide the opportunity to enhance legacy physical security systems. On the other hand, today's technology is years away from creating robots that can see and evaluate the physical world in the way R2-D2 or C-3PO did in the well-known "Star Wars" trilogy.
Understanding this Achilles heel of intelligent video analytics is the key to its ultimate adoption. There are major implications to teaching a computer to see and understand what a human would see and understand. The same false alarms and missed detections that people deal with every day with human security would have to be somehow designed out of the analytic software.
For example, if someone attempts to program a computer to identify an unattended object—one that doesn't move for a specified period of time—that person has to recognize that the computer will make the same associations that people would. Watching people at an airport alternately pick up and put down luggage is a perfect scenario to present the challenge for VA. How long should a VA algorithm, watching a given piece of luggage, wait for it to be deemed unattended? The reality is that even the best-trained human security guards often make mistakes with this scenario.
Fortunately, many VA behaviors operate successfully on much more binary situations, including whether or not an object or a human crosses a perimeter, people counting, reverse directional flow—walking the wrong direction in a queue—and tracking objects.
The lack of performance standards in measuring the actual performance of video analytics applications is another major hurdle to its adoption. Most high-technology industries begin to grow when the industry itself sets standards of performance. In a natural evolution of technology, media and independent test organizations emerge and provide test results of one product versus another and comparisons of vendor price/performance trade-offs. Additionally, standard interface protocols—message and alert definitions—are generally required for interoperability between products in a complex system. Without these standards, end users are often confused with conflicting marketing messages and freeze buying decisions.
The overall limited interoperability of physical security devices has slowed the use of IP-based devices, including VA. On a micro level, one can consider the shortcomings of security cameras with nonstandard input and output protocols requiring unique driver or interface software to be written for each application and product integration. On a macro level, interoperability could enable global monitoring centers to join forces during a major act of terrorism. Suspected terrorists might be followed from ground camera, to satellite camera to ground camera as they move. Exact GPS coordinates of each individual terrorist, identified with facial recognition software through satellite cameras powerful enough to read a license plate, will allow law enforcement and military to target exact positions.
Applications that Matter
Traditionally, security guards watch multiple cameras with multiple camera views. Proper implementation of video analytics applications, with enough powerful hardware and reliable behavior algorithms, can increase productivity without increasing security staff.
A system employing VA can monitor hundreds of cameras simultaneously and supply only alerts and relevant video to a security guard for specific situations requiring their attention. Security alerts can be sent via cell phones, pagers and e-mail to security personnel, identifying the location and suspected significant activity.
Safety in Retail Establishments
Analyzing behavior in the context of the camera's field of view or region of interest allows video analytics to identify unique behavior in retail establishments. If a customer slips and falls, the VA application can trigger an alert. A security guard in a warehouse might receive a unique loud beep and an alert on a pager that reads: "Possible customer injury: slip/fall. Location: Isle 9A." A secondary alert may contact 911 if the person who has fallen does not get up and lies unattended for more than a specified period of time.
Analyzing behavior from one camera to the next in a chain may allow tracking of an item's RFID tag from the shelf to a car. If a check-out scanner did not process the item, a parking lot camera may zoom in and save video footage of the individual, car and license plate.
Technology Mainstream
There is no question that in the past several years, video, video processing and video applications have entered the technology mainstream. The growing complexity of modern world demands constant improvement in human productivity to keep pace. Video analytics provides this productivity improvement across a wide range of applications. Video analytics, not unlike most technology products, has gone through the well-known cycle where early adopters face less than perfect performance. Based on a natural evolution of the marketplace, video analytics with certainty will enter the security system mainstream in the very near future.