Securing the City

How a new generation of video analytics is empowering the next generation of superheroes

• Aug 01, 2017

It happens every day. In the blink of an eye, a car accident can happen, gunshots go off, or an innocent bystander is assaulted. Thanks to the brave women and men who don their uniforms every day and put everything on the line to protect and serve the public, we can all worry a little less about the dangers of the world. But while these people are superheroes in their own right, they don’t possess superhuman capabilities—at least not yet, anyway.

Thanks to new advancements in video analytics technology and powerful Internet of Things (IoT) edge devices; these superheroes can now be armed with new superhuman capabilities to better protect us.

The Journey to a Smarter City

Sight is one of our first lines of defense and awareness is key. We look both ways before we cross the street. We avoid nefarious or aggressive- looking strangers. We keep a vigilant watch on those we care for when we’re out in public.

But our ability to leverage sight to keep us safe is limited.

When a crisis occurs, we must rely on our memories and limited field of vision to piece together who, what, where and why. The pieces can often feel like they don’t fit together or some may be missing entirely. As a result, devising how to respond to an emergency or investigate a crime becomes more reliant on intuition than cold hard facts.

Enter video. In 1951, the first Video Tape Recorder (VTR) was used to record live images from a television camera. By the mid-1960s closed-circuit video surveillance that could remote PTZ to monitor public spaces was becoming more and more common across cities. The technology enabled authorities to watch crowds for suspicious activities, take a closer look at crime incidents to better investigate cases and keep an eye on public events where crowds might become a target for threats. The capabilities, while limited, set the foundation for the continued use of video within the field of security and public safety.

Computer Vision Powers Superhuman Intelligence

With the progression of technologies like artificial intelligence, machine-learning and computer vision, video is quickly becoming more than an eye in the sky. Advanced analytics capabilities are now transforming video into IoT data to provide rich insights, offer realtime alerts and enhance safety like never before. These insights help us respond faster, act proactively to solve or prevent crimes, and help us make plans that make us more effective in the future—much like a superhero.

Until recently, the long-held industry standard for video analytics was pixel color recognition. The software would analyze individual pixels, or a portion of the pixels, to identify suspicious objects and people and an alert would then be sent so security teams could investigate or respond to the threat. The challenge with this type of technology is they could be easily tricked into falsely identifying an animal or object as a suspicious person.

For example, an alert that someone had entered a highly-restricted area monitored by an outdoor camera that could easily be triggered by a squirrel or raccoon that came into the camera’s field of view. The software would simply detect a change in pixel movement or colors and then trigger an alert— there was no intelligence behind it. All the algorithm knew was that something new was in the scene, but the ability to identify whether the intrusion was human, assess its threat level and determine if it warranted of action was missing.

Early video analytics technologies also created many false positives due to adverse weather conditions, like snow, wind and other daily environmental evets, which also triggered an inordinate amount of false positive alerts. People tend to start ignoring the alerts that cry wolf, essentially making the technology useless. These types of false positives would waste valuable time and resources, or in many cases, caused security personnel to ignore alerts. This ultimately slowed market adoption of video analytics. Video analytics that are useful need to provide critical situational awareness in all environmental conditions—even adverse weather—and ignore irrelevant intruders like squirrels or the neighborhood cat.

Today, computer vision, machine learning and artificial intelligence are dramatically lowering the rate of false positives and advancing the usefulness of video analytics. Rather than just seeing moving pixels, advanced video analytics can create 4-D reconstruction of two-dimensional video images (3-D plus time). Now, not only are objects able to be detected using particles, perspective, velocity, path deviation and travel distance, they can be accurately identified and in some cases classified through artificial intelligence. Security personnel can leverage these advancements to better understand if a person or suspicious object has been detected.

When a squirrel runs through a video feed, advanced video analytics capture multiple angles of the animal. On the back-end, it takes those two-dimensional images and reconstructs them to build a 4D version of the squirrel. The object is now identifiable against a database of known images—each with an assigned level of potential danger. The video system can not only see an object, it understands what that object means in the context of the area being monitored: a squirrel is only a squirrel, not a trespasser that might be armed and dangerous.

Unfortunately, not all objects are as innocent as a passing squirrel.

How many times have you been at the airport and heard the loud speaker come on reminding passengers to keep their luggage close and report any unattended or suspicious items? That’s because airports are a constant point of public safety concern. With thousands of people shuffling between gates and destinations—luggage and bags in hand—any passenger could be a threat or could have just forgotten their luggage in the confusion of travel.

So, how do you keep people safe when there is so much going on? Arm them with enhanced awareness, security intelligence, and the ability to better allocate resources.

Say a bag has been left unattended. With today’s advanced video analytics, the object would be recognized as a bag as soon as it enters into the purview of a video camera— then the clock starts ticking. Depending on the procedures in place at the airport, staff would be alerted once the threshold of designated time had passed that there was an object that has not moved or interacted with a passenger for longer than the allocated time. The video system could then send all first responders, in this case airport security, a realtime alert to their mobile devices, notifying them a bag has been abandoned, its location and how long it’s been there. Now everyone is on alert and the security team can immediately determine who should investigate based on geo-location. A plan of action can be formed in real-time, based on real insights.

If the situation goes beyond a passenger taking too long in the restroom, advanced video analytics can immediately arm investigators with the situational awareness they need to begin an investigation, while continuing to keep everyone safe. Through facial recognition, the software can also identify all the people who have come in contact with the bag—going so far as to backtrack all their steps from the point of abandoning the bag. Leveraging the airport’s parking lot cameras, a suspect’s car and license plate can then be identified. With a visual of a suspect and vehicle information in place, law officials can begin to investigate any criminal connections and evaluate if the bag might pose a bomb-level-threat worthy of evacuation.

While this might be an extreme scenario, the reality is that when a major event or everyday situations occur, we must prepare to stop them—and advancements in computer vision, artificial intelligence and analytics have been helping law enforcement and operations teams tackle these challenges more and more. For example, video analytics can also detect when lines become too long and alert staff to open another ticket counter, security gate, or passport stand in order to help people get from the door to their gate as quickly and conveniently as possible.

Smart Cities Fuel These Superpowers

For complex organizations like governments and cities that produce massive amounts of data, video, while valuable, is greatly enhanced when integrated within a broader smart city ecosystem. By connecting disparate video, data and public safety systems on a single platform, public safety officials can receive real-time alerts backed by the most up to date insights, map and predict crime to deploy resources and strategize the best approach to a crime that’s in progress or how to prevent the next one—maximizing the level of safety. Take the airport scenario. While video analytics may help set up police officers for success in identifying a suspect, integrating the system with a broader smart city platform will expedite results. Rather than having to wait for a suspect’s image to be located in a law enforcement database and try to find a license plate in another, an integrated platform can unify all data into a single platform, and allow all functions to be displayed simultaneously from a single command center, or even from a mobile device inside an officer’s vehicle. Similarly, integrating advanced video analytics across a city-wide platform would allow a suspect and any accomplices to be identified and tracked from the time they leave the airport to the time they reach their “hideout” location, enabling law enforcement to more swiftly apprehend and bring them in for questioning. This type of legwork automation instantly streamlines investigations and empowers the type of real- time response that’s needed in dangerous and emergency situations.

In a smart city, you wouldn’t deploy a single taskforce to find an abducted child— you’d deploy a technology-enhanced army. The first 24 hours in child abductions are critical and with video analytics an entire city can be searched in real time, providing leads in minutes or hours instead of days. With an integrated alert system police and the public can be notified in real-time to be on the look-out for the child. Amber Alert systems are already in place across the U.S., but the possibilities with video analytics for a more comprehensive and efficient response are tremendous, especially when integrating public and private cameras in the community through public-private partnerships.

Leveraging real-time social media, geolocational data and public video feeds in an integrated system could help them identify the location of a missing child faster. Upload the photos of the suspected kidnapper and missing child into the system and the video analytics could enable officials to quickly scan across public spaces to locate where they had been most recently recognized. Once the child’s location is identified, remote video analysis could help police assess any danger they might be in. The reconstruction of multiple video shots could help to detect whether the abductor is armed to inform a better, safer rescue strategy and help officers with realtime situational awareness when they close in on the suspect and rescue the child.

A New Type of Metropolis

These types of integrated city platforms aren’t just imaginary scenarios—they are quickly becoming a reality.

Cities are now deploying more sensors than ever and smart cameras are becoming the new standard for video surveillance, allowing them to continuously collect valuable IoT data. Yet more needs to be done to truly power smarter, safer cities. Despite the ability to now collect constant video data, analyzing it for the public good continues to be a challenge due to the lack of integration between systems across the city ecosystem. Many cities are implementing innovative smart city initiatives, but without an integrated centralized system to unify them, the vision of a smart city cannot be actualized. As cities look to become smarter and empower their first responders to be the superheroes that ensure the safety of their citizens and visitors, they must first look to implement a unified foundation that seamlessly integrates these systems and ensures fast, reliable communication and collaboration.

Together, humans and technology can power a safer metropolis for all of us.

This article originally appeared in the August 2017 issue of Security Today.