Securing the City
How a new generation of video analytics is empowering the next generation of superheroes
It happens every day. In the blink of an eye, a car accident can
happen, gunshots go off, or an innocent bystander is assaulted.
Thanks to the brave women and men who don their uniforms
every day and put everything on the line to protect and serve
the public, we can all worry a little less about the dangers of the
world. But while these people are superheroes in their own right, they
don’t possess superhuman capabilities—at least not yet, anyway.
Thanks to new advancements in video analytics technology and
powerful Internet of Things (IoT) edge devices; these superheroes can
now be armed with new superhuman capabilities to better protect us.
The Journey to a Smarter City
Sight is one of our first lines of defense and awareness is key. We look
both ways before we cross the street. We avoid nefarious or aggressive-
looking strangers. We keep a vigilant watch on those we care for
when we’re out in public.
But our ability to leverage sight to keep us safe is limited.
When a crisis occurs, we must rely on our memories and limited
field of vision to piece together who, what, where and why. The pieces
can often feel like they don’t fit together or some may be missing entirely.
As a result, devising how to respond to an emergency or investigate
a crime becomes more reliant on intuition than cold hard facts.
Enter video. In 1951, the first Video Tape Recorder (VTR) was
used to record live images from a television camera. By the mid-1960s
closed-circuit video surveillance that could remote PTZ to monitor
public spaces was becoming more and more common across cities. The
technology enabled authorities to watch crowds for suspicious activities,
take a closer look at crime incidents to better investigate cases and
keep an eye on public events where crowds might become a target for
threats. The capabilities, while limited, set the foundation for the continued
use of video within the field of security and public safety.
Computer Vision Powers
With the progression of technologies like artificial intelligence,
machine-learning and computer vision, video is quickly becoming
more than an eye in the sky. Advanced analytics capabilities are now
transforming video into IoT data to provide rich insights, offer realtime
alerts and enhance safety like never before. These insights help
us respond faster, act proactively to solve or prevent crimes, and
help us make plans that make us more effective in the future—much
like a superhero.
Until recently, the long-held industry standard for video analytics
was pixel color recognition. The software would analyze individual pixels, or a portion of the pixels, to identify
suspicious objects and people and an alert
would then be sent so security teams could
investigate or respond to the threat. The challenge
with this type of technology is they
could be easily tricked into falsely identifying
an animal or object as a suspicious person.
For example, an alert that someone had
entered a highly-restricted area monitored
by an outdoor camera that could easily be
triggered by a squirrel or raccoon that came
into the camera’s field of view. The software
would simply detect a change in pixel movement
or colors and then trigger an alert—
there was no intelligence behind it. All the
algorithm knew was that something new
was in the scene, but the ability to identify
whether the intrusion was human, assess its
threat level and determine if it warranted of
action was missing.
Early video analytics technologies also
created many false positives due to adverse
weather conditions, like snow, wind and other
daily environmental evets, which also triggered
an inordinate amount of false positive
alerts. People tend to start ignoring the alerts
that cry wolf, essentially making the technology
useless. These types of false positives
would waste valuable time and resources, or
in many cases, caused security personnel to
ignore alerts. This ultimately slowed market
adoption of video analytics. Video analytics
that are useful need to provide critical
situational awareness in all environmental
conditions—even adverse weather—and ignore
irrelevant intruders like squirrels or the
Today, computer vision, machine learning
and artificial intelligence are dramatically
lowering the rate of false positives and
advancing the usefulness of video analytics.
Rather than just seeing moving pixels, advanced
video analytics can create 4-D reconstruction
of two-dimensional video images
(3-D plus time). Now, not only are objects
able to be detected using particles, perspective,
velocity, path deviation and travel distance,
they can be accurately identified and in
some cases classified through artificial intelligence.
Security personnel can leverage these
advancements to better understand if a person
or suspicious object has been detected.
When a squirrel runs through a video
feed, advanced video analytics capture multiple
angles of the animal. On the back-end,
it takes those two-dimensional images and
reconstructs them to build a 4D version of
the squirrel. The object is now identifiable
against a database of known images—each
with an assigned level of potential danger.
The video system can not only see an object,
it understands what that object means
in the context of the area being monitored:
a squirrel is only a squirrel, not a trespasser
that might be armed and dangerous.
Unfortunately, not all objects are as innocent
as a passing squirrel.
How many times have you been at the
airport and heard the loud speaker come on
reminding passengers to keep their luggage
close and report any unattended or suspicious
items? That’s because airports are a
constant point of public safety concern.
With thousands of people shuffling between
gates and destinations—luggage and bags in
hand—any passenger could be a threat or
could have just forgotten their luggage in the
confusion of travel.
So, how do you keep people safe when
there is so much going on? Arm them with
enhanced awareness, security intelligence,
and the ability to better allocate resources.
Say a bag has been left unattended. With
today’s advanced video analytics, the object
would be recognized as a bag as soon as it
enters into the purview of a video camera—
then the clock starts ticking. Depending on
the procedures in place at the airport, staff
would be alerted once the threshold of designated
time had passed that there was an object
that has not moved or interacted with a
passenger for longer than the allocated time.
The video system could then send all first responders,
in this case airport security, a realtime
alert to their mobile devices, notifying
them a bag has been abandoned, its location
and how long it’s been there. Now everyone
is on alert and the security team can immediately
determine who should investigate based
on geo-location. A plan of action can be
formed in real-time, based on real insights.
If the situation goes beyond a passenger
taking too long in the restroom, advanced
video analytics can immediately arm investigators
with the situational awareness they
need to begin an investigation, while continuing
to keep everyone safe. Through facial
recognition, the software can also identify
all the people who have come in contact
with the bag—going so far as to backtrack
all their steps from the point of abandoning
the bag. Leveraging the airport’s parking lot
cameras, a suspect’s car and license plate can
then be identified. With a visual of a suspect
and vehicle information in place, law officials
can begin to investigate any criminal connections
and evaluate if the bag might pose a
bomb-level-threat worthy of evacuation.
While this might be an extreme scenario,
the reality is that when a major event or everyday
situations occur, we must prepare to
stop them—and advancements in computer
vision, artificial intelligence and analytics
have been helping law enforcement and operations
teams tackle these challenges more
and more. For example, video analytics can
also detect when lines become too long and
alert staff to open another ticket counter,
security gate, or passport stand in order to
help people get from the door to their gate as
quickly and conveniently as possible.
Smart Cities Fuel These
For complex organizations like governments
and cities that produce massive amounts
of data, video, while valuable, is greatly enhanced
when integrated within a broader
smart city ecosystem. By connecting disparate
video, data and public safety systems on a single
platform, public safety officials can receive
real-time alerts backed by the most up to date
insights, map and predict crime to deploy resources
and strategize the best approach to a
crime that’s in progress or how to prevent the
next one—maximizing the level of safety.
Take the airport scenario. While video
analytics may help set up police officers for
success in identifying a suspect, integrating
the system with a broader smart city platform
will expedite results. Rather than having
to wait for a suspect’s image to be located
in a law enforcement database and try to find a license plate in another, an integrated
platform can unify all data into a single platform,
and allow all functions to be displayed
simultaneously from a single command center,
or even from a mobile device inside an
officer’s vehicle. Similarly, integrating advanced
video analytics across a city-wide
platform would allow a suspect and any accomplices
to be identified and tracked from
the time they leave the airport to the time
they reach their “hideout” location, enabling
law enforcement to more swiftly apprehend
and bring them in for questioning. This type
of legwork automation instantly streamlines
investigations and empowers the type of real-
time response that’s needed in dangerous
and emergency situations.
In a smart city, you wouldn’t deploy a
single taskforce to find an abducted child—
you’d deploy a technology-enhanced army.
The first 24 hours in child abductions are
critical and with video analytics an entire
city can be searched in real time, providing
leads in minutes or hours instead of days.
With an integrated alert system police and
the public can be notified in real-time to be
on the look-out for the child. Amber Alert
systems are already in place across the U.S.,
but the possibilities with video analytics for
a more comprehensive and efficient response
are tremendous, especially when integrating
public and private cameras in the community
through public-private partnerships.
Leveraging real-time social media, geolocational
data and public video feeds in an
integrated system could help them identify
the location of a missing child faster. Upload
the photos of the suspected kidnapper and
missing child into the system and the video
analytics could enable officials to quickly
scan across public spaces to locate where they
had been most recently recognized. Once the
child’s location is identified, remote video
analysis could help police assess any danger
they might be in. The reconstruction of multiple
video shots could help to detect whether
the abductor is armed to inform a better, safer
rescue strategy and help officers with realtime
situational awareness when they close in
on the suspect and rescue the child.
A New Type of Metropolis
These types of integrated city platforms
aren’t just imaginary scenarios—they are
quickly becoming a reality.
Cities are now deploying more sensors
than ever and smart cameras are becoming
the new standard for video surveillance, allowing
them to continuously collect valuable
IoT data. Yet more needs to be done to truly
power smarter, safer cities. Despite the ability
to now collect constant video data, analyzing
it for the public good continues to be a challenge
due to the lack of integration between
systems across the city ecosystem. Many cities
are implementing innovative smart city
initiatives, but without an integrated centralized
system to unify them, the vision of
a smart city cannot be actualized. As cities
look to become smarter and empower their
first responders to be the superheroes that
ensure the safety of their citizens and visitors,
they must first look to implement a unified
foundation that seamlessly integrates
these systems and ensures fast, reliable communication
Together, humans and technology can
power a safer metropolis
for all of us.
This article originally appeared in the August 2017 issue of Security Today.