A Power User

How AI and deep learning are making airport operations an IP camera heavyweight

• By Anthony Incorvati
• Jan 27, 2022

On the Friday before Thanksgiving, the Transportation Security Administration (TSA) screened 2.24 million airline passengers in a single day. It was the highest number of people processed through checkpoints since the start of the coronavirus pandemic in March 2020, according to international news agency Reuters. (By comparison, on the Friday before Thanksgiving 2020, TSA screened just 984,369 passengers.)

Experts believe it was an indication of things to come. While increased air travel during the holidays is normal, the bigger news is that the transportation industry anticipates a substantial increase in air travel in the coming years. The International Air Transport Association (IATA), prior to the pandemic, said current trends suggest passenger numbers could double to 8.2 billion air travelers in 2037.

Certainly, the aftermath of COVID-19 caused the number of travelers to plunge downward; however, passenger loads are once again on the rise. Along with this significant increase in volume comes challenges that go well beyond health and safety compliance requirements, such as mask wearing.

In fact, the No. 1 issue faced by airports prior to the pandemic was capacity and as passenger counts ramp up, this issue is coming right back in focus. Airport expansions are few, and far between. When they do happen, they can take decades and cost billions of dollars, such as the O’Hare Airport expansion that cost $6 billion and took 16 years to complete.

The Role of Technology
Instead of building new terminals or expanding existing ones to accommodate a greater volume of travelers, airports and airlines are looking to technology for answers. Given the rising number of airline customers, they are asking how technology use to ease capacity constraints at airport terminals — a factor that directly affects the passenger experience. A passenger who has a pleasant experience at an airport, from curb to gate, is more likely to stay engaged with air travel as a mode of transportation and to develop an affinity for their airlines.

In addition, happy passengers may enrich non-aviation revenue streams by spending more money at airport shops and eateries, which in turn helps airports secure new tenants and other vendors.
Not surprisingly, advancements in surveillance technology play a huge role. Today’s high-performance IP cameras have powerful processing, which enables analytics based on artificial intelligence (AI) and deep learning at the edge. This means that although surveillance cameras are still very much part of the security fabric at airports, they are increasingly being used in a larger way by acting as IoT sensors that collect and analyze data. In these types of applications, they assist airports and airlines with optimizing their operations, maintenance and even marketing.

The Public Side
On the “public side” of an airport or better known as “Public Area Security,” includes the facilities outside of and up to and including the security checkpoints. Normally, IP cameras are installed for general surveillance of perimeters and fence lines, parking lots, entrances/exits, ticket counters, TSA checkpoints, baggage claims and other spaces. In addition to looking for a wide range of atypical behavior, such as perimeter breaches, suspicious vehicles, items left behind, reverse movement through checkpoints and corridors, and more, a recent trend in airport security is the detection of unruly passengers, making security surveillance a still heightened concern.

In addition, it has been widely reported that a record number of firearms, the majority loaded, have been confiscated by TSA over the past year. This may be attributable to a higher number of first-time leisure travelers. According to Forbes.com, the TSA said it caught 4,495 passengers with firearms at airport security checkpoints in the first nine months of 2021, the most in 20 years.

While the security and risk management needs for IP cameras at airports are clearly high, the operational needs may be even more commanding. In fact, AI may be the greatest impact on airport operations that the aviation industry has seen in decades, affecting everything from ramp activity to passenger flow.

AI and deep learning algorithms are being applied to these locations to better understand wait/dwell times, occupancy levels, and other factors — and as a tool to determine whether or not to allocate more resources to process passengers more efficiently.

One example is how IP cameras will help manage queues at airports. Cameras, as sensors, collect data on how quickly a queue is progressing. IP cameras are enabling smart algorithms. Integration with airport informational databases allows this data to be analyzed, and pushed out to traveler apps and flight displays.

Passengers can view wait times for shuttles or at security, checkpoints and better manage their time using the information provided. Again, more efficient processing of passenger flow, for example, helping ease capacity, contributing to a better customer experience.

The Sterile Side
On the “sterile side,” which includes areas beyond TSA checkpoints, airports are filling their terminals and ramp areas with IP cameras. In these applications, the surveillance cameras act as sensors, involving AI and deep learning to contribute data they collect to a larger structure. This effort does improve operational efficiency and ultimately enhance the customer experience.

Some airlines are installing as many as three to five cameras per gate to assist with ramp turnaround management. Like a racecar at a pit stop, when a plane lands and passengers deplane, the ground support equipment springs into action. Crews will replenish fuel, a cleaning team will perform their duties, the lavatory disposal service makes sure lavatories are ready for the next flight, the belt loader is activated for removal of luggage and cargo, jet bridge activity occurs, caterer services commence, and the list goes on.

IP cameras installed in strategic locations on the Jet Bridge and ramp, above and below the wing, generate a comprehensive overview of the entire ground-support operation, specifically looking for holdups that if addressed, could shave time off the turnaround process and contribute to the airline’s overall on-time performance. Additionally, operational surveillance provides data on gate activity, such as crew arrival, and on passenger movements, such as how many customers are carting carry-on luggage — important information that can identify trends.

IP cameras with AI algorithms are capturing and understanding events in what seems to be a very hectic environment. If the camera can see it, AI can sense and classify it. As these are collected, structure merges with this data and it becomes a tool to help manage, and most importantly, create accurate predictions and detect deviations from procedures, which improves operational performance. This is accomplished by taking the structured data and integrating with various airport operational databases and, in turn, performing deep regression analysis to better understand and predict the operation.

Areas of improvement include.
• Reducing the costs associated with poor on-time performance and delays, which can have a massive domino effect by causing other flight delays, causing passengers to miss flights. The direct and indirect costs, as well as lost opportunities with poor on-time performance, may cause unneeded stress.

• Improving service-level agreements with various support contractors can be realized, including accurate and verifiable billing from these support contractors and more importantly, increased use of aircraft and associated ground support equipment.

• Decreasing the extensive early-boarding calls will increase commercial dwell time in the terminal, which could support non-aviation revenues.

• Influencing the sustainability factor by improving operational performance will reduce fuel expenses and CO2 footprint.

The incentive to identify and fix operational issues is financial. If an airline’s ground-support contractors are not performing to the expectations spelled out in their service level agreements, penalties may be enforced. Ultimately, the airline wants to run on schedule, because ratings are based for on-time performance.

Other Use Cases
One example of using IP cameras to improve maintenance comes from a specialty provider that has a smart restroom solution. The camera contains a basic people-counting function and when a certain number of people have used that restroom, it triggers an alert to a maintenance crew to do a cleaning. It sounds simple, but it is actually sophisticated because the system uses different databases, including flight schedules and other data.

Marketing is another role for IP cameras. Most airports have a real estate group, which wants to understand passenger flow — where people hang out while they are waiting for their flights, as well as foot traffic in and out of retail establishments. Therefore, there is a movement to use more IP video technology as a tool to better support that informational need.

The Sky is the Limit
An airport is a major operation that relies on data collection and information management for optimizing all of its various processes. It is about efficiency and performance, and understanding where the chokepoints and bottlenecks are in the operation.

As well as their role in security and safety applications, IP cameras are equally proficient in collecting and analyzing data for this purpose. Today’s IP cameras are the ultimate sensors, supporting processing at the edge and enabling the latest AI and deep learning applications. By definition, an IP camera is an IoT device. By adding AI algorithms to the computer vision equation, there can be an interesting debate on how this IoT scheme is competing with other IoT devices.

For example, instead of ground-support equipment having their own IoT sensors, the camera/AI system can classify and analyze the activities of ground-support equipment.

With the avalanche that is coming with AI and deep learning algorithms, use cases are practically endless. This is particularly exciting in light of the Bipartisan Infrastructure Deal; signed into law in November 2021.

The legislation will upgrade U.S. infrastructure. Airports will share in the $25 billion windfall to address aging infrastructure and reduce congestion, among other uses. With the expanded use of IP cameras for optimizing capacity and enhancing the passenger experience, the next five years will be promising for the addition of this technology in airport environments.

This article originally appeared in the January / February 2022 issue of Security Today.