evolution of the face

Evolution of the Face

Technology puses science fiction out the door

Ten years ago, face recognition sounded like science fiction. Biometrics programs were expensive, not very accurate and largely the domain of researchers and specialists buried in the labs of government agencies and private companies. Now, biometrics is the most important security weapon of all—for detectives, soldiers and intelligence officers. As in industry, we’ve gone from working with highly-posed, mug shot-style images and limited photo banks to accurately mapping “faces in the wild” with mobile hand-held devices for instant checks against massive cloud-based databases.

Are our faces now like bar codes, easily scanned and instantly traceable? Not quite. But face recognition has grown into a tool that every investigator can benefit from. From U.S. Border Control and Homeland Security agents to local police, soldiers in the field and Department of Motor Vehicles clerks, there’s a need to identify faces. Our ability to make faster, more accurate facial identifications regardless of challenging conditions is helping law enforcement, military and security officials do their jobs more effectively.

From Mug Shots to Surveillance

Government has always been, and remains, the biggest user and innovator in the field of biometrics. A diverse array of governmental agencies are continually looking for more ways to put facial recognition to good use, mostly for criminal pursuit.

In 2004, mobile facial recognition was barely on anyone’s radar but biometrics really became a game changer in 2006 in Iraq, according to a recent annual report from the U.S. Biometrics Identity Management Agency (BIMA). That’s when the tool started to develop into a genuine weapons system. This came about largely through the development of rugged, portable and tactical devices. Battlefield deployment capability was essential to the growth of facial biometrics.

Face recognition really started with an Army program—the BAT, or Biometrics Activated Toolset—designed to gather enrollment images, or pictures taken when a person is booked or detained. The BAT also acquired fingerprint and iris biometrics. (BAT later became part of BIMA, which is part of the Army Provost Marshal.) The people who founded this database had a vision. They took five pictures of each suspect: one frontal, two profiles and two angulated pictures, left and right. It was a solid, forward-thinking start. The problem? They didn’t have a powerful facial recognition search tool for matching the images to a larger database. That’s where companies like Animetrics soon waded in, designing programs to work with these five pictures. The legacy facial recognition systems could only process one frontal image.

Since then, many development cycles have passed. The biggest innovations over the past decade have come in mobile and cloud technology, and in the ability to work with challenging images. We no longer need forward-facing images to compare to other static images; we have the technology to rotate faces—known as “pose correction— and to turn flat 2-D images into 3-D avatars. These advancements have given us the capability to work with challenging images, from a variety of sources, such as surveillance footage, security cameras and smartphones. The images we work with now are largely uncontrolled, unlike the old mug shot or driver’s license pictures.

Thanks to the Internet and explosion in digital and mobile cameras and video devices, there are so many more images to work with in the world, and a growing number of processing tools. The Web is now the major source for investigations. Ironically, it both fuels and fights crime.

Mobility is critical now. Agencies want investigators in the field to have biometric technology available at just the touch of a phone button. And it’s possible. Now, it’s all about integration, custom application development and harnessing the infinite processing power of cloud technology. Intelligence agencies are using video analytics to scour and analyze tons of online images, collecting, indexing, clustering and analyzing faces.

Accuracy has greatly improved, even as images become less precise. For the industry as a whole, you still get the best results with controlled imagery against a controlled database: a 99.7 percent identification rate, at a false accept rate of 1 in 1,000. (When heads are turned, the rate falls to 40-50 percent.) But the ability to rotate uncontrolled images and create avatars produces more accurate matches— an identification rate of 90 percent or higher at that same false accept rate. It’s not DNA and it’s not yet admissible in court, but these matches are incredibly effective in providing new leads for traditional investigatory follow-up.

Additionally, accessibility to facial recognition tools has increased greatly in the past decade. Biometrics screening used to be the purview of maybe one trained specialist in a platoon, or a technician in a crime lab. Now, we have the capability to give every soldier, border agent and police officer pocket-sized biometric tools—handheld devices that can be carried everywhere to make speedy identifications in the field. The implications for a soldier or officer of immediately knowing who they’re dealing with—and the ability to instantly match suspects to international watch lists and criminal records—are revolutionary.

Doing the Math

The science behind these advancements is based in mathematical concepts that many of us already possess a rudimentary understanding. It’s all about the math—mapping the face along the three-dimensional X, Y and Z axes.

Like most biometrics, such as fingerprint and iris matching, basic face recognition begins with a sensor-created biometric. In this case, the sensor is a camera that produces a photograph or video frame. Then, the digital image is scanned by face-detection software to find the faces, usually starting with the eyes in locating facial features. Once a face is determined, a set of measurements are created by algorithms, reducing the photo to a facial biometric template, usually consisting of a vector of many numbers. These templates are stored in a file, or an enrollment gallery.

Now, the application can search that gallery for any given face, seeking a match or close likeness. Searches are conducted by matching algorithms that compare the given—or suspect—face to the gallery templates, using a statistical analysis process. Matches are not binary, for example, match or no match. The matching is completed by finding a gallery face—or several—that are statistically very close to the given face, using a weighted score in reference to the other database subjects.

We now have the technology, through the “computational anatomy” developed by Animetrics founder Dr. Michael Miller at Johns Hopkins University, to take these images a stage further. Basically, 2-D images are converted to 3-D images by taking the X and Y coordinates, and synchronizing them into X, Y and Z coordinates. These 3-D templates, which visualize the subject’s facial structure, geometry and texture, allow for precise comparative analysis of facial features between a subject face and a larger database. Thus scars, moles, tattoos and distance measurements between facial features are compared. The technology is especially effective in increasing matching accuracy with uncontrolled imagery, such as faces with pose angulations or those obstructed and shadowed by severe lighting. The 3-D models can be viewed from any angle, rotated into any plane or used to create a wireframe image for overlay by a second image for comparison. These advanced capabilities have greatly expanded the usefulness and practicality of face recognition.

Street and Field Biometrics

The use cases for facial recognition are growing steadily, as the technology develops. So many people have smartphones, Androids and iPhones now, devices that are essentially little computers in our pockets. Biometrics tools are more accessible than ever before. But the Department of Defense (DoD), Homeland Security, law enforcement agencies and the international intelligence community, and forensic labs around the world, remain the biggest users of facial recognition.

From the FBI and NSA, to the CIA, local police to the intelligence agency of each armed forces branch, everyone has a use for this technology. Accurate identity resolution is an absolute necessity on the modern battlefield, where the ability to track terrorists remains crucial. It’s also become an important tool for fighting crime on the home front. Though agencies are largely differentiated by their databases— some of which are shared, some of which are not and some of which we wouldn’t want shared—they all share a commitment to the technology.

Here are just a few use cases.

State Department of Motor Vehicles

One hugely successful application of facial recognition is finding duplicate licenses in department or registry of motor vehicle databases. Almost all states now run these applications, which gained popularity after the attacks of Sept. 11, 2001. Local offices now issue temporary licenses to applicants, which gives staff a few days to research a person’s information, answering the question: does this driver have multiple identifications using the same face? Applications scour the motor vehicle license databases, measuring facial biometrics to identify duplicates, which can be used for fraudulent bank loans, check cashing and job applications.

This technology is useful both in preventing the issuance of duplicate identifications and also in identifying drivers on the road. The day may come when law enforcement can take the picture of a person pulled over without a driver’s license, and instantly compare that image to the state DMV database. The officer will immediately know if the driver does indeed have a valid license, and if the person really is who he or she claims to be.

The Military

Facial recognition technology is crucial to DoD, its military branches and intelligence gathering operations. Military investigators routinely use facial recognition as a search engine for processing imagery from surveillance cameras, smartphones, biometric sensors and hidden devices, checking captured faces against international watch lists of terrorists and criminals. It is especially critical for agents and soldiers in the field and check points, constantly on the look-out for suicide bombers and terrorists, to be able to quickly identify suspects. While other biometric measures, such as fingerprints, palm prints or irises, are great for making positive identifications, they’re not always feasible to collect. Facial biometrics can be used to investigate subjects in a more covert, less intrusive manner, alerting investigators that additional action may be warranted.

Rugged handheld hardware and smartphone or tablet devices designed for use in military intelligence are making in-the-field face recognition a reality, along with remote access to cloud-based databases. Powerful facial recognition tools can now run as localized applications, reducing soldiers’ dependency on technology at their home bases and allowing instant access to watch lists.

Access and control is another key application for government use. Facial recognition can be used to monitor access to countries, for example, in border control. But it also can be used to create smaller lists of internally “approved” faces, for access to Homeland Security or Transportation Security Administration offices and buildings. Additional benefits stem from providing government contractors access to restricted areas. Meanwhile, Security View Systems also are becoming more prevalent in restricting access to schools and other security-sensitive buildings.

Law Enforcement

Facial recognition is growing quickly in law enforcement agencies of all sizes. From Pennsylvania’s statewide JNET system, which allows all trained officers to compare enhanced 3-D images against a massive 3.5-million image database, to the Zurich police’s innovative use of 3-D facial rotation to help victims better identify suspects, criminal investigators around the world are putting face recognition to work. Prisons are also making their enrollment databases more widely available to those working with biometrics in the field.

The FBI is definitely leading the way, with its $1 billion Next Generation Identification (NGI) program, designed to allow law enforcement agencies around the country to compare suspect photos in a massive, national database, beginning in 2014. In the meantime, the agency also is working to offer free facial recognition software to law enforcement: the Universal Face Workstation (UFW).

From Sci-fi to Future

Facial recognition will continue to grow in speed, accuracy, accessibility and use. As the industry gets better at identifying faces from a distance—gathering images less intrusively—and continue improvements to surveillance footage and uncontrolled images, video analysis will continue to grow in importance.

The future is pocked-sized and mobile. In five or 10 years, it’s possible a soldier will be able to identify everyone around himself in seconds, merely by holding up a palm-sized device linked to augmented information in real time. Mobile will be the vehicle for all communications capabilities, and even most applications for people, businesses, educators, authorities and law enforcement. And mobile facial applications will be the norm for security, identity authorization, investigations and social networking.

Many new applications are on the way, particularly in the areas of locating missing and exploited children, catching human traffickers and identifying victims. The near infinite scaling of cloud technologies will open a wealth of opportunities for unlimited use and expansion of facial applications, including delivery as Web services and the indexing of unlimited numbers of photographs in online databases.

We are entering an age of even more information, cooperation and remote access. Linking field and regional data to national databases will be critical for faster identification of both our own people and our enemies.

This article originally appeared in the April 2013 issue of Security Today.


  • Maximizing Your Security Budget This Year

    The Importance of Proactive Security Measures: 4 Stories of Regret

    We all want to believe that crime won’t happen to us. So, some business owners hope for the best and put proactive security measures on the back burner, because other things like growth, attracting new customers, and meeting deadlines all seem more pressing. Read Now

  • 91 Percent of Security Leaders Believe AI Set to Outpace Security Teams

    Bugcrowd recently released its “Inside the Mind of a CISO” report, which surveyed hundreds of security leaders around the globe to uncover their perception on AI threats, their top priorities and evolving roles, and common myths directed towards the CISO. Among the findings, 1 in 3 respondents (33%) believed that at least half of companies are willing to sacrifice their customers’ long-term privacy or security to save money. Read Now

  • Milestone Announces Merger With Arcules

    Global video technology company Milestone Systems is pleased to announce that effective July 1, 2024, it will merge with the cloud-based video surveillance solutions provider, Arcules. Read Now

  • Organizations Struggle with Outdated Security Approaches, While Online Threats Increase

    Cloudflare Inc, recently published its State of Application Security 2024 Report. Findings from this year's report reveal that security teams are struggling to keep pace with the risks posed by organizations’ dependency on modern applications—the technology that underpins all of today’s most used sites. The report underscores that the volume of threats stemming from issues in the software supply chain, increasing number of distributed denial of service (DDoS) attacks and malicious bots, often exceed the resources of dedicated application security teams. Read Now

Featured Cybersecurity


New Products

  • PE80 Series

    PE80 Series by SARGENT / ED4000/PED5000 Series by Corbin Russwin

    ASSA ABLOY, a global leader in access solutions, has announced the launch of two next generation exit devices from long-standing leaders in the premium exit device market: the PE80 Series by SARGENT and the PED4000/PED5000 Series by Corbin Russwin. These new exit devices boast industry-first features that are specifically designed to provide enhanced safety, security and convenience, setting new standards for exit solutions. The SARGENT PE80 and Corbin Russwin PED4000/PED5000 Series exit devices are engineered to meet the ever-evolving needs of modern buildings. Featuring the high strength, security and durability that ASSA ABLOY is known for, the new exit devices deliver several innovative, industry-first features in addition to elegant design finishes for every opening. 3

  • ResponderLink


    Shooter Detection Systems (SDS), an Alarm.com company and a global leader in gunshot detection solutions, has introduced ResponderLink, a groundbreaking new 911 notification service for gunshot events. ResponderLink completes the circle from detection to 911 notification to first responder awareness, giving law enforcement enhanced situational intelligence they urgently need to save lives. Integrating SDS’s proven gunshot detection system with Noonlight’s SendPolice platform, ResponderLink is the first solution to automatically deliver real-time gunshot detection data to 911 call centers and first responders. When shots are detected, the 911 dispatching center, also known as the Public Safety Answering Point or PSAP, is contacted based on the gunfire location, enabling faster initiation of life-saving emergency protocols. 3

  • Luma x20

    Luma x20

    Snap One has announced its popular Luma x20 family of surveillance products now offers even greater security and privacy for home and business owners across the globe by giving them full control over integrators’ system access to view live and recorded video. According to Snap One Product Manager Derek Webb, the new “customer handoff” feature provides enhanced user control after initial installation, allowing the owners to have total privacy while also making it easy to reinstate integrator access when maintenance or assistance is required. This new feature is now available to all Luma x20 users globally. “The Luma x20 family of surveillance solutions provides excellent image and audio capture, and with the new customer handoff feature, it now offers absolute privacy for camera feeds and recordings,” Webb said. “With notifications and integrator access controlled through the powerful OvrC remote system management platform, it’s easy for integrators to give their clients full control of their footage and then to get temporary access from the client for any troubleshooting needs.” 3