evolution of the face

Evolution of the Face

Technology puses science fiction out the door

Ten years ago, face recognition sounded like science fiction. Biometrics programs were expensive, not very accurate and largely the domain of researchers and specialists buried in the labs of government agencies and private companies. Now, biometrics is the most important security weapon of all—for detectives, soldiers and intelligence officers. As in industry, we’ve gone from working with highly-posed, mug shot-style images and limited photo banks to accurately mapping “faces in the wild” with mobile hand-held devices for instant checks against massive cloud-based databases.

Are our faces now like bar codes, easily scanned and instantly traceable? Not quite. But face recognition has grown into a tool that every investigator can benefit from. From U.S. Border Control and Homeland Security agents to local police, soldiers in the field and Department of Motor Vehicles clerks, there’s a need to identify faces. Our ability to make faster, more accurate facial identifications regardless of challenging conditions is helping law enforcement, military and security officials do their jobs more effectively.

From Mug Shots to Surveillance

Government has always been, and remains, the biggest user and innovator in the field of biometrics. A diverse array of governmental agencies are continually looking for more ways to put facial recognition to good use, mostly for criminal pursuit.

In 2004, mobile facial recognition was barely on anyone’s radar but biometrics really became a game changer in 2006 in Iraq, according to a recent annual report from the U.S. Biometrics Identity Management Agency (BIMA). That’s when the tool started to develop into a genuine weapons system. This came about largely through the development of rugged, portable and tactical devices. Battlefield deployment capability was essential to the growth of facial biometrics.

Face recognition really started with an Army program—the BAT, or Biometrics Activated Toolset—designed to gather enrollment images, or pictures taken when a person is booked or detained. The BAT also acquired fingerprint and iris biometrics. (BAT later became part of BIMA, which is part of the Army Provost Marshal.) The people who founded this database had a vision. They took five pictures of each suspect: one frontal, two profiles and two angulated pictures, left and right. It was a solid, forward-thinking start. The problem? They didn’t have a powerful facial recognition search tool for matching the images to a larger database. That’s where companies like Animetrics soon waded in, designing programs to work with these five pictures. The legacy facial recognition systems could only process one frontal image.

Since then, many development cycles have passed. The biggest innovations over the past decade have come in mobile and cloud technology, and in the ability to work with challenging images. We no longer need forward-facing images to compare to other static images; we have the technology to rotate faces—known as “pose correction— and to turn flat 2-D images into 3-D avatars. These advancements have given us the capability to work with challenging images, from a variety of sources, such as surveillance footage, security cameras and smartphones. The images we work with now are largely uncontrolled, unlike the old mug shot or driver’s license pictures.

Thanks to the Internet and explosion in digital and mobile cameras and video devices, there are so many more images to work with in the world, and a growing number of processing tools. The Web is now the major source for investigations. Ironically, it both fuels and fights crime.

Mobility is critical now. Agencies want investigators in the field to have biometric technology available at just the touch of a phone button. And it’s possible. Now, it’s all about integration, custom application development and harnessing the infinite processing power of cloud technology. Intelligence agencies are using video analytics to scour and analyze tons of online images, collecting, indexing, clustering and analyzing faces.

Accuracy has greatly improved, even as images become less precise. For the industry as a whole, you still get the best results with controlled imagery against a controlled database: a 99.7 percent identification rate, at a false accept rate of 1 in 1,000. (When heads are turned, the rate falls to 40-50 percent.) But the ability to rotate uncontrolled images and create avatars produces more accurate matches— an identification rate of 90 percent or higher at that same false accept rate. It’s not DNA and it’s not yet admissible in court, but these matches are incredibly effective in providing new leads for traditional investigatory follow-up.

Additionally, accessibility to facial recognition tools has increased greatly in the past decade. Biometrics screening used to be the purview of maybe one trained specialist in a platoon, or a technician in a crime lab. Now, we have the capability to give every soldier, border agent and police officer pocket-sized biometric tools—handheld devices that can be carried everywhere to make speedy identifications in the field. The implications for a soldier or officer of immediately knowing who they’re dealing with—and the ability to instantly match suspects to international watch lists and criminal records—are revolutionary.

Doing the Math

The science behind these advancements is based in mathematical concepts that many of us already possess a rudimentary understanding. It’s all about the math—mapping the face along the three-dimensional X, Y and Z axes.

Like most biometrics, such as fingerprint and iris matching, basic face recognition begins with a sensor-created biometric. In this case, the sensor is a camera that produces a photograph or video frame. Then, the digital image is scanned by face-detection software to find the faces, usually starting with the eyes in locating facial features. Once a face is determined, a set of measurements are created by algorithms, reducing the photo to a facial biometric template, usually consisting of a vector of many numbers. These templates are stored in a file, or an enrollment gallery.

Now, the application can search that gallery for any given face, seeking a match or close likeness. Searches are conducted by matching algorithms that compare the given—or suspect—face to the gallery templates, using a statistical analysis process. Matches are not binary, for example, match or no match. The matching is completed by finding a gallery face—or several—that are statistically very close to the given face, using a weighted score in reference to the other database subjects.

We now have the technology, through the “computational anatomy” developed by Animetrics founder Dr. Michael Miller at Johns Hopkins University, to take these images a stage further. Basically, 2-D images are converted to 3-D images by taking the X and Y coordinates, and synchronizing them into X, Y and Z coordinates. These 3-D templates, which visualize the subject’s facial structure, geometry and texture, allow for precise comparative analysis of facial features between a subject face and a larger database. Thus scars, moles, tattoos and distance measurements between facial features are compared. The technology is especially effective in increasing matching accuracy with uncontrolled imagery, such as faces with pose angulations or those obstructed and shadowed by severe lighting. The 3-D models can be viewed from any angle, rotated into any plane or used to create a wireframe image for overlay by a second image for comparison. These advanced capabilities have greatly expanded the usefulness and practicality of face recognition.

Street and Field Biometrics

The use cases for facial recognition are growing steadily, as the technology develops. So many people have smartphones, Androids and iPhones now, devices that are essentially little computers in our pockets. Biometrics tools are more accessible than ever before. But the Department of Defense (DoD), Homeland Security, law enforcement agencies and the international intelligence community, and forensic labs around the world, remain the biggest users of facial recognition.

From the FBI and NSA, to the CIA, local police to the intelligence agency of each armed forces branch, everyone has a use for this technology. Accurate identity resolution is an absolute necessity on the modern battlefield, where the ability to track terrorists remains crucial. It’s also become an important tool for fighting crime on the home front. Though agencies are largely differentiated by their databases— some of which are shared, some of which are not and some of which we wouldn’t want shared—they all share a commitment to the technology.

Here are just a few use cases.

State Department of Motor Vehicles

One hugely successful application of facial recognition is finding duplicate licenses in department or registry of motor vehicle databases. Almost all states now run these applications, which gained popularity after the attacks of Sept. 11, 2001. Local offices now issue temporary licenses to applicants, which gives staff a few days to research a person’s information, answering the question: does this driver have multiple identifications using the same face? Applications scour the motor vehicle license databases, measuring facial biometrics to identify duplicates, which can be used for fraudulent bank loans, check cashing and job applications.

This technology is useful both in preventing the issuance of duplicate identifications and also in identifying drivers on the road. The day may come when law enforcement can take the picture of a person pulled over without a driver’s license, and instantly compare that image to the state DMV database. The officer will immediately know if the driver does indeed have a valid license, and if the person really is who he or she claims to be.

The Military

Facial recognition technology is crucial to DoD, its military branches and intelligence gathering operations. Military investigators routinely use facial recognition as a search engine for processing imagery from surveillance cameras, smartphones, biometric sensors and hidden devices, checking captured faces against international watch lists of terrorists and criminals. It is especially critical for agents and soldiers in the field and check points, constantly on the look-out for suicide bombers and terrorists, to be able to quickly identify suspects. While other biometric measures, such as fingerprints, palm prints or irises, are great for making positive identifications, they’re not always feasible to collect. Facial biometrics can be used to investigate subjects in a more covert, less intrusive manner, alerting investigators that additional action may be warranted.

Rugged handheld hardware and smartphone or tablet devices designed for use in military intelligence are making in-the-field face recognition a reality, along with remote access to cloud-based databases. Powerful facial recognition tools can now run as localized applications, reducing soldiers’ dependency on technology at their home bases and allowing instant access to watch lists.

Access and control is another key application for government use. Facial recognition can be used to monitor access to countries, for example, in border control. But it also can be used to create smaller lists of internally “approved” faces, for access to Homeland Security or Transportation Security Administration offices and buildings. Additional benefits stem from providing government contractors access to restricted areas. Meanwhile, Security View Systems also are becoming more prevalent in restricting access to schools and other security-sensitive buildings.

Law Enforcement

Facial recognition is growing quickly in law enforcement agencies of all sizes. From Pennsylvania’s statewide JNET system, which allows all trained officers to compare enhanced 3-D images against a massive 3.5-million image database, to the Zurich police’s innovative use of 3-D facial rotation to help victims better identify suspects, criminal investigators around the world are putting face recognition to work. Prisons are also making their enrollment databases more widely available to those working with biometrics in the field.

The FBI is definitely leading the way, with its $1 billion Next Generation Identification (NGI) program, designed to allow law enforcement agencies around the country to compare suspect photos in a massive, national database, beginning in 2014. In the meantime, the agency also is working to offer free facial recognition software to law enforcement: the Universal Face Workstation (UFW).

From Sci-fi to Future

Facial recognition will continue to grow in speed, accuracy, accessibility and use. As the industry gets better at identifying faces from a distance—gathering images less intrusively—and continue improvements to surveillance footage and uncontrolled images, video analysis will continue to grow in importance.

The future is pocked-sized and mobile. In five or 10 years, it’s possible a soldier will be able to identify everyone around himself in seconds, merely by holding up a palm-sized device linked to augmented information in real time. Mobile will be the vehicle for all communications capabilities, and even most applications for people, businesses, educators, authorities and law enforcement. And mobile facial applications will be the norm for security, identity authorization, investigations and social networking.

Many new applications are on the way, particularly in the areas of locating missing and exploited children, catching human traffickers and identifying victims. The near infinite scaling of cloud technologies will open a wealth of opportunities for unlimited use and expansion of facial applications, including delivery as Web services and the indexing of unlimited numbers of photographs in online databases.

We are entering an age of even more information, cooperation and remote access. Linking field and regional data to national databases will be critical for faster identification of both our own people and our enemies.

This article originally appeared in the April 2013 issue of Security Today.


  • Report: 15 Percent of All Emails Sent in 2023 Were Malicious

    VIPRE Security Group recently released its report titled “Email Security in 2024: An Expert Look at Email-Based Threats”. The 2024 predictions for email security in this report are based on an analysis of over 7 billion emails processed by VIPRE worldwide during 2023. This equates to almost one email for everyone on the planet. Of those, roughly 1 billion (or 15%) were malicious. Read Now

  • ASIS Announces ANSI-Approved Cannabis Security Standard

    ASIS International, a leading authority in security standards and guidelines, proudly announces the release of a pioneering American National Standards Institute (ANSI)-approved standard dedicated to cannabis security. This best-in-class standard, meticulously developed by industry experts, sets a new benchmark by providing comprehensive requirements and guidance for the design, implementation, monitoring, evaluation, and maintenance of a cannabis security program. Read Now

  • ISC West Announces Keynote Lineup

    ISC West, in collaboration with premier sponsor the Security Industry Association (SIA), announced this year’s dynamic trio of speakers that will headline the Keynote Series at ISC West 2024. Read Now

    • Industry Events
  • Government is Top Targeted Industry for DDoS Attacks in Q4 2023

    The government sector experienced a surge of DDoS attacks in Q4 according to Lumen Technologies (NYSE: LUMN), a global leader in integrated network and cybersecurity solutions. The Lumen Quarterly DDoS & Application Threat Report for Q4 2023 analyzes data from its DDoS mitigation platform and application protection partner, ThreatX, to provide an overview of the DDoS and application-layer attacks that targeted organizations in the last quarter of 2023. Read Now

Featured Cybersecurity


New Products

  • Hanwha QNO-7012R

    Hanwha QNO-7012R

    The Q Series cameras are equipped with an Open Platform chipset for easy and seamless integration with third-party systems and solutions, and analog video output (CVBS) support for easy camera positioning during installation. A suite of on-board intelligent video analytics covers tampering, directional/virtual line detection, defocus detection, enter/exit, and motion detection. 3

  • EasyGate SPT and SPD

    EasyGate SPT SPD

    Security solutions do not have to be ordinary, let alone unattractive. Having renewed their best-selling speed gates, Cominfo has once again demonstrated their Art of Security philosophy in practice — and confirmed their position as an industry-leading manufacturers of premium speed gates and turnstiles. 3

  • AC Nio

    AC Nio

    Aiphone, a leading international manufacturer of intercom, access control, and emergency communication products, has introduced the AC Nio, its access control management software, an important addition to its new line of access control solutions. 3