blog post background

Business guide to facial recognition: benefits, applications, and issues to consider

By Viktoria Shashkina, Innovation Analyst
Published on

Facial recognition is everywhere. What once started as an attribute specific to sci-fi movies is now a part of everyday life: we rely on facial recognition every time we unlock our phones or go through customs.

The numbers are only proving the ubiquitous nature of facial recognition. In 2022, the global facial recognition market was valued at $5.15 billion, and it is expected to grow at a CAGR of 14.9% from 2023 to 2030.

Along with gaining market volume, facial recognition algorithms are becoming more sophisticated. At the same time, they have been subject to numerous ethical concerns, amplified by reported uses of the technology for racial profiling and protester identification.

With more organizations turning to AI technology consultants, questions arise: how does AI-based facial recognition work, what are the benefits of facial recognition, and how does one implement the technology ethically? We’ve explored these points in this blog post.

How does facial recognition work?

facial recognition workflow

The meaning of the term ‘facial recognition’ is quite intuitive. The technology uses computer vision algorithms to map, analyze, and confirm the identity of a face in a photo or a video. Although every facial recognition solution (which often relies on proprietary algorithms) operates differently, we can distill the facial recognition process down to the following three steps:

  1. Detection refers to the process of locating a face in an input image. So, each face is placed into a bounding box. To complete this stage, the facial recognition algorithms are first trained to learn what a face looks like from various data entries.

  2. Analysis refers to mapping out the features of each face. This is done by measuring the distance between the eyes, the nose, and the mouse, as well as identifying the shape of the chin. Those distances are then combined and converted into a unique set of numbers—the so-called faceprint.

  3. Recognition refers to actually determining a person’s identity in the input photo. In some applications, this stage is replaced with categorization. In such cases, the algorithms don’t confirm a person’s identity but label the person as belonging to one of the distinct groups, for example, by gender or age.

The applications of facial recognition across industries

Let’s take a look at the key facial recognition applications across different industries.

Applications of facial recognition in the healthcare sector

1. Identifying patients

When incorporated into a hospital’s video surveillance system, facial recognition can simplify patient check-in, freeing hospital workers and patients from paperwork and preventing human error. By “looking” at a patient, a facial recognition system can verify their identity and insurance data, thus speeding up the admission process, laying the basis for a personalized experience, and preventing fraud.

Applications of facial recognition for patient tracking have high patient acceptance rates. Research published on Plos One states that almost 66% of patients find it acceptable for hospital systems to scan their faces for identity verification.

One example of such technology comes from ALCHERA, an AI-powered visual recognition firm. The company offers a facial recognition solution that helps hospitals streamline admission and discharge, identify unusual activities in patients, and control entry into the facilities while flagging unauthorized access.

Biometric technologies, including facial recognition, can also be used to verify the identities of surgical patients, identify patients who are unaccompanied by a medical worker, and track people entering and leaving the premises to prevent security threats.

2. Diagnosing genetic disorders

Facial recognition can help diagnose rare genetic disorders, especially those with mild symptoms.

Delaware’s Nemours Children’s Hospital deployed this technology. As clinicians struggled to diagnose a patient experiencing unusual symptoms, they turned to Face2Gene, a face recognition-powered mobile app that Dr. Karen Gripp, one of the hospital’s employees, helped develop.

The application scans a patient’s photo, mapping their face with 130 landmarks, and uses machine learning to match the detected facial characteristics to those of rare genetic conditions. As a result, the application generates a list of potential diagnoses, each with a probability score. With Face2Gene, the doctors diagnosed that patient with a rare case of Wiedemann-Steiner syndrome.

3. Facilitating mental therapy

Facial recognition helps track patients’ mental health patterns and behaviors. For example, the software can interpret the emotional state and improve the safety of patients prone to risky behaviors, such as removing a breathing tube.

People with special needs, too, can benefit from facial recognition. For example, researchers at Stanford University developed a facial recognition system that runs on Google Glass. It analyzes people’s facial expressions and prompts the wearer with respective cues, like ‘anxious’ or ‘happy.’ The research team claims their solution can help children with autism recognize facial expressions and improve the quality of their social interactions. The trials showed that children who relied on facial recognition software along with standard care showed improvement in socialization as opposed to the control group, who only received traditional care.

And just recently, Apple reported that its Vision Pro augmented reality headset can be potentially used to detect and alleviate mental distress. The headset includes cameras and sensors that can detect various facial expressions and emotions. For instance, the device can identify post-traumatic stress disorder, depression, and some other stress variations. Afterwards, the headset can soothe the user by showing images and playing calming sounds.

Applications of facial recognition in the retail sector

1. Checkout-free software solutions

Contactless payment technology relies on machine learning algorithms to let customers pay for the goods by simply scanning their faces. Since goods are RFID-tagged and a customer’s faceprint corresponds to a specific payment method in a database, it is no longer necessary to manually scan the purchased items or communicate with a cashier.

The first nation’s facial recognition payment system was rolled out in Pasadena in 2020. And since then, different financial organizations have rallied to build and test biometric-based payment systems. J.P. Morgan is piloting a system that will enable customers to pay using a face or a palm scan without the need to show their card. And Mastercard is also experimenting with fingerprint scans and facial recognition models as a payment system.

2. Loyalty programs

Integrated into a store’s CCTV cameras, facial recognition technology can help retailers reward loyal customers without interrupting their buying experience. The moment a loyalty club member enters a store, the facial recognition system identifies the customer and rewards them with a personalized discount or informs them about deals or products they might be interested in.

Cali Group was one of the first US companies to roll out a facial recognition loyalty program. They equipped their restaurants with AI-powered self-service kiosks that identify registered customers and activate their loyalty accounts as soon as they approach the kiosk. The software powering the kiosk may prompt customers to order their favorite meals, and they can pay via facial recognition as well.

3. Personalized shopping experience

A similar approach is taken in this facial recognition application. For example, knowing how much time a particular customer spends in a store helps tailor future experiences to their preferences. And by analyzing their purchase history, retailers may nudge buyers with push notifications advertising products similar to those they have recently purchased.

Retailers can also set up digital signage with face recognition technology to alter product promotions on the fly based on the gender and age of the prospective customer.

4. Store security and fraud prevention

Research shows that theft-induced financial losses in the US surpassed $112 billion in 2022, which is 19% up from the previous year.

Systems that prevent shoplifting are usually targeted at identifying repeat offenders whose photos are already stored in a database. A facial recognition system, thus, doesn’t attribute any personally identifiable information to the face of a shop visitor but searches for its equivalent in the database of known offenders.

The UK’s facial recognition retail security firm, Facewatch, provides a solution that operates on this basis. The company has access to the national facial recognition database. It scans every visitor and sends a notification if a suspicious person enters the facilities. The company claims their solution can reduce store theft by at least 35% during the first year of deployment.

Facial recognition applications in the education sector

1. Campus security

Since the frequent shooting incidents, thousands of schools across the country have deployed facial recognition systems as preventive measures. Here’s how they can operate in a school setting.

A facial recognition system analyzes the faces of people entering or navigating the campus and compares them to a database of authorized individuals, including students, the school’s current staff, and parents, to establish their identity. If a person doesn’t exist in the database or matches the identity of an unwanted person, like an expelled student or a former employee, the system immediately alerts security and automatically denies the visitor entry into the campus area. Modern facial recognition solutions can have additional features, such as object detection, to identify gun-shaped objects.

While US universities are hesitant to deploy this technology, the Chinese educational system is willing to give it a try. Beijing Normal University mounted facial recognition equipment at the entrance to its student dorms. And several other universities and schools experimented with the technology in their classrooms.

2. Attendance monitoring

Tracking attendance used to be a lengthy and tedious process that, despite a fair amount of time spent at the beginning of every class, leads to inevitable gaps and omissions when conducted manually. To fix that, educators are turning to AI-powered educational solutions. These facial recognition applications offer a faster and less disruptive way of tracking attendance.

One example of such a system comes from Australia. Victoria’s Department of Education resorted to facial recognition to monitor the whereabouts of students, letting teachers and staff access attendance data through a web dashboard or a mobile app. Another example comes from India, as Telangana’s government prepares a facial recognition-based attendance management system to be deployed in schools.

The core of the technology can be extended to enable more use cases, for instance, setting up automated book-lending library systems or managing other campus facilities.

3. Increasing learning engagement

Researchers report that analyzing students’ facial microexpressions, like raising eyebrows or tightening eyelids, may help highlight boredom, confusion, delight, frustration, surprise, and other emotions. This can be useful for professors and curriculum designers. For example, when conducting a lecture, a professor may evaluate the emotional state of the attendees and determine the parts of the lecture that spark or weaken interest.

As insights about student engagement come in, faculty may adjust the curriculum to better reflect student preferences and provide a more tailored learning experience.

Different universities are experimenting with this technology. For instance, a professor teaching Computer Technology Information Systems at Guilford College, North California, is working with his team to build a facial recognition system that can tell when students are confused or bored.

Facial recognition applications in the banking and finance sector

1. Customer verification

With financial services going almost entirely digital, it’s only natural for customer verification procedures to follow. Building on eKYC—a digital version of the “know your customer” standard that governs the verification and authentication of a customer’s data—financial institutions can now shift the customer onboarding process entirely online. Especially with research showing that three out of four bank clients accept biometric authentication. While eKYC uses fingerprints as an authentication method, facial recognition may become a viable alternative.

To give a few examples, the Bank of America uses a facial recognition solution from Samsung to allow customers to log into their banking mobile app. And the Orissa High Court in India is pushing for deploying facial recognition technology on ATMs to identify people involved in illegal withdrawals.

2. Cardless ATM transactions

Today, criminals routinely use skimming devices to hack into ATMs. Facial recognition could potentially replace plastic cards and PINs as a more secure option for preventing fraud.

Several banks have already started testing out facial recognition solutions. For example, National Australia Bank partnered with Microsoft to develop a system that allows the bank’s customers to scan their faces in order to access ATM services. The OCBC bank in Singapore allows customers to use ATMs through facial recognition. But there is a condition that the first transaction performed with the technology has to be a balance inquiry. And, since recently, Japan’s Seven Bank has been preparing to enable customers to withdraw and deposit cash through an ATM using face-based verification. This service is set to commence in March 2024. And by September, the bank is planning to expand facial recognition-based services to include opening new accounts.

Facial recognition pros and cons

Facial recognition technology can offer incredible benefits, but there are also challenges that you need to consider to succeed with implementation. Let’s explore.

The benefits of facial recognition technology

The proponents of rolling out facial recognition in enterprise environments say the technology brings about many benefits. Here are the essential ones:

  • Enhanced security. AI-based facial recognition solutions help identify suspicious behaviors, pinpoint known criminals, and ensure safety in crowded venues. Beyond that, facial recognition technology adds convenience and safety to everyday experiences, like using banking services, receiving healthcare, or shopping.

  • Faster service. By replacing current customer authentication procedures with facial recognition, businesses can make it easier for their clients to use the available services. The technology can also facilitate the transition to digital-first experiences, eliminating the need for a customer to be physically present at a venue in order to access the services.

  • Superior customer experience. As a continuation of the previous point, facial recognition also helps increase the quality of customer service, especially in such domains as retail and healthcare. For example, by knowing who enters a store and tapping into their buying habits, retailers may adjust their offer on the go to better suit the needs of the customer. In healthcare settings, facial recognition can also help craft personalized care plans.

  • Improving accessibility for people with visual impairments. Facial recognition applications have found wide use among visually impaired people. Powering your business with facial recognition can help make your services more accessible as well. This way, for example, instead of going through authentication procedures like entering a PIN or filling out papers, a visually impaired customer participates in a face scan and proceeds to use a service.

The challenges that facial recognition technology can bring about

Although capable of driving tangible benefits, facial recognition is not flawless. Among the issues halting a wider adoption of the technology are the following:

  • The accuracy of recognition. The National Institute of Standards and Technology found out that as of April 2020, the best-performing facial identification algorithm showed an error rate of just 0.08%. Still, recognition accuracy is higher when facial recognition technology relies on clear, static images, like ID photos or mugshots. But that is rarely the case in reality.

    A facial recognition software vendor experimented with the technology and discovered that the error rate of a facial recognition algorithm rose from 0.1% when faces were matched against high-quality static images to 9.3% when matched to photos of people captured in public. The error rates were especially high when people weren’t looking directly at the camera.

  • Algorithmic bias. In its 2020 research, the US Government Accountability Office found that facial recognition algorithms powering commercial solutions are subject to inherent bias. They found that the accuracy of facial recognition depends much on the race, ethnicity, gender, and age of the person being recognized. While having no trouble accurately identifying white men, commercially available facial recognition systems show high false-positive rates when applied to the images of people of color, women, children, and the elderly.

    And since 2020, this issue has not been resolved. Just recently, the US Labor Department Office of Inspector General warned about bias in facial recognition models used in unemployment programs.

  • Ethical issues of people monitoring. Another issue is connected to the ethics of managing sensitive biometric information. The opponents of the broader use of facial recognition point out that the problem lies in collecting personal data. Since comprehensive legislation regulating facial recognition technology is still being developed, people’s biometric data remains at risk. What adds up to the point is that some businesses overlook AI explainability and opt for closed-box solutions that can’t explain their outcome.

Implementing a facial recognition solution: tips from ITRex

If you consider rolling out an AI-based facial recognition solution, we strongly recommend committing to doing so ethically. Here are some aspects that you need to investigate before you start building your solution:

  • Select the right approach to development. You will have three options: choosing a readily available solution, going the custom route, or opting for library-based development. To make the right decision, weigh the options against the objectives you intend to achieve—the more specific the task, the higher the need for custom software. But if you develop a facial recognition solution aimed at the general public, it may be a faster option to use an off-the-shelf solution or API, for instance, Microsoft Face API or Amazon Rekognition, or build on an existing facial recognition library, for example, DeepFace, FaceNet, InsightFace, or others.

  • Decide how you will ask people for informed consent for collecting and storing biometric data. While some facial recognition systems can de-identify the information, biometric data can hardly be fully anonymized, so timely informing people is essential for maintaining trust and transparency.

  • Ensure your solution is explainable and involve human employees to resolve any issues. A user should understand why a system has come to a particular decision and revert it in case of false positives or false negatives. For example, when rolling out a facial recognition solution, National Australia Bank intentionally chose to refer all user verification requests that the system could not verify to a human operator rather than rejecting them. This approach reduced the error rate and increased customer satisfaction.

To sum it up

No matter what your line of business is, you are likely to discover exciting facial recognition applications that you can use in your sector. But even though the technology is maturing and getting more affordable, incomprehensive legislation still hinders its promising potential.

Amid public debate about the safety of facial recognition technology, businesses and tech vendors should prioritize building transparent and explainable solutions. As of now, the technology is not 100% accurate, and it can be fooled; a simple false mustache can trick some facial recognition systems, not to mention face spoofing and other more elaborate techniques. Depending on the implications of your facial recognition solution, you might want to add an extra verification layer, such as gait recognition or fingerprint scans.

Here at ITRex, we can analyze your business requirements and recommend an appropriate facial recognition model with a backup verification system, if needed. We are also experienced in explainable AI solutions and can implement a human-in-the-loop approach to assist with accuracy and transparency.

How does facial recognition work?The applications of facial recognition across industriesApplications of facial recognition in the healthcare sectorApplications of facial recognition in the retail sectorFacial recognition applications in the education sectorFacial recognition applications in the banking and finance sectorFacial recognition pros and consThe benefits of facial recognition technologyThe challenges that facial recognition technology can bring aboutImplementing a facial recognition solution: tips from ITRexTo sum it up
Contact us
background banner

If you still have questions about facial recognition technology or look for an AI technology solution boasting facial recognition, contact our consultants. We will answer any of your questions.