FRVT Evaluation Criteria

Previously, we provided an overview of the characteristics and structure of the Face Recognition Vendor Test (FRVT) conducted by the National Institute of Standards and Technology (NIST). The test evaluates the performance and functionality of face recognition algorithms and systems developed by various vendors. ALCHERA achieved high scores in two categories of the FRVT 1:1 verification and secured the top position. In Part 2, we will explore how ALCHERA was able to achieve outstanding results in the FRVT.

 

Evaluation Metrics for FRVT 1:1 Verification Test Metrics

The FRVT 1:1 verification evaluation aims to assess a technology's ability to determine if two distinct photos represent the same person. This assessment involves utilizing two images: a probe image and a gallery. The probe image is used for face recognition attempts, while the gallery consists of images already registered in the system and is used for comparison with the probe. The specific subtests for the 1:1 verification vary depending on the dataset used for the probe and gallery images.

The FRVT 1:1 verification evaluation metric employed is FNMR@FMR. FNMR (False Non-Match Error) represents the rate at which the algorithm erroneously identifies the same person as a different individual, while FMR (False Match Error) signifies the rate at which the algorithm mistakenly identifies a different person as the same individual. FNMR@FMR measures the FNMR at a particular FMR value. It's worth noting that FNMR and FMR exhibit an inverse relationship in relation to the threshold.

To ensure a fair comparison among multiple algorithms, it is essential to establish a standardized criterion. However, utilizing a threshold as the criterion presents a challenge as the FNMR and FMR values vary differently for each algorithm with changing thresholds. To overcome this issue, the metric FNMR@FMR is employed, which enables the comparison of algorithms at a consistent security level. FNMR@FMR measures the FNMR based on a specific FMR value, representing the rate at which a different person is mistakenly identified as the same individual. This approach allows for the evaluation and comparison of all algorithms on an equal footing, regardless of the threshold variations.

 

Evaluation Metrics for FRVT 1:N Identification Test Metrics

The FRVT 1:N Identification Test assesses the performance of algorithms in detecting specific faces from a large gallery of images. Similar to the 1:1 evaluation, the 1:N evaluation consists of specific subtests depending on the dataset used for probe and gallery images. In 1:N identification, the FNIR@FPIR metric is employed. FNIR (False Negative Identification Rate) represents the rate at which enrolled individuals are incorrectly identified as non-enrolled, while FPIR (False Positive Identification Rate) signifies the rate at which non-enrolled individuals are incorrectly identified as enrolled.

 

ALCHERA dominates FRVT with top rankings

ALCHERA's performance in the FRVT was outstanding. The company secured the top position in two categories of the 1:1 verification evaluation: Mugshots with 12+ years and Border. What sets ALCHERA apart is its remarkable stability and ability to achieve high scores even in environments with various variables.
 
ALCHERA has emerged as the top performer in the Korean market and is ranked twelfth globally in the Mugshot. This category evaluates performance in simulated immigration checkpoint scenarios, closely resembling real-world situations. The challenges within this category are significant, as the captured images often vary in terms of angles, lighting conditions, and poses of individuals, resulting in a substantial number of low-quality images. Overcoming these challenges requires a high level of technical expertise, as the field demands the ability to handle diverse environmental variables such as backlighting and changes in lighting.

To enhance accuracy in the Mugshots, advanced technical capabilities are crucial. The development of algorithms and image processing techniques that can adapt to the diverse conditions of capture devices plays a pivotal role. This category serves as a vital technology for security inspections and identity verification (crucial in detecting identity fraud and forged documents) as well as for identifying attempted immigration incidents.
 
Moving on, we come to the Mugshots with 12+ years category. In this category, the evaluation involves comparing frontal, profile, and CCTV images of individuals with a minimum age difference of 12 years. Precisely recognizing aging faces, including the alterations in facial features and skin texture, poses considerable technical complexity. Aging can manifest in diverse manners, such as wrinkles, freckles, and diminished skin elasticity, which vary from person to person. This technology has garnered significant attention as a crucial component for facial authentication solutions, encompassing tasks from remote identity verification to unmanned alcohol sales.

 

ALCHERA's FRVT Challenge

Model Size and Time Limit Configuration

The feature extraction time for images with a resolution of 640x480 was set to a limit of 1500ms. The hardware specifications for the test included the following CPU models:

Intel® Xeon® Gold 6254 CPU @ 3.10GHz
Intel® Xeon® E5-2630 v4 CPU @ 2.20GHz²
Intel® Xeon® E5-2680 v4 CPU @ 2.4GHz²
Intel® Xeon® Gold 6140 CPU @ 2.30GHz³

To accurately estimate the time required for feature extraction based on these hardware specifications, the clock speed of the test CPU was adjusted manually to match the clock speed of the specified CPU model. Additionally, the test was configured to utilize a single core only. Subsequently, the reported time in the FRVT report was compared to the test time using the previously submitted model. This approach enabled a prediction of the potential usage of larger models and provided insights into the expected processing time.

Establishing a Distributed Learning Environment

When it comes to training facial recognition models, the datasets used are extensive. In facial recognition, the number of faces (IDs) and the images associated with each ID have a significant impact on performance. However, training such massive datasets on a single server (node) can be extremely time-consuming. Moreover, limited memory necessitates training with small batch sizes. Unfortunately, smaller batch sizes result in fewer IDs being trained per iteration, making it challenging to achieve optimal performance.

To overcome these challenges, ALCHERA has leveraged the Distributed Data Parallel (DDP) functionality of PyTorch. DDP allows for distributed training across multiple servers (nodes), significantly reducing training time while enabling larger batch sizes.

Constructing a Large-scale Dataset

The importance of data in determining model performance cannot be overstated. However, acquiring and preparing a suitable dataset is both challenging and time-consuming. In an effort to enhance the comparison of their facial recognition models, ALCHERA has diligently constructed their own large-scale test dataset.

 

Demonstrating Leading Technological Capabilities

In Part 2, we explored ALCHERA's FRVT results and their significance. FRVT plays a vital role in the facial recognition field by providing a fair and competitive environment and contributing to remarkable advancements. As transparent and reliable tests like FRVT become more prevalent in facial recognition, the technology ecosystem continues to flourish.

ALCHERA has emerged as the top performer among domestic companies in two FRVT categories, solidifying their position as a prominent player in facial recognition. Leveraging their exceptional technological capabilities, ALCHERA actively supports business innovation in various sectors such as finance and security. By utilizing ALCHERA's advanced expertise, organizations can develop more accurate and efficient facial recognition systems, thereby enhancing their overall security measures.

ALCHERA's facial recognition AI technology is poised to make significant contributions to innovation and progress across diverse fields. With a focus on creating a positive impact on everyday life and society as a whole, ALCHERA remains at the forefront, promoting safety and convenience.