The widespread adoption of facial masks during the COVID-19 pandemic significantly challenged existing facial

recognition systems by occluding critical biometric features. This paper proposes a Vision Transformer (ViT)

based approach for robust Masked Face Recognition (MFR). Unlike traditional Convolutional Neural Networks

(CNNs) that rely on local receptive fields, the ViT architecture utilizes global self-attention to capture long-range

Facial recognition technology has faced unprecedented hurdles due to the global mandate for facial masks.

Masks occlude the nose, mouth, and chin, which are vital for identity verification, leading to a substantial decline

in the performance of standard recognition models [12], [13]. Research indicates that traditional CNN-based

systems, such as FaceNet, experience significant degradation when processing masked images because their

local feature extraction is easily disrupted by the non-linear occlusions of varying mask types [7].Vision

Transformers (ViTs) have emerged as a promising alternative for MFR due to their ability to model global

context. By processing images as sequences of patches and employing self-attention, ViTs can effectively

integrate information from non-occluded regions, such as the periocular and forehead areas, to compensate for

the missing data in the lower face [2], [6]. This paper details the implementation of a ViT-based recognition

recently, pure transformer models have shown superior performance. The Masked Face Transformer (MFT)

introduces Masked Face-compatible Attention (MFA) to suppress interactions between masked and non-masked

patches, thereby reducing noise in the final embedding [3], [6]. Additionally, "FaceT" employs a proxy task of

patch reconstruction to stabilize the training of the ViT backbone, which otherwise struggles to converge when

trained from scratch on small facial datasets [2], [4]. Other researchers have explored contrastive learning

(ViTEmbedding) to learn features that remain invariant to mask presence [1].

Vision Transformer (ViT) is a deep learning architecture that applies the transformer model to images. Instead

of relying on convolutions, ViTs use self-attention to capture relationships across all image patches, enabling a

global understanding of the image. ViT treats an image as a sequence of fixed-size patches and applies self-

of looking at the whole face at once, the model breaks it into a grid of pieces.

2. Linear Projection of Flattened Patches: Each 2D patch is flattened into a 1D vector. These vectors are

then passed through a linear layer to create patch embeddings, which are numerical representations of

the visual data in that specific square.

3. Positional Embedding: Since Transformers are permutation invariant, positional encodings inject spatial

order so the model knows the relative positions of patches. Since Transformers treat tokens as unordered

positional encodings are added to retain spatial structure and patch location information. ViT uses

learnable positional vectors to capture local and global spatial relationships adapting better than fixed

encodings across image resolutions.

4. The Transformer Encoder: This is the core engine. It uses Multi-Head Self-Attention to allow every patch

6. Feed-Forward Network (FFN): The FFN transforms each patch embedding to a higher-dimensional space

and back using two dense layers with a GELU activation, enabling complex feature learning. It operates

independently on each token with shared weights, allowing efficient non-linear transformations.

7. Layer Normalization: LayerNorm normalizes features across the input, stabilizing training and reducing

internal covariate shift. Pre-LN ensures well-conditioned gradients and consistent scaling across tokens

in deep Transformers.

8. MLP Head (classification): finally Converts the CLS token output into class probabilities using a small

final decision-making component.

The methodology for this study follows a systematic pipeline encompassing data preparation, architectural

while a separate unseen test set of 500 images is reserved for final performance validation. Both datasets are

organized using a directory structure, partitioned into two primary classes: masked_MFR2 and

unmask_MFR2.In the preprocessing pipeline, to ensure consistency across the input stream, all images undergo

a standardized transformation sequence:

 Resizing: Spatial dimensions are interpolated to 224 x 224 pixels to match the input requirements of the

transformer backbone.

 Tersorization: Raw pixel data is converted into PyTorch tensors.

 Normalization: A global normalization is applied across the RGB channels using a mean value of 0.5 and a

standard deviation of 0.5, which centers the data distribution and improves training convergence.

The model training is conducted using a supervised learning paradigm with the following hyperparameters:

 Optimization: We utilize the Adam optimizer with a fixed learning rate of 1 × 10⁻⁴ to manage weight updates.

To provide a comprehensive view of the model's diagnostic power, we employ a multi-faceted evaluation

strategy:

1. Global Performance: Overall test accuracy is calculated on the 500-image hold-out set.

2. Per-Class Granularity: A detailed classification report is generated to extract Precision, Recall, and

3. Inference Testing: Individual image inference is conducted on specific subjects like as

The Masked Face Recognition Dataset Version 2 (MFR2) is a real-world benchmark dataset designed to evaluate

the performance of face recognition systems under masked conditions. The dataset includes identities of 53

A key strength of MFR2 lies in its real-world complexity. The dataset includes a wide range of variations in

mask types, including surgical masks, cloth masks, and N95 masks. Additionally, it captures diverse conditions

in terms of facial poses and lighting, making it suitable for robust evaluation of face recognition models in

practical scenarios.

The performance of the ViT model on the MFR2 benchmark demonstrates the efficacy of global attention over

local convolution for occluded faces.

training loss curve demonstrates a highly efficient optimization process, characterized by a steep exponential

accurately mapped the input features to the Uddhav Thakare class label, demonstrating high predictive reliability

at the granular level. This individual success is supported by the broader class-specific metrics, the model

achieved a high accuracy of 96.49% for this specific category within the test set. This indicates that for images

to capture global dependencies, which are critical when key facial features are obscured. By leveraging the global

context provided by self-attention and utilizing a structured 70/15/15 dataset split, we demonstrated that

transformers can effectively overcome the challenges posed by facial masks. The model shows strong

10.1109/access.2021.3135255