Page 2763
www.rsisinternaonal.org
INTERNATIONAL JOURNAL OF LATEST TECHNOLOGY IN ENGINEERING,
MANAGEMENT & APPLIED SCIENCE (IJLTEMAS)
ISSN 2278-2540 | DOI: 10.51583/IJLTEMAS | Volume XV, Issue V, May 2026
Deepfake Detection System Using Hybrid CNN–VIT Architecture
Vaishali Kapure¹, Hemant Chaudhari², Pratik Bhosale³, Shivraj Jadhav⁴, Kshitij Hulawale⁵
¹ Dept. of CSE (Data Science) G H Raisoni College of Engineering and Management, Affiliated to
Savitribai Phule University Pune, India Pune, India
² Dept. of CSE (Cyber Security) G H Raisoni College of Engineering and Management, Affiliated to
Savitribai Phule University Pune, India Pune, India
³ Dept. of CSE (Cyber Security) G H Raisoni College of Engineering and Management, Affiliated to
Savitribai Phule University Pune, India Pune, India
⁴ Dept. of CSE (Cyber Security) G H Raisoni College of Engineering and Management, Affiliated to
Savitribai Phule University Pune, India Pune, India
⁵ Dept. of CSE (Cyber Security) G H Raisoni College of Engineering and Management, Affiliated to
Savitribai Phule University Pune, India Pune, India
DOI: https://doi.org/10.51583/IJLTEMAS.2026.150500224
Received: 21 May 2026; Accepted: 26 May 2026; Published: 18 June 2026
ABSTRACT
The increasing use of deepfake media is a serious threat because highly realistic fake images and videos can be
generated and posted online, leading to scams, identity theft, and misinformation. Because of how realistic these
images and videos are, it is becoming increasingly difficult to distinguish them manually. Our project provides
a softwarse application that can help determine whether an uploaded image or video is real or fake. It checks
facial characteristics and the overall facial structure to identify characteristic differences between real and fake
media. Additionally, it performs frequency analysis to identify characteristic artefacts that are usually created
when digital processing of images occurs. To ensure that the results are interpretable, the application provides
visual feedback about the facial areas that contributed to the determination, providing users with a clear
understanding of why a particular image or video is real or fake. The project aims to develop a useful and easy
to-use application that can be used for cybersecurity, digital forensics, and online media verification
INTRODUCTION
The increasing reliance on digital media for communication, security, and information systems has also raised
the risk of using manipulated or forged content for malicious purposes [3]. The recent development of artificial
intelligence has enabled us to create very lifelike synthetic images and videos that are difficult to distinguish
from real data using traditional verification methods [1]. This makes the development of accurate and automated
detection systems a high research priority [2].Traditional detection systems are mostly dependent on manual
analysis or hand-crafted features. These systems are very slow, prone to noise, and lack the ability to generalize
when the quality of data changes or new forgery techniques emerge. Even the early versions of deep learning
models, which used a single model such as Convolutional Neural Networks (CNNs), tend to overlook either the
fine details of the image or the overall context of the visual data [7], [8].
Recent developments in deep learning introduce Vision Transformers (ViTs), which use attention mechanisms
to capture long-range dependencies and global context [9]. These models are very effective at understanding
global structure but tend to overlook the local details that are very important for accurate detection [9]. This
problem statement leads to the development of a hybrid framework that combines the benefits of both CNNs
and ViTs [10].
Page 2764
www.rsisinternaonal.org
INTERNATIONAL JOURNAL OF LATEST TECHNOLOGY IN ENGINEERING,
MANAGEMENT & APPLIED SCIENCE (IJLTEMAS)
ISSN 2278-2540 | DOI: 10.51583/IJLTEMAS | Volume XV, Issue V, May 2026
Another major problem associated with automated detection systems is their lack of interpretability. The current
state-of-the-art deep learning models are essentially black boxes that provide predictions without any
explanation. This lack of explanation is a major setback in fields like security, forensics, and decision-making
systems. Explainable AI (XAI) deals with this problem by providing explanations for model decisions and
pointing out the regions of the image that influence the predictions [10].
In this paper, we present a Hybrid CNN-ViT-based Detection System with Explainable AI to overcome the
limitations of current detection systems. The objective of this paper is the design of a detection system that
achieves both high accuracy and maintain its explainability and robustness. By combining the ideas of local
features extraction, global context understanding, and explanations, this paper tries to offer a balanced approach
to detection problems.
LITERATURE SURVEY
Advances in deep learning technique have made significant contribution to the development of automatic
detection systems in various domains, particularly in visual-based applications [1],[2].The prior study on the
combination of machine learning ttechniques with hand crafted features is mostly focused. While the
combination produced some successful results, it was not efficient due to noise, data variations, lack of
adaptability to manipulation strategies.
CNN have resukted in a significant improvement in detection accuracy due to their abitility to learn the spatial
and textual features automatically from visual data [7]. CNN models have been extensively used since they have
been known to excel in capturing local patterns and details [5], [6]. However, certain research indicates that
using only CNN models may fail to capture any longrange dependencies and global context[8].
For this problem, Vision Transforms (VITs) have emerged as new deep learning solution [9]. The ViTs use self
attention techniques to learn representations by capturing relationships between the whole image [9]. Several
research papers have demonstrated the ability of ViTs to achieve similar performance levels as other methods,
particularly for problems tht demand a holistic understanding of the scene [9].
More recent studies have been conducted on hybrid architectures that combine CNNs and Vision Transformers,
aiming to exploit the complementary strengths of both [10]. In these studies, CNNs are used for low-level and
mid-level local feature extraction, while ViTs provide global contextual information via attention mechanisms
[10]. The experimental results of various studies show that hybrid CNN-ViT architectures are more accurate and
robust than standalone models [10]. However, most of these studies are still computationally expensive or lack
validation on real-world datasets [2].
Another important challenge that has been identified in the literature is the lack of interpretability of deep
learning-based detection models [10]. The most accurate models are often black boxes, providing no information
on how the predictions are made. This lack of interpretability hinders their use in high-stakes applications.
Various methods have been proposed using Explainable Artificial Intelligence (XAI) to address this challenge
[10]. These methods include attention visualization and gradient-based methods [10]. These methods are useful
in identifying important regions and features, hence improving interpretability.
Another important challenge that has been identified in the literature is the lack of interpretability of deep
learning-based detection models [10]. The most accurate models are often black boxes, providing no information
on how the predictions are made. This lack of interpretability hinders their use in high-stakes applications.
Various methods have been proposed using Explainable Artificial Intelligence (XAI) to address this challenge
[10]. These methods include attention visualization and gradient-based methods. These methods are useful in
identifying important regions and features, hence improving interpretability.
METHODOLOGY
The proposed methodology provides an automated and robust framework for the detection of B-cell Acute
Lymphoblastic Leukemia (B-ALL) from microscopic blood smear images [7], [8]. The aim of the proposed
Page 2765
www.rsisinternaonal.org
INTERNATIONAL JOURNAL OF LATEST TECHNOLOGY IN ENGINEERING,
MANAGEMENT & APPLIED SCIENCE (IJLTEMAS)
ISSN 2278-2540 | DOI: 10.51583/IJLTEMAS | Volume XV, Issue V, May 2026
system is to provide accurate malignant pattern detection while ensuring interpretability for decision-making
purposes [10]. The proposed framework aims to address the challenges associated with traditional single-model-
based approaches by integrating complementary feature learning strategies that focus on both fine-grained
cellular features and contextual information [10].
Firstly, the input microscopic blood smear images are standardized to ensure consistency within the dataset.
Image preprocessing is performed to improve visual interpretability and minimize variability due to differences
in staining and imaging conditions. This includes resizing images to a fixed resolution, normalizing color
intensity, and improving contrast to highlight morphological features of white blood cells [5], [6]. To address
generalization and minimize overfitting, data augmentation strategies are used to enable the system to learn
robust features from a small number of training samples [2].
After the preprocessing step, the improved images are passed to the feature extraction phase, where the features
are learned. Learning by convulation is used for learning the spatial information that includes edges, textures,
and cell boundaries, which are critical in recognizing leukemia infected cells [7]. At the same time, an attention-
based learning approach is also implemented to identify the global dependencies in the image [9]. The
combination of both learning approaches helps the model learn the structural and global aspects of blood cells
effectively [10].
The conversion from XYZ to CIELAB involves a set of standard equations. You begin with the lightness L*,
which is derived from 116 times a function f of the ratio Y/Yn,minus
16:L* = 116 · f(Y/Yn) – 16 (1)
Then, the a* part of the equation involves the difference between f(X/Xn) and f(Y/Yn), multiplied by 500:
a* = 500 [ f(X/Xn) − f(Y/Yn) ] (2)
Finally, the b* part of the equation involves the difference between f(Y/Yn) and f(Z/Zn), multiplied by 200:
b* = 200 [ f(Y/Yn) − f(Z/Zn) ] (3)
In these equations, Xn, Yn, and Zn are the reference white tristimulus values. The piecewise function f(t) in these
equations is given by:
f(t)={t^(1/3),if t > (6/29)
(1/3) (29/6) t + 4/29, otherwise }This is the standard f(t) function for the XYZ to CIELAB transformation.
The features obtained from the local and global learning approaches are combined to create a single feature
representation [10]. The combination of features helps the model learn the structural and global aspects of blood
cells effectively.
The combined feature vector is then passed through a classification layer, where the probabilistic output is
obtained for each class [1], [2]. This helps the system not only make a final diagnosis but also provide confidence
in the predictions. To make the system more transparent and meaningful from a clinical perspective,
interpretability approaches are also employed in the system [10]. These approaches help create visual
explanations of the system by pointing out the regions in the image that have the most influence on the system’s
predictions [10]. These visual explanations help clinicians confirm whether the system is looking at biologically
meaningful regions, thereby helping to build trust in the system
System Architecture
The architecture of the proposed Hybrid CNN–ViT Deepfake Detection Framework is designed to detect
manipulated facial media by combining spatial, global, and frequency analysis in a single system.
Page 2766
www.rsisinternaonal.org
INTERNATIONAL JOURNAL OF LATEST TECHNOLOGY IN ENGINEERING,
MANAGEMENT & APPLIED SCIENCE (IJLTEMAS)
ISSN 2278-2540 | DOI: 10.51583/IJLTEMAS | Volume XV, Issue V, May 2026
Fig. 4.1: System Architecture
A. Input Layer and Data Acquisition :
The system takes input from the user in the form of images or videos through a web interface. The video frames
are sampled periodically to enable the analysis of multiple frames.
B. Preprocessing Layer: Facial Region Standardization.
Detection and extraction of face, elimination of background noise, sizing of the face image and normalization of
pixel values takes place here.
C. Spatial Feature Extraction Layer: CNN Branch
The CNN extracts local features like edges, textures, and small facial artifacts to detect fine-level inconsistencies.
D. Global Feature Extraction Layer: Vision Transformer (ViT) Branch
The ViT analyzes overall facial structure by dividing the image into patches and understanding relationships
between regions.
Page 2767
www.rsisinternaonal.org
INTERNATIONAL JOURNAL OF LATEST TECHNOLOGY IN ENGINEERING,
MANAGEMENT & APPLIED SCIENCE (IJLTEMAS)
ISSN 2278-2540 | DOI: 10.51583/IJLTEMAS | Volume XV, Issue V, May 2026
E. Frequency-Domain Analysis Layer
FFT is applied to detect hidden patterns and artifacts not visible in normal images.
F. Feature Fusion and Integration Layer
Features from CNN, ViT, and FFT are combined to create a strong representation of the input.
G. Output Layer: Classification
The system classifies the input as real or fake using a probability-based prediction.
H. Explainability Module: Grad-CAM
Grad-CAM generates heatmaps to highlight important regions and explain the model’s decision.
RESULT
This section discusses the performance of the proposed deepfake detection system. We begin by testing different
pre-trained models on both the training and test data to identify the best-performing model. After hyperparameter
optimization, the efficiency and robustness of the identified model are measured using the test data.
A. Efficient Model Selection
Different pre-trained deep learning models were trained on the training data and tested on the test data for a
maximum of 30 epochs under the same experimental environment. Performance of each model was evaluated
on the basis of parameter like accuracy, precision, recall, and model size/complexity. These performance
statistics have beenlisted below in Table IV.
Upon evaluation of the performance, it was noticed that even though all models performed efficiently based on
their accuracy, the models performed efficiently based on their accuracy, the MobileNetV2 Proved to be the most
efficient model in the sense that it offered very high accuracy using very few parameters, less model size, and
lesser computational complexity.
B. Performance of the Selected Model
The behavior of the model selected during the training process is illustrated in Figure 3, which shows the training
and validation accuracy and loss vlues. With an increase in the number of epochs, there was a corresponding
reduction in the loss value and an increase in the accuracy value.
The model showed excellent generalization capabilities on novel test images, successfully identifying the
discriminative visual cues of manipulated images to distinguish real from deepfake images. The MobileNetV2
architecture provided excellent accuracy and efficiency, thus confirming its position as the final model for the
system.
C. Confidence Estimation
To improve the confidence of the deepfake classifier, confidence estimation strategies were employed.
Temperature scaling was used as a calibration technique to refine the predicted probability distribution. Adding
a temperature parameter (T > 1) to the softmax function helped to mitigate overconfidence, providing more
calibrated and interpretable confidence estimates.
This calibration helps to make the predicted probabilities more closely related to the actual probability of
correctness, thus improving the system’s credibility for practical applications.
Page 2768
www.rsisinternaonal.org
INTERNATIONAL JOURNAL OF LATEST TECHNOLOGY IN ENGINEERING,
MANAGEMENT & APPLIED SCIENCE (IJLTEMAS)
ISSN 2278-2540 | DOI: 10.51583/IJLTEMAS | Volume XV, Issue V, May 2026
Fig .5.1 Training and validation accuracy and loss curves for the proposed model. The model converged after a
small number of epochs, with high accuracy and low loss.
Table 4.1 Comparative Analysis of Deepfake Detection Models
CONCLUSION
This paper presents an efficient deepfake detection system solution that attempts to classify images automatically
either as authentic or fake. In this regard,the solution presented focuses on the process of detecting subtle clues
that are used during deepfake image creation through the use of deep learning techniques such as convulational
Neural Networks (CNNs) and vision transformer. The proposed solution applies the concept of transfer learning
where local spatial features can be learned using the models based on CNNs and global contextual dependencies
using models based on ViT.
Furthermore, the solution implements advanced training techniques including data augmentation, learning
methodologies, and temperature scaling for confidence calibration
From the experiment analysis, it is clear that the solution system designed exihibit high level of accuracy and
stability for deepfake image detection, even in difficult conditions.
Evaluation done experimentally prove that the suggested solution system is very accurate and stable for deep
fake images detection in very difficult situations. Also, the suggested full-stack solution system consists of
friendly web UI and a robust backend which is built using Flask/FastAPI, Streamliy, Gradio, and TensorFlow
for the real time deep fake images detection.
Page 2769
www.rsisinternaonal.org
INTERNATIONAL JOURNAL OF LATEST TECHNOLOGY IN ENGINEERING,
MANAGEMENT & APPLIED SCIENCE (IJLTEMAS)
ISSN 2278-2540 | DOI: 10.51583/IJLTEMAS | Volume XV, Issue V, May 2026
Overall, the suggested solution system offers a highly effective solution to tackle deepfakes' emerging threats
and can be employed in practical applications like digital forensics, surveillance on social media platforms, and
content verification systems.
REFERENCE
1. T. Nguyen and L. Tran, “Deepfake detection using CNN-based spatial and temporal features,” IEEE
Access, vol. 12, pp. 3456–3472, 2024.
2. Y. Zhang and H. Chen, “Multi-model ensemble learning for robust deepfake detection,” IEEE
Transactions on Artificial Intelligence, vol. 5, no. 2, pp. 112–130, 2024.
3. R. Kumar and P. Singh, “The arms race between deepfake generation and detection,” IEEE Signal
Processing Magazine, vol. 41, no. 1, pp. 45–61, 2024.
4. J. Li and F. Zhao, “Advanced Deepfake Video Detection Using Temporal
5. Models,” in Proc. IEEE/CVF CVPR Workshops, pp. 78–92, 2024.
6. A. Rössler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, “FaceForensics++: Learning
to detect manipulated facial images,” in Proc. IEEE Int. Conf. Computer Vision (ICCV), pp. 1–11, 2019.
7. B. Dolhansky, R. Howes, B. Pflaum, N. Baram, and C. Ferrer, “The DeepFake detection challenge
(DFDC) dataset,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 10438–
10447, 2020.
8. D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen, “MesoNet: A compact facial video forgery detection
network,” in Proc. IEEE Int. Workshop on Information Forensics and Security (WIFS), pp. 1–7, 2018.
9. Y. He et al., “TransForensics: Image forgery localization with dense self-attention,” in Proc. IEEE/CVF
Int. Conf. Computer Vision (ICCV), pp. 1501–1510, 2021.
10. Y. Liu, Y. Li, and S. Wang, “Vision transformer-based deepfake detection with frequency-aware
learning,” IEEE Access, vol. 11, pp. 99872–99884, 2023.
11. C. Zhao, G. Wang, and X. Li, “Hybrid CNN–transformer framework for robust deepfake image
detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 9, pp. 4562–
4575, 2023.
12. Y. K. Mali, V. U. Rathod, S. Dargad, V. N. Kapure, N. Vyawahare, and S. Gajarlewar, “Development of
ROS (robotic operating system) for fault detection of underwater cables,” in Proc. 2023 IEEE 11th
Region 10 Humanitarian Technology Conference (R10-HTC), pp. 956–961, 2023.
13. V. J. Shimpi and R. R. Ade, “Understanding 2016 drought in India by social media data mining,”
Communications on Applied Electronics (CAE), vol. 5, no. 6, pp. 1–5, 2016.
14. A. Patle, V. Kapure, K. B. Chaure, A. S. Patil, V. R. Pujari, and H. P. Deshmukh, “Minefield evacuation
proximity drift safety alert,” in Proc. 2024 International Conference on Computing, Sciences and
Communications, 2024.
15. V. J. Shimpi and R. Ade, “A review: Social media data mining for understanding and analyzing different
issues in society,” International Journal of Computer Applications.
