# AVTENet: Audio-Visual Transformer-based Ensemble Network Exploiting Multiple
Experts for Video Deepfake Detection

Ammarah Hashmi, Sahibzada Adil Shahzad, Chia-Wen Lin, , and Yu Tsao, , Hsin-
Min Wang Ammarah Hashmi is with the Social Networks and Human-Centered
Computing Program, Taiwan International Graduate Program, Academia Sinica,
Taipei 11529, Taiwan, and also with the Institute of Information Systems and
Applications, National Tsing Hua University, Hsinchu 30013, Taiwan. (e-mail:
hashmiammarah0@gmail.com). Sahibzada Adil Shahzad is with the Social Networks
and Human-Centered Computing Program, Taiwan International Graduate Program,
Academia Sinica, Taipei 11529, Taiwan, and also with the Department of
Computer Science, National Chengchi University, Taipei 11605, Taiwan. (e-mail:
adilshah275@iis.sinica.edu.tw). Chia-Wen Lin is with the Department of
Electrical Engineering and the Institute of Communications Engineering,
National Tsing Hua University, Hsinchu 300044, Taiwan. (e-mail:
<EMAIL_ADDRESS>Yu Tsao is with the Research Center for Information
Technology Innovation, Academia Sinica, Taipei 11529, Taiwan. (e-mail:
yu.tsao@citi.sinica.edu.tw). Hsin-Min Wang is with the Institute of
Information Science, Academia Sinica, Taipei 11529, Taiwan. (e-mail:
<EMAIL_ADDRESS>
###### Abstract

Forged content shared widely on social media platforms is a major social
problem that requires increased regulation and poses new challenges to the
research community. The recent proliferation of hyper-realistic deepfake
videos has drawn attention to the threat of audio and visual forgeries. Most
previous work on detecting AI-generated fake videos only utilizes visual
modality or audio modality. While there are some methods in the literature
that exploit audio and visual modalities to detect forged videos, they have
not been comprehensively evaluated on multi-modal datasets of deepfake videos
involving acoustic and visual manipulations. Moreover, these existing methods
are mostly based on CNN and suffer from low detection accuracy. Inspired by
the recent success of Transformer in various fields, to address the challenges
posed by deepfake technology, in this paper, we propose an Audio-Visual
Transformer-based Ensemble Network (AVTENet) framework that considers both
acoustic manipulation and visual manipulation to achieve effective video
forgery detection. Specifically, the proposed model integrates several purely
transformer-based variants that capture video, audio, and audio-visual salient
cues to reach a consensus in prediction. For evaluation, we use the recently
released benchmark multi-modal audio-video FakeAVCeleb dataset. For a detailed
analysis, we evaluate AVTENet, its variants, and several existing methods on
multiple test sets of the FakeAVCeleb dataset. Experimental results show that
our best model outperforms all existing methods and achieves state-of-the-art
performance on Testset-I and Testset-II of the FakeAVCeleb dataset.

###### Index Terms:

Deepfake detection, Video forgery detection, Audio-visual feature fusion,
Transformer, AVTENet

##  I Introduction

Figure 1: An example of audio-visual deepfakes. Cyber-attackers aim to create
convincing counterfeit videos by manipulating audio and visual streams.
Sharing such malicious content can have harmful consequences. Therefore, our
proposed method exploits manipulations in both audio and visual streams to
effectively detect forged videos.

Advances in social media and machine learning technology have made it easier
to generate and spread quickly hyper-realistic counterfeit content, commonly
known as Deepfakes. Deepfake refers to forged media content in which the face
and/or voice of the source person has been manipulated with the face and/or
voice of a target person using deep learning algorithms. The rapid development
of modern deep learning technology is driving multimedia processing, achieving
incredible results in transforming multimedia content (images, audio, and
video). Nowadays, it is a cinch to create or manipulate widely available
online media content through various advanced tools and techniques; on top of
that, the availability of large-scale datasets grants supplementary potential
to deepfake generative models. Models such as Generative Adversarial Networks
(GANs) [1] and Variational Autoencoders (VAEs) [2] can create highly realistic
synthetic media content that is indistinguishable. Face Swapping, Face
Reenactment, Face Editing, and Face Synthesis are different categories of
deepfake techniques that help produce convincing and well-crafted facial
deepfake media [3], with face swapping being the most common visually
manipulated deepfake today. Similarly, Text-to-Speech Synthesis (TTS) and
Voice Conversion (VC) are audio deepfake generation techniques that enable
deception by cloning an individual’s voice. In addition to deepfake audio
generation, automatic speaker verification (SV) systems help recognize the
speaker’s identity or identify the speaker in the audio [4, 5]. It has been
observed that existing SV systems are easily spoofed by morphing the audio
signal. State-of-the-art (SOTA) deepfake generative models have a wide range
of beneficial applications [6]; however, deepfakes can have potentially
serious adverse effects and can pose a threat when used for malicious
purposes, such as revenge porn, cyber-crime, misinformation, political
denigration, privacy, legal and ethical issues. Misinformation or
disinformation [7] spread through deepfakes can lead to chaos or political
instability [8], while cyber-attacks can escalate the threat of access to
sensitive information. Prevalent misinformation on online social media leads
to the spread of unverified rumors and can lead to social disputes. Therefore,
effective deepfake detection methods are urgently needed to mitigate these
potential harms, and video deepfake detection has become a focus for media
forensics researchers.

At first instance, deepfake content involves uni-modal manipulation in audio
or vision. But more recently, multi-modal manipulation has been noticed in
generating more realistic deepfake videos. The viral Instagram video of Mark
Zuckerberg (Meta’s CEO), as shown in Fig. 1, is the ultimate example of audio-
visual deepfakes that combine acoustic and visual forgeries. Detecting such
convincing deepfake videos is a crucial but challenging task. To combat the
trend of deepfakes, several deep learning-based solutions have been proposed
to detect forgeries in videos. However, most existing models only focus on
uni-modal detection of video deepfakes, and lack evaluation on multi-modal
deepfake datasets. Uni-modal methods limit the performance of detectors to
certain scenarios; for example, a video-only detector cannot detect acoustic
manipulation, while a detector that relies solely on the audio modality can
easily be fooled by visual manipulation. Moreover, the lack of benchmark
multi-modal datasets that contain both acoustic and visual forgeries hinders
the training of multi-modal video deepfake detectors, as existing video
datasets overlook audio deepfakes and multi-modal deepfakes. Therefore, to
address the above issues, our study focuses on an audio-visual approach
evaluated on a multi-modal deepfake dataset to take a step toward applications
in real-world scenarios.

Figure 2: Illustrations of the audio and visual streams of some samples
selected from the FakeAVCeleb Dataset, where real streams are indicated in
green, and fake ones are indicated in red.

Recently, transformers [9] have emerged as a promising alternative and marked
a dominant shift from Convolutional Neural Networks (CNNs) in computer vision
tasks due to their ability to model long-range dependencies and capture global
context. The sequential processing nature of CNN limits the network’s ability
to capture long-range dependencies; however, the attention mechanism in the
transformer allows it to model long-range global context and relationships.
Existing methods mainly use CNN-based solutions to deal with deepfake
detection [10, 11, 12, 13, 14, 15, 16, 17]. These CNN-based methods process
input data sequentially, while transformers can process it efficiently by
processing the entire input sequence in parallel. Inspired by transformers’
success in various fields, we intend to apply it to the video forgery
detection task by leveraging acoustic and visual features. We believe that
multiple modalities provide complementary information and stable inferences.
Therefore, to fill the above gap, we propose AVTENet, a novel Audio-Visual
Transformer-based Ensemble Network that utilizes both acoustic and visual
features for effective video forgery detection. The main contributions of our
work are summarized as follows:

  * •

We propose an audio-visual transformer-based ensemble network, AVTENet,
demonstrating that audio-only, video-only, and audio-visual transformer-based
networks can be effectively combined to improve robustness and detection
accuracy. The proposed approach can address scenarios where only a single
modality or both modalities in audio vision are manipulated.

  * •

To understand the strengths and weaknesses of various ensemble approaches, we
present four variants of AVTENet and show that the feature fusion approach
performs better than majority voting, score fusion, and average score fusion.

  * •

We experimentally confirm the effectiveness of AVTENet and empirically
demonstrate that our transformer-based ensemble model outperforms the previous
CNN-based ensemble network [10].

  * •

To evaluate the proposed method, we use the FakeAVCeleb dataset [18], which is
a multi-modal audio-visual deepfake dataset. We conduct a comprehensive
experimental comparison between the proposed method and existing single-modal
and multi-modal methods. The results confirm that our proposed method achieves
the highest performance on the test sets of the FakeAVCeleb dataset,
outperforming all compared methods.

The remainder of this paper is organized as follows. Sec. II reviews related
work. Sec. III illustrates the proposed methodology. Sec. IV outlines the
experimental setup and reports the results. Finally, Sec. V provides
conclusions and future work.

##  II Related Work

###  II-A Video Deepfake Generation

Deepfake technology has become more sophisticated as it uses artificial
intelligence (AI) techniques to generate, manipulate, or alter video content,
thereby adding significant threats to society, legislation, individuals,
community consensus, journalism, and cyber-security. Face2Face [19], Neural
Talking-Heads [20], First Order Motion Model for Image Animation [21], and
Deep Video Portraits [22] are some of the deepfake generation techniques.
Likewise, there exist different types of deepfake videos, including lip-
syncing, face-swapping, and voice manipulation. Generative Adversarial
Networks (GANs) [23] become a popular deep-learning method to generate
realistic images or videos. For instance, in [20] and [24] GAN-based video
deepfake generation methods were proposed. The method proposed in [24]
generates a dance video of a person from a single image. First, a sequence of
human poses is generated and rendered in the video. To generate realistic
animated dance videos, this method combines pose estimation, key-point
detection, and image synthesis. Similarly, the few-shot learning method
proposed in [20] uses a few reference images to generate realistic videos of
talking heads. Their method creates a realistic video sequence of a target
person speaking by mapping a number of reference images. Another GAN-based
method for image-to-image translation was proposed in [25]. To ensure high
perceptual qualities of the generated images, the model was trained with a
perceptual loss function. The authors evaluated their method on several image
generation tasks, such as image colorization, painting, and super-resolution.

###  II-B Video Deepfake Detection

####  II-B1 Uni-modal Deepfake Detection

Previous work has mainly focused on a single modality to detect forgeries in
videos since synthetic videos generally have less manipulation in the audio
track. Some researchers have worked to detect forgeries in videos by
exploiting visual artifacts such as inconsistent facial expressions and
movements [26, 27], irregular head movements or angles [28, 26], mismatched
shadows or lightening [29], distorted or blurred edges around the face or body
[30], unnatural eye blinking or movement [31], unnatural background or scenery
[32]. The authors of [33] argue that several facial editing techniques exhibit
visual artifacts in videos that can be tracked and used to detect video
forgeries. Considering this rationale, they demonstrated that even simple
visual cues can reveal deepfakes. The authors of [31] used eye blinking, a
physiological signal that is not well presented in synthetic videos, to detect
fake face videos. Other researchers have exploited emotional inconsistency
[26], synchronization between speech and lip movements [11], or motion
analysis [34]. A biometrics-based forensics technique was proposed in [35] to
detect faceswap deepfakes by exploiting appearance and behavior, especially
facial expressions and head movements. MesoNet and MesoInception are two
popular models proposed in [12]. Both are compact CNN-based models with a few
layers that exploit mesoscopic properties to detect facial tampering in
videos. MesoNet analyzes the residual high-frequency content of video frames.
XceptionNet [13] is built on depth-wise separable convolutional layers. The
authors of [14] and [15] also proposed other CNN-based methods for video
forgery detection. The method proposed in [14] employs composite coefficients
that effectively scale all dimensions of depth, width, and resolution
uniformly. The method proposed in [15] utilizes audio and video streams to
extract features and then combines these features to better detect deepfake
videos. A transformer network based on an incremental learning strategy is
proposed in [36], which utilizes face images and their UV texture generated by
a 3D face construction method using a single-face input image to detect
synthetic videos. The same authors later proposed a hybrid transformer network
to detect forged videos [37]. The transformer network is trained end-to-end
using XceptionNet and EfficientNet-B4 as feature extractors.

####  II-B2 Multi-modal Deepfake Detection

Considering that the auditory context plays a crucial role in visual object
perception [38], few recent attempts have addressed the challenge of detecting
multi-modal deepfake videos using multiple modalities such as video, audio,
and text. For instance, inspired by the Siamese networks and triplet loss, a
learning-based approach is proposed in [39], which uses acoustic and visual
features of the same video to enhance the information for learning. The
authors extracted affective cues and used them to compare the emotions
perceived in the two modes to discern whether the video was genuine or
manipulated. The authors of [40], [41] and [42] combined audio and visual
modalities to develop a video deepfake detector that can handle audio-only,
video-only, and audio-visual manipulations. In [40], a promising approach is
proposed to exploit the intrinsic synchronization between audio and visual
modalities to potentially detect increasingly prevalent deepfakes. In [41], an
audio-visual person-of-interest deepfake detector that exploits specific
biometrics of people is proposed to cope with the variety of manipulation
methods used in synthetic media generation. The high generalizability of the
detector is due to its independence from any manipulation techniques and
training on videos of real talking faces. Similarly, for audiovisual forgery
detection, authors in [42] incorporate audio loss, video loss and audiovisual
loss to capture the inconsistency of multi-modality and artifacts from
individual modalities. Additionally, ensemble approaches were used in [43] and
[10] to improve the accuracy of deep fake detection. In [43], incorporating
supplementary textual features has been shown to be beneficial in effectively
handling multi-modal fake content detection. The authors built an ensemble
network combining uni-modal and cross-modal classifiers to distinguish forged
from genuine video clips. On the other hand, the ensemble method in [10]
combines CNN-based audio-only, video-only, and audio-visual networks to detect
multi-modal deepfakes. Furthermore, the authors of [16] argue that the
manipulation of audio or visual flow in video can lead to a lack of harmony
between them, such as speech-lip non-synchrony and unnatural facial movements.
Therefore, they utilized modality dissonance scores to measure the
dissimilarity between two modalities to identify genuine or spoofed videos.
Another deep learning method that exploits the audio and visual modalities and
considers the importance of speech-lip synchronization for the task of video
forgery detection is proposed in [11]. Their audio-visual deepfake detector
checks the synchrony between the lip sequence extracted from the video and the
synthetic lip sequence generated from the audio track. Similarly, authors in
[44] adopted a self-supervised transformer-based approach that utilizes
contrastive learning. Their method allows paired acoustic and visual streams
to learn mouth motion representations propelling paired representation to be
closer and unpaired to stay farther in order to determine audio-visual
forgery. Another self-supervised learning approach in [45] exploits an audio-
visual temporal synchronization by evaluating consistency between the acoustic
stream and faces in a video clip to determine forgery. Although uni-modal
methods have been extensively studied in video forgery detection, the
underlying correlations implicit between different perception modes have not
been fully explored and exploited.

###  II-C Multi-modal Deepfake Datasets

To the best of our knowledge, DFDC [46] and FakeAVCeleb [18] are the only
multi-modal datasets that include both acoustic and visual manipulations in
videos.

The DFDC dataset [46] is larger than the FakeAVCeleb dataset, containing
100,000 video clips collected from 3,426 paid subjects. DFDC provides a
benchmark for evaluating the performance of deepfake detection models and
serves as a crucial resource for advancing research in the field of deepfake
detection. It has been released as part of a competition to promote the
development of deepfake detection methods. The deepfake videos in this dataset
were generated using various deepfake techniques, such as GAN-based methods
and non-learning methods. It consists of real and fake videos and considers
both acoustic and visual manipulations.

The FakeAVCeleb dataset [18] is a multi-modal audio-visual dataset released
after DFDC. The dataset was carefully generated through multiple manipulation
methods, taking into account the balance among ethnic backgrounds, gender, and
age groups. The real videos of 500 celebrities originating from the Voxceleb2
dataset [47] collected from YouTube form the base set. This base set was
further used to generate 20,000 deepfake videos through manipulation
techniques, including Faceswap [48], FSGAN [49], Wav2Lip [50], and RTVC (real-
time-voice-cloning) [51].

Based on its advantages mentioned above, we use the FakeAVCeleb dataset to
evaluate our multi-modal deepfake detector AVTENet. Fig. 2 shows a few samples
of each category from the FakeAVCeleb dataset. We do not use the DFDC dataset
due to its extreme environmental settings, e.g., some instances are poorly
illuminated or overexposed, or sometimes the subject’s face may not be facing
the camera. Another reason is that its labels lack separate labels for the
acoustic and visual modes in the video.

Figure 3: Overview of the proposed AVTENet model for detecting video forgeries
by simultaneously utilizing acoustic and visual cues. It consists of a video-
only network (VN), an audio-only network (AN), and an audio-visual network
(AVN), each of which is an independent transformer-based classifier, and a
decision-making module (DM) that produces final predictions based on the three
classifiers through various ensemble strategies.

##  III Proposed Method

The natural auditory context provides distinctive, independent, and diagnostic
information about the visual world, which directly impacts the perceptual
experience of visual objects [38]. To combat the emerging threat of video
deepfakes, we leverage both acoustic and visual information to detect
forgeries in videos. We use the transformer-based model because its self-
attention mechanism can catch inconsistencies or manipulations to identify
video forgeries. Furthermore, we employ ensemble learning to exploit the
fusion of complementary information, modality-specific patterns, and multi-
modal features to improve the robustness of the detection system. Accordingly,
we propose an audio-visual transformer-based ensemble network (AVTENet) that
fully exploits acoustic and visual information to detect video forgeries by
leveraging modality-specific cues and complementary insights of joint audio
and visual information. Fig. 3 shows an overview of our proposed AVTENet
model, which consists of three key networks, namely a video-only network (VN),
an audio-only network (AN), and an audio-visual network (AVN), as well as a
decision-making module (DM). The three key networks are different-modality
transformer-based networks integrated with pre-trained models through self-
supervised learning (SSL) or supervised learning (SL). Then, DM integrates the
outputs of these three classifiers according to different fusion strategies.
Given a test video $x$, the decision is made according to

${{\mathrm{AVTENet}(x)}={\mathrm{DM}(C_{v}(x)},{C_{a}(x)},{C_{av}(x))}},$ (1)

where $C_{v}$, $C_{a}$, and $C_{av}$ denote the video-only, audio-only, and
audio-visual classifiers, respectively, and $\mathrm{DM}$ denotes the function
of decision making.

###  III-A Video-only Network (VN)

The VN module extracts relevant spatiotemporal features from video frames
through its self-attention mechanism to capture spatiotemporal patterns and
long-range dependencies, and then outputs a vector representation that
captures essential visual information.

Inspired by the remarkable progress achieved by the video vision transformer
ViViT [52] in video classification tasks, we seamlessly integrate it into the
visual backbone of AVTENet. As shown in the top part of Fig. 3, we use a
factorized encoder model that consists of two separate transformer encoders.
The first is a spatial encoder, which delves into the interaction among tokens
extracted from the same temporal index. The second is a temporal encoder that
models interactions among tokens from different temporal indices. We first
divide the input video into small segments, a series of patches is extracted
from each segment in the form of a tubelet and passed through the spatial
encoder to produce an encoding vector representing the segment. The output
encoding vectors serve as input to the temporal transformer along with an
additional classification (cls) token to extract the final representation of
the entire video.

For VN training, a dataset $D^{v}=\\{v_{i},y_{i}\\}_{i=1}^{n}$ is extracted
from the training set of the FakeAVCeleb dataset, where $v_{i}$ denotes the
video stream of the $i$-th training sample $x_{i}$, and $y_{i}$ is the label
for $v_{i}$ (0 for fake and 1 for real). As shown in Fig. 3, the ViViT model
extracts the single-vector representation $E_{v}$ (the $cls$ token of the
temporal transformer) of each training video clip. A linear layer is used as
the classifier. The ViViT model is pre-trained on the Kinetics dataset [52].
During the VN training process, not only is the linear classification layer
trained, but also the pre-trained ViViT model is fine-tuned. The model is
trained with the binary cross-entropy loss defined as

${\mathcal{L}_{\mathrm{BCE}}=-\frac{1}{n}\sum_{i=1}^{n}y_{i}\log(\hat{y}_{i})+\left(1-y_{i}\right)\log\left(1-\hat{y}_{i}\right)},$
(2)

where $n$ is the total number of training samples, and $\hat{y}_{i}$ denotes
the predicted score.

In the inference phase of VN, given a test sample $x$, the prediction is
conducted by

$C_{v}(x)=\mathrm{VN}(x_{v}),$ (3)

where $x_{v}$ is the video stream (i.e., the image sequence) of $x$. Note that
in the inference phase of our ensemble model, the DM module utilizes either
the single-vector representation or the predicted score corresponding to the
“fake” of the video-only network.

###  III-B Audio-only Network (AN)

The AN module takes as input an audio spectrogram, where time and frequency
elements help learn and capture acoustic patterns, temporal dynamics, and
other audio-specific characteristics, and returns a high-dimensional feature
representation.

Transformers perform incredibly well in audio processing as their self-
attention mechanisms allow for capturing long-range dependencies in audio. In
this work, we use the Audio Spectrogram Transformer (AST) [53] as the audio
backbone of AVTENet. As shown at the bottom of Fig. 3, the AN module takes as
input a spectrogram that is further divided into a sequence of 16x16 patches
with an overlap of 6 in both the time and frequency dimensions. A linear
projection layer transforms each patch into a 1D embedding. Since the patch
sequence lacks input order information, trainable positional embeddings are
incorporated into each patch embedding to enable capturing spatiotemporal
structure from a 2D audio spectrogram. Additionally, a classification (cls)
token is added to the sequence. The output cls embedding of the transformer
encoder encapsulates information about characteristics that can be used to
determine challenging and deceptive alternations in the audio.

Figure 4: Four variants of the proposed AVTENet model for video forgery
detection based on different ensemble strategies: (a) $AVTENet_{mv}$ based on
majority voting, (b) $AVTENet_{asf}$ based on average score fusion, (c)
$AVTENet_{sf}$ based on score fusion, and (d) $AVTENet_{ff}$ based on feature
fusion.

For training AN, a dataset $D^{a}=\\{a_{i},y_{i}\\}_{i=1}^{n}$ is extracted
from the training set of the FakeAVCeleb dataset, where $a_{i}$ denotes the
audio track of the $i$-th training sample $x_{i}$, and $y_{i}$ is the label
for $a_{i}$ (0 for fake and 1 for real). As shown in Fig. 3, the AST model is
used to extract the single vector representation $E_{a}$ (the $cls$ token) of
the mel-spectrogram of each training audio clip. A linear layer is used as the
classifier. The AST model is pre-trained on Audioset [53]. During the training
process of AN, the linear classification layer is trained, and the pre-trained
AST model is fine-tuned. As with the video-only network, the cross-entropy
loss is used to train the audio-only network.

In the inference phase of AN, given a test sample $x$, the prediction is
conducted by

$C_{a}(x)=\mathrm{AN}(x_{a}),$ (4)

where $x_{a}$ is the audio track of $x$. As with the video-only network, in
the inference phase of our ensemble model, either the single-vector
representation or the predicted score corresponding to “fake” of the audio-
only network is used by the DM module.

###  III-C Audio-Visual Network (AVN)

The AVN module exploits both visual and acoustic information by jointly
learning paired audio and visual inputs that provide complementary information
and capture meaningful representations between these distinct modalities to
expose fabrication in videos.

We use the self-supervised learning framework AV-HuBERT [54] as the audio-
visual backbone of AVTENet. As shown in the middle of Fig. 3, the AVN module
simultaneously uses the audio spectrogram and the video frame stack of the lip
region extracted from the video as inputs to the lightweight modality-specific
encoders. The frames of the lip region are passed through a ResNet-based
feature extractor to extract relevant visual features. At the same time, the
audio spectrogram is passed through the feed-forward network (FFN) to extract
the acoustic features. The visual and acoustic features are then fused and fed
to a shared transformer encoder to extract jointly contextualized audio-visual
representations that encapsulate the correlation between acoustic and visual
modalities. These audio-visual representations are further passed through a
Multiscale Temporal Convolutional Network (MS-TCN) and a temporal pooling
layer to produce a single-vector representation.

For training AVN, a dataset $D^{av}=\\{a_{i},v_{i}^{lip},y_{i}\\}_{i=1}^{n}$
is extracted from the training set of the FakeAVCeleb dataset, where $a_{i}$
and $v_{i}^{lip}$, respectively, denote the audio track and the lip image
sequence of the $i$-th training sample $x_{i}$, and $y_{i}$ is the label for
$x_{i}$ (0 for fake and 1 for real). As shown in Fig. 3, the AV-HuBERT model,
MSTCN network, and temporal pooling layer are used to extract the vector
representation $E_{av}$ of the mel-spectrogram of the audio track and the lip-
image sequence of each training video clip. A linear layer is used as the
classifier. The AV-HuBERT model is pre-trained on the LRS3 dataset [55].
During AVN’s training process, not only MSTCN with linear classification layer
is trained, but also the pre-trained AV-HuBERT model is fine-tuned. As with
the video-only and audio-only networks, the cross-entropy loss is used to
train the audio-visual network.

In the inference phase of AVN, given a test sample $x$, the prediction is
conducted by

$C_{av}(x)=\mathrm{AVN}(x_{a},x_{v^{lip}}),$ (5)

where $x_{a}$ and $x_{v^{lip}}$ are the audio track and the lip-image sequence
of $x$, respectively. As with the video-only and audio-only networks, in the
inference phase of our ensemble model, either the single-vector representation
or the predicted score corresponding to “fake” of the audio-visual network is
used by the DM module.

In addition to the above AV-HuBERT-based AVN setup, we also study another
setup. The audio and video embeddings are extracted via AST (same as AN) and
ViViT (same as VN). The corresponding $cls$ tokens are concatenated as
$E_{av}$. The prediction is also conducted by Eq. (5), but using ($x_{a}$,
$x_{v}$) instead of ($x_{a}$, $x_{v^{lip}}$) as input. An overview of the
AVTENet model based on this AVN setting is shown in Fig. 5. Compared with the
AVTENet model in Fig. 3, the only difference is that the AV-HuBERT-based AVN
is replaced by the AST&ViViT-based AVN.

Figure 5: Overview of another AVTENet model that consists of AST-based AN,
ViViT-based VN, and AST&ViViT-based AVN.

###  III-D Decision-Making Module (DM):

As shown in Fig. 3, the DM module of AVTENet is used to integrate the outputs
of VN, AN, and AVN to produce final predictions that indicate potential
forgery in the video. In this work, we investigate several fusion strategies.

####  III-D1 Majority Voting

As shown in Fig. 4(a) and Algorithm 1, the DM module outputs “fake” if at
least two component classifiers (VN, AN, and AVN) output “fake”. The
corresponding AVTENet model is termed $\mathrm{AVTENet}_{mv}$. Note that no
additional models need to be trained for majority voting fusion.

Algorithm 1 Majority Voting Algorithm for $\mathrm{AVTENet}_{mv}$

0: $P_{v},P_{a}$, and $P_{av}\left(P_{i}\in 0\right.$ or 1$)$

0: AVTENet_Prediction $P\left(P\in 0\right.$ or 1$)$

1: Initialize $vote\\_count$ variable, $vote\\_count$ = $P_{v}+P_{a}+P_{av}$

2: if $vote\\_count$ $\geq 2$ then

3: $P$ = 1

4: else

5: $P$ = 0

6: end if

7: return $P$

####  III-D2 Average Score Fusion

As shown in Fig. 4(b), the DM module outputs “fake” if the average of the
output scores of the three component classifiers (VN, AN, and AVN) exceeds a
preset threshold. The corresponding AVTENet model is termed
$\mathrm{AVTENet}_{asf}$. Note that no additional models need to be trained
for average score fusion.

####  III-D3 Score Fusion

As shown in Fig. 4(c), in score fusion, the output scores of the three
component classifiers (VN, AN, and AVN) are concatenated as input to a linear
layer trained on the same training data as the three component classifiers.
When training the linear layer, the parameters of the three component
classifiers are fixed. The corresponding AVTENet model is termed
$\mathrm{AVTENet}_{sf}$.

####  III-D4 Feature Fusion

As shown in Fig. 4(d), in feature fusion, the output representations of the
penultimate layers of the three component classifiers (VN, AN, and AVN) are
concatenated as input to a linear layer trained on the same training data as
the three component classifiers. When training the linear layer, the
parameters of the three component classifiers are fixed. The corresponding
AVTENet model is termed $\mathrm{AVTENet}_{ff}$.

##  IV Experimental Results

This section presents the experiment setup, including the dataset, data
analysis and preprocessing, evaluation metrics, training hyperparameters, and
experimental results.

###  IV-A Dataset

Our experiments are conducted on the FakeAVCeleb dataset [18], an audio-visual
dataset released in 2022 specifically designed for the deepfake detection
task. It is based on a collection of 500 YouTube videos featuring 500
ethnically diverse celebrities from various fields, including music, film,
sports, and politics, taking into account age, geography, race, and gender
diversity. Four latest deepfake manipulation and synthetic speech generation
techniques, including Faceswap [48], FSGAN [49], Wav2Lip [50], and Real-Time-
Voice-Cloning (RTVC) [51], and their combinations were used to generate 19,500
fake video samples from these 500 videos. As a result, the dataset contains a
total of 20,000 video samples. The videos corresponding to 70 celebrities are
used for testing, and the videos corresponding to the remaining 430
celebrities are used for training. These video samples are divided into four
categories according to whether the audio and video modalities are
manipulated, namely RealVideo-RealAudio ($R_{v}R_{a}$), RealVideo-FakeAudio
($R_{v}F_{a}$), FakeVideo-RealAudio ($F_{v}R_{a}$) and FakeVideo-FakeAudio
($F_{v}F_{a}$), where each video is carefully labeled to indicate its
authenticity and facilitate training and testing forgery detection models.

TABLE I: Manipulation modalities in each test set. $\checkmark$ indicates
manipulation, while $X$ indicates no manipulation.

Testsets | Audio Manipulation | Video Manipulation
---|---|---
Testset-1 | $\checkmark$ | $\checkmark$
Testset-II | $\checkmark$ | $\checkmark$
faceswap | $X$ | $\checkmark$
faceswap-wav2lip | $\checkmark$ | $\checkmark$
fsgan | $X$ | $\checkmark$
fsgan-wav2lip | $\checkmark$ | $\checkmark$
RTVC | $\checkmark$ | $X$
wav2lip | $\checkmark$ | $\checkmark$

####  IV-A1 Testing Sets

To conduct a comprehensive evaluation of various uni-modal (AN and VN) and
multi-modal (AVN and AVTENet) detection methods using the FakeAVCeleb dataset,
we used eight different test sets, including Testset-I, Testset-II, faceswap,
faceswap-wav2lip, fsgan, fsgan-wav2lip, RTVC, and wav2lip. Except for
Testset-I and Testset-II, each test set contains the same set of 70 genuine
videos of 70 subjects unseen in training and a different set of 70 fake videos
generated using specific manipulation techniques. Testset-I and Testset-II are
the main evaluation test sets, which also contain the same 70 genuine videos
of 70 subjects unseen in training and 70 fake videos. However, Testset-I
contains the same number of fake samples from each manipulation technique,
while Testset-II contains an equal number of fake samples from the
$R_{v}F_{a}$, $F_{v}F_{a}$ and $F_{v}R_{a}$ categories. Table I summarizes the
manipulation modalities involved in each test set. Multiple test sets can
provide an exhaustive understanding of the strengths and limitations of
AVTENet and individual classifiers on fake video samples generated using
different deepfake techniques.

####  IV-A2 Training Sets for Different Models

In the experiments, the training data are video samples corresponding to 430
subjects in FakeAVCeleb. However, training uni-modal and multi-modal
classifiers requires specific data settings, so we customize the dataset
according to the requirements of each network. Table II shows the training
data setting for each network according to the categorical labels of
FakeAVCeleb.

TABLE II: Training data settings for different networks.

Classifier | Class | Category
---|---|---
Video Network | Fake | $F_{v}F_{a}$
$F_{v}R_{a}$
Real | $R_{v}F_{a}$
$R_{v}R_{a}$
Audio Network | Fake | $R_{v}F_{a}$
$F_{v}F_{a}$
Real | $F_{v}R_{a}$
$R_{v}R_{a}$
Audio-Visual Network | Fake | $F_{v}F_{a}$
$F_{v}R_{a}$
$R_{v}F_{a}$
Real | $R_{v}R_{a}$

Video-only Network (VN): The VN classifier is trained to exploit manipulations
in visual content. When constructing the training set for VN, as shown in
Table II, the fake class includes training samples in the $F_{v}F_{a}$ and
$F_{v}R_{a}$ categories because the visual content in these categories is
manipulated. In contrast, the real class includes samples from the
$R_{v}F_{a}$ and $R_{v}R_{a}$ categories, whose visual content is not
manipulated. As shown in Table III, this setting results in the VN training
set containing 17,809 video samples per class, of which the fake class
contains 9,411 samples from $F_{v}F_{a}$ and 8,398 samples from $F_{v}R_{a}$,
while the real class contains 430 samples from $R_{v}R_{a}$, 430 samples from
$R_{v}F_{a}$, and 16,949 samples from the external VoxCeleb1 dataset [45].
External data is used to balance the training data across classes.

Audio-only Network (AN): AN learns to detect acoustic manipulations in videos.
Therefore, the training set for the AN classifier is organized by treating
categories with manipulated acoustic streams as fake and categories with
unmanipulated acoustic streams as real. As shown in Table II, the fake class
contains all samples of $R_{v}F_{a}$ and $F_{v}F_{a}$, while the real class
contains samples from $F_{v}R_{a}$ and $R_{v}R_{a}$. As shown in Table III,
the training set for AN contains 9,841 samples in the fake class and 8,828
samples in the real class. For the fake class, 430 samples belong to
$R_{v}F_{a}$ and 9,411 samples belong to $F_{v}F_{a}$, while for the real
class, 8,398 samples belong to $F_{v}R_{a}$ and 430 samples belong to
$R_{v}R_{a}$.

Audio-Visual Network (AVN): As shown in Table II, for the AVN classifier, we
specify $F_{v}F_{a}$, $F_{v}R_{a}$ and $R_{v}F_{a}$ categories, where acoustic
and/or visual streams are manipulated, to the fake class. However, the real
class only contains video samples from the $R_{v}R_{a}$ category without
acoustic and visual manipulations. As shown in Table III, the AVN training set
contains 36,411 samples: 18,239 samples for the fake class and 18,172 for the
real class. For the fake class, 9,411 samples belong to $F_{v}F_{a}$, 8,398
samples belong to $F_{v}R_{a}$, and 430 samples belong to $R_{v}F_{a}$. For
the real class, in addition to the 430 samples from $R_{v}R_{a}$, we borrow
17,742 samples from the VoxCeleb1 dataset to balance the training data.

TABLE III: Number of training samples for different networks.

Classifier | Class | Samples | Total Training Samples
---|---|---|---
VN | Fake | 17,809 | 35,618
Real | 17,809
AN | Fake | 9,841 | 18,669
Real | 8,828
AVN | Fake | 18,239 | 36,411
Real | 18,172

####  IV-A3 Data Analysis and Preprocessing

Each classifier involves different data processing steps to ensure effective
training and evaluation. The video-only network takes as input a short clip of
video; therefore, we split the video into multiple short clips and extract a
sequence of consecutive frames from each clip to feed into the network. To
prepare the data for the audio-only network, we extract the audio track with a
sampling rate of 16kHz from each video. Furthermore, we extract mel-
spectrograms from these audios, which are then used as input to the audio-only
network. These mel-spectral features are also used as one of the inputs to the
audio-visual network. For the AV-HuBERT-based AVN, we extract the lip image
frames from each video and pair them with the mel-spectral features as input
to the network.

###  IV-B Hyperparameters

We use the Adam optimizer to train each classifier. AN (based on AST) is
trained with a learning rate of 0.00001, VN (based on ViViT) is trained with a
learning rate of 0.0001 and AVN (with AV-HuBERT) is trained with a learning
rate of 0.002. Similarly, AVN (with AST&ViViT) is trained with a learning rate
of 0.0001. The linear layer in AVTENet for combining AST-based AN, ViViT-based
VN, and AV-HuBERT-based AVN is trained with a learning rate of 0.002, while
the linear layer in AVTENet for combining AST-based AN, ViViT-based VN and
AST&ViViT-based AVN is trained with a learning rate of 0.002.

###  IV-C Evaluation Metrics

We use precision, recall, F1-score, and accuracy to evaluate the performance
of individual uni-modal/multi-modal classifiers and our proposed multi-modal
AVTENet classifier. For all metrics, a higher value indicates better
performance. For a given test input, a binary classifier produces a binary
prediction output, either 1 (for positive) or 0 (for negative). Compared to
the ground-truth label, the prediction can be true positive (TP: correct
positive prediction), false positive (FP: incorrect positive prediction), true
negative (TN: correct negative prediction), and false negative (FN: incorrect
negative prediction).

Accuracy is the primary metric. It calculates the proportion of correct
predictions out of total predictions:

$\mathrm{Accuracy}={\frac{TP+TN}{TP+TN+FP+FN}}.$ (6)

The precision score determines the capability of the model to correctly
identify positive samples. It is a measure of the proportion of true positive
predictions to the total number of positive predictions:

$\mathrm{Precision}={\frac{TP}{TP+FP}}.$ (7)

The recall score, also known as sensitivity or true positive rate, measures
how much a model captures positive instances. It is an estimate of the
proportion of true positive predictions to the actual positive instances in
the dataset:

$\mathrm{Recall}={\frac{TP}{TP+FN}}.$ (8)

The F1-score is the harmonic mean of precision and recall, which combines them
in one metric. It is calculated by taking into account false positives and
false negatives:

$\mathrm{F1}\text{-}\mathrm{score}={\frac{2\times Precision\times
Recall}{Precision+Recall}}.$ (9)

###  IV-D Experimental Results

This section provides detailed experimental results and their analysis.
Testset-I and Test-II are the main evaluation test sets of the FakeAVCeleb
dataset.

####  IV-D1 Video-only Detection Results

TABLE IV: Detection Results of the Video-only Network.

Test-Set type | Class | Precision | Recall | F1-Score | Accuracy
---|---|---|---|---|---
faceswap | Real | 1.00 | 0.87 | 0.93 | 0.94
Fake | 0.89 | 1.00 | 0.94
faceswap-wav2lip | Real | 1.00 | 0.87 | 0.93 | 0.94
Fake | 0.89 | 1.00 | 0.94
fsgan | Real | 0.95 | 0.87 | 0.91 | 0.91
Fake | 0.88 | 0.96 | 0.92
fsgan-wav2lip | Real | 1.00 | 0.87 | 0.93 | 0.94
Fake | 0.89 | 1.00 | 0.94
RTVC | Real | 0.50 | 0.87 | 0.64 | 0.50
Fake | 0.50 | 0.13 | 0.20
wav2lip | Real | 0.97 | 0.87 | 0.92 | 0.92
Fake | 0.88 | 0.97 | 0.93
Testset-I | Real | 0.87 | 0.87 | 0.87 | 0.87
Fake | 0.87 | 0.87 | 0.87
Testset-II | Real | 0.76 | 0.87 | 0.81 | 0.80
Fake | 0.85 | 0.73 | 0.78

The detailed results of the VN classifier on eight different test sets are
shown in Table IV. We observe that the VN classifier has better performance on
the test sets containing visual manipulations because the VN classifier
utilizes visual features to detect forgeries in videos. The accuracy on the
faceswap, faceswap-wav2lip, fsgan, fsgan-wav2lip, and wav2lip test sets is all
above 0.9, except for RTVC, which achieves an accuracy of 0.5. Table I
evidently shows that, except for RTVC, all these test sets contain visual
manipulations, making VN perform better on these test sets. The poor
performance on RTVC is due to the lack of visual manipulations. Here, it is
important to highlight that besides visual manipulations, both faceswap-
wav2lip and fsgan-wav2lip also contain acoustic manipulation, but the VN
classifier only detects visual manipulation in these instances. Furthermore,
the accuracy of the two main test sets (0.87 for Test set-I and 0.80 for
Testset-II) is relatively low compared to other test sets. This is because,
for both test sets, the fake class contains video instances with or without
video manipulation. The conclusion drawn from this experiment is that the VN
classifier performs well when the video is visually manipulated but fails
completely when the video only contains acoustic manipulation.

TABLE V: Detection Results of the Audio-only Network.

Test-Set type | Class | Precision | Recall | F1-Score | Accuracy
---|---|---|---|---|---
faceswap | Real | 0.50 | 1.00 | 0.67 | 0.50
Fake | 0.00 | 0.00 | 0.00
faceswap-wav2lip | Real | 1.00 | 1.00 | 1.00 | 1.00
Fake | 1.00 | 1.00 | 1.00
fsgan | Real | 0.50 | 1.00 | 0.67 | 0.50
Fake | 0.00 | 0.00 | 0.00
fsgan-wav2lip | Real | 1.00 | 1.00 | 1.00 | 1.00
Fake | 1.00 | 1.00 | 1.00
RTVC | Real | 0.97 | 1.00 | 0.99 | 0.99
Fake | 1.00 | 0.97 | 0.99
wav2lip | Real | 0.69 | 1.00 | 0.82 | 0.78
Fake | 1.00 | 0.56 | 0.72
Testset-I | Real | 0.71 | 1.00 | 0.83 | 0.80
Fake | 1.00 | 0.60 | 0.75
Testset-II | Real | 0.76 | 1.00 | 0.86 | 0.84
Fake | 1.00 | 0.69 | 0.81

####  IV-D2 Audio-only Detection Results

The AN classifier is a uni-modal method that uses acoustic features to detect
whether a video is real or fake. The detailed results of AN on eight different
test sets are shown in Table V. We observe a similar pattern to the results of
the VN classifier in Table IV. The detection accuracy is almost perfect on the
faceswap-wav2lip, fsgan-wav2lip, and RTVC test sets (all containing acoustic
manipulation in the fake class). But AN fails completely on the faceswap and
fsgan test sets, where fake instances are only visually manipulated while the
audio tracks are all real. Furthermore, the detection accuracies of Testset-I
and Testset-II (0.80 and 0.84, respectively) are lower than the strong
performance of the other test sets. This is because, for both test sets, the
fake class contains video instances with or without acoustic manipulation. The
results of this experiment show that the AN classifier can detect acoustically
manipulated videos well, but cannot detect fake videos containing only visual
manipulations.

TABLE VI: Detection Results of the AV-HuBERT-based Audio-Visual Network.

Test-Set type | Class | Precision | Recall | F1-Score | Accuracy
---|---|---|---|---|---
faceswap | Real | 0.79 | 0.96 | 0.86 | 0.85
Fake | 0.95 | 0.74 | 0.83
faceswap-wav2lip | Real | 1.00 | 0.96 | 0.98 | 0.98
Fake | 0.96 | 1.00 | 0.98
fsgan | Real | 0.91 | 0.96 | 0.93 | 0.93
Fake | 0.95 | 0.90 | 0.93
fsgan-wav2lip | Real | 1.00 | 0.96 | 0.98 | 0.98
Fake | 0.96 | 1.00 | 0.98
RTVC | Real | 0.96 | 0.96 | 0.96 | 0.96
Fake | 0.96 | 0.96 | 0.96
wav2lip | Real | 1.00 | 0.96 | 0.98 | 0.98
Fake | 0.96 | 1.00 | 0.98
Testset-I | Real | 0.92 | 0.96 | 0.94 | 0.94
Fake | 0.96 | 0.91 | 0.93
Testset-II | Real | 1.00 | 0.96 | 0.98 | 0.98
Fake | 0.96 | 1.00 | 0.98

####  IV-D3 Audio-Visual Detection Results

Compared to the VN and AN classifiers, which can only detect visual and
acoustic manipulations in fake videos, respectively, the AVN classifier
utilizes both acoustic and visual information to detect forgeries in videos.
Table VI shows the detailed results of the AV-HuBERT-based AVN classifier on
eight different test sets. Several observations can be drawn from the table.
First, the AVN classifier achieves good detection performance on all test sets
except the faceswap test set. One possible reason is that the faceswap test
set only involves visual manipulation, and the AV-HuBERT feature extractor
only extracts visual features from lip images, thereby losing information
outside the lip region. Second, although the AV-HuBERT-based AVN classifier
performs well on the test sets containing manipulation in a single modality,
the performance may be slightly lower than audio-only and video-only
classifiers focusing on one modality. For example, the AVN classifier achieves
an accuracy of 0.96 on RTVC (containing only acoustic manipulation), but the
AN classifier achieves an accuracy of 0.99 on the same test (see Table V).
Third, the AV-HuBERT-based AVN classifier achieves accuracy of 0.94 and 0.98
on Testset-I and Testset-II, whose fake instances contain various
manipulations in acoustic and visual streams, outperforming the VN classifier
(cf. 0.87 and 0.80 in Table IV) and the AN classifier (cf. 0.80 and 0.84 in
Table V). Therefore, the insight gleaned from this experiment is that the AVN
classifier successfully detects forgeries in videos by taking into account
both acoustic and visual information, generally outperforming VN and AN
classifiers that focus on uni-modal visual or acoustic manipulation.

TABLE VII: Detection Results of $AVTENet_{mv}$ (Majority Voting).

Test-Set type | Class | Precision | Recall | F1-Score | Accuracy
---|---|---|---|---|---
faceswap | Real | 0.80 | 1.00 | 0.89 | 0.87
Fake | 1.00 | 0.74 | 0.85
faceswap-wav2lip | Real | 1.00 | 1.00 | 1.00 | 1.00
Fake | 1.00 | 1.00 | 1.00
fsgan | Real | 0.88 | 1.00 | 0.93 | 0.93
Fake | 1.00 | 0.86 | 0.92
fsgan-wav2lip | Real | 1.00 | 1.00 | 1.00 | 1.00
Fake | 1.00 | 1.00 | 1.00
RTVC | Real | 0.96 | 1.00 | 0.98 | 0.98
Fake | 1.00 | 0.96 | 0.98
wav2lip | Real | 0.99 | 1.00 | 0.99 | 0.99
Fake | 1.00 | 0.99 | 0.99
Testset-I | Real | 0.92 | 1.00 | 0.96 | 0.96
Fake | 1.00 | 0.91 | 0.96
Testset-II | Real | 1.00 | 1.00 | 1.00 | 1.00
Fake | 1.00 | 1.00 | 1.00

TABLE VIII: Detection Results of $AVTENet_{asf}$ (Average Score fusion).

Test-Set type | Class | Precision | Recall | F1-Score | Accuracy
---|---|---|---|---|---
faceswap | Real | 0.79 | 1.00 | 0.88 | 0.86
Fake | 1.00 | 0.73 | 0.84
faceswap-wav2lip | Real | 1.00 | 1.00 | 1.00 | 1.00
Fake | 1.00 | 1.00 | 1.00
fsgan | Real | 0.88 | 1.00 | 0.93 | 0.93
Fake | 1.00 | 0.86 | 0.92
fsgan-wav2lip | Real | 1.00 | 1.00 | 1.00 | 1.00
Fake | 1.00 | 1.00 | 1.00
RTVC | Real | 0.96 | 1.00 | 0.98 | 0.98
Fake | 1.00 | 0.96 | 0.98
wav2lip | Real | 1.00 | 0.99 | 0.99 | 0.99
Fake | 0.99 | 1.00 | 0.99
Testset-I | Real | 0.92 | 1.00 | 0.96 | 0.96
Fake | 1.00 | 0.91 | 0.96
Testset-II | Real | 1.00 | 1.00 | 1.00 | 1.00
Fake | 1.00 | 1.00 | 1.00

####  IV-D4 Detection Results of AVTENet

AVTENet is an ensemble of the above VN, AN, and AVN networks. We compare four
fusion strategies, namely majority voting (mv), average score fusion (asf),
score fusion (sf), and feature fusion (ff). The resulting models are denoted
as $AVTENet_{mv}$, $AVTENet_{asf}$, $AVTENet_{sf}$, and $AVTENet_{ff}$,
respectively. AVN is based on AV-HuBERT.

$\mathbf{AVTENet_{mv}}$: Table VII shows the detailed results of
$AVTENet_{mv}$ on eight different test sets. $AVTENet_{mv}$ achieves perfect
accuracy of 1.00 on faceswap-wav2lip, fsgan-wav2lip, and Testset-II and high
accuracy of 0.99, 0.98, and 0.96 on wav2lip, RTVC, and Testset-I. This result
confirms the advantages of integrating VN, AN, and AVN networks in fully
considering visual and acoustic information in fake video detection. However,
$AVTENet_{mv}$ achieves relatively low performances on faceswap and fsgan. As
explained before, this may be because the faceswap and fsgan test sets only
involve visual manipulations, while the AV-HuBERT feature extractor ignores
visual information outside the lip region. Comparing Table VII with Tables IV,
V, and VI, $AVTENet_{mv}$ outperforms all its component networks in all test
conditions except for VN on faceswap and AN on RTVC. This result shows that
the ensemble $AVTENet_{mv}$ comes with some tradeoffs in fake detection
compared to modality-specific classifiers operating on the corresponding
single-modality manipulation. Finally, it is worth mentioning again that
$AVTENet_{mv}$ outperforms all its component networks on the two main tests,
Testset-I and Testset-II.

$\mathbf{AVTENet_{asf}}$: Table VIII shows the detailed results of
$AVTENet_{asf}$ on eight different test sets. Comparing Table VIII and Table
VII, we can see that $AVTENet_{asf}$ performs almost the same as
$AVTENet_{mv}$, except that the former achieves a slightly lower accuracy than
the latter on the faceswap test set (0.86 vs 0.87).

TABLE IX: Detection results of $AVTENet_{sf}$ (Score fusion).

Test-Set type | Class | Precision | Recall | F1-Score | Accuracy
---|---|---|---|---|---
faceswap | Real | 0.93 | 0.91 | 0.92 | 0.92
Fake | 0.92 | 0.93 | 0.92
faceswap-wav2lip | Real | 1.00 | 0.91 | 0.96 | 0.96
Fake | 0.92 | 1.00 | 0.96
fsgan | Real | 0.93 | 0.91 | 0.92 | 0.92
Fake | 0.92 | 0.93 | 0.92
fsgan-wav2lip | Real | 1.00 | 0.91 | 0.96 | 0.96
Fake | 0.92 | 1.00 | 0.96
RTVC | Real | 0.96 | 0.91 | 0.93 | 0.94
Fake | 0.92 | 0.96 | 0.94
wav2lip | Real | 1.00 | 0.91 | 0.96 | 0.96
Fake | 0.92 | 1.00 | 0.96
Testset-I | Real | 0.96 | 0.91 | 0.93 | 0.94
Fake | 0.92 | 0.96 | 0.94
Testset-II | Real | 1.00 | 0.91 | 0.96 | 0.96
Fake | 0.92 | 1.00 | 0.96

TABLE X: Detection Results of $AVTENet_{ff}$ (Feature fusion).

Test-Set type | Class | Precision | Recall | F1-Score | Accuracy
---|---|---|---|---|---
faceswap | Real | 0.93 | 0.97 | 0.95 | 0.95
Fake | 0.97 | 0.93 | 0.95
faceswap-wav2lip | Real | 1.00 | 0.97 | 0.99 | 0.99
Fake | 0.97 | 1.00 | 0.99
fsgan | Real | 0.93 | 0.97 | 0.95 | 0.95
Fake | 0.97 | 0.93 | 0.95
fsgan-wav2lip | Real | 1.00 | 0.97 | 0.99 | 0.99
Fake | 0.97 | 1.00 | 0.99
RTVC | Real | 0.96 | 0.97 | 0.96 | 0.96
Fake | 0.97 | 0.96 | 0.96
wav2lip | Real | 1.00 | 0.97 | 0.99 | 0.99
Fake | 0.97 | 1.00 | 0.99
Testset-I | Real | 0.96 | 0.97 | 0.96 | 0.96
Fake | 0.97 | 0.96 | 0.96
Testset-II | Real | 1.00 | 0.97 | 0.99 | 0.99
Fake | 0.97 | 1.00 | 0.99

$\mathbf{AVTENet_{sf}}$: Table IX shows the detailed results of $AVTENet_{sf}$
on eight different test sets. Comparing Table IX with Tables VII and VIII, we
can see that $AVTENet_{sf}$ performs more stably than $AVTENet_{mv}$ and
$AVTENet_{asf}$ across 8 test sets. The stability may be attributed to the
additional linear layer taking as input the scores output by the VN, AN, and
AVN networks, as it learns to balance the contributions of these three
component networks. However, although $AVTENet_{sf}$ outperforms
$AVTENet_{mv}$ and $AVTENet_{asf}$ on the faceswap test set (0.92 vs 0.87 and
0.86), it performs worse than $AVTENet_{mv}$ and $AVTENet_{asf}$ on all the
other test sets.

$\mathbf{AVTENet_{ff}}$: Table X shows the detailed results of $AVTENet_{ff}$
on eight different test sets. We can see that $AVTENet_{ff}$ performs best
among four ensemble models on faceswap, fsgan, and Testset-II, and achieves
near-perfect performance on faceswap-wav2lip, fsgan-wav2lip, wav2lip, and
Testset-II. While the other ensemble models may excel in performance on
specific test sets, $AVTENet_{ff}$ maintains stable and reliable performance
on all eight test sets. The good performance may be attributed to the
additional linear layer that takes as input the embeddings of the penultimate
layers of the VN, AN, and AVN networks, as it learns to balance the
contributions of these three component networks. In summary, feature fusion is
the best fusion strategy in this work.

Figure 6: Performance comparison of our models on two main test sets,
Testset-I and Testset-II. Figure 7: The comparison of our proposed approach
($AVTENet_{ff}$) with other existing unimodal, multimodal, fusion, and
ensemble benchmarks.

##### Comparison of Our Models

Fig. 6 depicts the accuracy of the above seven models evaluated on two main
test sets, Tesetset-I and Testset-II. The video-only model (VN) is good at
identifying visual manipulations in videos. Likewise, the audio-only model
(AN) is good at identifying manipulated acoustic content in videos. But they
only focus on a single modality. In contrast to VN and AN, the audio-visual
model (AVN) and all variants of AVTENet are able to identify acoustic and
visual manipulations in videos. Although $AVTENet_{mv}$ and $AVTENet_{asf}$
have higher accuracy than $AVTENet_{ff}$ on Testset-II, the latter shows
higher stability in detecting various manipulation types in videos.

TABLE XI: Performance comparison of AVTENet (AST_ViViT_AST&ViViT) and AVTENet (AST_ViViT_AV-HuBERT). Models |  | Ensemble
---
Strategies
faceswap |  | faceswap-
---
wav2lip
fsgan |  | fsgan-
---
wav2lip
RTVC | wav2lip | Testset-I | Testset-II
AVTENet (AST_ViViT_AST&ViViT) | $AVTENet_{mv}$ | 0.96 | 0.99 | 0.94 | 0.99 | 0.61 | 0.98 | 0.87 | 0.94
$AVTENet_{asf}$ | 0.96 | 0.99 | 0.93 | 0.99 | 0.63 | 0.98 | 0.88 | 0.94
$AVTENet_{sf}$ | 0.91 | 0.93 | 0.88 | 0.93 | 0.52 | 0.91 | 0.80 | 0.87
$AVTENet_{ff}$ | 0.92 | 0.97 | 0.88 | 0.96 | 0.52 | 0.90 | 0.80 | 0.87
AVTENet (AST_ViViT_AV-HuBERT) | $AVTENet_{mv}$ | 0.87 | 1.00 | 0.93 | 1.00 | 0.98 | 0.99 | 0.96 | 1.00
$AVTENet_{asf}$ | 0.86 | 1.00 | 0.93 | 1.00 | 0.98 | 0.99 | 0.96 | 1.00
$AVTENet_{sf}$ | 0.92 | 0.96 | 0.92 | 0.96 | 0.94 | 0.96 | 0.94 | 0.96
$AVTENet_{ff}$ | 0.95 | 0.99 | 0.95 | 0.99 | 0.96 | 0.99 | 0.96 | 0.99
TABLE XII: Performance comparison of our Ensemble model and several existing uni-modal, multi-modal, fusion, and ensemble models. DFD in the first column refers to the deepfake detection method. “V”, “A” and “AV” stand for visual, audio and audio-visual modalities, respectively. DFD Method | Model | Modality | Class | Precision | Recall | F1-score | Accuracy
---|---|---|---|---|---|---|---
Unimodal[56] | VGG16 | V | Real | 0.6935 | 0.8966 | 0.7821 | 0.8103
Fake | 0.8724 | 0.7750 | 0.8208
Unimodal[56] | Xception | A | Real | 0.8750 | 0.6087 | 0.7179 | 0.7626
Fake | 0.7033 | 0.9143 | 0.7950
Unimodal [57] | LipForensics | V | Real | 0.70 | 0.91 | 0.80 | 0.76
Fake | 0.88 | 0.61 | 0.72
Ensemble (Soft-Voting)[56] | VGG16 | AV | Real | 0.6935 | 0.8966 | 0.7821 | 0.7804
Fake | 0.8948 | 0.6894 | 0.7788
Ensemble (Hard-Voting)[56] | VGG16 | AV | Real | 0.6935 | 0.8966 | 0.7821 | 0.7804
Fake | 0.8948 | 0.6894 | 0.7788
Multimodal-1[56] | Multimodal-1 | AV | Real | 0.000 | 0.000 | 0.000 | 0.5000
Fake | 0.496 | 1.000 | 0.663
Multimodal-2[56] | Multimodal-2 | AV | Real | 0.710 | 0.587 | 0.643 | 0.674
Fake | 0.648 | 0.760 | 0.700
Multimodal-3[56] | CDCN | AV | Real | 0.500 | 0.068 | 0.120 | 0.515
Fake | 0.500 | 0.940 | 0.651
Multimodal-4[16] | Not-made-for-each-other | AV | Real | 0.62 | 0.99 | 0.76 | 0.69
Fake | 0.94 | 0.40 | 0.57
Multimodal [10] | Ensemble | AV | Real | 0.83 | 0.99 | 0.90 | 0.89
Fake | 0.98 | 0.80 | 0.88
Multimodal [11] | AV-Lip-Sync | AV | Real | 0.93 | 0.96 | 0.94 | 0.94
Fake | 0.96 | 0.93 | 0.94
Modality Mixing [58] | Multimodaltrace | AV | Real | - | - | - | 0.929
Fake | - | - | -
Ensemble[59] | AVFakeNet | AV | Real | - | - | - | 0.934
Fake | - | - | -
Fusion[60] | AVoiD-DF | AV | Real | - | - | - | 0.837
Fake | - | - | -
Fusion[61] | MIS-AVioDD | AV | Real | - | - | - | 0.962
Fake | - | - | -
Ensemble (ours) | $\mathbf{AVTENet_{ff}}$ | AV | Real | 1.00 | 0.97 | 0.99 | 0.99
Fake | 0.97 | 1.00 | 0.99

####  IV-D5 Comparison of AVTENet with two different AVNs

Our proposed AVTENet model is an ensemble of an AST-based AN model, a ViViT-
based VN model, and an AV-HuBERT-based AVN model. Although $AVTENet_{ff}$
performs well on all test sets, its performances on the faceswap and fsgan
test sets are relatively low, where the fake instances are only visually
manipulated while audio tracks are real. We speculate that the reason may be
that the AV-HuBERT feature extractor in AVN only extracts visual information
from lip images and ignores information outside the lip region. Therefore, we
also implement another AVN based on AST and ViViT. Here, we compare AVTENet
(AST_ViViT_AV-HuBERT) and AVTENet (AST_ViViT_AST&ViViT). From Table XI, it is
clear that although the AST_ViViT_AST&ViViT-based $AVTENet_{mv}$ and
$AVTENet_{asf}$ models do slightly improve the performance on the faceswap
test set, the AST_ViViT_AST&ViViT-based AVTENet models perform worse than the
AST_ViViT_AV-HuBERT-based AVTENet models. The reason may be that since the AST
and ViViT models are already used in the AN and VN models, they cannot provide
additional information in the AVN model. Surprisingly, the
AST_ViViT_AST&ViViT-based AVTENet models almost fail on the RTVC test set. To
fully utilize face images in the AVN model, we may need to apply new pre-
trained visual models.

####  IV-D6 Comparison of AVTENet and other Models

Finally, as shown in Fig. 7 and Table XII, we conduct a thorough comparison of
our proposed $AVTENet_{ff}$ model with various existing uni-modal, multi-
modal, fusion, and ensemble methods. Several uni-modal, multi-modal and
ensemble methods were presented in [56], but most of them failed to detect
acoustic and visual manipulations in videos well. VGG16, a uni-modal approach
exclusively trained on visual data, outperforms all other uni-modal approaches
reported in [56] for video forgery detection with an accuracy of 0.81.
Xception, another uni-modal method reported in [56], is trained exclusively on
the audio modality and achieves an accuracy of only 0.76. Similarly, Lip
Forensics [35] exploits visual lip movements to detect forgeries in videos
with an accuracy of 0.76. Although Ensemble (soft-voting and hard-voting),
Multi-modal-1, Multi-modal-2, and Multi-modal-3 utilize both audio and visual
modalities for deepfake detection, their performance is still unsatisfactory.
Not-made-for-each other [16] is an audio-visual dissonance-based approach that
detects video forgeries with an accuracy of 0.69. The multi-modal ensemble
approach proposed in [10] utilizes three CNN-based classifiers to identify
forgeries and achieves an accuracy of 0.89. Another audio-visual approach
proposed in [11] successfully identifies fake videos using lip
synchronization, and the model achieves an accuracy of 0.94. Moreover,
Multimodaltrace is an ensemble approach proposed in [58], which achieves an
accuracy of 0.929. AVFakeNet [59] is also an ensemble approach using audio and
video transformer encoders, which achieves an accuracy of 0.934. AVoiD-DF [60]
is another audio-visual fusion approach, but it only achieves an accuracy of
0.837. MIS-AVioDD [61] is also a fusion approach that jointly uses the
modality invariant and modality-specific representations to detect the audio-
visual forgery and achieves an accuracy of 0.962. From Table XII, it is
evident that our ensemble approach outperforms other existing models,
achieving an accuracy of 0.99 on Testset-II of the FakeAVCeleb dataset, which
is a new SOTA performance.

##  V Conclusions and Future Work

In this work, we demonstrated the effectiveness of ensemble learning and
studied deepfake video detection from an audio-visual perspective. We proposed
a novel audio-visual transformer-based ensemble learning solution for
effective and scalable video forgery detection, especially for videos
containing acoustic and visual manipulations, which cannot rely on existing
methods because they can only detect single-modal manipulations or lack
training diversity. Specifically, we have devised a transformer-based model
with a powerful detection ability due to its sequence modeling,
parallelization, and attention mechanisms. We have illustrated the
performances of our multi-modal transformer-based ensemble model on several
manipulation techniques and compared it with various existing uni-modal,
multi-modal, fusion, and ensemble models. Our model achieves state-of-the-art
performance on the FakeAVCeleb dataset. It should be emphasized that the
proposed solution is extensible via self-supervised learning models with large
amounts of training data. To further improve upon the results presented in the
paper, we plan to advance this research by integrating more recent self-
supervised learning models.

## References

  * [1] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, in: Proceedings of the Advances in Neural Information Processing Systems, 2014.
  * [2] D. P. Kingma, M. Welling, Auto-encoding variational bayes, in: Proceedings of the International Conference on Learning Representations, 2014.
  * [3] Y. Mirsky, W. Lee, The creation and detection of deepfakes: A survey, ACM Computing Surveys 54 (1) (2021) 1–41.
  * [4] R. K. Das, T. Kinnunen, W.-C. Huang, Z.-H. Ling, J. Yamagishi, Z. Yi, X. Tian, T. Toda, Predictions of subjective ratings and spoofing assessments of voice conversion challenge 2020 submissions, in: Proceedings of the Joint Workshop for the Blizzard Challenge and Voice Conversion Challenge, 2020.
  * [5] Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, et al., Tacotron: Towards end-to-end speech synthesis, in: Proceedings of the Interspeech Conference, 2017.
  * [6] A. O. Kwok, S. G. Koh, Deepfake: a social construction of technology perspective, Current Issues in Tourism 24 (13) (2021) 1798–1802.
  * [7] A. Ray, Disinformation, deepfakes and democracies: The need for legislative reform, The University of New South Wales Law Journal 44 (3) (2021) 983–1013.
  * [8] C. Vaccari, A. Chadwick, Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news, Social Media+ Society 6 (1) (2020) 2056305120903408.
  * [9] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al., An image is worth 16x16 words: Transformers for image recognition at scale, in: Proceedings of the International Conference on Learning Representations, 2021\.
  * [10] A. Hashmi, S. A. Shahzad, W. Ahmad, C. W. Lin, Y. Tsao, H.-M. Wang, Multimodal forgery detection using ensemble learning, in: Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, 2022, pp. 1524–1532.
  * [11] S. A. Shahzad, A. Hashmi, S. Khan, Y.-T. Peng, Y. Tsao, H.-M. Wang, Lip Sync Matters: A novel multimodal forgery detector, in: Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, 2022, pp. 1885–1892.
  * [12] D. Afchar, V. Nozick, J. Yamagishi, I. Echizen, MesoNet: a compact facial video forgery detection network, in: Proceedings of the IEEE International Workshop on Information Forensics and Security, 2018, pp. 1–7.
  * [13] F. Chollet, Xception: Deep learning with depthwise separable convolutions, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017, pp. 1251–1258.
  * [14] M. Tan, Q. Le, EfficientNet: Rethinking model scaling for convolutional neural networks, in: Proceedings of the International Conference on Machine Learning, 2019, pp. 6105–6114.
  * [15] X. Han, V. Morariu, P. I. Larry Davis, et al., Two-stream neural networks for tampered face detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 19–27.
  * [16] K. Chugh, P. Gupta, A. Dhall, R. Subramanian, Not made for each other-audio-visual dissonance-based deepfake detection and localization, in: Proceedings of the ACM International Conference on Multimedia, 2020, pp. 439–447.
  * [17] W. Ahmad, I. Ali, A. Shahzad, A. Hashmi, F. Ghaffar, ResViT: A framework for deepfake videos detection, International Journal of Electrical and Computer Engineering Systems 13 (9) (2022) 807–813.
  * [18] H. Khalid, S. Tariq, M. Kim, S. S. Woo, FakeAVCeleb: A novel audio-video multimodal deepfake dataset, in: Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 2021.
  * [19] J. Thies, M. Zollhofer, M. Stamminger, C. Theobalt, M. Nießner, Face2face: Real-time face capture and reenactment of RGB videos, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016, pp. 2387–2395.
  * [20] E. Zakharov, A. Shysheya, E. Burkov, V. Lempitsky, Few-shot adversarial learning of realistic neural talking head models, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9459–9468.
  * [21] A. Siarohin, S. Lathuilière, S. Tulyakov, E. Ricci, N. Sebe, First order motion model for image animation, in: Proceedings of the Advances in Neural Information Processing Systems, 2019.
  * [22] H. Kim, P. Garrido, A. Tewari, W. Xu, J. Thies, M. Niessner, P. Pérez, C. Richardt, M. Zollhöfer, C. Theobalt, Deep video portraits, ACM Transactions on Graphics 37 (4) (2018) 1–14.
  * [23] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial networks, Communications of the ACM 63 (11) (2020) 139–144.
  * [24] C. Chan, S. Ginosar, T. Zhou, A. A. Efros, Everybody dance now, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 5933–5942.
  * [25] C. Wang, C. Xu, C. Wang, D. Tao, Perceptual adversarial networks for image-to-image transformation, IEEE Transactions on Image Processing 27 (8) (2018) 4066–4079.
  * [26] B. Hosler, D. Salvi, A. Murray, F. Antonacci, P. Bestagini, S. Tubaro, M. C. Stamm, Do deepfakes feel emotions? a semantic approach to detecting deepfakes via emotional inconsistencies, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 1013–1022.
  * [27] Y. Li, S. Lyu, Exposing deepfake videos by detecting face warping artifacts, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2019.
  * [28] X. Yang, Y. Li, S. Lyu, Exposing deep fakes using inconsistent head poses, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2019, pp. 8261–8265.
  * [29] W. Wu, W. Zhou, W. Zhang, H. Fang, N. Yu, Capturing the lighting inconsistency for deepfake detection, in: Proceedings of the International Conference on Artificial Intelligence and Security, 2022, pp. 637–647.
  * [30] D.-K. Kim, K.-S. Kim, Generalized facial manipulation detection with edge region feature extraction, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 2828–2838.
  * [31] Y. Li, M.-C. Chang, S. Lyu, In ictu oculi: Exposing AI created fake videos by detecting eye blinking, in: Proceedings of the IEEE International Workshop on Information Forensics and Security, 2018, pp. 1–7.
  * [32] A. Borji, Qualitative failures of image generation models and their application in detecting deepfakes, Image and Vision Computing 137 (2023) 104771.
  * [33] F. Matern, C. Riess, M. Stamminger, Exploiting visual artifacts to expose deepfakes and face manipulations, in: Proceedings of the IEEE Winter Applications of Computer Vision Workshops, 2019, pp. 83–92.
  * [34] E. Prashnani, M. Goebel, B. Manjunath, Generalizable deepfake detection with phase-based motion analysis, arXiv preprint arXiv:2211.09363 (2022).
  * [35] S. Agarwal, H. Farid, T. El-Gaaly, S.-N. Lim, Detecting deep-fake videos from appearance and behavior, in: Proceedings of the IEEE International Workshop on Information Forensics and Security, 2020, pp. 1–6.
  * [36] S. A. Khan, H. Dai, Video transformer for deepfake detection with incremental learning, in: Proceedings of the ACM International Conference on Multimedia, 2021, pp. 1821–1828.
  * [37] S. A. Khan, D.-T. Dang-Nguyen, Hybrid transformer network for deepfake detection, in: Proceedings of the International Conference on Content-based Multimedia Indexing, 2022, pp. 8–14.
  * [38] J. R. Williams, Y. A. Markov, N. A. Tiurina, V. S. Störmer, What You See Is What You Hear: Sounds alter the contents of visual perception, Psychological Science 33 (12) (2022) 2109–2122.
  * [39] T. Mittal, U. Bhattacharya, R. Chandra, A. Bera, D. Manocha, Emotions don’t lie: An audio-visual deepfake detection method using affective cues, in: Proceedings of the ACM International Conference on Multimedia, 2020, pp. 2823–2832.
  * [40] Y. Zhou, S.-N. Lim, Joint audio-visual deepfake detection, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 14800–14809.
  * [41] D. Cozzolino, A. Pianese, M. Nießner, L. Verdoliva, Audio-visual person-of-interest deepfake detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 943–952.
  * [42] S. Muppalla, S. Jia, S. Lyu, Integrating audio-visual features for multimodal deepfake detection, arXiv preprint arXiv:2310.03827 (2023).
  * [43] S. Asha, P. Vinod, I. Amerini, V. G. Menon, A novel deepfake detection framework using audio-video-textual features, Research Square (2022).
  * [44] H. Zhao, W. Zhou, D. Chen, W. Zhang, N. Yu, Self-supervised transformer for deepfake detection, arXiv preprint arXiv:2203.01265 (2022).
  * [45] C.-S. Sung, J.-C. Chen, C.-S. Chen, Hearing and Seeing Abnormality: Self-supervised audio-visual mutual learning for deepfake detection, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2023, pp. 1–5.
  * [46] B. Dolhansky, J. Bitton, B. Pflaum, J. Lu, R. Howes, M. Wang, C. C. Ferrer, The deepfake detection challenge (DFDC) dataset, arXiv preprint arXiv:2006.07397 (2020).
  * [47] J. S. Chung, A. Nagrani, A. Zisserman, Voxceleb2: Deep speaker recognition, in: Proceedings of the Interspeech, 2018.
  * [48] I. Korshunova, W. Shi, J. Dambre, L. Theis, Fast face-swap using convolutional neural networks, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2017, pp. 3677–3685.
  * [49] Y. Nirkin, Y. Keller, T. Hassner, FSGAN: Subject agnostic face swapping and reenactment, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7184–7193.
  * [50] K. Prajwal, R. Mukhopadhyay, V. P. Namboodiri, C. Jawahar, A lip sync expert is all you need for speech to lip generation in the wild, in: Proceedings of the ACM International Conference on Multimedia, 2020, pp. 484–492.
  * [51] Y. Jia, Y. Zhang, R. Weiss, Q. Wang, J. Shen, F. Ren, P. Nguyen, R. Pang, I. Lopez Moreno, Y. Wu, et al., Transfer learning from speaker verification to multispeaker text-to-speech synthesis, in: Proceedings of the Advances in Neural Information Processing Systems, 2018.
  * [52] A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lučić, C. Schmid, ViViT: A video vision transformer, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 6836–6846.
  * [53] Y. Gong, Y.-A. Chung, J. Glass, AST: Audio spectrogram transformer, in: Proceedings of the Interspeech Conference, 2021.
  * [54] B. Shi, W.-N. Hsu, K. Lakhotia, A. Mohamed, Learning audio-visual speech representation by masked multimodal cluster prediction, in: Proceedings of the International Conference on Learning Representations, 2021.
  * [55] T. Afouras, J. S. Chung, A. Zisserman, LRS3-TED: a large-scale dataset for visual speech recognition, arXiv preprint arXiv:1809.00496 (2018).
  * [56] H. Khalid, M. Kim, S. Tariq, S. S. Woo, Evaluation of an audio-video multimodal deepfake dataset using unimodal and multimodal detectors, in: Proceedings of the 1st Workshop on Synthetic Multimedia-Audiovisual Deepfake Generation and Detection, 2021, pp. 7–15.
  * [57] A. Haliassos, K. Vougioukas, S. Petridis, M. Pantic, Lips Don’t Lie: A generalisable and robust approach to face forgery detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 5039–5049.
  * [58] M. A. Raza, K. M. Malik, Multimodaltrace: Deepfake detection using audiovisual representation learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 993–1000.
  * [59] H. Ilyas, A. Javed, K. M. Malik, AVFakeNet: A unified end-to-end dense swin transformer deep learning model for audio–visual deepfakes detection, Applied Soft Computing 136 (2023) 110124.
  * [60] W. Yang, X. Zhou, Z. Chen, B. Guo, Z. Ba, Z. Xia, X. Cao, K. Ren, AVoiD-DF: Audio-visual joint learning for detecting deepfake, IEEE Transactions on Information Forensics and Security 18 (2023) 2015–2029.
  * [61] V. S. Katamneni, A. Rattani, MIS-AVioDD: Modality invariant and specific representation for audio-visual deepfake detection, arXiv preprint arXiv:2310.02234 (2023).