11institutetext: Ehsan Hallaji 22institutetext: Department of Electrical and
Computer Engineering, University of Windsor, 401 Sunset Avenue, Windsor, ON
N9B 3P4, Canada, 22email<EMAIL_ADDRESS>33institutetext: Roozbeh Razavi-
Far 44institutetext: Department of Electrical and Computer Engineering and
School of Computer Science, University of Windsor, 401 Sunset Avenue, Windsor,
ON N9B 3P4, Canada, 44email<EMAIL_ADDRESS>55institutetext: Mehrdad Saif
66institutetext: Department of Electrical and Computer Engineering, University
of Windsor, 401 Sunset Avenue, Windsor, ON N9B 3P4, Canada, 66email:
<EMAIL_ADDRESS>
# Federated and Transfer Learning: A Survey on Adversaries and Defense
Mechanisms

Ehsan Hallaji  Roozbeh Razavi-Far and Mehrdad Saif

###### Abstract

The advent of federated learning has facilitated large-scale data exchange
amongst machine learning models while maintaining privacy. Despite its brief
history, federated learning is rapidly evolving to make wider use more
practical. One of the most significant advancements in this domain is the
incorporation of transfer learning into federated learning, which overcomes
fundamental constraints of primary federated learning, particularly in terms
of security. This chapter performs a comprehensive survey on the intersection
of federated and transfer learning from a security point of view. The main
goal of this study is to uncover potential vulnerabilities and defense
mechanisms that might compromise the privacy and performance of systems that
use federated and transfer learning.

##  1 Introduction

Machine learning has exploded in popularity as the information era has
matured. As a sub-discipline of machine learning, deep learning (DL) is
responsible for a number of achievements that have helped popularise the area.
The hierarchical feature extraction within the DL models enables them to learn
complex underlying patterns in the observed input space. This makes DL models
suitable for processing various data types and facilitating different tasks
such as prediction HASSANI2022118861 , detection 8608001 , imputation 9370000
, and data reduction HASSANI2021104150 . Although the success of DL-based
projects is contingent on several factors, one of the most important
requirements is usually to have access to abundant training samples.

With the advancement of technologies such as internet of things and increasing
number of intelligent devices, the diversity and volume of generated data is
growing at an astonishing pace 9608954 . This abundant data stream is
dispersed and diverse in character. When this data is evaluated as a whole, it
may provide knowledge and discoveries that might possibly accelerate
technological and scientific advancements. Nonetheless, the privacy hazards
associated with data ownership are increasingly becoming a major problem. The
conflict between user privacy and high-quality services is driving need for
new technologies and research to enable knowledge extraction from data without
jeopardising data-holding parties’ privacy.

Federated learning (FL) is perhaps the most recent approach presented to
potentially resolve this issue. FL allows for the collaborative training of a
DL model across a network of client devices. This multi-party collaboration is
accomplished by communication with a central server and decentralisation of
training data pmlr-v54-mcmahan17a . In other words, the FL design requires
that the training data be retained on the client, or local device, that
generated or recorded it. FL’s data decentralisation addresses a significant
portion of data and user privacy concerns, since model training at the
network’s edge eliminates the need for direct data sharing. Nevertheless, in
FL systems one should strike a balance between data privacy and model
performance. The appropriate balance is determined by a number of criteria,
including the model architecture, data type, and intended application. Beyond
FL restrictions, ensuring the confidentiality and security of the FL
infrastructure is critical for establishing trust amongst diverse clients of
the federated network.

The conventional FL imposes a constraint by requiring customers’ training data
to use similar attributes. However, most industries such as banking and
healthcare do not comply with this assumption. In centralized machine
learning, this was addressed by Transfer Learning (TL), which enables a model
obtained from a specific domain to be used in other domains of with the same
application 8608001 . Inspired by this, Federated TL (FTL) emerged as a way to
overcome this constraint 9076003 . FTL clients may differ in the employed
attributes, which is more practical for industrial application. TL
effectiveness is heavily reliant on inter-domain interactions. It is
worthwhile to mention that organizations joined in a FTL network are often
comparable in the service they provide and the data they use. As a result,
including TL within the FL architecture can be quite advantageous.

FTL is at the crossroads of two distinct and rapidly expanding research areas
of privacy-preserving and distributed machine learning. For this reason, it is
crucial to investigate more on these two topics to make best use of FTL.
Hence, this chapter studies different security and privacy aspects of FTL.

Information privacy and machine learning are distinct and rapidly expanding
research areas that derive FTL, and, thus, going through their connections to
FTL is necessary. Understanding the interplay between TL and FL, as well as
identifying potential risks to FTL in real-world applications, is crucial.
Knowing compatible diffense mechanisms with FTL is also vital for mitigating
potential cyber-threats. Hence, in this chapter, we present a comprehensive
survey on possible threats to FTL and the available defense mechanisms.

The rest of the chapter is organized as follows. The preliminaries of this
survey are explained in Section 2. Section 3 reviews known attack scenarios on
FL and TL w.r.t. performance and privacy. Section 4 presents tools and defense
mechanisms that are undertook in the literature for mitigating threats to FL
and TL. Section 5 explains the future directions in defending FTL. Finally,
Chapter 6 concludes the conducted survey.

##  2 Background

This chapter concisely reviews preliminaries of federated and transfer
learning to facilitate the discussions in the following sections.

###  2.1 Federated Learning

FL paradigm allows for collaborative model training across several
participants (i.e., also referred to as clients or devices). This multi-party
collaboration is accomplished by communication with a central server and
decentralization of training data pmlr-v54-mcmahan17a . Data decentralization
of FL mitigates a major part of user privacy issues. Moreover, the efficiency
of FL reduces the communication overhead in the network. FL can be categorizes
from different perspectives, as explained in the following.

#### Categories of Federated Learning

FL variations generally fall under three categories depending on the portion
of feature and sample space they share 10.1145/3298981 :

  1. 1.

Horizontal Federated Learning: Participants exchange data with comparable
properties captured from various users 1316832 . For instance, clinical
information of various patients is recorded using the same features across
several hospitals. Therefore, similar deep learning structures can be trained
on these datasets since they all process the same number and types of
features.

  2. 2.

Vertical Federated Learning: The vertical variation is utilized in
applications where participant datasets have considerable overlaps in the
sample space but each has a separate set of attributes 10.1145/775047.775142 .

  3. 3.

Federated Transfer Learning: FTL facilitates knowledge transfer between
participants when the overlap between sample and feature space is minimal
9076003 . FTL is discussed in further detail later in this section.

###  2.2 Transfer Learning

Most machine learning approaches work on the premise that the training and
test data are in the same feature space. In industry, data may be hard to
collect in certain applications, and, thus, there is a preference for using
the available data shared by large companies and organizations. The challenge,
however, is the difference between the data distributions despite the
similarity of the applications. In other words, the ideal model needs to model
from one domain with limited data resources, while abundant training data is
available in another domain. For instance, consider a factory that trained a
model to predict the market demand to adjust it production rate for different
items. However, it may be time consuming to obtain enough samples for each
produced item separately. Instead, TL can enable the model to be trained on
the data of other organizations that produce similar products, albeit samples
may not be recorded using the same features. The past decade has witnessed an
increasing attraction towards research on TL, which resulted in proposing
different variations TL under different names 5288526 .

###  2.3 Federated Transfer Learning

Similar to TL, clients of FTL may not use the same attributes in the training
data. This is mostly the case in organizations that are similar in nature but
are not identical DBLP:journals/corr/abs-2010-15561 . Due to such differences,
these organizations share only a small portion of each other’s feature space.
Therefore, under this condition, both samples and features are different in
the dataset. Note that the considered condition in FTL is in contrast to the
other variants of FL.

FTL takes a model that has been constructed on source data, and then aligns it
to be employed in a target domain. This allows the model to be utilized for
uncorrelated data points while exploiting the information gained from non-
overlapping features in the source domain. As a result, FTL transmits
information from the source domain’s non-overlapping attributes to new samples
within the target domain.

Existing literature on FTL mainly studies the customization of FTL for certain
applications 9076082 ; 9099064 . From a learning stand-point, only a limited
number of works present distinct FTL protocols. A secure FTL framework is
proposed in 9076003  that uses secret sharing and homomorphic encrypton to
protect privacy without sacrificing accuracy, which is a typical issue in
privacy-preserving techniques. Other benefits of this method include the
simplicity of homomorphic encryption and the fact that secret sharing ensures
zero accuracy loss and quick processing time. On the other had, Homomorphic
encryption, imposes significant computing overhead, and secret sharing
necessitated offline operations prior to online computation. A following
research 9006280  tackles the computation overhead of previous protocols and
extends the FTL model beyond the semi-honset setting by taking malicious users
into count as well. The authors use secret-sharing in the designed algorithm
to enhance the security and efficiency of multi-party communications in FTL.
An scalable heterogeneous FTL framework is also presented in 9005992 , which
uses secret sharing and homomorphic encryption.

##  3 Threats to Federated Learning

Table 1: Identification of sources of attacks on FL systems. Attacks | Source of Attack
---|---
Data poisoning | Malicious client
Model poisoning | Malicious client
Backdoor attack | Malicious client and malicious server
Evasion attack | Malicious client and model deployment
Non-robust aggregation | Aggregation algorithm
Training rule manipulation | Malicious client
Inference attacks | Malicious server and communication
GAN reconstruction | Malicious server and communication
Free-riding attack | Malicious client
Man-in-the-middle attack | Communication

Threats to FL and TL often compromise the functionality or privacy of the
system. Table 1 lists the sources of threats for common attacks to FL. FL
enables a distributed learning process without requiring data exchange,
allowing members to freely join and exit federations. Recent research has
shown, however, that resilience of FL against the mentioned threats in Table 1
can be questionable.

Existing FL protocol designs are vulnerable to rough servers and adversarial
parties. While both infer confidential information from participants’ updates,
the former mainly tampers with the model training whereas the latter resorts
to poisoning attacks to deviate the aggregation procedure.

During the training process, communicating model updates might divulge
critical information and lead to deep leakage. This can consequently
jeopardize the privacy of local data or lead to high-jacking the training data
NEURIPS2019_60a6c400 . The robustness of FL systems, on the other hand, can be
degraded using poisoning attacks on the model to corrupt the model or training
data DBLP:journals/corr/abs-2007-05084 ; pmlr-v108-bagdasaryan20a ;
pmlr-v97-bhagoji19a . In turn, these attacks lead to planting a backdoor into
the global model or degrade its convergence.

###  3.1 Threat Models

Attacks on FL can be launched in different fashions
DBLP:journals/corr/abs-2012-06337 ; https://doi.org/10.48550/arxiv.2003.02133
. To have a better grasp of the nature of FL attacks, we will first go through
the most frequent threat models in the following:

  * •

Outsider Adversaries include attacks by eavesdroppers on the line of
communication between clients and the FL server, as well as attacks by clients
of the FL model once it is provided as a service.

  * •

Insider Adversaries involve attacks initiated from the server or the edge of
the network. Byzantine NIPS2017_f4b9ec30  and Sybil
DBLP:journals/corr/abs-1808-04866  attacks can be mentioned as two most
important insider attacks.

  * •

Semi-Honest Adversaries are non-aggressive adversaries that attempt to
discover the hidden states of other users while being honest to the FL
protocol. Only the received information such as the global model’s parameters
is visible to the attackers.

  * •

Training Manipulation is the process of learning, affecting, or distorting the
FL model itself pmlr-v20-biggio11 . The attacker can damage the integrity of
the learning process by attacking the training data or the model during the
training phase pmlr-v97-bhagoji19a .

  * •

Inference Manipulation mainly consists of evasion or inference attacks
10.1145/1128817.1128824 . They usually deceive the model into making incorrect
decisions or gather information regarding the model’s properties. These
opponents’ efficacy is determined by the amount of knowledge given to the
attacker, which classifies them into white-box and black-box variations.

###  3.2 Attacks on Performance

Figure 1 shows the taxonomy of attacks on FL and TL. The common threats
between FL and TL that can jeopardize FTL are also specified within the red
area.

Figure 1: Taxonomy of the attacks on Federated and Transfer Learning. The
common threats between FL and TL are specified in red area.

#### Data Poisoning

Poisoning training data in FL often affect the integrity of the training data,
compromising model performance by injecting a backdoor for particular triggers
during inference and degrading the overall model accuracy. The most common
types of data poisoning attacks are as follows:

  * •

Denial-of-Service (DoS) attacks generally attempt to minimize the target
overall performance, impacting the recognition rate of all classes. In a FL
system, label noise in the training data may be induced to create a poisoned
model that cannot accurately predict any of the classes. This is also referred
to as label flipping DBLP:journals/corr/abs-1912-04977  in the literature.
When the parameters of this model are sent to the server, the performance of
other devices diminishes.

  * •

Backdoor attacks are designed to impose intentional and particular false
predictions for specific data patterns. DBLP:journals/corr/abs-1912-04977 .
Backdoor attacks, unlike DoS attacks, only influence the model’s recognition
ability for a specific group of samples or classes.

#### Model Poisoning

Poisoning a model refers to a wide range of techniques for tampering with the
FL training procedure. It is worth mentioning that in some literature data
poisoning is categorized as a type of model poisoning 9308910 . However, here,
we mainly target gradient and learning objective manipulation when referring
to model poisoning.

  * •

Gradient Manipulation: Local model gradients may be manipulated by adversaries
to degrade overall performance of the central model, for example, by lowering
detection accuracy DBLP:journals/corr/abs-1912-04977 . For instance, this
approach is used to inject hidden global model backdoors
pmlr-v108-bagdasaryan20a .

  * •

Training Objective Manipulation: involve manipulating model training rules
DBLP:journals/corr/abs-1912-04977 . Training rule manipulation, for example,
is used to successfully carry out a covert poisoning operation by appending a
deviating term to the loss function to penalize the difference of benign and
malicious updates pmlr-v97-bhagoji19a .

###  3.3 Attacks on Privacy

#### Model Inversion Attacks

It has been demonstrated that model inversion attacks can successfully define
sensitive characteristics of the classes and instances covered by the model
DBLP:journals/corr/abs-1912-04977 ; 9308910 . 10.1145/2810103.2813677  states
that using these attacks in a white-box setting on decision trees enables
reveal sensitive variables such as survey responses, which may be identified
with no false positives. Another study demonstrates that a hacker may
anticipate genetic data of a person simply using their demographic information
DBLP:journals/corr/abs-1912-04977 .

#### Membership Inference Attacks

Membership inference aims at disclosing the membership of a particular sample
to the training dataset or a certain class. Furthermore, this form of attack
can work even if the objective is unrelated to the basic characteristics of
the class 8835269 .

#### GAN Reconstruction Attacks

Model inversion is similar to GAN reconstruction; however, the latter is
substantially more potent and have been demonstrated to create artificial
samples that statistically resemble of the training data
10.1145/3133956.3134012 . Traditional model inversion approaches utterly fail
when attacking more sophisticated DL architectures, whereas GAN reconstruction
attacks may effectively create desirable outputs. It has been shown that even
with the presence of differential privacy, GAN may be able to reach the
objective. As a result, an adversary may be able to persuade benign clients to
mistakenly commit gradient modifications that leak more confidential details
than planned during collaborative learning 10.1145/3133956.3134012 .

##  4 Threats to Transfer Learning

###  4.1 Backdoor Attacks

Plenty of the pre-trained Teacher models ($\mathcal{T}$) employed for TL are
openly available, making them vulnerable to backdoor attacks 9112322 ;
10.1145/3319535.3354209 . In a white-box setup, the intruder has access to
$\mathcal{T}$, as is prevalent in modern applications. The intruder intends to
cause a erroneous decision making for a Student model ($\mathcal{S}$) that has
been calibrated via TL using a publicly available pre-trained $\mathcal{T}$
model.

The attacker may breach the publicly accessible pre-trained $\mathcal{T}$
model prior to the $\mathcal{S}$ system deployment phase. Because the
regulations of third-party platforms that store diverse $\mathcal{T}$ models
are often inadequate, the platforms contain multiple variations of the same
pre-trained neural networks. Since weights of a neural network are not self-
explanatory, distinguishing damaging models from refined models is
complicated, if not impossible. In this situation, we suppose that the
intruder is familiar with the structure and parameters of the $\mathcal{T}$
and has black-box access to $\mathcal{S}$, but is unaware of the specific
$\mathcal{T}$ who trained this model and which layers were fixed for training
$\mathcal{S}$ 10.5555/3277203.3277300 .

Adversaries might potentially get around the fine-tuning technique by
leveraging openly accessible pre-trained $\mathcal{T}$ models to construct
$\mathcal{S}$ models. The $\mathcal{S}$ models must be optimized using
particular $\mathcal{T}$ models, in which a portion of the $\mathcal{T}$
structure must be incorporated and retrained frequently. In a white-box
setting, we presume the intruder knows the certain $\mathcal{T}$ that trained
$\mathcal{S}$ and which layers were unchanged throughout the $\mathcal{S}$
training. The adversary, in particular, has access to the architecture and
weights of the $\mathcal{S}$ model and may change them.

###  4.2 Adversarial Attacks

In contrast to conventional adversarial attacks, which optimize false data to
be mistaken for benign samples, the central notion of adversarial attacks
against TL is to optimize a data matrix to imitate the intrinsic
representation of the target data. Models transferred by re-learning the last
linear layer have recently been shown to be sensitive to adversarial instances
produced exclusively using a pre-trained model rezaei2020targetagnostic . It
has been demonstrated that such an attack can fool models that have been
transported with end-to-end fine-tuning DBLP:journals/corr/abs-2002-02998 .
This discovery raises questions about the security of the extensively employed
fine-tuning approach.

###  4.3 Inference Attacks

An inference attack resorts to data analysis to gather unauthorized
information about a subject or database. If an attacker can confidently
estimate the true worth of a subject’s confidential information, it can be
termed as leaked. The most frequent variants of this approach are membership
inference and attribute inference.

#### Membership Inference

The goal of membership inference in machine learning is to establish whether a
sample was employed to train the target model. Discovering the membership
status of a particular user data might lead to serious information theft
zou2020privacy . For instance, revealing that a patient’s medical records were
utilized to train a model linked with an illness might disclose that the
patient has the condition.

In contrast with conventional machine learning, there are two attack surfaces
for membership inference in TL setting, that is discovering the membership
status of samples for both $\mathcal{S}$ and $\mathcal{T}$ models.
Furthermore, depending on the abilities of certain adversaries, access to
either the $\mathcal{T}$ or $\mathcal{S}$ model may be possible. Given both
attack surfaces and the extent of attackers’ access to the models, there are
three possible attack scenarios:

  1. 1.

The attackers can observe $\mathcal{T}$ and aim at ascertaining the state of
the $\mathcal{T}$ dataset’s membership. This approach is analogous to the
traditional membership inference attack, in which the target model is trained
from the ground up.

  2. 2.

The $\mathcal{S}$ model is visible to the attackers, and they attempt to
ascertain the status of the $\mathcal{T}$ dataset’s membership The target
model is not directly trained from the target dataset in this scenario.

  3. 3.

The attackers observes the $\mathcal{S}$ model and try to deduce the
$\mathcal{S}$ dataset’s membership status. In contrast to the first scenario,
here the target model is transferred from the $\mathcal{T}$ model.

#### Feature Inference

An adversary with partial prior knowledge of a target’s record can devise
feature inference to fill in the missing features by monitoring the model’s
behaviour yeom2018privacy ; 10.5555/2671225.2671227 ; 10.1145/2810103.2813677
. For instance, a description of attribute inference attack is given in
yeom2018privacy  and it has been demonstrated that by incorporating membership
inference as a subroutine, this attack may deduce missing attribute values.
Based on the missing attributes, a set of distinct feature vectors are
generated and passed to the membership inference adversary as input. The
output of this process is attribute values that correspond to the vector whose
membership is confirmed via membership inference. Experimental validations for
the effectiveness of an attribute inference of regression models are also
available yeom2018privacy .

##  5 Defense Mechanisms

Various defense mechanisms are proposed to fortify FL against privacy and
performance related threats. Figure 2 illustrates the taxonomy of the defense
mechanisms in TL and FL, and the common approaches between the two that can be
used to defend FTL.

Figure 2: Taxonomy of the defense mechanisms for FTL. The common defense
mechanisms between FL and TL are specified in red area.

###  5.1 Privacy Preserving

Despite the wide diversity of previous efforts on safeguarding FL privacy,
suggested methods typically fall into one of these categories: homomorphic
encryption, secure multiparty computation, and differential privacy. The
following paragraphs go through each of these groups.

#### Homomorphic Encryption

By processing on cyphertext, homomorphic encryption is commonly used to secure
the learning process. Clients can use homomorphic encryption to perform
arithmetic operations on encrypted data (i.e., cyphertext) without having to
decode it. The most prevalent techniques in Homomorphic encryption are
explained as in the following OGBURN2013502 .

Fully homomorphic encryption is capable of doing arbitrary calculations on the
encrypted data 10.1145/1536414.1536440 . This is while partially homomorphic
encryption can only execute one operation (e.g., addition or multiplication),
and substantially homomorphic encryption can do several operations
10.1007/978-3-642-32009-5_38 ; 10.1145/359340.359342 ;
10.1007/3-540-48910-X_16 . The latter, on the other hand, has a restricted
amount of additions and multiplications. While completely homomorphic
encryption offers greater flexibility, it is inefficient when compared to
other forms of homomorphic encryption 10.1145/1536414.1536440 .

Despite the benefits of homomorphic encryption, executing arithmetic on the
encrypted integers increases the memory and processing time costs. For this
reason, one of the main problems in homomorphic encryption is to find a proper
balance between privacy and utility 10.1145/2857705.2857731 ;
info:doi/10.2196/medinform.8805 . In 8241854 , for instance, additive
homomorphic encryption is used to secure distributed learning by securing
model changes and maintaining gradient privacy. Another example is
DBLP:journals/corr/abs-1711-10677 , which uses an additive homomorphic
architecture to defeat honest-but-curious adversaries using federated logistic
regression on the encrypted vertical FL data. However, the overburdening of
the system with additional computational and communication costs is a typical
downside of such systems.

#### Secure Multiparty Computation

Secure Multiparty Computation (SMC) 4568388  is a sub-field of cryptography,
in which multiple parties cooperate to estimate a function on their input,
without compromising privacy between participants. As an example of SMC is
proposed in 7958569 , which enables collaborative training without
compromising privacy. Nevertheless, SMC is followed by considerable
computational and communication burden, which may deter parties from
collaborating. This dramatic rise in communication and processing costs makes
SMC undesirable for large-scale FL.

For safe aggregation of individual model updates, 10.1145/3133956.3133982
suggested a protocol based on SMC that is secure, communication-efficient, and
failure-resistant. Their technique makes the communicating information
perceivable only when they are aggregated. Thus, their protocol is secured in
honest-but-curious and malicious setups. In other words, none of the
participants learns anything beyond the aggregate of the inputs of numerous
honest users 10.1145/3133956.3133982 .

Aside from the efficiency-related issues of SMC, another major problem for
SMC-based systems is the necessity for all participants to coordinate at the
same time during the training process. In practise, such multiparty
interactions may not be ideal, especially in FL contexts where the client-
server design is typical. Moreover, while the privacy of client data is
preserved, malicious parties may still infer sensitive information from the
final output 7286780 ; 10.1145/3196494.3196522 . As a result, SMC cannot
guarantee information leakage protection, necessitating the incorporation of
additional differential privacy mechanisms within the multiparty protocol to
overcome these issues 10.1145/1807167.1807247 ; 10.1007/978-3-642-24178-9_9 .
In addition, all cryptography-based protocols preclude the audition of
updates, during which a hacker can covertly inject backdoor features into the
shared model pmlr-v97-bhagoji19a .

#### Differential Privacy

The idea of Diffrential Privacy (DP) is to inject random noise into the
generating updates so that the data interpretation becomes infeasible for
malicious entities. DP is primarily used to safeguard DFL communications
against privacy attacks (e.g., inference attacks); however, the literature
also shows that DP is also beneficial against data poisoning, as these attacks
are usually designed based on the communicated gradients
10.5555/3367471.3367701 ; 10.1145/2976749.2978318 ;
DBLP:journals/corr/abs-1712-07557 .

In contrast to homomorphic encryption and SMC whose main disadvantage was
communication overhead, DP does not overburden the system in this sense.
Instead, DP comes at the cost of deteriorating the model quality since the
injected noise can potentially add up to the noise within the constructed
model. Moreover, DP provides resistance to poisoning attempts due to its group
privacy trait. As a result, as the number of attackers increases, this defense
will reduce significantly.

DP can be centralized, local, or distributed. In centralized DP, the noise
addition is performed via a server, which makes it impractical in FDL. On the
other hand, local 6736718  and distributed DP 10.1145/2873069 ; dwork2006our
both assume that the aggregator is not trusted, which perfectly complies with
the FDL paradigm. In the local variant, participants inject noise to their
estimated gradients before sharing them over the blockchain. However, research
on local DP indicates its impotency to provide privacy guarantee on large-
scale and heterogeneous models with numerous parameters
DBLP:journals/corr/abs-2009-05537 ; papernot2017semisupervised . In FDL, the
injected noise should be calibrated to ensure successful DP. Despite the
appealing security qualities of local DP, its practicality becomes
questionable when dealing with immense number of users.

It is also possible to integrate TL into DP. For instance, private aggregation
of $\mathcal{T}$ ensembles papernot2017semisupervised ; papernot2018scalable
initially training an ensemble of $\mathcal{T}$s on disjoint subsets of
private data, then perturbs the ensemble’s information by introducing noise to
the aggregated $\mathcal{T}$ votes before transferring the information to a
$\mathcal{S}$. The aggregated output of the ensemble then is used to train a
$\mathcal{S}$ model, which learns to precisely replicate the ensemble. To meet
the desired accuracy, this method requires a large number of clients, and each
of them must have sufficient training records. On the other hand, most
industrial applications deal with imbalanced data 8892480 , and similarly, FL
data is often imbalanced among parties that does not comply with this
assumption.

It has been established that the usage of DP helps prevent inference attacks
in TL 10.1145/2976749.2978318 ; 236254 , albeit at the cost of potential
utility loss 236254 ; 10.1145/2976749.2978355 . By definition, DP seeks to
conceal the presence or absence of a record in a dataset, which works against
the objective of membership inference attacks. 10.1145/2508859.2516686  draws
attention to the fact that these two concepts seem to counteract each other
and establishes a link between DP and membership inference attacks. This has
been often carried out by minimizing the bias of the model towards any
individual sample or feature by including adequate differential privacy noise.
The existing connection between records and features is elaborated in
yeom2018privacy .

###  5.2 Model Robustness

Defenses are classified into two types: proactive and reactive. The former is
an inexpensive method of anticipating attacks and associated consequences. The
reactive defense operates by detecting an invasion and taking preventative
steps. In the production environment, it is often deployed as a patch-up. FL
presents multiple additional attack surfaces throughout training, resulting in
complicated and unique countermeasures. In this part, we will look at some of
the most common types of FL defensive tactics and investigate their usefulness
and limits.

#### Anomaly Detection

Anomaly detection methods actively identify and stops malicious updates from
affecting the system Hallaji2022 ; article . These methods may be also used in
FL systems to identify potential threats 9411833 . One frequent technique for
handling untargeted adversaries is to calculate a specific test error rate on
updates and reject those disadvantageous or neutral to the global model
article .

In 10.1145/2991079.2991125 , a protection mechanism is proposed that clusters
participants based on their submitted suggestive attributes to identify
malicious updates. It produces groups of benign and malicious users with each
indicator attribute. Another detector monitors drifts in updates using a
distance measure for different participants NIPS2017_f4b9ec30 .
DBLP:journals/corr/abs-1910-09933  proposed producing low-dimensional model
weight surrogates to recognise anomalous updates from participants. An outlier
detection-based paradigm presented in 8418594  selects a number of updates
that work in favour of objective function among others. DL-based anomaly
detection is often performed using autoencoders app8122663 . These neural
network models represent data in a latent space, in which anomalies can be
discriminated. Examples of anomaly detection in FL are given in 247652 ;
li2020learning .

Backdoor attacks in TL may also be mitigated via anomaly detection. As an
example, 8119189  employs an anomaly detection technique to determine whether
the input is a possible Trojan trigger. If the input is identified as an
anomaly, it will not be passed to the neural network. This approach employs
support vector machines and decision trees to find anomalies.

#### Robust Aggregation

The security of FL aggregation techniques is of paramount importance.
Extensive research endeavours has been dedicated to research on robust
aggregation that can recognize and dismiss inaccurate or malicious updates
during training pillutla2019robust ; DBLP:journals/corr/abs-2009-08294 .
Furthermore, strong aggregation approaches must be able to withstand
communications disturbances, client dropout, and incorrect model updates on
top of hostile participants 9026922 . Existing constraints
DBLP:journals/corr/abs-1912-04977  of aggregation methods for integration with
FL lead to the emergence of more mature techniques such as adaptive
aggregation, have been developed. This technique incorporates repeated median
regression into an iteratively re-weighted least squares
DBLP:journals/corr/abs-1912-11464  and a resilient aggregation oracle
pillutla2019robust . This form of aggregation has been shown to be resistant
to distortion rates distortion up to fifty percent of the users. To assess
participants’ prospective contributions, 10.1145/3432291.3432303  recommends
employing a Gaussian distribution. They also provided layer-by-layer
optimization procedures to ensure that the aggregation works effectively.
Experiments reveal that this aggregation method surpasses the well-known
FedAvg in terms of robustness and convergence. Aggregation methods can also
help with the problem of FL client heterogeneity. FedProx was designed as a
re-parametrization and generalisation of FedAvg
DBLP:journals/corr/abs-1812-06127 . In comparison to FedAvg, it exhibits
substantially more consistent and accurate convergence behaviour in highly
heterogeneous FL systems.

Pruning also eliminates backdoors in TL by removing duplicate neurons that are
no longer relevant for normal classification 10.1007/978-3-030-00470-5_13 .
However, it has been discovered that when applied to particular models, it
significantly degrades the model performance 8835365 .

#### Pruning

Pruning decreases the size of a deep learning model by removing neurons in
order to reduce complexity, increase accuracy, and eliminate backdoors Clients
in the FL environment are abundant, and they are frequently linked to the
server via unreliable or costly connections. When it comes to training large-
scale deep neural networks (DNN), engineers encounter a huge challenge due to
the restricted processing power on some edge devices. Federated dropout
DBLP:journals/corr/abs-1812-07210  demonstrates that a good generalization can
be achieved by allowing users to perform partial training on the global model.
Both transmission and local processing costs are reduced by means of federated
dropout. Discarding inactive nodes of a network also make it robust against
backdoors 10.1007/978-3-030-00470-5_13 . Passing benign and malicious
behaviour into the same set of activations, one can combine pruning with fine-
tuning. It has been shown that using this approach the backdoor task accuracy
is reduced to zero in several circumstances.

#### Trusted Execution Environment

Trusted Execution Environment (TEE) secures linked devices in FL, establishing
digital trust 9411833 . By using an isolated and encrypted part of the main
processor, it safeguards devices from inserting incorrect training results.
TEE can be used in FL to mitigate algorithmic threats CHEN202069 ;
10.1145/3458864.3466628 . The validity of a participating device in a TEE
Authentication should be checked by the connected service with which it is
attempting to enroll. Furthermore, until the matching party provides a
message, the status of code execution stays hidden. The execution route of the
code cannot be changed until it takes explicit input or a validated
interruption. The TEE is in charge of all data access privileges.
Cryptographic technologies are used to secure TEE communications. Only the TEE
secure environment stores, maintains, and uses private and public encryption
keys. The TEE can show a remote client what code is presently being executed
as well as the starting state. TEE can aid in resolving a key challenge for FL
security since it is becoming progressively important in securing the central
server and clients against hackers and preventing data theft.

#### Zero-Knowledge Proofs

Zero-knowledge proofs allow one party to verify assertions made by another
party without exchanging or exposing underlying data 6547113 . In the
mid-1980s, MIT researchers initially promoted the notion of zero-knowledge
proofs doi:10.1137/0218012 . Zero-knowledge procedures are probabilistic
evaluations, meaning they cannot guarantee something with 100 percent
certainty that it will be discovered. Instead, they supply unconnected bits of
information that might add up to suggest that an statement’s truth is
overwhelmingly likely. Thus, zero-knowledge proofs offer a practical answer to
the problem of private data verifiability. For instance, zero-knowledge proofs
can be employed in FL to make sure the clients’ model used authentic feature
for training and generating an update. Even though this approach has many
appealing potentials for transforming secure update monitoring, we need to
better understand how to use these approaches and discover problems in how the
modules are constructed and deployed. Zero-knowledge proofs protocols mostly
maintain their performance regardless of the volume of data.

#### Adversarial Training

Adversarial training denotes a min-max optimization problem in which the
adversarial samples and model parameters are updated alternately. Generally,
adversarial samples are generated through maximizing a classification loss,
and model parameters are attained via minimizing a loss w.r.t. the generated
adversarial samples DBLP:journals/corr/abs-1812-06127 ; book ; 9609642 . This
approach can provide an acceptable resilience against evasion attacks
NEURIPS2019_7503cfac ; DBLP:journals/corr/abs-1812-03411 . While there are
different approaches to carry out adversarial training, including the so-
called generative adversarial networks Farajzadeh-Zanjani2022 ; 9563211 ;
9360878 ; FARAJZADEHZANJANI2021101 , non of them are flawless. To begin with,
this approach was mainly designed for independent and identically distributed
data. This is while FL data do not comply with this assumption, and, thus,
further research is required to investigate the practicality of adversarial
training in FL DBLP:journals/corr/abs-1903-10484 . Furthermore, this approach
can be very time-consuming. In addition, adversarial training often improves
resilience for cases utilized during the training. Furthermore, it can
possibly exhaust FL participants’ limited computational capabilities and
leaving the trained model exposed to various forms of adversarial noise
pmlr-v97-engstrom19a ; NEURIPS2019_5d4ae76f .

Adversarial training also aids in the prevention of TL inference attacks
zou2020privacy ; 10.1145/3243734.3243855 . For example,
10.1145/3243734.3243855  presents a technique for training models with
membership privacy, which assures that a model’s predictions are
indistinguishable on both training and unobserved samples of similar
distributions. This technique formulates a min-max problem and develops an
adversarial training procedure that minimizes the model’s prediction loss
along with the attack maximum gain. This method, which ensures membership
privacy, also functions as a powerful regularizer and aids in model
generalization.

Mitigating adversarial attacks is another use-case of adversarial training. As
an example, aithal2021mitigating  investigates the approach of introducing
white noise to DL results to counter these assaults and emphasises on the
noise-cost balance. The query count of the attacker is calculated analytically
based on the noise standard deviation. Consequently, the degree of noise
required to prevent attacks can be easily determined while maintaining the
appropriate extent of security defined by query count and limiting performance
deterioration.

#### Multi-Task Learning

The statistical and system difficulties of FL such as efficiency and fault
fault tolerance are addressed using Federated Multi-task Learning (FML)
NIPS2017_6211080f ; pmlr-v139-li21h . The goal of FML is to learn models for
numerous related activities at the same time. It can perfectly handle
statistical problems since it can immediately infer associations among
non-i.i.d. and imbalanced data. For instance, NIPS2017_6211080f  designs a FML
approach to accelerate convergence while managing devices that disconnect on a
regular basis. This approach is also flexible against data heterogeneity.

#### Moving Target Defense

Moving target defense 6578785 ; 10.1145/2663474.2663486 ; article2 ; 9047923
confuses malevolent adversaries by constantly re-configuring the system and
make it harder for intruders to infer system states. This may be accomplished
by randomly shifting the FL system’s components and nullify their knowledge of
the system. This defense mechanism also creates complexity and expense for
attackers and reduces the disclosure of vulnerabilities and the possibility of
an attack. It also improves the system resilience, specially against sniffing
attacks. This dynamic mechanisms disables intruders to make accurate
estimations regarding the required resources for attacking the FL training
process.

#### Client Trustworthiness Assessment

Poisoning attacks in FL are mostly studied in a centralized context. Only a
limited number of research endeavours, however, address these attacks in
decentralized systems Xie2020DBA: , where several adversarial parties follow
the same objective and attempt to poison the training data. Although these
attacks pose a greater risk in FL, their efficiency remains unknown compared
to their centralized variants. This protection approach works by detecting
authorized clients and drastically increasing the rate of failure for
poisoning attacks, even when the attack is initiated in a distributed fashion.

#### Federated Distillation

Exchanging model parameters becomes prohibitively expensive when communication
resources are limited, especially for contemporary big DNNs. In this sense,
federated distillation DBLP:journals/corr/abs-2009-05537  is an appealing FL
option since it only transmits model outputs, which are often considerably
less in size than the model sizes. Knowledge distillation is a fundamental
algorithm in federated distillation li2019fedmd . The goal of knowledge
distillation is to perform TL from a large model ($\mathcal{T}$) to a compact
model ($\mathcal{S}$). In FL, this idea translates into sharing the knowledge
of a model rather than the parameters, which improve FL’s resilience while
reducing communication and computing costs.

#### Regularization

Classifiers make more confident predictions when confronted with data samples
they have been trained on before. For this reasons, overfitting of a model can
lead to a successful membership inference. Classifiers make more reliable
predictions when confronted with records they have been trained on before. To
tackle this issue, researchers have investigated the usage of regularization
for preventing overfitting, which in turn eliminates membership inference. The
conventional $L_{2}$ regularizer, as an example, is examined during the
training process of a target classifier 7958568 .

Dropout is another regularization strategy intended to counter membership
inference attacks. It was employed in DBLP:journals/corr/abs-1806-01246  to
counter membership inference attacks. In each training cycle, dropout
dismisses a neuron with a particular probability.

Model stacking is a conventional ensemble approach that combines the findings
of multiple weak classifiers to form a strong model.
DBLP:journals/corr/abs-1806-01246  investigated the use of this method to
counter membership inference. The target classifier, in particular, is
comprised of three models grouped into a tree structure (i.e., one model at
top and two models at the bottom of the tree).

The original data samples are fed to leafs of the tree, while the results
obtained from the leafs are inputs to the top of the tree. The tree models are
trained using separate sets of data samples, which decreases the likelihood
that the target classifier will recall any particular point, which in turn
reduces overfitting.

#### Perturbing Posteriors

Rather than meddling with the target classifier’s training procedure, one can
introduce noise to the classifiers’ outputs 10.1145/3319535.3363201 . This
concept is referred to as perturbing posteriors. For instance,
10.1145/3319535.3363201 , designs a method to protect against membership
inference launched in a black-box setting. This defensive approach operates in
two stages and offers theoretical robustness guarantee. The first stage
involves locating a generated noise vector that may be used to convert a
vector of confidence scores into an adversarial example. This noise vector is
added to the confidence scores with a probability in the next stage.

##  6 Future Research

As mentioned previously, it is anticipated that FTL is most vulnarible against
backdoor, membership inference, feature inference, and adversarial samples
(refer back to Fig. 1). In this section, we outline future development
requirements that we believe will be promising for FTL in this sense.

###  6.1 Decentralized Federated Transfer Learning

Decentralized FL is a new study topic in which the system has no singular
central server. Decentralized FL may be more beneficial in business-based FL
instances when third parties are not trusted by the clients. Each client might
be selected as a server in a turn-based fashion. As of now, there are no
decentralized FTL protocols in the literature. It would be fascinating to see
if the same risks that exist in server-based FL also arise in decentralized
FTL.

###  6.2 Flaws in Current Defense Mechanisms

Because FL cannot review updates for privacy reasons, it is vulnerable to
poisoning attacks, which is often used to counter adversarial attacks in ML,
remains a questionable choice in FL since it was designed particularly for
i.i.d. data and its effectiveness in non-i.i.d. scenarios is unknown. This can
become problematic in the case of FTL furthermore. Besides, adversarial
training is computationally expensive and may degrade efficiency, regardless
of the type of FL.

Available privacy defenses for FTL are mostly based on homomorphic encryption
and secret sharing. Nevertheless, since DP is used as a privacy-preserving
method in both FL and TL, it may be also used for FTL. If future works extend
DP to FTL, there are a number of points to consider. Firstly, DP cannot handle
attribute inference. Secondly, client-level DP is designed for large-scale
systems with numerous clients, and using it in smaller systems may affect it
performance.

###  6.3 Optimizing Defense Mechanism Deployment

The servers will require additional computational resources while implementing
defense mechanisms to verify if any attacker is targeting the FTL system.
Additionally, different forms of defense systems may have varying degrees of
efficiency against different types of threats, as well as varying costs. It is
crucial to look into how to optimize the deployment of defensive systems or
the declaration of deterrent measures for FTL.

###  6.4 Achieving Simultaneous Objectives

There are no extant research on FL or FTL that can achieve the following
objectives at the same time DBLP:journals/corr/abs-2012-06337 :

  1. 1.

Rapid model convergence.

  2. 2.

Descent generalization of model.

  3. 3.

Efficient communication.

  4. 4.

Preserving privacy.

  5. 5.

Resilience to targeted and untargeted attacks.

  6. 6.

Fault tolerance.

Past efforts sought to tackle several objectives simultaneously 9130840 .
xu2021reputation  tackled cooperative fairness and privacy at the same time,
and a architecture has been developed to solve mitigate these problems. To cut
communication overhead and provide privacy perks, 10.5555/3327757.3327856
integrated DP with model compression approaches. Another research
bernstein2019signsgd  concentrates on enhancing convergence and preventing
gradient leakage. Nevertheless, it is crucial to remember that privacy and
robustness are incompatible by nature, as protecting against performance
attacks typically necessitates full access to the training samples, which is
irreconcilable with FTL’s privacy requirements. Even though encryption and DP-
based approaches can guarantee verifiably privacy-preserving, they are
vulnerable to poisoning techniques and may result in models with unfavourable
privacy-performance trade-off. Finding a cohesive design that meets all of the
aforementioned criteria is indeed undiscovered in the FTL domain.

###  6.5 Heterogeneity of Federated Transfer Learning

The vast majority of privacy and robustness studies have been conducted on FL
with homogenous designs. On the other hand, there is a common assumption of
feature co-occurrence among most of the available work on heterogeneous FL.
For FTL to be secure, the existing defense mechanisms should be compatible
with fully heterogeneous feature space 9005992 .

##  7 Conclusion

FTL is one of the latest fields of machine learning, it is evolving at a rapid
pace and will be a focal point of research in machine learning and privacy. As
FL and TL evolve, so will the dangers to FTL’s privacy and security. It is
critical to conduct a wide assessment of present FL and TL threats and
countermeasures so that upcoming FTL designs consider the possible weaknesses
in existing models. This survey provides a clear and straightforward review of
the privacy and robustness attack and possible defense mechanisms that may be
used in FTL. Designing a coherent FTL defensive mechanism that can withstand
various attacks without decreasing model performance would demand
multidisciplinary collaboration in the scientific community.

## Acknowledgements

This work is supported by the Natural Sciences and Engineering Research
Council of Canada (NSERC) under funding reference numbers CGSD3-569341-2022
and RGPIN-2021-02968.

## References

  * (1) Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., Zhang, L.: Deep learning with differential privacy.  In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS ’16, p. 308–318 (2016)
  * (2) Ács, G., Castelluccia, C.: I have a dream! (differentially private smart metering).  In: Information Hiding, pp. 118–132. Springer Berlin Heidelberg (2011)
  * (3) Agarwal, N., Suresh, A.T., Yu, F., Kumar, S., McMahan, H.B.: Cpsgd: Communication-efficient and differentially-private distributed sgd.  In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, p. 7575–7586 (2018)
  * (4) Aithal, M.B., Li, X.: Mitigating black-box adversarial attacks via output noise perturbation (2021).  ArXiv:2109.15160
  * (5) Ang, F., Chen, L., Zhao, N., Chen, Y., Wang, W., Yu, F.R.: Robust federated learning with noisy communication.  IEEE Transactions on Communications 68(6), 3452–3464 (2020)
  * (6) Aono, Y., Hayashi, T., Trieu Phong, L., Wang, L.: Scalable and secure logistic regression via homomorphic encryption.  In: Proceedings of the Sixth ACM Conference on Data and Application Security and Privacy, p. 142–144 (2016)
  * (7) Backes, M., Berrang, P., Humbert, M., Manoharan, P.: Membership privacy in microrna-based studies.  In: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, p. 319–330 (2016)
  * (8) Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., Shmatikov, V.: How to backdoor federated learning.  In: Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, _Proceedings of Machine Learning Research_ , vol. 108, pp. 2938–2948. PMLR (2020)
  * (9) Barreno, M., Nelson, B., Joseph, A., Tygar, J.: The security of machine learning.  Machine Learning 81, 121–148 (2010)
  * (10) Barreno, M., Nelson, B., Sears, R., Joseph, A.D., Tygar, J.D.: Can machine learning be secure?  In: Proceedings of the ACM Symposium on Information, Computer and Communications Security, p. 16–25 (2006)
  * (11) Benhamouda, F., Joye, M., Libert, B.: A new framework for privacy-preserving aggregation of time-series data.  ACM Trans. Inf. Syst. Secur. 18(3) (2016)
  * (12) Bernstein, J., Zhao, J., Azizzadenesheli, K., Anandkumar, A.: signsgd with majority vote is communication efficient and fault tolerant (2019).  ArXiv:1810.05291
  * (13) Bhagoji, A.N., Chakraborty, S., Mittal, P., Calo, S.: Analyzing federated learning through an adversarial lens.  In: Proceedings of the 36th International Conference on Machine Learning, vol. 97, pp. 634–643 (2019)
  * (14) Biggio, B., Nelson, B., Laskov, P.: Support vector machines under adversarial label noise.  In: C.N. Hsu, W.S. Lee (eds.) Proceedings of the Asian Conference on Machine Learning, _Proceedings of Machine Learning Research_ , vol. 20, pp. 97–112. PMLR (2011)
  * (15) Blanchard, P., El Mhamdi, E.M., Guerraoui, R., Stainer, J.: Machine learning with adversaries: Byzantine tolerant gradient descent.  In: Advances in Neural Information Processing Systems, vol. 30 (2017)
  * (16) Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H.B., Patel, S., Ramage, D., Segal, A., Seth, K.: Practical secure aggregation for privacy-preserving machine learning.  In: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, CCS ’17, p. 1175–1191 (2017)
  * (17) Bouacida, N., Mohapatra, P.: Vulnerabilities in federated learning.  IEEE Access 9, 63229–63249 (2021)
  * (18) Caldas, S., Konečný, J., McMahan, H.B., Talwalkar, A.: Expanding the reach of federated learning by reducing client resource requirements (2018).  ArXiv:1812.07210
  * (19) Chen, Y., Luo, F., Li, T., Xiang, T., Liu, Z., Li, J.: A training-integrity privacy-preserving federated learning scheme with trusted execution environment.  Information Sciences 522, 69–79 (2020)
  * (20) Chen, Y., Qin, X., Wang, J., Yu, C., Gao, W.: Fedhealth: A federated transfer learning framework for wearable healthcare.  IEEE Intelligent Systems 35(4), 83–93 (2020)
  * (21) Chin, T., Zhang, C., Marculescu, D.: Improving the adversarial robustness of transfer learning via noisy feature distillation.  CoRR (2020).  ArXiv:2002.02998
  * (22) Colbaugh, R., Glass, K.: Moving target defense for adaptive adversaries.  In: IEEE International Conference on Intelligence and Security Informatics, pp. 50–55 (2013)
  * (23) Damgård, I., Pastro, V., Smart, N., Zakarias, S.: Multiparty computation from somewhat homomorphic encryption.  In: Advances in Cryptology – CRYPTO, pp. 643–662 (2012)
  * (24) Duchi, J.C., Jordan, M.I., Wainwright, M.J.: Local privacy and statistical minimax rates.  In: 1st Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1592–1592 (2013)
  * (25) Dwork, C., Kenthapadi, K., McSherry, F., Mironov, I., Naor, M.: Our data, ourselves: Privacy via distributed noise generation.  In: Advances in Cryptology (EUROCRYPT 2006), _Lecture Notes in Computer Science_ , vol. 4004, pp. 486–503. Springer Verlag (2006)
  * (26) Engstrom, L., Tran, B., Tsipras, D., Schmidt, L., Madry, A.: Exploring the landscape of spatial robustness.  In: K. Chaudhuri, R. Salakhutdinov (eds.) Proceedings of the 36th International Conference on Machine Learning, _Proceedings of Machine Learning Research_ , vol. 97, pp. 1802–1811. PMLR (2019)
  * (27) Fang, M., Cao, X., Jia, J., Gong, N.: Local model poisoning attacks to byzantine-robust federated learning.  In: 29th USENIX Security Symposium (USENIX Security 20), pp. 1605–1622 (2020)
  * (28) Farajzadeh-Zanjani, M., Hallaji, E., Razavi-Far, R., Saif, M.: Generative-adversarial class-imbalance learning for classifying cyber-attacks and faults - a cyber-physical power system.  IEEE Transactions on Dependable and Secure Computing pp. 1–1 (2021).  DOI 10.1109/TDSC.2021.3118636
  * (29) Farajzadeh-Zanjani, M., Hallaji, E., Razavi-Far, R., Saif, M.: Generative adversarial dimensionality reduction for diagnosing faults and attacks in cyber-physical systems.  Neurocomputing 440, 101–110 (2021)
  * (30) Farajzadeh-Zanjani, M., Hallaji, E., Razavi-Far, R., Saif, M., Parvania, M.: Adversarial semi-supervised learning for diagnosing faults and attacks in power grids.  IEEE Transactions on Smart Grid 12(4), 3468–3478 (2021)
  * (31) Farajzadeh-Zanjani, M., Razavi-Far, R., Saif, M., Palade, V.: Generative adversarial networks: A survey on training, variants, and applications.  In: Generative Adversarial Learning: Architectures and Applications, pp. 7–29. Springer International Publishing, Cham (2022)
  * (32) Fredrikson, M., Jha, S., Ristenpart, T.: Model inversion attacks that exploit confidence information and basic countermeasures.  In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, p. 1322–1333. Association for Computing Machinery (2015)
  * (33) Fredrikson, M., Lantz, E., Jha, S., Lin, S., Page, D., Ristenpart, T.: Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing.  In: Proceedings of the 23rd USENIX Conference on Security Symposium, p. 17–32 (2014)
  * (34) Fu, S., Xie, C., Li, B., Chen, Q.: Attack-resistant federated learning with residual-based reweighting.  CoRR (2019).  ArXiv:1912.11464
  * (35) Fung, C., Yoon, C.J.M., Beschastnikh, I.: Mitigating sybils in federated learning poisoning.  CoRR (2018).  ArXiv:1808.04866
  * (36) Gao, D., Liu, Y., Huang, A., Ju, C., Yu, H., Yang, Q.: Privacy-preserving heterogeneous federated transfer learning.  In: IEEE International Conference on Big Data, pp. 2552–2559 (2019)
  * (37) Gentry, C.: Fully homomorphic encryption using ideal lattices.  In: Proceedings of the Forty-First Annual ACM Symposium on Theory of Computing, STOC ’09, p. 169–178. Association for Computing Machinery (2009)
  * (38) Geyer, R.C., Klein, T., Nabi, M.: Differentially private federated learning: A client level perspective abs/1712.07557 (2017).  ArXiv:1712.07557
  * (39) Goldwasser, S., Micali, S., Rackoff, C.: The knowledge complexity of interactive proof systems.  SIAM Journal on Computing 18(1), 186–208 (1989)
  * (40) Goryczka, S., Xiong, L.: A comprehensive comparison of multiparty secure additions with differential privacy.  IEEE Transactions on Dependable and Secure Computing 14(5), 463–477 (2017)
  * (41) Grama, M., Musat, M., Muñoz-González, L., Passerat-Palmbach, J., Rueckert, D., Alansary, A.: Robust aggregation for adaptive privacy preserving federated learning in healthcare (2020).  ArXiv:2009.08294
  * (42) Hallaji, E., Farajzadeh-Zanjani, M., Razavi-Far, R., Palade, V., Saif, M.: Constrained generative adversarial learning for dimensionality reduction.  IEEE Transactions on Knowledge and Data Engineering pp. 1–1 (2021).  DOI 10.1109/TKDE.2021.3126642
  * (43) Hallaji, E., Razavi-Far, R., Saif, M.: DLIN: Deep ladder imputation network.  IEEE Transactions on Cybernetics pp. 1–13 (2021).  DOI 10.1109/TCYB.2021.3054878
  * (44) Hallaji, E., Razavi-Far, R., Saif, M.: Embedding time-series features into generative adversarial networks for intrusion detection in internet of things networks.  In: Generative Adversarial Learning: Architectures and Applications, pp. 169–183. Springer International Publishing, Cham (2022)
  * (45) Hardy, S., Henecka, W., Ivey-Law, H., Nock, R., Patrini, G., Smith, G., Thorne, B.: Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption (2017).  ArXiv:1711.10677
  * (46) Hassani, H., Hallaji, E., Razavi-Far, R., Saif, M.: Unsupervised concrete feature selection based on mutual information for diagnosing faults and cyber-attacks in power systems.  Engineering Applications of Artificial Intelligence 100, 104150 (2021)
  * (47) Hassani, H., Razavi-Far, R., Saif, M.: Real-time out-of-step prediction control to prevent emerging blackouts in power systems: A reinforcement learning approach.  Applied Energy 314, 118861 (2022).  DOI https://doi.org/10.1016/j.apenergy.2022.118861
  * (48) Hitaj, B., Ateniese, G., Perez-Cruz, F.: Deep models under the gan: Information leakage from collaborative deep learning.  In: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, p. 603–618. Association for Computing Machinery (2017)
  * (49) Jacobsen, J., Behrmann, J., Carlini, N., Tramèr, F., Papernot, N.: Exploiting excessive invariance caused by norm-bounded adversarial robustness (2019).  ArXiv:1903.10484
  * (50) Jagielski, M., Oprea, A., Biggio, B., Liu, C., Nita-Rotaru, C., Li, B.: Manipulating machine learning: Poisoning attacks and countermeasures for regression learning.  In: IEEE Symposium on Security and Privacy (SP), pp. 19–35 (2018)
  * (51) Jayaraman, B., Evans, D.: Evaluating differentially private machine learning in practice.  In: 28th USENIX Security Symposium, pp. 1895–1912 (2019)
  * (52) Jere, M.S., Farnan, T., Koushanfar, F.: A taxonomy of attacks on federated learning.  IEEE Security Privacy 19(2), 20–28 (2021)
  * (53) Jia, J., Salem, A., Backes, M., Zhang, Y., Gong, N.Z.: Memguard: Defending against black-box membership inference attacks via adversarial examples.  In: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, p. 259–274 (2019)
  * (54) Kairouz, P., et al.: Advances and open problems in federated learning (2019).  ArXiv:1912.04977
  * (55) Kantarcioglu, M., Clifton, C.: Privacy-preserving distributed mining of association rules on horizontally partitioned data.  IEEE Transactions on Knowledge and Data Engineering 16(9), 1026–1037 (2004)
  * (56) Kim, M., Song, Y., Wang, S., Xia, Y., Jiang, X.: Secure logistic regression based on homomorphic encryption: Design and evaluation.  JMIR Med Inform 6(2), e19 (2018)
  * (57) Lei, C., Zhang, H.Q., Jinglei, T., Zhang, Y.C., Liu, X.H.: Moving target defense techniques: A survey.  Security and Communication Networks 2018, 1–25 (2018)
  * (58) Li, D., Wang, J.: Fedmd: Heterogenous federated learning via model distillation (2019).  ArXiv:1910.03581
  * (59) Li, N., Qardaji, W., Su, D., Wu, Y., Yang, W.: Membership privacy: A unifying framework for privacy definitions.  In: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, p. 889–900. Association for Computing Machinery (2013)
  * (60) Li, S., Cheng, Y., Liu, Y., Wang, W., Chen, T.: Abnormal client behavior detection in federated learning abs/1910.09933 (2019).  ArXiv:1910.09933
  * (61) Li, S., Cheng, Y., Wang, W., Liu, Y., Chen, T.: Learning to detect malicious clients for robust federated learning (2020).  ArXiv:2002.00211
  * (62) Li, T., Hu, S., Beirami, A., Smith, V.: Ditto: Fair and robust federated learning through personalization.  In: M. Meila, T. Zhang (eds.) Proceedings of the 38th International Conference on Machine Learning, _Proceedings of Machine Learning Research_ , vol. 139, pp. 6357–6368. PMLR (2021)
  * (63) Liu, K., Dolan-Gavitt, B., Garg, S.: Fine-pruning: Defending against backdooring attacks on deep neural networks.  In: M. Bailey, T. Holz, M. Stamatogiannakis, S. Ioannidis (eds.) Research in Attacks, Intrusions, and Defenses, pp. 273–294. Springer International Publishing, Cham (2018)
  * (64) Liu, Y., Kang, Y., Xing, C., Chen, T., Yang, Q.: A secure federated transfer learning framework.  IEEE Intelligent Systems 35(4), 70–82 (2020)
  * (65) Liu, Y., Xie, Y., Srivastava, A.: Neural trojans.  In: IEEE 35th International Conference on Computer Design, pp. 45–48 (2017)
  * (66) Lu, Y., Fan, L.: An efficient and robust aggregation algorithm for learning federated cnn.  In: Proceedings of the 3rd International Conference on Signal Processing and Machine Learning, p. 1–7 (2020)
  * (67) Lyu, L., Li, Y., Nandakumar, K., Yu, J., Ma, X.: How to democratise and protect ai: Fair and differentially private decentralised deep learning.  IEEE Transactions on Dependable and Secure Computing pp. 1–1 (2020)
  * (68) Lyu, L., Yu, H., Ma, X., Sun, L., Zhao, J., Yang, Q., Yu, P.S.: Privacy and robustness in federated learning: Attacks and defenses (2020).  ArXiv:2012.06337
  * (69) Lyu, L., Yu, H., Yang, Q.: Threats to federated learning: A survey (2020).  ArXiv:2003.02133
  * (70) Ma, Y., Zhu, X., Hsu, J.: Data poisoning against differentially-private learners: Attacks and defenses.  In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, p. 4732–4738 (2019)
  * (71) McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.y.: Communication-Efficient Learning of Deep Networks from Decentralized Data.  In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, _Proceedings of Machine Learning Research_ , vol. 54, pp. 1273–1282. PMLR (2017)
  * (72) Melis, L., Song, C., De Cristofaro, E., Shmatikov, V.: Exploiting unintended feature leakage in collaborative learning.  In: EEE Symposium on Security and Privacy (SP), pp. 691–706 (2019)
  * (73) Mo, F., Haddadi, H., Katevas, K., Marin, E., Perino, D., Kourtellis, N.: Ppfl: Privacy-preserving federated learning with trusted execution environments.  In: Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, p. 94–108 (2021)
  * (74) Mohassel, P., Zhang, Y.: Secureml: A system for scalable privacy-preserving machine learning.  In: IEEE Symposium on Security and Privacy, pp. 19–38 (2017)
  * (75) Nasr, M., Shokri, R., Houmansadr, A.: Machine learning with membership privacy using adversarial regularization.  In: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, p. 634–646. Association for Computing Machinery (2018)
  * (76) Ogburn, M., Turner, C., Dahal, P.: Homomorphic encryption.  Procedia Computer Science 20, 502–509 (2013).  Complex Adaptive Systems
  * (77) Paillier, P.: Public-key cryptosystems based on composite degree residuosity classes.  In: J. Stern (ed.) Advances in Cryptology — EUROCRYPT ’99, pp. 223–238. Springer Berlin Heidelberg (1999)
  * (78) Pan, S.J., Yang, Q.: A survey on transfer learning.  IEEE Transactions on Knowledge and Data Engineering 22(10), 1345–1359 (2010)
  * (79) Papernot, N., Abadi, M., Úlfar Erlingsson, Goodfellow, I., Talwar, K.: Semi-supervised knowledge transfer for deep learning from private training data (2017).  ArXiv:1610.05755
  * (80) Papernot, N., Song, S., Mironov, I., Raghunathan, A., Talwar, K., Úlfar Erlingsson: Scalable private learning with pate (2018).  ArXiv:1802.08908
  * (81) Parno, B., Howell, J., Gentry, C., Raykova, M.: Pinocchio: Nearly practical verifiable computation.  In: EEE Symposium on Security and Privacy, pp. 238–252 (2013)
  * (82) Phong, L.T., Aono, Y., Hayashi, T., Wang, L., Moriai, S.: Privacy-preserving deep learning via additively homomorphic encryption.  IEEE Transactions on Information Forensics and Security 13(5), 1333–1345 (2018)
  * (83) Pillutla, K., Kakade, S.M., Harchaoui, Z.: Robust aggregation for federated learning (2019).  ArXiv:1912.13445
  * (84) Preuveneers, D., Rimmer, V., Tsingenopoulos, I., Spooren, J., Joosen, W., Ilie-Zudor, E.: Chained anomaly detection models for federated learning: An intrusion detection case study.  Applied Sciences 8(12) (2018)
  * (85) Rastogi, V., Nath, S.: Differentially private aggregation of distributed time-series with transformation and encryption.  In: Proceedings of the CM SIGMOD International Conference on Management of Data, p. 735–746. Association for Computing Machinery (2010)
  * (86) Razavi-Far, R., Farajzadeh-Zanajni, M., Wang, B., Saif, M., Chakrabarti, S.: Imputation-based ensemble techniques for class imbalance learning.  IEEE Transactions on Knowledge and Data Engineering 33(5), 1988–2001 (2021)
  * (87) Razavi-Far, R., Ruiz-Garcia, A., Palade, V., Schmidhuber, J. (eds.): Generative Adversarial Learning: Architectures and Applications.  Springer, Cham (2022)
  * (88) Razavi-Far, R., Wan, D., Saif, M., Mozafari, N.: To tolerate or to impute missing values in v2x communications data?  IEEE Internet of Things Journal pp. 1–1 (2021).  DOI 10.1109/JIOT.2021.3126749
  * (89) Rezaei, S., Liu, X.: A target-agnostic attack on deep models: Exploiting security vulnerabilities of transfer learning (2020).  ArXiv:1904.04334
  * (90) Riazi, M.S., Weinert, C., Tkachenko, O., Songhori, E.M., Schneider, T., Koushanfar, F.: Chameleon: A hybrid secure computation framework for machine learning applications.  In: Proceedings of the Asia Conference on Computer and Communications Security, ASIACCS ’18, p. 707–721 (2018)
  * (91) Rivest, R.L., Shamir, A., Adleman, L.: A method for obtaining digital signatures and public-key cryptosystems.  Commun. ACM 21(2), 120–126 (1978)
  * (92) Saha, S., Ahmad, T.: Federated transfer learning: concept and applications (2020).  ArXiv:2010.15561
  * (93) Saha, S., Bovolo, F., Bruzzone, L.: Unsupervised deep change vector analysis for multiple-change detection in vhr images.  IEEE Transactions on Geoscience and Remote Sensing 57(6), 3677–3693 (2019)
  * (94) Sahu, A.K., Li, T., Sanjabi, M., Zaheer, M., Talwalkar, A., Smith, V.: On the convergence of federated optimization in heterogeneous networks (2018).  ArXiv:1812.06127
  * (95) Salem, A., Zhang, Y., Humbert, M., Fritz, M., Backes, M.: Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models (2018).  ArXiv:1806.01246
  * (96) Sengupta, S., Chowdhary, A., Sabur, A., Alshamrani, A., Huang, D., Kambhampati, S.: A survey of moving target defenses for network security.  IEEE Communications Surveys Tutorials 22(3), 1909–1941 (2020)
  * (97) Shafahi, A., Najibi, M., Ghiasi, M.A., Xu, Z., Dickerson, J., Studer, C., Davis, L.S., Taylor, G., Goldstein, T.: Adversarial training for free!  In: Advances in Neural Information Processing Systems, vol. 32 (2019)
  * (98) Sharma, S., Xing, C., Liu, Y., Kang, Y.: Secure and efficient federated transfer learning.  In: IEEE International Conference on Big Data (Big Data), pp. 2569–2576 (2019)
  * (99) Shen, S., Tople, S., Saxena, P.: A¡span class=”smallcaps smallercapital”¿uror¡/span¿: Defending against poisoning attacks in collaborative deep learning systems.  In: Proceedings of the 32nd Annual Conference on Computer Security Applications, ACSAC ’16, p. 508–519 (2016)
  * (100) Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models.  In: IEEE Symposium on Security and Privacy, pp. 3–18 (2017)
  * (101) Smith, V., Chiang, C.K., Sanjabi, M., Talwalkar, A.S.: Federated multi-task learning.  In: Advances in Neural Information Processing Systems, vol. 30 (2017)
  * (102) Sun, L., Lyu, L.: Federated model distillation with noise-free differential privacy (2020).  ArXiv:2009.05537
  * (103) Tramer, F., Boneh, D.: Adversarial training and robustness for multiple perturbations.  In: Advances in Neural Information Processing Systems, vol. 32 (2019)
  * (104) Vaidya, J., Clifton, C.: Privacy preserving association rule mining in vertically partitioned data.  In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, p. 639–644 (2002)
  * (105) Wang, B., Yao, Y., Shan, S., Li, H., Viswanath, B., Zheng, H., Zhao, B.Y.: Neural cleanse: Identifying and mitigating backdoor attacks in neural networks.  In: IEEE Symposium on Security and Privacy, pp. 707–723 (2019)
  * (106) Wang, B., Yao, Y., Viswanath, B., Zheng, H., Zhao, B.Y.: With great training comes great vulnerability: Practical attacks against transfer learning.  In: Proceedings of the 27th USENIX Conference on Security Symposium, SEC’18, p. 1281–1297. USENIX Association (2018)
  * (107) Wang, H., Sreenivasan, K., Rajput, S., Vishwakarma, H., Agarwal, S., Sohn, J., Lee, K., Papailiopoulos, D.S.: Attack of the tails: Yes, you really can backdoor federated learning (2020).  ArXiv:2007.05084
  * (108) Wang, S., Nepal, S., Rudolph, C., Grobler, M., Chen, S., Chen, T.: Backdoor attacks against transfer learning with pre-trained deep learning models.  IEEE Transactions on Services Computing pp. 1–1 (2020)
  * (109) Xie, C., Huang, K., Chen, P.Y., Li, B.: Dba: Distributed backdoor attacks against federated learning.  In: International Conference on Learning Representations (2020)
  * (110) Xie, C., Wu, Y., van der Maaten, L., Yuille, A.L., He, K.: Feature denoising for improving adversarial robustness (2018).  ArXiv:1812.03411
  * (111) Xu, J., Guo, P., Zhao, M., Erbacher, R.F., Zhu, M., Liu, P.: Comparing different moving target defense techniques.  In: Proceedings of the First ACM Workshop on Moving Target Defense, MTD ’14, p. 97–107. Association for Computing Machinery (2014)
  * (112) Xu, X., Lyu, L.: A reputation mechanism is all you need: Collaborative fairness and adversarial robustness in federated learning (2021).  ArXiv:2011.10464
  * (113) Yang, H., He, H., Zhang, W., Cao, X.: Fedsteg: A federated transfer learning framework for secure image steganalysis.  IEEE Transactions on Network Science and Engineering 8(2), 1084–1094 (2021)
  * (114) Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: Concept and applications.  ACM Transactions on Intelligent Systems and Technology 10(2) (2019)
  * (115) Yao, A.C.: Protocols for secure computations.  In: 23rd Annual Symposium on Foundations of Computer Science, pp. 160–164 (1982)
  * (116) Yao, Y., Li, H., Zheng, H., Zhao, B.Y.: Latent backdoor attacks on deep neural networks.  In: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, CCS ’19, p. 2041–2055. Association for Computing Machinery, New York, NY, USA (2019)
  * (117) Yeom, S., Giacomelli, I., Fredrikson, M., Jha, S.: Privacy risk in machine learning: Analyzing the connection to overfitting (2018).  ArXiv:1709.01604
  * (118) Zhu, L., Liu, Z., Han, S.: Deep leakage from gradients.  In: Advances in Neural Information Processing Systems, vol. 32 (2019)
  * (119) Zou, Y., Zhang, Z., Backes, M., Zhang, Y.: Privacy analysis of deep learning in the wild: Membership inference attacks against transfer learning (2020).  ArXiv:2009.04872