#  Sharing to learn and learning to share -
Fitting together Meta-Learning, Multi-Task Learning, and Transfer Learning : A
meta review

Richa Upadhyay1, Ronald Phlypo2, Rajkumar Saini1 and Marcus Liwicki1
1 Luleå University of Technology, Department of Computer Science, Electrical
and Space Engineering,
Embedded Intelligent Systems Lab, 97187 Lulea, Sweden.
Email -<EMAIL_ADDRESS>
2 Univ. Grenoble Alpes, CNRS, Grenoble INP*, GIPSA-lab, 38000 Grenoble, France
*Institute of Engineering and Management, Univ. Grenoble Alpes
Email -<EMAIL_ADDRESS>
###### Abstract

Integrating knowledge across different domains is an essential feature of
human learning. Learning paradigms like transfer learning, meta learning, and
multi-task learning reflect the human learning process by exploiting the prior
knowledge for new tasks, encouraging faster learning and good generalization
for new tasks. This article gives a detailed view of these learning paradigms
along with a comparative analysis. The weakness of a learning algorithm turns
out to be the strength of another, and thereby merging them is a prevalent
trait in the literature. This work delivers a literature review of the
articles, which fuses two algorithms to accomplish multiple tasks. A global
generic learning network, an ensemble of meta learning, transfer learning, and
multi-task learning, is also introduced here, along with some open research
questions and directions for future research.

###### Index Terms:

Multi-task learning, Meta learning, Transfer learning,

##  I Introduction

Machine Learning (ML) continuously draws inspiration from human cognition and
decision making [1] to develop more human-like, neurally-weighted algorithms.
One of the state-of-the-art models and successful tools in computer vision is
the Convolutional Neural Networks (CNNs), which are inspired by the biological
vision and neural activity. The traditional ML algorithms follow a single task
learning approach wherein they are trained to solve only one task at a time.
If there is a need to accomplish another task, the network needs to be re-
trained on a new dataset. This type of learning is quoted as isolated learning
in [2]. It does not utilize and preserve any prior information from the
previous learning for a future task. But humans don’t learn anything from
scratch, and they can rapidly learn new concepts due to their inherent
potential of seamlessly sharing the acquired knowledge across tasks. The more
related the tasks are, the easier it is to re-utilize the knowledge. For
example, while learning to drive a car, the knowledge acquired while riding a
bike or a motorbike comes in useful. The transfer of prior knowledge enables
humans to learn quickly and accurately in few instances because humans have a
bias that similar tasks have similar solutions (to some extent) therefore,
they acquire the concept by focusing on learning the differences between the
features of the tasks. Some of the learning techniques in ML or Deep Learning
(DL), such as transfer learning, meta learning, Multi-Task Learning (MTL),
lifelong learning, etc, are inspired from such human capability, where the aim
is to transfer the learnings of one task to another task rather than training
it from scratch.

###  I-A What is a Task ?

Before moving further with the discussion about these information sharing
learning paradigms, it is vital to understand the definition of a task.
Formally task can be defined as a piece of work performed to fulfill a
purpose. In this article, while discussing the learning paradigms, it should
be noted that task does not refer to the process of learning. In fact,
learning helps to acquire the competence required to execute a task. The most
common kinds of tasks performed by various ML or DL algorithms are
classification, regression, segmentation, machine translation, anomaly
detection, dimension reduction, and several others. If two tasks are similar
e.g., both are classification tasks, they are referred as homogeneous tasks.
But if they are different e.g., one is classification and the other is
segmentation, they are termed as heterogeneous tasks.

It should be noted that task and domain are considered different in this
article. When there is only one task from each domain, researchers
interchangeably use the term’s tasks and domain. In this article, domain
refers to the data distribution from where the training or test data is
sampled. It is possible to have multiple similar types or diverse tasks in a
domain. Overall it can be said that the tasks can be categorized depending on
the labels or ground truth in the case of supervised learning, whereas the
domain is related to the feature space of the data.

###  I-B Knowledge transfer in ML algorithms

Learn one task at a time; this is a generic approach in the field of ML. Big
problems are disassociated into smaller independent tasks that are learned
distinctly, and combined results are presented. In [3], the author introduces
MTL, wherein it is proposed that in order to have a better performance, all
the tasks should be trained simultaneously. The underlying concept is that if
all the smaller tasks share their learning, they may find it easier to learn
than learning in isolation. This idea of Multi-task learning is very similar
to the human vision system. For example, while looking at a scene, the brain
is not just able to identify objects. Indeed, it can also segment them,
understand them, identify people, classify the weather, and several other
things from a single visual. In ML classification, segmentation,
identification, etc are independent tasks, but MTL proposes to accomplish all
these tasks jointly by exploiting the fact that in the above example, from the
tasks point of view these are non identical but closely related tasks i.e.,
segmentation of a human from non-human in the picture will further aid in
identifying the person.

Similarly, learning a complex task like riding a bike relies on the motor
skills the human develops when they learn to walk as a baby. The task is
therefore learned by integrating a considerable amount of prior knowledge
across tasks. Also, while learning generalized concepts across many tasks,
humans evolve the ability to learn fast and in fewer instances. These concepts
are the bedrock for transfer learning, lifelong learning as well as meta
learning. The approach of how the prior knowledge is introduced while training
makes these learning algorithms mutually distinct.

###  I-C Scope of this work

The emphasis of this article is on learning algorithms that use previously
learned knowledge while learning an unseen but related task. Therefore, it
mainly discussed three learning paradigms: multi-task learning, meta-learning,
and transfer learning. Other related learning algorithms like Lifelong
Learning (LL), online learning, and Reinforcement Learning (RL) are not within
this work’s scope because their fundamental objective is slightly different,
i.e., progressive learning process. Also, reinforcement learning and online
learning are restricted to single task learning only, while the others involve
multiple tasks. LL has the key characteristics of consistent knowledge
accumulation across several tasks and reusing it while learning a new task
which makes it closely related to meta learning. However, the system
architecture of LL [4] is fundamentally different from the other learning
paradigms; it may require an ensemble of many learning algorithms and various
knowledge representation methods. As for the evaluation of LL, many tasks and
datasets are required to review the algorithm’s performance, while learning
the sequence of tasks is of great significance On the grounds of these
differences, LL is excluded from this study.

###  I-D Types of knowledge transfer

The information sharing between different tasks can result in two types of
knowledge transfers: positive and negative transfer [5, 6]. Positive transfer
is when the information shared between the tasks aids in improving the
performance of the tasks, while the negative transfer is when the performance
of the tasks suffers due to information flow within the tasks. Negative
transfer is also known as destructive interference; it occurs because possibly
even related tasks may have contradictory requirements, and when these are in
a knowledge sharing setting, improving the performance of one task may hinder
the performance of another. Negative transfer is a significant issue in
multiple tasks learning concepts like transfer learning, MTL, etc. Therefore,
to support positive transfer, it is important for an algorithm to fulfill two
complementary objectives: retain task-specific knowledge, and better
generalization across tasks. Accomplishing such goals is not at all that
straightforward; consequently, it is an active area of research.

###  I-E Contribution

The following are the important contributions of this work.

  * •

An outline of the three learning algorithms, i.e., transfer learning, MTL, and
meta learning, together with a comparative study of these learning paradigms
focusing on their strengths and weakness.

  * •

A detailed survey of the current research in multi-task meta learning, meta
transfer learning, and multi-task transfer learning. This article highlights
instances in the literature where because of abuse of terminology, these
algorithms are often misunderstood as similar or derivatives of each other.

  * •

This article proposes a novel generic learning network, an approach to combine
these three learning paradigms in such a manner that gives the flexibility to
utilize either all the three learning algorithms or combinations of two or
just one of them as required.

  * •

This work also presents the open questions in research related to knowledge
sharing in multiple tasks learning arrangement.

###  I-F Notations used in this paper

Before moving to the next section it is important to know some of the
notations used in this article. These are:

  * •

Variables in calligraphic ($\mathcal{D,X,Y}$) - sample space

  * •

Variables in italic ($D,x,y$) - one sample of the space

  * •

$p_{i}$ \- joint distribution of $i$ over the space

  * •

$p(A)$ \- marginal probability distribution of A

  * •

$p(A|B)$ \- conditional probability distribution of event A given B

###  I-G Structure

The structure of the article is as follows: Sec. II gives the definitions and
fundamentals of the learning paradigms along with supporting the explanation
with suitable examples and also a comparative study. A detailed literature
study of the learning algorithms when used together is presented in the Sec.
III. While Sec. IV explains the reason of fusing these algorithms and also
introduces a global learning network. At last Sec. V concludes the article
with some open questions and directions for future research.

##  II Learning paradigms

This section will elaborate on the three learning paradigms this article is
keying on, particularly transfer learning, MTL, and meta learning. Along with
an explicit definition, a comparison of these algorithms is also detailed
below. But before this, an introduction to conventional supervised learning
and its limitations, which direct to the reason for using information sharing
algorithm, is also discussed.

###  II-A Supervised Learning

The objective of ML algorithms is to learn from experiences. Here learning
refers to improving by experience at a particular task [7]. In supervised
learning, which is one of the primary categories of ML algorithms, these
experiences happen to be labeled datasets, wherein the algorithm is fed with
the inputs and the expected output, and the goal is to learn a mapping from
the input to the output. This article discusses the various knowledge
transferring learning algorithms in the supervised sense only.

#### Definitions

Consider a domain $\mathscr{D}$, which is a combination of the input feature
space $\mathcal{X}$, output space $\mathcal{Y}$, and an associated probability
distribution $p(x,y)$, i.e., $\mathscr{D}$ =
{$\mathcal{X},\mathcal{Y},p(x,y)$}. Here $p(x,y)$ represent the joint
probability distribution over the feature-label space, and can be decomposed
as $p(x,y)=p(x)p(y|x)$ or $p(x,y)=p(y)p(x|y)$ , where $p(.)$ is the marginal
distribution and $p(.|.)$ is the conditional distribution. The joint
probability is used because the learning algorithms implicitly assume that
each sample $(x_{i},y_{i})$ is drawn from a joint distribution. In supervised
setting the dataset $D$ consists of input-output pairs, i.e.,
$D=\\{(x_{i},y_{i})_{i=1}^{n}\\}$, where $x_{i}$ is a m-dimensional feature
vector and $y_{i}$ is the response or output variable which can be either a
categorical variable or a real-valued scalar and $n$ is the number of labelled
samples.

In supervised learning, for a specific domain $\mathscr{D}$ =
{$\mathcal{X},\mathcal{Y},p(x,y)$}, the aim is to learn a predictive function
$\mathcal{F_{\theta}}(x)$ from the training data {$(x_{i},y_{i})_{i=1}^{n}$},
where $x_{i}\in\mathcal{X}$ and $y_{i}\in\mathcal{Y}$, and $\theta$ are the
function parameters. From a probabilistic point of view,
$\mathcal{F_{\theta}}(x)$ can also be considered as the conditional
probability distribution $p(y|x)$. In case of a classification task, when the
response variable $y$ is categorical, i.e., $y\in\\{1,.....C\\}$ and C is the
number of categories, supervised learning aims to predict;

$\hat{y}=\mathcal{F_{\theta}}(x)=\operatorname*{arg\,max}_{c\in[\\![1,C]\\!]}~{}~{}p(y=c|x)$
(1)

This is known as Maximum A Posteriori (MAP) i.e., the most probable class
label.

In conventional supervised learning, it is assumed to solve one task using one
dataset, and so, the dataset is divided into development and test set. Here
the test set represents the future unseen data, but both the sets are drawn
from the same distribution (or the same domain), and both are labeled. But in
reality, such a dataset is difficult to find, and therefore the supervised
learning algorithm fails to generalize. For example, a model is trained using
the handwritten digits dataset but is further used for inference on license
plate images to detect the numbers automatically. So, for the model to perform
on the data from another distribution, it needs to be trained again from the
beginning using the new data. This loss of previous learning while learning
new information is often referred to as catastrophic forgetting [8]. It is the
biggest shortcoming in traditional supervised learning, which is said to be
overcome in learning algorithms like transfer learning, MTL, meta learning,
lifelong learning, etc, by sharing information between tasks.

###  II-B Transfer Learning

Most ML models work under the assumption that the training and testing data
are drawn from the same distribution (domain).

If the distribution is changed, then the model needs to be trained again from
scratch. Transfer learning helps to overcome this issue. Transfer learning
refers to exploiting what has already been learned in one setting to improve
the learning in another setting [9]. Information transfer happens from the
source domain (transfers knowledge to other tasks) to the target domain (use
knowledge from other tasks).

Assuming only one source domain $\mathscr{D}_{s}$ =
{$\mathcal{X}_{s},\mathcal{Y}_{s},p_{s}$} and target domain
$\mathscr{D}_{t}=\\{\mathcal{X}_{t},\mathcal{Y}_{t},p_{t}\\}$ with $p_{s}$ and
$p_{t}$ the joint distributions of the source and target data, respectively.
Let source data be $D_{s}$ = { $(x_{s_{i}},y_{s_{i}})_{i=1}^{n_{s}}$}, where
$x_{s_{i}}\in\mathcal{X}_{s}$ and $y_{s_{i}}\in\mathcal{Y}_{s}$ are the data
instances and associated labels respectively. Similarly let the target data be
$D_{t}$ = { $(x_{t_{i}},y_{t_{i}})_{i=1}^{n_{t}}$}, where
$x_{t_{i}}\in\mathcal{X}_{t}$ and $y_{t_{i}}\in\mathcal{Y}_{t}$. Usually in
transfer learning it is considered that $0\leq n_{t}<<n_{s}$ i.e., the source
data is much larger than the target data.

Definition of transfer learning, as discussed in [10]:  For a given source
domain $\mathscr{D}_{s}$, and target domain
$\mathscr{D}_{t}(\neq\mathscr{D}_{s})$, transfer learning aims to improve the
learning of the target prediction function $\mathcal{F_{\theta}}_{t}(x_{t})$
in domain $\mathscr{D}_{t}$ using the knowledge gained by performing a task in
domain $\mathscr{D}_{s}$. The task in the source domain is to learn the source
predictive function $\mathcal{F_{\theta}}_{s}(x_{s})$ from the training data
$D_{s}$. Also the dataset $D_{s}$ is not accessed during transfer.

In general, two domains are different if they differ in at least one of their
components, i.e., input space, label space, or the probability density
function. Based on these conditions, transfer learning can be classified as
follows;

  * •

Inductive transfer learning - in this $\mathcal{Y}_{s}\neq\mathcal{Y}_{t}$,
and it doesn’t matter whether the source and target input feature space are
the same or different. Here there are two possibilities;

    1. 1.

source domain has a lot of labeled data; this is similar to Multi-task
learning [11] (discussed in section II-C). But there is a difference,
inductive transfer learning shares the knowledge of the source task to improve
the learning of only the target task, while multi-task learning jointly learns
both the task and attempts to improve the performance of both.

    2. 2.

source domain has no labeled data; this converges to Self-taught learning.
[12]. In the case of unlabelled data, the source task is to learn a good
feature representation and use these learned feature representations to
accomplish the target task.

  * •

Transductive transfer learning [13] \- in this
$\mathcal{Y}_{s}=\mathcal{Y}_{t}$ but $\mathscr{X}_{s}\neq\mathscr{X}_{t}$, as
well as it is assumed that the target domain has unlabelled data at the time
of training. There can further be two cases-

    1. 1.

$\mathcal{X}_{s}\neq\mathcal{X}_{t}$ i.e., feature space is different

    2. 2.

$\mathcal{X}_{s}=\mathcal{X}_{t}$, but $p_{s}(x)\neq p_{t}(x)$ i.e., the
marginal probability distribution is different for both the domains. This case
is related to domain adaptation [14]. Furthermore, covariance shift is a
condition in domain adaptation when along with $p_{s}(x)\neq p_{t}(x)$, the
conditional distributions are constant $p_{s}(y|x)=p_{t}(y|x)$. And data drift
or concept shift is the case when $p_{s}(x)=p_{t}(x)$, while $p_{s}(y|x)\neq
p_{t}(y|x)$. These are the types of domain shifts explained in [14].

  * •

Unsupervised transfer learning - there is no labeled data in both the domains
and the aim is to solve unsupervised learning tasks in the target domain, for
example, clustering, dimension reduction etc. using a large amount of data in
the source domain. Self-taught clustering [15] is one such instance of
unsupervised transfer learning.

There are two primary approaches for transferring information between source
and target tasks. The first is feature extraction; it uses the source model
architecture and model parameters to extract good data features from the
target domain to accomplish the target task. The second approach is fine-
tuning; similar to the first, the source model shares its parameters and
architecture with the target task, but the source parameters only serve as
initialization to the target network, and further training is required using
the target domain data. Very often, only the higher (last) layers of the
network are modified and are trained, i.e., the parameters are altered to
adapt to the new data, while the parameters of the lower (initial) layers are
frozen, i.e., same as that from the source model. Therefore, the pre-trained
source model is said to be fine-tuned according to the target data and fine-
tuning also helps to improve generalization. Usually, transfer of learning,
along with fine-tuning leads to better performance than training the target
task from scratch.

#### Example of transfer learning

Consider a classification task of identifying cats from dogs as the target
task, with limited training data. And a source task, say of identifying the
breeds of dogs having a larger dataset than the target task. Since the
conventional supervised machine learning models are data-hungry, it won’t be
easy to accomplish the target task with less data, assuming that the source
task with larger data than the target performs sufficiently well. In this
scenario, knowledge transfer from the source to target, if performed
effectively, may enhance the performance of the target task. A common approach
to knowledge transfer is using the model as well as the parameters of the
source to extract features of the target data and appending a classifier,
which will learn to classify the images of dogs and cats.

###  II-C Multi-task Learning (MTL)

Figure 1: An illustration of the PAD-Net architecture proposed in [16], there
are four primary tasks of monocular depth estimation, semantic segmentation,
finding surface normal and contour prediction and further the outputs are
integrated for the prediction of two output tasks of depth estimation and
scene parsing. Here Loss 1 - Loss 6 represent the optimization losses for the
various tasks. All the images in the figure are from the dataset Taskonomy
[17]

MTL as explained in [11], is an inductive transfer approach that exploits the
domain information in the training data of related tasks as inductive bias to
improve the generalization of all the tasks. The underlying theory of MTL is,
the information gained while learning one task can help the other task to
learn better. In MTL all the tasks are trained (or learned) together, using
the shared representations.

Definition of MTL, as discussed in [4]:  Consider, $\mathcal{T}$ is an
ensemble of N related but not identical tasks i.e., $\mathcal{T}$ =
{$T_{1},T_{2}....T_{N}$} and each task $T_{i}\in\mathcal{T}$ has training data
$D^{Tr}_{i}$. MTL aims to jointly learn these multiple tasks
{$T_{1},T_{2}....T_{N}$} in order to maximize the performance of all the N
tasks.

The objective of MTL is to learn optimal parameters $\theta^{*}$, so as to
minimize the combined loss $\mathcal{L}$ across each task. It can be expressed
as:

$\theta^{*}=\min_{\theta\in\Theta=\cup_{i=1}^{n}\Theta_{i}}\sum_{i=1}^{N}\mathcal{L}_{i}(\theta_{i},D^{Tr}_{i})$
(2)

As discussed in section I-D, in multi-task arrangement despite the tasks being
related negative transfer can exist. It depends on the information sharing
between the tasks and can be controlled by better MTL architecture designs and
task relationship learning. In recent years there has been significant
research on creating shared architectures for MTL, the article [5] gives a
survey of common deep MTL architectures used in the computer vision, natural
language processing, reinforcement learning, etc. The deep MTL architectures
can be divided into two types of modules [9]:

Generic modules

these are shared across all tasks and the parameters are benefited from data
of all tasks (correspond to $\Theta_{g}=\bigcap_{i}\Theta_{i}$);

Task-specific modules

these are dedicated modules for each task and the parameters are benefited
from the instances of the particular task (corresponds to
$\Theta_{i}-\Theta_{g}$).

Note, in this context modules are combinations of layers of neural network or
the Convolutional Neural Network (CNN). As a result of these modules, the
parameters are divided as shared parameters and task specific parameters. The
best performance of MTL models is achieved only when there is balanced
sharing, because too much sharing can cause negative transfer and too little
sharing can inhibit the effective leveraging of information between tasks.
Therefore, in order to create an effective MTL architecture it is important to
analyze how to combine the shared modules (layers) and task specific modules
and what portion of model’s parameters will be shared between tasks. In
conventional MTL, the parameter sharing approach is classified as [18]-

  * •

hard parameter sharing \- model weights (or parameters) are shared between
multiple tasks and each weight is modified to minimize multiple loss
functions. This can be achieved as a result of MTL architectures.

  * •

soft parameter sharing \- tasks have separate weights and distance between the
weights of models for every task added to the joint loss function is
minimized, similar to introducing a regularization term in the combined loss.
Therefore there is no sharing of parameters explicitly, rather models of
different tasks are forced to have similar parameters. This is often
introduced when there is negative transfer between the task and need to share
less. The various optimization techniques help to achieve soft parameter
sharing.

#### Example of MTL

Consider a dataset with images of natural scenes. Every image is labelled for
a number of tasks like scene classification, semantic segmentation, instance
segmentation and pixel wise depth values for depth estimation. For
conventional machine learning all these are four different tasks, and it is
required to train different models for each of the problems. MTL, however,
exploits the facts that all the tasks are related and use the same input
image. Therefore, they can be trained together in a MTL architecture, so that
they share the representations between tasks and encourage the model to
generalize better than single task learning. A similar architecture is
proposed by [16] in Fig .1. Usually in MTL, it is also believed that there is
one main task and rest are auxiliary tasks which only contribute to improve
the performance of the main task. In this article, there is no such assumption
of main and auxiliary task. Furthermore in the above example, additional tasks
like learning image compression and decompression, colorization of gray-scale
images, denoising of images, etc can also be integrated in the architecture.

###  II-D Meta Learning

Meta Learning better known as “Learning to learn” [19], is a learning paradigm
that aims to improve the learning of new tasks with lesser data and
computation, by exploiting the experience gained over multiple training
episodes for various tasks. The conventional ML algorithm employs multiple
data instances for better model predictions, while meta learning uses multiple
learning instances to improve the performance of a learning algorithm.

Meta learning can be defined as:  Assuming a set of M source tasks
$\mathcal{T}_{s}$, sampled from a distribution $p(\mathcal{T})$, with source
datasets $D_{s}$. And Q target tasks $\mathcal{T}_{t}$ with target datasets
$D_{t}$. Meta learning is to train a model on the M source tasks using data
$D_{s}$, such that it generalizes well on a new unseen target task, which
leads to: • Computational efficiency - faster training of the target task
using data $D_{t}^{train}$ • Data efficiency - good training with less target
data instances • Effective knowledge transfer - good performance on the target
test data $D_{t}^{test}$

Let a task $\mathcal{T}$ be defined as, $\mathcal{T}=\\{\mathcal{L},D\\}$,
where $D=\\{(x_{1},y_{1}),..(x_{N},y_{N})\\}$ is training dataset and
$\mathcal{L}$ is the loss function. For a single task conventional supervised
ML algorithm, the aim of learning a model $\hat{y}=f_{\theta}(x)$
parameterized by $\theta$, is accomplished by solving:

$\theta^{*}(\mathcal{T})=\mathrm{arg}\min_{\theta}\mathcal{L}(\mathcal{D};\theta,\phi)$
(3)

Here $\phi$ denotes the “how to learn” assumptions [20], for example the
optimizer for $\theta$, choice of hyper-parameters, etc. A pre-specified
$\phi$ can help to achieve significant performance as compared to the case
when it is absent.

Meta learning involves learning a generic algorithm by training over several
tasks, that enables each new task to learn better than the previous.
Therefore,for a distribution of task $p(\mathcal{T})$, meta learning becomes:

$\min_{\phi}\displaystyle\mathop{\mathbb{E}}_{\mathcal{T}\sim
p(\mathcal{T})}\mathcal{L}(D;\theta^{\star}(\mathcal{T}),\phi)$ (4)

Where $\mathcal{L}(D;\theta^{\star}(\mathcal{T}),\phi)$ evaluates the model’s
performance trained using $\phi$ on task $\mathcal{T}$.
$\theta^{\star}(\mathcal{T})$ is the optimal parameter learnt for task
$\mathcal{T}$. Here the parameter $\phi$ is the meta knowledge or across task
knowledge [20].

To solve the meta learning problem, assuming M source tasks $\mathcal{T}_{s}$
sampled from $p(\mathcal{T})$, having dataset
$D_{s}=\\{(D_{s}^{train},D_{s}^{val})^{(1)},...,(D_{s}^{train},D_{s}^{val})^{(M)}\\}$,
with train (support) and validation (query) sets. Also, Q target tasks with
data
$D_{t}=\\{(D_{t}^{train},D_{t}^{test})^{(1)},...,(D_{t}^{train},D_{t}^{test})^{(Q)}\\}$,
i.e., each task with train and test set.

The meta learning objective in eq.[4] is obtained in two stages,

  * •

Meta training - The meta training stage can be posed as a bi-level
optimization problem, where one optimization contains another optimization as
a constraint. Here an inner learning algorithm solves a task, defined by
dataset $D_{s}^{train(i)}$ and objective function $\mathcal{L}^{task}$. While
in meta training an outer (meta) algorithm updates the inner algorithm in
order to improve the outer objective $\mathcal{L}^{meta}$. So, meta training
can be formulated as:
(outer Objective)

$\phi^{*}=\mathrm{arg}\min_{\phi}\mathop{\mathbb{E}}_{\mathcal{T}_{s}\sim
p(\mathcal{T})}\mathcal{L}^{meta}~{}(\theta^{*(i)}(\phi),\phi,D_{s}^{val(i)})$
(5)

Where,
(inner objective)

$\theta^{*(i)}(\phi)=\mathrm{arg}\min_{\theta}~{}\mathcal{L}^{task}~{}(\theta,\phi,D_{s}^{train(i)})$
(6)

$\phi^{*}$ has all the information of source tasks (or data) to solve new
tasks. So, the inner objective corresponds to task specific learning, while
the outer objective corresponds to multiple task learning.

  * •

Meta testing - The meta testing stage is often referred as adaptation stage.
This stage uses the meta knowledge or meta parameters ($\phi^{*}$), to train
the model on unseen target tasks. For an $\mathrm{i^{th}}$ target task, meta
testing involves training on the $D_{t}^{train(i)}$ to minimize the loss
$\mathcal{L}^{test}$ and evaluating the performance on $D_{t}^{test(i)}$, for
optimal parameter $\theta^{*(i)}$ given by:

$\theta^{*(i)}=\mathrm{arg}\min_{\theta}~{}\mathcal{L}^{test}(\theta,\phi_{\star},D_{t}^{train(i)})$
(7)

However, MTL can be also be seen as a special case of meta learning if
$\theta=\phi$ in the meta training phase, as there will be only one
optimization objective and multiple tasks for training. Despite this
similarity, there are many differences that persists between meta learning and
MTL which are discussed in Sec. II-E.

Figure 2: An example of meta learning illustrating 4 shot 2 class image
classification

#### Example of meta learning

Fig. 2, shows an example of meta learning, where the source tasks are: Task 1-
classification between cats and dogs, and Task 2- classifying flowers from
bikes. The target task is to classify between images of fish and bird which
the model has not seen during the meta training phase. This is a classic
example of 4 shot 2 class image classification, where the objective is to
learn to identify the categories only by 4 images and every tasks has 2
classes or labels. So, meta learning enables the model to learn fast in a few
instances of dogs and otters images during meta testing by utilizing the meta
knowledge gained from meta training of the source tasks. The number of source
tasks can be increased in order to achieve better generalization.

###  II-E Comparisons

Figure 3: A comparative representation of the learning paradigms

From the above sections which details about the learning paradigms, it is
clear that all the three transfer knowledge between tasks, the difference lies
in how, when and what knowledge is shared between the tasks. This section
presents a comparative view of the three learning paradigms. It might not
cover all the differences but focuses on the significant dissimilarities and
also similarities.

#### Transfer learning and MTL

The tasks involved in transfer learning and MTL can be heterogeneous i.e., the
nature of the tasks can be different like classification, segmentation,
regression, etc. Both of these learning paradigms can share features and
parameters between the tasks;. Because of the inductive transfer approach MTL
is also considered as a type of transfer learning [10], but they are very
different. MTL learns many tasks together while it is not the case in transfer
learning, where first the source task is trained and the information is
transferred for learning the target task. In other words, training of tasks in
transfer learning is sequential while in MTL the tasks are usually trained
simultaneously (or jointly). In MTL the goal is to generalize the performance
of all the tasks, while in transfer learning the focus is only on
generalization of the target domain. Transfer learning is a logical
explanation to multi-task learning but not vice versa.

#### Meta learning and Transfer learning

In both these learning paradigms, the target tasks are trained after the
successful training and testing of the source task i.e., sequentially. The aim
of both the algorithms is to achieve better generalization on the target task.
They key difference between both lies in the optimization algorithm. In
transfer learning there is no use of meta objective while deriving priors
(parameters) from learning the source task. In contrast, in meta learning the
priors are extracted as a result of the outer optimization and these are
evaluated while learning a new task. Only model parameters are shared in
transfer learning, while meta learning transfers a variety of meta
representations. Also in meta learning the meta training (source) and meta
testing (target) conditions must match e.g., if the source tasks are binary
classification problems, then the target task necessarily should be binary
classification. On the contrary, transfer learning is possible on diverse
tasks.

#### Meta learning and Multi-task learning

Similar to transfer learning, MTL has single level optimization i.e., no meta
objective. MTL aims to solve fixed number of known tasks, whereas meta
learning deals with solving unseen tasks. Meta learning as discussed earlier
only works with homogeneous tasks and same modality of inputs during meta
training and testing, while MTL can handle heterogeneous tasks and also
variety of inputs. In meta learning, the source tasks are trained sequentially
for many iterations during meta training and a new unseen task is learned at
the time of meta testing, on the contrary in MTL the training and testing is
respectively performed for all the tasks jointly. Therefore, the knowledge is
shared between the tasks at the time of training in MTL, but for meta learning
the model saves the prior knowledge from the source tasks to be used during
meta testing the target tasks.

##  III Ensemble of the learning paradigms

TABLE I: Strength and Weaknesses Paradigms | Strengths | Weakness
---|---|---
Multi-task | \- Can handle | \- Requires a huge dataset
learning | heterogeneous | for training
| tasks and | \- On adding a new task
| multimodal inputs | have to retrain the network,
| \- Aim is to enhance | as it may require
| performance of | architectural changes
| all tasks |
Meta | \- Easy addition of new | \- Works for homogeneous
learning | task | tasks only, as source and
| \- Less training data | target tasks must match
| required for a new | \- Only focuses on
| unseen task | enhancing performance of
| \- Robust generalization | target task
| across tasks |
| \- Meta learning |
| objective (bi-level |
| optimization) |
Transfer | \- Less training data | \- Pretrained models tend
learning | required for target task | to overfit on target task
| \- Good for feature | \- Complex source models
| extraction | with millions of parameters,
|  | not always required for
|  | target tasks
|  | \- Only focuses on
|  | enhancing performance of
|  | target task

The strength and weaknesses of the learning paradigms detailed in Sec. II are
summarized in Tab. I. Each of the algorithm has some drawbacks, which are
overcome by another learning algorithm. Like meta learning supports
homogeneous tasks and unimodal input but MTL may integrate heterogeneous tasks
and multimodal inputs. Similarly, introducing a new task is much simpler in
meta learning and transfer learning as compared to MTL. For these reasons, in
the literature these algorithms are coupled together, to utilize their best
features. This section will discuss the research performed in various
ensembles of these algorithms.

###  III-A Research in Multi-task meta learning or meta multi-task learning

Insights from both, meta learning and multitask learning can be fused to
achieve the best of both worlds, i.e., efficient training of multiple
heterogeneous tasks, a feature of MTL, and quickly adapting new tasks, a
feature of meta learning. Thereby offering a fast, efficient, and adaptive
learning mechanism. In the past there have been many studies which demonstrate
employing both of these learning paradigms together. These are discussed here
according to the field of application.

Figure 4: A multi-modal meta MTL framework proposed in [21] for rumor
detection in social media.

The article [22] proposes a function level information sharing scheme for MTL,
which employs a meta Long Short-Term Memory (LSTM) i.e., shared across all
tasks and a basic LSTM which is task specific and the parameters are generated
depending on the current factors by the meta LSTM. Here the meta LSTM is
considered as the prior knowledge while the basic LSTM is observed as the
posterior knowledge It focuses on two Natural Language Processing (NLP) tasks
i.e., text classification and sequence tagging. Overall the architecture they
propose is a multitask architecture and the two level optimization scheme is
from meta learning. But, the performance is not evaluated for an unseen task.
Nevertheless, they show that the performance of the proposed network is much
better than applying MTL and also single task learning. Because adding meta
learning to MTL, leads to meta knowledge transfer which is sharing semantic
composition function across tasks. In neural representation learning of
sequences of text, the semantic composition function are of extreme
significance.

Another work [21] introduces a multimodal meta multitask learning approach for
rumor detection and stance detection in social media, illustrated in Fig. 4.
The multimodal inputs here are text and images, which are converted into
embeddings and fused together to be fed to the model. This article discusses
the feature level, meta level and task level challenges related to the problem
statement and therefore, suggests to share the higher meta knowledge between
tasks i.e., sharing features in the initial layers of the model and further no
sharing in the task specific lower layers. Here the feature sharing between
the tasks is considered as meta knowledge and therefore the title of the
article mentions meta multi-task but actually there is no meta learning
applied, as it neither solves the bi-level meta objective nor evaluates the
performance on unseen tasks.

The Parameters Read-Write Networks (PRaWNs) are introduced in [23], that
presents a novel concept of communication between the tasks in a multi-task
setting. The idea here is to allow various tasks to pass the gradients
explicitly, thereby restricting the tasks to constantly update parameters.
Therefore the gradients are passed between tasks pairwise and list-wise, where
list-wise gradients account for task relatedness. The features learnt in the
shared space are usually entangled, and this proposed model helps to untangle
features in different domains, which as a result, for the given use-case
avoids escape of private information in the shared space. [23] investigates
the proposed network for text classification, sequence tagging and image
aesthetic assessment tasks for in-task (same dataset) and out-of-task (new
dataset) settings. It is concluded that for out-of-task setting, the
introduced gradient passing mechanism show better generalization to unseen
tasks.

In [24], the authors offer an approach to learn interactions between languages
and tasks by employing meta learning in a multi-task scenario. Question
answering, parts of speech tagging, name entity recognition, paraphrase
identification and natural language inference are the multitasks in
consideration for this work. They exploit the fact that these tasks can be
benefited from each other and also cross lingual embeddings in multiple
language learning can be helpful in case of languages with limited data.
Therefore the meta information in this work is the task relatedness plus the
language relatedness. They also propose sampling techniques like heuristic and
parameterized sampling, to be integrated in meta learning to improve the
performance of the model. For languages which has limited data resources, this
approach can be very useful as it performs significantly well on the zero-shot
new target languages.

Personalized dialogue generation is improved by the use of Meta and MTL in
[25], which seeks to over come the issue of large dataset for every person and
also including pre-defined persona information. In this work an auxiliary task
of persona reconstruction is added only at the meta training stage, with the
aim to gather the persona information as meta knowledge for the dialogue
generation task. Thereby making the model able to generate dialogues for new
users at the time of meta testing. This introduces two frameworks, Multi-Task
Meta-Learning (MTML) which combines losses from both the tasks and Alternating
Multi-Task Meta-Learning (AMTML) which operates alternatively on the tasks of
generation and persona reconstruction. Another similar work [26] aims at text
style transfer with limited data, which is basically paraphrasing text from
one writing style to another. It employs the Model Agnostic Meta Learning
(MAML) [27] algorithm for few shot text style transfer, and a task corresponds
to a pair of style, so viewing style transfer between each pair as a domain
specific task, hence employing MTL. This enables to transfer (to and from)
writing styles which have small training data and also which the model hasn’t
seen before due to the meta learning framework. The proposed methodology
outperforms the state-of-art results in text style transfer.

The article [28] combines concepts from meta learning, MTL and visual
attention for image classification, and estimation of depth, vanishing point
and surface normal form a single input image. The flexible attention mechanism
is used for adapting the network for a particular task and the features in the
generic modules (or backbone) of the network are weighted according to the
importance to the specific task. The task specific modules (or layers) are
adapted to unseen tasks by employing the Almost No Inner-Loop MAML (ANI-MAML)
[29] training procedure, hence introducing the role of meta learning in this
work. The attention mechanism along with meta learning made it possible for
the tasks heads to learn adapt new unseen tasks in lesser data instances.
Along with learning the task specific representations, the primary issue was
to learn task-invariant representations, the MTL architecture solves this by
providing inductive bias on selected features as directed by the attention
procedure.

[30] introduces meta learning and MTL for speech emotion recognition. In this
work, the emotion classification for each user is considered as the auxiliary
tasks which are used for meta training, also known as the multi-train stage.
Next is the knowledge transfer stage wherein the meta information from the
training stage is used to train and evaluate the model for a new user. So,
this article aims to model the relationship between the auxiliary tasks i.e.,
users and transfer knowledge to the target task. This is very much the concept
of meta learning as discussed in Sec. 2, this work considers the multiple
tasks during training as MTL and therefore mentions meta multi-task learning
in the title.

The article [31], solves the challenge of data shortage and disease diversity
in mortality prediction of rare diseases by employing a multi-task
architecture along with the meta learning optimization scheme MAML [27] . Here
the multiple tasks are detecting the temporal occurrences of rare diseases and
the input is multi-modal i.e., text, images, signals, etc. A learning methods
Ada-SiT (Adaptation to Similar Task) is introduced in this work, in which the
task similarity is measured during meta training and this is used to share
initialization for faster adaptation of new tasks.

All articles discussed above are the ones which appear (and are relevant) in
the ‘in-title’ search for meta learning + MTL on google scholar. Some [22, 30,
21, 23, 25] refer the combination as Meta Multi-task Learning which focuses on
improving multi-task learning by employing meta learning to gather meta
knowledge (e.g., task relationships) to transfer it to new task. while a few
[25, 26, 28] mention Multi-task Meta Learning that focus on upgrading meta
learning by introducing multi-task learning for allowing training with
heterogeneous tasks as well as efficient training mechanism for better
learning of features. In general, there is no logical explanation of the
taxonomy used for employing the two algorithms together. Moreover they can be
used interchangeably as the purpose of integrating these learning mechanisms
is to leverage the qualities of both meta learning and MTL.

There are also a few articles [32, 33, 34, 35, 36, 37] that by abuse of
terminology refer to the training of multiple tasks in meta training stage as
MTL, thereby considering the work as meta multi-task learning. Also, a few
articles [38, 39, 40, 41, 42, 43, 44, 45] in the google scholar search for
meta multi-task learning (or multi-task meta learning) simply mention these
terms in related work or future work sections. In the similar search, a
handful of articles [46, 47, 48] focusing on meta multi-task reinforcement
learning are also present. But these are not discussed here as it is beyond
the scope of this work.

###  III-B Research in multi-task transfer learning

Transfer learning and MTL usually differ in how and when the information is
shared between tasks. Yet, they can be used together in the following ways,

  1. 1.

transfer learning enables to extract features for tasks by employing models
pre-trained for some other related tasks on a large dataset, which can further
be used in a multi-task setting. This can be seen as when there are multiple
related target tasks and thereby employing MTL to solve them, and source task
is conventional supervised learning.

  2. 2.

Another way of fusing these algorithms is when there are multiple source tasks
such that MTL can be applied and the knowledge is shared with the target task
in terms of model parameters, features, etc.

In the past, there two learning paradigms are used together a number of times.
The article [49] aims to predict the pharmacokinetic parameters by learning a
model for quantitative structural activity relationships of drugs. The four
parameters i.e., oral bio-availability, plasma protein binding rate, apparent
volume of distribution at steady-state, and elimination half-life ought to be
estimated in this work and these four parameters are considered as multiple
tasks. To accomplish this, it follows the first approach discussed above of
feature extraction and multiple tasks in target domain. At first, a pretrained
model is learned on the large bioactivity data set, the knowledge from which
is further used in the multi-task DeepPharm model proposed in the article, as
depicted in Fig. 5. The integrated multi-task and transfer learning approach
proved to enhance the generalization of the model as compared to the
conventional models, as well as overcome the issue of lack of sufficient and
high quality data in Absorption, Distribution, Metabolism, and Excretion
(ADME) evaluation. . Such a well generalized model may be helpful to perform
the ADME calculations on the new drug structures using the DeepPharm model,
i.e., doing inference only, as there is no need of training the model again on
a different data. This article opens a totally new dimension of research in
the field of drug discovery and development using combining DL algorithms.

Figure 5: Illustration of the integration of transfer learning and MTL [49]

An example of the second type of fusion mentioned above i.e., when the source
task is MTL from which the learnt information is transferred to a target task,
in illustrated in [50]. The aim of this article is fault diagnosis of rolling
element bearings under uncertain working conditions. Firstly, MTL is enforced
to find out the speed and health type of the machines (in general) by using
the bi-spectrum based analysis of the vibration signals as inputs. Further,
using the proposed MTL-CNN as the source task, and identifying the bearing
faults under severe conditions as target task, transfer learning is used to
enhance the classification performance. Since the faulty bearing data in
extreme conditions is usually less (as this is a type of anomaly), transfer
learning technique turns out to be very useful. Employing the pre-trained
model for extraction features in the target dataset in a multi-task setting
lead to good performance on the unseen target data.

In [51], pretrained transfer learning models like BERT [52], ULMFiT [53] and
GPT-2 [54], for fake news detection. Along with this it also includes an
auxiliary language modelling task to adapt to the writing style of the
downstream task of fake new detection. Therefore introducing multi-task fine-
tuning in this work by combining losses from both the tasks. In a similar work
[55], transfer learning and MTL are used together for Named Entity Recognition
(NER) on Chinese Electronic Medical Records (EMR). It trains a bi-directional
LSTM in the general (source) domain and further use the acquired knowledge
from the source domain to improve the performance of NER in chinese EMR i.e.,
target domain. Parts-of-Speech (POS) tagging and NER are the two tasks in the
target domain which are trained alternatively, so that knowledge form one task
may enhance the knowledge gained by the other task. Since the aim of this work
is NER, the POS is treated as an auxiliary task that aids in better learning
of chinese NER. On similar grounds, [56] intends to classify the multi-word
expressions, with the help of two auxiliary tasks of dependency arcs and
labels. To attain this, it utilizes knowledge shared by the pretrained model
for POS and dependency parse tags for two different languages. This is because
the target language is has limited resource and cross lingual transfer
learning helps to overcome this issue.

Multi-task transfer learning is also exercised together in [57] for neural
decoding, which is decoding the brain activity for reconstructing the visual
information. To achieve this, first the fMRI voxel featues of the brain are
decoded into CNN features by using Structured Multi-output Regression (SMR) (
i.e., Voxel2Unit), further the predicted CNN features are converted to an
image using introspective conditional generation (i.e., Unit2Pixel). Multiple
CNN fetaures are decoded from the FMRI data using SMR, and every single output
prediction is considered as a task, thereby applying MTL for Voxel2Unit
process. For the Unit2Pixel process, a pretrained CNN called AlexNet [58],
trained on large image data ImageNet[59] is used for image reconstruction as a
part of deep generative models, particularly a combination of Variational
Auto-Encoders (VAEs) [60] and Generative Adversarial Nets (GANs) [61].

The article [62], analyzes the characters of a digital Japanese comics named
Manga, using Bi-modal inputs i.e., the graphics and text information. It uses
pretrained networks like BERT [52] and ResNet [63] for text and visual feature
extractor respectively, in other words to get the feature embeddings so as to
combine data from both the modes. Further use these embeddings in a multi-task
architecture for character retrieval, identification and clustering tasks.
These three tasks are independent tasks, each of the tasks can be accomplished
training three different models using the same images and text as inputs (uni-
modal inputs can also be used). But, since they share common multi-modal
inputs (image and text), they are suitable for MTL and certainly sharing of
parameters between tasks improves the performance of all the tasks as compared
to when they are trained isolated. They strengths of MTL i.e., handling multi-
modal inputs and multiple tasks and that of transfer learning i.e., good
feature representations are cherished in this work.

The article [64], exploits a few-shot Dirichlet-Net based MTL for hyper-
spectral image classification. An encoder-decoder network with the aim to
reconstruct hyper-spectral images is used, and the representations are shared
using the encoder in a classifier network. This ensemble of encoder-decoder
along with classifier is termed as MTL in this article. The encoder takes as
input two images of different domains and extracts both representative and
discriminative vectors representations from both the domains, these encoder
network parameters are shared with the classification network for extracting
features of the input image patches to predict output labels. The important
contribution of this article is extraction of shared representations from the
objects in different domains by applying transfer learning.

Another interesting application is [65], that aims to estimate the virus-human
protein interactions, by pre-training a source model UniRep [66] to produce
the features or protein embeddings. Due to the scarcity of training data for
virus-human protein interaction, the article first trains a network on
powerful statistical protein representations i.e., source task. And the target
task performs MTL by extracting the embeddings using the source network, to
find human protein-protein interactions (PPI) and human virus PPI. Likewise,
[67] uses VGGNet [68] for pre-training and the extracted features are used for
detection and identification of copper and plastic wires buried in the ground,
using Ground Penetrating radar (GPR) scans. In the target domain,
identification of the type of soil (wet or dry) is also added, making it a MTL
application. It is observed that, in many of the articles such as [51, 65,
68], a source network is trained on large dataset of one domain, which is
exploited by the target dataset of another domain having less data resources.
Thereby transfer learning helps to find good feature representations leading
to better generalization. Along with the multi-task architecture making it
possible to train many tasks jointly.

Some article that appear in search results for MTL \+ transfer learning on
google scholar, discuss these learning paradigms separately which is not the
aim of this article. This section is focusing on articles those have used the
two learning algorithms together. Like [69], gives a detailed survey of how
transfer learning and MTL individually are used in the field of bio-
informatics. [70] is a book chapter which discusses types of transfer learning
and describes MTL as a variant of transfer learning. While [71, 72], details
types of multi-view transfer learning and multi-view MTL. The articles [73,
74] gives theoretical concepts of improving the performance of MTL and
transfer learning in general.

###  III-C Research in Meta transfer learning

Meta learning and transfer learning are very similar, like both have source
tasks and target tasks which are unseen and the objective is better
generalization on the target task. But, what differs is the optimization
scheme, meta learning has a two level optimization, whereas transfer learning
doesn’t. However, an alliance of transfer learning with meta learning is
practiced in many researches in the literature.

As already discussed in Sec. III-B transfer learning can be a very crucial in
extracting features of the data using pre-trained model, and therefore is
capable of integrating with other learning algorithms. The article [75, 76],
follows a similar approach for performing few-shot learning. Transfer learning
is used during meta training by extraction representations of images trained
on a very large dataset like MiniImageNet [77]. The meta transfer learning
proposed in this work answers two important questions, i.e., what to transfer
and how to transfer. The Deep Neural Network (DNN) parameters trained on the
large scale data is the answer to what to transfer, whereas the scaling and
shifting operations learnt for each tasks, introduced in this work refer to
how to transfer. The meta knowledge from the training stage is shared in the
testing stage on the Fewshot-CIFAR 100 [78] dataset to learn new tasks using
less data instances. They also propose a hard task meta batch strategy in
which rather than randomly picking tasks for meta training, the algorithm
based on the past validation accuracy and failure, resamples the hard tasks.
The use of pre-trained DNN was proved to be very useful for tailoring the
learning experience for unseen tasks.

Likewise, [79] also uses transfer learning together with optimization based
meta learning method MAML [27] for zero shot super-resolution of images. For
faster adaptation and better generalization on new task, meta learning helps
to learn an effective initial parameters during training. At the time of meta
testing it takes only a few gradient steps to learn the image specific
information, even in case of external (or new) data instance. The learning
strategy introduced in this work, learns initialization parameters with
reference to different blur conditions, thereby making it possible to quickly
adapt to new (unseen) blur kernels. It uses a large-scale dataset ImageNet
[59] for transfer learning and meta training, during meta testing low
resolution images are used to train a model with corresponding blur kernel.
Therefore, at the testing stage the model can be trained for multiple types of
blur kernels.

A recent article [80] adopts meta transfer learning for predicting the
dynamics of water temperature of un-monitored lakes. Since the data for the
un-monitored lakes in insufficient and all the deep learning models need a lot
of data to learn, this work utilizes the adequately available data of the
monitored lakes to learn the models and transfer knowledge to the less
resourceful domain. It exploits the data of the monitored lakes by extraction
important characteristics and using two source models i.e., process based and
process guided deep learning, on each monitored lake and evaluate the
performance of the models, thereby applying transfer learning for the un-
monitored lakes. This meta knowledge of features and performance of the models
of the monitored lakes is used to select the best model for the un-monitored
lakes based on the lowest predicted error. These pre-trained models outperform
on the target un-monitored lake as compared to the when no transfer of
learning is performed.

For the purpose of vehicle tracking using Unmanned Aerial Vehicle (UAV) [81],
uses a pretrained model for vehicle tracking on ground images and employs the
model to adapt to the drone view images. From the deep learning viewpoint,
vehicle tracking in UAV is an under-explored research area, as these videos
have very less labelled data. To over come this availability of enough data
problem, transfer learning is employed. Large dataset of ground view vehicle
tracking are used to train a model, which is thereafter used by the drone-view
dataset. therefore transferring features across landscapes. Whereas to
adaptively extract shared features between both the domains (drone and ground
view), meta learning is used. Therefore, transfer learning helps to overcome
the data scarcity problem, while meta learning solves the issue of domain
shift.

A similar issue of difference in distributions during training and testing is
solved by using transfer and meta learning for down-link beam-forming in [82].
As in conventional supervised learning it is assumed that the training and
test data belongs to the same distribution, and if the distribution of the
test data changes the performance of the model is unsatisfactory. To address
the are changes in the wireless environment, and for faster adaptation to new
unseen data, this work proposes two types of models based on deep transfer
learning and meta learning. The results in the article prove that the
performance of the proposed algorithm is significantly better than the
traditional deep learning models.

Transfer of meta information (which is simply meta learning) is also termed as
meta transfer learning in articles such as [83, 84, 85, 86]. Like [83], the
meta information for facial emotion classification on one dataset is
transferred to another dataset. The article [84] discusses to transfer
knowledge across experiments for dynamic pricing application. [86] introduces
a variant of meta learning which learns how to learn kernels from data and
share the sequence of transformations to find a kernel for a new task. These
are solely meta learning applications, but due to abuse of terminology they
fall under the category of meta transfer learning, when actually it refers to
simply transfer of meta learnings (or knowledge).

##  IV Discussion - Why these learning paradigms should be used together?

As discussed in Sec. II-E, the learning algorithms have some strengths and
drawbacks, and when the union of two algorithms is employed, it helps to
overcome a few weakness. Like the integration of multi-task and meta learning,
helps in good generalization for unseen tasks, enables faster learning for new
tasks and also makes it possible to handle heterogeneous tasks, overall makes
it possible to introduce a new task in the multi-task and also have different
types of tasks in meta learning. But at the same time the issue of making
architectural changes for a new task in MTL still and large data requirement
still remains a challenge. For the combination of meta and transfer learning,
faster learning with less data and good performance, on a new task are
certainly the strengths of the merger, although the disadvantage that both of
these algorithms only focus on improving the target tasks and not able to have
multi-modal inputs, cannot be omitted. Similarly, the merger of transfer
learning with MTL brings the advantage of handling multi-modal inputs, various
heterogeneous tasks, requiring less data for training and aim to boost the
performance of all the tasks (because of MTL), but at the same time suffer
from the problem of unable to integrate an unseen task and good generalization
(as often pre-trained models tend to overfit).

A reliable solution that may help to beat the issues mentioned above, is an
ensemble of the three learning paradigms, i.e., MTL, meta learning and
transfer learning. The primary objectives of this ensemble are :

  1. 1.

Good performance on a new unseen task (due to meta learning)

  2. 2.

Ability to handle multi-modal inputs and heterogeneous tasks (due to MTL)

  3. 3.

Require less training data and good feature representation for learning (due
to transfer learning)

Figure 6: Proposed implementation, that ensembles MTL, meta learning and
transfer learning. Here the checkered rectangle boxes represent the pretrained
layers (or model), the solid rectangle boxes are the layers trained from
scratch and the glowing rectangle boxes denote the layers which are fine-
tuned, i.e., the parameters from the source model act as initialization for
the target network.

There are many approaches in which this ensemble may be implemented. One of
the them, as illustrated in Fig. 6 is when the architecture of the model is
with reference to MTL, it uses transfer learning, basically the pre-trained
models for better feature extraction for various inputs and then the backbone
network and multi-head modules are trained using a two level optimization that
is employed in meta learning. Assuming the inputs are multimodal, the pre-
trained networks (checkered boxes) extract important features from the inputs
and these are processed accordingly in the multimodal embedding layer before
forwarding to the backbone network, similar to what is presented in [21, 87].
The backbone network refers to the generic layers of the neural network or
CNN, whose input is the combination of embeddings from various inputs and the
output from the backbone network goes into the task specific layers. The
architecture of every task-specific layer can differ as required by the task.
Therefore, in the meta training stage, the multi-task architecture helps to
learn many tasks together, thereby improving the performance of all the tasks.
In the meta testing stage when new tasks (Task-4 and Task-5) are introduced,
it is easy to learn in fewer data instances and the inductive bias of the
multi-task architecture assists in better generalization than single task
learning. Also, in this stage it allows for both feature extraction and fine-
tuning variants of transfer learning. This proposed implementations gives the
liberty to add new heterogeneous tasks, allows for multi-modal inputs, because
of transfer learning and meta learning it also enables learning with less data
samples during meta testing, and MTL helps to improve the performance of both
the source and the target tasks. hence, it complies with all the three
objectives mentioned above.

Figure 7: A variant of the proposed implementation in Fig. 6, it represents
the instance when there is a single input and multiple tasks. The case-1 and 2
can be understood from two point of views one is transfer learning and the
other is meta learning, while the architecture represents MTL. Here the
checkered rectangle boxes represent the pretrained layers (or model), the
solid rectangle boxes are the layers trained from scratch and the glowing
rectangle boxes denote the layers which are fine-tuned for unseen target data.

A variant of the implementation in Fig. 6 is shown in Fig. 7, when there is
one input and many tasks related to the input (similar to the illustration in
Fig. 1). Here a multi-task source architecture can be meta trained for many
different datasets and similar tasks. Later the network is used for some data
of totally different domain, and the meta information from the training phase
will be useful for improved learning of the target tasks. The trained network
on the source data can either be employed as a feature extractor i.e., case -1
in Fig. 7 or fine-tuned for the target data i.e., case-2 in Fig. 7, this is
the transfer learning point of view. Another way of viewing this approach is
from the meta learning point of view, wherein the training of the source tasks
can be considered as the meta training stage, the case-1 can be treated as the
meta validation stage because there is no training and it is only doing
inference on a new data, and the case-2 is nothing but the meta testing stage
which uses the initial parameters from the source model and then trains the
network on new target dataset.

Fig. 6 is a plausible approach of how to fuse the learning paradigms together.
Adhering to the three primary objectives, certainly there can be several other
ways of accomplishing this. Not to forget, the approach should be able to
exploit all the advantages of these learning algorithms and try to overcome
the weakness of each other. Also, these approaches depend on the use-case and
the available dataset, i.e., the number of tasks, modality of the data, amount
of data samples, etc. To the best of our knowledge, there is no such work in
the literature, that employs the ensemble of meta learning, transfer learning
and MTL. Indeed, the related work or future scope section of many articles
mention these three learning algorithm, but none of them illustrate applying
them together.

##  V Open questions and future work

(a) Generic approach depreciated to MTL, by not using the pre-trained
networks, not employing the meta optimization and also avoiding the testing on
unseen tasks.

(b) Generic approach depreciated to meta learning, by avoiding the use of pre-
trained models and also the shared layers where the fused features from the
embeddings layer is given as inputs.

(c) Generic approach depreciated to transfer learning, by not using the shared
backbone network and the meta optimization scheme, along with using the pre-
trained models and fine-tuned layers as required for various tasks.

Figure 8: An illustration of reducing the generic structure of Fig. 6 to
individual learning paradigms. The black(hidden) area represent that the
blocks underneath are disabled.

This article discusses the fundamentals of transfer learning, MTL and meta
learning, gives a comparative view on these three algorithms, details the
literature study when pairs of these algorithms are employed together and also
proposes a few approaches of how all the three learning paradigms can be
jointly used to overcome the drawbacks of each other. Since, the proposed
approaches are a combination of transfer learning, meta learning and MTL,
along with the liberty to employ multi-modal inputs and heterogeneous task,
this study introduces the ensemble as  Multi-modal multi-task meta transfer
learning ($\mathrm{M^{3}TL}$). These suggested techniques in Sec. IV may
overcome most of the drawbacks faced by the learning algorithms when used in
isolation, to investigate this, this survey article enlists a few open
research questions. These are:

  1. 1.

How is the proposed $\mathrm{M^{3}TL}$ approach better than single task
learning?

  2. 2.

In $\mathrm{M^{3}TL}$, is there a possibility for modular learning? In
particular, for an unseen task, can the network automatically choose, based on
the meta knowledge which part of the network should be trained rather than
training the whole network and how to exploit the shared structure (using
transfer learning) ?

  3. 3.

Do similar tasks automatically resort to similar sub-architecture training?

  4. 4.

How can meta learning and transfer learning help to automatically alter the
network architecture for a new task?

  5. 5.

In case of multi-modal inputs, analyze the contribution of each modalities for
the outcome.

This work therefore proposes a novel generic approach to implement
$\mathrm{M^{3}TL}$ in Fig. 6, keeping in mind the three objectives required
for the ensemble. It is possible to reduce the global approach to any one of
the learning paradigm as shown in Fig. 8. For example, as illustrated in Fig.
8(a), only MTL can be achieved by not introducing the new tasks, by avoiding
the use of pre-trained models and also by not employing the bi-level meta
optimization scheme, i.e., when $\theta=\phi$ in the meta training phase Sec.
II-D, narrows down to only one optimization objective and multiple tasks for
training. Fig. 8(b) demonstrates how the generic implementation can be reduces
to meta learning, by adding no pre-trained models, no embedding layer to fuse
the embeddings and also no shared layers, as every isolated input is trained
for a task using two level optimization in the training phase and in the meta
testing phase new tasks can be introduced. On the same conditions, if even the
optimization is cut- down to single level optimization, along with giving the
ability to use the pre-trained and fine tune networks (howsoever required),
the approach is now shortened to transfer learning as in Fig. 8(c). It can be
considered as a global learning structure, which gives the flexibility to
choose between the learning algorithms and also combinations of them.

Since, the global network introduced in this work makes it possible to choose
the elements required to learn the task. It will certainly be worthwhile to
meta train the network to learn to evaluate which elements will do justice to
a task, rather than employing all the learning algorithms. Because there is a
possibility that for some tasks jointly learning is not possible and in such a
scenario the idea is to learn to activate only parts of the network, thereby
foreseeing a formulation of meta-meta learning algorithm. In consequence, as a
future research it will be interesting to explore how these learning paradigms
together share to learn and learn to share, along with knowing when to share
and thereby making it possible to develop human like learning techniques.

## References

  * [1] R. Fong, W. Scheirer, and D. Cox, “Using human brain activity to guide machine learning,” 2017.
  * [2] B. Liu, “Lifelong machine learning: A paradigm for continuous learning,” _Front. Comput. Sci._ , vol. 11, no. 3, p. 359–361, Jun. 2017. [Online]. Available: https://doi.org/10.1007/s11704-016-6903-6
  * [3] R. Caruana, “Multitask learning: A knowledge-based source of inductive bias,” in _Proceedings of the Tenth International Conference on International Conference on Machine Learning_ , ser. ICML’93. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1993, p. 41–48.
  * [4] Z. Chen, B. Liu, R. Brachman, P. Stone, and F. Rossi, _Lifelong Machine Learning_ , 2nd ed. Morgan & Claypool Publishers, 2018.
  * [5] M. Crawshaw, “Multi-task learning with deep neural networks: A survey,” 2020.
  * [6] W. Zhang, L. Deng, L. Zhang, and D. Wu, “A survey on negative transfer,” 2021\.
  * [7] T. M. Mitchell, _Machine Learning_ , 1st ed. USA: McGraw-Hill, Inc., 1997.
  * [8] M. E. Hasselmo, “Avoiding Catastrophic Forgetting,” pp. 407–408, jun 2017. [Online]. Available: http://dx.doi.org/10.1016/j.tics.2017.04.001
  * [9] I. J. Goodfellow, Y. Bengio, and A. Courville, _Deep Learning_. Cambridge, MA, USA: MIT Press, 2016, http://www.deeplearningbook.org.
  * [10] S. J. Pan, Q. Yang, W. Fan, and S. J. P. (ph. D, “A survey on transfer learning,” _IEEE Transactions on Knowledge and Data Engineering_ , 2010.
  * [11] R. Caruana, “Multitask learning,” _Mach. Learn._ , vol. 28, no. 1, p. 41–75, Jul. 1997. [Online]. Available: https://doi.org/10.1023/A:1007379606734
  * [12] R. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng, “Self-taught learning: Transfer learning from unlabeled data,” in _Proceedings of the 24th International Conference on Machine Learning_ , ser. ICML ’07. New York, NY, USA: Association for Computing Machinery, 2007, p. 759–766. [Online]. Available: https://doi.org/10.1145/1273496.1273592
  * [13] A. Arnold, R. Nallapati, and W. W. Cohen, “A comparative study of methods for transductive transfer learning,” in _Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007)_ , 2007, pp. 77–82.
  * [14] A. Farahani, S. Voghoei, K. Rasheed, and H. R. Arabnia, “A brief review of domain adaptation,” 2020.
  * [15] W. Dai, Q. Yang, G.-R. Xue, and Y. Yu, “Self-taught clustering,” in _Proceedings of the 25th international conference on Machine learning_ , 2008, pp. 200–207.
  * [16] D. Xu, W. Ouyang, X. Wang, and N. Sebe, “Pad-net: Multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing,” _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 675–684, 2018.
  * [17] A. R. Zamir, A. Sax, W. B. Shen, L. J. Guibas, J. Malik, and S. Savarese, “Taskonomy: Disentangling task transfer learning,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, 2018.
  * [18] M. Crawshaw, “Multi-task learning with deep neural networks: A survey,” _arXiv preprint arXiv:2009.09796_ , 2020.
  * [19] S. Thrun and L. Pratt, Eds., _Learning To Learn_. Boston, MA: Kluwer Academic Publishers, 1998.
  * [20] T. M. Hospedales, A. Antoniou, P. Micaelli, and A. J. Storkey, “Meta-learning in neural networks: A survey,” _IEEE Transactions on Pattern Analysis & Machine Intelligence_, pp. 1–1, may 5555.
  * [21] H. Zhang, S. Qian, Q. Fang, and C. Xu, “Multi-modal meta multi-task learning for social media rumor detection,” _IEEE Transactions on Multimedia_ , 2021\.
  * [22] J. Chen, X. Qiu, P. Liu, and X. Huang, “Meta multi-task learning for sequence modeling,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 32, no. 1, 2018.
  * [23] P. Liu and X. Huang, “Meta-learning multi-task communication,” _arXiv preprint arXiv:1810.09988_ , 2018.
  * [24] I. Tarunesh, S. Khyalia, V. Kumar, G. Ramakrishnan, and P. Jyothi, “Meta-learning for effective multi-task and multilingual modelling,” _arXiv preprint arXiv:2101.10368_ , 2021.
  * [25] J. Y. Lee, K. A. Lee, and W. S. Gan, “Generating personalized dialogue via multi-task meta-learning,” 2021.
  * [26] X. Chen and K. Q. Zhu, “St2: Small-data text style transfer via multi-task meta-learning,” 2020.
  * [27] C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in _Proceedings of the 34th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, D. Precup and Y. W. Teh, Eds., vol. 70. PMLR, 06–11 Aug 2017, pp. 1126–1135. [Online]. Available: https://proceedings.mlr.press/v70/finn17a.html
  * [28] K. Lekkala and L. Itti, “Attentive feature reuse for multi task meta learning,” 2020.
  * [29] A. Raghu, M. Raghu, S. Bengio, and O. Vinyals, “Rapid learning or feature reuse? towards understanding the effectiveness of maml,” in _International Conference on Learning Representations_ , 2020. [Online]. Available: https://openreview.net/forum?id=rkgMkCEtPB
  * [30] R. Cai, K. Guo, B. Xu, X. Yang, and Z. Zhang, “Meta multi-task learning for speech emotion recognition.” in _INTERSPEECH_ , 2020, pp. 3336–3340.
  * [31] L. Liu, Z. Liu, H. Wu, Z. Wang, J. Shen, Y. Song, and M. Zhang, “Multi-task learning via adaptation to similar tasks for mortality prediction of diverse rare diseases,” _AMIA … Annual Symposium proceedings. AMIA Symposium_ , vol. 2020, pp. 763–772, 2020.
  * [32] J. Bronskill, J. Gordon, J. Requeima, S. Nowozin, and R. Turner, “TaskNorm: Rethinking batch normalization for meta-learning,” in _Proceedings of the 37th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, H. D. III and A. Singh, Eds., vol. 119. PMLR, 13–18 Jul 2020, pp. 1153–1164. [Online]. Available: https://proceedings.mlr.press/v119/bronskill20a.html
  * [33] A. Ghadirzadeh, X. Chen, P. Poklukar, C. Finn, M. Björkman, and D. Kragic, “Bayesian meta-learning for few-shot policy adaptation across robotic platforms,” 2021.
  * [34] A. Kedia and S. C. Chinthakindi, “Keep learning: Self-supervised meta-learning for learning from inference,” in _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_. Online: Association for Computational Linguistics, Apr. 2021, pp. 63–77. [Online]. Available: https://aclanthology.org/2021.eacl-main.6
  * [35] B. Tian, Y. Zhang, J. Wang, and C. Xing, “Hierarchical inter-attention network for document classification with multi-task learning,” in _IJCAI_ , 2019\.
  * [36] Y. Lin, B. Zhong, G. Li, S. Zhao, Z. Chen, and W. Fan, “Localization-aware meta tracker guided with adversarial features,” _IEEE Access_ , vol. 7, pp. 99 441–99 450, 2019.
  * [37] D. S. Krueger, T. Maharaj, S. Legg, and J. Leike, “Hidden incentives for self-induced distributional shift,” 2020. [Online]. Available: https://openreview.net/forum?id=SJeFNlHtPS
  * [38] T. Bansal, R. Jha, T. Munkhdalai, and A. McCallum, “Self-supervised meta-learning for few-shot natural language classification tasks,” 2020.
  * [39] Y. Zou and X. Lu, “Gradient-em bayesian meta-learning,” 2020.
  * [40] L. Zintgraf, K. Shiarlis, M. Igl, S. Schulze, Y. Gal, K. Hofmann, and S. Whiteson, “Varibad: A very good method for bayes-adaptive deep rl via meta-learning,” 2020.
  * [41] W. Zhou, Y. Li, Y. Yang, H. Wang, and T. M. Hospedales, “Online meta-critic learning for off-policy actor-critic methods,” 2020.
  * [42] D. Krueger, T. Maharaj, and J. Leike, “Hidden incentives for auto-induced distributional shift,” 2020.
  * [43] P. Guo, C.-Y. Lee, and D. Ulbricht, “Learning to branch for multi-task learning,” in _Proceedings of the 37th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, H. D. III and A. Singh, Eds., vol. 119. PMLR, 13–18 Jul 2020, pp. 3854–3863. [Online]. Available: https://proceedings.mlr.press/v119/guo20e.html
  * [44] M. Kim and V. Pavlovic, “Recursive inference for variational autoencoders,” 2020\.
  * [45] ——, “Reducing the amortization gap in variational autoencoders: A bayesian random function approach,” 2021.
  * [46] F. Retyk, “On meta-reinforcement learning in task distributions with varying dynamics,” Ph.D. dissertation, UPC, Facultat d’Informàtica de Barcelona, Departament de Ciències de la Computació, Apr 2021. [Online]. Available: http://hdl.handle.net/2117/348143
  * [47] A. Ghosh, S. Tschiatschek, H. Mahdavi, and A. Singla, in _19th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS)_ , ser. Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, May 2020, pp. 447–455.
  * [48] ——, “Towards deployment of robust ai agents for human-machine partnerships,” 2020.
  * [49] Z. Ye, Y. Yang, X. Li, D. Cao, and D. Ouyang, “An integrated transfer learning and multitask learning approach for pharmacokinetic parameter prediction,” _Molecular Pharmaceutics_ , vol. 16, no. 2, p. 533–541, Dec 2018. [Online]. Available: http://dx.doi.org/10.1021/acs.molpharmaceut.8b00816
  * [50] M. J. Hasan, M. Sohaib, and J.-M. Kim, “A multitask-aided transfer learning-based diagnostic framework for bearings under inconsistent working conditions,” _Sensors_ , vol. 20, no. 24, 2020. [Online]. Available: https://www.mdpi.com/1424-8220/20/24/7205
  * [51] J. C. B. Cruz, J. A. Tan, and C. Cheng, “Localization of fake news detection via multitask transfer learning,” in _Proceedings of The 12th Language Resources and Evaluation Conference_. Marseille, France: European Language Resources Association, May 2020, pp. 2596–2604. [Online]. Available: https://www.aclweb.org/anthology/2020.lrec-1.316
  * [52] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” 2019.
  * [53] J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” 2018.
  * [54] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever _et al._ , “Language models are unsupervised multitask learners,” _OpenAI blog_ , vol. 1, no. 8, p. 9, 2019.
  * [55] X. Dong, S. Chowdhury, L. Qian, X. Li, Y. Guan, J. Yang, and Q. Yu, “Deep learning for named entity recognition on chinese electronic medical records: Combining deep transfer learning with multitask bi-directional lstm rnn,” _PLOS ONE_ , vol. 14, no. 5, pp. 1–15, 05 2019. [Online]. Available: https://doi.org/10.1371/journal.pone.0216046
  * [56] S. Taslimipoor, O. Rohanian, and L. A. Ha, “Cross-lingual transfer learning and multitask learning for capturing multiword expressions,” in _Proceedings of the Joint Workshop on Multiword Expressions and WordNet (MWE-WN 2019)_. Florence, Italy: Association for Computational Linguistics, Aug. 2019, pp. 155–161. [Online]. Available: https://aclanthology.org/W19-5119
  * [57] C. Du, C. Du, L. Huang, H. Wang, and H. He, “Structured neural decoding with multitask transfer learning of deep neural network representations,” _IEEE Transactions on Neural Networks and Learning Systems_ , pp. 1–15, 2020\.
  * [58] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in _Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1_ , ser. NIPS’12. Red Hook, NY, USA: Curran Associates Inc., 2012, p. 1097–1105.
  * [59] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in _2009 IEEE Conference on Computer Vision and Pattern Recognition_ , 2009, pp. 248–255.
  * [60] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” 2014.
  * [61] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” 2014.
  * [62] N.-V. Nguyen, C. Rigaud, A. Revel, and J.-C. Burie, “Manga-mmtl: Multimodal multitask transfer learning for manga character analysis,” in _Document Analysis and Recognition – ICDAR 2021_ , J. Lladós, D. Lopresti, and S. Uchida, Eds. Cham: Springer International Publishing, 2021, pp. 410–425.
  * [63] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015.
  * [64] Y. Qu, R. K. Baghbaderani, and H. Qi, “Few-shot hyperspectral image classification through multitask transfer learning,” in _2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS)_ , 2019, pp. 1–5.
  * [65] N. T. Dong and M. Khosla, “A multitask transfer learning framework for novel virus-human protein interactions,” _bioRxiv_ , 2021. [Online]. Available: https://www.biorxiv.org/content/early/2021/03/26/2021.03.25.437037
  * [66] E. C. Alley, G. Khimulya, S. Biswas, M. AlQuraishi, and G. M. Church, “Unified rational protein engineering with sequence-only deep representation learning,” _bioRxiv_ , 2019. [Online]. Available: https://www.biorxiv.org/content/early/2019/03/26/589333
  * [67] E. Aydin and S. E. Y. Erdem, “Transfer and multitask learning using convolutional neural networks for buried wire detection from ground penetrating radar data,” in _Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XXIV_ , S. S. Bishop and J. C. Isaacs, Eds., vol. 11012, International Society for Optics and Photonics. SPIE, 2019, pp. 259 – 270. [Online]. Available: https://doi.org/10.1117/12.2518875
  * [68] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2015.
  * [69] Q. Xu and Q. Yang, “A survey of transfer and multitask learning in bioinformatics,” _J. Comput. Sci. Eng._ , vol. 5, pp. 257–268, 2011.
  * [70] U. Kamath, J. Liu, and J. Whitaker, _Transfer Learning: Scenarios, Self-Taught Learning, and Multitask Learning_. Cham: Springer International Publishing, 2019, pp. 463–493. [Online]. Available: https://doi.org/10.1007/978-3-030-14596-5_10
  * [71] S. Sun, L. Mao, Z. Dong, and L. Wu, _Multiview Transfer Learning and Multitask Learning_. Singapore: Springer Singapore, 2019, pp. 85–104. [Online]. Available: https://doi.org/10.1007/978-981-13-3029-2_7
  * [72] E. Ricci, Y. Yan, A. K. Rajagopal, R. Subramanian, R. L. Vieriu, O. Lanz, and N. Sebe, “Chapter 4 - exploring multitask and transfer learning algorithms for head pose estimation in dynamic multiview scenarios,” in _Group and Crowd Behavior for Computer Vision_ , V. Murino, M. Cristani, S. Shah, and S. Savarese, Eds. Academic Press, 2017, pp. 67–87. [Online]. Available: https://www.sciencedirect.com/science/article/pii/B9780128092767000059
  * [73] B. Wang and J. Pineau, “Online boosting algorithms for anytime transfer and multitask learning,” _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 29, no. 1, Feb. 2015. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/9607
  * [74] A. Maurer, M. Pontil, and B. Romera-Paredes, “Sparse coding for multitask and transfer learning,” 2014.
  * [75] Q. Sun, Y. Liu, T.-S. Chua, and B. Schiele, “Meta-transfer learning for few-shot learning,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * [76] Q. Sun, Y. Liu, Z. Chen, T.-S. Chua, and B. Schiele, “Meta-transfer learning through hard tasks,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , pp. 1–1, 2020.
  * [77] O. Vinyals, C. Blundell, T. Lillicrap, k. kavukcuoglu, and D. Wierstra, “Matching networks for one shot learning,” in _Advances in Neural Information Processing Systems_ , D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, Eds., vol. 29. Curran Associates, Inc., 2016. [Online]. Available: https://proceedings.neurips.cc/paper/2016/file/90e1357833654983612fb05e3ec9148c-Paper.pdf
  * [78] B. N. Oreshkin, P. R. López, and A. Lacoste, “Tadam: Task dependent adaptive metric for improved few-shot learning,” in _NeurIPS_ , 2018.
  * [79] J. W. Soh, S. Cho, and N. I. Cho, “Meta-transfer learning for zero-shot super-resolution,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2020.
  * [80] J. D. Willard, J. S. Read, A. P. Appling, S. K. Oliver, X. Jia, and V. Kumar, “Predicting water temperature dynamics of unmonitored lakes with meta‐transfer learning,” _Water Resources Research_ , vol. 57, no. 7, Jun 2021. [Online]. Available: http://dx.doi.org/10.1029/2021WR029579
  * [81] W. Song, S. Li, Y. Guo, S. Li, A. Hao, H. Qin, and Q. Zhao, “Meta transfer learning for adaptive vehicle tracking in uav videos,” in _MultiMedia Modeling_ , Y. M. Ro, W.-H. Cheng, J. Kim, W.-T. Chu, P. Cui, J.-W. Choi, M.-C. Hu, and W. De Neve, Eds. Cham: Springer International Publishing, 2020, pp. 764–777.
  * [82] Y. Yuan, G. Zheng, K.-K. Wong, B. Ottersten, and Z.-Q. Luo, “Transfer learning and meta learning-based fast downlink beamforming adaptation,” _IEEE Transactions on Wireless Communications_ , vol. 20, no. 3, pp. 1742–1755, 2020\.
  * [83] D. Nguyen, K. Nguyen, S. Sridharan, I. Abbasnejad, D. Dean, and C. Fookes, “Meta transfer learning for facial emotion recognition,” in _2018 24th International Conference on Pattern Recognition (ICPR)_ , 2018, pp. 3543–3548.
  * [84] H. Bastani, D. Simchi-Levi, and R. Zhu, “Meta dynamic pricing: Transfer learning across experiments,” 2021.
  * [85] T. Duan, M. Chauhan, M. A. Shaikh, J. Chu, and S. Srihari, “Ultra efficient transfer learning with meta update for cross subject eeg classification,” 2021\.
  * [86] F. Aiolli, “Transfer learning by kernel meta-learning,” in _Proceedings of ICML Workshop on Unsupervised and Transfer Learning_ , ser. Proceedings of Machine Learning Research, I. Guyon, G. Dror, V. Lemaire, G. Taylor, and D. Silver, Eds., vol. 27. Bellevue, Washington, USA: PMLR, 02 Jul 2012, pp. 81–95. [Online]. Available: https://proceedings.mlr.press/v27/aiolli12a.html
  * [87] O. E. L. Team, A. Stooke, A. Mahajan, C. Barros, C. Deck, J. Bauer, J. Sygnowski, M. Trebacz, M. Jaderberg, M. Mathieu, N. McAleese, N. Bradley-Schmieg, N. Wong, N. Porcel, R. Raileanu, S. Hughes-Fitt, V. Dalibard, and W. M. Czarnecki, “Open-ended learning leads to generally capable agents,” 2021.

  *[ML]: Machine Learning
  *[CNNs]: Convolutional Neural Networks
  *[DL]: Deep Learning
  *[MTL]: Multi-Task Learning
  *[LL]: Lifelong Learning
  *[RL]: Reinforcement Learning
  *[CNN]: Convolutional Neural Network
  *[LSTM]: Long Short-Term Memory
  *[NLP]: Natural Language Processing
  *[MAML]: Model Agnostic Meta Learning
  *[ADME]: Absorption, Distribution, Metabolism, and Excretion
  *[NER]: Named Entity Recognition
  *[EMR]: Electronic Medical Records
  *[POS]: Parts-of-Speech
  *[SMR]: Structured Multi-output Regression
  *[VAEs]: Variational Auto-Encoders
  *[GANs]: Generative Adversarial Nets
  *[DNN]: Deep Neural Network
  *[$\mathrm{M^{3}TL}$]: Multi-modal multi-task meta transfer learning