# Grasping Extreme Aerodynamics on a Low-Dimensional Manifold

Kai Fukami  Kunihiko Taira<EMAIL_ADDRESS>
###### Abstract

Modern air vehicles perform a wide range of operations, including
transportation, defense, surveillance, and rescue. These aircraft can fly in
calm conditions but avoid operations in gusty environments, encountered in
urban canyons, over mountainous terrains, and in ship wakes. With extreme
weather becoming ever more frequent due to global warming, it is anticipated
that aircraft, especially those that are smaller in size, will encounter
sizeable atmospheric disturbances and still be expected to achieve stable
flight. However, there exists virtually no theoretical fluid-dynamic
foundation to describe the influence of extreme vortical gusts on wings. To
compound this difficulty, there is a large parameter space for gust-wing
interactions. While such interactions are seemingly complex and different for
each combination of gust parameters, we show that the fundamental physics
behind extreme aerodynamics is far simpler and lower-rank than traditionally
expected. We reveal that the nonlinear vortical flow field over time and
parameter space can be compressed to only three variables with a lift-
augmented autoencoder while holding the essence of the original high-
dimensional physics. Extreme aerodynamic flows can be compressed through
machine learning into a low-dimensional manifold, which can enable real-time
sparse reconstruction, dynamical modeling, and control of extremely unsteady
gusty flows. The present findings offer support for the stable flight of next-
generation small air vehicles in atmosphere conditions traditionally
considered unflyable.

## Introduction

Since the Wright brothers accomplished the first human-powered and controlled
flight in 1903, a wide variety of aircraft has been developed for
transportation, defense, observation, search, and rescue missions. What makes
these aircraft uniquely different from other modes of transportation is their
ability to stay aloft by taking advantage of aerodynamics. To support the
development of these aircraft, the field of aerodynamics has undergone
tremendous growth over the past century. Despite its expansive theory, the
current aerodynamics is generally based on steady (cruise) or quasi-steady
flight conditions with linear analysis or its nonlinear extensions for small
perturbations [1, 2, 3].

We are now at an important transition point for aerodynamics. With novel
materials and enhanced powerplants/batteries becoming available over the past
couple of decades, there have been tremendous efforts in developing and
operating smaller size personalized air vehicles and unmanned air systems [4,
5, 6, 7, 8]. They can traverse unconventional terrain, including mountainous
and urban environments [9, 10, 11] that were traditionally avoided by
conventional aircraft. As a matter of fact, flight demonstrations of such air
vehicles in calm weather have taken place in recent years [12]. These new and
amazing flying vehicle concepts will likely revolutionize air-based
transportation [13, 5, 14] and have already been realized for some cases [15,
16].

However, there are major challenges in operating small-scale aircraft under
these complex airspace when adverse weather generates highly turbulent
environments [17, 18, 19, 20, 21]. Moreover, the increased occurrence of
extreme weather caused by global warming limits the operations of aircraft.
Flying small-scale aircraft near large natural or manmade structures in
adverse weather face additional complications as these vehicles need to
navigate through severe turbulence comprised of gusts and vortical
disturbances, as illustrated in Fig. 1. These disturbances come in a variety
of forms [22] and are far more disruptive than what present-day commercial
aircraft experience in inclement weather [23]. These extreme aerodynamic
environments can be characterized by a manifestation of a large number of
strong vortices with different strengths, sizes, and orientations generated by
the surrounding structures [24]. Such a flight environment has been off-limits
due to the fact that there is virtually no available theory for extreme
aerodynamic problems and to avoid possible loss of aircraft.

Figure 1:  (Left) Illustration of possible extreme aerodynamic encounters by
modern air vehicles in urban environment during adverse weather. Air vehicles
operating in such an environment experience extreme level of unsteady
aerodynamic forces due to strong gusts with spatial variations comparable to
their vehicle size. (Right) Model problem of strong disturbance vortex
impinging on an airfoil with vorticity distribution being visualized.

With infinite scenarios of large and strong atmospheric disturbances hitting a
flying vehicle, we cannot only focus on a single cruise condition but must
also consider a whole array of cases in which wings experience extreme
aerodynamic disturbances. These disturbances are characterized by a variety of
parameters, including the size, strength, orientation, position, and geometry
of the disturbances, necessitating massive experimental and computational
campaigns if approached naively. With a single extreme aerodynamic simulation
already producing a very large amount of flow field data, extensive parametric
sweeps lead to an enormous collection of aerodynamic flow data and calls for
significant computational and experimental resources. These extreme
aerodynamic flows exhibit rich nonlinear behavior over a range of
spatiotemporal scales that cannot be easily analyzed and modeled with existing
theories.

One of the important parameters under such extreme aerodynamic situations is
the gust ratio $G=u_{\text{gust}}/u_{\infty}$, which is ratio between the
characteristic gust velocity $u_{\text{gust}}$ and the translational velocity
of the wing $u_{\infty}$. For conditions of $G\gtrsim 1$, sustaining stable
flight becomes challenging [18, 25]. In the present work, we consider high
levels of aerodynamic disturbance with $0\leq G\leq 10$, and refer to cases of
$G>1$ as _extreme aerodynamics_. Strong gusts with $G>1$ can be encountered in
urban canyons, mountainous environments, and severe atmospheric turbulence.
The goal of this study is to identify the unifying dynamics that a wing
experiences from extreme gust disturbances. At the most fundamental level, the
present problem requires the identification of the underlying nonlinear
dynamics of the complex separated flows from an enormous amount of data in an
efficient manner while gaining physical insights into extreme aerodynamics.

Although the aerodynamic influence of large-scale disturbances on lifting
bodies take various forms, the underlying dynamics are generally shared. In
this study, we seek these dominant dynamics embedded in complex extreme
aerodynamic flows. To achieve this objective, we examine the reduction of
massive fluid flow data into a low-dimensional space in which the right set of
variables describe the underlying extreme aerodynamic physics. This process is
enabled by incorporating physical observables and ensuring that the gained
insights are interpretable and beneficial for future aircraft operations and
designs. In fact, we find that extreme aerodynamic flows can be compressed by
a carefully designed nonlinear machine-learning technique to only three
variables for a model problem of a strong vortex hitting a canonical airfoil.
The present findings further suggest that the discovered manifold holds
potential to support downstream tasks such as real-time flow estimation,
dynamical modeling, flow control, and vehicle design.

## Results

### Extreme Vortex-Airfoil Interactions

We consider a strong vortex gust impacting an airfoil as a representative
model problem for wings experiencing extreme atmospheric disturbances. The
present model problem involves wake vortices shedding from a high-rise
building, ships in rough seas, and mountain ridges [18]. The size of such
strong vortices can be comparable to the size of the wing, exerting
tremendously large lift and drag forces. In this study, we simulate two-
dimensional incompressible flows with a vortex placed upstream of the wing
with varied vortex size, strength, and initial position, as shown in Fig. 1.
Because the airfoil wake responds differently for each combination of these
disturbance settings, the resulting flow fields exhibit vastly different wake
patterns and aerodynamic forces from case to case due to the nonlinear vortex
dynamics, as presented in Fig. 2. As the vortex passes around an airfoil, the
wing experiences massive flow separation (stall), which also causes the
emergence of additional vortical structures. All of these flow structures
interact nonlinearly, making the dynamics complex and difficult to predict and
control.

Figure 2:  Examples of lift responses and vortical flows. Visualizations of
the vorticity fields from times indicated by symbols $\circ$ and $\triangle$
on the lift responses (thick lines associated with the visualized cases). The
light-colored lift curves correspond to all lift responses considered in the
present study. The shown color for each angle of attack is shared with other
figures.

This study considers two-dimensional extreme aerodynamic flows around a NACA
0012 airfoil at a chord-based Reynolds number $\text{Re}=u_{\infty}c/\nu=100$.
Here, $u_{\infty}$ is the free-stream velocity, $c$ is the chord length, and
$\nu$ is the kinematic viscosity. The flow field is obtained with direct
numerical simulation using an incompressible flow solver [26, 27]. The airfoil
is positioned in the free stream with its leading edge at the origin with
angles of attack of $\alpha\in[20^{\circ},60^{\circ}]$, enabling us to cover
cases of steady and unsteady wakes in undisturbed (baseline) cases. These
wakes are disturbed with a gust vortex having an angular velocity profile [28]
of $u_{\theta}=u_{\theta,\max}(r/R)\exp[1/2-r^{2}/(2R^{2})]$, where the radius
of the vortex is $R$.

The present disturbance vortex is characterized by the gust ratio
$G\equiv{u_{\theta,\max}}/{u_{\infty}}\in[-10,10]$ with its size relative to
the wing chord $L\equiv{2R}/{c}\in[0.5,2]$ and is introduced upstream of the
wing at $x_{0}/c=-2$ and $y_{0}/c\in[-0.5,0.5]$. The combinations of these
parameters provide a wide range of large and strong gust vortices hitting the
wing at various locations. The flow field around the airfoil exhibits a rich
dynamical response to the disturbance vortex characterized by a large
parameter space comprised of $(\alpha,G,L,y_{0}/c)$. To fully resolve the
dynamics over this parameter space, a substantial number of flow cases for
different combinations of these parameters would be required. In general, Re
is also another parameter but is fixed for this study at 100. While the gust
vortices contained in actual atmospheric turbulence can be much more complex
than what is considered here, the primary dynamics of large vortex core
interacting with the wing is captured well at this Re at least in a two-
dimensional manner. What is particularly important is to resolve the large
vorticity source from the surface under local flow acceleration [29]. These
phenomena have relatively short time-scales compared to the much longer
viscous scales associated with $\text{Re}=100$, making the current problem
setup an appropriate test bed. In this study, we set the convective time to be
zero when the center of the vortex arrives at the leading edge of the wing
$x_{0}/c=0$. The snapshots of the vorticity field and the lift history are
examined in detail with data-driven analyses in what follows.

Without any external disturbances, the wing experiences steady aerodynamic
lift forces at low angles of attack ($\alpha\lesssim 20^{\circ}$) and moderate
unsteady aerodynamic force fluctuations at higher angles of attack
($\alpha\gtrsim 30^{\circ}$). These lift values are shown by the dashed lines
for the different angles of attack in Fig. 2. The unsteady lift fluctuations
are exerted by the von Karman vortices shedding from the leading and trailing
edges, as visualized in Fig. 2. At this Reynolds number, these cases are
periodic in time and constitute limit cycles.

When the wing encounters a strong gust vortex, the flow around the wing is
significantly modified. The approaching vortex strongly influences the
vorticity field around the wing and triggers large-scale vortex formation. For
example, let us consider the flow field for a case shown in Fig. 2 for which a
strong positive vortex with $G=3.8$ hits a wing at $\alpha=20^{\circ}$. Due to
this disturbance, two large vortices are formed shortly after impact,
generating massive separation due to the interaction of the gust vortex and
the wing wake. These dramatic transient wake dynamics exert sharp increase in
aerodynamic forces on the wing. In fact, the lift force increases 714% and
drops 656% within a short duration of 1.8 convective time. Such a significant
variation in the lift force makes controlling air vehicles tremendously
difficult. While not shown, the moment experienced by the wing also undergoes
a tremendous change. We also display a total of 100 force histories for cases
of flows disturbed by extreme levels of gust vortices in Fig. 2. In all of
these cases, the airfoil wake dynamics undergo large transient changes within
a short amount of time with large aerodynamic force fluctuations with similar
order of magnitudes. These violent disturbances not only destabilize flight
but can also damage vehicle structures, making the analysis of these flows
critically important.

Figure 3:  Nonlinear autoencoder. The vorticity field is taken as the input
and output. The green shaded portion becomes active when embedding lift into
the compression process.

Because of the nonlinear nature of the dynamics, flow around the airfoil
responds differently to each different gust vortex with massive flow
separation and large-scale formation of additional vortices, as visualized in
Fig. 2. There is no aerodynamic model or theory that can easily describe the
highly nonlinear nature of the extreme gust-airfoil interactions. What is
especially challenging is that there is no simple scaling that collapses the
collection of lift curves or the vortical flow fields due to the strong
nonlinearity. This is an enormous burden for studying extreme aerodynamic
flows since each and every case needs to be examined through painstaking
computational or experimental campaigns that require significant resources. We
re-emphasize that even for the present problem setup, there are a good number
of parameters (vortex strength $G$, size $L$, position $y_{0}/c$, and airfoil
angle of attack $\alpha$) that necessitate a very large number of cases of
extreme aerodynamic flows to map out the response dynamics. Practically
speaking, such a campaign may not be possible for all gust encounters with
limited computational resources. Therefore, it is desirable to capture the
underlying dominant dynamics that form the basis of extreme gust response
characteristics without having to rely on expensive simulations with a very
large degree of freedom, which in this case is proportional to $2.88\times
10^{4}$ grid (spatial) points to describe the instantaneous flow field and
$1.26\times 10^{5}$ temporal frames for sufficient spatiotemporal solutions
for all cases. While we do have the Navier–Stokes equations as the governing
partial differential equations to fully describe the dynamics, solving them in
real time for practical air vehicle operations is out of the question.

For the aforementioned reasons, it is important to extract the dominant low-
dimensional dynamics from the possible collection of extreme aerodynamic data
sets. The rich responses to different gust vortices appear uniquely different
from one snapshot to another but possess some common and identifiable
features, including the disturbance vortex, flow separation, wake vortices,
secondary vortices, shear layers, vortex roll-up, vortex pinch-off, and vortex
deformation. The fact that these features are indeed identifiable by the
trained eyes of fluid dynamicists suggests that there is likely some
underlying low-dimensional representation of the high-dimensional complex
dynamics. Thus, we aim to capture the key dynamics in a space comprised of a
very small number of variables that can estimate the full state of the flow
field and offer insights into the nonlinear dynamics of the vortex-gust
interaction. We find that a nonlinear autoencoder with physical observables
illustrated in Fig. 3 achieves the present objective.

### Identification of Extreme Aerodynamic Manifold

To find a low-dimensional space that captures the essential physics of extreme
aerodynamic interactions between the gust vortex and the airfoil wake, we
perform data-driven compression of the flow field. Herein, we consider cases
with randomly-sampled parameters from $G\in[-4,4]$, $L\in[0.5,2]$, and
$y_{0}/c\in[-0.5,0.5]$.

First, let us consider the most commonly used linear dimensionality reduction
technique, namely the principal component analysis (PCA), which is also known
as the proper orthogonal decomposition (POD) [30, 31, 32]. With this method,
the low-dimensional representation of a fluctuating variable is found by
identifying the primary modes (or vectors) that best capture the variance
about the mean. In the present study, we first analyze the vorticity fields
over $(x,y)/c\in[-1.4,4]\times[-1.2,1.2]$, shown in Fig. 2 by applying PCA to
determine the most vortically energetic components of the flow field.

Figure 4:  Compression of extreme aerodynamic flow data using (left) PCA
(linear autoencoder), (middle) nonlinear convolutional autoencoder, and
(right) lift-augmented nonlinear convolutional autoencoder. The undisturbed
cases are highlighted in color while all disturbed wake cases used for
training are shown with light gray curves. The lift-augmented convolutional
autoencoder identifies the extreme gust-airfoil interaction manifold.

Shown in Fig. 4 are the temporal variations of the first three PCA components
($\xi_{1}(t)$, $\xi_{2}(t)$, and $\xi_{3}(t)$) of the vorticity field data.
The gray curves represent the whole collection of extreme aerodynamic data
plotted in this coordinate space. Also highlighted are undisturbed baseline
cases for $\alpha=20^{\circ}$ to $60^{\circ}$. Here, we observe that the gray
curves span a range of values over $\xi_{1}$, $\xi_{2}$, and $\xi_{3}$ in a
seemingly incoherent manner. What is further problematic with this compression
is the overlap of the baseline cases. These observations reveal that PCA
struggles to compress the extreme aerodynamic data in a meaningful manner
while keeping different angles of attack cases distinct. Because different
flows over different angles of attack are collapsed as the same, the
overlapping low-dimensional representations produced by PCA cannot distinguish
important flow characteristics and yield grossly inaccurate flow
reconstructions, as shown in Fig. S.1.

The challenges of reducing degrees of freedom (dimension) of extreme vorticity
dynamics by PCA can be mitigated by utilizing a nonlinear compression
technique. For this purpose, we utilize a nonlinear convolutional autoencoder,
presented in Fig. 3 (excluding the green-shaded side network), to reduce a
large number of vorticity field data to very few variables. An autoencoder is
a neural network composed of an encoder and a decoder with a bottleneck in the
middle [33, 34, 35]. Generally, this neural network framework is used to take
an input data and replicate the same data at the output. The variables that
lie in the middle are referred to as latent variables (red circles in Fig. 3),
which hold the compressed information about the input data. When the nonlinear
autoencoder can replicate the same input data at the output, this means that
both the encoder and the decoder function effectively to nonlinearly transform
the full data set to a low-dimensional latent variable ${\bm{\xi}}$ and vice
versa effectively. The present autoencoder first compresses a flow field using
a convolutional neural network (CNN) [36] to capture global features of the
vortical flow field. The compressed vector (extracted feature) through the CNN
is then flattened at the reshape layer in Fig. 3 to pass into a multi-layer
perceptron (MLP) [37] towards the latent space. A similar operation is
performed for the decoder side to expand the dimension of the latent variable
back to the size of the original flow field. The present autoencoder is
trained with the same data sets as that used for PCA. The details of the
autoencoder setup used in the present study are provided in Supplemental
Material.

The nonlinear autoencoder is able to compress the flow field data and
reproduce the flow field accurately, as shown in Fig. S.1. We also present the
latent space comprised of only three latent variables ($\xi_{1}$, $\xi_{2}$,
and $\xi_{3}$) in Fig. 4 (middle). The ability of the nonlinear autoencoder to
compress the vorticity field to mere three variables is not only surprising
but also reaffirms that the flow field is indeed comprised of common flow
features. The compression capability offered by a nonlinear autoencoder is
promising to capture violent flow physics that appears tremendously rich.
Nonetheless, we should note that the full data set shown in Fig. 4 is
distributed over the latent space without a meaningful collapse of the latent
variables. This is sufficient if the objective is to ensure that data in
latent space are distinct, thus covering as much space as necessary to ensure
uniqueness of the information. However, with regard to this study, we are
aiming not only to compress the extreme aerodynamic flow data but also to
identify universal features among the large number of flow field data holding
dynamical information.

Figure 5:  The lift-augmented autoencoder-based manifold expression and its
output for a variety of wakes around a wing. Interpolation ($|G|<4$) and
extrapolation ($|G|>4$) cases with regard to training data are shown. The
trajectories for the undisturbed cases are shown in light color in latent
space. The value presented on each decoded flow field corresponds to the time-
averaged structural similarity index between the decoded and reference fields.

The above autoencoder analysis was performed purely from a data-centric
perspective. Instead, let us consider incorporating a physical measurement
(observable) into the autoencoder to facilitate the identification of a low-
dimensional subspace defined by the appropriate latent variable coordinates.
Capturing the low-dimensional nature of extreme aerodynamic flows can support
the flight stabilization of air vehicles in extreme levels of turbulence. For
this reason, we weigh the latent space variables $\bm{\xi}$ with lift force
acting on the wing. This is achieved by supplementing the autoencoder with a
multi-layer perceptron that outputs the lift force for each vorticity field
over time, as shown by the green-shaded network in Fig. 3. In this case,
training is performed to compress the vorticity field to the latent variables
and to estimate the lift force accurately from the latent variables.

Incorporating lift into the learning process of the autoencoder assists in
effectively extracting the essence of extreme aerodynamics due to three main
reasons. First, using the lift force with the vorticity input comes as a
natural choice since vorticity is theoretically known as the main mechanism
for generating lift on a body. We take advantage of this intimate relationship
between the vorticity field and lift. Second, the latent variables are guided
to retain important information held by the vorticity field correlated with
the lift force. This means that vortical structures that apply large vortical
forces are well-captured by the lift-augmented autoencoder. Because extreme
aerodynamic disturbances exert enormous amounts of transient forces as
presented in Fig. 2, the autoencoder weights these extreme event appropriately
and is able to accurately identify the responsible vortical structures. Third,
the latent variables are encouraged to distinguish cases that yield different
lift responses to impinging gust vortices. This is crucial to avoid latent
variables from overlapping unnecessarily as observed in the struggling cases
of PCA.

Now, let us examine the compression results from the lift-augmented
autoencoder. The latent space comprised of three variables $\xi_{1}$,
$\xi_{2}$, and $\xi_{3}$ is presented in Fig. 4. We observe that the entire
collection of extreme aerodynamic cases collapses well in these latent space
coordinates, confirming that the extreme aerodynamic responses to gust
vortices possess a fundamentally low-dimensional behavior if captured
appropriately with a nonlinear compression method. Here, the asymptotic
periodic shedding states of the airfoil wakes provide shows a “cone” or a
“chocolate cornet” like structure with the extreme aerodynamic trajectories
lying in its vicinity in the three-dimensional latent space. As the dynamical
trajectories converge to the cone-shaped structure, this structure serves as
the inertial manifold [38, 39, 40]. This manifold geometry can be considered
as an hour-glass shape since there is a mirrored manifold for negative angle
of attack cases. It should be noted that the geometry of this structure is not
specified a priori and is discovered in an unsupervised manner. Here, this
surface constitutes a manifold on which the key dynamics of extreme gust
response reside. That is, the trajectories of the presently considered extreme
aerodynamic flows are mapped onto the discovered geometry or to its vicinity.

Given the lift-augmented autoencoder collapsing all extreme aerodynamic
response data onto this cone-shaped manifold, let us examine the accuracy of
state reconstruction for the disturbed flow and lift based on the three latent
variables $(\xi_{1},\xi_{2},\xi_{3})$. As representative examples, we present
the performance of the autoencoder for the cases of
$(\alpha,G,L,y_{0}/c)=(40^{\circ},-2.2,0.5,0.3)$ and
$(60^{\circ},-2.8,1.5,0)$, which are unused in training but chosen from the
training parameter range. Here, the gust ratios $G$ for these two cases are
much higher than what are traditionally considered in gust response studies,
but are within the training data range of $|G|\leq 4$. The latent variable
trajectories for these cases and the reconstruction of lift and vortical flows
are also shown in Fig. 5. To assess the reconstruction performance, we
evaluate the structural similarity index (SSIM) [41] between the reference and
the decoded flow fields. The SSIM value for each decoded flow field is listed
under the visualized flow reconstruction. Even the flow states exhibiting
nonlinear interactions between the wing and the extreme vortex gust can be
reconstructed well by the present autoencoder, as presented in Figs. 5 and
S.1. Note that the structural similarity index for a regular autoencoder
without lift being higher compared to that for the lift-augmented autoencoder
is expected. This is because a regular autoencoder is able to tune its weights
solely to obtain accurate reconstruction of the flow field from the latent
variables. It is also possible to reconstruct lift solely from the vorticity
field. However, the lift-augmented autoencoder is critical for revealing the
manifold for extreme aerodynamic response dynamics. These successful
reconstructions indicate that high-dimensional extreme aerodynamic flows can
be compressed into only three variables without significant loss of key
physics.

The trajectory in the present latent space for the disturbed cases reflects
key features of nonlinear vortex-gust interaction appearing in the high-
dimensional space. For the extreme aerodynamic case of $\alpha=40^{\circ}$,
the latent vector first drops towards the direction of the undisturbed
periodic orbit of $\alpha=30^{\circ}$ then comes back to the original
undisturbed orbit of $\alpha=40^{\circ}$. This is due to the approach of
negative vortex disturbance to the wing, decreasing the effective angle of
attack. This indicates that the lift-augmented autoencoder captures the
relationship between high-dimensional extreme aerodynamic flows and lift force
in the low-order space. In fact, the reduction in $\xi_{3}$ towards the
direction of $\alpha=30^{\circ}$ in the latent space coincides with the
temporal evolution of lift responses, as shown in Fig. 4. A similar trend is
also observed in the case of $\alpha=60^{\circ}$ in which the latent vector
first heads to the direction of the periodic orbit of $\alpha=50^{\circ}$
corresponding to the decrease of the lift response. Being able to capture the
extreme aerodynamic response of the wing on this manifold enables us to relate
the instantaneous dynamics to the effective angle of attack, which is
critically important for the flight stability of air vehicles. It is
particularly encouraging that only three values in latent space are required
to accurately estimate the state of the violent flow around the wing and
transient lift force, which is promising for future development of sensors.

Figure 6:  Extrapolation assessments of the present nonlinear autoencoder with
lift augmentation. Three extrapolating cases are considered: 1. noisy data, 2.
two vortex gusts (vertically arranged), and 3. two vortex gusts (horizontally
arranged). The initial condition for the cases with two vortices is
illustrated in the plot of each latent space. The trajectories for the
undisturbed cases are shown in light color in latent space. The value
presented on each decoded flow field corresponds to the time-averaged
structural similarity index between the decoded and reference fields.

The discovered manifold captures dynamics beyond the trained gust strength
$G$. Let us demonstrate how the present autoencoder approach is able to
capture even more severe vortex gust conditions with $|G|\geq 4$ by presenting
two cases $(\alpha,G,L,y_{0}/c)=(40^{\circ},6.4,1.0,0.1)$ and
$=(60^{\circ},-6.0,2.0,-0.3)$, as shown in Fig. 5. The trajectory of these
seemingly extrapolative cases exhibits a larger radius on the
$\xi_{1}-\xi_{2}$ plane compared to the variables of the interpolation cases
while also presenting the effective angle of attack as the latent vector moves
in the $\xi_{3}$ direction. The wider radial trajectory is due to the stronger
disturbance, which produces a higher level of fluctuations in the vortical
flow response likely away from the body without affecting lift. The present
decoder can also recover the high-dimensional flow states while estimating the
very large transient lift dynamics as presented in Fig. 5. This indicates that
the present autoencoder can be robustly applied for such extreme gust vortex-
airfoil interactions while achieving a nearly lossless compression of high-
dimensional data. These findings also suggest that even under the
extrapolating condition of $|G|>4$, the underlying vortex dynamics shares
common physics and can be nonlinearly compressed to a low-dimensional and
universal manifold. This provides hope in estimating the flow states for cases
that were not part of the extreme aerodynamic training data.

To further examine the robustness of the identified manifold and the
autoencoder, let us consider cases that are different from the training cases,
namely flows with noise and two extreme vortices, as shown in Fig. 6. Here,
the noisy flow field is generated by adding Gaussian noise that is 30% of the
original extreme aerodynamic flow field (same as the case shown in Fig.
5$(b)$). The present lift-augmented autoencoder not only reconstructs a
vortical flow but also estimates the lift response well from a noisy flow. The
autoencoder noise rejection characteristics is beneficial in obtaining real-
time situational awareness and working with turbulent flows in which less
influential smaller-scale structures may also be present around the extreme
gust vortices.

We also consider cases of vortex-dominated gust flows that are challenging for
most reconstruction techniques trained only with single-gust disturbances.
Here, we take two vortices that are introduced vertically and horizontally
upstream of the airfoil, as depicted in Fig. 6. In both cases, the dynamical
lift responses can be accurately estimated while the decoder reproduces the
presence of two vortex disturbances very well, as shown in Figs. 6 and S.2. We
can also notice from the latent space that the trajectory of the two-
horizontal-vortex case presents two inflections at $\xi_{2}\approx 0$. This
coincides with the observation in the lift dynamics which possesses two
valleys due to the impingement of two negative vortices. We note that these
particular examples are difficult for linear techniques, including PCA which
completely fails to reconstruct the flow field as shown in Fig. S.2.

Figure 7:  Application of the present nonlinear autoencoder with lift
augmentation to extreme gust disturbance situation with multiple vortices,
simulating severe wake turbulence striking the airfoil. The initial condition
with the gust parameters for each vortex is provided (top left). The decoded
lift history and the latent space dynamics are shown (top right). The
trajectories for the undisturbed cases are shown in light color in latent
space. The value presented on the decoded flow field corresponds to the time-
averaged structural similarity index between the decoded and reference fields
(bottom).

Finally, let us demonstrate the potential of the present lift-augmented
autoencoder for handling a more challenging and realistic extreme flight
condition. In this last example, we introduce randomly generated five strong
vortices upstream of the wing to simulate severe wake turbulence striking the
airfoil, as shown in Fig. 7 (top left). The decoded lift, reconstructed flow
fields, and latent trajectory are presented in Fig. 7. Even under this extreme
operating condition, the present autoencoder robustly provides accurate
reconstruction of the flow variables despite the model being trained only with
single-gust disturbances. This success in flow compression/reconstruction and
lift estimation corroborates that the discovered low-dimensional manifold
universally captures the extreme gust vortex-airfoil interaction dynamics.
This discovery provides great hope in establishing flight in extreme gusts,
which was traditionally considered impossible.

## Discussion

Small-scale air vehicles flying in urban or mountainous environments need to
maneuver through a highly unsteady wake field full of strong vortices
generated by manmade or natural obstacles. The interaction of these vortices
with flying vehicles requires an understanding of extreme aerodynamic flows,
for which there are no established theories. In the current study, we
presented a data-driven approach to identify a low-dimensional manifold on
which the key dynamics between strong gust vortices and airfoil wakes can be
collapsed. This manifold was found with an autoencoder designed to retain the
knowledge of aerodynamic lift as part of the latent variables. The existence
of this low-dimensional manifold is significant in a few ways.

First, the fact that only three variables can represent the complex vortical
flow field confirms the low-dimensional nature of the strong gust vortices
interacting with the airfoil wakes. While the present study distilled the
dynamics to only three variables, it actually can be further reduced to two
variables if the three variables are projected on the identified manifold.
This significant compression of the extreme aerodynamic flow fields was
enabled with a nonlinear autoencoder-based approach that incorporates
aerodynamic insights embedded into its formulation. We also note that noisy
experimental data encountering a different type of gust can also be
coincidentally low-dimensionalized to be three-dimensional variables through a
nonlinear autoencoder with the assistance of a topology-based concept [42].
Second, this low-dimensional representation of the extreme aerodynamic flows
suggests that only a small number of sensors on the airfoil may be able to
accurately reconstruct the surrounding flow field in real time. In fact,
decoder-type neural networks, that take sparse sensors as the input and high-
resolution aerodynamic flows as the output, have been recently developed to
perform real-time fluid flow state estimation [43, 44, 45, 46]. The
observations in these studies imply that low-dimensional extreme aerodynamic
latent vectors can also be estimated from sparse sensor information, enabling
us to track the high-dimensional dynamics in a low-order, real-time manner.
Third, given the present findings, it is possible to develop a reduced-order
model that can capture the dynamics in the latent space to desired level of
accuracy and complexity. While we could model the latent dynamics using other
data-driven techniques such as sparse regression [47], modeling and
controlling the high-dimensional extreme aerodynamic flows on the present
manifolds from the perspective of phase-amplitude space appears interesting
[48, 49, 50]. It can be anticipated the phase-reduction analysis on the
present nonlinear manifold offers a new aspect in modifying wake dynamics by
providing the optimal timing and locations of actuation.

With the discovered manifold, active flow control and vehicle stability
control strategies can be developed for mitigating the effects of extreme
aerodynamic disturbances. While this study focused on extreme two-dimensional
vortex-airfoil interactions, there are other types of gust disturbances
present in severe atmospheric turbulence that require three-dimensional
analysis [9]. Three-dimensional flow extensions are important to further
examine the potential of the present approach. As shown in this study,
nonlinear data-driven compression techniques appear promising to support the
identification of other manifolds that capture the complex extreme aerodynamic
interactions between these other types of gusts and the aircraft. The present
findings offer a new perspective on modeling and taming extreme aerodynamic
flows in support of next-generation air vehicle operations in conditions
traditionally considered unflyable.

## Methods

### Simulations of Extreme Vortex-Airfoil Interactions

The present study considers the unsteady flow field generated by the extreme
gust vortex-airfoil interactions. The flow fields examined in this study are
obtained from direct numerical simulations of flows over a NACA 0012 airfoil
at a chord-based Reynolds number $\text{Re}\equiv u_{\infty}c/\nu=100$. Here,
$u_{\infty}$ is the free-stream velocity, $c$ is the chord length, and $\nu$
is the kinematic viscosity. The simulations are performed with an
incompressible flow solver [26, 27] for an airfoil at six different angles of
attack of $\alpha=20^{\circ}$, $30^{\circ}$, $40^{\circ}$, $50^{\circ}$, and
$60^{\circ}$. For the undisturbed cases, the flow at $\alpha=20^{\circ}$ is
steady while that at $\alpha\geq 30^{\circ}$ exhibits unsteady periodic wake
shedding. The computational domain extends over
$(x,y)/c\in[-15,30]\times[-20,20]$ with the leading edge of the wing
positioned at the origin.

To study the gust-airfoil interactions, a very strong vortex with an angular
velocity profile prescribed by equation [28] is introduced upstream of the
airfoil at $x_{0}/c=-2$ and $y_{0}/c\in[-0.5,0.5]$. The present disturbance
vortex is parameterized by the gust ratio $G\equiv
u_{\theta,\text{max}}/u_{\infty}\in[-10,10]$, its size $L\equiv
2R/c\in[0.5,2]$, and the vertical position of the disturbance $y_{0}/c$. From
the parameter space composed of these three variables, 40 randomly-sampled
cases of the disturbed flows are simulated for each angle of attack. For the
purpose of learning the extreme aerodynamics with the autoencoder, 20 cases
are used for training and the remaining 20 cases are used for testing. The
simulated flows were validated with previous studies [45, 51, 52, 53], in
particular with a study that considered a vortex-airfoil interaction problem
[45].

For each of the cases considered in the present study, we prepare 1200
snapshots of vorticity field over 10.2 non-dimensional convective time
$t^{*}\equiv u_{\infty}t/c$. We refer to this convective time as simply ‘time’
in the main text. Of the entire flow field, a subdomain
$(x,y)/c\in[-1.4,4]\times[-1.2,1.2]$ with spatial grid points
$(N_{x},N_{y})=(240,120)$ is considered for the data-driven analysis since
vortex-airfoil interactions primarily occur in this region. Moreover, the
history of the lift fluctuations is provided by the numerical simulations. The
non-dimensional lift coefficient $C_{L}\equiv{F_{\rm lift}}/{(\frac{1}{2}\rho
u_{\infty}^{2}c)}$, where $F_{\rm lift}$ is the lift force on the wing body
and $\rho$ is the density. In the main text, $C_{L}$ is referred to as ‘lift.’
Overall, the training data used for the present models amounts to $1.26\times
10^{5}$ frames comprised of 100 extreme aerodynamic gust response cases and 5
undisturbed wake cases with 1200 snapshots for each case.

### Autoencoder Setup

To discover the universal nonlinear manifold that represents the high-
dimensional extreme aerodynamic flows in a low-dimensional latent space, we
use an autoencoder [33] (see Fig. 3). Here, we consider a convolutional
neural-network-based autoencoder $\cal F$, which is trained to output
$\hat{\bm{q}}$ to be the same data as the input ${\bm{q}}\in{\mathbb{R}}^{n}$
such that $\hat{\bm{q}}\approx{\cal F}({\bm{q}};{\bm{w}})$, where ${\bm{w}}$
denotes the weights inside the autoencoder. This autoencoder is comprised of
an encoder ${\cal F}_{e}$ and a decoder ${\cal F}_{d}$ connected through a
low-dimensional variable $\bm{\xi}\in\mathbb{R}^{m}$ in the middle, where
$m\ll n$. Here, the high-dimensional input ${\bm{q}}$ can be compressed into
the latent vector ${\bm{\xi}}$ if the autoencoder $\cal F$ successfully
recovers the data accurately. That is, we seek to have an autoencoder that
achieves

${\bm{q}}\approx\hat{\bm{q}}={\mathcal{F}}({\bm{q}};{\bm{w}})={\mathcal{F}}_{d}(\bm{\xi})={\mathcal{F}}_{d}({\mathcal{F}}_{e}({\bm{q}})).$
(1)

The autoencoder $\cal F$ is found based on data such that its weights $\bm{w}$
are optimized to minimize a desired cost (loss) function $\cal E$, yielding
the following optimization problem

${\bm{w}}={\rm argmin}_{\bm{w}}[{\cal
E}({\bm{q}},{\mathcal{F}}({\bm{q}};{\bm{w}}))]={\rm
argmin}_{\bm{w}}\|{\bm{q}}-\hat{\bm{q}}\|_{2}.$ (2)

The weights ${\bm{w}}$ are determined with the Adam optimizer [54].

We use an autoencoder composed of convolutional neural networks (CNN) [36] and
multi-layer perceptrons (MLP) [37], as illustrated in Fig. 3. In the encoder,
the CNN captures global features of the extreme aerodynamic flow field and the
MLP is used to extract features from the CNN while further reducing the size
of the data. By leveraging nonlinear activation functions, an autoencoder can
achieve better compression than linear compression techniques such as
principal component analysis (PCA) [33]. Note that using autoencoder with
linear activation functions is mathematically equivalent to performing
principal component analysis (PCA) [33, 34]. As for the nonlinear activation
function, we use the hyperbolic tangent function
$\varphi(s)=(e^{s}-e^{-s})/(e^{s}+e^{-s})$, enabling us to consider the
positive and the negative gust influence in latent space. The hyperparameters
used in the MLP and CNN follow previous work with similar settings [34, 55].
Full details on the parameters of the present convolutional nonlinear
autoencoder are shown in table S.1.

In addition to PCA and a regular autoencoder, we develop a lift-augmented
convolutional autoencoder in this study. The present lift-augmented
autoencoder trains the model with a lift coefficient $C_{L}(t)$ in addition to
a vorticity field ${\bm{q}}(t)$ such that
$[\hat{\bm{q}}(t),\hat{C_{L}}(t)]={\cal F}({\bm{q}}(t))$. The additional side
network based on an MLP is illustrated in the green-shaded portion of Fig. 3.
This additional network ensures that the latent vector ${\bm{\xi}}(t)$ holds
relevant information related to the lift coefficient $C_{L}(t)$ to support the
manifold identification. The cost function in this case becomes

$\displaystyle{\bm{w}}^{*}={\rm
argmin}_{\bm{w}}\biggl{[}||{\bm{q}}-\hat{\bm{q}}||_{2}+\beta||C_{L}-\hat{C_{L}}||_{2}\biggr{]},$
(3)

where $\beta$ balances the vorticity field and lift reconstruction losses. In
this study, we choose $\beta=0.05$ based on the L-curve analysis [56]. With
the lift decoder ${\cal F}_{L}$, the reconstructed lift coefficient
$\hat{C_{L}}(t)$ is given by

$\displaystyle{\hat{C_{L}}(t)}={\cal F}_{L}({\bm{\xi}}(t))={\cal F}_{L}({\cal
F}_{e}({{\bm{q}}(t)})).$ (4)

## Appendix

### Autoencoder Setup

We use an autoencoder composed of convolutional neural networks (CNN) [36] and
multi-layer perceptrons (MLP) [37], as illustrated in Fig. 3 in the main text.
Full details on the parameters of the present convolutional nonlinear
autoencoder are shown in table 1.

Table 1:  The network structure of the present lift-augmented convolutional
autoencoder. The size of the convolutional filter $L_{c}$ and the number of
the filter $K$ are shown for each convolutional layer as $(L_{c},L_{c},K)$.
The maxpooling/upsampling ratio $M$ is shown for each maxpooling/upsampling
layer as $(M,M)$.
Encoder | Decoder | Lift decoder
---|---|---
Layer | Data size | Layer | Data size | Layer | Data size
Input | (240, 120, 1) | Fully connected | (16) | Fully connected | (32)
| Conv2D
---
(3, 3, 32)
(240, 120, 32) | Fully connected | (32) | Fully connected | (64)
| Conv2D
---
(3, 3, 32)
(240, 120, 32) | Fully connected | (64) | Fully connected | (32)
| MaxPooling2D
---
(2, 2)
(120, 60, 32) | Fully connected | (256) |  | Output 2
---
(Lift coefficient)
(1)
| Conv2D
---
(3, 3, 16)
(120, 60, 16) | Fully connected | (288) |  |
| Conv2D
---
(3, 3, 16)
(120, 60, 16) | Reshape | (12, 6, 4) |  |
| MaxPooling2D
---
(2, 2)
(60, 30, 8) |  | Conv2D
---
(3, 3, 4)
(12, 6, 4) |  |
| Conv2D
---
(3, 3, 8)
(60, 30, 8) |  | Conv2D
---
(3, 3, 4)
(12, 6, 4) |  |
| Conv2D
---
(3, 3, 8)
(60, 30, 8) |  | UpSampling2D
---
(5, 5)
(60, 30, 4) |  |
| MaxPooling2D
---
(5, 5)
(12, 6, 8) |  | Conv2D
---
(3, 3, 8)
(60, 30, 8) |  |
| Conv2D
---
(3, 3, 4)
(12, 6, 4) |  | Conv2D
---
(3, 3, 8)
(60, 30, 8) |  |
| Conv2D
---
(3, 3, 4)
(12, 6, 4) |  | UpSampling2D
---
(2, 2)
(120, 160, 8) |  |
Reshape | (288) |  | Conv2D
---
(3, 3, 16)
(120, 160, 16) |  |
Fully connected | (256) |  | Conv2D
---
(3, 3, 16)
(120, 160, 16) |  |
Fully connected | (64) |  | UpSampling2D
---
(2, 2)
(240, 120, 16) |  |
Fully connected | (32) |  | Conv2D
---
(3, 3, 32)
(240, 120, 32) |  |
Fully connected | (16) |  | Conv2D
---
(3, 3, 32)
(240, 120, 32) |  |
| Fully connected
---
(Latent vector)
(3) |  | Output 1
---
(Vorticity field)
(240, 120, 1) |  |

### Reconstruction of vorticity fields

Figure 8:  Reconstructed vorticity fields by PCA, a regular autoencoder, and
the present lift-augmented autoencoder for the cases of (a)
$(\alpha,G,L,y_{0}/c)=(40^{\circ},-2.2,0.5,0.3)$ and (b)
$(60^{\circ},-2.8,1.5,0)$. The values listed on the decoded flow fields are
the time-averaged structural similarity indices between the decoded and
reference fields.  Figure 9:  Reconstructed vorticity fields by PCA and the
present lift-augmented autoencoder for the cases of two gust vortices that are
$(a)$ vertically arranged and $(b)$ horizontally arranged. The values listed
on the decoded flow fields are the time-averaged structural similarity indices
between the decoded and reference fields.

Comparative assessment of the reconstructed flow fields using different
techniques is offered in this section. Here, we compare the reconstructed
vorticity field from PCA, a regular autoencoder, and the lift-augmented
autoencoder. To quantitatively assess the reconstruction performance, we
evaluate the accuracy of the structural similarity index (SSIM) [41] between
the reference and the decoded flow fields. The SSIM $\chi$ is defined as

$\displaystyle\chi^{\prime}=\frac{(2\mu_{r}\mu_{d}+C_{1})(2\sigma_{rd}+C_{2})}{(\mu_{r}^{2}+\mu_{d}^{2}+C_{1})(\sigma_{r}^{2}+\sigma_{d}^{2}+C_{2})}.$
(5)

Here, subscripts $r$ and $d$ are used for the reference and decoded flow
fields, $\mu$ and $\sigma$ are the mean and standard deviation of given data,
respectively, $\sigma_{rd}$ is the covariance of the reference and decoded
flow fields, and the constants $(C_{1},C_{2})=(0.16,1.44)$ are used to
stabilize division [41]. The SSIM $\chi$ lies between 0, representing no
similarity, and 1, representing an identical image. The value of the SSIM is
provided on every reconstructed vorticity field plot of the main text.

In this supplemental material, we present two cases of
$(\alpha,G,L,y_{0}/c)=(40^{\circ},-2.2,0.5,0.3)$ and $(60^{\circ},-2.8,1.5,0)$
in Figs. 8$(a)$ and $(b)$, respectively. We compare the performance of PCA,
regular autoencoder, and the lift-augmented autoencoder all with the number of
the latent vector being set to 3. As evident from these figures, PCA struggles
to reconstruct high-dimensional states over time for both cases of extreme
aerodynamic flows. PCA completely fails to detect the incoming gust vortex. In
addition to the failure of reconstruction for the presence of the vortex
disturbance upstream of the wing and the wake structures, limitations of
linear reconstruction techniques are also observed around the wing with
superpositions of wings appearing incorrectly.

In contrast, the reconstruction can be greatly improved with a nonlinear
convolutional autoencoder, as shown in Fig. 8. The position of extreme vortex
disturbance is estimated very accurately over time including when a vortex
gust impinges the wing causing strong nonlinear interactions among the wing,
gust, and wake. This observation shows that the collection of the present
extreme aerodynamic flow fields can be significantly compressed into only
three variables using a nonlinear autoencoder. The improvement of the
reconstruction by the nonlinear model can also be observed with an example of
disturbed flow fields with two extreme gust vortices.

The impingement with these multiple gust vortices causes challenges to stable
flight with extreme levels of nonlinear lift responses than single-gust cases.
These unseen situations cannot be generally analyzed with conventional linear
analysis. The reconstructed flow fields with two different types of vortex
gust placements, (1) vertical and (2) horizontal arrangements, using PCA and
the lift-augmented autoencoder are shown in Fig. 9. As expected, PCA cannot
detect the presence of two vortices and also completely fails to capture the
influence of extreme disturbance on the wake over time. On the other hand, the
present lift-augmented autoencoder is able to reconstruct the flow states,
achieving over 90% SSIM for both examples. The ability to reconstruct such
unseen complex gust situations based on a variety of single-vortex disturbed
flow training data suggests that the discovered manifold space is indeed
capturing the underlying extreme aerodynamic responses in not only a low-
dimensional manner but also in a universal fashion. It is hence anticipated
that the present manifold expression can be extended to wings flying into a
wide range of extreme vortex-dominated gusts.

## Data availability

Extreme aerodynamic data [57] used in the present study are available on the
Open Science Framework (https://doi.org/10.17605/OSF.IO/7VSH8). Source data
are provided with this paper, which is also available on the same link above.

## Code availability

Sample codes [58] for training the present models are available on GitHub
(https://github.com/kfukami/Observable-AE and
https://zenodo.org/badge/latestdoi/677509021 ).

## References

  * [1] Anderson, J. D.  _Fundamentals of Aerodynamics_ (McGraw Hill, 2016), 6th edn.
  * [2] Katz, J. & Plotkin, A.  _Low-speed aerodynamics_ , vol. 13 (Cambridge university press, 2001).
  * [3] Leishman, G. J.  _Principles of helicopter aerodynamics_ (Cambridge university press, 2006).
  * [4] Pines, D. J. & Bohorquez, F.  Challenges facing future micro-air-vehicle development.  _J. Aircraft_ 43, 290–305 (2006).
  * [5] Sage, A. T. _et al._ Testing the delivery of human organ transportation with drones in the real world.  _Sci. Robot._ 7, eadf5798 (2022).
  * [6] Mohamed, A., Abdulrahim, M., Watkins, S. & Clothier, R.  Development and flight testing of a turbulence mitigation system for micro air vehicles.  _J. Field Robot._ 33, 639–660 (2016).
  * [7] Prudden, S. _et al._ Measuring wind with small unmanned aircraft systems.  _J. Wind. Eng. Ind. Aerodyn._ 176, 197–210 (2018).
  * [8] Gavrilovic, N. _et al._ Avian-inspired energy-harvesting from atmospheric phenomena for small UAVs.  _Bioinspir. Biomim._ 14, 016006 (2018).
  * [9] Fernando, H. J. S. _et al._ C-FOG: life of coastal fog.  _Bull. Am. Meteorol. Soc._ 102, E244–E272 (2021).
  * [10] Gross, G.  On the estimation of wind comfort in a building environment by micro-scale simulation.  _Meteorol. Z._ 23, 51–62 (2014).
  * [11] Watkins, S. _et al._ Ten questions concerning the use of drones in urban environments.  _Build. Environ._ 167, 106458 (2020).
  * [12] Garrow, L. A., German, B. J. & Leonard, C. E.  Urban air mobility: A comprehensive review and comparative analysis with autonomous and electric ground transportation for informing future research.  _Transp. Res. Part C Emerg. Technol._ 132, 103377 (2021).
  * [13] Los Angeles Department of Transportation: Urban air mobility policy framework considerations.  https://ladot.lacity.org/sites/default/files/documents/ladot-uam-policy-framework-considerations.pdf (2021).  Accessed on 20-April-2023.
  * [14] Saitoh, T. _et al._ Real-time breath recognition by movies from a small drone landing on victim’s bodies.  _Sci. Rep._ 11, 5042 (2021).
  * [15] Yoo, W., Yu, E. & Jung, J.  Drone delivery: Factors affecting the public’s attitude and intention to adopt.  _Telemat. Inform._ 35, 1687–1700 (2018).
  * [16] Kugler, L.  Real-world applications for drones.  _Commun. ACM_ 62, 19–21 (2019).
  * [17] Floreano, D. & Wood, R. J.  Science, technology and the future of small autonomous drones.  _Nature_ 521, 460–466 (2015).
  * [18] Jones, A. R., Cetiner, O. & Smith, M. J.  Physics and modeling of large flow disturbances: Discrete gust encounters for modern air vehicles.  _Annu. Rev. Fluid Mech._ 54, 469–493 (2022).
  * [19] Gao, M. _et al._ Weather constraints on global drone flyability.  _Sci. Rep._ 11, 12092 (2021).
  * [20] De Croon, G. C. _et al._ Accommodating unobservability to control flight attitude with optic flow.  _Nature_ 610, 485–490 (2022).
  * [21] Di Luca, M., Mintchev, S., Su, Y., Shaw, E. & Breuer, K.  A bioinspired separated flow wing provides turbulence resilience and aerodynamic efficiency for miniature drones.  _Sci. Robot._ 5, eaay8533 (2020).
  * [22] Mohamed, A., Marino, M., Watkins, S., Jaworski, J. & Jones, A.  Gusts encountered by flying vehicles in proximity to buildings.  _Drones_ 7, 22 (2023).
  * [23] Hoblit, F. M.  _Gust loads on aircraft: concepts and applications_ (AIAA, 1988).
  * [24] Stutz, C. M., Hrynuk, J. T. & Bohl, D. G.  Dimensional analysis of a transverse gust encounter.  _Aerosp. Sci. Technol._ 137, 108285 (2023).
  * [25] Jones, A. R. & Cetiner, O.  Overview of unsteady aerodynamic response of rigid wings in gust encounters.  _AIAA J._ 59, 716–721 (2021).
  * [26] Ham, F. & Iaccarino, G.  Energy conservation in collocated discretization schemes on unstructured meshes.  In _Annual Research Briefs_ , 3–14 (Center for Turbulence Research, 2004).
  * [27] Ham, F., Mattsson, K. & Iaccarino, G.  Accurate and stable finite volume operators for unstructured flow solvers.  In _Annual Research Briefs_ , 243–261 (Center for Turbulence Research, 2006).
  * [28] Taylor, G. I.  On the dissipation of eddies.  _Meteorology, Oceanography and Turbulent Flow_ 96–101 (1918).
  * [29] Wu, J.-Z., Ma, H.-Y. & Zhou, M.-D.  _Vorticity and vortex dynamics_ (Springer Science & Business Media, 2007).
  * [30] Jolliffe, I. T.  _Principal component analysis_ (Springer, 2002).
  * [31] Berkooz, G., Holmes, P. & Lumley, J. L.  The proper orthogonal decomposition in the analysis of turbulent flows.  _Annu. Rev. Fluid Mech._ 25, 539–575 (1993).
  * [32] Taira, K. _et al._ Modal analysis of fluid flows: An overview.  _AIAA J._ 55, 4013–4041 (2017).
  * [33] Hinton, G. E. & Salakhutdinov, R. R.  Reducing the dimensionality of data with neural networks.  _Science_ 313, 504–507 (2006).
  * [34] Murata, T., Fukami, K. & Fukagata, K.  Nonlinear mode decomposition with convolutional neural networks for fluid dynamics.  _J. Fluid Mech._ 882, A13 (2020).
  * [35] Omata, N. & Shirayama, S.  A novel method of low-dimensional representation for temporal behavior of flow fields using deep autoencoder.  _AIP Adv._ 9, 015006 (2019).
  * [36] LeCun, Y., Bottou, L., Bengio, Y. & Haffner, P.  Gradient-based learning applied to document recognition.  _Proc. IEEE_ 86, 2278–2324 (1998).
  * [37] Rumelhart, D. E., Hinton, G. E. & Williams, R. J.  Learning representations by back-propagation errors.  _Nature_ 322, 533–536 (1986).
  * [38] Foias, C., Manley, O. & Temam, R.  Modelling of the interaction of small and large eddies in two dimensional turbulent flows.  _ESAIM: Math. Model. Numer. Anal._ 22, 93–118 (1988).
  * [39] Temam, R.  Do inertial manifolds apply to turbulence?  _Phys. D: Nonlinear Phenom._ 37, 146–152 (1989).
  * [40] De Jesús, C. E. P. & Graham, M. D.  Data-driven low-dimensional dynamic model of Kolmogorov flow.  _Phys. Rev. Fluids_ 8, 044402 (2023).
  * [41] Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P.  Image quality assessment: from error visibility to structural similarity.  _IEEE Trans. Image Process._ 13, 600–612 (2004).
  * [42] Smith, L., Fukami, K., Sedky, G., Jones, A. & Taira, K.  A cyclic perspective on transient gust encounters through the lens of persistent homology.  _arXiv:2306.15829_ (2023).
  * [43] Erichson, N. B. _et al._ Shallow neural networks for fluid flow reconstruction with limited sensors.  _Proc. Roy. Soc. A_ 476, 20200097 (2020).
  * [44] Fukami, K., Maulik, R., Ramachandra, N., Fukagata, K. & Taira, K.  Global field reconstruction from sparse sensors with Voronoi tessellation-assisted deep learning.  _Nat. Mach. Intell._ 3, 945–951 (2021).
  * [45] Zhong, Y., Fukami, K., An, B. & Taira, K.  Sparse sensor reconstruction of vortex-impinged airfoil wake with machine learning.  _Theor. Comput. Fluid Dyn._ 37, 269–287 (2023).
  * [46] Araujo-Estrada, S. A. & Windsor, S. P.  Aerodynamic state and loads estimation using bioinspired distributed sensing.  _J. Aircr._ 58, 704–716 (2021).
  * [47] Brunton, S. L., Proctor, J. L. & Kutz, J. N.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems.  _Proc. Nat. Acad. Sci. U.S.A_ 113, 3932–3937 (2016).
  * [48] Nakao, H.  Phase reduction approach to synchronisation of nonlinear oscillators.  _Contemp. Phys._ 57, 188–214 (2016).
  * [49] Taira, K. & Nakao, H.  Phase-response analysis of synchronization for periodic flows.  _J. Fluid Mech._ 846 (2018).
  * [50] Kurebayashi, W., Shirasaka, S. & Nakao, H.  Phase reduction method for strongly perturbed limit cycle oscillators.  _Phys. Rev. Lett._ 111, 214101 (2013).
  * [51] Kurtulus, D. F.  On the unsteady behavior of the flow around NACA 0012 airfoil with steady external conditions at $Re=1000$.  _Int. J. Micro Air Veh._ 7, 301–326 (2015).
  * [52] Liu, Y., Li, K., Zhang, J., Wang, H. & Liu, L.  Numerical bifurcation analysis of static stall of airfoil and dynamic stall under unsteady perturbation.  _Commun. Nonlinear Sci. Numer. Simul._ 17, 3427–3434 (2012).
  * [53] Di Ilio, G., Chiappini, D., Ubertini, S., Bella, G. & Succi, S.  Fluid flow around NACA 0012 airfoil at low-Reynolds numbers with hybrid lattice Boltzmann method.  _Comput. Fluids_ 166, 200–208 (2018).
  * [54] Kingma, D. P. & Ba, J.  Adam: A method for stochastic optimization.  _arXiv:1412.6980_ (2014).
  * [55] Fukami, K., Nakamura, T. & Fukagata, K.  Convolutional neural network based hierarchical autoencoder for nonlinear mode decomposition of fluid field data.  _Phys. Fluids_ 32, 095110 (2020).
  * [56] Hansen, P. C. & O’Leary, D. P.  The use of the l-curve in the regularization of discrete ill-posed problems.  _SIAM J. Sci. Comput._ 14, 1487–1503 (1993).
  * [57] Fukami, K. & Taira, K.  Grasping extreme aerodynamics on a low-dimensional manifold, XAero-Manifold.  https://doi.org/10.17605/OSF.IO/7VSH8/ (2023).
  * [58] Fukami, K. & Taira, K.  Grasping extreme aerodynamics on a low-dimensional manifold, Observable-AE.  https://zenodo.org/badge/latestdoi/677509021/ (2023).

## Acknowledgements

We thank A. Linot, H. Nakao, and L. Smith for stimulating discussions on
nonlinear dynamics and compression. Support for illustrations was provided by
T. Taira. K.T. acknowledges the generous support from the US Department of
Defense Vannevar Bush Faculty Fellowship (grant number: N00014-22-1-2798) and
the US Air Force Office of Scientific Research (grant number:
FA9550-21-1-0178). K.F. acknowledges the support from the UCLA-Amazon Science
Hub for Humanity and Artificial Intelligence.

## Author Contributions Statement

K.T. conceptualized the approach, K.F. and K.T. selected and analyzed the
problems, and wrote the manuscript. K.F. developed the software, curated the
data, and visualized the results. K.T. secured funding support and supervised
the project.

## Competing interests

Authors declare no competing interests.


# Learning Classifiers for Imbalanced and Overlapping Data

Pranav Murali${19BPS1035}$ Nitin Narayanan N${19BPS1050}$ Ajaykumar
M${19BPS1093}$ Shivaditya Shivganesh${19BPS1103}$ Shivaditya Shivganesh
19BPS1103  Nitin Narayanan N
19BPS1050  Pranav Murali
19BPS1035  Ajaykumar M
19BPS1093

###### Abstract

This study is about inducing classifiers using data that is imbalanced, with a
minority class being under-represented in relation to the majority classes.
The first section of this research focuses on the main characteristics of data
that generate this problem. Following a study of previous, relevant research,
a variety of artificial, imbalanced data sets influenced by important elements
were created. These data sets were used to create decision trees and rule-
based classifiers. The second section of this research looks into how to
improve classifiers by pre-processing data with resampling approaches. The
results of the following trials are compared to the performance of distinct
pre-processing re-sampling methods: two variants of random over-sampling and
focused under-sampling NCR. This paper further optimises class imbalance with
a new method called Sparsity. The data is made more sparse from its class
centers, hence making it more homogenous.

##  I INTRODUCTION

Supervised learning of classifiers from examples is one of the main tasks in
machine learning and data mining. However, their usefulness for obtaining high
predictive accuracy in real life data depends on different factors, including
also difficulties of the learning problem and its data characteristics. Class
imbalance is one of the sources of these difficulties. Many real life problems
are characterized by a highly imbalanced distribution of examples in classes.
Typical examples are rare medical diagnosis, recognition of oil spills in
satellite images, detecting specific astronomical objects in sky surveys or
technical diagnostics of equipment failures. Moreover, in fraud detection,
either in card transactions or in telephone calls the number of legitimate
transactions is much higher than the number of fraudulent ones. Similar
situations occur either in direct marketing where the response rate class is
usually very small in most marketing campaigns or information filtering where
some important categories contain few messages only.

###  I-A Issues with Class Imbalance

If imbalance in the class distribution is extensive, i.e. some classes are
strongly under-represented, then the typical learning methods do not work
properly. An even class distribution is often assumed (also non explicitly)
and the classifiers are “somehow biased” to focus searching on the more
frequent classes while “missing” examples from the minority class. As a result
constructed classifiers are also biased toward recognition of the majority
classes and they usually have difficulties (or even are unable) to classify
correctly new objects from the minority class.

###  I-B Possible Solutions

This paper concerns classifier-independent methods that rely on transforming
the original data to change the distribution of classes, e.g., by resampling
as these methods are more universal and they can be used in a pre-processing
stage before applying many learning algorithms. A small number of examples in
the minority class is not the only source of difficulties for classifiers.
Recent works also suggest that there are other factors that contribute to
difficulties. The degradation of performance was also related to other
factors, mainly to decomposition of the minority class into many sub-clusters
with very few examples. The rare sub-concepts correspond to, so called, small
disjuncts, which lead to classification errors more often than examples from
larger parts of the class.

###  I-C Goals

Studying the role of these factors in class imbalance is still an open
research problem. Therefore, the main aim of this study is to experimentally
examine which of these factors are more critical for the performance of the
classifier. Carrying out such experiments requires preparing a new collection
of artificial data sets which are affected by the above mentioned factors.
Proposing such data sets is another sub-aim of this paper. In this paper we
are particularly interested in focused (also called informed) resampling
methods, which modify the class distribution taking into account local
characteristics of examples. Representative methods such as SMOTE for
selective over-sampling of the minority class, one side sampling and NCR for
removing examples from the majority classes are used.

##  II Literature Review

Jerzy Stefanowski works on inducing classifiers from unbalanced data, in which
one class (a minority class) is underrepresented compared to the other classes
(majority classes). The minority group is usually the focus of attention, and
it must be recognised as accurately as possible. Because most algorithms
learning classifiers are biased toward the majority classes, class imbalance
is a challenge for them. The first section of this research focuses on the
main characteristics of data that generate this problem. Following the review,
related research resulted in the creation of numerous forms of artificial,
imbalanced data sets influenced by important aspects. These data sets were
used to create decision trees and rule-based classifiers. The results of the
initial studies suggest that a lack of examples from the minority class is not
the primary cause of problems.These findings support the initial premise that
the loss of classification performance is mainly related to the disintegration
of minority classes into small sub-parts, and we base our work on these
findings, employing a variety of strategies to address these concerns.

Chawla, N. V., et al offer a method for constructing classifiers from
unbalanced datasets. If the categorization categories are not roughly equally
represented, the dataset is unbalanced. ”Normal” examples make up the majority
of real-world datasets, with only a small number of ”abnormal” or
”interesting” examples. It’s also true that the cost of misclassifying an
unusual (interesting) sample as a normal example is frequently significantly
higher than the cost of making the opposite mistake. Under-sampling the
majority (normal) class has been offered as a good way to improve a
classifier’s sensitivity to the minority class; this strategy has also been
considered and tried alongside other strategies.

##  III Analysis

If one of the target classes has a significantly lower number of examples in
classes than the other, the dataset is called imbalanced. Minority courses are
those who are underrepresented, while majority classes are those who are over-
represented. The table below shows the results. When a sub-cluster dataset is
subjected to an imbalance ratio of (0.2), the model performance worsens as the
number of minority class sub clusters grows. Two of the primary elements for
this degradation are shown in the table below. One of them is sample size; as
the sample size shrinks, so does the model’s performance. As a result, both
the imbalance ratio and sample size play a role in creating an unbalanced
sample.

###  III-A Model Performance

Number of Sub Clusters | 1:5
---|---
| 600 | 400 | 200
2 | 0.803 | 0.845 | 0.742
3 | 0.767 | 0.765 | 0.712
4 | 0.787 | 0.712 | 0.721
5 | 0.742 | 0.694 | 0.655
6 | 0.651 | 0.684 | 0.59

###  III-B Generating Imbalanced Dataset

In order to solve to the issue of class imbalance in real world problems we
need to test and emulate the situations in order to devise solutions. So, we
have used an isotropic gaussian distribution in order to randomly sample data
points from it. The randomly sampled points are used to plot the cluster
centers with which we can make cluster blobs. Apart from the isotropic
distribution binomial or exponential distribution can also be used for random
sampling. We generate cluster centers based on which we create data points in
proximity to the cluster center in the coordinate space.

Figure 1: Datasets with subclusters Figure 2: Overlapping dataset

Below plots demonstrate the random data generation capability. Using the
method we are able to generate data with multiple subclusters. For plotting
the data we have using bandwidth estimation and MeanShift algorithm.

###  III-C Overlapping and Rare Examples

Rare and overlapping examples are crucial parts of the dataset in real world
situations. Most issues arise because of them and in order to make the method
resilient to such situations we need to introduce such examples in the
dataset. From the charts shown above you can see the algorithm’s capability to
generate data overlap and introduce rare examples into the dataset.

###  III-D Improving the Performance of Classifiers

We have chosen 4 methods which are simple random over-sampling, cluster based
oversampling and nearest cleaning rule.

##  IV Performance

###  IV-A Original Data Characteristics

  * •

Number of samples: 400

  * •

Number of features: 2

  * •

Majority class label: 0

  * •

Number of majority class samples: 316

  * •

Minority class label: 1

  * •

Number of Minority class sample: 84

  * •

Imbalance Ratio : 3.8

###  IV-B Data Characteristics after random Over Sampling

  * •

Number of samples: 632

  * •

Number of features: 2

  * •

Majority class label: 1

  * •

Number of majority class samples: 316

  * •

Minority class label: 0

  * •

Number of Minority class sample: 316

  * •

Imbalance Ratio : 1.0

###  IV-C Data Characteristics after NCR

  * •

Number of samples: 258

  * •

Number of features: 2

  * •

Majority class label: 0

  * •

Number of majority class samples: 174

  * •

Minority class label: 1

  * •

Number of Minority class sample: 84

  * •

Imbalance Ratio : 2.1

###  IV-D Data Characteristics after Cluster Based Oversampling

  * •

Number of samples: 632

  * •

Number of features: 2

  * •

Majority class label: 1

  * •

Number of majority class samples: 316

  * •

Minority class label: 0

  * •

Number of Minority class sample: 316

  * •

Imbalance Ratio : 1.0

##  V Result and discussion

Firstly, the impact of disturbing the borders of subregions in the minority
class was evaluated. It was simulated by increasing the ratio of borderline
examples from the minority class subregions. This ratio (further called the
disturbance ratio) was changed from 0 to 70rable to the width of the sub-
regions (sub-parts in data shapes). The constructed classifiers were combined
with the following focused pre- processing methods:

  * •

Standard random oversampling (abbreviated as RO)

  * •

Japkowicz’s cluster oversampling (CO)

  * •

Nearest cleaning rule NCR

Cluster oversampling was limited to the minority class, and for baseline
results (Base), both classifiers were ran without any pre-processing.

Oversampling RO and CO performed comparably on all data sets, and on non-
disturbed data sets they often over-performed focused method: NCR . If the
overlapping area is large enough (in comparison to the area of the minority
sub-clusters), and at least 30% of examples from the minority class are
located in this area, then oversampling method (NCR) strongly outperform
random and cluster over-sampled with respect to sensitivity and G-mean.

In the first series of experiments we used data sets of three shapes, 800
examples and the imbalance ratio of 1:7. We also employed rule- and tree-based
classifiers combined with the same pre-processing methods. In each training
set 30% of the minority class examples were safe examples located inside sub-
regions, 50% were located in the borderline/overlapping area (we denote them
with B), and the remaining 20% constituted the C rare examples.

NCR in handling rare examples located inside the majority class (also
accompanied with borderline ones). Such result has been in a way expected, as
both methods were introduced to handle such situations. The experiments also
demonstrated that even random oversampling better than NCR in classifying safe
examples from the minority class.

The results of experiments showed that although NCR often led to the highest
increase of sensitivity, at the same time it significantly deteriorated
specificity and overall accuracy. Moreover, it did not deteriorate the
recognition of the majority classes as much as NCR.

##  VI Conclusion

The issue of learning classifiers using skewed data has been discussed. It’s
one of the most difficult subjects in data mining and machine learning.
Furthermore, it is still ”open” from a theoretical standpoint, yet it is
critical in a variety of application domains. On the other hand, a review of
related literature reveals that it has sparked increased academic interest in
the recent decade, with various specialized methods already developed. Despite
the fact that some of them have been tested in experiments, additional general
research concerns like the class imbalance, data features, and the competency
of several popular approaches are still needed.

Other critical factors that affect classification performance include
decomposition of classes into smaller sub-parts with too few examples (so-
called small disjuncts), overlapping between classes (existence of too many
borderline examples in the minority class), and the presence of rare or noisy
examples located farther away from the decision boundary (deeper inside the
distribution of the opposite class)

The effect of important parameters in artificial data sets on the performance
of the most common rule-, tree-, and K-NN classifiers was investigated. First
and foremost, these findings corroborated previous findings on the importance
of the minority class decomposition into smaller sub-concepts based on simpler
contrived data. We also found that increasing the subdivision of the class
into small sub-parts decreased sensitivity or AUC values for more nonlinear
decision boundaries.

Learning is extremely challenging due to the combination of class
decomposition and overlapping (in particular for sensitivity measure and tree
classifiers). Our investigations revealed that the quantity of questionable
examples had a significant impact on the degradation of classification
ability. In terms of sensitivity and G-mean measurements, concentrated re-
sampling methods (such as NCR) outperformed random and cluster oversampling
when the overlapping area was large enough.

## ACKNOWLEDGMENT

Authors would like to thank , Dr. Shivani Gupta for their constant
encouragement towards the realization of this work.

## References

  * [1] Stefanowski, J. (2013). Overlapping, rare examples and class decomposition in learning classifiers from imbalanced data. In Emerging paradigms in machine learning (pp. 277-306). Springer, Berlin, Heidelberg.
  * [2] Chawla, N. V., Bowyer, K. W., Hall, L. O., Kegelmeyer, W. P. (2002). SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16, 321-357.
  * [3] ChR. Vilalta, M. -. Achari and C. F. Eick, ”Class decomposition via clustering: a new framework for low-variance classifiers,” Third IEEE International Conference on Data Mining, 2003, pp. 673-676, doi: 10.1109/ICDM.2003.1251005.
  * [4] B. Sowan, N. Matar, F. Omar, M. Alauthman and M. Eshtay, ”Evaluation of Class Decomposition based on Clustering Validity and K-means Algorithm,” 2020 21st International Arab Conference on Information Technology (ACIT), 2020, pp. 1-6, doi: 10.1109/ACIT50332.2020.9300084.
  * [5] P. Li, P. Qiao and Y. Liu, ”A Hybrid Re-sampling Method for SVM Learning from Imbalanced Data Sets,” 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discovery, 2008, pp. 65-69, doi: 10.1109/FSKD.2008.407