# Aero-Nef: Neural Fields for Rapid Aircraft Aerodynamics Simulations

Giovanni Catalani1,2, Siddhant Agarwal 1, Xavier Bertrand 1,
Frederic Tost 1, Michael Bauerheim 2, Joseph Morlier 2

1\. Airbus
2\. ISAE-Supaero, Toulouse
Corresponding email<EMAIL_ADDRESS>

###### Abstract

This paper presents a methodology to learn surrogate models of steady state
fluid dynamics simulations on meshed domains, based on Implicit Neural
Representations (INRs). The proposed models can be applied directly to
unstructured domains for different flow conditions, handle non-parametric 3D
geometric variations, and generalize to unseen shapes at test time. The
coordinate-based formulation naturally leads to robustness with respect to
discretization, allowing an excellent trade-off between computational cost
(memory footprint and training time) and accuracy. The method is demonstrated
on two industrially relevant applications: a RANS dataset of the two-
dimensional compressible flow over a transonic airfoil and a dataset of the
surface pressure distribution over 3D wings, including shape, inflow
condition, and control surface deflection variations. On the considered test
cases, our approach achieves a more than three times lower test error and
significantly improves generalization error on unseen geometries compared to
state-of-the-art Graph Neural Network architectures. Remarkably, the method
can perform inference five order of magnitude faster than the high fidelity
solver on the RANS transonic airfoil dataset.

##  1 Introduction

Computational Fluid Dynamics (CFD) has become an indispensable tool in modern
aircraft design, offering the possibility to perform high-fidelity simulations
of the complex physics of an aircraft. Numerical solutions can be leveraged to
evaluate aerodynamic performance, structural loads, and handling qualities at
a much lower cost than experimental and full-scale flight test campaigns.
These simulations primarily solve the Reynolds-averaged Navier-Stokes (RANS)
[1] equations to produce accurate flow field predictions, over the full flight
envelope [2]. However, the high computational cost associated with each
simulation restricts their use in the early design stages and time-sensitive
environments, such as the aerodynamic load estimation required over thousands
of flight conditions and design choices.

Given the prohibitive computational demands of traditional CFD simulations,
there is a critical need for efficient surrogate models. These models are
designed to approximate complex simulations quickly and accurately,
facilitating rapid iterations during the design process. Traditionally,
methods like Proper Orthogonal Decomposition (POD) [3] combined with
interpolation techniques (e.g., radial basis functions or Gaussian Processes
[4]) have been prevalent [5, 6]. Despite their efficiency and physical
interpretability, these models often fall short in scenarios involving
strongly non-linear phenomena, particularly at transonic flow conditions, due
to their inherent linear nature [7]. Additionally, the fixed geometry and
resolution constraints of modal decomposition methods limit their
applicability to scenarios involving geometric variations and multi-fidelity
data. Attempts to circumvent these drawbacks have focused on operating
snapshot clustering [8, 9], modifying the POD minimization metric [10], and
performing mesh morphing to a common reference mesh [11]. Despite the
potential of these proposed methods, their applicability can be limited to
specific cases, due to the underlying assumptions.

The recent advances in data-driven modeling, especially deep learning (DL),
offer new avenues for constructing surrogate models. Deep learning has
demonstrated exceptional capability in extracting and representing complex
hierarchical data features across various domains [12, 13, 14]. In the context
of fluid dynamics, DL methods have been used to perform a variety of tasks:
constructing reduced order models [15, 16], accelerating numerical solvers
[17, 18], identifying turbulence closure models [19] or flow control
strategies [20, 21], are only some of the most relevant examples.

Specifically for airfoil and aircraft aerodynamic predictions, Deep Learning
applications include modeling aerodynamic coefficients [22], surface pressure
distributions [23] pressure calibration [24] and shape optimization [25, 26,
27]. Building a surrogate model of the pressure fields, has a central
importance in all of the above tasks, replacing the need for a computationally
expensive numerical simulator. Convolutional Neural Networks (CNNs) have been
widely used to build surrogates, thanks to the strong capability of these
networks to capture local interactions and the possibility to
straightforwardly define multiscale operations. UNET architectures have been
applied to predict scalar and vector fields over airfoils and 3D
configurations [28, 29, 30]. Despite these successful attempts, CNNs expect
pixel-like input and output data at fixed resolution: CFD solutions are
typically defined on unstructured meshes, and interpolation routines have been
employed to map the two different types of data representations, with an
inevitable loss in performance [30]. Additionally, the different level of
refinement characteristic of numerical simulations, hinders the applicability
of CNN to 3D configurations, due to computational constraints.

Geometric deep learning, specifically Graph Neural Networks (GNNs), provide a
powerful paradigm to extend CNNs to unstructured domains, exploiting the
inductive biases inherent to graph-like data [31]. This flexibility allows
GNNs to model complex geometries typical of fluid dynamics simulations more
naturally than traditional neural network architectures. Mesh Graph Networks
[32] introduce convolution operations on meshes, involving relative node
distances as edge features, and demonstrate good performance for fluid
mechanics time-dependent problems defined on a common mesh, with a relatively
limited number of nodes. Extension of message passing GNNs to larger graphs,
inevitably requires pooling operations, in order to model multiscale behavior
of fluid flows [33, 34, 35] and long range interactions over the mesh. In a
monoscale architecture, a larger number of message passing layers would be
needed to cover an equivalent graph region, leading to oversmoothing [36] and
excessive computational overhead. The definition of pooling operations is,
however, dependent on the mesh topology and the kind of application, as no
general downsampling rule fits all cases. Despite their general formulation
and success across a wide range of applications, the effectiveness of GNN for
surrogate modeling of aerodynamic simulations is still limited. Firstly, GNNs
architectures encounter challenges in generalizing across different mesh
topologies and levels of discretization: this can be explained by the fact the
node connectivity and neighborhood definitions are based on the graph’s
structural metrics rather than the physical metrics of the domain [37].
Refining a mesh over a certain threshold often degrades the performance of the
GNN [33], as the neighborhood of each node shrinks, converging to a single
point in the limit of infinite refinement, failing the discretization
convergence criterion. Typical industrial aerodynamic applications in 3D
involve largely refined meshes (even surfacic-only meshes can exceed 1 Million
nodes per geometry) in order to account for the complex interactions between
lifting surfaces and the fluid flow at higher Reynolds Numbers. Training a
surrogate model at full resolution is often unfeasible with limited hardware
resources: scalability requires the possibility of accurately generating
predictions at higher levels of discretization while training at significantly
coarser levels. It must be noted that a substantial effort to scale GNN to
handle large meshes, in the content of aircraft aerodynamics, has been done in
the work of Hines and Berkemeyer [23], as dynamic subsampling has been applied
to predict the surface pressure distribution on the single NASA CRM aircraft
geometry [38] over a range of flight parameters variations.

Recent advances in the area of Operator Learning have allowed the introduction
of discretization invariant architecture, bypassing the limitations of GNN
which have been mentioned earlier. Neural Operator, learns the mapping between
the infinite-dimensional input and output function spaces, and can by
construction produce the value of the output function at any point in the
spatial domain. Fourier Neural Operator (FNO) [39] has been introduced to
solve parametric PDEs, by learning the integral kernel directly in the Fourier
Space, leveraging the computational efficiency of Fast Fourier Transforms
(FFT) on uniform grids. The Geometry Informed Neural Operator (GINO) [37]
employs a Graph Neural Network to map unstructured 3D domains to a structured
reference latent grid, where Fourier Layers can be efficiently applied.
Similarly, DeepOnet [40] learns operators from a sparse set of observations
using two subnetworks, but it is constrained to input functions evaluated
always in the same location, being poorly suited for problems involving shape
variations. Recently, Geom-DeepOnet was introduced [41], extending the
capabilities of the parent DeepOnet methodology to handle parametric geometry
variations in 3D.

Implicit Neural Representations (INRs), also known as Neural Fields, have
emerged as a powerful alternative to classical methods for learning spatial
representations of objects such as images, shapes, and 3D scenes through
radiance fields [42, 43, 44]. INRs achieve an impressive level of detail with
limited memory requirements by approximating functions at any point in the
domain. They take spatial coordinates as input and produce the corresponding
function value at those coordinates. Recent breakthroughs in the field,
particularly in input encoding techniques like SIREN [45] and Fourier Feature
Encoding (FFN) [46], have significantly enhanced the ability of INRs to
approximate high-frequency functions with unmatched efficiency. SIREN employs
periodic activation functions, enabling the representation of complex signals,
while FFN leverages a set of predefined frequency components typically sampled
from a Gaussian distribution.

However, tuning the hyperparameters of these architectures, especially the
frequency components in FFN, is non-trivial. The sampled frequencies can
significantly impact the model’s underfitting or overfitting behavior [46].
This aspect is particularly challenging in fluid dynamics datasets, which
often exhibit a wide spectrum of frequencies. While initially limited to
representing single samples, advancements such as latent modulation and meta-
learning have extended INRs’ capabilities to approximate entire classes of
objects and datasets [47, 43, 45, 48]. Serrano et al. introduced CORAL [49], a
flexible framework for learning initial value problems, modeling PDE dynamics,
and building surrogate models on 2D meshes using INRs. This framework
demonstrated computational efficiency and the ability to handle geometric
variations accurately. The INFINITY model [50] further specialized this
approach for the RANS equations over 2D airfoils in the incompressible regime,
achieving state-of-the-art accuracy.

##### In this study

we present a comprehensive methodology to build surrogate models of steady
aerodynamic simulations on meshed domains using Implicit Neural
Representations. Our contributions are twofold. Firstly, we introduce the
first application of INR-based surrogate models to 3D wing surface meshes,
incorporating shape variations and aileron deflections over a wide range of
flight conditions. This showcases the potential of our model for real
industrial applications. Our methodology is specialized into two main
frameworks: an encode-process-decode design suitable for general problems with
non-parametric shape variations, and an end-to-end design optimized for
scenarios with fixed geometry or parametric shape variations.

Secondly, we propose a Multiscale-INR backbone architecture to optimize
spectral convergence, addressing the challenges associated with tuning the
frequency hyperparameters of standard Fourier Feature Encoding architectures.
This architecture ensures that our model captures the necessary frequency
components to accurately represent complex aerodynamic phenomena without
extensive hyperparameter tuning.

Our extensive experimental study includes two key datasets: a transonic
airfoil dataset with fixed mesh configurations and a 3D wing dataset with
shape variations and aileron deflections. We chose the 2D transonic airfoil
dataset due to its industrial relevance, as accurately predicting shocks is
crucial for aircraft aerodynamics, with cruise speeds typically in the
transonic range. Traditional methods like POD often struggle to capture these
shock phenomena accurately. The 3D wing dataset presents extensive shape
variations, making it particularly challenging for most deep learning
architectures. We also test the model’s generalization capabilities on unseen
shapes. These experiments demonstrate the accuracy, efficiency, and
scalability of our proposed methodology. We compare our approach with state-
of-the-art Graph Neural Network (GNN) baselines, highlighting several aspects
crucial to the method’s scalability, such as discretization dependency.

Additionally, we conduct a discretization dependence study to quantify our
method’s ability to generalize across different levels of mesh refinement.
This study underscores the scalability potential of our framework, enabling
accurate predictions at higher levels of discretization while training at
significantly coarser levels. This capability is particularly important for
industrial applications, where high-resolution simulations are computationally
prohibitive.

##  2 Methodology

##### Problem Statement

In this work, we address the problem of finding data-driven solutions to
steady-state fluid dynamics simulations discretized on meshes. Let
$\Omega\subset\mathbb{R}^{d}$ denote the physical domain of the geometry under
consideration, where $d$ indicates the dimensionality of the spatial domain.

Let $\mathcal{A}=\mathcal{A}(\Omega;\mathbb{R}^{d_{a}})$ and
$\mathcal{U}=\mathcal{U}(\Omega;\mathbb{R}^{d_{u}})$ be separable Banach
spaces of functions taking values in $\mathbb{R}^{d_{a}}$ and
$\mathbb{R}^{d_{u}}$ respectively. Each element of $\mathcal{A}$ is a function
describing the input geometry (for instance through the Signed Distance
Function), the boundary conditions and the inflow conditions. Let
$\mathcal{G}:\mathcal{A}\rightarrow\mathcal{U}$ be a non-linear map, serving
as the solution operator of the associated partial differential equation (PDE)
that describes the physics of the problem.

Numerical methods seek for an approximate solution $u\in\mathcal{U}$ of the
governing PDE on a discretized version of the input domain $\Omega$, namely a
mesh $\mathcal{M}=(X,\mathcal{E})$, defined by the set of nodes coordinates
$X\in\mathbb{R}^{Nd}$ and their connectivities
$\mathcal{E}\in\\{0,1\\}^{N\times N}$ for specific instances of the input
parameter vector $\mu\in\mathbb{R}^{d_{p}}$.

Surrogate models are built to approximate the numerical solution, for
different values of the input parameters and different geometrical
configurations, typically at a much lower cost than the associated numerical
methods. Data driven methods learn this mapping from a dataset of observed
input-output pairs:

$\mathcal{D}=\\{(\mathcal{M}_{i},\mu_{i}),U_{i}\\}_{i=1}^{M}$ (1)

where each tuple consists of a mesh (${M}_{i}$), input parameters ($\mu_{i}$)
typically describing the boundary conditions and inflow conditions, and the
corresponding field solution ($U_{i}\in\mathbb{R}^{N_{i}d_{u}}$), evaluated on
the mesh, usually with a CFD solver. Additional scalars, namely lift and drag
coefficients, can be considered as the output of the surrogate model, but in
the following, we will focus on building parametric models for full fields
predictions.

Specifically, we aim to define a methodology to build surrogate models of
aerodynamic simulation on meshes, by directly approximating the solution
operator of the underlying PDE with a data-driven architecture based on
Implicit Neural Representations. It must be highlighted that although the
primary objective of a surrogate model is to obtain discretized solutions on a
given mesh, reformulating the problem in terms of approximating the continuous
solutions operator allows to obtain a flexible and scalable algorithm with
respect to discretization, as it will be shown in the following paragraphs.

##### Implicit Neural Representations

Implicit Neural Representations (INRs) are coordinate based neural networks
parameterized by weights $\theta\in\mathbb{R}^{p}$ mapping points $\mathbf{x}$
in Euclidean space (or equivalently on a manifold) to vector or scalar
quantities:

$f_{\theta}(\mathbf{x}):\mathbb{R}^{d}\rightarrow\mathbb{R}^{d_{u}}.$ (2)

INRs can be thought of as a continuous approximation of an underlying signal,
for which a discretely sampled version, is available for learning. Leveraging
the continuous nature of INRs, numerous data modalities can be modeled, such
as images, shapes, and physical fields without constraints on the type of
discretization. In this work, we employ INRs to learn concise representations
of physical solutions and shapes descriptors, that are invariant across levels
and types of discretization. Field predictions can be obtained by querying an
INR across multiple coordinate points:

$f_{\theta}(X)=[f_{\theta}(\mathbf{x_{1}}),...,f_{\theta}(\mathbf{x_{N}})].$
(3)

The Implicit Neural Architecture is implemented through a Multi-Layer-
Perceptron (MLP) which in its most basic form takes as input the spatial
coordinates and outputs the value of the physical field at the specific
coordinate. This formulation is naturally suited to fit single signals defined
over low-dimensional domains. For machine learning tasks, that require
learning classes of objects (eg. a dataset of various PDE solutions, instead
of a single element), the INR architecture can be conditioned through a latent
representation that encodes sample-specific features.

Hence, in the following we proceed to describe: (i) how to train INRs to
handle variations in geometry and input parameters, and (ii) how the proposed
architectural components make Neural Fields a powerful paradigm for learning
mesh-based simulations.

##### Input Encoding

Standard Neural Networks fail to represent high frequency oscillations on low
dimensional domains [46], being biased towards lower frequency signals. This
phenomenon, denominated Spectral Bias, hinders the convergence of vanilla MLP
architectures to higher harmonics, leading to blurry reconstructions. The
solutions proposed in the literature employ positional encoding [44] or
Fourier Feature encoding [46] to the inputs. [46] shows that incorporating
these encodings, leads to a stationary Neural Tangent Kernel [51] stationary,
thus improving the network convergence to the high frequency component of the
signal being approximated. The Fourier Encoding is implemented as follows:

$\gamma({\mathbf{x}})=[a_{1}cos(2\pi b_{1}^{T}\mathbf{x}),a_{1}sin(2\pi
b_{1}^{T}\mathbf{x}),...,a_{n}cos(2\pi b_{n}^{T}\mathbf{x}),a_{n}sin(2\pi
b_{n}^{T}\mathbf{x})]$ (4)

where the frequency components are sampled from a zero-centered Gaussian
Distribution $b_{i}\sim\mathcal{N}(0,\sigma)$. The standard deviation
($\sigma$) of the sampling distribution is a hyper-parameter that can have a
significant effect on the underfitting-overfitting behavior of the network
[46, 52]. In fact, its magnitude determines the range of frequency that the
network can represent (this is typically referred to as kernel bandwidth) in
the reconstructed signals. Low $\sigma$ values lead to the network acting a
low pass filter on the reconstructed signal, causing underfitting behaviour as
the network fails to capture the details of the target signal. Larger $\sigma$
values, on the other hand, can lead the network to incorporate high frequency
patterns, and produce noisy reconstruction characteristics of overfitting
behaviour. Attempts to learn the optimal value or infer it from the frequency
spectrum of the target signal are limited to simple cases in 1D or single
signal scenarios. It can be much more complex to tune this hyperparameter to
fit a dataset of CFD simulation, exhibiting multiscale phenomena. To
circumvent this issue we propose a multiscale architecture where multiple
input encoding are performed using distinct values of $\sigma$, passed through
the network leading to intermediate outputs that are concatenated through the
final layer. This approach, inspired by the work of [52] in the context of
PINNs, appears to simplify other proposed methods in the field of computer
vision [53, 54], based on a progressive increment of $\sigma$ during training.
We refer the reader to the corresponding literature for a deeper dive into
this topic. The encoding function for each scale $i$ is defined as:

$\gamma_{\sigma_{i}}(\mathbf{x})=\left[\sin(2\pi\mathbf{B}^{(i)}\mathbf{x}),\cos(2\pi\mathbf{B}^{(i)}\mathbf{x})\right]^{T},\quad\textit{for
}i=1,2,\dots,M$ (5)

where each element of $\mathbf{B}\in\mathbb{R}^{n\times d}$ is sampled from a
Gaussian distribution $\mathcal{N}(0,\sigma_{i})$. The output of the network,
$f_{\theta}(\mathbf{x})$, is then obtained by passing the concatenated
intermediate output at each scale through a final linear layer :

$f_{\theta}(\mathbf{x})=W_{L+1}[H_{L}^{(1)},H_{L}^{(2)},\dots,H_{L}^{(M)}]+b_{L+1}.$
(6)

Note that the hidden layers of the network are shared for all the encoded
inputs, and therefore this approach does not considerably increase the
parameter count. In Figure 1 a schematic illustration of the multiscale
architecture is presented. A simple pedagogical experiment is performed by
fitting a single 1D signal with large frequency bandwidth: choosing a priori
one single value of $\sigma$ is not trivial. Larger values of this
hyperparameter (eg. $\sigma=5$) can lead to overfitting behavior as high
frequency oscillations are present in the reconstructed signal, while smaller
sigma values (eg. $\sigma=0$) can result in underfitting. The Multiscale
architecture ($\sigma=[1,5]$)leads to better reconstructions, by balancing the
two behaviors: this advantage becomes even more relevant when fitting multiple
signals in higher dimensions.

##### Conditioning via Modulation

Conditioning the network to a specific sample is performed through shift
modulation of the intermediate layer outputs. The hidden layer outputs are
adjusted by a modulation vector $\phi$, which is added to the output of each
layer before applying the ReLU activation function. The first layer receives
the encoded input $\gamma_{\sigma_{i}}(\mathbf{x})$. The composition of layers
in the network is described as follows:

$\displaystyle H(\mathbf{x})$
$\displaystyle=W_{L}(\eta_{L-1}\circ\eta_{L-2}\circ....\circ\eta_{1}\circ\gamma_{\sigma}(\mathbf{x}))+b_{L}$
$\displaystyle\eta_{l}(\cdot)$
$\displaystyle=\textit{ReLU}(W_{l}(\cdot)+b_{l}+\phi_{l})$

where $W_{l},b_{l}\in\theta$ are the weights and biases of the $l$-th layer,
and $\phi_{l}$ is the shift modulation vector for the $l$-th layer. Here we
omit the index of the scale encoding. The modulation vectors carry the
specific information of each sample and allow to use a unique set of global
network parameters to account for all parametric or geometry variations. The
modulation is computed for each layer. It is a function of a latent embedding
($z$) for the encode-process-decode model or directly of the input parameters
($\mu$) for the end-to-end framework, through a hypernetwork $h_{\psi}$
parameterized by its parameters $\psi$. A detailed description of both
frameworks is provided in the following.

Figure 1: Left: Illustration of the multiscale architecture. Right:
Experimental results for fitting a single signal using a 3-layer INR with
different Fourier encoding. This experiment can be easily reproduced using the
notebook provided in the accompanying Git repository (https://gitlab.isae-
supaero.fr/gi.catalani/aero-nepf).

##### End-to-end framework

Several ways have been proposed to condition Neural Fields to handle classes
of shapes or images. In the original DeepSDF model proposed by Park et al.
[42], a distinct latent vector is jointly learned at training time together
with network parameters to minimize the loss between the network outputs and
the Signed Distance Function on a batch of sampled points around a specific
shape. Similarly, a hypernetwork $h$ of parameters $\mathbf{\psi}$ can be used
to map the input parameters $\mathbf{\mu}$ to the latent modulation vectors
$\phi$. In the case where the desired output is a physical field on a fixed
geometry, the learning task can be handled end-to-end by minimizing the
pointwise distance between the model predictions and the target values across
the sampled coordinates and training samples :

$\min_{\mathbf{\theta},\mathbf{\psi}}\mathcal{L}=\sum_{i}^{N}\sum_{\mathbf{x}\in\mathcal{M}}\ell\left(f_{\mathbf{\theta}}(\mathbf{x};h_{\mathbf{\psi}}(\mathbf{\mu}_{i})),u_{i}(\mathbf{x})\right).$
(7)

In practice, it is not necessary to provide the full set of sampled coordinate
points at each epoch. The model is able to fit the data with a much reduced
training resolution, while being able to perform super-resolution at test
time. This feature can greatly reduce training time and memory requirements,
as only a subset of points can be used for training. In particular, the best
strategy to ensure optimal data diversity is to dynamically downsample the
training samples at the required training resolution at the beginning of each
epoch.

Figure 2 illustrates the components of the Implicit end-to-end framework.

Figure 2: Top: The geometry is encoded by the input INR, where the spatial
coordinates $x,y,z$ are processed by a modulated neural network parameterized
by $\theta_{in}$ to produce a Signed Distance Function (SDF). The output INR,
parameterized by $\theta_{out}$, learns the mapping between the spatial
coordinates and the output physical fields $p$, for each training sample
through modulation. The modulation vectors $\phi$ are obtained from the (input
or output) latent codes $z$ through a hypernetwork parameterized by $\psi$.
The latent map, where $z_{in}$ and parameters $\mu$ is transformed into
$z_{out}$, is approximated via a Multilayer Perceptron (MLP). Bottom: In this
setup, the input parameters $\mu$ are directly fed into a hypernetwork
$h_{\psi}$ to produce the modulations $\phi$. This modulation, along with the
spatial coordinates, is then processed by a neural network parameterized by
$\theta_{out}$ to output the physical fields $p$.

##### Encode-Process-Decode framework

This framework, proposed by Dupont et al. [48] and specialized by Serrano et
al. [49] for mesh based simulations, splits the learning process into two main
steps: firstly learning compact representation of input and output field
quantities ($z_{in}$ and $z_{out}$ in Fig. 2), and secondly learning the
mapping between the two latent spaces ($z_{out}=MLP(z_{in},\mu$) conditioned
by the input parameters.

The encoding step is analogous for both input and output variables (e.g. for
the geometry and the pressure fields). We aim to model the geometry through
the Signed Distance Function. During the encoding step, for both input and
output variables, latent codes $z$, global networks $\theta$, and hyper-
network parameters $\psi$ are jointly optimized using a second-order Meta-
Learning approach based on the CAVIA algorithm [55]. The learning task can in
fact be split into a sample-specific regression problem: to find the optimal
$z$ specific to each distinct sample, and a global regression to learn the
optimal values of the global shared networks parameters ($\theta,\psi$) across
the whole dataset samples. A detailed description of the training algorithm
can be found in Algorithm 1 and in the provided references. As observed by
Serrano [49], this strategy offers several advantages over classical
approaches for training conditional Neural Fields: training history is
stabilized and the risk of overfitting is reduced, as the latent parameters
are re-initialized to zero at each epoch and learned in few gradient steps
inside the inner loop, thus acting as a regularization term on the optimized
latent vectors. Moreover, this speeds up remarkably the inference time, as
only few optimization steps are required to learn representations of the input
quantities compared to a classical approach as a larger amount of optimization
steps would be required to obtain the latent input representations. Encoding
the input and output variables allows to obtain input-output latent vector
pairs:

$\mathcal{D}^{\textit{enc}}=\\{(z^{in}_{i},\mu_{i}),z^{out}_{i}\\}_{i=1}^{M}\quad
z_{in}\in\mathbb{R}^{d_{in}},z_{out}\in\mathbb{R}^{d_{out}}.$ (8)

The regression step entails learning a mapping in the compressed space of
latent input vector ($z_{in}$), input parameters ($\mu$), and output latent
vectors ($z_{out}$). For this task, we employ train a simple residual MLP
$p_{\delta}$, with SiLu activations, by minimizing the latent loss:

$\min_{\mathbf{\delta}}\sum_{i}^{N}[p_{\delta}(z^{in}_{i},\mu_{i})-z^{out}_{i}]^{2}$
(9)

Figure 2 illustrates the architecture of the Encode-Process-Decode framework.

At inference, predictions for an unseen shape and flight parameters can be
performed using the pretrained Neural Fields and processor network. In
particular, the input latent code for an unseen shape can be inferred by
minimizing the loss function ($\ell$) between the reconstructed input function
(Signed Distance) and the target values:

$\hat{z}^{in}_{test}=argmin_{z}\sum_{\mathbf{x}\in\mathcal{M}_{test}}\ell\left(f_{\mathbf{\theta}^{in}}(\mathbf{x};h_{\mathbf{\psi}^{in}}(z)),u_{test}(\mathbf{x})\right)$
(10)

practically, this corresponds to the just performing the inner loop described
in Algorithm 1, while keeping the global network parameters fixed. Once the
geometry descriptor is obtained, it can be used as an input to the processor
network, together with the input parameters, in order to obtain the latent
representation of the output variable:

$\hat{z}^{out}_{test}=p_{\psi}(z^{in}_{test},\mu_{test}).$ (11)

Finally, the output fields can be decoded on the spatial domain, by querying
the pretrained output Neural Field:

$\hat{u}_{test}=f_{\mathbf{\theta}^{out}}(\mathbf{x};h_{\mathbf{\psi}^{out}}(\hat{z}^{out}_{test}))$
(12)

The inference steps are also sketched in Fig. 3 for a sample 3D wing shape.

Figure 3: At inference time, few gradient steps are needed to optimize the
input shape descriptors $\hat{z}^{in}$ to produce good reconstructions of the
Signed Distance Function. The processor network $p_{\delta}$ maps the input
latent and the parameters $\mu$ to the output $\hat{z}_{out}$, which allows
decoding the output fields everywhere in the spatial domain. Algorithm 1
Training Encoder Networks with CAVIA

1:Input: Dataset $\mathcal{D}$, epochs $E$, inner loops $K$, batch size $B$,
learning rates $\lambda_{\theta}$, $\lambda_{\psi}$,$\lambda_{z}$

2:Output: Optimized parameters $\theta$, $\psi$

3:for epoch $e=1,\ldots,E$ do

4: Sample $\mathcal{B}$ from $\mathcal{D}$

5: Initialize context parameters $z_{i}=0$

6: for $i\in i_{\mathcal{B}}$ and $k$ in $1,..,K$ do $\triangleright$ Inner
loop: Optimize local parameters

7: Compute sample loss
$\mathcal{L}_{i}=\sum_{\mathbf{x}\in\mathcal{M}_{i}}\ell\left(f_{\mathbf{\theta}}(\mathbf{x};h_{\mathbf{\psi}}(z_{i})),u_{i}\right)$

8: Update local parameters: $z_{i}\leftarrow
z_{i}-\lambda_{z}\nabla_{z_{i}}\mathcal{L}_{i}$

9: end for

10: Compute total loss for the batch: $\mathcal{L}_{\mathcal{B}}=\sum_{i\in
i_{\mathcal{B}}}\mathcal{L}_{i}$

11: Update global parameters:
$\theta\leftarrow\theta-\lambda_{\theta}\nabla_{\theta}\mathcal{L}_{\mathcal{B}}$,
and $\psi\leftarrow\psi-\lambda_{\psi}\nabla_{\psi}\mathcal{L}_{\mathcal{B}}$

12:end for

##  3 Experiments

In this section we demonstrate our methodology on two distinct test cases.
Firstly, a 2D RANS dataset over a fixed airfoil geometry, and a wide range of
Angles of Attack and Mach numbers. Here the end-to-end framework is used and
benchmarked against several baseline models. The second test case, is a more
challenging 3D wing dataset with shape and flight parameters variations: this
allows us to showcase the encode-process-decode framework against a state of
the art Graph Neural Network.

###  3.1 Transonic Airfoil Dataset

##### Dataset and Task Descriptiom

The first test case considered to demonstrate the proposed methodology
consists of Reynolds Averaged Navier Stokes (RANS) solution over a two
dimensional transonic airfoil, namely the RAE2822. This geometry is well-
documented in both numerical and experimental fluid mechanics research,
ensuring a robust basis for validation.

The dataset encompasses a broad spectrum of aerodynamic conditions, with
simulations carried out over a Reynolds number of 6.5 million, typical of
cruise flight scenarios. The angle of attack and Mach number, critical
parameters in aerodynamic studies, vary from 0° to 9° and 0.3 to 0.9,
respectively, providing a dense sampling across both weakly compressible and
transonic regimes. In total, the dataset consists in total of 1200 samples.

The RANS simulations, performed using the commercial software Fluent, leverage
the Spalart-Allmaras [56] turbulence model and a second-order upwind
discretization scheme on a hybrid mesh structure. The same hybrid CFD mesh,
consisting of 27499 cells, is used for all samples in the dataset, with a
refined structured block close to the airfoil in order to accurately capture
the boundary layer near the airfoil and an unstructured blocks, further away
from the airfoil surface. Such detail ensures the fidelity and granularity
required for high-quality machine learning training datasets. Each RANS
computation takes approximately 30 minutes to converge to the steady-state
solutions, utilizing an Intel Core i7-10850H processor with 6 cores. More
details about the dataset can be found in the original paper introducing the
dataset [30].

The machine learning task consists in learning a surrogate model capable of
predicting the output pressure field over the entire mesh (on and off the
airfoil surface) for any combination of Mach number $M$ and angle of attack
$\alpha$, given a training set of samples. We limit the discussion to one
output quantity, as it is generally the most relevant variable for
aerodynamics analysis and force estimation. It is possible, through the
presented model, to predict additional quantities, such as the velocity
components, using the same shared Neural Network: this is presented in the
Appendix for the sake of completeness. We also note that, increasing the
number of desired outputs, does not significantly change the memory
requirements of the method. This is not the case for GNN based surrogates
trained on large meshes, as storing the additional outputs on the full graph
can become a limitation when training on memory constrained GPUs.

####  3.1.1 Experimental Setting and Baselines

Experiments are conducted using a 70/10/20 train/validation/test dataset split
for all models. We compare the proposed INR end-to-end model with a simple MLP
baseline and a state of the art Graph UNET Architecture. A brief explanation
of the architectural choices for the proposed method and the baselines is
provide in the following.

The INR end-to-end model is implemented as 5-Layer MLP with Multiscale Feature
Encoding, ReLu activations and Shift Modulation, as outlined in Section 2. For
the input encoding, we sample Fourier Features with two distinct normal
distributions with standard deviations $\sigma_{1}=1,\sigma_{2}=5$, using in
total 128 components, aiming to cover the optimal frequency spectrum to fit
the dataset . We observed that using larger values of $\sigma$ introduced more
noisy reconstructions. The hypernetwork is implememnted as 3-Layer MLP,
processing the input parameters $\mathbf{\mu}=[M,\alpha]$ into latent
modulations of dimension 128. The model is trained on a downsampled version of
the original mesh, as 5000 nodes for each samples are randomly selected for
training. As it is shown in the following paragraphs, this allows to
drastically reduce the training time, while keeping approximately invariate
the test error at full resolution. A more detailed overview of the
hyperparamters and implementation details for the proposed method and the
baselines is provided in the Appendix.

##### Vanilla MLP

A standard Multi-Layer-Perceptron architecture serves as a simple baseline,
consisting of a 5-layer MLP with residual connections. The intermediate layer
width is the same as the INR model, in order to keep a similar number of
parameters and ensure a fair comparison. For this baseline, the input
parameters $\mathbf{\mu}$ are directly fed into the network along with the
spatial coordinates, making it essentially a basic INR without latent
conditioning and input encoding. This baseline helps to isolate and evaluate
the specific impact of input encoding and latent modulation on model
performance.

##### Graph UNET

Abbreviated GUNET, is a multiscale message passing graph convolutional neural
network introduced in 2019 [34] that extends the popular UNET architecture
[57] to handle non unfiform grids. Graph UNets are designed to handle graph-
structured data by performing hierarchical pooling operations, which reduce
the size of the graph at each level, and increase the receptive field of the
convolutions, thereby capturing multi-scale features. We use an encode-
process-decode architecture, where the encoder and decoder are implemented as
a 4-layer MLP. The encoded features are passed through a SAGE Convolutional
Layer [58] followed by a pooling layer that uses TopKPooling [59]. Each
pooling layers, halves the number of nodes, and doubles the number of
features. A multiscale GNN is particularly suited for this dataset, due to the
moderately large mesh size, which requires more than local message-passing as
in mono-scale architecture. However, we observe that it is possible (and
beneficial), to train the model on a downsampled version of the orginal mesh
and then to perform directly inference on the full mesh. This is not
surprising, and was obsrved in previous work on MultiScale GNNs [33]. For this
reason, the best performance of the model, is obtained when training on a mesh
of 5000 nodes, and using 2 levels of Pooling. It must be observed that the
mesh is dynamically downsampled at each epoch, in order to maximize variance
and improve generalization error. This approach is analogous to the training
strategy used in [23], where a MeshGraphNet is trained on a large number of
nodes representing a wing-shape. A multiscale approach is necessary for this
model because message passing on large graphs requires pooling to manage
computational complexity and memory usage effectively, making it feasible to
process large-scale graph data. The trained model is directly used for
inference at full resolution at test time. In Section 3.1.2 we provide
quantitative results in the model performance, highlighting the effect of
training resolution and number of pooling layers.

##### Evaluation Metrics

In our study, the performance of various models designed for predicting
transonic airfoil aerodynamics is quantitatively assessed using three primary
metrics: Mean Squared Error (MSE), training time, and inference time. These
metrics provide a robust framework for comparing the efficiency and
effectiveness of each modeling approach under consideration.

Training time refers to the total duration required to train a model on the
specified dataset. Depending on the model type, it is specified into the
different stages of the learning process. The experiments were conducted using
a single NVIDIA’s A100 GPU.

Inference time is defined as the average time required to perform a single
sample prediction. When possible within the computational resources, batched
inference can be performed, with additional speedup. In order to ensure
fairness in comparison between the different baselines, we do not perform
batching over the sample dimension. This metric is crucial for applications
where real-time predictions are necessary, and it can vary drastically between
different methodologies.

####  3.1.2 Results

The performance of the models on the transonic airfoil dataset is summarized
in Table 1.

In Figure 4, we observe the performance of the models in capturing the
pressure distribution in the spatial domain and on the airfoil surface. The
simple MLP approach shows high accuracy away from the shock region,
effectively capturing the pressure distribution in the weakly compressible
regime. However, due to the spectral bias, and the absence of input encoding,
the shock region is oversmoothed, failing to capture the sharp gradients
accurately. This limitation is visible in the surface pressure distribution
plot in Figure 5, where the shock is noticeably smeared, and in the contour
plots, as the isobaric lines are further apart in the vicinity of the
compression wave. The trailing edge region, as well as the pressure peaks are
well captured. Training and inference times for MLP-based approaches are
relatively fast, with the vanilla MLP being especially quick due to the
absence of input encoding and latent modulation.

The GUNET model, designed to handle graph-structured data with multi-scale
features, performs well in the shock region prediction. The hierarchical
pooling operations in GUNET enable it to capture multi-scale features, which
helps in modeling the sharp gradients near the shock. However, its overall
error is still higher than the MLP-based approaches, notably in the trailing
edge region. Training time for GUNET is the longest due to the inclusion of
pooling operations and multiple message passing steps. Despite this, GUNET
offers significant speedup at inference time compared to computing high-
fidelity solutions directly.

The INR model outperforms all other approaches, achieving the lowest MSE. The
multi-layer MLP with multiscale feature encoding and shift modulation enables
the INR to effectively model the complex aerodynamic phenomena present in the
dataset. The incorporation of Gaussian feature encoding and latent modulations
allows the INR to maintain high accuracy across different aerodynamic
conditions, including the challenging transonic and shock regimes. The speedup
of all tested surrogate models, compared to the high-fidelity RANS
computations is in the order of $10^{5}$, demonstrating the computational gain
of using a surrogate model.

Table 1: Performance comparison of different models in terms of error (MSE), training time (s), and inference time (ms). Model | MSE | Training Time (s) | Inference Time (ms)
---|---|---|---
INR | 0.0020 | 9497 | 4
Vanilla MLP | 0.0057 | 5749 | 2
GUNET | 0.0074 | 17100 | 17
Figure 4: Pressure field distribution prediction with different surrogate
models and comparison with CFD. Top: Angle of Attack 6.2 [deg], Mach Number
0.60. Center: Angle of Attack 4.94 [deg], Mach Number 0.84. Bottom: Angle of
Attack 7.2 [deg], Mach Number 0.85 Figure 5: Surface Pressure Coefficient
distribution prediction with different surrogate models and comparison with
CFD. Left: Angle of Attack [6.2 deg], Mach Number [0.60]. Center: Angle of
Attack [4.94 deg], Mach Number [0.84]. Right: Angle of Attack [7.2 deg], Mach
Number [0.85]

##### Discretization Invariance Analysis

A large class of Deep Learning based reduced order models can only predict
test signals at the same resolution of the training signals, and are often
coupled with interpolation routines to extract the value of the underlying
signals at finer resolutions: this process can perform poorly, as subgrid
dynamics is filtered out [60]. This consideration includes method based on
Convolutional Neural Network,transformer based architectures and modal
decomposition techniques. While it is true that the effect of subgrid dynamics
on larger scales can be modeled by those architectures, inferring the values
of the predicted signals at sub-pixel locations is problematic. Crucially,
fluid flows are modeled by PDEs on continuous domains, and numerical methods
seeks for the best approximation of the continuous operator mapping the input
to the output function spaces on a discretized domain, without the limitation
of mapping input and outputs of fixed resolution. These observations are of
primary importance in the development of efficient surrogate models with
powerful generalization properties: data used for training is in fact often
defined on unstructured domains, with different connectivity and topology.
Moreover, high resolution data might not always be abundant, preferring the
use of mixed resolution training dataset. An even more compelling
consideration regarding surrogate models for CFD must take into account the
scalability of the method to handle large mesh simulation, as the ones often
used for real aeronautical applications. In these scenarios, surface meshes
can be constituted by a number of nodes in the order of 1 Million. This
resolution is required in order to accurately model strongly non-linear
phenomenon in the boundary layer regions, or pressure discontinuities in the
transonic regime. The amount of computational resources, namely memory
footprint and training time, can represent the main limitation in training ML-
based surrogates for this class of applications. In practice, Graph Neural
Networks have been scaled to handle full graph training on moderately large
meshes by incorporating pooling techniques as part of multi-scale architecture
designed to improve message passing over larger regions of the domain (in an
analogous mannner of multiscale CNNs). The definition of pooling routines is
however non-trivial and mesh dependent, making the resulting architectures
only partially to generalize on graphs of similar topology. Additionally,
multiscale architecture, while requiring a smaller number of message passing
layers, do not solve the issue related to memory requirements (the graph at
original resolution is loaded in GPU memory) and often incur in additional
training time. Other techniques, based on graph mini-batching or domain
decomposition have been employed in the literature to improve the
computational efficiency of GNNs. Ideally, surrogate models that can
generalize across different resolutions present the advantage of allowing
training at much lower resolutions, while being able to predict accurately
solutions at full resolution. Implicit Neural Representations are naturally
formulated as coordinate based networks, without constraints on the signal
discretization, and can perform zero shot superesolution at finer scales at
test time. We quantify the trade-off between accuracy and efficiency, by
performing an analysis on the approximate discretization invariance of the
model, obtained on the RAE2822 dataset. The study is devised as follows:
multiple identical INR architecture (we use the end-to-end best performing
architecture) are trained on different version of the datasets, obtained by
randomly downsampling the mesh nodes at different resolution levels. The
models are then used to predict the CFD solutions at increasing resolutions
and the global metrics in terms of mean square error are summarized. In Table
2 the quantitative results are presented: the INR can be trained at much lower
resolutions (on less than 2 % of the full mesh), and perform super-resolution
at test time with a contained error increase. The test error at full
resolution converges to a minimum as the training resolution is increased,
demonstrating the approximate discretization convergence property of Neural
Operators. Optimal trade-off between test error and training time can be
achieved by training the model with a 6 times lower resolution than the full
mesh. It is important to stress that memory requirements do not scale with
training resolution, as the INR are coordinate-based and do not require any
sort of graph processing during training. The GUNET performance in terms of
test MSE does not improve when using finer training meshes. Using SAGE
Convolutional Layer aids the message passing scheme to propagate information
on the graph when training at higher resolutions, as the nodes neighborhoods
radii are defined in the physical space. However, the accuracy results are
biased towards the training resolution, with a test error that increases
considerably when performing inference on the full mesh. It is interesting to
note how the addition of a pooling level (compared to the 2-level baseline)
degrades the performance of the network, while increasing computational cost.

Table 2: Performance of models at different resolutions. The table shows the average mean squared error (MSE) and training time for INR and GUNET models at various training resolutions. GUNET levels refer to the levels of pooling used in the model. Model | Training Res | Test Res | Average MSE | Training Time (s)
---|---|---|---|---
INR | 500 | 500 | 0.00294 | 2899
Full | 0.00276
5000 | 5000 | 0.00200 | 9497
Full | 0.00202
Full | Full | 0.00197 | 46985
GUNET (2 levels) | 500 | 500 | 0.0059 | 6080
Full | 0.0078
5000 | 5000 | 0.0038 | 17100
Full | 0.0074
Full | Full | 0.0079 | 35580
GUNET (3 levels) | Full | Full | 0.0113 | 43320

###  3.2 XRF1 Wing Dataset

##### Dataset Description

The dataset comprises 8,640 surface pressure simulations over 120 wing shapes,
and different operating conditions. The numerical solver is based on the BLWF
method [61] developed at the Central Aerohydrodynamic Institute (TsAGI). This
code assumes external inviscid flow coupled with a viscous boundary layer. For
each computation, we only consider the pressure field on the wing, computed on
a triangular mesh of 6,600 nodes. An example of the mesh for a specific shape
is displayed in Figure 6. Despite using a lower fidelity numerical solver,
compared to typical 3D RANS computations on larger meshes, the relevant flow
structures and pressure patterns are well captured, as a function of the wing
geometries, ensuring the validity of the proposed surrogate on higher fidelity
datasets. Moreover, the dataset’s diversity and complexity make it highly
relevant for industrial aerospace applications. Testing generalization to
unseen geometries and configurations in 3D is in fact crucial for a surrogate
model to be effectively used within preliminary aerodynamic design and
analysis of an aircraft.

The dataset spans 120 distinct wing configurations characterized by systematic
variations of a reference wing, by variation of three shape parameters: Span
$S$,Thickness $T$ and Dihedral Sweep $D_{s}$. In Figure 6 a subset of wing
shapes used in this study is depicted. For each configuration, operational
conditions are defined with a simple Design of Experiment (DoE) approach, by
changing the Mach number within typical cruise values $M\in[0.50,0.86]$, the
Angle of attack $\alpha\in[-4^{\circ},8^{\circ}]$ and the Aileron Deflection
Angle $\delta_{ail}\in[0^{\circ},-15^{\circ}]$. The Reynolds number is kept
fixed at $Re=3\times 10^{6}$. The effect of the aileron deflection is modeled
on the resulting pressure distribution, but not present in the input geometry
and it is therefore kepts as a paramteric input for the surrogate model. This
provides a wide range of aerodynamic conditions, in the compressible and
transonic regime, including control surface deflection effects.

Figure 6: XRF Wing Dataset geometries.Left: Top and front view of 10 out of
120 different wing shapes. Span, shape and thickness are varied.Right: Sample
computational mesh for a given shape.

The machine learning task consists of predicting the pressure fields on the
surface mesh from the operational parameters
$\mathbf{\mu}=[M,\alpha,\delta_{ail}]$ and the geometry information. It is
important to note that here we assume non-parametric geometry definition:
although the shape variations are defined by modification of the shape
parameters, we decide to learn a representation of the geometry only from the
mesh node position, connectivity and surface normals. This demonstrated the
general capabilities of the proposed approach, since parametrization of CAD
degigned geometries is not in general available. Learning concise
representation of shapes is essential for downstream applications, such as
shape optimization or shape generation.

####  3.2.1 Experimental Setting and Baselines

In this study, we compare the performance of the INR model, using the encode-
process-decode framework, with a Graph Neural Network, specifically the
MeshGraphNet, which is widely used for this type of task [32, 23].

##### Geometry Preprocessing and SDF

Training the input INR model requires a preprocessing step of the surface
meshes to obtain the volumetric Signed Distance Function fields. We follow a
similar process to the one applied to train the DeepSDF model of Park et al.
[42]. In particular a bounding box is created around each mesh, prenormalized
to the unit sphere. A first random sampling, with uniform distribution, is
performed inside the box ($10\%$ of the total sampled points). The second
sampling is performed by randomly selecting points on the mesh surface, and
then adding random Gaussian noise with zero mean and pre-defined standard
deviation. Two different values ($\sigma_{s}=0.005,0.0005$) are used, defining
a progressively finer distribution of points near the surface. This is a
fundamental step to capture the finer geometrical details of a specific wing.
In total, around $N^{vol}=60000$ points are sampled for each mesh. Finally,
the SDF is computed in each of the sampled points, using the point-cloud-utils
[62] library. A set of input coordinates and ouptut SDF values is obtained for
each shape:

$\mathcal{D}^{SDF}_{i}=\\{(\mathbf{x}_{j},sdf_{j})\\}_{j=1}^{N^{vol}}\quad
i=1,...,M$ (13)

##### Encode-Process-Decode INR

The input INR is trained to learn the mapping between coordinates and sdf
values, conditioned by latent vectors specific for each geometry
$\\{z^{in}_{i}\\}_{i=1}^{M}$. The architecture is a 6-Layers MLP with a single
scale Fourier Feature Encoding. Latent modulations are obtained from latent
codes $z_{in}$ of dimension 64, with a single linear layer hypernetwork. In
order to make training more efficient, we randomly select mini-batches of
points $\mathcal{B}\in\mathcal{D}^{SDF}_{i}$ for each epoch. The output INR
maps the input node coordinates and the surface normals to the output pressure
values. The architecture is a 6-Layer MultiScale FFN with
$\sigma_{1}=1,\sigma_{2}=5$, single linear layer hypernetwork and latent codes
$z_{out}$of dimension 128. A processor network is used to learn the mapping
from input latent codes and the flight parameters to output latent codes. The
flight parameters are stored in a 3-dimensional vector, containing
respectively the Angle of Attack, the Mach Number, the Aileron Deflection.
This regressor model is implemented as a 4-Block MLP with skip connections and
SiLu activation. Training the processor is performed for a maximum 1000 epochs
and stopped when validation loss has not improved for more than 200 epochs.

##### MeshGraphNet

Abbreviated MGN, is employed as a baseline for this task, being widely used in
literature for learning unsteady fluid dynamic [32], and steady parametric
solutions [23] on general meshes. MGN follows an encode-process-decode
architecture without pooling layers. Given the manageable size of the
triangular mesh (6,600 nodes), pooling layers are not required for effective
message passing. The input features are specified on edges for relative node
distance magnitude and sign, and on nodes for parameter values and normals.
Two distinct 4-Layer residual MLP networks implement the edge and node
encoders, projecting the respective features to latent vectors (one latent
vector for each node or edge) of dimension 128.

The processor block includes 12 consecutive Graph Convolutional Layers with
multiple consecutive message passing layers with Edge Based Convolution.
Across each layer, edges representations are updated a function (parameterized
by an MLP) of their current state of the representations of the nodes
connected by said edge, while the nodes representations are updated as a
function of their current state and the sum of the edge representations
converging to the node. The decoder projects the output of the processor block
to the desired output features. The specific architectural choices and
hyperparameters are given in the appendix.

####  3.2.2 Results

The performance of the models on the XRF1 wing dataset is summarized in Table
3. The INR framework demonstrates far superior accuracy with the lowest MSE
for predictions on unseen shapes. In Figure 7, the surrogate models’
predictions are displayed against the ground truth CFD data. The MGN method
can capture the general trend of the pressure distribution, while failing to
model the more complex aerodynamic features. The shock region prediction is
the most challenging for both surrogate models. Nevertheless, the INR method
is capable to predict the sharp gradient with good accuracy. Towards the
leading edge, the accuracy of MGN degrades, as the pressure distribution shows
strongly non-linear pattern due to the presence of strong vorticity.

The INR method shows remarkable generalization capabilities to unseen shapes,
demonstrating the efficacy of the geometry latent representations. This is
visible in Figure 8, as the surrogate models’ predictions on three different
test shapes, at the same operating conditions, are presented against the CFD
ground truth.

The training time for the INR encode-process-decode model is split into the
three components: encoding the input (geometry), encoding the output (pressure
fields), and the processor network. The input INR, which learns the Signed
Distance Function (SDF) representation of the geometry, and the output INR,
which maps coordinates to pressure values, both require significant training
time. As the number of shapes is smaller than the number of total samples (for
each shape a number of operating conditions are simulated), training the input
encoder takes less time than the output encoder. The processor network, on the
other hand, can be trained to fit the output latent codes using relatively
limited computational resources. Similarly, the MeshGraphNet’s training time
is relatively high due to the extensive message passing and separate node and
edge encoding.The inference time, for the proposed INR approach, is dominated
by the geometry encoding step : as explained in the Section 2, thanks to the
meta-learning approach, the latent codes for a test geometry can be inferred
in few gradient steps while freezing the main network parameters. It must be
noted that this step is only required once per geometry. Thus, INR model can
be used to decode the output fields for different operating conditions, on the
same shape, without incurring in this computation. In a typical design
scenario, in fact, the same shape design is tested over a variety of flight
conditions relevant for the aircraft mission.

As a conclusion, these results highlight the effectiveness of the encode-
process-decode framework in handling complex aerodynamic datasets with varying
geometries and operating conditions.

Table 3: Performance comparison of different models in terms of error (MSE), training time (s), and inference time (ms). Model | MSE | Training Time (s) | Inference Time (ms)
---|---|---|---
INR | 0.008 | 28320 (INR Input) | 108 (Geometry)
73560 (INR Output)
3318 (Processor) | 9 (Output)
MGN | 0.035 | 90720 | 17

MGNINRCFD(c)(b)(a)

(c)(b)(a)

Figure 7: Comparison between MeshGraphNet, INR, and CFD predictions of the
surface pressure distribution at
$M=0.82$,$\alpha=4[deg]$,$\delta_{ail}=-15[deg]$ Left: Surface pressure
distribution on the upper surface of the wing (top view). Cutting planes
indicate the location of pressure distribution on the right. Right: Pressure
coefficient cuts at three different locations $\eta=0.32,0.55,0.77$ along the
wingspan.

MGNINRCFD

Figure 8: Pressure Distribution prediction on top surface, across different
shapes at the same operating conditions. Top:
$M=0.70$,$\alpha=8[deg]$,$\delta_{ail}=0[deg]$. Bottom:
$M=0.80$,$\alpha=3[deg]$,$\delta_{ail}=0[deg]$.

##### Geometry encoding and reconstruction

A key factor in the effectiveness of the INR methodology is the possibility to
effectively encode the geometrical information inside the input latent codes,
by fitting a modulated Neural Field on the Signed Distance Function scalar
field. In order to capture the shape variations in the dataset, a good level
of accuracy is required, so that the latent representations unambiguously
carry the specific geometrical features of the encoded objects. However, at
test time, inferring the latent code for an unseen shape should not be a
computationally demanding task, especially for surrogate modeling
applications, where the main focus is placed on speedup and accuracy on the
predicted physical fields rather than extremely detailed shape reconstruction.
For this reason, the employed training strategy detailed in Algorithm 1, which
allows to infer the latent codes with a small number of gradient updated
($K=3$ in all the experiments), fits particularly well the surrogate modeling
task. In Figure 9 the original shapes are reconstructed from the inferred
latent codes, obtained by fitting the geometry encoder (with frozen
parameters) on the SDF fields (see Fig. 3). The marching cube algorithm [63]
is applied on the decoded SDF fields to obtain the reconstructed surface
meshes. Despite some noisy regions towards the wing trailing edge and wingtip
regions, it can be observed that the main geometrical features, determining
the dataset shape variations, are well represented.

Figure 9:  Left: Ground Thruth (top) vs reconstructed (bottom) wing shapes,
top view. Right: Ground Thruth (left) vs reconstructed (right) wing shapes,
front view.

##  4 Conclusion

In this work, we introduced a robust methodology to build surrogate models for
the prediction of aerodynamic fields in 2D and 3D on unstructured meshes that
can generalize to unseen non-parametric geometric configurations. The
methodology is based on Implicit Neural Representations, enabling a continuous
approximation of the target function and approximate discretization
invariance. The proposed MultiScale backbone architecture allows optimal
signal reconstruction, on complex fluid dynamics dataset, without extensive
tuning of frequency hyperparameters.

Our experimental results on the Transonic Airfoil and XRF1 Wing datasets
demonstrate the advantages of the INR-based models over state-of-the-art Graph
Neural Networks (GNNs). The INR models achieve consistently the lowest mean
squared error (MSE) across various aerodynamic conditions, showcasing their
ability to accurately capture complex flow dynamics, including shock regions
and other nonlinear phenomena.

The approximate discretization invariance of the INR models allows for
training on lower resolution meshes, while maintaining high accuracy at full
resolution. This leads to substantial reductions in computational cost and
memory footprint.

Compared to high-fidelity CFD simulations, the surrogate model can achieve a 5
orders of magnitude speedup, allowing for real-time aerodynamic simulations.
This paves the way for building surrogate models of complex 3D industrial
simulations on large meshes, without substantial changes to the presented
method.

##### Limitations and future work:

Despite discussing and experimentally showing the scalability of the method,
the experimental section does not report results on large mesh experiments.
Due to the scarce availability of open-source datasets of aircraft
aerodynamics CFD simulations involving shape variations, this investigation is
saved for further work. Additionally, this work only deals with the analysis
of single component aerodynamic configurations (i.e. wings and airfoils),
while most aircraft aerodynamic applications involve the interaction of
multiple parts, such as fuselage, propellers and high-lift devices. Extending
the methodology to these cases is not trivial, as it requires learning
geometric representations of components with different length scales, as well
as their relative position.

## Acknowledgments

This was was supported in part by the French Government represented by Agence
Nationale de la Recherche (ANRT), through CIFRE PhD Fellowship sponsored by
Airbus Operations SAS and ISAE-Supaero.

## References

  * [1] Stephen B Pope.  Turbulent flows.  Measurement Science and Technology, 12(11):2020–2021, 2001.
  * [2] A Da Ronch, M Ghoreyshi, and KJ Badcock.  On the generation of flight dynamics aerodynamic tables by computational fluid dynamics.  Progress in Aerospace Sciences, 47(8):597–620, 2011.
  * [3] Gal Berkooz, Philip Holmes, and John L Lumley.  The proper orthogonal decomposition in the analysis of turbulent flows.  Annual review of fluid mechanics, 25(1):539–575, 1993.
  * [4] Ankit Chiplunkar, Elisa Bosco, and Joseph Morlier.  Gaussian process for aerodynamic pressures prediction in fast fluid structure interaction simulations.  In Advances in Structural and Multidisciplinary Optimization: Proceedings of the 12th World Congress of Structural and Multidisciplinary Optimization (WCSMO12) 12, pages 221–233. Springer, 2018.
  * [5] Saddam Hijazi, Giovanni Stabile, Andrea Mola, and Gianluigi Rozza.  Data-driven pod-galerkin reduced order model for turbulent flows.  Journal of Computational Physics, 416:109513, 2020.
  * [6] Omer San, Romit Maulik, and Mansoor Ahmed.  An artificial neural network framework for reduced order modeling of transient flows.  Communications in Nonlinear Science and Numerical Simulation, 77:271–287, 2019.
  * [7] David J Lucia, Paul I King, and Philip S Beran.  Domain decomposition for reduced-order modeling of a flow with moving shocks.  AIAA journal, 40(11):2360–2362, 2002.
  * [8] Romain Dupuis, Jean-Christophe Jouhaud, and Pierre Sagaut.  Aerodynamic data predictions for transonic flows via a machine-learning-based surrogate model.  In 2018 AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, page 1905, 2018.
  * [9] Giovanni Catalani.  Machine learning based local reduced order modeling for the prediction of unsteady aerodynamic loads.  2022\.
  * [10] Tan Bui-Thanh, Karen Willcox, Omar Ghattas, and Bart van Bloemen Waanders.  Goal-oriented, model-constrained optimization for reduction of large-scale systems.  Journal of Computational Physics, 224(2):880–896, 2007.
  * [11] Fabien Casenave, Brian Staber, and Xavier Roynard.  Mmgp: a mesh morphing gaussian process-based machine learning method for regression of physical problems under nonparametrized geometrical variability.  Advances in Neural Information Processing Systems, 36, 2024.
  * [12] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.  Deep learning.  nature, 521(7553):436–444, 2015.
  * [13] Yoshua Bengio, Aaron Courville, and Pascal Vincent.  Representation learning: A review and new perspectives.  IEEE transactions on pattern analysis and machine intelligence, 35(8):1798–1828, 2013.
  * [14] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [15] Stefania Fresca, Luca Dede’, and Andrea Manzoni.  A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized pdes.  Journal of Scientific Computing, 87:1–36, 2021.
  * [16] Hamidreza Eivazi, Soledad Le Clainche, Sergio Hoyas, and Ricardo Vinuesa.  Towards extraction of orthogonal and parsimonious non-linear modes from turbulent flows.  Expert Systems with Applications, 202:117038, 2022.
  * [17] Ekhi Ajuria Illarramendi, Antonio Alguacil, Michaël Bauerheim, Antony Misdariis, Benedicte Cuenot, and Emmanuel Benazera.  Towards an hybrid computational strategy based on deep learning for incompressible flows.  In AIAA Aviation 2020 Forum, page 3058, 2020.
  * [18] Rishikesh Ranade, Chris Hill, Lalit Ghule, and Jay Pathak.  A composable machine-learning approach for steady-state simulations on high-resolution grids.  Advances in Neural Information Processing Systems, 35:17386–17401, 2022.
  * [19] Martin Schmelzer, Richard P Dwight, and Paola Cinnella.  Discovery of algebraic reynolds-stress models using sparse symbolic regression.  Flow, Turbulence and Combustion, 104:579–603, 2020.
  * [20] Jean Rabault and Alexander Kuhnle.  Accelerating deep reinforcement learning strategies of flow control through a multi-environment approach.  Physics of Fluids, 31(9), 2019.
  * [21] Baptiste Corban, Michael Bauerheim, and Thierry Jardin.  Discovering optimal flapping wing kinematics using active deep learning.  Journal of Fluid Mechanics, 974:A54, 2023.
  * [22] Rebecca Zahn, Maximilian Winter, Moritz Zieher, and Christian Breitsamter.  Application of a long short-term memory neural network for modeling transonic buffet aerodynamics.  Aerospace Science and Technology, 113:106652, 2021.
  * [23] Derrick Hines and Philipp Bekemeyer.  Graph neural networks for the prediction of aircraft surface pressure distributions.  Aerospace Science and Technology, 137:108268, 2023.
  * [24] Xavier Bertrand, Frédéric Tost, and Steeve Champagneux.  Wing airfoil pressure calibration with deep learning.  In AIAA Aviation 2019 Forum, page 3066, 2019.
  * [25] Pierre Baque, Edoardo Remelli, Francois Fleuret, and Pascal Fua.  Geodesic convolutional shape optimization.  In International Conference on Machine Learning, pages 472–481. PMLR, 2018.
  * [26] Zhen Wei, Aobo Yang, Jichao Li, Michaël Bauerheim, Rhea Patricia Liem, and Pascal Fua.  Deepgeo: Deep geometric mapping for automated and effective parameterization in aerodynamic shape optimization.  In AIAA AVIATION Forum, 2024.
  * [27] Zhen Wei, Edouard Dufour, Colin Pelletier, Pascal Fua, and Michaël Bauerheim.  Diffairfoil: An efficient novel airfoil sampler based on latent space diffusion model for aerodynamic shape optimization.  In AIAA AVIATION Forum, 2024.
  * [28] Nils Thuerey, Konstantin Weißenow, Lukas Prantl, and Xiangyu Hu.  Deep learning methods for reynolds-averaged navier–stokes simulations of airfoil flows.  AIAA Journal, 58(1):25–36, 2020.
  * [29] Xiaoxiao Guo, Wei Li, and Francesco Iorio.  Convolutional neural networks for steady flow approximation.  In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 481–490, 2016.
  * [30] Giovanni Catalani, Daniel Costero, Michael Bauerheim, Luca Zampieri, Vincent Chapin, Nicolas Gourdain, and Pierre Baqué.  A comparative study of learning techniques for the compressible aerodynamics over a transonic rae2822 airfoil.  Computers & Fluids, 251:105759, 2023.
  * [31] Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst.  Geometric deep learning: going beyond euclidean data.  IEEE Signal Processing Magazine, 34(4):18–42, 2017.
  * [32] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W Battaglia.  Learning mesh-based simulation with graph networks.  arXiv preprint arXiv:2010.03409, 2020.
  * [33] Meire Fortunato, Tobias Pfaff, Peter Wirnsberger, Alexander Pritzel, and Peter Battaglia.  Multiscale meshgraphnets.  arXiv preprint arXiv:2210.00612, 2022.
  * [34] Hongyang Gao and Shuiwang Ji.  Graph u-nets.  In international conference on machine learning, pages 2083–2092. PMLR, 2019.
  * [35] Mario Lino, Stathi Fotiadis, Anil A Bharath, and Chris D Cantwell.  Multi-scale rotation-equivariant graph neural networks for unsteady eulerian fluid dynamics.  Physics of Fluids, 34(8), 2022.
  * [36] T Konstantin Rusch, Michael M Bronstein, and Siddhartha Mishra.  A survey on oversmoothing in graph neural networks.  arXiv preprint arXiv:2303.10993, 2023.
  * [37] Zongyi Li, Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Mohammad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, et al.  Geometry-informed neural operator for large-scale 3d pdes.  Advances in Neural Information Processing Systems, 36, 2024.
  * [38] John Vassberg, Mark Dehaan, Melissa Rivers, and Richard Wahls.  Development of a common research model for applied cfd validation studies.  In 26th AIAA applied aerodynamics conference, page 6919, 2008.
  * [39] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar.  Fourier neural operator for parametric partial differential equations.  arXiv preprint arXiv:2010.08895, 2020.
  * [40] Lu Lu, Pengzhan Jin, and George Em Karniadakis.  Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators.  arXiv preprint arXiv:1910.03193, 2019.
  * [41] Junyan He, Seid Koric, Diab Abueidda, Ali Najafi, and Iwona Jasiuk.  Geom-deeponet: A point-cloud-based deep operator network for field predictions on 3d parameterized geometries.  arXiv preprint arXiv:2403.14788, 2024.
  * [42] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove.  Deepsdf: Learning continuous signed distance functions for shape representation.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 165–174, 2019.
  * [43] Zhiqin Chen and Hao Zhang.  Learning implicit fields for generative shape modeling.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5939–5948, 2019.
  * [44] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng.  Nerf: Representing scenes as neural radiance fields for view synthesis.  Communications of the ACM, 65(1):99–106, 2021.
  * [45] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein.  Implicit neural representations with periodic activation functions.  Advances in neural information processing systems, 33:7462–7473, 2020.
  * [46] Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng.  Fourier features let networks learn high frequency functions in low dimensional domains.  Advances in neural information processing systems, 33:7537–7547, 2020.
  * [47] Matthew Tancik, Ben Mildenhall, Terrance Wang, Divi Schmidt, Pratul P Srinivasan, Jonathan T Barron, and Ren Ng.  Learned initializations for optimizing coordinate-based neural representations.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2846–2855, 2021.
  * [48] Emilien Dupont, Hyunjik Kim, SM Eslami, Danilo Rezende, and Dan Rosenbaum.  From data to functa: Your data point is a function and you can treat it like one.  arXiv preprint arXiv:2201.12204, 2022.
  * [49] Louis Serrano, Lise Le Boudec, Armand Kassaï Koupaï, Thomas X Wang, Yuan Yin, Jean-Noël Vittaut, and Patrick Gallinari.  Operator learning with neural fields: Tackling pdes on general geometries.  Advances in Neural Information Processing Systems, 36, 2024.
  * [50] Louis Serrano, Leon Migus, Yuan Yin, Jocelyn Ahmed Mazari, and Patrick Gallinari.  Infinity: Neural field modeling for reynolds-averaged navier-stokes equations.  In Workshop on Synergy of Scientific and Machine Learning Modeling (ICML 2023), 2023.
  * [51] Arthur Jacot, Franck Gabriel, and Clément Hongler.  Neural tangent kernel: Convergence and generalization in neural networks.  Advances in neural information processing systems, 31, 2018.
  * [52] Sifan Wang, Hanwen Wang, and Paris Perdikaris.  On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks.  Computer Methods in Applied Mechanics and Engineering, 384:113938, 2021.
  * [53] Amir Hertz, Or Perel, Raja Giryes, Olga Sorkine-Hornung, and Daniel Cohen-Or.  Sape: Spatially-adaptive progressive encoding for neural optimization.  Advances in Neural Information Processing Systems, 34:8820–8832, 2021.
  * [54] Zoe Landgraf, Alexander Sorkine Hornung, and Ricardo Silveira Cabral.  Pins: progressive implicit networks for multi-scale neural representations.  arXiv preprint arXiv:2202.04713, 2022.
  * [55] Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson.  Fast context adaptation via meta-learning.  In International Conference on Machine Learning, pages 7693–7702. PMLR, 2019.
  * [56] Philippe Spalart and Steven Allmaras.  A one-equation turbulence model for aerodynamic flows.  In 30th aerospace sciences meeting and exhibit, page 439, 1992.
  * [57] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer, 2015.
  * [58] Will Hamilton, Zhitao Ying, and Jure Leskovec.  Inductive representation learning on large graphs.  Advances in neural information processing systems, 30, 2017.
  * [59] Cătălina Cangea, Petar Veličković, Nikola Jovanović, Thomas Kipf, and Pietro Liò.  Towards sparse hierarchical graph classifiers.  arXiv preprint arXiv:1811.01287, 2018.
  * [60] Kamyar Azizzadenesheli, Nikola Kovachki, Zongyi Li, Miguel Liu-Schiaffini, Jean Kossaifi, and Anima Anandkumar.  Neural operators for accelerating scientific simulations and design.  Nature Reviews Physics, pages 1–9, 2024.
  * [61] Ke-shi Zhang and Martin Hepperle.  Evaluation of the blwf code-a tool for the aerodynamic analysis of transonic transport aircraft configurations.  2010\.
  * [62] Francis Williams.  Point cloud utils, 2022.  https://www.github.com/fwilliams/point-cloud-utils.
  * [63] William E Lorensen and Harvey E Cline.  Marching cubes: A high resolution 3d surface construction algorithm.  In Seminal graphics: pioneering efforts that shaped the field, pages 347–353. 1998.
  * [64] Paul Saves, Rémi Lafage, Nathalie Bartoli, Youssef Diouane, Jasper Bussemaker, Thierry Lefebvre, John T Hwang, Joseph Morlier, and Joaquim RRA Martins.  Smt 2.0: A surrogate modeling toolbox with a focus on hierarchical and mixed variables gaussian processes.  Advances in Engineering Software, 188:103571, 2024.

##  Appendix A Experimental Details

##### Transonic Airfoil Dataset

For each baseline model used in the experiments with the Transonic Airfoil
dataset, the hyperparameters are detailed in the Table 4,5,6. The number of
epochs is chosen such that training and validation losses achieve convergence.
The dataset is randomly split into the same train,validation, test (70,10,20)
for all methods. The input data and output data are featurewise normalized to
a standard normal distribution before being fed to the networks. The input
data has four dimensions: ($x,y,M,\alpha$) while the output data has one
dimension ($p$). For the end-to-end INR model, the parameters are fed through
the hypernetwork while the coordinates through the main network.

Table 4: Hyperparameters for end-to-end INR model Hyperparameter | Value
---|---
Layers Dimensions | [$2^{(in)}$,128,128,128,128,$1^{(out)}$]
Latent Dimension | 128
Activation Function | ReLu
Encoding Type | Gaussian FF
N. Sampled Frequencies (per scale) | 64
Sampling std (scales) | [1,5]
Hyper-network Layers Dimensions | [2,128,128,128]
Learning Rate | 2e-5
Batch Size | 16
Epochs | 5000
Table 5: Hyperparameters for Vanilla MLP Hyperparameter | Value
---|---
Layers Dimensions | [$4^{(in)}$,128,128,128,128,$1^{(out)}$]
Layers Type | Residual
Input Encoding | None
Activation Function | ReLu
Learning Rate | 2e-5
Batch Size | 16
Epochs | 5000
Table 6: Hyperparameters for GUNET model Component | Hyperparameter | Value
---|---|---
Encoder | Layers Dimensions | [$4^{(in)}$, 64, 64, 32]
Activation Function | Relu
Decoder | Layers Dimensions | [32, 64, 64, $1^{(out)}$]
Activation Function | Relu
Processor | Neighborhood Radius (List) | [0.5, 1]
Number of levels | 2
Pooling Ratio | 0.5
Pooling Type | TopKPooling
Convolution Type | SAGEConv
Max Neighbors | 32
Shared | Batch Size | 1
Number of Epochs | 600
Learning Rate | 0.001

##### XRF1 Wing Dataset

For each baseline model used in the experiments with the XRF1 Wing dataset,
the hyperparameters are detailed in Tables 7,8,9. For all models, featurewise
normalization to the standard normal distribution, for both inputs and outputs
is performed. The inputs of the INR input encoder are the 3D spatial
coordinates $x,y,z$ while scalar the outputs are the SDF values. The inputs of
the INR output encoder are the the 3D spatial coordinates $x$ (on the surface
only) and the surface normals $n_{x},n_{y},n_{z}$, while the output are the
scalar pressure values $p$. The processor network, takes as input the input
latent code $z_{in}$, concatenated with the flight parameters
$M,\alpha,\delta_{ail},Re$ and produces the output latent code $z_{out}$. For
MeshGraphNet, the surface normals and the flight parameters are specified as
node inputs, while the 3D relative node distances are specified on the edges.

Table 7: Hyperparameters for INR encoder (input and output) Component | Hyperparameter | Value
---|---|---
Input INR | Layers Dimensions | [$3^{(in)}$, 128, 128, 128, 128,128, $1^{(out)}$]
Latent Dimension | 64
Sampling Std (scales) | 1
Hyper-network Layers Dimensions | [128]
Epochs | 7500
Output INR | Layers Dimensions | [$6^{(in)}$, 256, 256, 256, 256, 256, $1^{(out)}$]
Latent Dimension | 128
Sampling Std (scales) | [1,5]
Hyper-network Layers Dimensions | [128]
Epochs | 1000
Shared | Learning Rate (Outer Loop) | 3e-5
Learning Rate (Inner Loop) | 0.01
Inner Step | 3
Batch Size | 32
Table 8: Hyperparameters for INR Processor Hyperparameter | Value
---|---
Layers Dimensions | [$68^{(in)}$,128,128,128,$128^{(out)}$]
Layers Type | Residual
Activation Function | Silu
Learning Rate | 5e-6
Batch Size | 128
Epochs | 1000
Table 9: Hyperparameters for MeshGraphNet model Component | Hyperparameter | Value
---|---|---
Node Encoder | Layers Dimensions | [$7^{(in)}$, 64, 64, 64]
Activation Function | Relu
Edge Encoder | Layers Dimensions | [3, 64, 64, 64]
Activation Function | Relu
Decoder | Layers Dimensions | [64, 64, 64, $1^{(out)}$]
Activation Function | Relu
Processor | Number of blocks | 12
Shared | Batch Size | 4
Number of Epochs | 150
Learning Rate | 0.001

##  Appendix B Additional Results

###  B.1 Transonic Airfoil Dataset

##### Multi-Output INR

The proposed INR model can be used to perform multi-output prediction of
physical quantities. This is particularly relevant for the definition of
physics based losses, as governing laws typically relate multiple physical
quantities. In 10 an end-to-end INR surrogate model is trained to output the
pressure field and the two-dimensional velocity vector field around the
RAE2822 airfoil. The predictions are in good accordance with the reference CFD
targets. The choice of the frequency embedding can be more challenging in this
case, as different physical variables can have different frequency
characteristics: pressure fields are smoother and do not present sharp
gradients across the boundary layer and in the wake flow, compared to velocity
fields. This is an additional justification of the flexibility of the proposed
MultiScale INR architecture, as multiple frequency encoding can balance the
reconstruction accuracy of signals with different bandwidths.

##### Effect of encoding scale

The parameter $\sigma$ controlling the range of sampled embedding frequencies
can have an important influence on the network reconstructions, namely in
terms of underfitting and overfitting behaviour of the model. This was more in
detail discussed in Section 2 and briefly shown in the 1D regression example
of Fig. 1. Here, we complement the discussion by presenting a typical error
distribution plot (Fig. 11) for the model predictions performed with different
choices of the $\sigma$ parameter: $\sigma=1$, $\sigma=5$ and a Multiscale
Architecture with $\sigma=[1,5]$. Is is clear that the reconstructions for
higher values of $\sigma$ become noisy, as the kernel is not filtering out the
spurious high-frequency components present in the training dataset. The result
with $\sigma=1$ presents more localized error peaks in proximity of the shock
region, where sharp variation of the pressure fields are present. The
multiscale architecture appears to mediate the effect of the two different
scale embeddings, by producing slightly lower error peaks in the strongly non-
linear regions and removing the noisy patterns introduced by the larger kernel
bandwidth.

(a) $\alpha=2.04$,$M=0.86$

(b) $\alpha=3.45$,$M=0.80$

Figure 10: Top: Pressure Distribution. Center: Horizontal Local Mach number
$M_{x}$ (Horizontaly velocity normalized by the speed of sound). Bottom:
Vertical Local Mach Number $M_{y}$. Figure 11: Contour plot of the magnitude
of the prediction error using different values of $\sigma$. Left: $\sigma=1$
at $\alpha=0[deg]$,$M=0.84$. Center: $\sigma=5$. Right: $\sigma=[1,5]$.

##### POD and GPR

Additional baseline comparison with a POD+GPR methodology are shown. This
surrogate combines Proper Orthogonal Decomposition (POD) with Gaussian Process
Regression (GPR) to perform dimensionality reduction via linear mode
decomposition and regression in the low-dimensional space of POD coefficients.
This is a traditional surrogate model, widely used for fluid dynamics
application, thanks to the capability of preserving the more salient features
in the training dataset, and to respect physical constraints such as Dirichlet
boundary conditions and conservation of mass [3]. In this study, the truncated
Singular Value Deomposition of the snapshot matrix containing the training
samples is performed, keeping the first 50 modes. A Gaussian Process
Regressor, mapping the input parameters to the reduced coefficients, is
implemented in the SMT [64] with a Matérn kernel. Due to the fixed geometry
and resolution constraints of the POD formulation, this model offline and
online stage are carried out on the same full mesh. In Table 10 a summary of
the surrogate model metrics on the RAE2822 dataset is provided. Overall, the
method achieves error metrics which are competitive with state of the art
model (as GNNs), on problems with fixed grid settings.

While providing physically plausible results away from the shock region, most
of the model error are located in proximity of the pressure jump, mainly due
to the linear formulation of the method, as visible in Figure 12 and Figure
13, where the POD-GPR sligthly oversmooth the shock discontinuity. Not
surprisingly, this method can be efficiently trained in around few minutes:
this is a big advantage that makes it particularly attractive for rapid
surrogate modeling development. We note here, that we did not try to
experiment with more advance POD and GPR formulations, namely clustered POD
[8, 9].It is reasonable to assume that ad-hoc modification of the linear
decomposition formulation can improve the method performance. However, this
does not hold much relevance for the scope of the current study.

Table 10: Performance comparison of different models in terms of error (MSE), training time (s), and inference time (ms). Model | MSE | Training Time (s) | Inference Time (ms)
---|---|---|---
INR | 0.0020 | 9497 | 4
POD+GPR | 0.0040 | 260 | 10
Figure 12: Pressure field distribution prediction with different surrogate
models and comparison with CFD. Top: Angle of Attack [0.75 deg], Mach Number
[0.74]. Center: Angle of Attack [2.78 deg], Mach Number [0.85]. Bottom: Angle
of Attack [5.58 deg], Mach Number [0.72]. Figure 13: Surface Pressure
Coefficient distribution prediction with different surrogate models and
comparison with CFD. Left: Angle of Attack [0.75 deg], Mach Number [0.74].
Center: Angle of Attack [2.78 deg], Mach Number [0.85]. Right: Angle of Attack
[5.58 deg], Mach Number [0.72].