In order to benefit from neural operators in infinite-dimensional spaces, Li et al., 2020 developed a neural operator in the Fourier space, dubbed as FNO, with a core architecture schematically shown in Fig. <ref>. The training starts with an input $X$, which is subsequently elevated to a higher dimensional space by a neural network $S$. The second phase entails the use of several Fourier layers of integral operators and activation functions. In each Fourier layer, the input is transformed using (i) a Fourier transform, $F$; (ii) a linear transform, $T$, on the lower Fourier modes that  filters out the higher modes; and (iii) an inverse Fourier transform, $F^{-1}$. The input is also transformed using a local linear transform, $W$, before the application of the activation function, $\sigma$. The Fourier layers are designed to be discretization-invariant due to the fact that they learn from functions that are discretized arbitrarily. Indeed, the integral operator is applied in convolution and is represented as a linear transformation in the Fourier domain, allowing the FNO to learn the mapping over infinite-dimensional spaces. The result of the Fourier layer is  projected back to the target dimension in the third phase using another neural network $M$, which eventually outputs the desired output $Y'$  [Li et al., 2020]. Unlike other DL methods, the FNO model's error is consistent regardless of the input and output resolutions (e.g., in PgNN methods, the error grows with the resolution).

A schematic architecture of the Fourier Neural Operator (FNO) [Li et al., 2020].
Here, $\textbf{X}$ is the inputs, S and M are the neural network for dimensional space operation, and $\textbf{Y'}$ is the predicted outputs. Each of the Fourier layers consists of v(x) as the initial state, $\mathcal{F}$ layer that performs Fourier transform, $T$ layer that performs linear transform, $\mathcal{F}^{-1}$ layer that performs inverse Fourier transform, W layer for local linear transform, + block that carries out element-wise addition on W, and the final output from the $\mathcal{F}^{-1}$ layer. The final product from element-wise addition is passed on to a $\sigma$ layer.

Li et al., 2020 employed FNO on three different test cases, including the 1D Burgers' equation, the 2D Darcy flow equation, and the 2D Navier-Stokes equations. For each test case, FNO was compared with state-of-the-art models. In particular, for Burgers' and Darcy's test cases, the methods used for comparison were the conventional ANN (i.e., PgNN),  reduced bias method [DeVore, 2017], fully convolutional networks [Zhu and Zabaras, 2018], principal component analysis as an encoder in the neural network [Bhattacharya et al., 2020], graph neural operator [Li et al., 2020], and low-rank decomposition neural operator (i.e., unstacked DeepONet [Lu et al., 2019]). In all test cases, FNO yielded the lowest relative error. The models' error comparisons for 1D Burgers' and 2D Darcy flow equations are depicted in Fig. <ref>, adapted from [Li et al., 2020].

Error comparison between FNO and other state-of-the-art methods (reduced bias method (RBM), fully convolutional network (FCN), graph neural operator (GNO), and graph convolutional network (GCN)) for (a) Burgers' equation and (b) Darcy's Flow equation at different resolutions (adapted from Li et al., 2020).

The FNO model, as stated, can be trained on a specific resolution and tested on a different resolution. Li et al., 2020 demonstrated this claim by training a FNO on the Navier-Stokes equations for a 2D test case with a resolution of 64$\times$64$\times$20 ($n_x,n_y,n_t$) standing for spatial (x, y) and time resolution, and then evaluating it with a resolution of 256$\times$256$\times$80, as shown in Fig. <ref>(a). In comparison to other models, the FNO was the only technique capable of performing resolution downscaling both spatially and temporally
[Li et al., 2020]. FNOs can also achieve several orders of magnitude speedup factors over conventional numerical PDE solvers. However, they have only been used for 2D or small 3D problems due to the large dimensionality of their input data, which increases the number of network weights significantly. With this problem in mind, Grady et al., 2022 proposed a parallelized version of FNO based on domain-decomposition to resolve this limitation. Using this extension, they were able to use FNO in large-scale modeling, e.g., simulating the transient evolution of the CO2 plume in subsurface heterogeneous reservoirs as a part of the carbon capture and storage (CCS) technology [Bui et al., 2018], see Fig. <ref>(b). The input to the network (with a similar architecture to the one proposed by Li et al., 2020) was designed to be a tensor containing both the permeability and topography fields at each 3D spatial position using a 60$\times$60 $\times$64 ($n_x,n_y,n_z$) resolution and the output was 60$\times$60$\times$64$\times n_t$. For a time resolution of $n_t = 30 ~s$, they found that the parallelized FNO model was 271 times faster (without even leveraging GPU) than the conventional porous media solver while achieving comparable accuracy. Wen et al., 2022 also proposed U-FNO, an extension of FNO, to simulate multiphase flows in porous media, specifically CO2-water multiphase flow through a heterogeneous medium with broad ranges of reservoir conditions, injection configurations, flow rates, and multiphase flow properties. They compared U-FNO with FNO and CNN (i.e., PgNN) and showed that the U-FNO architecture provides the best performance for both gas saturation and pressure buildup predictions in highly heterogeneous geological formations. They also showed that the U-FNO architecture enhances the training accuracy of the original FNO, but does not naturally enable the flexibility of training and testing at multiple discretizations.

A qualitative comparison between predictions made by the Fourier
neural operator (FNO) and ground truth values. Panel (a) shows the comparison for a FNO model trained on 64$\times$64$\times$20 resolution ($n_x,n_y,n_t$) and evaluated on 256$\times$256$\times$80 resolution to solve 2D Navier-Stokes equations (adapted from Li et al., 2020). Panel (b) shows the comparison for a parallelized FNO model trained to solve large-scale 3D subsurface CO2 flow modeling evaluated at 60$\times$60$\times$64$\times n_t$ resolution (adapted from Grady et al., 2022).

You et al., 2022 proposed an implicit Fourier neural operator (IFNO) to model the complex responses of materials due to their heterogeneity and defects without using conventional constitutive models. The IFNO model captures the long-range dependencies in the feature space, and as the network becomes deeper, it becomes a fixed-point equation that yields an implicit neural operator (e.g., it can mimic displacement/damage fields). You et al., 2022 demonstrated the performance of IFNO using a series of test cases such as hyperelastic, anisotropic, and brittle materials. Fig. <ref> depicts a comparison between IFNO and  FNO for the transient propagation of a glass-ceramic crack [You et al., 2022]. As demonstrated, IFNO outperforms  FNO (in terms of accuracy) and conventional constitutive models (in terms of computational cost) to predict the displacement field.

The FNO model has also been hybridized with PiNN to create the so-called physics-informed neural operator (PiNO) [Li et al., 2021]. The PiNO framework is a combination of operating-learning (i.e., FNO) and function-optimization (i.e., PiNN) frameworks that improves convergence rates and accuracy over both PiNN and FNO models. This integration was suggested to address the challenges in PiNN (e.g., generalization and optimization, especially for multiscale dynamical systems) and the challenges in FNO (e.g., the need for expensive and impractical large training datasets) [Li et al., 2021]. Li et al., 2021 deployed the PiNO model on several benchmark problems (e.g., Kolmogorov flow, lid-cavity flow, etc.) to show that PiNO can outperform PiNN and FNO models while maintaining the FNO's exceptional speed-up factor over other solvers.

A comparison between the implicit Fourier neural operator (IFNO) and FNO models to predict crack propagation and damage field within a region of interest (ROI) in a pre-cracked glass-ceramics experiment with randomly distributed material property fields (adapted from You et al., 2022).

A non-exhaustive  list of recent studies that leveraged Neural Operators to model different scientific problems.

NO Structure      Objective        Reference

DeepONets       Learning non-linear operators accurately and efficiently with low generalization error        [Lu et al., 2021]

CADeepONet       Learning dynamic development of a two-phase mixture and reducing solution time for predicting microstructure evolution      [Oommen et al., 2022]

Pi-DeepONets     Integrating DeepONets and PiNN to relax the requirement for a large training dataset while improving generalization and predictive accuracy      [Wang et al., 2021]

V-DeepONet     Generating a fast and generalizable surrogate model for brittle fracture mechanics      [Goswami et al., 2022]

FNO       Providing a mesh- and discretization-invariant model to be inferred at any arbitrary spatiotemporal resolutions        [Li et al., 2020]

Parallelized FNO       Extending FNO based on domain-decomposition to model large-scale three-dimensional problems       [Grady et al., 2022]

U-FNO     Enhancing the training and testing accuracy of FNO for large-scale, highly heterogeneous geological formations        [Wen et al., 2022]

IFNO     capturing the long-range dependencies in the feature space  that yields an implicit neural operator to model the complex responses of materials due to their heterogeneity and defects        [You et al., 2022]

PINO      Integrating PiNN and FNO to relax the requirement for a large training dataset while enhancing  generalization and optimization     [Li et al., 2021]

§.§ NOs Limitations

DeepONet [Lu et al., 2019] and FNO [Li et al., 2020], as the two most common neural operators to date, share some commonalities while also having significant differences. The  DeepONet architecture was inspired by Chen and Chen, 1995's universal approximation theorems, whereas the FNO was architecture established on parameterizing the integral kernel in Fourier space. However, FNO in its continuous form can be viewed as DeepONet with a specific architecture of the trunk  (expressed by a trigonometric basis) and  branch networks [Kovachki et al., 2021]. FNO, unlike DeepONet, discretizes both the input function and output function via point-wise evaluations in an equally spaced mesh.  Therefore, after network training, FNO can only predict the solution in the same mesh as the input function, but DeepONet can make predictions at any arbitrary location. FNO also requires a full field of observation data for training, whereas DeepONet is more flexible, with the exception of POD-DeepONet [Lu et al., 2022], which requires a full field of observation data to calculate the proper orthogonal decomposition (POD) modes [Lu et al., 2022]. DeepONet, FNO, and their various variants still face some limitations, especially when applied to large multi-physics problems, that  necessitate further investigations.

* Neural operators are purely data-driven and require relatively large  training datasets, therefore they face constraints when applied to problems where data acquisition is complex and/or costly   [Lu et al., 2022]. Integration with PiNN can resolve this issue to some extent for problems where the underlying physics is fully known and can be integrated into the loss function [Wang et al., 2021]. Also, for practical applications, training the NOs just based on the governing equations in the loss function may produce inaccurate predictions; instead, a hybrid physics-data training  is recommended [Wang et al., 2021].

* DeepONet and FNO are typically limited to basic geometry and/or structured data (e.g., 2D or small 3D problems)  due to the large dimensionality of their input data that increases the number of network weights significantly) [Lu et al., 2022]. They are also prone to over-fitting as the number of trainable parameters increases, making the training process more difficult [You et al., 2022]. IFNOs have addressed this challenge to some extent [You et al., 2022]. In IFNO, the solution operator is first formulated as an implicitly defined mapping and then  modeled as a fixed point. The latter aims to overcome the challenge of network training in the case of deep layers, and the former minimizes the number of trainable parameters and memory costs. Nonetheless, due to the finite size of the neural network architecture, the convergence of  NOs (e.g., DeepONet) error with respect to the size of the training data becomes algebraic  for large datasets; it is desired to be exponential [Lu et al., 2022].

* FNO might not be reliable for discontinuous functions as it relies on the Fourier transformation. This is mitigated to some extent by DeepONet, as it was shown to perform well for functions with discontinuity (e.g., compressible Euler equations) [Lu et al., 2022].

Despite these limitations, neural operators are the leading algorithms in a variety of real-time inference applications, including autonomous systems, surrogates in design problems, and uncertainty quantification [Goswami et al., 2022].

§ CONCLUSIONS AND FUTURE RESEARCH DIRECTIONS

A considerable number of research topics collectively support the efficacy of combining scientific computing and deep learning approaches. In particular, this combination improves the efficiency of both forward and inverse modeling for high-dimensional problems that are prohibitively expensive, contain noisy data, require a complex mesh, and are governed by non-linear, ill-posed differential equations. The ever-increasing computer power will continue to further furnish this combination by allowing the use of deeper neural networks and considering higher-dimensional interdependencies and design space.

A comparison of the main characteristics of PgNNs, PiNNs, PeNNs, and NOs to model non-linear multiscale phenomena in computational fluid and solid mechanics.

Feature     PgNNs     PiNNs     PeNNs     NOs

Accelerating Capability

Mesh-free Simulation

Straightforward Network Training

Training without Labeled Data

Physics-informed Loss Function

Continuous Solution

Spatiotemporal Interpolation

Physics Encoding

Efficient Operator Learning

Continuous-depth Models

Spatiotemporal Extrapolation

Solution Transferability

Efficient Real-time Predictions

The combination of scientific computing and deep learning approaches also surpasses traditional computational mechanics solvers in a number of prevalent scenarios in practical engineering. For example, a sparse dataset obtained experimentally for a complex (i.e., hard-to-acquire data) phenomenon cannot be simply integrated with traditional solvers. Whereas using DL, the following tasks can be performed: (i) PgNN-based models can be applied to the sparse data to extract latent interdependencies and conduct spatiotemporal downscaling or upscaling (i.e., interpolated data); (ii) PiNN-based models can be applied to the interpolated data to deduce governing equations and potentially unknown boundary or initial conditions of the phenomena (i.e., strong mathematical form); (iii) PeNN-based models can be used to combine the interpolated data and the strong mathematical form to conduct extrapolation exploration; and (iv) NO-based models can be applied to make real-time predictions of the complex dynamics. Therefore, the combination of DL-based methods and traditional scientific computing methods provides scientists with a cost-effective toolbox to explore problems across different scales that were deemed far-fetched computationally. To this end, several other breakthroughs in DL are required to enable the use of PgNNs, PiNNs, PeNNs, and NOs in large-scale three-dimensional (or multi-dimensional) problems. For instance, the training of complex DL models (e.g., PiNNs,  PeNNs, and NOs) should be accelerated using different parallelization paradigms.

Table <ref> compares the main characteristics of the PgNNs, PiNNs, PeNNs, and NOs. The PgNN-based models suffer mainly from their statistical training process, for which they require large datasets. They map carefully curated training datasets  only based on correlations in statistical variations, and hence, their predictions are naturally physics-agnostic. The PiNN-based models suffer mainly from the presence of competing loss terms that may destabilize the training process. PiNN is also a solution learning algorithm with limited generalizability due to its inability to learn the physical operation of a specific phenomenon. Models based on PeNNs and NOs, on the other hand, may experience low convergence rates and require a large volume of paired, structured datasets, leading to highly expensive training.

Considering the effectiveness of this new challenge of combining scientific computing and DL, future studies can be divided into three distinct categories: (i) Improving algorithms: Developing advanced variants of PgNNs, PiNNs, PeNNs and NOs  that offer simpler implementation with enhanced convergence rate; faster training in multi-dimensional and multi-physics problems; higher accuracy and generalization to unseen conditions while using sparse training datasets, more robust to be used in real time forecasting; better adaptability to multi-spatiotemporal-resolutions, more flexibility to encode various types of governing equations (e.g., all PDE types, closure laws, data-driven laws, etc.), and provide a closer tie with a plethora of traditional solvers; (ii) Considering causalities: Developing a causal training algorithm (e.g., causal Q-learning [Molina et al., 2020]) that restores physical causality during the training of PgNN, PiNN, and PeNN models by re-weighting the governing equations (e.g., PDEs) residual loss at each iteration. This line of research will allow for the development of causality-conforming variants of PgNN, PiNN, and PeNN algorithms that can bring new opportunities for the application of these algorithms to a wider variety of complex scenarios across diverse domains; (iii) Expanding applications: Leveraging the potentials of PgNNs, PiNNs,  PeNNs, and NOs in problems with complex anisotropic materials (e.g., flow in highly heterogeneous porous media, metal and non-metal particulate composites, etc.); problems with multiscale multi-physics phenomena (e.g., magnetorheological fluids, particle-laden fluids, dry powder dynamics, reactive transport, unsaturated soil dynamics,  etc.); problems with multi-resolution objectives and extensive spatiotemporal downscaling or upscaling  (e.g., global and regional climate modeling, geosystem reservoir modeling, etc.); and structural health monitoring (e.g., crack identification and propagation, hydrogen pipeline leakage, CO2 plume detection, etc.);  and (iv) Coupling solvers: Coupling PgNNs, PiNNs, and PeNNs as well as NOs  with open-source computational mechanics packages such as OpenIFEM, OpenFOAM, Palabos, LAMMPS, LIGGGHTS, MOOSE, etc. This line of research will allow for faster surrogate modeling and, hence, faster development of next-generation solvers. It also expedites community and industry adoption of the combined scientific-DL computational paradigm.

§ ACKNOWLEDGEMENTS

S.A.F. would like to acknowledge supports from the Department of Energy Biological and Environmental Research (BER) (award no. DE-SC0023044), National Science Foundation Partnership for Research and Education in Materials (PREM) (award no. DMR-2122041), and Texas State University Multidisciplinary Internal Research Grant (MIRG) (award no. 9000003028).

C.F. would like to acknowledge supports from FEDER funds through the COMPETE 2020 Programme and National Funds through FCT (Portuguese Foundation for Science and Technology) under the projects UID-B/05256/2020, UID-P/05256/2020 and MIT-EXPL/TDI/0038/2019-APROVA-Deep learning for particle-laden viscoelastic flow modelling (POCI-01-0145-FEDER-016665) under the MIT Portugal program.

§ CONFLICT OF INTEREST

The authors declare no conflict of interest.

[Vinuesa and Brunton, 2022]
Ricardo Vinuesa and Steven L Brunton.
Enhancing computational fluid dynamics with machine learning.
Nature Computational Science, 20 (6):0
358–366, 2022.

[Mianroodi et al., 2021]
J. R. Mianroodi, N. H. Siboni, and D. Raabe.
Teaching solid mechanics to artificial intelligence - a fast solver
for heterogeneous materials.
NPJ Computational Materials, 7:0 99, 2021.

[Kim et al., 2021]
Y. Kim, C. Yang, K. Park, G. X. Gu, and R. Seunghwa.
Deep learning framework for material design space exploration using
active transfer learning and data augmentation.
npj Computational Materials, 7:0 140, 2021.

[Dino et al., 2020]
H. I. Dino, S. R. Zeebaree, A. A. Salih, R. R. Zebari, Z. S. Ageed, H. M.
Shukur, L. M. Haji, and S. S. Hasan.
Impact of process execution and physical memory-spaces on os
Technology Reports of Kansai University, 620
(5):0 2391–2401, 2020.

[Im et al., 2021]
S. Im, J. Lee, and M. Cho.
Surrogate modeling of elasto-plastic problems via long short-term
memory neural networks and proper orthogonal decomposition.
Computer Methods in Applied Mechanics and Engineering,
385:0 114030, 2021.

[Karniadakis et al., 2021]
G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and
L. Yang.
Physics-informed machine learning.
Nature Reviews Physics, 30 (6):0 422–440,

[Arman and Anthony, 2022]
S. Arman and W. Anthony.
Physics-inspired architecture for neural network modeling of forces
and torques in particle-laden flows.
Computers and Fluids, 238:0 105379, 2022.

[Innes et al., 2019]
M. Innes, A. Edelman, K. Fischer, C. Rackauckas, E. Saba, V.B. Shah, and
W. Tebbutt.
A differentiable programming system to bridge machine learning and
scientific computing.
arXiv preprint arXiv:1907.07587, 2019.

[Brunton et al., 2020]
S. L. Brunton, B. R. Noack, and P. Koumoutsakos.
Machine learning for fluid mechanics.
Annual Review of Fluid Mechanics, 52:0 477–508, 2020.

[Cai et al., 2022]
S. Cai, Z. Mao, Z. Wang, M. Yin, and G. E. Karniadakis.
Physics-informed neural networks (pinns) for fluid mechanics: A
Acta Mechanica Sinica, pages 1–12, 2022a.

[Kutz, 2017]
J. N. Kutz.
Deep learning in fluid dynamics.
Journal of Fluid Mechanics, 814:0 1–4, 2017.

[Shi et al., 2019]
Z. Shi, E. Tsymbalov, M. Dao, S. Suresh, A. Shapeev, and J. Li.
Deep elastic strain engineering of bandgap through machine learning.
Proc. Natl. Acad. Sci., 116:0 4117–4122, 2019.

[Haghighat et al., 2021]
E. Haghighat, M. Raissi, A. Moure, H. Gomez, and R. Juanes.
A physics-informed deep learning framework for inversion and
surrogate modeling in solid mechanics.
Computer Methods in Applied Mechanics and Engineering,
379:0 113741, 2021a.

[Pilania et al., 2013]
G. Pilania, C. Wang, Jiang X., S. Rajasekaran, and R. Ramprasad.
Accelerating materials property predictions using machine learning.
Sci. Rep., 3:0 1–6, 2013.

[Butler et al., 2018]
K. T. Butler, D. W. Davies, H. Cartwright, O. Isayev, and A. Walsh.
Machine learning for molecular and materials science.
Nature, 559:0 547–555, 2018.

[Brunton and Kutz, 2019]
S. L. Brunton and J. N. Kutz.
Methods for data-driven multiscale model discovery for materials.
J. Phys. Mater., 2:0 044002, 2019.

[Bedolla et al., 2020]
E. Bedolla, L. C. Padierna, and R. Castaneda-Priego.
Machine learning for condensed matter physics.
Journal of Physics: Condensed Matter, 330
(5):0 053001, 2020.

[Kochkov et al., 2021]
D. Kochkov, J. A. Smith, A. Alieva, Q. Wang, M. P. Brenner, and S. Hoyer.
Machine learning–accelerated computational fluid dynamics.
Proceedings of the National Academy of Sciences, 118:0
e2101784118, 2021.

[Tran et al., 2017]
D. Tran, M. D. Hoffman, R. A. Saurous, E. Brevdo, K. Murphy, and D.M. Blei.
Deep probabilistic programming.
arXiv preprint arXiv:1701.03757, 2017.

[Jin et al., 2017]
K. H. Jin, M.T. McCann, E. Froustey, and M. Unser.
Deep convolutional neural network for inverse problems in imaging.
IEEE Transactions on Image Processing, 260
(9):0 4509–4522, 2017.

[Lai et al., 2022]
Z. Lai, Q. Chen, and L. Huang.
Machine-learning-enabled discrete element method: Contact detection
and resolution of irregular-shaped particles.
International Journal for Numerical and Analytical Methods in
Geomechanics, 460 (1):0 113–140, 2022.

[Faroughi et al., 2022]
S. A. Faroughi, A. I. Roriz, and C. Fernandes.
A meta-model to predict the drag coefficient of a particle
translating in viscoelastic fluids: a machine learning approach.
Polymers, 140 (3):0 430, 2022a.

[Taylor, 1715]
B. Taylor.
Methodus incrementorum directa & inversa. Auctore Brook
Taylor, LL. D. & Regiae Societatis Secretario.
typis Pearsonianis: prostant apud Gul. Innys ad Insignia Principis
in …, 1715.

[Alder and Wainwright, 1957]
B. J. Alder and T. E. Wainwright.
Phase transition for a hard sphere system.
The Journal of chemical physics, 270 (5):0
1208–1209, 1957.

[Clough, 1960]
R. W. Clough.
The finite element method in plane stress analysis.
In Proceedings of 2nd ASCE Conference on Electronic
Computation, Pittsburgh Pa., Sept. 8 and 9, 1960, 1960.

[Smagorinsky, 1963]
J. Smagorinsky.
General circulation experiments with the primitive equations: I. the
basic experiment.
Monthly weather review, 910 (3):0 99–164,

[Cundall and Strack, 1979]
P. A. Cundall and O. Strack.
A discrete numerical model for granular assemblies.
Géotechnique, 29:0 47–65, 1979.

[McDonald, 1971]
P. W. McDonald.
The computation of transonic flow through two-dimensional gas
turbine cascades, volume 79825.
American Society of Mechanical Engineers, 1971.

[Peskin, 1972]
C. S. Peskin.
Flow patterns around heart valves: a numerical method.
Journal of computational physics, 100 (2):0
252–271, 1972.

[Lucy, 1977]
L. B. Lucy.
A numerical approach to the testing of the fission hypothesis.
The astronomical journal, 82:0 1013–1024, 1977.

[D'Humieres et al., 1986]
D. D'Humieres, P. Lallemand, and U. Frisch.
Lattice gas models for 3d hydrodynamics.
EPL (Europhysics Letters), 20 (4):0 291, 1986.

[Bassi and Rebay, 1997]
F. Bassi and S. Rebay.
A high-order accurate discontinuous finite element method for the
numerical solution of the compressible navier–stokes equations.
Journal of computational physics, 1310 (2):0
267–279, 1997.

[Ivakhnenko and Lapa, 1967]
A. G. Ivakhnenko and V. G. Lapa.
Cybernetics and forecasting techniques, volume 8.
American Elsevier Publishing Company, 1967.

[Rumelhart et al., 1986]
D. E. Rumelhart, G. E. Hinton, and R. J. Williams.
Learning representations by back-propagating errors.
nature, 3230 (6088):0 533–536, 1986.

[Andersen et al., 1990]
K. Andersen, G. E. Cook, G. Karsai, and K. Ramaswamy.
Artificial neural networks applied to arc welding process modeling
and control.
IEEE Transactions on industry applications, 260
(5):0 824–830, 1990.

[LeCun et al., 1998]
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner.
Gradient-based learning applied to document recognition.
Proceedings of the IEEE, 860 (11):0
2278–2324, 1998.

[J. et al., 2014]
Goodfellow I. J., P. Jean, M. Mehdi, X. Bing, W. David, O. Sherjil, and
C. Courville Aaron.
Generative adversarial nets.
In Proceedings of the 27th international conference on neural
information processing systems, volume 2, pages 2672–2680, 2014.

[Raissi et al., 2017]
M. Raissi, P. Perdikaris, and G. E. Karniadakis.
Physics informed deep learning (part i): Data-driven solutions of
nonlinear partial differential equations.
arXiv preprint arXiv:1711.10561, 2017.

[Lu et al., 2019]
L. Lu, P. Jin, and G. E. Karniadakis.
Deeponet: Learning nonlinear operators for identifying differential
equations based on the universal approximation theorem of operators.
arXiv preprint arXiv:1910.03193, 2019.

[40]
C. Rao, H. Sun, and Y. Liu.
Hard encoding of physics for learning spatiotemporal dynamics.
arXiv preprint arXiv:2105.00557, a.

[Lienen and Gunnemann, 2022]
M. Lienen and S. Gunnemann.
Learning the dynamics of physical systems from sparse observations
with finite element networks.
In The Tenth International Conference on Learning
Representations. OpenReview, 2022.

[Hasson et al., 2020]
U. Hasson, S.A. Nastase, and A. Goldstein.
Direct fit to nature: an evolutionary perspective on biological and
artificial neural networks.
Neuron, 1050 (3):0 416–434, 2020.

[Li et al., 2021]
Z. Li, H. Zheng, N. Kovachki, D. Jin, H. Chen, B. Liu, K. Azizzadenesheli, and
A. Anandkumar.
Physics-informed neural operator for learning partial differential
arXiv preprint arXiv:2111.03794, 2021a.

[Raissi et al., 2019]
M. Raissi, P. Perdikaris, and G. E. Karniadakis.
Physics-informed neural networks: A deep learning framework for
solving forward and inverse problems involving nonlinear partial differential
Journal of Computational Physics, 378:0 686–707,

[Nabian and Meidani, 2020]
M. A. Nabian and H. Meidani.
Adaptive physics-informed neural networks for markov-chain monte
arXiv preprint arXiv:2008.01604, 2020.

[Cuomo et al., 2022]
S. Cuomo, V. S. Di Cola, F. Giampaolo, G. Rozza, M. Raissi, and F. Piccialli.
Scientific machine learning through physics-informed neural networks:
Where we are and what's next.
arXiv preprint arXiv:2201.05624, 2022.

[Baydin et al., 2015]
A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind.
Automatic differentiation in machine learning: a survey.
arXiv:1502.05767, 2015.

[Rao et al., 2020]
C. Rao, S. Hao, and L. Yang.
Physics-informed deep learning for incompressible laminar flows.
Theoretical and Applied Mechanics Letters, 10:0
207–212, 2020a.

[Faroughi et al., 2022]
Salah A Faroughi, Pingki Datta, Seyed Kourosh Mahjour, and Shirko Faroughi.
Physics-informed neural networks with periodic activation functions
for solute transport in heterogeneous porous media.
arXiv preprint arXiv:2212.08965, 2022b.

[McClenny and Braga-Neto, 2020]
L. McClenny and U. Braga-Neto.
Self-adaptive physics-informed neural networks using a soft attention
arXiv preprint arXiv:2009.04544, 2020.

[Yadav et al., 2022]
G. K. Yadav, S. Natarajan, and B. Srinivasan.
Distributed pinn for linear elasticity—a unified approach for
smooth, singular, compressible and incompressible media.
International Journal of Computational Methods, page 2142008,

[Bauer et al., 2021]
P. Bauer, P. D. Dueben, T. Hoefler, T. Quintino, T. C. Schulthess, and N. P.
The digital revolution of earth-system science.
Nature Computational Science, 10 (2):0
104–113, 2021.

[Chen et al., 2018]
R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud.
Neural ordinary differential equations.
Advances in neural information processing systems, 31, 2018.

[Chung et al., 2016]
H. Chung, S. J. Lee, and J. G. Park.
Deep neural network using trainable activation functions.
In 2016 International Joint Conference on Neural Networks
(IJCNN), pages 348–352. IEEE, 2016.

[Mattheakis et al., 2019]
M. Mattheakis, P. Protopapas, D. Sondak, G. M. Di, and E. Kaxiras.
Physical symmetries embedded in neural networks.
arXiv preprint arXiv:1904.08991, 2019.

[Lu et al., 2021]
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis.
Learning nonlinear operators via deeponet based on the universal
approximation theorem of operators.
Nature Machine Intelligence, 30 (3):0
218–229, 2021a.

[Goswami et al., 2022]
S. Goswami, A. Bora, Y. Yu, and G. E. Karniadakis.
Physics-informed neural operators.
arXiv preprint arXiv:2207.05748, 2022a.

[LeCun et al., 2015]
Y. LeCun, Y. Bengio, and G. Hinton.
Deep learning.
nature, 5210 (7553):0 436–444, 2015.

[Creswell et al., 2018]
A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and A. A.
Generative adversarial networks: An overview.
IEEE Signal Processing Magazine, 350 (1):0
53–65, 2018.

[Scarselli et al., 2008]
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini.
The graph neural network model.
IEEE transactions on neural networks, 200
(1):0 61–80, 2008.

[Goodfellow et al., 2020]
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair,
A. Courville, and Y. Bengio.
Generative adversarial networks.
Communications of the ACM, 630 (11):0
139–144, 2020.

[Rasamoelina et al., 2020]
A. D. Rasamoelina, F. Adjailia, and P. Sinčák.
A review of activation function for artificial neural network.
In 2020 IEEE 18th World Symposium on Applied Machine
Intelligence and Informatics (SAMI), pages 281–286. IEEE, 2020.

[He et al., 2020]
C. He, M. Ma, and P. Wang.
Extract interpretability-accuracy balanced rules from artificial
neural networks: A review.
Neurocomputing, 387:0 346–358, 2020.

[Li et al., 2014]
M. Li, T. Zhang, Y. Chen, and A. J. Smola.
Efficient mini-batch training for stochastic optimization.
In Proceedings of the 20th ACM SIGKDD international conference
on Knowledge discovery and data mining, pages 661–670, 2014.

[Huang et al., 2021]
K. Huang, M. Krugener, A. Brown, F. Menhorn, H. J. Bungartz, and D. Hartmann.
Machine learning-based optimal mesh generation in computational fluid
arXiv preprint: 2102.12923v1, 2021.

[Kumar and Kochmann, 2022]
S. Kumar and D. M. Kochmann.
What machine learning can do for computational solid mechanics.
In Current Trends and Open Problems in Computational
Mechanics, pages 275–285. Springer, 2022.

[Guo et al., 2021]
K. Guo, Z. Yang, C-H. Yu, and M. J. Buehler.
Artificial intelligence and machine learning in design of mechanical
Materials Horizons, 80 (4):0 1153–1172, 2021.

[Zhang et al., 2020]
Z. Zhang, Y. Wang, P. K. Jimack, and H. Wang.
MeshingNet: a new mesh generation method based on deep learning.
arXiv preprint: 2004.07016v1, 2020a.

[Wu et al., 2022]
T. Wu, X. Liu, W. An, Z. Huang, and H. Lyu.
A mesh optimization method using machine learning technique and
variational mesh adaptation.
Chinese Journal of Aeronautics, 35:0 27–41, 2022.

[Mendizabal et al., 2020]
A. Mendizabal, P. Márquez-Neila, and S. Cotin.
Simulation of hyperelastic materials in real-time using deep
Medical image analysis, 59:0 101569, 2020.

[Lu et al., 2021]
L. Lu, X. Gao, J-F. Dietiker, M. Shahnam, and W. A. Rogers.
Machine learning accelerated discrete element modeling of granular
Chemical Engineering Science, 245:0 116832,

[Li et al., 2021]
Z. Li, K. Meidani, P. Yadav, and A. B. Farimani.
Graph neural networks accelerated molecular dynamics.
arXiv preprint arXiv:2112.03383, 2021b.

[Menke et al., 2021]
H. P. Menke, J. Maes, and S. Geiger.
Upscaling the porosity–permeability relationship of a microporous
carbonate for darcy-scale flow with machine learning.
Scientific Reports, 110 (1):0 1–10, 2021.

[Cheng et al., 2022]
S. Cheng, J. Chen, C. Anastasiou, P. Angeli, O. K. Matar, Y. Guo, C. C. Pain,
and R. Arcucci.
Generalised latent assimilation in heterogeneous reduced spaces with
machine learning surrogate models.
arXiv preprint arXiv:2204.03497, 2022.

[Zawawi et al., 2018]
M. H. Zawawi, A. Saleha, A. Salwa, N. H. Hassan, N. M. Zahari, M. Z. Ramli, and
Z. C. Muda.
A review: Fundamentals of computational fluid dynamics (cfd).
In AIP conference proceedings, volume 2030, page 020252. AIP
Publishing LLC, 2018.

[He and Tafti, 2019]
L. He and D. K. Tafti.
A supervised machine learning approach for predicting variable drag
forces on spherical particles in suspension.
Powder technology, 345:0 379–389, 2019.

[Zhu et al., 2020]
L-T. Zhu, J-X. Tang, and Z-H. Luo.
Machine learning to assist filtered two-fluid model development for
dense gas–particle flows.
AIChE Journal, 660 (6):0 e16973, 2020.

[Roriz et al., 2021]
A. I. Roriz, S. A. Faroughi, G. H. McKinley, and C. Fernandes.
Ml driven models to predict the drag coefficient of a sphere
translating in shear-thinning viscoelastic fluids.

[Loiro et al., 2021]
C. Loiro, C. Fernandes, G. H. McKinley, and S. A. Faroughi.
Digital-twin for particle-laden viscoelastic fluids: Ml-based models
to predict the drag coefficient of random arrays of spheres.

[Webb et al., 2020]
M. A. Webb, N. E. Jackson, P. S. Gil, and J. J. de Pablo.
Targeted sequence design within the coarse-grained polymer genome.
Science advances, 60 (43):0 eabc6216, 2020.

[Srivastava et al., 2014]
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov.
Dropout: a simple way to prevent neural networks from overfitting.
The journal of machine learning research, 150
(1):0 1929–1958, 2014.

[Bejani and Ghatee, 2021]
M. M. Bejani and M. Ghatee.
A systematic review on overfitting control in shallow and deep neural
Artificial Intelligence Review, 540 (8):0
6391–6438, 2021.

[Ying, 2019]
X. Ying.
An overview of overfitting and its solutions.
In Journal of physics: Conference series, volume 1168, page
022022. IOP Publishing, 2019.

[Cati et al., 2022]
Y. Cati, S. aus der Wiesche, and M. Düzgün.
Numerical model of the railway brake disk for the temperature and
axial thermal stress analyses.
Journal of Thermal Science and Engineering Applications,
140 (10):0 101014, 2022.

[Chen et al., 2020]
X. Chen, J. Liu, Y. Pang, J. Chen, L. Chi, and C. Gong.
Developing a new mesh quality evaluation method based on
convolutional neural network.
Engineering Applications of Computational Fluid Mechanics,
14:0 391–400, 2020.

[Maddu et al., 2021]
S. Maddu, D. Sturm, B. L. Cheeseman, C. L. Müller, and I. F. Sbalzarini.
Stencil-net: Data-driven solution-adaptive discretization of partial
differential equations.
arXiv preprint arXiv:2101.06182, 2021.

[Bar-Sinai et al., 2019]
Y. Bar-Sinai, S. Hoyer, J. Hickey, and M. P. Brenner.
Learning data-driven discretizations for partial differential
Proceedings of the National Academy of Sciences, 1160
(31):0 15344–15349, 2019.

[Bernardin et al., 2010]
F. Bernardin, M. Bossy, C. Chauvin, J-F. Jabir, and A. Rousseau.
Stochastic lagrangian method for downscaling problems in
computational fluid dynamics.
ESAIM: Mathematical Modelling and Numerical Analysis,
440 (5):0 885–920, 2010.

[Wei et al., 2016]
X. Wei, C. Dong, Z. Chen, K. Xiao, and X. Li.
The effect of hydrogen on the evolution of intergranular cracking: a
cross-scale study using first-principles and cohesive finite element methods.
RSC advances, 60 (33):0 27282–27292, 2016.

[Shu, 1998]
C. Shu.
Essentially non-oscillatory and weighted essentially non-oscillatory
schemes for hyperbolic conservation laws.
Advanced numerical approximation of nonlinear hyperbolic
equations, pages 325–432, 1998.

[Zhang et al., 2021]
Z. Zhang, P. K. Jimack, and H. Wang.
Meshingnet3d: Efficient generation of adapted tetrahedral meshes for
computational mechanics.
Advances in Engineering Software, 157:0 103021,

[Triantafyllidis and Labridis, 2002]
D. G. Triantafyllidis and D. P. Labridis.
A finite-element mesh generator based on growing neural networks.
IEEE Transactions on neural networks, 130
(6):0 1482–1496, 2002.

[Srasuay et al., 2010]
K. Srasuay, A. Chumthong, and S. Ruangsinchaiwanich.
Mesh generation of fem by ann on iron—core transformer.
In 2010 International Conference on Electrical Machines and
Systems, pages 1885–1890. IEEE, 2010.

[Lee and Chen, 1993]
M. J. Lee and J. T. Chen.
Fluid property predictions with the aid of neural networks.
Industrial & engineering chemistry research, 320
(5):0 995–997, 1993.

[Yang et al., 2016]
C. Yang, X. Yang, and X. Xiao.
Data-driven projection method in fluid simulation.
Computer Animation and Virtual Worlds, 27:0 415–424,

[Tompson et al., 2016]
J. Tompson, K. Schlachter, P. Sprechmann, and K. Perlin.
Accelerating Eulerian fluid simulation with convolutional networks,

[Jacobs, 1980]
D.A.H. Jacobs.
Preconditioned conjugate gradient methods for solving systems of
algebraic equations.
Technical Report RD/L/N193/80, Central Electricity Research
Laboratories, 1980.

[Chen et al., 2019]
J. Chen, J. Viquerat, and E. Hachem.
U-net architectures for fast prediction of incompressible laminar
arXiv preprint arXiv:1910.13532, 2019a.

[Deng et al., 2019]
Z. Deng, C. He, Y. Liu, and K. Kim.
Super-resolution reconstruction of turbulent velocity fields using a
generative adversarial network-based artificial intelligence framework.
Physics of Fluids, 31:0 125111, 2019.

[Ling et al., 2016]
J. Ling, A. Kurzawski, and J. Templeton.
Reynolds averaged turbulence modelling using deep neural networks
with embedded invariance.
Journal of Fluid Mechanics, 807:0 155–166, 2016.

[Lévy-Leblond, 1971]
J. Lévy-Leblond.
Galilei group and galilean invariance.
In Group theory and its applications, pages 221–299.
Elsevier, 1971.

[Maulik et al., 2019]
R. Maulik, O. San, A. Rasheed, and P. Vedula.
Subgrid modelling for two-dimensional turbulence using neural
Journal of Fluid Mechanics, 858:0 122–144, 2019.

[Kraichnan, 1967]
R. H. Kraichnan.
Inertial ranges in two-dimensional turbulence.
Phys. Fluids, 10:0 1417–1423, 1967.

[Kim and Lee, 2020]
J. Kim and C. Lee.
Prediction of turbulent heat transfer using convolutional neural
Journal of Fluid Mechanics, 882:0 A18, 2020.

[Kingma and Ba, 2014]
D. P. Kingma and J. Ba.
Adam: A method for stochastic optimization.
arXiv preprint arXiv:1412.6980, 2014.

[Hoang, 2020]
N-D. Hoang.
Image processing-based spall object detection using gabor filter,
texture analysis, and adaptive moment estimation (adam) optimized logistic
regression models.
Advances in Civil Engineering, 2020, 2020.

[Priyadarshini and Cotton, 2021]
I. Priyadarshini and C. Cotton.
A novel lstm–cnn–grid search-based deep neural network for
sentiment analysis.
The Journal of Supercomputing, 770 (12):0
13911–13932, 2021.

[Sun et al., 2021]
Y. Sun, S. Ding, Z. Zhang, and W. Jia.
An improved grid search algorithm to optimize svr for prediction.
Soft Computing, 250 (7):0 5633–5644, 2021.

[Yousif et al., 2022]
M. Yousif, L. Yu, and H. Lim.
Physics-guided deep learning for generating turbulent inflow
Journal of Fluid Mechanics, 936:0 A21, 2022.

[Shi et al., 2016]
W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop,
D. Rueckert, and Z. Wang.
Real-time single image and video super-resolution using an efficient
sub-pixel convolutional neural network.
In Proceedings of the IEEE conference on computer vision and
pattern recognition, pages 1874–1883, 2016.

[Talab et al., 2019]
M. A. Talab, S. Awang, and S. M. Najim.
Super-low resolution face recognition using integrated efficient
sub-pixel convolutional neural network (espcn) and convolutional neural
network (cnn).
In 2019 IEEE international conference on automatic control and
intelligent systems (I2CACIS), pages 331–335. IEEE, 2019.

[Huang et al., 2015]
Z. Huang, W. Xu, and K. Yu.
Bidirectional lstm-crf models for sequence tagging.
arXiv preprint arXiv:1508.01991, 2015.

[Sherstinsky, 2020]
A. Sherstinsky.
Fundamentals of recurrent neural network (rnn) and long short-term
memory (lstm) network.
Physica D: Nonlinear Phenomena, 404:0 132306, 2020.

[Kou and Zhang, 2021]
J. Kou and W. Zhang.
Data-driven modeling for unsteady aerodynamics and aeroelasticity.
Progress in Aerospace Sciences, 125:0 100725, 2021.

[Wang et al., 2022]
Z. Wang, K. Gong, W. Fan, C. Li, and W. Qian.
Prediction of swirling flow field in combustor based on deep
Acta Astronautica, 201:0 302–316, 2022a.

[Chowdhary et al., 2022]
K. Chowdhary, C. Hoang, K. Lee, J. Ray, V. G. Weirs, and B. Carnes.
Calibrating hypersonic turbulence flow models with the HIFiRE-1
experiment using data-driven machine-learned models.
Computer Methods in Applied Mechanics and Engineering,
401:0 115396, 2022.

[Bond and Daniel, 2008]
B. N. Bond and L. Daniel.
Guaranteed stable projection-based model reduction for indefinite and
unstable linear systems.
In 2008 IEEE/ACM International Conference on Computer-Aided
Design, pages 728–735. IEEE, 2008.

[Beli et al., 2018]
D. Beli, J-M. Mencik, P. B. Silva, and J. Arruda.
A projection-based model reduction strategy for the wave and
vibration analysis of rotating periodic structures.
Computational Mechanics, 620 (6):0 1511–1528,

[Siddiqui et al., 2022]
M. F. Siddiqui, T. De Troyer, J. Decuyper, P. Z. Csurcsia, J. Schoukens, and
M.C. Runacres.
A data-driven nonlinear state-space model of the unsteady lift force
on a pitching wing.
Journal of Fluids and Structures, 114:0 103706, 2022.

[Wang et al., 2022]
X. Wang, J. Kou, and W. Zhang.
Unsteady aerodynamic prediction for iced airfoil based on multi-task
Physics of Fluids, 34:0 087117, 2022b.

[Stevens and Colonius, 2020]
B. Stevens and T. Colonius.
Enhancement of shock-capturing methods via machine learning.
Theoretical and Computational Fluid Dynamics, 34:0
483–496, 2020.

[Pawar and Faroughi, 2022]
N. Pawar and S. A. Faroughi.
Complex fluids latent space exploration towards accelerated
predictive modeling.
Bulletin of the American Physical Society, 2022.

[Fernandes et al., 2022]
C. Fernandes, S. A. Faroughi, L. L. Ferrás, and A. M. Afonso.
Advanced polymer simulation and processing, 2022a.

[Faroughi et al., 2020]
S. A. Faroughi, C. Fernandes, J. M. Nóbrega, and G. H. McKinley.
A closure model for the drag coefficient of a sphere translating in a
viscoelastic fluid.
Journal of Non-Newtonian Fluid Mechanics, 277:0
104218, 2020.

[Fernandes et al., 2019]
C. Fernandes, S. A. Faroughi, O. S. Carneiro, J. Miguel Nóbrega, and G. H.
Fully-resolved simulations of particle-laden viscoelastic fluids
using an immersed boundary method.
Journal of Non-Newtonian Fluid Mechanics, 266:0
80–94, 2019.

[Lin et al., 2017]
W. Lin, Z. Wu, L. Lin, A. Wen, and J. Li.
An ensemble random forest algorithm for insurance big data analysis.
Ieee access, 5:0 16568–16575, 2017.

[Chen et al., 2015]
T. Chen, T. He, M. Benesty, V. Khotilovich, Y. Tang, H. Cho, K. Chen, et al.
Xgboost: extreme gradient boosting.
R package version 0.4-2, 10 (4):0 1–4, 2015.

[Lennon et al., 2022]
K. R. Lennon, G. H. McKinley, and J. W. Swan.
Scientific machine learning for modeling and simulating complex
arXiv preprint arXiv:2210.04431v1, 2022.

[Cai et al., 2022]
Z. Cai, J. Chen, and M. Liu.
Least-squares ReLU neural network (LSNN) method for scalar
nonlinear hyperbolic conservation law.
Applied Numerical Mathematics, 174:0 163–176,

[Haber et al., 2022]
G. E. Haber, J. Viquerat, A. Larcher, D. Ryckelynck, J. Alves, A. Patil, and
E. Hachem.
Deep learning model to assist multiphysics conjugate problems.
Physics of Fluids, 34:0 015131, 2022.

[Lara and Ferrer, 2022]
F. M. Lara and E. Ferrer.
Accelerating high order discontinuous Galerkin solvers using neural
networks: 1D Burgers’ equation.
Computers & Fluids, 235:0 105274, 2022.

[List et al., 2022]
B. List, L. Chen, and N. Thuerey.
Learned turbulence modelling with differentiable fluid solvers:
Physics-based loss functions and optimisation horizons.
Journal of Fluid Mechanics, 949:0 A25, 2022.

[Vollant et al., 2017]
A. Vollant, G. Balarac, and C. Corre.
Subgrid-scale scalar flux modelling based on optimal estimation
theory and machine-learning procedures.
Journal of Turbulence, 18:0 854–878, 2017.

[Beck et al., 2018]
A. D. Beck, D. G. Flad, and C. D. Munz.
Deep neural networks for data-driven turbulence models.
arXiv preprint arXiv:1806.04482, 2018.

[Sekar et al., 2019]
V. Sekar, Q. H. Jiang, C. Shu, and B. C. Khoo.
Fast flow field prediction over airfoils using deep learning
Physics of Fluids, 31:0 057103, 2019.

[Zhu et al., 2019]
L. Zhu, W. W. Zhang, J. Kou, and Y. Liu.
Machine learning methods for turbulence modeling in subsonic flows
around airfoils.
Physics of Fluids, 31:0 015105, 2019a.

[Tadesse et al., 2012]
Z. Tadesse, K. Patel, S. Chaudhary, and A. Nagpal.
Neural networks for prediction of deflection in composite bridges.
Journal of Constructional Steel Research, 680
(1):0 138–149, 2012.

[Güneyisi et al., 2014]
E. M. Güneyisi, M. D'Aniello, R. Landolfo, K. Mermerdaş, et al.
Prediction of the flexural overstrength factor for steel beams using
artificial neural network.
Steel and Composite Structures, 170 (3):0
215–236, 2014.

[Hung et al., 2019]
T. V. Hung, V. Q. Viet, and T. D. Van.
A deep learning-based procedure for estimation of ultimate load
carrying of steel trusses using advanced analysis.
Journal of Science and Technology in Civil Engineering
(STCE)-HUCE, 130 (3):0 113–123, 2019.

[Chen et al., 2019]
G. Chen, T. Li, Q. Chen, S. Ren, C. Wang, and S. Li.
Application of deep learning neural network to identify collision
load conditions based on permanent plastic deformation of shell structures.
Computational Mechanics, 640 (2):0 435–449,

[Hosseinpour et al., 2020]
M. Hosseinpour, Y. Sharifi, and H. Sharifi.
Neural network application for distortional buckling capacity
assessment of castellated steel beams.
In Structures, volume 27, pages 1174–1183. Elsevier, 2020.

[Trahair and Bradford, 2017]
N. S. Trahair and M. A. Bradford.
The behaviour and design of steel structures to AS 4100.
CRC Press, 2017.

[White et al., 2006]
D. W. White, A. E. Surovek, B. N. Alemdar, C-J. Chang, Y. D. Kim, and G. H.
Stability analysis and design of steel building frames using the 2005
aisc specification.
Steel Structures, 60 (2):0 71–91, 2006.

[Eur, 2005]
Design of steel structures part 1–1: General rules and rules for buildings.
European Committee for Standardization (ECS), Brussels
Belgium., 2005.

[White et al., 2019]
D. A. White, W. J. Arrighi, J. Kudo, and S. E. Watts.
Multiscale topology optimization using neural network surrogate
Computer Methods in Applied Mechanics and Engineering,
346:0 1118–1135, 2019.

[Zhang et al., 2019]
Y. Zhang, H. Li, M. Xiao, L. Gao, S. Chu, and J. Zhang.
Concurrent topology optimization for cellular structures with
nonuniform microstructures based on the kriging metamodel.
Structural and Multidisciplinary Optimization, 590
(4):0 1273–1299, 2019.

[Sigmund and Maute, 2013]
O. Sigmund and K. Maute.
Topology optimization approaches.
Structural and Multidisciplinary Optimization, 480
(6):0 1031–1055, 2013.

[Abueidda et al., 2020]
D. W. Abueidda, S. Koric, and N. A. Sobh.
Topology optimization of 2d structures with nonlinearities using deep
Computers & Structures, 237:0 106283, 2020.

[Yu et al., 2019]
Y. Yu, T. Hur, J. Jung, and I. G. Jang.
Deep learning for determining a near-optimal topological design
without any iteration.
Structural and Multidisciplinary Optimization, 590
(3):0 787–799, 2019.

[Banga et al., 2018]
S. Banga, H. Gehani, S. Bhilare, S. Patel, and L. Kara.
3d topology optimization using convolutional neural networks.
arXiv preprint arXiv:1808.07440, 2018.

[Li et al., 2019]
B. Li, C. Huang, X. Li, S. Zheng, and J. Hong.
Non-iterative structural topology optimization using deep learning.
Computer-Aided Design, 115:0 172–180, 2019.

[Takano and Alaghband, 2019]
N. Takano and G. Alaghband.
Srgan: Training dataset matters.
arXiv preprint arXiv:1903.09922, 2019.

[Nagano and Kikuta, 2018]
Y. Nagano and Y. Kikuta.
Srgan for super-resolving low-resolution food images.
In Proceedings of the Joint Workshop on Multimedia for Cooking
and Eating Activities and Multimedia Assisted Dietary Management, pages
33–37, 2018.

[Kumar et al., 2020]
S. Kumar, S. Tan, L. Zheng, and D. M. Kochmann.
Inverse-designed spinodoid metamaterials.
npj Computational Materials, 60 (1):0 1–10,

[Ni and Gao, 2021]
B. Ni and H. Gao.
A deep learning approach to the inverse problem of modulus
identification in elasticity.
MRS Bulletin, 460 (1):0 19–25, 2021.

[Messner, 2020]
M. C. Messner.
Convolutional neural network surrogate models for the mechanical
properties of periodic structures.
Journal of Mechanical Design, 1420 (2):0
024503, 2020.

[Tcherniak, 2002]
D. Tcherniak.
Topology optimization of resonating structures using simp method.
International Journal for Numerical Methods in Engineering,
540 (11):0 1605–1622, 2002.

[Lininger et al., 2021]
A. Lininger, M. Hinczewski, and G. Strangi.
General inverse design of thin-film metamaterials with convolutional
neural networks.
arXiv preprint arXiv:2104.01952, 2021.

[Löper et al., 2014]
P. Löper, M. Stuckelberger, B. Niesen, J. Werner, M. Filipič,
S. Moon, J-H. Yum, M. Topič, S. De Wolf, and C. Ballif.
Complex refractive index spectra of ch3nh3pbi3 perovskite thin films
determined by spectroscopic ellipsometry and spectrophotometry.
The journal of physical chemistry letters, 60
(1):0 66–71, 2014.

[Smith and Smith, 2020]
K. E. Smith and A O Smith.
Conditional gan for timeseries generation.
arXiv preprint arXiv:2006.16477, 2020.

[Balaji et al., 2019]
Y. Balaji, M. R. Min, B. Bai, R. Chellappa, and H. P. Graf.
Conditional gan with discriminative filter generation for
text-to-video synthesis.
In IJCAI, volume 1, page 2, 2019.

[Oberai et al., 2003]
A. A. Oberai, N. H. Gokhale, and G. R. Feijóo.
Solution of inverse problems in elasticity imaging using the adjoint
Inverse problems, 190 (2):0 297, 2003.

[Liang et al., 2018]
L. Liang, M. Liu, C. Martin, and W. Sun.
A deep learning approach to estimate stress distribution: a fast and
accurate surrogate of finite-element analysis.
Journal of The Royal Society Interface, 150
(138):0 20170844, 2018.

[Mozaffar et al., 2019]
M. Mozaffar, R. Bostanabad, W. Chen, K. Ehmann, J. Cao, and M. A. Bessa.
Deep learning predicts path-dependent plasticity.
PNAS, 116:0 26414–26420, 2019.

[Chatterjee, 2000]
A. Chatterjee.
An introduction to the proper orthogonal decomposition.
Current science, pages 808–817, 2000.

[Liang et al., 2002]
Y. C. Liang, H. P. Lee, S. P. Lim, W. Z. Lin, K. H. Lee, and C. Wu.
Proper orthogonal decomposition and its applications—part i:
Journal of Sound and vibration, 2520 (3):0
527–544, 2002.

[Long et al., 2021]
X. Y. Long, S. K. Zhao, C. Jiang, W. P. Li, and C. H. Liu.
Deep learning-based planar crack damage evaluation using
convolutional neural networks.
Engineering Fracture Mechanics, 246:0 107604, 2021.

[Zhu et al., 2021]
S. Zhu, M. Ohsaki, and X. Guo.
Prediction of non-linear buckling load of imperfect reticulated shell
using modified consistent imperfection and machine learning.
Engineering Structures, 226:0 111374, 2021.

[Miller and Ziemiański, 2020]
B. Miller and L. Ziemiański.
Optimization of dynamic behavior of thin-walled laminated cylindrical
shells by genetic algorithms and deep neural networks supported by modal
shape identification.
Advances in Engineering Software, 147:0 102830, 2020.

[Nie et al., 2020]
Z. Nie, H. Jiang, and L. B. Kara.
Stress field prediction in cantilevered structures using
convolutional neural networks.
Journal of Computing and Information Science in Engineering,
200 (1):0 011002, 2020.

[Pizarro and Massone, 2021]
P. N. Pizarro and L. M. Massone.
Structural design of reinforced concrete buildings based on deep
neural networks.
Engineering Structures, 241:0 112377, 2021.

[Pathirage et al., 2018]
C. S. Pathirage, J. Li, L. Li, H. Hao, W. Liu, and P. Ni.
Structural damage identification based on autoencoder neural networks
and deep learning.
Engineering structures, 172:0 13–28, 2018.

[Jiang and Zhang, 2020]
S. Jiang and J. Zhang.
Real-time crack assessment using deep neural networks with
wall-climbing unmanned aerial system.
Computer-Aided Civil and Infrastructure Engineering,
350 (6):0 549–564, 2020.

[Perez-Ramirez et al., 2019]
C. A. Perez-Ramirez, J. P. Amezquita-Sanchez, M. Valtierra-Rodriguez, H. Adeli,
A. Dominguez-Gonzalez, and R. J. Romero-Troncoso.
Recurrent neural network model with bayesian training and mutual
information for response prediction of large buildings.
Engineering Structures, 178:0 603–615, 2019.

[Truong et al., 2020]
T. T. Truong, D. Dinh-Cong, J. Lee, and T. Nguyen-Thoi.
An effective deep feedforward neural networks (dfnn) method for
damage identification of truss structures using noisy incomplete modal data.
Journal of Building Engineering, 30:0 101244, 2020.

[Raissi, 2018]
M. Raissi.
Deep hidden physics models: Deep learning of nonlinear partial
differential equations.
The Journal of Machine Learning Research, 190
(1):0 932–955, 2018.

[Biros et al., 2011]
G. Biros, O. Ghattas, M. Heinkenschloss, D. Keyes, B. Mallick, L. Tenorio,
B. van Bloemen Waanders, K. Willcox, Y. Marzouk, and L. Biegler.
Large-scale inverse problems and quantification of
John Wiley & Sons, 2011.

[Vogel, 2002]
C. R. Vogel.
Computational methods for inverse problems.
SIAM, 2002.

[Franssen et al., 2009]
H. J. Franssen, A. A. Hendricks, M. Riva, M. Bakr, N. Van der Wiel,
F. Stauffer, and A. Guadagnini.
A comparison of seven methods for the inverse modelling of
groundwater flow. application to the characterisation of well catchments.
Advances in Water Resources, 320 (6):0
851–872, 2009.

[Li et al., 2020]
Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and
A. Anandkumar.
Fourier neural operator for parametric partial differential
arXiv preprint arXiv:2010.08895, 2020a.

[Randjbaran et al., 2015]
E. Randjbaran, R. Zahari, R. Vaghei, and F. Karamizadeh.
A review paper on comparison of numerical techniques for finding
approximate solutions to boundary value problems on post-buckling in
functionally graded materials.
Trends Journal of Sciences Research, 20 (1):0
1–6, 2015.

[Triebel, 2015]
H. Triebel.
Hybrid function spaces, heat and Navier-Stokes equations.

[Durran, 2013]
D. R. Durran.
Numerical methods for wave equations in geophysical fluid
dynamics, volume 32.
Springer Science & Business Media, 2013.

[Prato, 2004]
G. D. Prato.
The stochastic burgers equation.
In Kolmogorov Equations for Stochastic PDEs, pages 131–153.
Springer, 2004.

[Medková, 2018]
D. Medková.
The laplace equation.
Boundary value problems on bounded and unbounded Lipschitz
domains. Springer, Cham, 2018.

[Genovese et al., 2006]
L. Genovese, T. Deutsch, A. Neelov, S. Goedecker, and G. Beylkin.
Efficient solution of poisson’s equation with free boundary
The Journal of chemical physics, 1250 (7):0
074105, 2006.

[Jagtap et al., 2020]
A. D. Jagtap, E. Kharazmi, and G. E. Karniadakis.
Conservative physics-informed neural networks on discrete domains for
conservation laws: Applications to forward and inverse problems.
Computer Methods in Applied Mechanics and Engineering,
365:0 113028, 2020.

[Jagtap and Karniadakis, 2021]
A. D. Jagtap and G. E. Karniadakis.
Extended physics-informed neural networks (xpinns): A generalized
space-time domain decomposition based deep learning framework for nonlinear
partial differential equations.
In AAAI Spring Symposium: MLPS, 2021.

[Mahmoudabadbozchelou and Jamali, 2021]
M. Mahmoudabadbozchelou and S. Jamali.
Rheology-informed neural networks (rhinns) for forward and inverse
metamodelling of complex fluids.
Scientific reports, 110 (1):0 1–13, 2021.

[Katsikis et al., 2022]
D. Katsikis, A. D. Muradova, and G. E. Stavroulakis.
A gentle introduction to physics informed neural networks, with
applications in static rod and beam problems.
J Adv App Comput Math, 9:0 103–128, 2022.

[Salvati et al., 2022]
E. Salvati, A. Tognan, L. Laurenti, M. Pelegatti, and F. De Bona.
A defect-based physics-informed machine learning framework for
fatigue finite life prediction in additive manufacturing.
Materials & Design, page 111089, 2022.

[Datta et al., 2022]
P. Datta, N. Pawar, and S. A. Faroughi.
A physics-informed neural network to model the flow of dry particles.
In Fall Meeting 2022. AGU, 2022.

[Nguyen et al., 2022]
L. Nguyen, M. Raissi, and P. Seshaiyer.
Modeling, analysis and physics informed neural network approaches for
studying the dynamics of covid-19 involving human-human and human-pathogen
Computational and Mathematical Biophysics, 100
(1):0 1–17, 2022.

[Shaier et al., 2022]
S. Shaier, M. Raissi, and P. Seshaiyer.
Data-driven approaches for predicting spread of infectious diseases
through dinns: Disease informed neural networks.
Letters in Biomathematics, 90 (1):0 71–105,

[Dissanayake and Phan-Thien, 1994]
M. Dissanayake and N. Phan-Thien.
Neural-network-based approximations for solving partial differential
communications in Numerical Methods in Engineering,
100 (3):0 195–201, 1994.

[Owhadi, 2015]
H. Owhadi.
Bayesian numerical homogenization.
Multiscale Modeling & Simulation, 130 (3):0
812–828, 2015.

[Raissi et al., 2018]
M. Raissi, A. Yazdani, and G. E. Karniadakis.
Hidden fluid mechanics: A navier-stokes informed deep learning
framework for assimilating flow visualization data.
arXiv preprint arXiv:1808.04327, 2018.

[Iserles, 2009]
A. Iserles.
A first course in the numerical analysis of differential
Number 44. Cambridge university press, 2009.

[Nabian et al., 2021]
M. A. Nabian, R. J. Gladstone, and H. Meidani.
Efficient training of physics-informed neural networks via importance
Computer-Aided Civil and Infrastructure Engineering,
360 (8):0 962–977, 2021.

[Su and Gardner, 1969]
C. H. Su and C. S. Gardner.
Korteweg-de vries equation and generalizations. iii. derivation of
the korteweg-de vries equation and burgers equation.
Journal of Mathematical Physics, 100 (3):0
536–539, 1969.

[Constantin and Foias, 2020]
P. Constantin and C. Foias.
Navier-stokes equations.
University of Chicago Press, 2020.

[Stiasny et al., 2021]
J. Stiasny, G. S. Misyris, and S. Chatzivasileiadis.
Physics-informed neural networks for non-linear system identification
for power system dynamics.
In 2021 IEEE Madrid PowerTech, pages 1–6. IEEE, 2021.

[Mao et al., 2020]
Z. Mao, A. D. Jagtap, and G.E. Karniadakis.
Physics-informed neural networks for high-speed flows.
Computer Methods in Applied Mechanics and Engineering,
360:0 112789, 2020.

[Jagtap et al., 2022]
A. D. Jagtap, Z. Mao, N. Adams, and G. E. Karniadakis.
Physics-informed neural networks for inverse problems in supersonic
arXiv preprint arXiv:2202.11821, 2022a.

[Zhang et al., 2020]
T. Zhang, B. Dey, P. Kakkar, A Dasgupta, and A. Chakraborty.
Frequency-compensated pinns for fluid-dynamic design problems.
arXiv preprint arXiv:2011.01456, 2020b.

[Tancik et al., 2020]
M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan,
U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng.
Fourier features let networks learn high frequency functions in low
dimensional domains.
Advances in Neural Information Processing Systems,
33:0 7537–7547, 2020.

[Cheng and Zhang, 2021]
C. Cheng and G-T. Zhang.
Deep learning method based on physics informed neural network with
resnet block for solving fluid flow problems.
Water, 130 (4):0 423, 2021.

[Lou et al., 2021]
Q. Lou, X. Meng, and G. E. Karniadakis.
Physics-informed neural networks for solving forward and inverse flow
problems via the boltzmann-bgk formulation.
Journal of Computational Physics, 447:0 110676, 2021.

[Wessels et al., 2020]
H. Wessels, C. Weißenfels, and P. Wriggers.
The neural particle method–an updated lagrangian physics informed
neural network for computational fluid dynamics.
Computer Methods in Applied Mechanics and Engineering,
368:0 113127, 2020.

[Mahmoudabadbozchelou et al., 2022]
M. Mahmoudabadbozchelou, G. E. Karniadakis, and S. Jamali.
nn-pinns: Non-newtonian physics-informed neural networks for complex
fluid modeling.
Soft Matter, 180 (1):0 172–185, 2022.

[Haghighat et al., 2021]
E. Haghighat, D. Amini, and R. Juanes.
Physics-informed neural network simulation of multiphase
poroelasticity using stress-split sequential training.
arXiv preprint arXiv:2110.03049, 2021b.

[Almajid and Abu-Al-Saud, 2022]
M. M. Almajid and M. O. Abu-Al-Saud.
Prediction of porous media fluid flow using physics informed neural
Journal of Petroleum Science and Engineering, 208:0
109205, 2022.

[Depina et al., 2022]
I. Depina, S. Jain, V. S. Mar, and H. Gotovac.
Application of physics-informed neural networks to inverse problems
in unsaturated groundwater flow.
Georisk: Assessment and Management of Risk for Engineered
Systems and Geohazards, 160 (1):0 21–36, 2022.

[Tartakovsky et al., 2018]
A. M. Tartakovsky, C. O. Marrero, P. Perdikaris, G. D. Tartakovsky, and
D. Barajas-S.
Learning parameters and constitutive relationships with physics
informed deep neural networks.
arXiv preprint arXiv:1808.03398, 2018.

[Thakur et al., 2022]
S. Thakur, M. Raissi, and A. M. Ardekani.
Viscoelasticnet: A physics informed neural network framework for
stress discovery and model selection.
arXiv preprint arXiv:2209.06972, 2022.

[Fernandes et al., 2022]
C. Fernandes, S. A. Faroughi, R. Ribeiro, A. Isabel, and G. H. McKinley.
Finite volume simulations of particle-laden viscoelastic fluid flows:
Application to hydraulic fracture processes.
Engineering with Computers, pages 1–27, 2022b.

[Chiu et al., 2022]
P. Chiu, J. C. Wong, C. Ooi, M. H. Dao, and Y. Ong.
Can-pinn: A fast physics-informed neural network based on
coupled-automatic–numerical differentiation method.
Computer Methods in Applied Mechanics and Engineering,
395:0 114909, 2022.

[Van Der Hoeven, 2004]
J. Van Der Hoeven.
The truncated fourier transform and applications.
In Proceedings of the 2004 international symposium on Symbolic
and algebraic computation, pages 290–296, 2004.

[Raynaud et al., 2022]
G. Raynaud, S. Houde, and F. P. Gosselin.
Modalpinn: an extension of physics-informed neural networks with
enforced truncated fourier decomposition for periodic flow reconstruction
using a limited number of imperfect sensors.
Journal of Computational Physics, page 111271, 2022.

[Oldenburg et al., 2022]
J. Oldenburg, F. Borowski, A. Öner, K. Schmitz, and M. Stiehm.
Geometry aware physics informed neural network surrogate for solving
navier-stokes equation (gapinn).

[Wandel et al., 2022]
N. Wandel, M. Weinmann, M. Neidlin, and R. Klein.
Spline-pinn: Approaching pdes without data using fast,
physics-informed hermite-spline cnns.
In Proceedings of the AAAI Conference on Artificial
Intelligence, volume 36, pages 8529–8538, 2022.

[Jin et al., 2021]
X. Jin, S. Cai, H. Li, and G. E. Karniadakis.
Nsfnets (navier-stokes flow nets): Physics-informed neural networks
for the incompressible navier-stokes equations.
Journal of Computational Physics, 426:0 109951, 2021.

[Cheng et al., 2021]
C. Cheng, P. Xu, Y. Li, and G. Zhang.
Deep learning based on pinn for solving 2 d0f vortex induced
vibration of cylinder with high reynolds number.
arXiv preprint arXiv:2106.01545, 2021.

[Eivazi et al., 2022]
H. Eivazi, M. Tahani, P. Schlatter, and R. Vinuesa.
Physics-informed neural networks for solving reynolds-averaged
navier–stokes equations.
Physics of Fluids, 340 (7):0 075117, 2022.

[Wang et al., 2017]
J. Wang, J. Wu, and H. Xiao.
Physics-informed machine learning approach for reconstructing
reynolds stress modeling discrepancies based on dns data.
Physical Review Fluids, 20 (3):0 034603, 2017.

[Zhang et al., 2022]
X. Zhang, Y. Zhu, J. Wang, L. Ju, Y. Qian, M. Ye, and J. Yang.
Gw-pinn: A deep learning algorithm for solving groundwater flow
Advances in Water Resources, page 104243, 2022a.

[Aliakbari et al., 2022]
M. Aliakbari, M. Mahmoudi, P. Vadasz, and A. Arzani.
Predicting high-fidelity multiphysics data from low-fidelity fluid
flow and transport solvers using physics-informed neural networks.
International Journal of Heat and Fluid Flow, 96:0
109002, 2022.

[Tartakovsky et al., 2020]
A. M. Tartakovsky, C. O. Marrero, P. Perdikaris, G. D. Tartakovsky, and
D. Barajas-Solano.
Physics-informed deep neural networks for learning parameters and
constitutive relationships in subsurface flow problems.
Water Resources Research, 560 (5):0
e2019WR026731, 2020.

[Kashefi and Mukerji, 2022]
Ali Kashefi and Tapan Mukerji.
Prediction of fluid flow in porous media by sparse observations and
physics-informed pointnet.
arXiv preprint arXiv:2208.13434, 2022a.

[Kissas et al., 2020]
G. Kissas, Y. Yang, E. Hwuang, W. R. Witschey, J. A. Detre, and P. Perdikaris.
Machine learning in cardiovascular flows modeling: Predicting
arterial blood pressure from non-invasive 4d flow mri data using
physics-informed neural networks.
Computer Methods in Applied Mechanics and Engineering,
358:0 112623, 2020.

[Arzani et al., 2021]
A. Arzani, J-X. Wang, and R. M. D'Souza.
Uncovering near-wall blood flow from sparse data with
physics-informed neural networks.
Physics of Fluids, 330 (7):0 071905, 2021.

[Jagtap et al., 2022]
A. D. Jagtap, D. Mitsotakis, and G. E. Karniadakis.
Deep learning of inverse water waves problems using multi-fidelity
data: Application to Serre–Green–Naghdi equations.
Ocean Engineering, 248:0 110775, 2022b.

[Kashefi and Mukerji, 2022]
Ali Kashefi and Tapan Mukerji.
Physics-informed pointnet: A deep learning solver for steady-state
incompressible flows and thermal fields on multiple sets of irregular
Journal of Computational Physics, 468:0 111510,
ISSN 0021-9991.

[Haghighat et al., 2020]
E. Haghighat, M. Raissi, A. Moure, H. Gomez, and R. Juanes.
A deep learning framework for solution and discovery in solid
mechanics: linear elasticity.
arXiv preprint arXiv:2003.02751, 2020.

[Shukla et al., 2021]
K. Shukla, A. D. Jagtap, J. L. Blackshire, D. Sparkman, and G. E. Karniadakis.
A physics-informed neural network for quantifying the microstructural
properties of polycrystalline nickel using ultrasound data: A promising
approach for solving inverse problems.
IEEE Signal Processing Magazine, 390 (1):0
68–77, 2021.

[Henkes et al., 2022]
A. Henkes, H. Wessels, and R. Mahnken.
Physics informed neural networks for continuum micromechanics.
Computer Methods in Applied Mechanics and Engineering,
393:0 114790, 2022.

[Zhang and Gu, 2021]
Z. Zhang and G. X. Gu.
Physics-informed deep learning for digital materials.
Theoretical and Applied Mechanics Letters, 110
(1):0 100220, 2021.

[Rao et al., 2020]
C. Rao, H. Sun, and Y. Liu.
Physics informed deep learning for computational elastodynamics
without labeled data.
arXiv preprint arXiv:2006.08472, 2020b.

[Fang and Zhan, 2019]
Z. Fang and J. Zhan.
Deep physical informed neural networks for metamaterial design.
IEEE Access, 8:0 24506–24513, 2019.

[Lax and Nelson, 1976]
M. Lax and D. F. Nelson.
Maxwell equations in material form.
Physical Review B, 130 (4):0 1777, 1976.

[Zhang et al., 2020]
E. Zhang, M. Yin, and G. E. Karniadakis.
Physics-informed neural networks for nonhomogeneous material
identification in elasticity imaging.
arXiv preprint arXiv:2009.04525, 2020c.

[Abueidda et al., 2022]
D. W. Abueidda, S. Koric, E. Guleryuz, and N. A. Sobh.
Enhanced physics-informed neural networks for hyperelasticity.
arXiv preprint arXiv:2205.14148, 2022a.

[Abueidda et al., 2022]
D. W. Abueidda, S. Koric, R. A. Al-Rub, C. M. Parrott, K. A. James, and N. A.
A deep learning energy method for hyperelasticity and
European Journal of Mechanics-A/Solids, 95:0 104639,

[Yuan et al., 2022]
L. Yuan, Y. Ni, X. Deng, and S. Hao.
A-pinn: Auxiliary physics informed neural networks for forward and
inverse problems of nonlinear integro-differential equations.
Journal of Computational Physics, 462:0 111260, 2022.

[Lu et al., 2021]
L. Lu, X. Meng, Z. Mao, and G. E. Karniadakis.
Deepxde: A deep learning library for solving differential equations.
SIAM Review, 630 (1):0 208–228,

[Arora, 2022]
R. Arora.
Physrnet: Physics informed super-resolution network for application
in computational solid mechanics.
arXiv preprint arXiv:2206.15457, 2022.

[Madenci et al., 2019]
E. Madenci, A. Barut, and M. Dorduncu.
Peridynamic differential operator for numerical analysis,
volume 10.
Springer, 2019.

[Haghighat et al., 2021]
E. Haghighat, A. C. Bekar, E. Madenci, and R. Juanes.
A nonlocal physics-informed deep learning framework using the
peridynamic differential operator.
Computer Methods in Applied Mechanics and Engineering,
385:0 114012, 2021c.

[Huang et al., 2006]
G. Huang, Q. Zhu, and C. Siew.
Extreme learning machine: theory and applications.
Neurocomputing, 700 (1-3):0 489–501, 2006.

[Yan et al., 2022]
C. A. Yan, R. Vescovini, and L. Dozio.
A framework based on physics-informed neural networks and extreme
learning for the analysis of composite structures.
Computers & Structures, 265:0 106761, 2022.

[Rezaei et al., 2022]
S. Rezaei, A. Harandi, A. Moeineddin, B. Xu, and S. Reese.
A mixed formulation for physics-informed neural networks as a
potential solver for engineering problems in heterogeneous domains:
comparison with finite element method.
arXiv preprint arXiv:2206.13103, 2022.

[Mallampati and Almekkawy, 2021]
A. Mallampati and M. Almekkawy.
Measuring tissue elastic properties using physics based neural
In 2021 IEEE UFFC Latin America Ultrasonics Symposium (LAUS),
pages 1–4. IEEE, 2021.

[Li et al., 2021]
W. Li, M. Z. Bazant, and J. Zhu.
A physics-guided neural network framework for elastic plates:
Comparison of governing equations-based and energy-based approaches.
Computer Methods in Applied Mechanics and Engineering,
383:0 113933, 2021c.

[Vahab et al., 2021]
M. Vahab, E. Haghighat, M. Khaleghi, and N. Khalili.
A physics informed neural network approach to solution and
identification of biharmonic equations of elasticity.
arXiv preprint arXiv:2108.07243, 2021.

[Raj et al., 2021]
M. Raj, P. Kumbhar, and R. K. Annabattula.
Physics-informed neural networks for solving thermo-mechanics
problems of functionally graded material.
arXiv preprint arXiv:2111.10751, 2021.

[Zhang et al., 2022]
E. Zhang, M. Dao, G. E. Karniadakis, and S. Suresh.
Analyses of internal structures and defects in materials using
physics-informed neural networks.
Science advances, 80 (7):0 eabk0644,

[Bastek and Kochmann, 2022]
J. Bastek and D. M. Kochmann.
Physics-informed neural networks for shell structures.
arXiv preprint arXiv:2207.14291, 2022.

[Zhang et al., 2021]
X. Zhang, J. Gong, and F Xuan.
A physics-informed neural network for creep-fatigue life prediction
of components at elevated temperatures.
Engineering Fracture Mechanics, 258:0 108130,

[Haghighat et al., 2021]
E. Haghighat, A. C. Bekar, E. Madenci, and R. Juanes.
Deep learning for solution and inversion of structural mechanics and
arXiv preprint arXiv:2105.09477, 2021d.

[Zheng et al., 2022]
B. Zheng, T. Li, H. Qi, L. Gao, X. Liu, and L. Yuan.
Physics-informed machine learning model for computational fracture of
quasi-brittle materials without labelled data.
International Journal of Mechanical Sciences, 223:0
107282, 2022.

[Arora et al., 2022]
R. Arora, P. Kakkar, B. Dey, and A. Chakraborty.
Physics-informed neural networks for modeling rate-and
temperature-dependent plasticity.
arXiv preprint arXiv:2201.08363, 2022.

[Bai et al., 2022]
J. Bai, H. Jeong, C. P. Batuwatta, S. Xiao, Q. Wang, C. M. Rathnayaka,
L. Alzubaidi, G. Liu, and Y. Gu.
An introduction to programming physics-informed neural network-based
computational solid mechanics.
arXiv preprint arXiv:2210.09060, 2022.

[Dwivedi and Srinivasan, 2020]
V. Dwivedi and B. Srinivasan.
Solution of biharmonic equation in complicated geometries with
physics informed extreme learning machine.
Journal of Computing and Information Science in Engineering,
200 (6), 2020.

[Dwivedi et al., 2021]
V. Dwivedi, N. Parashar, and B. Srinivasan.
Distributed learning machines for solving forward and inverse
problems in partial differential equations.
Neurocomputing, 420:0 299–316, 2021.

[Shin et al., 2020]
Yeonjong Shin, Jerome Darbon, and George Em Karniadakis.
On the convergence of physics informed neural networks for linear
second-order elliptic and parabolic type pdes.
arXiv preprint arXiv:2004.01806, 2020.

[Fuks and Tchelepi, 2020]
O. Fuks and H. A. Tchelepi.
Limitations of physics informed machine learning for nonlinear
two-phase transport in porous media.
Journal of Machine Learning for Modeling and Computing,
10 (1), 2020.

[Long et al., 2018]
Y. Long, X. She, and S. Mukhopadhyay.
Hybridnet: integrating model-based and data-driven learning to
predict evolution of dynamical systems.
In Conference on Robot Learning, pages 551–560. PMLR, 2018.

[Zhu et al., 2019]
Y. Zhu, N. Zabaras, P. Koutsourelakis, and P. Perdikaris.
Physics-constrained deep learning for high-dimensional surrogate
modeling and uncertainty quantification without labeled data.
Journal of Computational Physics, 394:0 56–81,

[Geneva and Zabaras, 2020]
N. Geneva and N. Zabaras.
Modeling the dynamics of pde systems with physics-constrained deep
auto-regressive networks.
Journal of Computational Physics, 403:0 109056, 2020.

[Wang et al., 2020]
R. Wang, K. Kashinath, M. Mustafa, A. Albert, , and R. Yu.
Towards physics-informed deep learning for turbulent flow prediction.
In Proceedings of the 26th ACM SIGKDD International Conference
on Knowledge Discovery & Data Mining, pages 1457–1466, 2020.

[Ranade et al., 2021]
R. Ranade, C. Hill, and J. Pathak.
Discretizationnet: A machine-learning based solver for navier–stokes
equations using finite volume discretization.
Computer Methods in Applied Mechanics and Engineering,
378:0 113722, 2021.

[Gao et al., 2021]
H. Gao, L. Sun, and J. Wang.
Phygeonet: Physics-informed geometry-adaptive convolutional neural
networks for solving parameterized steady-state pdes on irregular domain.
Journal of Computational Physics, 428:0 110079, 2021.

[273]
Chengping Rao, Pu Ren, Yang Liu, and Hao Sun.
Discovering nonlinear pdes from scarce data with physics-encoded
arXiv preprint arXiv:2201.12354, b.

[Wang, 1994]
J. Wang.
A deterministic annealing neural network for convex programming.
Neural networks, 70 (4):0 629–641, 1994.

[Rangarajan et al., 1996]
Anand Rangarajan, Steven Gold, and Eric Mjolsness.
A novel optimizing network architecture with applications.
Neural Computation, 80 (5):0 1041–1060, 1996.

[Cranmer et al., 2020]
M. Cranmer, S. Greydanus, S. Hoyer, P. Battaglia, D. Spergel, and S. Ho.
Lagrangian neural networks.
arXiv preprint arXiv:2003.04630, 2020.

[Allen-Blanchette et al., 2020]
C. Allen-Blanchette, S. Veer, A. Majumdar, and N. E. Leonard.
Lagnetvip: A lagrangian neural network for video prediction.
arXiv preprint arXiv:2010.12932, 2020.

[Chen et al., 2019]
Z. Chen, J. Zhang, M. Arjovsky, and L. Bottou.
Symplectic recurrent neural networks.
arXiv preprint arXiv:1909.13334, 2019c.

[DiPietro et al., 2020]
D. DiPietro, S. Xiong, and B. Zhu.
Sparse symplectically integrated neural networks.
Advances in Neural Information Processing Systems,
33:0 6074–6085, 2020.

[Trask et al., 2022]
N. Trask, A. Huang, and X. Hu.
Enforcing exact physics in scientific machine learning: a data-driven
exterior calculus on graphs.
Journal of Computational Physics, 456:0 110969, 2022.

[Lin et al., 2013]
M. Lin, Q. Chen, and S. Yan.
Network in network.
arXiv preprint arXiv:1312.4400, 2013.

[Shi et al., 2015]
X. Shi, Z. Chen, H. Wang, D. Yeung, W. Wong, and W. Woo.
Convolutional lstm network: A machine learning approach for
precipitation nowcasting.
Advances in neural information processing systems, 28, 2015.

[He et al., 2016]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.
Deep residual learning for image recognition.
In Proceedings of the IEEE conference on computer vision and
pattern recognition, pages 770–778, 2016.

[Ren et al., 2022]
P. Ren, C. Rao, Y. Liu, J. Wang, and Hao Sun.
Phycrnet: Physics-informed convolutional-recurrent network for
solving spatiotemporal pdes.
Computer Methods in Applied Mechanics and Engineering,
389:0 114399, 2022.

[Baydin et al., 2018]
A. G. Baydin, B. A Pearlmutter, A. A. Radul, and J. M. Siskind.
Automatic differentiation in machine learning: a survey.
Journal of Marchine Learning Research, 18:0 1–43,

[Rackauckas et al., 2019]
C. Rackauckas, M. Innes, Y. Ma, J. Bettencourt, L. White, and V. Dixit.
Diffeqflux. jl-a julia library for neural differential equations.
arXiv preprint arXiv:1902.02376, 2019.

[Pontryagin, 1987]
L. S. Pontryagin.
Mathematical theory of optimal processes.
CRC press, 1987.

[Ma et al., 2021]
Y. Ma, V. Dixit, M. J. Innes, X. Guo, and C. Rackauckas.
A comparison of automatic differentiation and continuous sensitivity
analysis for derivatives of differential equation solutions.
In 2021 IEEE High Performance Extreme Computing Conference
(HPEC), pages 1–9. IEEE, 2021.

[Poli et al., 2020]
M. Poli, S. Massaroli, A. Yamashita, H. Asama, and J. Park.
Torchdyn: A neural differential equations library.
arXiv preprint arXiv:2009.09346, 2020.

[Lai et al., 2021]
Z. Lai, C. Mylonas, S. Nagarajaiah, and E. Chatzi.
Structural identification with physics-informed neural ordinary
differential equations.
Journal of Sound and Vibration, 508:0 116196, 2021.

[Roehrl et al., 2020]
M. A. Roehrl, T. A. Runkler, V. Brandtstetter, M. Tokic, and S. Obermayer.
Modeling system dynamics with physics-informed neural networks based
on lagrangian mechanics.
IFAC-PapersOnLine, 530 (2):0 9195–9200, 2020.

[Dulny et al., 2021]
A. Dulny, A. Hotho, and A. Krause.
Neuralpde: Modelling dynamical systems from data.
arXiv preprint arXiv:2111.07671, 2021.

[K. et al., 2016]
He K., Zhang X., Ren S., and Sun Jian.
Identity mappings in deep residual networks.
In European conference on computer vision, pages 630–645.
Springer, 2016.

[Goswami et al., 2020]
S. Goswami, C. Anitescu, S. Chakraborty, and T. Rabczuk.
Transfer learning enhanced physics informed neural network for
phase-field modeling of fracture.
Theoretical and Applied Fracture Mechanics, 106:0
102447, 2020.

[Bhattacharya et al., 2020]
K. Bhattacharya, B. Hosseini, N.B. Kovachk, and A.M. Stuart.
Model reduction and neural networks for parametric pdes.
arXiv preprint arXiv:2005.03180, 2020.

[Li et al., 2020]
Z. Li, N. Kovachki, K. Azizzadenesheli, Burigede Liu, K. Bhattacharya,
A. Stuart, and A. Anandkumar.
Neural operator: Graph kernel network for partial differential
arXiv preprint arXiv:2003.03485, 2020b.

[Migus et al., 2022]
Leon Migus, Yuan Yin, Jocelyn Ahmed Mazari, and Patrick Gallinari.
Multi-scale physical representations for approximating pde solutions
with graph neural operators.
arXiv preprint arXiv:2206.14687, 2022.

[Chen and Chen, 1995]
T. Chen and H. Chen.
Universal approximation to nonlinear operators by neural networks
with arbitrary activation functions and its application to dynamical systems.
IEEE Transactions on Neural Networks, 60 (4):0
911–917, 1995.

[Lin et al., 2021]
C. Lin, Z. Li, L. Lu, S. Cai, M. Maxey, and G. E. Karniadakis.
Operator learning for predicting multiscale bubble growth dynamics.
The Journal of Chemical Physics, 1540 (10):0
104118, 2021.

[Oommen et al., 2022]
V. Oommen, K. Shukla, S. Goswami, R. Dingreville, and G. E. Karniadakis.
Learning two-phase microstructure evolution using neural operators
and autoencoder architectures.
arXiv preprint arXiv:2204.07230, 2022.

[Wang et al., 2021]
S. Wang, H. Wang, and P. Perdikaris.
Learning the solution operator of parametric partial differential
equations with physics-informed deeponets.
Science advances, 70 (40):0 eabi8605, 2021.

[Goswami et al., 2022]
S. Goswami, M. Yin, Y. Yu, and G. E. Karniadakis.
A physics-informed variational deeponet for predicting crack path in
quasi-brittle materials.
Computer Methods in Applied Mechanics and Engineering,
391:0 114587, 2022b.

[DeVore, 2017]
R.A. DeVore.
The theoretical foundation of reduced basis methods.
Model reduction and approximation: theory and algorithms,
15:0 137, 2017.

[Zhu and Zabaras, 2018]
Y. Zhu and N. Zabaras.
Bayesian deep convolutional encoder–decoder networks for surrogate
modeling and uncertainty quantification.
Journal of Computational Physics, 366:0 415–447,

[Grady et al., 2022]
T. J. Grady, R. Khan, M. Louboutin, Z. Yin, P. A. Witte, R. Chandra, R. J.
Hewett, and F. J. Herrmann.
Towards large-scale learned solvers for parametric pdes with
model-parallel fourier neural operators.
arXiv preprint arXiv:2204.01205, 2022.

[Bui et al., 2018]
M. Bui, C. S. Adjiman, A. Bardow, E. J. Anthony, A. Boston, S. Brown, P. S.
Fennell, S. Fuss, A. Galindo, L. A. Hackett, and others.
Carbon capture and storage (ccs): the way forward.
Energy & Environmental Science, 110 (5):0
1062–1176, 2018.

[Wen et al., 2022]
G. Wen, Z. Li, K. Azizzadenesheli, A. Anandkumar, and S.M. Benson.
U-fno—an enhanced fourier neural operator-based deep-learning model
for multiphase flow.
Advances in Water Resources, 163:0 104180, 2022.

[You et al., 2022]
H. You, Q. Zhang, C.J. Ross, C-H. Lee, and Y. Yu.
Learning deep implicit fourier neural operators (ifnos) with
applications to heterogeneous material modeling.
arXiv preprint arXiv:2203.08205, 2022a.

[Kovachki et al., 2021]
N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, and
A. Anandkumar.
Neural operator: Learning maps between function spaces.
arXiv preprint arXiv:2108.08481, 2021.

[Lu et al., 2022]
L. Lu, X. Meng, S. Cai, Z. Mao, S. Goswami, Z. Zhang, and G. E. Karniadakis.
A comprehensive and fair comparison of two neural operators (with
practical extensions) based on fair data.
Computer Methods in Applied Mechanics and Engineering,
393:0 114778, 2022.

[You et al., 2022]
H. You, Y. Yu, M. D'Elia, T. Gao, and S. Silling.
Nonlocal kernel network (nkn): a stable and resolution-independent
deep neural network.
arXiv preprint arXiv:2201.02217, 2022b.

[Molina et al., 2020]
A. M. Molina, I. F. Avelino, E. F. Morales, and L. E. Sucar.
Causal based q-learning.
Research in Computing Science, 149:0 95–104, 2020.