# NITO: Neural Implicit Fields for Resolution-free Topology Optimization

Amin Heyrani Nobari  Giorgio Giannone  Lyle Regenwetter  Faez Ahmed

###### Abstract

Topology optimization is a critical task in engineering design, where the goal
is to optimally distribute material in a given space for maximum performance.
We introduce Neural Implicit Topology Optimization (NITO), a novel approach to
accelerate topology optimization problems using deep learning. NITO stands out
as one of the first frameworks to offer a resolution-free and domain-agnostic
solution in deep learning-based topology optimization. NITO synthesizes
structures with up to seven times better structural efficiency compared to
SOTA diffusion models and does so in a tenth of the time. In the NITO
framework, we introduce a novel method, the Boundary Point Order-Invariant MLP
(BPOM), to represent boundary conditions in a sparse and domain-agnostic
manner, moving away from expensive simulation-based approaches. Crucially,
NITO circumvents the domain and resolution limitations that restrict
Convolutional Neural Network (CNN) models to a structured domain of fixed size
– limitations that hinder the widespread adoption of CNNs in engineering
applications. This generalizability allows a single NITO model to train and
generate solutions in countless domains, eliminating the need for numerous
domain-specific CNNs and their extensive datasets. Despite its
generalizability, NITO outperforms SOTA models even in specialized tasks, is
an order of magnitude smaller, and is practically trainable at high
resolutions that would be restrictive for CNNs. This combination of
versatility, efficiency, and performance underlines NITO’s potential to
transform the landscape of engineering design optimization problems through
implicit fields.

Machine Learning, ICML

##  1 Introduction

Figure 1: NITO framework leveraging BPOM. Left: ground truth obtained using a
SIMP optimizer. Second From Left: the raw output of neural fields using BPOM.
Right Two: the NITO framework output leveraging a few steps (5 and 10 steps)
of optimization. We see that NITO is amenable to deep optimization and can
generate high-quality, constraint-satisfying, and high-performance topologies
with fast inference. See Appendix C for more visualizations.

Recent years have witnessed remarkable advancements in vision and language
domains, marked by a series of influential studies (Child, 2020; Brock et al.,
2018; Ho et al., 2020; Rombach et al., 2021; Devlin et al., 2018; Brown et
al., 2020; Raffel et al., 2020). These developments have fundamentally
transformed our approach to processing multimodal and unstructured data. This
transformation has been pivotal in fostering innovative generative
methodologies for such data (Ramesh et al., 2022; Nichol et al., 2021;
Blattmann et al., 2022; Ramesh et al., 2021). With the advent of these deep
generative models (DGM) and their success in language and vision, significant
research has been developed specifically in the application of these deep
generative models in scientific and engineering problems (Regenwetter et al.,
2022; Song et al., 2023; Hu et al., 2024; Wu et al., 2021).

Engineering design seeks to create designs that satisfy a set of constraints
while maximizing functional performance. It is therefore easy to see how this
kind of problem lends itself to optimization and is effectively addressed by
optimization algorithms (Boyd & Vandenberghe, 2004), making them cornerstones
in computational design. A classic objective for structural topology
optimization (TO) is to distribute a limited amount of material in a physical
domain such that the resulting structure is as stiff as possible. This
objective is called compliance minimization, but other objectives such as heat
transfer are often considered. A popular TO method is the Solid Isotropic
Material with Penalization method (Bendsøe & Kikuchi, 1988; Rozvany et al.,
1992) (SIMP). While methods such as SIMP are powerful engineering tools, they
struggle with computational cost when solving large-scale problems due to
their iterative nature and expensive per-step computations (Sigmund & Maute,
2013).

In recent years, many works have investigated the prospect of tackling
topology optimization problems using machine learning (Regenwetter et al.,
2022). Most of these approaches are built around DGMs trained on optimized
topologies and conditioned on a set of loads and boundary conditions. (Nie et
al., 2021b; Mazé & Ahmed, 2023; Giannone et al., 2023; Giannone & Ahmed, 2023;
Hu et al., 2024). The main justification for these approaches is significant
time savings compared to TO solvers like SIMP and the ability to generate
multiple near-optimal solutions. Indeed, any DGM able to quickly generate
near-optimal solutions delivers immense value in the form of time and cost
savings (Woldseth et al., 2022b).

However, compared to classic iterative optimizers like SIMP, pure data-driven
approaches typically do not generate optimal topologies, despite being faster.
This is primarily attributable to the fact that _data-driven models typically
focus on density estimation and remain agnostic to the physics of a given
problem_. To address this, researchers have proposed to incorporate physics
into these models, such as through guidance in diffusion models (Mazé & Ahmed,
2023) or aligning the trajectory of diffusion models with the optimizer
trajectory (Giannone et al., 2023). Despite these efforts, the aforementioned
approaches suffer from some important limitations.

When applying TO most problems are defined in uniquely shaped domains and
require different levels of detail (i.e., resolution) depending on the
application and the underlying problem. Most existing deep learning methods
discretize the physical domain into pixels or voxels to leverage convolutions
(CNNs) as basic building block (Mazé & Ahmed, 2023; Giannone & Ahmed, 2023;
Giannone et al., 2023; Nie et al., 2021b; Hu et al., 2024). This limits their
generation capabilities to a specific resolution and shape (such as a square
domain) – a new resolution or aspect ratio necessitates an entirely new
dataset and model. Moreover, these DGMs typically rely on physical fields (or
approximations) as a representation of the boundary condition of the
underlying problems (Mazé & Ahmed, 2023; Giannone & Ahmed, 2023; Giannone et
al., 2023; Nie et al., 2021b; Chen et al., 2023; Hu et al., 2024). These
physical fields are represented as images to be handled by CNNs. Not only is
this approach computationally expensive, particularly for high dimensionality,
but it further exacerbates the lack of generalizability across problems and
domains. This lack of generalizability is a common criticism directed at such
methods by computational structure engineering (Woldseth et al., 2022b).

In this work, we propose a “deep optimization” approach that couples the
strengths of deep learning models and optimization in an effective learning
scheme. To avoid the limitations of CNN-based generative schemes, we propose
an approach based on neural implicit fields, that can be applied to physical
domains of any size, shape, and resolution. We also propose a generalizable
scheme for representing sparse boundary conditions in numerical physics
domains, eliminating the need for computationally expensive and domain-
limiting physical field calculations, previously thought to be crucial for
constraint representation (Chen et al., 2023). Instead, we show that our
computationally- and memory-efficient scheme is capable of representing sparse
boundary conditions effectively, without loss of performance. Indeed our
approach outperforms the state-of-the-art by more than 80% while running
2.5-50 times faster than the fastest methods in the state-of-the-art.

We also re-implement an updated SIMP optimizer. This optimizer improves upon
existing open-source Python implementations (Hunter et al., 2017) using the
latest advances in structural engineering optimizers for TO (Wang et al.,
2021a). Leveraging this updated optimizer, we introduce a new dataset using
our improved optimizer and retrain and benchmark previous state-of-the-art
methods using this updated dataset.

#### Contributions.

Our contributions can be summarized as follows:

  * 1.

We introduce a novel framework for TO called “Neural Implicit Topology
Optimizer (NITO),” a model capable of generating near-optimal topologies. NITO
leverages “Boundary Point Order-Invariant MLP (BPOM),” a simple, generalizable
architecture to represent sparse physical boundary conditions, that can
perform on par with computationally expensive physical fields.

  * 2.

We show that NITO and BPOM enable resolution-free and generalizable generation
of topologies and demonstrate that NITO can be trained on multiple
resolutions/domains and generate topologies for multiple resolutions, making
it one of the first resolution-free and truly generalizable frameworks for TO
with deep learning addressing a major limitation in prior works.

  * 3.

We empirically show that NITO is an effective framework, producing topologies
with up to 80% lower compliance errors than SOTA learning-based models, while
requiring an order of magnitude fewer parameters which enables it to run
2.5-50x faster while retaining high performance. More notably we show that
NITO is a generalizable framework, retaining similar performance metrics for
higher resolution with the same architecture and number of parameters.

  * 4.

Finally, we implement a new SIMP optimizer for minimum compliance TO, which is
based on the latest literature on the topic. We demonstrate that our new
optimizer is more capable than the previous optimizer used as the gold
standard in recent literature. The new optimizer produces topologies that are
on average 3% lower in compliance and runs up to 6x faster than prior Python
implementations.

##  2 Background

This section will provide some background information on topology optimization
and neural implicit fields. We will also review existing methods for optimal
topology generation using deep generative models. For more related work and
limitations of previous approaches, see Appendix A.

Figure 2: Topology Optimization. Given a domain, boundary conditions, loads,
and volume fraction, TO aims to find the design variables $\phi$ that maximize
performance (in this case minimizing compliance $f$) for the system,
fulfilling all the prescribed constraints and respecting the underlying
physics (Static Equilibrium).

#### Structural Topology Optimization

Topology optimization (TO) is a powerful computational method that identifies
an efficient material layout, given a material budget, constraints, boundary
conditions, and loads. In structural applications, TO often aims to minimize
compliance (structural deformation) given a set of forces and supports (Fig.2)
(Liu & Tovar, 2014). A prominent technique in TO is the Solid Isotropic
Material with Penalization (SIMP) method, introduced by Bendsøe (1989). This
method models material properties through a density field, where the density
value signifies the amount of material in a specific region. The optimization
iteratively simulates the system to evaluate the objective, then adjusts the
density field based on the gradient of the objective score for the field.
Mathematically, we represent the density distribution as $\rho(\phi)$, where
$\phi$ is a set of design variables, each representing a value within the
problem domain $\Omega$. The optimization task is structured as follows:

$\begin{array}[]{cl}\underset{\phi}{\operatorname{min}}&f=\mathbf{F}^{T}\mathbf{d}\\\
\text{ s. t. }&\mathbf{K}(\phi)\mathbf{d}=\mathbf{F}\\\
&\sum_{e\in\Omega}\rho^{e}(\phi)v^{e}\leq V\\\
&\phi_{\min}\leq\phi_{i}\leq\phi_{\max}\quad\forall i\in\Omega\end{array}$ (1)

The objective is to minimize the compliance $\mathbf{F}^{T}\mathbf{d}$, with
$\mathbf{F}$ being the load tensor and $\mathbf{d}$ the nodal displacement, a
solution to the equilibrium equation $\mathbf{K}(\phi)\mathbf{d}=\mathbf{F}$,
where $\mathbf{K}(\phi)$ is the stiffness matrix contingent on the design
variables $\phi$. The constraint $\sum_{e\in\Omega}\rho^{e}(\phi)v^{e}\leq V$
ensures the total volume does not exceed a maximum limit $V$ (often expressed
as a fraction of the domain volume). The optimization seeks design variables
within specified bounds ($\phi_{\min}$ and $\phi_{\max}$) for every element
$i$ in the domain $\Omega$. By allowing continuous variation of design
variables between 0 and 1, this formulation supports gradient-based
optimization. Nonetheless, the expensive FEA simulation (solving
$\mathbf{K}(\phi)\mathbf{d}=\mathbf{F}$) at each iteration causes the overall
optimization to be computationally intensive. Notably, solving the linear
system of the equations scales cubically ($O(n^{3})$) with the size of the
problem, in this case, the number of equations that must be solved (two
equations per node in 2D).

#### Implicit Neural Fields

A neural field is a field that is partially or fully defined by a neural
network (Xie et al., 2022). This neural network takes some form of coordinate
representation in space $\mathbf{x}\in\mathbb{R}^{n}$ as input and outputs a
set of values (i.e. a measure of the desired field)
$\Phi(\mathbf{x})\in\mathbb{R}^{m}$:

$\tilde{\Phi}(\mathbf{x})=f_{\Theta}(\mathbf{x})$ (2)

where $f_{\Theta}$ is the neural network function given parameters $\Theta$.
These neural fields have proven effective in representing audio (Du et al.,
2021), images (Skorokhodov et al., 2021; Sitzmann et al., 2020), videos (Yu et
al., 2022), 3d objects (Park et al., 2019), 3D scenes (Mildenhall et al.,
2020), and many more applications (see the review by Xie et al. (2022) for a
more comprehensive picture). It has also been shown that neural implicit
representations can be directly optimized using gradient-based optimizers
similar to SIMP to represent the optimal topology for any given problem
(Zehnder et al., 2021; Joglekar et al., 2023), which shows clearly that these
implicit neural representations are indeed capable of generating density
fields that represent individual optimal topologies. In this work, we propose
a conditional implicit neural field approach that is meant to generate
different optimal topologies based on conditions (i.e., material budget,
loads, supports).

##  3 Methodology

Figure 3: The NITO framework for topology optimization. The boundary
conditions are processed as point clouds using the BPOM approach and a neural
field is conditioned on these representations by modulating layer
normalization based on the latent representation of the constraints to predict
the optimal density field given the boundary conditions of the problem.
Finally, the predicted density field is further refined through a few steps of
direct optimization.

Our framework aims to enhance inference speed by eliminating iterative
sampling for topology generation, ensuring linear scalability with sampled
points. Additionally, it introduces a method for managing sparse conditioning
in dense models without relying on fields. The framework, based on neural
fields, overcomes size and shape constraints in images, enabling application
to any spatial domain and improving generalizability in topology optimization
(TO).

###  3.1 NITO: Neural Implicit Topology Optimization

Our main objective in this problem is to learn how to distribute material
inside a domain such that the mechanical compliance of the resulting structure
is minimized. As we discussed, the distribution of material is often described
by a density field, which we denote as $\rho(\mathbf{x})$, where $\mathbf{x}$
is the coordinate of a given point in space and $\rho(\mathbf{x})$ is the
density of material in that part of space. This formulation is highly
compatible with neural fields. However, we do not intend to just learn a
single distribution of material. Rather, we need to learn a conditional
distribution based on the boundary conditions and volume ratio for a given
problem. This means that we have to learn a conditional density field for a
given problem:

$\hat{\rho}(\mathbf{x}|\mathbf{C};\theta)=f_{\theta}(\mathbf{x},\mathbf{C}),$
(3)

where $f$ denotes the neural field function which is defined by the neural
network architecture and $\mathbf{C}$ is a condition vector that includes
information about the domain shape, boundary conditions, and desired volume
ratio. In practice, this density field is desired to have a density of exactly
0 or 1 at any point, representing where material should or should not be
placed. As such, the density fields that are generated by the SIMP optimizer
for a given problem have binary values. In practice, it is more practical to
formulate the problem as the probability of material being placed at any given
coordinate. As such, the formulation of the objective can be written as:

$\begin{split}\mathcal{L}(\theta)=&-\mathbb{E}_{\mathbf{x},\mathbf{C}}\left[\rho(\mathbf{x}|\mathbf{C})\log
f_{\theta}(\mathbf{x},\mathbf{C})\right.\\\
&\left.+(1-\rho(\mathbf{x}|\mathbf{C}))\log(1-f_{\theta}(\mathbf{x},\mathbf{C}))\right]\end{split}$
(4)

where $\rho(\mathbf{x}|\mathbf{C})$ is the probability of material existing at
$\mathbf{x}$, which in this case is the same as the output of the SIMP
optimizer. A detailed schematic of how the NITO framework operates is shown in
Figure 3. We discuss implementation details of the neural fields and
conditioning mechanism in Appendix D.

###  3.2 BPOM: Latent Constraint Representation

Having discussed the neural field implementation, we now consider boundary
conditions in discretized domains, as seen in TO problems. In particular, we
discuss how arbitrary discrete boundary conditions can condition generative
models in a sparse and generalizable way. This is particularly important for
large domains, where defining constraints over grids is computationally
cumbersome and memory-expensive.

#### Conditioning on Physics Fields

Previous works (Mazé & Ahmed, 2023; Nie et al., 2021b; Chen et al., 2023) have
argued that representing the boundary conditions of TO problems through
simulation-derived physical fields such as stress and strain energy is _the
most effective_ (or only) way to generate high-performing topologies using
conditioning generative models. This has consequences for the generalizability
of these methods since the fields often have to be handled by CNN models which
are fixed to one resolution and domain.

Figure 4: Comparison of field-based such as stress fields (middle), and point-
cloud-based (right) representations, given a TO problem (left). Unlike the
expensive and domain-limiting iterative FEA method, the point clouds offer a
generalizable and memory efficient, representation of the boundary conditions.

We believe that field-based conditioning’s popularity is largely attributable
to a conditioning mechanism limitation. In most existing studies, authors have
only considered conditioning models at the input of the first layers. Using
this approach, field-based conditioning was the only effective method.
However, it is well understood that this input-only conditioning allows models
to ignore the input conditions to a great extent (Perez et al., 2017), and
that more efficacious approaches exist. Therefore, in this paper, we propose a
simple and effective model to represent conditions and incorporate them more
effectively while enabling more generalizable and widespread use.

#### Constraints as Point Clouds

In our approach, we aim to represent the boundary conditions of TO problems in
a manner that is _agnostic to the problem domain_. This enables our method to
easily generalize to different domains without needing to train separate
models on individual datasets for each domain shape and resolution. In all
problems, conditioning is based on loads, displacement constraints, and volume
fraction (see Figures 4 and 2). Since a problem may have arbitrarily many
loads or boundary conditions, we need a mechanism that can take all loads and
boundary conditions for a problem and reduce them to a single latent
representation. To do this, we treat these sparse conditions as a set of three
point clouds – one for loads, x supports, and y supports (in higher
dimensional problems, more point clouds can be used) (see Figures 3 and 4).
Then we process each point cloud using ResP Layers proposed by Ma et al.
(2022) in their point cloud model, PointMLP. However, unlike PointMLP, we did
not find a need for the geometric affine module (as proposed by the authors
for complex point cloud geometries), given the simpler geometric nature of our
sparse boundary condition point clouds.

#### Order-Invariant Aggregation

As discussed, the point clouds can have any size. We therefore take the output
of the point cloud model and perform order-invariant pooling to reduce every
point cloud into one single vector representing the boundary conditions. We do
this by computing minimum, maximum, and average pooling and concatenating the
three pooling results. In the end, the three point clouds are each reduced,
then concatenated into one vector. This vector is then input to the modulated
layer normalization mechanism in our conditional neural fields (see Appendix D
and Figure 3). We call this boundary condition representation Boundary Point
Order-Invariant MLP (BPOM) and we show that this representation works just as
well as physical fields. Finally, for the volume fraction condition (which is
always a single floating point value), we use a single fully-connected layer
and concatenate its output to the BPOM outputs of the boundary condition point
clouds (Figure 3).

###  3.3 Few-Step Direct Optimization On Neural Field

Despite the success of generative models and other deep learning techniques
for TO, we observe that the performance of these models always lags behind the
optimization baseline. To address this, Giannone et al. (2023) proposed
applying a few steps (5-10 steps) of optimization to the outputs of deep
generative models with SIMP (compared to 200-500 iterations for full
optimization). This allows SIMP to rapidly hone in on more accurate structures
for the given boundary conditions using near-optimal generated topologies as a
starting point. We see this ‘optimization-seeding’ approach as a crucial step
in topology generation using generative models and inherit it in our approach.
To ensure that the optimization itself is performed optimally, we implement
the SIMP method based on the latest research on TO (Wang et al., 2021b) and
use this solver for our experiments and training. This completes the NITO
framework as a “deep optimization” approach (Figure 3).

##  4 Experiments

In this section, we present an assortment of experiments to compare the
performance of NITO to existing state-of-the-art models in the literature.
With these experiments, we provide compelling evidence that:

  * 1.

NITO is a scalable, resolution-free paradigm that outperforms convolution-
based methodologies, with a significantly smaller parameter count.

  * 2.

NITO is fast and can generate constraint-satisfying, high-performance
topologies much faster than a state-of-the-art SIMP optimizer, and

  * 3.

NITO merges the power of deep learning models as an efficient first stage, and
the precision and guarantees of optimization as a reliable second stage to
form an effective ‘deep optimization’ framework. Deep optimization is a
promising research direction for large-scale adoption and deployment of deep
learning methodologies in engineering design.

For more experiments and discussion, see Appendix B. See Appendix C for
extensive visualizations.

###  4.1 Experimental Details

Before presenting results, we will discuss the different aspects of our
experiments in this section.

#### Data

We created a dataset of SIMP-optimized topologies for the 64x64 and 256x256
unit square domains. We include loading conditions, boundary conditions,
volume fractions, and stress and strain energy fields in our dataset. Noting
that previous works used older implementations of SIMP, specifically ToPy
(Hunter et al., 2017), which lacks recent improvements and uses a slower
solver, we developed a new SIMP optimizer in Python. This implementation
utilizes the most current and efficient SIMP algorithm available (Wang et al.,
2021b). We then re-created the 64x64 dataset from Mazé & Ahmed (2023) with our
solver, finding significant improvements in optimality. For fairness in
comparison, we retrained the TopoDiff model(Mazé & Ahmed, 2023) (the best-
performing model in literature) on our new dataset, while also acknowledging
the original performance metrics reported by other authors. Further details on
the dataset and solver are included in Appendix E.

#### Evaluation Metrics

We evaluate the models in terms of performance (i.e., minimum compliance) and
constraint satisfaction. First, to measure how well the models perform in
compliance minimization, we measure compliance error (CE), which is calculated
by subtracting the compliance of a generated sample from the compliance of the
SIMP-optimized solution for the corresponding problem. We also measure the
volume fraction error (VFE), which quantifies the absolute error between the
generated topology’s actual volume fraction and the target volume fraction for
the given problem. Beyond these base performance metrics, we quantify
inference time. The speed of the SIMP optimizer is also benchmarked for
comparison. We remove outlying samples, which have extremely high compliance
errors (above 1000%) due to models failing to place material in locations
where the load is applied. We do this for all models including other SOTA
models that we compare against, as is common practice (Mazé & Ahmed, 2023;
Giannone et al., 2023; Nie et al., 2021b).

#### Setup

We train NITO for 50 epochs in three different scenarios: (1) We train on
30,000 topologies with a resolution of 64x64. (2) We train on 60,000 256x256
topologies. (3) We train on both datasets simultaneously to demonstrate the
generalizability of our approach. To compare our approach to the state of the
art, we also train TopoDiff on the 64x64 in the manner the original authors
did (Mazé & Ahmed, 2023). Notably, TopoDiff is currently the state-of-the-art
model and outperforms all existing models in literature (Giannone et al.,
2023). Training a diffusion model like TopoDiff on large images of size
256x256 is rather difficult and can be prohibitive for most practitioners
given the memory requirements and time needed to train these models. Despite
this, we train TopoDiff on the 256x256 dataset for 500,000 steps and report
the results.

###  4.2 Performance

Table 1: Quantitative Evaluation on 64x64 datasets. All TopoDiff variants have
been re-trained on optimized topologies obtained using our improved SIMP
optimizer. ∗TopologyGAN and cDDPM results from Mazé & Ahmed (2023) and
Giannone et al. (2023). w/ G: using a classifier and regression guidance. FS-
SIMP: Few-Steps of optimization. CE: Compliance Error. VFE: Volume Fraction
Error. NITO achieves SOTA performance on topology optimization in terms of
compliance and volume fraction error.

Model | FS-SIMP | CE % Mean | CE % Med | VFE % Mean | VFE % Med
---|---|---|---|---|---
TopologyGAN∗ | - | 48.51 | 2.06 | 11.87 | -
cDDPM∗ | - | 60.79 | 3.15 | 1.72 | -
TopoDiff | - | 3.23 | 0.45 | 1.14 | 0.94
TopoDiff w/ G | - | 2.59 | 0.49 | 1.18 | 0.99
NeuralField | - | 6.28 | 0.38 | 1.37 | 0.75
NITO (ours) | 5 | 0.81 | 0.098 | 0.50 | 0.32
NITO (ours) | 10 | 0.48 | 0.067 | 0.41 | 0.28

Table 2: Quantitative Evaluation on 256x256 datasets. The columns are the same
as Table 1. NITO with BPOM and a few steps of optimization is effective at
generating topologies with high performance irrespective of the considered
problem resolution. On the other hand, the performance of TopoDiff has
degraded so much that raw neural fields (highlighted in green) outperforms it.

Model | FS-SIMP | CE % Mean | CE % Med | VFE % Mean | VFE % Med
---|---|---|---|---|---
TopoDiff | - | 16.62 | 0.59 | 2.92 | 2.29
NeuralField | - | 7.52 | 0.72 | 1.39 | 0.89
NITO (ours) | 5 | 0.38 | 0.035 | 0.56 | 0.37
NITO (ours) | 10 | 0.10 | 0.011 | 0.38 | 0.28

In Table 1 we present a comprehensive quantitative evaluation of SOTA models
for topology optimization on the 64x64 datasets. TopologyGAN and cDDPM show
higher values in CE % Mean and VFE % Mean compared to other models, which is
expected since TopologyGAN is a much older benchmark and cDDPM is a naive
conditioning of diffusion models and is not expected to perform well in such a
complex problem. A vanilla neural fields model without optimization does not
perform as well as TopoDiff on average, however, the median performance of the
neural fields model is better than all other baselines. This is due to a small
number of problems that incur higher compliance errors, while the majority of
samples outperform those generated by the state-of-the-art as reflected by the
median values (highlighted in green on table 1). A similar pattern is apparent
in volume fraction error scores as well, however, the mean values are not as
far apart in this realm. Unambiguously though, Table 1 demonstrates that our
NITO framework achieves a significant leap in performance compared to the
SOTA. Neural fields start with average compliance errors of more than double
TopoDiff. However, after even 5 steps of direct optimization, NITO outperforms
TopoDiff and other methods by a large margin. When taking 10 steps of direct
optimization, NITO achieves results that are very close to the optimizer,
which takes up to 500 iterations to converge.

Next, we proceed to test NITO and TopoDiff on the much more challenging
dataset of 256x256 topologies. Table 2 presents the results of this study for
NITO and TopoDiff (the leading SOTA approach). In this case, we train TopoDiff
with a larger model size to allow for effective learning on the 256x256
images. Despite this, we see a significant performance degradation of TopoDiff
with increased resolution. This performance decrease is so severe that the
vanilla neural fields (highlighted in green in table 2) outperform TopoDiff.
This is while NITO achieves results on par with or slightly better than the
64x64 tests, all while using the same model size architecture and training.

###  4.3 Direct Optimization Study

Table 3: In this table, we present the results of applying direct optimization
on different approaches. This table demonstrates that the topologies predicted
by neural fields are closer to the optimal. This indicates that despite
lacking some finer details, the few-step optimizer quickly converges to a very
good solution. This is contrasted with the topologies generated by TopoDiff
which are detailed, but further from an optimal solution, reducing the
benefits of additional optimizer steps.

Model | FS-SIMP | CE % Mean | CE % Med | VFE % Mean
---|---|---|---|---
TopoDiff | - | 3.23 (-) | 0.45 (-) | 1.14 (-)
TopoDiff | 5 | 3.55 (+9.91%) | 0.42 (-6.67%) | 0.67 (-41.2%)
TopoDiff | 10 | 1.38 (-57.3%) | 0.33 (-26.7%) | 0.45 (-60.5%)
TopoDiff w/ G | - | 2.59 (-) | 0.49 (-) | 1.18 (-)
TopoDiff w/ G | 5 | 2.24 (-13.5%) | 0.44 (-10.2%) | 0.69 (-41.5%)
TopoDiff w/ G | 10 | 1.05 (-59.5%) | 0.32 (-34.7%) | 0.45 (-61.9%)
NeuralField | - | 6.28 (-) | 0.38 (-) | 1.37 (-)
NITO (ours) | 5 |  0.81 (-87.1%) |  0.098 (-74.2%) |  0.50 (-63.5%)
NITO (ours) | 10 |  0.48 (-92.4%) | 0.067(-82.4%) |  0.41 (-70.1%)

In Table 3, we provide performance metrics for NITO and TopoDiff with direct
optimization. Optimization of topologies generated by TopoDiff yields a much
smaller performance boost compared to NITO and even reduces performance in
some cases. As a result, compared to TopoDiff with 10 iterations of direct
optimization, NITO outperforms with only 5 steps and widens the gap with 10.
Importantly, NITO, despite being a much smaller model without insight from
physical fields, generates topologies that are closer to optimal than CNN-
based generative methods like TopoDiff. However, the topologies generated by
neural fields lack more refined details (see Figure 1), which is quickly
resolved through a few iterations of optimization (5-10 iterations compared to
500 iterations of the full optimization). This establishes NITO as a more
robust, generalizable, and efficient framework for optimal topology
generation.

###  4.4 Inference Speed & Efficiency

Table 4:  Average Inference time for different problem resolutions. For the
baselines, the inference time is estimated without considering the few steps
of SIMP to improve performance. We include 10 SIMP iterations when computing
NITO inference time. NITO is resolution-free, i.e. we can leverage the same
small model for 64x64, 256x256, and any intermediate resolution. These times
are measured using an RTX 4090 GPU and an Intel Core i-9-13900K CPU. We run
the SIMP optimizer for 300 iterations. We compute the conditioning fields for
TopoDiff using our fast solver, greatly speeding up inference for the
baselines.

| 64x64 Resolution | 256x256 Resolution
---|---|---
Model | Params (M) | Inference (s) | Params (M) | Inference (s)
TopoDiff | 121 | 1.86 | 553 | 10.81
TopoDiff w/ G | 239 | 4.79 | 1092 | 22.04
DOM | 121 | 0.82 | 553 | 7.82
NeuralFields | 22 | 0.005 | 22 | 0.16
SIMP (Hunter et al., 2017) | - | 18.12 | - | 316.02
SIMP (our implementation) | - | 3.45 | - | 69.45
NITO (ours) | 22 | 0.14 | 22 | 2.88

In Table 4 we present a comparative analysis of the generation speed of NITO
against classic optimization methods and generative models, specifically
focusing on inference time across different problem resolutions (64x64 and
256x256). The main objective of these kinds of deep learning schemes for data-
driven design is to generate topologies quickly, in a manner that is faster
than conventional optimization. In Table 4 we see that in both 64x64 and
256x256 resolutions, NITO is significantly faster than the state-of-the-art
models. Specifically for the 64x64 resolution, NITO is 83% faster than the
fastest state-of-the-art model DOM (Giannone et al., 2023), and 97% faster
than our fast SIMP implementation. This demonstrates the speed and efficiency
of NITO which is orders of magnitude faster than the SOTA, while
simultaneously generating superior topologies. These gaps are widened in
higher resolution tests with NITO significantly outperforming baselines in
inference speed, compliance error, and volume fraction error. NITO’s superior
scaling with resolution reflects the performance degradation of CNN-based
models at high resolutions.

Table 4 highlights another important metric, which is the number of parameters
each model uses in their architecture. NITO can be trained on both image
resolutions with the same number of parameters, 22 million, achieving SOTA
performance, while CNN-based models have to be made larger and still face
significant performance degradation as we demonstrated in prior sections. This
is further evidence of the efficiency and scalability of NITO. We train NITO
for both 64x64 and 256x256 resolution for the same number of steps while
sampling the same number of points for each batch during training, which means
that the model trains roughly for the same amount of time and with the same
memory requirements. In fact a _single consumer GPU_ (we use an RTX 4090) is
enough to train NITO. On the contrary, diffusion models like TopoDiff or DOM
must grow to match larger resolutions and therefore require more memory and
time to run and train. For resolutions above 256x256 or 3D TO problems, these
frameworks can be impractical to train or run for most practitioners. In
contrast, NITO is built to generalize to different domains/resolutions without
issue, allowing for practical training of large problems with consumer-level
computational resources. In the following, we specifically showcase the
versatility of NITO when handling different domains.

###  4.5 Resolution-Free Generalization

Table 5: Quantitative Evaluation on both datasets using models trained on
mixed resolution data and different resolutions. NITO variants in this table
use 10 steps of direct optimization. The results show that NITO trained on
mixed-resolution data performs well across different resolutions.

Model | Train Res. | CE % Mean | CE % Med | VFE % Mean
---|---|---|---|---
64x64 Test
Neural Fields | 64 | 6.28 | 0.38 | 1.37
NITO | 64 | 0.48 | 0.067 | 0.41
Neural Fields | 256 | 10.5 | 0.70 | 1.67
NITO | 256 | 0.51 | 0.085 | 0.50
Neural Fields | 256,64 | 7.11 | 0.39 | 1.36
NITO | 256,64 | 0.74 | 0.069 | 0.52
256x256 Test
Neural Fields | 64 | 10.47 | 0.81 | 1.46
NITO | 64 | 0.26 | 0.10 | 0.34
Neural Fields | 256 | 7.52 | 0.72 | 1.39
NITO | 256 | 0.10 | 0.011 | 0.38
Neural Fields | 256,64 | 8.47 | 0.72 | 1.43
NITO | 256,64 | 0.13 | 0.012 | 0.40

A foundational pillar of NITO is its generalizability and adaptability to
multiple resolutions and physical domains. We specifically conduct a study to
showcase this generalizability and resolution-free adaptability. As mentioned
before, we trained NITO on both 64x64 and 256x256 data separately, but here we
also present results for training NITO on both the 64x64 and 256x256 datasets
simultaneously. Then we use all three models to test on both the 64x64 test
dataset and the 256x256 test dataset. These results are presented in detail in
Table 5. We see that the model trained on both datasets simultaneously
performs on par and better than other SOTA models on both the 256x256 data and
the 64x64 data. This proves NITO’s capacity to be trained on multiple domains
and perform just as well on both domains. Leveraging NITO’s flexibility and
BPOM’s adaptability, models trained at one resolution can be effectively
applied to tasks at different resolutions. This capability is demonstrated by
testing NITO models trained on 64x64 data with 256x256 resolution tasks, and
vice versa, showcasing the model’s resolution-free generalization. Remarkably,
NITO’s performance is consistent across resolutions: it performs similarly
when trained on lower-resolution data compared to higher or mixed-resolution
data, and likewise when tested on higher-resolution or lower-resolution
problems. This underscores NITO’s exceptional adaptability, indicating that
its architecture not only supports training across multiple domains but also
facilitates the transfer of learning from one domain to another, provided the
problems share related distributions. This ability signifies a critical
advantage of such frameworks—the potential to train on cost-effective low-
resolution data and subsequently refine the models for higher resolutions with
minimal additional data and reduced time.

##  5 Conclusion & Limitations

In this paper, we introduce Neural Implicit Topology Optimization (NITO), a
novel resolution-free and domain-agnostic deep-learning framework for topology
optimization. Leveraging our proposed Boundary Point Order-Invariant MLP
(BPOM) to represent boundary conditions, NITO eliminates the need for costly
physical fields, overcoming the resolution and domain limitations of CNNs.
NITO significantly outperforms state-of-the-art models in multiple resolutions
and domains. Notably, it does so with remarkable speed and efficiency,
requiring far fewer parameters. NITO’s scalability and generalizability offer
a robust foundation for future models in topology optimization and other
physics-based problems, conferring a solution to high-dimensional problems
that were previously insurmountable with CNN-based methods.

#### Limitations

While NITO presents significant advancements in topology optimization, it is
not without potential shortcomings. Firstly, NITO is not a generative model,
which means that the outputs of NITO are deterministic and not diverse for a
given set of constraints. This limits NITO’s ability to generate solutions
that enable exploration of the design space. This characteristic could
potentially hinder NITO’s efficacy in addressing problems outside its training
distribution, given the lack of a generative mechanism that enhances
performance in entirely new tasks (See Appendix B for a broader discussion on
this). Future work could be devoted to addressing this matter, potentially by
leveraging recent advances in generative frameworks for neural implicit fields
(You et al., 2023; Kosiorek et al., 2021). The second limitation arises from
the fact that NITO requires a small number of direct optimization steps, and
without this, the neural field-generated topologies with BPOM lack the
necessary details. Therefore, future improvements should aim at refining
NITO’s training procedures and architecture. Such enhancements would reduce
the reliance on direct optimization, empowering the neural fields to produce
topologies with greater detail and precision.

## Impact Statement

While the introduction of Neural Implicit Topology Optimization (NITO)
represents a significant breakthrough in topology optimization, bringing
efficiency and resolution-free solutions, it is crucial to consider the
potential implications for society and the engineering field.

Firstly, the automation and efficiency of NITO could lead to significant
shifts within the engineering sector. The traditional roles of engineers,
especially those involved in iterative design and optimization processes, may
evolve or diminish as automation becomes more prevalent. This could lead to a
need for re-skilling or up-skilling of the workforce to ensure that engineers
remain relevant and can work synergistically with AI-driven systems like NITO.

Moreover, like with most advanced technologies, there is a risk of over-
reliance on our framework. If engineers rely too heavily on NITO without fully
understanding its internal workings, there might be a knowledge gap that could
lead to oversights in design or an inability to troubleshoot and innovate
beyond what the model offers. The “black box” nature of deep learning models
can sometimes lead to a lack of interpretability, making it difficult to
diagnose errors or understand the model’s decision-making process in depth.
This is crucial as TO is often used in safety-critical components, from
brackets to bridges. Deploying models without testing their performance using
experiments creates a risk.

## References

  * Abueidda et al. (2020) Abueidda, D. W., Koric, S., and Sobh, N. A.  Topology optimization of 2d structures with nonlinearities using deep learning.  _Computers & Structures_, 237:106283, 9 2020.  ISSN 00457949.  doi: 10.1016/j.compstruc.2020.106283.  URL https://linkinghub.elsevier.com/retrieve/pii/S0045794920300869.
  * Ahmed et al. (2016) Ahmed, F., Fuge, M., and Gorbunov, L. D.  Discovering diverse, high quality design ideas from a large corpus.  In _International Design Engineering Technical Conferences and Computers and Information in Engineering Conference_ , volume 50190, pp. V007T06A008. American Society of Mechanical Engineers, 2016.
  * Ates & Gorguluarslan (2021) Ates, G. C. and Gorguluarslan, R. M.  Two-stage convolutional encoder-decoder network to improve the performance and reliability of deep learning models for topology optimization.  _Structural and Multidisciplinary Optimization_ , 63(4):1927–1950, 4 2021.  ISSN 1615-147X.  doi: 10.1007/s00158-020-02788-w.  URL http://link.springer.com/10.1007/s00158-020-02788-w.
  * Aulig & Olhofer (2013) Aulig, N. and Olhofer, M.  Evolutionary generation of neural network update signals for the topology optimization of structures.  In _Proceedings of the 15th annual conference companion on Genetic and evolutionary computation_ , pp. 213–214, New York, NY, USA, 7 2013. ACM.  ISBN 9781450319645.  doi: 10.1145/2464576.2464685.  URL https://dl.acm.org/doi/10.1145/2464576.2464685.
  * Banga et al. (2018) Banga, S., Gehani, H., Bhilare, S., Patel, S., and Kara, L.  3d topology optimization using convolutional neural networks.  _Preprint_ , 8 2018.  URL http://arxiv.org/abs/1808.07440.
  * Barmada et al. (2021) Barmada, S., Fontana, N., Formisano, A., Thomopulos, D., and Tucci, M.  A deep learning surrogate model for topology optimization.  _IEEE Transactions on Magnetics_ , 57(6):1–4, 6 2021.  ISSN 0018-9464.  doi: 10.1109/TMAG.2021.3063470.  URL https://ieeexplore.ieee.org/document/9367238/.
  * Behzadi & Ilieş (2021a) Behzadi, M. M. and Ilieş, H. T.  Real-time topology optimization in 3d via deep transfer learning.  _Computer-Aided Design_ , 135:103014, 6 2021a.  ISSN 00104485.  doi: 10.1016/j.cad.2021.103014.  URL https://linkinghub.elsevier.com/retrieve/pii/S0010448521000257.
  * Behzadi & Ilieş (2021b) Behzadi, M. M. and Ilieş, H. T.  Real-time topology optimization in 3d via deep transfer learning.  _Computer-Aided Design_ , 135:103014, 2021b.
  * Behzadi & Ilieş (2021) Behzadi, M. M. and Ilieş, H. T.  Gantl: Toward practical and real-time topology optimization with conditional generative adversarial networks and transfer learning.  _Journal of Mechanical Design_ , 144(2), 12 2021.  ISSN 1050-0472.  doi: 10.1115/1.4052757.  URL https://doi.org/10.1115/1.4052757.  021711\.
  * Bendsøe (1989) Bendsøe, M. P.  Optimal shape design as a material distribution problem.  _Structural optimization_ , 1:193–202, 1989.
  * Bendsøe & Kikuchi (1988) Bendsøe, M. P. and Kikuchi, N.  Generating optimal topologies in structural design using a homogenization method.  _Computer Methods in Applied Mechanics and Engineering_ , 71(2):197–224, 1988.  ISSN 0045-7825.  doi: https://doi.org/10.1016/0045-7825(88)90086-2.  URL https://www.sciencedirect.com/science/article/pii/0045782588900862.
  * Berthelot et al. (2017) Berthelot, D., Schumm, T., and Metz, L.  Began: Boundary equilibrium generative adversarial networks.  _arXiv preprint arXiv:1703.10717_ , 2017.
  * Blattmann et al. (2022) Blattmann, A., Rombach, R., Oktay, K., and Ommer, B.  Retrieval-augmented diffusion models.  _arXiv preprint arXiv:2204.11824_ , 2022.
  * Boyd & Vandenberghe (2004) Boyd, S. and Vandenberghe, L.  _Convex optimization_.  Cambridge university press, 2004.
  * Brock et al. (2018) Brock, A., Donahue, J., and Simonyan, K.  Large scale gan training for high fidelity natural image synthesis.  _arXiv preprint arXiv:1809.11096_ , 2018.
  * Brown et al. (2020) Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.  Language models are few-shot learners.  _arXiv preprint arXiv:2005.14165_ , 2020.
  * Chandrasekhar & Suresh (2021a) Chandrasekhar, A. and Suresh, K.  Tounn: Topology optimization using neural networks.  _Struct. Multidiscip. Optim._ , 63(3):1135–1149, mar 2021a.  ISSN 1615-147X.  doi: 10.1007/s00158-020-02748-4.  URL https://doi.org/10.1007/s00158-020-02748-4.
  * Chandrasekhar & Suresh (2021b) Chandrasekhar, A. and Suresh, K.  Multi-material topology optimization using neural networks.  _Computer-Aided Design_ , 136:103017, 7 2021b.  ISSN 00104485.  doi: 10.1016/j.cad.2021.103017.  URL https://linkinghub.elsevier.com/retrieve/pii/S0010448521000282.
  * Chandrasekhar et al. (2022) Chandrasekhar, A., Sridhara, S., and Suresh, K.  Gm-tounn: Graded multiscale topology optimization using neural networks.  _arXiv preprint arXiv:2204.06682_ , 2022.
  * Chen et al. (2022) Chen, H., Whitefoot, K. S., and Kara, L. B.  Concurrent build direction, part segmentation, and topology optimization for additive manufacturing using neural networks.  In _International Design Engineering Technical Conferences and Computers and Information in Engineering Conference_ , volume 86229, pp. V03AT03A029. American Society of Mechanical Engineers, 2022.
  * Chen et al. (2023) Chen, H., Joglekar, A., and Burak Kara, L.  Topology Optimization Using Neural Networks With Conditioning Field Initialization for Improved Efficiency.  _Journal of Mechanical Design_ , 146(6):061702, 12 2023.  ISSN 1050-0472.  doi: 10.1115/1.4064131.  URL https://doi.org/10.1115/1.4064131.
  * Child (2020) Child, R.  Very deep vaes generalize autoregressive models and can outperform them on images.  _arXiv preprint arXiv:2011.10650_ , 2020.
  * Deng & To (2021) Deng, H. and To, A. C.  A parametric level set method for topology optimization based on deep neural network (dnn).  _Preprint_ , 1 2021.  URL http://arxiv.org/abs/2101.03286.
  * Devlin et al. (2018) Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ , 2018.
  * Du et al. (2021) Du, Y., Collins, K., Tenenbaum, J., and Sitzmann, V.  Learning signal-agnostic manifolds of neural fields.  In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), _Advances in Neural Information Processing Systems_ , volume 34, pp. 8320–8331. Curran Associates, Inc., 2021.  URL https://proceedings.neurips.cc/paper_files/paper/2021/file/4639475d6782a08c1e964f9a4329a254-Paper.pdf.
  * Elingaard et al. (2022) Elingaard, M. O., Aage, N., Bærentzen, J. A., and Sigmund, O.  De-homogenization using convolutional neural networks.  _Computer Methods in Applied Mechanics and Engineering_ , 388:114197, 1 2022.  ISSN 00457825.  doi: 10.1016/j.cma.2021.114197.  URL https://linkinghub.elsevier.com/retrieve/pii/S0045782521005284.
  * Fujita et al. (2021) Fujita, K., Minowa, K., Nomaguchi, Y., Yamasaki, S., and Yaji, K.  Design concept generation with variational deep embedding over comprehensive optimization.  In _International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-21_ , Virtual, Online, Aug 2021. ASME.
  * Giannone & Ahmed (2023) Giannone, G. and Ahmed, F.  Diffusing the optimal topology: A generative optimization approach.  _arXiv preprint arXiv:2303.09760_ , 2023.
  * Giannone et al. (2023) Giannone, G., Srivastava, A., Winther, O., and Ahmed, F.  Aligning optimization trajectories with diffusion models for constrained design generation.  _arXiv preprint arXiv:2305.18470_ , 2023.
  * Hertlein et al. (2021) Hertlein, N., Buskohl, P. R., Gillman, A., Vemaganti, K., and Anand, S.  Generative adversarial network for early-stage design flexibility in topology optimization for additive manufacturing.  _Journal of Manufacturing Systems_ , 59:675–685, 4 2021.  ISSN 02786125.  doi: 10.1016/j.jmsy.2021.04.007.  URL https://linkinghub.elsevier.com/retrieve/pii/S027861252100087X.
  * Ho et al. (2020) Ho, J., Jain, A., and Abbeel, P.  Denoising diffusion probabilistic models.  In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), _Advances in Neural Information Processing Systems_ , volume 33, pp. 6840–6851. Curran Associates, Inc., 2020.  URL https://proceedings.neurips.cc/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf.
  * Hu et al. (2024) Hu, J., He, Y., Xu, B., Wang, S., Lei, N., and Luo, Z.  If-tonir: Iteration-free topology optimization based on implicit neural representations.  _Computer-Aided Design_ , 167:103639, 2024.  ISSN 0010-4485.  doi: https://doi.org/10.1016/j.cad.2023.103639.  URL https://www.sciencedirect.com/science/article/pii/S0010448523001719.
  * Huang & Belongie (2017) Huang, X. and Belongie, S.  Arbitrary style transfer in real-time with adaptive instance normalization, 2017.
  * Hunter et al. (2017) Hunter, W. et al.  Topy-topology optimization with python, 2017.
  * Jiang et al. (2021) Jiang, J., Chen, M., and Fan, J. A.  Deep neural networks for the evaluation and design of photonic devices.  _Nature Reviews Materials_ , 6(8):679–700, 8 2021.  ISSN 2058-8437.  doi: 10.1038/s41578-020-00260-1.  URL https://www.nature.com/articles/s41578-020-00260-1.
  * Joglekar et al. (2023) Joglekar, A., Chen, H., and Kara, L. B.  Dmf-tonn: Direct mesh-free topology optimization using neural networks.  _Engineering with Computers_ , Oct 2023.  ISSN 1435-5663.  doi: 10.1007/s00366-023-01904-w.  URL https://doi.org/10.1007/s00366-023-01904-w.
  * Joo et al. (2021) Joo, Y., Yu, Y., and Jang, I. G.  Unit module-based convergence acceleration for topology optimization using the spatiotemporal deep neural network.  _IEEE Access_ , 9:149766–149779, 2021.  ISSN 2169-3536.  doi: 10.1109/ACCESS.2021.3125014.  URL https://ieeexplore.ieee.org/document/9599692/.
  * Kallioras et al. (2020) Kallioras, N. A., Kazakis, G., and Lagaros, N. D.  Accelerated topology optimization by means of deep learning.  _Structural and Multidisciplinary Optimization_ , 62(3):1185–1212, 9 2020.  ISSN 1615-147X.  doi: 10.1007/s00158-020-02545-z.  URL http://link.springer.com/10.1007/s00158-020-02545-z.
  * Kench & Cooper (2021) Kench, S. and Cooper, S. J.  Generating three-dimensional structures from a two-dimensional slice with generative adversarial network-based dimensionality expansion.  _Nature Machine Intelligence_ , 3(4):299–305, 2021.
  * Keshavarzzadeh et al. (2021) Keshavarzzadeh, V., Alirezaei, M., Tasdizen, T., and Kirby, R. M.  Image-based multiresolution topology optimization using deep disjunctive normal shape model.  _Computer-Aided Design_ , 130:102947, 2021.  ISSN 0010-4485.  doi: https://doi.org/10.1016/j.cad.2020.102947.  URL https://www.sciencedirect.com/science/article/pii/S0010448520301408.
  * Kosiorek et al. (2021) Kosiorek, A. R., Strathmann, H., Zoran, D., Moreno, P., Schneider, R., Mokrá, S., and Rezende, D. J.  Nerf-vae: A geometry aware 3d scene generative model, 2021.
  * Li et al. (2019) Li, B., Huang, C., Li, X., Zheng, S., and Hong, J.  Non-iterative structural topology optimization using deep learning.  _Computer-Aided Design_ , 115:172–180, 2019.  ISSN 0010-4485.  doi: https://doi.org/10.1016/j.cad.2019.05.038.  URL https://www.sciencedirect.com/science/article/pii/S001044851930185X.
  * Liu & Tovar (2014) Liu, K. and Tovar, A.  An efficient 3d topology optimization code written in matlab.  _Structural and Multidisciplinary Optimization_ , 50:1175–1196, 2014.
  * Ma & Zeng (2020) Ma, F. and Zeng, Z.  High-risk prediction localization: evaluating the reliability of black box models for topology optimization.  _Structural and Multidisciplinary Optimization_ , 62(6):3053–3069, 12 2020.  ISSN 1615-147X.  doi: 10.1007/s00158-020-02648-7.  URL https://link.springer.com/10.1007/s00158-020-02648-7.
  * Ma et al. (2022) Ma, X., Qin, C., You, H., Ran, H., and Fu, Y.  Rethinking network design and local geometry in point cloud: A simple residual mlp framework, 2022.
  * Mazé & Ahmed (2023) Mazé, F. and Ahmed, F.  Diffusion models beat gans on topology optimization.  In _Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)_ , Washington, DC, 2023.  URL https://arxiv.org/abs/2208.09591.
  * Mildenhall et al. (2020) Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R.  Nerf: Representing scenes as neural radiance fields for view synthesis.  In _ECCV_ , 2020.
  * Napier et al. (2020) Napier, N., Sriraman, S.-A., Tran, H. T., and James, K. A.  An artificial neural network approach for generating high-resolution designs from low-resolution input in topology optimization.  _Journal of Mechanical Design_ , 142(1), 1 2020.  ISSN 1050-0472.  doi: 10.1115/1.4044332.  URL https://asmedigitalcollection.asme.org/mechanicaldesign/article/doi/10.1115/1.4044332/955332/An-Artificial-Neural-Network-Approach-for.
  * Nichol et al. (2021) Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M.  Glide: Towards photorealistic image generation and editing with text-guided diffusion models.  _arXiv preprint arXiv:2112.10741_ , 2021.
  * Nie et al. (2021a) Nie, Z., Lin, T., Jiang, H., and Kara, L. B.  Topologygan: Topology optimization using generative adversarial networks based on physical fields over the initial domain.  _Journal of Mechanical Design_ , 143(3), 02 2021a.  ISSN 1050-0472.  doi: 10.1115/1.4049533.  URL https://doi.org/10.1115/1.4049533.  031715\.
  * Nie et al. (2021b) Nie, Z., Lin, T., Jiang, H., and Kara, L. B.  Topologygan: Topology optimization using generative adversarial networks based on physical fields over the initial domain.  _Journal of Mechanical Design_ , 143(3), 2021b.
  * Nobari et al. (2021a) Nobari, A. H., Chen, W., and Ahmed, F.  Pcdgan: A continuous conditional diverse generative adversarial network for inverse design.  _arXiv preprint arXiv:2106.03620_ , 2021a.
  * Nobari et al. (2021b) Nobari, A. H., Rashad, M. F., and Ahmed, F.  Creativegan: Editing generative adversarial networks for creative design synthesis.  In _International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-21_ , Virtual, Online, Aug 2021b. ASME.
  * Nobari et al. (2022) Nobari, A. H., Chen, W., and Ahmed, F.  Range-constrained generative adversarial network: Design synthesis under constraints using conditional generative adversarial networks.  _Journal of Mechanical Design_ , 144(2), 2022.
  * Oh et al. (2018) Oh, S., Jung, Y., Lee, I., and Kang, N.  Design automation by integrating generative adversarial networks and topology optimization.  In _International Design Engineering Technical Conferences and Computers and Information in Engineering Conference_ , volume 51753, pp. V02AT03A008. American Society of Mechanical Engineers, 2018.
  * Oh et al. (2019) Oh, S., Jung, Y., Kim, S., Lee, I., and Kang, N.  Deep generative design: Integration of topology optimization and generative models.  _Journal of Mechanical Design_ , 141(11), 09 2019.  ISSN 1050-0472.  doi: 10.1115/1.4044229.  URL https://doi.org/10.1115/1.4044229.  111405\.
  * Olhofer et al. (2014) Olhofer, M., Oñate, E., Oliver, J., Huerta, A., and Aulig, N.  Topology optimization by predicting sensitivities based on local state features.  Technical report, 2014.  URL https://www.researchgate.net/publication/265593998.
  * Park et al. (2019) Park, J. J., Florence, P., Straub, J., Newcombe, R., and Lovegrove, S.  Deepsdf: Learning continuous signed distance functions for shape representation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * Parrott et al. (2022) Parrott, C., Abueidda, D., and James, K. A.  Multi-head self-attention gans for multiphysics topology optimization.  In _AIAA AVIATION 2022 Forum_ , pp. 3726, 2022.
  * Perez et al. (2017) Perez, E., Strub, F., De Vries, H., Dumoulin, V., and Courville, A.  Film: Visual reasoning with a general conditioning layer.  _arXiv preprint arXiv:1709.07871_ , 2017.
  * Raffel et al. (2020) Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _The Journal of Machine Learning Research_ , 21(1):5485–5551, 2020.
  * Ramesh et al. (2021) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., and Sutskever, I.  Zero-shot text-to-image generation.  In _International Conference on Machine Learning_ , pp. 8821–8831. PMLR, 2021.
  * Ramesh et al. (2022) Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M.  Hierarchical text-conditional image generation with clip latents.  _arXiv preprint arXiv:2204.06125_ , 2022.
  * Rawat & Shen (2019a) Rawat, S. and Shen, M. H.  A novel topology optimization approach using conditional deep learning.  _CoRR_ , abs/1901.04859, 2019a.  URL http://arxiv.org/abs/1901.04859.
  * Rawat & Shen (2018) Rawat, S. and Shen, M. H. H.  A novel topology design approach using an integrated deep learning network architecture.  _Preprint_ , 8 2018.  URL http://arxiv.org/abs/1808.02334.
  * Rawat & Shen (2019b) Rawat, S. and Shen, M. H. H.  A novel topology optimization approach using conditional deep learning.  _Preprint_ , 1 2019b.  URL http://arxiv.org/abs/1901.04859.
  * Regenwetter et al. (2022) Regenwetter, L., Nobari, A. H., and Ahmed, F.  Deep generative models in engineering design: A review.  _Journal of Mechanical Design_ , 144(7):071704, 2022.
  * Rombach et al. (2021) Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B.  High-resolution image synthesis with latent diffusion models.  _arXiv preprint arXiv:2112.10752_ , 2021.
  * Rozvany et al. (1992) Rozvany, G. I. N., Zhou, M., and Birker, T.  Generalized shape optimization without homogenization.  _Structural optimization_ , 4:250–252, 1992.
  * Sabiston & Kim (2020) Sabiston, G. and Kim, I. Y.  3d topology optimization for cost and time minimization in additive manufacturing.  _Structural and Multidisciplinary Optimization_ , 61(2):731–748, 2020.
  * Sasaki & Igarashi (2019) Sasaki, H. and Igarashi, H.  Topology optimization accelerated by deep learning.  _IEEE Transactions on Magnetics_ , 55(6):1–5, 6 2019.  ISSN 0018-9464.  doi: 10.1109/TMAG.2019.2901906.  URL https://ieeexplore.ieee.org/document/8673771/.
  * Sharpe & Seepersad (2019) Sharpe, C. and Seepersad, C. C.  Topology design with conditional generative adversarial networks.  Volume 2A: 45th Design Automation Conference, 08 2019.  doi: 10.1115/DETC2019-97833.  URL https://doi.org/10.1115/DETC2019-97833.  V02AT03A062.
  * Sigmund & Maute (2013) Sigmund, O. and Maute, K.  Topology optimization approaches: A comparative review.  _Structural and Multidisciplinary Optimization_ , 48(6):1031–1055, 2013.  ISSN 1615-147X.  doi: 10.1007/s00158-013-0978-6.
  * Sitzmann et al. (2020) Sitzmann, V., Martel, J. N. P., Bergman, A. W., Lindell, D. B., and Wetzstein, G.  Implicit neural representations with periodic activation functions, 2020.
  * Skorokhodov et al. (2021) Skorokhodov, I., Ignatyev, S., and Elhoseiny, M.  Adversarial generation of continuous images, 2021.
  * Song et al. (2023) Song, B., Zhou, R., and Ahmed, F.  Multi-modal machine learning in engineering design: A review and future directions.  _arXiv preprint arXiv:2302.10909_ , 2023.
  * Sosnovik & Oseledets (2017) Sosnovik, I. and Oseledets, I.  Neural networks for topology optimization.  _Preprint_ , 9 2017.  URL http://arxiv.org/abs/1709.09578.
  * Sun & Ma (2020) Sun, H. and Ma, L.  Generative design by using exploration approaches of reinforcement learning in density-based structural topology optimization.  _Designs_ , 4(2):10, 5 2020.  ISSN 2411-9660.  doi: 10.3390/designs4020010.  URL https://www.mdpi.com/2411-9660/4/2/10.
  * Tancik et al. (2020) Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J. T., and Ng, R.  Fourier features let networks learn high frequency functions in low dimensional domains, 2020.
  * Ulu et al. (2016) Ulu, E., Zhang, R., and Kara, L. B.  A data-driven investigation and estimation of optimal topologies under variable loading configurations.  _Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization_, 4(2):61–72, 3 2016.  ISSN 2168-1163.  doi: 10.1080/21681163.2015.1030775.  URL http://www.tandfonline.com/doi/full/10.1080/21681163.2015.1030775.
  * Wang et al. (2021a) Wang, C., Zhao, Z., Zhou, M., Sigmund, O., and Zhang, X. S.  A comprehensive review of educational articles on structural and multidisciplinary optimization.  _Structural and Multidisciplinary Optimization_ , 64(5):2827–2880, Nov 2021a.  ISSN 1615-1488.  doi: 10.1007/s00158-021-03050-7.  URL https://doi.org/10.1007/s00158-021-03050-7.
  * Wang et al. (2021b) Wang, C., Zhao, Z., Zhou, M., Sigmund, O., and Zhang, X. S.  A comprehensive review of educational articles on structural and multidisciplinary optimization.  _Structural and Multidisciplinary Optimization_ , 64(5):2827–2880, 11 2021b.  ISSN 1615-147X.  doi: 10.1007/s00158-021-03050-7.
  * Wang et al. (2021c) Wang, D., Xiang, C., Pan, Y., Chen, A., Zhou, X., and Zhang, Y.  A deep convolutional neural network for topology optimization with perceptible generalization ability.  _Engineering Optimization_ , 54(6):973–988, 2021c.  doi: 10.1080/0305215X.2021.1902998.  URL https://doi.org/10.1080/0305215X.2021.1902998.
  * Woldseth et al. (2022a) Woldseth, R. V., Aage, N., Bærentzen, J. A., and Sigmund, O.  On the use of artificial neural networks in topology optimisation.  _Structural and Multidisciplinary Optimization_ , 65(10):294, 2022a.
  * Woldseth et al. (2022b) Woldseth, R. V., Aage, N., Bærentzen, J. A., and Sigmund, O.  On the use of artificial neural networks in topology optimisation.  _Structural and Multidisciplinary Optimization_ , 65(10), October 2022b.  ISSN 1615-1488.  doi: 10.1007/s00158-022-03347-1.  URL http://dx.doi.org/10.1007/s00158-022-03347-1.
  * Wu et al. (2021) Wu, R., Xiao, C., and Zheng, C.  Deepcad: A deep generative network for computer-aided design models, 2021.
  * Xie et al. (2022) Xie, Y., Takikawa, T., Saito, S., Litany, O., Yan, S., Khan, N., Tombari, F., Tompkin, J., Sitzmann, V., and Sridhar, S.  Neural fields in visual computing and beyond, 2022.
  * Xue et al. (2021) Xue, L., Liu, J., Wen, G., and Wang, H.  Efficient, high-resolution topology optimization method based on convolutional neural networks.  _Frontiers of Mechanical Engineering_ , 16(1):80–96, 3 2021.  ISSN 2095-0233.  doi: 10.1007/s11465-020-0614-2.  URL http://link.springer.com/10.1007/s11465-020-0614-2.
  * Yildiz et al. (2003) Yildiz, A., Öztürk, N., Kaya, N., and Öztürk, F.  Integrated optimal topology design and shape optimization using neural networks.  _Structural and Multidisciplinary Optimization_ , 25(4):251–260, 10 2003.  ISSN 1615-147X.  doi: 10.1007/s00158-003-0300-0.  URL http://link.springer.com/10.1007/s00158-003-0300-0.
  * Yoo et al. (2021) Yoo, S., Lee, S., Kim, S., Hwang, K. H., Park, J. H., and Kang, N.  Integrating deep learning into cad/cae system: generative design and evaluation of 3d conceptual wheel.  _Structural and Multidisciplinary Optimization_ , 64(4):2725–2747, 10 2021.  ISSN 1615-147X.  doi: 10.1007/s00158-021-02953-9.  URL https://link.springer.com/10.1007/s00158-021-02953-9.
  * You et al. (2023) You, T., Kim, M., Kim, J., and Han, B.  Generative neural fields by mixtures of neural implicit functions, 2023.
  * Yu et al. (2022) Yu, S., Tack, J., Mo, S., Kim, H., Kim, J., Ha, J.-W., and Shin, J.  Generating videos with dynamics-aware implicit generative adversarial networks, 2022.
  * Yu et al. (2018) Yu, Y., Hur, T., Jung, J., and Jang, I. G.  Deep learning for determining a near-optimal topological design without any iteration.  _Structural and Multidisciplinary Optimization_ , 59(3):787–799, oct 2018.  doi: 10.1007/s00158-018-2101-5.  URL https://doi.org/10.1007%2Fs00158-018-2101-5.
  * Yu et al. (2019) Yu, Y., Hur, T., Jung, J., and Jang, I. G.  Deep learning for determining a near-optimal topological design without any iteration.  _Structural and Multidisciplinary Optimization_ , 59(3):787–799, 2019.
  * Zehnder et al. (2021) Zehnder, J., Li, Y., Coros, S., and Thomaszewski, B.  Ntopo: Mesh-free topology optimization using implicit neural representations.  In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), _Advances in Neural Information Processing Systems_ , volume 34, pp. 10368–10381. Curran Associates, Inc., 2021.  URL https://proceedings.neurips.cc/paper_files/paper/2021/file/55d99a37b2e1badba7c8df4ccd506a88-Paper.pdf.

##  Appendix A Related Work

In this section, we will discuss some of the related works in literature that
work towards similar goals to our work.

The recent successes of Deep Learning (DL) in the field of vision have sparked
significant interest in adapting these techniques to engineering applications.
Specifically, DL methods have been applied to direct design (Abueidda et al.,
2020; Ates & Gorguluarslan, 2021; Behzadi & Ilieş, 2021b; Ma & Zeng, 2020; Ulu
et al., 2016), accelerating the optimization process (Banga et al., 2018; Joo
et al., 2021; Kallioras et al., 2020; Sosnovik & Oseledets, 2017; Xue et al.,
2021), enhancing shape optimization post-processing (Hertlein et al., 2021;
Yildiz et al., 2003), achieving super-resolution (Elingaard et al., 2022;
Napier et al., 2020; Yoo et al., 2021), conducting sensitivity analysis (Aulig
& Olhofer, 2013; Olhofer et al., 2014; Barmada et al., 2021; Sasaki &
Igarashi, 2019), generating 3d topologies (Kench & Cooper, 2021; Sabiston &
Kim, 2020; Behzadi & Ilieş, 2021b), among others (Chen et al., 2022;
Chandrasekhar et al., 2022; Chandrasekhar & Suresh, 2021b; Deng & To, 2021).
Generative Models, in particular, have been highlighted for their capability
to increase design diversity in engineering design (Ahmed et al., 2016; Nobari
et al., 2021b, a, 2022). Within Topology Optimization (TO), efforts have been
concentrated on enhancing diversity through data-driven approaches (Jiang et
al., 2021; Rawat & Shen, 2018, 2019b; Keshavarzzadeh et al., 2021; Sun & Ma,
2020). Additionally, Generative Models are being utilized for TO problems,
conditioning on constraints such as loads, boundary conditions, and volume
fraction for structural cases, to directly generate topologies from a training
dataset of optimized topologies, apply superresolution methods to increase
fidelity (Yu et al., 2019; Li et al., 2019), and use filtering and iterative
design methods to improve quality and diversity (Oh et al., 2018, 2019;
Berthelot et al., 2017; Fujita et al., 2021). Proposals for methods addressing
3D topologies have also been put forward (Behzadi & Ilieş, 2021a;
Keshavarzzadeh et al., 2021). Recent advancements include GAN (Nie et al.,
2021b) and DDPM-based approaches (Mazé & Ahmed, 2023), which have shown
promise in modeling the TO problem by conditioning on constraints and
incorporating physical information. A detailed review and critique of this
evolving field is provided by (Woldseth et al., 2022a).

#### Deep Learning For Topology Optimization

The limitations associated with conventional optimization have motivated a
significant body of work around deep learning approaches for topology
optimization (TO). Most relevant to our work are other methods that perform
the task in an end-to-end manner, meaning they take constraints and boundary
conditions as input and produce near-optimal topologies to minimize compliance
(Oh et al., 2019; Sharpe & Seepersad, 2019; Chandrasekhar & Suresh, 2021a;
Parrott et al., 2022). For example, Yu et al. (2018) propose an auto-encoding
GAN approach that uses an autoencoder for topology generation and a GAN for
super-resolution which is applied simultaneously. Rawat & Shen (2019a), Li et
al. (2019) take a similar approach but use GANs for both initial generation
and super-resolution. This is while Sharpe & Seepersad (2019), Nie et al.
(2021a), and Behzadi & Ilieş (2021) propose directly generating topologies
using conditional GANs and Wang et al. (2021c) introduce U-Net-based
architectures for TO. More recently, researchers have looked at using implicit
neural fields to generate topologies. Specifically, Hu et al. (2024) use
implicit neural representations to produce topologies in their approach called
IF-TONIR. In their approach, however, the authors use stress and strain fields
for their representations of boundary conditions which rely on CNNs and
somewhat reduce the generalizability of the implicit neural fields’
versatility.

#### Diffusion-based Topology Optimization

New methods that utilize diffusion models have recently achieved significant
performance improvements over the aforementioned methods (Mazé & Ahmed, 2023;
Giannone & Ahmed, 2023; Giannone et al., 2023). Mazé & Ahmed (2023) were the
first to do so and demonstrated that diffusion models are significantly better
than GANs at optimal topology generation. They also showed that introducing
guidance based on a regression model for predicting compliance and a
classifier trained to identify floating material significantly improves the
performance of such models (Mazé & Ahmed, 2023). Like iterative optimization,
however, diffusion models are slow, requiring the models to be run as many as
1000 times to generate a sample. This puts their value into question when the
time savings are not notable compared to the optimizer itself. Giannone et al.
(2023) propose aligning the diffusion models denoising with the optimizer’s
intermediate designs as a way to reduce the number of iterations needed for
the diffusion model and show that they can indeed reduce the number of
sampling steps significantly.

#### Limitations

Despite the upsides of the best-performing models which are diffusion-based,
they lack speed and are all based around CNNs, which means that they treat the
problem as images. Furthermore, the mechanism they use for conditioning based
on boundary conditions is field-based, meaning these methods compute some
physical or energy field to represent the boundary conditions of the problem,
treat these fields as images, and employ CNN-based architectures to handle
them. This method introduces two limitations. These models generate images
instead of density fields, restricting them to a specific problem domain and
resolution (e.g. a square-shaped domain treated as 64x64 images) and making
them unsuitable for different domain shapes and resolutions. Additionally, the
use of image-based fields for boundary conditions further narrows their
generalizability. This approach fails to capture boundary conditions in their
raw form, limiting applicability to problems with boundary conditions defined
for the exact domain they are trained on and excluding problems with irregular
boundary conditions.

##  Appendix B Additional Experiments

Here we will discuss some further experiments and discuss some limitations of
our approach.

###  B.1 Out of Distribution Experiments

Table 6: Quantiative Evaluation on out-of-distribution 64x64 datasets. w/ G: using a classifier and/or regression guidance. FS-SIMP: Few-Steps of optimization. CE: Compliance Error. VFE: Volume Fraction Error. Model | Train Res. | FS-SIMP | CE % Mean | CE % Median | VFE % Mean | VFE % Median
---|---|---|---|---|---|---
out-of-distro |  |  |  |  |  |
NeuralField | 64 | - | 73.24 | 12.81 | 8.12 | 6.87
TopoDiff | 64 | - | 8.57 | 1.14 | 1.14 | 0.97
TopoDiff w/ G | 64 | - | 7.79 | 1.26 | 1.21 | 1.00
TopoDiff | 64 | 5 | 6.20 | 1.07 | 0.93 | 0.55
TopoDiff w/ G | 64 | 5 | 5.44 | 1.05 | 0.89 | 0.55
NITO (ours) | 64 | 5 | 9.33 | 2.37 | 2.22 | 1.32
TopoDiff | 64 | 10 | 2.91 | 0.71 | 0.64 | 0.33
TopoDiff w/ G | 64 | 10 | 2.25 | 0.71 | 0.63 | 0.35
NITO (ours) | 64 | 10 | 6.38 | 1.43 | 1.55 | 0.85

Something that should be looked at when it comes to these models is their
performance generalizability to problems that are very different from the
distribution of data used for training. To do this we test the performance of
different models on an out-of-distribution test set for the 64x64 dataset.
These results are presented in Table 6 for TopoDiff and NITO. As it can be
seen NITO’s performance has significantly deteriorated on out-of-distribution
data. This can be attributed to two matters. The first and most impactful is
the nature of these models. TopoDiff generalizes better when it comes to these
out-of-distribution tests given the fact that the model is generative in
nature. This allows TopoDiff to handle out-of-distribution conditions better.
To understand this we can look at generative models as a sort of retrieval
approach, which allows the models to generate detailed and high-quality
samples by finding the most similar topologies from the training data even
when faced with very different inputs. On the other hand, NITO is a
deterministic model that learns to map specific boundary conditions to near-
optimal density fields. This makes it rather challenging for models like NITO
when it comes to generating topologies for unseen and very different inputs
since the mapping is a deterministic one. This highlights the importance of
future work focusing on transforming our framework to be generative, which
should be possible since many works have shown implicit neural approaches can
be made generative (You et al., 2023). The second potential reason
contributing to TopoDiff’s better performance on out-of-distribution samples
is the use of physics-based fields for conditioning. This is because mapping
stress fields to topologies makes it easier to handle very different boundary
conditions since the stress fields may still be similar to samples in the
dataset. However, this creates the kind of limitation that we discussed
before, as such it is better to focus on making more robust generative schemes
rather than using physical fields.

Figure 5: Visual comparison of samples generated for the out-of-distribution
test. Each row is labeled. Ground truth samples are SIMP-optimized samples.

###  B.2 Further Examination Of Results & Outperforming The Optimizer

Figure 6: Distributions of volume fraction error (Bottom) and Compliance
Error(Top). We see that NITO is capable of out-performing SIMP 19% of the time
as highlighted by yellow in the negative compliance errors (meaning better
than SIMP) on the top histogram. This data is for the 64x64 test set.

In Figure 6, we show the distribution of compliance error and volume fraction
error for NITO on the 64x64 dataset. We see that the majority of volume
fraction error is below 1% and a small number of high error samples skew the
average. Similarly, we see that the majority of the compliance errors for NITO
are below 1% as well. Most notably, we observe that on 19.2% of the samples in
the test set NITO actually outperforms SIMP as indicated by the negative
compliance errors visible on the histogram. This is a rather interesting
outcome where NITO is capable of doing better than the optimizer that the
training data came from. In Figure 21, we visualize a few instances of this
phenomenon. It can be seen that in some problems NITO actually comes up with a
different solution which as it happens out-performs SIMP, while in other
instances the topologies are very similar and NITO has adjusted some of the
finer details and redistributed the material differently to achieve better
performance.

Figure 7: Example topologies where NITO outperforms SIMP. It can be seen that
in some instances NITO finds very different topologies that outperform SIMP
and in some instances, NITO has removed some details and redistributed the
material in a way that has improved performance.

##  Appendix C Visualizations

Here we will provide a set of different visualizations for each test case.

###  C.1 Direct Optimization Visualization

In this section we provide some visualizations, demonstrating the effect of
direct optimization on neural field-generated density fields. The main
objective here is to showcase the importance and effectiveness of a few steps
of direct optimization on the generated samples which completes the NITO
framework. As can be seen in Figures 8 and 9, neural fields tend to have some
smoothing and averaging in areas of high detail in the topology, which makes
the baseline performance of neural fields worse. With only a small number of
direct optimization iterations, however, we see that NITO can resolve the
complex details effectively, showcasing why direct optimization is a crucial
aspect of our framework.

In contrast, when we look at the effect of direct optimization on TopoDiff
(Figures 10 and 11) we see that in cases where the details predicted by
TopoDiff are accurate, the optimizer does not provide much benefit, and when
TopoDiff has not generated the correct details the generated topology is far
from optimal which means that few steps of optimization does not provide
significant benefits.

Figure 8: Visualization of the effect of direct optimization on samples
generated by neural fields. The first column of each set of images shows the
ground truth, the next column shows the raw predictions from neural fields,
and the two columns that follow show the effects of 5 and 10 steps of direct
optimization on the samples respectively. These samples are from the 64x64
test set. Figure 9: Visualization of the effect of direct optimization on
samples generated by neural fields. The first column of each set of images
shows the ground truth, the next column shows the raw predictions from neural
fields, and the two columns that follow show the effects of 5 and 10 steps of
direct optimization on the samples respectively. These samples are from the
256x256 test set. Figure 10: Visualization of the effect of direct
optimization on samples generated by TopoDiff. The first column of each set of
images shows the ground truth, the next column shows the samples generated by
TopoDiff, and the two columns that follow show the effects of 5 and 10 steps
of direct optimization on the samples respectively. These samples are from the
64x64 test set. Figure 11: Visualization of the effect of direct optimization
on samples generated by TopoDiff. The first column of each set of images shows
the ground truth, the next column shows the samples generated by TopoDiff, and
the two columns that follow show the effects of 5 and 10 steps of direct
optimization on the samples respectively. These samples are from the 256x256
test set.

###  C.2 Generated Samples Visualizations

In this section, we provide visualizations of samples predicted by different
configurations of our model on different problems.

Figure 12: Ground truth images from the 64x64 SIMP datasets. Images that
follow visualize NITO generated samples for the same problems. Figure 13: NITO
generated topologies using a model trained on 64x64. Tested on the 64x64 data.
Figure 14: NITO generated topologies using a model trained on 256x256. Tested
on the 64x64 data. Figure 15: NITO generated topologies using a model trained
on both 64x64 and 256x256. Tested on the 64x64 data. Figure 16: Ground truth
images from the 256x256 SIMP datasets. Images that follow visualize NITO
generated samples for the same problems. Figure 17: NITO generated topologies
using a model trained on 256x256. Tested on the 256x256 data. Figure 18: NITO
generated topologies using a model trained on 64x64. Tested on the 256x256
data. Figure 19: NITO generated topologies using a model trained on both 64x64
and 256x256. Tested on the 256x256 data.

##  Appendix D Implementation Details

Here we will briefly discuss some implementation details of the neural fields.
First, the neural fields in our implementation are based on simple multi-layer
preceptrons (MLP), and amongst the different variations of implicit neural
fields, we based our implementation on SIREN (Sitzmann et al., 2020) layers
which apply sine activation functions to the output of each layer.
Furthermore, it has been shown that implicit neural fields tend to ignore
higher-frequency features and Tancik et al. (2020) have shown that
transforming the spatial coordinates using Fourier feature mapping can be
effective at mitigating this issue. As such we also apply Fourier feature
mapping to the input coordinates of the model.

Another important thing to note at this stage is the mechanism we use to
incorporate the conditioning. Neural fields possess the capability to
assimilate prior behaviors and exhibit generalization to novel fields through
conditioning on a latent variable $\mathbf{C}$, which encapsulates the
characteristics of a field. Specifically, this includes the boundary
conditions and the volume ratio in our problem. Perez et al. (2017) propose
Feature-wise Linear Modulation (FiLM) which applies conditioning by modulating
the outputs of the different layers of the model. To do this the conditioning
mechanism includes two networks $\alpha(\mathbf{C})$ and $\beta(\mathbf{C})$,
which predict a multiplicative and additive adjustment to the outputs of each
layer. Our implementation is inspired by similar mechanisms built around
normalization layers such as adaptive instance normalization (AdaIN) (Huang &
Belongie, 2017). Instead of modulating the outputs of each layer we apply
layer normalization and modulate the scale and shift of the layer norm for
each feature of the layer output. Putting this all together our neural field
can be described as:

$f_{\theta}(\mathbf{x},\mathbf{C})=f^{(L)}\circ f^{(L-1)}\circ\cdots\circ
f^{(0)}(\mathbf{x},\mathbf{C}))$ (5)

where $f^{(i)}$ for $i\in\\{1,2,...,L-1\\}$ indicates the function applied at
each layer of the neural field except the first and last layer. Each layer in
the model takes a hidden state $h^{(i)}$ as input and performs a linear
transformation and layer normalization with modulation based on the condition
vector:

$\begin{split}f^{i}(h^{i},\mathbf{C})=\sin(LN_{1,0}(W^{i}h^{i}+b^{i})\times\alpha(\mathbf{C})+\beta(\mathbf{C})),\end{split}$
(6)

where $LN_{1,0}$ is layer normalization with scale 1 and shift zero and
$\alpha$ and $\beta$ are fully-connected (FC) layers that determine the
feature-wise scale and shift based on the condition inputs. The first layer
has the same function however the input coordinates are transformed by Fourier
feature mapping. The final layer simply lacks the conditioning modulation
layer normalization and $\sin$ activation function, instead, the activation
function is $sigmoid$. As evident these neural fields are easily applicable to
any domain shape as long as that information is contained in the condition
latent representation $\mathbf{C}$. This is because the input to the neural
field is simply coordinates, which allows this kind of density field
prediction to be sampled arbitrarily in space, therefore, making this kind of
modeling truly generalizable at least in theory to different domains, so long
as the condition representation $\mathbf{C}$ is also generalizable. This is
the topic is discussed in the main body of the paper. In our implementation,
we use 8 layers of size 1024 for the neural fields and use 4 layers of size
256 for the point cloud models of which there are 3 one for force and one for
constraints in each x and y direction. Further detail can be found in our
code.

###  D.1 Training Details

The training of the conditional neural implicit model is performed in 3
stages. In the first stage, we perform training by sampling points of a 16x16
grid. That is to say that we sample points in space that each belong to one of
the 16x16 divisions in space for the unit square domain. Furthermore, when
sampling, if the material exists in a given sampling grid we randomly choose a
point with material rather than picking points that are void. The first stage
of training is carried out for 20 epochs. Followed by 20 epochs of sampling on
a 32x32 grid and finally we sample on a 32x32 grid but rather than preferring
material points we sample completely randomly. The final stage of training is
10 epochs. We use AdamW optimizer with a starting learning rate of $10^{-4}$
which is reduced on a cosine annealing schedule to be reduced at the end of
each epoch to reach $5\times 10^{-6}$ at the final epoch. The learning rate is
stepped at the end of each epoch.

##  Appendix E Dataset & Optimizer Details

Our experiments use a dataset of SIMP-optimized topologies in a unit square
domain with a size of 64x64 and 256x256. For each topology in the dataset
information about the loading condition, boundary condition, and volume
fraction, are included. Furthermore, we include the stress and strain energy
fields. Our dataset is similar to the one proposed by Mazé & Ahmed (2023),
however, we noted that the dataset used in prior works has been generated
using an older version of SIMP, namely ToPy (Hunter et al., 2017).

This solver, although a robust implementation of SIMP, does not implement the
latest improvements to the SIMP algorithm and uses a slower solver, which
causes two issues. Firstly, the topologies that are used in the dataset
proposed by Mazé & Ahmed (2023) are lower-performing topologies in comparison
to what the latest solvers produce, hence overselling the performance of these
models. Secondly, the SIMP method itself that prior studies compared their
inference time with did not use the fastest solvers, making those comparisons
also inaccurate. To ensure that this is not the case in our studies we
implement the SIMP optimizer from scratch in Python (The code for which will
be publicly available), which performs the optimization using the latest and
fastest implementation of the SIMP algorithm as far as the authors are aware
(Wang et al., 2021b).

As such we re-create the 64x64 dataset proposed by Mazé & Ahmed (2023) using
our solvers and find that the resulting topologies using our method are
significantly better performing in comparison to the prior datasets used by
many other works of research (Giannone et al., 2023; Giannone & Ahmed, 2023;
Mazé & Ahmed, 2023; Nie et al., 2021b). Given this, it is safe to assume that
the performance of the models in prior studies that compare to the inferior
dataset may have been overestimated. However, to allow for a fair comparison
we retrain the best-performing model in the literature TopoDiff (Mazé & Ahmed,
2023) on our new dataset and rerun their experiments on this new dataset.
However, we report the performance of other models as the original authors
measured them.

The 64x64 dataset includes 30,000 training samples and 1,800 test samples
which we use for testing our models. Additionally, we include a dataset of
1,000 out of distribution samples for out of distribution testing. The 256x256
dataset includes 60,000 training samples and 1,800 samples for testing. The
figures that follow visualize some of the training samples for each dataset.
In the following figures, we include figures showing samples from both
datasets.

Figure 20: Random samples from the 64x64 dataset. Green arrows show the
locations and directions of the loads applied in each problem. Blue triangles
indicate points that are constrained in x and red triangles indicate points
that are constrained in y. Figure 21: Random samples from the 256x256 dataset.
Green arrows show the locations and directions of the loads applied in each
problem. Blue triangles indicate points that are constrained in x and red
triangles indicate points that are constrained in y.