# RetinaGS: Scalable Training for Dense Scene Rendering with Billion-Scale 3D
Gaussians

Bingling Li Shengyi Chen ††footnotemark:  Luchao Wang Kaimin Liao
Sijie Yan Yuanjun Xiong

MThreads AI
Equal contribution.

###### Abstract

In this work, we explore the possibility of training high-parameter 3D
Gaussian splatting (3DGS) models on large-scale, high-resolution datasets. We
design a general model parallel training method for 3DGS, named RetinaGS,
which uses a proper rendering equation and can be applied to any scene and
arbitrary distribution of Gaussian primitives. It enables us to explore the
scaling behavior of 3DGS in terms of primitive numbers and training
resolutions that were difficult to explore before and surpass previous state-
of-the-art reconstruction quality. We observe a clear positive trend of
increasing visual quality when increasing primitive numbers with our method.
We also demonstrate the first attempt at training a 3DGS model with more than
one billion primitives on the full MatrixCity dataset that attains a promising
visual quality.

##  1 Introduction

Figure 1: Left: Different sizes of datasets require varying levels of
computational power and numbers of 3DGS Primitives. Larger and higher-
resolution datasets can no longer be trained using just a single GPU, which
limits the pursuit of scale and fidelity in 3DGS reconstruction. Right: The
billion-level model bring better visual experience than million-level model on
MatrixCity-ALL dataset, which is trained via our distributed modeling with 64
GPUs.

3D scene reconstruction with Gaussian Splatting (GS) [11] has drastically
improved rendering quality and rendering speed over previous neural 3D
representation [21, 42, 5]. However, this success has been largely limited to
reconstructing scenes with limited image or video resolution (typically
$<=1600$ pixels wide), data volume, and view distance. Viewing the scene at
high resolution or close range remains an unsolved challenge. To achieve
imaging effects flawless to the human retina, which we refer to as the goal of
retina-level reconstruction, we would desire to train the GS models with
higher spatial resolution, larger datasets, and more varying viewing
distances.

However, all these demands lead to more compute and memory costs. Fig. 1
illustrates the compute cost and memory footprint when training GS models
across different rendering resolutions and different data volume on several
datasets. We can observe that high-resolution training indeed improves the
rendered views’ visual quality. It also inevitably requires more splats, which
means higher parameter counts. As the total compute of training grows roughly
by the product of the number of pixels per view and the number of splats,
training time on a single GPU will quickly become unacceptable. The increased
splat numbers also lead to a growing memory footprint that could become
infeasible for even the best single modern GPU. To enable effective scaling,
we need a method for distributed GS training.

In this paper, we present a method to achieve efficient distributed training
of GS models while retaining equivalence to the existing single GPU training
scheme. We start by dividing the model space into a set of convex subspaces
and identify one subset of splats for each subspace so that all the subsets
collectively form an overlapping cover of the set of all splats. We then show
that, with a proper subset assignment strategy, each ray’s original iterative
alpha-blending process can be transformed into a hierarchical form, in which
we can first compute partial color and alpha values for each subset and then
obtain the rendered pixel color value by orderly merging these partial values.

We derive our training method by exploiting this property and designating one
worker for each subset. In the forward process, all workers compute subset-
level color and alpha values for all rays in parallel. The partial values,
sufficient to compute rendered pixel values, can then be merged through cross-
worker communication with a minimal message size. After computing the
reconstruction loss and its gradients w.r.t. the partial values, we can
distribute the corresponding gradients to each worker and run the backward
process in parallel. We further improve training efficiency by utilizing KD-
tree to produce subspaces that induce subsets with more even cardinality. This
method allows us to efficiently train arbitrarily large GS models on multiple
GPUs without any assumption on the reconstructed scene.

We validate the training method on multiple 3D reconstruction datasets with
high training resolutions. The trained GS models outperform 3DGS[11] on Mip-
NeRF360[3], Mega-NeRF [32], ScanNet++[40] and MatrixCity[17] datasets. Through
scaling study, we observe a clear tendency to improve visual quality when the
GS models grow in splat numbers. Finally, we demonstrate to our knowledge the
first attempt at training GS models with billion-scale primitives on city-
level dataset with 140k images, resulting in unprecedented visual experience
using million-scale primitives.

##  2 Related Work

##### Distributed Neural Radiance Fields.

NeRF [21] has revolutionized 3D scene reconstruction and novel viewpoint
generation with its photorealistic rendering capabilities, sparking a series
of subsequent innovations focused on enhancing various aspects of the
technology. These advancements strive to improve rendering quality [42, 4, 2,
3], optimize training speed [29, 8, 5, 22], increase memory efficiency [24,
15, 25] and expand the scale of reconstructed scenes [44, 37, 30, 32, 38, 43,
28, 36]. While these fields might benefit from larger models or enhanced
computational power, it is primarily the pursuit of scaling scene sizes that
has driven researchers to explore ways to speed up or scale up NeRF models.
Switch-NeRF [44] utilizes a Mixture of Experts (MoE) to increase the capacity
of NeRF models, enabling them to represent urban-scale scenes effectively.
Bungee-NeRF [37] employs a hierarchical assembly of submodels, granting the
model extensive multiscale representational capabilities. Recognizing the
limitations imposed by GPU computational power and memory on model scale and
training velocity, some researchers have begun deconstructing models into
smaller components, adopting distributed approaches to address these
challenges. Block-NeRF [30] segments cities into multiple overlapping blocks,
each represented by its own NeRF model, and uses neural networks to fuse the
outputs of multiple NeRF models in image space, achieving seamless visual
results. Mega-NeRF [32] introduces a simple geometric clustering algorithm and
partitions training pixels into various NeRF submodules, which can then be
trained in parallel.

##### Distributed Point-Based Representation.

NeRF utilizes a volumetric rendering approach that inherently limits their
inference speed, which makes it challenging to achieve real-time performance.
Several studies [41, 24, 39, 31] have focused on optimizing inference speeds,
yet achieving both real-time performance and high-quality rendering remains
elusive. In contrast, the 3D Gaussian Splatting (3DGS) [11] method, which
employs a point-based scene representation, achieves state-of-the-art
rendering quality and speed and has significant advantages in training speed.
However, the "shallow and wide" structure of the 3DGS model and its explicit
representation lead to a larger parameter count and increased training memory
footprint. VastGaussian [18] is the first attempt to large-scale scene
reconstruction with 3DGS. It divides the ground plane into a series of 2D
cells with one splat subset for each cell. Unlike our method, all subsets are
mutually exclusive and considered as independent 3DGS models in training. To
mitigate the deviation from the original 3DGS rendering equation at subspace
boundaries, VastGaussian introduces additional nearby cameras and Gaussian
primitives to assist in training at the cost of increased compute and memory.
CityGaussian [20] uses a similar partitioning strategy but trains an
additional coarse GS model as the baseline to regularize submodels. However,
artifacts caused by deviation from the proper rendering equation could still
intensify at certain view angles and overcome the mitigations. As a result,
these approaches mostly assume bird-eye-view urban scenarios due to their
simplicity in view angles and splat distribution. Our method is based on an
equivalent form of the 3DGS rendering equation. Thus it can be applied to any
scene with arbitrary splat distribution.

##### Distributed Deep Learning.

Our work is also related to distributed deep learning, which aims to scale up
the training system for deep neural network models [6, 1, 16]. Early
approaches revolve around training multi-GPU convolutional neural networks
(CNNs) [13] on image [10] and video [34] datasets, with data parallelism being
highly efficient thanks to CNNs’ high compute to bandwidth ratio. Recent
development of large language models presented new challenges in distributing
model parameters and larger clusters, resulting in dedicated model parallelism
[12] approaches for Transformers [33] models, such as tensor-parallelism [27],
pipeline parallelism [9], or hybrid parallelism [27, 19, 14]. Fully sharded
data parallelism [7], a type of redundancy-free data parallelism [23], also
works great with LLM training. Our method is best described as model
parallelism as it partitions GS model parameters onto multiple workers to
distribute workload and incurs communication cost only for Gaussian on the
subspace boundaries. This is well suited to GS models’ relatively low compute-
to-parameter ratio111also known as operational intensity [35]., which makes
data parallelism less efficient.

##  3 Approach

###  3.1 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) utilizes a series of anisotropic 3D Gaussian
primitives to explicitly characterize scenes. Each Gaussian primitive, known
as a splat, is defined by its central position $\mu\in\mathbb{R}^{3}$, and a
covariance matrix $\Sigma\in\mathbb{R}^{3\times 3}$. A splat’s influence at
any given point $x$ within the scene’s world coordinate system is attenuated
by the Gaussian function

$G(x)=e^{-\frac{1}{2}(x-\mu)^{T}\Sigma^{-1}(x-\mu)}.$ (1)

In practice, the function is truncated to save computation. Each splat also
carries an opacity $\alpha\in\mathbb{R}$. Its color attributes
$F\in\mathbb{R}^{C}$ are expressed through spherical harmonics (SH)
$c\in\mathbb{R}^{3}$ to allow view-dependent textures. A view is rendered
through rasterization the Gaussian primitives onto a 2D imaging plane, during
which the 3D Gaussians are projected to 2D Gaussians $G^{\prime}(x^{\prime})$
through Jacobian linearization as described in [45]. As a result, the
influence weight of each 3D primitive on a given ray $l$ needs to be computed
through path integration, whereas a 2D primitive only requires a single
sampling, that is, $g(l)=G^{\prime}(x^{\prime})$, where $x^{\prime}$ is the
intersection point of the ray $l$ with the 2D imaging plane. By employing
alpha-blending, these primitives are rendered with the following rendering
equation

$C(l)=\sum_{g_{i}\in{N_{l}}}c_{i}\sigma_{i}\prod_{g_{j}\in
N_{l},~{}j<i}(1-\sigma_{j}),\quad\sigma_{i}=\alpha_{i}g_{i}(l),$ (2)

where ${N_{l}}$ denotes the set of Gaussian primitives that contribute to ray
$l$, arranged in order of their depth.

###  3.2 Distributed Training of 3DGS

Given the set of all splats $N=\\{g_{i}\\}$ and $K$ workers
$\\{w_{1},\ldots,w_{K}\\}$, we aim to devise a distributed training method
with minimal communication overhead that still conforms to the rendering
equation in Eq. 2. We first divide the scene space into a set of convex
subspaces $S_{1},\ldots,S_{K}$. One worker is expected to only work on a
subspace and incur minimal communication needs. To achieve this, we generate
$K$ subsets $N_{1},\ldots,N_{K}$ and $N_{1}\cup N_{2}\cup...\cup N_{K}=N$,
which are allowed to overlap. Below we show how to create $N_{k}$ and
manipulate Eq. 2 for distributed computation for the workers.

Given a ray $l$ and a subspace $S_{k}$, we can always obtain a subset of $N$
as $N_{lk}$ which denotes all splats that, when projected to a truncated 2D
Gaussian according to $l$, intersects with $l$ within $S_{k}$. Note since the
Gaussians are truncated, not all splats will intersect with $l$. Since
$N_{lk}$ is ray dependent, we define $N^{*}_{k}=\cup_{l}N_{lk}$ as the union
of $N_{lk}$ for all possible rays. Going through every possible ray to obtain
$N^{*}_{k}$ is infeasible. However, we know that any intersection point must
reside on its corresponding 3D ellipse. So, we can instead define $N_{k}$ as
the set of splats whose corresponding 3D ellipsoid intersects with $S_{k}$.
Then it is obvious that $N^{*}_{k}\subseteq N_{k}$ and $N_{k}$ is not
dependent on any specific ray. To render the color for a given ray, We can
first calculate the partial color $C_{k}(l)$ and the partial opacity
$T_{k}(l)$ on $N_{k}$ as

$\displaystyle C_{k}(l)$ $\displaystyle=\sum_{g_{i}\in
N_{k}}c_{i}\sigma_{i}\mathbbm{1}(g_{i}\in N_{lk})\prod_{g_{j}\in
N_{k},~{}j<i}(1-\sigma_{j}\mathbbm{1}(g_{j}\in N_{lk})),$
$\displaystyle=\sum_{g_{i}\in N_{lk}}c_{i}\sigma_{i}\prod_{g_{j}\in
N_{lk},~{}j<i}(1-\sigma_{j})$ (3) $\displaystyle T_{k}(l)$
$\displaystyle=\prod_{g_{i}\in{N}_{k}}(1-\sigma_{i}\mathbbm{1}(j\in N_{lk}))$
$\displaystyle=\prod_{g_{i}\in{N}_{lk}}(1-\sigma_{i}),$ (4)

where $\sigma_{i}=\alpha_{i}g^{k}_{i}(l)$ and $\mathbbm{1}(\cdot)$ denotes the
indicator function which equals to 1 when the condition specified is true and
otherwise equals to 0. Note splats in $N_{k}$ are ordered by their distance to
the origin of $l$ to attain the index $i,~{}j$ in Eq. 3 and Eq. 4. The
introduction of the indicator function allows the accumulation to be carried
on the entire $N_{k}$ for any ray $l$. The partial values can then be merged
in the order that the ray passes through them, which can be represented as a
permutation $o_{l}$ of size $K$:

$C(l)=\sum_{k\in o_{l}}C_{k}(l)\prod_{m\in o_{l},~{}m<k}T_{m}(l).$ (5)

Substituting Eq. 3 and 4 into Eq. 5, we can observe that Eq. 2 is equivalent
to Eq. 5 for any $l$. The relevant proof process has been placed in the
appendix. Therefore we have transformed the rendering equation of 3DGS into
the independent computation of subset-level partial colors and opacities and
the subsequent merging. All workers can compute their corresponding partial
values in parallel for each subset $N_{k}$ and perform the merge step through
cross-worker communication. The result will be identical to when rendered on a
single worker.

Evaluating the condition of $g_{i}\in N_{lk}$. Since the subspaces
$S_{1},S_{2},\ldots,S_{K}$ seamlessly partition the entire space and each
subspace is convex, it necessitates that all dividing surfaces are planar.
Each subspace $S_{k}$ can be represented by a set of plane constraints:
$S_{k}=\\{x\in\mathbb{R}^{3}:n^{\top}_{km}\cdot x+d_{km}\leq 0,\text{ for all
}m\\}$ , which allows us to transform the indicator function $\mathbbm{1}$ in
Eq 3 into a form more amenable to computation:

$\displaystyle\mathbbm{1}(g_{i}\in N_{lk})=\mathbbm{1}(x_{i}\in
S_{k})=\prod_{m}\mathbbm{1}(n^{\top}_{km}\cdot x_{i}+d_{km}\leq 0),$ (6)

where $x_{i}$ represents the world coordinates of the intersection between the
ray $l$ and the 2D Gaussian primitive $G^{\prime}_{i}$. Expressing the ray $l$
as the equation $l(t)=o+td$, which is defined by the camera center
$o\in\mathbb{R}^{3}$ and the unit ray direction $d\in\mathbb{R}^{3}$, then
$x_{i}$ precisely equals the projection center point $u_{i}$ of $g_{i}$ onto
$l$, which can be represented as: $x_{i}=u_{i}+(d^{\top}\cdot(o-u_{i}))d$.

###  3.3 Distributed Training with Sub-models

To balance each subset’s size and each worker’s workload, we employ a KD tree
to determine the partition of the subspaces $\\{S_{k}\\}$. Initially, we
construct a three-dimensional KD tree with a depth of $L$ using the center
coordinates of Gaussian primitives. The KD tree recursively uses hyperplanes
perpendicular to the X, Y, and Z axes to bisect the space and equally divided
primitives, ultimately resulting in a series of rectangular subspaces
$S_{1},S_{2},\ldots,S_{K}$, where $K=2^{L}$. Then, the corresponding subsets
for each worker can be derived as: $N_{k}=\\{i:n^{\top}_{km}\cdot
u_{i}+d_{km}\leq D_{i},\text{ for all }m\\}$, where $u_{i}$ is the center of
primitives and $D_{i}$ is the truncation threshold for $g_{i}$. In our
implementation, we set $D_{i}$ to be three times the length of the major axis
of the Gaussian ellipsoid. Appendix.Fig 10 provides an intuitive illustration
of how the primitives near the subspace boundary work.

###  3.4 The Complete Training Pipeline

Figure 2: By employing planes generated using KD-Tree, we spatially
partitioned the initial 3DGS model to a set of sub-models. These sub-models
share certain primitives only when these primitives cross boundaries. The
rendering results of sub-models are then merged to form the final rendered
image. After the loss is computed uniformly, the corresponding gradients are
returned to each sub-model to update their primitive parameters.

The actual training pipeline is shown in Figure 2. We denote each subset as a
sub-model and assign it to a separate GPU, while a central manager is
responsible for managing the KD-Tree and the subspaces $\\{S_{n}\\}$. This
manager also handles the parsing of incoming rendering requests and
distributes rendering tasks to the relevant sub-models. The computational
results from all sub-models, $\\{T_{k}\\}$ and $\\{C_{k}\\}$, are sent back to
the central manager. These results can be represented in a 2D map with the
same resolution as the target image, which consumes only minimal communication
bandwidth. The central manager then completes the final rendering based on Eq
5, calculates the loss, and sends the gradient back to each sub-model for
model parameter updates. After a predetermined number of training epochs, we
repeat the partitioning process to accommodate any significant shift of
primitive centers. Note that primitives belonging to intersections of multiple
subsets require gradient synchronization after each training step.
Empirically, we found this could be omitted without significantly affecting
reconstruction quality.

####  3.4.1 Primitive Initialization

In the original 3D Gaussian Splatting (3DGS) approach, initialization and
densification processes involved numerous heuristic strategies and
hyperparameters to ensure that Gaussian primitives were appropriately
positioned. Such strategies made it difficult to control the number of
primitives, impeding our ability to effectively scale the model size and
leverage the advantages of distributed modeling to enhance rendering fidelity.
Furthermore, we found that the densification process does not always promote
an increase in primitives in high-resolution training, which makes it
counterproductive.

In this paper, we adopt a simple yet effective strategy for initialization as
shown in Fig 2 and do not adjust the number afterwards. We performed Multi-
View Stereo (MVS) on all training data to obtain depth estimations for all
training viewpoints, which were then transformed into dense 3D point clouds.
These point clouds could be flexibly sampled and initialized as Gaussian
primitives. This streamlined approach allowed us to control the number of
primitives as desired. It is also easier to balance workers’ workloads thanks
to a predefined number of primitives.

##  4 Experiments

##### Datasets.

Our performance evaluation spanned 4 datasets that comprising indoor and
outdoor scenes. These datasets include all scenes from full-resolution
MipNeRF-360 [3]. We extended our analysis to high-resolution scenarios based
on the ScanNet++ dataset [40], with a focus on scenes labeled 108ec0b806 and
8133208cb6. After distortion correction, two scenes provided 863 and 476 high-
resolution images respectively (8408 pixels wide). Furthermore, we conducted
tests in large-scale environments, including the Residence, Building, and
Rubble datasets from Mega-NeRF [32], as well as the entire MatrixCity Small
City Aerial dataset (referred to as MatrixCity-Aerial, 1920$\times$1080),
consisting of 6,362 images. We also utilized the MatrixCity Small City Dense
Street dataset, sampling 5 angles every meter along the centerline of streets
to gather 135,290 images (1000$\times$1000). From the Small City Dense Street
dataset, we selected a focused test set of 2,480 images (referred to as
MatrixCity-M). By combining all images from Aerial and Dense Street (referred
to as MatrixCity-ALL), we conducted a comprehensive Billion GS level
reconstruction. We followed the official Train/Test splits for Mega-NeRF and
MatrixCity-Aerial, and used every eighth image for testing in other datasets
as recommended by MipNeRF-360. To obtain superior MVS initial points, we reran
Colmap’s sparse reconstruction based on full-resolution images of MipNeRF-360
and ScanNet++ to obtain poses (using official provided poses of Mage-NeRF and
MatrixCity), and subsequently performed dense reconstruction [26] on all
datasets except MatrixCity-ALL. For MatrixCity-ALL, we replaced depth
estimations of MVS with official provided depth estimations to avoid super
long time cost of Colmap’s dense reconstruction (using MVS for MatrixCity-M
and MatrixCity-Aerial).

##### Baselines and Metrics.

Our comparisons include 3DGS and NeRF-related works. For fair comparisons,
results from an equal number of iterations from our own 3DGS runs are also
presented. We primarily assessed the rendered image quality using three
metrics consistent with 3DGS: PSNR, SSIM, and LPIPS.

##### Implementation Details.

Our method is based on 3DGS. We extended the number of training iterations to
60k on MipNeRF-360, ScanNet++ and Maga-NeRF and 20 epochs on all MatrixCity
datasets for both 3DGS and ours to ensure adequate convergence. We do not
adjust the number of primitives during training. Since the primitives are
initialized with relatively accurate position parameters from MVS, we reduce
the learning rate for the position parameters in all primitives from from
$1.6\times 10^{-4}$ to $1.6\times 10^{-6}$ with a exponential decay function,
which is 1/100th of the original setting in 3DGS. All experiments are
conducted on NVIDIA A100 GPUs.

###  4.1 Experimental Results

|  |
---|---|---

(a) Garden

(b) ScanNet++

(c) MatrixCity-M

Figure 3: PSNR vs. #GS analysis on various datasets. The markers represent
3DGS baseline, and the curves denote our method. Results at various training
resolutions are upsampled and evaluated at the full resolution to unify the
metrics. Our method achieves superior PSNR simply by naively scale the model
size. This trend is more pronounced on the high-fidelity dataset ScanNet++ and
larger-scale dataset MatrixCity-M.

Fig 3presents our quantitative results on the MipNeRF-360 (Garden), ScanNet++
(108ec0b806), and MatrixCity-M with various training resolution. To
standardize the metrics and align them more closely with direct visual
perception, we upsample all rendered results from different resolutions to the
full resolution using nearest-neighbor upsampling and evaluate on the full
resolution. Utilizing MVS for initializing Gaussian points and disabling
densification allows us to easily control the number of model points.
Furthermore, our distributed training approach enables the use of a large
number of Gaussian primitives. We observed a strong positive correlation
between the number of Gaussian primitives and the final model’s PSNR. When the
number of primitives is similar, our PSNR closely matches the quality of 3DGS
trained on a single GPU; however, we can effectively achieve higher PSNRs by
simply increasing the number of primitives. Table 1 shows results on more
datasets, uniformly showing that our method consistently performs better
across all datasets, especially on high-resolution and large-scale datasets.

Figure 4: Visualization of models and with various number of primitives and
training resolution on Garden and ScanNet++ dataset (top and bottom metrics:
training resolution/splats count/PSNR). As we get close to objects or zoom
into camera, higher training resolutions and more primitives help maintain
rendering clarity and reveal more details, which bring better visual
experience and better quantitative results than the 3DGS baseline.

We further analyze the relationship between the number of Gaussian points and
subjective visual effects, as illustrated in Fig. 4. As revealed in Fig. 3, at
a fixed number of GS primitives, higher resolutions yield better image
quality; similarly, at a fixed high resolution, an increased number of
primitives enhances image quality. We believe that the number of primitives
determines the capacity of the 3DGS model, while higher image resolutions
bring a greater amount of information, necessitating a larger model capacity
to achieve adequate fitting. Therefore, to ensure a Retina quality effect,
both high resolution and high-quality rendering are essential, which in turn
imposes substantial demands on the number of GS primitives.

Table 1: PSNR vs. #GS analysis. * indicates that the results are average values across multiple scenes. See Appendix for complete results. Datasets | MatrixCity-Aerial | MatrixCity-M | Mega-NeRF* | ScanNet++* | MipNeRF-360*
---|---|---|---|---|---
Pixels | 13.19B | 2.48B | 2.25B | 33.44B | 2.32B
Metrics | PSNR | #GS | PSNR | #GS | PSNR | #GS | PSNR | #GS | PSNR | #GS
3DGS† | 26.56 | 25.06M | 27.81 | 1.53M | 22.56 | 6.57M | 28.42 | 1.87M | 27.33 | 3.02M
Ours | 27.70 | 217.30M | 31.12 | 62.18M | 23.03 | 35.57M | 28.91 | 39.89M | 27.78 | 27.79M

###  4.2 Exploration study

Initialization and Densification. In practice, we observed issues with the
original 3DGS’s point-growing strategy, as shown in Appendix.Fig 11. Excessive
iterations lead to deteriorating results. Using an aggressive densification
strategy in 3DGS did not yield better outcomes, and excessive point splitting
made the training unstable. In Fig 5, we show that initialization via MVS
results in a more stable training run and better model quality. By simply
initializing using MVS with more primitives, our model surpasses the original
3DGS even with careful tuning.

|
---|---

(a) Garden

(b) MatrixCity-M

Figure 5: Comparative Analysis of Desification Strategies. The curves denote
our MVS initialization, showing a clear positive correlation between the
number of primitives and visual quality. Baseline results from the 3DGS method
with its default densification threshold of 0.0002 are marked as black dots.
Lower thresholds were tested to assess their impact on point densification
compared to the default setting. The findings suggest that reducing the
threshold does not consistently increase model size and fails to outperform
our method.

Validity of Distribute Rendering we devise a simple test to validate the
correctness of the underlying rendering equation in our method. We precisely
positioned the camera’s optical axis on the dividing plane between two
subspaces sharing this plane, ensuring that each ray passes through only one
subspace. In this case, each image pixel should only be rendered by primitives
on one side of the plane. If we do not perform the step, we would expect a
crisp boundary between color pixels and completely dark pixels on the
partially rendered images from the two subspaces. As shown in Fig. 6, our
method exhibits the expected behavior while baseline partition approaches [18,
20] that deviates from the proper rendering equation fails the test.

|  |  |  |  |
---|---|---|---|---|---
Our Partition | Cell-based | Cell-based + Fine-tune
PSNR: 27.053 | PSNR: - | PSNR: 26.969
Figure 6: Our two submodels produced distinctly bounded outputs, and result in
better PSNR results. In contrast, spatial-based partitions exhibited blurry
boundaries that could not be entirely eliminated by comprehensive end-to-end
refinement, demonstrating the superiority of our approach.

##### KD-Tree Partition vs. Fixed-size Partition.

We compare our KD-Tree partitioning approach to a naive grid division strategy
that uniformly divides the scene into blocks of fixed size in Table 2. The KD-
Tree effectively balances the number of primitives in each sub-model,
resulting in an optimal balance in peak memory usage and the best training
speed. KD-tree is used throughout all experiments work without further notice.

Table 2: Partition efficiency. Range of primitive count, memory usage, total time, and communication time of submodels under different partition strategies in distributed training. The fixed-size division resulted in sub-models with significantly varied number of primitives and computation times, which increased time spent blocking on waiting, thereby reducing the overall system efficiency. Partition | Batch Size | GPU | Primitives (M) | Mem.(GiB) | Total Time (s) | Comm. (s)
---|---|---|---|---|---|---
KD-Tree | 2 | 2 | 2.87$\sim$2.88 | 6.26$\sim$6.32 | 26.74$\sim$26.76 | 4.39$\sim$6.04
Fixed-size | 2 | 2 | 0.81$\sim$4.93 | 3.57$\sim$9.01 | 32.45$\sim$32.52 | 1.04$\sim$23.83
KD-Tree | 4 | 4 | 1.45$\sim$1.47 | 5.26$\sim$5.40 | 18.87$\sim$18.90 | 5.33$\sim$7.34
Fixed-size | 4 | 4 | 0.05$\sim$4.79 | 3.08$\sim$10.95 | 32.26$\sim$32.41 | 4.77$\sim$26.83

##### Model parallelism vs. Data parallelism.

We compare data parallel training and training with our method, which is model
parallelism, on the MipNeRF-360-garden dataset. As shown in Table 4, our
method achieve lower peak memory usage and higher training throughput. It is
evident that even with such a small GS model, our method maintains its
advantages over DP and single GPU training. However, as the size of submodels
decreases and their quantity increases, it becomes increasingly challenging
for MP to achieve workload balance. As shown in Table 4, a straightforward
solution is to increase the batch size, which statistically leads to a more
balanced workload.

Table 3: Parallel mechanism efficiency.  Efficiency of data parallel(DP) and our model parallel(MP) with various setting. Parallel | GPU | Batch Size | Mem. | Time | Comm.
---|---|---|---|---|---
- | 1 | 1 | 8.33 | 32.35 | 0
DP | 2 | 2 | 9.66 | 45.36 | 29.22
MP | 2 | 2 | 6.26 | 26.76 | 6.038
DP | 4 | 4 | 9.66 | 30.62 | 21.56
MP | 4 | 4 | 5.26 | 18.90 | 7.34
DP | 8 | 8 | 9.66 | 20.38 | 16.51
MP | 8 | 8 | 5.05 | 17.738 | 12.73

Table 4: Efficiency vs. Batch Size. Increasing the batch size enhances the balance of computation across processes, resulting in reduced communication blocking.  Bacth Size | GPU-ID | Mem. | Time | Comm.
---|---|---|---|---
1 | 0 | 5.52 | 31.20 | 6.51
1 | 5.31 | 31.84 | 10.94
2 | 0 | 6.26 | 26.76 | 6.04
1 | 6.32 | 26.74 | 4.39
4 | 0 | 7.79 | 25.29 | 6.20
1 | 7.93 | 25.30 | 3.02

##  5 Training Billion-Scale 3DGS

Our distributed modeling method enables the GS model to scale up to extremely
large sizes. Typically, scenes in previous work are trained on a single GPU
using 10 to 200 million pixels with a GS model with millions of primitives. In
this work, we use 64 A100 GPUs for 10 days to train a 3DGS model with a
billion primitives on the MatrixCity-ALL dataset, which includes 141,652
images containing over 148 billion pixels. There have been no previous reports
of successful training at this scale of training set or model size. After
training the model with our method for 20 epochs, billion-scale model achieve
superior quantitative results compared to million-scale model as shown in
Table. 5. We provide more visualization results in the Fig 7.

|  |  |
---|---|---|---
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
GT | 1B | 100M | 10M
Figure 7: 1 Billion-Scale vs. 100 Million-Scale vs. 10 Million-Scale 3DGS Model on Maxtricity-ALL Dataset. Table 5: Qualitative results on MatrixCity-All. More primitives can effectively improve performance, although it requires more GPUs. Datasets | MatrixCity-All
---|---
Pixels | 148.24B
Metrics |  SSIM$\uparrow$ |  PSNR$\uparrow$ |  LPIPS$\downarrow$ | #GS | GPU
Ours-10M | 0.608 | 16.53 | 0.536 | 10.00M | 1
Ours-100M | 0.761 | 23.07 | 0.397 | 100.00M | 8
Ours-1B | 0.815 | 25.50 | 0.282 | 1023.13M | 64

##  6 Conclusion and Limitation

In this paper, we study the problem of distributed training of 3DGS models. We
devise a model parallelism-based training method that utilizes a proper
rendering equation to avoid artifacts. This allows us to significantly expand
the model scale in terms of primitive numbers and seamlessly support large-
scale scene reconstruction and detailed rendering. We also explore training
3DGS model at unprecedented scales. Despite the promising results of the
method, load balancing among sub-models remains a challenging problem when the
worker number increases. Additionally, the MVS used for primitive
initialization has a relatively low throughput, warranting a study of a better
initialization method.

##  7 Appendix

###  7.1 More Visualizations

|  |
---|---|---
|  |
|  |
|  |
|  |
GT | Ours | 3DGS
Figure 8: Comparisons of high-primitive-number RetinaGS models and 3DGS baseline on ScanNet++ dataset. Note the superior rendering quality, especially on high-frequency textures like text and leaves. |  |  |  |
---|---|---|---|---
|  |  |  |
|  |  |  |
|  |  |  |
Dividing plane | Our Partition | Cell-based | Cell-based + Fine-tune
Figure 9: Blending results at the sub-space boundaries for different partition
approaches. Approximate methods leak through the boundaries in partial
rendering, resulting in obvious artifacts after merging. RetinaGS uses the
equivalent form the 3DGS rendering equation, so it does not have this issue
and shows no artifacts in the final rendering.  Figure 10: In RetinaGS, one
primitive may be involved in partial value computation of more than one
overlapping subsets, but the indicator function in Eq 6 ensures sure its color
and opacity will only take effect once. For each ray, it can be illustrated as
each subspace only possesses a fraction of the Gaussian ellipsoid, and the
fragments collaboratively and distributively accomplish the task of a single
primitive.  Figure 11: In 3DGS, as the densification threshold decreases, the
growth rate of GS increases. However, due to the rapid growth rate, floaters
are generated in the model that cannot be eliminated through training (marked
with red rectangle), which deteriorates the reconstruction effect. The MVS
initialization strategy introduced in RetinaGS will mitigate this issue.

###  7.2 Equivalence Derivation

Starting from Eq. 3 for partial color $C_{k}(l)$:

$C_{k}(l)=\sum_{g_{i}\in N_{lk}}c_{i}\sigma_{i}\prod_{g_{j}\in
N_{lk},~{}j<i}(1-\sigma_{j}),$

Eq. 4 for partial opacity $T_{k}(l)$:

$T_{k}(l)=\prod_{g_{i}\in{N}_{lk}}(1-\sigma_{i}),$

and Eq. 5 for merging formula:

$C(l)=\sum_{k\in o_{l}}C_{k}(l)\prod_{m\in o_{l},~{}m<k}T_{m}(l),$

we substitute the expressions for partial color and partial opacity into the
merging formula:

$C(l)=\sum_{k\in o_{l}}\left(\sum_{g_{i}\in
N_{lk}}c_{i}\sigma_{i}\prod_{g_{j}\in
N_{lk},~{}j<i}(1-\sigma_{j})\right)\prod_{m\in
o_{l},~{}m<k}\left(\prod_{g_{j}\in{N}_{lm}}(1-\sigma_{j})\right).$

Now we expand this expression step by step. First, applying the distributive
property of multiplication, we can pull out the inner summation symbol:

$C(l)=\sum_{k\in o_{l}}\sum_{g_{i}\in
N_{lk}}c_{i}\sigma_{i}\left(\prod_{g_{j}\in
N_{lk},~{}j<i}(1-\sigma_{j})\right)\left(\prod_{m\in
o_{l},~{}m<k}\prod_{g_{j}\in{N}_{lm}}(1-\sigma_{j})\right).$

Then, utilizing the commutative property of multiplication, we can rewrite the
above equation as:

$C(l)=\sum_{k\in o_{l}}\sum_{g_{i}\in
N_{lk}}c_{i}\sigma_{i}\left(\prod_{g_{j}\in
N_{l1}}(1-\sigma_{j})\right)...\left(\prod_{g_{j}\in
N_{l(k-1)}}(1-\sigma_{j})\right)\cdot\left(\prod_{g_{j}\in
N_{lk},~{}j<i}(1-\sigma_{j})\right).$

Since $N_{l}$ in Eq 2 and $N_{lk}$ in Eq 3 follow a consistent rule for the
ordering of elements, we have $N_{l}=(N_{l1},N_{l2},...N_{lK})$. This allows
us to reduce the product symbols to one:

$C(l)=\sum_{k\in o_{l}}\sum_{g_{i}\in
N_{lk}}c_{i}\sigma_{i}\left(\prod_{g_{j}\in
N_{l},~{}j<i}(1-\sigma_{j})\right).$

Similarly, nested summations can also be simplified into one:

$C(l)=\sum_{g_{i}\in N_{l}}c_{i}\sigma_{i}\left(\prod_{g_{j}\in
N_{l},~{}j<i}(1-\sigma_{j})\right),$

which has the exact same form as Eq 2. Now we prove the original rendering
equation of 3DGS is equivalent to the hierarchical form underlying the
distributed training framework of RetinaGS.

###  7.3 More Experimental Results

In the main text, for the sake of brevity, we did not present the experimental
results for each individual scene on several datasets. Instead, we provided
the average results across all scenes contained in each dataset (as
highlighted with * in the experimental tables of the main text). Here, we
present the detailed experimental data for each individual scene.

Table 6: Experimental results(60K iterations) on MatrixCity. R & P means resolution and training pixels. Datasets | MatrixCity-Aerial | MatrixCity-M
---|---|---
R & P |  1920$\times$1080 & 13.19B |  1000$\times$1000 & 2.48B
Metrics |  SSIM$\uparrow$ |  PSNR$\uparrow$ |  LPIPS$\downarrow$ | #GS |  SSIM$\uparrow$ |  PSNR$\uparrow$ |  LPIPS$\downarrow$ | #GS
3DGS | 0.833 | 26.56 | 0.244 | 25.06M | 0.851 | 27.81 | 0.271 | 1.53M
Ours | 0.840 | 27.70 | 0.177 | 217.3M | 0.932 | 31.12 | 0.110 | 62.18M
Table 7: Experimental results(60K iterations) on Mega-NeRF. Datasets | Residence | Rubble | Building
---|---|---|---
Metrics |  SSIM$\uparrow$ |  PSNR$\uparrow$ |  LPIPS$\downarrow$ |  SSIM$\uparrow$ |  PSNR$\uparrow$ |  LPIPS$\downarrow$ |  SSIM$\uparrow$ |  PSNR$\uparrow$ |  LPIPS$\downarrow$
Mega-NeRF | 0.628 | 22.08 | 0.489 | 0.553 | 24.06 | 0.516 | 0.547 | 20.93 | 0.504
Switch-NeRF | 0.654 | 22.57 | 0.457 | 0.562 | 24.31 | 0.496 | 0.579 | 21.54 | 0.474
GP-NeRF | 0.661 | 22.31 | 0.448 | 0.565 | 24.06 | 0.496 | 0.566 | 21.03 | 0.486
3DGS | 0.751 | 21.43 | 0.274 | 0.709 | 24.47 | 0.337 | 0.723 | 21.74 | 0.302
Ours | 0.781 | 21.87 | 0.217 | 0.760 | 25.09 | 0.234 | 0.754 | 22.14 | 0.227
Table 8: PSNR vs. Primitive numbers(60K iterations) on Mega-NeRF. Datasets | Residence | Rubble | Building
---|---|---|---
R & P | 1368$\times$912 & 3.19B | 1152$\times$864 & 1.64B | 1152$\times$864 & 1.91B
Metrics | PSNR | #GS | PSNR | #GS | PSNR | #GS
3DGS | 21.43 | 6.42M | 24.47 | 4.7M | 21.44 | 8.6M
Ours | 21.87 | 51.41M | 25.09 | 27.9M | 22.14 | 27.4M
Table 9: Experimental results(20 epochs) on MatrixCity. ScanNet++. Datasets | 108ec0b806 | 8133208cb6
---|---|---
R & P |  8408$\times$5944 & 43.13B |  8408$\times$5935 & 23.75B
Metrics |  SSIM$\uparrow$ |  PSNR$\uparrow$ |  LPIPS$\downarrow$ | #GS |  SSIM$\uparrow$ |  PSNR$\uparrow$ |  LPIPS$\downarrow$ | #GS
3DGS | 0.881 | 28.95 | 0.408 | 2.65M | 0.908 | 27.89 | 0.355 | 1.09M
Ours | 0.883 | 29.71 | 0.395 | 47.59M | 0.908 | 28.11 | 0.340 | 32.19M
Table 10: #GS for Mip-NeRF360 scenes under 1.6K resolution and full resolution. The full marks the results obtained in full resolution. The dagger †marks the results obtained in our own experiments (60K iterations).  Method | Scenes | bicycle | garden | flowers | stump | treehill | room | counter | kitchen | bonsai
---|---|---|---|---|---|---|---|---|---|---
Resolution-Wide | 4946 | 5187 | 5025 | 4978 | 5068 | 3114 | 3115 | 3115 | 3118
Pixels-full | 3.15B | 3.22B | 2.87B | 2.05B | 2.37B | 2.00B | 1.55B | 1.80B | 1.89B
3D-GS† | 7.03M | 6.92M | 4.11M | 5.34M | 4.17M | 1.80M | 1.27M | 1.95M | 1.05M
Ours | 31.67M | 62.94M | 20.53M | 15.32M | 22.75M | 22.41M | 22.83M | 28.31M | 23.43M
3D-GS†-full | 5.04M | 7.39M | 3.11M | 3.30M | 2.61M | 1.70M | 1.20M | 1.91M | 0.97M
Ours-full | 31.67M | 62.94M | 20.53M | 15.32M | 22.75M | 22.41M | 22.83M | 28.31M | 23.43M
Table 11: SSIM scores for Mip-NeRF360 scenes under 1.6K resolution and full resolution. The full marks the results obtained in full resolution. The dagger †marks the results obtained in our own experiments (60K iterations). Method | Scenes | bicycle | garden | flowers | stump | treehill | room | counter | kitchen | bonsai
---|---|---|---|---|---|---|---|---|---|---
3D-GS | 0.771 | 0.868 | 0.605 | 0.775 | 0.638 | 0.914 | 0.905 | 0.922 | 0.938
3D-GS† | 0.770 | 0.866 | 0.623 | 0.771 | 0.641 | 0.931 | 0.919 | 0.933 | 0.950
Mip-NeRF360 | 0.685 | 0.813 | 0.583 | 0.744 | 0.632 | 0.913 | 0.894 | 0.920 | 0.941
iNPG | 0.491 | 0.649 | 0.450 | 0.574 | 0.518 | 0.855 | 0.798 | 0.818 | 0.890
Plenoxel | 0.496 | 0.606 | 0.431 | 0.523 | 0.509 | 0.841 | 0.759 | 0.648 | 0.814
Ours | 0.776 | 0.870 | 0.624 | 0.777 | 0.650 | 0.935 | 0.927 | 0.941 | 0.958
3D-GS†-full | 0.732 | 0.817 | 0.604 | 0.795 | 0.696 | 0.922 | 0.917 | 0.926 | 0.941
Ours-full | 0.751 | 0.827 | 0.635 | 0.800 | 0.703 | 0.927 | 0.925 | 0.932 | 0.947
Table 12: PSNR scores for Mip-NeRF360 scenes under 1.6K resolution and full resolution. The full marks the results obtained in full resolution. The dagger †marks the results obtained in our own experiments (60K iterations). Method | Scenes | bicycle | garden | flowers | stump | treehill | room | counter | kitchen | bonsai
---|---|---|---|---|---|---|---|---|---|---
3D-GS | 25.25 | 27.41 | 21.52 | 26.55 | 22.49 | 30.63 | 28.70 | 30.32 | 31.98
3D-GS† | 25.33 | 27.58 | 21.85 | 26.74 | 22.46 | 31.93 | 29.54 | 31.52 | 33.10
Mip-NeRF360 | 24.37 | 26.98 | 21.73 | 26.40 | 22.87 | 31.63 | 29.55 | 32.23 | 33.46
iNPG | 22.19 | 24.60 | 20.34 | 23.63 | 22.36 | 29.27 | 26.44 | 28.55 | 30.34
Plenoxel | 21.91 | 23.49 | 20.09 | 20.66 | 22.24 | 27.59 | 23.62 | 23.42 | 24.67
Ours | 25.41 | 27.74 | 21.94 | 26.86 | 22.67 | 32.86 | 29.91 | 32.49 | 34.09
3D-GS†-full | 24.47 | 26.67 | 20.78 | 26.23 | 22.34 | 31.57 | 29.33 | 31.72 | 32.92
Ours-full | 24.86 | 27.06 | 21.58 | 26.58 | 22.42 | 31.65 | 29.90 | 32.23 | 33.76
Table 13: LPIPS scores for Mip-NeRF360 scenes under 1.6K resolution and full resolution. The full marks the results obtained in full resolution. The dagger †marks the results obtained in our own experiments (60K iterations). Method | Scenes | bicycle | garden | flowers | stump | treehill | room | counter | kitchen | bonsai
---|---|---|---|---|---|---|---|---|---|---
3D-GS | 0.205 | 0.103 | 0.336 | 0.210 | 0.317 | 0.220 | 0.204 | 0.129 | 0.205
3D-GS† | 0.200 | 0.107 | 0.320 | 0.223 | 0.317 | 0.186 | 0.172 | 0.112 | 0.168
Mip-NeRF360 | 0.301 | 0.170 | 0.344 | 0.261 | 0.339 | 0.211 | 0.204 | 0.127 | 0.176
iNPG | 0.487 | 0.312 | 0.481 | 0.450 | 0.489 | 0.301 | 0.342 | 0.254 | 0.227
Plenoxel | 0.506 | 0.386 | 0.521 | 0.503 | 0.540 | 0.418 | 0.441 | 0.447 | 0.398
Ours | 0.181 | 0.100 | 0.316 | 0.218 | 0.287 | 0.160 | 0.139 | 0.099 | 0.134
3D-GS†-full | 0.375 | 0.268 | 0.438 | 0.386 | 0.447 | 0.277 | 0.261 | 0.188 | 0.263
Ours-full | 0.290 | 0.195 | 0.379 | 0.334 | 0.362 | 0.233 | 0.212 | 0.163 | 0.223

## References

  * [1] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng.  TensorFlow: Large-scale machine learning on heterogeneous systems, 2015\.  Software available from tensorflow.org.
  * [2] Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan.  Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5855–5864, 2021.
  * [3] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman.  Mip-nerf 360: Unbounded anti-aliased neural radiance fields.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5470–5479, 2022.
  * [4] HK Baumgartner, OT Starodub, JS Joehl, L Tackett, and MH Montrose.  Cyclooxygenase 1 is required for ph control at the mouse gastric surface.  Gut, 53(12):1751–1757, 2004.
  * [5] Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su.  Tensorf: Tensorial radiance fields.  In European Conference on Computer Vision, pages 333–350. Springer, 2022.
  * [6] Jeffrey Dean, Greg S. Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Quoc V. Le, Mark Z. Mao, Marc’Aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, and Andrew Y. Ng.  Large scale distributed deep networks.  In NIPS, 2012.
  * [7] FairScale authors.  Fairscale: A general purpose modular pytorch library for high performance and large scale training.  https://github.com/facebookresearch/fairscale, 2021.
  * [8] Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa.  Plenoxels: Radiance fields without neural networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5501–5510, 2022.
  * [9] Yanping Huang, Yonglong Cheng, Dehao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V. Le, and Zhifeng Chen.  Gpipe: Efficient training of giant neural networks using pipeline parallelism.  CoRR, abs/1811.06965, 2018.
  * [10] Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell.  Caffe: Convolutional architecture for fast feature embedding.  arXiv preprint arXiv:1408.5093, 2014.
  * [11] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis.  3d gaussian splatting for real-time radiance field rendering.  ACM Transactions on Graphics, 42(4):1–14, 2023.
  * [12] Alex Krizhevsky.  One weird trick for parallelizing convolutional neural networks.  ArXiv, abs/1404.5997, 2014.
  * [13] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012.
  * [14] Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen.  Gshard: Scaling giant models with conditional computation and automatic sharding.  CoRR, abs/2006.16668, 2020.
  * [15] Lingzhi Li, Zhen Shen, Zhongshu Wang, Li Shen, and Liefeng Bo.  Compressing volumetric radiance fields to 1 mb.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4222–4231, 2023.
  * [16] Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, and Soumith Chintala.  Pytorch distributed: Experiences on accelerating data parallel training.  CoRR, abs/2006.15704, 2020.
  * [17] Yixuan Li, Lihan Jiang, Linning Xu, Yuanbo Xiangli, Zhenzhi Wang, Dahua Lin, and Bo Dai.  Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3205–3215, 2023.
  * [18] Jiaqi Lin, Zhihao Li, Xiao Tang, Jianzhuang Liu, Shiyong Liu, Jiayue Liu, Yangdi Lu, Xiaofei Wu, Songcen Xu, Youliang Yan, et al.  Vastgaussian: Vast 3d gaussians for large scene reconstruction.  arXiv preprint arXiv:2402.17427, 2024.
  * [19] Hao Liu and Pieter Abbeel.  Blockwise parallel transformer for large context models.  Advances in neural information processing systems, 2023.
  * [20] Yang Liu, He Guan, Chuanchen Luo, Lue Fan, Junran Peng, and Zhaoxiang Zhang.  Citygaussian: Real-time high-quality large-scale scene rendering with gaussians.  arXiv preprint arXiv:2404.01133, 2024.
  * [21] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng.  Nerf: Representing scenes as neural radiance fields for view synthesis.  Communications of the ACM, 65(1):99–106, 2021.
  * [22] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller.  Instant neural graphics primitives with a multiresolution hash encoding.  ACM transactions on graphics (TOG), 41(4):1–15, 2022.
  * [23] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He.  Zero: memory optimizations toward training trillion parameter models.  In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’20. IEEE Press, 2020\.
  * [24] Christian Reiser, Rick Szeliski, Dor Verbin, Pratul Srinivasan, Ben Mildenhall, Andreas Geiger, Jon Barron, and Peter Hedman.  Merf: Memory-efficient radiance fields for real-time view synthesis in unbounded scenes.  ACM Transactions on Graphics (TOG), 42(4):1–12, 2023.
  * [25] Daniel Rho, Byeonghyeon Lee, Seungtae Nam, Joo Chan Lee, Jong Hwan Ko, and Eunbyung Park.  Masked wavelet representation for compact neural radiance fields.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20680–20690, 2023.
  * [26] Johannes L Schönberger, Enliang Zheng, Jan-Michael Frahm, and Marc Pollefeys.  Pixelwise view selection for unstructured multi-view stereo.  In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, pages 501–518. Springer, 2016.
  * [27] Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro.  Megatron-lm: Training multi-billion parameter language models using model parallelism.  CoRR, abs/1909.08053, 2019.
  * [28] Kaiwen Song and Juyong Zhang.  City-on-web: Real-time neural rendering of large-scale scenes on the web.  arXiv preprint arXiv:2312.16457, 2023.
  * [29] Cheng Sun, Min Sun, and Hwann-Tzong Chen.  Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5459–5469, 2022.
  * [30] Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben Mildenhall, Pratul P Srinivasan, Jonathan T Barron, and Henrik Kretzschmar.  Block-nerf: Scalable large scene neural view synthesis.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8248–8258, 2022.
  * [31] Jiaxiang Tang, Hang Zhou, Xiaokang Chen, Tianshu Hu, Errui Ding, Jingdong Wang, and Gang Zeng.  Delicate textured mesh recovery from nerf via adaptive surface refinement.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17739–17749, 2023.
  * [32] Haithem Turki, Deva Ramanan, and Mahadev Satyanarayanan.  Mega-nerf: Scalable construction of large-scale nerfs for virtual fly-throughs.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12922–12931, 2022.
  * [33] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  * [34] Limin Wang, Yuanjun Xiong, Zhe Wang, and Yu Qiao.  Towards good practices for very deep two-stream convnets.  CoRR, abs/1507.02159, 2015.
  * [35] Samuel Williams, Andrew Waterman, and David Patterson.  Roofline: An insightful visual performance model for multicore architectures.  Commun. ACM, 52:65–76, 04 2009.
  * [36] Xiuchao Wu, Jiamin Xu, Xin Zhang, Hujun Bao, Qixing Huang, Yujun Shen, James Tompkin, and Weiwei Xu.  Scanerf: Scalable bundle-adjusting neural radiance fields for large-scale scene rendering.  ACM Transactions on Graphics (TOG), 42(6):1–18, 2023.
  * [37] Yuanbo Xiangli, Linning Xu, Xingang Pan, Nanxuan Zhao, Anyi Rao, Christian Theobalt, Bo Dai, and Dahua Lin.  Bungeenerf: Progressive neural radiance field for extreme multi-scale scene rendering.  In European conference on computer vision, pages 106–122. Springer, 2022.
  * [38] Linning Xu, Yuanbo Xiangli, Sida Peng, Xingang Pan, Nanxuan Zhao, Christian Theobalt, Bo Dai, and Dahua Lin.  Grid-guided neural radiance fields for large urban scenes.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8296–8306, 2023.
  * [39] Lior Yariv, Peter Hedman, Christian Reiser, Dor Verbin, Pratul P Srinivasan, Richard Szeliski, Jonathan T Barron, and Ben Mildenhall.  Bakedsdf: Meshing neural sdfs for real-time view synthesis.  In ACM SIGGRAPH 2023 Conference Proceedings, pages 1–9, 2023.
  * [40] Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai.  Scannet++: A high-fidelity dataset of 3d indoor scenes.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22, 2023.
  * [41] Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa.  Plenoctrees for real-time rendering of neural radiance fields.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5752–5761, 2021.
  * [42] Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun.  Nerf++: Analyzing and improving neural radiance fields.  arXiv preprint arXiv:2010.07492, 2020.
  * [43] Yuqi Zhang, Guanying Chen, and Shuguang Cui.  Efficient large-scale scene representation with a hybrid of high-resolution grid and plane features.  arXiv preprint arXiv:2303.03003, 2023.
  * [44] MI Zhenxing and Dan Xu.  Switch-nerf: Learning scene decomposition with mixture of experts for large-scale neural radiance fields.  In The Eleventh International Conference on Learning Representations, 2022.
  * [45] Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross.  Ewa volume splatting.  In Proceedings Visualization, 2001. VIS’01., pages 29–538. IEEE, 2001.