# FRACTAL

Charles Gaydon

(April 2024)

declare autowrapped toks=pic me, pic dir tree/.style=for
tree=folder,font=,grow’=0,, before typesetting nodes=for tree=edge
label+/.option=pic me,,, , pic me set/.code n args=2#1/.style=inner xsep=2pic
me=pic #2, , pic me set=directoryfolder, pic me set=filefile,

FRACTAL: An Ultra-Large-Scale Aerial Lidar Dataset for 3D Semantic
Segmentation of Diverse Landscapes

Charles Gaydon Michel Daab Floryne Roche
Institut national de l’information géographique et forestière (IGN), France
<EMAIL_ADDRESS>

Abstract—Mapping agencies are increasingly adopting Aerial Lidar Scanning
(ALS) as a new tool to monitor territory and support public policies.
Processing ALS data at scale requires efficient point classification methods
that perform well over highly diverse territories. To evaluate them,
researchers need large annotated Lidar datasets, however, current Lidar
benchmark datasets have restricted scope and often cover a single urban area.
To bridge this data gap, we present the FRench ALS Clouds from TArgeted
Landscapes (FRACTAL) dataset: an ultra-large-scale aerial Lidar dataset made
of 100,000 dense point clouds with high quality labels for 7 semantic classes
and spanning 250 km². FRACTAL is built upon France’s nationwide open Lidar
data. It achieves spatial and semantic diversity via a sampling scheme that
explicitly concentrates rare classes and challenging landscapes from five
French regions. It should support the development of 3D deep learning
approaches for large-scale land monitoring. We describe the nature of the
source data, the sampling workflow, the content of the resulting dataset, and
provide an initial evaluation of segmentation performance using a performant
3D neural architecture.

Key Figures
---

➔ 250 km² of data sampled from 17,440 km² in 5 French regions
---
➔ 100,000 point clouds - 50 x 50 m each
➔ 9261M points from high density ALS (37 pts/m²)
➔ 7 semantic classes: other, ground, vegetation, building, water, bridge,
permanent structure

##  I Introduction

###  I-A Aerial Lidar Scanning for land monitoring

High-density ALS is now recognized as a powerful remote sensing modality to
support important public actions such as ecological monitoring [1] and risk
management (floods [2], forest fires [3]). Detailed 3D mapping also contribute
to the consolidation of existing geographic databases. The reduction in costs
associated with the acquisition, storage, processing, and dissemination of
Aerial Lidar Scanning (ALS) data has paved the way for its production at
unprecedented scales. In recent years, ALS has been increasingly used by
public authorities at the regional and national levels. In a comprehensive
report, [4] describe the availability of non-commercial Lidar data across
Europe: at least a dozen European countries have already made nationwide Lidar
acquisitions, with densities of 10 to 40 pts/m². In France, the national
mapping agency (Institut national de l’information géographique et forestière,
IGN) aims to map the entire French territory with high-density ALS point
clouds (10 pulses/m², about 40 pts/m²) by 2026, in a program called Lidar HD
(for "High Density") [5].

###  I-B Challenges and methods for point cloud classification

Point cloud classification is a prerequisite to any downstream use of Lidar
products. Ground points must be identified to produce accurate Digital Terrain
Models (DTMs) and separated from vegetation points to normalize tree heights
in forest applications, e.g. for forest biomass estimation. Buildings are
reconstructed from building points for urban modeling, while city trees are
mapped from clusters of vegetation points to manage urban biodiversity.

Understanding Lidar data requires specialized knowledge and tools. Many
softwares exist for Lidar feature extraction (e.g. TopoDOT [6], TerraScan
[7]…), however, each has its own strengths and weaknesses, and they all
require some form of manual input to reach satisfying results [8]. The
complexity, volume, and lack of structure of Lidar data make manual labelling
time and labor-intensive. There is therefore a critical need for more
automated methods.

Recently, there has been a surge in the development of deep learning
approaches for the efficient semantic segmentation of point clouds. PointNet
in 2016 [9] and PointNet++ in 2017 [10] paved the way by proposing a framework
to process unordered point clouds without costly preprocessing. We refer the
reader to Section 4 of a previous work [11] for a short history of the
development of 3D deep learning architectures.

###  I-C The need for diversity in ALS benchmark datasets

ALS differs from ground-based Lidar in critical ways: airborne Lidar has nadir
orientation instead of lateral orientation, and a lower but more homogeneous
point density [12]. Most importantly, airborne Lidar can be used to collect
data at a much larger scale than ground-based Lidar, which adds specific
challenges to their classification. Vast territories the size of a country
have diverse landscapes and vegetation, along with unique signs of human
activity, leading to high intraclass heterogeneity. Additionally, geographic
data are characterized by spatial autocorrelation, and a long-tail
distribution, resulting in a variety of rare scenes that may be spatially
concentrated i.e. globally rare but locally frequent like wind turbines or
greenhouses.

The specific nature and scope of ALS must be addressed in the evaluation of 3D
point cloud semantic segmentation methods. In doing so, having large
representative annotated datasets is critical, however, current benchmark
datasets come up short: they are limited in size and spatial diversity,
typically covering a single area, and thus cannot attest to the capacity of 3D
deep learning models to deal with the challenges of large-scale land
monitoring.

[13] make a comprehensive review of current Lidar benchmark datasets for 3D
point cloud semantic segmentation as of October 2023. They list 26 datasets of
which only 6 have ALS data, which we report in Table I. To date, OpenGF [14]
is the largest of them: it spans almost 50 km² and explicitly includes diverse
landscapes. Interestingly, OpenGF is built from open large-scale ALS archives
from 4 countries in 3 continents, via a careful selection of areas with high-
quality point annotations in diverse landscapes (i.e. metropolis, small city,
village, mountain). Authors demonstrate the effectiveness of leveraging open
Lidar assets to quickly create high-quality, diverse, large-scale benchmark
datasets for 3D semantic classification.

OpenGF is, however, tailored for the specific task of ground/non-ground
classification and was stripped of the semantic classes used in most
applications (e.g., building, vegetation…). The other ALS benchmark datasets
only span a single urban area, and we observe that rich semantics and high
point densities are only made available in small volumes. The Dayton Annotated
Lidar Earth Scan (DALES) [12] attempts to offer the level of detail required
for typical urban applications, with 8 semantic classes and a high point
density of 50 pts/m². Covering an area of 10 km², DALES is the largest dataset
of its kind. However, with a single urban area, it falls far short of the
landscape diversity found in EO applications.

TABLE I: Benchmark datasets for 3D semantic segmentation of ALS data. We include the datasets listed by [13] along with their proposed benchmark dataset CENAGIS-ALS, and our own dataset FRACTAL. ALS: Aerial Lidar Scanning; ULS: Unmanned Lidar Scanning. Reference | Year | Sensor Platform | Area (km²) | Classes | Points | Density (pts/m²)
---|---|---|---|---|---|---
Vaihingen (ISPRS) [15] | 2012 | ALS | 0.1 | 9 | 1.16M | 4-7
DublinCity [16] | 2015 | ALS | 2 |  9 $\in$ 7 $\in$ 4 | 260M | 348
LASDU [17] | 2020 | ALS | 1.02 | 5 | 3.12M | 3-4
DALES [12] | 2020 | ALS | 10 | 8 | 500M | 50
Hessigheim 3D [18] | 2021 | ULS | 0.19 | 11 | 74M | 800
OpenGF [14] | 2021 | ALS | 47.7 | 2 | 542M | 11
CENAGIS-ALS [13] | 2023 | ALS | 2 |  49 $\in$ 28 $\in$ 7 | 550M | 275
FRACTAL (Ours) | 2024 | ALS | 250 | 7 | 9261M | 37

###  I-D Contributions

This data gap motivates us to release the FRench ALS Clouds from TArgeted
Landscapes, or FRACTAL. FRACTAL is a large dataset for the semantic
segmentation of ALS data that spans 250 km². FRACTAL is made of 100,000
distinct point clouds, each spanning 50 $\times$ 50 m and typically containing
20k to 40k labelled points (37 pts/m² on average).

The dataset is built from the French Lidar HD data [5], using a sampling
scheme that explicitly concentrates rare classes, rare objects, and
challenging landscapes. Being sampled from an initial area of 17,440 km² in 5
French regions, it is characterized by a high diversity of scenes.

Point clouds are labelled with seven common semantic classes: ground,
vegetation, building, water, bridge, permanent structure, and other. The
labels were produced with automated processes and then verified and corrected
by Lidar operators to achieve high quality classification. FRACTAL is overall
the largest open benchmark dataset for 3D semantic segmentation, and the first
to offer the diversity of landscapes inherent to large-scale land monitoring.
It provides a challenging benchmark dedicated to the development of 3D deep
learning methods.

We make the following contributions:

  * •

Define a general framework to catalog and sample diverse point clouds from
large open ALS archives.

  * •

Introduce FRACTAL: a benchmark dataset for 3D point cloud semantic
segmentation, sampled from verified areas of the French Lidar HD data. The
sampling has explicit consideration for spatial diversity and landscape
diversity, making FRACTAL suitable for the evaluation of point cloud
classification methods against the specific challenges of land monitoring.

  * •

Set a baseline evaluation of segmentation performance on the dataset using a
state-of-the-art deep learning method to show its potential for benchmarking.

Section II presents the data sources used to create FRACTAL. Section III
describes the strategy used for cataloguing and sampling data patches at scale
into a deep learning benchmark dataset. Section IV presents the resulting
dataset itself.

##  II Data Sources

We achieve efficient, large scale dataset creation by leveraging an open ALS
data archive: the Lidar HD data. To reflect real-world Lidar processing
practices, clouds are colorized from national Very High Resolution (VHR)
aerial imagery from the ORTHO HR database.

###  II-A ALS point clouds

The Lidar HD program [5] is a national initiative that aims to provide a 3D
description of the French territory by 2026, using high-density ALS (10
pulses/m² or about 40 pts/m²). Designed to meet the needs of public actions
(flood prevention, estimation of forest resources, etc.), it is operated by
the IGN with both internal resources and subcontracted assistance. The data
acquired and produced as part of this program (i.e., point clouds, Digital
Terrain Models, Digital Surface Models) are made available as open data with
extensive documentation [19].

The program covers mainland France and its overseas territories for a total of
550,000 km². The program consists of four phases: data acquisition, storing,
processing, and dissemination, along user support. The data acquisition and
processing are sequenced in blocks of 50 x 50 km and must be compatible with a
variety of Lidar sensors (Leica, Riegl, and Teledyne/Optech), acquisition
seasons (leaf-on or leaf-off), and landscapes (urban, rural, mountains,
seashores, overseas territories, etc.).

Processing includes the semantic segmentation of point clouds into 11 classes:
unclassified, ground, vegetation (low, medium, and high), buildings, water,
bridge deck, permanent structures, artefact, synthetic. We detail the
specification of this nomenclature in Appendix
LABEL:appendix:lhd_nomenclature.

In addition, the classification of Lidar HD data may come in one of two
flavors:

Classified Lidar HD: results from a fully automated classification process
operated by the IGN, consisting of explicit detection methods using the IGN’s
national databases and the results of AI deep learning models.

Optimized Lidar HD: results from automatic classification processing followed
with manual corrections by subcontractors. The classification is audited by
the IGN via a visual inspection of 10% of the delivered point clouds. In the
audit, all classification errors are listed and rated for severity, with
particular attention to confusions in buildings, bridges and ground.
Consolidated error scores are computed, first for each of these three critical
semantic classes, then globally. These scores are required to be under
specific thresholds for the classification to be validated. Particular
attention is given to the exhaustivity of individual buildings, with a target
recall of 99.9% in high-stakes areas (e.g. flood-prone, urban…) and a target
recall of 99.5% elsewhere.

###  II-B VHR aerial images

The ORTHO HR® [20] is a mosaic of VHR aerial images acquired during national
surveys. Every year, about a third of French metropolitan territory is
updated. The individual images are mapped onto a cartographic coordinate
reference system and projected on the RGE ALTI Digital Terrain Model for
orthorectification. ORTHO HR images have a high spatial resolution of 0.20 m,
and near infrared, red, green and blue channels. Radiometric processing
methods, including equalization and global correction, are applied to obtain
the final product. The radiometry of the red, green, and blue channels cannot
be considered a physical measurement of channel reflectance due to variations
in acquisition dates and specific radiometric corrections applied for visual
appealing.

Aerial surveys are not synchronized with Lidar HD acquisitions and a variable
time lag of up to 3 years might separate them. This is because aerial images
from the ORTHO HR are renewed every three years, while Lidar data are
processed as soon as they are acquired. In the interim, buildings may have
been constructed and vegetation may have expanded or been cleared. Moving
elements such as vehicles are unlikely to be consistent. Also, the appearance
of cultivated fields is likely to be inconsistent between modalities due to
different seasons of acquisition.

##  III Sampling Methodology

###  III-A Sampling area

At the time of dataset creation, Lidar HD data is available for about half of
metropolitan France, mainly in its southern half. To have the highest possible
annotation quality, we restrict our sampling to Optimized Lidar HD i.e., to
the areas that went through human verification and audit of the
classification. Figure 1 presents the 5 spatial domains we consider, each
spanning 3456 km² on average, and distant from at least 100 km from one
another. The spatial domains compose a sampling area of 17,440 km² in Southern
France.

To define a common setting for the benchmark of deep learning models, we
reserve an area for testing in each spatial domain. Each of the 5 test areas
span 210 km² on average, for a total test sampling area of 1049 km². Having
only a limited number of spatial domains with their own unique characteristics
(seashores, mountains, field crops, etc.), test data and train data share the
same spatial domains. Test areas are however contiguous, large, and distinct
from the train areas. This enables spatial block validation and reduces the
influence of spatial autocorrelation on model evaluation, which is critical
when evaluating remote sensing solutions [21].

Figure 1: The five spatial domains composing the 17,440 km² of the sampling
area. The 1,049 km² of area reserved to sample the test set are highlighted in
red.

###  III-B Dataset size and sample unit

We aim to produce a manageable dataset with suitable size for common research
practices and computing infrastructures. Based on our own experience with
training semantic segmentation models on Lidar HD data, we set a target
dataset size of 250 km² - a 70-fold reduction compared to the initial sampling
area.

The sample unit is a 50 × 50 m square patch. We have found that such a width
provides a good balance between detailed structures and long-range
dependencies. A square shape facilitates data extraction and visualization of
data patches. Also, training models on square point clouds favors the use of
non-overlapping tiling at inference time, thus gaining efficiency by limiting
redundant predictions.

###  III-C Considerations for dataset diversity

We adopt the following design principles to sample the dataset:

Class rebalancing: We explicitly address class imbalance, which is often
detrimental to model training, by up-sampling rarer classes such as water,
bridge, and permanent structure.

Uncertainty sampling: We aim to increase the representation of scenes known to
be challenging for classification models by upsampling specific landscapes and
hardscapes, such as mountainous areas, seashores, or complex urban scenes.

Spatial sampling as a proxy for scene diversity: Given the spatial
autocorrelation of geographic data, we aim for the broadest spatial
distribution of samples. This maximizes diversity within each category of
scenes and globally.

###  III-D Cataloguing of available data

For efficient consideration of classes and landscapes, we need a detailed
representation of all available data of the sampling area with patch
resolution. This representation takes the form of a geographical database
called the Lidar Patch Catalog (LiPaC).

Cataloging data boils down to two steps, listing and describing, which we
illustrate in Figure 2. We start by regularly dividing the sampling area into
$50\times 50$ m patches. We then describe all patches with two sets of
descriptors: Lidar descriptors and vector descriptors. Lidar descriptors are
derived from the Lidar data patches themselves: number of points in each
semantic class, elevation (above sea level), and elevation gain. Vector
descriptors are derived by spatial crossing with the BD TOPO® [22], a 3D
vector description of the French territory and its infrastructures (buildings,
roads, etc.). In the LiPaC, we include information about the following classes
of objects: greenhouses, antennas, pylons, highways, parking lots, water
surfaces, greenhouses, pylons, antennas, and industrial, commercial, or
agricultural buildings.

Figure 2: Key steps involved in setting up a Lidar Patch Catalog (LiPaC) i.e.
the geographical database that we use to a) list and b) describe all candidate
Lidar patches for sampling.

The LiPaC is a comprehensive tool for flexible and efficient sampling of all
Lidar tiles under consideration. Its implementation as a PostGIS database
gives us the full power of SQL syntax to select data and further characterize
each Lidar patch with bespoke SQL queries that combine simple descriptors into
more complex ones. We define two categories of high-level Boolean flags, which
we report in Figure II. The first category contains 12 scene flags related to
the semantic classification of the point cloud and the presence of certain
objects within a semantic class. For example, we define a BUILD flag for
patches that contain some points classified as buildings (n $\geq$ 500 pts)
and a BUILD_GREENHOUSE flag for patches that intersect greenhouses. The second
category contains 7 scene flags that characterize patches in terms of
landscapes, using heuristic definitions. For example, a patch is flagged as
SEASHORE if it contains water (n $\geq$ 50 pts) and ground (n $\geq$ 100 pts)
and is within 10 m of sea level.

TABLE II: Scene types and their target minimal concentration for targeted sampling. Motivation | Scene Type | Definition | Target (%)
---|---|---|---
Classes | BUILD |  building $\geq$ 500 pts | 8 | 10
BUILD_GREENHOUSE | greenhouse (BD TOPO) | 1
BUILD_BIG | non-residential building (BD TOPO) | 1
BRIDGE |  bridge $\geq$ 50 pts | 5 | 5
WATER |  eau $\geq$ 50 pts | 4 | 5
WATER_SURFACE |  water area (BD TOPO) & eau $\geq$ 50 pts | 1
PERMSTRUCT |  permanent structure $\geq$ 50 pts | 3 | 5
PERMSTRUCT_PYLON |  pylon (BD TOPO) & permanent structure $\geq$ 50 pts | 1
| PERMSTRUCT_ANTENNA |  antenna (BD TOPO) & permanent structure $\geq$ 50 pts | 1
| OTHER |  unclassified $\geq$ 250 pts | 3 | 5
| OTHER_PARKING |  parking lot (BD TOPO) & unclassified $\geq$ 250 pts | 1
| OTHER_HIGHWAY |  highway (BD TOPO) & unclassified $\geq$ 400 pts | 1
Landscapes | FOREST |  high vegetation $\geq$ 90% of points | 5 | 20
HIGHSLOPE1 |  35 m $\leq$ elevation gain < 45 m | 2
HIGHSLOPE2 |  elevation gain $\geq$ 45 m | 2
MOUNTAIN |  elevation $\geq$ 1000 m | 4
WATER_ONLY |  water $\geq$ 50 pts & ground $=$ 0 pts | 1
SEASHORE |  -10 $\leq$ elevation < 10 & water $\geq$ 50 & ground $\geq$ 100 pts | 1
| URBAN |  building $\geq$ 25% of points | 5 |

###  III-E Patch sampling

Having a detailed description of each patch makes it easy to rebalance
semantic classes and ensure that challenging or rarer types of scenes are
represented in the dataset, following the design principles established in
Section III-C. We adopt a simple but effective sampling scheme consisting of
targeted sampling followed by completion sampling:

Targeted sampling: For each scene type (i.e., flag), we perform spatially
stratified sampling, i.e., we sample patches with stratification on 1 $\times$
1 km Lidar HD tiles until a minimum concentration of that scene type is
reached. Each sampling is performed independently, with replacement, to ensure
that the spatial diversity within each group is optimal. As a result, the
final proportion of each scene type may be higher than the minimum
requirement. We set a target requirement for each scene type so that
approximately 50% of the final dataset is found using targeted sampling (see
Table II).

Completion sampling: The rest of the dataset is found by spatially stratified
sampling without considering the flags. This is to ensure that we include more
ordinary scenes such as field crops and rural areas, which are the majority.

###  III-F Dataset split

We define a reference split into train, val, and test sets with an 80/10/10
ratio: 225 km² of data is used for model training, of which 25 km² is reserved
for in-training evaluation, and 25 km² of data is sampled from test areas and
kept for model evaluation. Train and test areas are sampled independently with
the same sampling parameters.

Patches for in-training validation are sampled from the train areas with
spatially stratified stratification, independently for each scene type. This
results in the widest possible spatial distribution for each type of scene and
ensures that all landscapes are equally represented in the train and val sets.
Refer to Appendix LABEL:appendix:descriptors_trainvaltest for the proportions
of scene types in each set.

###  III-G Data extraction and colorization

From the sampled $50\times 50$ m geometric patches, we extract the
corresponding ALS point clouds. We colorize point clouds using near infrared,
red, green and blue channels from the ORTHO HR images, based on the vertical
alignment of points and pixels. Colorization is done regardless of potential
obstructions: for instance, a ground point beneath a tree will be colored with
the spectral information of the tree above it.

Point clouds are made available in the LAZ 1.4 format defined by the American
Society for Photogrammetry and Remote Sensing (ASPRS) [23]. The following
dimensions are particularly relevant for semantic segmentation: xyz
coordinates in Lambert 93 spatial reference system (EPSG:2154), intensity
(16-bits encoding), return number and number of returns, scan angle, and near-
infrared, red, green and blue (16-bits encoding). The Lidar HD classification
nomenclature, described in Appendix LABEL:appendix:lhd_nomenclature, is left
intact in the data patches.

Except for its colorization, point clouds in FRACTAL inherit the
characteristics of the Lidar HD data, and we refer users to Section 2.2 of
their official product description [19] for more information.

##  IV FRACTAL: The Dataset

In this subsection, we present the resulting dataset, which we name FRACTAL:
FRench ALS Clouds from TArgeted Landscapes.

Section IV-A presents the characteristics of the dataset itself and emphasizes
its diversity. Section IV-B documents the file structure itself. Section IV-C
list limitations to consider when using the dataset.

###  IV-A Spatial and semantic diversity

Table III presents key figures of the dataset, namely its size expressed in
number of points, number of tiles, number of patches, and total area. We
compare these figures with those of the sampling area to highlight that, while
the dataset has been substantially reduced in size to facilitate manageable
research, it still retains spatial diversity. Despite being 70 times smaller,
the dataset incorporates data from all 17,440 initial tiles, ensuring
comprehensive representation.

TABLE III: Size, areas, and number of points in FRACTAL compared to the sampling area. $r$ = ratio. | Sampling area | $\rightarrow$ | FRACTAL | $\mathbf{r}$
---|---|---|---|---
Tiles | 17440 | $\rightarrow$ | 17440 | 1
Area (km²) | 17440 | $\rightarrow$ | 250 |  70
Patches | 6976000 | $\rightarrow$ | 100000
Points (M) | 661998 | $\rightarrow$ | 9261 | 71

Half of the dataset is found by targeted sampling of scene descriptors (47,253
patches, or 47.3%), the rest by completion sampling (52,747 patches, or
52.7%). The target minimum concentration is achieved for all descriptors
except one: scenes with permanent structure points and with an antenna in the
BD TOPO could not be concentrated to 1% in the train (target of $n=1000$
patches) and test (target of $n=100$ patches). Only 830 and 65 patches
matching this description were found, respectively.

Figure 3: Proportional gain of each scene type in FRACTAL after sampling.

Figure 3 illustrates the proportional gain of each descriptor. The gain is
most significant for rarer classes, with up to a 22-fold concentration of
bridges. There is a reduction in the proportion of scenes with a slope in the
$[35\leavevmode\nobreak\ m;45\leavevmode\nobreak\ m[$ range: these are common
scenes, concentrated in mountainous areas, which may therefore be undersampled
by the full sampling.

Figure 6 shows the distribution of FRACTAL’s patches in a small area of 4 × 6
Lidar HD tiles. The 182 patches are highlighted in yellow (targeted sampling)
and cyan (completion sampling). They are scattered and cover only 1.9% of the
24 km² (52-fold reduction). Validation patches are marked with a black cross
and are homogeneously distributed across both Lidar HD tiles and scene types.
Two Lidar HD tiles are framed in red, which we look at in more detail in
Figure 6. We observe that the targeted sampling (in yellow) over-concentrates
specific landscapes such as water surfaces, specific classes such as
buildings, and specific human-made structures such as parking lots, industrial
buildings, and electricity pylons. 5 patches were selected by completion
sampling (in cyan). Among them, 4 do not contain any scene type flag. This
illustrates the scarcity of more complex scenes in geographical data and the
need to explicitly target them to obtain a challenging and diverse dataset.

###  IV-B File Structure

pic dir tree, where level=0directory,, for tree= s sep=0.05cm, l sep=0.65cm,
font= [FRACTAL [data [train [00 … 79 [TRAIN-{tile_id}-{patch_id}.laz, file] ]
] [val [00 … 09 [VAL-{tile_id}-{patch_id}.laz, file] ] ] [test [00 … 09
[TEST-{tile_id}-{patch_id}.laz, file] ] ] ] [fractal.gpkg, file] ]
---
Figure 4: Structure of files and directories in FRACTAL.

Figure 4 illustrates the dataset’s file structure. The point clouds are
organized by set (train, val, and test), in 100 subdirectories of 1000 point
clouds each. The naming convention for data patches is shown in the figure,
where tile_id gives the X and Y kilometer northwest coordinates of the Lidar
HD tile in the Lambert 93 projection (EPSG:2154), and patch_id is a unique,
arbitrary patch identifier. For instance, patch TEST-0744_6246-006804306.laz
belongs to the test set, was extracted from Lidar HD tile whose top left
coordinates are $X=0744$ and $Y=6246$, and its unique identifier is
$006804306$.

The geometries of all patches and their descriptions are provided in a
metadata file: fractal.gpkg.

###  IV-C Limitations and recommendations

FRACTAL is sampled from an existing open ALS archive, and shares its
limitations. The dataset only includes data that underwent human corrections
and auditing (i.e. “Optimized Lidar HD”), however, the specification of the
Lidar HD classification includes a tolerance for errors. This tolerance is
class specific: for instance, a perfect 100% recall for individual buildings
is not mandatory as explained in Section II-A). Beyond actual errors, the very
definition of “unclassified” points leads to potential confusions with other
classes which we discuss in Section V-A. Considering the size of the dataset,
these imperfections are expected to have a negligible impact for model
evaluation, but users should be aware that they exist when inspecting the
data.

Point clouds are colorized with asynchronous aerial imagery, which can lead to
color artifacts (typically for moving objects such as vehicles) and
misalignment between colors and shapes that can blur segmentation boundaries
(e.g., roof points colored by a nearby tree). Mitigation strategies such as
color augmentation and randomly dropping color dimensions may help reduce
models’ reliance on color information.

Using FRACTAL to train models for production purposes should be done with
caution. Indeed, the dataset covers 5 spatial domains from 5 southern regions
of metropolitan France. While large and diverse, it only covers a fraction of
the French territory and is not representative of its full diversity regarding
either landscapes or human-made structures. Furthermore, domain shifts are
frequent for aerial images due to different acquisition conditions and
downstream data processing. When using models trained on the dataset to
predict a classification in unseen regions, one should make the adequate
verification and not assume its capacity to generalize. Similarly, considering
new Lidar sensors should be done carefully. ALS data of comparable point
densities (about 40 pts/m²) are expected to have consistent geometric
characteristics, but users should still assess the model’s accuracy when
predicting from alternative 3D data sources.

Figure 5: Patches in FRACTAL on a subset area of 4 × 6 Lidar HD tiles. Patches
selected via targeted sampling (yellow) are more present in complex urban
areas. Patches from completion (cyan) sampling are homogeneously distributed.

Figure 6: Enlarged view of 2 Lidar HD tiles showing the difference between
targeted sampling and completion sampling. 25 patches are selected from the
800 possible patches, most of which contain scenes of particular interest.

##  V Baseline evaluation

###  V-A Task and metrics

FRACTAL is a benchmark dataset for semantic segmentation of ALS scenes into 7
semantic classes: other, ground, vegetation, building, water, bridge, and
permanent structure. This reduced nomenclature is adapted from the Lidar HD
nomenclature, which has 11 classes (described in Appendix
LABEL:appendix:lhd_nomenclature). Low, medium, and high vegetation are grouped
together since they only differ in height above ground, artefact and synthetic
points are simply filtered out, and the “Unclassified” ASPRS class is renamed
"other" for clarity. This correspondence between the two nomenclatures is
summarized in Figure IV.

TABLE IV: Relation between the Lidar HD classification nomenclature and its adaptation in FRACTAL. Lidar HD | $\rightarrow$ | FRACTAL
---|---|---
Unclassified | $\rightarrow$ | Other
Ground | $\rightarrow$ | Ground
Low vegetation |  | Vegetation
Medium vegetation | $\rightarrow$
High vegetation |
Buildings | $\rightarrow$ | Building
Water | $\rightarrow$ | Water
Bridge deck | $\rightarrow$ | Bridge
Permanent structures | $\rightarrow$ | Permanent structure
Artefact |  | -
Synthetic |

We adopt the evaluation practices of similar Lidar datasets (e.g. [12]) and
use the mean IoU (mIoU) as our main metric. Importantly, we decide not to
exclude class other from metric computations. This class is semantically ill-
defined and may include points with blurred boundaries with adjacent classes
such as building, ground, and vegetation. However, it also includes well-
defined objects such as vehicles and containers. Such objects present a known
challenge to segmentation methods, and we believe it is important to account
for them in model evaluation.

For consistency with other benchmarks, we also report Overall Accuracy (OA),
and precision, recall, and F1 score (for each class and macro-averaged). All
metrics should be computed from the accumulated point confusion matrix over
all point clouds from the test set, which we also report as part of our
baseline evaluation.

###  V-B Baseline model

The baseline performance is established using Myria3D [24], a 3D deep learning
library developed at IGN. Myria3D leverages the Pytorch-Lightning framework
[25], and Pytorch-Geometric [26], a powerful library for 3D deep learning.

####  V-B1 Neural architecture

Myria3D was explicitly built for the semantic segmentation of Lidar HD data,
and scalability informed its design, including the choice of its neural
architecture: RandLa-Net [27]. Since the PointNet++ architecture [10]
succeeded to the ground-breaking PointNet [9] to operate directly on unordered
point clouds, there were many attempts to improve over point-based
architectures, characterized by PointNet-like operations hierarchically
organized in a U-shaped architecture. Conceptually simple, RandLa-Net makes
some interesting additions to PointNet++. It uses a lightweight module for
local spatial encoding and achieves performance gains thanks to random
sampling. Importantly, authors demonstrated its performance in large-scale
outdoor Lidar benchmark datasets like SemanticKITTI [28] and Semantic 3D [29].

####  V-B2 Data processing and training hyperparameters

In our experiments, we kept most defaults setting from Myria3D, which were
described in extenso in a previous work. We thus refer the reader to Section
Benchmark Models in [11] for details on the processing of colorized Lidar HD
tiles as well as training hyperparameters (optimizer, learning rate,
scheduler, early stopping, etc.).

####  V-B3 Infrastructure

Training is conducted with 6 NVIDIA Tesla V100 GPUs, each equipped with 32 GB
of memory, using Pytorch-Lightning’s distributed data parallel (ddp) strategy.
We log all metrics using Comet, a machine learning experiment tracking tool
[30]. With this configuration the approximate learning time is 30 minutes per
epoch.

###  V-C Experimental results

We report the results obtained using the 80,000 patches for training and
10,000 patches for validation, and testing on the remaining 10,000 patches of
the dataset.

####  V-C1 Quantitative results

The baseline achieves a test mIoU of 77.5% and a test OA of 96.1%. Figure 7
details the IoU results per class. We observe highly accurate results for the
three most common classes: ground, vegetation, and building have an IoU above
90%, with a maximum of 93.8% for vegetation. Similarly, class water has an IoU
of 90.1%, despite its extreme initial rarity (0.6%) in the sampling area. For
classes bridge and permanent structure, also initially rare (0.01% each), the
model achieves decent performance with an IoU above 60%. Compared to the six
other classes, the baseline underperforms for class other, but still achieves
an IoU of 47.5%, which is remarkable considering the fuzzy definition of this
class. Based on our own experience with Lidar HD semantic segmentation, these
performances can not be achieved without the class rebalancing that happens in
the sampling of FRACTAL. For an in-depth performance assessment, all metrics
(IoU, precision, recall and F1 score) are reported in Appendix
LABEL:appendix:metrics.

othergroundvegetationbuildingwaterbridgeperm.
struct.$0$$50$$100$$47.5$$91.9$$93.8$$90.4$$90.1$$65.2$$63.5$IoU (%) Figure 7:
Baseline per-class test IoUs.

Figure 8 shows the baseline confusion matrix on the test set, normalized by
both rows (a) and columns (b). Vegetation, ground, and building have low
relative confusion, as expected from their high IoU. Permanent structure has
its highest confusion with vegetation, due to the vertical nature and
intricate geometries shared by both classes e.g., between pylons and trees.
High interclass confusion is also found between classes that may have
imprecise geometric boundaries. For example, bridge points are misclassified
as ground, of which they are often an extension. Points of class other have
high confusion with vegetation and ground points. This is mainly due to the
Lidar HD classification specification, which favors specificity over recall
for the ground and vegetation classes. As a result, the model may be penalized
for accurately identifying ground and vegetation whose target class is other.
Note that with only 0.6% of all points in FRACTAL, class other is a small
minority, and these errors have a negligible effect on the measured
performance for the ground and vegetation classes.

Figure 8: Baseline test confusion matrices normalized by rows (a) and columns
(b). Diagonal values in row-normalized matrix correspond to recall. Diagonal
values in column-normalized matrix correspond to precision.

####  V-C2 Qualitative results

Finally, Figure 9 presents sample predictions of the baseline model for five
types of Lidar scenes: a) OTHER_PARKING, b) WATER and BRIDGE, c) URBAN, d)
BUILD_GREENHOUSE, and e) HIGHSLOPE1. The 5 patches are selected at random
based solely on their scene descriptors, among test patches with at least 10k
points (i.e. at least 4 pts/m²). We present the inference results without
cherry-picking.

Figure 9: Input point cloud, target classification, and predictions of the
baseline model for a subset of test patches. Patches are selected at random
and without cherry-picking, to match the following scene types: a)
OTHER_PARKING, b) WATER and BRIDGE, c) URBAN, d) BUILD_GREENHOUSE, and e)
HIGHSLOPE1. Color scheme is: other (white), ground (orange), vegetation
(green), building (red), water (cyan), bridge (yellow), permanent structure
(purple). Predictions errors are circled in red.

##  VI Conclusion

We present FRACTAL, an ultra-large-scale benchmark dataset for the 3D semantic
segmentation of ALS point clouds. The dataset is based on high quality open
data from the French Lidar HD nationwide acquisition program. It is a
distillation of a larger initial area of 17,440 km² in five French regions. We
show that a simple targeted sampling with spatial stratification at all levels
effectively preserves the spatial and semantic diversity of regional-scale
volumes of Lidar data. FRACTAL’s size is compatible with deep learning
research practices, and it is the largest Lidar benchmark dataset to date,
with 9261 million points in 100,000 point clouds and a total span of 250 km².
While other ALS benchmark datasets typically cover a single urban area, the
large diversity of urban and rural landscapes in FRACTAL is representative of
the challenges of 3D semantic segmentation for land monitoring. Thanks to
class rebalancing, our methodology opens the possibility of robust assessment
of semantic segmentation performance, even for initially rare classes water,
bridge, and permanent structure. The baseline evaluation of a 3D neural
network (RandLa-Net) further demonstrates the quality of the dataset for model
benchmarking. We invite the research community to benchmark both state-of-the-
art and novel methods against this dataset. We hope that FRACTAL will advance
the field of deep learning for airborne Lidar and ultimately benefit public
Lidar-based 3D mapping programs for land monitoring.

## Code and data access

The dataset is made available on the HuggingFace platform as IGNF/FRACTAL. The
sampling, extraction, and colorization of point clouds were conducted with the
Patch Catalog Sampling (PaCaSam) code repository available at:
github.com/IGNF/pacasam. The baseline model was trained with the Myria3D code
repository available at github.com/IGNF/myria3d. Its weights are made
available on HuggingFace as IGNF/FRACTAL-LidarHD_7cl_randlanet.

All assets are released under permissive open licences.

## Acknowledgements

The authors thank Léa Vauchier for her code reviews of the data engineering
code, and Marouane Zellou for maintaining the in-house high-performance
computing server, and Matthieu Porte and Anatol Garioud for reviewing this
data paper. The authors also thank Anatol Garioud for paving the way for the
open release of deep learning datasets for land monitoring at IGN with the
FLAIR dataset [31].

## Authors’ contribution

C.G. established the sampling framework. C.G. and F.R defined the data
descriptors and their target proportions. C.G. implemented the tools for patch
sampling and data extraction. F.R. and M.D. defined the specifications for the
Lidar Patch Catalog, and M.D. implemented it. C.G. implemented and evaluated
the deep learning baseline. C.G. wrote the data paper and released the
dataset.

## References

## References

  * [1] M. Melin, A. C. Shapiro, and P. Glover-Kapfer, “LiDAR for ecology and conservation - WWF conservation technology series (3),” pp. 24–25, 2017. [Online]. Available: www.researchgate.net/publication/320347018
  * [2] N. A. Muhadi, A. F. Abdullah, S. K. Bejo, M. R. Mahadi, and A. Mijic, “The use of LiDAR-derived DEM in flood applications: A review,” 7 2020. [Online]. Available: www.mdpi.com/2072-4292/12/14/2308
  * [3] M. Yebra, S. Marselis, A. van Dijk, G. Cary, and Y. Chen, “Using Lidar for forest and fuel structure mapping: Options, benefits, requirements and costs,” 2015. [Online]. Available: www.bnhcrc.com.au/sites/default/files/managed/downloads/using_lidar_for_forest_and_fuel_structure_mapping_final.pdf
  * [4] G. Kakoulaki, A. Martinez, and F. Petro, “Non-commercial Light Detection and Ranging (LiDAR) data in Europe,” 2021. [Online]. Available: https://doi.org/10.2760/212427
  * [5] Institut national de l’information géographique et forestière (IGN), “Lidar HD [Database],” 1 2023. [Online]. Available: www.geoservices.ign.fr/documentation/donnees/alti/lidarhd
  * [6] “TopoDOT [software].” [Online]. Available: www.topodot.com
  * [7] “TerraScan [software].” [Online]. Available: terrasolid.com/products/terrascan
  * [8] K. Yen, “Automated LiDAR extraction software,” 2021. [Online]. Available: https://dot.ca.gov/-/media/dot-media/programs/research-innovation-system-information/documents/preliminary-investigations/pi-0324-lidarsoftwarepi.pdf
  * [9] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “PointNet: Deep learning on point sets for 3D classification and segmentation,” _Arxiv_ , 12 2016. [Online]. Available: https://arxiv.org/abs/1612.00593
  * [10] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “PointNet++: Deep hierarchical feature learning on point sets in a metric space,” _Arxiv_ , 6 2017. [Online]. Available: https://arxiv.org/abs/1706.02413
  * [11] C. Gaydon and F. Roche, “PureForest: A large-scale aerial lidar and aerial imagery dataset for tree species classification in monospecific forests,” 4 2024. [Online]. Available: https://arxiv.org/abs/2404.12064
  * [12] N. Varney, V. K. Asari, and Q. Graehling, “DALES: A large-scale aerial LiDAR data set for semantic segmentation,” _ArXiv_ , 4 2020. [Online]. Available: http://arxiv.org/abs/2004.11985
  * [13] P. Zachar, K. Bakuła, and W. Ostrowski, “CENAGIS-ALS Benchmark - new proposal for dense ALS benchmark based on the review of datasets and benchmarks for 3D point cloud segmentation,” vol. 48. International Society for Photogrammetry and Remote Sensing, 10 2023, pp. 227–234. [Online]. Available: https://doi.org/10.5194/isprs-archives-XLVIII-1-W3-2023-227-2023
  * [14] N. Qin, W. Tan, L. Ma, D. Zhang, and J. Li, “OpenGF: An ultra-large-scale ground filtering dataset built upon open ALS point clouds around the world.” [Online]. Available: https://arxiv.org/abs/2101.09641
  * [15] J. Niemeyer, F. Rottensteiner, and U. Soergel, “Contextual classification of lidar data and building object detection in urban areas,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 87, pp. 152–165, 1 2014. [Online]. Available: https://doi.org/10.1016/j.isprsjprs.2013.11.001
  * [16] S. M. I. Zolanvari, S. Ruano, A. Rana, A. Cummins, R. E. da Silva, M. Rahbar, and A. Smolic, “DublinCity: Annotated LiDAR point cloud and its applications,” 9 2019. [Online]. Available: http://arxiv.org/abs/1909.03613
  * [17] Z. Ye, Y. Xu, R. Huang, X. Tong, X. Li, X. Liu, K. Luan, L. Hoegner, and U. Stilla, “LASDU: A large-scale aerial LiDAR dataset for semantic labeling in dense urban areas,” _ISPRS International Journal of Geo-Information_ , vol. 9, 7 2020. [Online]. Available: https://doi.org/10.3390/ijgi9070450
  * [18] M. Kölle, D. Laupheimer, S. Schmohl, N. Haala, F. Rottensteiner, J. D. Wegner, and H. Ledoux, “The Hessigheim 3D (H3D) benchmark on semantic segmentation of high-resolution 3D point clouds and textured meshes from UAV LiDAR and Multi-View-Stereo,” _ISPRS Open Journal of Photogrammetry and Remote Sensing_ , vol. 1, p. 100001, 10 2021. [Online]. Available: https://doi.org/10.1016/j.ophoto.2021.100001
  * [19] Institut national de l’information géographique et forestière (IGN), “LiDAR HD version 1.0 - descriptif de contenu des nuages de points LiDAR,” 10 2023. [Online]. Available: www.geoservices.ign.fr/sites/default/files/2023-10/DC_LiDAR_HD_1-0_PTS.pdf
  * [20] ——, “ORTHO HR [Database],” 1 2023. [Online]. Available: www.geoservices.ign.fr/bdortho
  * [21] T. Kattenborn, F. Schiefer, J. Frey, H. Feilhauer, M. D. Mahecha, and C. F. Dormann, “Spatially autocorrelated training and validation samples inflate performance assessment of convolutional neural networks,” _ISPRS Open Journal of Photogrammetry and Remote Sensing_ , vol. 5, 8 2022. [Online]. Available: https://doi.org/10.1016/j.ophoto.2022.100018
  * [22] Institut national de l’information géographique et forestière (IGN), “Bd topo® [database],” 2024. [Online]. Available: www.geoservices.ign.fr/bdtopo
  * [23] The American Society for Photogrammetry and Remote Sensing, “LAS specification 1.4 - R15,” 2019. [Online]. Available: www.asprs.org/wp-content/uploads/2019/07/LAS_1_4_r15.pdf
  * [24] C. Gaydon, “Myria3D: Deep learning for the semantic segmentation of aerial Lidar point clouds [software],” 2022. [Online]. Available: www.github.com/IGNF/myria3d
  * [25] W. Falcon and T. P. L. team, “PyTorch Lightning [software],” 3 2019. [Online]. Available: www.github.com/Lightning-AI/lightning
  * [26] M. Fey and J. E. Lenssen, “Fast graph representation learning with PyTorch Geometric,” 2019. [Online]. Available: www.github.com/pyg-team/pytorch_geometric
  * [27] Q. Hu, B. Yang, L. Xie, S. Rosa, Y. Guo, Z. Wang, N. Trigoni, and A. Markham, “RandLA-Net: Efficient semantic segmentation of large-scale point clouds,” _Arxiv_ , 11 2019. [Online]. Available: https://arxiv.org/abs/1911.11236
  * [28] J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall, “SemanticKITTI: A dataset for semantic scene understanding of LiDAR sequences,” 2019. [Online]. Available: https://arxiv.org/abs/1904.01416
  * [29] T. Hackel, N. Savinov, L. Ladicky, J. D. Wegner, K. Schindler, and M. Pollefeys, “SEMANTIC3D.NET: A new large-scale point cloud classification benchmark,” vol. IV-1-W1, 2017, pp. 91–98. [Online]. Available: http://www.semantic3d.net
  * [30] Comet ML, “Comet [ML platform],” 2024. [Online]. Available: www.comet.com
  * [31] A. Garioud, N. Gonthier, L. Landrieu, A. D. Wit, M. Valette, M. Poupée, S. Giordano, and B. Wattrelos, “FLAIR: a country-scale land cover semantic segmentation dataset from multi-source optical imagery,” 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2310.13336

## Appendices

###  A Specifications of the Lidar HD classification.

RGB | Value | Name |  Content
---|---|---|---
255,255,255 | 1 | Unclassified |  All points that do not belong in any of the other classes. For example, it includes vehicles, animals or people, temporary objects, wood piles, etc.
255,128,0 | 2 | Ground |  Points located on the surface of natural and artificial ground. Bridge decks are not part of this class.
0,255,0 | 3 | Low vegetation |  Trees, shrubs and low vegetation (e.g. bushes, ferns, reeds, etc.). Vegetation at ground level (less than 20 cm high, typically grass) is classified as vegetation only if there are enough ground points locally for ground modeling. The class also includes cultivated trees (orchards, vineyards, field crops, etc.). Vegetation is further divided into 3 classes based on height above ground: low (< 0.5m), medium (between 0.5 m and 1.5 m), and high ($\geq{1.5\leavevmode\nobreak\ m}$) vegetation.
0,193,0 | 4 | Medium vegetation
0,134,0 | 5 | High vegetation
255,0,0 | 6 | Building |  A building is defined as a permanent structure with an area greater than 10 m². Besides residential buildings it includes monuments, castles, mills, water towers, lighthouses, industrial chimneys, ramparts and fortifications. The specification also includes roofs and façades, as well as chimneys, dormer windows, skylights and balconies. As the permanent nature of a structure cannot be established from lidar data only, this class may include lightweight structures without walls such as garden sheds, bungalows, market canvases or awnings.
0,225,225 | 9 | Water |  All points located on the surface of rivers, bodies of water, sea, or ocean.
255,255,0 | 17 | Bridge deck |  A bridge is an engineering structure passing over one or more elements of the road, rail or waterway network. Point on bridge decks are included, while structural elements such as piers and parapets are assigned to the "Unclassified" class. Very high structural elements ($\geq$ 5 m above deck level) such as piers and abutments are classified as “Permanent structures”. Tunneled passages (including nozzles, which are openings in the ground generally to allow water to drain) are considered part of the ground and therefore excluded.
128,0,64 | 64 | Permanent structures |  All aboveground objects other than buildings, vegetation and bridges, that are identified as perennial and of such a nature that they characterize the landscape. This class includes (but is not limited to): wind turbines, cable cars, telecommunication antennas, electricity distribution networks (cables and pylons), bridge elements above the deck (cables, piers, etc.).
64,0,128 | 65 | Artefact |  All points that do not correspond to an actual object or terrain.
255,0,255 | 66 | Synthetic |  Artificial points created under bridges and and on water surfaces to have coherent digital models.

Refer to the Lidar HD product description [19] for more specifications.

###  B Sensors used to acquire Lidar HD data in each of the 8 50 $\times$ 50
km blocks represented in FRACTAL.

|  | Subcontractor(s)
---|---|---
Block | Sensor | Acquisition | Classification
GN | CityMapper 2H: Hyperion 2+ | APEI + Avineon | Avineon
MQ | RIEGL VQ1560 II-S | Eurosense + SFS | Eurosense + SFS
MP | RIEGL VQ-1560 II | Eurosense | Sintégra
PK | RIEGL VQ1560 II | Eurosense | Eurosense
PO | Leica CityMapper-2 | Avineon + APEI | Avineon + APEI
PP | Leica CityMapper-2 | Avineon + APEI | Eurosense + SFS
QO | RIEGL VQ780 II-S | Sintégra + Bluesky | Sintégra + Bluesky
UT | RIEGL VQ780 II-S | Sintégra + Bluesky | Avineon + APEI

N.B. Data acquisition and its classification were often conducted by distinct
subcontractors, which we also report.

###  C Number of points by class in the sampling area, in the dataset, and in
the train, val and test sets.

| FRACTAL
---|---
| Sampling Area | FRACTAL | Train | Val | Test
Name | Points (M) | % | Points (M) | % | Points (M) | % | Points (M) | % | Points (M) | %
Other | 2,137 | 0.32 | 53 | 0.57 | 41 | 0.56 | 5 | 0.53 | 6 | 0.66
Ground | 258,751 | 39.1 | 3,625 | 39.1 | 2,874 | 39.0 | 359 | 39.1 | 391 | 40.5
Vegetation | 394,425 | 59.6 | 5,248 | 56.7 | 4,203 | 57.0 | 523 | 56.9 | 523 | 54.1
Building | 4,939 | 0.75 | 264 | 2.85 | 206 | 2.80 | 26 | 2.80 | 32 | 3.33
Water | 1,504 | 0.23 | 54 | 0.59 | 38 | 0.52 | 5 | 0.49 | 12 | 1.20
Bridge | 47 | 0.01 | 12 | 0.13 | 9 | 0.13 | 1 | 0.10 | 2 | 0.16
Permanent structure | 33 | 0.01 | 3 | 0.04 | 3 | 0.04 | 0 | 0.04 | 0 | 0.03
Total | 661998 | - | 9261 | - | 7376 | - | 919 | - | 966 | -

###  D Proportions of scene types in the train, val, and test sets.

Descriptor | Train (%) | Val (%) | Test (%)
---|---|---|---
BUILD | 22.5 | 22.8 | 25.7
BUILD_BIG | 5 | 4.8 | 4.9
BUILD_GREENHOUSE | 1.3 | 1.3 | 1.2
BRIDGE | 5.5 | 5.4 | 5.4
WATER | 8.2 | 8.1 | 11
WATER_SURFACE | 6 | 5.9 | 7.7
PERMSTRUCT | 5 | 4.9 | 4.6
PERMSTRUCT_ANTENNA | 0.9 | 0.9 | 0.6
PERMSTRUCT_PYLON | 1.5 | 1.4 | 1.4
OTHER | 22.7 | 22.6 | 27.5
OTHER_HIGHWAY | 1.3 | 1.2 | 1.7
OTHER_PARKING | 1.1 | 1.1 | 1.5
FOREST | 7.3 | 7.4 | 7.1
HIGHSLOPE1 | 11.5 | 11.8 | 11
HIGHSLOPE2 | 5.5 | 5.4 | 4.8
MOUTAIN | 17.4 | 17.6 | 14.5
WATER_ONLY | 1.7 | 1.7 | 4.5
SEASHORE | 1.4 | 1.4 | 4.5
URBAN | 6.4 | 6.5 | 7.4

###  E Baseline test IoU, precision, recall and F1 score for each semantic
class and macro-averaged.

Class | IoU | Accuracy | Precision | Recall | F1 Score
---|---|---|---|---|---
other | 47.5 | 54.9 | 77.8 | 54.9 | 64.4
ground | 91.9 | 97.7 | 93.8 | 97.7 | 95.8
vegetation | 93.8 | 95.6 | 98.0 | 95.6 | 96.8
building | 90.4 | 93.7 | 96.2 | 93.7 | 95.0
water | 90.1 | 92.6 | 97.1 | 92.6 | 94.8
bridge | 65.2 | 96.1 | 79.3 | 78.6 | 79.0
permanent structure | 63.5 | 76.6 | 78.9 | 76.6 | 77.7
Macro Average | 77.5 | 86.7 | 88.7 | 84.2 | 86.2