# Observed Patterns of Surface Solar Irradiance under Cloudy and Clear-sky
Conditions

Wouter Mol<EMAIL_ADDRESS> Meteorology and Air Quality Group, Wageningen
University, Wageningen, The Netherlands  Bert Heusinkveld  Meteorology and Air
Quality Group, Wageningen University, Wageningen, The Netherlands  Mary Rose
Mangan  Meteorology and Air Quality Group, Wageningen University, Wageningen,
The Netherlands  Oscar Hartogensis  Meteorology and Air Quality Group,
Wageningen University, Wageningen, The Netherlands  Menno Veerman  Meteorology
and Air Quality Group, Wageningen University, Wageningen, The Netherlands
Chiel van Heerwaarden  Meteorology and Air Quality Group, Wageningen
University, Wageningen, The Netherlands

###### Abstract

Surface solar irradiance varies on scales down to seconds or meters mainly due
to clouds, but also via moisture structures in the atmospheric boundary layer.
The highly variable nature of irradiance is not resolved by most atmospheric
models, yet heterogeneity in surface irradiance impacts the overlying cloud
field through feedback with the land surface. Atmospheric model resolution and
radiative transfer calculations are simplified, necessary due to high
computational demands, but the development of fast models capable of
accurately resolving irradiance variability is limited by our understanding of
cloud-driven solar irradiance variability. Spatial and spectrally resolving
observational datasets of solar irradiance at such high resolution are rare,
but they are required for characterizing observed variability, understanding
the mechanisms, and model validation. In 2021, we deployed a spatial network
of low-cost radiometers at the FESSTVaL (Germany) and LIAISE (Spain) field
campaigns, specifically to gather data on cloud-driven surface patterns of
irradiance, including spectral effects, with the aim to address this gap in
observations and understanding. This work discusses the measurement strategies
at both campaigns, the performance and calibration of these radiometers,
analysis techniques to construct spatial patterns despite limited network
size, and our interpretation of these observations. We find that cumulus,
altocumulus, and cirrus clouds generate large spatiotemporal variability in
irradiance, but through different mechanisms, and with spatial scales of
patterns ranging from 50 m to 30 km. Under clear-sky conditions, solar
irradiance varies significantly in water vapour absorption bands at the minute
scale, due to local and regional variability in atmospheric moisture. In
conclusion, observing detailed spatiotemporal irradiance patterns is possible
using a relatively small, low-cost sensor network, and these network
observations provide important insights and validation for the development of
models capable of resolving irradiance variability.

Keywords: Solar Radiation, Local or boundary layer scale, Observations

00footnotetext: Funding from the Dutch Research Council (NWO) (grant:
VI.Vidi.192.068)

##  1 Introduction

Surface solar irradiance varies on scales down to seconds or meters mainly due
to clouds, which influences the coupled land-atmosphere system (i.e., the
energy, water, and carbon cycle), atmospheric photochemistry, and solar energy
production. Cloud fields generate complex, high contrast spatial patterns that
range from stationary to dynamically evolving depending on cloud type, cloud
velocity, and cloud shape evolution. The spatiotemporal scales of surface
irradiance variability involved are directly linked to cloud size
distribution, which means a wide range of scales contribute to the total
variance [Wood and Field, 2011, Tabar et al., 2014, Mol et al., 2023c]. Light
scattering and absorption in the atmosphere by gas molecules, clouds, or
aerosols, also result in changes in the light spectrum, which has implications
for photosynthesis [Durand et al., 2021] and wavelength-dependant photovoltaic
technologies [Dirnberger et al., 2015]. Cloud-driven heterogeneity in surface
heat fluxes can feed back to cloud development [Lohou and Patton, 2014, Jakub
and Mayer, 2017, Veerman et al., 2022], and the highly variable nature of
solar irradiance negatively impacts electricity grid stability and solar
energy yield [Liang, 2017, Kreuwel et al., 2021, Yang et al., 2022].

Heterogeneity in surface irradiance is amplified by, most notably, the three-
dimensional scattering and redistribution of solar radiation by clouds,
resulting in darker shadows and a feature called cloud enhancement [Gueymard,
2017]. Cloud enhancement is caused by the scattering of irradiance by clouds
that locally combines with direct irradiance to exceed clear-sky irradiance
(hence ’enhancement’), and potentially even extra-terrestrial irradiance
[Yordanov et al., 2015, Gueymard, 2017]. Cloud shadows, mainly caused by the
(partial) blocking of direct irradiance, are darker due to scattered light
(diffuse irradiance) reaching the surface in an area around the cloud shadow
rather than only in the shadow. The physics of radiative transfer is well-
known, though the 3D path photons take through an atmosphere filled with
liquid water and ice, aerosols, and gas molecules is highly complex.
Qualitatively, the spatial patterns of cloud fields can be visible by eye,
e.g. from an aerial view or on the side of mountains, but a quantitative
characterization and explanation of how exactly they arise remain a challenge.
To add to the complexity, in particular absorption and Rayleigh scattering
strongly depend on wavelength.

Spatial observations at the scale of cloud shadow and enhancement patterns
(101 \- 104 m) [Mol et al., 2023c], with an adequate temporal resolution of 1
s or better [Tomson, 2010, Yordanov et al., 2013], are rare. Many high quality
solar irradiance observations exist, for example the Baseline Surface
Radiation Network [Driemel et al., 2018], but these are single point
measurements at typically 1 minute resolution. Other examples are the 99
pyranometers network deployed during the HOPE campaign [Lohmann et al., 2016],
or the 17 photodiode pyranometers used by Weigl et al. [2012], Tabar et al.
[2014], both deployed on $\sim$ 1 km2 scale areas. Photodiode pyranometers are
fast responding ($<<$ 1 s) sensors, and thus capture even the fastest
atmospheric driven fluctuations in irradiance, but they do not provide
spectral information. None of these datasets provide enough temporal,
spectral, and spatial resolution to characterize surface irradiance
heterogeneity at the scale of clouds, in part due to the high cost and
operational burden of performing such measurements.

As for using modelling as a research tool, the accurate modelling (and
forecasting) of solar irradiance requires a fully resolved and realistic
simulation of clouds, information about atmospheric composition, and accurate
3D radiative transfer calculations such as Monte Carlo ray tracing. In
practise, the most detailed operational weather models operate at a resolution
of approximately 1 km, too coarse to resolve clouds. Apart from that, the
physics of radiative transfer is simplified to a two-stream approach (up and
down) [Hogan and Bozzo, 2018], which by design means it cannot resolve cloud
enhancement and will thus underestimate the contrast between shaded and
sunlight surfaces. There are attempts to improve this by subgrid
parameterization of clouds and 3D radiative effects, such as SPARTACUS
[Villefranque et al., 2021] or ecRAD [Ukkonen and Hogan, 2023] for operational
weather models. More accurately resolving 3D radiation is done in academic
setups by e.g. Veerman et al. [2022], who have coupled a Monte Carlo ray
tracer to a cloud resolving model, or Pincus and Evans [2009], who demonstrate
an alternative to ray tracing. Development of such parameterizations or
optimized 3D radiative transfer calculations to work at finer scales depend on
our understanding of radiative transfer in the atmosphere and validation using
observations.

To quantitatively characterize spatial surface irradiance variability and
understand how exactly it arises, we have developed low-cost radiometers,
optimized to capture variability and to be deployed in spatial network setups
[Heusinkveld et al., 2022]. The design philosophy is similar to that of the
APOLLO (Autonomous cold POoL LOgger) network [Kirsch et al., 2022]:
autonomously running low-cost instruments optimized to accurately capture
fluctuations rather than high accuracy single-point data. With calibration
against expensive, high quality reference stations, these instruments give
useful information at a fraction of the cost while also being flexible in
their setup, necessary to densely cover and maintain a large area.

In this paper, we describe the deployment, calibration, and first findings of
a 20 to 25 radiometer network at two field campaigns in 2021: FESSTVaL and
LIAISE. Both campaigns were organized to observe and understand local
meteorology, with FESSTVaL focused on sub-mesoscale variability and cold pools
from convective storms, and LIAISE with the aim to observe the land-atmosphere
coupling from irrigated crop fields to the regional scale in complex terrain.
The detailed observations of the atmosphere done by other groups at these
campaigns are essential for understanding what we see our own measurements.
Section 2 covers the design of the radiometers and our network measurement
strategy at both campaigns. Calibration and validation of the radiometers is
detailed in Section 3, where we derive global horizontal irradiance and total
column water vapour after calibrating the raw data. Findings resulting from
these campaigns using the broadband signal is presented and discussed in
Section 4, followed by an analysis of atmospheric moisture observations in
Section 5. Conclusions and projected impacts of our results mean for solar
irradiance research are discussed in Section 6. All data in this study are
open access, see Section 7 for details.

##  2 Measurement strategy and campaigns

Figure 1: Spectral solar irradiance and the radiometer wavelength bands.
Normalized top of atmosphere and surface solar spectra are based on the
reference spectra ASTM G-173-03 [National Renewable Energy Laboratory, 2023].
Each wavelength band response curve of the FROST radiometer is color-coded
according to the three subsensor groups.

###  2.1 Solar irradiance sensors

The sensors we use have been specifically designed to capture the fastest
cloud-driven fluctuations of incoming sunlight, and variations in the light
spectrum induced by the atmospheric composition, clouds, aerosols, and
vegetation. At a sampling frequency of 10 Hz, the Fast Response Optical
Spectroscopy Time synchronized instrument (FROST, Heusinkveld et al. [2022]),
measures incoming shortwave irradiance at 18 wavelengths in the visible to
near-infrared spectrum (410 to 940 nm). The locations of the 18 wavelengths
are detailed in Figure 1, which shows the response curves for the factory
specification of 20 nm full-width half maximum, combined with the solar
spectrum under a reference cloud-free atmosphere (ASTM G-173-03, National
Renewable Energy Laboratory [2023]). The 18 wavelengths are spread over 3
subgroups of 6 bands on the spectrometer chip, which are color-coded in the
figure. Since these three subsensors are spatially separated by about a
centimeter (in a triad), we use a teflon diffuser on top to equally distribute
incoming sunlight.

Material costs for one sensor are under €200 in total, they are powered by
their own small solar panel, and are all time-synchronized using a GPS chip.
This makes them scalable and easy to deploy on tripods in field campaigns in
flexible setups compared to conventional high quality pyranometers or
spectrometers. The low-cost design philosophy is a trade-off against accuracy
compared to high-end instrumentation, but Section 3 will show that after
calibration, performance is good enough to capture and analyse spatial surface
irradiance patterns driven by clouds and spectral signals of variations in
atmospheric moisture. We focus on the practical application of the sensor in
this work, but a complete and technical reference with more use cases is
described in Heusinkveld et al. [2022].

###  2.2 Network design

We participated in two major field campaigns aimed at observing local to
regional scale atmospheric dynamics and land-atmosphere coupling. Two weeks in
June 2021 during FESSTVaL (Field Experiment for Submesoscale Spatio-temporal
Variability in Lindenberg, https://fesstval.de) in north-east Germany, and two
weeks in July 2021 during LIAISE (Land surface Interactions with the
Atmosphere over the Iberian Semi-arid Environment, https://liaise.aeris-
data.fr/) in north-east Spain. Differences between the two campaigns in
climate, local atmospheric dynamics, time of year, and geographical location
have offered a diverse range of solar irradiance conditions to observe through
the network, resulting in a total of $\sim$ 4 weeks of spatial measurements.
The sensor network measurement strategy at each campaign is described next.

####  2.2.1 FESSTVaL

During the FESSTVaL campaign (Hohenegger et al., 2023), we deployed a network
of 20 sensors in a simple, equidistant rectangular grid. Our measurements took
place between June 14 and June 29, 2021, at the Falkenberg supersite of the
Deutscher Wetterdienst, $\sim$ 50 meters above sea level. Figure 2a shows the
4 by 5 sensor network layout with a 50 meter horizontal grid spacing. The
choice of grid spacing is a combination of aiming for something that resembles
the grid of a high resolution cloud resolving model, an a priori estimate of
the required resolution to resolve shadow/sunlight transitions, practical
constraints the Falkenberg site, and amount of sensors we had available.
Technically, the grid spacing was 49 meters due to the constraints of rolling
out the sensor network that is as little as possible obstructed by, or in the
way of, other instrumentation on the field.

We deployed two consumer action cameras (with an on-board GPS clock) at the
northern two grid corners to take time-lapse photos of the sky at a 5 second
interval, so we can relate the cloud field to surface irradiance. These
cameras were oriented up towards the sun in the south-east for the north-
western camera, and south-west for the north-eastern camera. Calibration of
the sensors is done against the Falkenberg suntracker, equipped with high
quality instrumentation and located in the south-east corner of the field.
Instrument locations relative to the grid are illustrated in Figure 2a.

In addition to this setup, 4 sensors were located several kilometers to the
west, south, east and north of the field (Figure 2b), to capture part of the
larger scale variability in the campaign area. Two other supersites discussed
in this paper are Lindenberg and Birkholz, each equipped with a microwave
radiometer for integrated water vapour measurements. There were many more
instruments deployed, see https://fesstval.de and Hohenegger et al., (2023)
for details. Our two weeks at FESSTVaL featured one fully clear-sky day, one
rainy day with thick cloud cover, but was otherwise characterized by many
different (broken) cloud covers (Figure 4a).

Figure 2: Radiometer network measurement layout at FESSTVaL. The 4 by 5 sensor
grid at Falkenberg with a horizontal spacing of $\approx$ 50 meters is shown
in (a), with the 98 meter tower, suntracker, two cloud cameras, and microwave
radiometer (MWR) locations. Additional sensors set up in the FESSTVaL campaign
area around Falkenberg are shown in (b), and includes the location of the
FESSTVaL supersites Lindenberg and Birkholz. Background satellite data: Google
© 2023.

####  2.2.2 LIAISE

Between 14 and 30 July 2021, we set up a network of sensors at the La Cendrosa
site of the LIAISE campaign (https://liaise.aeris-data.fr/). La Cendrosa is
located within an irrigated part of an otherwise semi-arid region, with
complex local, regional, and mesoscale dynamics [Mangan et al., 2023]. The
typical expected, and observed (see Figure 4b), summertime weather in this
region in north-eastern Spain is cloud-free, dry, and hot. Specifically at La
Cendrosa, due to local topography and sea breeze dynamics (locally called
’Marinada’), prevailing day time winds are westerly, shifting via a southern
sea breeze to easterly night time winds. While the goal is primarily to
observe cloud-driven irradiance variability, frequent clear-sky days offer a
good calibration opportunity and analyses in spectral variations due to day-
to-day variations in aerosol and water vapour content. We had a bit more space
to set up the network compared to Falkenberg, so in an attempt to capture
larger patterns, we decided on a grid spacing of 100 meters. With the
prevailing westerly daytime winds, we oriented the grid in a similar
direction, in hopes of tracking cloud shadows and enhancements over a length
of 400 meters, illustrated in Figure 3a. An additional group of radiometers
was set up in footprint of the the scintillometer, which in the context of
this study gives additional resolution in the grid center, but makes the total
network non-equidistant.

Two action cameras were mounted on the energy balance station (EBS), west and
east oriented, but pointed straight ahead rather than at the sun to include a
visual record of the vegetation growth and irrigation during the campaign
period. The EBS measures incoming broadband irradiance as part of the
radiation balance measurements, which is used in this study as a calibration
reference. Hourly boundary layer soundings were deployed at La Cendrosa during
Intensive Observation Periods (IOPs), which we combine with hourly full
troposphere soundings at the non-irrigated site Els Plans (Figure 3b), located
14.1 km to the south east. There are many more observations and sites within
the campaign area, which can be found on https://liaise.aeris-data.fr/.

Figure 3: Sensor network measurement layout at LIAISE. The west-east oriented
5 by 2 sensor grid at La Cendrosa has a horizontal spacing of $\approx$ 100
meters, with an additional set of 5 sensors within the footprint of the
scintillometer (Sn to Ss), shown in (a). The reference pyranometer is mounted
on the EBS (energy balance station). La Cendrosa and Els Plans are shown in
(b), which illustrated also the irrigated (green) versus non-irrigated (beige)
area. Background satellite data: Google © 2023. Figure 4: Overview of weather
variables most relevant for irradiance measurements at FESSTVaL and LIAISE.
All data is during day time (solar elevation angle $>$ 15o). Box plots are the
surface temperatures at (a) Falkenberg and (b) Els Plans. The blue bars are
accumulated precipitation during day time at the sensor network. The color
shading at the bottom is the observed percentage of clear-sky irradiance (CAMS
McClear), where darker colors indicate more cloudiness, labelled with numbers
for values $<$ 95%. Vertical gray bars at June 18 and July 16 are the
reference cloud-free calibration dates.

###  2.3 Supplementary irradiance data

Clear-sky irradiance is required for identifying cloud enhancement events in
the measurements, which we base on the CAMS McClear product [Gschwind et al.,
2019]. In addition to the solar position for a given geographical location and
time of year, this product includes 3-hourly reanalysis of the atmospheric
composition (aerosols, water vapour, and various other gases) in its
calculations of clear-sky irradiance. It is available at a 1 min resolution,
which we linearly interpolate to 1 or 10 Hz when necessary. For sensor cosine
response corrections and calibration, we need accurate solar positions (zenith
and azimuth angles), which we calculate using PySolar [Pingswept, 2022].

##  3 Sensor calibration and accuracy

This sections details quality control, the calibration process of raw
measurement data, and the derivation of global horizontal irradiance and total
column water vapour products. We discuss the main sources of measurement error
we encountered when processing the raw observations, these form the motivation
of our calibration approach and explain the measurement accuracy that we are
able to achieve. After the calibration laid out on Sections 3.2 and 3.3, we
have two main sensor network datasets for each campaign. You can find these
FROST campaign data at Mol and Heusinkveld [2022] for FESSTVaL and Mol et al.
[2023a] for LIAISE.

###  3.1 Sources of measurement error

####  3.1.1 Cosine response

Given a solar zenith angle $\rm\theta$, the horizontally measured signal
strength of a constant light source $\rm Q$ as function of $\theta$ would
ideally be $\rm Q$ $\cdot$ cos($\theta$), see also Figure 5. In practise,
there is an increasing relative underestimation of irradiance for high
$\theta$ in our instrument (Figure 5a), referred to as the cosine response,
for which we correct in post processing. While this cosine response correction
is in principle a function of cos($\theta$), the triad sensor design results
in a unique response curve for each subsensor, despite the diffuser, which is
also a function of the orientation of the sensor with respect to the sun. This
means the sensor orientation is important to keep constant throughout a
measurement campaign, and ideally all sensors in a network are placed in the
same orientation to minimize the variations among sensors. Only the variation
between subgroups in cosine response is relevant for the derivation of
spectral products, which depend on the ratio between wavelengths, discussed in
Section 3.3.

####  3.1.2 Build consistency

All instruments are hand-made, which leads to small imperfections or
inconsistencies, such as the exact distance and position of the spectrometer
from the diffuser. This results in measurable variations in build quality
among instruments, and imposes a limit on what is achievable through post
processing. While ideally there would be one universal calibration for all
instruments, we find the best overall results when calibration is fine-tuned
per individual instrument. This, combined with the other sources of error,
leads to a limitation in accuracy.

Figure 5: Schematic of surface solar irradiance measurements using FROST (a)
and a suntracker with a pyranometer and pyrheliometer (b), both located at
gray square in the axis origin. The solar elevation angle $\alpha$ depicts the
origin of direct irradiance Qdir, of which a fraction (typically 10 to 30%,
see Appendix Figure 18) scatters and results into diffuse irradiance Qdif, in
this clear-sky example. FROST does not distinguish between Qdir and Qdif, and
misses part of the signal originating from low angles $\alpha$, illustrated by
the red part of the semi circle in (a). A suntracker measures Qdir and Qdif
separately, illustrated by the 5o degree arc and semi circle in (b), and is
less biased for small $\alpha$. The blue background gradient illustrates
decreasing air density with height, htropo and habl are the approximate
tropopause and atmospheric boundary layer heights. $\Delta$t denotes the
temporal resolution of each instrument.

####  3.1.3 Cross-talk

Especially the first group of wavelengths (Figure 1) experience significant
cross-talk, meaning a sensor sensitivity to wavelengths outside the specified
range. Under worst-case conditions, e.g. the 410 nm band can receive nearly
60% of its signal from $>$ 1000 nm. While this issue is addressed in a new
version of the instrument by using certain wavelength filters over the
affected subsensors, it is a problem for the measurements presented in this
study. Some spectral bands are unaffected and therefore suitable for spectrum
analyses, as will be shown in Section 5. This cross-talk has no negative
impact for estimating broadband irradiance, as the sensitivity stays within
the spectrum of solar irradiance.

####  3.1.4 Temperature sensitivity

There are two components introducing a temperature sensitivity in the
instrument, with no significant difference across wavelengths. A small, linear
change in signal strength of -0.25% per +10 K comes from the spectrometer
itself. The teflon diffuser has a +2% jump in transmittance from 20 to 21 oC,
and slowly declines at a rate of $\approx$ -0.4% per +10 K afterwards. While
10 cm or 2 m air temperature is known for all measurements, it does not
directly translate to teflon diffuser and spectrometer temperature, making a
temperature correction not trivial. We use surface temperature
$\mathrm{T_{s}}$ (10 cm) as a proxy for qualitative assessment of measurement
quality, while keeping in mind $\mathrm{T_{s}}$ is still an underestimation.
The role of temperature in measurement accuracy is discussed further in the
Section 3.2 on broadband calibration.

####  3.1.5 Factory calibration

Sensor to sensor and wavelength band to band variations in factory calibration
accuracy are $\pm$ 10% for most, with some outliers up to $\pm$ 20%. Even
though each sensor can be treated separately, it helps to have homogeneous raw
output among sensors for a given light signal. We homogenise the factory
calibration as well as possible, shown in Section 3.3.

####  3.1.6 Maintenance and quality control

We performed irregular but frequent maintenance on the sensors during the
field campaigns, usually early or late in the day, to check whether they were
still running, level, and free of dirt or dust (birds or flies liked to sit on
some particular sensors). All data is provided with quality flags that mark
data points with bad or unreliable data quality, which is mostly due to this
sensor maintenance. In addition, temporary displacement of sensors or
obscuration of direct sunlight by nearby objects at low solar angles (trees,
crops, other instrumentation) is flagged.

###  3.2 Global horizontal irradiance

Broadband solar irradiance is the total shortwave surface solar irradiance,
often called global horizontal irradiance (GHI). GHI is measured using
pyranometers, or in combination with pyrheliometers (Figure 5b), and typically
has a spectral range encompassing the full shortwave irradiance spectrum, i.e.
200 to 4000 nm [Kipp & Zonen, 2004]. This range exceeds the spectral range
covered by the 18 wavelength bands of our sensor (410 to 940 nm, a bit more
including the cross-talk sensitivity), but these wavelengths cover the most
energetic part of the spectrum (Figure 1). We derive the GHI by first taking
the mean of all spectral bands (Figure 1), correcting for the cosine response,
and converting raw measurement units, counts bin-1 $\Delta t^{-1}$, to W m-2.
This calibration generalises under the assumption that the spectrum shape of
short-wave irradiance remains constant.

For both FESSTVaL and LIAISE, we have at least one clear-sky day with high
quality reference GHI measurements available from pyranometers to calibrate
against. The cosine response is a function of solar zenith angle, so we take
the ratio between the spectral average of our instrument and the reference GHI
measurement as a function of this angle. Figure 6a shows for a single
radiometer at FESSTVaL the resulting ratio curve relative to the suntracker
reference for the clear-sky day (June 18, 2021). The best curve fit is
manually extended below solar elevation angles of 15 degrees, because here the
absolute signal gets too low and relative measurement errors, including that
of the pyranometer cosine response, amplify. Especially differences between
the cosine response of each subgroup become more pronounced at such low
angles. Furthermore, when the sensor is not perfectly level, measured incoming
irradiance between morning and evening for the same solar elevation angle
introduces an asymmetry, which explains part of the hysteresis effect visible
in Figure 6a. We find that a fitting technique that takes the solar azimuth
angle into account and fits on subgroups seperately does not generalize beyond
the calibration data, and thus does not improve data accuracy for days with
clouds. This is likely due to slight variations in sensor orientation from day
to day, and different combinations of solar elevation and azimuth angle as the
Earth’s orbit around the Sun progresses. Figure 6b illustrates the resulting
time series when the reference date best fit is applied to a different day
(June 27, 2021). This examples shows sensor 3 captures both the daily cycle
and cloud driven fluctuations.

Figure 6: Broadband global horizontal irradiance calibration example.
Calibration factor as function of solar elevation angle between raw instrument
data and the irradiance data from the Falkenberg suntracker for June 18 is
shown in (a). The best fit curve is manually extended at the lower and upper
end of the data range (dashed line), and follows the function with best fit
coefficients between 15 and 60 degrees elevation angle. Time at the bottom is
in UTC for morning until noon. The scatter colors indicate the solar azimuth
angle. In (b), the best fit calibration from (a) is applied to measurements of
June 27, and compared again to the Falkenberg suntracker reference. All FROST
data shown is for sensor 3, resampled to 1 minute.

We apply this calibration routine to all data for each sensor individually,
for both campaigns separately. Figures 7 give an overview of the performance
of all grid sensors across all campaign days for FESSTVaL and LIAISE, compared
to their respective reference station. Overall, the bias is within 2%, and
mean absolute error between 5 to 15 W m-2 for most days and sensors. Notable
outliers, June 22 or July 26 and 27, are explained by overcast and rainy
weather (Figure 4), which leads to increased absorption in mostly near-
infrared wavelength bands that our instrument does not sample. GHI is thus
overestimated, because clear-sky conditions with less relative absorption in
the near-infrared are the basis for calibration, violating our assumption that
the solar spectrum shape is constant under all conditions. Sensor to sensor
variations are typically smaller or equal to the error with respect to the
reference station, and likely originate from variations in construction and
minor day to day changes in sensor orientation. Changes in calibration quality
due to the sensors’ temperature sensitivity are small, as for most days the
measured 10 cm temperature, and therefore the teflon diffuser temperature, was
well above the 2% signal jump at 20 degrees Celsius (Figure 4).

Figure 7: Global horizontal irradiance calibration performance of all sensors
for FESSTVaL and LIAISE. Panels (a, b) show the mean bias of daily sums of
irradiance compared to a reference pyranometer for all sensors, and panels (c,
d) the mean absolute error. Vertical shading indicates the reference clear-sky
calibration date for FESSTVaL in (a, c) and LIAISE in (b, d). Box plots span
the 5-95th percentile range of the spread among sensors. Only data with solar
elevation angle above 15 degrees is used for verification.

###  3.3 Total column water vapour

Because of limitations of the spectral quality in the sensor version used in
this research, it is not easy to derive reliable measurements in W m-2 nm-1
(see Section 2.1). However, ratios between certain wavelengths contain
valuable information still, and can be done in native sensor units, as will be
shown shortly for the spectral signature of water vapour. Since the factory
calibration accuracy varies significantly at wavelength level (Figure 8a), the
ratios between certain wavelengths at identical atmospheric conditions will
vary from sensor to sensor. This noise is reduced from $\sim$ 20% to $\sim$
1-2%, by homogenizing all sensors at wavelength level using a reference
period. The result applied to a different date is shown in Figure 8b.
Variations from sensor to sensor are still at $\pm$ 2%, similar to the
calibration accuracy for GHI discussed in Section 3.2. This puts a limit on
how accurate any wavelength-ratio derived variable might be in the absence of
high quality reference data. This dataset, which we label as ’precalibration’,
is the starting point further analyses.

Figure 8: Improving the factory calibration of FROST spectra. Factory
calibrated wavelength level spread among sensors shown in (a). Improved
calibration coefficients are derived from the training data (June 18, 7:28 to
14:48 UTC) in (a). The application to test data (June 27 data 7:30 - 9:00 UTC)
is shown in (b). Both training and test data are under clear-sky conditions
with a solar elevation angle above 40 degrees, at FESSTVaL. Note the values in
(b) are an order of magnitude smaller.

####  3.3.1 Deriving total column water vapour

Total column water vapour (TCWV) in the atmosphere is indirectly measurable
using water vapour absorption bands, where heightened levels of water vapour
result in significantly reduced signal within absorption bands compared to
other wavelengths. Figure 1 highlights two of such bands for water, at 940 and
1130 nm. A strong absorption band is captured by the 940 nm channel of FROST,
which in theory means we are able to detect changes in atmospheric moisture
when comparing the 940 nm signal to a reference channel outside the absorption
band. Choosing a reference channel is limited by some sensor design and
performance limitations. The cosine response varies between the three
subgroups, such that changes in the ratio between two channels (e.g. 940 vs.
860 nm) are partially a result of instrument imperfections (see also Section
2.1) rather than changes in absorption strength. The choice of suitable
channels is further limited by cross-talk at the shorter wavelengths of the
subsensor with 940 nm, making the 900 and 940 nm channels the best options,
despite 900 nm also partially being in a (weak) absorption band. Figure 9
illustrates the signal of 900 and 940 nm, where early in the day the ratio
under clear-sky conditions is lower (more absorption) than noon due to the
longer path length of irradiance. Shading from (semi-transparent) cumuli also
give distinct absorption signals, though seemingly only for the most optically
thick cumulus passages in this example. This might be due to the change in
diffuse/direct partitioning, but any stronger statement requires a more
careful analysis. But this highlights why only clear-sky conditions are
suitable for estimating TCWV, as we currently can not separate the effect of
liquid water (or ice) from that of water vapour. One more limitation is the
signal strength for the individual bands being low, even at high solar
elevation angles. For a clear signal we therefore take a moving average of 120
seconds or more, thereby effectively reduce the temporal resolution. In a new
version of FROST, the cross-talk and weak signal issues have been addressed
[Heusinkveld et al., 2022].

Figure 9: Spectral signature of water vapour measured by the 900 and 940 nm
bands of FROST. The time series shows part of a daily cycle for a single
sensor in the Falkenberg network, which is clear-sky until 10 UTC, but
features brief cumulus passages thereafter. The ratio of the 940 nm / 900 nm
bands is shown in grey on the right axis.

In Figure 10a we show we can derive an absorption signal that is highly
correlated to accurate reference measurements of TCWV. Higher values in the
940 nm / 900 nm ratio indicate less TCWV, so by flipping the y-axis this
correlation (r2 = 0.75) is more clearly visible. The reference TCWV
measurement comes from a co-located microwave radiometer [Löhnert et al.,
2022], with a 5 minute moving average applied to both time series to get a
comparable signal. The microwave radiometer measures along a single straight
vertical path (Figure 14a), but the signal our sensors measure is a function
of the path that light travels through the atmosphere, mostly from direct
irradiance (Figure 5a). For fitting a model to the data we therefore include
the atmospheric mass fraction (AMF), i.e. the path length light travels
through the atmosphere given a solar elevation angle $\alpha$. This also means
that the light our instruments measure has a horizontal footprint of several
kilometers, whereas the microwave radiometer is a vertical integral
measurement at one fixed horizontal point. Furthermore, under clear-sky
conditions, diffuse irradiance still typically constitutes about 15 to 30% of
GHI (see Figure 18 in the Appendix), and would have travelled a longer
distance through the atmosphere (by definition, as it is scattered light
compared to direct light). This effect is implicitly taken into account, as
the diffuse fraction is also a function of solar elevation angle, and thus the
atmospheric mass fraction. We expect some bias in our model fit for extremely
clear or hazy days, but we have no clear signal or quantification of its
effect within our observational dataset.

The relationship between TCWV, AMF, and measured water vapour absorption (WVA)
is captured by a function of the form f(x, y) = a x + b + c y2 \+ d y + e x y,
where x = WVA and y = AMF. Figure 10b illustrates the best fit of this model,
based on 30-minute averages of all available clear-sky data during FESSTVaL
for sensor 11 at Falkenberg compared to the microwave radiometer. Measurement
uncertainties are based on the standard deviation within the 30-minute
windows.

Figure 10: Total column water vapour (TCWV) derived from the 940 / 900 nm band
ratio. In (a), an example time series of measured spectral signal compared to
TCWV from the microwave radiometer [Löhnert et al., 2022] is shown for June 18
at Falkenberg (FESSTVaL). In (b), the scatters represent 30-minute averages
and standard deviations of the measured spectral signal and atmospheric mass
fraction, coloured by the TCWV from the microwave radiometer. The results from
the model best-fit are shown in the curved shading.

Since there is a limitation to the accuracy of the spectra from sensor to
sensor and within sensors (Section 3.3), we find no gain by training the model
from data of all sensors together, or calibrating sensors individually as we
do for GHI, likely to due over-fitting. Instead, we apply the best fit based
on sensor 11 at FESSTVaL, and use the uncertainty in the ratio between
spectral bands as an error estimation. Similar to broadband irradiance
estimates, one could fine-tune the calibration on a case-specific basis if a
high quality reference is available. We estimate uncertainty by taking the
standard deviation between all sensors in a network, which results in $\pm$
0.5 kg m-2 for sufficiently high solar elevation angles ($\alpha$ $>$ 30o),
and increases towards sunset and sunrise. Analyses of moisture variability at
FESSTVaL and LIAISE using this spectrally derived TCWV dataset are shown in
Section 5.

##  4 Spatial patterns of surface solar irradiance

To get a first impression of cloud-driven patterns of surface solar
irradiance, we plot the sensor network directly on a map combined with the
cloud camera images, and render frames for each time step to create a video.
For June 27, 2021, a day with cumulus clouds at FESSTVaL, such a video is
available on https://vimeo.com/827602111 (or supplementary material). One
frame is displayed here in Figure 11, but in particular the video shows how
the sensor network captures irradiance patterns made by the dynamic cumulus
field. Some interesting features are how some cloud passages show a clear
temporary increase in diffuse irradiance, while for others the diffuse
irradiance remains constant but slightly elevated. There is an extra step on
complexity once cirrus fields pass over, superimposing their effect onto that
of the boundary layer cumuli. These are observations are difficult to
illustrate or analyse in the static format of a paper, though we will attempt
to do so by post-processing three examples into a kind of time-integrated
visualisation, explained in the next section.

Figure 11: Network measurements of GHI combined with cloud imagery for June
27, 2021, at Falkenberg (FESSTVaL campaign). The time series (c) are centered
around the 11:00 UTC snapshot, and features diffuse irradiance from the
Falkenberg sun tracker, clear-sky GHI, and the FROST network measurements
(spatial mean and min/max range). The network measurements are plotted
relative to clear-sky GHI (CAMS McClear) in an absolute (d) and relative way
(e). For an animated version, see supplementary material or
https://vimeo.com/827602111.

###  4.1 Increasing domain size and spatial resolution

What is clear from the video (or frame in Figure 11), is that cloud shadow and
enhancement patterns both exceed the network size and have details finer than
the network resolution. We apply a data processing technique that makes use of
the high temporal resolution and an estimate of the cloud velocity in order to
increase the effective network size and spatial resolution. Effectively, the
following technique is a way to merge many snap shots of the network
measurements into a single figure. The technique assumes clouds retain their
shape (analogous to Taylor’s hypothesis of frozen turbulence) when moving over
the measurement network, schematically shown in Figure 12 for a 1D time series
example. We can ’advect’ the FESSTVaL Falkenberg network of 20 sensors in
space (Figure 2 or 11), for example with a time step of 5 s and 5 m s-1 cloud
velocity for 10 steps, resulting in effectively a spatial network of 200
points (20 sensors $\times$ 10 steps) that spans $\sim$ 450 m (5 s $\times$ 5
m s-1 $\times$ 10 steps + $\sim$ 200 m) in the advection direction and has
approximately double the resolution (one step is 25 m). The virtual 200 points
can then be interpolated to a new equidistant grid for a easier comparison and
visualization. The main challenge with this technique is the determination of
the cloud velocity vector, which we initially base on wind speed at cloud
level from nearby soundings, and then manually fine-tune through trial and
error to produce a smooth looking result. The cloud velocity step fine-tuning
needs to be repeated frequently, every 10 to 30 minutes or so, due to wind and
cloud velocity (subtly) changing or simply varying among clouds. Keeping the
total integration time short also minimizes our violation of the static cloud
shapes assumption. We apply this technique to three distinct cases, showcased
in Figure 13, using a time integration step of 5 seconds, and horizontal
interpolation resolution of 5 meters.

Figure 12: Schematic illustration of a 1D translation from a single point time
series (a) to a spatial pattern (b), assuming the cloud velocity is known and
the cloud’s shape is constant in time. At t0, a sensor is underneath a cloud
shadow, preceded by cloud enhancement at the front side of the cloud at t-1,
and followed up by a (weaker) cloud enhancement after the passage at t1. The
asymmetry in enhancement is commonly, but not always, seen in reality, and
used here for illustrative purposes. The reconstructed spatial pattern in (b)
has effectively transformed a single point measurement into a three point
measurement (x-1, x0, x1) using the temporal information. Figure 13: Spatial
patterns of cloud enhancements and shadows for three cases: cumulus (a-c),
altocumulus (d-f), and cirrus (g-i). (a, d, g) show a representative photo.
The time series include grid measurements (FROST network at $\Delta$t = 1 s),
clear-sky GHI (CAMS McClear at $\Delta$t = 60 s), and diffuse irradiance
(Falkenberg sun tracker (c, f) at $\Delta$t = 60 s or La Cendrosa energy
balance station at $\Delta$t = 5 s (i)). Data in (b, e, h) are plotted
relative to clear-sky irradiance. The temporal range of each pattern segment
is labelled in HH:MM:SS on top.

###  4.2 Three pattern examples

The three cases, boundary level cumulus, mid level altocumulus, and high level
cirrus, give an impression of how distinctly different surface patterns can be
between cloud types, illustrated in Figure 13. Case 1 is from the same date
and time as Figure 11, and features slow-moving (4.5 m s-1) fair weather
cumuli at FESSTVaL (June 27, 2021), with cloud bases at $\approx$ 1750 meters,
and a relatively clean atmosphere (deep blue sky). Cloud enhancement to shadow
transitions are very sharp, about 40 to 60 meters as estimated from Figure
13b, making cumulus cloud shadows slightly smaller than their size with a
length scale similar to what we found in Mol et al. [2023c] based on long-term
time series. Diffuse irradiance does not vary significantly on the
spatiotemporal scales of this example, suggesting the total light scattering
off of clouds in this case has a very wide horizontal range without much
contribution to the total enhancement from local forward scattering at
transparent cloud edges. We think this may be due to the relatively high cloud
base, where diffuse forward scattering is spread out over a large surface area
rather than more locally focused. Around the smaller cloud fragments (11:00 to
11:05 UTC, bottom pattern in Figure 13b), we observe increases in cloud
enhancement of $\approx$ 30 to 40 W m-2 (4% of clear-sky). The contrast
between cloud enhancement and shadow is large, approximately 750 W m-2, or 80%
of clear-sky irradiance. Some of the artefacts arise from small biases between
sensors, such as the horizontal stripes at 10:50 and 11:10 UTC, or incorrect
cloud velocity and changing cloud shape, such as the noisy pattern around
11:06 UTC.

The second case (Figure 13d) is of an altocumulus field at 5 km altitude
(estimated using the ceilometer of Falkenberg), moving at 14.3 m s-1, and
under overall hazier conditions than the first case. Individual altocumulus
clouds are about 1 to 5 times the area of the sensor network based on the
distance between cloud enhancement peaks, whereas these peaks are up to
$\approx$ 200 m in diameter and thus mostly fit within the network area. Apart
from the spatial scales being significantly smaller than the cumulus case, the
shadow patterns are weak (250 W m-2 or 35% below clear-sky), and cloud
enhancements very strong, locally more than 300 W m-2 or 40% above clear-sky.
The mechanism appears to be a consistently high diffuse irradiance (500 W m-2,
two-thirds of clear-sky) generated by forward scattering in the semi-
transparent altocumulus field as a whole, with superimposed gaps in the clouds
that let up to 650 W m-2 of direct irradiance briefly pass through. These are
thus particularly extreme variations at small spatiotemporal scales, with a
similar relative magnitude (cloud shadow to enhancement contrast) to that of
the previous example with cumulus clouds.

The third and last case (Figure 13g) is on the other end of the spectrum,
featuring an optically thick cirrus field moving over the LIAISE network setup
at 35 m s-1 (July 25, 2021). Only weak spatial patterns of about 500 meters in
length are visible in some parts of the cirrus (8:43-8:44 and 8:55-8:56 UTC,
Figure 13h). Despite the high cloud velocity, it takes 15 minutes to go from a
120 W m-2 or 18% of cloud enhancement (8:39 UTC) to a near complete blocking
of direct irradiance (8:54 UTC). The area of influence of this patch of cirrus
far exceeds the radiometer grid size, with the transition from cloud
enhancement to shadow minimum covering 35 m s-1 $\times$ 900 s = 31.5 km. The
scale of this pattern suggests errors made by radiative transfer models using
the independent column approximation (no 3D effects) will already become
apparent in coarser resolution medium range numerical weather prediction
(e.g., ECMWF’s IFS at $\sim$ 9$\times$9 km2), not just in cloud resolving
models.

##  5 Patterns driven by water vapour variability

In this section, we will look at two examples of variability in atmospheric
moisture as measured by the FROST network at FESSTVaL and LIAISE. These
examples have a scope that is more towards regional scale (campaign area)
land-atmosphere coupling and atmospheric dynamics under clear-sky conditions,
rather than cloud-driven irradiance variability.

Figure 14: Schematic of total column water vapour measurements using a
microwave radiometer (MWR) (a) and radiosondes (b). The MWR is an active
instrument, shooting a vertically oriented laser beam, and measures specific
humidity through backscatter. To get the same measurement from soundings at La
Cendrosa, shown in (b), the boundary layer soundings (left, orange) are
extended with tropospheric soundings (right, red) of Els Plans, 14.1 km apart
and at slightly higher elevation. The colored cones illustrate the potential
horizontal displacement with respect to the initial position, with the solid
curved lines an example path a sounding could take. Measurement duration is
approximately 10 to 15 minutes or 1 hour, respectively. The remainder of this
plot is similar to Figure 5.

###  5.1 Sub-mesoscale water vapour variability at FESSTVaL

June 18 at FESSTVaL was a clear-sky, warm summer day (maximum 2 m temperature
of 31 oC). There are intra-hourly variations in TCWV of 1-3 kg m-2, as
illustrated in Figure 15 for various locations within the campaign area. With
a predominantly southerly wind of $\approx$ 9 m s-1 average over the lower
troposphere, we tried to track moisture patterns across a 10 km south-north
transect in the FESSTVaL domain (Figure 2b), south of Falkenberg to
Lindenberg, at a 10 and 20 minute time lag respectively. The southern and
Falkenberg measurements are based on the IWV derived from our instruments,
with at Falkenberg and Lindenberg two high quality measurements taken from the
microwave radiometers. While our instruments and the microwave radiometers at
Falkenberg are in agreement on the local variations (despite a bias of $\sim$
0.5 kg m-2), there appears to be no correlation between the southern and
northern locations in the variability. We therefore think that the temporal
variability in TCWV is more locally driven by turbulence structures in the
convective boundary layer rather than the advection of (sub)-mesoscale
horizontal patterns in moisture. The more subtle differences in variability
between irradiance and microwave radiometer based measurements may be
explained by the larger footprint measurement of FROST sampling different
moisture plume structures, given its horizontal component in the diagonal
cross-section through the atmosphere (Figure 5a).

Figure 15: Time series of total column water vapour (TCWV) from a south to
north transact in the FESSTVaL campaign area. FROST-derived tcwv is compared
to high quality microwave radiometer measurements, both resampled to 15 minute
moving averages. The shading is the FROST grid standard deviation. The south
to north transact (south - Falkenberg - Lindenberg) is approximately 10 km,
with Falkenberg in the middle (see also Figure 2b), and the wind was $\approx$
10 m s-1 from the south.

###  5.2 Variability in water vapour at LIAISE

During the LIAISE campaign, most days were clear-sky (Figure 4), which gives
ample data to study intra-day variability in atmospheric moisture. The
campaign area is characterized by complex meteorology due to a combination of
topography, strong heterogeneity in surface fluxes at various scales due to
irrigation (Bowen ratio between 0.01 and 30, Mangan et al. [2023]), and late
afternoon south-eastern sea breeze dynamics with varying timing and strength.
This collection of complex factors affecting local weather is not captured by
high resolution weather models, and even less so by the one ERA5 grid cell
covering the campaign area. Measuring TCWV helps to identify internal boundary
layers and moisture plumes at the local scale to synoptic scale advection,
that both contribute to variability. We apply the best fit derived from
FESSTVaL (Figure 10) to the sensor network at the irrigated La Cendrosa site.
Figure 16 shows the resulting time series for four (clear-sky) IOP days
compared against ERA5 and hourly radiosondes, using a 3 minute moving average
to get a clear signal. FROST-derived TCWV time series and soundings do not at
all agree with the magnitude and trends of ERA5, which highlights the
difficulty (coarse) models have with the complex meteorology in the LIAISE
domain. Radiosondes and our spectrally derived TCWV time series are in much
better agreement overall, and provides a good validation of our calibration
methodology. Derivation of TCWV from radiosondes is precarious, though, since
the hourly boundary layer radiosondes at La Cendrosa only reach to about 1.5 -
4 km and need to be supplemented with hourly tropospheric radiosondes from Els
Plans, a non-irrigated location 12.5 km to the south-east. The atmospheric
conditions in Els Plans and La Cendrosa converge above their respective local
boundary layers to a regional atmospheric profile, described by a blending
height of approximately 1.5 km [Mangan et al., 2023]. For each sounding in La
Cendrosa, we supplement its information the mid to upper tropospheric data
gathered from the closest (in time) Els Plans sounding, as schematically
illustrated in Figure 14b). Timing and footprint differences between
instantaneous spectrum-derived TCWV and hourly combined soundings are, we
believe, the main reason for differences between their measurements.

Figure 16: Time series of total column water vapour (tcwv) at La Cendrosa
during LIAISE. Irradiance based measurements are compared to ERA5
(interpolated to La Cendrosa), and hourly soundings of La Cendrosa combined
with Els Plans. The uncertainty range of the FROST measurements is the
standard deviation among sensors in the network. The four dates are ideal
clear-sky days and official Intensive Observation Periods.

In terms of variability, there seem to be various time scales at play. On
hourly time scales, trends in TCWV can be up to 5 kg m-2 h-1, whereas at the
minute time scales the constant fluctuations do not exceed 1 kg m-2. The
latter we believe to be driven by moist boundary layer thermals and subsequent
dry air entrainment. Daily cycles of boundary layer drying through local
advection [Mangan et al., 2023] and synoptic scale advection of air masses
(captured by ERA5), explain the multi-hour and day to day variations,
respectively. Figure 17 illustrates a detailed look at short term variability
of TCWV, including an analysis in which we attempt to correlate variability of
measured specific humidity at 45 meters ($\rm q_{45\rm m}$) in the well mixed
boundary layer to variation in TCWV. Both time series are resampled to a
common resolution of 2 minutes. To exclude hourly or daily trends, we define
variations in TCWV or $\rm q_{45\rm m}$ as the deviations with respect to
their 60 minute moving average. The boundary layer contains a significant
portion of the total vertical moisture, but the resulting correlation is only
very weak, with an r2 of 0.17 (Figure 17b), and thus can’t be explained by
local moist or dry plumes. Rather, the TCWV from FROST has larger scale
variations due to its diagonal cross-sectional footprint, as compared to a
single point measurement. Figure 17c shows the vertical structure of $\rm q$
for a single point in an idealized convective boundary layer simulation in
north-eastern Spain, where the complex turbulence structures illustrate why
variations between $\rm q_{45\rm m}$ and TCWV do not correlate. FROST is thus
able to capture moisture variability that is representative for the kilometer
scale and above, and not at individual field level. In an improved version of
FROST, we have stronger better signal to noise performance and thus require
shorting time averaging, which may improve its ability to capture more small
scale variations. This could offer a flexible, low-cost alternative to a
microwave radiometer or soundings.

Figure 17: Moisture variability of total column compared to 45 meters at La
Cendrosa. (a) shows a detailed time series of the same data as Figure 16b,
combined with specific humidity at 45 meters above ground level. The
correlation between deviations from hourly mean values for both total column
water vapour and specific humidity at 45 meters are shown in (b). To
illustrate the difference between total column and single point moisture, (c)
shows a (time, height) cross section of a convective boundary layer from large
eddy simulation.

##  6 Conclusions and Outlook

Using low-cost radiometers calibrated against high-quality reference stations,
we have gathered two high-resolution, spectrally resolving datasets of surface
solar irradiance. In combination with supplementary observations, such as
cloud cameras at FESSTVaL and soundings at LIAISE, we are able to derive
insights into spatial patterns of surface irradiance caused by various types
of clouds and atmospheric moisture variability. This work demonstrates how
low-cost instruments can provide accurate and detailed spatial measurements
and, given their flexible setup, can be an effective addition to field
campaigns, particularly in complex areas.

Boundary layer cumulus clouds, mid level altocumulus, and high level cirrus
all create distinctly different but large spatiotemporal variability at the
surface at scales between 50 m and 30 km. All three presented cases have
distinctly different surface irradiance patterns, spatiotemporal scales of
variability, cloud types, direct/diffuse partitioning, and mechanism through
which these patterns are formed. None of these commonly occurring cases are
well-represented by state-of-the-art numerical weather prediction, nor by most
cloud-resolving models in academic setups due to simplifications in radiative
transfer calculations. Despite some limitations of our sensor, we are also
able to capture variability in the irradiance spectrum that arises from
significant changes in variations of atmospheric moisture. These local
variation are often larger than synoptic scale moisture advection, and
illustrate heterogeneity in moisture fluxes and boundary layer turbulence.

The presented datasets provide observations of cloud- and moisture-driven
irradiance variability that can guide the development of radiative transfer
variability parameterizations and better constrain the input for land-surface,
photosynthesis, or dynamic vegetation models that are currently driven by
incorrect irradiance distributions. A more comprehensive understanding of
cloud-driven irradiance variability will require analyses of many more clouds
and irradiance patterns in these datasets, possibly aided by cloud resolving
models and 3D ray tracing to quantitatively characterize the mechanisms.

##  7 Open Data

All data measured with FROST sensors at FESSTVaL and LIAISE are available at
the locations listed below. These include ready to use calibrated quality
controlled data, and the raw instrument data, which requires pyranometer
calibration references. Additional data used in this study is listed as well.

  * •

Radiometer data FESSTVaL [Mol and Heusinkveld, 2022]:
https://doi.org/10.25592/uhhfdm.10272, new version in the process of uploading

  * •

Radiometer data LIAISE [Mol et al., 2023a]:
https://doi.org/10.5281/zenodo.7966437

  * •

FESSTVaL Microwave Radiometer [Löhnert et al., 2022]:
https://doi.org/10.25592/uhhfdm.10197

  * •

La Cendrosa solar irradiance is not yet available at the time of writing, but
will be available here in due time: https://liaise.aeris-data.fr/page-
catalogue/?uuid=d9608a55-b836-427b-a186-e007462012b9.

  * •

Els Plans radiosondes [Price, 2023]: https://doi.org/10.25326/429

  * •

La Cendrosa radiosondes [Canut and Garrouste, 2022]:
https://doi.org/10.25326/322

  * •

Code to process raw data, analyse data, and generate publication figures:
https://zenodo.org/record/7992184

##  8 Author contributions

W.M. has designed and operated the sensor network measurements at both
campaigns, processed and analyzed the data, and wrote the manuscript. B.H. has
designed and calibrated the sensors, and helped with the measurements at
FESSTVaL. M.V. has helped with sensor maintenance at FESSTVaL, MR. M. and O.H.
have led the Dutch team at LIAISE and helped with analyses of LIAISE
observations. C.v.H. has helped with designing the fieldwork, sensor
deployment at FESSTVaL, analyzing results, and writing the final manuscript.
All authors have contributed to the final manuscript.

##  9 Acknowledgements

Our thanks goes out to the FESSTVaL and LIAISE campaign organizers and host
institutes for enabling us to gather observations alongside many others,
without which this work would not have been possible. W.M., B.H., M.V, and
C.v.H, acknowledge funding from the Dutch Research Council (NWO) (grant:
VI.Vidi.192.068).

## References

  * Canut and Garrouste [2022] G. Canut and O. Garrouste.  Liaise_la-cendrosa_cnrm_rs_l2.  2022\.  doi: 10.25326/322.  URL https://liaise.aeris-data.fr/liaise-product?uuid=a5403106-755e-46d2-a2d1-c35bb4352c3c.
  * Dirnberger et al. [2015] D. Dirnberger, G. Blackburn, B. Müller, and C. Reise.  On the impact of solar spectral irradiance on the yield of different PV technologies.  _Solar Energy Materials and Solar Cells_ , 132:431–442, Jan. 2015.  ISSN 0927-0248.  doi: 10.1016/j.solmat.2014.09.034.  URL https://www.sciencedirect.com/science/article/pii/S0927024814005169.
  * Driemel et al. [2018] A. Driemel, J. Augustine, K. Behrens, S. Colle, C. Cox, E. Cuevas-Agulló, F. M. Denn, T. Duprat, M. Fukuda, H. Grobe, M. Haeffelin, G. Hodges, N. Hyett, O. Ijima, A. Kallis, W. Knap, V. Kustov, C. N. Long, D. Longenecker, A. Lupi, M. Maturilli, M. Mimouni, L. Ntsangwane, H. Ogihara, X. Olano, M. Olefs, M. Omori, L. Passamani, E. B. Pereira, H. Schmithüsen, S. Schumacher, R. Sieger, J. Tamlyn, R. Vogt, L. Vuilleumier, X. Xia, A. Ohmura, and G. König-Langlo.  Baseline Surface Radiation Network (BSRN): structure and data description (1992–2017).  _Earth System Science Data_ , 10(3):1491–1501, Aug. 2018.  ISSN 1866-3508.  doi: 10.5194/essd-10-1491-2018.  URL https://essd.copernicus.org/articles/10/1491/2018/.  Publisher: Copernicus GmbH.
  * Durand et al. [2021] M. Durand, E. H. Murchie, A. V. Lindfors, O. Urban, P. J. Aphalo, and T. M. Robson.  Diffuse solar radiation and canopy photosynthesis in a changing environment.  _Agricultural and Forest Meteorology_ , 311:108684, Dec. 2021.  ISSN 0168-1923.  doi: 10.1016/j.agrformet.2021.108684.  URL https://www.sciencedirect.com/science/article/pii/S0168192321003701.
  * Gschwind et al. [2019] B. Gschwind, L. Wald, P. Blanc, M. Lefèvre, M. Schroedter-Homscheidt, and A. Arola.  Improving the McClear model estimating the downwelling solar radiation at ground level in cloud-free conditions – McClear‑v3.  _metz_ , 28(2):147–163, June 2019.  ISSN 0941-2948.  doi: 10.1127/metz/2019/0946.  URL http://www.schweizerbart.de/papers/metz/detail/28/90593/Improving_the_McClear_model_estimating_the_downwel?af=crossref.
  * Gueymard [2017] C. A. Gueymard.  Cloud and albedo enhancement impacts on solar irradiance using high-frequency measurements from thermopile and photodiode radiometers. Part 1: Impacts on global horizontal irradiance.  _Solar Energy_ , 153:755–765, Sept. 2017.  ISSN 0038092X.  doi: 10.1016/j.solener.2017.05.004.  URL https://linkinghub.elsevier.com/retrieve/pii/S0038092X1730381X.
  * Heusinkveld et al. [2022] B. G. Heusinkveld, W. B. Mol, and C. C. van Heerwaarden.  A new accurate low-cost instrument for fast synchronized spatial measurements of light spectra.  _EGUsphere_ , pages 1–23, Sept. 2022.  doi: 10.5194/egusphere-2022-726.  URL https://egusphere.copernicus.org/preprints/2022/egusphere-2022-726/.  Publisher: Copernicus GmbH.
  * Hogan and Bozzo [2018] R. J. Hogan and A. Bozzo.  A Flexible and Efficient Radiation Scheme for the ECMWF Model.  _Journal of Advances in Modeling Earth Systems_ , 10(8):1990–2008, 2018.  ISSN 1942-2466.  doi: 10.1029/2018MS001364.  URL https://onlinelibrary.wiley.com/doi/abs/10.1029/2018MS001364.  _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1029/2018MS001364.
  * Jakub and Mayer [2017] F. Jakub and B. Mayer.  The role of 1-D and 3-D radiative heating in the organization of shallow cumulus convection and the formation of cloud streets.  _Atmospheric Chemistry and Physics_ , 17(21):13317–13327, Nov. 2017.  ISSN 1680-7316.  doi: 10.5194/acp-17-13317-2017.  URL https://acp.copernicus.org/articles/17/13317/2017/.  Publisher: Copernicus GmbH.
  * Kipp & Zonen [2004] Kipp & Zonen.  CM22 precision pyranometer instruction manual, 2004.  Retrieved 26-07-2022 from https://www.kippzonen.com/Download/55/CM-22-Pyranometer-Manual.
  * Kirsch et al. [2022] B. Kirsch, C. Hohenegger, D. Klocke, R. Senke, M. Offermann, and F. Ament.  Sub-mesoscale observations of convective cold pools with a dense station network in Hamburg, Germany.  _Earth System Science Data_ , 14(8):3531–3548, Aug. 2022.  ISSN 1866-3508.  doi: 10.5194/essd-14-3531-2022.  URL https://essd.copernicus.org/articles/14/3531/2022/.  Publisher: Copernicus GmbH.
  * Kreuwel et al. [2021] F. P. M. Kreuwel, W. B. Mol, J. Vilà-Guerau de Arellano, and C. C. van Heerwaarden.  Characterizing solar PV grid overvoltages by data blending advanced metering infrastructure with meteorology.  _Solar Energy_ , 227:312–320, Oct. 2021.  ISSN 0038-092X.  doi: 10.1016/j.solener.2021.09.009.  URL https://www.sciencedirect.com/science/article/pii/S0038092X21007593.
  * Liang [2017] X. Liang.  Emerging Power Quality Challenges Due to Integration of Renewable Energy Sources.  _IEEE Transactions on Industry Applications_ , 53(2):855–866, Mar. 2017.  ISSN 1939-9367.  doi: 10.1109/TIA.2016.2626253.
  * Lohmann et al. [2016] G. M. Lohmann, A. H. Monahan, and D. Heinemann.  Local short-term variability in solar irradiance.  _Atmospheric Chemistry and Physics_ , 16(10):6365–6379, May 2016.  ISSN 1680-7316.  doi: https://doi.org/10.5194/acp-16-6365-2016.  URL https://acp.copernicus.org/articles/16/6365/2016/.  Publisher: Copernicus GmbH.
  * Löhnert et al. [2022] U. Löhnert, C. Knist, T. Böck, and B. Pospichal.  Microwave radiometer observations during fesstval 2021.  2022\.  doi: 10.25592/UHHFDM.10197.  URL https://www.fdr.uni-hamburg.de/record/10197.
  * Lohou and Patton [2014] F. Lohou and E. G. Patton.  Surface Energy Balance and Buoyancy Response to Shallow Cumulus Shading.  _Journal of the Atmospheric Sciences_ , 71(2):665–682, Feb. 2014.  ISSN 0022-4928, 1520-0469.  doi: 10.1175/JAS-D-13-0145.1.  URL https://journals.ametsoc.org/view/journals/atsc/71/2/jas-d-13-0145.1.xml.  Publisher: American Meteorological Society Section: Journal of the Atmospheric Sciences.
  * Mangan et al. [2023] M. R. Mangan, O. Hartogensis, A. Boone, O. Branch, G. Canut, J. Cuxart, H. J. de Boer, M. Le Page, D. Martínez-Villagrasa, J. R. Miró, J. Price, and J. Vilà-Guerau de Arellano.  The surface-boundary layer connection across spatial scales of irrigation-driven thermal heterogeneity: An integrated data and modeling study of the LIAISE field campaign.  _Agricultural and Forest Meteorology_ , 335:109452, May 2023\.  ISSN 0168-1923.  doi: 10.1016/j.agrformet.2023.109452.  URL https://www.sciencedirect.com/science/article/pii/S0168192323001442.
  * Mol and Heusinkveld [2022] W. Mol and B. Heusinkveld.  Radiometer grid at falkenberg and surroundings, downwelling shortwave radiation, fesstval campaign.  2022\.  doi: 10.25592/UHHFDM.10272.  URL https://www.fdr.uni-hamburg.de/record/10272.
  * Mol et al. [2023a] W. Mol, B. Heusinkveld, and C. Van Heerwaarden.  Radiometer network dataset of 10 hz spectral irradiance and derived variables (liaise campaign).  2023a.  doi: 10.5281/ZENODO.7966437.  URL https://zenodo.org/record/7966437.
  * Mol et al. [2023b] W. B. Mol, W. H. Knap, and C. C. van Heerwaarden.  Ten years of 1 Hz solar irradiance observations at Cabauw, the Netherlands, with cloud observations, variability classifications, and statistics.  _Earth System Science Data Discussions_ , pages 1–19, Jan. 2023b.  doi: 10.5194/essd-2022-456.  URL https://essd.copernicus.org/preprints/essd-2022-456/.  Publisher: Copernicus GmbH.
  * Mol et al. [2023c] W. B. Mol, B. J. H. van Stratum, W. H. Knap, and C. C. van Heerwaarden.  Reconciling Observations of Solar Irradiance Variability With Cloud Size Distributions.  _Journal of Geophysical Research: Atmospheres_ , 128(5):e2022JD037894, 2023c.  ISSN 2169-8996.  doi: 10.1029/2022JD037894.  URL https://onlinelibrary.wiley.com/doi/abs/10.1029/2022JD037894.  _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1029/2022JD037894.
  * National Renewable Energy Laboratory [2023] National Renewable Energy Laboratory.  AM1.5 Spectral Irradiance Data.  https://www.nrel.gov/grid/solar-resource/spectra-am1.5.html, 5 2023\.  [Accessed on: May 9, 2023].
  * Pincus and Evans [2009] R. Pincus and K. F. Evans.  Computational Cost and Accuracy in Calculating Three-Dimensional Radiative Transfer: Results for New Implementations of Monte Carlo and SHDOM.  _Journal of the Atmospheric Sciences_ , 66(10):3131–3146, Oct. 2009.  ISSN 0022-4928, 1520-0469.  doi: 10.1175/2009JAS3137.1.  URL https://journals.ametsoc.org/view/journals/atsc/66/10/2009jas3137.1.xml.  Publisher: American Meteorological Society Section: Journal of the Atmospheric Sciences.
  * Pingswept [2022] Pingswept.  Pysolar is a collection of Python libraries for simulating the irradiation of any point on earth by the sun.  https://github.com/pingswept/pysolar, 2022.  Accessed 06-Jul-2022.
  * Price [2023] J. Price.  Liaise_els-plans_ukmo_radiosondes_l1.  2023\.  doi: 10.25326/429.  URL https://liaise.aeris-data.fr/liaise-product?uuid=41b5a6e6-86d6-4d83-aec1-4a30f747c990.
  * Tabar et al. [2014] M. R. R. Tabar, M. Anvari, G. Lohmann, D. Heinemann, M. Wächter, P. Milan, E. Lorenz, and J. Peinke.  Kolmogorov spectrum of renewable wind and solar power fluctuations.  _Eur. Phys. J. Spec. Top._ , 223(12):2637–2644, Oct. 2014.  ISSN 1951-6401.  doi: 10.1140/epjst/e2014-02217-8.  URL https://doi.org/10.1140/epjst/e2014-02217-8.
  * Tomson [2010] T. Tomson.  Fast dynamic processes of solar radiation.  _Solar Energy_ , 84(2):318–323, Feb. 2010.  ISSN 0038-092X.  doi: 10.1016/j.solener.2009.11.013.  URL https://www.sciencedirect.com/science/article/pii/S0038092X09002795.
  * Ukkonen and Hogan [2023] P. Ukkonen and R. J. Hogan.  Fast computation of cloud 3D radiative effects in dynamical models by optimizing the ecRad scheme.  preprint, Preprints, May 2023.  URL https://www.authorea.com/users/95958/articles/639895-fast-computation-of-cloud-3d-radiative-effects-in-dynamical-models-by-optimizing-the-ecrad-scheme?commit=d9eaa6a428167e876e825a8b436fc362a076ca9a.
  * Veerman et al. [2022] M. A. Veerman, B. J. H. van Stratum, and C. C. van Heerwaarden.  A case study of cumulus convection over land in cloud-resolving simulations with a coupled ray tracer.  _Geophysical Research Letters_ , n/a(n/a):e2022GL100808, Dec. 2022.  ISSN 1944-8007.  doi: 10.1029/2022GL100808.  URL https://onlinelibrary.wiley.com/doi/abs/10.1029/2022GL100808.  _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1029/2022GL100808.
  * Villefranque et al. [2021] N. Villefranque, S. Blanco, F. Couvreux, R. Fournier, J. Gautrais, R. J. Hogan, F. Hourdin, V. Volodina, and D. Williamson.  Process-Based Climate Model Development Harnessing Machine Learning: III. The Representation of Cumulus Geometry and Their 3D Radiative Effects.  _Journal of Advances in Modeling Earth Systems_ , 13(4):e2020MS002423, 2021.  ISSN 1942-2466.  doi: 10.1029/2020MS002423.  URL https://onlinelibrary.wiley.com/doi/abs/10.1029/2020MS002423.  _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1029/2020MS002423.
  * Weigl et al. [2012] T. Weigl, L. Nagl, J. Weizenbeck, M. Zehner, M. Augel, P. Ochsner, B. Giesler, G. Becker, O. Mayer, T. R. Betts, and R. Gottschalg.  Modelling and validation of spatial irradiance characteristics for localised irradiance fluctuations and enhancements.  _Proceedings of the 27th Photovoltaic Solar Energy Conference_ , Jan. 2012.  URL https://www.researchgate.net/publication/237353116_Modelling_and_Validation_of_Spatial_Irradiance_Characteristics_for_Localised_Irradiance_Fluctuations_and_Enhancements.
  * Wood and Field [2011] R. Wood and P. R. Field.  The Distribution of Cloud Horizontal Sizes.  _Journal of Climate_ , 24(18):4800–4816, Sept. 2011.  ISSN 0894-8755, 1520-0442.  doi: 10.1175/2011JCLI4056.1.  URL https://journals.ametsoc.org/view/journals/clim/24/18/2011jcli4056.1.xml.  Publisher: American Meteorological Society Section: Journal of Climate.
  * Yang et al. [2022] D. Yang, W. Wang, C. A. Gueymard, T. Hong, J. Kleissl, J. Huang, M. J. Perez, R. Perez, J. M. Bright, X. Xia, D. van der Meer, and I. M. Peters.  A review of solar forecasting, its dependence on atmospheric sciences and implications for grid integration: Towards carbon neutrality.  _Renewable and Sustainable Energy Reviews_ , 161:112348, June 2022.  ISSN 1364-0321.  doi: 10.1016/j.rser.2022.112348.  URL https://www.sciencedirect.com/science/article/pii/S1364032122002593.
  * Yordanov et al. [2013] G. H. Yordanov, T. O. Saetre, and O. Midtgård.  100-millisecond Resolution for Accurate Overirradiance Measurements.  _IEEE Journal of Photovoltaics_ , 3(4):1354–1360, Oct. 2013.  ISSN 2156-3403.  doi: 10.1109/JPHOTOV.2013.2264621.  Conference Name: IEEE Journal of Photovoltaics.
  * Yordanov et al. [2015] G. H. Yordanov, T. O. Saetre, and O.-M. Midtgård.  Extreme overirradiance events in Norway: 1.6 suns measured close to 60°N.  _Solar Energy_ , 115:68–73, May 2015.  ISSN 0038-092X.  doi: 10.1016/j.solener.2015.02.020.  URL https://www.sciencedirect.com/science/article/pii/S0038092X15000900.

*Appendix

Figure 18: Observed diffuse fraction under clear-sky conditions for BSRN
Cabauw 1 Hz dataset [Mol et al., 2023b], 2011-2022 average. Clear-sky defined
as in the dataset.