# Super-resolved multi-temporal segmentation with deep permutation-invariant
networks

###### Abstract

Multi-image super-resolution from multi-temporal satellite acquisitions of a
scene has recently enjoyed great success thanks to new deep learning models.
In this paper, we go beyond classic image reconstruction at a higher
resolution by studying a super-resolved inference problem, namely semantic
segmentation at a spatial resolution higher than the one of sensing platform.
We expand upon recently proposed models exploiting temporal permutation
invariance with a multi-resolution fusion module able to infer the rich
semantic information needed by the segmentation task. The model presented in
this paper has recently won the AI4EO challenge on Enhanced Sentinel 2
Agriculture.

Index Terms—  Super-resolution, image segmentation, deep neural networks.

##  1 Introduction

Image super-resolution is the process of recovering information from one or
more degraded images, sampled at a lower spatial resolution (LR). This process
is typically ill-posed so strong priors on the data are needed to effectively
address the problem and, for this reason, deep learning has shown impressive
results in recent years. Multi-temporal image super-resolution (MISR) [1, 2]
is particularly interesting in the context of remote sensing, since multiple
versions of the same scene are naturally acquired by revisits of a satellite.
Thanks to sub-pixel phase shifts, multiple images carry complementary
information about the scene and impressive super-resolution results can be
achieved with suitable fusion techniques. Very recent works have developed
deep-learning models that can combine multiple images in complex non-linear
fashions to reconstruct the higher-resolution version of a scene. In
particular, DeepSUM [3] proposed a modular architecture and the idea of
adaptive filters that are computed directly from the features extracted by the
neural network to register and interpolate the multiple low-resolution images.
Highres-net [4] developed a recursive fusion approach to the problem. RAMS [5]
improved the state of the art by careful usage of the feature attention
mechanism. Most recently, PIUnet [6] showed that temporal ordering of the
input images does not matter and designing an architecture that is fully
invariant to temporal permutation captures richer representations, improving
reconstruction performance and data efficiency.

The aforementioned methods and most of the literature have been focused on
image generation, whereby the output of the model is the image itself, sampled
at a higher spatial resolution. On the other hand, in this paper, we study a
problem of super-resolved inference, i.e., inferring some property with
increased spatial resolution, which is a problem that has been largely
neglected by MISR models in remote sensing. In particular, we work in the
context of the AI4EO challenge on Enhanced Sentinel 2 Agriculture [7], where
multi-temporal Sentinel 2 images are used for the detection of cultivated
areas. However, the segmentation map of cultivated areas must be produced with
a resolution of 2.5 meters per pixel, while the input Sentinel 2 bands have
resolutions up to 10 meters. Thus, an effective model for the problem needs to
suitably combine the multiple input images to extract spatially super-resolved
features which, at the same time, are also semantically discriminative for the
detection of cultivated areas.

##  2 Proposed method

Fig. 1: SPInet architecture. Fig. 2: Multi-resolution fusion block. Features
are extracted at four different scales and scales are made to densely interact
with other before returning features at the highest scale only. This promotes
more semantically meaningful features, requiring larger receptive field and
multi-resolution reasoning.

In this section, we present an overview of the proposed model to address
super-resolved segmentation of cultivated areas from Sentinel 2 images. The
model is called SPInet (Segmentation by Permutation Invariance) and an
overview is shown in Fig.1. SPInet treats the super-resolved segmentation
problem in an end-to-end fashion, i.e., it is trained to directly regress the
super-resolved segmentation maps, rather than generating super-resolved images
and then applying a segmentation model on those. The reason for this design is
efficiency, since a single end-to-end model can use its parameters in the most
efficient manner, reducing computational and data requirements.

Existing models for MISR have largely neglected the issue of temporal
permutation, whereby the temporal ordering of the input images does not carry
any relevant information for the super-resolution task and causes such models
to be inefficient with the, often scarce, ground truth data available for
training. Valsesia et al. [6] have recently shown how models ought not to
learn feature extractors that rely on temporal ordering, but rather should be
invariant to temporal permutation by design. In this work, we build a model
that is fully invariant to temporal permutation for the task of image
segmentation at an increased spatial resolution. The backbone of the model is
the PIUnet architecture [6], but the segmentation problem significantly
differs from image generation. While super-resolved image generation largely
exploits localized features, segmentation requires larger receptive fields and
interaction between features across scales in order to infer semantic
properties.

Following Fig.1, the first part of the architecture up to the temporal mean
operation closely follows PIUnet, with the exception of the first
convolutional layer which processes multiple bands instead of a single one.
One of the main design patterns is the use of a temporally-shared 2D
convolution operation followed by batch normalization, leaky ReLU non-
linearity and a residual self-attention operation [8] in the temporal
dimension. This block is, by design, equivariant to temporal permutation and
serves as a fundamental building block to extract spatio-temporal features.
The Temporally-Equivariant Feature Attention (TEFA) block uses this
fundamental building block to construct a variant of classic feature attention
that is equivariant to temporal permutations. The repetition of 16 TEFA blocks
serves as the backbone of our network. The Temporally-Equivariant RegNet
(TERN) is a module that proposes and applies adaptive registration filters to
the features and allows to compensate for misregistration, always maintaining
temporal equivariance. The temporal mean operation, being symmetric over the
temporal dimension, allows to transform the model from equivariant to
invariant. Notice how there is no invariance in the spectral dimension because
ordering does matter when processing spectral information.

The promotion of semantically meaningful features is performed with the
inclusion of a multi-resolution feature fusion block (see Fig.2). This serves
as an enhancement for the segmentation task, with respect to pure super-
resolution, because it allows features to exploit a more global context,
rather than localized properties. This block is inspired by state-of-the-art
operations used in semantic segmentation networks such as HRNet [9]. It
generates four dyadic scales where features at different resolution and
spatial receptive fields can be extracted before merging them to create the
output logits. More in detail, after the pixel shuffling upsampling operation,
four strided convolutions create four parallel branches whose spatial
resolution is either the original (i.e. the super-resolved one), $1/2$, $1/4$,
or $1/8$. Each branch is further downsampled or upsampled to generate versions
of itself at all the four scales. Then, features from highest resolution from
all branches are concatenated into a new high-resolution branch. Similarly,
the features from all versions at $1/2$ resolution are concatenated into a
$1/2$ resolution branch and so on for all the scales. After a convolution on
each branch, the lower scales are upsampled and the features from all branches
are concatenated and projected to generate the segmentation map.

Finally, we remark that the model does not require a fixed number of input
images and can work with any number of low-resolution images, depending on
availability and quality, without retraining or redesigning the architecture.
Expensive temporal self-ensembling operations where the model performance is
boosted by averaging the results of different input temporal permutations are
avoided thanks to the invariance property which already captures this richness
in the model.

##  3 Experimental results

###  3.1 Experimental setting

We tested the proposed model on the dataset provided by the AI4EO challenge on
Enhanced Sentinel 2 Agriculture [7]. The public dataset is composed of 100
scenes, each with a variable number of multi-temporal images (usually more
than 15) collected between March 1st and September 1st 2019. All the Sentinel
2 bands are either native or interpolated to 10m per pixel and they are
Bottom-of-Atmosphere L2A products with $500\times 500$ pixels. Ground truth
segmentation maps are provided with size $2000\times 2000$ at 2.5m per pixel.
We partitioned the public data into training and validation sets, with 90 and
10 scenes each. For training purposes, a fixed number of multi-temporal
images, equal to 20, is used. Cloudless images are selected whenever possible.
Testing used all available multi-temporal images with at most $25\%$ cloud
coverage. All the 12 bands of Sentinel 2 have been used in this work,
including the 20m and 60m ones, which have been interpolated to 10m.

SPInet has been trained for about 60 epochs with a batch size equal to 8 and
learning rate equal to $10^{-4}$ with the binary cross-entropy loss function.
The number of feature channels is set to 48 throughout the network. Feature
attention uses a bottleneck with 6 features. Patches of size $32\times 32$
have been extracted for the training process. Model training required about
two days with a Titan RTX GPU with peak VRAM occupation equal to 16GB.

###  3.2 Segmentation results

We compare the performance of SPInet on our validation set against two
relevant baselines. The first baseline is a model-based segmentation approach
which relies on bicubic interpolation and thresholding of the temporally-
averaged normalized difference vegetation index (NDVI). Additionally, we
include the results of a deep learning baseline consisting in using a state-
of-the-art neural network for semantic segmentation, namely DeepLabv3 [10]
applied on the bicubically-interpolated and temporally-averaged images.

Performance is evaluated in terms of the Matthews correlation coefficient
(MCC) which better accounts for true and false positives (TP,FP) and negatives
(TN,FN) with respect to other measures like F1 score or accuracy, especially
with unbalanced classes. The MCC is defined as:

$\displaystyle\text{MCC}=\frac{TP\times TN-FP\times
FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}.$

Table 1 shows the MCC obtained by the different methods on the validation set.
It can be noticed that the proposed end-to-end approach with SPInet
significantly outperforms the simple model-based baseline and also improves
upon the deep-learning segmentation approach.

Table 1: Cultivated land detection - MCC | Model-based | Deeplabv3 | SPInet
---|---|---|---
eopatch-902 | 0.636 | 0.785 | 0.895
eopatch-903 | 0.413 | 0.704 | 0.802
eopatch-904 | 0.550 | 0.721 | 0.819
eopatch-905 | 0.604 | 0.804 | 0.895
eopatch-907 | 0.534 | 0.692 | 0.802
eopatch-908 | 0.544 | 0.702 | 0.787
eopatch-914 | 0.474 | 0.652 | 0.778
eopatch-915 | 0.325 | 0.507 | 0.689
eopatch-916 | 0.525 | 0.659 | 0.819
eopatch-924 | 0.496 | 0.605 | 0.736
Avg. MCC | 0.510 | 0.683 | 0.802
Table 2: Effectiveness of multi-resolution fusion (MRF) - MCC | SPInet with MRF | SPInet w/o MRF
---|---|---
Avg. MCC | 0.802 | 0.783

Figures 3 and 4 show the segmentation results obtained by the various methods
on two validation images when compared against the ground truth segmentation
map. It can be noticed how the proposed method is able to capture finer
details of the cultivated areas map, showing its effectiveness in generating a
super-resolved result. Existing segmentation techniques such as the tested
DeepLab model heavily rely on downsampling because the classic semantic
segmentation tasks studied in computer vision do not carry much fine spatial
information. This is undesirable for the super-resolved cultivated land
detection problem we study and the proposed joint super-resolution and
segmentation approach avoids such operations and results in better
performance.

Table 2 reports an ablation study that highlights the importance of the multi-
resolution fusion module. In this experiment, the multi-resolution fusion
module is replaced by two convolutional layers interleaved with batch
normalization and ReLU maintaining the full image resolution and with overall
a comparable number of weights with respect to the fusion block. This solution
appears to be suboptimal, as expected, because there is no mechanism to
exploit multi-resolution representations which are important to capture scene
semantics.

Finally, as a reference, we report that SPInet has achieved an MCC value of
0.862 on the private test set of the AI4EO challenge.

Fig. 3: Results on eopatch-902. Left to right: one of the LR images,
cultivated areas map produced by model-based approach, bicubic+Deeplab deep
learning approach, SPInet and ground truth.

Fig. 4: Results on eopatch-914. Left to right: one of the LR images,
cultivated areas map produced by model-based approach, bicubic+Deeplab deep
learning approach, SPInet and ground truth.

##  4 Conclusions

In this paper, we presented an end-to-end deep learning approach for a super-
resolved inference problem consisting in detecting cultivated areas with
higher spatial resolution than what is offered by the input Sentinel 2 images.
We leveraged recent results on the importance of temporal permutation
invariance in the design of deep neural network that deal with multi-temporal
data, and specifically for MISR. We also showed how multi-resolution fusion is
important in this inference problem with respect to pure image super-
resolution. Future work will focus on estimating and exploiting aleatoric
uncertainty which was shown in [6] to be an important factor in presence of
temporal variation in the input data.

## References

  * [1] S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar,  “Fast and robust multiframe super resolution,”  IEEE Transactions on Image Processing (TIP), vol. 13, no. 10, pp. 1327–1344, Oct 2004.
  * [2] Kato Toshiyuki, Hino Hideitsu, and Murata Noboru,  “Multi-frame image super resolution based on sparse coding,”  Neural Networks, vol. 66, pp. 64 – 78, 2015.
  * [3] Andrea Bordone Molini, Diego Valsesia, Giulia Fracastoro, and Enrico Magli,  “Deepsum: Deep neural network for super-resolution of unregistered multitemporal images,”  arXiv preprint arXiv:1907.06490, 2019.
  * [4] Md Rifat Arefin, Vincent Michalski, Pierre-Luc St-Charles, Alfredo Kalaitzis, Sookyung Kim, Samira Ebrahimi Kahou, and Yoshua Bengio,  “Multi-image super-resolution for remote sensing using deep recurrent networks,”  in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2020, Seattle, WA, USA, June 14-19, 2020. 2020, pp. 816–825, IEEE.
  * [5] Francesco Salvetti, Vittorio Mazzia, Aleem Khaliq, and Marcello Chiaberge,  “Multi-image super resolution of remotely sensed images using residual attention deep neural networks,”  Remote Sensing, vol. 12, no. 14, 2020.
  * [6] Diego Valsesia and Enrico Magli,  “Permutation invariance and uncertainty in multitemporal image super-resolution,”  arXiv preprint arXiv:2105.12409, 2021.
  * [7] “AI4EO Enhanced Sentinel 2 Agriculture,” https://platform.ai4eo.eu/enhanced-sentinel2-agriculture,  Accessed: 2010-10-13.
  * [8] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin,  “Attention is all you need,”  in Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp. 6000–6010.
  * [9] Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, et al.,  “Deep high-resolution representation learning for visual recognition,”  IEEE transactions on pattern analysis and machine intelligence, 2020\.
  * [10] Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam,  “Rethinking atrous convolution for semantic image segmentation,”  arXiv preprint arXiv:1706.05587, 2017.


[a]G. Grolleron

#  nectarchain, the scientific software for the Cherenkov Telescope Array -
NectarCAM

H. Ashkar  F. Brun  H. Costantini  D. Dumora  P. Jean  D. Kerszberg  J.-P.
Lenain  V. Marandon  S. R. Patel  L. Tibaldo

###### Abstract

The NectarCAM is a camera that will be mounted on the Medium-Sized Telescopes
of the Cherenkov Telescope Array (CTA) observatory. Along with the hardware
integration of the camera, the scientific software, nectarchain, is being
developed. The software is responsible for transforming the raw data from the
camera into analysis-ready calibrated data. In this contribution, we present
the structure of the software, which consists of two modules: the calibration
pipeline and the data quality check pipeline. The calibration pipeline reduces
the data, performs flat fielding, and determines the gain for the analysis.
The data quality monitoring pipeline is used to select the data that meets the
necessary standards for analysis. Additionally, we discuss the format of the
downstream data and the integration of the nectarchain modules in the general
software framework of CTA. We also present the necessary tests for validating
each part of the code. We conclude by mentioning the prospects for the future
of the software.

##  1 Introduction

The Cherenkov Telescope Array (CTA) will be the next generation ground-based
imaging atmospheric Cherenkov telescopes (IACT) observatory. In comparison to
the current IACT arrays, it will improve the sensitivity by a factor five to
ten depending on the energy. Therefore it will bring a new outlook of the
Universe. CTA will be composed of three types of telescopes, the Small-Sized-
Telescopes (SST), the Medium-Sized-Telescopes (MST) and the Large-Sized-
Telescopes (LST). They will be sensitive to the lower, median and higher part
of the CTA energy range respectively from 20 GeV to beyond 100 TeV.

NectarCAM [1] will be the camera mounted on the MSTs located in la Palma, the
northern site of the Cherenkov Telescope Array Observatory (CTAO). This camera
has a 8 degrees field of view and consists of 265 modules each equipped with 7
photomultiplier tubes (PMTs) able to detect Cherenkov light produced by very
high energy (VHE) photons entering in the atmosphere. Each module is equipped
with a trigger and readout electronic system, which operates with GHz sampling
on 60 ns windows. The camera has two gain channels : the high gain to measure
signals at the single photo-electron (p.e.) level and the low gain which aims
at reconstructing signal up to 2000 p.e.. A full camera prototype is at the
CEA Paris Saclay test bench where some tests have been performed (timing,
temperature, etc.) and where flat-field (FF), pedestal (Ped) and gain
calibration data acquisitions are taken to calibrate the camera and to develop
the software which will be used on site to generate the calibrated data (R1)
from the raw data produced by the camera (R0).

In this proceeding, we are focusing on the software pipeline
nectarchain111https://github.com/cta-observatory/nectarchain. In section 2,
the main goal of the calibration pipeline is presented. Then in section 3, the
nectarchain workflow is presented and the current development status is
explained. Next the interface with ctapipe in presented in section 4. Monte
Carlo simulations are briefly presented in section 5 and the Conclusion is
done thereafter in section 6.

##  2 Main Goal : R0 to R1 calibration

The main goal is to convert the signal in analog-to-digital-conversion ADC to
the number of $N_{\mathrm{p.e.}}$ defined by :

$N_{\mathrm{p.e.}}=\frac{ADC-Ped}{gain}\times FF$

The pedestal has to be estimated to be substracted to the signal in ADC. Then,
the gain has to be calibrated to perform the conversion ADC to p.e., as well
as the FF coefficients which are used to balance the inhomogeneities of the
camera response.

Ultimately, the on-site raw data streamed from the CTA camera servers to the
CTA Array Control and Data Acquisition system (ACADA), dubbed R1 data in the
CTA data model, will include pre-calibrated event data. The pre-calibration
requires pedestal subtraction, gain linearisation (conversion from ADC to
photo-electron number), gain equalisation among the camera pixels, and the
selection of the most appropriate among the two gain channels.

The current version of the camera server software does not perform the pre-
calibration required for R1 data. We implemented in ctapipe_io_nectarcam (cf.
4) a prototype for the pre-calibration software and methods that can read
calibration coefficients produced by nectarchain (cf. 3) and apply them to the
raw-data to fill R1 containers. This makes it possible to ingest the data from
the camera prototype in the dark room in ctapipe, and thus test the entire
analysis chain.

##  3 nectarchain modules

The entire workflow is described in Figure 1. In this part the calibration
workflow is presented and the emphasis is made on the main part already
implemented.

Figure 1: Description of the NectarCAM calibration workflow.

###  3.1 Pedestal estimation

An accurate pedestal estimate is required to correctly evaluate the charges
produced by Cherenkov photons from electromagnetic showers. The pedestal is
composed of a deliberately set non-zero offset with electronic noise and,
during observations, background light from the night sky. The electronic
component can be measured when the camera is in dark conditions (e.g. when the
shutter of the camera is closed). For NectarCAM, the average value of the
pedestal baseline is set to a value of about 250 ADC. However, it may slightly
change with time (warm up of electronics) and with temperature (see Figure 2).
Figure 2 also shows that the pedestal waveform is affected by fluctuations in
the 60 ns window due to slight offsets between the memory cells of the NECTAr
chip [2] (a part of the NectarCAM front end board which produces ADC
conversion). During an observation, pedestals are evaluated with periodic
pedestal triggers interleaved with regular triggers. A pedestal waveform
template is obtained for each pixel and for each channel by averaging
waveforms that are not affected by pulses due to noise photons. This template
is then used for the analysis of that observation.

Figure 2: Example of pedestal waveforms of a pixel of NectarCAM extracted at 5
different periods of an observation. The waveform labelled "Average"
corresponds to the template.

###  3.2 Flat-field and timing

Additional information needed during the calibration of the data are the
relative efficiency between pixels and the relative timing between pixels.
Both are measured with a device called the flat-fielding unit which deliver
short pulses of uniform blue light across the camera. The relative
inhomogeneity is obtained by computing the ratio of a given pixel measured
intensity to the average across the camera. The relative timing is obtained by
computing the difference between the average maximum time of the waveform to
the average across the camera.

###  3.3 ADC to photo-electron conversion

####  3.3.1 Description of the method

The Single photo-electron (SPE) calibration method is based on low
illumination calibration data acquisitions where an illuminating device is set
to give about 1 p.e. per pixels. Thus, after the accumulation of a reasonable
number of events, the charge for each event for each pixel is computed by
integrating the ADC signal over a fixed-width window centered on the main
pulse. Several charge extraction methods can be used, by varying window width,
with a global window for the whole camera or adapted to each pixel for
instance. Charge histograms show two main peaks : one created by the pedestal
signal, and another one created by the SPE signal. The separation between the
two peaks makes it possible to measure the gain, which is the charge in ADC
per p.e., with a two Gaussian modelling presented in [3]. However, as the
illumination is low, this method can only be used to reconstruct the gain in
the high gain channel. Derivation of the gain in the low gain channel will
then use the High-Low method described in section 3.4.

####  3.3.2 Single photo-electron Calibration devices

Two different types of SPE calibration data acquisitions can be used, the
first one is simply a FF one where the flasher is set at low intensity, in
this case each pixel is illuminated simultaneously. The gain computed with
this method for the entire camera in presented on Figure 3. Moreover, this
method can only be performed in the darkroom. On site, the SPE measurement
will not be possible with the shutter open (NSB contamination/light pollution)
and a special device, embedded in the camera was created to be able to make
SPE measurement with the shutter closed. This alternative method does not
require simultaneous illumination of the entire camera and is described below.

Figure 3: Gain computed with the SPE spectrum fit method applied to a FF data
acquisition at low illumination.

Analysis of a few of these data acquisitions will be presented in another
contribution in these proceedings [4], as well as the comparison between
results from white target runs, FF data acquisitions at low illumination (cf
Figure 3), and another method presented in [4] to cross-check all that
procedures.

###  3.4 High-Low ratio

The measurement of the low gain channel amplification cannot be achieved by
direct measurement using e.g the SPE device, as it is expected to be a factor
15 lower in amplification compared to the high gain channel. However, by using
the regime in which both gain channels are linear ($\sim$30-200 p.e.), we are
able to measure the ratio of amplification between the high and low gain
channels using events that fall in this range. In NectarCAM, this measurement
is done using events from the flat-fielding device set to provide an intensity
of $\sim$100 p.e. The low-gain channel gain value is then the multiplication
of the previously calculated high-gain amplification value by this ratio.

###  3.5 Data quality monitoring

Figure 4: DQM semi-automatic pipeline.

The data quality monitoring (DQM) pipeline is a semi-automatic pipeline
developed for the monitoring of the data taken by the camera and presented in
Fig. 4. It is data acquisition independently of the calibration module and is
still in development. It monitors the data for low and high gains and for
different types of triggers such as pedestal and physical triggers. The main
script reads the NectarCAM events and defines processor classes that are used
to derive the quantities to be monitored. At this stage, the processor
contains five classes presented below.

Mean waveforms: The mean waveform class computes the average waveform for all
channels and the overall camera average. The waveform of different trigger
types can be compared. It allows for the verification of the quality of the
camera sampling.

Mean display: This class averages all the signal seen by the channels through
all events in a given data acquisition period. It is used to flag defective
pixels, abnormal pedestals and anomalous behavior in general.

Charge integration: In this class, the charge is integrated around the peak of
the signal for each event. An additional option to subtract pedestals,
computed from the signal itself, is available. The class computes the charge
power spectrum of the camera throughout the entire data acquisition period and
computes statistical entities such as the mean, the median, the RMS and the
standard deviation of the charge through the whole period. This module can be
used to monitor the gain (computation described in section 3.3.2) of the
camera.

Camera monitoring: This class monitors the temperature of the modules of the
camera. It uses the measurements of the camera by two sensors on the modules
saved daily in the log files and matches the temperature with the times of the
data acquisition. It also computes the average module temperature during these
times.

Trigger statistics: This class studies the camera trigger rates as a function
of trigger ID and time. The triggers are flagged by their nature: pedestal,
physical or other.

###  3.6 Continuous integration

Workflows dedicated to continuous integration (CI) are being developed for the
nectarchain package. For instance, the package is published on PyPI on
releases, as well as a conda package on the conda-forge channel. Notably, a
Singularity/Apptainer [5] container image is also built and deployed on the
GitHub Container Registry on releases and merged push requests, which greatly
eases software deployment. When processing NectarCAM data on the EGI grid
through DIRAC [6], where the data are stored, such an image can readily be
instantiated without the need of embedding the software in DIRAC jobs or of
deploying it to distributed file systems prior to the job processing.
Ultimately, nectarchain could be deployed on the CTA Observatory
CVMFS222CernVM File System, https://cvmfs.readthedocs.io/en/stable/ instance
once stabilized. Some work in progress on the CI side concerns well covering
unit tests as well as an extensive code documentation.

##  4 Interface with ctapipe : ctapipe_io_nectarcam

The interface between the data acquisition system and the CTA data processing
pipeline for NectarCAM is ensured by a dedicated plugin ctapipe_io_nectarcam.
The plugin reads the data in output from the camera server and fills the
containers used by ctapipe, the base library upon which the CTA data
processing pipeline is built. The ctapipe_io_nectarcam module is distributed
through PyPI and as a conda package using the same kind of CI workflows as
nectarchain.

##  5 Monte Carlo simulations

To characterize the performance of the NectarCAM camera, in the past we have
produced a Monte Carlo model of the camera that was compared to data taken
with a partially equipped prototype of the camera at the testbench in CEA
Paris-Saclay (France) and on a prototype of the MST structure in Adlershof
(Germany) [7]. As the atmosphere is an integral part of the detection
principle, therefore, Monte Carlo simulations are needed to convert the number
of photons from the shower to the number of p.e. received in the camera. The
NectarCAM camera is now complete and we are updating the Monte Carlo model
comparing the simulations to the data taken with the fully equipped camera at
the CEA Paris-Saclay testbench taking into account the improved timing
resolution analysis [8].

##  6 Conclusion

In these proceedings, the calibration pipeline for NectarCAM has been
presented. The most important part is the nectarchain pipeline which is
currently used to extract the calibration quatity needed to produce R1 pre-
calibrated data. Moreover, nectarchain makes possible study the camera
performances (pedestal estimation and evolution in time or with temperature,
flat-fielding, gain estimation). ctapipe_io_nectarcam is the next part of the
calibration workflow and use the output of nectarchain to produce the R1 data
in a ctapipe format to be CTAO compliant. The Monte Carlo simulations are
still on the rise and will be the next piece of the calibration pipeline. The
current development status has been presented and we aim to have an
operational workflow in the following year.

## Acknowledgments

This work was conducted in the context of the CTA Consortium. We gratefully
acknowledge financial support from the agencies and organizations listed here:
https://www.cta-observatory.org/consortium_acknowledgments/.

## References

  * [1] J.-F. Glicenstein and M. Shayduk, _NectarCAM, a camera for the medium sized telescopes of the Cherenkov telescope array_ , in _6th International Symposium on High Energy Gamma-Ray Astronomy_ , vol. 1792 of _American Institute of Physics Conference Series_ , p. 080009, Jan., 2017, DOI [1610.04173].
  * [2] J.-F. Glicenstein, M. Barcelo, J.A. Barrio, O. Blanch, J. Boix, J. Bolmont et al., _The NectarCAM Camera Project_ , in _International Cosmic Ray Conference_ , vol. 33 of _International Cosmic Ray Conference_ , p. 3036, Jan., 2013, DOI [1307.4545].
  * [3] S. Caroff, S. Fegan, P. Jean, J.-F. Olive and A. Tsiahina, _Determination of the single photo-electron spectrum and gain measurement for the Cherenkov Telescope Array camera NectarCAM_ , in _Optics for EUV, X-Ray, and Gamma-Ray Astronomy IX_ , S.L. O’Dell and G. Pareschi, eds., vol. 11119 of _Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series_ , p. 111191W, Sept., 2019, DOI.
  * [4] S. Patel, et al. and for the NectarCAM Collaboration, _A single photo-electron calibration of NectarCAM_ , in _these proceedings_ , 2023\.
  * [5] G.M. Kurtzer, cclerget, M. Bauer, I. Kaneshiro, D. Trudgian and D. Godlove, _hpcng/singularity: Singularity 3.7.3_ , Apr., 2021.  10.5281/zenodo.4667718.
  * [6] L. Arrabito, J. Bregeon, P. Maeght and M. Sanguillon, _The Cherenkov Telescope Array production system proto-type for large-scale data processing and simulations_ , in _European Physical Journal Web of Conferences_ , vol. 251 of _European Physical Journal Web of Conferences_ , p. 02029, May, 2021, DOI.
  * [7] T. Armstrong, H. Costantini, J.-F. Glicenstein, J.-P. Lenain, U. Schwanke, T. Tavernier et al., _Monte Carlo Simulations and Validation of NectarCAM, a Medium Sized Telescope Camera for CTA_ , in _37th International Cosmic Ray Conference_ , p. 747, Mar., 2022, DOI [2108.00426].
  * [8] F. Bradascio, H. Rueda, J.A. Barrio, J. Biteau, F. Brun, C. Champion et al., _The NectarCAM Timing System_ , _arXiv e-prints_ (2023) arXiv:2301.13828 [2301.13828].