Further rate reduction (see Figure <ref> for the improvement in L1 Muon endcap trigger rate) in the range $1.3 < |\eta| < 2.4$ is achieved in by replacing the existing muon endcap inner stations (the Small Wheels) by the NSW described in Section <ref>, comprising sTGC and MM detectors with high-rate tolerance and improved resolution. The upgrades to the L1Muon endcap trigger are illustrated in Figure <ref>. At , the total L1 trigger rate for single muons with $\pT > \SI{20}{\GeV}$ was expected to be 18 before these improvements; with these improvements, the expected rate is 13. The additional fake muon rejection provided by the upgrade will be even more important during  and beyond.
The expected performance of the combined upgrades to the L1Muon system is also shown in Figure <ref>.

Schematic overview of the upgraded L1Muon endcap system, illustrating the NSW and Tile-Muon coincidence.

In the L1Muon trigger, muon candidates are identified by measuring the degree to which their paths deviate (in both $R$ and $\phi$) from the pattern of hits expected from a muon with infinite momentum; this deviation is inversely proportional to the of the muon.
Using the deviations expected for different muon , so-called “muon roads” can be defined for different thresholds.
The trigger electronics permitted the use of six programmable thresholds; in ,  the upgraded sector logic (see Section <ref>) will allow 15 programmable thresholds in the endcap.

§.§.§ TGC EI-FI Coincidence

The main background source in the L1Muon endcap trigger is low-momentum charged particles produced in the endcap toroid magnets and beam shielding.
In order to reject backgrounds due to these particles, a coincidence requirement between the Big Wheel and the TGC-FI chambers of the old EI wheels was introduced in 2015, in part as a proof of principle for the NSW, which ultimately replaced the EI wheels.
An additional coincidence requirement between the Big Wheel and the EIL4 TGC chambers was introduced in 2016, and remains in effect for and beyond. papers refer to the combination of these two vetoes as the “TGC EI-FI coincidence”; however, for only the “TGC EIL4 coincidence” remains.

§.§.§ Tile-Muon Coincidence

fig:TDAQL1MuonTileMuonRoIVsEta The pseudorapidity distribution of L1Muon RoI with $\pT > \SI{20}{\GeV}$ before and after the deployment of the Tile calorimeter coincidence requirement.  The reduction in the range $1.05 < |\eta_{\mathrm{RoI}}| < 1.3$, where the coincidence requirement is applied, is highlighted.
fig:TDAQL1MuonTileMuonEfficiency The efficiency of the L1 trigger selection for muon RoI with $\pT > \SI{20}{\GeV}$ in this pseudorapidity range.

In the region $1.05 < |\eta| < 1.3$, where the inner layer of the muon system provides incomplete coverage due to the presence of the toroid magnets, a coincidence between the EM-TGC chambers and the Tile calorimeter D-cell layers is required in order to improve the rejection of fake muons, which consist primarily of low-protons.  The distribution of this background can be seen in Figure <ref>.

As shown in Figure <ref>, a high-muon originating from the IP and passing through the endcap in the region $1.0 < |\eta| < 1.3$ can be expected to traverse the D5 or D6 cells of the Tile calorimeter extended barrel.  These modules have a granularity of $\eta \times \phi = 0.2 \times 0.1$, thus providing finely segmented energy measurements.  Muon candidates with $\pt > 20$ GeV are required to coincide with an energy deposit satisfying a pre-determined threshold in at least one of the corresponding Tile modules mapped in $\phi$ to the associated muon trigger sector.

New TMDB were installed in to perform matching in $\phi$ between the Tile extended barrel modules and the L1Muon endcap sector logic.  Each extended barrel consists of 64 modules in $\phi$ (128 in total), while the L1Muon endcap region consists of 48 trigger sectors.  Each TMDB receives the D5 and D6 inputs from eight Tile modules and three L1Muon endcap sector logic boards.  Hence, 16 TMDB in total are required.  The TMDB are 9U VME modules situated in two crates in USA15.  They perform the following tasks:

* Receive and digitise the signals from the Tile D5 and D6 cells.

* Provide calibration for the Tile signals.

* Provide signal detection for each Tile cell.

* Provide BCID information using timing information from the TTC receiver <cit.>.

* Provide $\eta$, $\phi$, and BCID information from cells in which a signal has been detected to the corresponding three muon sector logic boards.

* Share information with neighbouring receiver boards to accommodate for non-perfect matching in $\phi$ between the eight Tile modules and three muon sector logic boards.

* Provide readout data fragments to the DAQ system.

For single muons with $\pt > \SI{20}{\GeV}$, the trigger rate was reduced by 6% after this coincidence requirement was introduced, with only a 2.5% efficiency loss in the range $1.05 < |\eta| < 1.3$, as shown in Figure <ref>.

§.§.§ NSW Trigger

Trigger inputs from both the sTGC and the MM chambers in a sector are processed by algorithms running on the NSW-TP cards (Figure <ref>).

Functional diagram of the NSW-TP.

NSW Trigger Processor requirements
The NSW-TP provides track segments from the NSW detectors (as described in Section <ref>) to be matched in the SL with coincidences found in the EM-TGC.
Each track segment is characterised by its radial position $\eta$, its angle $\Delta\theta$ with respect to an ideal infinite-momentum track (a line from the IP to the segment's radial position in the NSW), measured by the precision strips of the sTGC or MM detectors, and by its azimuthal position, $\phi$, measured by the sTGC pad towers (see also Section <ref>) or by the output of MM “diamond” fitter (explained in Section <ref>).
The required $\eta$-resolution of both the sTGC and MM strip triggers is 0.005.
The sTGC pad towers have an $r\phi$-resolution of 738, increasing with radius (based on the size of the logical pads), and the MM trigger diamonds provide an $r\phi$-resolution of better than 11 at all radii.
The angular resolution of the NSW segments required for Phase-II is 1 in order to match the expected angular resolution of the EM-TGC.
For ,
$\Delta\theta$ is passed to the SL but not used in the Phase-I trigger decision.

NSW-TP mezzanine cards and input interfaces
The NSW-TP is housed in two ATCA <cit.>
crates in USA15.
The NSW-TP processes information from the MM and sTGC layers in the sector in separate algorithms (one for the eight MM layers, and one for the eight sTGC layers) in separate Virtex-7 XC7V690 FPGA
on a mezzanine card.
Each blade contains two mezzanine cards, to serve one NSW octant (one Large sector and one adjacent Small sector); a crate of eight blades serves a full wheel.

One FPGA on each mezzanine card takes 32 MM fibre inputs (from the sixteen ADDC per sector).
Each ADDC GigaBit transceiver packet can contain ART data from up to eight triggered VMM.
When a packet is received, the ART data are decoded, for each strip, into a strip number and the slope associated with a line from the hit strips to the IP.
The decoded data are provided to the MM TP algorithm (Section <ref>), which runs on the FPGA.

The other FPGA on the mezzanine takes 32 sTGC fibre inputs from the Routers (four for each of the eight layers), plus two redundant fibres from the sTGC Pad Trigger (see Section <ref>) for the sector.
Only data from strips passing through the tower selected by the sTGC Pad Trigger (see Section <ref>) are transmitted to the NSW-TP. A maximum of four such track segments per sector may be transmitted per BCID.
The data from the Routers consist of a Band-ID identified by the Pad Trigger, the 6-bit ADC values from 14 strips, a flag indicating whether the 14 high or low strips of the 17 strips in the band are transmitted, along with the six low bits of the BCID, and the $\phi$-ID (see Section <ref>). The decoded data are used by the sTGC TP algorithm (Section <ref>), which runs on the FPGA.

NSW sTGC Strip Trigger Algorithm
The sTGC strip trigger uses the digitised peak data describing the quantity of charge deposited
on each active strip selected by the Pad Trigger to compute a centroid for each of the eight sTGC layers and, from the centroids in the eight layers, track segments.
A configurable charge threshold is used to define the “active” strips used in the centroid calculation.
A configurable look-up table defines which patterns of active strips are accepted: wide clusters due to neutrons or $\delta$-rays can be rejected;
certain patterns with isolated hits near the cluster may be configured to be accepted.
An average centroid is calculated from the valid layer centroids in each of the two wedges.
An $R$-index, defined as the radial position of the trajectory projected onto a virtual plane passing through the nominal $z$-coordinate of the most downstream wire plane of the NSW, and $\Delta\theta$ are calculated directly from the values of the two wedge-centroids, using LUT.

The sTGC trigger produces at most four candidates per sector; the MM trigger can transmit up to eight candidates. The FPGAs are connected by a high-speed, low-latency 64-bit LVDS link. Since the MM TP algorithm results are available sooner, MM candidates are sent to the sTGC TP for merging, with no impact on the latency due to the data transfer.
The merging algorithm limits the total number of merged candidates to at most eight, removing duplicates.
Since $\phi$ from the MM is more precise and $R$ from the sTGC is more precise (see Tables <ref> and <ref>), a duplicate segment can take its $\phi$ from the MM and its $R$-index and $\Delta\theta$ from sTGC.
Further details of the merging algorithm are given in Ref. <cit.>.

Micromegas Trigger Algorithm
The trigger algorithm first converts incoming hits to slope values, based on the slope of the line between the hit strip and the ATLAS IP.
Hits are stored in a buffer with a tuneable number of bunch crossings (currently five). A set of overlapping roads, eight strips wide, is formed for the $\eta$ strips;
the number of strip hits within a road required for further processing can be set between two and four (currently three).

The MM trigger identifies “roads”: sets of strips in the eight MM layers of a sector that are consistent with the path of a particle (with momentum above a given threshold) coming from the IP.
In practice, MM roads are twelve strips wide (5.1 or 5.4), with the four outer strips at each side overlapping the neighbouring roads.
Each wedge contains two layers of “eta” strips and two layers of “stereo” strips (one with a stereo angle of 1.5 and the other -1.5 with respect to the eta strips).
When a coincidence of three $\eta$ strips is found (of the four $\eta$ layers in the sector), there is a range of possible stereo hits in both stereo projections.
The intersection of two opposite-angle roads of stereo strips (also 12 strips wide) forms a diamond (see Figure <ref>).
The number of these diamonds required to span the full azimuthal extent of the chamber (see Figure <ref>) depends on the radius.
Seven diamonds are sufficient to span the full azimuth at the innermost radius of the Large sectors, while seventeen are needed at the outer radius.
For the $\eta$ road, all diamonds are sequentially parsed for possible stereo hits. The number of stereo hits required is also tuneable from two of opposite direction to a maximum of four.  Currently the default is three, but there must be at least two with opposite inclination angles. Once the coincidences in the $\eta$ roads and diamonds satisfy the coincidence requirements, the hit strips are then used to calculate slopes. See also Figures figMuon:oneDiamond and figMuon:Ndiamonds.
Candidate diamonds are required to have hits in a configurable number of the eta layers in the sector and a configurable number of the corresponding stereo layers.
A high-efficiency configuration would be at least two out of four hits in the eta layers and at least two out of four in the stereo layers, whereas a high-rejection configuration would require three out of four in both.
Coincidences are considered within a configurable sliding window of time, in the range of four to eight BC. These configurable parameters are chosen based on detector and electronics performance, and on LHC conditions.

The MM trigger is built from “roads” 12 strips wide. figMuon:oneDiamond shows an $\eta$-road (grey), and two opposite-angled stereo roads (red and blue) which intersect to form a diamond.
figMuon:Ndiamonds shows one $\eta$-road (grey), and five positive and five negative-angled overlapping stereo roads, forming five diamonds spanning the full length of the $\eta$-road. Both axes on each plot use units of strip pitch.
The $\eta$-road overlaps by four strip pitches with the $\eta$-roads immediately above and below it. The stereo roads also overlap by four strip pitches with their immediate neighbours, as shown.
The trigger diamonds are the intersections of twelve stereo strips in each direction. The diamonds overlap with their neighbours, as shown by the black outlines.
The MM TP FPGA performs a fit to the strips in the roads within the selected diamonds. This fit determines the azimuthal resolution of the MM trigger.

A local slope is calculated from the hits on the horizontal $\eta$ strips and used to calculate $\Delta\theta$. The stereo hits are used to find the $\phi$ of the track segment. If a fit is found to be consistent with a projection back to the IP using $\Delta\theta$, its $R$-index is determined using a lookup table and sent along with  $\Delta\theta$ for merging with the sTGC segments.

NSW-TP output interfaces
The outputs of both TP algorithms are segment candidates.
The NSW-TP requires a valid track segment to be within a configurable angle (up to ±15) of the corresponding infinite momentum track through the IP.
Each 24-bit segment includes a radial $R$-index, an azimuthal $\phi$-index, a track angle $\Delta\theta$, and resolution flags that indicate whether the segment is from the sTGC (sTGC segments have one bit lower $\phi$ resolution than MM), or whether one quadruplet had only a 3-out-of-4 coincidence, which implies worse $\Delta\theta$ resolution.
The sTGC TP algorithm produces up to four segment candidates, and then accepts up to eight segment candidates from the output of the MM TP algorithm running on the same mezzanine card.
The TP algorithm running on the sTGC FPGA then merges the two lists of segments.
The merging procedure removes duplicates and drops segments beyond the eight allowed.
Priority is currently given to sTGC segments. There are options to ignore one or the other of MM or sTGC segments and in case of duplicates, to take the $\phi$-id from MM and the other variables from the sTGC segment. <cit.>

The surviving segments are then sent via fast serial links from the NSW-TP to the SL described in Section <ref>, where endcap Muon candidates are formed by successfully matching NSW track segments with coincidences found in the EM-TGC (Big Wheels).
Each NSW sector sends the track segment data to the SL via optical fibres: up to eight candidates per sector per BCID on two fibres, each running at 6.4Gb/, four candidates per fibre,
including the sector ID and BCID.
Seven copies (14 links) are sent out from the NSW-TP  sTGC FPGA
to up to seven different endcap SL modules.
(A single SL board receives data from at most six NSW trigger sectors, but each NSW trigger processor may need to deliver data to up to seven SL boards.
This is needed to cover the overlap of NSW sectors with EM-TGC sectors, with multiple scattering, misalignments and magnetic field bending also taken into account.)
For an overview of the NSW trigger-path electronics see Figure <ref>.
The whole process repeats for every BCID with a fixed latency of 1075
from the time of collision to the time the signal reaches the SL.

Via the ATCA card's two FPGA and Rear Transition Module, each mezzanine has two fibre connections to FELIX, one for sTGC and one for MM, carrying, on different e-link, the L1A event readout, exception messages, statistics, sampled events for monitoring, algorithm parameters, and TTC signals.
The BC clock recovered from these links is distributed to the various FPGA and transceiver clocks via jitter cleaners.

A Zynq SOC FPGA on the ATCA card handles configuration of all the FPGA and communicates various temperatures and voltages from elements of the  NSW-TP via Ethernet to monitoring applications.
An IPMC card on the ATCA card communicates critical temperatures and voltages to DCS via the ATCA Shelf Manager.
If any temperature exceeds a configurable threshold, the shelf manager can shutdown a complete ATCA board or specific mezzanines.

§.§.§ New Endcap Sector Logic

Muon track candidates are reconstructed based on coincidence logic and assigned to transverse momentum intervals by the
sector logic boards.  For , new sector logic boards have been designed in order to receive the information from new detector components, and apply additional coincidence requirements.  Inputs to the new sector logic boards include information from the legacy EM-TGC, EIL4 TGC, and Tile calorimeters, as well as the new RPC-BIS78 and NSW chambers.  The new sector logic boards utilise a Xilinx® Kintex® -7 FPGA, featuring Multi-Gigabit Transceiver technology (GTX); each board has 12 GTX interfaces and 14 G-Link receivers.  A sector logic board is shown in Figure <ref>.

A L1Muon sector logic board.

The “endcap” region ($|\eta| < 1.9$) is subdivided into 48 independent trigger sectors per side  in $\phi$, as shown in Figure <ref>, and 24 sectors per side in the “forward” ($|\eta| > 1.9$) region.  A sector logic board processes information from two adjacent sectors.  In total, 72 new sector logic boards are required to cover the entire L1Muon endcap trigger system.

Schematic of the L1Muon endcap trigger segmentation for one octant of the detector, showing independent trigger sectors outlined in green and RoI outlined in red.  An example of the grouping of the detector elements into a trigger sector is shown on the left side of the figure.

A block diagram of the L1Muon trigger sector logic is shown in Figure <ref>.  Hit positions in the third station of the EM-TGC ($\eta_{\mathrm{M3}}, \phi_{\mathrm{M3}}$) are extrapolated back to the IP, as shown in Figure <ref>, thus determining the “road” corresponding to the straight track of a muon with infinite momentum.  Deviations from the centre of these straight roads ($\Delta R, \Delta\phi$) depend on the track momentum and are computed at the first TGC station (TGC-M1).
Inputs from the NSW to the new sector logic include the position of the NSW track segment ($\eta_{\mathrm{NSW}}, \phi_{\mathrm{NSW}}$) and the deviation of the reconstructed track segment with respect to the IP ($\Delta\theta$).

Block diagram of the L1Muon sector logic.  The Big Wheel (G-Link) inputs are shown in blue and the NSW (GTX) inputs are shown in red.

Overview of the L1Muon endcap trigger system, showing the tracking information used to compute the trigger decision.
The solid line shows the trajectory of a muon with finite momentum, the dotted line shows the trajectory of a hypothetical infinite momentum muon used to define the centre of the trigger road.

Look-up tables are used to determine the $R-\phi$ coincidence between signals from the EM-TGC and the NSW, EIL4 TGC chambers, Tile calorimeter D-layer cells, or RPC-BIS78 chambers. Tracks accepted by the sector logic are divided into intervals corresponding to the programmable thresholds.  Up to four of the highest-tracks per sector are sent to the MuCTPI.  RoI, spanning an area of $\eta \times \phi = 0.025 \times 0.033$, are sent to the HLT.

§.§.§ RPC Feet and Elevator Chambers

Muon triggers in the barrel region ($|\eta| < 1.05$) are provided by three concentric layers of RPC doublets.  The L1 trigger decision in the barrel region relies on coincidence logic.  For the low-thresholds, a coincidence of three out of four layers in the middle station is required.  The high-trigger thresholds require the low-trigger logic to be satisfied, as well as an additional hit in one of two layers in the outer barrel station.

During , the sectors 12, 13 and 14 of the barrel spectrometer had a trigger coverage approximately 20% lower than the other sectors due to the presence of the toroid feet support structure (sectors 12 and 14, corresponding to $-2.16 < \phi < -1.77$ and $-1.37 < \phi < -0.98$) and the elevator shaft ($|\eta| \simeq 0.7, \phi \simeq -1.57$).  In LS1, additional chambers were installed in the toroid feet and elevator regions in order to increase the L1Muon barrel trigger coverage by about 3%, as described in Section <ref>.

In the feet region, the new chambers consist of two layers of RPC doublets, instead of the usual three layers.  A two-layer coincidence is required for the high-thresholds, but since there are fewer doublet layers in this region, the fraction of fake muon triggers in this region is higher than in the rest of the barrel.  The impact of the new chambers in the feet region can be seen in Figure <ref>.

Trigger efficiency for muons with $\pT > \SI{15}{\GeV}$ and satisfying a three-station coincidence requirement in the barrel region.  The increased efficiency due to the additional chambers in the feet region can be seen for the two $\phi$-sectors where the chambers were added: fig:TDAQL1MuonBarrelFeetSector12 Sector 12 and fig:TDAQL1MuonBarrelFeetSector14 Sector 14.

§.§.§ RPC BIS78

In the regions of the inner barrel (BI) that receive the highest background rates, additional RPC modules have been deployed, as discussed in Section <ref>.  These new BIS78 (Barrel Inner Small <cit.>) chambers, covering $\eta$ stations 7 and 8, are located on the A-side of ATLAS, in the transition region between the barrel and endcap ($1.0 < \eta < 1.3$).  The high background rate in this region is due to secondary charged tracks originating from beam halo protons and a lack of detector shielding and instrumentation.  The BIS78 chambers provide stand-alone detection and localisation of charged tracks and have an angular resolution of 3, which provides improved trigger rejection of low-muons.  These new chambers are a pilot project for the Phase-2 BI upgrade, which is planned to compensate for the reduced efficiency of the RPC in the HL-LHC environment, and increase the geometrical acceptance of the muon trigger in the barrel with BI-BO combinations.

The trigger logic for the BIS78 chambers is performed on Pad boards <cit.>, which are installed on each station.  The Pad is an FPGA-based board which collects the BIS78 RPC hit data from the front-end electronics over 18 serial links at 1.6Gb/.  It selects muon trigger candidates by requiring a local 2/3 coincidence of the RPC hits, applies a zero-suppression algorithm, and then sends the trigger information to the endcap sector logic board (described in Section <ref>), located off the detector.  Readout data are sent to FELIX (Section <ref>) through optical links via a gigabit transceiver (GBTx) chip <cit.>.

The Pad board utilises a Xilinx® Kintex® -7 FPGA, which supports optical transmission with fixed latency and provides robustness against radiation, including both single-event upsets and total ionising dose effects.

The expected rate reduction expected from the BIS78 RPC is illustrated in Figure <ref>.

§.§ L1 Topological Trigger

The ATLAS physics programme relies significantly on electroweak-scale processes involving hadronically-decaying tau leptons, jets, and , such as $H \rightarrow \tau\tau$ and $ZH \rightarrow \nu\overline{\nu}b\overline{b}$.  Also of importance are processes with unique topologies, such as $B$- or $J/\psi$-meson decays to low-leptons or vector-boson fusion production of a Higgs boson, which then decays invisibly (VBF $H \rightarrow \mathrm{invisible}$).  These signatures have large multi-jet backgrounds, and so the ability to reject the background and improve signal purity while still maintaining low trigger thresholds is of critical importance.

To accomplish this, a new L1Topo system was introduced in and commissioned in 2016.  It consisted of a single ATCA shelf with two FPGA-based processor modules which performed selections based on geometric or kinematic observables of TOB received from the L1Calo and / or L1Muon systems, e.g $\Delta\eta, \Delta\phi, \Delta R$, and invariant mass ($M_{\mathrm{inv}}$).  The details of the hardware design for the system may be found in <cit.>.

The L1Topo system has been redesigned for .
It consists of three ATCA modules: one module computes L1Calo trigger multiplicities (Topo1)  and the other two (Topo2 and Topo3)  apply topological selections as in .
(In the legacy system, the multiplicity task was performed by the L1Calo CMX <cit.>.)
Each module includes two Xilinx® Ultrascale+ processor FPGA for algorithm computation, which provide increased processing power compared to the version.  Each FPGA has 118 input and 24 output fibres.  Each module also contains two mezzanine cards: one for power and control, and another for communication with the CTP.  An L1Topo module is shown in Figure <ref>.

A production L1Topo module.

L1Topo receives TOB for jets, ,  e/$\gamma$ clusters, and muons from the L1Calo and L1Muon systems via the TopoFOX optical plant (described in Section <ref>).  The data received include the object type, ($\eta, \phi$) coordinates, and transverse energy of the object, and its isolation (in the case of e/$\gamma$ objects).  Table <ref> summarises the number of TOB per object type sent to L1Topo.

A summary of the maximum number of TOB sent to L1Topo for each object type.

Object     Number of TOB

e/$\gamma$ (eFEX)     144

Tau (eFEX)     144

Forward e/$\gamma$ ( jFEX)      5

Large Tau (jFEX)     6

Small-radius jet (jFEX)     168

Large-radius jet (jFEX)     24

(jFEX)     7

(jFEX)     7

Small-radius jet (gFEX)     6

Large-radius jet (gFEX)     3

(gFEX)     3

(gFEX)     1

Muon     32

The L1Topo algorithms are implemented in a pipelined, deadtime-free manner using a fixed latency.  Algorithms are distributed across the FPGA as evenly as possible such that optimal resource usage is achieved.  The configuration of algorithms, as well as their configurable parameters, are stored in the trigger menu.

The real-time output data sent by L1Topo to the CTP consist of individual bits indicating the algorithm decisions as well as an overflow bit for the topological algorithms.  The output of the multiplicity algorithms includes multiple bits indicating the number of TOB fulfilling the requirements of a given algorithm.  The outputs are sent to the CTP via optical or electrical cables.

Upon receipt of an L1A, the L1Topo real-time output data are captured and sent to the DAQ system.  Readout to FELIX and the SW ROD is done via the ROD as described in Section <ref>.  The clock and TTC signals are received from the Hub module via the backplane.

Control and configuration are performed via an IPbus interface on every FPGA, as well as on a control FPGA which controls the entire module.  Voltage and temperature are monitored by the IPMC via an I2C bus and are provided to DCS by the shelf manager.

Three types of L1Topo algorithms exist:

* Sort / select / no-sort algorithms These algorithms take all input TOB and convert the various TOB formats to a single global data format.  They also sort the TOB by or ,  or select all TOB with or exceeding a configurable threshold value, or satisfying object-specific criteria, such as isolation or substructure requirements.  The output consists of reduced lists of sorted TOB, which may then be used as inputs to the decision algorithms, described below.  This reduction is necessary to handle the otherwise excessively large combinatorics.  The final lists contain six TOB in the case of “sort” algorithms and 10 TOB for “select” algorithms.  “No sort” algorithms only perform the conversion to a global data format and are used as inputs to the multiplicity algorithms.  The latency of these algorithms is two bunch crossings (50).

* Decision algorithms These algorithms determine whether a given trigger condition has been satisfied; examples include selections on $\Delta\eta$, $\Delta\phi$, and $M_{inv}$.  Their outputs include decision and overflow bits which are sent to the CTP.  The latency of the decision algorithms is limited to one bunch crossing (25).

* Multiplicity algorithms These algorithms count the number of objects passing a given threshold, e.g. , or located within a given region in $\eta$.  The output multiplicity bits are transmitted to the CTP.  The latency of the multiplicity algorithms is limited to three bunch crossings (75).

Configurable parameters (e.g. threshold values) of the algorithms are specified in the trigger menu.  The three algorithm types are illustrated in block format in Figure <ref>.

The L1Topo algorithm structure is illustrated in fig:TDAQL1TopoAlgoBlock1. TOB inputs are first passed to sort / select / no sort algorithms, which produce shortened lists of TOB, which are in turn passed to the decision algorithms. Subfigure fig:TDAQL1TopoAlgoBlock2 shows the multiplicity algorithm structure
which has as input all TOB produced by eFEX, jFEX and gFEX.   These TOB inputs are first passed to “no-sort” algorithms (labelled as “Decoding” in the figure).  The resulting lists of TOB in a global format are then passed to the multiplicity algorithms; the output multiplicity bits are sent to the CTP.

§.§ Central Trigger & TTC

The L1 CTP receives trigger information from the other L1 trigger systems and forward detectors and executes the final L1 trigger decision.  Muon trigger inputs are received via the MuCTPI, which is described in Section <ref>. The CTP itself is described in Section <ref>.

§.§.§ Muon to central trigger processor interface

Functional overview
The muon barrel and endcap trigger processors send their results as input to the
MuCTPI, which has been redesigned and replaced during the Phase-I upgrade <cit.>.  The replacement was necessary in order to provide full-granularity muon information at the BC rate to the L1 topological processor and to be able to interface to the sector logic modules using high-speed optical links.

The MuCTPI receives information for up to four muon track candidates per muon trigger sector from the endcap trigger sector logic modules, and up to two candidates from the barrel trigger sector logic modules.  The information includes the position and threshold passed by the track candidates (15 thresholds for the endcap, and six thresholds for the barrel), along with additional information, such as track quality flags, geometrical flags, and in the case of the endcap, a flag indicating the electric charge of the muon candidate.

The MuCTPI processes the information in three parallel paths:

* It combines the information from all trigger sectors to calculate the total multiplicity of muon candidates per muon threshold and sends the multiplicities to the CTP for each bunch crossing.  Up to 64 bits of multiplicity information can be sent to the CTP for each bunch crossing.  The multiplicity can either be a 2- or 3-bit value, and the maximum multiplicity value for a given threshold includes the cases with even more muon candidates.

* It sends muon position and transverse momentum information of selected muon candidates to the L1 topological trigger system, to be used in subsequent topological trigger algorithms in combination with calorimeter information.  The muon candidates sent are ordered according to decreasing .

* It can apply muon-only topological trigger algorithms to be sent to the CTP.

Care is taken to avoid double-counting of muons which traverse more than one detector region due to geometrical overlap of the chambers and the deflection of the muons in the magnetic field.  Many cases of overlaps are resolved within the barrel and endcap muon trigger processors.  The remaining overlaps to be treated by the MuCTPI are those between neighbouring trigger sectors.

The MuCTPI also provides data to the HLT and to the data acquisition system for events selected at L1.
A subset of the muon candidate information, ordered by is sent to the  HLT to be used as RoI for further processing.  The DAQ system records a more complete set of information, including the computed multiplicity values, which is used to monitor the functions of the  MuCTPI.

System implementation

The MuCTPI module shown fig:TDAQMUCTPIBlock as a block diagram and fig:TDAQMUCTPIPhoto in a photograph.

The Phase-I MuCTPI, shown in Figure <ref>, is integrated on a single ATCA blade, replacing the 18 9U VME cards of the system.  The architecture is based on a highly integrated generation of FPGA, featuring a large number of on-chip MGT as well as 12-channel ribbon fibre optics receiver and transmitter modules (MiniPODs) for the data transfer.

The two Muon Sector Processor FPGA – each taking care of one detector hemisphere – receive and process muon trigger data from 208 inputs from sector logic modules connected through high-speed serial optical links using MiniPOD receiver modules.  The Muon Sector Processor FPGA also copy information on selected muon trigger objects to several L1Topo modules using MiniPOD transmitter modules.

The Trigger and Readout Processor FPGA merges the muon multiplicity information received from the two Muon Sector Processor FPGA and sends the results to the CTP.  In addition, it can also implement muon topological trigger algorithms.  This is possible because all the trigger information is available in a single module with low latency.  The same FPGA also receives, decodes and distributes the TTC information.

A SOC is used for configuration, control and monitoring of the module.  The device integrates a programmable logic part with a dual-core ARM processor subsystem.  The processor subsystem runs the required software to interface the MuCTPI to the ATLAS run control system through a GbE interface.  It will also be used for environmental monitoring of components of the board such as the power supply, optical modules, and FPGA.  The values read include voltages, currents, temperatures, optical input power, clock status, etc.

§.§.§ Central Trigger Processor

Functional overview
The CTP is the last stage of processing of the Level-1 trigger system.  It receives digital trigger information from the L1Topo, legacy L1Calo, MuCTPI and ALFA systems, and from various forward detectors.  The CTP system used during has been described in <cit.> and has since been upgraded.

Table <ref> shows the various inputs of the CTP.  Three different input paths are available:

* The traditional electrical trigger path via the CTPIN modules is used for trigger signals coming from various forward detectors, for calibration signals from some sub-detectors, and for special triggers such as a filled-bunch trigger based on beam-pickup monitors, and a minimum-bias trigger based on scintillation counters.  It is also used for the legacy L1 calorimeter trigger system during the start-up phase of and the commissioning of the L1 topological trigger system.

* The electrical low-latency input via the CTPCORE+ module, which is used for latency-critical trigger signals, such as those from the L1 topological processor, muon-only topological triggers from the  MuCTPI system, and signals from the ALFA detector.  For the L1 topological processor, these are signals from the Topo2 and Topo3 modules: Topo2 provides topological algorithms that combine jets with one or more other objects, comprehensive combinations of  jet and $\tau$, and jet and
EM clusters, along with muons and missing transverse energy where appropriate, while Topo3 is largely dedicated to topological triggers typically involving electrons, ranging from the standard model performance triggers, to exotics triggers involving many lepton flavours.

* The optical input via the CTPCORE+ module is used for the muon threshold multiplicities from the MuCTPI and for non-latency critical L1 topological algorithms, mainly coming from the Topo1 module.  These include simple multiplicity triggers on electromagnetic,
hadronic $\tau$, and jet objects, energy sums, missing transverse energy, and other fast topological algorithms.

Overview of the trigger inputs to the CTP in .

CTP input     Cable origin     Number of bits     Trigger information

CTPIN      Various sources     36     Forward detectors, calibration, special triggers

20.15CTPCORE electrical     MUCTPI/ALFA     20.1577$-$149     Muon-only topological algorithms and ALFA
Topo2, Topo3        Topological algorithms

CTPCORE optical     MuCTPI     64     Muon threshold multiplicities
Topo1     263$-$335     Simple multiplicity triggers, energy sums, missing energy

Block diagram of the trigger path in the CTP.

The trigger path of the CTP is shown in Figure <ref>.  Up to 512 trigger inputs can be used in programmable look-up tables to form trigger conditions from the input signals, as described in more detail below.

Each trigger item can be put in coincidence with any combination of the 16 bunch groups, which are programmable collections of bunch-crossings.

In the next step, the rate of each trigger item is controlled by pre-scaling, where an algorithm rejects pseudo-randomly on average a certain fraction of the trigger item signals.  The fraction is called the pre-scale factor and can be individually set for each trigger item.  Each trigger item can subsequently be enabled or disabled to be used in the trigger decision.  The L1A signal generated by the CTP is the logical OR of all enabled trigger items.

For each L1A signal, the CTP provides an 8-bit trigger-type word signal, whose bits indicate certain categories of types of triggers and can be used to select options in the event data processing and monitoring in the sub-detector readout chain.  For each L1A signal, the CTP also sends information about the trigger decision for all trigger items to the HLT and to the data acquisition system.  The CTP can support up to three data-taking partitions for calibration and test runs.  The trigger items after pre-scaling are partitioned and assigned to the three partitions, each of which individually enables the trigger items of interest, gates them with the veto signal, forms the L1A of the partition, and calculates the deadtime taking into account the L1A and the BUSY signals of the corresponding partition.

In addition to its function as trigger processor, the CTP is also the timing reference of the detector.  It receives the beam-synchronous clock and timing signals from the LHC and distributes them together with the L1A signal to the ATLAS sub-detectors, preserving their timing information.  It also interfaces with the LHC GPS timing system and attaches a nano-second precision absolute time-stamp to the read-out data of each L1-accepted event.  In addition, it is in charge of generating luminosity blocks, which divide a data acquisition run into small time intervals of typically one minute.  These luminosity blocks are the shortest time interval for which the integrated luminosity, corrected for deadtime and pre-scale effects, can be determined.  They are also used as the time interval for online and offline data quality assessments.

The CTP features many monitoring facilities that allow the monitoring of the rate of each incoming trigger signal, and of each trigger item at the different stages of trigger processing in the CTP.  These rates are determined, published online, and preserved for offline usage, every few seconds and on luminosity block boundaries.  Some of the trigger rates, such as those before and after veto, are used to correct the integrated luminosity for deadtime losses.  The deadtime and its components are monitored directly by corresponding deadtime counters.  In addition, the CTP has counter facilities that allow the monitoring of trigger rates and deadtime fractions per BCID and thus the study of bunch-by-bunch effects of the trigger rates and deadtime.

System implementation

Block diagram of the various components of the CTP. Components marked with $+$ were newly built during LS1, while components marked with $*$ use the hardware, but with upgraded firmware. The firmware of the CTPCORE+ was further upgraded during LS2.

The CTP for has been described in detail in <cit.> and has since been modified during LS1 and LS2.  The upgraded CTP system consists of six types of modules which are housed in a single 9U VME crate, as shown in Figure <ref>:

* The CTPMI module as the machine interface module.

* Three CTPIN modules to receive trigger input signals.

* The CTPCORE+ module to receive additional direct trigger input signals and implement the trigger logic.

* The CTPMON module for per-bunch-monitoring of trigger input signals from the CTPIN.

* Five CTPOUT+ modules as interfaces to the sub-detector TTC distribution.

* One CTPCAL module to receive calibration request signals from sub-detectors and receive electrical signals.

* Two NIM2LVDS boards to receive further electrical signals.

The controller CPU interacts with the modules via the VME backplane.  Internal communication between the modules proceeds by custom bus systems implemented as custom-built backplanes installed in the CTP crate: The PITbus is used for routing synchronised and aligned trigger inputs from the CTPIN boards to the CTPCORE+ module, the COMbus+ is for the interchange of common timing, trigger and control signals between all the modules, and the CALbus is used for sub-detector calibration requests.

For , the following components of the CTP were upgraded:

* The firmware of the 3 CTPIN modules was upgraded to drive trigger input signals to the PITbus in double-data rate at 80, hence doubling the number of input signals available on the PITbus from 160 to 320.

* The CTPCORE board was replaced by the newly designed CTPCORE+ board, which allows trigger logic to be applied to up to 512 trigger inputs (to be compared with 256 inputs in ).  The inputs are a programmable selection of signals from the PITbus and additional 192 direct electrical inputs from the L1Topo trigger processor and the ALFA sub-detector.  The CTPCORE+ can form up to 512 trigger items (256 in ) that can be put into coincidence with up to 16 bunch groups (eight in ).  It also features the possibility to support up to three data-taking partitions for calibration and test runs.

* The four CTPOUT boards have been replaced by five newly designed CTPOUT+ boards, each supporting running with up to three concurrent data-taking partitions, and featuring additional monitoring and testing facilities.

* The COMbus backplane was replaced by the newly designed COMbus+ backplane, which allows the distribution of timing signals for up to three concurrent data-taking partitions, and includes the communication with a fifth CTPOUT+ board.

* The CTPMON firmware was updated to select up to 160 trigger inputs from the PITbus for per-bunch monitoring.

For , the CTPCORE+ firmware has been updated to receive optical trigger signals from L1Topo via up to 12 optical fibres running at a link speed of 6.4GBaud, corresponding to 96 trigger bits per optical fibre.  A programmable switch matrix maps the incoming trigger signals to up to 512 usable trigger inputs.

The timing signals from the LHC are received by the CTPMI, which can also generate these signals internally for stand-alone running.  This board also controls and monitors the internal and external busy signals; for example, the busy signal transmitted from a sub-detector in case of overload on its data acquisition system.  The module sends the timing signals to the COMbus+, thereby making them available to all of the other modules in the CTP.

The CTPIN modules receive trigger inputs from forward detectors, the legacy L1Calo system, and from various other sources, such as calibration systems, the beam-pickup monitors, and a minimum-bias trigger based on scintillation counters.  The input boards select and route the trigger inputs to the PITbus, after synchronising them to the clock signal and aligning them with respect to the bunch-crossing.  Three boards with four connectors of 31 trigger input signals each allow for a total of 372 input signals to be connected, of which up to 320 can be made available on the PITbus at any given time, using double-data rate at 80.

The trigger decision module (CTPCORE+) receives the trigger inputs from the PITbus and from two external sources: via three electrical connectors of 64 trigger input signals each, and via up to 12 optical fibres containing up to 96 trigger bits each.  A selection of these trigger input signals is made using a programmable switch matrix.  The CTPCORE+ module combines the trigger input signals from the PITbus, the electrical and the optical CTPCORE+ inputs, and internally generated signals, using several programmable look-up tables to form up to 512 trigger conditions.  In a further step the trigger conditions are combined using content-addressable memories to form up to 512 trigger items.  Any of the up to 512 trigger conditions may participate in any of the up to 512 trigger items.  The trigger items are put into coincidence with individual combinations of up to 16 bunch groups, subsequently undergo pseudo-random prescaling and deadtime gating, and can be individually enabled or disabled to take part in the final L1A trigger decision.  The trigger results are transmitted to the COMbus+.  The CTPCORE+ module also acts as the readout driver of the system, sending information to the HLT trigger and the data acquisition for each accepted event.

The output module (CTPOUT+) receives the timing and trigger signals from the COMbus+ and fans them out to the sub-detectors.  The module receives back from the sub-systems the busy signals, which are sent to the COMbus+, and 3-bit calibration trigger requests, which are routed to the CALbus.  The calibration module time-multiplexes the calibration requests on the CALbus and sends them via a front-panel cable to one of the input modules.  The calibration module also has front-panel inputs for beam pick-up monitors, minimum-bias scintillators, other forward detectors, and test triggers.  In addition, two NIM2LVDS modules provide further front-panel inputs for such trigger signals.

The TTC signals of the CTP are distributed electrically to the sub-detectors via the CTPOUT+ links, through which in return the sub-detector busy signals and some calibration request signals are received.  A local trigger system acts as an interface between the CTP and the TTC distribution of each sub-detector, and imitates the function of the CTP during stand-alone data taking of the sub-detector.

During , the local trigger system consisted for all sub-detectors of a series of custom 6U VME electronics boards <cit.>: an LTP module as the TTC switch board and generator of local signals, an LTPI module for interconnections between sub-detector partitions, a TTCvi for the serialisation of the TTC information, and a TTCex for the optical transmission to the sub-detector front- and back-end electronics, where TTC receiver chips (TTCrx) decode the transmitted information and make it available as electrical signals for further use.

During LS2, the ALTI has been introduced as the local trigger system for new Phase-I sub-detector partitions and to replace the legacy set of TTC modules (LTPI, LTP, TTCvi and TTCex) for all sub-detectors, except for the RPC, TGC, and LUCID detectors.

The ALTI module provides the same electrical and optical interfaces as the legacy set of TTC modules in a single 6U VME electronics board.  Using a single modern FPGA, all the functions of the legacy set of TTC modules are replicated.  New, useful functions were added, such as an optical input and memory to analyse optical TTC signals, memories to store incoming electrical signals, additional monitoring features including per-bunch monitoring of trigger signals, and a range of specific functions serving to generate L1A sequences similar to the ones from the CTP.  These latter functions include a programmable look-up table for defining the trigger logic of the trigger inputs and generated trigger signals of the ALTI board, pseudo-random trigger generators, pseudo-random pre-scaling, bunch group masking, and simple and complex deadtime generation.  Where possible, the same algorithms as in the CTP were used.

From the ALTI module or the legacy set of TTC modules, the timing signals are distributed to the detector electronics using the TTC system.  The implementation and use of the TTC system is sub-system specific.  As an example, the muon trigger systems use TTC standard components to transmit the timing signals all the way to the electronics mounted on the chambers, while in case of the inner tracking detector a custom-built distribution system is used to transmit the signals from the counting rooms to the cavern.

§.§ DAQ/HLT

§.§.§ *FELIX/*SW ROD

FELIX and the SW ROD are new detector readout components introduced into the ATLAS DAQ system for . FELIX is designed to act as a configurable data router, receiving packets from detector front-end electronics and transferring them to peers on a commodity high-bandwidth ethernet network. Whereas previous detector readout implementations relied on diverse custom hardware platforms, known as ROD, as the interface between the detector electronics and the common DAQ chain, the motivation for FELIX is to unify all readout across one well supported and flexible platform. As well as its readout function, FELIX will also serve as a relay for trigger accept and clock information from the TTC system to front-end electronics. It will also be possible to use FELIX to send general purpose control data to front-end electronics to manage modules throughout data taking and calibration.

Another key aspect of the FELIX concept is to take advantage of advances in technology, for example: larger and faster FPGA, the advent of multicore CPU and high-performance networking, to move tasks which were previously performed in customised hardware (the aforementioned ROD) into the more flexible firmware and software domains. In this new architecture, with FELIX acting as a simple router, detector data processing, monitoring and control functions are instead implemented in software hosted by commodity server systems subscribed to FELIX data. The primary peer on this network will be the , which will perform functions such as event fragment building, data formatting and other detector-specific processing to prepare data and facilitate online selection. The will also buffer event fragments and supply them on request to HLT nodes, via an identical interface to the legacy ROS.

FELIX and -based readout paths will exist alongside the legacy ROS system for the duration of . An overview of the architecture is presented in Figure <ref>. FELIX and installations were deployed during LS2 for systems undergoing significant detector or readout upgrades in preparation for operation. These are the NSW, LAr digital readout and new L1Calo systems described earlier in this chapter. Smaller scale demonstrators for upgraded Barrel RPC-BIS78 and the Tile Calorimeter were also installed during this shutdown. In total the installation comprises about 60 FELIX servers (hosting 100 I/O cards between them), with 30 software ROD servers. This is similar in size to the legacy ROS system in , which consists of approximately 100 servers. The remaining ATLAS systems will then all be migrated to FELIX
during the next long shutdown before the HL-LHC, resulting in the final decommissioning of the legacy ROS.

Comparison of Readout Architecture in (right-hand side, with hardware ROD) and (pink box on left-hand side) showing new FELIX readout paths. HLTPU are able to sample event data from both paths via an identical interface.

FELIX systems are able to interface with front-end electronics over one of two optical link protocols: GigaBit Transceiver (GBT <cit.>), a radiation-hard standard developed at CERN, where multiple lower speed links (E-links) from separate pieces of electronics can be aggregated into a single 4.8Gb/ link; and FULL mode, an in-house design with no link substructure for higher bandwidth (9.6Gb/) communication between FPGA. Data streams for either protocol can be configured to use different encoding, although 8b10b is typically used for normal dataflow.

*FELIX & *SW ROD Server Hardware

Each FELIX server is 2U high and hosts custom I/O cards (FLX-712, described in Section <ref>), with firmware to interface with either of the two link protocols. For the GBT case each server hosts two cards, for the higher bandwidth FULL mode case each server hosts one card (driven primarily by the number of available PCIe lanes). Each server also hosts high-bandwidth network interface cards (dual-port 25 GbE for GBT, dual-port 100 GbE for FULL mode). Each FELIX server has an Intel® Xeon® E5-1660 V4 CPU (8 cores  3.2) and 32 GB of DDR4 ECC RAM. Each server is 1U high, featuring dual Intel® Xeon® Gold 5218 CPUs (16 cores  2.3) and 96 GB of DDR4 ECC RAM. Each server also hosts a dual-port 100 GbE network interface.

*FELIX I/O Card

The FLX-712, shown in Figure <ref>, is a PCIe  card supporting a 16-lane Gen 3 interface, able to reach a throughput of up to 100 Gb/s. An MTP 24 or 48 coupler provides the interface to external data fibres, after which the light is internally routed to one of eight MiniPOD transceivers (four for reception and four for transmission) handling 12 links each. A maximum of 48 bi-directional optical links can therefore be connected to each board. A Xilinx® Kintex® Ultrascale™ (XCKU115) FPGA provides the platform for all on-board firmware features. An on-board PEX8732 PCIe  switch makes it possible to map two separate PCIe  8-lane endpoints into one 16-lane bus. A JTAG connector is provided to facilitate FPGA configuration, though this may also be stored in an on-board FLASH chip. FPGA programming and card health monitoring and control are also possible over PCIe.

Finally, an interchangeable mezzanine card provides an interface for a number of timing and control systems. The ATLAS TTC system in connects to FELIX via an optical fibre for the distribution trigger and clock information and a LEMO connector for the receipt of BUSY signals. An SI5345 jitter cleaner on-board the FLX-712 itself ensures a sufficiently good quality clock for all FELIX use cases.

FELIX hardware platform (FLX-712), with key components labelled.

During LS2,
approximately 60 FELIX servers hosting
a total of 100 FLX-712 cards
deployed, routing data to 30 systems. A
significantly larger number (of order 6 times more) will be deployed in LS3 to service all remaining ATLAS systems.

*FELIX Firmware

The FELIX firmware, a diagram of which can be found in Figure <ref>, is designed to be modular and flexible. Separate components manage different key functions, such as the link wrapper (GBT or FULL mode) and the PCIe  and DMA <cit.> engines. Between these lies the Central Router module, which performs the most data intensive workload. Here data arriving over different links are decomposed according to protocol and converted into regular 1kB elements for optimal DMA transfer to the host server's memory. In order to optimise FPGA resource utilisation and timing, the FELIX firmware deployed in the FLX-712 consists of two duplicate paths with identical modules, each servicing half the input links and reading out to an 8-lane PCIe  interface. As such, each FLX-712 card appears to the host server as two 8-lane devices.

Diagram of the firmware deployed on the FPGA of the FLX-712 in GBT mode. In FULL mode the incoming (RX) GBT link wrapper is replaced with a dedicated FULL mode module. The Central Router is also much simpler in this case, as the data processing requirements are less severe.

Alongside the primary dataflow path, a separate common module interacts with the TTC system, injecting trigger and clock signals into the data path and relaying BUSY signals back to the central trigger should operating conditions require a pause in dataflow. The TTC module is also responsible for generating an information packet for each trigger accept received on a special stream for downstream subscribers to use to facilitate event fragment building and synchronisation. Other common modules also manage configuration registers, clock control/distribution and general housekeeping.

By re-using the basic blocks above it is possible to flexibly produce firmware designs for different use cases. For ATLAS, separate designs are maintained for both GBT and FULL mode, where the primary differences are the link wrapper and the complexity of the Central Router (which is significantly lower for FULL mode). Due to FPGA resource utilisation constraints the maximum number of GBT links which can be supported for primary dataflow is 24. For FULL mode the limitation comes from the PCIe  bandwidth of the FLX-712, which can accommodate a maximum of 12 links. However, the standard approach is to build firmware to support up to 24 links, this giving the option to operate more than 12 at lower occupancy. Thus a higher link density can be provided within the same bus constraints.

*FELIX Software

The FELIX software suite comprises high- and low-level components. Alongside a dedicated device driver, low-level tools make it possible to test all firmware features in a laboratory setting and debug any issues which may arise. At a higher level, a high-performance daemon operates in an “always on” fashion in order to receive data from the FLX-712 over DMA and provide onward routing. DMA transfers are received via a separate ring buffer for each PCIe  device visible to the host server (hence two per FLX-712). The software daemon, based on the NetIO architecture <cit.> is designed to be event driven, able to react to hardware interrupts from the card, indicating incoming data, or signals from the network interface or operating system. The design is such that copies of the data in memory are kept to a bare minimum to maximise throughput. Finally, RDMA technology is used to transfer data to connected network peers without intermediate processing. An overview of the complete software stack is presented in Figure <ref>.

Diagram showing the different layers of the FELIX software architecture, with the central event loop (shown in green) able to interact with low level network software (supporting RDMA) and operating system signals (including interrupts from the FELIX I/O card). The higher-level layer implements application-specific logic and handles subscriptions from external clients via dedicated sockets, available both in peer-to-peer or publisher/subscriber mode. Sockets can also either operate in “buffered” mode, whereby data are packed into an output buffer before transfer across the network (useful to optimise throughput for small packets) or “unbuffered” mode, where data are transferred immediately (useful for larger packets).

The is implemented as an ATLAS Run Control-aware application running on dedicated servers. The application is designed to be able to support flexible workloads, with configurable event fragment building and processing. The overall architecture of the application is shown in Figure <ref>. At the input stage, dedicated “Reader” threads subscribe to FELIX data streams and write incoming packets into “slice” buffers, with each buffer made up of “vectors” of data packets corresponding to a specific L1ID for a specific input link. From here packets are placed into an associative “ROBFragment assembly” map structure, with all data in a previously assembled slice associated with their corresponding L1ID. Thus the aggregation from many packets per L1ID to one ROBFragment object happens in two steps. At this point, if required, fragments are placed into a “Ready” queue for handling by a subdetector-specific plugin, operating in a dedicated user processing thread. Within this thread, a variety of fragment analysis, error checking, monitoring and statistics gathering can take place, making it possible to replicate the actions of the old generation of hardware ROD which the replaces. Should such processing not be required fragments can be sent directly to the output stage. At this stage, fragments are held in a “ROBFragments” map while the HLT performs selection operations on the events of which they form part. A pool of request processing threads transfers any requested fragment data from the map to the HLT during this period. This final stage is designed to be functionally identical to the legacy ROS buffers, making use of a common interface to the HLT, such that any HLT node requesting data makes no distinction as to whether it comes from a ROS or a .

Architecture with subdetector plugins. Data are aggregated by common L1ID in the input stage before being passed (if needed) to dedicated subdetector code, before being buffered and transferred onward as needed during HLT processing. In this example only two Reader Threads are used for clarity, but in principle more can be configured as needed.

§.§.§ Network

As part of the preparations for , the TDAQ network is undergoing a major programme of updates. The overall design is based on two physical levels, with machines operating in the electronics cavern (FELIX, , ROS and various DCS and detector infrastructure nodes) communicating with systems on the surface (HLT, core infrastructure, control calibration and monitoring nodes) via a 40 GbE backbone. Different communication workloads are supported, from bulk dataflow to diverse control and monitoring traffic.

While the overall core throughput between the electronics cavern and the surface has not changed since (remaining at 40 GbE), the overall number of nodes connected to the network has increased significantly, with more extensive deployment of virtual networks to serve different functions, eliminating the need for further physical interconnects. The network's router infrastructure has also been replaced, both at the core router level (now two Juniper QFX10016s) and all client switches at various levels (now made up of 50 Juniper QFX5100), with improved performance and the introduction of active-active redundancy (based on MC-LAG technology). All routers now make use of the next generation of ASIC (Broadcom Trident II in the client switches and Juniper Q5 in the core routers). A mixture of copper and optical fibre connectivity is used, so as to optimise the cost for each use case. The overall architecture of the network for is shown in Figure <ref>, along with a more detailed description of node connectivity.

TDAQ Network for , showing upgraded core routers (QFX10016 in the centre) and new FELIX/network (top right). Client routers (QFX5100) act as the interface between the core and the processing farms performing online control, monitoring and calibration functions. Nodes participating in primary dataflow (such as ROS, , TPUs and SFO) are connected directly to the core. In the case of ROS, FELIX, and , separate connections for the purposes of node control are also present. The ATCN operates in parallel, connecting DCS and ATLAS core infrastructure, and interfaces to the TDAQ network via the core and via connections to hardware ROD components.

The majority of new clients on the TDAQ network belong to the new FELIX and readout paths. Each FELIX server hosts a dual 25 or 100 GbE network interface card, with each hosting a dual 100 GbE interface. The servers are connected via dedicated high-performance switches (a total of six Juniper QFX 5200), which feature lossless operation over RDMA and active-active redundancy. Each then features a separate dual 40 GbE interface to the common dataflow network, for control and bulk dataflow through to the HLT and beyond. Further new nodes may also be added to the network in future as the HLT farm continues to expand on the surface. As such, additional contingency has been factored into the specification of the core network to allow for such evolution to take place.

As mentioned before, the physical network is subdivided into separate VLAN, each with a dedicated quality of service policy, to enable efficient management of separate workloads without extra cabling. There are five separate VLAN in operation: Management, for network control and monitoring; Control, for server control (DHCP, NFS, DNS, etc.) and DAQ control; Data, for DAQ traffic; Monitoring, for Low-priority monitoring and backup (operated on a best-effort basis)
and finally SimP1: for simulation jobs running in the HLT farm. With this implementation, virtual router instances also provide traffic isolation and enhanced security (e.g. the ATLAS technical network is protected against traffic flooding).

Beyond the upgrades described above, significant effort has gone into reworking the cabling structure in the surface cavern and improving uninterruptible power supply coverage for the DAQ network.

§.§.§ HLT/AthenaMT

Event processing frameworks for most LHC experiments have traditionally been designed to process single events serially on a single CPU core, with events distributed between independent processing nodes. Unfortunately, for offline processing, such a model no longer matches trends in computer architecture. Crucially, while the number of CPU cores available in a standard compute node has increased, the amount of memory per core is increasing at a lower rate. In order to effectively make use of the increasing core count, the average memory utilisation per job must therefore be reduced. The most effective way to achieve this is to share memory between cores. This can be achieved by sharing event processing between multiple threads, i.e. multi-threading, thus reducing the overall memory footprint per core. Memory has never been a limiting factor for the ATLAS HLT, due to the different operational model, but the system can still benefit from any optimisation to gain additional margin for future evolution, and to aid with other workflows with differing resource limitations such as HLT simulation as part of MC event generation.

To tackle the memory issue, a significant redesign of the ATLAS software framework (Athena) was undertaken to allow it to process events across multiple threads. The new AthenaMT framework to be used for positions ATLAS software to optimally exploit future server technology evolution. An additional feature of AthenaMT is its built-in hooks to facilitate potential future use of co-processors, such as GPU and FPGA. Asynchronously offloading compute-intensive tasks to these devices can free up CPU cores for work better suited to the CPU. While the use of co-processors is not planned for the start of , the structure of AthenaMT means they can be integrated in future without major architectural changes.

The Athena framework is mainly written in C++, with a configuration layer in Python.  The underlying Gaudi framework <cit.> is shared with the LHCb experiment. The Gaudi layer defines the basic classes used for event processing, and also provides a component known as the scheduler, which is responsible for optimal algorithm execution.  In , the Gaudi scheduler was used by offline reconstruction,  but the HLT used a custom implementation: “Trigger Steering”. The steering layer was needed to implement additional HLT functionality, to facilitate features such as RoI-based processing and the sequencing of trigger selection criteria to arrive at a decision for each event. The Trigger Steering layer fulfilled these requirements successfully during , but with a significant development and maintenance overhead.

Taking into consideration the above, the key drivers of the design of the new AthenaMT framework were to reduce maintenance overhead and to make effective use of hardware. As already discussed, effective use of modern multi-core machines requires multi-threading. From the beginning, the new framework was designed to meet both offline and trigger requirements, eliminating the need for a custom trigger-specific layer. Data and control flow, as well as regional reconstruction, were designed to be part of the scheduler. AthenaMT makes it possible to implement three different kinds of parallelism: inter-event, where multiple events are processed in parallel; intra-event, where multiple algorithms can run in parallel for an event and in-algorithm, where algorithms can utilise multi-threading and vectorisation. A pictorial representation of the different options is presented in Figure <ref>.

Diagram demonstrating the three different kinds of parallelism available with AthenaMT: inter-event, where multiple events are processed in parallel; intra-event, where multiple algorithms can run in parallel for an event and in-algorithm, where algorithms can utilise multi-threading and vectorisation.

As described in Section <ref>, the trigger menu translates the physics priorities of the experiment into allocations of the total L1 and HLT rates. Each unique selection combination is called a chain, with the set of chains making up the menu. During , more than 2000 unique combinations of trigger selection criteria were employed to distinguish interesting events.  The configuration for each selection was generated from a large
Python “TriggerMenu” package. As it is important to know exactly which menu is used both for data-taking and production of simulated samples, the configuration is stored in a database.  As part of the software upgrade for , both the actual configuration and its database storage mechanism have been redesigned. The key requirement was to reduce the complexity of the configuration and to ease maintenance and development overhead while also improving the performance of the configuration serialisation/deserialisation. The reduced complexity is the result of a redesign of the configuration (both in the trigger and across Athena as a whole), while the improved serialisation/deserialisation performance comes from storing the configuration in a JSON <cit.> representation rather than in a relational format.

The AthenaMT software framework must interact with the online trigger and data-acquisition infrastructure, as well as new hardware triggers installed for . During , each node in the HLT forked multiple sub-processes, thus sharing memory between them via the copy-on-write mechanism. In , each forked sub-process will additionally run with multiple threads, and potentially process multiple events across these threads.  This design implies that tuning of the number of forks, threads and concurrent events will be necessary to ensure maximum performance. It also implies that if a sub-process crashes or takes too long, all concurrently processed events must be saved for offline reprocessing. A comparison of the and architectures is presented in Figure <ref>.

fig:hltmppu Athena process operating within the wider HLT processing framework in , where multiple forked instances performed all processing and fig:hltputree equivalent, potentially with fewer forks, with each fork capable of spawning multiple processing threads..

Image includes an icon made by Smashicons from <flaticon.com>.

§.§.§ Online Software & Monitoring

The ATLAS Control & Configuration software suite underwent a significant
overhaul for the start of . The features introduced have since been
refined throughout data taking; thus a modern and flexible system was in place at the
the start of . The goals of the upgrade
were three-fold: first, to properly accommodate additional requirements that
could not be seamlessly included during data taking; to re-factor software that
had been repeatedly modified to include new features, and make it more
maintainable; and to take the opportunity to modernise the software base,
making the most of the rapid evolution in information technology during the lifetime
of the LHC. This upgrade was carried out retaining the important constraint
to minimally impact the operational mode of the system or any public API, thus
making it as easy as possible for the large user community to integrate all the
changes. An example of the changes is the introduction of a completely redesigned
online monitoring archiving engine (P-BEAST) able to aggregate and serve data to
a number of clients (principally a Grafana web interface for the production
of time series displays). P-BEAST interfaces with the existing statistics distribution
system (IS) without API changes in the latter, but provides enormous improvements
in user experience and monitoring capability. An example of a typical online monitoring
time series display with P-BEAST and Grafana is shown in Figure <ref>.

Example dashboard produced with the Grafana web interface.

Alongside the tools already mentioned, ATLAS also deploys specific Data Quality monitoring tools
to aid shifters in the task of monitoring the performance of a data-taking session. The Data Quality Monitoring Framework automates data quality assessment by applying data quality algorithms to the histograms produced by event selection applications running in the HLT. The results produced by these algorithms are displayed to the shift crew in a hierarchical manner, so that the shift operator can quickly spot problems and easily identify their origin. This framework was further enhanced throughout with the addition the Data Quality Monitoring Archiver, a new feature to store results in ROOT format for offline analysis. An example of a web display produced by the tool's integrated archive viewer is shown in Figure <ref>.

Example screenshot of the Data Quality Monitoring Application.

§.§ Detector Control System

The purpose of the DCS <cit.> is to ensure coherent and safe operation of ATLAS and to serve as a homogeneous interface to all subdetectors and to the technical infrastructure of the experiment.
The DCS must be able to bring the
detector into any desired operational state, to continuously monitor
and archive the operational parameters, to signal any abnormal
behaviour to the operator, and to allow manual or automatic actions to
be taken.
It must also provide bi-directional
communication between DCS and run control to synchronise the state of the detector with the
operation of the data acquisition system.
The DCS also handles interactions between ATLAS
sub-detectors and other systems that are controlled externally,
such as the LHC accelerator, CERN technical services, the ATLAS
magnets and the DSS.

Apart from consolidation and modernisation efforts for the DCS hardware and software components, several conceptual upgrades have been implemented since . First, the standard middleware employed by all systems was migrated from OPC Data Access to the OPC UA standard <cit.>. Middleware applications interface front-end components with the DCS back-end software applications based on the WinCC OA SCADA package <cit.> (formerly known as PVSS). Second, for the new detector systems introduced for , DCS functions of most of the new or upgraded front-end electronics components are covered by the GBT-SCA ASIC<cit.> which communicates with the back-end electronics using the same optical links as the readout or trigger data processed by the FELIX system. These monitoring and control data are relayed to and from the DCS back-end by the GBT-SCA software suite, a dedicated set of middleware applications. Finally, a significant part of the new back-end electronics of the upgrade systems is based on crates and blades following the ATCA standard for which a dedicated integration solution was conceived. The ATCA equipment significantly extends the DCS functionality compared to the previous implementation based on VME crates. These three aspects are described in more detail in the following three sub-sections.

§.§.§ Standard Middleware *OPC UA

The middleware standard for DCS systems is OPC UA, an industry
standard for machine-to-machine communication in the controls domain, allowing independence from the operating system and development in various programming environments. Features like robust data modelling,
custom hardware embedding and secure communications are among the advantages of this standard.

In order to reduce development and maintenance efforts, a framework
for OPC UA  server creation is available – the
<cit.>. Development starts with
creation of a design file, in XML format, describing an
object-oriented information model of the target system or
device. Using this model, the framework generates an executable OPC UA
server application, which exposes the per-design OPC UA  address
space, without the developer writing a single line of
code. Furthermore, the framework generates skeleton code into which
the developer adds the required target device/system integration
logic.  This approach allows both developers unfamiliar with the OPC UA  standard, and advanced OPC UA  developers, to create servers
for the systems they are experts in while greatly reducing design and
development effort as compared to developments based purely on COTS
OPC UA  toolkits. Higher level software may further benefit from
the explicit device model by using the XML design description as the
basis for generating client connectivity configuration and server data
representation. Moreover, having the XML design description at hand
facilitates automatic generation of validation tools.

OPC UA  server architecture within the  framework.

Figure <ref> gives an overview of the different layers
of  put into context. Controllable devices or
systems are accessed using their specific access layer – often
provided together with the specific device.  The device logic layer
functions as an interface with the high level layers of
, which comes in several modules covering
different functional aspects.  The address space module sits in the
OPC UA  end of the server, exposing data towards OPC UA
clients, and is implemented using an OPC UA  back-end layer with
exchangeable back-end implementations. A configuration module
facilitates address space and device instantiation and the definition
of their relations.  XML is used as the configuration format backed by
XML schema definitions. A  XML schema to C++ mapping generator is used to build actual instances from configuration files.  An additional subsystem called Calculated Items, operating entirely in the address space, enables creation of new variables which are derived from existing ones using mathematical
functions.  further comes with optional modules
such as component based logging, certificate handling, server
meta-data, embedded Python processing, WinCC OA integration
tools and SQL/NoSQL archiving with historic data access. A
ready-to-use build system based on CMake along with pre-configured
tool-chains for several platforms such as  or
ARM-based Linux and Microsoft Windows are provided. Finally, an
OPC UA  client generation facility called UaoForQuasar is
available for building C++ clients for -based

By the start of , all middleware applications for Java Card OpenPlatform-supported devices such as power supplies and the ELMB <cit.>, as well as their WinCC OA integration components were migrated to the OPC UA  standard. Furthermore, at the time of writing, more than 20 different OPC UA  server implementations for numerous custom device types used in ATLAS have been developed and integrated, including servers running on embedded platforms such as system-on-chip devices.

§.§.§ Controls Software for *GBT-SCA-based Front-End Electronics

The GBT-SCA is a radiation-tolerant ASIC and part of a chip-set of the GBT project, providing simultaneous transfer of readout data, timing and trigger signals as well as slow control and monitoring data, by multiplexing multiple logical electrical data links, onto a single optical link using the rad-tolerant GBTx ASIC on the front-end side.
The GBT-SCA serves as an interface to the control and monitoring signals of front-end electronics on the detectors, using two redundant e-link to connect to a GBTx.

The  GBT-SCA software suite <cit.> provides a high level of abstraction and an interface to all communication channels of a GBT-SCA, profiting from the hardware parallelism between independent channels. To ensure reliability, the software does the necessary bookkeeping for the synchronous communication and transaction tracking.

Moreover, the software achieves high performance and low latency, including features such as grouping requests for lengthy operations requiring transfers of large amounts of data over JTAG, such as FPGA programming. Since thousands of  GBT-SCA are used in the detector systems, scalability is an important design aspect. At the same time, monitoring and control tasks require availability close to 100%, implying the need for a high level of robustness. For the final production system, the GBT-SCA software is interfaced with the optical link receiver system, FELIX, via a dedicated communication link called netIO.
Figure <ref> shows an overview of the  GBT-SCA integration chain, illustrating the interplay of the components of the GBT-SCA software suite.

Global picture of the software suite. The GBT-SCA Software package, in light blue, comprises the GBT-SCA Software API to communicate with the  GBT-SCA via different back-ends, the GBT-SCA Simulator to emulate  GBT-SCA traffic for testing and development, and the Demonstrator tools which are used for standalone operations. The GBT-SCA OPC UA server and its ecosystem, in orange, is the middleware of choice to exchange data with the front-ends. UaoClientForScaOpcUa is a library that clients use to communicate with the  GBT-SCA server. Finally, the fwSca module automates the integration of the server data into  SCADA systems.

*GBT-SCA *OPC UA  Ecosystem

In the GBT-SCA software package core there is a library that is structured in modules that implement the required functionality in various layers. The library was designed to be flexible and easily adaptable to the diverse systems intended to use it by its polymorphic high-level data link control back-end. Moreover, the GBT-SCA software package contains the Demonstrators, which are tools that directly use the library and are used for testing and for low-level diagnostics. Finally, as part of the package, a GBT-SCA Simulator was developed that is able to generate  GBT-SCA traffic, simulating realistic  GBT-SCA behaviour, in order to allow for development and testing without real hardware.

The GBT-SCA OPC UA server was implemented using the  framework, taking advantage of 's built-in features such as calculated variables, threading, different types of variables and methods. The server architecture divides the  GBT-SCA channels into device classes, according to their respective hardware functions. In addition, a Global Statistician module was developed to collect and measure general statistics across the setup and to expose the collected metrics to the clients. Finally, a GBT-SCA Supervisor software module oversees the state of the system and provides supervisory functionality such as automatic recovery from communication loss with GBT-SCA,  GBT-SCA ID validation and other administrative tasks.

GBT-SCA OPC UA stack.

For any sub-system application using the  GBT-SCA functionality – the  GBT-SCA OPC UA  Clients – the GBT-SCA OPC UA server is used as the hub to transmit and synchronise the data, and a choice was made to decentralise all specialised applications (as shown in Figure <ref>). This choice facilitates maintenance, has the advantage of dividing responsibilities among different communities, allows for staging by creating higher-level wrapper applications, and allows for interoperability between diverse clients.
To support the concept, a -generated C++ library, namely UaoClientForOpcUaSca is provided for building ad-hoc OPC UA  clients. This library supplies the interface to the GBT-SCA OPC UA  server and is created based on information sourced from the design of the server. Applications in ATLAS, that use the aforementioned library, are TDAQ OPC UA  clients used for configuration, or peripheral servers which perform sub-detector specific higher-level operations.
The simplified architecture of the OPC UA GBT-SCA server and an example of a GBT-SCA OPC UA client
is depicted in Figure <ref>. The server allows for the usage of any general-purpose OPC UA  clients for diagnostic purposes such as the publicly available UaExpert tool <cit.>. Finally, the most common way that an OPC UA  server is used by the DCS is through  SCADA  WinCC OA-based systems. These systems employ OPC UA  clients which connect to the servers to retrieve the data and visualise the information in a user interface, usually deployed in the control room where the system is monitored by a shifter. In the case of WinCC OA, the OPC UA  connectivity is realised via a software module, fwSca, which is
supplied by the GBT-SCA software suite. This module allows for fast integration as it creates all the necessary configuration in the WinCC OA application based on a priori information of the GBT-SCA OPC UA information schema.

*GBT-SCA Software Performance

The server has been designed to serve setups of various sizes and types. The biggest challenge is the  NSW detector upgrade (see Section <ref>). In this system, 6976 GBT-SCA are employed, distributed over different types of front-end electronics boards. The traffic of the GBT-SCA is handled by 30  FELIX hosts with 18 optical fibre connections each, and a corresponding number of GBT-SCA OPC UA servers.

In an early integration setup, the GBT-SCA OPC UA server was tested against a full-sector slice of the NSW MM sub-system with eight detector layers fully equipped with their front-end electronics. The slice serves 160 GBT-SCA, handled by a single server which was used in various realistic scenarios. The  GBT-SCA are separated in three categories/types of electronics with different functionality and interfaces as described in Table <ref>.

The setup used one FELIX host equipped with an Intel(R) Xeon(R) CPU E5-1650 v4 @3.60. The GBT-SCA OPC UA server runs in the  FELIX host machine along with  FELIX software. In a first constant-throughput scenario, a WinCC OA SCADA application monitored the inputs from a separate host while three OPC UA  clients were used for diagnostics. In the second burst traffic scenario, the server was used by 128 additional configuration clients.

GBT-SCA channel usage in the ATLAS  NSW MM full-sector slice. The setup was used to evaluate the performance of the server.

Board Name				    MMFE8	    ADDC						    L1DDC

2*Functionality	    2*readout		   trigger					   data

aggregator				   aggregator

GBT-SCA Numbers					   128								   16								   16

ADC Inputs				   15								   10								   9

Calculated 				    2*15		    2*10		    2*9

I2C Master 				   2								   6								   2

I2C Slave 					   44+60						   6								   2

SPI Slave					   8								   -									   -

GPIO			   19								   18								   -

Constant-throughput Monitoring Traffic: Even when no configuration activities are performed, the server is used constantly to provide monitoring data from the detector electronics. These data, mostly from inputs, correspond to the minimum possible activity of the server. In the MM full-sector slice setup the global request rate was measured to be around 7800requests/ for 2192 ADC inputs (each input consists of two GBT-SCA requests). That resulted in an actual refresh rate of about 2 per input. The CPU usage of the server reached about 25% on average and the share of available physical memory used was 340MB, a metric that is stable and not dependent on the usage.

On-demand  GBT-SCA Traffic - Front-end Configuration: The most challenging aspect, in terms of open sessions and process complexity, is the configuration of the front-end boards. Emulating the cold start of the  NSW MM detector, a full-sector configuration was attempted in addition to the constant-throughput monitoring traffic as described above. During the process, up to 58 concurrent sessions were established from various OPC UA  clients. The configuration clients were programming the front-end electronics using a combination of interleaving operations between GPIO, I2C, and SPI totalling around 2700 requests for each GBT-SCA. The global request rate reached about 35000requests/. The instantaneous CPU usage peaked at 218%. The total time to initialise all the front-ends was measured to be 10. In ATLAS, each sector is independent, so these times and rates are also applicable to the full detector.

§.§.§ Controls for *ATCA Back-End Electronics

The ATCA standard is employed as a back-end platform by many of the system upgrades for (the NSW Trigger Processor, the LAr LATOME, and the TDAQ L1 gFEX, eFEX, MuCTPI, jFEX, and L1Topo systems), replacing the VME standard as the preferred back-end technology for new electronics systems. To allow the DCS to work with ATCA, an integration solution based on the common OPC UA  toolset was developed, managing ATCA shelves via their shelf manager SNMP interface and providing control and monitoring of shelf and blade functions. The solution covers the “CERN-standard” Pigeon Point Shelf Managers (ShMM 500, ShMM 700R) and, while designed for ATCA, is compatible with the broader device family. An overview of this scheme is illustrated in Figure <ref>.

Overview of the scheme based on OPC UA to manage ATCA shelves via their shelf manager interface. During , some systems still use the IPbus-based pathway shown in grey, but this is deprecated and will be phased out by the end of LS3.

The ATCA OPC UA server, a modularised software application
(see also <ref>), models selected parts of the ATCA standard functionality for compliant devices in an object-oriented design
and interfaces with the shelf manager SNMP agent via the experiment controls network. Code generation techniques are used to implement the selected device functions based on the SNMP Management Information Base provided by the shelf manufacturer. The SNMP back-end within the server implementation is a C++ wrapper of the Net-SNMP open-source library and provides a generic interface to any SNMP device. In addition, the ATCA OPC UA server provides features such as automatic hardware discovery that queries a given set of shelf managers and automatically creates a map of all blades, fan trays, power supplies and other field-replaceable units, and makes their functionality available to OPC UA  clients. The supervision of custom blade functions such as on-board sensors controlled by the blade IPMC is also supported.

Finally, a set of associated tools, allowing for easy client integration with the  WinCC OADCS applications, are part of the ATCA software solution. These tools take care of datapoint creation, alarm handling, archiving, and Finite State Machine integration, and facilitate the creation of operator interfaces with pre-built generic user interface panels, making shelf integration into the ATLAS DCS an easy and efficient task.

The LAr Calorimeter back-end electronics (see also Section <ref>) and the TDAQ Level-1 Calorimeter Trigger electronics (see also Section <ref>) use this method to monitor onboard component parameters such as temperature and voltage.

§ OUTLOOK

Extensive upgrades were performed to ready the ATLAS detector for based on the experience gained from .
While the running parameters foreseen for are not expected to subject the detector to more extreme conditions than it has experienced already, it is expected that with luminosity levelling, ATLAS will perpetually experience conditions close to the extremes of during .
The emphasis of the Phase-I upgrades was therefore on making the detector and its trigger as robust as possible, so that ATLAS can run comfortably during with un-prescaled single-lepton trigger thresholds comparable to those of . The Phase-I upgrades were designed to last for the remaining lifetime of ATLAS, and constitute the first step in preparing ATLAS for the rigours of data-taking at the HL-LHC, when it is expected that the instantaneous luminosity could rise as high as , with an average pileup of .

The remaining upgrades required for ATLAS to run at the HL-LHC are being prepared now, and will be installed during LS3, at the end of .
The largest of these is the complete replacement of the ATLAS ID by the new, all-silicon, ITk <cit.> that is designed for a similar or even improved tracking performance compared to the current ATLAS ID, but in the challenging pileup conditions of the HL-LHC. The ITk pixel system, which builds on the technology used in the IBL, will be composed of five barrel layers plus endcap rings covering up to $\abseta = 4$. It will be surrounded by a large-area strip detector composed of four barrel layers and six endcap discs, extending up to a maximum radius of 1m and and covering up to $\abseta < 2.7$.
There will be substantial upgrades to the electronics of numerous subsystems for Phase-II, with a focus on radiation-hard readout electronics for the LAr <cit.> and Tile <cit.> Calorimeters that must operate at the trigger rates and latencies required for Phase-II luminosities.
The already excellent timing resolution of the calorimeters will be augmented with the addition of a dedicated HGTD <cit.>.
The Muon Spectrometer <cit.> will see further improvements to trigger coverage and redundancy, with the replacement of on- and off-detector readout and trigger electronics, the replacement of the TGC EIL4 doublet modules by more efficient triplets, and with the addition of RPC to the entire inner barrel layer, building on the BIS78 pilot project described in Section <ref>.
Information from the MDT detectors will be incorporated in the first level of the Muon trigger, increasing its granularity and improving the sharpness of the trigger transverse momentum thresholds.
The replacement of the front-end electronics of the existing ATLAS detector systems during the Phase-I and Phase-II upgrades allows for a substantially higher hardware-level trigger rate and a longer latency.
The TDAQ system for ATLAS in the HL-LHC <cit.> will consist of a single-level hardware trigger that analyses calorimeter and muon detector information at 40 within 10 latency. After the hardware-based trigger decision, the resulting full detector and trigger data will be read out at a rate of 1. An upgraded high-level trigger system will be implemented using commodity hardware, refining the trigger objects in order to achieve a maximum output rate of 10.

The Phase-I upgrade project is an essential part of a broad upgrade program through the lifetime of the Large Hadron Collider and is fully compatible with the future Phase-II upgrade program of the ATLAS experiment. The collection of $\rts=$13.6 collisions in July 2022 is the realisation of the Run 3 configuration of the ATLAS detector (see Figure <ref>) and has marked the start of the vibrant physics programme planned for Run 3 and beyond.

Event display (Run 427514, Event 68319093) of a collision event recorded in ATLAS on 7 July 2022, when stable beams of 6.8 protons were delivered to ATLAS by the LHC. The red line shows a muon candidate with a transverse momentum of 15 reconstructed using information from the inner tracking detectors and the ATLAS MS endcap. The muon candidate was among the first reconstructed using hits in the MM chambers of the NSW on side C. The NSW is outlined in white and the MM hits are shown as orange lines. Additional muon chambers associated with the track are shown as green (MDT endcap) and purple (TGC endcap) boxes.

§ ACKNOWLEDGEMENTS

We thank CERN for the very successful operation of the LHC, as well as the
support staff from our institutions without whom ATLAS could not be
operated efficiently.

We acknowledge the support of
ANPCyT, Argentina;
YerPhI, Armenia;
ARC, Australia;
BMWFW and FWF, Austria;
ANAS, Azerbaijan;
CNPq and FAPESP, Brazil;
NSERC, NRC and CFI, Canada;
ANID, Chile;
CAS, MOST and NSFC, China;
Minciencias, Colombia;
MEYS CR, Czech Republic;
DNRF and DNSRC, Denmark;
IN2P3-CNRS and CEA-DRF/IRFU, France;
SRNSFG, Georgia;
BMBF, HGF and MPG, Germany;
GSRI, Greece;
RGC and Hong Kong SAR, China;
ISF and Benoziyo Center, Israel;
INFN, Italy;
MEXT and JSPS, Japan;
CNRST, Morocco;
NWO, Netherlands;
RCN, Norway;
MEiN, Poland;
FCT, Portugal;
MNE/IFA, Romania;
MESTD, Serbia;
MSSR, Slovakia;
ARRS and MIZŠ, Slovenia;
DSI/NRF, South Africa;
MICINN, Spain;
SRC and Wallenberg Foundation, Sweden;
SERI, SNSF and Cantons of Bern and Geneva, Switzerland;
MOST, Taiwan;
TENMAK, Türkiye;
STFC, United Kingdom;
DOE and NSF, United States of America.
In addition, individual groups and members have received support from
BCKDF, CANARIE, Compute Canada and CRC, Canada;
PRIMUS 21/SCI/017 and UNCE SCI/013, Czech Republic;
COST, ERC, ERDF, Horizon 2020 and Marie Skłodowska-Curie Actions, European Union;
Investissements d'Avenir Labex, Investissements d'Avenir Idex and ANR, France;
DFG and AvH Foundation, Germany;
Herakleitos, Thales and Aristeia programmes co-financed by EU-ESF and the Greek NSRF, Greece;
BSF-NSF and MINERVA, Israel;
Norwegian Financial Mechanism 2014-2021, Norway;
NCN and NAWA, Poland;
La Caixa Banking Foundation, CERCA Programme Generalitat de Catalunya and PROMETEO and GenT Programmes Generalitat Valenciana, Spain;
Göran Gustafssons Stiftelse, Sweden;
The Royal Society and Leverhulme Trust, United Kingdom.

The crucial computing support from all WLCG partners is acknowledged gratefully, in particular from CERN, the ATLAS Tier-1 facilities at TRIUMF (Canada), NDGF (Denmark, Norway, Sweden), CC-IN2P3 (France), KIT/GridKA (Germany), INFN-CNAF (Italy), NL-T1 (Netherlands), PIC (Spain), ASGC (Taiwan), RAL (UK) and BNL (USA), the Tier-2 facilities worldwide and large non-WLCG resource providers. Major contributors of computing resources are listed in Ref. <cit.>.

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

family=ATLAS Collaboration,

journaltitlePhys. Lett. B
titleObservation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
family=CMS Collaboration,

journaltitlePhys. Lett. B
titleObservation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
family=ATLAS Collaboration,

titleThe ATLAS Experiment at the CERN Large Hadron Collider

noteTo be published in the same volume as this paper.
titleLHC Upgrades in preparation of Run 3
family=ATLAS Collaboration,

titleATLAS data quality operations and performance for 2015–2018 data-taking

noteCERN General Photo
titleBatch Compression Merging and Splitting (BCMS).

titleSlip Stacking: A New Method of Momentum Stacking
family=ATLAS Collaboration,

titleATLAS Insertable B-Layer: Technical Design Report
typeATLAS-TDR-19; CERN-LHCC-2010-013

titleProduction and integration of the ATLAS Insertable B-Layer
family=ATLAS Collaboration,

journaltitleEur. Phys. J. C
titlePerformance of the ATLAS trigger system in 2015
family=ATLAS Collaboration,

titleThe ATLAS Collaboration Software and Firmware
family=ATLAS Collaboration,

titleATLAS Liquid Argon Calorimeter: Technical Design Report
typeATLAS-TDR-2; CERN-LHCC-96-041
family=ATLAS Collaboration,

titleATLAS Tile Calorimeter: Technical Design Report
typeATLAS-TDR-3; CERN-LHCC-96-042
family=ATLAS Collaboration,

journaltitleEur. Phys. J. C
titleReadiness of the ATLAS liquid argon calorimeter for LHC collisions
family=ATLAS Collaboration,

titleATLAS New Small Wheel: Technical Design Report
typeATLAS-TDR-020; CERN-LHCC-2013-006

journaltitleIEEE Transactions on Nuclear Science
titleCorrelation of Particle-Induced Displacement Damage in Silicon

journaltitleIEEE Trans. Nucl. Sci.
titleDisplacement Damage in Silicon Detectors for High Energy Physics

journaltitleNucl. Instrum. Meth. A
titlePion induced displacement damage in silicon devices

journaltitleNucl. Instrum. Meth. A
titleSimulation of non-ionising energy loss and defect formation in silicon

journaltitleNuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment
titleRadiation damage in scintillating crystals

journaltitleNucl. Instrum. Meth. A
titleComputational method to estimate Single Event Upset rates in an accelerator environment

booktitleProceedings of Topical Workshop on Electronics for Particle Physics, Naxos, Greece
titleThe ATLAS radiation dose measurement system and its extension to SLHC experiments
family=ATLAS Collaboration,

titleModelling radiation damage to pixel sensors in the ATLAS detector

family=Pedro on behalf of the ATLAS Collaboration,

journaltitleJ. Phys.: Conf. Ser.
titleOptics robustness of the ATLAS Tile Calorimeter

titleFLUKA: A multi-particle transport code

journaltitleNucl. Data Sheets
titleThe FLUKA Code: Developments and Challenges for High Energy and Medical Applications

titleRadiation calculations for the ATLAS detector and experimental hall

journaltitleNucl. Instrum. Meth. A
titleThe GEANT-CALOR interface and benchmark calculations of ZEUS test calorimeters
family=GEANT4 Collaboration,

journaltitleNucl. Instrum. Meth. A
titleGeant4 – a simulation toolkit

journaltitleComput. Phys. Commun.
titleAn introduction to PYTHIA 8.2

Universität Hamburg

titleRadiation damage in silicon particle detectors: Microscopic defects and macroscopic properties
typePh.D. Thesis
thesis (INSPIRE) / semiconductor detector: microstrip (INSPIRE) / semiconductor detector: pixel (INSPIRE) / radiation: damage (INSPIRE) / dosimetry (INSPIRE) / bibliography (INSPIRE)
family=ATLAS Collaboration,

titleMeasurements of sensor radiation damage in the ATLAS inner detector using leakage currents
family=ATLAS Collaboration,

titleATLAS pixel detector electronics and sensors
family=ATLAS Collaboration,

titleOperation and performance of the ATLAS semiconductor tracker

family=The ATLAS TRT collaboration,

IOP Publishing

titleThe ATLAS Transition Radiation Tracker (TRT) proportional drift tube: design and performance
family=ATLAS Collaboration,

typeATLAS-TDR-19-ADD-1; CERN-LHCC-2012-009

noteProceedings of the Third International Workshop on Semiconductor Pixel Detectors for Particles and X-rays
title3D — A proposed new architecture for solid-state radiation detectors

note7th International ""Hiroshima"" Symposium on the Development and Application of Semiconductor Tracking Detectors
number1, Supplement
titleThe FE-I4 pixel readout integrated circuit
Pixel detector,ATLAS upgrades,High luminosity,130nm
family=Ian Dawson (ed.),

journaltitleCERN Yellow Reports: Monographs
titleRadiation effects in the LHC experiments: Impact on detector performance and operation
family=ATLAS Collaboration,

journaltitleEur. Phys. J. C
titleAlignment of the ATLAS Inner Detector in Run-2

note1st International Conference on Technology and Instrumentation in Particle Physics
titleThe ATLAS ReadOut System Performance with first data and perspective for the future

titleCERN S-LINK homepage
family=SFF Committee,

titleQuad Small Form-factor Pluggable (QSFP) Transceiver Specification
family=B. Hallgren, H. Boterenbrood, H.J. Burckhart and H. Kvedalen,

booktitleProceedings of the 7th Workshop on Electronics for LHC Experiments: Stockholm, Sweden 10 - 14 Sep 2001.
titleThe embedded local monitor board (ELMB) in the LHC front-end I/O control system
family=ETM professional control GmbH,

titleSIMATIC WinCC Open Architecture SCADA System

titleThe Detector Safety System of the ATLAS experiment
family=The ATLAS TRT collaboration,

IOP Publishing

titleThe ATLAS TRT Barrel Detector
family=The ATLAS TRT collaboration,

IOP Publishing

titleThe ATLAS TRT end-cap detectors
family=The ATLAS TRT collaboration,

IOP Publishing

titleThe ATLAS TRT electronics

journaltitleProceedings of the IRE
titleA Method for the Construction of Minimum-Redundancy Codes

titleHOLA High-speed Optical Link for Atlas
family=ATLAS Collaboration,

titlePerformance of the ATLAS Transition Radiation Tracker in Run 1 of the LHC: tracker properties
family=ATLAS Collaboration,

titleStudy of the material of the ATLAS inner detector for Run 2 of the LHC
family=ATLAS Collaboration,

titleOperation and performance of the ATLAS semiconductor tracker in LHC Run 2
family=ATLAS Collaboration,

titleATLAS Liquid Argon Calorimeter Phase-I Upgrade: Technical Design Report
typeATLAS-TDR-022; CERN-LHCC-2013-017

titleThe Phase-I trigger readout electronics upgrade of the ATLAS Liquid Argon calorimeters

titleSignal processing considerations for liquid ionization calorimeters in a high rate environment
family=ATLAS Collaboration,

titleMonitoring and data quality assessment of the ATLAS liquid argon calorimeter

IOP Publishing

journaltitleJournal of Physics: Conference Series
titleRadiation hardness of plastic scintillators for the Tile Calorimeter of the ATLAS detector

IOP Publishing

journaltitleJournal of Physics: Conference Series
titleA comparative study of the radiation hardness of plastic scintillators for the upgrade of the Tile Calorimeter of the ATLAS detector
family=ATLAS Collaboration,

titleElectron and photon energy calibration with the ATLAS detector using data collected in 2015 at \(\sqrt{s} = 13~\text{TeV}\)
family=ATLAS Collaboration,

titleATLAS Muon Spectrometer Phase-II Upgrade: Technical Design Report
typeATLAS-TDR-026; CERN-LHCC-2017-017
family=ATLAS Collaboration,

howpublishedCERN-LHCC-2011-012. LHCC-I-020
titleLetter of Intent for the Phase-I Upgrade of the ATLAS Experiment
family=ATLAS Collaboration,

titleATLAS TDAQ System Phase-I Upgrade: Technical Design Report
typeATLAS-TDR-023; CERN-LHCC-2013-018

booktitle2007 IEEE Nuclear Science Symposium Conference Record
titleDevelopment of precision drift tube detectors for very high background rates at the super-LHC
family=ATLAS Collaboration,

titleATLAS TDAQ Phase-II Upgrade: Technical Design Report
typeATLAS-TDR-029; CERN-LHCC-2017-020

journaltitleNuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment
titleMICROMEGAS: a high-granularity position-sensitive gaseous detector for high particle-flux environments

abstractThe ATLAS collaboration at the Large Hadron Collider at CERN has endorsed the resistive-strip micromegas technology for the high luminosity upgrade of the first muon station in the high-rapidity region, the so called “New Small Wheel” project. It requires detectors with a spatial resolution of ∼100μm, fully efficient up to a particle rate of ∼20kHz/cm2. In order to demonstrate that the resistive-strip micromegas technology fulfils these requirements, small resistive bulk micromegas have been studied with radioactive sources and with high energy beams. The micromegas chambers were operated with an Ar+7%CO2 gas mixture and read out using the APV25 chip. Results on the detection efficiency and the position resolution are presented for track impact angles from 0° to 40°. A position reconstruction method has been developed for inclined tracks, called the “micro-TPC method”. A description of the method along with performance studies is presented. In addition, the impact of the unavoidable presence of pillars and the relative alignment of readout and resistive strips on the micromegas performance has been quantified. In view of the fact that the micromegas detectors will also contribute to the trigger in ATLAS their time response has been studied.
titlePerformance studies of resistive-strip bulk micromegas detectors in view of the ATLAS New Small Wheel upgrade
Micromegas,Resistive-strip,Performance,Muon spectrometer,ATLAS experiment

journaltitleZ. Phys.
titleDie Entwicklung der Elektronenlawine in den Funkenkanal

titleFurther evaluation of a THGEM UV-photon detector for RICH - comparison with MWPC

IOP Publishing

journaltitleJ. Phys.:Conf. Ser.
titleATLAS NSW Micromegas readout boards: Industrialisation and Quality Control and Quality Assurance

titleThe large inner Micromegas modules for the Atlas Muon Spectrometer upgrade: Construction, quality control and characterization
MPGD,Micromegas,Resistive anode,High rate capability,HL-LHC,ATLAS,NSW,Cosmic bench,Validation tests
family=De Cecco,
family=De Fazio,
family=Del Gaudio,
family=Di Donato,
family=Di Nardo,
family=Kourkoumeli Charalampidi,
family=Safai Tehrani,

titleConstruction and test of the SM1 type Micromegas chambers for the upgrade of the ATLAS forward muon spectrometer
Muon detector,Micromegas

titleMicromegas chambers for the ATLAS New Small Wheel upgrade

IOP Publishing

abstractTo exploit the full discovery potential of the Large Hadron Collider an upgrade towards high luminosity (HL-LHC) is scheduled for 2024–25. Simultaneously to the accelerator, the experiments have to adapt to the expected higher particle rates and detector occupancy. Within the next long shutdown in 2019-20 the innermost end-cap regions of the ATLAS Muon spectrometer will be replaced by the New Small Wheels (NSW) including Micromegas detector modules of several m2 size.The Micromegas readout anode boards, representing the core components of the detector, are manufactured in industry, making the NSW Micromegas the first Micro Pattern Gaseous Detector (MPGD) for a major LHC experiment with a crucial industrial contribution. Production of the up to 2.2 m long boards is a serious challenge for industrialization technology and quality control methods.
titleProduction and quality control of Micromegas anode PCBs for the ATLAS NSW upgrade

IOP Publishing

abstractThe Rasnik alignment system was developed initially in 1983 for the monitoring of the alignment of the muon chambers of the L3 Muon Spectrometer at CERN. Since then, the development has continued as new opto-electronic components become available. Rasnik systems are 3-point optical displacement monitors and their precision ranges from below nanometers to several micrometers, depending on the design and requirements of the systems. A result, expressed in the range/precision ratio of 2 × 106, is presented. According to the calculations of the Cram   apos;er-Rao limit, and by means of MonteCarlo simulations, a typical Rasnik image should have enough information to reach deep sub-nanometer precision. This paper is an overview of the technological developments and achievements since Rasnik was applied in high energy physics experiments.
titleThe Rasnik 3-point optical alignment system

La Rochelle, France

booktitleJ. Phys.:Conf. Ser.
seriesMicroPattern Gaseous Detectors Conference 2019
titleVMM ASIC
family=ATLAS Collaboration,

titleStereo Information in Micromegas Detectors

National Technical University, Athens

note(pp. 180–185)
titlePerformance characterization of the Micromegas detector for the New Small Wheel upgrade and Development and improvement of the Muon Spectrometer Detector Control System in the ATLAS experiment

titleResistive Micromegas for the Muon Spectrometer Upgrade of the ATLAS Experiment
family=De Fazio,
family=Del Gaudio,
family=Di Donato,
family=Di Nardo,
family=La Rotonda,

journaltitleNuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment
titleConstruction techniques and performances of a full-size prototype Micromegas chamber for the ATLAS muon spectrometer upgrade
Muon detector,Micromegas

abstractThe idea of a thin multiwire chamber with thick wires as a possible sampling detector for calorimeters was checked out in practice. Two small test detectors with gas sample thickness of 2 and 3 mm were constructed and tested in the laboratory. Avoiding argon, several pure hydrocarbons or binary mixtures with CO2 were found to quench photons efficiently enough to make it possible to have charge multiplication factors of the order of 107. Efficiency of operation in this probably limited streamer mode for minimum ionizing electrons and with a single avalanche (streamer) response was found to be higher than 98%, with pulse-height resolutions down to 60% fwhm. The effect of gap-length variation is quite small; a gap change of upto 25% results in less than a 20% change in the mean charge released in a streamer.
titleA thin multiwire chamber operating in the high multiplication mode

abstractThe instantaneous luminosity of the Large Hadron Collider at CERN will be increased up to a factor of five with respect to the present design value by undergoing an extensive upgrade program over the coming decade. The most important upgrade project for the ATLAS Muon System is the replacement of the present first station in the forward regions with the so-called New Small Wheels (NSWs). The NSWs will be installed during the LHC long shutdown in 2019/2020. Small-Strip Thin Gap Chamber (sTGC) detectors are designed to provide fast trigger and high precision muon tracking under the high luminosity LHC conditions. To validate the design, a full-size prototype sTGC detector of approximately 1.2×1.0m2 consisting of four gaps has been constructed. Each gap provides pad, strip and wire readouts. The sTGC intrinsic spatial resolution has been measured in a 32GeV pion beam test at Fermilab. At perpendicular incidence angle, single gap position resolutions of about 50μm have been obtained, uniform along the sTGC strip and perpendicular wire directions, well within design requirements. Pad readout measurements have been performed in a 130GeV muon beam test at CERN. The transition region between readout pads has been found to be 4mm, and the pads have been found to be fully efficient.
titlePerformance of a full-size small-strip thin gap chamber prototype for the ATLAS new small wheel muon upgrade
LHC,ATLAS upgrade,Muon spectrometer,Gaseous detectors,Tracking,Trigger

IOP Publishing

abstractThe muon spectrometer of the ATLAS detector at the Large Hadron Collider (LHC) at CERN consists of 1182 muon chambers for precision track measurements, arranged in three concentric cylinders in the barrel region, and in four wheels in each of the two endcaps. The endcap wheels are located between 7 m and 22 m from the interaction point, and have diameters between 13 m and 24 m. Muon chambers are equipped with a complex optical alignment system to monitor their positions and deformations during ATLAS data-taking. We describe the layout of the endcap part of the alignment system and the design and calibration of the optical sensors, as well as the various software components. About 1% of the system has been subjected to performance tests in the H8 beam line at CERN, and results of these tests are discussed. The installation and commissioning of the full system in the ATLAS cavern has been completed, and the analysis of the first data indicates that it performs already now at a level close to the goal of a 40 μm alignment accuracy, ultimately required for reconstructing high-momentum final-state muons with the desired momentum resolution of 10% at 1 TeV.
titleThe Optical Alignment System of the ATLAS Muon Spectrometer Endcaps

IOP Publishing

titleThe New Small Wheel electronics

titleGBTX MANUAL V0.15
family=S. Bonacini, A. Caratelli et al.,

titleGBT-SCA User Manual V. 2019-002

IOP Publishing

titleThe GBT-SCA, a radiation tolerant ASIC for detector control and monitoring applications in HEP experiments

journaltitleIEEE Transactions on Nuclear Science
noteFor more details about the FEAST, see <https://espace.cern.ch/project-DCDC-new/>
titleOptimization of DC-DC Converters for Improved Electromagnetic Compatibility With High Energy Physics Front-End Electronics

titleThe GBT: A proposed architecure for multi-Gb/s data transmission in high energy physics

journaltitleIEEE Transactions on Nuclear Science
titleThe VMM3a ASIC

titleThe Read Out Controller for the ATLAS New Small Wheel

abstractA small-strip thin gap chamber (sTGC) will be used for both triggering and precision tracking purposes in the upgrade of the ATLAS forward muon spectrometer. Both sTGC pad and strip detectors are read out by a trigger data serializer (TDS) application-specific integrated circuit (ASIC) in the trigger path. This ASIC has two operation modes to prepare trigger data from pad and strip detectors, respectively. The pad mode (pad TDS) collects the firing status for up to 104 pads from one detector layer and transmits the data at 4.8 Gbps to the pad trigger extractor every 25 ns. The pad trigger extractor collects pad-TDS data from eight detector layers and defines a region of interest (ROI) along the path of a muon candidate. Data defining the ROIs is returned to the strip TDS. In the strip mode (strip TDS), the deposited charges from up to 128 strips are buffered and time-stamped, and a trigger matching procedure is performed to read out strips underneath the ROI. The strip-TDS output is also transmitted at 4.8 Gbps to the following field-programmable gate array (FPGA) processing circuits. Details of ASIC design and test results are presented in this paper.
journaltitleIEEE Transactions on Nuclear Science
titleDesign of a Trigger Data Serializer ASIC for the Upgrade of the ATLAS Forward Muon Spectrometer
application specific integrated circuits;field programmable gate arrays;muon detection;nuclear electronics;readout electronics;transition radiation detectors;trigger circuits;region of interest;field-programmable gate array;field-programmable gate array;FPGA processing circuits;detector layers;pad-TDS data;pad trigger extractor;detector layer;pad TDS;pad mode;operation modes;trigger path;trigger data serializer application-specific integrated circuit;strip detectors;triggering precision tracking purposes;small-strip thin gap chamber;ATLAS forward muon spectrometer;trigger data serializer ASIC;ASIC design;strip-TDS output;trigger matching procedure;strip mode;muon candidate;ROI;Digital integrated circuits;Detectors;Large Hadron Collider;Application specific integrated circuits;Application-specific integrated circuits (ASICs);ATLAS;digital integrated circuits

booktitle2019 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC)
titleThe Development and Production of the ADDC for the Micromegas Detector of the ATLAS New Small Wheel Upgrade

titleLEVEL-1 DATA DRIVER CARD - A high bandwidth radiation tolerant aggregator board for detectors

journaltitleIEEE Transactions on Nuclear Science
titleFPGA Implementation of a Fixed Latency Scheme in a Signal Packet Router for the Upgrade of ATLAS Forward Muon Trigger Electronics
family=ATLAS Collaboration,

titleThe Serial and LVDS repeaters for the ATLAS New Small Wheel sTGC trigger

howpublishedATL-ELEC-PUB-2007-002, ATL-COM-ELEC-2007-003
titleGrounding of the ATLAS experiment

journaltitleNucl. Instrum. Meth. A
titleConstruction and test of a full prototype drift-tube chamber for the upgrade of the ATLAS muon spectrometer at high LHC luminosities

titleNew high-precision drift-tube detectors for the ATLAS muon spectrometer

journaltitleNucl. Instrum. Meth. A
titlePerformance of drift-tube detectors at high counting rates for high-luminosity LHC upgrades

titlePerformance studies of RPC detectors operated with new environmentally-friendly gas mixtures in presence of LHC-like radiation background

journaltitleIEEE Trans. Nucl. Sci.
titleNew results on ATLAS RPC's aging at CERN's GIF

titlePerformance study of HL-LHC ATLAS RPC prototype

journaltitleNucl. Instr. and Meth. A
titleUpgrades of the ATLAS Muon Spectrometer With sMDT Chambers

journaltitleNucl. Instr. and Meth. A
titleUpgrades of the ATLAS Muon Spectrometer with New Small-Diameter Drift Tube Chambers

titleDevelopment of a new Front End electronics in Silicon and Silicon-Germanium technology for the Resistive Plate Chamber detector for high rate experiments

titleHPTDC – High Performance Time to Digital Converter

titleATLAS Muon Drift Tube Electronics

seriesIEEE NSS Real Time Conference
titleFELIX: The new approach for interfacing to front-end electronics for the ATLAS experiment

titleA radiation tolerant gigabit serializer for LHC data transmission
titleXilinx Kintex-7 FPGA product information

titleThe ATLAS beam conditions monitor

titleCalibration of the effective beam height in the ISR
family=ATLAS Collaboration,

journaltitleEur. Phys. J. C
titleLuminosity determination in \(pp\) collisions at \(\sqrt{s} = 8\,\text{TeV}\) using the ATLAS detector at the LHC
family=ATLAS Collaboration,

titleLuminosity determination in \(pp\) collisions at \(\sqrt{s} = 13\,\text{TeV}\) using the ATLAS detector at the LHC
family=ATLAS Collaboration,

titleLuminosity determination in \(pp\) collisions at \(\sqrt{s} = 13\,\text{TeV}\) using the ATLAS detector at the LHC

titleChoice and characterization of photomultipliers for the new ATLAS LUCID detector

titleThe new LUCID-2 detector for luminosity measurement and monitoring in ATLAS
titleCaen SY 1527 Technical Information Manual
family=ATLAS Collaboration,

titleThe LUCID 3 detector for the ATLAS Phase-II Upgrade

journaltitleIEEE Trans. Nucl. Sci.
titlePrecision Luminosity of LHC Proton–Proton Collisions at 13 TeV Using Hit Counting With TPX Pixel Devices
family=ATLAS Collaboration,

titleLuminosity monitoring using \(Z\to{\ell^+\ell^-}\) events at \(\sqrt{s} = 13\,\text{TeV}\) with the ATLAS detector

titleThe ALFA Roman Pot Detectors of ATLAS

Islamic University of Gaza

Gaza City

titleRadiation resistance of the ATLAS-ALFA electronics and trigger efficiency analysis at $\sqrt{s}=13$ TeV and $\beta$*=2.5 km
family=ATLAS Collaboration,

titleATLAS Forward Proton Phase-I Upgrade: Technical Design Report
typeATLAS-TDR-024; CERN-LHCC-2015-009

abstractThe ATLAS Forward Physics (AFP) project plans to install 3D silicon pixel detectors about 210 m away from the interaction point and very close to the beamline (2–3 mm). This implies the need of slim edges of about 100–200 μm width for the sensor side facing the beam to minimise the dead area. Another challenge is an expected non-uniform irradiation of the pixel sensors. It is studied if these requirements can be met using slightly-modified FE-I4 3D pixel sensors from the ATLAS Insertable B-Layer production. AFP-compatible slim edges are obtained with a simple diamond-saw cut. Electrical characterisations and beam tests are carried out and no detrimental impact on the leakage current and hit efficiency is observed. For devices without a 3D guard ring a remaining insensitive edge of less than 15 μm width is found. Moreover, 3D detectors are non-uniformly irradiated up to fluences of several 1015 neq/cm2 with either a focussed 23 GeV proton beam or a 23 MeV proton beam through holes in Al masks. The efficiency in the irradiated region is found to be similar to the one in the non-irradiated region and exceeds 97 percent in case of favourable chip-parameter settings. Only in a narrow transition area at the edge of the hole in the Al mask, a significantly lower efficiency is seen. A follow-up study of this effect using arrays of small pad diodes for position-resolved dosimetry via the leakage current is carried out.
title3D silicon pixel detectors for the ATLAS Forward Physics experiment

titleBeam tests of an integrated prototype of the ATLAS Forward Proton detector

titleEngineering Change Request - Installation of the ATLAS/AFP stations, Phase-2

titleThe RD53A Integrated Circuit

IOP Publishing

titleTimepix4, a large area pixel detector readout chip which can be tiled on 4 sides providing sub-200 ps timestamp binning

CTU, Prague

numberinternal CTU note
titleCommissioning of cooling system AIRCOOLER SPLIT before installation for TOTEM detectors

Optica Publishing Group

journaltitleOpt. Express
titleUpgraded Cherenkov time-of-flight detector for the AFP project
Detectors; Fused silica; Optical constants; Photodetectors; Pulsed operation; Radiation detectors

journaltitleProceedings of Science
titlePerformance study of the ATLAS Forward Proton Time-of-Flight Detector System
notePHOTONIS Defense, Inc., 1000 New Holland Ave., Lancaster, PA 17601-5688, USA
titleMini PLANACON XPM85112-S-R2D2: a long-life, high-rate, 4x4-channel multi-anode multi-channel plate photomultiplier

University of West Bohemia, Pilsen

Belgrade, Serbia

booktitle27th Telecommunications Forum TELFOR 2019
titleMultichannel Majority Coincidence Circuit for ToF AFP Detector

titleDesign of the SLAC RCE Platform: A general purpose ATCA based data acquisition system

journaltitleJ. Phys. Conf. Ser.
titleDetector Control System for the AFP detector in ATLAS experiment at CERN
noteHamamatsu, 325-6, Sunayama-cho, Naka-ku, Hamamatsu City,Shizuoka Pref., 430-8587, Japan
titleH6559 Photomultiplier tube assembly

titleThe BRAN luminosity detectors for the LHC
Luminosity Detector,LHC,Fast Ionization Chamber,Gas Detector,Ionization Chamber
family=S. Ask et al.,

titleThe ATLAS central level-1 trigger logic and TTC system
family=ATLAS Collaboration,

titlePerformance of the upgraded PreProcessor of the ATLAS Level-1 Calorimeter Trigger
family=ATLAS Collaboration,

journaltitleEur. Phys. J. C
titlePerformance of the ATLAS Level-1 topological trigger in Run 2
family=ATLAS Collaboration,

titleATLAS Tile Calorimeter Phase-II Upgrade: Technical Design Report
typeATLAS-TDR-028; CERN-LHCC-2017-019

noteAccessed: 2021-11-04
titleAurora 8B/10B Protocol Specification
family=C. Ghabrous Larrea, K. Harder, D. Newbold, D. Sankey, A. Rose, A. Thea and T. Williams,

titleIPbus: a flexible Ethernet-based control system for xTCA hardware

noteAccessed: 2021-11-04
family=D. Bertolini, T. Chan, and J. Thaler,

titleJet Observables Without Jet Algorithms
family=Intel Corporation, Hewlett-Packard Company, NEC Corporation, Dell Inc.,

noteAccessed: 2021-11-04
titleIntelligent Platform Management Interface Specification, v. 2.0
family=ATLAS Collaboration,

titlePerformance of the ATLAS muon triggers in Run 2

titleAdvanced Telecom Computing Architecture
family=Salvatore Loffredo, ATLAS Muon Collaboration,

journaltitle2019 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC)
titleThe BIS78 Pad Trigger Board for the Phase-I Upgrade of the Level-1 Muon Trigger of the ATLAS Experiment at the LHC

journaltitlePhysics Procedia
noteProceedings of the 2nd International Conference on Technology and Instrumentation in Particle Physics (TIPP 2011)
titleA Gigabit Transceiver for Data Transmission in Future High Energy Physics Experiments
family=A. Armbruster et al.,

journaltitle2017 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC)
titleThe ATLAS Muon to Central Trigger Processor Interface Upgrade for the Run 3 of the LHC

Morgan Kaufmann

booktitleIntel Xeon Phi Coprocessor High Performance Programming
titleChapter 8 - Coprocessor Architecture

booktitle2017 IEEE 25th Annual Symposium on High-Performance Interconnects (HOTI)
titleUtilizing HPC Network Technologies in High Energy Physics Experiments

journaltitleComputer Physics Communications
titleGAUDI — A software architecture and framework for building HEP data processing applications
LHCb,GAUDI,Architecture,Components,Abstract interfaces,Framework
family=Bray, T.,

titleThe JavaScript Object Notation (JSON) Data Interchange Format, STD 90, RFC 8259
family=A. Barriuso Poy et al.,

titleThe detector control system of the ATLAS experiment
titleOPC UA Online Reference

booktitleProceedings, 15th International Conference on Accelerator and Large Experimental Physics Control Systems (ICALEPCS 2015): Melbourne, Australia, October 17-23, 2015
titlequasar - A Generic Framework for Rapid Development of OPC UA Servers

IOP Publishing