The upgrade of the CMS trigger system

The LHC accelerator at CERN in Geneva is being upgraded to increase its energy and luminosity, which requires that the CMS (Compact Muon Solenoid) detector and its trigger system are also upgraded to make full use of higher collision rates. The trigger upgrade will proceed in several stages. At present the hardware-based Level-1 trigger is being upgraded to improve the resolution and thus allow for efficient data taking at higher collision rates while keeping the present Level-1 trigger rate of 100 kHz. The new system will be running in parallel to the existing Level-1 trigger for commissioning in 2015, when LHC starts running again, and will take over the full trigger functionality in 2016. A fundamental change in the trigger is planned for the time when the CMS silicon tracker is replaced in 2022. While at present the tracker does not send data to the Level-1 trigger and is only read out when a positive Level-1 trigger decision is received, the new tracker will be integrated into the Level-1 trigger. For that second upgrade stage, a significant increase of the Level-1 trigger rate by a factor of up to ten is planned.


.1 Physics motivation
The successful discovery of the Higgs boson at the Large Hadron Collider (LHC) has ended the 50-year long hunt for this elusive particle. However, this does not mean that all the secrets of elementary particle physics have been unraveled. The very existence of a Higgs particle at the relatively low mass of 125 GeV/c 2 opens new problems concerning the stabilisation of its mass. A promising model to answer this question is Supersymmetry. Another reason for this theory being so popular at the moment is that a stable lightest supersymmetric particle could provide a good candidate for solving the puzzle of Dark Matter: this particle could be the WIMP, the Weakly Interacting Massive Particle. However, so far no signs of supersymmetric particles have been found at the LHC or anywhere else. The observed existence of neutrino masses is yet another motivation for looking for physics beyond the Standard Model. As most of the parameter space accessible so far has been experimentally excluded, the obvious next step is to go to higher energies but also to higher luminosities in order to look for processes with extremely small cross sections.

The upgrade of the Large Hadron Collider
From 2009 to 2012, LHC was operating at a center-of-mass energy of initially 7 and later 8 TeV. At the moment, the collider is being refurbished and should reach a collision energy of 13 TeV in 2015. As cross sections generally rise with energy, this change by itself will result in an increase in interaction rates of roughly a factor of two.
At the same time, the luminosity will increase from the value of 0.7×10 34 cm −2 s −1 reached in 2012 to more than its original design value of 10 34 cm −2 s −1 in 2015. Later on, LHC is planned to evolve into the "High-Luminosity LHC (HL-LHC)" and reach a luminosity of over 5×10 34 cm −2 s −1 .

Pileup
A point of particular concern for LHC experiments is the increase in pileup to be expected (see figure 1). So far CMS had to deal with about 30 individual proton-proton collisions in each bunch crossing (each "event"). In the future this number may go up to 100 or more individual collisions. According to the original design, the LHC should collide proton bunches every 25 ns. However, this running mode poses problems due to the effect of the so-called "electron cloud" and therefore the LHC has so far usually been operated at a bunch collision spacing of 50 ns. It is still not completely clear if in the future the LHC will allow for operation at 25 ns bunch spacing or not. Obviously, running with the same luminosity at twice the distance between bunches and thus half the number of bunches per orbit results in an increase in pileup of a factor of two. Disentangling tracks from so many individual collisions in one event is hard enough for the High-Level Trigger and during data analysis when full resolution and tracker information are available. For the present Level-1 trigger, which operates at reduced data precision and does not have access to the tracker information, this is a major challenge.

The upgrade of the Compact Muon Solenoid (CMS) experiment and its trigger systems
The higher collision energies and higher interaction rates will result in a significant increase in the radiation load to which the detector is exposed. This will necessitate the replacement of some parts of the detector, in particular in the inner and forward regions, with more radiation-hard components. At the same time, changes in both the detector and the trigger and data acquisition structure will be needed to make full use of the available higher rates of physics processes.

The upgrade schedule
At present, the LHC is being upgraded to a collision energy of 13 TeV and a luminosity of 10 34 cm −2 s −1 or more. For carrying out this work, LHC ended operations of its "Run 1" in early 2013 and entered into its "Long Shutdown 1 (LS1)". CMS has been using this period for initiating its "Phase-1 Upgrade". Certain detector parts (outer muon chambers in the forward region) are being added and parts of the trigger and data acquisition electronics is being replaced. This work will continue in parallel to data taking when LHC starts operations again for its "Run 2" in early 2015. Some of the new electronics will be run in parallel for commissioning purposes through 2015 and CMS will switch over to using these new systems in production mode during LHC's annual technical stop at the end of 2015. The detailed upgrade plans for Phase 1 have been fixed in a Technical Design Report [1]. The CMS silicon pixel detector will be replaced at the end of 2016.
-2 - Further upgrade work on the LHC will take place during Long Shutdown 2 (LS2) scheduled for 2019.
A third Long Shutdown (LS3) of the LHC is planned for 2023-2025. During this period, the LHC should be modified to allow for attaining a luminosity of over 5 × 10 34 cm −2 s −1 . CMS will make use of this period for major reconstruction projects. The most important item will be the complete replacement of the silicon tracker (strips and pixel detectors). This will allow the experiment to withstand the much higher radiation levels expected at the HL-LHC. At the same time, this will offer an opportunity to include the tracker into the first-level trigger of the experiment. This "Phase-2 Upgrade" will allow for much better background suppression already at the hardware level. Obviously, technical solutions for Phase 2, which will happen in almost a decade from now, are not quite as concrete as for Phase 1 yet. However, the complexity of the work to be done requires long-term planning, and a detailed Technical Proposal for the CMS Phase 2 upgrade (with an important chapter on the trigger) is in preparation [2].
3 The existing CMS trigger system CMS relies on a two-level trigger system. The Level-1 (L1) Trigger is implemented in hardware (ASICs and programmable logic chips (FPGAs)) and serves to reduce the data rate from the 40 MHz of the LHC bunch crossing rate down to 100 kHz [3]. In case of a positive Level-1 decision ("L1 Accept (L1A)") all data for the corresponding bunch crossing time is read out from the CMS detector and transferred to the High-Level Trigger (HLT), which consists of a large computer farm. The HLT performs a full reconstruction of events and writes data out to permanent storage at a typical rate of several hundred Hz [4].
-3 - Figure 2. The existing Level-1 trigger system of CMS. Candidate trigger objects from calorimeter systems (left) and muon detectors (right) are combined and forwarded to the Global Trigger, which calculates the trigger decision and sends out "Level-1 Accept" signals to read out the complete CMS detector at a maximum rate of 100 kHz.
The existing Level-1 Trigger can only use part of the detector information (see figure 2). Data from the three muon detector systems (Drift Tubes (DTs), Cathode Strip Chambers (CSCs) and Resistive Plate Chambers (RPCs)) and the calorimeters (the electromagnetic calorimeter (ECAL), the hadronic calorimeter (HCAL) and the forward hadronic calorimeter (HF)) are available at reduced precision for the L1 decision while information from the silicon tracker (pixels and strips) is read out only in case of a positive Level-1 decision and therefore available only at the High-Level Trigger.
Data from the calorimeters are combined in the electronics of the "Regional Calorimeter Trigger (RCT)" [5] and the "Global Calorimeter Trigger (GCT)" [6] and forwarded to the "Global Trigger (GT)" [7]. Muon candidates are calculated by separate Track Finder electronics for DTs and CSCs and by a Pattern Comparator system for the RPCs. These muon candidates are merged by the "Global Muon Trigger (GMT)" [9], which sends the four best muon candidates to the Global Trigger. The GT calculates a maximum of 128 trigger paths ("Algorithms"), which can consist of single trigger objects or of combinations of several muon and/or calorimeter objects. Topological conditions (such as differences in the coordinates ϕ (azimuth) and η (pseudorapidity)) can also be applied at this level. Algorithms can be enabled, disabled or prescaled by a constant factor and are then combined in a "Final OR". The "Trigger Control System (TCS)" [10] checks if all parts of the CMS detector are ready to receive a trigger signal and if there are no other reasons why the trigger decision should be suppressed, and in this case sends out an L1A signal via the "Trigger, Timing and Control (TTC)" system [11] of CMS to all components of the detector. This triggers the readout of all detector data and the transfer to the computer farm of the High-Level Trigger.

JINST 9 C08002
4 Aims and methods of the Level-1 trigger upgrade project 4

.1 The L1 trigger upgrade strategy
With LHC collision rates rising, the task of the trigger upgrade is to keep the rates for data to be recorded at a manageable level while conserving a high efficiency for the individual physics processes. The relatively low mass of the observed Higgs boson makes it mandatory to keep comparably low energy thresholds in the trigger in order to remain sensitive to the Higgs decay products and be able to carry out precision studies of Higgs parameters. Several approaches will have to be combined to reach this goal: • better resolution and precision in the geometrical coordinates (azimuth ϕ and pseudorapidity η) and transverse momentum (p T ) or transverse energy (E T ) of trigger objects; eventually use of the full detector resolution at L1 level; • more complex operations at an early level, such as pileup subtraction in the calorimeter trigger; • combination of complementary trigger data (such as data from redundant muon systems) at an early stage; • more sophisticated trigger algorithms such as complex correlations between different types of trigger data, calculation of invariant masses or transverse masses of pairs of trigger objects; • a larger number of trigger objects (such as muon or electron candidates) and a larger number of Algorithms (trigger paths) to allow for more combinations; • use of information from additional parts of the CMS detector, in particular from the silicon strip tracker.

The L1 trigger upgrade technology
The present CMS trigger electronics consists largely of custom-built VME modules. The large number of different modules is a problem for maintenance and spares management. Large numbers of bulky galvanic cables make servicing and trouble shooting difficult.
In the future, CMS is planning to use standardised electronics wherever possible. Electronics modules built by an institute will be used for several different subsystems. Also, commercial offthe-shelf components (COTS) will be used wherever possible. The VME standard will be replaced by the new form factor µTCA (Micro Telecommunications Computing Architecture, see figure 3). Optical links will provide increased bandwidth and less space will be needed for bulky connectors.
Use of new generations of electronics components, in particular of larger FPGAs (Field Programmable Gate Arrays) will make it possible to achieve higher performance using a smaller number of electronics modules. Higher integration inside a chip will make it possible to reduce the number of error prone interconnections between different boards. Figure 4 shows an example of a generic µTCA board using a single powerful FPGA.
An important constraint is the available trigger latency of about 4 µs, which is largely determined by the limited length of the silicon tracker's readout pipeline. When the tracker will be -5 - Figure 3. A µTCA crate equipped with HCAL trigger ("µHTR") modules. The "MCH" is a commercial crate controller module. The "AMC13" will be used as standard interface for trigger data readout in CMS. replaced for the Phase-2 upgrade around the year 2022, this pipeline length will be substantially increased (to 10-20 µs, according to present plans). All modifications before that date will have to fit into the limited present latency budget. While the time needed for calculations can be speeded up by newer electronics components, serial optical links require some extra time for the serialization and deserialization (SerDes) steps.

.1 The calorimeter trigger
The performance of the calorimeter trigger has been suffering from the growing pileup during the luminosity ramp-up of the LHC, which has made it increasingly harder to identify jets and electron showers and discriminate them against background. Also, the original design of the electronics did not offer the flexibility to react to this problem by modifying algorithms. Therefore the upgrade of the calorimeter trigger is one of the highest priorities for the CMS Phase-1 upgrade [8]. During Phase 1 the size of the "trigger towers" will remain unchanged (a trigger tower collects the signals from 5 × 5 ECAL crystals, which corresponds to a width of 0.087 × 0.087 in ϕ × η) but more sophisticated algorithms will increase the trigger performance. A particular improvement is expected for hadronic decays of τ-leptons, which produce significantly narrower jets than normal quark jets.
Due to the urgency with which these improvements are needed and the fact that not all new electronics will be finalized during LS1, it has been decided to implement the calorimeter trigger upgrade in a staged way. Stage 1 is scheduled to be ready by early 2015 and go into operation with the "legacy" electronics of the other Level-1 trigger sub-systems while Stage 2 will be introduced together with other trigger system upgrades one year later.
For Phase 2, the endcaps of the electromagnetic calorimeter will have to be replaced due to radiation damage. This will also be a good time to change the ECAL front-end trigger electronics and deliver full crystal resolution to the Level-1 trigger.

The muon trigger
CMS uses three muon systems not only for data analysis but also in the trigger. Drift Chambers (DTs) cover the barrel and Cathode Strip Chambers (CSCs) the endcaps while the redundant system of Resistive Plate Chambers (RPCs) covers the whole CMS detector. So far, each of these systems reconstructed muon candidates, which were merged only afterwards, in the Global Muon Trigger (GMT). After the Phase-1 upgrade, three track finders covering the barrel, the endcaps and the overlap region between barrel and endcaps will each use all signals available in their region, which should provide better discrimination of signal and background by the muon system. The data of these three track finders will be merged in the upgraded version of the GMT [12], which will forward them to the GT.

The tracker trigger
The tracker trigger will be implemented for the Phase-2 upgrade when the CMS silicon tracker is replaced during Long Shutdown 3 of the LHC. The new silicon tracker will consist of a barrel and two endcaps equipped with strip sensors on the outside and with combined strip and pixel sensors on the inside. In the center, a silicon pixel detector will be inserted (see figure 5).
For the trigger, CMS will be pursuing a "push" approach where for each bunch crossing the tracker sends track candidate signals of a certain minimum energy (see figure 6).
One solution that is being investigated is using associative memories to compare event data to pre-calculated patterns locally in the tracker modules (see figure 7).

JINST 9 C08002
The track candidates from the silicon strip tracker will be sent to a special trigger stage where they will be used to validate muon and calorimeter objects from the respective systems. This will improve the quality of the trigger data in several ways: • Trigger objects can be validated, for instance by discriminating electrons against hadronic (π 0 → γγ) background in jets.
• The p T assignment will be strongly improved (the erroneous assignment of high p T values to low-p T background tracks by the muon trigger is one of the major sources of high background rates at the moment).
• Tracker data should allow for much refined pileup corrections for calorimeter objects.
• The z-vertex location within the collision region will be determined with a precision of about 1 mm. Thus the tracker will make it possible for the Level-1 trigger to separate signals from different proton collisions in an event. While at the moment "missing E T (MET)" in the Level-1 trigger can only be calculated globally for a collision of two proton bunches, the tracker data will allow the trigger to calculate this parameter for individual proton collisions.
In addition to using the silicon strip tracker, it might be useful to also include the pixel detector in the Level-1 trigger. Due to the enormous number of channels a "push" approach does not appear feasible here. It would, however, be possible to create a level-0 request from other trigger sources to read out the pixel detector in "pull" mode. A drawback of this approach is that it would require an additional increase in latency. At present studies are underway to determine if the benefit from including the pixel detector into the Level-1 trigger would justify the additional expense and complication of the system.

Global Trigger
While the legacy Global Trigger (GT) consists of a number of custom-built VME modules in a 9U VME crate, during the Phase-1 upgrade all the logic functionality will be transferred into one µTCA module equipped with a powerful FPGA. To allow for future extension of the functionality and the parallel calculation of an arbitrary number of Algorithms, several such modules will be able to run in parallel. The optical input data from calorimeters and muons (merged with tracker data after the Phase-2 upgrade) will be split to serve all the parallel GT boards. The outputs will be merged electrically and forwarded to the Trigger Control and Distribution System (TCDS).
Modern powerful FPGAs with built-in DSPs (Digital Signal Processors) will allow the GT to calculate complex correlations between different trigger objects and physical quantities such as invariant masses or transverse masses of pairs of trigger objects. It will also be possible to base a trigger decision not only on the data of the trigger bunch crossing itself but also of the preceding or subsequent bunch crossings. Typical use cases would be the search for relatively long-lived particles as predicted in some scenarios of Supersymmetry or possibly corrections for inaccurate detector timing.

Trigger Control and Distribution System
In the existing Level-1 trigger system, the Trigger Control System is incorporated into the Global Trigger as a special electronics module [10]. It receives status signals from all parts of the CMS detector. Upon a trigger request by the trigger logic, it issues Level-1 Accept signals only if all CMS sub-detectors are ready to be read out and if certain additional conditions are fulfilled (minimum spacing between triggers, no orbit gap etc.) It also produces control signals and calibration triggers.
This functionality is now being integrated together with the TTC system [11] into the "Trigger Control and Distribution System (TCDS)". This will allow CMS to establish additional subdetector partitions (for additional detector systems, or in order to break down groups into smaller units for easier serviceability). It will also make use of bi-directional messaging via optical fibers: the same fibers will be used to send L1As from TCDS to the various detector partitions and to receive their status information back at TCDS. The TCDS is scheduled to be ready for use in data taking at the beginning of LHC Run 2, in early 2015.

High-Level Trigger
As LHC rates increase the High-Level Trigger (HLT) will evolve by increasing the present number of CPU cores (about 13 000 at the end of Run 1). This will also allow to use more complex HLT algorithms. While the average time spent by the HLT on an event was about 100 ms in 2012, it might be allowed to go up to as much as 1 second in the future.
After the Phase-2 upgrade, the HLT will also be helped by the tracker trigger as its track information will be available already at the very first stage of HLT calculations.
The tracker trigger will help to significantly reduce background rates for certain trigger sources such as muons, for which studies predict more than a factor of 10 in rate reduction, electrons, and taus. The expected rate reduction for photons, jets and missing E T is not quite as large and the tracker trigger by itself will probably not allow to keep an overall L1 output rate of no more than 100 kHz while keeping the same efficiency for these objects. Therefore CMS envisages to significantly increase the L1 output rate and so more data will have to be processed by the HLT. Considering the progress in computing power to be expected ("Moore's Law"), at the time of the Phase-2 upgrade the HLT farm could be made to process up to 1 MHz of events, ten times more than at present. As the HLT requires all CMS detector data as input, the speed of the detector readout will have to increase as well, and certain detector electronics components will have to be upgraded for this purpose.
In this context, one might ask the question if a hardware-based Level-1 trigger will actually still be indispensable or if CMS should aim at a "triggerless architecture" like LHCb and other experiments, reading out the full 40 MHz of bunch crossing rate and doing all the selection in computer farms. Studies have shown that this will not be feasible for CMS, however. The biggest problem would be the amount of front-end electronics needed in the tracker and the power dissipation arising from it. For readout, power supply and cooling so much material would have to be installed in the tracker that its performance would suffer significantly due to particle scattering. So, while the HLT will develop further and play a very important role in event selection, it will not allow CMS to eliminate the Level-1 trigger.

Conclusions
The Phase-1 upgrade of the CMS trigger is already underway. Commissioning work will continue in parallel with data taking during the beginning of LHC Run 2 in 2015 and care will have to be taken to avoid interference.