The CUOF optical link for the electronic system upgrade of the CMS Muon DT

The upgrade of the acquisition for the muon Drift Tube foresees the relocation of the Sector Collector electronics, from the cavern towards the counting room. The project requires an electrical to optical conversion by the Copper to Optical Fibre (CUOF) board, developed and tested in a radiation environment for components qualification. More tests were performed inserting the prototype in the actual acquisition system, with such a good result that a substantial production is in progress. The installation for a large number of channels is planned for the 2013 autumn, with the new system that will be active beside the actual one.

. The relocation for the sector collector electronics, and the introduction of the optical link.
In order to maintain the current high efficiency for the full lifetime of LHC and for the next High Luminosity LHC (HL-LHC), an upgrade program has been planned. This foresees the relocation of the complete SC electronics from the CMS cavern (UXC) to the CMS counting room (USC), leaving the hazardous environment close to the experiment.
In this way the maintenance and the eventual intervention on the electronics may be accomplished without waiting machine technical stops avoiding, in this way, the risk of losses of large fraction of the detector.
Moreover the relocation of the SC electronics to the counting room will allow the redesign and the upgrade of the SC system together with the Drift Tube Track Finder (DTTF) at a later stage allowing the development of an upgrade for the entire L1 trigger system.
The fulfilment of the CuOF upgrade project, planned for the Long Shutdown 1 (LS1) of LHC in the period 2013 ÷ 2014, envisages to turn electrical signals (in LVDS standard on copper cables) into optical ones (on optical fibres). Globally that conversion concerns a total number of 3500 channels, that will be routed toward the counting room. The data streams run at a rate of 240 Mb/s for the readout link, and a rate of 480 Mb/s for the trigger link.
The Copper to Optical Fiber (CUOF) converter system will be located in the towers beside the magnet into the CMS cavern, inside the crates for the SC electronics. To assure the robustness of the data link, the electronic components have to be qualified for a moderate radiation tolerance and a low magnetic field (figure 1).
The performances of the CUOF system were evaluated in details to have a good comprehension of the behaviour with respect to the different data streams, corresponding to the readout path and the trigger path.
A complementary Optical Fiber to Copper (OFCU) system 1 was designed to recover the electrical signals. By this way the full information coming from DT chambers will be delivered again directly to the present SC electronics, that are now relocated in the counting room. Two specific OFCU boards, with a similar electronics but different form factor and channel granularity, were designed to cope with the readout path and trigger path allowing to convert back the data into an LVDS standard [4].

The CUOF system
The main goal of the CuOF system if to convert the large number of the readout and trigger electrical signals coming from the DT chambers into optical signals.
That allows a one to one independent translation, and the availability of the complete DT chamber information in the counting room. The CUOF system is designed as a motherboard providing the slow control services, and hosting a set of mezzanines which contains the sensitive electronics running at high speed.

The CUOF motherboard in VME format
The CUOF motherboard has the function to support mechanically and provide power supply to 4 CUOF mezzanines and it provides all the functions of the slow control which sets and monitors all the parameters of the optical link.
The motherboard has a 9U VME form factor and hosts the mezzanines on 4 slots via easily pluggable connectors. On every motherboard a set of 3 optical fibre fanouts is arranged, each of them composed of 12 independent fibres terminated on one side with LC connectors and on the other side with a single MTP connector. The MTP connectors are fixed on the rear side of the motherboard, and allow the signal transfer to the CMS counting room through the 60 m long optical trunk cables. The CUOF motherboard requires a custom backplane, with a single din41612 connector in the central position J2, for power supply lines and slow control buses (figure 2). The slow control system establishes the laser driver settings to manage the working point of the optical link, and monitors the environmental parameters such as the driver temperature and supply voltage. The slow control data are collected by several ADCs sitting on the motherboard, and connected to two Field Programmable Gate Arrays (FPGA) from Microsemi through Inter Integrated Circuit (I 2 C) ports. The control logic is based on the two FPGA circuits and it is designed to be redundant in order to increase the robustness of the system, and to avoid failures due to single event upsets.
The slow control circuit is interfaced with two independent Controller Area Network (CAN) buses managed from the counting room and distributed, via optical fibres and converters, on the custom crate backplane. The redundancy adopted on the FPGA component is applied, for the same reason, also on the CAN bus.
-3 -Typically the slow control functions are managed by just the primary FPGA and its corresponding bus, while the secondary FPGA is used as backup and it becomes active only when the primary one has a problem and it does not respond properly.

The CUOF mezzanine
Each CUOF mezzanine receives 8 signals on copper cables from the readout or trigger muon barrel electronics of one DT chamber.
Up to 55 m long Category 5 Shielded Twisted Pair (STP), connected to RJ45 connectors are used to carry Low Voltage Differential Signalling (LVDS) signals up to the CUOF.
Every optical link has, as input stage, a line equalizer that is used for compensating the distortion due to the long copper line and for restoring the DC levels. Then the equalizer output is sent to a laser driver, which in turn controls the laser diode setting through the bias current and the modulation current. The bias current determines the low level of the digital light signal emitted from the output diode, setting the DC working point near the laser emission threshold. The modulation current is added to the previous one and it determines the high level of the light signal, representing the dynamic component applied to the AC coupled laser.
The Vertical Cavity Surface Emitting Laser (VCSEL) diode was chosen due to the low power consumption and high radiation tolerance with respect to the Edge Emitting Laser (EEL) [5]. The emitted light has a 850 nm wavelength, and the typical level for the working optical power is chosen to be in the range from 300 µW up to 500 µW. Due to the radiation exposure and ageing problems, the light emitted by the VCSEL may change and degrade. In order to maintain the optical power at a level with a good quality of the optical link, the bias current and the modulation current may be tuned through the FPGA via a two wire protocol managed by the CUOF slow control. Inside the VCSEL package a photodiode is present also and can be used for measuring the light, and to implement a feedback loop for the automatic power compensation.
The 8 VCSEL on the mezzanine are matched to the fibres using LC type connectors through a custom made adapter, which has been accurately designed and manufactured to maintain the diode and the fibre ferule at the optimal distance. The adapter was made of PolyEther Ether Ketone (Peek) which is an advanced thermoplastic polymer, well known for having a good radiation resistance.
The selected fibres for the data transfer towards the counting room are of multi-mode type and are bundled in ribbon of 12 channels, that in turn are gathered in a large trunk cable where some spares are available. The connection with the optical fibre fanout is made on the back side of the motherboard, where the MTP to MTP junctions are positioned.

Qualification test for the CUOF system
In this section the certification tests on the CUOF prototypes and the certification tests for the production of the whole system are described. During the prototyping phase of the CUOF boards some certification tests have been defined in order to guarantee the goodness of the optical link transmission with the patterns generated by the DT chambers from readout and trigger paths. Then the prototypes have been tested under irradiation in order to certify the lifetime of the CUOF system during the HL-LHC era. Finally an integration test within the CMS complete electronic chain in the experiment cavern has been performed.
-4 - The good performance of the tests on the prototypes has allowed to start the full CUOF system production and the setup of the protocol of the qualification tests to be performed during the board production.
Two main tests must be fulfilled by the CUOF mezzanine boards.
The first one, called scan test, is an overall error evaluation of each channel for every combination of the main laser diode parameters such as modulation current and bias current.
For each point, in the modulation bias plane (figure 3), the amount of errors were registered to find the optimal working area.
In figure 3 an example of the scan test is shown for a typical optical link channel. In the figure the modulation current and the bias current of the laser driver are varying within the laser diode allowed intervals, and the number of bit errors are measured with respect to the input pattern for each (modulation, bias) current value.
The binning for the scan test is chosen to have the best detailed topology answer in the two parameters plane, however this test indicates only qualitatively that the optical link is working because the test is performed for each (modulation, bias) current value during only a short period of a few seconds. The bit error response is translated into a color logarithmic scale which is shown in figure 3 on the right side. The optical link is performing in a good way when the bit error rate is 0 which corresponds to the white colour in the scale. In figure 3 it is shown a wide white area indicating large intervals of bias and modulation currents in which the link may have the optimal working point.
The working point of each optical link is chosen with the second test that is the Bit Error Ratio (BER) test. It is a precise measurement of the link error rate in a small number of (modulation, bias) current points that have to be less than 1 · 10 −12 with a Confidence Level of 95 %. This test is performed like the previous one injecting in the optical link the bit pattern and reading it back at the link output. The pattern used both for the scan test and for the BER test is the one which is the typical trigger pattern of the DT chambers. It is a strongly unbalanced pattern made principally of idle bits '0' interleaved with some random bit data.
-5 - The test set up to perform the scan and the BER test consists of a custom Pattern Unit 2 system which was produced to emulate the input data coming from the DT chambers on LVDS cables which are transmitted by the optical link and the receiver part which receives back the pattern simulating the present ROS and TSC readout and trigger DT boards. The PU boards are triggered by an external FPGA and can generate and acquire serial streams of bits under the control of a Labview application which, after the evaluation of the link latency, sends the input streams, reads the output ones and compare the two computing the bit error rate.
In order to test in more details the goodness of the light emitted by the laser diodes which are mounted on the CUOF mezzanines the eye diagram was studied.
In figure 4 the eye diagrams generated injecting in the laser diode the readout pattern (top) and the trigger pattern (bottom) are shown.
That results were obtained acquiring the optical signal at the output of the laser diodes, and the eye diagrams are changing their shapes with an increase of the total jitter for the trigger pattern that is the most unbalanced pattern. This behaviour is due to the front-end ASIC that simply sends the original data, and cannot implement a balanced data encoding.
An ageing test was carried out keeping the card in the climatic chamber at 70 • C and 70 % RH for 1000 hours without any significant change [6].
The CUOF system will be placed in the CMS cavern beside the main body of the experiment and will be subject to a moderate magnetic field and a low level of radiation.
Therefore two irradiation tests were performed on the mezzanines and motherboards prototypes: the first corresponding to an integrated dose of a 10 years period of LHC and the second to an integrated dose of 10 years of HL-LHC.
The evaluation for Total Ionizing Dose (TID) in the region close to SC crates is expected to be, at LHC condition, approximately 1 Gy for 10 years assuming an uncertainty factor of 3. For the same time interval is estimated an overall 1 MeV equivalent fluence of around 3·10 10 neutrons/cm 2 .
In the same location beside the CMS magnet and with the same uncertainty factor the TID is foreseen to be, at HL-LHC condition, about 10 Gy for a duty of 10 years. Again the hypothesis relative to the same time period for the 1 MeV equivalent global fluence is approximately 3 · 10 11 neutrons/cm 2 , assuming a collected luminosity of 5000 fb −1 .
For the radiation tolerance of the system, several prototype tests were performed at the H4Irrad facility at CERN aiming at evaluating the behavior for the present condition and for a 10 years period in the HL-LHC environment. After the last run an evaluation of the Single Event Upset (SEU) occurrence was accomplished for the selected a3p600l FPGA, based on non volatile flash technology. An 8-b word was written in an instantiated register and continuously monitored, searching for a bit flipping under irradiation. When such an event appears it is registered, the word is restored and the monitoring resumes. The SEU trend matches quite precisely with the beam integrated luminosity, showing a proportional relationship, and a 10 SEU/day rate for the full system was calculated. This is considered admissible due to the large number of channels.
Finally the CUOF optical links have been tested in the CMS experiment environment inside the tower crates in the experimental cavern. In this context the CUOF boards were receiving the signals of cosmic rays detected by the DT chambers. They translated the electrical signals into optical signal and transmitted them to the OFCU receiver boards which turn back the signal to electrical one. Finally everything was input to the present ROS and TSC boards and acquired by the CMS central DAQ.
This test concerned 9 different chambers distributed in 5 different sectors for a total of 72 optical links.
The acquired data, analysed with the standard software tools, fulfilled completely the test since it is not possible to distinguish them in terms of parameter as occupancy, efficiency and residuals (figure 5).

Certification tests during CUOF boards production
All the certification tests described in the previous section allowed to optimize the design of the CUOF optical links and the positive outcome of the tests performed on the prototypes put confidence for the launch of the boards mass production.
The full system consists of 540 mezzanines and 150 motherboards which translates the whole amount of signals coming from the 250 DT chambers in the CMS experiment.
The protocol defined to certify the CUOF optical links foresees three kind of tests: the first concerning the mezzanines, the second the motherboards itself, and the last one the mezzanines mounted on the motherboards together with the fibre fanouts which interface the links with the main optical trunk cables.
Firstly the mezzanines have to pass the scan test and the BER test as described in the previous section. The BER test is performed in the region of (modulation, bias) currents which is most suit--7 - Figure 5. The comparison between the run executed with transmission by the CUOF system with respect to the present system. able for the trigger type of patterns which is strongly unbalanced. The typical range of (modulation, bias) currents goes from 8 mA to 9 mA for modulation and from 2 mA to 3 mA for bias current.
The stand alone validation of the motherboard foresees, as the first step, a check of all the ADC channels reading the parameters of the 32 laser diodes. Each laser driver provides information such as a scaled replica of the bias current, a photodiode current proportional to the emitted optical power, the internal temperature and a fault flag that is raised in case of over current. To have a predictable answer a mock up of the mezzanine, generating a copy of the predicted voltages and consuming expected current, was prepared and inserted into the respective slots. Taking into account that also the power lines for each mezzanine are monitored, together with the FPGA supply, a global number of more than 140 ADC channels are repeatedly acquired for a noise evaluation. The last step of the CUOF motherboard test foresees the switching of the CAN bus control from the primary FPGA to the secondary FPGA, where all the previous measurements are repeated again to pass the final verification.
After the checking of the single motherboards and of the single mezzanines, a set of 4 from the latter ones are selected and assembled on the former one together with the optical fibre fanouts and the MTP junctions. For that reason another scan test is performed on the fully equipped motherboard to avoid any possible problem on the optical path, and to verify the setting capability of the laser diodes by means of the CAN bus.

Summary
The first batches of CUOF motherboard and CUOF mezzanine are already received, and have been tested according to the protocol and have been assembled according to the trigger and readout channel distributions.
At present the full amount of CUOF optical links needed for the installation of 20 % of the CMS DT chambers have been produced. The installation procedure began in October 2013, with a -8 -

JINST 9 C02020
general test in November 2013 consisting in a combined global run with the full CMS experiment. Currently the whole production of the system is in progress, and all the cards are expected to be installed in time within the LHC Long Shutdown 1.