SPIDR: a read-out system for Medipix3 & Timepix3

The realisation of the Timepix3 chip opened the way for new opportunities in research areas such as particle tracking with both semiconductor sensors and gas filled time projection chambers, electron microscopy and imaging mass spectrometry. To exploit the full capability of the Timepix3 chip, Nikhef developed a compact read-out system, called SPIDR that can deal with the high data output of 80 Mhits per chip per second. The main read-out board connects to both 10 Gb Ethernet and 1 Gb Ethernet devices. The latter obviously at a reduced rate. The main board connects to individual chip-carrier boards via a standard FMC connector. The system is designed such that support for other readout chips is foreseen via reprogramming of the FPGA. Besides the Timepix3 chip also the Medipix3 chip is currently supported. Both the main board and the chip carrier boards are cooled, via the housing and a fan to obtain a stable temperature of around 40 ± 0.2 °C for the Timepix3 chips. We will present the system and the results obtained with the LHCb beam telescope at CERN and proton radiography data obtained with a time projection chamber based on GEM technology.


System
The system consists of the housing with the cooling and electronic boards. The electronics need firmware in the FPGA and software to be operated. The electronics consist of a main board to which a number of mezzanine boards are connected with dedicated functions like power distribution and bias voltage supply. Each of these components will be described below.

Main electronics board
The main electronics board hosts the FPGA. We had a choice between an FPGA with links that can transfer more than 12 Gb and one with lower speed transceivers. We have chosen a mid-range FPGA with an external 10 GbE connection. This is a more cost effective solution than a high end FPGA with an integrated PHY. The chosen FPGA is the Xilinx Artix7 and we use the type XC7A200T because it has the greatest number of available I/O pins. For the 10 GbE connection, we have chosen the Marvell Alaska 88X2010. This combination is significantly cheaper than a single integrated device.
To connect to the chipboards on which the Medipix3 or Timepix3 chips are mounted, we have chosen FMC connectors according to the Vita 57 standard. These FMC connectors have 400 pins, 80 of which are differential pairs and 10 are transceivers. Additionally, we have a second FMC connector with 160 pins, which we use for accessing multiple chips.
The connection to the data-acquisition system uses the aforementioned 10 GbE link and an additional 1 GbE link. If both are connected, the 10 GbE link is exclusively used to transfer data to the data-acquisition system, while the slow control data and monitoring is run via the 1GbE link.
There are three data streams: slow control, monitoring and data. The links are configurable to determine which stream goes to which link.
Additionally, we have two HDMI connectors that are used to transfer fast signals like clock signals and synchronisation signals to run e.g. multiple systems via a common timing control unit. There are eight differential pairs available on the HDMI connectors. A separate TDC channel can be used to supply timestamps for applications that require a trigger. The current resolution is 260 ps -2 -and we aim for 50 ps. This channel is connected via an SMA connector. For debugging purposes there is also a mini USB connection.
To store data either temporarily or more permanently a number of memory options are available. Three banks of Flash memory of 128 Mb each can be used to store the bit-files specific for Medipix3 and Timepix3 operation. This can be used to switch from one type of chip to the other without the need to reprogram the FPGA from scratch from a PC. The third bank is a user specific memory, which can be used to store calibration and chip settings. The 4 GB DDR3 memory can be used as a buffer e.g. in case the data is produced in bursts and the data-acquisition system cannot keep up.
The last block, named peripherals, is actually a collection of different functions. We can measure both the ambient temperature and the temperature near the pixel chip. This can be used to drive the fan, which is currently used. It is also possible to drive a Peltier element in case that would be required. In addition, the air pressure and the humidity are measured as well as the orientation of the main board. There are four ADC channels to monitor e.g. the DAC values set in the Medipix3 or Timepix3 chips. In addition, for the attached chips i.e. Medipix3 or Timepix3, the current and voltage are being monitored of the analogue and digital power domains.

Xilinx Artix7 FPGA
The Xilinx Artix7 provides the means to communicate between the pixel chips and the PC as indicated in figure 2. The Leon3 CPU is a 32 bits SPARC V8 architecture [4] soft-core processor that is programmed into the FPGA. The Leon3 takes care of the slow control data communication with the read-out chips and the peripherals. The communication with the PC uses a TCP/IP stack as this is less prone to packet loss than a UDP implementation. The router is the main traffic controller for the Ethernet frames. There are three data streams: slow control (bi-directional TCP), data (UDP) and monitoring (UDP). Each of the 3 streams can be assigned to either the 1 or the 10 GbE link. It can be configured such that certain packages go over the 1 GbE link and/or the 10 GbE link. The chip data is transferred to the selected Ethernet link using UDP because TCP implementation generates too much overhead for a large data stream. The timing information from the Time to Digital Converter (TDC) is merged into the data stream coming from the pixel chip(s). This timing information can be e.g. a trigger signal from a scintillator. The fast control signals are transferred directly to the read-out chips via the differential pairs in the HDMI cable, but they can also be generated in the FPGA. These signals are e.g. the clock signals, shutter etc.

Peripheral electronics boards
In addition to the main board, we have designed two peripheral boards. One, the power board, provides the power for the FPGA, and supplies the voltages for the Medipix3 or Timepix3 chips. The other one is the bias voltage board, which can provide the required bias voltage for thin silicon sensors from few tens of Volts up to +/-1.2 kV for high Z material sensors like CdTe with a maximum current of 100 µA. For the first version of the board that provides the bias voltage the voltage/current monitoring is done with a 10-bit ADC. The current can be measured with a precision of 97 nA and has a maximum of 100 µA. The bias voltage can be measured with a resolution of 1.2 V.
-3 -  Behind the area for the ASICs a heat sink can be mounted.

Chip carrier boards
To enable the use of different read-out chips in various configurations, we have designed a number of chip carrier boards that can be connected to the main board via the FMC connector. Both for Medipix3 and for Timepix3 single-chip boards and quad-chip boards are available. The single boards are made of FR4, while the quad boards have in addition a flexible kapton PCB between the part where the chips are mounted and the part that connects to the main board, which can be seen in figure 3. This is to enable the detection of particles or photons in various geometrical configurations. One configuration is as an X-ray camera, as shown in figure 4, where the cooling is integrated for the ASICs. Another configuration would be as a beam telescope. In the latter configuration, the chip-carrier board needs to be separately supported and as one would like to have as little material as possible between a number of carrier boards it requires a different cooling method.
-4 - As there are a number of applications that require the detectors to be in vacuum, we are working on a vacuum compatible chip-carrier board solution, but none is available as yet.

Housing
The housing, which holds all the electronic components, is a rectangular box made by folding sheet metal into a U-shaped form and closing the top with a ribbed cooling plate. To this plate the FPGA is connected with heat conducting tape. The sides have slits to let out the air that is brought in by a fan from the back and the sides. The backside and front side are 3D-printed parts, made of Nylon PA2200. For an ambient temperature of 25˚C, the temperature of the FPGA is 50˚C and the temperature of the detector itself is 42˚C. The housing is shown in figure 4. The black part with the metal hatch is the 3D-printed part to which the heat sink is bolted. The chip-board is mounted on the heat sink with thermally conductive paste and screws. The protective hatch can be inserted as protection for when the device is not in use. At the back side all the power and data connections are made. Also there is an OLED display on the side of the housing to show IP addresses or status info. The overall size is 203 × 74 × 67 mm 3 (length × width × height).

Tests in various applications
During the development of the SPIDR we used a VC707 development board from Xilinx. We tested the firmware in various applications with both the VC707 based systems and with the first prototypes of the custom developed boards.

LHCb beam telescope
To test the Timepix3 chip in its capacity as the predecessor of the VeloPix chip for the upgrade of the vertex detector of the LHCb collaboration [5], a beam telescope was built and tested at the PS and SPS beam-lines at CERN. This telescope consists of eight detector planes; four behind and four in front of the device-under-test. Each VC707 board reads out two Timepix3 detector planes. Though the main purpose of the telescope is to test prototype sensors for the upgrade of the LHCb Velo, it also provides a nice testbed for the Timepix3 chip itself. One of the tests was to see to which particle hit rate the Timepix 3 chip could perform, relative to the design value of 80 Mhits per second per chip.
-5 - Limited by the particle rate that could be generated in the used test area, the maximum rate at which the Timepix3 chips were tested was about 20 Mhits per second per chip. At this rate the efficiency was still well above 99% [5].

Proton beam radiography
For Proton Beam Radiography in the field of cancer treatment with protons, Nikhef has developed a system that can reconstruct proton tracks both before and behind a phantom and determine the energy deposited in the phantom by subtracting the residual energy from the initial beam energy [6]. For the tracking system, Time Projection Chambers are used. In those detectors measuring the arrival time at the anode of the liberated electrons is a crucial parameter. The anode is completely covered with Timepix3 chips that provide the x and y coordinates. The chips also provide the time of arrival of each of the hits, and combined with a constant drift velocity and a trigger from the residual energy detector the location of the ionisation centres along the proton trail through the gas can be reconstructed. Combining these centres over a distance of about 3 cm, a 3D track can be fit. The time projection chambers built with Timepix3 chips and read out by the SPIDR system had a height of 5 cm, and a width and depth of 3 cm. So a phantom with a size of up to 5 × 3 cm 2 could be studied. The single electrons liberated drift towards the anode and in this case are multiplied with triple GEM foils to generate the required signal of more than one thousand electrons to trigger the read-out chips.
The Timepix3 chip can simultaneously measure the time of arrival and the time over threshold. The time of arrival is used to reconstruct the height of the initial ionisation in the gas volume by combining the arrival time relative to a trigger and the constant drift velocity of the electrons in the electric field. The time over threshold gives a measure of the number of collected electrons by a certain pixel. The measured time of arrival depends on the amount of electrons collected because of the time-walk effect The time over threshold provides a mean to correct for the timewalk contribution in the time of arrival measurement. In figure 5 one track is shown in both these modes. One can see that the passing proton had an interaction with an atom, kicking out an electron that went out of the plane parallel to the proton track as can be deduced from the different arrival time. In figure 6, a 3D reconstructed track is shown.

Spectral X-ray Imaging
To obtain more information from an X-ray irradiation than just photon intensities, one must discriminate the different photon energies that are present in the spectra from X-ray tubes. With an energy sensitive detector one can do this. Until recently, this domain was exclusively covered by silicon drift detectors with limited or no imaging capabilities. With the development of read-out chips that provide spectral capabilities to hybrid pixel detectors such as the Medipix3 chip [1] the combination of imaging and spectral information could be made, albeit with a lower energy resolution. The presented SPIDR system is also capable to read out Medipix3 chips. The first images taken with the development set-up based on the Xilinx ML605 development board are shown in figure 7. We used a single silicon sensor on top of four Medipix3RX chips, one of which could unfortunately not be read out due to a connector issue. To access the spectroscopic mode the sensor has to be bump-bonded on read-out chip on every fourth pixel. In this way the infrastructures of four pixels are joined to form one pixel of 110 x 110 µm 2 . This makes four times two thresholds available to set eight thresholds for a single exposure. In the example of a chicken leg shown in figure 7, four thresholds were set. Taking the data from a single exposure the collected photons can be sorted into a number of energy bins, as indicated in the top two plots. Each line is a histogram representing a subset of the collected data at a different threshold. Thus they show different spectral information from the same object. The area between the vertical blue lines indicate the part of the histogram that was used for the images of the chicken leg below. In the left image, the energy domain is chosen such that the inner structure of the bone can be observed. On the right, the lower energy part of the collected spectrum is shown, where the less dense area of the flesh is represented with its density differences. It must be noted that for the representation of these images no open beam or beam hardening correction has been applied. Such corrections will be implemented to improve the image quality even further.

Summary and conclusion
In this paper we have presented the newly developed SPIDR system that can be used to read out Medipix3 and Timepix3 chips at their full output capacity. The system is made such that with a -7 - dedicated chip board and reprogramming of the FPGA other chips can also be read out relatively easily. We have shown that the system is fully operational as it was used successfully in the tests for the LHCb test beam at CERN, the proton radiography experiment at KVI and the spectral images taken at Nikhef.