An FPGA-based Instrumentation Platform for use at Deep Cryogenic Temperatures

We describe a cryogenic instrumentation platform incorporating commercially-available field-programmable gate arrays (FPGAs) configured to operate well beyond their specified temperature range. The instrument enables signal routing, multiplexing, and complex digital signal processing at temperatures approaching 4 kelvin and in close proximity to cooled devices or detectors within the cryostat. The cryogenic performance of the system is evaluated, including clock speed, error rates, and power consumption. Although constructed for the purpose of controlling and reading out quantum computing devices with low latency, the instrument is generic enough to be of broad use in a range of cryogenic applications.


I. INTRODUCTION
Electronic instrumentation at cryogenic temperatures is widespread in astronomy 1 , experimental cosmology 2,3 , and essential to the performance of particle 4,5 , antimatter 6 and single photon 7 detectors as well as quantum information devices 8 . In most configurations, the devices or detectors that require cooling are separated from their room temperature interface and control electronics, typically using low thermal conductivity wiring to cross the often significant thermal gradient. Owing to the Wiedemann-Franz law, thermally resistive wiring must also be electrically lossy, limiting its bandwidth and power carrying capability. For complex instrumentation systems that employ large numbers of wires 4,9 , wide bandwidth transmission lines 10,11 , or lowlatency measurement and control, the physical separation between room temperature electronics and the cryogenic device environment poses practical challenges that can impact performance.
Integrating much of the interface electronics inside the high-vacuum stage of the cryostat can partially address these challenges. Embedded cryogenic amplifiers 12 and multiplexing circuits 1,10,13,14 , for instance, are commonly used to boost weak signals over lengthy transmission lines 15 or to minimise the number of separate cables and feedthrough connectors traversing the vacuum space and temperature gradient. Including in this approach the possibility of operating digital-to-analog converters (DACs) 16,17 and analog-to-digital converters (ADCs) cryogenically 18,19 , as well as cryogenic logic and memory systems opens the prospect of digital signal processing, feedback, and realtime control without the need to bring signals up and out of the cryostat. In this configuration the exclusive use of superconducting cables and interconnects also becomes feasible 20,21 , greatly reducing the thermal conductivity and cross-section of signalcarrying cables in comparison to lossy normal metals.
Here we describe the design and operation of a modular instrumentation platform for supporting digital sig-nal processing applications at deep cryogenic temperatures, including the functionality of a soft-core processor. Key components of the system are commercially available field-programable gate arrays (FPGAs), configured to function well-beyond their specified operating temperature. The instrument incorporates expansion ports for connection to peripheral data converters such as DACs and ADCs, of use in the autonomous operation and feedback control of quantum information devices [22][23][24] . Beyond quantum science, the platform is sufficiently generic to be of wide applicability in the read out and control of various cryogenic detectors and devices.

A. Overview
The instrument comprises a motherboard equipped with an Artix-7 FPGA (Xilinx Inc.) and ports for connecting up to five 'daughterboards', as shown in Fig.  1. This modular design allows the system to be configured for specific cryogenic applications while providing a generic platform that offers power, digital logic, communication links, and associated thermal management. There are two high-speed and three low-speed connectors for the daughterboard modules on the rear of the motherboard (see Fig. 1(b)(ii)). The two high-speed connectors each have 32 dedicated differential pairs and 8 power pins, and are suitable for high-bandwidth modules including giga-sample per second data converters. The three low-speed connectors share 18 single-ended signalling lines and 8 power pins, have daisy-chained JTAG (IEEE 1149.1) lines, and are suitable for modules which do not need high-bandwidth communication with the motherboard. Various communication protocols are possible using the low-speed connectors; for example, we have made use of a clock-line, a sync-line, and 16 data-lines to transmit 16-bit words using a two-word address/command packet followed by a variable-length data (e) Modular enclosure, partially filled, and mounted beneath the 4-K stage of a Leiden Cryogenics CS450 dilution refrigerator. A high-speed DAC daughterboard and a low-speed DAC daughterboard fill one high-speed and one low-speed expansion slot, respectively. Details of the components used are given in Table I. packet. In this paper we do not describe further the separate daughterboard modules, which can be customised for specific applications.

B. FPGA
Several semiconductor devices, such as bipolar junction transistors and diodes, suffer from carrier freeze-out at deep cryogenic temperatures, owing to the small fraction of donors that remain ionized. In contrast, the presence of large electric fields in complementary metal-oxide semiconductor (CMOS) devices leads to field-induced donor ionization 2 . Carrier freeze-out effects can be sup-pressed by these fields to the extent that digital circuits can continue to operate at deep cryogenic temperatures. In selecting devices that are compatible with cryogenic operation, the presence of high-K dielectrics to suppress transistor gate-leakage provides an indication that large electric fields are present. The FPGA device central to our instrument, for instance, is manufactured on the TSMC Ltd. 28 nm process node, which makes use of hafnium-oxide between gate and channel 25 .
The sole active component on the motherboard is a Xilinx Artix-7 FPGA in a 484-pin ball grid array package, and is pin-compatible with 15 k -100 k (wire-bond package) and 200 k (flip-chip package) logic element versions of the integrated circuit (IC) 26 . The Artix-7 is the lowest power of Xilinx's 7-series FPGAs and has 0.9 to 13.1 Mb of on-chip block random access memory (RAM), and 45 to 740 digital signal processing (DSP) slices, each with a 25 × 18-bit multiplier. DSP configuration options include pre-and post-adders and a 48-bit accumulator 27 . High speed grade (-3) and low power (-2L, -1L) variants are available 28 . The specific device used in our instrument is an XC7A50T-2FGG484C, which has 50 k logic elements, has 2.7 Mb of block RAM and 120 DSP slices, and comes from the common '-2' speed-grade bin in the commercial temperature range (0-85°C). We also make use of Spartan-3 and Spartan-6 devices, which are configured to be operational at deep cryogenic temperatures. These simpler FPGAs are ideal for adding functionality to the daughterboards, for example, in parsing data to a DAC.

C. Printed Circuit Board Design
The motherboard is an 80 mm × 80 mm, 8-layer FR4 printed circuit board (PCB), with all layers using 35 µm (1 oz) of copper, as shown in the stack-up illustrated in Fig. 2. The board is finished with electroless nickel immersion gold (ENIG) plating. A simplified bill of materials is provided in Table I, listing the manufacturers and part numbers of the components.
Capacitors used on the motherboard and daughterboards for decoupling and filtering are a mix of NP0 ceramic and tantalum polymer (TP). At cryogenic temperatures NP0 capacitors lose negligible capacitance, and maintain low equivalent series resistance (ESR) 29,30 , but the capacitance density is small. TP capacitors also retain stable ESR but suffer from a reduction in capacitance when cooled 30 . Their higher capacitance density however, makes them more suitable when large capacitance is required. Thin-film resistors are used for their temperature stability, as opposed to thick-film resistors, which have been observed to vary dramatically in resistance when cooled.

D. Communication and Clocking
Communication between the cryogenically operated FPGA and room temperature instruments is provided via stainless steel coaxial cables that mate with the SMA connectors, with optional 50 Ω termination, on the motherboard. A global clock signal is also provided to the instrument in this way. Although alternative clocking and communication protocols are possible, our typical configuration brings three coaxial cables into the cryostat for the FPGA clock, for a serial input signal, and for a serial output. The clock is used to generate various other internal clocks required: for operation of the serial interface, clocking data for the low-speed connectors, and to run internal logic. The power supply for the instrument as well as the FPGA programming signals are carried from room temperature using beryllium-copper cryogenic loom wire to the 37-pin Micro-D connector on the power-distribution module (see Fig. 1(a)). The module distributes power and programming signals via two 10-pin sockets on the front of the motherboard; one socket for digital power and programming, and one socket for analog power. Five digital and seven analog voltage lines, plus analog and digital ground lines, are supplied. Four-terminal sensing, with force and sense pairs tied at the power distribution board, compensates for loom wire resistance ensur- ing correct voltages at the instrument.
Programming and debugging of the motherboard and daughterboards is performed using a Xilinx Platform Cable USB II. The cable uses a standard 4-pin interface: the clock (TCK) and mode-select (TMS) signals are shared between the motherboard FPGA and the low-speed connectors; the test-data-in (TDI) and test-data-out (TDO) signals are daisy-chained from the motherboard FPGA to each of the low-speed connectors. In case a daughterboard module is not installed in a low-speed connector slot, a jumper is provided on the front of the motherboard to connect the unused TDI and TDO, ensuring a complete JTAG chain (see Fig. 3).

F. Thermal Management
The entire instrument comprising power-distribution module, motherboard, and daughterboard modules is housed in a gold-plated copper chassis, which ensures good thermal coupling between the active electronic components and cryostat. The overall length and width of the instrument is 96.5 mm and 88 mm, and the height is 92 mm. The motherboard and each daughterboard have their own copper mounts which feature extrusions to make direct thermal contact to the packaging of the integrated circuits. In Fig. 1(d), extrusions are shown which thermally connect individual DAC ICs on a lowspeed multi-channel DAC daughterboard module. The instrument is installed at the 4-K stage of a cryogenfree dilution refrigerator, as shown in Fig. 1(e). Unused slots can be covered with blank copper panels to reduce electromagnetic interference. The instrument is cooled slowly from room temperature in the presence of helium exchange gas, which is evacuated when a temperature of 4 K is reached.

III. FPGA OPERATION
Modern FPGAs have complex internal architectures with many subsystems for specialised tasks. The reconfigurable general-purpose digital logic comprises a large number of configurable logic blocks (CLBs). A switch matrix for each CLB connects it to the general routing matrix. Each CLB contains flip-flops (FFs), look-up tables (LUTs), multiplexers, basic logic, and memory 31 . FPGAs incorporate DSP slices which contain hardware multipliers and accumulators, for specialised high-throughput operations. In addition, input and ouput (IO) buffers can be configured to suit various single-ended and differential voltage specifications. A summary of the operation of these components at cryogenic temperatures is given in Table II, for Artix-7 and Spartan FPGAs.

A. IO Voltage Characterisation
We first investigate if cooling the instrument leads to variations in the FPGA switching voltage levels associated with the single-ended low-voltage CMOS (LVC-MOS) and low-voltage differential-signalling (LVDS) logic standards. Input thresholds are measured by applying a dc input voltage and measuring the minimum voltage which always gives a high output (V IH ) and the maximum voltage which always gives a low output (V IL ). A common-mode voltage of 1.2 V is used for LVDS. Results are presented in Table III. Both LVCMOS and LVDS input thresholds change negligibly with cooling to cryogenic temperatures allowing standard operation. We observe a decrease in the resistance of pull-up resistors and differential termination resistors with cooling, but note that these variations can be compensated for with careful circuit design. LVCMOS outputs function normally.
The parameter that varies the most with cryogenic operation is the LVDS output voltage. Below 50 K, the common-mode and differential output voltages both decrease dramatically on the Artix-7, and both increase dramatically on the Spartan-6. Only the Spartan-3 FPGA exhibits functioning differential output signalling. The operation of LVDS outputs are likely linked to internal bandgap voltage reference offsets occurring at low temperature.

B. Performance and Soft Processor Operation
To demonstrate the functionality of the instrument's general-purpose digital logic at deep cryogenic temperatures, we have implemented an embedded soft processor, a Xilinx 'MicroBlaze' 32 , executing standard C-code, and loaded onto the FPGA via the JTAG interface. We have run implementations with logic utilisation of up to 8569 FFs (13% utilisation), 7190 LUTs (22% utilisation), and 1476 kb of block RAM (55% utilisation). Based on this demonstration we expect that similar IP-cores, for instance the Cortex-M1 implementations available from ARM Ltd. 33 , could be embedded in our instrument and run in the cryogenic environment.
Since the instrument generates heat in proportion to its clock speed, performance is constrained by the available cooling power of the cryostat for a given temperature. With our instrument mounted at the 4-K stage of a standard cryogen-free dilution refrigerator however, we find that significant computational performance is possible without adversely affecting the mixing chamber base temperature. To benchmark performance, we carry out a series of tests involving multiply-and-accumulate (MAC) blocks, noting that these MAC operations form the basis of many DSP applications including filters, window functions, down-converters and Fourier transforms. The setup for our test comprises the execution of 30 DSP48E1 slices 27 , configured as 16×16-bit MAC operations and clocked with an external (room temperature) variable source from 0 -∼400 MHz. We proceed by comparing the output of 1000 accumulated multiplications of pregenerated random numbers to the expected result, with error rates recorded. The test is performed as a function of core voltage, comparing instrument behaviour at room temperature and 4 K.
The maximum operating frequency of the instrument is determined as the FPGA clock frequency at which there are no errors over 32 repeat test runs, corresponding to a total of 960,000 MAC operations. The maximum clock frequency is a function of both the core voltage and temperature of the FPGA, as shown in Fig. 4(a). We note that slightly higher frequency clocking is possible at cryogenic temperatures, when operating at the nominal core voltage of 1.0 V. Instrument input and output logic thresholds were measured for LVCMOS and LVDS standards, for the Artix-7, Spartan-6 and Spartan-3, at a temperature of 300 K and 4 K. The following characteristics are tabulated: input-high (VIH) and input-low (VIL) thresholds, for 3.3 V LVCMOS signals; the differential input thresholds for LVDS signals with a 1.2 V common-mode voltage; the resistance of an internal pull-up resistor for 3.3 V LVCMOS, when the input is held at 0 V; the dc differential input resistance of an internal differential termination, for a 400 mV differential signal and 1.2 V common-mode voltage; and the differential and common-mode output voltages of LVDS signals, measured using an oscilloscope with a 100 Ω terminating resistor at room temperature. Asterisk indicates an average reading for voltages that fluctuate in time.

C. Power Dissipation and Operating Temperature
We have compared the performance and power dissipation of the motherboard Artix-7 FPGA when operating at room temperature and inside the dilution refrigerator, as shown in Table IV and Fig. 4(b). Cooling the instrument to 4 K increases the static power but decreases the dynamic power. In the case of the dynamic power, we evaluate power dissipation using only the core voltage data from the DSP-block test, expressed as power per clock-rate (mW/MHz) or equivalently, average energy per MAC operation (nJ/MAC). Although we can ensure that the outer casing of the FPGA is well-thermalised to the 4 kelvin stage of the dilution refrigerator, it is likely that the die exhibits hotspots and an overall elevated temperature with respect to its packaging. We provide a course measure of the die temperature by directly accessing a semiconductor diode via external pins of the FPGA, performing a calibration of the resistance of the diode as a function of the refrigerator temperature with the FPGA unpowered (see Fig.5 (a)). During the DSP-block test, the diode current was recorded as a function of clock frequency to determine the relationship between the on-chip temperature (( Fig.  5(b)) and the power dissipation, as shown in Fig. 5(c). We estimate that the core temperature rises from 16 K at 170 mW (idle) to 27.5 K at 380 mW. We thus determine an approximate thermal resistance between the chip and the 4-K stage of the refrigerator to be 55 K/W. Despite the FPGA die having an elevated internal temperature, the instrument and its connectors remain in close thermal contact with the 4-K stage, even at the highest clock rates.

IV. DISCUSSION
Integrating FPGA-based instrumentation directly in the cryogenic environment with cooled devices or detectors has technical advantages such as the use of miniaturised, high-density superconducting cabling, and in enabling the operation of cryogenic multiplexers, DACs and ADCs. In addition to these practical aspects, the functionality of FPGAs at cryogenic temperatures provides a path to establishing complete instrumentation solutions that take advantage of reduced temperatures to improve performance. Cooling analog circuits, for example, leads to a reduction in thermal noise 34 and an increase in the transconductance and gain of transistors 35 . For digital systems, lower temperatures improve carrier mobility, reduce the interconnect resistance, and lower the subthreshold swing, leading to higher clock speeds and lower power dissipation 36,37 . Integrated with cryogenic FPGA, it is anticipated then that these improvements to semiconductor-based circuits can lead to enhanced performance of the classical control and readout hardware needed to scale-up quantum computing devices. Further improvements are likely found via the use of structured-ASICs (application-specific integrated circuits), that hard-wire program implementations during fabrication. Such devices can lead to significant reductions in power dissipation at cryogenic temperatures.
We also draw attention to the potential use of cryo- shows the diode current in response to a voltage, measured as the refrigerator temperature is increased and decreased to provide a calibration (arrows show direction of temperature sweep). As some hysteresis in the diode response is observed with heating and cooling, our calibration is taken from a lineof-best fit. In (b) the diode current is monitored as a function of FPGA power, allowing a functional dependence of core temperature on power to be determined. This function is plotted in (c), extrapolating to conditions at maximum operating frequency and idle (only static power).
genic FPGA in the context of superconducting digital electronics, interfacing with rapid single flux quantum (RSFQ) 39 or reciprocal quantum logic (RQL) devices 40 .
Cryogenic systems featuring such devices are already commercially available 41 , controlled via room temperature FPGAs. Operating the control electronics cryogenically would enable more tightly-integrated and compact instruments of potential use in systems requiring lowlatency, high-speed feedback control.

V. CONCLUSION
We have developed a cryogenic FPGA-based modular instrumentation platform for the readout and control of detectors and devices at temperatures approaching 4 K. Our instrument makes use of three models from Xilinx Inc. (Artix-7, Spartan-6, and Spartan-3), which have been shown to operate in the cryogenic, high-vacuum environment of a dilution refrigerator. It is possible to use much of the functionality of the more powerful, Artix-7-FPGA, including the operation of a soft-core processor and DSP blocks. We anticipate that such FPGA-based instruments can enhance the performance of the classical control hardware needed in the operation of nextgeneration quantum technologies.