Synchronization of scanning probe and pixelated sensor for image-guided diffraction microscopy

Graphical abstract


Hardware in context
Scanning transmission electron microscopy (STEM) uses a focused beam as a probe to measure electron scattering in transmission through a specimen. The scattering wave projects a convergent-beam electron diffraction (CBED) pattern in the far field [1]. Classical STEM techniques such as high angle annular dark field (HAADF) or bright field (BF) STEM use area detectors to record the scattered intensity by essentially integrating the electron flux over specific regions of the diffraction pattern. The emerging method of 4D-STEM uses a pixelated camera to record a 2D image of the CBED for each position, i.e., pixel, of the electron probe in real space. Integrations over areas specified a posteriori create virtual detectors, from which the real space images (scan images) display a variety of contrast aspects. The projected electrical potential induced by the atomic nuclei in the specimen affects the phase of the electron wave. A phase image may be obtained using a quadrant detector, real or virtual, placed at the diffraction plane. This already offers a possibility to compensate for defocus aberration and reveal depth contrast from a single scan [2]. More precise phase reconstruction can be made by the methods of integrated center of mass (iCOM) imaging [3] or Ptychography [4], which require recording the full CBED pattern. Acquiring the full 2d diffraction information enables, at least in theory, to apply reconstruction schemes such as inverse multislice reconstruction [5], multislice electron tomography [6] and atomic model iterative reconstruction [7], for enhanced resolution in the final images. Finally, specimens are susceptible to damage by the electron beam. The approach of compressed sensing suggests to reduce electron exposure with minimal loss in information via sparse sampling in the real space acquisition, i.e., by skipping probe positions [8]. The combined approaches are now offered in a single a system, which acquires 2d diffraction information, guides the microscope during specimen tilting, and manages the scanning pattern. The system is based on available components and common interfaces of a standard transmission electron microscope, and is supported by open-source software.
The goal in this work is to synchronize the collection of diffraction images with the pixel clock of a STEM scan generator at the relevant pixel time on the order of 10 ls, while maintaining the ability to guide the microscope based on real-time image feedback during acquisition. It offers a first of a kind solution for a tilt-series acquisition in 4D-STEM. We build here upon a previously reported flexible scan system, called SavvyScan [2], which combines an arbitrary wave generator (AWG) to control the scan coils of the microscope, with an eight-channel analog to digital converter (ADC) acquisition card for signal recording from multiple electron detectors. In the present implementation, we acquire a single channel of data from a conventional high angle annular dark-field (HAADF) detector, while simultaneously triggering the clock of a fast electroncounting hybrid camera (ARINA, DECTRIS AG, Switzerland). In particular, we describe the software and hardware design to collect only useful data during the scans, i.e., to avoid distortions and overdose, and to acquire the signals simultaneously. The synchronization of scan position, analog channel acquisition, and pixelated detector recording is more challenging than it might appear. First of all, a typical raster scan for STEM includes a fly-back time where the beam returns or accelerates and the image information is distorted, due to inductive inertia in the scan coils [2]. Previous designs select the information during post-processing [9], but here we require selective recording in advance because the overload of storage and processing time is significant. Second, we require a real-time feedback for purposes of focusing and navigation across the grid. This is based on images provided by the HAADF detector (note, however, that in some ultra-high resolution applications a careful aberration corrections in HAADF may not be optimized for bright field images [10]). The HAADF signal is collected by an analog to digital converter (ADC) acquisition card that integrates up to 64 samples per pixel at a fast oversampling rate in order to reduce electronic noise; therefore, the same pulses cannot serve for both ADC and camera. As a solution, a master sampling clock from the ADC serves as a reference for the clock of the arbitrary wavefunction generator (AWG) that controls the scan coils of the microscope. The clock pulses of the AWG card are then used to trigger the pixelated detector, but the trigger is enabled only at useful locations of the probe. Specifically, sampling is skipped during the flyback and acceleration phases of the raster scan, thereby avoiding distortions in the image. The same scheme of editing the trigger enable list may be employed for compressive sensing applications, which propose to reproduce a full image from a small fraction of sampled data points. In order to reduce electron exposure in practice, the sub-sampling must be coupled in addition with a fast shutter to blank the beam while the non-recorded points are visited.
The synchronization system is called SavvyGate. Fig. 1 provides a connections scheme of the flexible scanning SavvyScan system. The hybrid detector camera is installed on the transmission electron microscope (Tecani F20, Thermo Fisher Scientific) and stores a stream of images (up to 120,000 images per second at 96 Â 96 pixels, and at least 20,000 per second at 192x192 pixels) on a dedicated server, in real time, in compressed HDF5 file format. The server is attached to a local Ethernet network, which includes the SavvyScan computer that sets the acquisition mode and initiates full data recording, and a separate computer that controls the microscope optics and mechanical stage via the SerialEM software framework and scripting interface to the microscope [11]. The HAADF image sourced from the ADC card (Spectrum Instruments M2p.5923-x4) is fed back to SerialEM for purposes of navigation, tracking, focus, etc. The AWG card (Spectrum Instruments M2p.6541-x4) generates the scanX,Y signals. The usual synchronization between these cards is based on a component called STAR-HUB; however, it is outside specification for clock rate slower than 128 kHz, so the synchronization is set as shown in the diagram. The ADC trigger, which indicates the start of an acquisition on command from SerialEM, is fed to the trigger input of the AWG. The output trigger from the AWG is a frame HIGH signal during the run of the scan pattern, and is fed to the SavvyGate microcontroller unit (MCU). In order to allow both synchronization and oversampling, the ADC clock rate must be an integer multiple of the AWG clock rate. To maintain stability, the output clock from the ADC is fed as a reference to a phase-locked loop of the AWG clock. The output clock signal of the AWG is then fed to SavvyGate, which generates selective pulses of 3 ls width according to a sequence table delivered from the SavvyScan computer via a Universal Serial Bus (USB) port.
The SavvyScan system functions as an ad hoc camera server for SerialEM software and provides an immediate image, which is necessary for the user as well as for automated image guided tasks in SerialEM such as tracking alignment, autofocus, and eucentric height alignment. The HAADF detector is often a favorable option to generate rapid image guidance while the diffraction images are collected for off-line processing. The HAADF signal is also useful for image alignments in tomography based on strongly-scattering fiducials such as gold nanoparticles. However, in specimens thicker than the elastic scattering mean free path some details are lost in the HAADF, and it is necessary to use bright field image for microscope guidance. Accordingly, the system should ideally return a real-time image signal such as iCOM or virtual bright field computed on the fly from the pixelated detector measurements. Such capability is recently offered in the LiberTEM-live project [12], which aims further to achieve resolution enhancement based on ptychography on the fly [13]. SavvyScan version 2b can mediate between LiberTEM-live output and SerialEM to facilitate real-time tilt-series acquisition and montage mapping based on 4D-STEM data stream. Nevertheless, HAADF is still required for the quick intermediate scans and the SavvyGate hardware supports synchronization with the scanning probe. A further route of development is the extension to unconventional scan modes such as spirals, sliding circles, or Hilbert patterns. In such cases the real-time generation of a HAADF image for SerialEM will always be available without complication. SavvyScan is already compensating for the distortions due to inductive inertia in the scan coils for the analog signal [2], and similar compensations would have to be applied to the camera-generated images. Hardware description The purpose of SavvyGate is to send trigger pulses to the camera. The functional block diagram is shown in Fig. 2. Programming and communication are provided by the USB port, which also supplies the power. A sequence table in memory must be loaded before each scan using a direct memory access (DMA) streaming from the Universal Serial Asynchronous Receiver Transmitter module connected to ST-Link USB module. In the context of STEM the sequence table is determined by the SavvyScan system. The sequence starts once the sequence table is loaded and the frame state is high. The rising edge of the clock input activates an interrupt, which upon a condition defined by the frame signal and position in the sequence table the trigger is followed by generation of a trigger pulse to the camera.
The sequence table contains 20 pages of 2500 words in a little endian byte order. The last word in each page serves to indicate if the last packet of information is received, so only the necessary number of pages are transferred. In all other 16-bit words the most significant bit specifies if to transmit (1) or skip (0) a train of pulses. The count of the pulse train is provided in the least significant 15 bits of the word. For example, a raster scan of 1024X1024 pixels requires 2048 words. Each line of scan requires one word for the skip train (meaning the number of pulses to skip, which are 400 pulses including flyback from the previous line and the travel of the probe outside the range of interest), and one word for the transmit train of 1024 pulses. Using 8 pages of 2500 words we can acquire 8 k scan images. Pulse trains longer than 32,767 clocks may be achieved by concatenation as needed. In more sophisticated patterns that involve compressed sensing, the number of trains may be larger, and it would be possible to control the beam illumination with a fast shutter using SavvyGate to reduce the electron dose.  [14]. 1024X1024 useful pixels. The overall time of our raster scan pattern includes 29.3% part overhead for the flyback and acceleration periods, so the actual duration of the probe over a pixel is 14.1 ls. The field of view is exposed to electron damage only during 70% of the overall scan time. We allowed for a setting time of 3 ls (for demonstration) in the ARINA detector before the trigger, so the integration time in ARINA was set to 10 ls, in which the electrons are counted. The setting time can be reduced further to 0.5 ls, which in addition to 1 ls propagation delay implies a minimum 1.5 ls ''dead time" in which a point of interest in the specimen is exposed to radiation but electrons are not counted. The experiment was performed on a scanning transmission electron microscope (Tecnai T20-F, Thermo Fisher Scientific) set to acceleration voltage of 200 kV (De Broglie wavelength of 2.5 pm) in microprobe mode (convergence angle of 0.8 mrad) and about 5pA electron current. In order to test synchronization between the HAADF detector (Fischione 3000) and the ARINA camera (DECTRIS), a virtual dark field image (vDF) is computed from the diffraction pattern recorded on the latter. The intensity of each pixel is determined by the sum of all electron counts falling between radii of 12 and 48 pixels on the diffraction images (measured from the center of the diffraction disc). A common term used to specify the magnification of the diffraction pattern is the camera length, in analogy to an equivalent camera obscura setup. At an effective camera length setting of 390 mm, the corresponding scattering range included in the vDF is between 0.26 and 1.03 Å À1 (6.4 and 25.6 mrad), whereas the HAADF covers scattering range between 1.1 and 4.4 Å À1 . Also, the analog HAADF signal is oversampled by 64 digital samples per pixel by the SavvyScan ADC for noise reduction, whereas the ARINA camera is counting electrons. In Fig. 4 we show the scan images acquired simultaneously for a specimen of replica grating with latex spheres from Electron Microscopy Sciences (EMS cat. #80055). The vDF image is shifted to the left by 8 pixels because of delay in the AWG Spectrum card with relation to its own frame signal. This has little implications for using the HAADF signal as guiding image. HAADF images are also useful for acquiring the relative alignment shifts of the tilt series. However, the HAADF images should be aligned with the vDF images before combining the detector inputs for extended diffraction information.

Validation and characterization
The 4D-STEM modality is demonstrated on aggregates of ferritin found on a purchased dry specimen (Ferritin on a grid, EMS cat #80042). The size of the aggregates is about 300 nm while the known diameter of a single ferritin molecule is 12 nm. In Fig. 5 we show vDF scan image and various CBED diffraction images averaged over squares of 5X5 scan pixels at the loca- tions indicated by the colored arrows, which correspond to the frame color of the CBED pattern. At some probe locations the Bragg reflections are dominant, exhibiting a twofold symmetry that has been reported previously in polymers [15]. The twofold diffraction pattern could arise from a structure with translational symmetry (2 R ) as well as other point groups [16]. The effective camera length was 1940 mm so the scattering vector magnitude at the Bragg disc locations is 0.16 Å À1 , which conforms to a lamellar structure with periodic interlayer spacing of 6.2 Å. The Bragg condition is met where the 30 Å wide beam is almost parallel to the scattering planes, thus the orientation of the discs reveal the normal direction to the lamellar structure. A possible explanation for the emergence of scattering planes is a periodic variation in the iron content within the ferritin molecules [17]. Using a much larger probe size (and camera length) would reveal the global crystalline order among ferritin molecules [18]; however, the current system is unique in mapping the variation in the internal structure within every pixel in the scan.
Crystalline grains diffract at specific tilt angles so it is interesting to rotate the specimen. Starting from version 2 of SavvyScan [14] the tilt series acquisition with ARINA detector is completely automated. The ferritin specimen was tilted between À 36⁰ and 36⁰ at steps of 2⁰ and scanned using the tilt series acquisition function in SerialEM [11]. A ''trial" scan preceded each ''record" scan to align the microscope position; whereas the SavvyScan system operated the electron camera only during ''record" scans based on the scan settings reported to the system by the serialEM plugin. SerialEM was able to track and correct the scanning location as for its usual application to tomography. The HDF5 files recorded by the DECTRIS server of 40 GB total size were processed according to virtual detector segments to generate MRC files. The tilt series MRC stacks were aligned according to a rigid body rotation model by ClusterAlign [19]. The region of interest was selected at the same location for all tilt views. The results are shown in Fig. 6 for the HAADF scans in comparison to the vDF and virtual bright field images at several tilt views. The dominant shining features may conform to crystalline grains of a few ferritin molecules, within which the lamellar structure happens to meet the Bragg reflection condition (i.e., beam almost parallel to the scattering planes). For those molecules, we were able to map the orientation of the scattering planes by the displayed color code.
In summary, the ongoing trend in electron microscopy to study of beam sensitive materials, including cryogenic biological specimens [4,20], largely motivates the development of dose-efficient methods for STEM. 4D-STEM is especially attractive for such because it can employ essentially every scattered electron for generation of useful contrast, but until recently the approach was severely limited by long image acquisition times. The system presented here addresses the significant technical problem of synchronizing the scan with the diffraction image acquisition at the relevant rates on the order of 100 kHz. Building on the previously-reported scan generator, which was designed for acquisition from a segmented diode STEM detector (Opal, El-Mul Technologies, Israel) and integrated with the microscope control platform SerialEM [2], the system provides a complete image acquisition solution for 4D STEM.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.