Detectors—The ongoing revolution in scanning transmission electron microscopy and why this important to material characterization

Detectors are revolutionizing possibilities in scanning transmission electron microscopy because of the advent of direct electron detectors that record at a high quantum efficiency and with a high frame rate. This allows the whole back focal plane to be captured for each pixel in a scan and the dataset to be processed to reveal whichever features are of interest. There are many possible uses for this advance of direct relevance to understanding the nano- and atomic-scale structure of materials and heterostructures. This article gives our perspective of the current state of the field and some of the directions where it is likely to go next. First, a wider overview of the recent work in this area is given before two specific examples of its application are given: one is imaging strain in thin films and the other one is imaging changes in periodicity along the beam direction as a result of the formation of an ordered structure in an epitaxial thin film. This is followed by an outlook that presents future possible directions in this rapidly expanding field.


INTRODUCTION
Transmission electron microscopy was founded based on the invention of the cylindrical magnetic lens, 1 and this was soon used to construct a transmission electron microscope as an analog of a compound light microscope; 2 pretty quickly, this surpassed the resolution available with light. 3 Nevertheless, it was soon shown that the resolution of such a microscope was unavoidably limited by intrinsic spherical and chromatic aberration. 4 While the path to the correction of such aberrations was already clear by the late 1940s, 5,6 the practical realization thereof had to wait until the 1990s and early 2000s. [7][8][9][10][11][12] In recent years, aberration correction has revolutionized both imaging and spectroscopy in the transmission electron microscope by allowing subangstrom resolution and therefore allowing routine direct imaging of the atomic lattice [13][14][15][16][17] in almost any crystalline material that is not excessively beam-sensitive (also indeed the atomic structure for very thin non-crystalline materials in certain cases [18][19][20]. Aberration correction also enables a large increase in the beam current of a focused probe, greatly improving analytical microscopy using electron energy loss spectroscopy [21][22][23] or energy dispersive x-ray analysis 24 (for instance, the following review summarizes some of the advances in the application to functional oxide materials 25 ). This has had a huge influence on the development of materials science and thereby products used in real-world application, as atomic-resolution characterization of materials and engineered nanostructures has become widely available worldwide. This has contributed to fields as diverse as catalysis, 26 precipitation hardening in metals, [27][28][29] and growth of high-k gate oxides for the latest generations of MOS-FETs, 30 which have direct relevance to widely used consumer products.
However, while there has been a step change in the routinely achievable image resolution of a factor of at least 3-4 in the past 20 years and an increase in the available current in a small probe of an order of magnitude (of great importance to spectroscopy in the STEM), prospects for major further improvements in the resolution in the immediate future are more modest. Any further increases in the numerical aperture of the objective lens will require the correction of chromatic aberration, which is still not easy, and improvements in correcting geometrical aberrations will only give limited increases in the resolution. This is especially true because Johnson noise in the conducting elements in the corrector is an additional limiting factor on corrector performance, and short of cooling the microscope to cryogenic temperatures, it is difficult to avoid this. 31 However, another major advance has been taking place in recent years, which is also revolutionizing electron microscopy. This is the ongoing revolution in detector technology, which now gives access to much more detailed information about the structure, chemical, and magnetic ordering in materials at the atomic scale, and this is the subject of the present article. Specifically, much of STEM imaging has concentrated on the spatial resolution to date while somewhat neglecting the angular resolution (or k-space resolution). However, with the advent of faster pixelated detector technologies, there is no longer any need for this to remain the case. The richness of the angular-resolved electron scattering information in the back focal plane of the sample can be used in the formation of STEM images, revealing a wide range of useful information about the sample. This can include magnetization, strain, crystallographic phase changes and orientation, charge transfer in covalent bonding, compositional information, point defect location, and three-dimensional crystal ordering. This has the potential to completely change material characterization in the next decade.
This article therefore provides a perspective of how we got to this exciting position, where we are currently in both technique development and applications and where the next advances are likely to be made in the coming few years. It commences with a brief review of detector technologies for electron microscopes, specifically focusing on recent developments in pixelated direct electron detectors. There is then a broader survey of the applications and advances arising from this across a wider range of types of electron microscope. This is then followed by a detailed illustration of two specific examples of the application of pixelated detectors in scanning transmission electron microscopy. Finally, an outlook is given about the importance and possible future directions of this rapidly developing field.

A SHORT REVIEW OF DETECTOR TECHNOLOGIES
A brief review is presented here of advances in detectors for electron microscopes for the benefit of providing a coherent narrative within this paper, but fuller and far more detailed reviews of the subject of detectors in electron microscopy have been published, 32,33 covering this topic in far more detail.
Prior to digital detectors, transmission electron microscopy mainly used blue sensitive photographic film to record images. This had many advantages, especially as suitable fine-grained film in large photographic plates had both high resolution and high dynamic range, and could be used to generate images containing a very high information content. Nevertheless, film was not totally linear in the conversion of the electron beam flux to optical intensity. 34 Additionally, typical microscopes in the 1980s and 1990s had between 30 and 50 plates in the camera limiting the number of images that could be taken in a session. More seriously, taking a photograph took several seconds including loading the plate into position, exposing it and then moving it into an exposed plate cannister. Thus, time resolution was very poor. Better time resolution could sometimes be achieved with video cameras, but for these dynamic range, sensitivity and resolution were poor.
Electronic recording of images came first to scanning electron microscopy and scanning transmission electron microscopy, because in these instruments, images are collected sequentially, and a single electronic signal can be attached to each pixel in a scan. In particular, scanning electron microscopy used secondary electrons collected with an Everhart-Thornley detector 35 and STEMs used annular dark field (ADF) detectors (usually a phosphor and a photomultiplier tube) for this purpose. However, some aspects of SEM and STEM operation required 2D images. For example, using the Ronchigram has long been recognized as a method for tuning a STEM probe, 36 but this requires a live image of the back focal plane of the probe. 10,11 Similarly, electron backscatter diffraction patterns in the SEM are two dimensional images and acquiring them for every point on a scan was only enabled once electronic recording of images became possible. 37 Thus, the introduction of charge-coupled-device (CCD) sensors into electron microscopes (from the early 1990s onwards) was significant, as it allowed electronic recording of images, and doing that repeatedly so that better time resolution was possible. However, standard CCDs are not able to record electrons directly and require the conversion of the fast electrons into light through a coupled phosphor which then generates stored charge in the CCD for later readout and amplification or conversion into a digital signal. This indirect process causes spreading of the beam in the phosphor and the coupling causes further optical blurring leading to a triggering several pixels for each electron, worsening the resolution. Even with improved coupling between the phosphor and CCD, this problem is not entirely absent. Additionally, the readout speed of standard CCDs is relatively slow and most commercially available sensors do not exceed 30 frames/s (at least for full frames). This is not suitable for high speed imaging or for use of the detector in any scanned beam technique. Finally, all CCDs produce dark noise which needs to be subtracted from the images, and readout and amplification also adds noise to the final signal.

THE INTRODUCTION OF DIRECT ELECTRON DETECTORS
It was realized, however, that it may be preferable to convert the electron beam directly into electronic signals, rather than using an indirect approach. First, a CCD has blurring from both the spreading of the electron beam in the scintillator and spreading of the photons in the optical coupling to the CCD, so a direct detector will clearly give a better modulation transfer function (MTF). 38 Second, direct conversion and detection of the energy from a high energy electron in a single pixel will result in a large signal that can be easily distinguished from low level thermal generation of electronhole pairs, giving a lower noise. A number of approaches for doing so were developed in the 2000s, mainly as spin-outs from particle physics, nuclear physics, or astrophysics, which needed to detect and PERSPECTIVE scitation.org/journal/apm track high energy particles using high speed electronic systems. A number of different approaches were pursued in this field, but all share the approach that an electron beam produces electron-hole pairs directly in a semiconductor and that these are then detected and converted to a signal. Two main variants were developed. In the first one, a thick absorber layer is placed in front of the detector, and the beam is completely absorbed in this and converted to electron-hole pairs, which are separated and read out. In the second, a thinner semiconductor chip is used in which only part of the energy from the electron is absorbed and converted into a charge pulse. The first has the advantage of being fully radiationhard and achieves a large charge pulse for each electron, but the pulse may spread laterally and the spatial resolution on the detector may be less than optimal. In the second one with a thinner sensitive region, the pulse will be weaker but there will be less beam spreading. There is, however, concern about the possibility of high energy electrons damaging the sensitive counting electronics behind each pixel.
As one example of an early investigation, initial experiments (pre 2011) on integrating a direct electron detector into an electron microscope at Glasgow used a Medipix2 detector mounted on a fixed housing at the base of the column. 39 This detector had a small number of pixels (256 × 256), but with the ability to record very short exposures (less than 1 μs). This was used to examine domain wall mobility in magnetic materials. 40 This detector, however, was restricted by a slow readout system, so although short exposure images could be recorded, the number of images per second was still limited. Other investigations were done before 2010 at Cambridge 41 and Oxford 42 on integrating direct electron detectors into electron microscopes and characterizing their performance.
Meanwhile, several commercial developments went into the production of large format thin pixel sensors specifically for imaging large areas at a high resolution for use in the then rapidly expanding field of cryo-electron microscopy. This is not the major focus of this Perspective article, but it suffices to note that several manufacturers produce high pixel count direct electron detectors with excellent modulation transfer function (MTF), which are optimized for large area imaging at a high voltage (>200 keV) with low electron doses, achieving a very good transfer of fine detail into the final image. The availability of such detectors was a critical part of the award of the Nobel Prize for the development of cryo-electron microscopy (https://www.nobelprize.org/prizes/chemistry/2017/ summary/).
Small direct electron detectors with lower pixel counts also have an important role to play particularly in scanned diffraction applications in which two key requirements must be met: (1) The detector must read out fast to reduce the deleterious effects of sample drift and stability and also to allow a reasonable area to be scanned at a reasonable resolution in a sensibly short time. Traditional STEM imaging tends to use pixel dwell times of tens of μs (i.e., 10 000 s −1 -100 000 s −1 ). The current typical performance for small detectors is of the order 1000 s −1 which, while not quite competitive with traditional STEM imaging for speed, is certainly much faster than both previous generations of imaging detectors and most analytical spectroscopies [e.g., electron energy loss spectroscopy (EELS) or energy dispersive x ray (EDX) mapping]. (2) The detector must be radiation hard so that it can cope with very bright diffracted beams without damage (either short or long-term).
There are a number of detectors available that possibly fill these requirements, and they each have specific strengths and weaknesses.
One design is essentially a modified CCD, which works without a scintillator and counts electrons individually. 43 An advantage of this design is an exceptionally high frame rate, 44 but a downside is that the well depth for each pixel is small and can only hold a few electrons, so high dynamic range imaging is not a strength of this detector. One application in which this detector excels is ptychography, [45][46][47] which is extremely dose-efficient and can be used for both atomic resolution 48,49 and lower resolution biological imaging. 50 A commercial version of this is sold by a microscope manufacturer (JEOL Ltd., Akishima, Japan).
An alternative design [the Electron Microscope Pixel Array Detector (EMPAD, Thermo Fisher Scientific, Hillsboro, OR, USA)] collects the absorbed high energy electrons and converts them into electron-hole pairs in a 500 μm Si absorber layer and reads them out as an integrated charge pulse collected over a period of time and integrates this to a total signal per pixel that should be linearly dependent on the number of electrons per pixel in the integration time. 51 At time of initial publication, that detector had 128 × 128 pixel 2 . This has been demonstrated to work in a range of applications. 48,52 This detector has the advantage that at very high beam currents or very bright areas of a diffraction pattern, it is not restricted by the counting speed. A commercial version of this detector is sold by one of the microscope manufacturers (Thermo Fisher, Inc., Hillsboro, OR).
Finally, the Medipix3 detector used in the experimental work described in this paper also collects electrons in a thick absorber layer, but in contrast to the above, each charge pulse above a userset threshold is counted individually, and the counting electronics can count up to 24 bits of data per pixel (16.7 M gray levels). Characterization of the MTF and detective quantum efficiency (DQE) of this detector with 60 kV and 80 kV electrons has been published, and the very high dynamic range has been demonstrated in practice using a diffraction pattern out to a high angle. 53 In standard format, this detector is a 256 × 256 pixel 2 array although it can be tiled in a 2 × 2 detector layout (as used in one of the studies shown in this work) if a larger mounting is used and space is available in the microscope. This detector is restricted in that it cannot count electrons arriving faster than 1 MHz/pixel. In this case, more than one electron will be counted simultaneously, electrons will be missed, and the linear relationship between beam current and counts will be lost. In a standard readout mode with the Merlin hardware (Quantum Detectors Ltd., Harwell, UK), this detector is limited to about 1200 patterns/s, although faster is possible in a 1-bit mode (which may be useful in some applications). 44 For many applications, this is acceptable, but there are others where a detector with a faster readout of images may be advantageous. A recent publication demonstrates a suite of software for control of this detector and live view of the resulting data. 54

APPLICATIONS OF PIXELATED DIRECT ELECTRON DETECTORS IN SCANNING BEAM TECHNIQUES IN ELECTRON MICROSCOPY
As described above, classic scanned probe techniques take a single electronic signal (or a small number of them simultaneously) attached to each pixel in a scan. For example, classic scanning transmission electron microscopy has generally used annular dark field (ADF) detectors 55 to produce images, which are mainly incoherent and dependent on the atomic number 56,57 [especially high angle ADF (HAADF) 58,59 ]. More recently, bright field (BF) 60,61 or annular bright field (ABF) imaging 62,63 have also been used, which have more coherent diffraction information in them (although low angle inelastic scattering also has an influence). Differential imaging using split detectors has also been used to determine magnetization, 64,65 but there is always a wealth of information in the back focal plane not recorded by such techniques, whether using a highly convergent beam and high spatial resolution or with a lower convergence angle and higher angular resolution at the expense of poorer spatial resolution. The advantage of recording a scanned dataset with a pixelated detector that reads out fast enough that drift does not distort the images is that a large dataset is produced containing all the detail already present in the classic STEM techniques, but which can also be examined for other features and processed in a range of different ways. There is an associated challenge with this in that the technique generates a large volume of data, and this brings practical challenges to data handling and live view of information. The section titled Software for data acquisition and processing in this paper will give a short overview of different software packages currently available with links to the URLs for the code and documentation, as well as references to relevant journal publications.
While this article mainly focuses on detectors in the scanning transmission electron microscope, it should be noted that direct detection of electrons is potentially also of use in scanning electron microscopes 66 to perform electron backscatter diffraction. 67 It has been shown that higher quality patterns were produced over wider angular ranges, 68 and this has been recently been used for orientation mapping beam sensitive materials. 69 In TEM or STEM, these 4D-STEM and scanned diffraction techniques have been well reviewed recently by Ophus et al. 70 A brief list of some of these techniques is given here, approximately in the order of increasing beam convergence angle.
First, the bright field disk when focused contains information that helps in the tuning of the microscope for optimal STEM imaging, the so-called Ronchigram. 10,11,71 It has recently been shown that a scanned dataset of Ronchigrams can be used for rapid aberration measurement, which may be used in the future in automated microscope tuning. 72 For differential phase contrast (DPC), a classic problem is that phase contrast within the bright field disk can reduce the ability to measure the disk shift by moving the apparent center of mass. When using this for magnetic imaging, this gives increased "speckle" in the images due to the grain structure in a nanocrystalline film. 73 When using this in an attempt to study electric fields, this can lead to entirely erroneous interpretation. 74 Recent studies using a direct detector to collect the diffraction patterns, followed by edge detection and cross correlation techniques to determine the disk shifts with sub-pixel accuracy, show improved sensitivity and reduced noise in differential phase contrast imaging. 73 Movement of the bright field disk or intensity within it can also be used to map the electric fields seen by the fast electron in its path through the sample. [75][76][77] This works by calculating the movement of the center of mass (or first moment) of the diffraction pattern. It should just be noted here that these are not the same electric fields as those seen by a valence or conduction electron in these materials, as the screening of the nucleus is much less for a fast electron. Hence, while this does reveal atomic structures in some detail, the relationship to actual materials properties requires more investigation. It is true that first moment determination may be quicker than the DPC method described above and may also be useful for rapid mapping of magnetic and electric fields in cases where atomic resolution data are not the dominating feature.
Scanned nanodiffraction has sometimes been used in studies of strain in materials, 78 although it has largely been superseded by scanned precession electron diffraction where the electron beam is tilted away from the optic axis and precessed during scanning to reduce the effects of tilt and dynamical diffraction on the resulting patterns. [79][80][81][82] Scanning precession electron diffraction can also be used for orientation mapping in crystalline materials [83][84][85][86] and was originally developed for this purpose. Recent work has demonstrated that using a direct electron detector for this improves the quality of the recorded patterns and thereby the reliability of indexing and the sensitivity of the detection of phase changes in nanoscale regions in a two-phase alloy. 87 Recent work has also demonstrated that the improved signal to noise ratio results in the ability to map subtle crystallographic features in both scanning electron diffraction 88,89 and scanning precession electron diffraction. 90 Ptychography, 91,92 as mentioned above, uses interference in the diffraction pattern to generate contrast based on phase differences measured from the overlapping diffraction disks in a scanned convergent beam electron diffraction dataset. The initial ideas for focused probe ptychography were demonstrated long before fast pixelated direct detectors were developed 93-95 but were then applied in theory and practice more recently once fast pixelated detection became a reality. 45,49,96,97 This can work at very low doses of very few electrons per pixel and the effect of dose on resolution has been studied along with extending the resolution using higher order reflections. 48 This has been applied to atomic resolution imaging of a wide range of specimens including metal nanoparticles, heavy metal-containing mixed oxides, 47 and carbon or light element based nanomaterials, 45,49 as well as lower resolution, low dose images of biological samples. 50 Low angle diffraction rings can be used to measure lattice parameter changes in different directions in a polycrystalline diffraction pattern, especially if the convergence angle is low enough to allow high resolution in k-space at the expense of some resolution in real space. This was used by Nord et al. 98 to study lattice parameter changes due to ion-beam induced disordering of nanopatterned areas in Fe-Al thin films. Additionally, this study showcased one of the key advantages of pixelated detection of diffraction patterns in STEM-the ability to synthesize multiple signals from the same dataset and thereby correlate different effects. In this case, the magnetization could be measured from the deflection of the bright field disk at the same time as the radius of the first order diffraction rings PERSPECTIVE scitation.org/journal/apm from the {111} planes of the face centered cubic structure. Thus, a direct correlation was possible between local crystal strain and magnetization leading to the discovery of strain-driven control of magnetization in thin ion-beam written nanowires. Another investigation performed with low angle diffraction using low convergence angles is fluctuation microscopy of amorphous structures. Fluctuation microscopy is based on the concept that glasses are ordered at sufficiently short ranges, so it is possible to use either dark field TEM or pixelated STEM to investigate the real space variation in the diffracted intensity at specific points in k-space and to determine the typical length over which order persists. The STEM implementation of this technique was initially introduced by Voyles and Muller. 99 As with any other STEM diffraction technique, doing this on a faster detector is a major advantage and has recently been used to study local ordering in silicide superconductor thin films 100 and ordering in mixed metal carbide films. 101 Integration of different angular ranges in the diffraction pattern is also possible with pixelated detection, allowing the user to examine the data after acquisition and adjust the angular ranges for virtual detectors to best extract the data required. Indeed, the possibility exists to match the details of the angular variation in scattering with simulations and thereby separate the effects of strain and composition in a more complete version of what was attempted by Grillo et al. 102 In that case, the variable angle was created by scanning the same area multiple times at different camera lengths using the same ADF detector. Similarly, Müller-Caspary et al. 103 used similar ideas to separate strain and composition in GaNxAs 1−x using an iris aperture above the HAADF detector to change the angular range sampled.
Some examples of variable angle annular detection using a fast pixelated detector are shown in Fig. 1. The microscope used was a JEOL ARM300F operated at 200 kV. The sample was a film of double perovskite La 2 MnNiO 6 grown on a SrTiO 3 substrate, and the area imaged also contained a NiO impurity inclusion within the film; the sample orientation is a ⟨100⟩ direction in SrTiO 3 . Figure 1(a) shows the averaged diffraction pattern from all pixels in the image. The bright ring at a high angle in the image corresponds to the first order Laue zone for a perovskite, which is at 113 mrad along this direction. This was used to calibrate the angular scale in the image. The data eventually cut off at high angle due to aberrations at high angles in the double-corrected JEOL ARM300F used for this work.

PERSPECTIVE scitation.org/journal/apm
Some pre-processing was performed to reduce the original 515 × 515 pixel 2 data to a binned 128 × 128 pixel 2 dataset since the analysis did not require high resolution in angular space, reducing the file size from 4.05 GB to 679 MB. The dataset was then processed in a similar way to that performed by Nord et al., 104 where the center was found for each diffraction pattern. A radial sum was then calculated about this center for each diffraction pattern to give a new smaller dataset where each pixel contains a 1D plot of intensity vs angle, rather than a full 2D diffraction pattern. This is much smaller and easier to work with (reducing the dataset to 41.2 MB). A plot of the radial integration of Fig. 1(a) is shown in Fig. 1(b) with rectangles marked representing the different angular ranges used for forming images; the cutoff at about 160 mrad is just visible at the extreme right. Figures 1(c)-1(f) then show ABF and low angle, medium angle, and high angle ADF (LAADF, MAADF, and HAADF, respectively) images calculated by integrating each of these regions for each pixel in the dataset. As can be seen, the ABF image of Fig. 1(c) resembles a classic ABF image, and the MAADF and HAADF images of Figs. 1(e) and 1(f) resemble classic ADF images that one would expect from a perovskite along ⟨100⟩. A key feature of the dataset is seen in the LAADF image of Fig. 1(d), where there is a bright rim just outside the NiO inclusion. Two ellipse segments are drawn in the same position in each image to make it clear which feature corresponds to this in each image. If this region is examined in detail in the other images, it is clear it corresponds to a drop-off in intensity in the ABF and HAADF regions while not changing the contrast of the perovskite much in the MAADF region. This is as expected for highly strained regions. Initially, this effect was demonstrated for an interface of crystalline and amorphous silicon by Yu et al. 105 and was then subsequently studied by Fitting et al. 106 and Grillo et al. 102 The parallel case to this has been simulated by Grillo and Rotunno 107 for inclusions straining the surrounding lattice (in that case quantum dots), bending lattice planes, deflecting more intensity from the bright field disk into the low angle diffraction, and reducing the amount of channelling, thus reducing the HAADF intensity. This effect was also observed in imaging of quantum dots by Trevisi et al. 108 The fact that this can now be done with the full angular resolution and at an atomic resolution means that these systems could be investigated in much more detail with careful matching of simulation and experiment to disentangle any compositional gradient effects from the strain effects. This analysis showed that a large amount of strain is present in the perovskite film for about 2-3 unit cells width outside the NiO inclusion, but very little strain is present within the NiO. This suggests that some deformation of the film happened as a result of the NiO formation possibly because of differential thermal expansion between the growth temperature and room temperature. The implication appears to be that the perovskite is far more deformable than the rock-salt-structured NiO. It would be similarly expected that strain at epitaxial interfaces with larger misfits or at internal crystal defects resulting from strain and/or chemical effects 109 may also be revealed in LAADF contrast, which would allow the localization of this contrast very straightforwardly, without needing atomic resolution measurements of atomic column displacements. 110 Another use for variable angle STEM imaging has been developed and demonstrated by Stemmer and co-workers. This uses the change in collection angle between two annular detectors to change depth of field and thus provide more precise location of point defects along a column of atoms in a crystal. In their work, this was used to locate single Gd dopant atoms and Sr vacancies in SrTiO 3 in three dimensions. 111,112 With access to the full angular range, rather than the limits of two ADF detectors, it would be possible to improve the precision of this analysis further, and this is promising as an interesting application for pixelated detectors.
Beyond simply integrating annular ranges in the bright or dark field regions of the diffraction pattern, it is also possible to focus on specific features at specific angles. In particular, Nord et al. 104 recently showed that it was possible to record the Laue zone rings [as already noted above in Fig. 1(a)] using a pixelated detector and then extract the intensity in the rings with subtraction of the general high angle scattering. This relates specifically to the periodicity of the sample along the beam direction and thus allows 3D crystallography to be studied from a single observation direction. More specifically, detailed instructions on how to perform this are provided at https://pixstem.org and a worked example is given in the study of Paterson et al. 54 Much like for the variable angle annular dark field imaging, the range of angles present in the two reciprocal space dimensions of the four-dimensional dataset allow far more information to be extracted than would be possible using standard STEM imaging alone. It should further be noted that this imaging method originated with a method performed with thin annular detectors 114 based on earlier theoretical ideas, [115][116][117] but no separation of the high-order Laue zone (HOLZ) intensity from the rest of the high angle scattering (which is typically much stronger) was then possible, and this approach then ceases to be quantitative. Figure 2 shows this method applied to a film of doubleperovskite La 2 CoMnO 6 grown on LSAT. The dataset was acquired on a JEOL ARM200F operated at 200 kV at a beam convergence angle of 29 mrad, using a scanning step size of 6 Å, using a MerlinEM (Quantum Detectors Ltd., Harwell, Oxfordshire, UK, https://quantumdetectors.com/n/products/merlinem) system with a single Medipix 3 chip to record the diffraction data controlled by the software developed by Nord et al. 118,119 This sample was prepared as part of a previous study on promoting double perovskite ordering in film growth in order to provide optimum magnetic properties and was characterized by atomic-resolution STEM, 113 but the STEM characterization available at the time did not include this technique. Figure 2(a) shows a HAADF image calculated from the dataset, and this is much as expected, showing similar intensity in both the film and substrate, as would be expected since both contain are rich in La, the highest atomic number element present. It should be noted that although this appears to be an atomic resolution image, this is not true. The scan step size was 6 Å, larger than the atomic spacings in this projection (La-La column spacings would be about 3.9 Å and 2.76 Å in two orthogonal directions). However, an image showing features of the lattice is still produced in such cases by a Moiré effect, i.e., an interference between the sampling frequency and the actual periodicity, which has sometimes been used deliberately to perform atomic-resolution STEM over larger areas of periodic structures 120 (e.g., for strain measurements 121 or lowering dose per unit area 122 ). A rough calculation is that the distance between atoms along the xaxis of the plot should be about 4.8 Å, but the observed distance is about 4 pixels or 24 Å, where a Moiré calculation predicts this to be 23.4 Å, supporting this interpretation. Thus, the areas that are bright in the HAADF image are clearly Moiré images of lines of heavy A-site atoms in the perovskite. The film and the substrate display quite different diffraction patterns, and patterns averaged from many pixels over larger areas of the film and substrate are shown in Figs. 2(b) and 2(c). The pattern from the LSAT substrate just shows a single HOLZ ring at the angle expected for a disordered perovskite (about 96 mrad). However, the pattern from the film shows an additional HOLZ ring at a lower angle demonstrating a longer periodicity of the structure along the beam direction, presumably as a consequence of the double perovskite ordering. The strength of the two HOLZ rings is then mapped using the modeling method described above, where the high angle scattering is reduced to a 1D plot of intensity vs scattering angle, and then modeled as a decaying power-law background plus a Gaussian peak for the HOLZ ring. The area of that peak is plotted for the inner HOLZ ring in Fig. 2(d) and shows a strong peak in the film, but nothing in the substrate. However, it is clear that the HOLZ ring does not become immediately strong at the interface between the film and the substrate, but that the maximum strength occurs at several nanometers above the interface. This is plotted for the box shown in Fig. 1(d) in the direction of the arrow in Fig. 1(e), which shows that although some intensity in the Laue zone appears immediately, it only reaches a maximum after 10 nm. This is perhaps not surprising, as in the original study, it was found that for films grown on both (111) LSAT and SrTiO 3 , the octahedral tilting and the modulation of A-site spacings (thus B-site sizes) took a few nm to develop. 113 For comparison, the strength of the Asite modulation from the original study is plotted on Fig. 2(e) as an orange trace-these data were measured from atomic resolution images and only covered a few nm. It should also be noted that the Laue zone intensity for LaFeO 3 was also suppressed near interfaces in a SrTiO 3 -LaAlO 3 -(La, Sr)MnO 3 trilayer structure. 104 This tells us something fundamental and important about the epitaxial growth of such oxides as thin films on substrates: the octahedral tilting, the modulation of A-site spacings in the out-of-plane direction, and the modulation of A-site positions along the beam direction are all connected phenomena and part of the same structural transformation from an untilted primitive cubic structure to a tilted perovskite such as the double perovskite. It also means that measuring the inner Laue zone intensity is a very powerful way to map the development of this ordering over areas of tens of nanometers in size in a way that would never be possible by measuring octahedral tilting directly from ABF images 113,123 or positional modulation of A-site planes in HAADF images. 113

PERSPECTIVE scitation.org/journal/apm
The radius of this inner Laue zone was remarkably constant at about 68 mrad (67.8 ± 0.4 mrad). A radius of 67.75 mrad would be expected for a material with the primitive lattice constant of LSAT, but with a doubling of the unit cell along this direction. Slight changes are seen in some areas where the HOLZ ring is weaker, close to the interface, and in the central part, it is possible that this is affected by strain at the interface, the possible presence of a domain boundary in the LCMO in the central part of the image, or the CoO second phase previously observed. 113 In contrast to these effects on the inner HOLZ ring, Fig. 1(g) shows that the outer HOLZ ring is much stronger in the LSAT substrate than in the film, although, of course, it is still present in the film as the fundamental perovskite periodicity remains in the presence of the cell doubling. As for the inner HOLZ ring, the strength of the outer HOLZ ring is suppressed close to the interface, and this may be because the HOLZ ring intensity is formed by dynamical diffraction not just from single atom columns but by coherent diffraction from electron waves traveling along this and surrounding columns of atoms. The reason the outer HOLZ ring gets weaker in the substrate is analogous to the effect shown in Fig. 1 with LAADF intensity strengthening at the expense of ABF and HAADF intensity at the interface. In both cases, the total number of electrons is conserved, so if extra electrons go to one place in the diffracted plane, fewer go somewhere else. Thus, forming an extra HOLZ ring at a lower angle due to ordering takes some of the electrons that would otherwise be coherently scattered into the HOLZ ring at a larger angle, resulting in a weakening of this ring. It should be noted that the HAADF image was slightly brighter at the interface. This may also arise from a related effect: where there is less scattering into HOLZ rings, then there will be more into general thermal diffuse scattering, which forms the main basis for high angle incoherent scattering that forms the bulk of the intensity detected by a HAADF detector.
The radius of this outer HOLZ ring is also constant at 94.1 ± 0.6 mrad in the LCMO film (the same area as used for the calculation of the radius of the inner HOLZ ring above) and 94.0 ± 0.3 mrad in the LSAT. It increases slightly just above the interface to above 95 mrad. First, it should be noted that the expected radius for the LSAT would be 95.8 mrad, so the measured values for the film and substrate are slightly below expectations. However, it should also be noted that a non-linear relationship between pixels and angle is not surprising at higher angles, as radial distortion can be difficult to avoid at larger angles in electron lenses (as shown in the previous work on electron microscope optics for electron energy loss spectroscopy 124,125 ). A detailed characterization of the radial distortion in the lens program used in this study is beyond the scope of this work. However, it is interesting that the HOLZ ring radius rises in a small but significant manner just above the interface, indicating a slight reduction in the lattice parameter along the beam direction in this area. This can be more precisely estimated using the standard formulae for HOLZ ring radii, where the radius depends on the square root of the lattice period along the beam direction. 54,126 This would suggest a 2% reduction in the lattice parameter along this direction for about the first 1 nm or 2 nm of the film. In the original work on these films, 113 it was found that the first few atomic layers above the interface were richer in Mn than the rest of the film. Autret et al. 127 also found that the lattice parameter of La(Co, Mn)O 3 mixed oxides went down with increasing Mn content, which is not surprising as Mn 4+ is nominally a smaller ion than Co 2+ . Hence, a subtle relaxation of the film just above the interface to the substrate because it is Co-poor at the nucleation of the film appears in the HOLZ STEM data in a robust and reliable way. As with the development of the ordering of the period doubling, where it was easier to do HOLZ STEM than atomic resolution STEM and subsequent quantification, it is also easier in this case than doing the careful EELS mapping needed to detect the composition change or would at least form a valuable counterpart to it.
These two examples demonstrate a key strength of performing STEM imaging with pixelated detectors: the angular resolution and detail in the diffraction plane allows easy access to information that was either difficult or impossible to acquire with older detectors that simply integrated large areas of the back focal plane to form either sum or differential images. Moreover, the information gained thereby is directly useful in understanding localized strain concentrations, locations of dopant atoms and vacancies, changes in crystal periodicity and in the strengths of atomic movements that lead to these changes, and changes in the lattice parameter along the beam direction because of changes in chemistry. It is therefore anticipated that increased use of such techniques will become a key part of materials characterization in the electron microscope in the next decade.

Software for data acquisition and processing
One thing that is implicit in all the work on STEM imaging with pixelated detectors is that it works by producing a diffraction point for every point on the scan, and by any measure, this produces a relatively large volume of data. A scan of just 128 × 128 pixel 2 to a 256 × 256 pixel 2 detector recording 2 bits of data (i.e., more than 256 gray levels) per pixel requires 2 GB of storage, which is already significant. Larger area scans quickly grow into much bigger data files. It is therefore easy to acquire hundreds of GB of data in a single day. While this is not that large compared to some major international facilities such as particle physics colliders, it is much larger than that electron microscopists have been used to and moving such volumes of data with standard computer networking is slow and inefficient. Hence, simply acquiring, transferring, and storing data is a challenge that has to be explicitly considered. Consequently, significant attention has been directed to issues with data handling in acquisition of data. An additional challenge is that, unlike in conventional STEM, there is not typically a live view of an image, especially as typically all computer resources are dedicated to fast and seamless acquisition. Thus, Nord et al. 118,119 also considered the issue of simple and fast processing to give live or near live views of the data prior to later full processing. Nevertheless, this is not just a software challenge, and performing this kind of work in an optimal way will require fast network connections and access to a suitable processing workstation for fast, near-live imaging of the results.
Once the data have been acquired, transferred, and stored appropriately, then processing the data from a four-dimensional dataset (basically, x, y, kx, and ky) into useful information is a significant challenge. A number of packages have been produced for this purpose, and an incomplete list of some current packages is provided below. HyperSpy 128 (https://hyperspy.org) is a PERSPECTIVE scitation.org/journal/apm commonly used (also longstanding) open source python library for handling hyperspectral data and does include routines for handling data with one or two signal dimensions. While much of HyperSpy is concerned with spectroscopic data, its routines are also useful for handling pixelated STEM data, especially after reduction to a one-dimensional profile of signal vs angle. One of the big advantages with HyperSpy is "lazy loading," meaning that the processing can be done without loading the full dataset into memory. Rather, using this option, the data are processed in "chunks," meaning that very big datasets can be processed without manually having to split the data into different smaller files, which is advantageous for performing analysis on regular personal computers with limited RAM (although the advantages of dedicated computing facilities for the more computationally intensive operations are undoubted). Pycroscopy (https://pycroscopy.github.io/pycroscopy/about.html) is also a python library of slightly longer standing for processing of multidimensional imaging datasets from Oak Ridge National Laboratory in the USA. In the case of this package, pixelated STEM is just one 129,130 of the several applications catered for by the package, and this also has plenty of support for data from other techniques, including scanned probe microscopies 131 (such as piezo force microscopy). pixStem (https://pixstem.org) is a set of tools (again a python library) from Magnus Nord specifically for analyzing some aspects of pixelated STEM data, including the code used for the HOLZ STEM shown in Fig. 2 above. fpd (https://fpdpy.gitlab.io/fpd/index.html) is a package of python routines for storage and handling of pixelated data from Gary Paterson. The use of these two packages is described in a recent paper. 54 Another set of routines has been described by Savitzky et al., 132 which mainly arises from the work at the National Center for Electron Microscopy at the Lawrence Berkeley National Laboratory, again in the form of an open source python library (https://github.com/py4dstem/py4DSTEM). A group at Forschungszentrum Jülich in Germany has been developing LiberTEM (https://libertem.github.io/LiberTEM/index.html), which is designed for efficient processing of larger pixelated STEM datasets using GPUs and more powerful computer architectures with parallel processing. pyXem (https://pyxem.github.io/pyxemwebsite/) is a python library specifically for crystallographic studies in electron microscopy, especially with scanning (precession) electron diffraction data, from a group at the University of Cambridge 133 as used in some recent studies, 88,89,134,135 although it is merging with some other libraries (such as pixStem mentioned above) to enable a wider range of possibilities and reduce duplication of some commonly used operations. ptychoSTEM (https://gitlab.com/ptychoSTEM/ptychoSTEM) is a Matlab package for ptychographic STEM imaging used in several recent papers by the Oxford group. 44,49 The Muller Group at Cornell has produced a number of packages for specific applications, including strain mapping with nanobeam diffraction 136 and ptychography, 52 which can be found at https://github.com/muller-group-cornell.
In the longer term, it is likely that some of these projects may merge or become specialized for specific applications. It is noteworthy that many of these projects all use similar file structures, which are variants of the Open Source hdf5 file structure (https://www.hdfgroup.org/solutions/hdf5/), and some newer OEM software also uses this format for data storage (such as the TopSpin software for precession electron diffraction, https://nanomegas.com/all-in-one-topspin-platform-for-precessionadvanced-analytical/). It is likely that manufacturers of detectors, electron microscopes, and add-ons such as spectrometers will have to recognize that users will use custom routines for processing data and will need to export hdf5 files for analysis and that this will become more routine. It is also likely that as electron microscope manufacturers sell more microscopes with an integrated pixelated detector, they will produce OEM software for performing some standard operations on the data, including live viewing of simple operations. It is recognized that many of the software packages mentioned above require some proficiency in python or Matlab to use them, and this may be a barrier to wider adoption. Thus, there is an opportunity for manufacturers to build software with GUI operation, which makes pixelated STEM more accessible to a wider cross section of electron microscope users.

Acquisition rate and hardware limitations
For current detectors, there is an issue about the speed of data acquisition, which restricts some current applications. With the detectors used in this work, standard imaging is restricted to about 1 kHz frame rates (1 ms frame times), although in some cases, it is possible to acquire significantly faster by reducing the number of pixels or dynamic range (see below). As long as the main focus is on acquiring high quality diffracted data, not absolute length measurements in the real space dimensions, this is fine. However, as has been shown in the previous work on absolute position measurement of atomic columns and in real-space crystallography in the sample plane, there are several restrictions due to long term drift of the sample, electromagnetic disturbances of the scan apparatus from AC pickup and other time-varying magnetic field sources, and shorter period distortions (perhaps from electronic noise in the scan control electronics). This has been corrected in earlier quantitative STEM work by acquiring multiple frames of the same area with a very short acquisition time per pixel (often 5 μs-10 μs), cross-correlating them to remove just the drift distortion and summing them, which tends to average out the short period distortions from EM pickup or electronic noise. 61,137,138 Later work has then gone further using non-rigid registration to better correct the short duration distortions [139][140][141][142] and rotating the scan several times during acquisition in order to remove any effects of affine distortions in the scanning system. 143,144 However, at acquisition times greater than ∼100 μs/pixel, this can still be challenging, although some useful work has been done in the area, principally for EELS and EDX spectrum imaging (which tend to acquire at tens of ms per pixel or slower). 145,146 This is because drift, low frequency electrical interference, and stochastic events can all interfere. Drift may smear individual pixels over such time frames and nothing can be done to recover that information within the pixels. Low frequency interference may modulate the atom positions over a length scale of many atom columns (as shown in Fig. 3 below). Stochastic events (such as mains spikes or mechanical disturbances) can affect one local region of an image (although this could probably be removed with multiple frame acquisition). Moreover, although good enough correction can be achieved in an atomic resolution spectroscopic image to allow the recovery of atomic resolution spectral information, it is quite a different challenge to actually recover atomic resolution positional information with picometer precision (as has routinely been possible with the alignment of atomic resolution HAADF images over the past decade 138 ). Indeed, the dataset shown in Fig. 1(e) was analyzed using quantitative atomic position measurement with the Atomap package, 110 and the results are shown in Fig. 3. There is a significant oscillation over a wavelength of many pixels in the atom position in the x direction of the scan. The actual oscillation is only of an amplitude of about ±0.25 pixels, but this is already enough to make any reliable determination of a difference in lattice parameter between the film and substrate difficult (also a difference is expected and is actually seen in scanning precession diffraction data). Thus, while there are many strengths to pixelated STEM with the current generation of fast pixelated detectors, accurate 2D atom position measurement across a larger image field is problematic without increasing acquisition rates and/or tackling any longer period instabilities in the microscope system. Going beyond this limitation is a significant technical challenge and will require further developments in both detector and readout hardware and software. There are applications in pixelated STEM where a relatively small number of pixels give sufficient information about the diffraction pattern for good results, for instance, in ptychography. 96 In such cases, if a detector could be produced with a smaller number of pixels, then reading out at a higher frame rate would be possible. A similar strategy was used in some recent work on ptychography with the JEOL 4DCanvas detector, where binning by a factor of 4 in one direction allowed frame acquisition at 4 kHz and 7.5 kHz rates, and the data were still suitable for ptychography. 147 Another strategy using detectors such as the Medipix with a counting mode is to reduce the counting bit depth-with the Merlin readout system, readout times are 1.64 ms at 24 bits, 822 μs at 12 bits (standard mode), 412 μs at 6 bits, and 70.8 μs at 1 bit (i.e., binary count or no count mode). This was recently used to perform ptychography at 12.5 kHz frame rates. 44 However, it should also be recognized that the current generation of pixelated detectors, whether those that work in counting mode or those that integrate total charge, arose from developments in other fields, especially in particle physics where active pixel sensors have been used for high frame rate counting of particles arising from collisions for a longer time. With the development of particle accelerators to higher luminosity, there is already a drive to faster counting with such detectors. Thus, when the current generation of particle physics and nuclear physics detectors becomes the next generation of x-ray and electron imaging detectors, one can already expect an increase in frame rates. In this vein, some recent work on delay line detectors has revealed a time resolution for individual events of 150 ps, but the detector could only cope with relatively low total beam currents. 78 This technology might be one way forward for faster counting. In a different vein, a prototype CMOS active pixel sensor detector and readout system has been reported, which can deliver full frames at 87 kHz. 148 In the light of the observations about handling and transfer of large volumes of data mentioned above, this detector is explicitly provided with fast data interconnects to a supercomputer to allow writing of the raw data to useable files and to allow simple live processing. Thus, the field is an active one and developing rapidly, and having observed the rapid development of detector technology over the past 20 years, it is safe to assume that significant advances in framerate will be seen in the next 10 years.
However, there is a fundamental trade-off to be made between speed of acquisition and getting sufficient signal for a specific analysis. Some analyses using the bright field disk (e.g., ptychography, first moment analysis, and differential phase contrast) need few electrons in order to work, and thus, speeding up frame rates may be advantageous. On the other hand, some features at higher scattering angles are intrinsically weak, such as the HOLZ rings, and going faster would mean fewer electrons per frame, and thus, the HOLZ ring would no longer be reliably located. Additionally, such experiments require the center of the bright field disk to not be saturated so that a pattern center can be reliably determined for the azimuthal integration. Consequently, purely increasing the beam current and the frame rate is not enough, and the count rate needs to be high enough to cope with any increase in beam current.
Another point that may come into play is the material used in detectors. In most of the current generation of pixelated detectors, the absorber layer for converting the high energy electron's kinetic energy into electron-hole pairs is silicon. Obviously, this has the advantage that it is relatively inexpensive and the expertise of the semiconductor industry with Si is such that large crystals can be grown with miniscule defect density, meaning that there will be few areas on the detector where a defect causes anomalous behavior (e.g., a defect that causes increased recombination of electron-hole pairs and reduced signal). However, the low atomic number means that the electron beam penetration is large and with significant lateral spread, meaning that MTF is worse at higher energies (e.g., 200 kV) and a thicker detector would be needed for energies higher than 200 kV. This could, in principle, be improved using alternative semiconductors for the absorber made from higher Z elements (e.g., GaAs). 149 Early results show that Si absorbers are better for thick hybrid detectors for use at 60 kV or 80 kV, but for higher accelerating voltages of 120 kV or 200 kV, both the MTF and DQE are better for a GaAs absorber on such a detector. However, problems with a higher defect density in GaAs means that pixel response is less uniform. It is clear that while this is a promising research direction,

PERSPECTIVE
scitation.org/journal/apm more work is needed before a detector with a higher Z absorber than Si can be recommended for pixelated STEM at higher accelerating voltages (>100 kV).

Possibilities for spectroscopy
This review has not focused on electron energy loss spectroscopy (EELS), but one of the big limitations for this application, as for diffraction, is the quality of the data coming from the detectors at the end of the spectrometer. (It should be noted that very fast readouts have been achieved in spectrometers with CCD cameras using a stripe acquisition mode and shuffling data across the CCD while the next spectrum is being acquired to allow up to 1000 spectra/s to be recorded 150 -this operation mode would not work for 2D imaging on a CCD, only for 1D spectra.) No matter how good the optics is in the spectrometer and in the microscope coupling to the spectrometer, 124,125 if there is excess electronic noise arising from the detector or readout, this will limit the performance, especially for weaker signals at the high energy end of the spectrum. This may then lead to longer dwell times per pixel to get sufficient signal to noise to obtain a signal that is reliably separable from the background. This may subsequently place limits on how much information may be acquired from beam-sensitive materials by requiring a higher beam dose to get a detectable signal than would be the case if all electrons were detected without additional noise. Thus, Hart et al. installed the first EEL spectrometer with a direct electron detector as the sensor in 2017. 151 However, the sensor used in that case is better optimized for imaging than for high frame rate imaging or recording bright areas with high electron arrival rates (such as a zero-loss peak in a spectrum). Nevertheless, this spectrometerdetector combination is clearly promising for some applications, and a detailed characterization of the latest version of that Gatan direct detector spectrometer has been published by Cheng et al. 152 The use of other detector architectures better suited to high electron arrival rates and fast readout has been picked up by other manufacturers, and the authors are aware of two manufacturers with prototype systems and orders for delivery of new systems to electron microscope laboratories. One of these has used a Medipix-3 chip inside a spectrometer, and some initial results have been published, 153 although no detailed characterization of detector performance in this application is available at this time. That work was also just performed on a single chip of 256 × 256 pixel 2 , although a 1024 × 256 pixel 2 detector is possible by tiling four Medipix3 chips together in a line, and initial experiments with this configuration have been performed (M. Tencé, private communication). A similar setup has been experimented with by NION, Inc. working with DECTRIS Ltd. on a 1024 × 512 detector made from tiling 4 × 2 detectors, each 256 × 256 in size, and characterization results for the detector and application examples for EELS have been reported. 154 There are issues here common to the future development of detectors for STEM imaging. In principle, it would be possible to develop optimized detectors for various different applications. For example, for EELS, a stripe format (high aspect ratio) detector with no junctions would be ideal, with 1000 pixels-2000 pixels in the dispersion direction and a smaller number in the perpendicular direction (perhaps 100), to give enough resolution for tuning aberrations in the spectrometer, to ensure proper focus and spectrometer setup, and to allow spreading of the spectrum across the lateral direction to allow higher beam currents.

CONCLUSIONS
It has been shown that scanning transmission electron microscopy is rapidly moving from an era where there was limited information in the back focal plane to one in which this information is recorded in detail on a fast pixelated detector and used to provide additional and often previously inaccessible information about the sample at a high spatial resolution. A range of different current and developing uses for this advance are surveyed before two specific examples are given from work carried out by the authors, which give some perspectives on the versatility and utility of these techniques in understanding the structure of materials at the atomic scale. First, the possibilities for angular-resolved bright and dark field imaging are investigated using a dataset recorded at the atomic resolution along a ⟨100⟩ axis of an epitaxial perovskite film grown on a perovskite substrate. This shows the presence of strain just outside a second phase inclusion in the film, clearly highlighted in a way that can be investigated with variable angle annular dark field imaging. Second, it is demonstrated how the modulation of La-atom positions along the beam direction can be directly mapped in a thin epitaxial perovskite film using an additional HOLZ ring, which appears at about 68 mrad as a result of this modulation. Moreover, an unexpected extra signal was found, suggesting a slight reduction in lattice parameter in the first 1 nm-2 nm of the film, which correlated with previous observations of excess Mn forming at the base of the film. This illustrates the power of pixelated detection in STEM in that a full set of data is recorded, which can then be processed and may well reveal other features that were previously unanticipated. Finally, an outlook is given for the future of this field, including developments in software, hardware, and transferring these advances to electron energy loss spectroscopy.