A nonlinear filtering algorithm for denoising HR(S)TEM micrographs

doi:10.1016/j.ultramic.2014.11.012

Ultramicroscopy

Volume 151, April 2015, Pages 62-67

https://doi.org/10.1016/j.ultramic.2014.11.012 Get rights and content

Highlights

•
A nonlinear filtering algorithm for denoising HR(S)TEM images is developed.
•
It can simultaneously handle both periodic and non-periodic features properly.
•
It is particularly suitable for quantitative electron microscopy.
•
It is of great interest for beam sensitive samples, in situ analyses, and atomic resolution EFTEM.

Abstract

Noise reduction of micrographs is often an essential task in high resolution (scanning) transmission electron microscopy (HR(S)TEM) either for a higher visual quality or for a more accurate quantification. Since HR(S)TEM studies are often aimed at resolving periodic atomistic columns and their non-periodic deviation at defects, it is important to develop a noise reduction algorithm that can simultaneously handle both periodic and non-periodic features properly. In this work, a nonlinear filtering algorithm is developed based on widely used techniques of low-pass filter and Wiener filter, which can efficiently reduce noise without noticeable artifacts even in HR(S)TEM micrographs with contrast of variation of background and defects. The developed nonlinear filtering algorithm is particularly suitable for quantitative electron microscopy, and is also of great interest for beam sensitive samples, in situ analyses, and atomic resolution EFTEM.

Introduction

Aberration-corrected high resolution (scanning) transmission electron microscopy (HR(S)TEM) enables quantitatively imaging atomic structures of condensed matters at sub-angstrom resolution [1], [2]. Nowadays, electron microscopes are widely equipped with CCD cameras. HR(S)TEM micrographs recorded from a CCD camera tend to be degraded by noise. A low signal-to-noise ratio (SNR) often makes the accurate quantification difficult. The SNR can be improved by either increasing the signal intensity or decreasing the noise level. When it is possible, the increase of signal intensity is of higher priority than the decrease of noise level in achieving high SNR. However, for the majority of beam sensitive samples or in situ analyses, the increase of signal intensity may not be practical. Under these circumstances, any improvement of the SNR through noise reduction by filtering is incredibly valuable.

There are several important sources of noise in a micrograph [3]: (i) quantum noise (shot-noise) of electron beam; (ii) dark current noise from thermally generated electrons; (iii) the so-called Fano noise from electron–photon and photon–electron conversions; (iv) read-out noise from electronic devices to read the image from a CCD. The Poisson statistics would apply for the first three sources of noise, whereas Gaussian for the read-out noise. The dark current noise and the read-out noise are independent of beam intensity. The Fano noise is empirically described as a function of photon energy, while the shot noise is proportional to the square root of the number of recorded electrons per pixel.

At low to moderate electron dose, the shot-noise tends to dominate the noise in electron micrographs because of its dependence on the beam intensity. For more than 10 electrons detected, the Poisson distributed shot-noise appears to approach a Gaussian distribution with its standard deviation proportional to the square root of the number of detected electrons in each pixel. Therefore it usually is valid to assume an experimental electron micrograph ( $I_{\exp}$ ) as a certain theoretically predicted image ( $I_{th}$ ) from a specific model plus uncorrelated additive Poisson or Gaussian ( $I_{noise}$ , $σ^{2} = I_{th}$ ) noise for simplification, so that $I_{\exp} = I_{th} + I_{noise}$ Noise reduction can be considered as an inverse process to obtain an estimated theoretical micrograph ( $I_{est}$ ) by applying a filter to the $I_{\exp}$ : $I_{est} = I_{\exp} \otimes F$ The noise reduction algorithms in general can be categorized into spatial and temporal filtering [4]. Frequently used spatial filters for denoising HR(S)TEM images include Bragg filter [5], Wiener filter [6], [7], [8], [9], and Gaussian filter [10], [11]. Whereas the simplest method of temporal filtering is frame averaging [4]. Registration of frames to one another is often essential before frame averaging to account for drift of the sample between frames. Both rigid [12] and non-rigid [13], [14] registration methods have been reported for frame averaging to obtain high SNR images.

Because HR(S)TEM studies are often aimed at resolving periodic atomistic columns and their non-periodic deviation at defect areas, it is important to develop a noise reduction algorithm that can simultaneously handle both periodic and non-periodic features properly. It is possible, often justified, to find out how complete the noise is removed and how well the periodic and non-periodic atomistic information is preserved by inspecting the difference ( $I_{diff}$ ) between the recorded $I_{\exp}$ and the denoised $I_{est}$ images: $I_{diff} = I_{\exp} - I_{est}$ The $I_{diff}$ actually is estimated noise. In this work, simulated instead of experimental images were used as the first testing ground in order to justify the performance of filters by exactly knowing the true signal, $I_{th}$ , so that an error image ( $I_{err}$ ) can be obtained: $I_{err} = I_{th} - I_{est}$

Section snippets

Tests of Gaussian and Wiener filters

A super cell of MgO with terrace and an edge dislocation with Burgers vector of a/2 $〈 1 \bar{1} 0 〉$ was constructed (Fig. 1a). A HRTEM image of the super cell with size of 512 ×512 pixels (0.01 nm/pixel) was simulated using the optimized FEI Titan 80–300 parameters at 300 kV under a negative spherical aberration (C_s) imaging (NCSI) condition (Fig. 1b), which is considered to be $I_{th}$ . The maximum intensity of the noise-free image was normalized to 100. Poisson noise was included so that at each pixel the

Nonlinear filtering algorithm

Reduction of peak attenuation of 1D spectra has been reported by adding low-pass filtered residuals to the original low-pass filter output [17]. The nonlinear filtering algorithm described in this paper is by adding residuals of Wiener filtering ( $F_{w}$ ) to the output of low-pass filtering ( $F_{lp}$ ), or vice versa through an iterative process.

Fig. 3 shows the flowchart of the nonlinear filtering algorithm. At each iteration, the output is used as input for the next iteration. The mathematic form of the

Simulated images

Tests were made to investigate the performance of the nonlinear filter in noise reduction and quantification of the simulated HR(S)TEM micrographs. Fig. 4a shows the nonlinear filtered image of the testing HRTEM image of the MgO super cell (Fig. 1c). No noticeable peak attenuation and artifacts are seen in the error image (Fig. 4b). For comparison, Fig. 4c shows the details at the dislocation core from the Poisson noise included (S+N), noise-free (S), nonlinear (NL), Wiener (W), and Gaussian

Conclusions

A nonlinear filtering algorithm has been developed by iteratively adding Wiener filtered residuals to the low-pass filtered output or vice versa. The developed nonlinear algorithm overcomes peak intensity attenuation and artifacts of low-pass and Wiener filters so that it can simultaneously handle both periodic and non-periodic features properly. Moreover, the present results have shown that both peak position and intensity can be more accurately determined by the nonlinear filtering, which

Acknowledgments

The author greatly appreciates C.L. Jia for fruitful discussion and L. Houben for help in the EFTEM imaging with the PICO microscope. The author also thanks the reviewers for their extremely valuable comments and suggestions. Financial support from the Deutsche Forschungsgemeinschaft (SFB 917) is kindly acknowledged.

References (21)

A. De Jong et al.
Image processing of HRTEM images with non-periodic features
Ultramicroscopy
(1989)
L. Marks
Wiener-filter enhancement of noisy HREM images
Ultramicroscopy
(1996)
F. Lin et al.
An improved wiener deconvolution filter for high-resolution electron microscopy images
Micron
(2013)
O.L. Krivanek et al.
Scanning transmission electron microscopyAlbert crewe's vision and beyond
Ultramicroscopy
(2012)
K. Kimoto et al.
Local crystal structure analysis with several picometer precision using scanning transmission electron microscopy
Ultramicroscopy
(2010)
B. Berkels et al.
Optimized imaging using non-rigid registration
Ultramicroscopy
(2014)
P.L. Galindo et al.
The peak pairs algorithm for strain mapping from HRTEM images
Ultramicroscopy
(2007)
B. Forbes et al.
Elemental mapping in achromatic atomic-resolution energy-filtered transmission electron microscopy
Ultramicroscopy
(2014)
C.L. Jia et al.
Atomic-resolution measurement of oxygen concentration in oxide materials
Science
(2004)
P.E. Batson et al.
Sub-angstrom resolution using aberration corrected electron optics
Nature
(2002)

There are more references available in the full text version of this article.

Cited by (52)

Ru/MgO catalyst with dual Ru structure sites for efficient CO production from CO<inf>2</inf> hydrogenation
2024, Chemical Engineering Journal
The development and comprehension of supported metal catalysts for CO₂ hydrogenation is of paramount importance in mitigating the net CO₂ emissions. Supported Ru catalysts have been widely recognized in facilitating CO₂ methanation, on which recent findings suggest that the CO₂ hydrogenation process can be manipulated to favor the reverse water–gas shift (RWGS) pathway by precisely adjusting the size of Ru particles. However, the size-dependent impact of Ru remains a topic of lively debate. In this work, Ru/MgO catalysts with Ru in the form of single atoms (Ru₁) and few-atom cluster (Ru_FAC) structures were prepared for CO₂ hydrogenation. The 1.0Ru/MgO catalyst (with 1 wt.% of Ru), featuring a mixture of Ru₁ and Ru_FAC with a size of 0.6–1.0 nm, showed the highest CO yield (38% at 500 °C) with balanced CO₂ conversion and CO selectivity. Transient CO₂ hydrogenation and temperature-programmed surface reaction (TPSR) studies suggested that the adsorbed CO₂ species participated in CO₂ hydrogenation. On Ru₁ sites, CO₂ hydrogenation followed the RWGS pathway, resulting in the production of CO. In contrast, on Ru_FAC sites, the enhanced H₂ dissociation ability, along with the presence of adsorbed bidentate and monodentate carbonate species at the Ru−MgO interfaces, facilitated the formation of CH₄ through the CO₂ methanation pathway. This study highlights the critical roles of Ru structure and local environment in defining the CO₂ hydrogenation pathways and provides new design principles for highly active Ru-based catalysts.
Quantifying noise limitations of neural network segmentations in high-resolution transmission electron microscopy
2023, Ultramicroscopy
Motivated by the need for low electron dose transmission electron microscopy imaging, we report the optimal frame dose (i.e. $e^{-} / Å^{2}$ ) range for object detection and segmentation tasks with neural networks. The MSD-net architecture shows promising abilities over the industry standard U-net architecture in generalising to frame doses below the range included in the training set, for both simulated and experimental images. It also presents a heightened ability to learn from lower dose images. The MSD-net displays mild visibility of a Au nanoparticle at 20–30 $e^{-} / Å^{2}$ , and converges at 200 $e^{-} / Å^{2}$ where a full segmentation of the nanoparticle is achieved. Between 30 and 200 $e^{-} / Å^{2}$ object detection applications are still possible. This work also highlights the importance of modelling the modulation transfer function when training with simulated images for applications on images acquired with scintillator based detectors such as the Gatan Oneview camera. A parametric form of the modulation transfer function is applied with varying ranges of parameters, and the effects on low electron dose segmentation is presented.
Coherent twin-oriented Al<inf>3</inf>Sc-based precipitates in Al matrix
2023, Scripta Materialia
Al₃(Sc,Zr,Ti) nanoparticles with an ideal twin-type orientation relationship to Al host matrix were found in cold-rolled and subsequently annealed Al-based alloy. Atomic-scale investigations using high-resolution scanning transmission electron microscopy identified particles that form prominent coherent (111) twin-type interfaces along their longer facets and semi-coherent twin interfaces on their shorter facets. Ab-initio calculations showed that a coherent Al/Al₃Sc twin-like phase boundary corresponds to a local energy minimum. A model is proposed explaining the formation of the twin orientation relationship of an Al₃Sc nanoparticle with the Al host matrix.
Three-dimensional stacked filter: A non-linear filter for series images obtained using a transmission electron microscope
2022, Ultramicroscopy
Citation Excerpt :
Among these, shot and dark current noises can be described using Poisson statistics, whereas Fano noise is empirically smaller than a Poisson distribution noise, as a function of photon energy. For higher doses, Fano noise approaches a Gaussian distribution for higher dose and the read-out noise is often directly depicted using Gaussian statistics [8,9]. With the development in cameras [10], noise has been reduced; however, the incident electron beams’ Poisson noise may become dominant on the image.
De-noising is an important issue in quantitative high-resolution transmission electron microscopy (HRTEM), and its roles become even more important in applications such as beam-sensitive materials and dynamic characterisations, where the attainable signal-to-noise ratio (SNR) of HRTEM images is frequently limited. In this study, we introduce a three-dimensional stacked filter (3DSF), a novel non-linear filter in which a series of HRTEM images is stacked into a 3D data cube and then treated with the Wiener filter in the 3D domain. In comparison to the traditional Wiener filter, which is widely used for individual images, this filter can accurately estimate the power spectral density of noise and filter images with a higher SNR, fewer artefacts and greater computation efficiency, which works particularly well for HRTEM images containing periodic information and feature similarities in successive micrographs, as demonstrated by simulated and experimental images of graphene and metal–organic framework (MOF). When applied to an ultra-low dose (∼8 e/Å²) HRTEM image stack of MOF MIL-101, the 3DSF could distinguish 40 consecutive frames, revealing the trajectory of subtle lattice shrinkage during the exposure.
High-resolution transmission electron microscopy and spectroscopy of epitaxial metal oxides
2022, Epitaxial Growth of Complex Metal Oxides
Quantitative determination of local atomic and electronic structures at surfaces, interfaces, and defects and elucidation of their dynamics and influences on materials, nanostructures, and devices, thereby to develop structure–property relationships at the atomic level remains the ultimate goal of microscopy in materials sciences. The reigning paradigms are imaging and spectroscopy techniques based on aberration-corrected transmission electron microscopy (TEM). Aberration-corrected TEM provides scientists great opportunities to routinely resolve structural details, to map both compositional and bonding information, and to explore properties of thin films at the atomic scale.
In chapter 13, we introduce first imaging and spectroscopic techniques based on both the conventional coherent and the scanning imaging modes of current state-of-the-art aberration-corrected transmission electron microscopes. We show then examples for application of these techniques in characterization of structural, compositional, and bonding details of interfaces and defects in thin films of metal oxides at the atomic scale.
Anisotropic point defects in rhenium diselenide monolayers
2021, iScience
Citation Excerpt :
Images used for point defects statistics were processed by removing the effect of the probe tail to more accurately identify the intensity on each atom position(Krivanek et al., 2010). Each frame of the time-sequence video was filtered using a nonlinear filter to increasing the signal to noise ratio(Du, 2015). The 4D-STEM experiments were performed on the same microscope with the same settings.
Point defects in 1T″ anisotropic ReSe₂ offer many possibilities for defect engineering, which could endow this two-dimensional semiconductor with new functionalities, but have so far received limited attention. Here, we systematically investigate a full spectrum of point defects in ReSe₂, including vacancies (V_Se1-4), isoelectronic substitutions (O_Se1-4 and S_Se1-4), and antisite defects (Se_Re1-2 and Re_Se1-4), by atomic-scale electron microscopy imaging and density functional theory (DFT) calculations. Statistical counting reveals a diverse density of various point defects, which are further elaborated by the formation energy calculations. Se vacancy dynamics was unraveled by in-situ electron beam irradiation. DFT calculations reveal that vacancies at Se sites notably introduce in-gap states, which are largely quenched upon isoelectronic substitutions (O and S), whereas antisite defects introduce localized magnetic moments. These results provide atomic-scale insight of atomic defects in 1T″-ReSe₂, paving the way for tuning the electronic structure of anisotropic ReSe₂ via defect engineering.

View all citing articles on Scopus

^☆: It is a great pleasure to dedicate this paper to Prof. Harald Rose in celebration of his 80th birthday.

View full text

A nonlinear filtering algorithm for denoising HR(S)TEM micrographs☆

Highlights

Abstract

Introduction

Section snippets

Tests of Gaussian and Wiener filters

Nonlinear filtering algorithm

Simulated images

Conclusions

Acknowledgments

Ultramicroscopy

Ultramicroscopy

Micron

Ultramicroscopy

Ultramicroscopy

Ultramicroscopy

Ultramicroscopy

Ultramicroscopy

Atomic-resolution measurement of oxygen concentration in oxide materials

Science

Sub-angstrom resolution using aberration corrected electron optics

Nature