Three-dimensional focal stack imaging in scanning transmission X-ray microscopy with an improved reconstruction algorithm

Focal stack (FS) is an effective technique for fast 3D imaging in high-resolution scanning transmission X-ray microscopy. Its crucial issue is to assign each object within the sample to the correct position along the optical axis according to a proper focus measure. There is probably information loss with previous algorithms for FS reconstruction because the old algorithms can only detect one focused object along each optical-axial pixel line (OAPL). In this study, we present an improved FS algorithm, which utilizes an elaborately calculated threshold for normalized local variances to extract multiple focused objects in each OAPL. Simulation and experimental results show its feasibility and high efficiency for 3D imaging of high contrast, sparse samples. It is expected that our advanced approach has potential applications in 3D X-ray microscopy for more complex samples. © 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

soft X-ray regime due to the low efficiency of the available focusing optics and more importantly the extremely low soft x-ray fluorescence yields [15,23].
Focal stack (FS) method has been proposed since 1980s and recently used in synchrotron X-ray regime [24][25][26][27][28]. The FS method requires collecting a series of 2D microscopic images with shifting the sample to different positions along the optical axis (usually referred to as "zdirection") so that the incident beam is focused at different depths within the sample. In STXM, it means that the sample is not only raster-scanned through the focal spot in x-and ydirection, but also in z-direction. These images with different Z-positions are then combined to get a better 2D image or form a composite volume to distinguish the 3D spatial positions of individual features. A number of FS-based studies have been carried out using light-filed cameras in the visible light regime [29,30]. A variance-based algorithm has been introduced to reconstruct gas filled microspheres and a flake of a butterfly wing by the FS scheme of STXM [31]. However, their algorithm will only detect one optimum focused position along each optic-axial pixel line (OAPL) for a stack of images. If the sample has two or more features along one particular OAPL, only the one with the highest absorption can be found and parts of the structural features will be lost after reconstruction. A similar approach for transmission X-ray microscope (TXM) is the extended depth of focus (DOF) method to generate fully in-focus micrographs by fusion of a series of sample images at different focal positions based on a fast discrete wavelet transform algorithm [28]. Thus, a high resolution 3D image of a complete biological cell can be obtained from tomographic reconstruction of multiple fusion images at different angles [32].
Here we present an improved FS algorithm to efficiently reconstruct the full 3D structures of thick samples based on 2D STXM micrographs. In this algorithm, a threshold calculation method is introduced to extract multiple focused features along each OAPL. We conduct a simulation and experiments for various samples with ferroferric oxide (Fe 3 O 4 ) nanoparticles to verify the effectiveness of the approach we proposed.

Principles
In STXM, if a sample is thicker than the DOF of the Fresnel zone plate (FZP), only the features within the DOF can be imaged clearly at the best level. The DOF for an FZP is given by [33] where Δr n represents the outermost zone width of the FZP, n is the total number of zones, λ is the wavelength of the incident light. According to the Rayleigh criterion, the lateral resolution of STXM is 1.22Δr n with a given zone plate [34]. Therefore, imaging of nanostructures with a resolution of 25 ~30 nm (routine resolution for standard STXM experiments) will result in a DOF of several hundreds of nanometers. Many test samples relevant for soft X-ray imaging can still be investigated with a thickness up to several micrometers when the transmitted signal is high enough [12,35,36]. In consequence, only parts of the specimen can be in proper focus within a specific STXM micrograph. Figure 1 illustrates this issue. The focused parts of a scanning image ( Fig. 1(a)), marked with a red box) are within the DOF range of the beam propagation path ( Fig. 1(b)), marked also by a red box, z 1 ), corresponding to a focus point to be detected by FS algorithm. The particles marked with purple and blue boxes are out of focus ( Fig. 1(a)), which should be at different focused z-positions ( Fig. 1(b)), purple and blue frames, z 2 and z 3 ). For a series of 2D images collected at different focused depths inside the sample along the optical axis, extracting focused specimen features axially with the FS algorithm will lead to the reconstruction of a 3D structure of the specimen. The local variance R n (x, y) is often used as a focus measure in FS approach. It reflects the density gradients of the neighborhood surrounding a given pixel and enables us to choose the best focused features of each OAPL. However, extracting the maximal R n (x, y) for a series of axially-stacked images can only pick out the most prominent focused feature which is applicable only for samples with a single object along each OAPL. Apparently, there may be information loss when a sample is complex enough that multiple features could appear along the same OAPL. The expression of local variance R n (x, y) is as follows [31,37]  where I is the photon intensity recorded for a certain pixel, n represents the index of the image sequence.⎯I denotes the local mean, which can be calculated from the following equation: In addition, the size of the measured window (2a + 1) × (2a + 1) is an important factor influencing the precision of the extracted depth positions of the features. This is because the smaller the window size, the more sensitive the local measure is to the noise; the larger the window size, the smoother the edge areas of a focus map will be. In the simulation of this paper, the parameter a was set as 3; while in the reconstructions of the two experiment data sets, a was taken as 5. A larger value of a for experimental data can relieve the influence of noise which is unavoidable to experimental data.
Based on the above considerations, we proposed an improving scheme for the FS algorithm by thresholding the normalized local variances r n to extract and locate multiple focused feature points (high-density featured structures) along each OAPL. As illustrated in Fig. 2(a), the peaks of r n higher than the threshold T n (red points in Fig. 2(a)) correspond to the positions in the z-direction where sample features need to be extracted. The normalized local variance r n has the following form: where R n (x,y) max is the maximal R n along a given OAPL. The range of r n (x,y) is 0 to 1. The first step of reconstructing the 3D image cube V(x,y,z) is to calculate the threshold T n for each OAPL. We introduce an iterative threshold method [37] to calculate T n . This method works as follows: (1) Start with the average of r n over an OAPL as an initial threshold T 0 and the sequence of r n is divided into two parts, r 1 and r 2 (r 1 : r n <T 0 ; r 2 : r n ≥T 0 ). (2) Calculate the averages of r 1 and r 2 respectively, obtaining u 1 and u 2 . (3) Calculate the average of u 1 and u 2 as a new T n . (4) Repeat steps (2)-(3) until T n remains stable, then the resulting T n is the desired threshold along this OAPL. In the next step, by extracting the peaks of r n higher than T n along an OAPL, the focused positions of multiple structural features can be obtained on this OAPL. Then the corresponding measured image intensity (or optical density of the sample under transmission in 2D measurements, usually proportional to the sample density) values are filled in the V(x 0 ,y 0 ,z) at these focused positions ((x 0 ,y 0 ) is the lateral position of the OAPL) ( Fig. 2(b)). That is to say, the voxel values are the measured image intensity at these positions of the OAPL, V(x 0 ,y 0 ,z) = I(x 0 ,y 0 ,z), where I(x,y,z) is the stacking of the measured 2D image intensity along the z-direction. On the other hand, for the positions with r n < T n along this OAPL, the corresponding voxel values are set as zero, i.e. V(x 0 ,y 0 ,z) = 0 ( Fig. 2(b)). After all the OAPLs of I(x,y,z) are treated in this way, the 3D image V(x,y,z) is generated. Finally, V(x,y,z) can be rendered and visualized with a common 3D software (cf. Figure 3(a) for details of the algorithm). The schematic of the FS imaging process is shown in Fig. 3(b) and the STXM experimental setup for FS is animated in a movie (Visualization 1).

Simulation
In order to validate the improved FS algorithm described above, we performed a simulation for FS imaging in soft X-ray regime. In the simulation, a cubic sample model was constructed with a size of 3 × 3 × 3 μm 3 . It consisted of a carbon matrix with randomly distributed iron (Fe) and manganese (Mn) nanoparticles. The particle size ranged from about 100 to 600 nm and their shapes were all cubic ( Fig. 4(a)). The X-ray propagation through the thick sample was modelled by the multi-slice approach [38,39] combined with the angular spectrum method of Fresnel diffraction [8,40] and finally the intensity of the exit wave was integrated and recorded (see Appendix A for details). The incident X-ray probe produced by an FZP was modeled by the Fresnel-Kirchhoff diffraction theory (see Appendix A for details).
The incident X-ray flux used was 10 9 photons/s and its photon energy was 708 eV at the Fe L 3 -edge. The applied FZP had a diameter of 60 μm, a central-stop of 24-μm-radius and an outermost zone-width (Δr n ) of 10 nm. According to Eq. (1), the DOF of this focusing optics was calculated to be 230 nm. Some parameters related to the simulated sample, such as the densities ρ and absorbencies μ, were taken from an X-ray database [41]. Taking into account the DOF and ~5 times oversampling need, the step size of the sample moving in z-direction was set as 50 nm in this simulation. The scanning step size in x-and y-direction was set as 10 nm, determined by the outermost zone-width of the FZP. Therefore, the modelled sample had a volume of 300 × 300 × 60 voxels. By using the multi-slice method to model the propagation of X-rays in the sample and moving the sample along the optical axis, 61 images were collected sequentially with a 50 nm z-direction step for moving over 3 μm to ensure that each part of the sample was within the DOF in one of the 2D micrographs. The 2D representation of the sample model thicker than the DOF is shown in Fig. 4(b). Figures 4(c) and 4(d) present two STXM micrographs imaged at different focused positions z 1 and z 2 in the sample, respectively. The colorbars on the right side of the two images represent the transmission coefficients: white corresponds to high transmission, gray to the medium values and black to low transmission. Comparing the two images, it can be seen that the cubic particle marked with a red box at the top of Fig. 4(d) is in focus, which is clearer than that in Fig. 4(c), while the one with a blue box is nearer to focus than that in Fig. 4(d). The 3D reconstructed image by our improved algorithm (Fig. 5(a)) was in good agreement with the sample model shown in Fig. 4(a). It is animated in Visualization 2. While employing the algorithm from a previous literature [31], there were something wrong with the reconstruction. As shown in Fig. 5(b), some particles are lost and some are wrongly hollow since only one focused feature (with the highest local variance) was extracted along each OAPL in the old algorithm. The colors in the reconstructed 3D images represent the density of the sample, which is proportional to the optical density (OD) or the absorption intensity of the sample under transmission. From the color scale, it can be seen that the calculated densities of Fe and Mn have little difference in this simulation. Furthermore, the 3D resolutions were estimated by picking out a slice of the reconstructed image perpendicular or parallel to the optical axis and fitting the derivatives of three linesegments across the edges of an exemplary particle on this slice ( Fig. 9 in Appendix B). The full width at half maximum (FWHM) values of the Gaussian fitting curves gave the resolutions of the reconstructed 3D image as 11 nm, 14 nm and 95 nm in the x, y and z directions, respectively.

Experiments
To verify the imaging capabilities of the improved FS approach for real specimens and explore its applications, we reconstructed and analyzed the data set once used in the literature [31], and carried out another soft X-ray experiment at the soft X-ray spectromicroscopy (SM) beamline of Canadian Light Source (CLS) [42].

Experiment 1
The sample for the first data set was a 5-μm-diameter gas filled PVA microballoon with Fe 3 O 4 nanoparticles on the shell. The STXM micrographs used for reconstruction have been measured at the PolLux beamline of Swiss Light Source by moving the sample with a step of 400 nm in z-direction (DOF ≈500 nm) and recording 2D images for an area of 6 × 6 μm 2 and 250 × 250 pixels. We reconstructed the specimen with the previous algorithm and the improved one, respectively, for comparison, as shown in Figs. 6(a) and 6(b) (Visualization 3). From both reconstructed images, we can see that the Fe 3 O 4 particles are generally distributed on the surface of the microballoon and there are more particles dispersed on the left semisphere than on the right one. Moreover, there are lots of large or small agglomerations of the Fe 3 O 4 nanoparticles. However, in general, the image reconstructed by the previous algorithm ( Fig. 6(b)) shows a sparser distribution of Fe 3 O 4 particles than that by our new algorithm ( Fig. 6(a)), especially in the part marked by a white circle. The color scales depict the corresponding voxel values and the larger the voxel values, the higher the absorption intensity at that position. Analyzing the enlarged half microballoons, which were obtained by cutting the reconstructed balloons through the center, it was found that some particles were lost under the reconstruction by the previous algorithm ( Fig. 6(d)) as compared to the reconstruction by the improved one (Fig. 6(c)). For example, in the green box of Fig. 6(d), an agglomerated particle is lost when compared to the same area of Fig. 6(c), and it is more obvious for the features marked by a white ellipse. Apparently, the disappearance of particles is due to the limited detection capability of the old FS algorithm.
Two slices perpendicular to each other were selected to estimate the spatial resolutions with the edge profile-fitting method, giving 3D resolutions of 35 nm in the x and y dimensions and 496 nm in the z-direction (Fig. 10 in the Appendix B). Fig. 6. Comparison between the reconstructed images by the previous algorithm and the improved one. (a-b) The reconstructed images by our improved algorithm (Visualization 3) and the previous one, respectively. The part marked by a white circle in (b) has fewer particles than that in (a). (c-d) Details comparison between the two enlarged reconstructed half microballoons by the two algorithms.

Experiment 2
The test sample of the second experiment was a film consisting of cellulose nanofibrils (CNF) embedded with Fe 3 O 4 nanoparticles (CNF/Fe 3 O 4 film). It was produced by adding 0.2g of Fe 3 O 4 nanoparticles aqueous dispersion (particle size: about 100 nm) at 5 wt % to 50g of CNF aqueous suspension with a solids content of 0.2 wt %, then stirring this mixture for 5 hours at 580 rpm. The film was collected on a disposable petri dish with dehumidification overnight.
In the second experiment, the photon energy was 710 eV and a 20-nm-FZP with a diameter of 120 μm was used to focus the X-ray beam, which resulted in a DOF of 917 nm. Figure 11(a) in the Appendix B shows a scanning electron micrograph of the CNF/Fe 3 O 4 film cross-section, from which the specimen thickness was estimated to be 4 μm. Agglomerations of the Fe 3 O 4 nanoparticles can also be seen in this graph. Obviously, the film is significantly thicker than the DOF of the FZP and the whole volume cannot be simultaneously imaged clearly. For the focal stack imaging, 51 STXM micrographs were collected at a series of axial positions by moving the specimen along the optical axis with a step of 300 nm. The 2D image size is 7 × 7 μm 2 which was scanned with a step of 40 nm. The first scan was based on the position where the specimen was completely off-focus, as shown in Fig. 7(a), then it was stepped in the z-direction until the specimen position was again off-focus. Four representative STXM images are shown in Fig. 7, where Figs. 7(a) and 7(d) are two off-focus images, and Figs. 7(b) and 7(c) are partly focused micrographs within the specimen (Δz b,c = 2.7 μm). A detailed comparison between Figs. 7(b) and 7(c) shows that the particles marked with a purple box at the right side of Fig. 7(b) are clearer than that in Fig. 7(c), while the ones marked with a red box is in focus in Fig. 7(c) and should be out of focus in Fig. 7(b). Magnified views showing more details are in the respective insets. After pre-processing with normalization, de-noising and subpixel image registration for the 51 focus-stacked images, the proposed and previous FS algorithms were applied for reconstruction, respectively. The 3D reconstructed images were rendered and visualized in movie files. From the 3D image reconstructed by our improved algorithm ( Fig. 8(a)), Visualization 4), we can see that all locations of the Fe 3 O 4 particles within the specimen are obtained, and the 3D image clearly shows these nanoparticles. Most importantly, multiple particles on the same OAPL can be seen clearly in this image, while in the image retrieved by the previous algorithm ( Fig. 8(b)), some particles are lost as compared to those in Fig. 8(a), indicating information loss during reconstruction. Furthermore, by picking out two slices perpendicular to each other and fitting the gradients of the profiles across the edges of an Fe 3 O 4 particle, we estimated the 3D spatial resolutions of the FS reconstruction to be 80 nm in the x and y dimensions and 214 nm in the z dimension (Fig. 11 in the Appendix B).
Additionally, the data acquisition time of the stacked images was 66 minutes, which is usually faster (at least no slower) than STXM-based nanotomography under the similar resolution condition. More importantly, compared to the tomography method, the FS method allows a larger imaging field of view, has more flexible requirements on the sizes or shapes of samples and no need for sample rotations. It is undoubted that the 3D imaging using the improved FS approach is more convenient than the traditional nanotomography by STXM in terms of easy operations and no need for rotation.

Conclusions
In summary, we have presented an improved focal stack algorithm which can extract multiple focused features along each optic-axial pixel line by thresholding the normalized local variances so that it can reconstruct 3D images from soft x-ray STXM micrographs. Simulated and X-ray experimental data demonstrated the viability and validity of the new algorithm. The proposed FS approach is more reliable for the complete reconstruction of thick sparse samples with complex axial nanostructures as compared to previous FS approaches. So far the axial resolution of the approach is still limited by the relatively large DOF of the Fresnel zone plate optics. The present work serves as a significant step to improve the STXM 3D imaging with high resolution and flexibility and can be hopefully further extended to complexstructure 3D imaging of dense specimens. It is expected that the focal stack imaging, either in STXM or in other mode, will be of considerable value to both morphologic and spectral microscopy with improved quality and flexibility.