Reconstruction of Axial Tomographic High Resolution Data from Confocal Fluorescence Microscopy: A Method for Improving 3D FISH Images

Fluorescent confocal laser scanning microscopy allows an improved imaging of microscopic objects in three dimensions. However, the resolution along the axial direction is three times worse than the resolution in lateral directions. A method to overcome this axial limitation is tilting the object under the microscope, in a way that the direction of the optical axis points into different directions relative to the sample. A new technique for a simultaneous reconstruction from a number of such axial tomographic confocal data sets was developed and used for high resolution reconstruction of 3D‐data both from experimental and virtual microscopic data sets. The reconstructed images have a highly improved 3D resolution, which is comparable to the lateral resolution of a single deconvolved data set. Axial tomographic imaging in combination with simultaneous data reconstruction also opens the possibility for a more precise quantification of 3D data. The color images of this publication can be accessed from http://www.esacp.org/acp/2000/20‐1/heintzmann.htm. At this web address an interactive 3D viewer is additionally provided for browsing the 3D data. This java applet displays three orthogonal slices of the data set which are dynamically updated by user mouse clicks or keystrokes.

Reconstruction of axial tomographic high resolution data from confocal fluorescence microscopy: a method for improving 3D FISH images Fluorescent confocal laser scanning microscopy allows an improved imaging of microscopic objects in three dimensions. However, the resolution along the axial direction is three times worse than the resolution in lateral directions. A method to overcome this axial limitation is tilting the object under the microscope, in a way that the direction of the optical axis points into different directions relative to the sample. A new technique for a simultaneous reconstruction from a number of such axial tomographic confocal data sets was developed and used for high resolution reconstruction of 3D-data both from experimental and virtual microscopic data sets. The reconstructed images have a highly improved 3D resolution, which is comparable to the lateral resolution of a single deconvolved data set. Axial tomographic imaging in combination with simultaneous data reconstruction also opens the possibility for a more precise quantification of 3D data.
The color images of this publication can be accessed from http://www.esacp.org/acp/2000/20-1/heintzmann.htm. At this web address an interactive 3D viewer is additionally provided for browsing the 3D data. This java applet displays three orthogonal slices of the data set which are dynamically updated by user mouse clicks or keystrokes.

Introduction
Confocal microscopy is an important method for analytical cellular pathology. It offers the possibility to acquire three-dimensional data of a cell and to perform a 3D analysis of various features. The confocal microscope allows for optical sectioning of the specimen. A flat structure can easily be discriminated using a confocal microscope (in a standard wide field microscope this is not possible). But even a confocal microscope has still a very limited axial resolution. This anisotropic resolution along the optical axis (Zdirection) is by a factor of about three worse than the lateral resolution. A high and isotropic resolution is, however, desirable for a precise imaging and analysis of cellular specimens.
To avoid the fundamental problem of reduced axial resolution, the technique of axial tomography has been introduced (see [16]). By a rotation of the specimen under the microscope it is possible to acquire a number of different views, where the axis of reduced resolution points into different directions relative to the specimen. This technique has been applied to wide-field microscopic data ( [5,9,16]) but is also well applicable to confocal data ( [2,5]). The apparatus for specimen rotation has further been improved ( [1]) by placing the specimen inside a capillary or on a glass fiber. In the experiments presented here the specimen has been placed on a glass fiber (see Fig. 1).
When such axial tomographic data is acquired, it still remains a difficult problem to reconstruct a consistent three-dimensional image of the sample with isotropic high resolution. Most axial tomographic approaches for image reconstruction were based on different data selection criteria in Fourier space ( [5,15,16]). This Fourier space based reconstruction methods have, however, a number of drawbacks and do not ex- plicitely account for the photon noise present in the measured data. In addition these methods do require a very accurate alignment of the measured data sets.
In the following a method is described to overcome some of the difficulties described above by using a technique that does not operate directly in Fourier space. In addition this new method combines a successful iterative deconvolution method with the aim of high resolution reconstruction from multiple angles. The necessary alignment is achieved by procedures based on the interactive analysis of the measured data sets. Using this technique, however, no precise adjustments of angles or positions are necessary in the experiment.
The developed method was tested in two different ways.
-Computer simulations of human nuclear structure [10] were convolved with a measured confocal point spread function (PSF) and photon noise was simulated (see Section 5.1). -For an experimental application a well known microscopic structure (a moss spore) was registered by two color confocal microscopy (see Section 5.2).
These data sets were used for high resolution reconstruction.

Experimental setup
Moss spores (Polytrichum commune, gratefully provided by V. Sarafis, see [14]) have been diluted in water and placed on a glass fiber by evaporation of the solvent. The glass fiber was inserted into a specially constructed sample holder for axial tomography [1]. This sample holder supports the fiber which is placed in a V-groove and surrounded by the mounting medium. The mounting medium immersion oil (Zeiss 518 C, n = 1.518) was used to achieve a matching of the index of refraction with the immersion fluid and the glass fiber. The refraction index of the glass (AR-Glass, n = 1.516) was chosen to match with the index of the mounting medium. Above the fiber a standard coverslip was placed.
The capillary holder lifted the capillary 4.2 mm above the normal imaging plane of the confocal laser scanning microscope (LEICA TCS, Leica Lasertechnik GmbH, Heidelberg, Germany). This resulted in a shearing of the confocal data sets, because in this confocal set up used, the axial movement is achieved not by a translation stage but by a rotation about an axis that is in about 70 mm distance from the lateral sample mounting position. This shearing was corrected after data acquisition using standard algorithms.
Confocal imaging was performed using an objective lens with magnification 63× and a numerical aperture of 1.4. Between the lens and the coverslip, immersion oil of refraction index n = 1.518 (Zeiss 518 C) was used. An argon-krypton mixed ion gas laser was used for excitation of the imaged specimens with a wavelength 1 of 488 nm. This wavelength was used to excite both detectable fluorescence channels.
A spore on the glass fiber was selected by visual inspection using the conventional wide-field mode of the microscope. This region was imaged three times for each rotation angle of the fiber using the confocal mode of the microscope with a pixel step size of 78.6 nm in lateral and 162 nm in axial direction. The image size was 128 × 128 pixel in every image plane. About 40 slices were measured in every view. Every slice in every stack was averaged four times before advancing to the next lateral slice of a data set.
A total number of three rotation angles for the specimen were imaged (estimated to be around −45 • , 0 • and +45 • ). Each rotational view was imaged three times in succession. To locate the same region on the fiber again after rotation, very faint illumination light was used, while repositioning the lateral and axial position using the conventional epi-fluorescent mode.

Correction of the experimental data
The confocal data obtained had to be corrected prior to reconstruction to eliminate all factors that were not accounted for in the reconstruction process.
The measured values where first rescaled in such a way that to every voxel the number of detected photons after subtraction of an offset was assigned. This could be especially important at low photon counts for the maximum likelihood (ML-) algorithm, because the ML-algorithm is based on the assumption of Poisson distributed photon noise. The algorithm does not account for photo-bleaching, and the intensity reduction between successive data sets was therefore determined. In the red channel the bleaching was about 7% per data set and in the green channel 20% per stack. The intensities in different views were rescaled to correct for bleaching. No bleaching correction was applied to the planes inside a single data set.
These data sets were then corrected for the shift introduced in each lateral slice of the data set by the misaligned direction of axial movement due to the misplacement of the capillary 4.2 mm above the normal image plane.
The individual views (under different rotation angles) of the specimen were aligned by a rotation and a displacement. The angles of rotation were chosen by selecting the three-dimensional coordinates of three points manually in each data set. From these vectors the angles of rotation and the displacements were computed. A further alignment of the the data sets rotated back to the 0 • position, was achieved by a crosscorrelation based on shifting in Fourier-space (see [8]). A rescaling based on zero padded expansion in the Fourier domain yielded data sets with equal sampling rates (78 nm) in all three dimensions of space.
For the following maximum likelihood reconstruction a point spread function (PSF) is needed. This was taken from a number of fluorescent beads (TetraSpeck ∅ = 90 nm, Molecular Probes, Eugene, Oregon, USA), which were imaged simultaneously in both fluorescent channels (red and green). Obtained data sets were added after alignment and resampled to 78×78× 78 nm voxel spacing.

The ML reconstruction algorithm
For the reconstruction of one data set with high resolution along all three directions of space an image reconstruction algorithm based on the maximum likelihood (ML) method was developed. Iterative ML reconstruction is a widely used tool for deconvolution of confocal data (see [17]). This technique has been further extended to include the data of the specimen imaged under different rotational views in the pro-cess of ML-reconstruction (a related technique was already proposed in the discussion of [16]). A mathematical analysis of the likelihood for an axial tomographic measured data set resulted in the following iterative reconstruction process: Prior to the iterations: -In a first step all data sets have to be rotated and translated in a way that their geometry is aligned. It is useful (but not necessary) to use the 0 • view as a reference reconstruction coordinate system. -The PSF used for data reconstruction has also to be turned by the same angles. The center of intensity of the PSF is, however, not changed. -An initial "guess" data set (G 0 ) is chosen. It has been proven useful to choose the data set to be equal to a constant (one). Another possibility would be to choose the mean value of all data sets.
One iteration step number n consists of : -A forward projection P of the guessed data set is calculated. This is done by computation of a convolution of the actual guess G n with the microscopical point spread function PSF( x). This convolution is computed with PSFs, which were adapted to the rotation angle for every measured data set imaging orientation (see above).
The over-relaxation factor q is used for improving the speed of convergence of the algorithm.
This iteration process is depicted in Fig. 2. As noted in [7], ML reconstruction algorithms tend to a non-stable result due to an ill-defined reconstruction goal. One way to deal with this problem is to stop the iteration after a specified number of steps before the distortion becomes too big.
Another approach is a regularization of the problem. By placing a number of constraints on the resulting data set a stable result can be obtained. In our algorithm the resulting data set was constrained to contain nonnegative values only and no spatial frequencies above the band-pass limit of the optical system. Spatial frequencies above this limit where deleted during reconstruction after each iterative step. This method is somehow similar to the common usage of a sieve kernel.

Reconstruction results
After verification of the algorithm by simulations of random point positions and their reconstruction, the technique was tested on microscopic bead samples. The reconstructed bead data sets revealed a clear improvement of axial resolution with almost no loss of lateral resolution 3 .

Simulated data sets
To test the usefulness of this method for the imaging of structures of cell nuclei, a simulation of a chromosomal structure model was performed. This procedure allowed a tight control of the performance of the algorithm. Virtual confocal images of such a simulation based on the "Spherical 1Mbp-Chromatin Granule (∅ = 500 nm) Domain" model (see [10]) were computed. The size of the simulated nucleus was ten micrometers in diameter. In the simulation the whole territory of chromosome 7 was labeled and the process 3 As compared to deconvolutions using only one confocal view but three times more photons. of confocal imaging was computed (see Fig. 3). This was achieved by a convolution of the simulated original structure with an experimentally measured PSF. These data sets were then subjected to simulated photon noise. The maximal expected photon count was equal to 100 photons per voxel in every view. For the single view reconstruction the simulated photon count was greater by a factor of three 4 . Reconstructed images from these simulated chromosome models are shown in Figs 3, 4, 5. Single view deconvolution resulted already in a significant improvement. Note, for example, that the gap that is visible as indentation in Fig. 5a cannot be seen in the virtual microscopic image (Fig. 5b). Further considerable progress was achieved by the technique of tomographic reconstruction (note, for example, the contrast of the gap in Fig. 5d).
For reconstruction the data sets were corrected, resampled and rotated to the 0 • position. Then the iterative ML-based reconstruction process was applied to the data (either to one or multiple views simultaneously). During iteration a list of empirically obtained over-relaxation factors (q): (1, 2, 1, 4, 1, 8, 1, 1, 4, 1, 16,  1, 2, . . .) was used. The last four factors were repeated until the total number of 30 iterations was reached. The iteration was then finished with 10 iterations using only an over-relaxation factor of 1. The image quality of axial tomographic reconstruction can be seen to be superior to the single view reconstruction.

Experimental data sets
In addition to virtual microscopy the reconstruction method was used on an experimental data set of a biological specimen. As an object moss-spores were used for the following reasons: -The structure of these objects is well known to consist of four chloroplasts, which have distinct boundaries. -The total size corresponds to that of a typical human cell nucleus. -The spore can be embedded in immersion oil as mounting medium, therefore aberrations caused by refraction index mismatch [6] can mostly be avoided. -The object and its constituents are small enough to test the microscopic resolution. -Fluorescence can be investigated in two color channels.   -The structure is truly three dimensional.
-No artificial staining procedure is needed.
-A high photon statistics can be obtained from these specimens.
A moss spore placed on a glass fiber was imaged from three different views (see Section 2). No staining had to be applied to the samples. Only the autofluorescence of the chloroplasts (red fluorescence) and other material of the spore (mainly green fluorescence) were measured. The maximal photon counts per pixel and data set were 2641 photons (red detection channel) and 194 (green detection channel), respectively.
The data sets were corrected and then treated in the same way as simulated data sets.
A ray-traced version (ray tracing using [12]) after segmentation of the reconstructed axial tomographic result can be seen in Fig. 6. The four chloroplasts are clearly visible embedded in the surrounding material of the spore. A comparison of red color YZ slices through the data using axial tomographic reconstruction and single-view ML reconstruction can be seen in Fig. 8 (for two color images see http://www.esacp.org/ acp/2000/20-1/heintzmann.htm, Fig. 7). For appropriate comparison with a tomographic reconstruction the triple amount of collected photons were used in the single-view reconstruction. Because of the low photon statistics in the green fluorescent light an iteration number of only 15 iterations was used for the reconstruction of the green channel. The computational time necessary for the axial tomographic reconstruction of the spore (128 × 128 × 128 voxels) was 20 min (for the green channel) and 50 min (for the red channel) on a standard PC equipped with a 200 MHz PentiumPro processor.
As it can be observed, there is an improvement of this method especially along the direction of the optical axis (Z-direction). The image quality of the axial tomographic reconstruction is now nearly isotropic. The internal structure of the chloroplasts can clearly be seen. It can well be seen that in the axial tomographic reconstruction the boundaries and the internal structures are much better defined. No significant loss in lateral resolution was observed even in comparison to single-view deconvolution (see Fig. 10).  Due to the intense fluorescence excitation in the chloroplasts, the transmitting excitation light in a single view is diminished (see Fig. 9, also Fig. 8b) in deeper layers (increasing Z-values) of the sample. In a single view ML-deconvolution this effect can cause artifacts especially of the deeper embedded boundaries. When the sample is rotated under the microscope and the acquired data is recombined, this destructive effect can be partially compensated 5 . In different rotational views, different regions of the specimen are affected by this absorption effect. The axial tomographical reconstruction is therefore superior.

Discussion and outlook
The results obtained indicate the usefulness of the developed algorithm for 3D reconstruction of micro- 5 For other ways of absorption correction see [11]. scopic images. Presently we intend to use these algorithms for the 3D reconstruction of radiation induced changes in nuclear architecture. A further potential application in analytical cellular pathology is the 3D reconstruction of tumor cell nuclei labeled by FISH.
The data alignment was still performed by an interactive process. Presently it can take up to a few hours to define the matching points in the datasets. It is, however, intended to considerably reduce the time requirements by a fully automatic alignment of the data.
Artificial microscopic beads could be added in a moderate concentration to the sample preparation. These beads can be detected and segmented automatically, and their position can be used for the data alignment process. The mapping between many of those points could for example be determined with a method using the technique of Complete Moment Invariants ( [3]). Methods based on the moment of inertia ( [13] and [4]) are also a promising approach.
A further approach would be an automatic alignment that does not require any special sample preparation techniques. Such an approach is described in [16] by making use of a high frequency enhancing crosscorrelation approach iterating over the angles of rotation.
The rotational alignment of the data is at the moment performed by a trilinear interpolation technique. This interpolation can introduce small artifacts by high frequencies reduction and introduction of frequencies   9. (a) Projection for a data set (red channel) imaged at a rotation angle, where the chloroplasts partially overlap. The incident excitation light comes from the top of this image along Z-direction. One of the chloroplasts is much reduced in intensity due to the extinction of the light by fluorescence of upper layers. (b) Single-view deconvolution of the data. The image is distorted due to this extinction effect. A colored version of (b) is shown at http://www.esacp.org/acp/2000/20-1/heintzmann.htm. Fig. 10. XY-Slices of the red fluorescence channel data of the moss spore shown in Fig. 6. For (a) and (b) the approximate triple amount of photons was collected and displayed. above the sampling limit, when resampling is used (see [8]). To avoid this, a rotation algorithm should be used that is based on Fourier-space shearing (see [5]).
The algorithm could further be improved by modifying the ML-iteration to account for the photobleaching of the sample. The speed of the algorithm can also significantly be improved when the necessary convolution steps are performed in parallel. When the calculations are performed in parallel on ten PCs with PentiumPro 200 MHz double processors, the time requirement for the axial tomographic deconvolution can be estimated to be below five minutes.
The same reconstruction method can also be applied to the reconstruction of epifluorescent (wide-field) data sets, either from a single focus series or from many series taken under different tilt angles.