Real-time Visualization of 3-d Dynamic Microscopic Objects Using Optical Diffraction Tomography References and Links

3-D refractive index (RI) distribution is an intrinsic bio-marker for the chemical and structural information about biological cells. Here we develop an optical diffraction tomography technique for the real-time reconstruction of 3-D RI distribution, employing sparse angle illumination and a graphic processing unit (GPU) implementation. The execution time for the tomographic reconstruction is 0.21 s for 96 3 voxels, which is 17 times faster than that of a conventional approach. We demonstrated the real-time visualization capability with imaging the dynamics of Brownian motion of an anisotropic colloidal dimer and the dynamic shape change in a red blood cell upon shear flow. Quantitative phase imaging techniques for the study of cell pathophysiology: from principles to applications, " Sensors (Basel) New approach to optical diffraction tomography yielding a vector equation of diffraction tomography and a novel tomographic microscope, " J. refractive index tomography by digital holographic microscopy, " Opt. Refractive index maps and membrane dynamics of human red blood cells parasitized by Plasmodium falciparum, " Proc. Natl. High-resolution three-dimensional imaging of red blood cells parasitized by Plasmodium falciparum and in situ hemozoin crystals using optical diffraction tomography, " J. Pf155/RESA protein influences the dynamic microcirculatory behavior of ring-stage Plasmodium falciparum infected red blood cells, " Sci Rep (2012).


Introduction
3-D live-cell imaging has been an invaluable tool for understanding the mechanisms involved in biological cells and tissues [1].Among techniques for 3-D live cell imaging, confocal scanning microscopy employs the principle of point-illumination and point-detection of fluorescent probes that tags specific targets, and axial scanning of focused beams yields the optical sectioning of biological samples [2].Recently, several super-resolution microscopy techniques have revealed 3-D structures at a nano-meter resolution [3].However, due to limited measurement and processing speed, 3-D imaging using those techniques is not suitable for real-time application.
Refractive index (RI) is an intrinsic optical property of materials that is sensitive to the chemical composition and structure of biological samples.The 3-D RI distribution of microscopic particles can be measured by various interferometry microscopy techniques [4,5], which utilizes a series of images either from various illumination angles [6][7][8][9][10] or axial translation of samples based on the transport-of-intensity [11].The measurement of the 3-D RI distribution of biological cells has been implemented for studying chemical and structural change of red blood cells (RBCs) [12][13][14][15], HeLa cells with various chemical exposures [16] and quantification of the dry mass of chromosomes during mitosis [17].However, acquisition of hundreds of images from various illumination angles and tomogram reconstruction based on several Fourier transforms require a large amount of calculation time, which hinders realtime application of optical diffraction tomography.Recently, researchers have tried to reduce acquisition time by a spatial fringe demodulation method [16] or by reducing the number of scanning views [18].In addition, a graphics processing unit (GPU) has been utilized to significantly reduce the computation time for field retrieval of off-axis holography and tomogram reconstruction [19][20][21].Unfortunately, despite considerable effort, attempts to push the acquisition and visualization of dynamic 3-D RI distribution into the realm of real time have been stymied by technical limitations, especially time-consuming 3-D reconstruction.Consequently, studying the dynamic 3-D RI distribution remains an unexplored area.
In this paper, we demonstrate the real-time reconstruction of dynamic 3-D RI maps of individual cells via optical diffraction tomography.Sparse angle illumination and GPU implementation can reconstruct an optical diffraction tomogram in 0.21 sec for 96 3  which is about 17 times faster than the execution time of the conventional method using CPU.Here, we first validate the use of sparse angle illumination and a reconstruction algorithm for optical diffraction tomography.Then, the proposed method is exploited for the real-time 3-D visualization of micron-sized objects: Brownian motions of anisotropic colloids and shape change of RBCs under a shear flow.

Experimental setup
Complex optical fields from various illumination angles are recorded by Mach-Zehnder interferometry (Fig. 1).A laser beam from a diode-pumped solid state (DPSS) laser (λ = 532 nm, 100 mW, Shanghai Dream Laser Co., Shanghai, China) is divided into two arms by a beam splitter.One arm is used as a reference beam.The other arm is tilted by a dual-axis scanning galvanometer (GVS012, Throlabs, Newton, NJ, USA) to illuminate samples with various angles.A condenser lens (UPLFLN, 60 × , NA = 0.9, Olympus Inc., Center Valley, PA, USA) with a tube lens (f = 200 mm) is used to illuminate the sample with a plane wave.The diffracted beam from a sample is collected by a high numerical aperture (NA) objective lens (UPLSAPO, 100 × , oil immersion, NA = 1.4,Olympus Inc., Center Valley, PA, USA).The beam is further magnified by an additional 4-f telescopic system, and the total magnification is 400 × .The diffracted beam interferes with the reference beam at an image plane and generates spatially modulated holograms, which are recorded by a high-speed CMOS camera (1024 PCI, Photron USA Inc., San Diego, CA, USA) with a frame rate of 1,000 Hz.The detailed experimental setup for optical diffraction tomography was described in the previous work [13].In the proposed technique, the galvanometer scans only 10 beams of different azimuthal angles, as shown in Fig. 1(a), with a scanning rate of one cycle at 10 milliseconds.The polar angle of illumination is limited by the NA of both the condenser and the objective lens, which is maintained at approximately 70° at the sample plane.

Optical diffraction tomography
3-D RI distribution of samples is reconstructed via optical diffraction tomography.The complex optical fields are extracted from measured holograms using a field retrieval algorithm [22].Figure 1(b) shows typical optical fields from different illumination angles.Then, 2-D Fourier spectra of the extracted optical fields are mapped into 3-D Fourier space according to the Fourier diffraction theorem [6,23], as shown in Fig. 1(c).Due to the sparse scanning of 10 different illumination angles and the limited acceptance angle of the optical system, the 3-D Fourier space has missing information.This missing information is filled by an iterative non-negative constraint algorithm [9,13,24] as shown in Fig. 1(d).It has been proved that the use of non-negative constraint can obtain the unique solution when sparse solution exists [25].The iterative algorithm takes 20 iterations to fill the missing information.
Finally, the 3-D inverse Fourier transform yields the 3-D RI distribution of the samples.
To accelerate the speed of the reconstructing diffraction tomogram, a graphics processing unit (GPU, GeForce GTX 680, nVidia Corp., Santa Clara, CA, USA) was utilized.First, the extracted optical fields were transferred to the GPU memory as shown in Fig. 1(c), and then all the data analyses were performed by the GPU using custom-made scripts in MatLab R2013a and the parallel computing toolbox (MathWorks Inc., Natick, MA, USA) on a desktop computer (Intel Core i5-2600 CPU, 3.4 GHz, 8 GB RAM).Recent version of the parallel computing toolbox in MatLab provides the Compute Unified Device Architecture (CUDA) based built-in functions such as fft, ifft and matrix calculations.The computation times for reconstructing a typical tomogram (96 × 96 × 96 voxels) takes 0.21 sec, which are 17 times faster than the previously reported calculation time for the diffraction tomogram used in a recent work [13].Table 1 summarizes the comparison of the computation time of tomogram reconstruction between a central processing unit (CPU) and the GPU for various numbers of voxels from 96 3 to 256 3 .The difference of computation time using CPU and GPU increases as the number of voxels increases because the tomogram reconstruction algorithm performs several Fourier transforms that can be significantly accelerated at a large scale using the GPU.Although the computational speed of tomogram reconstruction is considerably increased, there are inevitably additional time delays from the camera recording and communication.Thus, the final refreshing time of each tomogram, including the measurement, 3-D reconstruction, and visualization, was 1.3 sec for a sample with 96 3 voxels.While the present refreshing time of 1.3 sec is enough to visualize the slow dynamics of microscopic biological samples in real time, there is room for further enhancement of computation speed to further expand the applicability of the method.Parallel processing for measurement and reconstruction should be a direct solution.In this case, the reconstruction time will be same as the final refresh rate.In addition, the present method uses a CPU for 2-D unwrapping of phases, but the calculation speed can be faster using GPU [19].Furthermore, there are several algorithms for retrieving optical fields from interferograms without using 2-D Fourier transform [26,27], and those algorithms perform 4 to 10 times faster than Fourier transform-based methods.In addition, as an instrumentation issue, utilizing high-performance computing GPU (i.e., Tesla K20, nVidia Corp.) and a detector with high speed communication can further enhance the tomogram reconstruction speed.

Validation of sparse angle illumination for optical diffraction tomography
We first show that the reduced number of illumination angles provides acceptable tomogram reconstruction without shape distortion in 3-D RI maps.To systematically compare the difference between tomograms reconstructed from full illumination (500 angles) and sparse angle illumination (10 angles), we calculated the correlation coefficient of two 3-D RI tomograms of a healthy RBC reconstructed with the decreased number of illuminations and the full number of illuminations, respectively.Figure 2 shows that 10 views are reliable for tomogram reconstruction with a correlation coefficient of 0.95.RI distribution of slice images in the x-y plane also shows that the proposed tomogram using 10 views is comparable to the tomogram obtained with 500 illuminations.To further prove the validity of the reconstruction of diffraction tomograms with fewviews illumination, the RI tomograms of a polystyrene bead (Sigma-Aldrich, St. Louis, MO, USA, diameter = 3 μm) in immersion oil (n oil = 1.518 at λ = 532 nm) and a healthy RBC in a phosphate-buffered saline (PBS) solution (n PBS = 1.337 at λ = 532 nm) were measured.Figure 3 clearly shows that our method successfully reconstructed tomograms of the polystyrene bead and RBC with correct RI values (polystyrene bead: n = 1.599, healthy RBC in the oxygenated condition: n = 1.39-1.4at λ = 532 nm) [12,28].In addition, shape distortion in 3-D RI tomograms such as axial elongation due to the limited NA of the objective lens is diminished, which resulted from using the iterative non-negativity constraint method.The isosurfaces of the samples were also rendered for the purpose of visualization using reconstructed 3D RI distributions by commercial software (Amira 5, Visage Imaging Inc., San Diego, CA, USA).

Brownian motion of PMMA dimers
In order to demonstrate the capability of the present real-time reconstruction of optical diffraction tomograms, the diffusive motion of individual micron-sized non-spherical objects is visualized in real-time.Brownian motion is the random trajectory of a small particle immersed in a fluid, directed by random collisions of molecules.Brownian motion of spherical particles can be well described by defining a diffusion coefficient, D = k B T/γ, where k B is Boltzmann's constant, T is the temperature, and γ is the hydrodynamic friction coefficient related to the size of spherical particles and the viscous properties of the immersing fluid.In contrast, it is difficult to predict the diffusive motion of geometrically anisotropic particles such as dimers, trimers and most biological samples.The translational and rotational Brownian motion of anisotropic particles have been studied using bright-field microscopy for colloidal ellipsoids in 2-D confinement [29] and in-line digital holographic microscopy for dimers and triangular trimers in three-dimension [30,31].However, those techniques can be only applied to particles with known geometry.For example, in [30,31], the position and orientation of dimers and trimers are extracted from fitting digital holograms with Lorentz-Mie scattering theory for known samples.
Using the present approach, we visualized Brownian motion of polymethyl methacrylate (PMMA) dimers in real time.Diluted PMMA dimers were prepared in a similar way in [32].The major and minor axes of the dimer were 4.3 μm and 2.3 μm, respectively.Figure 4 and Media 1 show the Brownian motion of a PMMA dimer for 320 s.The position and the orientation of the dimer in each frame were calculated as the centroid and principal axes of the reconstructed tomogram, respectively.Our technique does not require a priori information about the optical and geometrical properties of the sample for measuring the position and orientation of the samples.Therefore, the translational and rotational diffusion coefficient of anisotropic particles in 3-D suspension can be directly measured from a series of reconstructed tomograms.
Typically, the speed of data acquisition for investigating Brownian motion of microscopic particles should be determined less than the time for the particles displaces an effective radius [33].Putting a radius for the minor axis of PMMA dimers, the diffusion coefficient of the PMMA dimer is 0.114 μm 2 /s, and corresponding time lag for a dimer to displace a radius is 1.45 sec, which is still longer than the refreshing rate of our technique, 1.3 sec.Therefore, it is possible to visualize the 3-D Brownian motion of an anisotropic particle in real time using our technique.Brownian dynamics of a micron-sized particle is often experimentally difficult to archive due to the limited scanning rate of confocal microscopy [34] that it has to adopt the solvent viscosification method to delay the motion of particles.Thus, real-time visualization by optical tomogram is promising technique to investigate fast dynamics of micron-sized colloids without any label.

Dynamics of RBC deformation in shear flow
Our method can also be applied to the study of dynamic biological samples.Here, an optical diffraction tomogram is used to visualize the shape change of RBCs under the accelerating or decelerating blood flow in real time.The customized microfluidic channel was fabricated by sandwiching two cover glasses, and an additional cover glass was placed between the sandwiched coverslips as a spacer.As a result, the height of the channel was 170 μm, and the width of the channel was 5 mm.The inlet and outlet of the channel were generated by drilling at the opposite ends of the channel, and syringe tips were attached to the two holes.First, we inserted healthy RBCs into the channel by a syringe pump (PHD Ultra Syringe Pump, Harvard Apparatus, Holliston, MA, USA), and then we waited for a few minutes for the RBCs to settle and stably sediment on the substrate.Due to electrostatic force between the RBC membrane and the glass, RBCs were weakly attached to the lower cover glass so that their positions became fixed independent of the slow flow [35].Then, the external flow rate was accelerated for 45 s to reach the maximum flow rate of 100 μl/min, and then decelerated for 45 s to a zero volume flow rate.
Figure 5 and Media 2 show the dynamic shape change of a RBC in response to the accelerating and decelerating fluid described above.Initially, when the shear flow was stationary, the RBC had a discoid shape (Fig. 5b).When the flow rate of the shear flow reached the maximum value, the RBC acquired a streamline shape, as shown in Fig. 5c.It can be clearly seen that a portion of the RBC membrane is attached to the lower cover glass while the center of the mass of the RBC is shifted along the shear flow direction ( + x-axis).When the flow rate was decreased to zero, the RBC recovered its original shape as a discocyte (Fig. 5d). Figure 5 suggests that it is possible to visualize the dynamics of the 3-D structure of biological samples in real time.For example, analysis of a series of real-time tomograms of RBCs may give more information on the studies of viscoelastic properties of RBCs in response to laminar flow [36] and small capillaries [37,38], which is now accessible to direct experimental study.Furthermore, the dynamic membrane fluctuation in RBCs has been only measured in 2-D projection geometry [39][40][41][42][43].The dynamic measurement of 3-D membrane fluctuation in RBCs can be investigated with the present approach, which would open up many unexplored studies in hematology and membrane biophysics.The present method, however, has some technical limitations.First of all, RI distribution does not provide molecular specificity to identify specific proteins in biological samples, while fluorescent imaging (i.e., confocal microscopy) can image different proteins by tagging various fluorescent dyes.Recently, spectroscopic phase microscopy has records phase images with different illumination wavelengths to measure RI dispersion of samples [44][45][46][47][48]. Since RI dispersion is sensitive to the type of proteins, tomogram reconstruction with different illumination wavelengths can identify 3-D distribution of various types of proteins label-free.In addition, imaging with a coherent light source, such as a laser, has inevitable speckle noise.Recently, researchers have tried to minimize coherent speckle noise using temporallyincoherent light for quantitative phase microscopy [49,50].Thus, reconstruction of a realtime tomogram with complex optical fields via incoherent light illumination is expected to reduce the speckle noise.

Conclusion
We presented and experimentally demonstrated a novel approach to measure and reconstruct 3-D RI tomograms in real time.To achieve this goal, optical fields are measured with sparse illumination angles and then reconstructed with the GPU implementation.The execution time for the tomographic reconstruction was 0.21 s for 96 3 voxels, which is 17 times faster than the conventional method.Including the measurement and visualization processes, we achieved a total refresh rate of 1.3 s.We performed the real-time visualization of Brownian motion of a PMMA dimer and investigated the shape change of a RBC in shear flow in real time.We expect the method can be used to investigate the dynamics of individual cells in response to chemical, electrical [51], and mechanical stimulus in real time.In addition, with the method one can investigate the 3-D dynamics of the structural and chemical information about individual cells associated with pathophysiological states, such as cell migration [52], cell growth [53,54], and malaria [15,55], in real time.Furthermore, this approach of using sparse illumination angles and then reconstructed with the GPU implementation can also be extended to other quantitative phase imaging techniques that synthesize multiple coherent images [56][57][58].

Fig. 1 .
Fig. 1.The schematic diagram of real-time tomography.(a) Experimental setup for illuminating samples from different angles.(b) Retrieved phase of a typical RBC from 10 different illumination angles.(c) Retrieved complex optical fields are transferred to GPU memory, and the 2-D Fourier transform of each complex field is mapped in 3-D Fourier space according to Fourier diffraction theory.(d) An iterative non-negative constraint processing method is implemented for filling the rest of 3-D space.(e) A 3-D inverse Fourier transform yields 3-D complex RI distribution of a RBC.The scale bar in (b) indicates 5 μm.

Fig. 2 .
Fig. 2. (a) Cross-correlation coefficient of 3-D RI tomograms reconstructed from 500 illuminations and a reduced number of illuminations.The inset shows the same data in log scale.(b) Cross-sectional images of the x-y plane of the reconstructed tomogram with the cross-correlation coefficients as a function of the number of illuminations used for the reconstruction.The red box indicates the tomogram reconstructed with 10 illuminations.

#Fig. 3 . 3 -
Fig. 3. 3-D rendering of the RI surfaces of (a) a polystyrene bead and (e) a RBC.For visualization purpose, the threshold for the iso-surfaces is set to the 70% of the maximum RI value.(b-d) cross-sectional slices of RI distribution of the polystyrene bead in the x-y, y-z and x-z planes, respectively.(f-h) cross-sectional slices of RI distribution of the RBC in the x-y, y-z and x-z planes, respectively.

Fig. 4 .
Fig. 4. The real-time measurement and visualization of the 3-D trajectory and orientation of Brownian motion of a PMMA dimer.Colored lines indicate orientation of the dimer in each tomogram.Each color corresponds to the position in the z-axis.Black dashed lines indicate the trajectory of dimer centroids.The dimer in the initial and final positions are indicated as a red and blue isosurface, respectively.Real-time visualization of the Brownian motion of the dimer is presented in Media 1.

Fig. 5 .
Fig. 5. Reconstructed 3-D RI tomogram for a RBC under a flow rate along the applied flow direction (a).The RBC in (b) stationary fluid, (c) accelerating flow rate with the maximum rate, and (d) decelerating fluid with zero rate.(left) 3D rendering of the RI surfaces of RBCs (three right panels) Optical phase delay of red blood cells in the x-y, x-z and y-z planes, respectively.The dashed lines indicate the initial shape of the RBC.Real-time visualization of the motion of the RBC under the shear flow is also presented in Media 2.