Multi-Physical Parameter Cross-Sectional Imaging of Quantitative Phase and Fluorescence by Integrated Multimodal Microscopy

Integrated multimodal cross-sectional or volumetric imaging techniques give us fruitful information to understand the behavior or status of target objects such as biological samples. Most of the reported systems for this purpose are either time consuming due to scanning or use additional reference beams such as in interferometry. Therefore, fast, simple, highly efficient, and powerful multimodal imaging systems that can perform cross-sectional imaging with simple algorithms are worth to be investigated. In this paper, a multimodal technique for cross-sectional quantitative phase and fluorescence imaging with computational microscopy is presented. We combine cross-sectional fluorescence and quantitative phase imaging by using the transport of intensity equation (TIE) and numerical wave propagation. The amplitude and phase of the fluorescence light wave with partially spatial coherence are obtained from three defocused intensity patterns. The proposed hybrid imaging system is simple, compact, and non-iterative. We present experimental results of microbeads and fluorescent protein-labeled living cells of the moss Physcomitrella patens to demonstrate the performance of the proposed imaging system.

and structures. In fluorescence imaging for biological applications, an excitation light beam generally illuminates the sample through an objective lens that is also used to detect the emission from the sample [1]- [13]. A band pass filter separates the light by wavelength so that the emitted light can be imaged without interference from the excitation light [1]- [13]. Most of the fluorescence imaging techniques such as laser scanning confocal microscopy, two-photon microscopy, and other related techniques require a scanning process to get three-dimensional (3D) information [1]- [10]. These techniques are time-consuming for obtaining the 3D features of objects. Fast imaging techniques are highly desirable in the field of fluorescence imaging for measuring dynamic behavior of cellular activity and cellular network. For more large-scale observation in living animals, computational tomography of fluorescence and luminescence can be used [14]- [16].
On the other hand, phase imaging techniques have been used for obtaining structural information by exploiting optical path-length shifts through the specimen of interest [17]. Several techniques such as optical interferometry [18], single-pixel imaging and wavefront sensing [19], [20], ptychography [21] and transport of intensity equation imaging (TIE) [22]- [28], have been utilized for the visualization of phase information. Most of these techniques have been combined with fluorescence imaging to get additional functional information of the biological sample to be imaged [29]- [37].
When two or more physical parameters of a specimen are imaged together, the technique is usually named multimodal imaging [32]- [39]. Over the past decade, multi-modal imaging techniques that use phase imaging in combination with fluorescence imaging have been developed considerably due to several advantages [32]- [37]. In most of the multimodal imaging techniques, two or more setups are combined lowering the number of components, which can reduce the cost of the system up to some extent [32]- [39]. However, it enhances the complexity of the system. Besides, interferometric or scanning based configuration compromise the light efficiency or the temporal resolution of the imaging system, respectively. Therefore, imaging techniques that can perform two or more separate imaging modalities, by using a single optical setup to avoid complexity of the system, need to be investigated. Especially, investigations on multi-modal imaging systems to visualize and delineate the structural and functional information in the biological specimen on a single platform at low light condition with simultaneous fast recording are in high demand. Furthermore, obtaining both features with simple and innovative 3D methods are at utmost demand for biological applications.
In this work, we propose a multimodal cross-sectional imaging of quantitative phase and fluorescence in an integrated system. Here, cross-sectional observation of both features are obtained by using the TIE algorithm and numerical refocusing using Fresnel propagation. The proposed system is simple and highly efficient due to non-involvement of any reference beam, in contrast with interferometric based multimodal imaging techniques. Our imaging technique also has the capability to obtain 3D features like holographic multimodal systems at lower cost (due to the use of inexpensive components) and with reduced light requirements (as it uses a single light beam technique). The performance of the proposed multimodal system is verified by conducting various experiments on fluorescent microspheres and fluorescent protein-labeled living cells of the moss Physcomitrella patens (Physcomitrella). In the case of fluorescence imaging, it is assumed that some degree of spatial coherence is available after the propagation with sufficient distance when the fluorescent objects are small enough and sparsely distributed.

II. PROPOSED 3D IMAGING SYSTEM
The schematic diagram of the proposed imaging system, performing simultaneously quantitative phase and fluorescence imaging in arbitrary cross-sectional planes, is shown in Fig. 1. Two light sources and two image sensors are used. We note that it is possible to use a single color image sensor to capture two light waves with different wavelengths. However, in the plant cell experiment, a high-sensitive image sensor is required to capture the weak fluorescence light to avoid optical damage of the plant cell. For fluorescence, a blue light-emitting diode (LED), denoted LED1 in Fig. 1, is used to excite the sample and another LED (LED2) is used for phase imaging. The light from LED1 is reflected from the dichroic mirror (DM1), which allows the light to enter the microscope objective lens (MO) to excite the fluorescent objects under study. The fluorescent light emitted by the objects passes through DM1 and then through the 4f system after passing through a tube lens (TL). The 4f imaging system also includes an electrically tunable lens (ETL) that is inserted in the focal plane between the two lenses. It should be noticed that the use of an ETL in this configuration allows us the recording of the stacks of images with slightly different focus without changing the magnification. A bandpass/wavelength filter (WF) is used to selectively record fluorescent light with the image sensor after passing through another dichroic mirror DM2. For phase imaging, LED2 is used to illuminate the sample. In this case, the light passes through the same components as in the case of fluorescence up to DM2, minimizing the requirement of components unlike other multimodal systems performing interferometric or holography based multimodal imaging. Finally, the reflected light from DM2 is recorded for phase imaging by another image sensor.
Firstly, we would like to discuss briefly the principle of TIE imaging and then how it can be used for 3D phase and fluorescence visualization. Basically, the TIE describes the relationship between the derivative of intensity with respect to the optical axis and the phase at the image plane. It can be written as follows [22]- [28] − 2π λ where (x, y) are the spatial coordinates perpendicular to the optical axis, λ is the wavelength, I z (x, y) is the in-focus intensity distribution, and φ z (x, y) is the phase distribution at the same plane. The symbol denotes the 2D gradient operator and z indicates the positions along the optical axis of the lateral plane (x, y).
To solve the TIE for obtaining the phase distribution, φ 0 (x, y) [22]- [31], the following equation is used: Here, (u, v) represents the coordinates in the Fourier transform (FT) domain. Now, we assume that the object is located at a distance z along the optical axis. The intensity distribution, I 0 (x,y) can be recorded by the optical setup as shown in Fig. 1 with the image sensor located at axial position z = 0, that is, at an out-of-focus plane. The intensity derivative used in (2) is approximated using the difference of two additional intensity distributions, I Δz (x, y) and I -Δz (x, y), obtained at different axial positions shifted Δz and -Δz. The relation between the derivative and the axially shifted intensity distributions can be written as- Therefore, the retrieved phase distribution is obtained from (2) by using three intensity distributions shifted along the axial direction around z = 0. The complex amplitude distribution for the fluorescence and phase imaging, H(x, y), is constructed by combining the phase distribution φ 0 (x, y), obtained by TIE and the corresponding amplitude distribution I 0 (x, y). The retrieved complex amplitude distribution is Fresnel propagated to get the focused input image. The propagation distance gives us the depth information. The free space Fresnel propagation of the complex amplitude distribution, H(x, y), is implemented by using the following equation: The free space Fresnel propagation can be calculated numerically using a fast Fourier transform based algorithm [40].
The optical setup for simultaneous visualization of phase and fluorescence features is shown in Fig. 1. In both cases, three defocus intensity images are recorded and used to apply the TIE algorithm. The same procedure is followed to obtain the complex amplitude, as described through (1)(2)(3)(4), for the cases of fluorescence and phase modalities. For the case of phase imaging, the phase is retrieved by calculating the angle of the propagated complex distribution in (4), as follows- It should be noted that the retrieved phase is wrapped continuously between 0 and 2π and, therefore, a phase unwrapping process is required. The process of phase unwrapping involves the retrieval of the original (unwrapped) phase starting from the corresponding restricted (wrapped) phase in the (−π, π or 0, 2π) interval. For the purpose of quantitative phase imaging, we remove the 2π phase discontinuity by applying the Goldstein phase unwrapping algorithm [41,42]. The phase distribution can be obtained at planes located at different axial distances z on the sample to get quantitative 3D phase information.
For the case of cross-sectional fluorescence imaging, the complex wave function obtained applying the TIE algorithm to the fluorescence intensity distribution is further Fresnel propagated to get intensity information at different depths. The retrieved intensity distribution, I r (ξ, η) can be obtained as follows- Here, it is straightforward to reconstruct the different intensity images at planes located at different z distances to get crosssectional distributions of the sample of interest.
It is well known that the light emitted from fluorescence objects is incoherent. However, in this study, our observation target is constituted by fluorescent protein-labeled nucleus in living cells or fluorescence bead, which is assumed as a collection of quasi-point like fluorescent light sources due to its size of the order of 10 µm. Further, a band-pass filter is used to increase the temporal coherence. Hence, we can consider that the light emitted by the fluorescence objects, after propagation through the optical system is partially coherent both spatially and temporally. Therefore, it is possible to measure the phase distribution associated to this partially coherent light proceeding from fluorescence of small objects [29], [31]. If the light source has points located at different distances, our technique based on TIE is able to detect the defocusing phase and, therefore, to reconstruct the light source at the original distance. This is also similar to other conventional 3D imaging techniques with incoherent light such as confocal imaging or integral imaging. In our system, the measured phase is related with the defocus transfer function, which will be different for fluorescent light sources located at different axial distances. By measuring the phase, we are measuring the transfer function of the optical system, and therefore we are able to focus to different distances.

III. EXPERIMENTAL RESULTS
To perform the experiment for the demonstration of simultaneous cross-sectional fluorescence and quantitative phase imaging, the optical setup shown in Fig. 1 is used. A blue LED of 470 nm wavelength is used as excitation light source. The MO used in the experiment has a numerical aperture of 0.4 and a magnification of 20x, adapted for imaging living plant cells and microbeads of 10.4 μm in diameter. The light emitted by the microbeads has a spectral bandwidth with wavelengths ranging from 450 nm to 600 nm. The strongest central wavelength emitted from the yellow fluorescent protein Citrine [43], [44] that labels nuclei in living cells is 529 nm. A band-pass/wavelength filter centered at 580 nm with a bandwidth of 10 nm is used to pass only fluorescence light in case of beads. On the other hand, for the imaging of fluorescent protein-labelled plant cells, a band pass filter of 525 nm with bandwidth of 39 nm is used.
For phase imaging, a LED with a wavelength of 560 nm is used as light source. And a band-pass/wavelength filter centered at 550 ± 10 nm is used to block or reduce light from other sources. We have to note that part of fluorescence light, especially that arising from beads, also pass through the filter and reach the image sensor for phase imaging. However, the fluorescence light is much weaker than the transmitted light for phase imaging and does not affect the phase imaging technique. In the case of bead imaging, the size of the image sensor is 1500 × 1500 pixels for fluorescence and phase, with a pixel pitch of 6.5 μm, while in the case of plant-cell imaging, the size of the image sensor is 700 × 700 pixels for fluorescence with a pixel pitch of 3.45 μm and 920 × 920 pixels for phase with a pixel pitch of 4.54 μm. In both cases, three defocus intensity images are recorded by changing the optical power of the ETL (EL-16-40-TC by Optotune). The focal length of the lenses used for phase and fluorescence illumination is 100 mm. While the focal length of the lenses used in the 4f system is 150 mm.  Figs. 2(a-c). The extracted phase distribution obtained from these defocus images by the TIE algorithm is shown in Fig. 2(d) and the corresponding surface plot is shown in Fig. 2(e). The phase change (high phase value) can be noticed for all the beads falling under the field of view of the imaging system. This phase distribution and the corresponding intensity distribution constitute the complex distribution of propagated optical wave at a particular distance. Then, the intensity distributions at different distances are reconstructed numerically by propagating the complex amplitude distributions. Here, we present the reconstructed intensity images in two planes. The reconstruction distances are -33 mm and 32 mm from the defocus image, which correspond to -82.5 μm and 80 μm in the sample domain, respectively. The reconstruction results are shown in Figs. 3(a-f). The reconstructed fluorescence image for the fluorescent beads of selected area in focus, is shown in Fig. 3(a). Fig. 3(c) is the reconstructed fluorescence image for the second plane, where another selected area is in focus. Fluorescence images are drastically changed by numerical refocusing. For comparison, in Figs. 3(b) and (d) we show the images recorded experimentally by focusing the microscope directly onto the same axial planes considered in  Fig. 3(e) for the first focused plane and in Fig. 3(f) for the second plane. Another comparison in terms of cross-correlation (CC) has also been carried out whose results are shown in Fig. 3(g) and (h). Firstly, we confirm that the autocorrelation of the selected region in Fig. 3(a) has a correlation peak value equal to 1, as it can be seen in the autocorrelation plot in Fig. 3(g). Second, we verify that the CC peak value is only slightly reduced to 0.93 when the correlation of the selected region is calculated between the image retrieved computationally, in Fig. 3(a), and that obtained optically, in Fig. 3(b). The correlation plot is shown in Fig. 3(h). From these results, it can be seen that fluorescence images at different distances can be reconstructed from defocus fluorescence images.
For phase imaging, the recorded three out of focus intensity images in transmission mode are shown in Figs. 4(a-c). The recording interval of axial distances for the out of focus images is 2 μm. The output phase image obtained from these defocus images using the TIE algorithm is shown in Fig. 4(d) and the  Fig. 2(a), (f) comparison of phase profiles of images focused optically vs computationally corresponding to the selected area in Fig. 2(c). corresponding quantitative phase distribution (surface plot) is shown in Fig. 4(e). The complex function obtained by multiplying the phase distribution in Fig. 4(d) with the defocus image in Fig. 4(b) is Fresnel propagated to reconstruct the desired phase distributions in different planes. We choose to reconstruct the phase in planes located at two different distances with different fluorescent beads in focus. The reconstruction distances are -12 mm and 55 mm from the defocus image, which correspond to -30 μm and 137.5 μm in the sample domain, respectively. The reconstruction results are shown in Figs. 5(a-d). The reconstructed phase distribution for the first plane, where the fluorescent bead of the selected area is in focus, is shown in Fig. 5(a) and the corresponding phase distribution (surface plot) is shown in Fig. 5(b). Fig. 5(c) is the phase distribution for the second plane, where the fluorescent bead of the selected area is in focus, while the corresponding phase distribution (surface plot) is shown in Fig. 5(d). We have obtained also the phase distribution of these images focused optically in order to compare quantitatively the result obtained with both methods. The result of the comparison between profiles of the phase images obtained by computational and optical focusing are shown in Figs. 5(e) and (f). Fig. 5(e) shows the comparison of phase profiles corresponding to the selected area in Fig. 5(a) while Fig. 5(f) shows the comparison for the selected area in Fig. 5(c). From these results, it can be seen that quantitative phase images can be obtained at different axial distances starting from a defocused image obtained by TIE algorithms. Hence, the proposed system can successfully retrieve the multimodal 3D fluorescence and phase information simultaneously.
Another experiment is also carried out for the simultaneous quantitative phase and cross-sectional fluorescence imaging of living cells. Physcomitrella is chosen as an observation target [43], [45], where the Citrine Yellow fluorescent protein gene [44] was inserted into a histone H3.3 locus (Pp3c18_14481) to label only nuclei [46]. Physcomitrella has clear cell identity and relatively simple body structure [43], both of which are beneficial for molecular and developmental biological studies using imaging.
The results of plant-cell imaging are shown in Figs. 6 to 8. The results of cross-sectional quantitative phase imaging are shown in Figs. 6 and 7 while cross-sectional fluorescence imaging results are shown in Fig. 8. Figs 6(a-c) show the three defocus images of the plant cells recorded with the image sensor for phase imaging in the multimodal microscope shown in Fig. 1. Fig. 6(d) shows the quantitative phase distribution obtained from the defocus images in Figs. 6(a-c) by applying the TIE algorithm. The 3D profile of the phase distribution is shown in Fig. 6(e). Fig. 7(a) shows the quantitative phase distribution in a first plane, located in the upper part of the sample, obtained after Fresnel propagation of the complex amplitude distribution. Fig. 7(b) shows the quantitative 3D profile of the phase distribution. The phase shifts are in the range of 5.5 radians and, therefore, we can estimate that the thickness of the cell is about 18.47 µm, assuming that the refractive index of the plant cell is 1.36 [47], [48]. The obtained thickness is in the appropriate range of the actual thickness of the plant cell [43]. Fig. 7(c) shows the phase distribution in a second plane, located on the lower part of the sample, obtained after Fresnel propagation to the proper distance. Fig. 7(d) is the quantitative 3D phase profile of the lower area marked in Fig. 7(c). Movie data of reconstructed cross-sectional phase images is available (Visualization 1). The obtained reconstruction distances, for the first plane (upper part) and the second plane (lower part) are -3.6 mm and 2.9 mm, respectively, from the central out of focus image shown in Fig. 6(b).
Figs 8(a-c) are the three defocused images of plant cells recorded in fluorescence mode by the image sensor in Fig. 1. Fig. 8(d) shows the phase distribution obtained from the previous defocus images by the TIE algorithm. Fig. 8(e) shows the 3D profile of the quantitative phase distribution. Figs 8(f-h) show the recovered fluorescence images reconstructed at planes located at distances of -17 mm, -10 mm, and 6 mm, respectively, from the central defocused image. Movie data of reconstructed cross-sectional fluorescence images is available (Visualization 2). From these results obtained for plant cells, it can be claimed that the proposed multimodal imaging system is not only suitable for imaging standard objects but also for imaging biological samples in modern biology.

IV. CONCLUSION
In the presented work, we have proposed a noninterferometric multimodal computational imaging system for simultaneous quantitative phase and cross-sectionalfluorescence imaging that enables to visualize structural and functional features of biological samples. The proposed multimodal computational microscopy system is based on the TIE phase retrieval algorithm and Fresnel propagation. The 3D features of phase and fluorescence information are reconstructed by free space propagation from the complex amplitude distribution obtained previously with the help of the TIE algorithm. Both features of a sample in terms of fluorescence and phase images can be recorded and visualised simultaneously by using the same optical setup and by the same algorithm. The proposed imaging system is simple and shows a high efficiency, unlike holographic or interferometric imaging systems, and does not use iterative algorithms. We have presented experimental multimodal images of microbeads and fluorescent protein-labeled living cells of Physcomitrella. From the presented results, it can be claimed that the proposed multimodal imaging system works successfully, providing simultaneous 3D fluorescence and phase images of a sample.
We hope that the strategies used in our proposed work will allow estimating phase from intensity to get crosssectional/volumetric information in different experiments of advanced fluorescence imaging and other incoherent imaging methods.
Conflicts of interest:The authors declare no conflicts of interest.