Improving spatial resolution of confocal Raman microscopy by super-resolution image restoration

A new super-resolution image restoration confocal Raman microscopy method (SRIR-RAMAN) is proposed for improving the spatial resolution of confocal Raman microscopy. This method can recover the lost high spatial frequency of the confocal Raman microscopy by using Poisson-MAP super-resolution imaging restoration, thereby improving the spatial resolution of confocal Raman microscopy and realizing its super-resolution imaging. Simulation analyses and experimental results indicate that the spatial resolution of SRIR-RAMAN can be improved by 65% to achieve 200 nm with the same confocal Raman microscopy system. This method can provide a new tool for high spatial resolution micro-probe structure detection in physical chemistry, materials science, biomedical science and other areas. ©2016 Optical Society of America OCIS codes: (170.5660) Raman spectroscopy; (300.6320) Spectroscopy, high-resolution; (300.6450) Spectroscopy, Raman; (100.3020) Image reconstruction-restoration; (100.6640) Superresolution. References and links 1. S. Sil and S. Umapathy, “Raman spectroscopy explores molecular structural signatures of hidden materials in depth: Universal Multiple Angle Raman Spectroscopy,” Sci. Rep. 4, 5308–5314 (2014). 2. C. Neumann, S. Reichardt, P. Venezuela, M. Drögeler, L. Banszerus, M. Schmitz, K. Watanabe, T. Taniguchi, F. Mauri, B. Beschoten, S. V. Rotkin, and C. Stampfer, “Raman spectroscopy as probe of nanometre-scale strain variations in graphene,” Nat. Commun. 6, 8429 (2015). 3. J. J. Schaefer, A. C. Crawford, M. D. Porter, and J. M. Harris, “Confocal Raman Microscopy for Investigating Synthesis and Characterization of Individual Optically Trapped Vinyl-Polymerized Surfactant Particles,” Appl. Spectrosc. 68(6), 633–641 (2014). 4. X. H. Xianwei Hu, Z. Y. Zongxin Yu, B. G. Bingliang Gao, Z. S. Zhongning Shi, J. Y. Jiangyu Yu, and Z. W. Zhaowen Wang, “Equilibrium between NO3-and NO2-in KNO3-NaNO2 melts: a Raman spectra study,” Chin. Opt. Lett. 12(9), 093001 (2014). 5. J. C. C. Day and N. Stone, “A subcutaneous Raman needle probe,” Appl. Spectrosc. 67(3), 349–354 (2013). 6. M. J. Banholzer, L. Qin, J. E. Millstone, K. D. Osberg, and C. A. Mirkin, “On-wire lithography: synthesis, encoding and biological applications,” Nat. Protoc. 4(6), 838–848 (2009). 7. N. Gierlinger, T. Keplinger, and M. Harrington, “Imaging of plant cell walls by confocal Raman microscopy,” Nat. Protoc. 7(9), 1694–1708 (2012). 8. N. Everall, J. Lapham, F. Adar, A. Whitley, E. Lee, and S. Mamedov, “Optimizing depth resolution in confocal Raman microscopy: a comparison of metallurgical, dry corrected, and oil immersion objectives,” Appl. Spectrosc. 61(3), 251–259 (2007). 9. K. Watanabe, A. F. Palonpon, N. I. Smith, L. D. Chiu, A. Kasai, H. Hashimoto, S. Kawata, and K. Fujita, “Structured line illumination Raman microscopy,” Nat. Commun. 6, 10095 (2015). 10. J. Lu, W. Min, J. A. Conchello, X. S. Xie, and J. W. Lichtman, “Super-Resolution Laser Scanning Microscopy through Spatiotemporal Modulation,” Nano Lett. 9(11), 3883–3889 (2009). 11. T. Ichimura, S. Fujii, P. Verma, T. Yano, Y. Inouye, and S. Kawata, “Subnanometric near-field Raman investigation in the vicinity of a metallic nanostructure,” Phys. Rev. Lett. 102(18), 186101 (2009). #260084 Received 26 Feb 2016; revised 29 Apr 2016; accepted 2 May 2016; published 9 May 2016 © 2016 OSA 16 May 2016 | Vol. 24, No. 10 | DOI:10.1364/OE.24.010767 | OPTICS EXPRESS 10767 12. R. Zhang, Y. Zhang, Z. C. Dong, S. Jiang, C. Zhang, L. G. Chen, L. Zhang, Y. Liao, J. Aizpurua, Y. Luo, J. L. Yang, and J. G. Hou, “Chemical mapping of a single molecule by plasmon-enhanced Raman scattering,” Nature 498(7452), 82–86 (2013). 13. J. L. Harris, “Diffraction and Resolving Power,” J. Opt. Soc. Am. 54(7), 931–936 (1964). 14. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill,1968). 15. A. Govil, D. M. Pallister, and M. D. Morris, “Three-dimensional digital confocal Raman microscopy,” Appl. Spectrosc. 47(1), 75–79 (1993). 16. J. P. Tomba, J. M. Carella, and J. M. Pastor, “Interphase evolution in polymer films by confocal Raman microspectroscopy,” Appl. Spectrosc. 60(2), 115–121 (2006). 17. L. Zhang, X. Feng, W. Zhang, and X. M. Liu, “Improving spatial resolution in fiber Raman distributed temperature sensor by using deconvolution algorithm,” Chin. Opt. Lett. 7(7), 560–563 (2009). 18. M. Offroy, M. Moreau, S. Sobanska, P. Milanfar, and L. Duponchel, “Pushing back the limits of Raman imaging by coupling super-resolution and chemometrics for aerosols characterization,” Sci. Rep. 5, 12303–12316 (2015). 19. B. H. Su and W. Q. Jin, “POCS-MPMAP based super-resolution image restoration,” Guangzi Xuebao 32(4), 502–504 (2009). 20. G. M. P. van Kempen, H. T. M. van der Voort, J. G. J. Bauman, and K. C. Strasters, “Comparing maximum likelihood estimation and constrained Tikhonov-Miller restoration,” IEEE. Eng. Med. Biol. 15(1), 76–83 (1996). 21. M. Gu, Principles of Three-Dimensional Imaging in Confocal Microscopes (World Scientific Publishing Co. Pte. Ltd., 1996). 22. R. Dosselmann and X. D. Yang, “Existing and emerging image quality metrics,” in Proceedings of IEEE Conference on Electrical and Computer Engineering (IEEE, 2005), pp. 1906–1913. 23. Medda, and V. DeBrunner, “Color image quality index based on the UIQI. Image Analysis and Interpretation,” in Proceedings of IEEE Conference on Image Analysis and Interpretation (IEEE, 2006), pp. 213–217.


Introduction
Confocal Raman microscopy (CRM) is widely used as an important tool in frontier areas such as physical chemistry, materials science, biomedical science [1][2][3][4][5][6][7], because of its micro-spectral optical section detection capability.However, the pinhole size of conventional confocal Raman microscopy has to be a few hundred microns to ensure the sensitivity of micro-spectrum detection, so it restricts the improvement of CRM spatial resolution.Therefore, CRM still keeps the micron or sub-micron spatial resolution since it invented and it has impeded its further application in high spatial resolution micro-spectrum detection fields.In fact, how to improve the CRM spatial resolution has attracted much interest in Raman spectroscopy research.
Several methods to improve the CRM spatial resolution have been proposed [8][9][10][11][12].In far-field CRM, compressing the focused spot is the most frequently used method to enhance spatial resolution.In [8], N. Everall et al. used a large numerical-aperture oil immersion objective to increase the numerical aperture and improved the CRM spatial resolution by 35%.In [9], K. Watanabe et al adopted structured line illumination Raman microscopy to modulate high frequency, and then reached the theoretical limitation of confocal microscopy spatial resolution.In [10], J. Lu et al used spatiotemporal modulation microscopy to shift the out-of-band specimen frequency into the pass-band of the microscope, and achieved spatial resolution improvement by a factor of 2 over conventional wide field microscopy.In the above methods, CRM spatial resolution is improved significantly, however, the excitation energy of the focused spot is lost and thus CRM spectrum detection sensitivity is reduced with the improvement of spatial resolution.In near-field CRM, tip-enhanced Raman spectroscopy (TERS) is the often used method to enhance CRM spatial resolution.In [11], T. Ichimura et al used TERS to improve the spatial resolution to 3 nm.In [12], R. Zhang et al used plasmon-enhanced Raman scattering to spectrally match the resonance of the nanocavity plasmon to the molecular vibronic transitions, and achieved spatial resolution of approximately 0.5 nm.In general, TERS makes the spatial resolution of Raman spectroscopy reach at the nanometer level, but it requires that the distance between the AFM tip and specimen to be at nanometer scale during image processing.So, it reduces the detection efficiency and cannot realize tomography imaging, with the result that the requirements for the spectrum detection of internal information in biology, materials science and other fields could not be met.Therefore, in the far-field region, improving the CRM spatial resolution without reducing spectrum detection sensitivity is attracting more attention, and the appearance of super-resolution image restoration technology provides a new approach.
The super-resolution image restoration method was first reported by Harris and Goodman [13,14], which was based on the non-negativity and boundedness of an optical imaging system, and realized frequency extrapolation by solving the linear equations based on the sampling theorem, and then recovered the missing high frequency of the specimen, that is, achieved super-resolution image restoration.However, in super-resolution image restoration processing, some factors including image noise and error would result in bad convergence and low restoration accuracy.Therefore, many super-resolution image restoration algorithms have been proposed [15][16][17][18] to achieve good convergence and restoration effect.In [15], A. Govil et al increased the CRM spatial resolution by using the Van Cittert iterative deconvolution algorithm.In [16], J. P. Tomba et al used a regularized deconvolution algorithm to improve the CRM spatial resolution by 33%.In [17], L. Zhang et al used frequency-domain deconvolution algorithm (called ASFDA) to improve the CRM spatial resolution by 50%.In [18], M. Offroy et al used the super-resolution concept and the multivariate curve resolution (MCR-ALS) algorithm to realize super-resolution restoration with several confocal Raman images, and eventually improved the CRM spatial resolution by 65%.The above-mentioned methods increase the CRM spatial resolution without changing the optical imaging system.However, neither the capability of noise immunity and convergence speed nor the sub-pixel registration among image sequences is satisfied simultaneously while it improves the spatial resolution.Therefore, the quality of image restoration reduces.
To meet the urgent needs of improving the CRM spatial resolution, we propose a super-resolution image restoration confocal Raman microscopy method (SRIR-RAMAN) with high spatial resolution in this paper, which combines the super-resolution image restoration technology and CRM.The method uses a Maximum a posterior algorithm based on the Poisson-Markov model (MPMAP) [19] to increase the capability of noise immunity and the convergence speed, and conducts super-resolution image restoration on single Raman imaging at the same time, and thereby achieving high CRM spatial resolution imaging without changing the optical imaging system or reducing spectrum detection sensitivity.

SRIR-RAMAN theory
CRM is essentially a low-pass filtering optics, which cuts off the high spatial frequency during the image processing and then forms a degraded image.Based on non-negativity and boundedness of CRM [20], SRIR-RAMAN could realize high spatial resolution Raman imaging by recovering the lost high frequency information of degraded image through nonlinear operations.This method includes three steps: 1) A degraded image is obtained by the CRM system.2) The degraded image is restored to obtain the super-resolution image by using a super-resolution image restoration algorithm.3) The restored image is evaluated according to image quality index.

Degradation model of CRM
A degraded image of CRM is obtained by the CRM system shown in Fig. 1.In this system, a laser beam is focused onto a sample, and the reflected light and Raman light are typically collected by a microscope objective along the original optical path.Raman light is transmitted through a Notch Filter, focused on a pinhole by a lens and finally detected by a spectrometer.CRM system is assumed to be an ideal non-coherent imaging system, the point spread function (PSF) of the CRM system is given by Eq. ( 1) [21].
where h 1 is the PSF of the microscope objective, h 2 is the PSF of the lens, x 2 , y 2 are the coordinates in image space, and D(x 2 , y 2 ) is detector intensity function.
A confocal Raman image can be obtained from Eq. ( 1) and the degradation model of CRM; the result is shown in Eq. ( 2).
( ) , where O f (x, y) is the object function, ⊗ 3 denotes the 3-D convolution operation, and n(x,y) is system noise.

Theory of super-resolution image restoration
In order to recover the high frequency information over the cut-off frequency and improve the noise immunity of the algorithm, the MPMAP super-resolution algorithm [19] is used to restore the confocal Raman image.The confocal Raman image obtained by Eq. ( 2) could be restored by using iterative kernel shown in Eq. ( 3).
( ) where * denotes a correlation operator, g is the degraded image, f is the target image, h is the system PSF, β is a step factor which controls the convergence behavior and speed of the algorithm, and α is the balance factor, i and j refer to pixel index.The energy function U (f) is shown in Eq. (4).
, ( , ) ( , ) where d is the image smoothness parameter, c is the clusters of Poisson-Markov model, γ is the shape control parameter, k and l refer to pixel index, ∅ (.) is the penalty function.The Geman-McClure model, which can effectively reduce or eliminate noise and oscillation stripes, is used in the penalty function shown in Eq. ( 5).
( ) In CRM system, the degraded image g and the system PSF h in Eq. ( 3) can be expressed as Eqs.( 6) and ( 7).

( , )
i h h x y = (7) Besides, the balance factor is α = 0.1 and the control parameter of the shape is γ = 10.The restored image could be acquired by restoring the degraded image according to Eqs.

Image quality index
The super-resolution restoration capability is usually measured by an objective image quality index, including the Mean Square Error (MSE), the peak signal-to-noise ratio (PSNR), the improvement signal to noise rate (ISNR), or the frequency spectrum correlation coefficients r [22,23], and they are defined as Eqs.( 9) and ( 10).
( ) ( ) where N × M is image size, f (i, j) is the target image, ( ) , f i j ∧ is the restored image, and g is the degraded image.
where m = (M−1)/2 represents the size of the relevant sub-area of M × M.

Noise impact analysis
The CRM main noise sources include optoelectronic noise and electronic thermal noise.Optoelectronic noise is caused by the statistical properties of light and the photoelectric #260084 conversion of the photodetector, and electronic thermal noise is caused by the electronic random thermal motion during electronic processing such as A/D conversion and amplification of electrical signal.The target image is the picture-lena with 512 × 512 pixels and 256 grayscale.After being blurred by the PSF of the CRM system, noises with Gaussian distribution are introduced.As shown in Fig. 2, the degraded image I d is obtained, and the restored image I R is acquired from the degraded image after the super-resolution image restoration processing, and the frequency spectrum correlation coefficients r of restored image and target image can be acquired according to Eq. (10).
It can be seen from Fig. 2 that when the signal-to-noise ratio (SNR) is high (SNR > 40 dB), the restored image and the target image have a high correlation spectrum, and low frequency information can be fully recovered, high frequency information can be recovered significantly, and there are no oscillation stripes in the restored image.When the SNR is low, a relatively good restoration can be also obtained.So an ideal restoration can be obtained in the CRM system by SRIR-RAMAN.By using Eq. ( 9), the image quality of Fig. 2 can be evaluated, which shows a comparison between the degraded image and the restored image in terms of the MSE, PSNR, and ISNR, as shown in Table 1.When the noise is low, the restored image has a low MSE, and high PSNR and ISNR.These show that the restored image is closer to the target image than the degraded image.When the noise is high, the restored image quality declines, but a good restoration still can be obtained.It can be seen from Fig. 2 and Table 1 that SRIR-RAMAN can obtain a significant restoration and effectively suppress noise in the CRM system, and it can acquire an ideal restoration when SNR > 40 dB.

Spatial resolution analysis
We use a vertical bar as a target to verify that SRIR-RAMAN can improve the spatial resolution of the CRM.The theoretical spatial resolution of CRM is 272 nm [18] obtained with the 532 nm wavelength laser and an objective (NA = 0.9).Therefore, in the simulation, the target image is a vertical bar with 250 nm period, and the CRM is equipped with an excitation wavelength of 532 nm, an objective (NA = 0.9), SNR > 40 dB, and the image size is 64 × 64 pixels, with a step size of 10 nm.It can be seen from Fig. 3 that the restored image [Fig.3(c)] obtained after super-resolution image restoration has clear edges and obvious contrast improvement of the image compared with the degraded image [Fig.3(b)], and the shape of the restored image is almost the same as the target image [Fig.3(a)].As shown in Fig. 3(d), the horizontal distance between two identification points of the degraded image is D mn = 120 nm, while that of the restored image is D m'n' = 20 nm.So, the edge of restored image is narrower and steeper than that of degraded image, i.e., the restored image is closer to square wave than the degraded image, and a notable super-resolution restoration is realized by SRIR-RAMAN in the CRM system.

Materials and instrumentation
As shown in Fig. 4, we use a sample produced by electron beam lithography to verify the proposed method, which is a Poly methyl methacrylate (PMMA) sub-micronic pattern coated on Si wafer.Figure 4(a) is sample morphology acquired by confocal microscope (LEXT OLS4000, Olympus).Figure 4(b) shows a vertical bar with different period between 0 and 1000 nm. Figure 4(c) is the design image of the vertical bar, and Fig. 4(d) is Raman spectrum of point A in Fig. 4(c).As shown in Fig. 4(d), the Raman peak of Si is 520.7 cm −1 , which is used to construct the Raman images in the following section.Based on the diagram shown in Fig. 1, we construct a CRM system as shown in Fig. 5.A CW DPSS laser (532 nm, Cobolt, Sweden) is used as a light source for all the experiments.A spectrometer coupled with a homemade confocal microscope equipped with a 100 × objective (NA = 0.9, Olympus) is used for detecting Raman spectrum, and the sample is placed on an XY piezo motorized stage from PI (Physik Instrument).

Results and discussion
Vertical bars with periods 520 nm and 200 nm shown in Fig. 4 are placed on the homemade CRM system shown in Fig. 5 to obtain Raman images, and the results are shown in Figs. 6 and  7.The mapping size is 30 × 60 pixels, with a step size of 25 nm and 10 nm respectively, and then every pixel in Fig. 6 and Fig. 7 represent 25 nm and 10 nm respectively.
Figure 6 shows that the degraded image shown in Fig. 6(a) has edge blur and low SNR, while the restored image shown in Fig. 6(b) has sharp edges and a substantial increase in SNR.As shown in Fig. 6(d), the horizontal distance between two identification points of the degraded image is D mn = 250 nm, while that of the restored image is D m'n' = 50 nm.So, the edge of restored image is narrower and steeper than that of degraded image.In short, the spatial resolution is improved after super-resolution image restoration processing, and the restored image can better describe the sample topography and be closer to the target image.Figure 6(c) shows the three-dimensional restored image.By using Eq. ( 9), the image quality of the restored image in Fig. 6 can be evaluated, and the results are shown in Table 2.And it can be seen from Table 2 that the restored image has a decreased MSE and an increased PSNR compared with the degrade image, and the high ISNR indicates that the restored image is closer to the target image than the degraded image.Both Fig. 6 and Table 2 demonstrate that a better Raman image can be obtained and the spatial resolution can be improved by super-resolution image restoration processing.It can be seen from Fig. 7 that, a conventional CRM system shown in Fig. 7(a) cannot distinguish vertical bars with period 200 nm, while the sample topography are probably get from the restored image shown in Fig. 7(b), and both SNR and edge sharpness of restored image have increased obviously.As shown in Fig. 7(d), the valley between adjacent peaks is less than 73.5% [the red dotted line in Fig. 7(d)], which can be judged to distinguish peaks effectively according to the Rayleigh criterion.Thus, the CRM system can distinguish vertical bars with period 200 nm, and the spatial resolution of 200 nm can be achieved after super-resolution image restoration processing.Compared with a conventional CRM system that has a spatial resolution of 0.59 μm [18], the spatial resolution is improved by 65% with super-resolution image restoration processing.Figure 7(c

Conclusions
In this paper, we have proposed a method (SRIR-RAMAN) for improving the spatial resolution of CRM based on MPMAP super-resolution image restoration.Experiments and analysis indicate that this method only makes super-resolution image restoration with single Raman imaging and eventually improves spatial resolution by 65% to achieve 200 nm without changing the CRM system.SRIR-RAMAN provides a new way for improving the CRM spatial resolution.

Fig. 3 .
Fig. 3. Super-resolution image restoration simulation results of a vertical bar with 250 nm period: (a) target image, (b) degraded image (SNR = 100 dB), (c) restored image, (d) representative cross-sectional line profiles taken along the dotted line in (b) and (c).

Fig. 4 .
Fig. 4. (a) microscopic observation image, (b) zoom-in the area shown in red area of (a), (c) design image from the area of the green square in (b), (d) Raman spectrum of point A in (c).