Machine learning guided rapid focusing with sensor-less aberration corrections

Non-invasive, real-time imaging and deep focus into tissue are in high demand in biomedical research. However, the aberration that is introduced by the refractive index inhomogeneity of biological tissue hinders the way forward. A rapid focusing with sensorless aberration corrections, based on machine learning, is demonstrated in this paper. The proposed method applies the Convolutional Neural Network (CNN), which can rapidly calculate the low-order aberrations from the point spread function images with Zernike modes after training. The results show that approximately 90 percent correction accuracy can be achieved. The average mean square error of each Zernike coefficient in 200 repetitions is 0.06. Furthermore, the aberration induced by 1-mm-thick phantom samples and 300-μm-thick mouse brain slices can be efficiently compensated through loading a compensation phase on an adaptive element placed at the back-pupil plane. The phase reconstruction requires less than 0.2 s. Therefore, this method offers great potential for in vivo real-time imaging in biological science. © 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement


Introduction
In recent years, the development of biological imaging was focusing on real-time, high resolution and deep in vivo imaging [1,2].Over the past two decades, researchers have overcome the diffraction limit and provided new insights into subcellular structures, yet the spatial resolution was improved at the cost of the temporal resolution.Adaptive optics (AO) becomes a valuable technique for high-resolution microscopy.It compensates the aberrations introduced by the specimens and obtains high-resolution images in deep biological tissue [3].AO is originally developed for telescopes to overcome the atmospheric distortions, which degrade the image qualities of the extraterrestrial objects.Recently, it has been applied in optical microscopy to recover diffraction-limited imaging deep in the biological tissue [4][5][6] by using an active element such as a deformable mirror (DM) or a spatial light modulator (SLM).However, the imaging speed is fundamentally limited by the refresh rate of the active element.Moreover, the total fluorescent photon budget is also limited, which means that to obtain a higher signal to background ratio, fewer photons should be used as the feedback signal to measure the wavefront aberrations.Traditional adaptive optics systems utilize a wavefront sensor such as a Shack-Hartman wavefront sensor to measure the aberrations [7,8].For example, Kai Wang et al. used the de-scanning, laser-guided star and the direct wavefront detection methods to achieve the rapid adaptive optical recovery [9].However, the wavefront sensor is high An alternative al. proposed a wavefront and increasing the cause much m In this pa algorithm, wh non-linear m aberrations ex are collected wavefront abe are some prev throughput an the machine Due to the ge for any AO sy applied our m thick mouse b  polarized light to exit.To further ensure the polarization direction of light is consistent with the direction required by the SLM, we placed a polarizer after the PBS.The incident laser beam is phase-modulated by using a spatial light modulator (SLM, PLUTO-NIR-011-A, pure phase, 60Hz, Holoeye photonics), on which the compensation patterns are loaded.The BS is used to make the laser beam perpendicularly project to the SLM and reflect.The reflected laser beam focuses on the focal plane after passing through the relay lenses L5 and L6, objective OBJ1 (RMS4X, Olympus, 4X / 0.10 NA) and sample.To detect the point spread function, another objective OBJ2 (RMS4X, Olympus, 4X / 0.10 NA) and relay lens L7 are mounted and the intensity information is collected by a CMOS camera.Before experiments, the SLM has been calibrated.We utilized the interferometric method to calibrate the phase modulation [15] and set the SLM for a linear 2π phase distribution over all 8-bit gray level to assure the phase response as stable as possible.Furthermore, the number of the pixels and the usage area of the SLM match the pixel interval of the CMOS camera.

Machine learning-guided fast AO compensation algorithm
The aberrations of the wavefront can be quantized as the difference in its phase or optical path length from the ideal (e.g., spherical or planar) form.Mathematically, it can be described as a summation of the Zernike polynomials, a set of basic functions that are orthogonal within a unit circle [16] (in this case, the objective back pupil).The phase distribution can be expressed as: phase x y a Z x y a Z x y a Z x y a Z x y In other words, if we can reconstruct the phase distribution with proper Zernike modes and coefficients at the backpupil plane, the aberrations induced by the scattering can be compensated.Therefore, the aim is to establish a mapping between the Zernike coefficients and the point spread function.
Zernike mode 1 Z means a piston where the size of its coefficient has no effects on the point spread function at the back-pupil plane.Therefore, the phase reconstruction is without regard to mode 1 Z .Then the other Zernike modes are divided into two categories: The tip-tilt modes (Zernike modes 2 Z , 3 Z ) and the high-order modes (Zernike modes ( 4,5,6, ) The aberrations in tip-tilt modes have a linear relationship with their coefficients, which can be calculated directly by Eq. ( 2)-(3): where λ is the wavelength of the laser beam, f is the focal length of lens L7 and D is the beam diameter on the SLM.dx and dy are the displacements of the center of the point spread function in horizontal and vertical directions as illustrated in Fig. 3     ase-mask, 15th) and berrations d through objective OBJ2 and then be detected by the CMOS camera.After the compensation phase pattern was obtained, it was superimposed onto the phase-mask and then loaded to SLM.We conducted 200 repeated experiments by only changing the phase-mask and get a statistical result that more than 80% distorted point spread functions were improved.Figure 4 shows four groups of the compensation results in 200 repetitions.Figure 4(a) records the point spread functions before (left) and after (right) compensation, respectively.Comparing the intensity profile at the center section of the point spread functions in Fig. 4(d), we can find that the center of the point spread function moved to the ideal spot location after tip-tilt correction and there is not only an increase on the intensity and a decrease on full width at half maxima (FWHM) after compensation.The reconstructed first ten Zernike coefficients (in orange bar) and corresponding 1st-15th Zernike coefficients (in the blue bar) of phase-mask are illustrated in Fig. 4(b).The mean square error (MSE) of coefficients between the reconstructed phase and the phase-mask illustrate the ability to reconstruct the aberration.MSE of these four groups are 0.034, 0.14, 0.011, 0.015 and the average MSE of 200 repetitions is 0.060.Figure 4(c) is the compensation phase pattern applied on the SLM consists of the reconstructed first ten Zernike modes.According to the experimental performance, we infer that when the main parts of coefficients are calculated correctly, the distortion can be compensated even though some minors are inaccurate.
To verify the performance of our method in real scattering media, 1-mm-thick phantoms and 300-µm-thick mouse brain slices were applied.We directly mounted the real scattering medium on a vertical stage between two objectives and the system would calculate the proper Zernike coefficients according to the distorted point spread function.5(a)-5(c) are more irregular than that induced by phasemask and the corrected point spread functions were not as smooth as that in Fig. 4(a).This is because that the phantom induces multiply scattered light, which contains high-order aberrations.The section intensity profile of the point spread functions in Fig. 5(d) illustrates that the proposed AO method dramatically improves the point spread function quality where the intensity increased by 3-5 times.The 300-µm-thick brain tissue slices are prepared as follows: Mice were rapidly anesthetized with chloral hydrate (5%, w/v, 0.01 ml/g, i.p.) and transcardially perfused with ice-cold 0.01 mol phosphate-buffered saline (PBS, Solarbio) and paraformaldehyde (4% in PBS w/v, Sinopharm Chemical Reagent Co.Ltd).Brain tissues were collected and incubated in the paraformaldehyde solution at 4°C overnight for uniform fixation through the sample.
To remove the water remained in the brain tissue, incubated the sample in sucrose (30% (w/v) in PBS) at 4 °C for 24-48 hours until the specimen sank to the bottom of the tube.After that, 300-µm-thick brain slices were sectioned by using a cryostat (CM3050S, Leica).Sections were immediately embedded into optical clearing reagent in 2 minutes and mounted in the holder.Figure 6 presents two typical compensation results within 300-µm-thick mouse brain slices.The distortions induced by mouse brain slices are more complicated than phase-mask and phantoms.As shown in Fig. 6(c)-6 (d), although our method only compensates the first ten orders, we can still improve the intensity and FWHM of the point spread function of the samples, which contain Zernike modes more than 10 orders.

Conclusion
We proposed a rapid AO aberration compensation method based on the machine learning algorithm.The time consumption for each phase reconstruction is less than 0.2 s (CPU Intel Xeon(R) E5-2667 v4, NVIDIA Tesla P4).The experimental correction accuracy is larger than 85% and each compensation process with CMOS camera (DMK 23UV024, 640 × 480 (0.3 MP) Y800 @ 115 fps) signal collection and SLM (PLUTO-NIR-011-A, pure phase, 60Hz) pattern loading is approximately 0.08 s.It is also capable of compensating low-order aberrations from both 1-mm-thick phantoms and 300-µm-thick mouse brain slices, respectively.If we utilize GPU acceleration or FPGA control acceleration, we are able to further expand the order of the Zernike modes as training sets, thus achieving more complex aberration corrections, and deeper imaging depth.Based on the advantages of the machine learning method, although the training set takes a few hours of training, the illumination time required for the corrections on the experiment is very short, thus dramatically reducing photobleaching and photodamage.
In conclusion, we can achieve high-speed wavefront aberration corrections with machine learning and recover near diffraction-limited focal spots.With these advantages, our method has great potential to be applied to rapid deep tissue imaging in biological science.

Funding
Fig. 1 beam a half reflec SLM.object functi OBJ1 AP, a splitte ) phase x y ψ is the phase distribution on the pupil plane; ( , )( 1,2,3, ) ⋅⋅⋅ are Zernike coefficients.Low-order Zernike modes are related to the primary aberrations such as spherical aberration, coma, and astigmatism.Different combinations of the Zernike coefficients in a phase distribution at the back-pupil plane gives distinguishable point spread function at the focal plane.
Fig. 3 ideal vertic model Zernik Training i pixel images random phase Zernike mode Fig. 4 Four g plane coeffi bars).profile functi guided3.Results aThe proposed which is a ph loaded on the leading to a d

Fig. 5 .
Fig. 5. Experimental compensation results of the 1-mm-thick phantom slice.(a)-(c) Point spread functions scattered by three different areas in a 1-mm-thick phantom sample (up) and corrected by our machine learning guided AO system (down).Inside the colored dotted boxes are the enlarged views of each point spread function.(d) Section intensity profile of the point spread functions without correction (NO AO), after tip-tilt correction (T-T corr) and after machine learning fully correction (ML-AO).The scale bar in (a)-(c) is 100 μm.

Figure 5
Figure5provides the compensation results of a 1-mm-thick phantom.The distorted point spread functions depicted in Fig.5(a)-5(c) are more irregular than that induced by phasemask and the corrected point spread functions were not as smooth as that in Fig.4(a).This is because that the phantom induces multiply scattered light, which contains high-order

Fig. 6 .
Fig. 6.Experiment compensation results of 300-µm-thick mouse brain slices.(a)-(b) are two typical scattered (NO AO) and corrected (ML-AO) point spread functions.The corresponding blue-dashed and magenta-dashed ROI are enlarged as below.(c)-(d) Intensity profile at the center of the point spread function before and after correction (indicated with blue and magenta arrows respectively).(e)-(f) Amplitude distribution of Zernike coefficients calculated with our method.The inserted pictures demonstrate the compensate phase pattern loaded on SLM.The scale bar in (a)-(b) is 100 μm.