Compact multi-band fluorescent microscope with an electrically tunable lens for autofocusing

: Autofocusing is a routine technique in redressing focus drift that occurs in time-lapse microscopic image acquisition. To date, most automatic microscopes are designed on the distance detection scheme to fulfill the autofocusing operation, which may suffer from the low contrast of the reflected signal due to the refractive index mismatch at the water/glass interface. To achieve high autofocusing speed with minimal motion artifacts, we developed a compact multi-band fluorescent microscope with an electrically tunable lens (ETL) device for autofocusing. A modified searching algorithm based on equidistant scanning and curve fitting is proposed, which no longer requires a single-peak focus curve and then efficiently restrains the impact of external disturbance. This technique enables us to achieve an autofocusing time of down to 170 ms and the reproductivity of over 97%. The imaging head of the microscope has dimensions of 12 cm × 12 cm × 6 cm. This portable instrument can easily fit inside standard incubators for real-time imaging of living specimens. m length multimode fiber (M71L01, Ø1000 µm, 0.48 NA, Thorlabs Inc., USA), and collimated by a fiber collimator (F240SMA, 0.51 NA, Thorlabs Inc., USA). The collimated light passes through a dichroic mirror and is focused by the objective lens to illuminate the sample. A quad-band filter sets (including a 390/482/563/640 nm quad-band bandpass excitation filter F1, a 446/523/600/677 nm quad-band bandpass emission filter F2, and a R405/488/561/635 nm quad-edge dichroic beamsplitter DM1, Semrock Inc., USA) are used to separate the excitation and fluorescence beams. The ETL (EL-10-30-C-VIS-LD-MV, Optotune AG, Switzerland) is mounted to the objective rear stop. The focus of the objective can be shifted axially by tuning the control current of the ETL. A USB3.0 CMOS camera with a maximum full-frame rate of 60 fps (DCC3240M, 1280 × 1024 pixels, 12 bits gray depth, Thorlabs Inc., USA) is used to record the 2D images. In addition, an optional dichroic mirror DM2 and a second camera can be attached to the microscope to observe dual-channel fluorescence simultaneously.


Introduction
Over the years, time-lapse imaging [1,2] and real-time observation have become widely used approaches to monitor living cells and developing organelles. These techniques suffer from 'focus drift' where the specimen moves out of the focal plane during image acquisition [3]. Actually, focus drift occurs in almost every microscope system due to various factors, including initiative motion of the living specimen, thermal drift, mechanical instability and immersion media fluctuations, etc [3][4][5].
To redress focus drift in the time-lapse imaging process, researchers have developed a variety of autofocusing solutions including both software-based and hardware-based approaches [6]. Hardware-based autofocusing systems are generally based on the measurement of the distance between the objective and the specimen by detecting reflected light from the specimen or the surface of coverslip. This scheme has been used in most commercial autofocusing microscopes like the IX81-ZDC2 autofocus microscope provided by Olympus [7] and the CRISP autofocus system provided by ASI [8]. Success requires a near-infrared laser [7] or LED [8] as specialized illumination source. These approaches are precise when the signal to noise ratio (SNR) of the reflected signal is sufficiently high. However, the SNR may decline as a result of multiple reflections due to the refractive index mismatch at the water/glass interface. Therefore the flexibility of such schemes is limited.
In contrast, the software-based category is more flexible [9][10][11]. An autofocusing algorithm and a searching scheme are primarily required for a software-based autofocusing system. Varieties of autofocusing algorithms based on diverse principles have been developed, including Variance [12], Vol-4 [13], and Brenner gradient [14], etc. These algorithms extract focus-related information of the captured images like sharpness or brightness at different layers and in turn drive the axial scanning mechanism to the focal plane (FP) with the guidance of special searching algorithms.
Typical axial scanning control mechanisms include motorized and piezoelectric stages. The primary advantages of motorized stages are their high reproducibility and the long travel range. However, their limited travelling velocity tremendously impedes the autofocusing speed. When more rapid autofocusing is required, the better choice may be piezoelectric stages, which offer greater precision and higher speed than the stepper motors. Piezoelectric stages are normally confronted with challenges for their finite travel range (generally under 200 micrometers) and high cost. Finally, both motorized and piezoelectric stages may suffer from motion artifacts, which are detrimental to autofocusing when using immersion objectives.
Here, we present a scheme of compact automatic microscope by using a commercially available electrically tunable lens (ETL) for autofocusing. The ETL is a variable focal length lens [15,16] that can realize "optical axial scanning" rather than the conventional "mechanical axial scanning", which has been implemented in applications like confocal [17], two-photon [18,19] and light-sheet [20] microscopy for its compact structure, low price and fast tuning speed. The focal length of the ETL tends to be a nonlinear function of the supplied control current. We calibrated the influence of the nonlinearity on autofocusing and promoted a modified searching scheme. By driving the ETL to refocus at 100 Hz and utilizing the modified searching scheme, an autofocusing time of 170 ms and the reproductivity of over 97% were achieved. The compact multi-band fluorescent microscope apparatus is shown in Fig. 1. A custom-built, 4-wavelength LEDs array is used as the excitation source. The four LEDs chips with different wavelength (405, 470, 565 and 630 nm) are assembled in one base module and encapsulated together. The four beams are coupled into a 1 m length multimode fiber (M71L01, Ø1000 µm, 0.48 NA, Thorlabs Inc., USA), and collimated by a fiber collimator (F240SMA, 0.51 NA, Thorlabs Inc., USA). The collimated light passes through a dichroic mirror and is focused by the objective lens to illuminate the sample. A quad-band filter sets (including a 390/482/563/640 nm quad-band bandpass excitation filter F1, a 446/523/600/677 nm quadband bandpass emission filter F2, and a R405/488/561/635 nm quad-edge dichroic beamsplitter DM1, Semrock Inc., USA) are used to separate the excitation and fluorescence beams. The ETL (EL-10-30-C-VIS-LD-MV, Optotune AG, Switzerland) is mounted to the objective rear stop. The focus of the objective can be shifted axially by tuning the control current of the ETL. A USB3.0 CMOS camera with a maximum full-frame rate of 60 fps (DCC3240M, 1280 × 1024 pixels, 12 bits gray depth, Thorlabs Inc., USA) is used to record the 2D images. In addition, an optional dichroic mirror DM2 and a second camera can be attached to the microscope to observe dual-channel fluorescence simultaneously.

System configuration
Autofocusing and image collection are carried out by custom developed software programmed in Matlab. All the opto-mechanic components are assembled into the imaging head of ~12 cm × 12 cm × 6 cm shown in Fig. 1(a). Because the imaging head of the microscope is compact, the whole microscope system can be easily arranged in either upright or inverted geometries for different applications, as is shown in Fig. 1(b) and Fig. 1(c). Figure  1(d) is the photograph of the microscope instrument with an upright arrangement. The photographs of the fiber-coupled output 4-wavelength LEDs array light source are shown in Fig. 2. The 4 vertical structured LED chips with different wavelength (405, 470, 565 and 630 nm) are mounted on a copper substrate with excellent heat dissipation efficiency and the thermal resistance of the module is less than 0.8 °C/W. The upper surface of each chip is cathode and attached to the cathode of the electrical source with gold thread. The back surface of each chip is anode and packaged on the copper substrate. The 4-LED chips are common anodes and can be turned on separately with the help of independent cathode control. The size of each chip is 1 × 1 mm 2 , yielding a total light-emitting area of 2.1 × 2.1 mm 2 . The light divergence angle of each LED chip is 160°. An optical cone (wide diameter of 4.5 mm, narrow diameter of 1.2 mm, and length of 13 mm) is directed against the lightemitting area. The on-off and output power of each wavelength is individually controlled by the LEDs controller, which enables the LEDs either a single-band output or multi-band output simultaneously. The output powers of the four bands of 405, 470, 565 and 630 nm are ranged in 0~8.2 mW, 0~6.4 mW, 0~7.7 mW and 0~5.4 mW, respectively, which are measured after the fiber collimator by using an optical power meter (S130C, Thorlabs Inc., USA). More power up to 12 mW can be obtained by further increasing the LED current. The diameter of the LED beam at the back aperture of the objective is about 10 mm. Considering the transmission of the objectives, we can normally get 3~6 mW light power for each wavelength in the sample plane, which is sufficient for wide-field fluorescence imaging and autofocusing in our present circumstance. The ETL produced by Optotune AG is a liquid lens as shown in Fig. 3. By controlling the current of the motor bobbin, liquid inside the container is squeezed in and out the lens volume, resulting axial shift of the focus [21]. We employ the C-Mount ETL system with the tunable focal length from 80 mm to 200 mm and mount it close to the back focal plane of either a 10 × /NA0.3 or a 40 × /NA0.6 objective (Nikon Inc., Japan). In this manner, we can get an effective axial scan range of 2.17 mm and 128 μm for the 10 × and 40 × objectives respectively by tuning the ETL control current from 0 to 200 mA, as is shown in Fig. 4(a) and Fig. 4(b).

Characteristics of ETL
While using the ETL to achieve fast "optical axial scanning", several points must be taken into consideration. The most common issue is the nonlinear relation between the focal position and the control current, which has been evaluated as a parabolic equation by Jabbour et al. [17] and Grewe et al. [19] to character the axial scan range in the confocal and the twophoton microscopes, respectively. As verification, we measured the relative change of the focal position when tuning the control current and found that the nonlinear relation did exist. To describe the nonlinearity quantitatively, parabolic fittings are employed as shown in Fig.  4(a) and Fig. 4(b). Equation (1) and Eq. (2) are respectively the fitting results for the 10 × and 40 × objectives: where I represents the control current (mA), z 10 × and z 40 × are respectively focal positions (mm) for 10 × and 40 × objectives. In both equations we define focal position at 0 mA as zero. As most objectives are designed to be telecentric, direct appending of the ETL will lead to an non-telecentric assemblling, finally resulting in a relative variation of the magnification. This problem can be avoided by employing an additional 4f configuration [20] with the cost of increasing the physical size of the whole system. The change of the magnification in our system is also quantitated (Fig. 4(c)). The magnification varies from a factor of 1 at 0 mA to 0.83 at 200 mA, which reveals 17% variation over the whole scan range. Similar research on this issue was performed by Naikai [22] when observing intracellular structures and protein complexes. For autofocusing, the impact of magnification variation is negligible compared with the defocusing. Over the past years, numerous autofocusing algorithms have been proposed to redress focus drift. These algorithms employed the focus value (FV) to distinguish the in-focus image and the defocused ones. Although different algorithms use different parameters as their focus values, the common characteristic of the focus values is that it should reach the maximum at the focal plane and decreases rapidly and monotonously with defocusing. For example, the Variance algorithm calculates the variance of the acquired image as the focus value, while the Image Intensity algorithm sum up the square of the image gray values [10,11], The ideal outline of the focus value curve is illustrated in Fig. 5. Thus, the position of the focal plane can be precisely obtained by searching the maximum focus value of the whole curve.

Autofocusing algorithms
Typically, the focus value curve can be described by different mathematical models such as the polynomial [23], Gaussian [24] or Lorentzian models [14]. In this letter, we adopted the Gaussian model as it more closely approximates the measured curve:  On account of the nonlinearity between the focal position and control current of the ETL, the focus curve needs to be calibrated in the beginning. Figure 6 where p and q are respectively known quadratic and monomial coefficients in Eq. (1) (or Eq. (2)), a, b and z 0 are undetermined parameters, which imply that at least three focus values at different control currents are required to fit Eq. (4). In the next section, we will utilize the calibrated focus curve to determine the focal plane.

Autofocusing rapidly and correctly
A superior autofocusing algorithm should get the maximal focus value right at the focal plane. To find this maximum efficiently, researchers often employ a sort of one-dimensional searching algorithms such as Fibonacci, mount-climbing or golden-section methods. Speeds of such schemes are rather high, which only takes several steps to reach the focus. However, they are quite sensitive to the local maximums of the focus curve. The fluctuation of illumination and the mechanical vibration of environment may be the most common inducements of the local maximums, which usually manifest as jitters in the focus curve (see Fig. 7). In addition, we found it is difficult to autofocus correctly when observing thick samples. One reason is the image degration caused by inhomogeneous optical properties of the thick sample. "Guide star" technique, which employs a reference point-source behind the inhomogeneous medium for measuring the shape of the wave-front, combined with the adaptive optics method is usually used to correct the aberrations and improve the imaging quality [25,26]. Yet, the more important cause comes from the multi-peaks of the focus curve, which is mainly formed by two or more projections of the sample at different depth in the field-of-view, as is illustrated in Fig. 7. Any local maximum in the focus curve may lead to a mistaken convergence of the searching algorithm, ultimately resulting in a failure to autofocus precisely. Fig. 7. Measured multi-peaks focus curve. Multiple peaks caused by multi-layer specimen and jitters caused by system instability are the main reasons for local maximums, which probably lead to a mistaken convergence of the searching algorithm.
To circumvent this restriction, we developed a modified searching scheme, illustrated as a flow chart in Fig. 8. This algorithm is based on coarse equidistant scanning and curve fitting rather than simple iteration and as such is different from the conventional searching algorithms. It does not require the focus curve to present single peak and is no more sensitive to jitters caused by vibrations of the system. It is worth noting that although more frequent equidistant scanning can provide better accuracy, it will require much time. Thus, it is crucial to adjust the space of scanning for a given searching range to reach a balance between speed and accuracy. In general, nine steps of calculation are enough for a searching range of 20 mA (equivalent to a defocusing distance of 12 μm for the 40 × objective). That is, a step interval of 2~3 mA is acceptable in most cases to reach a comparable speed as the conventional scheme but achieves much higher stability. Besides, sub-window technique is also adopted to shrink the FV's computing time and avoid local maximums caused by multi-layer samples. Fig. 8. Flowchart of the promoted searching scheme. The whole process could be divided into two stages. In the first stage, raw images are captured at even-spaced control currents and corresponding focus values are calculated. Maximal point and its four neighbors are preserved for later use. In the second stage, the five points are assigned into two groups. F(I) in the flow chart presents the function in Eq. (4) and a, b, and z 0 are the corresponding unknown coefficients. By fitting the measured points to F(I), the unknown coefficients in F(I) can be determined. Further, via solving their stationary points of the determined functions, we can get the maximum points of the fitted curves I M1 and I M2 . Arithmetic mean value of the two is set as the final control current. To demonstrate the multi-band fluorescent imaging capability of the microscope, we employ the multi-dye labeled specimens, a mouse kidney cryosection and a bovine pulmonary artery endothelial (BPAE) cell (Molecular Probes Inc., USA) to embody the multi-band information. The color-fused fluorescent images are shown in Fig. 9. The glomeruli and convoluted tubules of the mouse kidney cells are stained with the green-fluorescent lectin, Alexa Fluor 488 wheat germ agglutinin, thus a instinct view of these parts can be recognised by the green color in Fig. 9(a). Similarly, the nuclei (counterstained with DAPI) can be distinguished by the blue color. For the BPAE cells, mitochondria, F-actin and the nuclei were respectively labeled with MitoTracker Red CMXRos, Alexa Fluor 488 phalloidin and DAPI, which represent the red, green and blue components in Fig. 9(b).  Figure 10 is the main interface of the autofocusing software written in Matlab. The CMOS camera is connected via the ActiveX component and the ETL is controlled though serial port communication. Control panels for both devices are designed for parameter adjustment. The sub-window for autofocusing can be selected either by entering parameters or by simply selecting a rectangular region of interest (ROI). Being designed to be easy to use, this interface enables the observer to implement autofocusing by simply clicking on the specified button. In addition, time-lapse imaging is also integrated into the graphical interface for further application. As performance of autofocusing algorithms are susceptible to specimen type, illumination scenarios and noise levels, we integrate sixteen autofocusing algorithms [10][11][12][13][14] in the software to meet different circumstances. These sixteen algorithms can be classified into four groups, that is, derivative-based algorithms, statistical algorithms, histogram-based algorithms and intuitive algorithms. The detailed information on the algorithms is listed in Table 1. Performance of the autofocusing algorithms and their ranking process under different circumstances can be found in the prior literatures [10][11][12][13][14]. For the fluorescent scenario in this article, the Variance, Normalized Variance, Vol-4, Vol-5, and Brenner gradient algorithms are proved to have better performance than the others.

Autofocusing software
To offer an aid when selecting the autofocusing algorithm, a rapid one-click analysis module to find the optimal algorithm is attached in Algorithm Selection tab control. With the autofocusing algorithm selected, autofocusing can be completed accurately and rapidly with the guidance of the promoted searching scheme.

Speed and reproductivity
Besides the mouse kidney cells and the BPAE cells, the custom-prepared fluorescent beads are also used to embody a variety of information. In the autofocusing experiment, the size of the sub-window is unified as 100 × 100 pixels, while the CMOS camera works at the full resolution of 1280 × 1024 pixels. For concision, only the 40 × objective is used for demonstration of autofocusing here. The best algorithms for beads, mouse kidney cells and BPAE cells are testified to be the Variance, Vol-5 and Brenner gradient, respectively. Fig. 11. Time-sequence chart of the autofocusing process based on the camera's time shaft. Camera works in the trigger mode and is triggered periodly. One autofocusing process contains nine steps, and each step includes three events, i.e., camera exposure, FV calculation and ETL settling.
The average execution time of autofocusing varies with the type of specimen. Runtime for autofocusing of fluorescent beads is about 170 ms, that is, (10 ms ETL settling time + 6 ms camera exposure time + 3 ms focus value computing time) × 9 axial moving steps, as is shown in Fig. 11. This turns out to be fastest among the three specimens and is attributed mainly to the strong fluorescent signal from the beads and the sufficent illumination power, both of which reduce the camera's exposure time. In contrast, for the mouse kidney cells and BPAE cells, the fluorescent signals are relatively weak and the exposure time must be increased to obtain a sufficient SNR. Under this circumstance, autofocusing runtime for mouse kidney cells is 1.04 s, that is, (10 ms ETL settling time + 102 ms camera exposure time + 3 ms focus value computing time) × 9 axial moving steps. As is analyzed above, autofocusing speed of the system is basically limited by the camera's sensitivity, thus better results can be achieved by employing more sensitive cameras. To verify the autofocusing reproductivity of the fluorescent microscope, 1200 repetitive tests are conducted on the three types of specimens, i.e., 3 groups of experiments are organized and each group repeats 400 times. The excitation wavelength is selected to be 405 nm. A periodic defocusing of the specimen is carried out by a motorized z-stage on which the specimen is placed. The whole test can be treated as an alternate process of autofocusing and defocusing between two focal planes denoted as FP_A and FP_B, that is, firstly move the specimen to FP_A (defocusing process) and consequently make autofocusing to FP_A by using the ETL, afterwards move the specimen to FP_B and then make autofocusing to FP_B with the ETL, and so forth. For each focal plane, both autofocusing and defocusing take place 200 times. For every autofocusing, what we can obtain include the final control current of ETL and the in-focus image. Both of them provide clues to justify whether the autofocusing is successful. However, judgment by the image quality is a subjective approach that depends on the observer or the merit of criteria, which is not consistent with our purpose to quantify the reproductivity. The control currents of ETL for autofocusing are proved to distribute tightly around two values corresponding to the two focal planes in practice. The statistic histograms of the control currents for the beads are shown in Fig. 12. We assume that the FP_A and FP_B stand at the centroid (I A and I B ) of the measured control currents, and consider a successful autofocusing to occur if the final control current falls into a range corresponding to the depth-of-field (DOF) of the objective lens, which is determined by Eq.
where λ indicates the emission wavelength (λ = 446 nm), n denotes the refractive index of the medium (n = 1), NA is the numeral aperture (NA = 0.6), P presents the pixel size of the camera (P = 5.3 μm), and M is the magnification of the system (M = 40). Thus, the DOF turns out to be 1.25 μm for the test, which corresponds to 1.95 mA in control current (saying ΔI DOF ). We define the reproductivity as the rate of successfully autofocusing counts to the testing number. Eventually, the reproductivity values for the three groups of tests are measured to be 97.5% (the beads), 97% (the mouse kidney cells) and 96.8% (the BPAE cells), resulting in an average reproductivity of 97.1%.

Summary
In summary, we have developed a compact multi-band fluorescent microscope by using an electrically tunable lens (ETL) device for autofocusing and a fiber-coupled quad-band LEDs as illumination source. The combination of the ETL and the fiber-coupled output 4wavelength LEDs array light source can realize fast axial scan and make the microscope more compact and cost-effective. With the new promoted searching scheme based on equidistant scanning and curve fitting, we demonstrate that the setup has a minimum autofocusing time of 170 ms and a reproductivity of over 97%. The whole microscope system can be easily arranged in either upright or inverted geometries and applied to time-lapse imaging and realtime observation.