Image-based autofocusing system for nonlinear optical microscopy with broad spectral tuning

We present an image-based autofocusing system applied in nonlinear microscopy and spectroscopy with a wide range of excitation wavelengths. The core of the developed autofocusing system consists of an adapted two-step procedure maximizing an image score with six different image scorings algorithms implemented to cover different types of focusing scenarios in automated regime for broad wavelength region. The developed approach is combined with an automated multi-axis alignment procedure. We demonstrate the key abilities of the autofocusing procedure on different types of structures: single nanoparticles, nanowires and complex 3D nanostructures. Based on these experiments, we determine the optimal autofocusing algorithms for different types of structures and applications. © 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

multiple pump photons are combined into a single shorter wavelength signal photon. Compared to optical linear microscopy, the spatial resolution in nonlinear microscopy is higher as the wavelength of the photons is spectrally shifted. Additionally, the penetration depth is increased for materials and structures that absorb in the infrared regime.
Another research field that employs methods of nonlinear optical characterization is nonlinear nanophotonics which studies the mechanisms of manipulating light at the nanoscale. The characterization methods in nonlinear nanophotonics require different approaches, such as the characterization of wavelength dependent resonances, emission enhancement or directionality of the generated nonlinear signals [20][21][22][23][24]. These types of studies require the recording of the nonlinear optical response over a broad range of wavelengths and polarizations of the incoming excitation laser beam. Current commercial multiphoton microscopes and corresponding software are not adapted for these types of measurements as they consist of achromatic objectives, adapted to certain precise wavelength ranges and adequate filters. Autofocusing systems are commonly used for routine measurements in life sciences, usually with attenuation, fluorescence or multiphoton imaging contrast. However, these systems do not allow to autofocus in a broad range of the laser excitation and collection wavelengths. That limits their application for nonlinear optical microscopy, especially in the fields of material analysis and nonlinear photonics structures characterization. The development of new approaches of automated focusing procedures will allow to overcome these limitations.
For an automated focusing procedure, there are many possible systems with various strategies and algorithms. Automated focusing procedures can be divided into two main classes: active/reflection-based or passive/image-based techniques. In the first category, active autofocusing systems measure the distance between a reference point near the sample and the objective with light or ultrasonic sound waves. For passive autofocusing systems a camera captures the signal at different sample or objective positions' and evaluates the focused position with an image analysis algorithm. On the one hand, active/reflection-based techniques are faster and more direct because no images need to be taken, on the other hand, they are less accurate because the distance is measured with respect to a reference point (or reference plane), not the sample itself. Passive image-based systems rely on algorithms to determine if a sample is focused and, on a strategy, to find the best position of the focus. The focusing strategy governs the choice of images that are captured and given an image, the scoring algorithm returns a figure of merit called the focus score. This numerical value indicates the focus quality.
In our work, we present an image-based autofocusing system developed for highresolution nonlinear optical microscopes. This system allows in a fully automated regime to characterize the nonlinear optical responses while sweeping the excitation laser wavelength, polarization power and position of the sample. To perform the autofocusing for a broad region of changing excitation parameters, we developed an autofocusing procedure with an image-based technique. This procedure includes a modified two-step hill-climbing strategy (coarse to finer search) with six different robust focusing algorithms also called focus measure functions. In our work we combined one strategy to maximize the focus score with the methods of multiphoton imaging microscopy. This approach allows to perform in a fast and reliable manner the nonlinear optical characterizations in a fully automated regime without any limitations to a specific wavelength region. We developed a software that allows to control all essential components such as laser, camera, stages for excitation and collection objectives, and stages to control the sample's position. The autofocusing system is incorporated in this software as a plug-in and allows to perform nonlinear optical characterization in a fully automated regime for different types of structures and in a fully automated regime for a broad range of wavelengths. Furthermore, we developed a procedure to control position of the sample by controlling the different stages (axes) to keep the sample focused and aligned with the laser beam during the entire measurement procedure. In the next sections, we describe the autofocusing procedure and demonstrate how we applied it to characterize different types of structures. We tested the developed autofocusing system for different excitation, specifically laser and white light excitation, and different types of signals, such as a transmitted laser beam, bright field image or a generated SHG signal. We demonstrate the abilities of the system by characterizing structures with different dimensions and geometry. The InGaN nanowires were used as an example of an elongated 1D structure, single barium titanate (BaTiO 3 or BTO) nanoparticles as an example of small single spot structures, layers of an epitaxially grown GaAs film and a woodpile photonic crystal structure, as an example of a layered structure with complex SHG signal. Our approach of the autofocusing can be easily adapted to different types of optical systems without limitations of wavelength range or contrast mechanism. The developed method can significantly reduce the duration measurements' time and increase the quality as well as the reproducibility of the nonlinear optical characterizations in material science and other nonlinear photonics research areas.

Method
To demonstrate the abilities of the developed autofocusing system, we used a home-built nonlinear optical microscope with transmission and reflection measurement configurations. This setup is equipped with a tunable Ti:sapphire laser and with an optical parametric oscillator (OPO) system (wavelengths 700 -1080 nm for laser region and 1080 -1600 nm for the OPO region). Figure 1 presents the typical schematic of the home-built microscope [25][26][27], which was used to test the autofocusing procedure. The nonlinear microscope allows to perform the characterization with wide-field excitation in transmission regime, using a low magnification objective or lens to excite the studied structure.
The wide-field excitation principle has two advantages over the scanning system. It is significantly faster and allows to study wavelength interference effects inside the structure [28]. As shown in Fig. 1(a), the excitation laser beam passes firstly through the power and polarization control unit, then it is focused with the excitation objective on the sample. The nonlinear response generated by the sample is collected with an objective, filtered to remove the pump wavelength and finally recorded with a camera. The core part of the autofocusing system consists of controlling the position of the sample, the excitation and collection objectives. The excitation and collection objectives are placed on motorized translation stages that move in horizontal direction (along the optical axis) while the sample holder is mounted on translation stages moving in the plane perpendicular to the optical axis. Similarly, the nonlinear microscope can be built in a reflection configuration ( Fig. 1(b)). The main difference here is that only one objective is required for excitation and collection. All controllers/drivers for the translation stages, camera and tunable sources are combined in one software environment. This software environment allows to tune the excitation wavelength, to reposition the objectives and the sample, and capture the corresponding nonlinear optical signal. All components for the autofocusing are combined in the software in one loop, that allows for all hardware devices to communicate with each other to perform the measurements with the procedure set by the operator.

Wavelen
A key challen maximize the objectives, ha automated reg the sample fo performed at images of the intensity of t distances bet excitation and corresponding wavelength. W indicated by t during the a procedure are spot is focuse 1 algorithms [29][30][31][32]. However, there is not one optimal algorithm for all possible configurations and focusing scenarios, as their performance depends on the specific situation, although some algorithms perform very well in multiple situations. In our nonlinear optical microscope, we implemented six focus measure functions to cover a wide variety of different scenarios, for autofocusing on the different types of structures. These six focus measure functions algorithms are listed in Table 1. For more tuning in the autofocusing, a threshold θ can be set for the Brenner Gradient and the Image Power so that only terms bigger than i x y is the intensity at a given pixel. The threshold allows these methods to be used in a wider variety of focusing scenarios. We also evaluated the performance of other algorithms, for example the neighbor pixel difference algorithm (Squared Gradient), normalized Variance and Autocorrelation [33] algorithms, but they demonstrated a lower performance for our autofocusing method, both in terms of quality of results and speed. Therefore, we excluded these algorithms from further implementations. We have selected six different algorithms that demonstrated the best efficiency for different studies. The choice of the focusing criteria, is based on the sharpness of the image that we can see by eye or the shape of the focus curve, given by the optimal algorithm and threshold. That will give a focus score curve, in which the focused and non-focused positions can be clearly distinguished.
Regarding the convergence of the algorithms, the intensity of the SHG varies with the peak power and the pulse duration of the excitation, which depends on the alignment and the related material dispersion in the optical path. We implemented in our system different algorithms to be able to focus on different aspects of the signal in case of intensity fluctuations. For example, the Maximum Pixel Intensity algorithm will be affected the most by such fluctuation, while the Image Power of Brenner Gradient algorithms will allow to overcome this problem. Variance method [31,34]: The focused image has the strongest pixel intensity variations in comparison with a mean value 2.
Variance & edge filter Statistical and Derivative-based The focused image possesses the sharpest contours. 3.
Brenner Gradient [35]: Image Power [36]: The focused image has the highest pixel intensities. Usually used to focus SHG signals.
6. image is foc rison) as show this region is e tify a local max oints compariso m Fig. 4(b), in mum, the user c over many po e, the value n focusing and a t will make th lows us to find teps to disting or the Brenner ble threshold. T the backgrou parameter θ i Lower signal can be exper or different valu er Gradient or the ulated from several everal loops, focus cutive values are es are compared to tep, this region of ved to the position hest focus x different coarse to ds [37]. In ons of the der to be tegory of Figure 3 using, the cused by wn in Fig.  evaluated ximum as on with a n order to can select oints with = 3 was analyzing he coarse d a region guish the Gradient To define und noise, is set to a l to noise imentally ues. We offer t fine focusing increase as th searches for a point is far fr region of inte should posses coarse and fin the objectives use a ratio of sample positio sizes which is coarse and fin coarse and fin

Multi-axi
During the a wavelength c procedure as d positions of o in Fig. 5  tions for the co m should con using algorithm ceed even if th m will reach th sured focus sco sired final posi on the depth of d in Fig. 1 excitation bea example of th 6(e) and 6(f), during the aut  To demonstrate how the developed multi-axis autofocusing method can be used to characterize different samples, we applied it to different categories of the structures. Depending on the ratio between the sizes of the excitation laser spot and the dimension of the studied structure, we can split the structures in four main categories: 0D, 1D, 2D and 3D. In our work, we experimentally tested it on the following structures: a spherical nanoparticle with sizes much smaller than the beam diameter (3-5 µm) as an example of a 0D structure, a long and thin nanowire as an illustration of a 1D structure, a layer of GaAs material to show how the autofocusing system will work for a 2D structure, and a photonic crystal as an example of a complex 3D structure. The number of stages that should be involved in the autofocusing procedure is given by the shape and morphology of the investigated structure. In the transmission mode of the setup, objectives with different magnification can be used for the excitation and the collection. This allows to achieve both wide-field excitation and high resolution in collection simultaneously. In reflection mode, the excitation and collection are usually performed with the same objective, which decreases the number of axes involved in the multi-axis autofocusing procedure. For the excitation, we used a focusing lens, that allows to perform wide focusing. The changes in the focused position during the wavelength sweep can be calculated with the equation for the wavelength dependency of the focal length of a thick lens [38]. The analytical solution could be used as a first step in the autofocusing procedure if the wavelength was swept with big steps. Table 2 presents an overview of the axes (translational stages) that should be involved in the multi-axis autofocusing procedure depending on the type of structure. For instance, measuring 0D particles in transmission mode requires autofocusing by moving the collection and excitation objective plus the x and y position of the sample. In reflection mode, the excitation and collection are performed by the same objective. Also, in nanowire-like materials (1D), the motion of the x and y translational stages of the sample can be coupled to optimize the position perpendicular to the direction of the wire.

Experimental demonstration
The autofocusing system was tested in real conditions on samples with different geometries and dimensionalities. We demonstrate autofocusing for the nanowire under bright light illumination, under 800 nm laser beam and similarly with 400 nm SHG recorded. After that, we concentrate on focusing SHG from a nanoparticle, a layer and a woodpile structure. This show the capability of the developed system to work first with very different types of signals, and second with very different geometries and dimensions. To evaluate the performance of autofocusing, the true focused position was first determined for each axis. The focused image in a first step, when the sample is placed in the system, can be determined manually based on the sharpness perceived by human eye and after that the position of the objectives and the sample can be set precisely from the final position returned by the algorithm. Images were recorded around this position for some axes (with 0.1 µm step size) and then the focus score was measured at each position with each algorithm. Since the focus score curves reflect the performance of the autofocusing system using the corresponding algorithm, we can discuss the quality for each algorithm independently from the parameters such as the start position or the coarse and fine step size. As mentioned in the previous section, on the one hand the focus score measured with the coarse algorithm should continuously increase as the image gets more and more focused, because this indicates the direction of the maximum region. On the other hand, the focus score curve measured with the fine focusing algorithm should reach a global maximum corresponding to the true focused position. Additionally, the sharpness of the maximum defines the uncertainty of the algorithm and the wings around the maximum the size of the search range.

Nanowire
We tested the autofocusing system on a 20 µm long InGaN nanowire [39,40] with a hexagonal cross section of 500 nm high between two sides. Images were taken in transmission for three different situations: under bright light, under 800 nm laser beam and similarly with 400 nm SHG being recorded. In the first two situations, the excitation and collection objectives were moved simultaneously in the same direction by the same amount (equivalent to moving the sample along the optical axis) and in the last one, only the collection objective was moved around the focused position. The focus score curves for the images of the nanowire around the focused position for different autofocusing algorithms are shown in Fig. 7. In the first situation (see Fig. 7(a)), the intuitive algorithms (Maximum Pixel Intensity and Image Power) could not be used to focus as the calculated focus score was almost constant over the whole range of images, as can be seen from the difference between the true focused position and the one returned by the algorithm is indicated by an error arrow and a shaded region. These two algorithms were not suitable because all the captured light signal was coming from the transparent substrate. Similarly, the Image Power algorithm with very high thresholds ( θ > 90%) returned a wrong focused position as the signal originated from a bright spot in the image. The Variance and Brenner Gradient (low threshold θ ) algorithms could not focus the image correctly. The focus score curve indicated indeed a broad and almost flat maximum. However, these algorithms could be used for coarse focusing (only) as the measured focus score curve possessed decreasing wings (offset > 2 µm) which indicated the focused region. The Variance & Edge and Brenner Gradient (high threshold θ ) algorithms returned a correct position. The measured focus score curve indicated a clear maximum that also corresponded to the true focused position. The calculated focus curve showed two other peaks that did not correspond to any sharp image. To avoid a fine focusing step around one local maximum only, the coarse focusing step was set to 10 µm. Focus score curves and images of the nanowire under a laser spot are shown in Fig. 7(b). Two clear positions appeared focused on the camera, either a thin line or a thicker still sharp line. This is caused by the particular hexagonal cross section of the nanowire [39,40], because the laser light can be focused either on the top facet or on the edges of the nanowire with hexagonal cross section. Only the Brenner Gradient algorithm was able to correctly identify the two positions. The two different threshold parameter values (20% and 50%) enable to focus on one or the other focused position of the nanowire, that correspond to the top and side facets of the nanowire. The other methods found neither of the two positions. Indeed, the Image Power and Variance algorithms returned a position in-between the two focused positions, as can be seen from the difference between the focused position and the one returned by the algorithm. This is indicated by an error arrow and a shaded region. nd thin layer was tested on emonstrates the the excitation asured with the hm could retur re sharp. We n am center to m cusing on the x ted on an Al 0.2 th the objectiv algorithms retu n. This can be n at pump's wa d and collected (Fig. 8(b)

Conclusio
In this paper optical micro the adequate different type crystals.
The key f excitation wa nanoparticle, system allow procedure eva limitation of time of the structures, the when the sign modified two functions) allo geometries fro results presen focusing depe focusing proc several optica developed in focusing devi Funding Schweizerisch (150609, 163 639052); H20 9. The SHG imag on and the focus s xcitation objective cases, yet it was al lines and frame on we demonstr scopy with a b algorithms, th es of sample feature of the p avelengths. Th excited in the ws to autofocu aluates in gen the autofocusi camera, which e autofocusing nal is very low -step hill-climb ows to focus th om single 0D nted in our pa ending on the g cedure enables al components our work can ce.
her Nationalfo 3916, 179966) 020 Marie Skło ges of a woodpile score curves calcul or b) the collectio more difficult to s indicate the focu rated a two-ste broadband rang he system was geometries l presented appro his feature w range 780 to us for any ot eral around 60 ing routine is t h directly dep g time can be w the autofocu bing strategy w he excitation a nanoparticles t aper allows to geometry of th the automatio aligned as wel n be also impl onds zur Förd ); Horizon 20 odowska-Curie