Video-rate remote refocusing through continuous oscillation of a membrane deformable mirror

Abstract This paper presents the use of a deformable mirror (DM) configured to rapidly refocus a microscope employing a high numerical aperture (NA) objective lens. An Alpao DM97-15 membrane DM was used to refocus a 40×/0.80 NA water-immersion objective through a defocus range of −50–50 μm at 26.3 sweeps s−1. We achieved imaging with a mean Strehl metric of >0.6 over a field of view in the sample of 200 × 200 μm2 over a defocus range of 77 μm. We describe an optimisation procedure where the mirror is swept continuously in order to avoid known problems of hysteresis associated with the membrane DM employed. This work demonstrates that a DM-based refocusing system could in the future be used in light-sheet fluorescence microscopes to achieve video-rate volumetric imaging.


Introduction
The ability to refocus an optical microscope rapidly, on sub-second timescales is essential for example when using optically-sectioning microscopy to acquire 3D images of the specimen at multiple volumes per second [1]. The most straightforward approach to achieve refocusing is axial translation of the sample or microscope objective with a piezoelectric actuator [2]. However, moving the mass of the sample or lens at the required frequency and amplitude needs an actuator of relatively high power and can introduce vibrations that can perturb the sample.
An alternative to moving the sample or lens is to employ an adjustable optical element elsewhere in the optical system. Electrically tunable lenses (ETL) provide convenient electronic control of the amount of defocus [3] and have been successfully demonstrated to achieve remote refocusing for multiphoton microscopy [4], light-sheet microscopy [5] and confocal microscopy [6]. The ETL in [5] was able to scan 17 planes within a zebrafish heart at 30 volumes s −1 . The authors used a medium/low numerical aperture (NA) lens which corrected for primary defocus and they reported that astigmatism, coma and field curvature restricted the use of the lens to the central region of the field of view (FOV): these aberrations increased towards the limits of the ETL focal range. ETLs in general only provide low-order quadratic primary defocus wavefront correction whereas spherical aberration and higher orders are required to remotely refocus high-NA objectives [7]. The proposed bi-actuator design of a liquid ETL in [8] consisting of two concentric piezo rings was able to provide simultaneous correction for defocus and spherical aberration. However, more degrees of freedom are required for greater aberration correction of high-NA lenses and when imaging in non-homogeneous media. ETLs exhibit significant hysteresis; a study of ETLs produced by Optotune [9] reported hysteresis up to 1D of optical power that was dependent on the history of the applied current used to refocus the lens.
We will refer to the total wavefront correction required as high-NA defocus, see equation (1) in [7]. Figure 1 plots the maximum diffraction-limited refocus that can be achieved as a function of NA when applying only quadratic defocus wavefront correction (see appendix for calculation). For an NA of 0.75 the application of primary defocus alone only allows a diffraction-limited remote refocus distance of 22 µm with Spatial light modulators (SLMs) are able to provide higher-order wavefront correction. However, this technology typically only provides phase modulation up to 2π and therefore the pattern displayed by the SLM must be wrapped if significant amounts of defocus are to be applied. The discontinuities occurring between pixels and those when the phase is wrapped cause a decrease in the optical efficiency of the system. In addition, SLMs are typically not achromatic, which means that they do not work well over a broad range of fluorescence excitation or emission wavelengths. In two-photon fluorescence microscopy, losses in the SLM can be overcome by increasing the laser power used, and this approach has been demonstrated to apply high-NA defocus to the excitation beam [10]. This approach is effective for two-photon imaging, which, due to the inherent sectioning afforded by two-photon excitation, does not require refocusing of the return signal, but SLM losses limit the application of this approach to other modalities.
The elegant remote-refocusing approach of Botcherby et al [7] can also achieve axial refocusing. Here, three microscopes are placed in series with an ideal intermediate image being formed between the 2nd and 3rd microscope objectives. Axial translation of the 2nd or 3rd microscope objectives achieves remote refocusing with no mechanical perturbation of the sample. In this configuration the actuator driving the translation of the 2nd or 3rd microscope objective must be sufficiently powerful to accelerate the mass of the objective at the required frequency and amplitude and ultimately it is the power of this that limits the range achievable for a given refocusing frequency. This approach has been demonstrated working over the range of 2.5-50 volumes s −1 by Millett-Sikking et al [11]. At 50 volumes s −1 ten image planes were acquired per 16.6 µm. The mass being moved can be reduced by folding the optical system about the intermediate image, with remote refocusing achieved by scanning the axial position of the small fold mirror, but then a beam-splitter arrangement is required to separate the refocused beam. Remote refocusing of the excitation beam path can be performed without loss of light using a polarising beam splitter arrangement e.g. for scanning a polarised excitation spot in multiphoton microscopy [12], but if employed on the detection path it typically leads to a reduction in light throughput [13]. An alternative to using a folded remote-refocusing setup with an axially displaced remote mirror is to introduce a galvo mirror in the remote-refocusing path in a plane conjugate to the pupils of the remote-refocusing objectives [14]. A remote mirror with a stepped profile can then be used to achieve rapid discrete axial translation of an illumination beam, or a tilted remote mirror can be used in combination with the remote objective to achieve continuous axial scanning. The tilted remote mirror was used to demonstrate rapid axial sweeping of a light-sheet illumination beam and also for rapid axial refocusing of a two-photon excitation beam. As for conventional folded remote-refocusing, this approach can be performed without light loss for polarised illumination, but will lead to a reduction in light throughput if it were to be employed on the detection path.
Rapid refocusing using a deformable mirror (DM) has been demonstrated previously for applying low-order defocus in optical coherence tomography [15]; for achieving an active focus lock in a confocal microscope [16]; and to extend the depth of field in an epi-fluorescence microscope [17]. Higher-order defocus wavefront correction with a DM has been demonstrated previously for rapid adaptive focusing in a multiphoton microscope employing temporal focusing [18]. The authors used open-loop control of an Alpao DM to acquire ten optically-sectioned images axially separated by 5 µm in 10 ms from a fluorescently labelled pollen grain and five optically-sectioned images axially separated by 5 µm in a live fruit fly brain expressing a genetically encoded calcium indicator at 2 volumes s −1 .
Higher-order membrane DMs additionally offer the possibility to correct for spherical, and higher-order aberrations, provide average corrections to some field-dependent aberrations such as field curvature, and compensate for system aberrations.
Changes in refractive index caused by local structures, e.g. proteins, nuclear acids and lipids have a refractive index different to water, can give rise to optical aberrations. Therefore, as well as refocusing the objective, another benefit of using DMs is that they can be used to correct for sample-induced aberrations, and this becomes more important when employing higher NA microscope lenses. For example, Débarre et al [19] demonstrated an adaptive optics-based two-photon microscopy approach using a DM to correct for sample-induced aberrations when imaging to a depth of 140 µm in a mouse embryo. Sample-induced aberrations have been shown to be stable over hours when performing two-photon imaging in vivo in mouse brain [20], so once determined, aberrations can be corrected by a DM over a period of time-lapse imaging.
Alpao DMs are known to suffer from a viscoelastic creep on a timescale of minutes. This is suspected to be due to a polymer material used between the actuators and the mirror membrane surface, and so their response is dependent upon the history of previous shapes/poses [21]. They are also known to change their shape depending on temperature and that the temperature of the mirror depends on the electric current flowing through the actuator coils [22]. Both effects complicate their use for sustained high-speed refocusing devices. Bitenc et al [22] present a detailed software-based method to greatly reduce these issues, but it requires extensive prior characterisation of the DM to quantify the creep processes.
In this paper, we present an image-based optimisation approach using an Alpao DM to achieve refocusing for wide-field microscopy. DMs have the benefits that they are achromatic, present low insertion loss and offer the potential to achieve rapid refocusing. Our work presents a new method for determining the DM command sequence that is insensitive to the creep processes of the mirror and that enables sustained refocusing at 26.3 refocus sweeps s −1 over a defocus range −50-50 µm. The optimised DM command sequence was found to be stable over an extended timeframe (many hours). The refocusing and optimisation procedure was applied to both a 40×/0.85 NA air objective and also a 40×/0.80 NA water immersion objective. Through the imaging of a star-test mask we determined that the mean of the estimated Strehl metric across the FOV for the 40×/0.80 NA water immersion objective was >0.7 over an FOV of 200 × 200 µm 2 for a refocus range of 52 µm and >0.6 over a 200 × 200 µm 2 FOV for a refocus range of 77 µm.

Optical setup
The optical setup used for DM optimisation and testing is shown in figure 2. A pair of aspheric lenses (AL1&2, 350230, Geltech/Thorlabs) imaged the LED (625 nm, M625L3, Thorlabs) onto a Lambertian diffuser (flashed opal glass diffuser, 50 DO 50, Comar Optics) that was placed in contact with and behind a custom star-test mask (JD Photo Data). The star-test mask is a 1.6 mm thick glass slide with one surface coated in a chrome layer that has an array of 1 µm diameter circular pinholes spaced on a hexagonal lattice with a period of 20 µm, which was used as the object during the DM optimisation procedure. The axial position of the star-test mask was controlled using a motorized stage (ESP100, Newport).
Light transmitted through the star-test mask was collected by objective O1, which was either a 40×/0.85 NA air objective (1-UB827, Olympus) or a 40×/0.80 NA water-immersion objective (LUMPLFLN, Olympus). For the water immersion lens, as the system was aligned in a horizontal plane, we used ultrasound gel (UGEL250, Ana Wiz Ltd) as the immersion medium (which has a refractive index within 1% of that of water). The exit pupil of O1 was relayed via 4 f tube lenses TL1 (Thorlabs TTL100A) and TL2 (Thorlabs TTL200A), onto the DM (DM97-15, Alpao), which applied a phase correction. Ideally, the pupil of O1 (diameter 7.65 mm and 7.20 mm for the air and water immersion objectives respectively) would be imaged onto the DM so that the pupil image matches the size of the DM (diameter 13.5 mm). However, in this setup the image of the pupil overfilled the DM, thereby reducing the NA of the system to 0.75.
In order to avoid using a beamsplitter, the DM was angled with its normal at 10 • to the optical axis of O1 and TL1&2, and reflected the refocused wavefront to TL3 (f = 200 mm, TTL200A, Thorlabs), which produced an image on the sCMOS camera (ORCA-Flash4.0 v3, Hamamatsu) with an overall lateral magnification from the star-test mask of 22.2×. The camera was operated in its global reset edge-trigger mode with a 0.5 ms LED illumination pulse timed to occur when all rows of the sensor were exposed. To correct for the distortion introduced by the tilt of the DM with respect to the incident beam, the patterns applied to the DM were scaled by a factor of 1/cos(10 • ) in the direction parallel to the plane of the optical table.

Model describing DM surface profile
The optical path difference (OPD), required in the exit pupil of an objective lens to achieve a specific amount of defocus can be estimated for an on-axis defocused point by exploiting the sine condition, the expression for which will be referred to as high-NA defocus [7], and is, Here, n is the refractive index in the sample, ρ is the normalised radial pupil coordinate and z is the axial displacement of the object from the focal plane towards the objective. To account for aberrations of the objective lens and optical relay system, the dynamic response of the continuously oscillating DM as well as the potential need to under or over-drive the DM in order for it to reach the correct shape at the correct time, a series of Zernike modes were added to this to give a combined OPD for the mirror command signal, The Zernike modes Z are indexed by their Noll index i and each mode is normalised so that its inner product with itself over the unit circle is π. The amplitudes a and b i were determined by the optimisation procedure below.

Calculation of Strehl ratio and optimization metric
The Strehl ratio is defined as the ratio of the intensity at the centre of a system's point spread function (PSF) to the theoretical maximum diffraction limited intensity that would be obtained in the absence of any aberration. A system with a Strehl ratio greater than 0.80 is commonly considered as being diffraction limited [23]. Figure 3(a) shows an example of a typical image acquired by the camera of the star-test mask using the 40×/0.8 NA air objective.
The Strehl ratio was estimated for each pinhole using the following procedure. First, as the Strehl ratio is very sensitive to the background light in the image, we estimated the local background by taking the mean of pixel values lying at half the period of the star-test pattern from each pinhole, see yellow circle shown in inset to figure 3(a), and subtracted this from the image of that pinhole lying within the yellow circle. Next, the normalised maximum pixel value for each pinhole was found by dividing the maximum pixel value within each circle by the sum of all pixel values within the circle. A Strehl metric was then estimated by dividing the normalised maximum pixel value by the theoretical diffraction-limited value for the normalised maximum pixel value in the absence of any aberration, which was estimated using the theoretical Airy PSF convolved with the 1 µm diameter top hat function for the pinhole, and then allowing for the pixel size of the detector (6.5 µm). Finally, the Strehl metric for each pixel was displayed on a map (Voronoi diagram), where each hexagon corresponds to a specific pinhole; the Strehl metric for the pinhole marked with a blue circle in figure 3(a) is shown by the false-colour of the hexagon identified with a blue circle in figure 3(b). These Strehl-metric maps show the spatial variation of Strehl metric across the FOV and were used to evaluate the performance of the DM when refocusing the optical system by different amounts.
Ideally, the DM should refocus the objective with an as large as possible diffraction-limited FOV at each defocus. We chose to use the mean Strehl value of the pinholes contained within a central 60 µm square of the camera FOV (shown in figure 3(a) as a red square) as the metric for the optimisation algorithm.

DM performance
The Alpao DM97-15 is provided with a factory-measured influence matrix, and it is initially necessary to find the set of actuator commands that flatten the mirror, which was performed using a Shack-Hartmann wavefront sensor (HASO4VIS, Imagine Optics). To achieve this, light from a white-light source (Ocean Optics, Halogen HL-2000-FHSA) coupled via a 400 µm diameter multimode step-index fibre was collimated by an achromatic doublet (AC254-200A-ML, Thorlabs), transmitted through a 50:50 non-polarising beam splitter and incident normal to the surface of the DM. After the DM, the reflection from the 50:50 non-polarising beamsplitter was directed through a 4× de-magnifying telescope, consisting of a pair of achromatic doublets spaced by the sum of their focal lengths (AC254-200A-ML, AC254-050A-ML, Thorlabs), to create an image of the DM at the front focal plane of the Shack-Hartmann lenslet array. An iterative optimisation was then employed to flatten the DM based on the wavefront sensor measurement. The manufacturer-supplied influence matrix was used to calculate actuator commands for a particular desired surface and were added to the best flat commands in order to produce accurate representations of the desired surface. This was validated by commanding the DM to adopt a range of Zernike modes, and then confirmed using the Shack-Hartmann wavefront sensor.
The DM's viscoelastic creep, which has a time constant of the order of 6 min [21], and the heating effects of the actuator coil currents, which occur on a scale of 10 s of seconds [22], mean that the mirror response depends on the previous actuator command history, which can be a significant issue.  Even when the mirror is on average flat, the mirror still takes some time to return to the initial flat pose. This was attributed to the effect of thermal creep of the mirror surface, as the average drive current to the mirror actuators is higher when the mirror is oscillating compared to when it is flat. Figure 4(b) shows the effect of thermal creep of the mirror. The mirror was oscillated for 120 s through different levels of high-NA defocus, ranging from −10-10 µm to 50 to −50 µm, where each pose was held for 100 ms and where the mirror is always on average flat. The plots show the normalized mean Strehl ratio over the central 200 × 200 µm 2 of the image as the mirror returned to a flat pose once the oscillation had ceased (normalized to the value of the mean Strehl when the mirror was flat). The greater the amplitude of the oscillation the more heat is generated by the mirror and so the longer it takes for the mirror to flatten.
We found (data not shown) that the viscoelastic and thermal creep prevented us from optimizing the mirror unless we waited at least 1 min between each change of the mirror pose, which made the optimization of multiple mirror poses through a range of defocuses prohibitively slow (>12 h) and also meant that the mirror was not able to oscillate rapidly through the range of poses.
Provided that the time-average shape of the mirror remained constant over a timescale shorter than the time constant of the creep, then the viscoelastic creep did not present a problem. We therefore chose to oscillate the DM continuously through a series of poses chosen so that the time-averaged DM pose remained flat-we found this to be necessary to successfully optimize the DM poses. The thermal creep could be minimised by allowing the mirror to oscillate through this series of poses for several minutes prior to any measurement in order to allow the system to reach thermal steady state. Figure 4(b) shows that at least 60 s was required for this. We chose to use a period of 10 min as this would also allow any viscoelastic creep to occur if the mirror had been switched off prior to measurements.
The dependence of the thermal creep on oscillation amplitude meant re-optimisation was necessary if the amplitude or frequency of the mirror oscillation was changed.

Optimisation algorithm
The approach used involved optimising a set of control mirror poses corresponding to a discrete set of defocus positions of the star-test mask (−50, −40, …, 40, 50 µm, positive defocus is defined here in the direction from the object towards the objective). For each defocus position, first the motorized translation stage was used to move the star-test mask to the required defocus position. A range of high-NA defocus poses were then applied to the DM centred on the required defocus position and the Strehl metric measured for each one. The amount of high-NA defocus providing the highest Strehl metric was then determined from a quadratic fit to the data points spanning the maximum and three points either side of the maximum, see figure 5. This value was then applied to the DM and the Strehl metric found. If this was greater than the previous best metric, then this was taken to be the new best pose, otherwise the previous value was retained-we found that this reduced the chances of noise affecting the convergence of the optimisation. The above procedure was then carried out for Zernike modes with Noll indices from 4 to 20 in turn. The whole optimisation process was then repeated until there was no further change, which typically required four cycles.

Static optimisation
The DM was optimized initially for each amount of desired defocus in turn without continuously oscillating through a sequence of poses. The optimisation approach described in section 2.5 was modified, adding a 1 min pause after each change in mirror pose, to allow for most of the visco-elastic creep to occur, and to then apply the reverse profile to the mirror for 1 min, in order to keep the long-term average pose flat. This made the optimisation procedure very time consuming (>12 h) and changes in ambient conditions such as temperature could cause the optimisation to fail. Results obtained for the Olympus 40×/0.85 air objective are shown in figure 6. It can be seen that the largest FOV is achieved for zero refocus. The decrease in the FOV is more pronounced for positive defocus. The maximum and mean Strehl values over the central 200 × 200 µm square of the FOV as a function of defocus are shown in figure 7. We achieved a maximum Strehl of >0.8 over a range of 59 µm, better than would be expected correcting only for primary defocus (table 1). As well as applying high-NA defocus, the optimisation procedure corrected for system aberrations. The final performance was limited by field-dependent aberrations, in particular field curvature; the DM could only provide an average correction over the FOV for these kinds of aberrations.

Dynamic optimisation
In order to achieve rapid refocusing, a periodic sequence of poses was sent to the mirror at its internal update period of 65 µs. We chose to optimise the outward sweep using a sequence of 11 control poses, corresponding to defocus positions of −50-50 µm in steps of 10 µm. For the return sweep, we chose to use four (non-optimised) poses of high-NA defocus in steps of 20 µm. We then used linear interpolation of the actuator commands between each consecutive control or calculated pose to generate 38 intermediate poses in order to achieve a smooth motion of the mirror surface. This resulted in a defocus sweep over the range −50-50 µm-including return to the start position-every 38.0 ms, i.e. at a sweep rate of 26.3 Hz. The total number of poses per sweep was 585.
To avoid viscoelastic creep, an offset was applied to the calculated return poses to ensure that the temporally averaged pose of the DM over the whole cycle remained flat. The control software (MATLAB), was implemented to ensure that the DM was continually oscillating at 26.3 Hz throughout the entirety of the optimisation procedure and afterwards in order to ensure that thermal creep effects settled out and remained constant. The initial 10 min warm-up used unoptimized control poses for the calculated high-NA defocus. During optimisation, the illumination LED was synchronised with the mirror poses so that images of the star test mask were only acquired when the mirror had the pose corresponding to the control pose that was currently being optimised. This strobing method allowed the optimisation of each control pose separately. The whole optimisation procedure took approximately 90 min and re-optimisation, including re-flattening    of the DM, was required on a daily basis, which was attributed to variations in ambient temperature due to the laboratory not being air-conditioned. During a period when the conditions in the laboratory were relatively stable, we tested the performance of the optimisation of the DM at 12 h after the optimisation was performed and, in a separate experiment, at 24 h after optimisation. In the 12 h experiment, the average over all defocus values in the range −50-50 µm of the mean Strehl metric over the central 200 × 200 µm 2 region was within 4% of the value obtained immediately post optimisation. In the 24 h experiment this value was 3%. Following the optimisation, there was an evaluation step where images of the star-test mask were obtained for each defocus position with the star-test mask displaced (∆z) by −4-4 µm from the position used during optimisation, and Strehl maps obtained, see columns in figure 8(a) for the results for an Olympus 40×/0.85 air objective. The data from each defocus position were then combined to produce the best Strehl map for each optimised defocus position ( figure 8(b)). The final column (figure 8(c)) shows the ∆z value at which the best Strehl value was obtained and therefore indicates the amount of field curvature present at each optimised defocus position. The field curvature increased with defocus and reversed in sign through the focal plane. Figure 9 shows results for the 40×/0.80 W water dipping objective-where ultrasound gel was used as the immersion medium. The results also show some field curvature. Figure 7 shows a plot of (a) the maximum, and (b) the mean Strehl metric over the central 200 × 200 µm 2 of the camera FOV for both objectives as a function of optimised defocus position for both static and dynamic optimisations. For the water dipping objective, a mean Strehl of >0.6 is obtained over a defocus range of 80 µm; for the air objective this range is 45 µm. For the 40×/0.85 air objective, the performance achieved with the dynamic optimisation is lower than that achieved with the static optimisation, which we attribute to the motion of the DM during the 0.5 ms LED illumination during both optimisation and  For both the air and water dipping objectives in the dynamic optimisation, the amount of each mode applied for each defocus position is shown in figure 10. Beyond Z 13 , additional Zernike modes provided no further improvement in the mean Strehl metric. The dashed horizontal lines in this figure show the amount of each mode alone that would be required to exceed the diffraction-limit root-mean square (RMS) error in order to give an indication of the scale of the correction applied.
To further validate the performance of the system for the 40×/0.80 W objective, the DM was set to sweep at 26.3 Hz through the set of dynamically optimised poses with the camera acquiring an image of the star-test mask (with a reduced FOV) for every optimised DM pose, i.e. 11 images per oscillation of the mirror. During this process, the star-test mask was moved with a constant axial velocity from −50 to 50 µm at a rate of 6 µm s −1 . Figure 11 shows a montage of the data acquired, with the images from each sweep (defocus position) shown as a column and with data from every 30th sweep shown moving from left to right. It can be seen that the images where the star-test mask is in focus (diagonal) tracks the motion of the stage and validates the DM defocusing at video frame rate. Images over a total of 440 refocus sweeps were acquired over 16.7 s, demonstrating the capability of this approach to perform sustained refocusing.

Discussion and conclusions
We have investigated the use of an Alpao DM97-15 DM to perform rapid remote refocusing of a microscope at a rate of 26.3 Hz through a defocus range of −50-50 µm. The mirror was optimised for 11 control poses per sweep, with 4 high-NA defocus plus offset poses being used for the return sweep and with linear interpolation to generate intermediate poses. The entire optimisation procedure was conducted with the mirror oscillating continuously at the desired sweep rate in order that the temperature of the DM remained constant and so to avoid issues with thermal creep of the DM. Visco-elastic creep was avoided by keeping the temporal average profile of the mirror constant.
The performance of the system was tested using a 40×/0.85 air objective and a 40×/0.80 water objective. The air objective enabled a mean Strehl metric of >0.6 over a FOV of 200 × 200 µm 2 and for a refocus range of 40 µm to be achieved. The water objective with ultrasound gel immersion fluid achieved a mean Strehl metric of >0.6 over 200 × 200 µm 2 over a larger refocus range of 77 µm.
The results showed increasing field curvature for increasing defocus and showed that there is a limit to the amount of defocus that can be corrected with the DM for the configuration used here. The DM in our setup is unable to correct for field dependent aberrations such as field curvature in its location conjugate to the pupil of the objective. The refocus rate of the DM was limited by the power of the LED light source used during optimisation and much faster sweep rates should be achievable with a brighter source.
We plan to apply this DM refocusing system in a light-sheet fluorescence microscope (LSFM) where the illumination light sheet position is swept along the optical axis of the detection objective in synchrony with the refocusing provided by the DM in order to achieve video-rate volumetric LSFM with high fluorescence collection efficiency and further, to use the DM to correct for previously-determined/estimated depth-dependent sample-induced aberrations.

Data availability statement
The data that support the findings of this study are available upon reasonable request from the authors.