In situ retrieval and correction of aberrations in moldless lenses using Fourier ptychography

Liquid droplets cured at low temperatures or using ultraviolet light are primary approaches for fabricating refractive lenses without molds. Until now the performance of moldless lens fabrication process relied heavily on this step to precisely control the shape of each liquid droplet. Hence, a major hurdle in lenses fabricated from liquid droplets is the large variability of droplet shapes because they are sensitive to small amounts of interfacial forces. The shape of the final droplet critically affects the imaging performance of the lenses and cannot be reversed easily. Here, we aim to overcome this hurdle by performing in situ aberration correction using Fourier ptychography techniques. We demonstrate, for the first time, that computational optics can reverse high amounts of optical aberrations in moldless lenses and achieve high resolution imaging. In terms of imaging resolution, we successfully increased the resolving power of low powered moldless elastomer lenses by almost threefold, from a numerical aperture of 0.035 to 0.099. The computational approach directly elucidates the spatially varying wavefront aberrations from each lens using the same imaging system. This provides direct feedback of droplet lens fabrication techniques without the need for advanced wavefront correction methods. The application of computational imaging onto moldless lenses, using consumer digital imaging systems, lends itself to the global efforts in decentralising high resolution image intensive scientific tools to the wider community. © 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement OCIS codes: (110.1758) Computational imaging; (070.0070) Fourier optics and signal processing; (100.5070) Phase retrieval; (120.4610) Optical fabrication; (170.0110) Imaging systems. References and links 1. R. Richards-Kortum and M. Oden, “Engineering. Devices for Low-Resource Health Care,” Science 342(6162), 1055–1057 (2013). 2. R. Amarit, A. Kopwitthaya, P. Pongsoon, U. Jarujareet, K. Chaitavon, S. Porntheeraphat, S. Sumriddetchkajorn, and T. Koanantakool, “High-Quality Large-Magnification Polymer Lens from Needle Moving Technique and Thermal Assisted Moldless Fabrication Process,” PLoS One 11(1), e0146414 (2016). 3. S. Ekgasit, N. Kaewmanee, P. Jangtawee, C. Thammacharoen, and M. Donphoongpri, “Elastomeric PDMS Planoconvex Lenses Fabricated by a Confined Sessile Drop Technique,” ACS Appl. Mater. Interfaces 8(31), 20474–20482 (2016). 4. Z. Hong and R. Liang, “IR-laser assisted additive freeform optics manufacturing,” Sci. Rep. 7(1), 7145 (2017). 5. T. Kamal, R. Watkins, Z. Cen, J. Rubinstein, G. Kong, and W. M. Lee, “Design and fabrication of a passive droplet dispenser for portable high resolution imaging system,” Sci. Rep. 7, 41482 (2017). 6. W. M. Lee, A. Upadhya, P. J. Reece, and T. G. Phan, “Fabricating low cost and high performance elastomer lenses using hanging droplets,” Biomed. Opt. Express 5(5), 1626–1635 (2014). 7. S. Nagelberg, L. D. Zarzar, N. Nicolas, K. Subramanian, J. A. Kalow, V. Sresht, D. Blankschtein, G. Barbastathis, M. Kreysing, T. M. Swager, and M. Kolle, “Reconfigurable and responsive droplet-based compound micro-lenses,” Nat. Commun. 8, 14673 (2017). 8. Y. Sung, F. Campa, and W.-C. Shih, “Open-source do-it-yourself multi-color fluorescence smartphone microscopy,” Biomed. Opt. Express 8(11), 5075–5086 (2017). 9. Y. L. Sung, J. Jeang, C. H. Lee, and W. C. Shih, “Fabricating optical lenses by inkjet printing and heat-assisted in situ curing of polydimethylsiloxane for smartphone microscopy,” J. Biomed. Opt. 20(4), 047005 (2015). Vol. 26, No. 3 | 5 Feb 2018 | OPTICS EXPRESS 2708


Introduction
Decentralizing high performance optical instrument especially for low resource setting is a global challenge that is a complex engineering task [1]. While there are many ways to deliver a cost effective high resolution microscopy system, they generally fall into two main categories. The first category is to exploit economics of scale through bulk purchases which drives down the overall cost for each instrument. The second category is through decentralization where high resolution microscope can be constructed through a series of cost-efficient steps. The second approach opens up ways to tailor high performance imaging without being limited to fixed parameter determined by manufacturers. A recent surge in a variety of moldless lens techniques [2][3][4][5][6][7][8][9] opened up new lens fabrication techniques on demand, which has been proven to be economical, efficient and repeatable. This has enabled the rapid decentralization of lens-fabrication and at the same time leverage upon consumer electronics to provide high resolution imaging devices. However, the crux of existing moldless approaches is to cure transparent droplets, either polydimethylsiloxane (PDMS) droplets through thermally activated crosslinking or using UV curing. Whilst moldless lenses have dual purposes, i.e. imaging and illumination, each droplet is subjected to a range of interfacial forces, i.e. capillary forces, surface tension and gravity, before being cured. Due to the complexity of the interface forces, the majority of fabrication techniques both physical [2-5, 9, 10] or chemical [7] are focused on fine tuning of the droplet curvature. Still, moldless lenses with shorter curvatures (or high focusing power) will assume shapes closer to a hemisphere and therefore exhibit higher off-axis aberrations. Off-axis aberrations are particularly difficult to remove as it requires the shape of the lenses to have an aspheric profile [6]. All in all, these physical constraints of lens aperture and off-axis aberrations severely limit the optical resolving power of moldless lenses.
Fourier Ptychographic (FP) imaging has emerged as a new form of imaging technique that combines iterative phase retrieval methods [11,12] and non-interferometric synthetic scanning to dramatically increase the space bandwidth product (pixel density) of an image. The iterative phase retrieval approaches have been the most powerful aspect of the technique because it can retrieve the phase of an imaging system, via the pupil plane, using only intensity of the sample. The difference between FP and light-field techniques is that, in lightfield techniques, a single image is split onto an array detector that reduces the overall imaging resolution; whereas the sequential processing approach in FP increases the final imaging resolution. In comparison with other wavefront correction methods that use coherent light (laser interferometry [13]) or specialised optical devices (Shack-Hartmann, adaptive optics) [14], wavefront correction using FP imaging uses readily available devices such as partially coherent light sources (light emitting diodes) with computational methods [15]. This implies that FP can be implemented without resorting to advanced instrumentation. At its core, Fourier Ptychography (FP) borrow concepts from synthetic aperture by acquiring multiple low-resolution images taken at different angular illuminations. The illumination angles are set by a set of LEDs [16,17]. This provides diversity in each low resolution image that contributes to a different segment in the imaging Fourier space. The intensity information from each acquired image also sets the constraints and thereafter increases the possibility of convergence towards a solution and in turn retrieves the optimal phase. Ou et al [18] verified that FP is capable of retrieving the full complex function of light that has propagated through biological samples with accuracy comparable to traditional optical holography. In addition to that, they also demonstrated the retrieval of spatially varying aberrations across the imaging lens [19]. On a side note, most FP microscopy imaging systems were built on traditional microscopy platforms, with the exception of an LED array. There have been efforts made to reduce the overall footprint of a FP microscope which would undoubtedly increase their widespread use in non-specialist laboratories [20]. Here, we built upon our prior work in developing decentralized imaging systems based on moldless lenses [5] using miniature consumer computing systems (Raspberry Pi). This aligns with the primary motivation of our work of leveraging consumer devices and low cost moldless lenses to conduct high resolution imaging at low-cost and off-optical table.
We previously established that the optical aberrations in moldless lenses have large degrees of variations (peak to valley ~0.7 µm). This is especially true for those with shorter focal lengths and higher optical magnification [4,5,7]. These aberrations are more severe than traditional lenses which could be randomly distributed across the imaging plane. One approach is to devise new fabrication techniques to manipulate the droplet curvatures, and this requires extensive investment into complex processing and fabrication equipment. Instead, we apply a holistic approach to remove these optical distortions using computational techniques. We aim to apply FP techniques and reverse optical aberrations of moldless lenses within the same compact imaging system in situ. This approach would, in principle, increased the imaging performance of moldless lenses and thereby reduces the barrier for individuals to construct high resolution imaging systems of their own.
In this paper, we attempted, for the first time, to reverse the optical distortions of images acquired with elastomer lenses integrated with compact digital imaging system using FP techniques. Previous FP recovery approach uses commercial lenses and bulk microscope with low amounts of optical aberrations. In contrast, there are a much higher degree of variations in the spatial aberrations across moldless lenses. We also perform majority of the acquisition process using a fully standalone compact imaging system that is constructed using off-the-shelf components (Raspberry Pi, ICstation LED matrix -4 mm pitch). The moldless lenses were made using passive droplet approaches [5]. Here, we adopted the FP methodology proposed by Tian et al [17] who have shown that sub-level optimisation using 2nd order Newton's method are used to achieve higher speed of convergence. We observed that this approach resulted in significant improvement of imaging contrasts for large variations of offaxis aberrations across the entire imaging field of view are retrieved and corrected. Fig. 1. Optical setup for digital imaging with moldless lens. a) Schematic of imaging setup, a LED matrix used for illumination, a USAF target card is used as the sample, L1 is the imaging objective lens, L2 is the tube lens. The lenses (L1 and L2) are manufactured with polydimethylsiloxane (PDMS) droplets using passive droplet lenses dispensing process and the CMOS is a Raspberry Pi camera (without the camera lens). c) and d) are cropped images of USAF target card 1951 using two different moldless lenses as L1 lens (scale bars indicate 1 mm and 2 mm). Lens in Fig. 1(c) possesses a clear aperture of width 3 mm and focal length of 12 mm and lens in Fig. 1(d) has a diameter of ~2.8 mm and focal length of ~13 mm. e) shows the wider field of view of image shown in Fig. 1 Recently, Kim et al [21] began constructing compact FP systems using traditional objective lenses and miniature scientific cameras to fit into an bio-incubator for longitudinal imaging of biological samples. In contrast, we used consumer equipment and lens made from a passive droplet dispenser. The passive droplet dispenser is a set of 3D printed tools that is capable of dispensing multiple droplets of PDMS simultaneously with minimal operator training [5]. The dispenser includes a droplet holder that is designed to balance the surface forces such that each droplet retains a parabolic shape during curing at a temperature of 70°C. Additional details of the fabrication techniques using moldless lenses can be found elsewhere [3,5,9].

Compact imaging system with moldless lenses
Next, we describe the optical setup used to conduct FP. Figure 1(a) provides the layout of the illumination and imaging components of the imaging setup. An array of LEDs is first placed at a distance (80 mm) away from the sample. The distance is to ensure that there is sufficient overlap in the Fourier plane among adjacent LEDs. Light emerging from sample are collected by moldless lenses and focused onto the imaging sensor (CMOS) whilst maintaining the appropriate coherence. Figure 1(a) shows a compound lens arrangement with two thin PDMS lenses (L1 and L2) where L1 acts as an objective lens and L2 as the relay lens. The tube lens sets up the appropriate magnification on the imaging sensor (CMOS). The traditional imaging NA based on Rayleigh criterion can be defined by smallest resolvable spot [16]. On the other hand, the maximum synthetic NA of a FP system is set by maximum angle between optical axis and LED. In an ideal FP recovery, the NA from the Rayleigh criterion would match the maximum synthetic NA [16]. Figure 1(b) shows the imaging system constructed onto an optical table. The whole footprint of the imaging system is 120 mm by 120 mm by 150 mm. The majority of the setup is to accommodate the LED matrix (96 mm by 96 mm). Figures 1(c) and 1(d) show two digitally cropped images of USAF 1951 target card imaged using two moldless lenses. The two types of moldless lenses are shown in the insets of Figs. 1(c) and 1(d). The focal lengths of the lenses are of 12 mm and 13 mm respectively. Figure 1(e) shows the uncropped image of Fig. 1(d) that displays a large amount of optical distortions at the perimeter of the lenses. We aim to remove majority of the optical distortions using the same imaging system i.e. in situ. The resolution performance of an FP system is highly dependent on identifying optical and digital sampling parameters such as numerical aperture, optical magnification, pixel size and optical wavelength. We first determined the appropriate sampling parameters by using an aspheric lens (Thorlabs 352280 -A, focal length = 18 mm and NA 0.15) with similar focal length as the moldless lenses. The aspheric lens is placed as an imaging lens (L1) as shown in Fig. 1(a). A high quality objective lens is used as the relay lens (L2). The combination of the two lenses gave a total magnification of 2.6. The magnification factor has been calculated by dividing the size of the image projected onto the CMOS sensor with the actual size of the object. For the illumination, we also programmed a circular set of LEDs to conform to the circular symmetry of the lens aperture.

FP imaging with molded small diameter aspheric lens
Maximum angle between optical axis and LED provide a calculated synthetic NA of the 0.39. From the full imaging field of view and the expected imaging resolution, we can redefine the size of matrix of the improved image after FP optimisation. A large part of FP optimisation is based on iterative phase retrieval algorithms that is similar to established techniques i.e. Gerchberg-Saxton (GS) [12] and later improved by Fienup [11,22]. Lately, a second order Newton's method was implemented in FP imaging which results in lower iterations and lesser image artifacts [23]. These methods are capable of retrieving phase information from intensity images only and utilized the first-order derivative of the cost function for convergence to a solution. In this work, we adopted Levenberg-Marquardt based second-order Newton's method previously used in multiplexed illumination-based FP systems [17,23]. Figure 2(a) shows the positions of three LEDs that were used to illuminate the sample from different positions. Figure 2(b) shows the corresponding images from each LED (LED 1, 2 and 3) imaged by the commercial aspheric lens. The central LED is coincident with the central axis of the CMOS sensor. For each LED, a low resolution image of a fixed region of interest is recorded and then optimized by an iterative phase retrieval process [17]. The size of the ROI is determined by coherence illumination area of the LED; calculated based on the wavelength of the LED, λ, height between the LED matrix, z and the LED illumination area d [24]. The expected ROI is chosen to be around 280 µm by 280 µm, which translates to 200 × 200 pixels. We note that each LED element used in the imaging setup is approximately 4 mm in diameter which was chosen to maximise illumination intensity. In doing so, the spatial coherence of the system could be lower than previous FPM which uses smaller LED elements ~200 µm [16]. The amount of overlap between neighbouring spatial frequency spectrums generated by individual LEDs also affects the FP process. To ensure sufficient overlap, the relationship of the separation between each adjacent LEDs (s) and the height (h) between sample and LED is determined by Eq. (1) [25].
According to [25], for R overlap greater than 31.81%, there was noticeable improvement in the rate of convergence. This was also previously mentioned that a minimum of ~35% overlap is required for quality reconstruction [26]. The calculated values of R overlap have been articulated in Table 1. Figure 2(c) illustrates a representation of the iterative process in a graphical flow chart based on each ROI. Apart from defining the ROI, the background noises from images are to be removed. For each ROI, the process iterates for n Iter times. For each iteration, we calculate the difference between the actual and estimated intensity in each image by subtraction. The goal is to have minimal intensity difference of the intensity between each iteration. After each iteration (i), the optimised complex intensity function is used to fill up the Fourier space. In this approach the least square error optimization has been used where the squared difference between measured and actual measurement is minimized per iteration. The whole reconstruction process runs for nIter times for all the LEDs (all of 293), filling up the Fourier space with higher frequency information. The pupil function, P is retrieved from each ROI.
Next, we describe in detail each step within a single iteration i. At each iteration i, for a given LED q, we calculate the transmitted wave exiting from imaging lens to be ( ) , where I measured is corrected q-th raw image. The difference ( ∇ ) between the new and previous transmitted complex intensity for each q-th image is used to update the object and pupil functions. The Tikhonov regularization parameters constraints required for object and pupil recovery are the two independent parameters, α and β shown in Eq. (2) and Eq. (3) [27]. The ratio of α and β is important to ensure convergence and quality of reconstruction, which has been observed that for our imaging setup 10 α β  ensures convergence. The iteration terminates when the calculated error reaches a predefined minimum or the number of iterations reaches a predefined value (~90 as suggested in [23]). We chose the latter to stop the iteration due to the high degree of variations between each image where the error values alter significantly. After each ROIs are optimised, they are stitched together to generate the full FOV image which is shown in Fig. 2(d)-(i). We also compared the improvement of image contrast before and after FP optimisation in Fig. 2(d)-(ii) by plotting the intensity profile of image of elements 5 and 6 of group 7 with structure width of 2.46 µm and 2.19 µm respectively. From the line plot, we calculated a 38.5% improvement in imaging contrast. The doubling of spatial product bandwidth after FP reconstruction showed improvement in both imaging contrast and resolution, as shown in Fig. 2(d)-(iii). In circular symmetrical lenses, it may be expected that the majority of spatial aberrations arises from the non-convergence marginal and paraxial rays. Spatial aberrations in moldless lenses generate undesirable image distortions over the whole FOV especially along the peripheral of the lenses. Commercial lenses have small sources of aberrations because of more stringent fabrication standards. On the other hand, moldless lenses can exhibit a large degree of aberrations due to non-standardised fabrication processes [4][5][6]10]. Although conventional wavefront correction methods [14] that have shown the ability to reverse higher amounts of optical aberrations, the measurement tools do not offer any correction in situ. On the other hand, FP have demonstrated the success in correcting image distortions and aberrations in commercial fabricated objective lenses by retrieving optical aberrations in situ by using the same imaging platform system [15,17]. However, the feasibility of FP imaging techniques in retrieving aberrations in moldless lenses, with much higher optical aberrations, has yet to be studied. A clear benefit for FP imaging could be used to remove large variations in optical aberrations across the lenses through the images without resorting to conventional bulky wavefront correction devices. The use of FP to perform phase retrieval satisfies the size constraints for a portable imaging system but at the cost of computational time. In our experiment, the time for convergence could take up to 20 mins after 90 complete iterations. We also summarised the values of RL and R overlap for the two types of moldless lenses, as shown in Figs. 1(c) and 1(d), in Table 1. Based on the NA of the two moldless lenses, we calculated R overlap of 73% and 68%, respectively that adequately fulfilled conditions for FP process. Next, we describe two types of imaging setup; a single lens and a compound lenses arrangement, each offering different optical magnifications as described in Table 1. We calculate the magnification using the same approach previously described. Figure 3(a) shows the schematic of a single moldless lens imaging system that exhibits non-flat field of view that provides a magnification of 1.75x. The pincushion effect is clearly evident over the whole field of view. This is the result of high spherical aberrations as shown in Figs. 3(b) and 3(c) where there are two or more image planes. Using the FP methodology, described before in Fig. 2, we successfully reconstructed the whole field of view that is both flat and possesses minimal on and off axis aberrations that is shown in Fig. 3(d). Due to the variability of the aberrations, we empirically observed that there was a need to vary the initial guess for each ROI [23]. The optimization of the initial guess for each ROI is crucial to optimally retrieve the full FOV. This has been summarized in the Appendix. To highlight the amount of improvements, we compared, in Fig. 3(e) a cropped region (group 7 element 1) with an image of the same region before and after FP retrieval as well as the corresponding line profile of the intensity to quantify the resolution and contrast improvement. The line plot demonstrates that only after FP, the structures were resolved with the modulation transfer function (MTF) of 81.9%. Figure 3(f) shows two compound lenses arrangement with two set of moldless lenses. The combination of the lenses was shown to increase the magnification to 3.5x. As compared with Fig. 3(b), the second lens arrangement shown in Fig. 3(g) was able to image much finer features on the USAF1951 card as shown in Fig. 3(h). Using the same FP routine, we show the retrieval and reversal of the optical aberrations in Fig. 3(i). Again, we highlight the improvement by comparing the intensity image (group 7 element 3) with its corresponding MTF as shown in Fig. 3(j). The MTF for the second lens-set arrangement is given by 82%. The single moldless lens and compound moldless lens achieved an imaging resolution of ~3.91 µm and ~2.76 µm respectively that was not resolvable before FP correction.  (Fig. 3(a)), compound lenses (a pair of moldless lenses- Fig. 3(g)), aspheric lens (shown in Fig. 2) and an objective 5x lens (not shown).

FP on moldless elastomeric lenses
Since the FP method operates by optimising on a localized ROI, this provides the possibility of recovering the pupil function from each image segment. In conventional wavefront sensing process, one would require taking multiple laser scanning patterns. But here, it is possible to retrieve the aberration mask directly. In Figs. 4(a) and 4(b), we consolidated the aberrations from on axis to off axis positions across the sample for two different imaging setups. We represent the different aberrations in the form of pupil functions from each of five positions including the central field of view (on-axis) and four at different corners of the image (off-axis). The pupil functions here are derived from approximate amplitude of orthogonal Zernike polynomials. In both cases, the pupil functions vary significantly across the lenses. The aberration has been summarised in Fig. 4(c) where we compare the aberration amplitude (peak to valley) of the moldless lenses with the aspheric lens (Fig. 2) and a commercial microscopic objective 5x. It is noteworthy that the magnitude of aberrations for the objective 5x is significantly lower than the others (~0.02).
A major focus in the field of aberration correction is the measurement and optimisation of correction methods so that phase distortion can be efficiently corrected in situ. When the optical aberrations vary across an imaging field in both amplitude and frequency, the optimization process would find it difficult to predict the correct optical aberrations. Until now, FP aberration correction results, have only been demonstrated for the removal of lower order aberrations (Zernike modes <10) which are predictable aberrations in conventional lenses [15,20]. To investigate further, we show the randomly distributed aberration modes across the moldless lenses by summarizing them into Zernike modes (for data set shown in Fig. 3(a)). Figure 5 shows that FP can remove much higher order aberrations (Zernike >10) that are randomly distributed across the entire field of view. Such large amount of optical aberrations across the sample is high and less predictable than conventional lenses. From our assessment of the images based on Rayleigh criterion, we measured that the imaging NA has improved by 2.8 folds (from 0.035 to 0.099). FP has clearly restored many off-axis features that were not visible. However, by comparing with the synthetic NA of 0.39, FP was not able to fully recover all the distortions. There are few likely possible reasons. One of which is that the illumination NA is not fully satisfied because of small aperture of the imaging lens and larger LED illumination area. Another reason could be, is likely attributed by even higher distribution of aberration modes larger than 35, that has not been used in the reconstruction process. We did not perform this due to extensive computational cost involved.
In the next step, we shall employ larger aperture moldless lenses as well as FP phase diversity [28] techniques that can provide more accurate phase information.

Conclusion
The simplicity of fabrication of moldless lenses provides a low cost alternative to create high quality lenses [5], but this comes at the cost of variability over the optical performance. In this work, we demonstrated, for the first time, the power of FP imaging in retrieving and reversing aberrations in moldless lenses and thereby retrieve the full field of view with imaging resolution, from 0.035 to 0.099, and SBP, from 5 megapixels to 15 megapixels. Hence, this advantage provides a new avenue for moldless optics to increase the imaging space product bandwidth which cannot be achieved through improvements in lens fabrication. This work shows that computational imaging can overcome large aberrations in moldless lenses without resorting to classical wavefront correction. This work paves ways to development of numerous low cost high performance imaging systems using moldless lenses. From the results, we showed that FP imaging can be used on single or multi-lensed moldless lens-based systems with resolution improvement by approximately 3 folds, recovering almost all off-axis aberrations. A current drawback of the FP imaging is the lengthy time during acquisition but that could be solved by parallel processing i.e. increasing the number of microprocessor in the system. Due to high distortion and aberrations spanning across the curvature of the lenses the resolution improvement has been limited; this discrepancy could be addressed by employing phase diversity approaches and moldless lenses of larger apertures. Furthermore, we plan to use smaller LED areas to increase the spatial coherence of the system to improve the FP reconstruction performance on our imaging system. In our next work, we plan to make use of the concept to develop a low-cost, portable, microscopic pathology scanner by using of an array of the existing setup. Although the idea of digital pathology using FP has been discussed in [29], it uses the existing microscope platforms. Whereas, by integrating low cost moldless lenses imaging system into several compact Raspberry Pi based systems, there is potential of creating a disposable pathology slide scanner. This will aid high throughput digital imaging and open up new opportunities in development of biomedical applications for the developing world.

Appendix A: Demonstration of the impact of an initial guess on an object
In Fig. 6, we demonstrated how the choice of an initial guess plays a crucial outcome for optimization of the final imaging resolution. We found that the choice of initial guess for a particular region of interest (ROI) needed special care. Initial guess as discussed in [17] works best from an image acquired during the experimentation that closely resembles the final image. From the set of images, we selected an image closest to the solution such as to provide the optimum solution. For ROI, group 4 element 6, we showed 3 sample initial guesses acquired from 3 different brightfield images. From the results, we observed that the one closer to the solution, gives us the best reconstruction results (in this case the guess number = 8). The raw and reconstructed image contrasts (MTF) demonstrate that there have been improvements in imaging contrast.