Ultralow-light-level color image reconstruction using high-efficiency plasmonic metasurface mosaic filters

As single-photon imaging becomes progressively more commonplace in sensing applications such as low-light-level imaging, three-dimensional profiling, and fluorescence imaging, there exist a number of fields where multispectral information can also be exploited, e.g., in environmental monitoring and target identification. We have fabricated a high-transmittance mosaic filter array, where each optical filter was composed of a plasmonic metasurface fabricated in a single lithographic step. This plasmonic metasurface design utilized an array of elliptical and circular nanoholes, which produced enhanced optical coupling between multiple plasmonic interactions. The resulting metasurfaces produced narrow bandpass filters for blue, green, and red light with peak transmission efficiencies of 79%, 75%, and 68%, respectively. After the three metasurface filter designs were arranged in a 64×64 format random mosaic pattern, this mosaic filter was directly integrated onto a CMOS single-photon avalanche diode detector array. Color images were then reconstructed at light levels as low as approximately 5 photons per pixel, on average, via the simultaneous acquisition of low-photon multispectral data using both three-color active laser illumination and a broadband white-light illumination source.


INTRODUCTION
Single-photon imaging is a rapidly expanding field of research with emerging applications in three-dimensional profiling [1], kilometer-range depth profiling [2], three-dimensional imaging of moving targets [3], astrophysics [4], and fluorescence lifetime imaging (FLIM) [5].Simultaneous single-photon acquisition of multiple wavelengths has been used for target identification [6][7][8][9] and physiological foliage parameter measurement [10]; however, these applications involve complex and lossy optical routing of the received signal to independently acquire discrete multiple wavelengths.Large-format complementary metal-oxide semiconductor (CMOS) single-photon avalanche diode (SPAD) arrays provide a new approach for fast image acquisition.These large-format SPAD arrays are becoming more technically feasible [11] and are already finding use in areas such as 2D imaging [12], depth profiling [13,14], and FLIM [15,16].
Higher transmission and low out-of-band rejection produce better color contrast and selectivity when metasurfaces are used as visible color filters.Plasmonic metasurfaces have the advantage of requiring only a single lithography step to filter any wavelength, and they are significantly thinner and easier to fabricate compared with alternative technologies such as dichroic films and polymer dye filters.Dichroic filters require different thicknesses for different wavelengths; additionally, for both dichroic and dye filters, multiple lithographic steps are required when filtering multiple wavelengths.Indeed, the need for only a single lithography step has significant benefits in the design and fabrication of color filter arrays, enabling the freedom to alter filter designs without the need for complex process modifications.The number of fabrication processes required by more mature color filter technologies, such as thin-film or dye-based filters, increases with the number of filter designs required, restricting most arrays to three-color designs.The development of high-quality plasmonic-metasurface-based filters, however, would permit the inclusion of filters selected for any number of wavelengths, including the near-IR and shortwave IR regions [44], enabling hyperspectral imaging.It is also relatively simple to incorporate additional optical features into plasmonic metasurfaces, such as polarization sensitivity for multispectral polarimetry, or even lensing to increase photodetection.Furthermore, plasmonic metasurfaces are more robust than existing technologies, as they are not susceptible to degradation after long-term exposure to UV light, high temperatures, or humidity [40], and could therefore be used in more extreme environments.
Plasmonic filters have been fabricated using nanoholes [45-47] and coaxial nanoholes [47]; however, the optical transmission is poor at ∼45%.Higher optical transmission can be achieved using asymmetric cross voids [33,48] and gratings [49][50][51] (∼60%), though such structures are highly polarization dependent and there is generally a trade-off between the bandwidth of the filter and the level of peak transmission [44].This means that we may have to reduce the overall transmission to achieve narrower bandpass features.Polarization insensitivity with 60% transmission has been achieved using circular nanoholes [52] in a metal-insulator-metal configuration, but this design required a complex, multistage fabrication approach and had large out-of-band transmission (poor rejection ratios).

PLASMONIC METASURFACE DESIGN A. Theory and Simulation
Herein, we present a design that requires only a single lithography step, is polarization insensitive, and (to our knowledge) has the highest recorded transmission in the visible spectrum.In this design, ellipses alternately oriented at 45 • and 135 • , together with circular nanoholes, were defined in a thin sheet of aluminum [44,53].The holes were created by etching the metal film and then extending the etch into the substrate below the film.Figure 1(a) shows a schematic of the plasmonic metasurface design.By scaling the dimensions of both the elliptical and circular nanoholes, different wavelengths of light can be selectively transmitted.The unit cell, shown in Fig. 1(b), relies on the design principles presented in A ratio b a of 0.85 was used to maintain polarization insensitivity [44].Circular nanoholes were created with control radii r 1 and r 2 .(c) As shown in the schematic, unpolarized light (E i x ,y ) is normally incident in the k z direction.The nanoholes are over-etched into the substrate by 50 nm.The Al thickness, d , is 70 nm.[44], wherein the unit cell period p is obtained from the following equation [54]: where λ SPP is the wavelength corresponding to the transmission dip that occurs immediately before the extraordinary optical transmission (EOT) [54] at the desired transmission wavelength, (i, j ) is the diffraction grating order, which is (1, 0) in our case, and ε m and ε d are the permittivities of the metal and dielectric, respectively.In order to obtain a high E-field confinement, the nanoholes must be subwavelength [44], so the short axis of the elliptical nanoholes (b) were initially approximated as λ/3.The ratio of the long (a ) to short (b) axis of the elliptical holes is fixed at 0.85, because a ratio b a between 0.8 and 0.9 is shown to maintain polarization insensitivity [44].In addition to these ellipses, six circular holes are present; the middle holes have radius r 1 and the four corner holes have radius r 2 [see Fig. 1(b)].In this study we used two designs, designated as A and B. For design A, r 1 = r 2 = r such that 2r a ≈ 1, whereas for design B, 2r 1 ≈ a , and r 2 < r 1 .From these starting values, simulations were performed using Lumerical finite-difference time domain (FDTD) with the geometry of b varied in steps of ±10 nm, with a and r altered according to the relations stated.A 250 nm SiO 2 cap layer is required to ensure the normalized optical impedance of the metasurface is approximately 1 [42,44], resulting in maximum transmission at resonant wavelengths with minimal backreflection.The cross section of a typical metasurface is shown in Fig. 1(c) with the unpolarized light, E i x ,y , normally incident from the substrate onto the nanoholes.By varying the period p and scaling the dimensions of the nanoholes, metasurfaces were designed to select for blue, green, and red light.As a final consideration, it was necessary to choose a suitable metal film thickness d .
For this work, 70 nm was chosen based on an experimental study, presented in Fig. S1 of Supplement 1.
The high transmission observed in these devices is attributed to several factors, one of which is the over-etching into the substrate.The differences in the spectral response between the over-etched and optimally etched devices are shown in Figs.S2, S3, and S4 in Supplement 1.The increased transmission of the over-etched devices compared to that of the standard etched devices is ascribed to the incident light being scattered more efficiently into the nanoholes [55].The incident light is found to combine with quasi-cylindrical waves (QCWs) launched from the nanoholes, as E i x ,y interacts with the guided modes confined in the nanoholes.A closer analysis of the design reveals that destructive interference between QCWs [56][57][58][59] and surface plasmon polaritons (SPPs) oscillating on the surface produces a dip in transmission, aiding out-of-band rejection, while a high transmission is observed due to constructive coupling between the QCW, SPPs, and localized surface plasmon resonance (LSPR).This differs from the standard designs, in which only the resonant coupling between the incident light and SPPs contributes to the EOT [54].

B. Fabrication
The substrate was a borosilicate glass slide with the dimensions 20 mm × 20 mm and a thickness of 515 µm.Dry-etching tools were used to etch the elliptical and circular nanohole designs in a thin layer of aluminum.70 nm of Al was deposited using a Plassys electron beam (e-beam) evaporator, followed by an inductively coupled plasma deposition of a 50 nm thick Si 3 N 4 film.The Si 3 N 4 acted as an adhesion layer for the e-beam resist ZEP520A.The e-beam resist was spin coated at 2250 rpm for 60 s and baked in the oven at 18 • C for 40 min.Exposed patterns were developed in o -Xylene for 35 s at 23.5 • C, followed by a rinse in isopropyl alcohol (IPA) for 30 s.An oxygen ash in the Oxford Instruments Plasmalab 80 Plus reactive ion etcher (RIE) was followed by a Si 3 N 4 etch using CHF 3 /O 2 chemistry.Immediately after the Si 3 N 4 etch, the samples were transferred into another RIE etcher (Oxford Instruments PlasmaPro 100 System) to etch the Al layer using a SiCl 4 chemistry.In order to achieve the over-etching into the substrate, the samples were given another CHF 3 /O 2 etch (80 Plus RIE), followed by an O 2 ash and another CHF 3 /O 2 etch (1 min 40 s) in the same RIE to remove any residue of the ZEP-520A resist and the Si 3 N 4 adhesion layer, respectively.The substrates were then cleaned in acetone and IPA and imaged with the FEI Nova NanoSEM 630.A cap layer of 250 nm of silicon dioxide was deposited on top of the Al layer using a plasma-enhanced chemical vapor deposition process.

C. RGB Design Specs
Figure 2 shows scanning electron microscopy images along with experimental versus simulated transmission spectra of each filter.Blue and green color filters followed design A specifications, while design B was found to achieve a higher transmission and narrower linewidth for the red filter.The full design specifications for each filter are provided in Table 1.It was found that for dimensions smaller than the proposed designs, we observe a reduction in transmission, whereas for larger dimensions we observe no increase in transmission but an exponential increase in bandwidth and hence a reduction in color fidelity.A 10% increase in transmission is observed for the red filter using design B over design A, due to the higher E-field confinement in the larger elliptical holes combined with a narrower LSPR that is expected from smaller circular holes (as shown in Fig. S5 of Supplement 1).When applied to the green and the blue filters, the same modification did not lead to an improvement, as described in Fig. S6 of Supplement 1.

MOSAIC FILTER FABRICATION
High-transmission plasmonic metasurface filters are ideal for creating mosaic filters, particularly in applications where the signals are very weak.To highlight the practicality of using such a device, three designs for blue, green, and red were randomly arranged in a large area to form a mosaic filter covering the active regions of a 64 × 64 SPAD array [60], with each design covering one pixel of the array.The mosaic filter in this instance uses a random pattern of approximately equal quantities (33.33%) of each design (blue, green, and red).The arrangement is further explained in Supplement 1.The random location of the three designs in the mosaic filter reduces the probability of false coloring and Moiré interference [61] when the full RGB image is reconstructed.Although the image reconstruction step, known as demosaicing, can become more complex when using random masks, the additional overhead is negligible in practice and this approach remains beneficial, as the reduction in errors such as Moiré patterns related to periodic color features in a target scene can be valuable [62].Note that if a larger number of designs are used, it might be more efficient to adopt a pseudorandom approach to filter allocation to ensure homogeneous coverage of each design.In that case, mask design tools for coded apertures [63] may be used.

SINGLE-PHOTON IMAGING
The resulting mosaic filter was integrated with a SPAD image sensor, which was used to perform multispectral imaging of targets in two different approaches.Targets were imaged under active, targeted laser illumination and under passive, white-light flood illumination.

A. Filter-SPAD Integration
The image sensor was a 64 × 64 Si CMOS SPAD array with a 61.5 µm pixel pitch and a square active area with a 11.6 µm side length, resulting in a fill factor of 3.55% [60].The imager used in this work is capable of counting single photons within a predetermined frame duration, with no precise timing circuitry.An ACCµRA100 flip-chip bonder was used to integrate the two components with the use of a UV-sensitive optical adhesive.The 11.6 µm active area within a 61.5 µm pixel pitch allows a linear alignment tolerance of ∼25 µm in each direction.The flip-chip bonder provides a consistent bonding accuracy of ≤1 µm, while the e-beam write process has a maximum error of ∼0.5 µm in the positioning of each filter.These small error margins, compared to the ∼25 µm tolerance, ensure there is no filter cross talk between adjacent pixels in this integrated assembly.An example of an integrated device is shown in Fig. 3, and further details about the flip-chip bonding process are provided in Section 4 of Supplement 1.

B. Active Laser Illumination
An NKT supercontinuum laser combined with an acousto-optic tunable filter provided the illumination source for the active imaging system.This allowed the transmission spectra of each filter set (R, G, B) to be recorded by the illumination of the device at discrete wavelengths from λ = 460 nm to λ = 1100 nm.A 4f optical system is employed in combination with a pair of engineered diffusers (D1, D2) to ensure a flat, uniform illumination profile and a central variable aperture stop to control the f -number.The optical system is shown in Fig. 4.
The transmission of each pixel was calculated as the number of counts recorded at each wavelength, divided by the average number of counts recorded by an identical device without an integrated filter under the same illumination conditions.Hot pixels-those with significantly higher than average dark count rates-can skew   results if overlooked.All such pixels are identified from the detector dark count distributions using the method described in [64] and discarded prior to any analysis being conducted.This led to the removal of 206 (5%) of the pixels on the mosaic filter bonded chip and 149 (3.6%) of those on the bare chip.

F-Number
The average transmission spectra of each filter set are shown in Fig. 5 at f /8 and f /2, corresponding to maximum angles of incidence (i.e., half cone angles) of 3.6 • and 14.0 • , respectively.The spectra show negligible angular sensitivity with respect to transmission and peak wavelength shift, which are typically found in thin-film-based bandpass filters, demonstrating the robustness of the metasurface design.The peak transmission wavelengths were identified at λ = 470 nm for the blue filter set, λ = 580 for the green filter set, and λ = 680 nm for the red filter set.

Uniformity
The filters demonstrate high levels of uniformity, which were determined by measuring each of the three filter sets to provide a histogram of the peak transmission values.These histograms were fitted using a Gaussian, as shown in Fig. 6.The standard deviation  σ of the transmission values of each filter set is equivalent to that of the pixels in the bare chip over all wavelengths.We can therefore attribute the variation in transmission peak values predominantly to non-uniformities in SPAD detection efficiency, rather than any spread in peak transmission across the metasurface filters.

Imaging
The optical setup used for target imaging is shown in Fig. 7, with two diffusers (D1, D2) used to expand the beam and ensure uniform illumination and a 75 mm focal length lens used to direct the light on the target scene located at an ∼50 cm stand-off distance.A 50 mm focal length Canon single lens reflex (SLR) camera lens was used to image the return photons from the target onto the SPAD array.The NKT supercontinuum laser source provides the three-color illumination of the target scene, emitting the three discrete wavelengths of peak filter transmission simultaneously: λ = 470, 580, and 680 nm.A stand-off distance of ∼50 cm was used, resulting in a field of view of ∼3 cm × 3 cm.
The SPAD imager was initially set to record 100 frames at 5 ms exposure per frame.These frames were then summed to provide a total exposure time of 500 ms per target.Figure 9 shows the corresponding raw intensity measurements prior to any image reconstruction or preprocessing.
Using the proposed image reconstruction algorithm (described in more detail in Section 5), we are able to extract a set of RGB values for each pixel.Examples of color reconstructions are shown in Fig. 10.In order to study the number of photons required to extract high-fidelity color images, additional measurements were acquired using the toy figure with reduced total exposure time.The average detected photons per pixel were calculated from returns on the target only, ignoring those returning from the white backplane, which would normally be expected to return a far higher number of photons.In Fig. 11 it can be seen that a reasonable color quality is maintained down to as few as ∼20 detected photons per pixel (i.e., ∼7 photons per channel, on average), using a single 5 ms exposure.Below this level, significant image degradation is observed.Combined background photons and dark counts contribute on average 0.67 counts/ms per pixel.It is worth noting here that when 5 photons per pixel are detected on average, the impact of shot noise becomes significant and the image restoration algorithm must still decide how to allocate these potentially spurious counts to the three RGB channels.

C. Passive White-Light Illumination
While laser illumination is useful for many applications including lidar, other areas such as astronomy will require a passive imaging approach.To demonstrate passive imaging, a thermal white-light source was introduced to flood illuminate the scene, replacing the direct laser illumination and its optics.
Data were collected using the same five targets, with the illumination power set approximately equal to that used for the active imaging system, determined by count rate recorded by the detector.A 500 ms total exposure time was used, resulting in ∼2500 average photons per pixel returning from the targets (compared to 2176 at the same exposure with the laser).The resulting reconstructions are provided in Fig. 12.
The differences in terms of image reconstruction, compared to Fig. 10, are clear.The broadband nature of the white-light source accesses the entire spectral width of each filter, allowing for a much more realistic reconstruction of the white parts of the targets, which proved difficult to obtain using the three discrete wavelengths of the active illumination setup.This is mitigated, however, by the narrowband nature of the laser illumination, which effectively reduces the spectral broadness of the filters, providing sharper colors and a greater contrast.

IMAGE RECONSTRUCTION ALGORITHM
To extract color information from a single intensity reading per pixel, we model I (n), the measured intensity in the nth pixel, as a linear combination of the contributions of the individual illumination wavelengths and background as follows: where the parameters {A i (n), B(n)} are obtained during the calibration.Here, A i (n) represents the transmission coefficient A of the filter over pixel n at illumination wavelength λ i .The parameter I i (n) represents the incident light intensity on pixel n at that wavelength, and B(n) represents the background counts at that pixel.By considering the intensities recorded in neighboring pixels, in combination with an inpainting (matrix completion) technique, it was possible to recover {I 1 (n), I 2 (n), I 3 (n)} for each pixel from the single intensity reading.The estimation problem based on the equation above can be recast as a linear source separation problem with compressively sampled measurements, and it can be solved efficiently using, for instance, convex optimization techniques.
Here we developed an algorithm similar to that used in earlier work, where the inpainting is performed using a total-variation optimization, which promotes smooth intensity variations [7].In this work, the method was adapted to use multiple A t values that were assumed to be nonzero in each filter.The contributions from each wavelength are then considered to represent the three constituents of the RGB color model [65] and combined accordingly to provide the "true" color.

CONCLUSION
We have demonstrated plasmonic metasurface wavelength filters comprising nanocircles and nanoellipses that achieve ultrahigh transmission efficiency for blue, green, and red light by simultaneously enhancing three optical phenomena: LSPR, SPPs, and QCWs.Transmission efficiencies of 68%, 75%, and 79% were achieved for red, green, and blue wavelengths, respectively.One advantage of the single lithography approach using plasmonic metasurfaces is that we are not limited to only three colors, but rather can access a range that covers the visible and IR parts of the spectrum by simply changing the geometry of the elliptical and circular nanoholes and the unit cell period.
To demonstrate the applicability of these metasurfaces to color imaging on a single focal plane array, a mosaic filter array was fabricated and flip-chip bonded to a 64 × 64 SPAD array to provide a robust, compact integrated detector assembly.Active and passive color imaging in the sparse photon regime was demonstrated using a single exposure.This work demonstrates the potential benefits of using metasurface mosaic filter arrays, which may be used to enhance a number of low-light-level imaging applications.These applications could include lidar-based 3D imaging, low-lightlevel color passive imaging, or FLIM.The potential for utilizing metasurfaces in this manner extends beyond simple spectral filtering, due to the ability of metasurfaces to incorporate multiple optical features such as micro-concentration or phase shifting.The metasurfaces described in this paper were deliberately designed for polarization insensitivity; however, simple alterations to the shape and orientation of the elliptical nanoholes allow polarization control, which can be utilized for full-field polarimetric imaging.Furthermore, the metasurface color filter array designs may even be realized using photolithography rather than the e-beam lithography presented here, with modern techniques able to attain sub-10-nm feature sizes [66].
To the best of our knowledge, this represents the first singlephoton color image reconstruction achieved using a SPAD image sensor integrated with a mosaic filter array, thus facilitating the rapid acquisition of sparse photon color images in both two and three dimensions.

Fig. 1 .
Fig. 1.Schematic of the proposed plasmonic metasurface design.(a) Schematic of the visible plasmonic metasurface designed on a borosilicate substrate.(b) The unit cell is highlighted with period p, long axis a , and short axis b of the elliptical nanohole array.A ratio b a of 0.85 was used to maintain polarization insensitivity [44].Circular nanoholes were created with control radii r 1 and r 2 .(c) As shown in the schematic, unpolarized light (E i x ,y ) is normally incident in the k z direction.The nanoholes are over-etched into the substrate by 50 nm.The Al thickness, d , is 70 nm.

Fig. 2 .
Fig. 2. Characterization of the fabricated visible metasurface filters.Scanning electron microscopy (SEM) images and micrographs (insets) of the fabricated metasurface design for (a) blue, (b) green, and (c) red color filters.Experimental and Lumerical FDTD transmission spectra of the (d) blue, (e) green, and (f ) red plasmonic metasurfaces.The difference in transmission between design A and design B for red plasmonic metasurface is shown.

Fig. 4 .
Fig. 4. 4f optical sytem used for filter transmission measurements and calibration.The variable aperture stop allowed varying-illumination f -numbers to be incident on the SPAD array.

Fig. 6 .
Fig. 6.Uniformity of each of the three filter sets measured at (a) λ = 500 nm, (b) λ = 580 nm, and (c) λ = 680 nm.(d) Normalized distribution of average counts in the bare chip at all three wavelengths.In each case the data are displayed as transmission histograms with the Gaussian trend fitted.The standard deviation σ of the fit is provided for each case.

Fig. 7 .
Fig. 7. Active imaging system in a bistatic configuration.The illumination is provided by a tunable laser source, diffusers (D1, D2), and an illumination lens.The receiving channel is composed of an objective lens ( f = 50 mm, f /1.8) and the SPAD array.

Fig. 8 .
Fig. 8. Targets used for imaging with multispectral laser illumination.Dashed lines on (a) and (b) represent approximate camera fields of view of 3 cm × 3 cm.

Fig. 9 .
Fig. 9. Raw intensity images from the SPAD imager for multispectral laser illumination.Intensity values are shown as detected photons per pixel.

Fig. 10 .
Fig. 10.Active imaging color reconstructions of each target for multispectral laser illumination.The target field was 3 cm × 3 cm.

Fig. 11 .
Fig. 11.Color image reconstruction of the toy figure when the exposure duration is progressively reduced using multispectral laser illumination.

Table 1 .
Filter Design Specifications