Dynamic brain spectrum acquired by a real-time ultra-spectral imaging chip with reconfigurable metasurfaces

Spectral imaging paves way for various fields and particular in biomedical research. However, spectral imaging mainly depending on spatial or temporal scanning, cannot achieve high temporal, spatial and spectral resolution simultaneously. In this study, we demonstrated a silicon real-time ultra-spectral imaging chip based on reconfigurable metasurfaces, comprising of 155,216 (356$\times$436) image-adaptive micro-spectrometers with ultra-high center-wavelength accuracy of 0.04 nm and spectral resolution of 0.8 nm. It is employed for imaging brain hemodynamics, and the dynamic spectral absorption properties of deoxyhemoglobin and oxyhemoglobin in a rat barrel cortex were obtained, which enlighten the spectroscopy in vivo studies and other real-time applications.


Introduction
Spectral imaging technology captures both spatial and spectral information for all points in the field of view [1,2], and has been applied in various fields, such as health, remote sensing, military, environmental monitoring [3], mining and geology [4], agriculture [5], and astronomy [6]. Real-time spectral imaging (RTSI) has shown great potential in biomedical research. The cortical spectral imaging technique has been used to study brain behavior and cortical activity patterns by continuously recording the hemodynamic responses to reveal the underlying mechanism of complex brain functions [7][8][9][10][11][12][13].
However, because only a few wavelength band data can be acquired continuously for the indicators [14], it is necessary to develop a broad spectral imaging method with high temporal and spatial resolutions for dynamic brain spectra. Nonetheless, spectral imaging still mainly depends on spatial or temporal scanning [15][16][17][18][19][20], which cannot achieve high temporal, spatial, and spectral resolution simultaneously.
In recent years, spectral devices based on micro/nano filters have been used to improve the integration and miniaturization of on-chip spectral devices. There are two main types of micro-nano filters: resonant filters and broadband filters. Resonant filters, such as micro-ring resonators [21][22][23], optical microcavities [24,25], and resonant metasurface structures [26][27][28], perform spectral analysis by filtering light of different wavelengths separately, and offer a relatively high spectral resolution. However, it is difficult to simultaneously produce a broad spectrum and high resolution using resonant filters considering the number of spectral channels corresponds to the number of filters. In contrast, broadband filters, such as quantum dot arrays [29], photonic crystal plate arrays [30,31], disordered scattering structures [32], and nanowires with tunable band gaps [33] encode the spectral information of incident light into the response of a set of filters at different detector positions, and uses a computational spectral algorithm to reconstruct the incident spectrum [29][30][31][32][33][34][35]. Because spectral information can be reconstructed for multiple wavelength points using fewer filter structures, it makes developing a micro spectrometer possible. However, despite extensive research on the potential of integrated microspectrometers to replace current spectrometers with complex structures and large volumes [29,30,32,33,36], the RTSI requirements could not be met. Therefore, developing new mechanisms to implement RTSI to analyze highly complex brain activities and various practical applications is still a big challenge.
In this study, we fabricated an RTSI chip based on a reconfigurable metasurface supercell on a CMOS image sensor (CIS) for imaging brain hemodynamics. Several locally distributed metasurface units can be dynamically combined and multiplexed in the metasurface supercell to reconfigure image-adaptive micro-spectrometers. As a result, we realized 155,216 (356 436) micro-spectrometers on a CIS chip no larger than 0.5 cm 2 with an ultra-high center-wavelength accuracy of 0.04 nm and spectral resolution of 0.8 nm. The RTSI chip was used for imaging a rat's brain, which continuously monitored the dynamic absorption properties of deoxyhemoglobin and oxyhemoglobin over different wavelength bands, which are the indicators of neural activities. The result showed that the proposed chip is a promising method for studying brain functions pertaining to hemodynamics. Additionally, this scheme of reconfigurable metasurfaces allows the device to be directly extended to almost any commercial camera to seamlessly switch the information between the image and spectral image. Furthermore, spectral reconstruction can be easily combined with image recognition, which is significant in practical real-time applications.

Design of the proposed RTSI chip with reconfigurable metasurfaces
As shown in Figs. 1a and 1b, the proposed RTSI chip was designed by integrating a reconfigurable metasurface supercell on top of a CIS. All metasurface units (Fig. 1b) on the supercell can be dynamically combined and multiplexed to form thousands of image-adaptive micro-spectrometers (shown inset of  can be fabricated entirely using the CMOS-compatible processing technology, which reduces the production cost.
The intensities of modulated incident light were recorded by the underlying CIS chip in the reconfigurable metasurface supercell, as shown in Fig. 1c. Then, the recorded data was processed and reconstructed into the incident spectrum using an algorithm. Herein, we introduced the working principle of a single microspectrometer. The transmission response of the i-th metasurface unit is denoted as ℎ , where i = 1, 2, 3, ..., N, N is the number of different patterns in a base cell (Fig. 2a),  is the wavelength, and is the incident spectrum to be measured. The signal intensity received by the CIS below the i-th metasurface unit is expressed as: where is the average signal intensity received by 3× 3 CIS pixels below one metasurface unit to improve the signal-to-noise ratio (SNR) and the relaxed alignment requirements during fabrication, is the absorption quantum efficiency of the CIS for wavelength , is the dispersion curve of the lens imaging system, and and are the lower and upper limits of the incident spectral distribution, respectively.
We set ℎ , which is the modified transmission spectral curve that can be predetermined through measurement, as shown in Fig. 1d (see Supplementary S1 for details).
Furthermore, the integral equations were discretized to produce the matrix equation (Eq. 2), as shown in  sensing, H can be treated as a compressed sensing matrix [37] (i.e., CS matrix), used to sense the optical spectrum. In the proposed design, the transmission of every metasurface unit was simulated using a fullwave simulation software [38,39] to optimize the requirements of C 4 symmetry and compressed sensing [37] (see details in Supplementary S1). where abrupt changes occur in the spectrum (see Supplementary S5). In addition, the multiplexing of metasurface can also be applied to improve the spatial resolution, as shown in Fig. 2c.
Finally, the algorithm of compressed sensing for the optical spectrum is grafted after setting up the optimal micro-spectrometers in the supercell. Dictionary learning based on sparse coding [40][41][42] was used to recreate the original spectrum (see Supplementary S4). Considering the ultraspectral imaging chip can resolve two monochromatic lights with very similar wavelengths, we reduced the sampling interval to 0.2 nm and the spectral band to 2 nm (544.6-546.6 nm) to resolve ultra-fine spectral lines. A pair of peaks were formed using the 546 nm line of a mercury lamp and a tunable monochromatic light source. The micro-spectrometer effectively resolved the aforementioned double peaks with a wavelength interval of only 0.8 nm, as shown in Fig. 3c.

Experimental results
Furthermore, the linewidths of the reconstructed double peaks (blue curves) were broadened to 0.23 and 0.24 nm. To the best of our knowledge, this double-peak resolution is the best result obtained for on-chip spectral imaging devices, and is approximately an order of magnitude higher than that obtained using a nanowire spectrometer (15 nm) [33]. Additionally, the acquired results of double-peak resolution surpass the results obtained using commercial portable spectrometers (OceanView QE Pro) by approximately 1.2 nm. Additionally, the proposed ultraspectral imaging device was able to accurately reconstruct complex broad-spectrum signals in the visible light region. Fig. 4 shows the reconstruction results for several types of broad spectra. The spectral range was maintained to 450-750 nm (0.5 nm interval). The blue and red curves were obtained using the micro-spectrometer of the proposed device and a commercial spectrometer, respectively, and served as a reference spectrum. The concept of fidelity was introduced to quantitatively compare the original and reconstructed spectra as: Furthermore, we investigated the effect of the number of metasurface units k in a single microspectrometer on the fidelity of the reconstructed spectra to evaluate the spatial pixel reconstruction ability of the proposed device. k units were randomly selected from N= 400 different metasurface patterns in the region of interest. The error bars represented the variance of the reconstruction fidelity obtained for multiple possible random combinations for each k value, as shown in Fig. 4d. As k increased, the reconstruction error decreased and the fidelity value increased, which was consistent with the principle of compressed sensing [37]. Moreover, the k value of a single micro-spectrometer can be dynamically adjusted depending on the reconfigurable metasurface supercell by considering the noise level for achieving an optimal trade-off between the pixel density and reconstruction quality of the reconstructed spectra. A small k value (large micro-spectrometer density), and a large k value (small microspectrometer density), can be used at low and high noise levels, respectively. Moreover, a high fidelity can still be realized with a k value as low as 25 for simple spectra, as shown in Fig. 4b. Lastly, we demonstrated an RTSI video of an in vivo rat brain using the fabricated device. Fig. 5 shows the RTSI results for the barrel cortex of the rat brain. The RTSI video (see Supplementary Video) was recorded using a microscope with a tenfold objective amplification, as shown in Fig. 5a (see Supplementary S6 for details). Herein, the original images, modulated images and multi-frame spectral images are shown respectively, where the pattern of micro-spectrometers is loomed in the latter and the cerebrovascular regions are denoted by red dashed lines. We obtained the ultraspectral information over a spectral range of 450-750 nm at a sampling interval of 0.5 nm using a snapshot for all points in the field of view. The spatial resolution was 87.9m and we set k = 100 in the spectrum reconstruction to ensure spectral resolution. Moreover, we converted the spectrum data to a post-colored spectral image and a data cube, wherein we selected spectral images at five single wavelengths of 510, 540, 570, 600, and 630 nm. can be used to monitor the excited brain regions and study them based on a millisecond-level real-time response. Therefore, the proposed RTSI chip can non-invasively analyze brain activity and open a new avenue for neuroscience and brain science.

Conclusion
This study proposed and demonstrated an ultraspectral imaging chip based on a reconfigurable metasurface supercell. Further, an image-adaptive strategy was employed to provide the best tradeoff between spatial and spectral resolutions. The micro-spectrometers of the proposed device exhibited a high center-wavelength accuracy of 0.04 nm, a high spectral resolution of 0.8 nm, and a broad wavelength range of 300 nm. Compared to the latest report of on-chip spectrometers [26,29,31,33], this spectral resolution for a single microspectrometer is considered the best, and is approximately an order of magnitude higher than that obtained using a nanowire spectrometer (15 nm resolution [33]), while the spatial resolution (87.9m) is maintained at the same magnitude (100 m for nanowire spectrometer [33]). Meanwhile, the temporal resolution was improved from a scanning scenario (< 1 Hz [33]) to a real-time scenario (typical value 30 Hz for normal CIS, maximum value 1000 Hz for high-speed CIS).
Therefore, the proposed reconfigurable chip can be a potential new method for RTSI with both high spectral and spatial resolutions.
Furthermore, we performed an in vivo experiment on a rat brain (barrel cortex region) to obtain realtime ultraspectral video with Δλ /λ~0.001 (λ: 450-750 nm, Δλ: 0.5 nm interval). The results show a correlation between the optical spectrum and hemoglobin concentration. Affected by some defects of the spectral chip and optical aberration of microscopic imaging systems, the quality of the video is not very high, which would be improved by further iterated experiments in order to eventually achieve the dynamic detection of brain hemoglobin concentration. Finally, we seamlessly integrated the reconfigurable metasurface supercell with commercial cameras to avoid system incompatibility issues and enable real-time dynamic spectrum measurement in all optical imaging systems.

Data and materials availability:
The data supporting the findings of this study, including the data used for the plots and custom code, are available with the corresponding authors upon reasonable request.