Adaptive hyperspectral imager: design, modeling, and control

The proposed system takes after the coded aperture snapshot spectral imager originally introduced in [9]: it is comprised of two gratings (or prisms) separated by a spatial light modulator (SLM). The first half of the setup is a 4f system [10] centred around a grating which forms a spectrally dispersed image on the SLM. Figure 1 illustrates the case in which the SLM allows only a slice of the dispersed image to pass through into the second half of the setup which is another 4f system ending with the detector. The purpose of the second grating is to undo the spectral spreading of the first one, allowing for an undispersed image at the detector. With this setup, the spectral components selected by the SLM are placed in their original positions relative to each other, thus reconstructing a spectrally filtered image. If all of the light is transmitted by the SLM then an undispersed, unfiltered image is formed.

Zoom In Zoom Out Reset image size

In this section a basic analysis of the system is used to obtain an efficient reconstruction algorithm. It is shown that the system acts as a programmable, spatially dependent, spectral filter for which spatial resolution is independent of spectral resolution and for which there is no wavelength dependent shift of the image at the detector. The analysis of this system closely follows that of Gehm et al [9] which models the propagation of the spectral density through the system. The intensity, I_n, at the nth plane of propagation is given in terms of the spectral density, S_n, as

$$\begin{eqnarray}&&{I}_{n}({x}_{n},{y}_{n})=\int {\rm{d}}\lambda \;{S}_{n}({x}_{n},{y}_{n},\lambda ),\end{eqnarray} \tag{ 1 }$$

where x_n and y_n are perpendicular to the optical axis of propagation and λ is the wavelength. Using geometrical optics propagation, the spectral density of the dispersed image at the SLM, S₁, is given by

$$\begin{eqnarray}{S}_{1}\left({x}_{1},{y}_{1},\lambda \right) & = & \displaystyle \int \displaystyle \int {\rm{d}}{x}_{0}{\rm{d}}{y}_{0}\;\delta \left({x}_{0}-\left[{x}_{1}+\alpha \left(\lambda -{\lambda }_{C}\right)\right]\right)\\ & & \times \delta \left({y}_{0}-{y}_{1}\right){S}_{0}\left({x}_{0},{y}_{0},\lambda \right),\\ & = & {S}_{0}\left({x}_{1}+\alpha \left(\lambda -{\lambda }_{C}\right),{y}_{1},\lambda \right),\end{eqnarray} \tag{ 2 }$$

where S₀ is the spectral density at the source, and the central wavelength, ${\lambda }_{C}$, is the wavelength of light which follows the optical axis of the system. The dispersion of the system (along the x direction) is given by $\alpha =f/d\mathrm{cos}({\theta }_{{dC}})$ where f is the focal length of the lenses, d is the grating periodicity, and ${\theta }_{{dC}}$ is the angle at which the central wavelength is diffracted from the grating [10]. The spectral density of the diffracted image after a narrow slice has been selected, S₂, is given by

$$\begin{eqnarray}{S}_{2}\left({x}_{2},{y}_{2},\lambda \right) & = & {\rm{T}}\left({x}_{2},{y}_{2}\right){S}_{1}\left({x}_{2},{y}_{2},\lambda \right),\\ & = & {\rm{T}}\left({x}_{2},{y}_{2}\right){S}_{0}\left({x}_{2}+\alpha \left(\lambda -{\lambda }_{C}\right),{y}_{2},\lambda \right),\end{eqnarray} \tag{ 3 }$$

where ${\rm{T}}(x,y)$ is the transmission function corresponding to the SLM. The second grating reverses the spectral shift applied by the first thus giving S₃, the spectral density at the detector:

$$\begin{eqnarray}{S}_{3}\left({x}_{3},{y}_{3},\lambda \right) & = & \displaystyle \int \displaystyle \int {\rm{d}}{x}_{2}{\rm{d}}{y}_{2}\;\delta \left({x}_{2}-\left[{x}_{3}-\alpha \left(\lambda -{\lambda }_{C}\right)\right]\right)\\ & & \times \delta \left({y}_{2}-{y}_{3}\right){S}_{2}\left({x}_{2},{y}_{2},\lambda \right),\\ & = & {\rm{T}}\left({x}_{3}-\alpha \left(\lambda -{\lambda }_{C}\right),{y}_{3}\right){S}_{0}\left({x}_{3},{y}_{3},\lambda \right),\end{eqnarray} \tag{ 4 }$$

For a simple spectralcunction is given by a rectangular function of width $\Delta {x}_{m}$ centred at x_m.

$$\begin{eqnarray}&&{\rm{T}}(x,y)={\rm{R}}\left(\displaystyle \frac{x-{x}_{m}}{\Delta {x}_{m}}\right).\end{eqnarray} \tag{ 5 }$$

Applying this transmission function to the spectral density at the detector gives

$$\begin{eqnarray}{S}_{3}\left({x}_{3},{y}_{3},\lambda \right) & = & {\rm{R}}\left(\displaystyle \frac{\alpha }{\Delta {x}_{m}}\left(\lambda -{\lambda }_{C}-\displaystyle \frac{{x}_{3}-{x}_{m}}{\alpha }\right)\right)\\ & & \times {S}_{0}\left({x}_{3},{y}_{3},\lambda \right)\end{eqnarray} \tag{ 6 }$$

The intensity at the detector is determined by integrating the spectral density with respect to wavelength.

$$\begin{eqnarray}{I}_{3}\left({x}_{3},{y}_{3}\right) & = & \displaystyle \int {\rm{d}}\lambda \;{S}_{3}\left({x}_{3},{y}_{3},\lambda \right)g(\lambda ),\\ & = & \displaystyle \int {\rm{d}}\lambda \;{\rm{R}}\left(\displaystyle \frac{\alpha }{\Delta {x}_{m}}\left(\lambda -{\lambda }_{C}-\displaystyle \frac{{x}_{3}-{x}_{m}}{\alpha }\right)\right)\\ & & \times {S}_{0}\left({x}_{3},{y}_{3},\lambda \right)g(\lambda ),\\ & = & {\displaystyle \int }_{{\lambda }_{1}}^{{\lambda }_{2}}{\rm{d}}\lambda \;{S}_{0}\left({x}_{3},{y}_{3},\lambda \right)g(\lambda ),\end{eqnarray} \tag{ 7 }$$

where the bounds of integration are determined by the width and position of the rectangular function used as the transmission function and $g(\lambda )$ is the spectral response function of the detector.

$$\begin{eqnarray}{\lambda }_{1} & = & {\lambda }_{C}+\displaystyle \frac{{x}_{3}-{x}_{m}}{\alpha }-\displaystyle \frac{\Delta {x}_{m}}{2\alpha },\\ {\lambda }_{2} & = & {\lambda }_{C}+\displaystyle \frac{{x}_{3}-{x}_{m}}{\alpha }+\displaystyle \frac{\Delta {x}_{m}}{2\alpha }.\end{eqnarray} \tag{ 8 }$$

Applying ${\lambda }_{1}$ and ${\lambda }_{2}$ as the end points of the integral in equation (7) gives the final intensity ${I}_{3}\left({x}_{3},{y}_{3}\right)$ which, assuminsg linearity of the integrand within the bounds of integration, has a spectral resolution $\Delta \lambda$ and is given by

$$\begin{eqnarray}{I}_{3}\left({x}_{3},{y}_{3}\right) & = & {\displaystyle \int }_{{\lambda }_{1}}^{{\lambda }_{2}}{\rm{d}}\lambda \;{S}_{0}\left({x}_{3},{y}_{3},\lambda \right)g(\lambda ),\\ & \approx & {S}_{0}\left({x}_{3},{y}_{3},\bar{\lambda }\right)g\left(\bar{\lambda }\right)\Delta \lambda ,\\ \bar{\lambda } & = & \displaystyle \frac{{\lambda }_{1}+{\lambda }_{2}}{2}={\lambda }_{C}+\displaystyle \frac{{x}_{3}-{x}_{m}}{\alpha },\\ \Delta \lambda & = & {\lambda }_{2}-{\lambda }_{1}=\displaystyle \frac{\Delta {x}_{m}}{\alpha }.\end{eqnarray} \tag{ 9 }$$

This equation can be slightly rearranged to give

$$\begin{eqnarray}&&{I}_{3}\left({x}_{3},{y}_{3}\right)\approx \displaystyle \frac{\Delta {x}_{m}}{\alpha }g\left(\bar{\lambda }\right){S}_{0}\left({x}_{3},{y}_{3},\bar{\lambda }\right),\end{eqnarray} \tag{ 10 }$$

from which the key features of the dual disperser setup can be highlighted.

First, the recorded image ${I}_{3}\left({x}_{3},{y}_{3}\right)$ for a given slit position x_m on the SLM gives a direct access to a 2D subset of the hyperspectral cube that corresponds to a slanted cut inside the cube along the $\bar{\lambda }$ direction (figure 2). The dual disperser acts as a narrow band spectral filter with a linear gradient in wavelength along x₃ and the position of the slits on the SLM determines the offset of this gradient.

Zoom In Zoom Out Reset image size

Second, all the spectral components of the hyperspectral cube sharing the same x and y coordinates are colocated at the same position on the image: a point $({x}_{e},{y}_{e},{\lambda }_{e})$ of the hyperspectral cube can only affect the intensity at the image position $({x}_{3}={x}_{e},{y}_{3}={y}_{e})$. In other words, there is no wavelength dependent spatial shift on the image: the position at which we retrieve an element from the hyperspectral cube on the image does not depend on its wavelength coordinate in the cube.

This implies that there is no need for a complicated reconstruction algorithm to recover the spatial coordinates of points in the hyperspectral cube. Using a simple slit pattern, the whole hyperspectral cube can be readily retrieved by scanning the slit across the SLM and recording the images corresponding to each slit position.

It is also obvious from equation (10) that the spatial resolution of the system does not depend on the parameters which determine spectral resolution (i.e. slit width, grating periodicity, and focal length). The spectral resolution is given by the difference in wavelength corresponding to two neighbouring slit positions at any single point on the detector, $\delta \lambda =\Delta {x}_{m}/\alpha$. The dependence of the spectral resolution on slit width and dispersion is the same as that of a conventional spectrometer.

Although, thus far, a finite numerical aperture has not been accounted for in this system, that is the primary limiting factor in terms of spatial and spectral resolutions. It is also important to evaluate the effect of diffraction from the source and SLM. In order to illustrate the scanning method described in the previous section, a simulation was run to mimic the dual disperser system and test the reconstruction method. The model used for this simulation is chosen because it is well established in the field of pulse shaping to describe the effect of an SLM inside a 4-f line [10]. This model, relying on Gaussian optics propagation is fairly fast and naturally reproduces the effect of diffraction and limited numerical aperture. It also reproduces the geometrical distortion introduced by a grating. The propagation of the electric field through a lens of focal length f, as illustrated in figure 3, is given by [10, 11]:

$$\begin{eqnarray}{E}_{i}\left({x}_{i},{y}_{i},\lambda \right) & = & \displaystyle \int \displaystyle \int {\rm{d}}{x}_{o}{\rm{d}}{y}_{o}{E}_{o}\left({x}_{o},{y}_{o},\lambda \right){{\rm{e}}}^{{\rm{i}}2\pi \frac{{x}_{o}{x}_{i}+{y}_{o}{y}_{i}}{f{\lambda }_{C}}}\\ & & \mathrm{for}\;{x}_{i}^{2}+{y}_{i}^{2}\lt {D}_{l}^{2},\\ & = & 0\quad \mathrm{for}\;{x}_{i}^{2}+{y}_{i}^{2}\geqslant {D}_{l}^{2},\end{eqnarray} \tag{ 11 }$$

where ${E}_{i}({x}_{i},{y}_{i})$ and ${E}_{o}({x}_{o},{y}_{o})$ are respectively the electric fields at the image and object planes (both separated from the lens by a distance f) and ${\lambda }_{C}$ is the central wavelength of the field. The limited numerical aperture of the lens is taken into account by limiting the Fourier transform integral to components that fits within the lens diameter D_l[11]. Similarly, the field immediately after the diffraction by a grating, as illustrated in figure 4, is given by [10, 12, 13]

$$\begin{eqnarray}{E}_{d}\left({x}_{d},{y}_{d},\lambda \right) & = & {E}_{i}\left({x}_{d}/\beta ,{y}_{d},\lambda \right){{\rm{e}}}^{-{\rm{i}}{k}_{C}\alpha \Delta \lambda {x}_{d}/f},\\ \beta & = & \displaystyle \frac{\mathrm{cos}\left({\theta }_{{dC}}\right)}{\mathrm{cos}\left({\theta }_{{iC}}\right)},\\ \alpha & = & \displaystyle \frac{f}{d\mathrm{cos}({\theta }_{{dC}})},\\ \Delta \lambda & = & \lambda -{\lambda }_{C},\end{eqnarray} \tag{ 12 }$$

where d is the grating period and E_i and E_d are the incident and diffracted fields, respectively, and are located at the grating on a plane perpendicular to the direction of propagation of the central wavelength respectively before and after diffraction. ${\theta }_{{dC}}$ and ${\theta }_{{iC}}$ are the angles of diffraction and incidence corresponding to the central wavelength, respectively, and k_C is the wavenumber associated with the central wavelength ${\lambda }_{C}$.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Using equations (11) and (12) one can propagate the initial field to the SLM plane, and using the same equations propagate the filtered field from the SLM to the detector plane. It should be noted that if we leave aside the limited numerical aperture by the lenses, the main difference with the analysis done in the previous section is the presence of the scaling factor β that magnifies the image formed at the SLM without affecting the dispersion. The inverse magnification occurs on the way to the detector. Accounting for this in the analysis of the previous section only changes the position dependence of the measured wavelength. Thus, the measured intensity for a given slit position in the disperser in the limit of infinite numerical aperture for the whole setup is

$$\begin{eqnarray}{I}_{3}\left({x}_{3},{y}_{3}\right) & \approx & \displaystyle \frac{\Delta {x}_{m}}{\alpha }{S}_{0}\left({x}_{3},{y}_{3},{\lambda }_{C}+\displaystyle \frac{{x}_{3}/\beta -{x}_{m}}{\alpha }\right)\\ & & \times g\left({\lambda }_{C}+\displaystyle \frac{{x}_{3}/\beta -{x}_{m}}{\alpha }\right).\end{eqnarray} \tag{ 13 }$$

When accounting for the limited numerical aperture of the lenses, the solution cannot be written as simply as the above equation, as it contains many integral terms. However, the core linear dependence in the equation (13) remains valid as the main effect of the limited numerical aperture is to reduce the spatial and spectral resolution of the reconstructed image, exactly like in an imaging spectrometer.

By applying equations (11) and (12) in the right order, propagation through the dual disperser was simulated, using the object image shown in figure 5. An arbitrary central wavelength of $2\;\mu {\rm{m}}$ was used for the simulation. The initial object cube contained 31 spectral channels ranging from $1.8\;\mu {\rm{m}}$ to $2.2\;\mu {\rm{m}}$. The focal length of the lenses, periodicity of the gratings and width of the slit were 40 cm, $100\;\mu {\rm{m}}$ and $200\;\mu {\rm{m}}$ respectively. The lens diameter was 1 cm. Images were retrieved for 41 different slit positions which resulted in the retrieval of 41 spectral channels ranging from $1.293\;\mu {\rm{m}}$ to $2.707\mu {\rm{m}}$. Spectral images for channels centred at $1.853\;\mu {\rm{m}}$, $2\;\mu {\rm{m}}$, and $2.147\;\mu {\rm{m}}$ of the initial object cube are shown figures 6(a)–(c), and the corresponding retrieved object cube channels are shown figures 6(e)–(g). The initial object cube is defined by 31 spectral planes, while the scanning allows to retrieve 41 retrieved spectral planes: the retrieved cube is hence spectrally not sampled exactly as the initial cube, and the figure compares three close channels. The unfiltered initial and reconstructed images are also shown respectively on figures 6(d) and (h). The model takes into account the limited numerical aperture of the lenses: the imager is not collecting all the light out of the initial cube, which introduces, like in a real device, some vigneting and loss of spatial resolution. This simulation demonstrates the effectiveness of the model described in section 2 in terms of reconstructing a hyperspectral image, as well as getting an unfiltered image of the object.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

The scheme presented here relies on a scanning process: the slit position on the SLM needs to be scanned to retrieved the whole hyperspectral cube. The scanning can be extremely fast (well beyond video rate) by using modern SLM like digital micromirror devices (DMD) or liquid crystal on silicon found in most projectors. But contrary to traditional scanning hyperspectral imagers, we can easily access the unfiltered image in a single shot, and also set the trade-off between number of retrieved spectral channels and acquisition time while keeping the maximal spatial resolution available. Additionally, near snapshot acquisition of the hyperspectral cube can be achieved under certain conditions by using more subtle patterns than a single slit, as described in the next section.

To demonstrate the models and techniques described in sections 2 and 3 a prototype has been set up (figures 9 and 10). The observed object consists of a light source placed behind a lithographic mask. The mask is followed by a $10\;\mathrm{cm}$ focal-length lens, a $1.695\;\mu {\rm{m}}$ periodicity grating set for a deviation angle of around $50^\circ$, a $5\;\mathrm{cm}$-lens, and a DLP lightcrafter DMD. The DMD was set up to reflect the first diffraction order straight back along its incident path, thus reusing all of the optics of the first half of the dual disperser as the second half. A beam splitter is placed between the source and the first lens in order to direct the final image onto a camera. The camera is a standard CMOS camera (monochromatic IDS uEYE).

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

A lithography mask serves as a test pattern with sharp spatial features. To test our setup with different spectral contents, two collimated beams were alternatively used to illuminate the mask: a Superlum BroadLighter S840, providing an incoherent $50\;\mathrm{nm}$-wide spectrum centred at $840\;\mathrm{nm}$ or a Superlum BroadSweeper 840 providing coherent and narrow spectrum (down to $0.05\;\mathrm{nm}$), tunable between $820\;\mathrm{nm}$ and $870\;\mathrm{nm}$. This latter source can also quickly sweep to generate dual wavelength spectra or broad square spectra. The resulting test object resembles the one of figure 5, where an illuminating beam is spatially filtered by an aperture.

A simple linear calibration of the dispersion on the SLM was carried out by recording CMOS images as a function of the slit position on the SLM for several monochromatic spectra between $820\;\mathrm{nm}$ and $840\;\mathrm{nm}$. This calibration gives $\lambda ={\lambda }_{C}+{x}_{3}/(\alpha \beta )-{x}_{m}/\alpha$ with ${\lambda }_{C}=0.861\;\mu {\rm{m}}$, $\alpha =-31860$ and $\beta =2.99$, and where x₃ is the x-position on the CMOS camera and x_m the slit position on the DMD as in equation (13), all dimensions being in $\mu {\rm{m}}$. The spectra measured on each point of the CMOS image with our setup were compared with independent measurements made on the light sources using an optical spectrum analyser (OSA Ando AQ6315-A).

To assess the validity of the system model and calibration, first measurements were made with the scanning technique described in section 2. Two typical results are shown in figures 11 and 12, respectively for a dual wavelength spectral content and for a broad spectrum. In both figures, sub-figure (a) show the unfiltered image that integrates the whole spectral content image (all pixels of the DMD being opened), and the region of interest (ROI) in which the spectrum is recovered. Sub-figures (b) show the recovered $(\lambda ,x)$ slice of the cube corresponding to the ROI, and sub-figures (c) show the averaged spectrum retrieved over the ROI, compared to the OSA measurements.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

In both figures, one can see that within the ROI, the spectral and spatial variations of the spectral density are not coupled: the spectral density can be represented by a product of one spatial and one spectral functions. This is coherent with our test hyperspectral cube that can be separated in a spectral function (the spectral content of the tunable laser or superluminescent diode) and a spatial one (the Gaussian-like spatial profile of the illuminating beam, clipped by our lithography mask). One can also observe (in subfigure (b)) that the accessible spectral range shifts linearly with the x-position on the CMOS camera, in agreement with equation (13).

In figure 11(c), one can see a good agreement between the reported wavelength by our prototype and the OSA, but also notice a small difference in the relative intensities of the two spectral peaks.

Another noticeable difference appears in figure 12(c), where a significant intensity mismatch appears for wavelengths above $830\;$ nm. This is due to the lack of spectral efficiency calibration in our setup (contrary to the OSA, which is properly calibrated). This is compatible with the spectral efficiency of the CMOS camera used in the setup, that drops by a factor of 2 from $800\;\mathrm{nm}$ to $900\;\mathrm{nm}$.

To better illustrate the fidelity of the retrieved spectrum, we measured it at the centre of the ROI defined in figure 12 for different sources and compared it with independent OSA measurements of the source spectrum. Results are presented in figure 13, with several 5 nm wide square spectra, and with the super-luminescent source in high and low power mode. These measurements confirm that apart from a the lack of intensity calibration (visible for wavelengths above $830\;$ nm), the prototype reproduces the spectral content of the test object. This validates the basic scanning method proposed in section 2, and this especially shows that a straightforward coarse calibration of the system allows to recover precise spectral information.

Zoom In Zoom Out Reset image size

A second set of measurements illustrates the near snapshot technique described in section 3 to retrieve in just two acquisitions the full spectrum of a dual wavelength source (830 and $835\;$ nm) in a ROI detected in the unfiltered image. Figure 14 illustrates the step of the process: (a) shows the unfiltered image (first acquisition), which is coarsely segmented into regions of homogeneous intensities (b). The selected ROI is the narrow vertical portion of the lithographic mask. This ROI is not wide enough to allow the recovery of a large spectrum by opening only a single slit in the DMD: we are in a case similar to the one of figure 7 (bottom). Hence various slit positions are required to recover its spectrum: three areas are selected in the ROI (green, blue and red rectangles of (b)), and a slit position is defined in the DMD for each of these areas, so as to independently recover a part of their spectrum over three intervals $\left[{\lambda }_{A}^{i},{\lambda }_{B}^{i}\right]$, i denoting the area index (the width of the areas defines the width of the recoverable spectral interval, the position of the slit defines the central wavelength of this interval; see section 3). The slit arrangement is shown in (c), and the resulting filtered image captured by the CCD is shown in (d) (second acquisition). The spectrum recovered for each area is shown in (e), and the whole spectrum of the ROI is shown in (d): it is coherent with the illuminating source spectrum. An issue here is the precise intensity ratio between the $830\;\mathrm{nm}$ and $835\;\mathrm{nm}$ peaks, due to the slight intensity inhomogeneity within the ROI: intensity in the red area is slightly lower than in the two other areas, which leads to an underestimation of the spectrum part measured in the red area.

Zoom In Zoom Out Reset image size