On Space-spectrum Uncertainty Analysis for Coded Aperture Systems

We introduce and analyze the concept of space-spectrum uncertainty for certain commonly-used designs for spectrally programmable cameras. Our key finding states that, it is impossible to simultaneously capture high-resolution spatial images while programming the spectrum at high resolution. This phenomenon arises due to a Fourier relationship between the aperture used for obtaining spectrum and its corresponding diffraction blur in the (spatial) image. We show that the product of spatial and spectral standard deviations is lower bounded by {\lambda}/4{\pi}{\nu_0} femto square-meters, where {\nu_0} is the density of groves in the diffraction grating and {\lambda} is the wavelength of light. Experiments with a lab prototype for simultaneously measuring spectrum and image validate our findings and its implication for spectral filtering.


Introduction
Spectrum is often a unique feature of materials and is used for identification and classification across a broad range of fields such as geology [1], biological imaging [2,3] and material identification [4,5]. Tools such as the hyperspectral camera capture the spectrum of a scene which is subsequently used for identification and classification purposes.
Capturing the full spectrum, while useful, is also quite wasteful especially if we are only interested in measuring similarity of the spectral profile at each pixel to a small collection of reference spectra. Such cameras, called spectrally-programmable cameras, have been demonstrated [6,7] with compelling applications in computer vision. This paper evaluates a spectrally programmable cameras involving a dispersive element, such as a diffraction grating or a prism, and a spatial light modulator (SLM).

Problem setting
The analysis in this paper is for the optical setup shown in Fig. 1, commonly used in prior art for spectral filtering [4,[7][8][9]]. The optical system consists of a series of lenses of focal length f , each subsequent pair separated by 2 f . The setup relays the image plane from plane P1 to P3 with a pupil code in P2. A diffraction grating placed on P3 provides a spectral dispersion of the light. The dispersed light is focused on plane P4 to form the so-called rainbow plane, where each point corresponds to the average intensity of light of the whole scene for a single wavelength. The image on P3 is simply relayed on to plane P5. Arbitrary spectral filtering can then be performed by placing an (SLM) on the rainbow plane (P4) and measuring image on plane P5. Intuitively, planes P1 to P3 is a simple camera with an aperture in its Fourier plane, and planes P2 to P4 is a spectrometer with the aperture replacing a slit. The setup provides an insight into the tradeoff between spatial and spectral resolutions. While a camera requires a large and open aperture for a compact spatial blur, this would lead to severe loss in spectral resolvability, as a spectrometer requires a narrow opening. Our goal is to formalize the role played by the shape of the pupil code in deciding spatial and spectral resolution. In a camera configuration, the pupil code consists of an open aperture and leads to sharp image but blurred spectrum. In spectrometer configuration, the pupil code is a slit and leads to sharp spectrum but blurred image. Our paper formalizes the role played by pupil code for spatial and spectral resolutions.

Main Result
We show that the pupil code a(x, y) introduces a spectral and spatial blur, h λ (λ) and h x (x) respectively with standard deviations σ λ and σ x (detail expressions in eq. (7)). Our main contribution is in the form of a space-spectrum bandwidth product that relates the spectral resolution at which light can be modulated and the spatial resolution of captured image. This is encapsulated in the following theorem, Theorem 1 For the spectrally-coded imaging architecture shown in Fig. 1, the product of spatial and spectral standard deviations σ x and σ λ respectively is related as where ν 0 is the density of slits in the diffraction grating.
This result was first explored in [7] and [9] where they stated that a slit trades off spatial and spectral resolution. Our paper builds on their results by providing a concise expression for the tradeoff. We prove that a Gaussian-shaped pupil code achieves the lower bound and leads to most compact spatial blur for a targeted spectral blur. We also show that for narrowband filtering, the spatial blur is affected by the pupil code as well as the shape of the narrowband filter, and that a slit, a commonly used narrowband filter shape leads to a spectrally-varying spatial blur. Instead, using a Gaussian-shaped narrowband filter achieves spectrally-independent spatial blur, thereby being the optimal candidate for spectral programming.

Implications
The space-spectrum bandwidth product introduces an uncertainty in spectrally-modulated cameras, stating that one cannot arbitrarily code spectrum at high resolution without loss in spatial resolution.
We demonstrate the impact of uncertainty by building a spectrally-programmable camera and showing that blocking one of two closely-spaced narrowband sources cannot be done without severe loss in spectral resolution.
Hyperspectral imagers. Apart from programmable spectral filtering, several hyperspectral imaging architectures [8][9][10] rely on obtaining spectrally-modulated images. Our findings have a direct implication on such setups, as a key requirement of such setups is to capture high resolution images without sacrificing spectral resolution. Hence, the space-spectrum bandwidth product can serve as a design guide to carefully choose the pupil code to obtain desired spatial and spectral resolutions.
Spatially-coded cameras. We note that the analysis in the paper does not apply to many spectral cameras where there is no pupil plane coding. Cameras such as the pushbroom camera and the coded aperture snapshot spectral imager (CASSI) [11,12] only perform spatial coding and, as such, are not affected by this result. Since such systems code space and then measure its spectrally-sheared image, the spatial code only affects the spectral resolution and not the spatial resolution.

Prior Work
We start our discussion by talking about capturing images with arbitrary spectral filters and then briefly state its applications. We then state the fundamental tradeoffs based on system parameters.

Measurement model. Consider a scene's hyperspectral image (HSI) represented by
where (x, y) represent spatial coordinates and λ represents wavelength. Our goal is to optically obtain a spectrally-filtered image of the scene with a spectral filter. Specifically, given a spectral filter f (λ), our aim is to implement a camera that captures the following grayscale image, Applications of spectral filtering. The ability of arbitrarily code spectrum enables a wide gamut of applications. This includes adaptive color displays [6], programmatically blocking illuminants [7], and detecting materials [4,5]. The key advantage in all these applications is to not measure the complete HSI, but only the desired spectral projections, which leads to fewer measurements and higher signal to noise ratio (SNR). Such a system can also be used for compressively sensing the complete HSI [8,9] which relies on capturing projection of a scene's HSI on random or designed spectral filters.
Spectrally-programmable camera architecture. Spectral programming is an ubiquitous technique that is often used in imaging applications, such as Bayer filters for RGB cameras or narrowband spectral filters for fluorescence microscopy [3]. Static filters offer arbitrarily high spectral resolution, but are not tailored for applications that require changing filters rapidly; while this can be achieved with filter wheels, the speed of such devices is constrained by the speed at which the filters can be changed. Electronically tuning filters, in part can be achieved by using a tunable filter [13] where liquid crystal (LC) cells are used to obtain a combination of narrowband spectral filters. LC filters however are typically slow as they require large settling times. The most practical way of implementing rapidly changeable programmable spectral filters is to rely on the setup shown in Fig 1. Here, a dispersion element such as a grating or prism is used to create the so-called rainbow plane [6] where each point corresponds to intensity of a single wavelength of the whole scene. By placing a spatial modulator (SLM) on this plane, one can achieve arbitrary spectral coding. This approach is similar to replacing a sensor in a spectrometer with an SLM, and has been the defacto way of spectral coding in some of the past works [7,8].
The SLM-based approach has certain advantages. Since SLMs are fast (often in excess of 60fps), one can achieve high frame rates, which is crucial for imaging dynamic scenes, or settings that require several rapidly switching spectral filters. Two, the system is potentially capable of achieving high spectral resolution without sacrificing capture time. However, as we will see next, a high spectral resolution has a debilitating effect in the form of a severe loss in spatial resolution. The focus of this paper will be on the fundamental trade-off of spectral and spatial resolutions when using an SLM-based programmable camera. For brevity, we will henceforth refer to SLM-based programmable camera as just spectrally-programmable camera.
Time-bandwidth product. Our main result is based on the time-bandwidth product [14], which we state here for completeness. Let x(t) be a time-domain signal, and let X(ω) be its continuous-time Fourier transform. We define the spread of time-domain and frequency-domain signals as, Then the time-bandwidth product states that, As a consequence, one cannot achieve simultaneous localization in time and frequency. Our result is a translation of the time-bandwidth product applied to spatial and spectral signals which arises Fourier transform property of a thin lens [15]. Therefore, the blurs in space and spectrum have a scaled Fourier-pair relationship.

Space-spectrum tradeoff.
A key component of a spectrometer is a slit that leads to high resolution spectral measurements. In similar spirit, a programmable camera would also necessitate a narrow slit to ensure that spectrum can be modulated at high resolution. However, such a narrow slit leads to a severe loss in spatial resolution, since imaging at high resolution requires a large and open aperture. The authors in [6] note that a large slit leads to loss of spectral resolution, but they do not mention what happens to the spatial resolution. The authors in [7] identified this tradeoff and stated an approximate relationship between spectral and spatial tradeoff for fully open aperture and demonstrated that high spectral resolution lead to blurry images. We formalize the result and show that such a tradeoff applies to any pupil code shape and can be concisely stated as a space-spectrum bandwidth product. In the upcoming sections, we will formalize the spatial and spectral resolutions that result from the choice of a pupil code.
Spectral and spatial blurs. We present the spatial and spectral blurs here and refer the interested readers to Appendix A for a detailed derivation. For brevity, we show the blur along x-axis alone, as there is no spectral dispersion along y-axis. Let a(x) be the shape of the aperture function and let A(·) be its Fourier transform. Then the spectral and spatial blur functions are, We observe that the blur in space and spectrum are not independent; in fact, they behave similar to a Fourier-transform pair, with appropriate scaling. Our goal is to show that this interdependence between spatial and spectral blur has a very specific structure and can be lower boundedimplying that we cannot arbitrarily resolve in both domains.

The space-spectrum uncertainty principle
Our main result, stated in Theorem 1, suggests that the spatial and standard deviations are related by the inequality, σ x σ λ ≥ λ 4πν 0 . We now outline the proof of our theorem.
Proof. We define the spectral and spatial standard deviations as, which are similar to time and frequency spreads defined in eq. (3) with appropriate scaling. Given that σ t is the spread of x(t), the spread of a scaled function x(t) = x(st) is σ t = sσ t . From eq. (4) and substituting t = λ f ν 0 and ω = x f λ , 1 Implication. We make some observations about the uncertainty principle here.
• Invariance to scaling. The bandwidth product does not change even if the aperture is stretched or squeezed. If the aperture a(x) is replaced by a(sx), then spectral blur changes to h λ (λ) = |a(−sλ f ν 0 )| 2 and the spatial blur changes to A − x sλ f 2 . This changes the spectral and spatial variances to s 2 σ 2 λ and σ 2 x /s 2 , thereby keeping the product a constant. • Invariance to power of lenses. The bandwidth product is independent of focal length of the system, implying that one cannot expect any increase in product of standard deviations by changing the lenses.
• Dependence on groove density. The bandwidth product is a function of groove density ν 0 , and decreases with increasing density. In theory, one can achieve arbitrarily low space-bandwidth product by having high groove density, but the limiting factor becomes the aperture size of lenses.
• Dependence on wavelength. The bandwidth product is directly proportional to wavelength. This is not a surprising result. The limiting case of our statement, where σ λ is several hundreds of nanometers is just a normal grayscale imager, and in that case, the expression looks very similar to Abbe's diffraction limit [16]. However, one may make the expression independent of wavelength by using lower bound of the spectral range, blur for various window sizes. The blur kernel was computed for λ = 500 nm, f = 100 mm and a groove density of 300 grooves/mm. There is a visible trade off between the two resolutions. The appropriate window size depends on the application; a camera with low spectral resolution requirement can use a σ = 500µm window, while one with stringent spatial resolution requirements may use a σ = 1000µm window. Achievability of lower bound. As in the case of time-frequency uncertainty, there exists a pupil code function that has its space-spectrum bandwidth product equal to λ 4πν 0 . This is achieved by the family of Gaussian windows: The spectral and spatial blur are then given by, (12) Figure 2 shows the simulated "uncertainty" box at 500 nm of Gaussian windows of various widths. We simulated a system comprising of 75 mm lenses and a diffraction grating with a groove density of 300 grooves/mm. Evidently, as we squeeze along one axis, the other axis stretches with the product of widths being a constant at 145.9 nm·µm. Next, we validate our findings with an optical setup that implements the schematic in Fig. 1 and capture scenes with various aperture shapes.

Experiments
Optical setup. We built an optical setup shown in Fig. 3 with relevant components marked. The setup is a minor modification of the schematic shown in Fig. 1. We placed a spatial light modulator (SLM) on plane P2 which enabled display of various coded apertures, and a diffraction grating in plane P3. The spectral measurement camera is on plane P4. Instead of placing spatial camera on P5, we place it on P3 (using beamsplitter BS1). Since we do not code the rainbow plane P4, image on P3 and P5 are equivalent. Focal length of all our lenses was 75 mm and the diffraction grating had 300 grooves/mm.

Visualization of spectral and spatial resolutions.
To illustrate our hypothesis, we placed a USAF resolution chart on the image plane P1. The scene was illuminated with a cool white compact fluorescent lamp (CFL) which is comprised of several narrow peaks. This setup enabled us to simultaneously visualize sharp spectrum as well as sharp spatial features. Figure 4 shows images and spectra for some representative cases. Each row shows results for a specific coded aperture, whereas each column shows results for a fixed spectral resolution. The trend of  Fig. 4: Visualization of spectral and spatial resolutions. We illuminated a USAF resolution target with a CFL lamp. We varied aperture types and widths to get spatial and spectral measurements. Each row shows image and spectrum for a specific aperture, and each column shows image and spectrum for a fixed spectral standard deviation. The results clearly illustrate the tradeoff between spatial and spectral resolution. decreasing spectral resolution with increasing spatial resolution is clearly visible. Further, a Gaussian aperture is superior to slit in terms of greater spatial resolution for the same spectral resolution, which agrees with our theoretical findings.

Quantitative verification.
We illuminated a pinhole with a spectrally-narrowband light source with a central wavelength of 670 nm and an FWHM of 3 nm. We then captured both spectrum of the light source and image of the pinhole, which we then used for computing the corresponding standard deviations. Figure 5 compares reciprocal of spatial resolution against spectral resolution. The two plots show a straight line, thereby verifying that the product of spatial and spectral resolutions is a constant. We also observe that the line for Gaussian aperture is very close to the theoretically optimal line, thereby confirming that the lower bound is tight, even in practice. We captured spatial and spectral measurements using the setup in Fig. 3. We illuminated a pinhole with a narrowband light source at 670 nm and swept across various aperture shapes and sizes. We then plotted reciprocal of spatial standard deviation against spectral standard deviation, clearly showing a straight line.  Fig. 6: Schematic and image of our prototype for spectral programming. We illuminated a USAF target with 520 nm and 532 nm lasers. We then blocked 532 nm with a spatial mask on the rainbow plane. Images were captured with various slit widths to show the effect of space-spectrum uncertainty.

Spectral Programming
Effect of edge-pass filter. The tradeoff between spectral and spatial resolution affects how well the spectrum can be coded. To test this, we illuminate two closely spaced spatial points in a scene with two narrowband light sources (520 nm and 532 nm). We then attempt to block the 520 nm laser with various coded apertures and observe the spatial image. Figure 6 shows the schematic and our lab prototype for spectral programming. The setup is built very similar to the schematic shown in Fig. 1 with a spatial mask placed in plane P4. The results are shown in Fig. 7. With a broad aperture, it is not possible to effectively block one of the two lasers, shown in fourth column. A narrow aperture can lead to effective blocking (compare first and last columns) but with loss in resolution.
Effect of the shape of a narrowband filter. We now show that a slit has unintended implication, when used as a narrow band filter. In order to perform narrowband spectral filtering, an intuitive choice is to place a narrow slit on the rainbow plane P4 and a camera on plane P5. This results in a spectra that looks similar to the example in Fig. 8(a). While such a mask works well for the Illum. target wavelength, the spatial images corresponding to adjacent wavelengths have severe loss in spatial resolution.
To understand the effect of a narrowband filter, consider a scene illuminated by a monochromatic light source of wavelength λ 1 . The resulting field on rainbow plane, Now let a spatial mask a(x) centered around λ 2 be placed on the rainbow plane. Then the output just after the mask, If both a(x) and a(x) are slits of width W, then the effective width is W − |λ 1 − λ 2 |, which decreases with increasing gap between the two wavelengths. This is illustrated by the plots of PSFs in Fig. 8(c). We utilized the setup in Fig. 6, where we illuminated a pinhole with a 520 nm laser. We placed a spatial mask on a horizontal translation stage to block various adjacent wavelengths. We then measured the image of a pinhole and fit an appropriate curve to the measured PSF. Evidently, the PSF has a larger spread as the gap between target wavelength and central wavelength of filter increases. This is true even if the pupil code were Gaussian mask and the filter were a slit, as shown in Fig. 8(d). Ideally we require the effective width to be independent of λ 1 and λ 2 . This is achieved if both pupil plane and the filter have a Gaussian  Fig. 8: Effect of narrow-band spectral filter on spatial resolution. Spatial resolution is affected by the pupil code as well as the spatial mask used for creating a narrow-band spectral filter. We illuminated a pinhole with a 520 nm laser and then swept various spatial masks to filter wavelengths around 520 nm. For configurations where either the pupil code or the spatial mask was a slit, the spatial resolution got worse with increasing gap between desired and laser wavelength. In contrast, a configuration with both masks being Gaussian resulted in a wavelength-independent spatial blur.
The output field has a spread that is independent of λ 1 , λ 2 which results in a wavelengthindependent PSF. This is illustrated by the plot of PSFs in 8 (e) with a gaussian aperture as well as filter shape. Figure 8 (f) compares PSF spread in terms of spatial standard deviation for various positions of filter, clearly illustrating the wavelength-indepedent blur arising due to Gaussian-shaped pupil-code and filter.

Conclusion
We formalized the tradeoff between spectral and spatial resolution associated with a spectrallyprogrammable camera of the type shown in Fig. 1 and stated the space-spectrum uncertainty principle. We showed through theory, simulations and real experiments that one can finely resolve space or spectrum, but not both. Our analysis then showed that a Gaussian-shaped aperture achieves the theoretical lower bound. We concluded our analysis by showing that a Gaussian-shaped narrowband filter introduces a wavelength-independent spatial blur.