Filters with random transmittance for improving resolution in filter-array-based spectrometers

In this paper, we introduce a method for improving the resolution of miniature spectrometers. Our method is based on using filters with random transmittance. Such filters sense fine details of an input signal spectrum, which, when combined with a signal processing algorithm, aid in improving resolution. We also propose an approach for designing filters with random transmittance using optical thin-film technology. We demonstrate that the improvement in resolution is 7fold when using the filters with random transmittance over what was achieved in our previous work. ©2013 Optical Society of America OCIS codes: (300.6320) Spectroscopy, high-resolution; (120.6200) Spectrometers and spectroscopic instrumentation; (100.6640) Super-resolution. References and links 1. D. J. Brady, Optical Imaging and Spectroscopy (John and Wiley Sons, 2009). 2. S. W. Wang, C. Xia, X. Chen, W. Lu, M. Li, H. Wang, W. Zheng, and T. Zhang, “Concept of a highresolution miniature spectrometer using an integrated filter array,” Opt. Lett. 32(6), 632–634 (2007). 3. W. L. Wolfe, Introduction to Imaging Spectrometers (SPIE, 1997). 4. H. N. Lee, Introduction to Compressed Sensing (Lecture notes; Spring Semester, GIST, Korea, 2011). http://infonet.gist.ac.kr/?page_id=843 5. J. Oliver, W. B. Lee, S. J. Park, and H. N. Lee, “Improving resolution of miniature spectrometers by exploiting sparse nature of signals,” Opt. Express 20(3), 2613–2625 (2012). 6. C. C. Chang, N. T. Lin, U. Kurokawa, and B. I. I. Choi, “Spectrum reconstruction for filter-array spectrum sensor from sparse template selection,” Opt. Eng. 50(11), 114402 (2011). 7. C. C. Chang and H. N. Lee, “On the estimation of target spectrum for filter-array based spectrometer,” Opt. Express 16(2), 1056–1061 (2008). 8. U. Kurokawa, B. I. Choi, and C.-C. Chang, “Filter-based miniature spectrometers: spectrum reconstruction using adaptive regularization,” IEEE Sens. J. 11(7), 1556–1563 (2011). 9. C. Bendjaballah, “Information rates in optical channels,” Opt. Commun. 17(1), 55–58 (1976). 10. Y. Aizu, K. Ogino, and T. Asakura, “A laser velocimeter using a random pattern,” Opt. Commun. 64(3), 205–210 (1987). 11. J. Ojeda-Castañeda and A. Sauceda, “Random gratings as correlator sensors,” Opt. Lett. 22(5), 257–258 (1997). 12. D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006). 13. R. Baraniuk, “Compressive sensing,” IEEE Sig. Proc. Mag. 24(4), 118–121 (2007). 14. E. Candes and J. Romberg, “11-magic: Recovery of sparse signals via convex programming,” Technical report (2005). http://users.ece.gatech.edu/~justin/l1magic/ 15. S. Park and H. N. Lee, “Designing an algorithm to solve basis pursuit denoising with a nonnegative constraint,” IEEE Sig. Proc. Letters. (submitted to). 16. R. Tibshirani, “Regression shrinkage and selection via the lasso,” J. R. Stat. Soc., B 58, 267–288 (1996). 17. A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Ima. Sciences 2(1), 183–202 (2009). 18. A. Juditsky and A. Nemirovski, “First Order Methods for Nonsmooth Convex Large-Scale Optimization, I: General Purpose Methods,” in Optimization for Machine Learning, S. Sra, S. Nowozin, and S.J. Write, eds. (MIT Press, 2011), pp. 1–28. 19. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Founda. and Tren. Mach. Learn. 3(1), 1–122 (2010). 20. M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, “Singlepixel imaging via compressive sampling,” IEEE Sig. Proc. Mag. 25(2), 83–91 (2008). 21. R. Dikpal, A. Veeraraghavan, and R. Chellappa, “P2C2: Programmable pixel compressive camera for high speed imaging,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 329–336. #177709 $15.00 USD Received 11 Oct 2012; revised 7 Dec 2012; accepted 13 Jan 2013; published 11 Feb 2013 (C) 2013 OSA 25 February 2013 / Vol. 21, No. 4 / OPTICS EXPRESS 3969 22. C. Li, T. Sun, K. Kelly, and Y. Zhang, “A compressive sensing and unmixing scheme for hyperspectral data processing,” Technical report. (http://www.caam.rice.edu/~zhang/reports/tr1101.pdf). 23. A. Rajwade, D. Kittle, T.-H. Tsai, D. Brady, and L. Carin, “Coded hyperspectral imaging and blind compressive sensing,” submitted (2012). 24. K. Madanipour and M. T. Tavassoly, “Determination of modulation transfer function of a printer by measuring the autocorrelation of the transmission function of a printed Ronchi grating,” Appl. Opt. 48(4), 725–729 (2009). 25. J. R. Barry and J. M. Kahn, “Link design for non-directed wireless infrared communications,” Appl. Opt. 34(19), 3764–3776 (1995). 26. H. A. Macleod, Thin-Film Optical Filters (Institute of Physics Publishing, 2002). 27. Z. B. Haim, Y. C. Eldar, and M. Elad, “Coherence-based performance guarantees for estimating a sparse vector under random noise,” IEEE Trans Sig. Proc. 58, 5030–5043 (2010). 28. C. Z. Microscopy, “Fundamentals of mercury arc lamps,” http://zeisscampus.magnet.fsu.edu/articles/lightsources/mercuryarc.html.


Introduction
Miniature spectrometers are key instruments that are required in various academic and industrial applications such as bio-medical, chemical, and environmental engineering [1].These spectrometers, built with integrated-filter arrays, provide superior portability, flexibility, and cost-effectiveness [2].The spectrometers also provide fine details about the various spectral components of the incident light.These spectral components reveal a wealth of information concerning the composition and the structure of the various objects being observed [3].The ability of a spectrometer to reveal fine information is usually attributed to its resolution.Thus, in recent years, researchers have focused on improving the resolution of spectrometers.
State-of-the-art filter-array-based spectrometers rely on digital signal processing (DSP) algorithms.These algorithms may run on a general-purpose computing platform of cellular phones and laptop computers, as shown in Fig. 1, before the spectrum estimate is displayed on the screen.They can also run on a specialized DSP chip integrated into the spectrometer unit in the case of stand-alone spectrometers.These algorithms process the raw spectrum acquired by the spectrometer to reduce noise and distortions and thereby aid in improving the resolution.
In the filter-array-based spectrometers, the factors that limit the resolution are the number of filters in the filter array and the shapes of the transmittance functions of these filters [4].In practical miniature spectrometers, the number of filters is fixed.Thus, in order to improve the resolution, it is often not a realistic option to increase the number of filters.The second factor that affects the resolution is the transmittance functions of the filters.While the transmittance function (TF) is usually designed to meet certain criteria, in practice, owing to low-cost integrated-array fabrication, the TF of each of the filters is non-ideal.These non-ideal filters highly distort the raw spectrum and thereby necessitate the digital processing of the raw spectrum to determine information concerning the spectral components of the input signal spectrum.DSP algorithms or estimators have been reported in the literature [5][6][7][8].The algorithms in [5,6] are centered on L 1 -norm minimization, which is a modern optimization tool in DSP for solving underdetermined system of linear equations, whereas the algorithms in [7,8] are based on conventional least squares and are designed with an aim to reconstruct the signal spectrum but not to improve resolution.
In this paper, we assume the use of the L 1 -norm-minimization-based DSP algorithm in [5] as the spectral estimator and aim to address the following relevant questions: Is it possible to enhance the resolution of the spectrometers by shaping the TFs in a particular way?Namely, is there a certain shape of the TF that is suitable for use with the L 1 -normminimization algorithm at the back end?If yes, what is that shape, and how can such a TF be designed?Is there a practical way to implement such a TF? Also of great importance is the discussion of suitable performance metrics to compare different TFs.This is necessary to reflect the use of the L 1 -norm-based spectral estimator we aim to use.With the new metric, one should be able to investigate the maximum possible resolution achieved by employing different TFs.
As per the discussion of the questions above, the role of TFs in improving the resolution of the spectrometers under the DSP-based spectral estimator needs to be carefully examined.We observe that the TFs whose auto-covariance functions resemble that of a Dirac delta function acquire independent information about the closely spaced spectral components of the input signal spectrum under the DSP-based spectral estimator.Filters with random-TFs possess such auto-covariance.Based on our study, we show that it is possible to design filters with random-TFs using thin-film technology and improve the resolution up to 7-fold compared to the filters with non-ideal transmittances in [5].
The concept of random-TFs goes back to the 1970s.Different aspects of the random-TFs have been studied in the past [9][10][11].In [9], the author has studied the maximum information transmission rate of an optical filter with random transmittance.A laserbased velocimeter is designed in [10] based on random transmittance patterns.In [11], the authors have designed a holographic-based sensor using random transmittances that are generated by using random gratings.As per our knowledge, however, no one has designed or reported filters with random transmittance for improving the resolution of a spectrometer.
This paper is organized as follows: Section 2 presents the system model, and Section 3 discusses in detail the need for designing filters with random transmittances for improving resolution.Section 4 discusses the design of random-transmittance filters using thin-film technology, and Section 5 presents the concept of the resolution of a spectrometer.Experimental results are discussed in Section 6, and Section 7 concludes the paper.

System description
We consider a spectrometer that consists of a planar filter array with M filters arranged in a 2D fashion as shown in Fig. 1.Each of these filters acts as a wavelength-selective device.Each filter is specified in terms of a transmittance function (TF), ( ), 1,2, , ,  Each sample of the raw spectrum denotes the projection (inner product) of the input signal spectrum onto a TF.That is, the ith sample of the raw spectrum is obtained as where ( ) x λ is the input signal spectrum and i w denotes a Gaussian noise sample with zero-mean and variance 2  σ .Thus, each raw spectrum sample contains information about the input signal spectrum acquired by a single filter in the filter array.We observe that the model in Eq. ( 1) for the raw spectrum can be used to describe the raw spectrum of a grating-based spectrometer as well.Grating-based spectrometers are a more conventional way of implementing spectrometers [1].The raw-spectrum model for the grating-based spectrometer is given [1, p. 339, Eq. (9.5)] by where i g is the detected spectral intensity at the ith CCD element, ( ) x λ is again the input signal spectrum, ( ) h λ is the system impulse response or the instrumental function, and Δ is the sampling interval.The system impulse response ( ) h λ denotes the overall impulse response of the complete system from the slit to the CCD sensor, including the transmittances of the slit and the grating.We note that the raw-spectrum models of the modern filter-based and conventional grating-based spectrometers are similar in their integral form input-output relationship.The role of ( ) h λ in the grating-based spectrometer is taken by the TF, ( ) i T λ , in the filter-array-based spectrometers.Modern filter-array-based spectrometers [4][5][6][7][8] process the raw spectrum using a DSP algorithm or an estimator to recover the input signal spectrum.The data model for the raw spectrum, y , which is an input to the DSP algorithm can be represented [4] as a system of linear equations: In Eq. ( 3), the 1 N × vector x contains the samples of the original signal spectrum having an operating bandwidth of W λ , and the matrix D is an M N × TF matrix.Each row of D is the TF of a filter.We observe that the matrix D contains only non-negative values as light transmittance is always non-negative.
In this paper, since we are dealing with improving resolution, the value of N is set to be greater than M, so N > M. That is, the number of samples of x is greater than the number of filters.The reason for setting N > M is that resolution in filter-array-based spectrometers is limited by the number of filters M (as will be discussed in detail in Section 3.1).Thus, in order to increase resolution beyond this limit set by the fixed number of filters M, the value of N should be greater than M. By setting N > M, we note that the noise-free system of equations in Eq. ( 3) is underdetermined.We also note then that in Eq. ( 3), the spacing between the samples of x is W N N λ λ Δ = , while the spacing between the samples of the raw spectrum y is Recently, sparse representation of the input signal spectrum, x , has drawn considerable attention [4][5][6].The aim of sparse representation is to decompose the signal spectrum into its constituent spectral (wavelength) components.For example, the signal spectrum of a low-power mercury lamp can be decomposed into its seven wavelength components, whereas the rest of the wavelength components are zero-valued.In general, any signal spectrum vector x in Eq. ( 3) can be equivalently represented as a sparse vector s using a linear transform matrix G, i.e., G = x s .The vector s is called a K- sparse vector.In this paper, we refer to s as a sparse spectrum.It contains K non-zero values and N K − zero-valued components, K N  .The K non-zero values of s indicate the intensities of the spectral (wavelength) components that constitute x.The set of K wavelengths corresponding to those intensities is called the support set.Thus, each element of the support set indicates the wavelengths (in nm) that are present in the signal spectrum.The intensities of the remaining N -K wavelength components of s are zero.
The matrix G is called the kernel matrix, and it is of size N N × .The columns of G are chosen as functions that preserve the natural shape of the signal spectrum.Gaussian kernels are widely used for signal spectrum modeling [8].The other possible non-Gaussian kernels are secant hyperbolic and Lorentzian [1].To construct G, we need to know only a single kernel function, the width of which is controlled by a parameter called full-width at half-maximum (FWHM).When the FWHM is small, the width of the kernel function is also small, and vice versa.Once the kernel function is chosen, it is sampled, and the sampled version of the kernel function is considered as the first column of G.The remaining 1 N − columns of G are just shifted versions of the first column [6].We observe that the Gaussian kernel has a uniform width that models narrowband light sources.For modeling broadband light sources, one may require kernels with varying widths, as discussed in [6].In this paper, we consider only narrowband sources.With the sparse modeling of the input signal spectrum, Eq. ( 3) can be written as where s 0.
The goal of the DSP algorithm or the spectral estimator is to obtain an estimate ŝ f s from the raw spectrum y, given the matrices D and G.

Introduction: Transmittance functions
The transmittance function (TF) of an optical filter is a waveform that has a shape that dictates the fraction of input light that the filter transmits at various wavelengths.The conventional approach is to attempt to design TFs with an ideal shape, i.e., a sharp brick wall as shown in Fig. 2(a).In Fig. 2(a), we show only three filters out of 40 in a filter array.Ideal filters allow light only for desired wavelengths, called the passband, and completely stop the remaining wavelengths, called the stopband.In addition, from Fig. 2(a), we note that the passbands of the ideal TFs do not overlap with each other.Ideal filters are desired for two reasons.First, the raw spectrum (Eq.( 3)) obtained by employing the ideal TFs is a direct estimate of the signal spectrum, and hence, there is no need for a DSP algorithm.This is because each sample of the raw spectrum is a projection (Sec.2) of the input signal spectrum with an ideal TF.Since ideal TFs are nonoverlapping by design, each sample of the raw spectrum only contains information about the spectral (wavelength) components that correspond to a single passband.Second, any two spectral components of the input signal spectrum that are one passband spacing apart are distinguishable (resolvable) in the raw spectrum.If there are any two spectral components separated by less than a passband width, they cannot be resolved.In order to resolve such spectral components, the passband width of each filter should be decreased.Consequently, this increases the number of TFs (number of filters) in order to cover the entire operating bandwidth.Thus, the resolution of the filter-array-based spectrometers with ideal TFs is limited by the number of filters, M. Since the spacing between the samples of the raw spectrum is resolution of filter-array-based spectrometers.This limit is the same for the more conventional grating-based spectrometer with a CCD array consisting of a total of M pixels.That is, by using the ideal system impulse response ( ( ) ( ) , the Dirac delta), the spacing between the samples of the raw spectrum of a grating-based spectrometer is also However, filters with ideal TFs are not easy to realize in practice [2].Therefore, nonideal TFs are inevitable in spectrometers.The shapes of the practical non-ideal TFs employed in a typical filter-array spectrometer [7] are shown in Fig. 2(b).These TFs are not purposely designed in such shapes but are instead accidentally obtained in the microlevel processing of implementing the ideal filters.Unlike the ideal TFs, the waveforms of non-ideal TFs are smooth functions of wavelength.In addition, as seen from Fig. 2(b), the passband of each filter's TF leaks into the passbands of neighboring filters.Hence, nonideal TFs have considerable overlap with each other.Non-ideal TFs are considered worse for the signal spectrum estimation process in conventional wisdom.The reason is that each sample of the raw spectrum now contains information about the spectral components not only from its own band but also from the neighboring filters' bands, viz.interferences.The interference from the use of non-ideal TFs causes severe distortion in the raw spectrum.Therefore, for the spectrometers with non-ideal TFs, post-processing of the raw spectrum using DSP algorithms is inevitable.These algorithms are designed with the aim of extracting the constituent spectral components (sparse spectrum) of the input signal spectrum from the raw spectrum y, via Eq.(4).
A recent recovery algorithm, reported in [5], is tailor-made for spectrometers.This algorithm is based on a DSP optimization tool called 1 L -norm minimization.1 L -norm minimization is a recent approach more suitable for solving a noise-corrupted underdetermined system of linear equations such as Eq. ( 4) than a classical least-squares approach [12][13][14].The minimization problem for the recovery of the sparse signal spectrum s in Eq. ( 4) can be expressed as: where ε is a small positive constant.
Note that the problem in Eq. ( 5) is a bit different from the standard L 1 -norm minimization approach with the inclusion of the additional non-negative constraint, 0 ≥ s .In order to find the non-negative spectral estimate ŝ , the authors in [5] have used a DSP algorithm that was reported in [15].In [15], the authors have derived an algorithm using the modern interior point method.They have shown that the performance of their algorithm is best when compared to the standard L 1 -norm minimization algorithms such as L 1 magic [14] and LASSO [16], which do not use the additional information.In addition, they have shown that their algorithm is robust against the observation noise, stable, and competitive with the existing algorithms.Using the algorithm in [15] for spectrometers, the authors in [5] have shown an improvement in resolution to about a factor of six below the resolution limit.In addition, the authors in [5] have provided a hint that the recovery performance (and hence the resolution) of the algorithm depends on the design of the TF matrix D.
Recently, fast algorithms for L 1 -norm minimization such as FISTA [17], FOM [18], and ADMM-based algorithms [19] have gained considerable interest among the imaging community.In imaging systems [20][21][22][23], the number of measurements (typically more than a few million) is much more than the number of measurements used in the miniaturized spectrometers.Consequently, these imaging systems should be capable of handling large amounts of data efficiently; for spectrometers, however, the accuracy is more important than the efficiency.It is well known that the interior-point-method-based algorithms provide good accuracy.Thus, in this paper, we use the same algorithm that was used in [5].Readers who may be interested in implementing fast algorithms for spectrometers can refer to the approaches in [17][18][19].
In this paper, our central goal is to design the TF matrix D in order to enhance the resolution.That is, we use the spectral estimation algorithm in [5] and show how to design the TF matrix D (which in turn determines the filter transmittances) that helps the algorithm to improve resolution.We show that by using our proposed TF matrix, it is possible to enhance resolution 7-fold compared to what was achieved in [5].

Motivations for new TF design
In this paper, we aim to present a new approach to further enhance (as compared to what was achieved in [5]) the resolution of the spectrometers by designing TFs.The motivations behind designing new TFs are based on the following observations: 1. We recall that each sample of the raw spectrum is modeled as a projection of the input signal spectrum onto a filter TF.In spectrometers with non-ideal TFs, the projection captures not only the in-band but also the out-of-band information about spectral components in each sample of the raw spectrum.Traditionally, when no DSP is used, unintentional capturing of the out-of-band information that is mixed with the information of the desired band is considered as pure distortion of the spectral components.In a completely different viewpoint, however, it can be considered as additional source from which useful information can be extracted.Namely, because of the shapes of the non-ideal filters, each sample of the raw spectrum contains extra information about the entire signal spectrum rather than the only about a particular band like the sample of an ideal TF does.That is, each non-ideal filter collects information from the entire signal spectrum and maps it into a single sample of the raw spectrum.Since the shape of each non-ideal filter is different from that of all other non-ideal filters, we can get many such independent "holistic" views of the entire signal spectrum from each sample of the raw spectrum.This led us to the very natural question: What kind of TFs should provide more holistic and independent information about the spectral components in each sample of the raw spectrum?
2. The modern, L 1 -norm-minimization-based DSP spectral estimator [5] recovers the spectral components very well from the distorted raw spectrum, prompting us to investigate the reason for this unexpected level of performance.We found that the problem of resolving (identifying) the spectral components essentially reduces to the unique identification of s from y.The success of this identification depends on the matrices D and G in Eq. ( 4).For a given class of signal spectrum, the matrix G is fixed.However, the matrix D can be determined from the design and fabrication of the filter TFs.Hence, we conclude that a good design for the TFs is important in order for the DSP algorithm to recover the spectral components from the raw spectrum.Thus, it is crucial to design TFs that 1) capture holistic and independent information about the constituent spectral components and 2) aid in recovering those spectral components.

Design of new TFs
We observe from the previous section that the amount of additional information acquired in each sample of the raw spectrum is solely proportional to the shape of the TFs.By shaping the TFs through a proper design procedure, it is possible to extract much information about the input signal spectrum.Therefore, the TF matrix D that senses the maximum amount of information about the signal spectrum appears to be the best choice for the spectrometer case, and hence, it is our design goal.We note here that a similar design notion arises in the emerging signal processing research area called compressive sensing (CS) where a sensing matrix [12,13] takes the role of the TF matrix in acquiring a signal.
In CS, the raw signal samples acquired using the matrix D is modeled as DG = + + A y s w= s w, which is the same as the raw spectrum model in Eq. ( 4).In CS, the matrix D is referred to as the sensing matrix; the matrix G, the sparsifying basis; the vector s, a K-sparse signal; and As is called the compressed signal.In CS, the sensing matrix D is chosen such that it captures enough information for the unique identification of the signal s with as few samples of As (the compressed signal) as possible.In order to quantify the ability of the sensing matrix to acquire enough information about the signal in a minimum number of raw samples, the mutual coherence metric, μ , is used [12,14].
The mutual coherence of the matrix A = DG is defined as follows: max , , 1, , where i a is the ith column of A. The μ is a measure of the maximum possible correlation among pairs of columns of A. The smaller μ is, the smaller the correlation among the columns of A is.The smaller the correlation is, the better the reconstruction accuracy of s from y is.Thus in CS, it is desirable to design the sensing matrix D, such that the matrix A = DG has low coherence.
It is well known in CS that sensing matrices D, the entries of which are drawn from i.i.d.samples of a random variable, exhibit low coherence.Such matrices are called random sensing matrices.These matrices are capable of capturing enough information about the signal s to perform reconstruction from a small number of samples of y.Therefore, random sensing matrices are widely employed in CS-based applications.
It appears that the goal of designing a sensing matrix in CS resembles that of TF matrix design.That is, both the TF and the sensing matrix should be designed to capture sufficient information to permit the faithful recovery of the signal.Since both of these goals are met by random matrices [12], in this paper, we consider a random matrix as the TF matrix rather than using the non-ideal TF matrix as in [5][6][7][8].Since each row of the TF matrix denotes a transmittance function of a filter, a row of the random TF matrix is termed a random transmittance function.That is, a filter with a transmittance that exhibits random fluctuations as opposed to possessing a pre-designed transmittance shape is termed a random transmittance function.Thus, our goal of improving resolution reduces to the design of a set of M filters with random TFs.A set of TFs is referred to as random when 1) The auto-covariance function (ACF) of each TF in the set is close to the Diracdelta function.That is, where ( ) .δ is the Dirac-delta function, i m is the mean of ( ) i T λ , and λ Δ is the wavelength difference, and, 2) when the cross-covariance function (CCF) between each pair of TFs in the set is very small.That is, ε is a small number close to zero.We discuss in detail the design of filters with such random transmittance in Section 4 using thin-film technology.

Significance of the proposed approach
The ideas in this paper mirror some of the prior work in CS with regard to the use of random matrices for signal acquisition and recovery.We briefly compare our approach to other well-known CS imaging approaches such as the single-pixel camera (SPC) [20], the P2C2 video system [21], the hyperspectral design (HSD) [22], and the CASSI system [23].First, we note that the SPC is a well-known CS-based imaging system [20].In SPC, in order to compressively sample a scene by optical random linear projection, a digital micromirror device (DMD) was used together with a biconvex lens.The DMD is programmed with random 0/1 Walsh patterns [20].With the availability of such optical devices, random projections of a scene were made almost effortlessly without going through a pixel-by-pixel scanning, and as a result, it was possible to realize the concept of SPC in practice.The P2C2 video system [21] extends the SPC architecture to video acquisition, and hence it also employs the random 0/1 Walsh patterns.We observe that in both the systems the measurement matrices are 0/1 Walsh matrices.
HSD [22] and CASSI [23] are spectral imaging systems.These systems collect a scene as a set of spatial images.Each image represents a range of an electromagnetic spectrum also known as a spectral band.These spatial images are arranged along various spectral bands to form a three-dimensional structure called a hyperspectral data cube.In HSD, the data cube is modeled as a linear combination of the spectral signatures of endmembers (that is, objects in a scene such as a road, grass, trees, or a roof), where each spectral signature can be obtained from a publicly accessible database such as HYDICE Urban hyperspectral data as used in [22].Then, the HSD exploits the spatial and spectral redundancies of the data cube by using CS.In particular, HSD adopts the concept of SPC for acquisition of the data cube, and hence they use the DMD with 0/1 Walsh patterns.In contrast to HSD, the CASSI system models the data cube as a linear combination of twodimensional wavelets and then acquires the data cube using the coded aperture masks.Thus, the measurement matrices in the CASSI system are also random binary 0/1 code matrices.We observe that both the HSD and CASSI systems compressively acquire the spectral data by means of spatial domain random measurement matrices, each element of which is either 0 or 1.
In contrast to the above imaging systems, we note that in our proposed spectrometer, we acquire the spectral data or signal spectrum using random spectral filters (not spatial).That is, as discussed in Section 2, the sensing of the signal spectrum is performed using random analog optical TFs, which are non-negative continuous functions of wavelength.We observe that the set of optical TFs play a sensing role as significant as that of the DMD in single-pixel cameras or the coded aperture masks.In addition, the TFs have values that can be any real number between 0 and 1, in contrast to the 0/1 patterns in the imaging applications.Table 1 summarizes the measurement matrices and their implementation in various imaging applications.
Second, in optical applications the sensing of signals and their subsequent recovery is accomplished with a sensing matrix stored beforehand.For example, in [20, p. 85], the DMD array stores Walsh, Hadamard, or noiselet patterns.The best matrices or the matrices of interest were designed in the digital domain.They were then used to dither the micromirrors back and forth in the DMD.We call this the digital design first approach here, as depicted in Fig. 3.In spectrometers, the actual mechanism of sensing is done using the analog TFs, as per Eq. ( 1), and then the signal spectrum recovery is done using its digital model.We note here that the digital TFs can be losslessly obtained from the TFs of analog thin-film filters (discussed in the next section).Thus, once the analog TFs are designed, they can be digitally represented in the TF matrix in Eq. ( 4) for spectrum recovery.We call this the analog design first approach in this paper (see Fig. 3), in contrast to the digital design first approach.The comparison of the analog design first approach with the digital design first approach may unearth a noteworthy characteristic of the spectrometer designs.Let us elaborate a little further.For spectrometers, it is desirable to design a set of random analog TFs with the ACF of each TF shaped as close as possible to the Dirac delta function.The narrower the width of the ACF is, the better the set is at resolving the fine details of the input spectrum.Thus, the resolution of a spectrometer is closely related to the width of the ACF of the TF.In the analog design first approach, due to the optical nature of the thin-film filters, there is an inevitable practical limit on how narrow the width of an ACF can be made.
We note that there is no such limit for the digital design.That is, in the digital case, the width of the ACF can be made as arbitrarily narrow as one wishes.For example, this can be achieved by dividing the operating wavelength range by the number of samples, say N, and hence obtaining any resolution we wish to have, and then by digitally generating a TF of length N by drawing independent samples from a probability distribution such as the 0/1 Walsh matrix with N = 65,536 as in [20] or as in P2C2 video systems [21].Since the samples of the TF are independently and identically generated from the given distribution, the ACF of the TF is always as close as possible to the Dirac delta function.This means that one can increase the resolution just by increasing N.This may work for image recovery applications, subject to the creation of a DMD array with so many more elements, but not for spectrometers.For spectrometers, the inherent width of the ACF of the analog TFs must be preserved in its digital TFs.If we first design a random (white) digital TF with an ACF very close to the Dirac delta, the digital TF may not be realizable as the TF of an analog filter.Thus, the resolution in the digital domain is a useless concept unless a way of constructing the practical analog counterpart of the digital TF is given.Our main contribution is twofold.The first part shows a specific way to design a set of optical analog filters with TFs that are white and uncorrelated with each other.The second part illustrates that significant improvement in resolution can be achieved with such a filter design.

Characterizing random TFs
Usually, the degree of correlation of a function over an interval is characterized in terms of the auto-covariance function (ACF).A function can be correlated with a time lagged (or advanced) version of itself.Thus, the ACF is a function of the time difference (time lag).The ACF reveals the correlation among the samples of the function.Similarly, a random TF can be characterized in terms of their ACF, which is now a function of wavelength difference.The shape of the ACF reveals the degree of similarity (correlation) between the light intensities at two different wavelengths of the random TF.If the shape of the ACF is wide with slowly decaying tails, then the TF is said to be highly correlated.On the other hand, if the TF has low correlation at different wavelengths, then the shape of the ACF is narrow with rapidly decaying tails, resembling a Dirac delta function.The shape of the Dirac delta function is ideal for an ACF.The width of the ACF measures the degree of correlation.That is, an ACF with narrow width exhibits low correlation, and vice versa.A Dirac-delta-like ACF is required in applications such as determining the modulation transfer function of an optical system [24] or detecting displacements or phase gradients in holography [11].This evidence shows that filters with a Dirac-delta-like ACF are crucial in optical systems and can be designed in practice.
Filters having ACFs that resemble that of a Dirac delta are preferred for L 1 -normminimization-based spectrometers.The reason is that a Dirac-delta-like ACF has narrow width, and hence two spectral components that are separated by more than this width are sensed differently by the TF.This independent information about the spectral components aids the L 1 -norm-based algorithms to resolve closely spaced spectral components.Thus, the resolution of a spectrometer has a close connection to the width of the ACF.By using random TFs, we can reduce the width of the ACF and hence the spacing between two spectral components that are independently sensed.On the other hand, non-ideal TFs exhibit high correlation, and hence the widths of the ACFs are wider than those of the random TFs.Hence, information gathered about the two nearby spectral components is not too independent.This reduces the ability of the spectrometer to distinguish the two spectral components.Thus, our goal in this paper is to design the transmittances of filters with Dirac-delta-like ACFs.We call them random TFs, while the TFs in [5], correlated TFs, in the subsequent discussions.

Design of the proposed random transmittance
In the previous section, we introduced random TFs with the goal of improving the resolution of a spectrometer system.Towards this end, we ask ourselves the following question: is it possible, in practice, to design and implement random TFs in the analog domain?If yes, how should one be designed?We have found that filters with random transmittance can be designed and implemented by using thin-film optical filters [25,26] in the way explained in this section.
A thin-film optical filter consists of multiple layers of high-and low-refractive index materials (dielectrics) deposited on a substrate [26].Each layer has a thickness usually of one quarter-wavelength.Thin-film filters work based on the interference of light transmitted or reflected at the boundaries between the layers.This interference is wavelength-dependent; that is, depending on the number of layers, the index of refraction, and the thicknesses of the layers, the transmission (or reflection) of light through the filter changes with wavelength, and hence the overall transmittance changes.By controlling the thickness of each layer and the number of layers, optical engineers design various filter transmittances such as band-pass filters.In this section, we show how varying the thickness of each layer generates random transmittances using the design methodology in [25].

Thin-film-based random transmittance filter design
In this section, we first briefly discuss the generic design of thin-film filters to generate band-pass-like transmittances.We then motivate and show how to design random transmittances using the same thin-film structure by varying only the thicknesses of the layers.Let us consider a thin-film filter with m dielectric layers between the input medium (air) and the output medium (substrate).Let a n and s n denote the refractive indices of the input and output medium, while H n and L n refer to the refractive indices of the high-refractive-index and low-refractive-index intermediate dielectric layers, respectively, and k d is the thickness of layer k.Assuming no dielectric losses, the transmittance of the thin-film filter as a function of wavelength ( )  λ and the angle of the incident light ( ) where TE ρ and TM ρ are power parameters that measure the powers that are contained in the TE and TM polarization modes of the input light.
The power parameters in Eq. ( 8) are calculated using a recursive set of equations shown in Table 2.We note from Table 2 that the power parameters (and hence the transmittance) depend on the thickness and the refractive index of the different media, as well as on the wavelength and angle of the incident light.Optical engineers use these recursive equations to design the transmittance shapes, such as the band-pass profile, that are required in practice.For example, the typical band-pass transmittances designed using Eq. ( 8) for various angles of incidence are provided in [25, p. 3765, Fig. (1)].Also, it is shown in [25, p. 3767, Fig. (3)] that the band-pass transmittances designed using the equations in Table 2 are accurate and are close to the actual transmittances obtained in practice.In Table 2, k θ denotes the angle that the incident light makes with the normal as it travels from layer k to k + 1, k η is the effective complex-valued refractive index seen by the light as it enters layer k.In order to calculate TE ρ and TM ρ , we first start with m m N η = , and we apply the equation for k η recursively until we arrive at 2 η , which when substituted into the equation for ρ in Table 2 yields either TE ρ or TM ρ .In this paper, we adopt the design equations in Table 2 to generate the random transmittances.We are able to generate the random transmittances based on the following observation: We first observed that thin-film filters can provide band-pass-like transmittance because of the principle of interference of light beams.That is, the incident light, when traveling through the thin-film filter, suffers different phase shifts as a result of propagating through different layers.The phase shifts depend on the refractive indices, the thicknesses of the layers, and the wavelength of the incident light.After transmission through the layers, the different beams that reappear at the filter output combine (add) either constructively or destructively depending on their path lengths.The constructive addition leads to increased transmission, and destructive addition leads to decreased transmission of the incident light.Since these constructive and destructive additions are wavelength-dependent, optical engineers usually design the thicknesses of the layers such that constructive interference occurs only for certain wavelengths and thereby achieve band-pass-like transmittances.
To avoid boundary reflections (and hence to obtain efficient transmission), it is preferred to have a 180° phase shift between the reflected beams at the boundary between the layers.This phase difference directly corresponds to a / 2 λ phase shift of a sinusoidal wave, which can best be accomplished by setting the optical thickness of the layers to refractive index of the layer.Thus, we note that the filter transmittances have a direct relationship with the thicknesses of the layers.Based on these observations, we inferred that it is possible to create variations in the filter transmittances by just changing the thicknesses of the layers.
In this paper, in order to generate random transmittances, we propose to vary the thicknesses of the filters, in contrast with the conventional fixed quarter-wavelength thickness.Specifically, we vary the thicknesses of the layers as normal / U λ , where U is a uniform random variable that has a mean and variance that are chosen such that the average thickness of a layer is on the order of m μ , similar to the thickness of the quarterwavelength layer.We note that once we generate the random thicknesses ( normal / U λ ), they are inserted into the design, and hence the thicknesses are no longer random after they are generated.We use the same design equations shown in Table 2.The only parameter that changes in the design is k β (in Table 2), which depends on the thickness.The other recursive equations stay the same.We note that we can produce M independent random filters by generating M different sets of random thicknesses.

Designed random transmittances and their ACF and CCF
Figure 4(a) shows the random TFs of three filters (5, 20, and 34) generated by the thinfilm method for the specifications given in Section 6.It is apparent from Fig. 4(a) that it is possible to generate a random transmittance just by varying the thickness of the layers in the thin-film filters.Figure 4(b) shows the ACF of the transmittance of Filter-5.It is evident from Fig. 4(b) that the shape of the ACF of thin-film-based transmittances is

Resolution of filter-array-based spectrometers
Traditionally, the resolution of a spectrometer characterizes its ability to distinguish the peaks of two closely spaced spectral components of the input signal spectrum.The spectrometers that identify the closely spaced spectral components reveal fine details about the input signal spectrum.Thus, the quality of the spectrometers is usually specified in terms of this resolution.This was perhaps good enough for the conventional spectrometers but not appropriate for those that use a DSP algorithm.In this section, therefore, we first introduce a metric useful for defining the resolution that can be achieved by using various TF matrices introduced in Section 3. We then provide a new definition of resolution suitable for DSP-based filter-array spectrometers.

Performance metric useful for defining resolution
As discussed in Section 3.2, the problem of resolving distinct spectral components is equivalent to the exact recovery of the sparse spectral components of Eq. ( 4) by DSP algorithms.There are infinitely many different sparse spectra s.Thus, to determine the resolution, it is required to test the recovery of as many sparse spectra s as possible for a given TF matrix.Usually, a Monte Carlo simulation is used to evaluate if, on average, a specified resolution can be achieved by a TF matrix.The drawback of Monte Carlo simulation is that it is time-consuming.Therefore, in this section, we introduce a metric called genie-aided mean square error (g.MSE).It is a closed-form mean square error expression obtained assuming that the estimator knows the support set, i.e., a set of constituent wavelengths of the input signal spectrum.Thus, g.MSE can be used to indicate the minimum MSE obtainable by the 1 L -norm-based DSP estimator.We will  4).Given y and A, the aim of the 1 L -norm-based DSP algorithms is to find an estimate  s of s.The DSP algorithm should find: 1) a set of constituent wavelengths (the support set) and 2) the non-zero intensity values corresponding to those constituent wavelengths.Let us suppose that the estimator exactly knows the support set.Then, by removing all the non-support set elements from the data model, the raw spectrum can be written as where k s is a 1 K × vector that contains the K non-zero intensity values of s, and k A is the M K × submatrix, the columns of which are the K corresponding columns of A. Note that this is an over-determined problem now, and finding  s essentially reduces to estimating the intensity vector k s from y.An estimator that has complete knowledge of the support set is called an oracle estimator [27].By making use of this prior information, the oracle estimator determines k s from y using the conventional least-squares approach.The minimum MSE of the oracle estimator is termed a genie-aided MSE (g.MSE).This g.MSE provides a tight lower bound on the accuracy of an estimator that finds k s from y.In this paper, we use the g.MSE to predict the maximum achievable resolution, because 1.It is often used as a golden standard against which the performance of practical algorithms can be compared [27].
2. It can be pre-computed, with just the knowledge of A.
We now define the g.MSE.Let k  s be the estimate of k s , the 1 K × intensity vector.Let ( ) s s be the error vector.The co-variance of the error vector s e is then given by taking only those K columns that correspond to K wavelengths in the support set.We note that the expectation operation is taken over the probability distribution of the noise vector.The matrix C s is of size K K × .Now, the average MSE, ( ) s s e e , where ( ) is the trace of the matrix s C .Since the input signal spectrum is s , the minimum MSE between x and  x with the known support set can be calculated.We define the error vector ( ) ( ) x x s s .The co-variance of the error vector x e is then given by ( ) . Now, the average genie-aided MSE is calculated as x x e e (9) For simplicity, we drop the arguments of g.MSE in the subsequent discussions.We note that the MSE that we would obtain by using any spectrum estimator is always bounded below by the g.MSE.
We note from Section 2 that a sparse spectrum s contains only K non-zero wavelength components, and the remaining N-K wavelength components are zero.In addition, these K non-zero wavelength components can be present anywhere among the N possible wavelength components.That is, for a given N and K, there are ( ) N K possible sets of locations (support sets) where each set contains K wavelengths.Therefore, we can obtain ( ) N K g.MSEs.Each g.MSE is for a signal spectrum with K wavelengths.We note that for a fixed K, increasing N increases the number of support sets exponentially.In the next section, we define the resolution of a DSP-based spectrometer based on the g.MSE.In this paper, we always express the g.MSE in decibels (dB) for convenience.

Resolution: A DSP-based definition
As discussed in Section 3.1, when no DSP is used, the resolution is determined by the number of filters, M, in the filter array.That is, is the minimum distance between two spectral components that can be distinguished by the spectrometer.The larger the number of filters is, the better the resolution is.However, in miniature spectrometers, the number of filters is fixed, and hence the resolution is limited.Thus, M λ Δ is called the resolution limit of the filter-array-based spectrometers.In order to further improve the resolution (beyond the resolution limit), DSP estimators are used [5].These estimators exploit the sparse nature of the signal spectrum (Section 2) and employ L 1 -norm-based estimators to improve the resolution.In this section, we introduce a new definition to measure the resolution that can be obtained by using the DSP estimators for the filter-array-based spectrometers.
We recall from Section 2 that the spacing between the samples of the input signal spectrum x (or equivalently s) is A way to define the resolution of a DSP-based spectrometer therefore is to find the minimal distance between the spectral components that can be accurately detected by the DSP estimator.Since the minimal distance is the distance between any two adjacent spectral components, viz.

W N N
λ λ Δ = , as N increases, the minimal distance decreases, and hence the resolution increases.We may come to the point of diminishing returns when going beyond a certain N. We call this N max .In addition, we define the minimum distance to be max min . Thus, to find this "digital resolution," it is sufficient to find N max .
We obtain N max using the g.MSE measure introduced in Section 4.1 as follows: We first fix the parameters N, M, K, G, 2  σ , and a TF matrix, which are required to calculate the g.MSE.We then check if the condition g.MSE ≤ δ is met for a given percentage of the total ( ) N K possible support sets, where δ is an MSE specified by the user.If the condition is met, it indicates that the DSP estimator is able to detect all the spectral components.We then increase the value of N further and check the condition again.We keep repeating this process (that is, increasing the value of N and checking the condition) until the condition is no longer met.We finally declare the largest value of N that met the condition as N max .The minimum spacing and hence the maximum achievable resolution of the DSP-based spectrometer is then max min In this paper, we increase the value of N starting from M that corresponds to the resolution limit of filter-array-based spectrometers.We note that it is not possible to increase N indefinitely with the hope of obtaining a minimum spacing.There is a limit for increasing N .This limit depends only on the TF matrix for the fixed parameters M, K, G, and 2 σ .Since the g.MSE captures all these parameters, it is enough to define the resolution based on the g.MSE measure.
For a signal spectrum with K spectral components, we define the maximum resolution ( min λ Δ ) of the spectrometer as follows: { } denotes the probability of the event g.MSE δ ≤ .The sample space for the event is the set of all possible support sets.Each support set is mapped into a g.MSE value.The variable ρ , 0 1 ρ ≤ ≤ , denotes the probability of the event g.MSE ≤ δ .That is, the variable ρ represents the fraction of the total support sets that satisfy the event.We express ρ in terms of percentage as 100 % ρ × .If 1 ρ = , then 100%; that is, all the ( ) support sets should satisfy the event.If 0.95 ρ = , then 95% of the total number of support sets should satisfy the event.We say that the K spectral components are resolvable if we can find an N such that the probability of the event g.MSE δ ≤ is greater than or equal to a specified ρ .We note that min λ Δ in Eq. ( 10) is the resolution of the DSP-based spectrometer whereas is the limit on the resolution of filter-array spectrometers.
The definition of the resolution given by Eq. ( 10) is general in nature in that it can include prior information.For example, if we know beforehand that the signal spectral components occupy only a certain range of the spectrum, then this definition automatically incorporates this prior information.Since prior information reduces the number of support sets and therefore the change is only in the event g.MSE δ ≤ , the definition of the resolution remains unaffected.Since we use g.MSE, the minimum possible MSE, the corresponding resolution obtained is the maximum possible by employing any kind of DSP-based estimator.In the next section, we evaluate the resolution that can be achieved by the correlated TF matrix in [5] and the proposed random TF matrix.

Results and discussion
In this section, we aim to investigate how much gain in terms of resolution improvement can be obtained by using random TFs in comparison with the correlated, non-ideal TFs in [5].We first investigate the maximum possible resolution by using the definition in Eq. (10).We then use the L 1 -based DSP algorithm reported in [5] and see how much of this resolution can be realized in practice.We set the number of filters in the array to M = 40.The non-ideal TFs, for the 40 filters, have been obtained from [7], which are also used in [5].The random TFs are generated using thin-film filters with m = 4 layers.We used the following refractive indices: The thicknesses of the four layers are chosen as explained in Section 4.1, with a different thickness for each filter.After we generate the random TFs, we sample them, and arrange them in rows to form a random TF matrix.
We choose a Gaussian kernel with an FWHM of 1 nm to generate the kernel matrix G.The operating wavelength range of the spectrometer is from 300 nm to 700 nm, corresponding to a W λ of 400 nm.We vary N in order to find the maximum possible resolution min λ Δ as per Eq. ( 10).The value of N begins with M (which corresponds to the resolution limit) and is set to increase in steps of 40, i.e., N = 40, 80, 120, etc.For 40 N = , the minimum spacing between two adjacent spectral components is 400 nm 40 N λ Δ = = 10 nm (from Section 3), which is the resolution limit (maximum possible resolution without using a DSP estimator).We define the signal-to-noise ratio (SNR) for the data model in Eq. (4) as We consider a sparse signal spectrum with K = 5 spectral (wavelength) components.The locations of these spectral components within the spectrum are random.Thus, the total number of possible locations (support sets) is ( ) 5 N for a given N. We choose 50,000 support sets at random from this total population.For each of these support sets, we calculate the g.MSE as per Eq. ( 9).We set 0.95 ρ = .As we vary N, we note the value of the g.MSE below which the g.MSEs of 95% of the support sets lie. Figure 5 shows the plot of the 95th-percentile MSE versus N obtained for the TF matrix in [5] and for the random TF matrix.
We now set 5 dB δ = − and find N max .That is, we aim to find the maximum N such that the 95th-percentile MSE is less than 5 dB − .We observe from Fig. 5 that for the TF matrix in [5], the 95-percentile MSE is less than 5 dB − when N max = 62.Thus, the maximum resolution of the DSP-based spectrometer obtained by using the TF matrix in [5] is max 400 nm min 62 6.5 nm . Therefore, the improvement in resolution over the resolution limit is 1.54 (10 nm /6.5 nm).In the case of random TFs, N max = 405 and the maximum resolution obtained is 400 nm min 405 0.99 nm λ Δ = = .The improvement in resolution in this case is 10.1 (10 nm/0.99 nm).We observe that the random TFs provide a resolution improvement of approximately a factor of 7 over that of the TFs in [5].Table 3 summarizes the resolution obtained by using various design approaches and algorithms for the same filter and signal settings as before.From Table 3, we note that the proposed random TFs achieve 7-fold resolution improvement compared to the TFs in [5] and 10-fold resolution improvement when compared to the ideal TFs.
We have also considered the non-Gaussian kernels such as Lorentzians and secant hyperbolic (mentioned in Section 2) for the same FWHM.Here, we report that we have obtained a similar resolution improvement to those obtained by using the Gaussian kernel.In addition, we note that comparing the resolution obtained by using ideal TFs with that of the random TFs is equivalent to comparing the resolution of ideal gratingbased spectrometers with the random filter-array-based spectrometer.This is because, as discussed in Section 3.1, the resolution limit of both of the spectrometers is the same.Thus, we are in one sense indirectly comparing our proposed approach with the best possible conventional grating-based spectrometers.We now investigate the practical spectral estimate obtained by using the DSP estimator reported in [5] when both the non-ideal and the random TFs are used in the filter-array spectrometer.For this purpose, we consider a signal spectrum generated by a mercury arc lamp, commonly used in fluorescence microscopy applications.The original signal spectrum of the mercury arc lamp has five (K = 5) prominent spectral components located at 365 nm, 405 nm, 436 nm, 546 nm, and 579 nm [22].We consider N = 360 and M = 40.Figure 6 shows the original signal spectrum of the mercury arc lamp and an estimate of the signal spectrum obtained by using the correlated TF matrix in [5]. Figure 7 shows the estimated signal spectrum obtained by using the proposed random TF matrix.
From Figs. 6 and 7, we observe the following: First, it is evident that the estimate obtained by using the random TFs is closer to the original signal spectrum than that of the estimate using the non-ideal, correlated TFs.Second, the five spectral components of the original signal spectrum are determined correctly by the random TFs.However, the nonideal TFs miss all five of the spectral components.Both the non-ideal and random TFs detect additional spectral components that are not present in the original signal spectrum.These additional spectral components appear as small bumps in Figs. 6 and 7.
In order to investigate how much resolution can be obtained using the DSP algorithm, we computed the MSE of the spectral estimates shown in Figs. 6 and 7 and compared them with the theoretical MSE shown in Fig. 5.The MSE of the spectral estimate by the random TFs (in Fig. 7) is about -5.5 dB which is just 1.3 dB away from the theoretical 95th-percentile MSE of -6.8 dB (in Fig. 5) at N = 360.For the non-ideal filters, the calculated MSE of the spectral estimate is 14 dB, which is 1.1 dB away from the MSE (12.9 dB) shown in Fig. 5. Thus, we observe that the resolution improvement deduced from Fig. 5 is actually achievable by using the practical DSP algorithms.The difference in MSE between the theoretical and the practical estimates is because of the detection of additional spectral components by the DSP algorithm.The difference in the practical MSE between the non-ideal TF in [5] and the random TFs using the DSP algorithm is about 18 dB.In order to show that a resolution of 0.99 nm (see Table 3) is achievable by using random TFs, we consider an original mercury lamp spectrum [28] with prominent and weak spectral lines as follows: 404.656 nm, 407.781 nm, 435.835 nm, 491.604 nm, 546.074 nm, and a pair at 576.959 nm and 579.065 nm.Thus, the sparse spectrum of the mercury lamp contains seven spectral lines or components as shown in Fig. 8(a).We note that the least separation among these spectral lines is 2.106 nm, which is between the pair 576.959 nm and 579.065 nm.We aim to resolve these components in the sparse domain.We consider the case of N = 400.Figures 8(b) and 8(c) show the estimate of the sparse spectrum using the TFs in [5] and using the proposed thin-film-based random TFs, respectively.It is apparent from Fig. 8(c) that random TFs correctly estimate the seven dominant wavelength components in the sparse spectrum.In addition, random TFs clearly resolve the pair of two closely spaced spectral components at 576.959 nm and 579.065 nm.This evidence supports the fact shown in Table 3 that random TFs are capable of resolving any two spectral components that are more than 0.99 nm apart.In contrast, the estimate obtained by using the TFs in [5] misses all seven of the dominant spectral components as shown in Fig. 8(b).In addition, the estimate contains spectral components that were not originally present in the signal spectrum.Based on these results and observations, we conclude that the filters with random TFs outperform the non-ideal TFs in terms of resolution and reconstruction performance.

Summary and conclusions
In this paper, we have proposed random-transmittance-based filters to improve the resolution of a spectrometer.We have shown that the optical filters with random TFs are designed to acquire holistic information about the signal spectrum, rather than the localized information that was the target of traditional designs.The holistic information captured by each filter when put together at the spectral estimator provides multiple independent views and helps the DSP algorithm to differentiate the details in the spectral components.We have shown via examples that spectrometers with random TFs provide 7-fold resolution improvement when compared to the use of filters that aim at implementing the ideal TFs.Rather than trying to design filters with the ideal brick wall TF, which has been the design paradigm in the past, our study shows that designing filters with random TFs should be the design paradigm for modern DSP-based spectrometers.This shift in the design paradigm not only brings forth an order-of-magnitude improvement in terms of resolution and MSE but also may relax the difficulty of the filter engineering process, resulting in less stringent design requirements.
fraction of input light that the filter allows at each unit of wavelength.We note that ( ) i T λ is a continuous (analog) function of wavelength.Each filter is attached to a CCD element that converts light into electrical energy.The output of each of the CCD elements is sampled to form an 1 M × vector y , which is called a raw spectrum.

Fig. 3 .
Fig. 3. Summary of analog and digital design first approaches.

/ 4 λ.
Hence, in practice, stacks of quarter-wavelength layers are used as a basic building block[26]  for many types of thin-film filters such as band-pass, long-wave-pass.For example, thin-film layers with thickness normal / 4n λ are very common in practice, where normal λ is the center wavelength of the band-pass filter with normal incidence and n is the #177709 -$15.00USD Received 11 Oct 2012; revised 7 Dec 2012; accepted 13 Jan 2013; published 11 Feb 2013 (C) 2013 OSA

#
177709 -$15.00USD Received 11 Oct 2012; revised 7 Dec 2012; accepted 13 Jan 2013; published 11 Feb 2013 (C) 2013 OSA 25 February 2013 / Vol.21, No. 4 / OPTICS EXPRESS 3983 , and therefore, the raw spectrum is in a low-resolution state ( M λ Δ ), while the signal spectrum x is in a high-resolution( N λ Δ ) state.The high-resolution state can be realized and determined when a quality estimate of x is obtained for all possible x.

(
Section 4.1).The angle of incidence of the light at the input medium is 45°.The center wavelength normal 850 nm λ = .
keep the SNR at 30 dB.

Table 2 . Recursive Equations for Calculating Power Parameters
Vol. 21, No. 4 / OPTICS EXPRESS 3982provide a comparison based on the more practical MSE as well and discuss in Section 6 how well the g.MSE predicts the MSE that is achieved in practice.