Responsivity-based criterion for accurate calibration of FTIR emission spectra : theoretical development and bandwidth estimation

An analytical expression for the variance of the radiance measured by Fourier-transform infrared (FTIR) emission spectrometers exists only in the limit of low noise. Outside this limit, the variance needs to be calculated numerically. In addition, a criterion for low noise is needed to identify properly calibrated radiances and optimize the instrument bandwidth. In this work, the variance and the magnitude of a noisedependent spectral bias are calculated as a function of the system responsivity (r) and the noise level in its estimate (σr). The criterion σr /r <0.3, applied to downwelling and upwelling FTIR emission spectra, shows that the instrument bandwidth is specified properly for one instrument but needs to be restricted for another. ©2011 Optical Society of America OCIS codes: (120.0280) Remote sensing and sensors; (120.3180) Interferometry; (120.5630) Radiometry; (120.6200) Spectrometers and spectroscopic instrumentation; (280.4991) Passive remote sensing. References and links 1. H. E. Revercomb, H. Buijs, H. B. Howell, D. D. Laporte, W. L. Smith, and L. A. Sromovsky, “Radiometric calibration of IR Fourier transform spectrometers: solution to a problem with the High-Resolution Interferometer Sounder,” Appl. Opt. 27(15), 3210–3218 (1988). 2. L. A. Sromovsky, “Radiometric errors in complex Fourier transform spectrometry,” Appl. Opt. 42(10), 1779– 1787 (2003). 3. R. O. Knuteson, H. E. Revercomb, F. A. Best, N. C. Ciganovich, R. G. Dedecker, T. P. Dirkx, S. C. Ellington, W. F. Feltz, R. K. Garcia, H. B. Howell, W. L. Smith, J. F. Short, and D. C. Tobin, “Atmospheric Emitted Radiance Interferometer (AERI) Part I: instrument design,” J. Atmos. Ocean. Technol. 21(12), 1763–1776 (2004). 4. R. O. Knuteson, H. E. Revercomb, F. A. Best, N. C. Ciganovich, R. G. Dedecker, T. P. Dirkx, S. C. Ellington, W. F. Feltz, R. K. Garcia, H. B. Howell, W. L. Smith, J. F. Short, and D. C. Tobin, “Atmospheric Emitted Radiance Interferometer (AERI) Part II: instrument performance,” J. Atmos. Ocean. Technol. 21(12), 1777–1789 (2004). 5. D. L. Cohen, “Noise-equivalent change in radiance for sampling noise in a double-sided interferogram,” Appl. Opt. 42(13), 2289–2300 (2003). 6. P. M. Rowe, Department of Geography, University of Idaho, 875 Perimeter Drive, Moscow, Idaho 83844, S. P. Neshyba, C. J. Cox, and V. P. Walden are preparing a manuscript to be called, “A responsivity-based criterion for accurate calibration of FTIR emission spectra: identification of in-band low-responsivity wavenumbers.” 7. G. Lesins, L. Bourdages, T. Duck, J. Drummond, E. Eloranta, and V. Walden, “Large surface radiative forcing from topographic blowing snow residuals measured in the High Arctic at Eureka,” Atmos. Chem. Phys. 9(6), 1847–1862 (2009). 8. A. Shimota, H. Kobayashi, and S. Kadokura, “Radiometric calibration for the airborne interferometric monitor for greenhouse gases simulator,” Appl. Opt. 38(3), 571–576 (1999). 9. V. P. Walden, R. L. Tanamachi, P. M. Rowe, H. E. Revercomb, D. C. Tobin, and S. A. Ackerman, “Improvements in the data quality of the Interferometric Monitor for greenhouse Gases,” Appl. Opt. 49(3), 520– 528 (2010). 10. P. Rowe, L. Miloshevich, D. Turner, and V. Walden, “Dry bias in Vaisala RS90 radiosonde humidity profiles over Antarctica,” J. Atmos. Ocean. Technol. 25(9), 1529–1541 (2008). 11. P. Antonelli, H. E. Revercomb, L. A. Sromovsky, W. L. Smith, R. O. Knuteson, D. C. Tobin, R. K. Garcia, H. B. Howell, H. L. Huang, and F. A. Best, “A principal component noise filter for high spectral resolution infrared measurements,” J. Geophys. Res. 109(D23), D23102 (2004). #137173 $15.00 USD Received 17 Nov 2010; revised 15 Feb 2011; accepted 18 Feb 2011; published 8 Mar 2011 (C) 2011 OSA 14 March 2011 / Vol. 19, No. 6 / OPTICS EXPRESS 5451 12. D. D. Turner, R. O. Knuteson, H. E. Revercomb, C. Lo, and R. G. Dedecker, “Noise reduction of Atmospheric Emitted Radiance Interferometer (AERI) observations using principal component analysis,” J. Atmos. Ocean. Technol. 23(9), 1223–1238 (2006). 13. M. Jackson, Mathematics Department, University of Puget Sound, 1500 N. Warner, Tacoma, WA 98416 (personal communication, 2010).


Introduction
Measurements of atmospheric infrared emission made by Fourier transform infrared (FTIR) spectrometers are often calibrated using spectra of hot and cold calibration sources to produce spectral radiances (e.g.[1]).Error in the calibrated radiance is thus due to noise in uncalibrated, or raw, scene and calibration-source spectra, in addition to drift of the response of the instrument with time and other sources such as the temperatures and emissivities of calibration sources (e.g.[2][3][4]).In particular, error in calibrated radiances due to noise in raw spectra depends not only on detector noise, but also on other instrument characteristics and on experimental protocols.Instrument characteristics include the response of the detector, which tapers off at the limits of detector sensitivity, and the optical system, while experimental protocols include the number of coadditions (averaged raw spectra), the concentrations of trace gases in the instrument, and the temperatures of the hot and cold calibration sources.Together, these determine how the noise propagates into the calibrated spectra and, thus, the variance of the calibrated radiance due to noise.
An analytical expression has been derived for the variance of the calibrated radiance when random errors (e.g.due to detector noise) in the calibration-source spectra are negligible [5], or small [2].However, a quantitative threshold is needed for how small these errors need to be for the expression to be accurate, and the threshold must depend on the instrument characteristics and experimental protocols given above.Such a threshold is needed to optimize the spectral bandwidth, to make the spectral range as broad as possible while ensuring that radiances can still be calibrated properly at wavenumbers that are within the band, but near its limits.In addition, an estimate of the variance is needed for cases when errors in calibration-source spectra are not small enough to use the expression given in [2].
In this work, we propose a criterion for accurate calibration in terms of the system responsivity (r ) and the uncertainty (σ r ) in the measurement-based estimation of r (r m ).In addition, the variance and bias in calibrated radiances due to noise in raw spectra are determined numerically as a function of σ r /r.In section 2, a rationale for setting the criterion in terms of σ r /r is discussed, after introducing the notation that will be used here and expressing the variance in the limit of low noise in this notation.In section 3, the error in the calibrated radiance due to noise in the raw spectra is derived.It is shown that high noise causes biases in calibrated spectra; the magnitude of the bias is calculated numerically, as a function of σ r /r.The variance of the calibrated radiance is determined as a function of three error terms that are calculated numerically and depend, in turn, on σ r /r.The variance is used to set a quantitative criterion for low noise.In section 4 this criterion is applied to downwelling and upwelling FTIR emission spectra to choose an optimal bandwidth, which is then compared to that specified for each instrument.Finally, section 5 presents conclusions of this work.A companion paper ( [6]; hereafter referred to as RNCW) shows how the responsivity criterion can also be used for identifying spectral data points in-band that cannot be calibrated properly due to absorption by trace gases inside the instrument.

A criterion for accurate calibration in terms of responsivity
A rationale for setting a criterion in terms of the system responsivity is given in this section.First, the notation used here is developed and expressions for the error and variance in calibrated spectra, developed by Sromovsky [2], are expressed using our notation.Following this, we provide a rationale for using σ r /r as a figure of merit for the instrument.Finally, we examine measurements of the system responsivity.

Notation
The uncalibrated, or raw, spectrum measured by an FTIR spectrometer, in the absence of noise, is given by Sromovsky (Eq.( 18) of [2]) as 0.5 exp( ) , where the subscript x is replaced with h for the raw spectrum measured when viewing the hot calibration source, c for the cold source, or s for the scene.Furthermore, η is the overall system efficiency, R d is the detector response, L x is the radiance, ϕ is the instrument phase, and R d O is the background or instrument spectrum.The term t AI represents the transmittance of air inside the instrument, which is not included in [2] but is discussed in RNCW.The quantity 0.5ηR d t AI defines the system responsivity, r.The prime on V x indicates that errors are not taken into account.As pointed out by Sromovsky (first paragraph of section 6C of [2]), a term needs to be added to account for detector noise.Sromovsky shows how detector noise in a measured interferogram transforms into noise in a spectrum, and specifies a noise term, ε x , which has the same units as the calibrated radiance (e.g.L x ).To account for detector noise, the quantity rε x is added to the right hand side of Eq. ( 1), where r converts the noise term to the units of the raw spectrum (e.g.V' x ), and we drop the prime on V x to indicate that noise is now included.We define an equivalent noise term, n x ≡ rε x exp(-iφ), which is already in the units of the raw spectrum.The raw spectrum, including noise, can then be written exp( ) exp( ).
The advantage of the present formulation is that the detector noise in a raw spectrum, and thus the standard deviation of n x exp(iφ) is expected to be fairly constant with wavenumber.By contrast, the standard deviation of ε x varies with wavenumber (because the proportionality constant r varies with wavenumber).The noise in the raw spectrum due to detector noise, n x exp(iϕ), is assumed to be random with real and imaginary parts of equal variance that are uncorrelated, both of which have a normal distribution with a mean of zero.These statistical characteristics are a good approximation for some instruments, including the Atmospheric Emitted Radiance Interferometer (AERI [3,4]; ), but may not be true for all instruments (for example, if the error is correlated with moving-mirror position [2]).For convenience, the term "noise" hereafter refers to n x exp(iϕ), which is the detector noise for an average of M coadded spectra; that is, the detector noise for a single scan divided by the square root of M. Thus the standard deviations of n h , n c , and n s should be the same if the number of coadds are the same.For the purpose of this paper, we assume that the same number of coadditions will be taken for the hot and cold calibration sources (M h and M c are the same) but a different number may be used for the scene (M s may differ), thus the standard deviation of n s may be different.Although errors exist in uncalibrated spectra due to other sources, in this work we focus on detector noise.
We will use the term "error" to refer to the effects of noise on calibrated radiances.It will be shown that propagation of random noise in raw spectra does not generally lead to random error in calibrated spectra; in particular a bias is introduced at high noise levels.Importantly, averaging raw spectra (coadding) is not equivalent to averaging calibrated radiances.

Error in calibrated spectra in the low-noise limit
Calibration of the scene spectrum, V s , is accomplished using spectra of hot and cold blackbodies, V h and V c , of known radiances, L h and L c .The calibration equation is given by Revercomb et al. [1] as In the limit that (n h -n c )/r << L h -L c , which we will hereafter refer to as the "low-noise limit", Sromovsky [2] shows that -Re Re , for 1, ( -) where we have written the equation in our notation (compare to Eq. ( 53) of [2]).We can use Eq. ( 4) or (5) to calculate statistical properties of the error in the calibrated radiance.The variance of a coadded calibrated radiance is <ε , where <ε L > represents the bias (brackets represent the mean over a large number of samples).
From Eq. ( 5), we can see that in the low-noise limit, <ε L > = 0, since <Re[n]> = 0, indicating that no bias exists in calibrated spectra due to noise.Sromovsky develops an analytical expression for <ε L 2 >, (which is thus the variance in the low-noise limit), where the σ n terms represent the standard deviations of the real parts of n s , n h , and n c .(For the derivation, see [2]; we discuss Eq. ( 6) in section 3 in the context of this work).The σ n can be calculated from the standard deviations of V s , V h and V c over a short enough time period that L s , L h , and L c are expected to be fairly constant.The variance in the low-noise limit given by Eq. ( 6) is useful because it can be calculated for real measurements.It is therefore important to determine when the requirement for low noise is met.

Rationale for setting a criterion in terms of the system responsivity
The requirement for low noise is that The system responsivity cannot be measured directly but can be estimated from a measurement we term r m (in contrast to Revercomb et al. [1], who make no notational distinction between the true system responsivity and its estimation).The requirement (n h -n c )/[r(L h -L c )] << 1 can then be expressed succinctly in terms of r and the standard deviation of r m , as shown below.This makes the system responsivity a useful quantity for setting the criterion for low noise.
The system responsivity can be estimated by taking the difference of uncalibrated blackbody spectra [as in Eqs. ( 8) and ( 9) of [1], but including the noise terms explicitly], We define a measured responsivity, r m , where the (complex) error in r m is given by We can then express the requirement for low noise in the compact form e r /r << 1.
Since the individual errors n cannot be measured, we need to relate e r /r to quantities that are experimentally accessible.The variance of r m is given by where we have left out a subscript "m" from e r and σ r for notational convenience.The asterisk represents the complex conjugate.We can insert Eq. ( 8) for r m into Eq.( 10) to show that where the double vertical bars indicate magnitude, and we have assumed that the phase is constant over the average and noted that <e r > = 0. Thus we can write a criterion for low noise in terms of quantities that are experimentally accessible as Equation ( 12) is a statistical statement of the condition It is experimentally accessible, in that we can estimate r and σ r from measured values of r m (σ r can also be estimated from <||e r ||>, if known).Moreover, the value of σ r /r incorporates important instrument characteristics and experiment protocols, as described in the introduction.In particular, σ r depends on detector noise, number of coadditions M, and the temperaturedependent radiance difference between hot and cold calibration sources.The system responsivity, r, depends on detector sensitivity, the optical system, and the influence of trace gases in the instrument.Thus, σ r /r represents uncertainties embedded in each of these factors into a comprehensive, wavenumber-dependent figure of merit, which allows us to define a quantitative criterion for low noise.Next, we examine these quantities using experimental measurements to identify a practical threshold for the inequality appearing in Eq. ( 12).[8,9]).The solid line in Fig. 1(a) shows ||r m || for a summertime AERI measurement, while the dashed curve is an estimate of σ r based on the standard deviation of sequential measurements of r m .The bandwidths of instruments vary, depending on the characteristics of the optics and detector, the number of coadditions, and the error tolerance of the user.For this particular AERI, the lower bandwidth limit has been specified as 500 cm −1 [10], as shown by the vertical dash-dotted lines.Near the centers of strong absorption lines of water vapor (1300 to 2000 cm −1 ) and carbon dioxide (667 cm −1 ), the responsivity drops to very low values due to absorption by these gases in the instrument; this is discussed in RNCW.Away from strong line centers, r varies slowly with wavenumber.Outside the sensitive region of the detector, r is close to 0 and thus ||r m || = ||e r exp(iϕ)||.

The responsivity estimation
Figure 1(b) shows ||r m || for band 1 of the IMG instrument.The dashed curve is an estimate of σ r based on the standard deviation of ||r m || about neighboring spectral data points (in ~10 cm −1 intervals) from 2350 to 2500 cm −1 .Assuming the noise level <||n||> is fairly constant with wavenumber, this can be used to estimate σ r at all wavenumbers, since L h and L c are known (L c = 0, since the cold calibration source is space for the IMG).Vertical dash-dotted lines represent the bandwidth specified for the instrument [8].For this instrument, the lowwavenumber cut-off is set close to the maximum of the responsivity.At the high-wavenumber extreme (3030 cm −1 ), r is still large but σ r has also become large, as L h decreases with wavenumber (i.e.r m has become very noisy).Overall, σ r / ||r m || is considerably larger in-band than for the AERI.As shown in Fig. 1(b), for band 1 of the IMG, σ r /r ~0.5 at ~3000 cm −1 , thus calling into question whether this meets the low-noise criterion σ r /r <<1 [Eq.( 12)].Outside the low-noise limit, Eq. ( 6) does not apply and an alternate estimate of the variance is needed.However, a method for calculating <ε L 2 > and <ε L > outside the low-noise limit has not been previously published.An estimate of the variance that applies to all error levels can also be used to set the criterion for how much less than 1 the inequality must be for the low-noise limit to be valid.In addition, an estimate of <ε L > is needed to determine if a bias exists in calibrated spectra outside the low-noise limit.These issues are addressed in section 3.

Bias and variance for all noise levels
In this expressions for <ε L > and <ε L 2 > are derived that are valid for all noise levels.The expressions are written in terms of three dimensionless quantities that can be calculated independently of any experiment, as a function of σ r /r.In the low-noise limit when σ r /r <<1, we show that these results are equivalent to those of previous work [2].
Rearranging Eq. ( 4), the error in a calibrated spectrum due to noise is given as ( ) We define three dimensionless quantities where we have used Eq. ( 9) and defined analogous terms e c ≡ n c /(L h -L c ) and e s ≡ (n s /s) /(L h -L c ), and where s is defined so that n s /s has the same standard deviation as n h .We can then write ε L in a more compact form,

Bias error
The mean of ε L , < ε L >, represents the mean bias in spectra, that is, the bias if many spectra are averaged.If s, L h , L c , and L s are assumed to be constant with time, then In the low-noise limit, the approximate expressions for the <f s >, <f r >, and <f c > are zero, indicating that errors average to zero, and no bias exists.Outside of the low-noise limit, the terms can be calculated numerically.For example, the expectation value for f s is / Re , 1 / s r s s e s e r r e r f P de P de e r where each integral sign actually represents two integrations, over the real and imaginary parts, each from -∞ to ∞ and where the probability distributions, P, are complex Gaussians.
Taking the real part of the term in brackets and rearranging gives a sum of terms, one multiplied by an integral whose integrand is odd about Re(e s ) and the other an integrand that is odd about Im(e s ).The two terms, and therefore <f s >, are zero.Thus, Eq. ( 18) reduces to The value of <f c > can be shown to be proportional to <f r > as follows.We define a term <f h > by analogy to <f c > so that <f r > = <f h-c > = <f h >-<f c >.Because we are taking the mean, and n h and n c are statistically identical, we can interchange them in <f h > and replace each with its negative, since they are equally likely to be positive or negative.Thus <f h > = -<f c >. Then <f r > = -<f c >-<f c >, or <f c > = −0.5<fr >.Substituting −0.5<f r > for <f c > in Eq. ( 20) and rearranging gives ( ) The expectation value for f r can be calculated numerically and is found to be finite.(The singularity in the expression for <f r > at e r = −1 is integrable).Figure 2 shows <f r > as a function of σ r /r. Figure 2(a) shows the increase in <f r > from ~0 to ~1 as σ r /r increases from 0.1 to 30.In Fig. 2(b), the y-axis has been converted to a log scale to emphasize the rapid decrease in <f r > as σ r /r decreases, for small σ r /r.If many calibrated radiances are averaged over a time period for which L h and L c are approximately constant (i.e. the temperatures of the calibration sources are constant), and L s is constant (i.e. the scene radiance is constant), then the mean error is given by Eq. ( 21).Thus, this represents the mean bias in the calibrated radiance due to noise.
In the low-noise limit, f r ≈Re[e r /r], and <f r > ≈0, since the real part of the error is random.Therefore, in the low-noise limit, biases are zero, as mentioned in section 2. This is borne out in the numerical calculations shown in Fig. 2, where <f r > approaches zero as σ r /r decreases.To determine the mean bias in a given measurement, σ r /r must be calculated at each wavenumber and the corresponding values of <f r > determined from Fig. 2. Finally, these, together with an estimate of L s , can be inserted into Eq.( 21) to obtain the bias.
In this limit, the expected average of calibrated radiances, <L s + ε L >, has no dependence on the scene radiance, but approaches 0.5(L h + L c ). Figure 3(a) shows the average of 1816 calibrated AERI spectra taken at Eureka, Canada on 1 July 2008 (blue curve).For this case, 0.5(L h + L c ) (red dashed curve) is greater than the blue curve at most wavenumbers within the band (black vertical dashed lines).Outside the band, at wavenumbers smaller than 492 cm −1 and greater than 1820 cm −1 , r~0 because the instrument is unresponsive.Thus, in these regions σ r /r is quite large, and we expect <L s + ε L > to be close to 0.5(L h + L c ).In the figure, the blue curve and red dashed curve are approximately the same in these regions, showing that this is indeed the case and validating the result predicted by Eq. ( 22).There are also wavenumbers near strong absorption linecenters in-band where the mean is close to 0.5(L h + L c ); these are discussed in RNCW.
In Fig. 3(a), it is fortuitous that the red dashed curve represents an upper limit to the blue curve; this is because the Eureka AERI experimental protocols are such that L c ≥ ~Ls and L h is large.However, L h + L c is unrelated to L s , which means that in other experimental circumstances the red dashed curve may not be an upper limit to the blue curve.For example, Fig. 3(b), shows the average of 3 calibrated IMG spectra taken at −68.8° latitude and 93.4° longitude (blue curve; because the scene changes quickly for the IMG as the satellite moves, only 3 were averaged).In this case, the red dashed curve is sometimes larger than, equal to, or lower than the blue curve merely because 0.5(L h + L c ) happens to be of the same order as L s .The important point is that when σ r /r is extremely large, the average of measured radiances, <L s + ε L >, has no dependence on L s .Instead <L s + ε L >≈0.5(L h + L c ); this can be seen in Fig. 3(b) above 3000 cm −1 (it is not apparent outside the bandwidth at the low-wavenumber end due to other errors in IMG spectra).As long as σ r /r is small, the bias will be negligible.For example, at 1835 cm −1 , for the AERI measurement of 0357 UTC on 1 July 2008, σ r /r = 0.34 corresponds to an uncertainty σ L (in the low-noise limit) of 1.9 radiance unit [RU; 1 RU = mW m −1 sr −1 (cm −1 ) −1 ], but only corresponds to a bias error of ~0.0008 RU [noting from Fig. 2 that f r = 10 −4 , and further that L h + L c = 11.5 and L s ~Lc ~5.9 RU, and using Eq. ( 21)].However, at 1841 cm −1 (above the standard cut-off for AERI instruments of 1800 cm −1 ), the uncertainty is 6.0 RU (in the lownoise limit) and the bias is 2.1 RU.It is therefore important to restrict the spectral range so that it only includes wavenumbers where σ r /r is small enough that the bias error is negligible compared to the total error budget; such a decision will likely depend on how the data will be used.In particular, it is desirable that the bias be smaller than the random component of the noise after the averaging of calibrated spectra or spectral data points, or removal of noise through techniques such as principal component analysis [11,12].

Variance
The variance of a calibrated spectrum is σ L 2 = <ε L 2 >-<ε L > 2 , which is approximately equal to <ε L 2 >.Squaring Eq. ( 17) and taking the mean leads to ( . Like <f s >, all averages that include f s to the first power are zero: <f s f r > = <f s f c > = <f h f s > = 0. Reducing Eq. ( 23) then gives If we omit a small region about the singularity to allow numerical computation (i.e., calculate a truncated mean), the integrations can be performed for <f s 2 >, <f c 2 >, <f r 2 >, and <f c f r >.The truncated mean and low-noise approximations are shown in Fig. 4. In the limit of small σ r /r, all truncated-mean terms converge to the low-noise approximations for the <f 2 > terms and the cross-term in Eq. (24).For example, in the low-noise limit, <f s 2 > ~<Re[e s ] 2 >.To set a threshold for what is considered low noise, we can compare the uppermost curves in Fig. 4 (equivalent to ignoring errors that occur once in 100,000 measurements) to those for the low-noise approximation.At σ r /r = 0.3, the differences are less than about 20% for the four terms, and thus the variance of the calibrated spectrum (σ L 2 ) can reasonably be approximated by that for the low-noise limit [Eq.( 6)].Outside this limit, <ε L 2 > can be calculated by determining σ r /r, choosing appropriate expectation values from the four plots, and plugging the results into Eq.( 24).The variance will actually be slightly smaller, since <ε L > 2 , which is generally much smaller than <ε L 2 >, also needs to be estimated (as described above) and subtracted.Thus, the low-noise criterion is set to σ r /r < 0.3, corresponding to a variance that agrees with that for the low-noise approximation to within about 20% for 99.999% of all data.If the low-noise approximations for the <f 2 > terms are used, plugging these into Eq.( 24) and rearranging gives the expression for the variance in the low-noise limit given by Sromovsky [2] and, in our notation, as Eq. ( 6).
In contrast to the above, outside the low-noise limit the <f 2 > terms and the cross-term, <f c f r >, each have non-integrable singularities (at e r /r = −1), resulting in infinite values [13].For an actual measurement, ignoring the singularity in the low-noise limit (σ r /r <<1) amounts to assuming that |e r /r| will never be close to 1.As σ r /r gets larger, the probability increases of |e r /r| being close to 1, corresponding to the possibility of arbitrarily large errors in measurements (one extreme example is an error of 2800 RU observed in a calibrated AERI spectrum on 1 July 2008; more moderate errors are shown in Fig. 2(a) of RNCW).

Application to FTIR emission spectra
Figure 5 shows σ r /r for (a) the AERI and (b) the IMG.Vertical dashed lines indicate the limits specified for the bands, as discussed previously.The horizontal dashed line in each panel represents σ r /r = 0.3.Thus the values indicated by asterisks show the bandwidths that would result in σ r /r < 0.3 everywhere within the band, so that bias errors are negligible and the variance due to noise is within ~20% of that for the low-noise limit.For this particular AERI, the bandwidth specified for the instrument, 500 to 1800 cm −1 [10] is within the bandwidth meeting the criterion, 492.3 to 1829.8 cm −1 .For the IMG the lower bandwidth (2325 cm −1 [8]) is acceptable but the upper bandwidth needs to be restricted (from 3030 cm −1 [8]) since the range meeting the criterion is 2156 to 2853 cm −1 .To determine the variance for real measurements, calculation of the low-noise approximation to the variance given by Eq. ( 6) is preferable to the time-intensive numerical calculations needed for Eq. ( 24).We have used numerical calculations to show that Eq. ( 6) should be within ~20% of the numerically-determined variance [Eq.( 24)] when σ r /r < 0.3.To validate this result using AERI measurements, Fig. 6 shows an expanded view from 480 to 520 cm −1 of the standard deviation for an AERI measurement calculated using both Eq. ( 6) and Eq. ( 24), where we have omitted 0.001% of the total probability in calculating Eq. (24).Also shown is an estimate of the bias, calculated numerically according to Eq. (21).As expected, where σ r /r > 0.3 (between 480 and 492 cm −1 ) the two variance estimates disagree (by as much as 100%), while where σ r /r < 0.3 (above 492 cm −1 ) the two variance estimates agree to within better than 20%.Furthermore, the bias indicated by <ε L > is found to be less than 0.01 RU for wavenumbers greater than 492 cm −1 .
The accuracy of the estimate of σ r /r depends on the accuracy with which the noise in raw spectra can be estimated as well as the stability of the instrument temperature and phase.In particular, σ r /r becomes more difficult to estimate as it gets larger; that is, as r gets smaller.In addition, since r m is complex, it must be taken into account that ||r m || may overestimate r by as much as σ r [2].The challenge of accurately estimating σ r /r is especially important given that σ r /r can change quickly (for example, by an order of magnitude over 40 cm −1 , as shown in Fig. 6) and that small changes may correspond to shifts from a value that corresponds to a negligible bias and statistically predictable errors to one that results in a large bias and the potential of arbitrarily large errors, as shown in Figs. 2 and 4. Fig. 6.The standard deviation (√<εL 2 >) and mean bias (<εL>) of calibrated Atmospheric Emitted Radiance Interferometer (AERI) spectra, where εL is the error due to noise.The standard deviation is calculated numerically excluding 0.001% of probability about a singularity in <εL 2 >, and in the low-noise limit.The vertical dotted line indicates the wavenumber where σr /r = 0.3.
The criterion that σ r /r < 0.3 can be used as a check on the bandwidth, or to set a maximum range for the bandwidth.However, since σ r only includes random errors in raw spectra of hot and cold calibration sources, and therefore does not account for other sources of error, it will often be desirable to restrict the bandwidth further than indicated by σ r /r.In particular, sources of error other than noise may cause a bias that can be observed in averages of calibrated spectra.For example, for a measurement made using band 3 of the IMG instrument, a bias of ~0.25 L h is evident at 1900 cm −1 even though σ r /r ~0.1, which should indicate a negligible bias, and a bias of ~0.5L h is evident at 2100 cm −1 , where σ r /r ~0.2Such biases could be due to a background spectrum [R d O in Eq. ( 1)] that is different for calibration and scene views, or to other sources of error such as detector nonlinearity.For the IMG instrument, evidence for such errors can be seen in the raw spectra.

Conclusions
An equation for the variance of (the real part of) a calibrated radiance spectrum is given by [2].The equation for the variance is an approximation, valid only if the noise in uncalibrated spectra (n/r) is small compared to L h -L c .In this paper, we restate this requirement as σ r /r << 1 where r is the theoretical, real, noise-free system responsivity, estimated from a measurement-derived responsivity r m , and where σ r is the standard deviation of r m .We use numerical simulations to show that as σ r /r gets larger, the likelihood of biases and arbitrarily large noise spikes in calibrated radiances increases.Numerical simulations are used to calculate the magnitude of the mean bias as a function of σ r /r, showing that the bias is negligible (compared to typical error budgets) for σ r /r < 0.3.However, for large σ r /r, the mean calibrated radiance approaches 0.5(L h + L c ), regardless of the actual sky radiance.In confirmation, averages of many calibrated radiances measured with surface and satellite-based instruments are found to be ~0.5(Lh + L c ) outside the detector bandwidth.Numerical simulations are used to calculate the variance, excluding a small region of probability where the error due to noise is infinite.This variance is found to agree with the low-noise approximation to the variance to within 20% for 99.999% of measurements when σ r /r = 0.3.Thus a reasonable criterion for the low-noise approximation is given as σ r /r < 0.3.
The criterion σ r /r < 0.3 is applied to downwelling radiance spectra (measured by an AERI at Eureka) and upwelling spectra (measured by the IMG).The bandwidth specified for the AERI, 500 to 1800 cm −1 , is found to be within the bandwidth meeting the criterion, 492.3 to 1829.8 cm −1 .For the IMG the bandwidth specified is 2325 to 3030 cm −1 but the range meeting the criterion is 2156 to 2853 cm −1 , indicating that the upper bandwidth needs to be reduced.If other sources of error are small and well known and if σ r /r can be calculated with good accuracy, as for the Eureka-AERI instrument, this criterion can be used to push the limits of the bandwidth when it is desirable to use data near its limits.More generally, uncertainty in the estimate of σ r /r and the presence of other errors (as for the IMG) make the criterion most useful as a check on the bandwidth and to set its maximum range.

Figure 1
Figure1shows ||r m || and estimates of σ r for both a ground-based and a space-borne FTIR spectrometer.The ground-based spectrometer is an AERI[3,4,7] operated at Eureka, Canada from March 2006 through April 2009 as part of the Canadian Network for the Detection of Atmospheric Change (CANDAC) program.The space-borne spectrometer is the Interferometric Monitor for Greenhouse gases (IMG[8,9]).The solid line in Fig.1(a) shows ||r m || for a summertime AERI measurement, while the dashed curve is an estimate of σ r based on the standard deviation of sequential measurements of r m .The bandwidths of instruments vary, depending on the characteristics of the optics and detector, the number of coadditions, and the error tolerance of the user.For this particular AERI, the lower bandwidth limit has been specified as 500 cm −1[10], as shown by the vertical dash-dotted lines.Near the centers of strong absorption lines of water vapor (1300 to 2000 cm −1 ) and carbon dioxide (667 cm −1 ), the responsivity drops to very low values due to absorption by these gases in the instrument; this is discussed in RNCW.Away from strong line centers, r varies slowly with wavenumber.Outside the sensitive region of the detector, r is close to 0 and thus ||r m || = ||e r exp(iϕ)||.Figure1(b) shows ||r m || for band 1 of the IMG instrument.The dashed curve is an estimate of σ r based on the standard deviation of ||r m || about neighboring spectral data points (in ~10 cm −1 intervals) from 2350 to 2500 cm −1 .Assuming the noise level <||n||> is fairly constant with wavenumber, this can be used to estimate σ r at all wavenumbers, since L h and L c are known (L c = 0, since the cold calibration source is space for the IMG).Vertical dash-dotted lines represent the bandwidth specified for the instrument[8].For this instrument, the lowwavenumber cut-off is set close to the maximum of the responsivity.At the high-wavenumber extreme (3030 cm −1 ), r is still large but σ r has also become large, as L h decreases with wavenumber (i.e.r m has become very noisy).Overall, σ r / ||r m || is considerably larger in-band than for the AERI.

Fig. 1 .
Fig. 1.Measurements of the instrument response (rm) and its standard deviation (σr) for (a) the Atmospheric Emitted Radiance Interferometer (AERI) and (b) band 1 of the Interferometric Monitor for Greenhouse Gases (IMG).

Fig. 2 .
Fig. 2. The mean of fr (described in the text) as a function of the uncertainty in the measured instrument response relative to the true instrument response (σr /r) with a) the x-axis on a log scale and b) both axes on log scales.

Fig. 3 .
Fig. 3. Averaged calibrated radiance, <Ls + εL> and the limit approached at high noise, 0.5(Lh + Lc).Panel (a) shows the average of 1816 downwelling radiance spectra taken over a 24-hour period on 01 July 2008 with the Atmospheric Emitted Radiance Interferometer (AERI), while (b) shows the average of 3 upwelling radiance spectra measured by the Interferometric Monitor for Greenhouse Gases (IMG) at −68.8° latitude and 93.4° longitude.

Fig. 4 .
Fig. 4. (a)-(d).The means of four functions, described in the text, each excluding a small region of the probability space about a singularity [see legend in (a)].The position of the singularity depends on the values of two random, complex variables with combined standard deviation σr /r.For small σr /r, the mean is proportional to (σr /r) 2 , as shown (low-noise).

Fig. 5 .
Fig. 5.The relative uncertainty in the system responsivity (σr /r) for measurements using a) an Atmospheric Emitted Radiance Interferometer (AERI) and (b) band 1 of the Interferometric Monitor for Greenhouse Gases (IMG).