Ensemble uncertainty of inherent optical properties

: We present a method to evaluate the combined accuracy of ocean color models and the parameterizations of inherent optical proprieties (IOPs), or model-parametrization setup. The method estimates the ensemble (collective) uncertainty of derived IOPs relative to the radiometric error and is directly applicable to ocean color products without the need for inversion. Validation shows a very good ﬁt between derived and known values for synthetic data, with R 2 > 0 . 95 and mean absolute difference (MADi) < 0.25 m − 1 . Due to the inﬂuence of observation errors, these values deteriorate to 0 . 45 < R 2 < 0 . 5 and 0 . 65 < MADi < 0 . 9 for in-situ and ocean color matchup data. The method is also used to estimate the maximum accuracy that could be achieved by a speciﬁc model-parametrization setup, which represents the optimum accuracy that should be targeted when deriving IOPs. Application to time series of ocean color global products collected between 1997-2007 shows few areas with increasing annual trends of ensemble uncertainty, up to 8 sr m − 1 decade − 1 . This value is translated to an error of 0 . 04 m − 1 decade − 1 in the sum of derived absorption and backscattering coefﬁcients at the blue wavelength 440 nm. As such, the developed method can be used as a tool for assessing the reliability of model-parametrization setups for speciﬁc biophysical conditions and for identifying hot-spots for which the model-parametrization setup should be reconsidered.


Abstract:
We present a method to evaluate the combined accuracy of ocean color models and the parameterizations of inherent optical proprieties (IOPs), or model-parametrization setup. The method estimates the ensemble (collective) uncertainty of derived IOPs relative to the radiometric error and is directly applicable to ocean color products without the need for inversion. Validation shows a very good fit between derived and known values for synthetic data, with R 2 > 0.95 and mean absolute difference (MADi) <0.25 m − 1 . Due to the influence of observation errors, these values deteriorate to 0.45 < R 2 < 0.5 and 0.65 < MADi < 0.9 for in-situ and ocean color matchup data. The method is also used to estimate the maximum accuracy that could be achieved by a specific model-parametrization setup, which represents the optimum accuracy that should be targeted when deriving IOPs. Application to time series of ocean color global products collected between 1997-2007 shows few areas with increasing annual trends of ensemble uncertainty, up to 8 sr m −1 decade −1 . This value is translated to an error of 0.04 m −1 decade −1 in the sum of derived absorption and backscattering coefficients at the blue wavelength 440 nm. As such, the developed method can be used as a tool for assessing the reliability of model-parametrization setups for specific biophysical conditions and for identifying hot-spots for which the model-parametrization setup should be reconsidered.

Introduction
Deriving inherent optical properties (IOPs) from ocean color data requires accurate atmospheric correction, reliable retrieval algorithms and a consistent method for uncertainty estimation [1]. Commonly, a subset of the IOPs is retrieved from ocean color data, namely the absorption and backscattering coefficients of the water upper mixed layer. These bulk optical quantities are then related to biophysical characteristics of suspended and dissolved materials in the water. The choice of the retrieval algorithm and IOP parameterizations influences the accuracy of derived IOPs (backscattering and absorption coefficients). Consequently, quantifying the combined effect of both model and parametrization uncertainty on the accuracy of derived IOPs supports the assessment of ocean-color models and facilitates the design of future ocean color retrieval algorithms and IOP parameterizations. Various methods have been developed to evaluate the uncertainty of derived IOPs. Mélin [2] estimated the uncertainty embedded within chlorophyll-a (chla) concentrations derived from Sea viewing Wide Field-of-view Sensor (SeaWiFS) and Moderate Resolution Imaging Spectroradiometer (MODIS) measurements. Moore et al. [3] quantified the uncertainty of Chla concentrations derived from MODIS based on a fuzzy-logic approach to define memberships to specific optical water types. Other studies [4][5][6][7] have employed gradient-based methods to evaluate IOP uncertainties via application of a specific ocean color model, utilizing model derivatives and the difference between model best-fit and observed radiances. Alternatively, Salama and Stein [8] developed a non-gradient based method to quantify and separate uncertainty sources within derived IOPs based on a prior estimate of the radiometric uncertainties and their propagation through the retrievals. However, they also showed that the gradient based method has two inherent limitations: it produces uncertainty values that are proportional to the magnitude of derived IOPs, and it cannot handle radiometric residuals resulting from sensor noise and atmospheric correction.
The uncertainties in derived IOPs are determined by radiometric uncertainties, imperfection in the forward ocean color model and the adopted parametrization. The effect of the used ocean color model and set of parameterizations, or model-parametrization setup on the accuracy of retrieved IOPs has, so far, not been quantified in a single measure. In this paper, we present a method that quantifies the uncertainty of IOPs that follows from a model-parametrization setup. As such, the method provides a single measure of ensemble (collective) uncertainty of IOPs and avoids computing the radiometric uncertainty.
Our method is validated using three data sets. The first is a set of radiative transfer simulations of synthetic IOPs obtained from the International Ocean Color Coordination Group (IOCCG), report-5 [9, IOCCG data set]. The second is in-situ measured data of water-leaving radiance and IOPs obtained from the NASA bio-Optical Marine Algorithm Data set (NOMAD), Version 2.a [10, NOMAD data set]. The third consists of concurrent SeaWiFS observations and NO-MAD measured inherent and apparent optical properties, Version 1.3 [10, SeaWiFS matchup data set]. Finally, the operational application of the method is demonstrated using time series of IOPs derived from SeaWiFS monthly acquisitions from 1997 to 2007 [11]. From hereon, we assume the difference between known and derived IOPs per unit of radiometric error is equivalent to the uncertainty.

Ensemble Uncertainty of IOPs
Inversion of ocean color models is used to derive IOPs from radiometric observations of water surfaces. These models are based on: (i) approximations that link remote sensing reflectance just above the water surface, Rs w (λ ), to the IOPs (generally absorption and backscattering coefficients); (ii) parameterizations of the IOPs as functions of their values at a reference wavelength λ 0 . So we have: Rs w (λ ) = f (iop), with λ being the wavelength and iop being the set of derived IOPs at the reference wavelength λ 0 : iop = [iop i=1 , ..., iop i=n ]. As such, the radiometric uncertainty is propagated towards the derived IOPs as follows, where ΔRs w (λ ) and Δiop i (λ 0 ) represent the infinitesimal-change in Rs w (λ ) and ith IOP at the reference wavelength λ 0 , iop i (λ 0 ), respectively; w i is the partial derivative of remote sensing reflectance with respect to ith IOP; i.e. w i = ∂ Rs w (λ )/∂ iop i (λ 0 ). The term δ (λ ) is an error component that represents the accuracy of the used forward ocean color model in describing the relationship between apparent and inherent optical properties. To simplify the mathematical derivations, the term δ (λ ) will be imbedded in ΔRs w (λ ) and its spectral dependence will be dropped. Equation (1) facilitates estimating the uncertainties of all IOPs if ΔRs w (λ ) is provided for at least n wavelengths, with n being the number of derived IOPs which requires prior knowledge on the magnitude of ΔRs w (λ ). It is, therefore, more convenient to evaluate the uncertainty of IOPs with respect to ΔRs w (λ ). We divide both sides of Eq. (1) by ΔRs w (λ ), and denote the ratio Δiop i (λ 0 )/ΔRs w (λ ) as φ i (λ ), we have: ∑ w i (λ )φ i (λ ) = 1. Dividing both sides by ∑ w 2 i , yields the collective uncertainty of derived IOPs per unit of radiometric error: The term Φ(λ ) in Eq. (2) is referred to as the ensemble uncertainty of IOPs and can be expressed as, where Δiop(λ ) is the weighted sum of IOPs errors, with the ith weight being the ratio Assuming that the IOPs are mutually independent, we have from the Taylor series approximation of the second moment: where σ 2 r (λ ) is the radiometric variance and σ 2 i (λ 0 ) is the variance of the ith IOP at the reference wavelength λ 0 ; δ 2 is an error component analogous to δ in Eq. (1); represents the covariance terms in the Taylor series expansion. Assuming that the water observed-radiance is governed by independently varying IOPs, gives ≈ 0. For now, the term δ 2 is embedded in σ 2 r (λ ) for the following steps, but is further elaborated on in the analysis and discussion. Dividing both sides of Eq. (4) by the radiometric variance term yields, The ensemble uncertainty of IOPs per radiometric error, Ψ(λ ), is derived from Eq. (5) by normalizing both sides by the squared sum of partial derivatives and taking its square-root: Both, Φ(λ ) and Ψ(λ ) represent the ensemble uncertainty of IOPs per unit error of remote sensing reflectance and have the unit of sr m −1 . Since underestimation of the absorption coefficient is generally associated with overestimation of the backscattering and vice versa, Φ(λ ) is expected to be smaller than the individual uncertainties of absorption and backscattering coefficients. In other words, the under/overestimations cancel each other out. Conversely, Ψ(λ ) is additive; errors always add up. From hereon, Ψ(λ ) will be used as the measure of uncertainty instead of Φ(λ ).

Relative Measure of Uncertainty
From the definition of Ψ(λ ), Eq. (6) can be rewritten as, where σ iop (λ 0 ) is the sum of weighted IOPs uncertainties: Dividing both sides of Eq. (7) by the sum of derived IOPs, cb where the parameter CV(λ ) is the ratio, is a measure of the relative ensemble uncertainty per radiometric error and has units of sr. The reciprocal of Eq. (9) is a measure of the radiometric uncertainty with respect to CV(λ ), σ N r (λ ) in Eq. (10) is called the normalized radiometric uncertainty, and quantifies the radiometric uncertainty per unit of relative error of IOPs with the sr −1 unit.

Used Data Sets and Ocean Color Model
Four data sets are used in this study, three to validate the proposed method and one to demonstrate its potential for assessing global ocean color products. The validation data include simulated, in-situ measured and ocean color matchup data. Simulated data are radiative transfer simulations with the synthetic IOPs [9, IOCCG data set] as input, performed for a 30 • sun zenith over the 400 nm to 720 nm spectral range with 10 nm interval. Inelastic scattering, such as Raman scattering, chlorophyll fluorescence etc, were excluded from the simulations. In-situ measured data of water-leaving radiance and IOPs are taken from the NO-MAD data set, Version 2.a [10, NOMAD data set]. Ocean color matchup data are concurrent SeaWiFS observations and NOMAD measured inherent and apparent optical properties, Version 1.3 [10, SeaWiFS matchup data set]. Information on the different versions of NOMAD data sets can be found on SeaWiFS Bio-optical Archive and Storage System (SeaBASS): http://seabass.gsfc.nasa.gov/seabasscgi/nomad.cgi. Global ocean color products of monthly IOPs data are downloaded from the Goddard Earth Sciences Data and Information Services Center, Interactive Online Visualization and Analysis Infrastructure (Giovanni) [11] and covered the period between 1997 and 2007. This set includes SeaWiFS monthly products of IOPs derived from the Garver-Siegel-Maritorena (GSM) [12] model-parametrization setup.
The GSM model is used to relate the IOPs to the radiometric quantities of ocean color data as [12,13], where t is the transmission function from water to air and taken equal to t = 0.95 for the nadir viewing angle; n is the water index of refraction and is taken equal to 1.334; g i are model expansion parameters, for which g 1 = 0.0949 and g 2 = 0.0794 are adopted [12,13]; and u(λ ) is the ratio b b (λ )/(a(λ ) + b b (λ )), with a(λ ) and b b (λ ) as the bulk absorption and the backscattering coefficients of the water upper layer, respectively. Three IOPs will be considered at the reference wavelength λ 0 = 440 nm: the absorption of chlorophyll-a (Chla), a chla (440), the lumped absorption effect of detritus and gelbstoff, a dg (440) and the backscattering of suspended particulate matter (SPM), b bp (440). The parameterizations of Salama et al. [6,14] are used for simulated, in-situ measured and SeaWiFS matchup data. For global ocean color products of IOPs, however, the original parameterizations of the GSM model [12] are applied. The difference between these two parameterizations is only in the expression of a chla (λ ) at the reference wavelength λ 0 . We present an application of our method to the existing global products (SeaWiFS) to demonstrate its potential for evaluating the uncertainty of existing model-parametrization setups. In this context, slight differences in the model-parameterizations setup will not affect the conclusions of this application. Parameterizations of IOPs with respect to the reference wavelength λ 0 = 440 nm are as follows: the coefficient a chla (λ ) is obtained from [15]: a ph (λ ) = a ph (440)[a 0 (λ ) + a 1 (λ ) log a ph (440)] with a 0 and a 1 tabulated; the coefficient a dg (λ ) is defined as in [16]: a dg (λ ) = a dg (440)ζ dg , where ζ dg = exp [−s(λ − 440)] describes the spectral shape via s = 0.021 nm −1 ; the SPM backscattering coefficient is parameterized as in [17]: b bspm (λ ) = b bspm (440)ζ spm , in which ζ spm = (440 · λ −1 ) y describes the spectral dependency with y = 1.1.

Validation
The cross-entropy method of Salama [14] is used to derive the IOPs from radiometric measurements. The partial derivatives, w i , of Eq. (11) are computed with respect to the derived IOPs. The ensemble uncertainty is derived from Eq. (6) as derived error = Ψ(λ )σ r (λ ). The radiometric uncertainty, σ 2 r (λ ), is computed as (x d − x k ) 2 with x being the reflectance. The quantity, x d , is the best-fit spectrum derived from inverting Eq. (11), whereas x k refers to the known reflectance, in this case from the IOCCG-simulated or NOMAD-measured spectra. For the SeaWiFS-matchup set, these two quantities (x d and x k ) are replaced by in-situ measured and satellite observed spectra, respectively. The known values of ensemble-uncertainties are estimated as, where Δiop i is the difference between the derived and measured ith IOPs, and w * i is the ith partial derivatives computed using the measured values of IOPs. The error parameter, δ 2 , is estimated as δ 2 = (x m − x k ) 2 , in which x m is the output of Eq. (11) using the measured IOPs and x k is the observed (or known) spectra (e.g. IOCCG-simulated, NOMAD-measured or SeaWiFSobserved).
The NOMAD data subset is selected such that each site has radiometric observations and three IOPs measurements, a chla (440), a dg (440) and b bp (440). On the other hand, the SeaWiFSmatchup subset is composed such that each site has radiometric observations and at least two measured absorption coefficients, a chla (440) and a dg (440). Missing measurements of b bp (440) in the SeaWiFS-matchup subset are substituted by their estimates as derived from the measured spectra. This is justified by studies showing that the uncertainties associated with the derivation of the backscattering coefficient, b bp (440), are much lower than those found for absorption [9,18]. In total, there are 90 NOMAD and 123 SeaWiFS-matchup sites satisfying this criterion.
Derived and known values for the wavelength 440 nm are log-transformed and compared in Fig. 1 for the three data sets.
The fits between derived and known uncertainties in Fig. 1 are assessed using three goodnessof-fit parameters: i-R 2 , coefficient of determination; ii-MADi, mean absolute differences between derived values and the regression line; iii-RMSE, root mean of squared error. The regression line was computed using type-II model [19] and following the practice of the IOCCG report [9]. The goodness-of-fit parameters are computed for each data set separately as well as for all data points and are given in Table 1.  Figure 1 and Table 1 show that derived and known uncertainty values have a good linear relationship, with MADi and RMSE values ranging from minima for IOCCG to maxima for SeaWiFS-matchup. Correspondingly, the values of R 2 decrease from about 0.96 for IOCCG to 0.50 for NOMAD and reaches 0.45 for SeaWiFS-matchup. Table 1 and Fig. 1 confirm that the proposed method produces acceptable estimates of uncertainty for the three data sets. Overall, the simulated, measured and ocean color matchup yield a R 2 ∼ 0.61, MADi < 0.7 and RMSE < 1.2.

Formulation
The error term, δ 2 , was included in Eq. (4) to account for the uncertainty of the forward model. Figure 2 shows the comparison of σ 2 r on the X-axis, computed as (x d − x k ) 2 , against Eq. (4) on the Y-axis, with δ included (red dots) and without δ (grey circles). This figure is produced using the IOCCG data set and serves as a reference. The best-fit parameters, R 2 , MADi and RMSE, between known and derived radiometric uncertainties and the effect of including δ are shown in Table 2. It is obvious from Fig. 2 and Table 2 that adding δ improves the results for all wavelengths, with 13-35 % increase in R 2 and 13-44% decrease in MADi. The same improvement in RMSE is, however, more difficult to note. This can be attributed to the nature of RMSE which depends, apart from accuracy, also on the distribution of errors [20]. Willmot and Matsuura [20] recommend using mean absolute error (MADi in this study) instead of RMSE.
Recall that the following approximations were used: δ 2 = (x m − x k ) 2 and σ 2 r = (x d − x k ) 2 , from which the following relation can be derived For this to hold, i.e. ∑ w 2 i σ 2 i 0, the following condition should be satisfied x k 0.5 (x d + x m ). So assuming δ 2 = (x m − x k ) 2 and σ 2 r = (x d − x k ) 2 implies that the forward and inversion models should on average produce radiance that is equal to or larger than the actual one. This implicit assumption does not reflect the practice of model inversion. Equation (11) may under-or overestimate the observed radi-  ance based on noise level and water contents of dissolved and suspended matters. We avoid, however, this implication by using absolute values, such as δ = |x m − x k | and σ r = |x d − x k |, which prohibits expanding δ 2 and σ 2 r , and yet guarantees positive value.

Validation Results
The ensemble uncertainty, Ψ in Eq. (6), is a function of the weights, w i , which are obtained from the partial derivatives of Rs w (λ ) = f (iop) with respect to IOPs. The weight w i is only a function of IOPs and Ψ (uncertainty per radiometric error) is nearly independent of the error on Rs w . As such, Ψ computed for the three data sets should be comparable for similar IOPs, which is confirmed in Fig. 3. For validation, we express the uncertainty in terms of IOPs, which is done by multiplying Ψ by σ r for the derived error, and by expressing the known error as a sum of weighted IOPs's errors, Eq. (12). The reason behind this weighting is that the ensemble uncertainty of IOPs is also a weighted sum. The term Δiop 2 i in Eq. (12) includes the uncertainties associated with Rs w (λ ) (for NOMAD and SeaWiFS-matchup), so that the error will increase as the input spectra might have their spectral shapes affected by errors from various sources (sensor noise, atmospheric correction and spatial scale differences). For the SeaWiFS sensor, and the same model-parametrization setup as used in this paper, residuals from atmospheric correction and sensor noise are on average 50% of the total error [8, their Table 2]. The discrepancy in the spatial scale accounts for about 20% of the total error [21,22]. Thus, the effect of the model-parametrization setup contributes for 30% of the total uncertainty in the derived IOPs. On the other hand, the sensor noise signal in the NOMAD data set contributes by an average of 20% to the total error on derived IOP [8, their Table 4], Moreover, the NOMAD data are affected by surface reflectance, which can be in the same order of magnitude of water signal in case-1 waters [25]. This leaves about 40% of the total error to model-parametrization setup. Two additional error components will be added due to approximating σ 2 i by Δiop 2 i , and truncating the covariance terms in the Taylor expansion, in Eq. (4). These reasons could explain the large discrepancy shown in Fig. 1 between known and derived values for the NOMAD and SeaWiFS-matchup data sets.

Optimum Accuracy of Model-Parametrization Setup
Application of a gradient-based method for estimating the uncertainties in derived IOPs produces values proportional to the magnitude of IOPs and radiometric uncertainty [6,8]. This proportionality is compensated for by normalizing Ψ to the sum of derived IOPs, Ψ N (λ ). Doing so will result in a measure of relative ensemble uncertainty per unit error of radiance. However, for most of the ocean waters, where a b b , the sum of IOPs (cb) is equivalent to the total absorption coefficient, which makes cb hard to interpret statistically. The reciprocal of Ψ N (λ ) is the radiometric uncertainty normalized to the relative ensemble error, σ N r (λ ). The normalized radiometric uncertainty, σ N r , is expected to increase with water load of dissolved and suspended matters (turbidity), and approaches a constant value for waters with a high turbidity. Figure 3 shows σ N r , computed from derived values, versus the sum of known IOPs, cb k . For the three data sets, the normalized radiometric uncertainty approaches a constant value for cb k (440) ≥ 0.4 m −1 , with σ N r of about 0.0783 sr −1 (-2.547 on the log scale). This value, 0.0783, is very high and close to the saturation-of-reflectance (SoR) of the employed ocean color model, i.e. Eq. (11), SoR is defined here as being the highest radiometric value that can be produced by the ocean color model. This situation occurs in waters loaded with non-absorbing particles such that the fraction u in Eq. (11) approaches unity (i.e. absorption by constituents is small in comparison to backscattering b b a). In this case, SoR = lim u→1 Rs w = 0.0922 sr −1 . Conversely, the reciprocal of σ N r (λ ), Ψ N λ , decreases with increasing water turbidity reaching a constant value of 1/0.0922 = 10.85 sr. It is clear from the above discussion that higher radiometric uncertainty (σ N r (λ )) is expected for turbid waters, and the relative ensemble uncertainty is expected to be higher for clear water. In addition, Fig. 3 allows us also to provide an estimate of the optimum accuracy that is achievable with a model-parametrization setup. The maximum value of normalized radiometric uncertainty (at 440 nm) is computed as 0.0783 sr. This value can be converted to ensemble uncertainty using cb k (440) ≈ 0.4 m −1 as: 0.4/0.0783 = 5.11 sr m −1 . The minimum value of normalized radiometric uncertainty (also at 440 nm, see Fig. 3) is 0.0369 sr which corresponds to cb k (440) ≈ 0.0123 m −1 . In the same way we compute the minimum value of ensemble uncertainty as: 0.0123/0.0369 = 0.33 sr m −1 . The lower limit of the ensemble uncertainty shows that the ocean color model and IOPs parameterizations have an inherent error at 440 nm of at least 0.333 sr m −1 . Therefore, each error of 1 sr −1 results in 0.33 m −1 collective error of IOPs. This lower limit of error represents the optimum (maximum) accuracy that can be achieved by a model-parametrization setup.
Nominal values for radiometric error at 440 nm are in the range of 0.

Conclusions
A method is presented for the quantification of the combined accuracy of ocean color models and the parameterizations of inherent optical proprieties (IOPs), or model-parametrization setup. Its application produces a single (or ensemble) uncertainty measure for the collective errors in the derived IOPs relative to the radiometric uncertainty without the need for model inversion or prior information on the radiometric errors. As such, the method is directly applicable to existing satellite based IOP products. It can be used to compare different satellite data products and in a context of data merging where the respective uncertainties associated with the input data streams are required.
A thorough validation of the method is presented using simulated, in-situ measured and ocean color matchup data. For the synthetic data a very good fit is obtained between the derived and known values (R 2 > 0.95 and mean absolute difference (MADi) < 0.25 m −1 ). A reduced performance (0.45 < R 2 < 0.5 and 0.65 < MADi < 0.9) is, however, found for the in-situ and ocean color matchup data, which is attributed to additional error sources such as sensor noise, atmospheric correction and spatial scale differences. Further, we employ the method also for estimating optimum accuracy that could be achieved with the three data sets for a specific model-parametrization setup, which could be seen as the target accuracy in retrieving IOPs.
Application to time series (1997)(1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005)(2006)(2007) of global IOP products illustrates the capability of the method in identifying areas subject to large uncertainties. Specifically, the analysis of the annual trend reveals regions with ensemble uncertainties increasing at rates up to 8.0 sr m −1 decade −1 . Such trends are likely related to changes in the biophysical characteristics of waters associated with, for example, climatic changes or anthropogenic influences.
Quantifying the ensemble uncertainty of IOPs provides the optimum accuracy which can be achieved by a model-parametrization setup. Moreover, it has the potential of detecting areas for which the performance of a given model-parametrization setup is suboptimal or deteriorating. This provides the information needed for updating the global model-parametrization setup, which contributes to the overall uncertainty reduction within IOP products.