Uncertainty Evaluation on Temperature Detection of Middle Atmosphere by Rayleigh Lidar

Abstract

Measurement uncertainty is an extremely important parameter for characterizing the quality of measurement results. In order to measure the reliability of atmospheric temperature detection, the uncertainty needs to be evaluated. In this paper, based on the measurement models originating from the Chanin-Hauchecorne (CH) method, the atmospheric temperature uncertainty was evaluated using the Guide to the Expression of Uncertainty in Measurement (GUM) and the Monte Carlo Method (MCM) by considering the ancillary temperature uncertainty and the detection noise as the major uncertainty sources. For the first time, the GUM atmospheric temperature uncertainty framework was comprehensively and quantitatively validated by MCM following the instructions of JCGM 101: 2008 GUM Supplement 1. The results show that the GUM method is reliable when discarding the data in the range of 10–15 km below the reference altitude. Compared with MCM, the GUM method is recommended to evaluate the atmospheric temperature uncertainty of Rayleigh lidar detection in terms of operability, reliability, and calculation efficiency.

1. Introduction

The middle atmosphere, located in the range of 10–100 km, contains complex physical, chemical, and dynamic processes, which have a great impact on global climate change. In addition, it offers the main area of activity for the rapidly developing field of hypersonic vehicles; thus, it is extremely important in the military and aerospace industries [1]. Perception and cognition of the primary properties of the middle atmosphere, such as density, temperature, and wind-speed distribution, lay the basis for the above applications. With respect to temperature measurement, various measures have been developed, including sounding rockets [2], satellites [3], and lidar [4]. Sounding rockets are costly and can only provide limited data during the time they spend in the region, so they are generally used to calibrate the other remote sensing instruments. Satellite remote sensing has low spatial and temporal resolution and is mainly used to study atmospheric dynamics on a large scale. Compared with the former methods, lidar, especially the Rayleigh lidar, has the advantages of high space-time resolution, continuous monitoring, and good flexibility by being equipped on ground-based, shipborne, airborne, and even balloon platforms [5,6,7]. At present, Rayleigh lidar systems all over the world have been incorporated into a global observation network to study the dynamics of the middle atmosphere [8].

To evaluate the performance of Rayleigh lidar, the general method involves comparing the in situ detection data with rocket-sounding measurements [9], a standard atmospheric model [10], and satellite databases [11]. However, this method suffers from varying validation standards and numerical errors. Measurement uncertainty, which describes the dispersion of measured quantity values, provides another effective means to evaluate the reliability of measurement results. Other than comparing with standard values that may be unavailable or do not exist, measurement uncertainty only depends on the potential sources of uncertainty involved in the setup and inversion algorithms of the lidar. Measurement uncertainty is scientific, reasonable, and conducive to the uniform evaluation of measurement results. In atmospheric temperature measurement, the error transfer formula is used to calculate the uncertainty [12], where the main source is considered to be the random statistical uncertainty of detected photon counts [13,14,15]. Some publications have also given expressions for the atmospheric temperature uncertainty owing to the ancillary temperature uncertainty [16], but there was confusion between the concepts of error and measurement uncertainty. Other reports intentionally introduced initial errors to investigate the effect of ancillary temperature uncertainty on temperature retrieval, although specific analytical expressions were not given [17,18,19]. In order to quantify the contribution of each source of uncertainty and to give the atmospheric temperature uncertainty, a more efficient method is required. The internationally recognized uncertainty evaluation method currently follows the Guide to the Expression of Uncertainty in Measurement (GUM). The GUM method has been widely used in various fields for uncertainty analysis [20,21,22]. In 2016, Leblanc et al. evaluated the uncertainty of the lidar temperature data products in the Network for the Detection of Atmospheric Composition Change (NDACC) database using the GUM method [23]. The literature was quite detailed in analyzing the potential uncertainty sources, and the combined standard uncertainty was given, including the uncertainty components that made contributions, but lacked rigorous validation for the GUM atmospheric temperature uncertainty framework. It is important to note that there are prerequisites for the GUM method; that is, the nonlinearity of the measurement model is negligible, the Central Limit Theorem applies, and the Welch-Satterthwaite formula is sufficient to calculate the degrees of freedom [24]. In practice, there are cases where the above conditions are not satisfied and where there is uncertainty as to whether the conditions are appropriate. To overcome these problems, the Joint Committee for Guides in Metrology (JCGM) published the JCGM 101:2008 GUM Supplement 1 [25], which proposed that the Monte Carlo Method (MCM) is capable of evaluating the uncertainty of complex measurement models and that the GUM uncertainty framework can be validated by MCM.

In general, there are few reports on evaluating the atmospheric temperature uncertainty by the GUM method, and if there have been any, no rigorous validation of this method has been accomplished. In this paper, the GUM uncertainty framework for evaluating atmospheric temperature uncertainty was validated quantitatively by MCM based on simulation data, considering both the detection noise and ancillary temperature uncertainty, following the instructions of Supplement 1. Furthermore, we gave a calculation example with measured data to evaluate the atmospheric temperature uncertainty enabled by a ground-based lidar. The comparison of GUM and MCM in terms of operability and reliability indicates that the GUM method is more favorable, with a dominant advantage in efficiency.

2. Atmospheric Temperature Measurement Models

2.1. Lidar Equation

The lidar equation, which builds the relation between a backscattered signal and the system parameters of the lidar as well as the associated atmospheric characteristics, is the basis of lidar detection. The Rayleigh lidar equation is

$$P_R(\lambda ,z)=P_L\left(\lambda \right){\sigma }_M\left(\lambda \right)n_A\left(z\right)\Delta z\frac{A}{{(z-z_L)}^2}\eta (\lambda ,z)T^2(\lambda ,z)G\left(z\right)+N_B$$

(1)

where $P_R(\lambda ,z)$ is the number of photons received from altitude z , $P_L\left(\lambda \right)$ is the number of photons emitted by the lidar, ${\sigma }_M\left(\lambda \right)$ is the Rayleigh cross section, $n_A\left(z\right)$ is the number density of air molecules, A is the area of the receiving telescope, $z_L$ is the altitude of the lidar, $\eta (\lambda ,z)$ is the lidar system efficiency, $T(\lambda ,z)$ is the atmospheric transmittance, and $G\left(z\right)$ is the geometry parameter of the lidar system. $N_B$ is the background noise in echo signal, calculated from

$$N_B=N_b+N_d$$

(2)

where $N_b$ and $N_d$ are the background radiation noise and detector thermal noise, respectively, given by

$$N_b=\frac{\eta \lambda }{hc}{p}_b\pi {\left(\frac{{\theta }_r}{2}\right)}^2\Delta \lambda AT\Delta t$$

(3)

$$N_d=CPS\times \Delta t$$

(4)

respectively, where $P_b$ is the sky background radiance, ${\theta }_r$ is the field of view of the receivers, $\Delta \lambda$ is the transmission bandwidth of the optical filter, $CPS$ is the dark count of the detector, and $\Delta t$ is the system collection time. When the lidar operates in photon counting mode, the noise following the Poisson distribution should be added to the signal to acquire the actual lidar echo. It is assumed that the lidar is carried by a floating platform at an altitude of 20 km, which prevents the impacts of the troposphere and most of the noise from sunlight scattering to realize continuous monitoring. The basic parameters are shown in

Table 1. Simulation parameters of the lidar system.

System Parameter	Value
Laser wavelength/nm	532
Repetition frequency/Hz	50
Pulse energy/mJ	40
System efficiency	0.191
Range resolution/m	100
Receiver diameter/m	0.35
Field of view/$\mathsf{\mu }$rad	165
Dark counts	50
Filter bandwidth/nm	0.25
Lidar-site altitude/km	20

.

2.2. Chanin-Hauchecorne Method for Atmospheric Temperature Retrieval

Since the atmosphere above 30 km can be regarded as pure and free of aerosol, the backscattering signal of lidar mainly comes from the Rayleigh scattering of atmospheric molecules, which provides the basic concept of calculating molecule number density at different altitudes from the echo signals received by Rayleigh lidar. Assuming there is a constant relation between atmospheric composition and altitude, the atmospheric density is proportional to the molecular number density, and thus a relative atmospheric density profile can be obtained. The reference altitude is usually at the highest point with a reasonable signal-to-noise ratio (SNR), and the temperature at this point is used as an auxiliary temperature for initializing the temperature profile, i.e., $T_a\left(z_{ref}\right)$=$T_a\left(z_{TOP}\right)$, where T is temperature. Here, the auxiliary temperature is taken from the NRLMSISE-00 standard atmosphere model [26]. Furthermore, combining the ideal gas law and the hydrostatic equation, the atmospheric temperature can be retrieved from the integration technique, which is also called the Chanin-Hauchecorne (CH) method [9].

$$T\left(z\right)=\frac{N\left(z_{ref}\right)}{N\left(z\right)}T\left(z_{ref}\right)+\frac{M_0\Delta z}{RN\left(z\right)}S\left(z\right)$$

(5)

where N is the relative number density measured by the lidar, $M_0$ is the atmospheric molecular weight, $\Delta z$ is the distance resolution, S is the discrete summation term, and R is the molar gas constant. If there is a complete overlap between the laser beam and the telescope field of view, $N\left(z\right)$ can be written as a function of the returning photon counts $P\left(z\right)$:

$$N\left(z\right)={(z-z_L)}^{2}P\left(z\right)$$

(6)

Irrespective of signal saturation or the pulse pile-up effect [27], the photon counts $P\left(z\right)$ can be expressed as a function of the raw signal $R\left(z\right)$ recorded in the data files at altitude z:

$$P\left(z\right)=R\left(z\right)-B\left(z\right)$$

(7)

$S\left(z\right)$ in Equation (5) is the discrete summation term:

$$S\left(z\right)=\sum _{z^{\prime }=z}^{z_{ref}-1}\overline{N}\left(z^{\prime }\right)g\left(z^{\prime }\right)$$

(8)

in which $\overline{N}\left(z\right)$ represents the layer-averaged number density of the vertical layer comprised between $z^{\prime }$ and $z^{\prime }+1$ , and the variation of gravity acceleration g with altitude is expressed as

$$\overline{N}\left(z^{\prime }\right)=\sqrt{N\left(z^{\prime }\right)N(z^{\prime }+1)}$$

(9)

$$g\left(z\right)=g_0{\left(\frac{r_0}{r_0+z}\right)}^2$$

(10)

where $r_0$ is the effective radius of the earth given by the Lambert equation at a particular latitude, and $g_0$ is the acceleration of gravity at sea level.

3. Atmospheric Temperature Uncertainty Evaluation with Simulation Data

From Equation (5), the accuracy of the retrieval results can be related to the relative measurement number density and the auxiliary temperature uncertainty. The detection noise accompanying the return signal affects the signal intensity, which in turn affects the magnitude of the relative measured number density. In addition, depending on aspects of the lidar measurement environment and inversion algorithms, uncertainty sources in atmospheric temperature retrieval by Rayleigh lidar may also involve ozone absorption, auxiliary number density, and background noise extraction. However, they contribute very little to the combined standard uncertainty of atmospheric temperature [23]. Therefore, only the detection noise and ancillary temperature uncertainty are the main uncertainty sources considered in this evaluation.

3.1. Temperature Uncertainty Evaluation by GUM

3.1.1. The Law of Uncertainty Propagation

GUM provides a framework for evaluating uncertainty. The best estimates of the input quantities and their standard uncertainties are propagated following the propagation law to provide an estimate of the output quantity and its standard uncertainty [24], expressed as

$$u_y=\sqrt{\sum _{n=1}^N{\left(\frac{\partial y}{\partial x_n}\right)}^2{u_n}^2+2\sum _{m=1}^{N-1}\sum _{n=m+1}^N\frac{\partial y}{\partial x_n}\frac{\partial y}{\partial x_m}{r}_{nm}{u}_n{u}_m}$$

(11)

where $x_n$ is the best estimate of the input quantity $X_n$, $u_n$ is the standard uncertainty associated with $x_n$, $r_{nm}$ is the correlation coefficient between two input quantities $x_n$ and $x_m$, y is the best estimate of the output quantity Y, and $u_y$ is the standard uncertainty associated with y. The mathematical relationship between the input and output quantities can be described by a measurement model.

3.1.2. Temperature Uncertainty Owing to Auxiliary Temperature Uncertainty

The auxiliary temperature $T_a\left(z_{TOP}\right)$ is mostly taken from the atmospheric model, so the auxiliary temperature used in retrieval has an unavoidable deviation from the real atmospheric temperature at that right moment. Khanna et al. showed that the auxiliary temperature uncertainty resulted in non-negligible uncertainty for temperature retrieval in the range of 10–15 km below the reference altitude [28]. Therefore, the acquired temperatures in this part must be discarded, which is an inherent disadvantage of the CH method for retrieving atmospheric temperatures. Substituting Equation (5) into Equation (11), the auxiliary temperature uncertainty is propagated to the temperature retrieval profile:

$$u_{T\left(AUX\right)}\left(z\right)=\frac{N\left(z_{TOP}\right)}{N\left(z\right)}{u}_{Ta}\left(z_{TOP}\right)$$

(12)

where $u_{T_a}\left(z_{TOP}\right)$ is conservatively estimated to be 20 K and the auxiliary temperature is from the MSISE series models covering the detection altitude of 30–100 km [29].

3.1.3. Temperature Uncertainty Owing to Detection Noise

The detection noise, which follows a Poisson distribution, is introduced during the photon-detecting process. The uncertainty in the raw signal R owing to detection noise is

$$u_{R\left(DET\right)}\left(z\right)=\sqrt{R\left(z\right)}$$

(13)

Starting from Equation (13), the uncertainty owing to detection noise is propagated step by step to the atmospheric temperature T. Firstly, the propagation law of uncertainty, that is, Equation (11), is applied to Equation (7) to transfer the uncertainty component $u_{R\left(DET\right)}$ to the signal $P\left(z\right)$:

$$u_{P\left(DET\right)}\left(z\right)=\sqrt{R\left(z\right)}$$

(14)

Secondly, Equations (6) and (14) are applied to Equation (11), and the uncertainty component $u_{P\left(DET\right)}$ continues to propagate to the relative measured number density N:

$$u_{N\left(DET\right)}\left(z\right)=\frac{N\left(z\right)}{P\left(z\right)}{u}_{P\left(DET\right)}\left(z\right)$$

(15)

Thirdly, assuming that the relative number density N between adjacent altitudes is not correlated, $u_{N\left(DET\right)}$ propagates to the layer-averaged density $\overline{N}$:

$$u_{\overline{N}\left(DET\right)}\left(z^{\prime }\right)=\frac{1}{2}\sqrt{\frac{N(z^{\prime }+1)}{N\left(z^{\prime }\right)}{u}_{N\left(DET\right)}^2\left(z^{\prime }\right)+\frac{N\left(z^{\prime }\right)}{N(z^{\prime }+1)}{u}_{N\left(DET\right)}^2(z^{\prime }+1)}$$

(16)

Then, considering the propagation of uncertainty component $u_{\overline{N}\left(DET\right)}$ to the summation term S [23]:

$$u_{S\left(DET\right)}\left(z\right)=\sqrt{2\sum _{z^{\prime }=z}^{z_{TOP}-1}{g}^2\left(z^{\prime }\right)u_{\overline{N}\left(DET\right)}^2\left(z^{\prime }\right)}$$

(17)

Finally, $u_{S\left(DET\right)}$ propagates to the temperature profile $T\left(z\right)$, and the atmospheric temperature uncertainty owing to detection noise is calculated:

$$u_{T\left(DET\right)}\left(z\right)=\frac{1}{N\left(z\right)}\begin{array}{c}\sqrt{T^2\left(z\right)u_{N\left(DET\right)}^2\left(z\right)+T_a^2\left(z_{TOP}\right)u_{N\left(DET\right)}^2\left(z_{TOP}\right)+{\left(\frac{M_0\Delta z}{R}\right)}^2{u}_{S\left(DET\right)}^2\left(z\right)}\hfill \end{array}$$

(18)

3.1.4. Combined Standard Uncertainty

The combined standard uncertainty for the retrieved atmospheric temperature is formed by combining both uncertainty components, the auxiliary temperature uncertainty $u_{T\left(AUX\right)}$ and detection noise $u_{T\left(DET\right)}$:

$$u_T\left(z\right)=\sqrt{u_{T\left(DET\right)}^2\left(z\right)+u_{T\left(AUX\right)}^2\left(z\right)}$$

(19)

3.2. Temperature Uncertainty Evaluation by MCM

Although the GUM method is considered perfectly suitable in many cases, it is not straightforward to determine all the conditions for its application.The applicability of the MCM is broader than that of GUM, it is recommended in Supplement 1 that both the GUM and MCM be applied and the results compared. If the comparison is favorable, the GUM uncertainty framework can be used in this case and for sufficiently similar problems. Otherwise, MCM or another appropriate approach should be considered [25]. Given that the temperature measurement model is complex and the GUM temperature uncertainty framework has never been validated, a comprehensive comparison between the GUM and MCM is mandatory before the GUM is applied to temperature uncertainty assessment.

The MCM evaluates measurement uncertainty based on the principle of distribution propagation [25]. The probability density function (PDF) of each input quantity is discretized for sampling, and the discrete PDF value of the output quantity is obtained through the measurement model. The best estimate of $y_M$, coverage interval $[y_{low}$, $y_{high}]$, and standard uncertainty $u_M\left(y\right)$ of the output quantity is obtained from this process. The Adaptive Monte Carlo (AMC) procedures have been implemented to improve operational efficiency while ensuring the reliability of the results. The steps to evaluate uncertainty are shown in Figure 1.

The MCM validation of the GUM uncertainty framework is as follows [25]:

Step 1: Obtain a $p\%$ coverage interval $y\pm U_p$ for the output quantity using the GUM uncertainty framework, where $p\%$ is the stipulated coverage probability, y is the best estimate, and $U_p$ is the extended uncertainty, which is calculated from the standard uncertainty $u\left(y\right)$. In this paper, y is the atmospheric temperature retrieved using Equation (5): p = 95, $U_p$ = $1.96\times u\left(y\right)$.

Step 2: Provide the standard uncertainty $u_M\left(y\right)$ and the two endpoints $y_{low}$ and $y_{high}$ of the same coverage interval for the output quantity by the AMC procedures.

Step 3: Determine the numerical tolerance $\delta$, defined as the half-width of the shortest interval containing all numbers that can correctly be expressed to a specified number of significant decimal digits, according to $u_M\left(y\right)$ expressed in the form of $u_M\left(y\right)$ = $c\times {10}^l$. l is an integer, and c is a decimal integer with the same digits as the significant decimal digits of $u_M\left(y\right)$; then $\delta$ = $\frac{1}{2}\times {10}^l$;

Step 4: Acquire the absolute deviations ($d_{low}$ and $d_{high}$) of the endpoints ($y_{low}$ and $y_{high}$) of the coverage intervals using the following formulas. If neither is larger than the numerical tolerance $\delta$, the comparison is successful, and the results obtained by the GUM method are reliable.

$$d_{low}=\left|(y-U_p)-y_{low}\right|$$

(20)

$$d_{high}=\left|(y+U_p)-y_{high}\right|$$

(21)

3.2.1. Temperature Uncertainty Owing to Auxiliary Temperature Uncertainty

To validate the GUM uncertainty framework for evaluating the atmospheric temperature uncertainty owing to the ancillary temperature uncertainty, ideal return photon counts without noise were simulated and used to retrieve the atmospheric temperature in running the Monte Carlo experiments. The auxiliary temperature at the top of the profile, $T_a\left(z_{TOP}\right)$ follows a normal distribution with the mean of $T_{00}\left(z_{TOP}\right)$ taken from the NRLMSISE-00 atmospheric model and the standard deviation of $u_{T00}\left(z_{TOP}\right)$ = 20 K. The numerical results were stabilized after 4,360,000 Monte Carlo trials at detection altitudes of 30–60 km and an integration time of 300 s. The standard uncertainty evaluated by MCM is shown in Figure 2, together with the results obtained by GUM for comparison. Obviously, two curves perfectly match each other under the same conditions.

The atmospheric temperature uncertainty owing to the auxiliary temperature uncertainty varies at different altitudes. Depending on the standard uncertainty, one significant digit was taken for 30–50 km, and the corresponding numerical tolerance was calculated to be 0.05 in the altitude range of 30–36.9 km and 0.5 in the altitude range of 37–50 km. As shown in Figure 3, the absolute deviation at each altitude was smaller than the corresponding numerical tolerance, indicating that the GUM uncertainty framework was ideally validated.

3.2.2. Temperature Uncertainty Owing to Detection Noise

In terms of the atmospheric temperature uncertainty owing to the detection noise, simulated photons $N_p$ with only Poisson noise in per altitude bin were replaced with $N_p$ + $\alpha \sqrt{N_p}$, where $\alpha$ is a random number following a normal distribution with the standard deviation of one and a mean of zero. After running the 460,000 Monte Carlo experiments, Figure 4 compares the temperature uncertainties owing to detection noise by both the GUM and MCM methods, which were almost exactly coincident under the same conditions.

The standard uncertainty was not constant at different altitudes, as with the numerical tolerances. The numerical tolerance was 0.05 at 30–37.1 km and 0.5 at 37.2–50 km. Taking one significant digit for the measurement uncertainties, the simulation results are shown in Figure 5. The absolute deviation values of the interval endpoints at the corresponding attitudes are smaller than the corresponding numerical tolerances, indicating that the calculated results of the GUM method were validated. This also proves that the atmospheric temperature uncertainty owing to detection noise was effectively propagated in each sub-measurement model of Section 3.1.3.

3.2.3. Combined Standard Uncertainty

Both sources of uncertainty mentioned above were introduced, and the combined standard uncertainty of the atmospheric temperature was evaluated, as shown in Figure 6, where the results of 1,980,000 Monte Carlo tests and the GUM method were compared. Two methods keep in close agreement, which can be predicted by the former results in Figure 2 and Figure 4.

At the detection altitudes of 30–34.8 km and 34.9–50 km, one significant digit was taken for the combined standard uncertainties, and the corresponding numerical tolerances were calculated to be 0.05 and 0.5, respectively. As shown in Figure 7, the absolute deviation values of the interval endpoints at the corresponding attitudes were smaller than the respective numerical tolerances. The successful evaluation of GUM indicates that the form of Equation (19) is correct and that the two uncertainty sources are independent. So far, it has been validated that the GUM method can be used to evaluate the temperature uncertainty caused by the two uncertainty sources, and combining the sum of squares of each component gives the combined standard uncertainty.

According to Figure 3, Figure 5, and Figure 7, all the absolute deviations of the endpoints increase with altitude. The explanation is that the atmospheric density is higher at lower altitudes, so that the SNR of the echo signal is higher and the source of uncertainty has less influence on the retrieved temperature, leading to a smaller uncertainty. The atmospheric density decreases exponentially with the detection altitude, and as the SNR decreases, the effect of the uncertainty source becomes more significant and the uncertainty increases. When we were calculating $y_{low}$ and $y_{high}$, the extended uncertainty [25] (the half-width of the coverage interval) was used, and thus the absolute deviation also showed an increasing tendency with altitude.

The GUM uncertainty framework was further validated by MCM while varying the detection altitude and corresponding integration time, as shown in Figure 8, Figure 9 and Figure 10. The significant digits were chosen according to the magnitude of the atmospheric temperature uncertainties, and the calculated numerical tolerances are listed in

Table 2. Altitude, integration time and the corresponding numerical tolerance in simulation. The range resolution is 0.1 km.

Altitude Range (km)	Integration Time (s)	Component	Altitude Section (km)	Significant Digit	Numerical Tolerance
30–50	60	Auxiliary	30–40	1	0.5
		Detection	30–31.2	1	0.05
			31.3–40	1	0.5
		Combined	30–40	1	0.5
30–60	420	Auxiliary	30–36.9	1	0.05
			37–50	1	0.5
		Detection	30–38.3	1	0.05
			38.4–50	1	0.5
		Combined	30–35.1	1	0.05
			35.2–50	1	0.5
30–70	2400	Auxiliary	30–30.8	2	0.005
			30.9–46.7	1	0.05
			46.8–60	1	0.5
		Detection	30–45.8	1	0.05
			45.9–60	1	0.5
		Combined	30–43.4	1	0.05
			43.5–60	1	0.5

. The results show that the GUM atmospheric temperature uncertainty calculation frameworks were all validated perfectly by the MCM after removing the 10–15 km range below the reference altitude. It should be noted that the retrieved data in the range of 10–15 km below the reference altitude should also be discarded according to the CH method itself.

4. Calculation Example of the Temperature Uncertainty with Measured Data

Finally, measured data were substituted into these theoretical models to verify their applicability in evaluating temperature uncertainty. Since there were no experimental results of lidar working on a floating platform at an altitude of 20 km, measured data collected by a ground-based Rayleigh lidar were used. The data on returning photons were provided by the National Space Science Center of the Chinese Academy of Sciences and collected at 17:30 on 3 September 2018. The lidar is located at Yanqing, Beijing (40°18′N, 116°40′E) with a vertical resolution of 122.88 m and an effective detection altitude range of around 40–80 km. The return signal above 100 km was averaged as the background and subtracted from the raw signal. Furthermore, the photon data were used for the subsequent evaluation of atmospheric temperature uncertainty.

4.1. Temperature Uncertainty Evaluation by GUM

Figure 11 demonstrates the atmospheric temperature uncertainty evaluated by the GUM method, in which (a) and (b) are the uncertainties owing to the auxiliary temperature uncertainty and detection noise, respectively, (c) is the combined standard uncertainty, and (d) synthesizes all the plots for an intuitional comparison. According to Figure 11d, the curves corresponding to two uncertainty components overlap at around 63 km. The detection noise contributes more below 63 km, while the auxiliary temperature uncertainty dominates above this altitude. It is worth noting that this intersection does not always exist, as the relative magnitudes of two uncertainty components are determined by various factors, such as the lidar configuration, data processing method, auxiliary dataset (atmospheric models), and range of detection altitude.

4.2. Comparison between GUM and MCM

Figure 12 shows the comparison of atmospheric temperature uncertainty evaluated by GUM and MCM, respectively. The uncertainty introduced by the auxiliary temperature uncertainty and detection noise, and the combined standard uncertainty are all included as retrieved from the measured data. A near-perfect agreement between the two methods was achieved. The minor difference generally increases with altitude, the reason for which has been given in Section 3.2.3. With the decrease in SNR, the difference between two methods becomes more noticeable, but it is limited at the order of ${10}^{-2}$ K.

The coverage intervals containing $95\%$ probability were obtained using the GUM and MCM methods; the absolute deviation curves of the endpoint values for their coverage intervals are shown in Figure 13. The curves have a similar, varying trend to the simulation results. At an altitude of 40.06 km, the standard uncertainty of the atmospheric temperature owing to the auxiliary temperature uncertainty was 0.0948 K. Taking two significant digits for the uncertainty, the numerical tolerance at this point was calculated to be 0.005. Since $d_{low}$ = 0.00097 and $d_{high}$ = 0.00088, both were less than the numerical tolerance. According to the uncertainty magnitudes induced by auxiliary temperature uncertainty, one significant digit was taken, and the numerical tolerances were 0.05 and 0.5 in the detection ranges of 40.18–58.25 km and 58.37–69.92 km, respectively. As to the uncertainty caused by the detection noise, one significant digit was taken in the ranges of 40.06–54.56 km and 54.68–69.92 km, corresponding to the numerical tolerances of 0.05 and 0.5, respectively. When focusing on the combined standard uncertainty at different detection altitudes, one significant digit was taken in the range of 40.06–69.92 km. The numerical tolerance was calculated to be 0.05 in the range of 40.06–52.84 km, while it was 0.5 in the range of 52.96–69.92 km. It can be seen from Figure 13 that the absolute deviation at each altitude is smaller than the corresponding numerical tolerances, indicating that the GUM framework to evaluate the uncertainty of the atmospheric temperature has been validated by MCM with measured data.

5. Conclusions

Measurement models for evaluating atmospheric temperature uncertainty were developed based on the CH method. According to JCGM 101: 2008 GUM Supplement 1, we quantitatively validated the applicability of the GUM uncertainty framework by MCM based on both the simulation data and measured data provided by the National Space Science Center of the Chinese Academy of Sciences. By considering two major components, the auxiliary temperature uncertainty and the detection noise, the results show that the GUM method is reliable after discarding the data in the range of 10–15 km below the reference altitude, which is consistent with the requirements of the CH method. When comparing the GUM with the MCM, both methods can quantitatively describe the contribution of an uncertainty component to the combined standard uncertainty. However, usually it is necessary to run hundreds of thousands or even millions of Monte Carlo trials to obtain the $95\%$ coverage interval of the atmospheric temperature retrieval. As a result, the running time required for MCM is significantly longer than GUM, accompanied by the requirement of a larger amount of data storage space. Therefore, the GUM method is the preferred method to evaluate the uncertainty of the atmospheric temperature detected by Rayleigh lidar. It should be noted that all the conclusions are transferrable to different platforms, especially the ground-based Rayleigh lidar being used in temperature measurements of the middle atmosphere.