Comparison of Desroziers and Three-Cornered Hat Methods for Estimating COSMIC-2 Bending Angle Uncertainties
1. Introduction
In this paper we compare two seemingly different methods of estimating random error statistics (uncertainties) of observations, the three-cornered hat (3CH) method (Gray and Allan 1974; Sjoberg et al. 2021) and Desroziers (Desroziers et al. 2005) method and show several examples of estimated uncertainties of Constellation Observing System for Meteorology, Ionosphere and Climate 2 [COSMIC-2 (C2)] radio occultation (RO) observations. We also compare the uncertainty estimates from both methods to the RO error models used by several operational forecasting systems, in order to guide adjustments of these error models toward more accurate estimates of RO uncertainties and to improve the impact of RO observations on model analyses and forecasts.
We also investigate the variation of these uncertainty estimates with latitude and in two different seasons.
Finally, we consider the relationship between the vertical variation of the C2 uncertainties and atmospheric variability, as measured by the standard deviation of the C2 sample between 45°S and 45°N.
a. Estimation of uncertainties in observational or model datasets
In modern data assimilation schemes, observations are inversely weighted by their random error covariances (Fletcher 2017). Accurately estimating the error variances of observations is crucial for data assimilation systems and numerical weather prediction (NWP). It is also important in research and other applications to know the error characteristics of the datasets used. There are several ways of estimating observational uncertainties. With the assumption that observation and forecast errors are not correlated, the apparent error is defined as the difference between observations and short-range model forecasts (Hollingsworth and Lönnberg 1986; Kuo et al. 2004). The apparent error method requires two collocated datasets and an independent estimate of the error statistics of the model forecast. In another method, observation-minus-background (O − B) and observation-minus-analysis (O − A) statistics are used to estimate observation error statistics in operational forecast models (Desroziers et al. 2005).
The Desroziers method has been used to estimate the observation error covariance matrices of microwave imager and infrared sounder radiances, Doppler radial winds, RO bending angles (e.g., Stewart et al. 2013; Weston et al. 2014; Bormann et al. 2011, 2016; Bonavita 2014; Cordoba et al. 2017; Campbell et al. 2017; Simonin et al. 2019; Bowler 2020; El Aabaribaoune et al. 2021).
A third technique to estimate random error statistics from three datasets (models or observations) simultaneously is the 3CH method (Gray and Allan 1974). Anthes and Rieckh (2018) were the first to use 3CH method for atmospheric datasets. They estimated the observation errors of COSMIC (Anthes et al. 2008) RO, radiosondes (RS), and two models at four RS stations in the tropics and subtropics. Sjoberg et al. (2021) summarize the history of the 3CH method, the factors that limit its accuracy, and its similarity and differences from a similar, but developed independently, triple collocation method (Stoffelen 1998). Rieckh et al. (2021) used the 3CH method to estimate the uncertainties of COSMIC, COSMIC-2 (Schreiner et al. 2020), and RS together with 10 different model forecast or reanalyses datasets.
These methods estimate the random error statistics or uncertainties of observations, which are measures of precision, rather than the accuracies, or biases. It is impossible to determine accuracy in atmospheric properties, because truth is never known, and in fact depends on the horizontal, vertical and temporal scales of the property being measured (representativeness), modeled or estimated (O’Carroll et al. 2008). However, it is not necessary to know truth when computing the uncertainties with the Desroziers or 3CH methods.
Because of the variations in the scale or representativeness of observations and models, which affect the estimated error uncertainties, it is important to distinguish between resolution and footprint. These are defined as follows:
-
Vertical or horizontal resolution: The average spacing between observations or model grid points. Examples include the average horizontal spacing between a network of radiosonde observations or radio occultation profiles, or model grid points.
-
Vertical or horizontal footprint: The vertical or horizontal scale of the observation or model grid point that a value represents. Examples include a single radiosonde observation represents essentially a point value. A single RO observation represents a weighted average over a vertical distance of ∼200 m and a horizontal distance of ∼200 km. A model grid point represents an average over the model cell volume (typically 30 km × 30 km × 500 m).
An observing system may have low horizontal resolution with small footprints (e.g., a radiosonde network with an average spacing between RS of 500 km), or high horizontal resolution with large footprints (e.g., an RO network with average spacing between RO profiles of 50 km or a swath of microwave satellite soundings with a pixel size of 40 km). Sjoberg et al. (2021) and Rieckh et al. (2021) discuss the effect of representative differences on the 3CH uncertainty estimates and provide examples.
b. Radio occultation observations
Radio occultation observations are obtained when radio signals emitted from Global Navigation Satellite System (GNSS) satellites pass through the atmosphere and are bent (refracted) and delayed by the atmosphere. By measuring the Doppler shift of these signals with receivers on low-Earth-orbit (LEO) satellites, vertical profiles of bending angles and refractivity can be obtained and assimilated in NWP models. The RO observations contain information on temperature and water vapor, have high vertical resolution (∼200 m) and corresponding small vertical footprints, are relatively unbiased and unaffected by clouds and precipitation, and have high accuracy and precision (Kursinski et al. 1997; Anthes 2011). They have had a significant positive impact on operational NWP models and reanalyses in many centers, e.g., ECMWF (Healy 2008; Healy and Thépaut 2006), NCEP (Cucurull et al. 2007), the Met Office (Rennie 2010), Environment and Climate Change Canada (Aparicio and Deblonde 2008), Météo-France (Poli et al. 2009, 2010), Center for Weather Forecasts and Climate Brazil (Banos et al. 2019), and the U.S. Naval Research Laboratory (Ruston and Healy 2020). The impact of RO observations on NWP depends critically on the estimated RO random error variances specified in the data assimilation process (Bowler 2020).
The assumed RO uncertainties used by different operational NWP modeling systems vary considerably, as shown by the examples in Fig. 1. The uncertainty models shown for bending angle are empirical functions in the NRL, ECMWF, and NOAA Gridpoint Statistical Interpolation (GSI) systems and statistically derived by the Met Office. The NRL and ECMWF models estimate the bending angle error directly as a percent, up to a noise floor following Ruston and Healy (2020), while the NOAA GSI errors are estimated in radians. A mean profile on bending angle was calculated for 15 December 2020–15 January 2021 to find the percentage error for the noise floor of 3 microradians (ECMWF) and 6 microradians (NRL), and was also used to transform the NOAA GSI model in radians to a percentage. Also, it is critical to note that the NOAA GSI model adds an inflation factor based on the number of occultation points within the model vertical layer, which can significantly increase the uncertainty up to 2 times in the final estimate. The uncertainty estimates from the Met Office are shown for two sensors, and are provided at discrete latitude bands at 2-km vertical resolution in impact height. Though broadly similar in vertical structure with all exhibiting maxima at the surface, a rapid decay to around 10-km impact height and then rapid growth above 30-km impact height, the differences between the error models throughout the column vary greatly. Therefore there is considerable interest in refining the error estimates and understanding how they vary with latitude, season, and atmospheric conditions, e.g., warm versus cold or dry versus moist conditions (Bowler 2020).
(right) RO bending angle error standard deviations (uncertainties) assumed by four operational modeling centers in global NWP models. NRL is the U.S. Naval Research Laboratory, ECMWF is the European Centre for Medium-Range Weather Forecasts, GSI is the National Centers for Environmental Prediction (NCEP) model, and UKMO is the Met Office. Different latitudes are presented. (left) An enlarged portion of the right panel to show the details of the different error models for low (0%–5%) assumed errors.
Citation: Journal of Atmospheric and Oceanic Technology 39, 7; 10.1175/JTECH-D-21-0175.1
c. Outline of this study
In this study we compare the Desroziers and 3CH methods for estimating uncertainties of C2 RO bending angle observations. COSMIC-2 was launched 25 June 2019 into an equatorial orbit and is producing ∼5000 RO profiles per day (Schreiner et al. 2020). Ruston and Healy (2020) showed that C2 observations have a significant positive impact on NRL and ECMWF operational forecasts.
We first summarize these two methods, the assumptions they make, and the factors that limit their accuracy. We discuss how the methods are similar and how they differ and confirm the theoretical analysis of Todling et al. (2022) which was developed during Ricardo Todling’s review of the first draft of this paper. We then compare the two methods’ estimates of C2 bending angle uncertainties using several combinations of ancillary datasets, including the ERA5 analysis and background fields, for the 3CH estimates. We consider the variation of these estimates with latitude and for two separate months of data (16 August–15 September 2020 and April 2021).
2. Summary of Desroziers and 3CH methods
a. Desroziers
The Desroziers method uses values of observations and their background and analysis counterparts obtained from the data assimilation process in an NWP model to estimate consistent diagnostics for the covariances of observations, background, and analysis errors. Desroziers uncertainty estimates are computed from NWP forecasts and analyses fields, routinely produced at operational forecast centers. However, they require many forecast cycles, and hence experiments with variations in observations, specified uncertainties, and other factors are expensive and time consuming. In addition, the method is applicable only to the three datasets used in the data assimilation (observations, background, and analysis).
b. Three-cornered hat
With three datasets, one can compute all the terms in (2) and the corresponding equations for Y and Z except the error covariance terms [called error cross covariance by Todling et al. (2022)]; they are assumed to be zero. If the errors of the datasets are correlated, the estimated error statistics will be different from the true statistics. Error correlations may occur in four ways (Rieckh et al. 2021): 1) actual error correlations, as occurs between model and observational datasets when models assimilate the observations; 2) representativeness errors in which two datasets have a similar horizontal or vertical footprint that is different from the third dataset; 3) errors introduced by the collocation process, especially vertical interpolation and filtering; and 4) correlations arising by chance in small sample sizes.
The effect of different footprint sizes of the datasets on the uncertainty estimates can be mitigated by adjusting the footprints of one or more datasets to make them more comparable to the others. For comparison of observations of high vertical resolution and small footprints such as RO or RS with models, increasing the size of the observations’ vertical footprints while reducing their vertical resolution to make them more comparable to the models can be done through vertical filtering (e.g., Kitchen 1989; Lohmann 2007; Gilpin et al. 2018) or thinning and superobbing (e.g., Liu and Rabier 2002).
The 3CH uncertainty estimates [Eq. (2)] are easily calculated once the three datasets are collocated. The collocation process is the most difficult part of the process and must be done carefully to minimize errors introduced by the collocation process. When RO observations are used, the tangent point drift, which can be more than 100 km during an occultation, can be taken into account to determine the varying location of each observation. In the Desroziers method, collocation errors are minimized through the forward modeling to observation space in the NWP model integration to the time and location of the observations (Healy 2016).
For estimating the error variances of an observational dataset, suitable ancillary datasets are short (∼6–18 h) forecasts from either an operational model or a reanalysis. The short forecast is initialized prior to the time of the observations being studied (t = 0), in order that it will not have assimilated the observations at t = 0 and hence their random errors should be nearly uncorrelated. To collocate the model datasets to the observation location, the model analyses can be interpolated in space and time to the observation positions. Experience has shown that the spatial sampling differences in the range 0–500 km are larger than temporal sampling differences in the range of 0–18 h.
c. Relationship of the Desroziers and 3CH methods
Todling et al. (2022) compare the theoretical basis for the two methods and show mathematically that they give the same results for the observational and model background error estimates under certain conditions: 1) the 3CH method uses the background and analysis datasets from the model that assimilates the observations as ancillary datasets and 2) the model data assimilation system is optimal. However, the methods differ greatly in their error estimates of the third dataset, the model analysis, with the 3CH method giving analysis estimates that are a negative of the Desroziers estimate, a result of neglected error covariance terms between the observations and analysis and background and analysis. Table 1 compares the two methods.
Summary of the Desroziers and 3CH methods.
The factors limiting the accuracy of both methods are related to their assumptions. Desroziers assumptions include unbiased observation, background and analysis errors, and uncorrelated observation and background errors as shown in Table 1. The limitations of using Desroziers estimates to refine only observation uncertainties and overlooking estimates of background uncertainties, are discussed by Ménard (2015), Waller et al. (2015), and Bathmann (2018), showing the iteration on observation error variance generally converges, but will converge to an overestimate (underestimate) if the background error variance is underestimated (overestimated).
Rieckh et al. (2021) discuss the assumptions made in the 3CH methods and factors that limit its accuracy, mainly that the 3CH method requires uncorrelated errors in the three datasets. Violations of these assumptions will weaken the validity of both methods. For small sample sizes random correlations may be significant, thus large sample sizes greater than 500 are desirable. Errors introduced by the various steps in both computational processes may introduce errors. These include, for example, truncation errors, errors in the forward model used to convert model data to observation space, collocation errors due to spatial and temporal interpolation in the 3CH method, and difference in footprints in the datasets. For observations of atmospheric properties not predicted explicitly by models, such as radiances or RO bending angles, an accurate forward model to simulate these observations from the model variables is required for both methods.
3. Examples comparing uncertainty estimates of COSMIC-2 radio occultation observations
a. COSMIC-2 bending angle uncertainties for April 2021
We first consider the results for April 2021, a month close to the vernal equinox. We compute the Desroziers and 3CH uncertainty estimates for the entire C2 sample, which includes profiles from 50°S to 50°N. We obtained the RO bending angle data (bfrPrf) from the COSMIC Data Analysis and Archive Center (CDAAC) on a 247-level grid (average spacing about 200 m). For the 3CH estimates, we used the same C2 bending angle data that were used in the Desroziers estimates and data from short-range forecasts from ERA5 (6–18 h) and the NCEP Global Forecast System (GFS) version 15 or 16 (6–12 h). We followed the procedures described in Rieckh et al. (2021). All data were normalized at each level by the sample mean of the ERA5 data. We interpolated the model data to the C2 RO locations, computed bending angles from the model data using a 1D forward model accounting for tangent point drift, and computed the 3CH estimates. We filtered the 3CH results with two passes of a 900-m window Savitsky–Golay (Savitsky and Golay 1964) filter to remove small-scale irregularities in the profiles of the 3CH uncertainty estimates.
We computed the Desroziers estimates from the ERA5 statistics saved in internal ECMWF files. Observation-minus-background and observation-minus-analysis residuals are accumulated for C2 over the two periods 16 August–15 September 2020 and April 2021. The diagnosed uncertainties are globally averaged for each period as a function of impact height.
Figure 2 shows the vertical profiles of the Desroziers (black) and 3CH (red) C2 bending angle error STD (uncertainties), with the corresponding assumed uncertainties in the ERA5 data assimilation system (blue). These uncertainties are all averaged over 1-km layers. The assumed uncertainty profile is also the one used in the current ECMWF operational model (Ruston and Healy 2020). The agreement in the two methods when 3CH uses ERA5 and GFS short-term forecasts is remarkably close, except for the ranges of 2–4-km impact height (corresponding to the lowest 2 km of the atmosphere) and 11–19-km impact height [upper troposphere–lower stratosphere (UTLS)]. The two uncertainty profiles are closer to each other than they are to the assumed uncertainty over most of the vertical range, and much closer than the range of assumed uncertainties in other models shown in Fig. 1.
Estimated COSMIC-2 bending angle random error standard deviations (uncertainties) from the Desroziers (black) and 3CH (red) methods for April 2021. The assumed ECMWF uncertainty model is shown in blue. The standard deviations of the COSMIC-2 bending angles are shown by the green profile. These estimates are for all COSMIC-2 latitudes (50°S–50°N).
Citation: Journal of Atmospheric and Oceanic Technology 39, 7; 10.1175/JTECH-D-21-0175.1
The standard deviations (STD) of the C2 data are also shown in Fig. 2 by the green profile; these are a measure of the atmospheric variability in time and space. The C2 STD are computed on the C2 vertical grid and averaged over the same 1-km layers as the uncertainty estimates. The C2 bending angle uncertainty estimates are closely related to atmospheric variability, larger where the atmosphere is most variable.
The C2 uncertainties show a maximum in the lower troposphere around 4-km impact height (∼2 km above mean sea level height). This level is in the vicinity of the top of the atmospheric boundary layer (ABL) where temperature and water vapor are highly variable. The uncertainties decrease with height as temperature and water vapor variability decrease, and are smallest in the layer from about 10- to 35-km impact height, the so-called RO sweet spot. However, a small relative maximum exists around 18-km impact height (the UTLS), which is related to temperature variability associated with the tropopause. Above 35 km, the two uncertainty estimates increase rapidly due to the increasing effect of ionospheric residuals in the RO bending angle retrievals (Syndergaard 2000; Rieder and Kirchengast 2001). The two estimates are nearly identical, and much less than the assumed ERA5 uncertainties in this region.
If C2 is represented as a signal plus noise with the sample mean of noise equal to zero, then the variance of C2 is equal to the variance of the signal plus the variance of the noise plus 2 times the covariance between them. The variance of the noise is what the Desroziers and 3CH methods are estimating, while the covariance term is not included in either method. At high levels (above 30 km) the C2 signal decreases exponentially, but the C2 noise is relatively constant (Kuo et al. 2004); thus, for negligible covariance between the signal and noise, the C2 variance will converge to the true C2 error variance. This is what we see in Fig. 2, with the C2 STD approaching the estimated C2 uncertainties above 40 km. This convergence is evidence of the accuracy of both methods’ estimates of C2 uncertainty and suggests that the assimilation of RO observations will have a decreasing impact of reducing the background uncertainty above 40 km.
The differences between the two uncertainty estimates near the surface and the UTLS are likely related to two quite different factors. In the lowest 2 km, the Desroziers estimates are closer to the assumed uncertainties than the 3CH estimates. The Desroziers method is somewhat sensitive to the assumed observation and background uncertainties, as discussed by Healy (2016) and N. Semane and S. Healy (2022, unpublished manuscript), and this sensitivity could be part of the reason for the differences seen here. The assumed observation uncertainties are very large for the 2–4-km impact height, larger than the assumed background uncertainties (which vary with atmospheric conditions). This pushes the Desroziers diagnostic toward the innovation uncertainty [first limiting case discussed in Healy (2016)], which results in background uncertainties being attributed as observation uncertainties.
In the 11–19-km impact height range, the reason for the larger 3CH estimates is likely different. In this layer the 3CH estimates may be overestimated because of the smaller C2 footprints compared to the ERA5 and GFS datasets in this region of high variability. The C2 vertical footprints are ∼200 m, while the vertical footprints of the models are considerably larger (ERA5 ∼ 700 m; GFS ∼ 2 km). Different footprints of the DS in the 3CH method will create the largest effect in layers where the atmospheric variability is greatest (Rieckh et al. 2021), which is the case between 10 and 20 km because of the variable tropopause heights and complex tropopause structures. This reasoning is supported by the following calculation in which the two ancillary DS in the 3CH method are chosen to be equal to the background and analysis DS used in the ERA5 assimilation.
To provide further insight into the Desroziers and 3CH methods, we computed additional 3CH estimates of the error variances of C2 using other ancillary datasets, including the two that are used in Desroziers method, the model background and analysis data. We also computed the Desroziers estimates of the background and analysis errors. The results (Fig. 3) show that the 3CH and Desroziers estimates of the C2 and background errors are nearly identical when the 3CH method uses the background and analysis as ancillary datasets in the 3CH triplet (overlap of solid and dashed black and solid and dashed green profiles). These results confirm the theoretical results of Todling et al. (2022). The 3CH estimate of ERA5-Analysis using (C2, ERA5-Background) are the negative of the Desroziers estimates because the neglected error covariance terms E(εAεB) and E(εC2εA) add rather than cancel, where εA, εB, and εC2 are the ERA5-Analysis, ERA5-Background, and C2 errors, respectively.
Profiles of different error STD estimates of C2 and ancillary datasets. These include the Desroziers estimates of C2 (dashed black), ERA5-Analysis (dashed blue), and ERA5-Background (dashed green). Also shown are four 3CH estimates of C2 using different ancillary datasets: GFS, MERRA-2 (magenta), GFS, ERA5-Background (red), ERA5-Background, MERRA-2 (turquoise), and ERA5-Analysis, ERA-Background (solid black). The Desroziers and 3CH estimates of C2 (dashed and solid black) and the ERA5-Background profiles (dashed and solid green) are nearly identical showing the equivalence of the estimates of the Desroziers and 3CH methods for these two datasets under certain conditions.
Citation: Journal of Atmospheric and Oceanic Technology 39, 7; 10.1175/JTECH-D-21-0175.1
b. Variation of COSMIC-2 uncertainties with latitude
We next investigated how the Desroziers and 3CH uncertainty estimates varied with latitude for April 2021. Because the uncertainty estimates are so close, we first show the overall variation with latitude of the 3CH estimates by 5° latitude bands (Fig. 4), and then compare the Desroziers and 3CH estimates for broader (15°) latitude bands. In the troposphere between 6 and 10 km the largest 3CH uncertainty estimates (Fig. 4) are in the tropics (20°S–20°N). However, near the top of the ABL between 3- and 4-km impact height, the variation with latitude is more complex, with the largest uncertainty estimates occurring between 20° and 25°S.
3CH uncertainty estimates for COSMIC-2 bending angles by 5° latitude bands for April 2021. (left) The entire range from 0 to 60-km impact height; (right) an enlargement of the 0–10-km impact height range.
Citation: Journal of Atmospheric and Oceanic Technology 39, 7; 10.1175/JTECH-D-21-0175.1
In the stratosphere and lower mesosphere (40–60 km) the order is reversed, with the lower latitudes tending to have smaller uncertainties compared to those of the higher latitudes.
The latitudinal variation of the Desroziers and 3CH uncertainty estimates is shown in Fig. 5 by the differences of the estimates over 15° latitudinal bands from the overall sample mean (all latitudes). The 3CH and Desroziers uncertainty profiles show essentially the same relationship to each other in each latitude band as they do in the overall dataset covering all latitudes (Fig. 2). Agreement is close at all altitudes except the lowest 2 km of impact height and the 11–19-km layer. The largest latitudinal variations occur in the 10–20-km layer, where the tropopause height varies with latitude, and in the troposphere, where the higher latitude band (30°–45°N/S) shows smaller uncertainties (2%–3%) and the subtropical latitude band (15°–30°N/S) shows larger uncertainties (∼1.5%) than the overall mean. The differences in the two estimates are generally less than the differences in the latitudinal band variations.
Latitudinal differences of Desroziers (solid) and 3CH (dashed) estimated COSMIC-2 bending angle uncertainties from the overall sample mean (all latitudes, 50°S–50°N) for April 2021. Also shown are differences of COSMIC-2 STD (dotted profiles): (top left) 30°–45°S and (top right) 30°–45°N, (middle left) 15°–30°S and (middle right) 15°–30°N, and (bottom left) 0°–15°S and (bottom right) 0°–15°N.
Citation: Journal of Atmospheric and Oceanic Technology 39, 7; 10.1175/JTECH-D-21-0175.1
c. Tropical cyclone season uncertainty estimates (16 August–15 September 2020)
To see how the uncertainty estimates from the two methods vary by season, we compared the two methods for a 31-day period at the peak of the Northern Hemisphere tropical cyclone season, 16 August–15 September 2020. As in April 2021, the Desroziers and 3CH uncertainty estimates are quite close, except in the lowest 2 km and in the 11–19-km range (figure not shown). The differences in both estimates between the August–September and April periods are also similar, as shown in Fig. 6. The sign of the seasonal differences in the two estimates are highly correlated and the differences between the profiles is generally less than a tenth of a percent.
Differences (August–September 2020 minus April 2021) in uncertainty estimates by 15° latitude bands for the Desroziers (solid profiles) and 3CH (dashed profiles). The corresponding differences in bending angle standard deviations are also shown (dotted profiles): (top left) 30°–45°S and (top right) 30°–45°N, (middle left) 15°–30°S and (middle right) 15°–30°N, and (bottom left) 0°–15°S and (bottom right) 0°–15°N.
Citation: Journal of Atmospheric and Oceanic Technology 39, 7; 10.1175/JTECH-D-21-0175.1
In the troposphere below 10 km, both estimates are larger by several percent in the Northern Hemisphere (NH) between 30° and 45°N and smaller by a similar amount in the Southern Hemisphere (SH) at these latitudes (Fig. 6, top row). At other altitudes in this latitude band the differences are smaller, less than 1%. The Desroziers and 3CH methods show the largest differences between 2 and 7 km in the 30°–45°N band. This is related to the sensitivity of the two methods to datasets with fairly noisy and steeply sloped profiles. Small changes in the slopes of the difference profiles can make a large difference in the values at any given level.
In the subtropical latitudes (Fig. 6, middle row) the differences between the two periods are small (generally less than 0.1%) above 10 km in both hemispheres. Below 10 km, the differences in the SH are negative by as much as 1%, similar to the differences in the 30°–45° band. In the NH subtropics, the differences are positive by up to 1% above 6 km, and negative by up to 1% below 6 km.
In the deep tropics (0°–15°), the differences are small (less than 0.5%) above 6 km (Fig. 6, bottom row). Below 6 km, the changes are asymmetric with respect to hemisphere, with the SH showing an increase and the NH a decrease.
As noted above, both the Desroziers and 3CH uncertainty estimates show larger values where the atmosphere is most variable. This relationship between COSMIC-2 uncertainty and atmospheric variability is illustrated in Fig. 6 by showing the seasonal variations in standard deviation of the bending angles. The variations in the two uncertainty estimates are closely related to the variations in standard deviations of the bending angles themselves.
4. Summary
We found that Desroziers and 3CH methods of estimating the random error statistics (uncertainties) of observations gives very similar results for COSMIC-2 (C2) radio occultation bending angles for all latitudes and for two different seasons. When the 3CH method uses the same two datasets that are used in Desroziers method (the model background and analysis), the results give identical results, in agreement with theory (Todling et al. 2022).
The Desroziers and 3CH uncertainty estimates are considerably different from the assumed RO error models used by several NWP centers, suggesting that adjustment of these error models to more closely follow the estimated error statistics may improve the impact of RO observations on model analyses and forecasts. The similarity of results of the more general 3CH method to the Desroziers method indicates that the 3CH method is useful for error estimation studies that do not require sophisticated model data assimilation systems, as well as to guide modelers in their choice of observation error model.
The diagnosed RO uncertainties in both methods vary in similar ways with latitude. In the troposphere, uncertainties are higher in the tropics than subtropics and middle latitudes, and estimated error STD can vary with latitude from as low as 5% to as high as 14% in the lower troposphere. In the upper stratosphere–lower mesosphere, we find the reverse, with typical uncertainties slightly less (e.g., 8.5% vs 9.0% at 48 km) than those of the subtropics and higher latitudes.
Both methods showed similar variations between a month during the Northern Hemisphere tropical cyclone season (August–September 2020) compared to a month near the vernal equinox (April 2021). In the troposphere the August–September uncertainties are 1%–2% higher between latitudes 30° and 45°N, and slightly lower or nearly the same at other latitudes. In the stratosphere (30–48 km) the differences between the two periods are less than 0.5%.
Finally, the diagnosed uncertainties of C2 bending angles are related to the atmospheric variability, with high uncertainties associated with high variability and vice versa. The convergence of the error estimates and the standard deviations above 40 km indicates a lessening impact of assimilating C2 in reducing the model uncertainties above this level.
Acknowledgments.
We thank Ricardo Todling (NASA GMAO) for his careful review and analysis showing the close relationship between Desroziers and the 3CH methods and the discussions that followed, which clarified this relationship. We also thank Joe Nielsen (Danish Meteorological Institute) and Neill Bowler (Met Office) for their helpful discussions. Anthes and Sjoberg are supported by NSF Grant AGS-2054356, NASA Grant C22K0658, and NOAA Contracts 16CN0070 and R4310483. Semane is supported by the Radio Occultation Meteorology Satellite Application Facility (ROM SAF), which is a decentralized processing center under the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT).
Data availability statement.
COSMIC-2 data were obtained from CDAAC (https://cdaac-www.cosmic.ucar.edu). ERA5 and GFS data were obtained from the NCAR Research Data Archive (https://rda.ucar.edu/).
REFERENCES
Anthes, R. A., 2011: Exploring Earth’s atmosphere with radio occultation: Contributions to weather, climate and space weather. Atmos. Meas. Tech., 4, 1077–1103, https://doi.org/10.5194/amt-4-1077-2011.
Anthes, R. A., and T. Rieckh, 2018: Estimating observation and model error variances using multiple data sets. Atmos. Meas. Tech., 11, 4239–4260, https://doi.org/10.5194/amt-11-4239-2018.
Anthes, R. A., and Coauthors, 2008: The COSMIC/FORMOSAT-3 mission: Early results. Bull. Amer. Meteor. Soc., 89, 313–333, https://doi.org/10.1175/BAMS-89-3-313.
Aparicio, J. M., and G. Deblonde, 2008: Impact of the assimilation of CHAMP refractivity profiles on Environment Canada global forecasts. Mon. Wea. Rev., 136, 257–275, https://doi.org/10.1175/2007MWR1951.1.
Banos, I. H., L. F. Sapucci, L. Cucurull, C. F. Bastarz, and B. B. Silveira, 2019: Assimilation of GPSRO bending angle profiles into the Brazilian global atmospheric model. Remote Sens., 11, 256, https://doi.org/10.3390/rs11030256.
Bathmann, K., 2018: Justification for estimating observation-error covariances with the Desroziers diagnostic. Quart. J. Roy. Meteor. Soc., 144, 1965–1974, https://doi.org/10.1002/qj.3395.
Bonavita, M., 2014: On some aspects of the impact of GPSRO observations in global numerical weather prediction. Quart. J. Roy. Meteor. Soc., 140, 2546–2562, https://doi.org/10.1002/qj.2320.
Bormann, N., A. Geer, and P. Bauer, 2011: Estimates of observation error characteristics in clear and cloudy regions for microwave imager radiances from numerical weather prediction. Quart. J. Roy. Meteor. Soc., 137, 2014–2023, https://doi.org/10.1002/qj.833.
Bormann, N., M. Bonavita, R. Dragani, R. Eresmaa, M. Matricardi, and A. McNally, 2016: Enhancing the impact of IASI observations through an updated observation‐error covariance matrix. Quart. J. Roy. Meteor. Soc., 142, 1767–1780, https://doi.org/10.1002/qj.2774.
Bowler, N. E., 2020: Revised GNSS-RO observation uncertainties in the Met Office NWP system. Quart. J. Roy. Meteor. Soc., 146, 2274–2296, https://doi.org/10.1002/qj.3791.
Campbell, W. F., E. A. Satterfield, B. Ruston, and N. L. Baker, 2017: Accounting for correlated observation error in a dual-formulation 4D variational data assimilation system. Mon. Wea. Rev., 145, 1019–1032, https://doi.org/10.1175/MWR-D-16-0240.1.
Cordoba, M., S. L. Dance, G. A. Kelly, N. K. Nichols, and J. A. Waller, 2017: Diagnosing atmospheric motion vector observation errors for an operational high-resolution data assimilation system. Quart. J. Roy. Meteor. Soc., 143, 333–341, https://doi.org/10.1002/qj.2925.
Cucurull, L., J. C. Derber, R. Treadon, and R. J. Purser, 2007: Assimilation of global positioning system radio occultation observations into NCEP’s Global Data Assimilation System. Mon. Wea. Rev., 135, 3174–3193, https://doi.org/10.1175/MWR3461.1.
Desroziers, G., L. Berre, B. Chapnik, and P. Poli, 2005: Diagnosis of observation, background and analysis-error statistics in observation space. Quart. J. Roy. Meteor. Soc., 131, 3385–3396, https://doi.org/10.1256/qj.05.108.
El Aabaribaoune, M., E. Emili, and V. Guidard, 2021: Estimation of the error covariance matrix for IASI radiances and its impact on the assimilation of ozone in a chemistry model. Atmos. Meas. Tech., 14, 2841–2856, https://doi.org/10.5194/amt-14-2841-2021.
Fletcher, S. J., 2017: Data Assimilation for the Geosciences. Elsevier, 941 pp.
Gilpin, S., T. Rieckh, and R. A. Anthes, 2018: Reducing representativeness and sampling errors in radio occultation–radiosonde comparisons. Atmos. Meas. Tech., 11, 2567–2582, https://doi.org/10.5194/amt-11-2567-2018.
Gray, J. E., and D. W. Allan, 1974: A method for estimating the frequency stability of an individual oscillator. 28th Annual Symp. on Frequency Control, Atlantic City, NJ, IEEE, 243–246, https://doi.org/10.1109/FREQ.1974.200027.
Healy, S. B., 2008: Forecast impact experiment with a constellation of GPS radio occultation receivers. Atmos. Sci. Lett., 9, 111–118, https://doi.org/10.1002/asl.169.
Healy, S. B., 2016: Estimates of GNSS radio occultation bending angle and refractivity error statistics. ROM SAF Rep., 26 pp., https://www.romsaf.org/general-documents/rsr/rsr_26.pdf.
Healy, S. B., and J. N. Thépaut, 2006: Forecast impact experiment with GPS radio occultation measurements. Quart. J. Roy. Meteor. Soc., 132, 605–623, https://doi.org/10.1256/qj.04.182.
Hollingsworth, A., and P. Lönnberg, 1986: The statistical structure of short-range forecast errors as determined from radiosonde data 1. The wind field. Tellus, 38A, 111–136, https://doi.org/10.3402/tellusa.v38i2.11707.
Kitchen, M., 1989: Representativeness errors for radiosonde observations. Quart. J. Roy. Meteor. Soc., 115, 673–700, https://doi.org/10.1002/qj.49711548713.
Kuo, Y.-H., T.-K. Wee, S. Sokolovskiy, C. Rocken, W. Schreiner, D. Hunt, and R. A. Anthes, 2004: Inversion and error estimation of GPS radio occultation data. J. Meteor. Soc. Japan, 82, 507–531, https://doi.org/10.2151/jmsj.2004.507.
Kursinski, E. R., G. A. Hajj, K. R. Hardy, J. T. Schofield, and R. Linfield, 1997: Observing Earth’s atmosphere with radio occultation measurements. J. Geophys. Res., 102, 23 429–23 465, https://doi.org/10.1029/97JD01569.
Liu, Z.-Q., and F. Rabier, 2002: The interaction between model resolution, observation resolution and observation density in data assimilation: A one-dimensional study. Quart. J. Roy. Meteor. Soc., 128, 1367–1386, https://doi.org/10.1256/003590002320373337.
Lohmann, M., 2007: Analysis of global positioning system (GPS) radio occultation measurement errors based on Satellite de Aplicaciones Cientificas-C (SAC-C) GPS radio occultation data recorded in open-loop and phase-locked-loop mode. J. Geophys. Res., 112, D09115, https://doi.org/10.1029/2006JD007764.
Ménard, R., 2015: Error covariance estimation methods based on analysis residuals: Theoretical foundation and convergence properties derived from simplified observation networks. Quart. J. Roy. Meteor. Soc., 142, 257–273, https://doi.org/10.1002/qj.2650.
O’Carroll, A. G., J. R. Eyre, and R. W. Saunders, 2008: Three-way error analysis between AATSR, AMSR-E, and in situ sea surface temperature observations. J. Atmos. Oceanic Technol., 25, 1197–1207, https://doi.org/10.1175/2007JTECHO542.1.
Poli, P., P. Moll, D. Puech, F. Rabier, and S. B. Healy, 2009: Quality control, error analysis, and impact assessment of FORMOSAT-3/COSMIC in numerical weather prediction. Terr. Atmos. Ocean. Sci., 20, 101–113, https://doi.org/10.3319/TAO.2008.01.21.02(F3C).
Poli, P., S. B. Healy, and D. P. Dee, 2010: Assimilation of global positioning system radio occultation data in the ECMWF ERA-Interim reanalysis. Quart. J. Roy. Meteor. Soc., 136, 1972–1990, https://doi.org/10.1002/qj.722.
Rennie, M. P., 2010: The impact of GPS radio occultation assimilation at the Met Office. Quart. J. Roy. Meteor. Soc., 136, 116–131, https://doi.org/10.1002/qj.521.
Rieckh, T., J. Sjoberg, and R. A. Anthes, 2021: The three-cornered hat method for estimating error variances of three or more atmospheric datasets. Part II: Evaluating radio occultation and radiosonde observations, global model forecasts, and reanalyses. J. Atmos. Oceanic Technol., 38, 1777–1796, https://doi.org/10.1175/JTECH-D-20-0209.1.
Rieder, M. J., and G. Kirchengast, 2001: Error analysis and characterization of atmospheric profiles retrieved from GNSS occultation data. J. Geophys. Res., 106, 31 755–31 770, https://doi.org/10.1029/2000JD000052.
Ruston, B., and S. Healy, 2020: Forecast impact of FORMOSAT-7/COSMIC-2 GNSS radio occultation measurements. Atmos. Sci. Lett., 22, e1019, https://doi.org/10.1002/asl.1019.
Savitzky, A., and M. Golay, 1964: Smoothing and differentiation of data by simplified least squares procedure. Anal. Chem., 36, 1627–1639, https://doi.org/10.1021/ac60214a047.
Schreiner, W. S., and Coauthors, 2020: COSMIC-2 radio occultation-first results. Geophys. Res. Lett, 47, e2019GL086841, https://doi.org/10.1029/2019GL086841.
Simonin, D., J. A. Waller, S. P. Ballard, S. L. Dance, and N. K. Nichols, 2019: A pragmatic strategy for implementing spatially correlated observation errors in an operational system: An application to Doppler radial winds. Quart. J. Roy. Meteor. Soc., 145, 2772–2790, https://doi.org/10.1002/qj.3592.
Sjoberg, J. P., R. A. Anthes, and T. Rieckh, 2021: The three-cornered hat method for estimating error variances of three or more atmospheric datasets. Part I: Overview and evaluation. J. Atmos. Oceanic Technol., 38, 555–572, https://doi.org/10.1175/JTECH-D-19-0217.1.
Stewart, L. M., S. L. Dance, N. K. Nichols, J. R. Eyre, and J. Cameron, 2013: Estimating interchannel observation-error correlations for IASI radiance data in the Met Office system. Quart. J. Roy. Meteor. Soc., 140, 1236–1244, https://doi.org/10.1002/qj.2211.
Stoffelen, A., 1998: Toward the true near-surface wind speed: Error modeling and calibration using triple collocation. J. Geophys. Res., 103, 7755–7766, https://doi.org/10.1029/97JC03180.
Syndergaard, S., 2000: On the ionosphere calibration in GPS radio occultation measurements. Radio Sci., 35, 865–883, https://doi.org/10.1029/1999RS002199.
Todling, R., N. Semane, S. Healy and R. Anthes, 2022: The relationship between Desroziers and three-cornered hat methods. Quart. J. Roy. Meteor. Soc., in press.
Waller, J. A., S. L. Dance, and N. K. Nichols, 2015: Theoretical insight into diagnosing observation error correlations using observation‐minus‐background and observation‐minus‐analysis statistics. Quart. J. Roy. Meteor. Soc., 142, 418–431, https://doi.org/10.1002/qj.2661.
Weston, P. P., W. Bell, and J. R. Eyre, 2014: Accounting for correlated error in the assimilation of high-resolution sounder data. Quart. J. Roy. Meteor. Soc., 140, 2420–2429, https://doi.org/10.1002/qj.2306.
