The mainstay of the global radio occultation (RO) system, the COSMIC constellation of six satellites launched in April 2006, is already past the end of its nominal lifetime and the number of soundings is rapidly declining because the constellation is degrading. For about the last decade, COSMIC profiles have been collected and their retrievals assimilated in numerical weather prediction systems to improve operational weather forecasts. The success of RO in increasing forecast skill and COSMIC’s aging constellation have motivated planning for the COSMIC-2 mission, a 12-satellite constellation to be deployed in two launches. The first six satellites (COSMIC-2A) are expected to be deployed in December 2017 in a low-inclination orbit for dense equatorial coverage, while the second six (COSMIC-2B) are expected to be launched later in a high-inclination orbit for global coverage. To evaluate the potential benefits from COSMIC-2, an earlier version of the NCEP’s operational forecast model and data assimilation system is used to conduct a series of observing system simulation experiments with simulated soundings from the COSMIC-2 mission. In agreement with earlier studies using real RO observations, the benefits from assimilating COSMIC-2 observations are found to be most significant in the Southern Hemisphere. No or very little gain in forecast skill is found by adding COSMIC-2A to COSMIC-2B, making the launch of COSMIC-2B more important for terrestrial global weather forecasting than that of COSMIC-2A. Furthermore, results suggest that further improvement in forecast skill might better be obtained with the addition of more RO observations with global coverage and other types of observations.
To quantitatively evaluate the benefits of new observations in our understanding and prediction of Earth’s atmosphere, both observing system experiments (OSEs) and observing system simulation experiments (OSSEs) are necessary. OSEs are data-denial studies that allow the evaluation of existing data but cannot be used to analyze the impact of future observing systems. Atmospheric OSSEs are modeling experiments used to perform an objective evaluation of the potential benefits of proposed observing systems in weather forecasting. These experiments explore the value of enhancing the current global observing system with additional observations that do not yet exist. As stated in Hoffman and Atlas (2016), OSSEs provide a rigorous, cost-effective approach to evaluating the potential impact of new observing systems and alternate configurations and deployments of existing systems, and to optimizing observing strategies. They are also used to prepare for the assimilation of new types of data, and to optimize the assimilation of existing data.
The methodology currently employed for OSSEs using global data assimilation systems was redesigned in the early 1980s (Atlas 1997; Atlas and Pagano 2014) in order to increase their realism, and this methodology has yielded accurate estimates of the potential impact of several space-based instruments years before their launch (Atlas et al. 2001). For OSSEs to produce accurate quantitative results, all of the components of the OSSE system must be realistic. This includes the following: 1) a long atmospheric model integration using a “state of the art” numerical model to provide a complete record of the assumed true state of the atmosphere, usually referred to as the nature run, where the nature run would represent the main characteristics of the real atmosphere (e.g., its climatology, general patterns of storm tracks, etc.); 2) observations simulated from the nature run for existing conventional and satellite platforms, where these synthetic observations need to be simulated with the same spatial and temporal coverage, resolution, and accuracy as the real-world observations have; 3) simulation of new, proposed observations from the nature run with realistic coverage and accuracy; 4) an assimilation system that includes a numerical model for the generation of first-guess forecasts, where the synthetic observations from both the current and proposed instruments are assimilated and evaluated; this assimilation system should be of lower (or at least of similar) resolution than the nature run, where realistic differences between the nature run and the model used for assimilation and forecasting should exist and, ideally, where they should approximate the differences between a state-of-the-art assimilation system and the real atmosphere; and 5) validation of the entire OSSE system to ensure that the accuracy of the analyses and forecasts, and the impact of existing observing systems in the OSSE environment, are comparable to the accuracies and impacts of the same observing systems in the real world.
Over time, OSSE systems have become more realistic and robust. In global forecasting, OSSEs are now performed with operational data assimilation systems (e.g., Boukabara et al. 2016) and hurricane OSSEs investigate observing techniques to improve hurricane track and intensity (e.g., Atlas et al. 2015). An excellent review of the OSSE methodology and its current status and anticipated progress is provided by Hoffman and Atlas (2016). Historically, most OSSEs have addressed numerical weather prediction questions. However, there are emerging needs for new OSSE capabilities (e.g., ocean, renewable energy, climate) that will also require the development of appropriate verification metrics. For example, Halliwell et al. (2014, 2015) are now conducting OSSEs for the ocean.
The assessment of the potential impact of observations from current and future instruments is not without challenges (English et al. 2013). Modern data assimilation techniques and the large volume of observations currently being used make the task of showing the impact of a new observing system on forecast accuracy very challenging. In addition, quantitative impact studies often show small changes in measures of forecast accuracy, and these changes can differ from one study to another and on the metric selected for the verification.
Global Navigation Satellite Systems (GNSSs) radio occultation (RO) observations are among the observing systems making the largest contributions to efforts to improve global forecast skill (e.g., Healy and Thépaut 2006; Cucurull and Derber 2008; Aparicio and Deblonde 2008; Rennie 2010; Anlauf et al. 2011; Cucurull et al. 2013). During a radio occultation event, the rays connecting the transmitter (a GNSS satellite) and a low-Earth-orbit (LEO) receiving satellite scan the atmosphere quasi vertically, providing information on the thermodynamic state of the atmosphere. While scanning the atmosphere, rays are refracted as a result of the different densities of the atmospheric layers. The total amount of bending accumulated during the ray path trajectory can be derived from Doppler-shift measurements and from the knowledge of the exact positions and velocities of the LEO and GNSS satellites, along with the assumption of spherical symmetry around the ray path tangent point (e.g., Melbourne et al. 1994; Kursinski et al. 1997; Rocken et al. 1997). Profiles of refractivity are then derived from profiles of the atmospheric bending angle under global spherical symmetry of Earth’s atmosphere and with the use of some auxiliary climatology information.
Soundings of the refractivity or bending angle have been used worldwide in operational global data assimilation since the launch of the Constellation Observing System for Meteorology, Ionosphere and Climate (COSMIC) in 2006 (Rocken et al. 2000; Anthes et al. 2008). However, COSMIC is already past the end of its nominal lifetime and the number of profiles is rapidly declining, which has motivated the COSMIC-2 mission, a 12-satellite constellation, to be deployed in two launches. The first six satellites are planned to be deployed in December 2017 in a low-inclination orbit for dense equatorial coverage. The second six will be deployed later in a high-inclination orbit and will provide global coverage. However, funding for the second launch of COSMIC-2 is not yet guaranteed. Recent studies have shown the negative effects on global weather forecasting of a possible gap in RO data (e.g., Cucurull and Anthes 2015).
Harnisch et al. (2013) used an ensemble of data assimilations approach to show that no saturation in forecast skill exists for RO soundings even with 128 000 profiles per day. However, and to the best knowledge of these authors, no global OSSEs have yet been performed with RO observations. To evaluate the potential impact of the COSMIC-2 observations, we have conducted a series of preliminary OSSEs. First, a simplified OSSE configuration is used to analyze the information content associated with the RO observations from the COSMIC-2 mission in global weather forecasting. To quantify the impact of RO data coverage and data density, both the equatorial (COSMIC-2A, first launch) and polar (COSMIC-2B, second launch) components of the COSMIC-2 mission are considered. Furthermore, the sensitivity of our results to the number of RO soundings will be addressed.
This paper is structured as follows. A description of the OSSE system used in this study is provided in section 2. Results from two different experimental configurations, a baseline with roughly no satellite or conventional data and a more realistic series of experiments, which include the current observing system, are provided in sections 3 and 4, respectively. The sensitivity of the results to the number of COSMIC-2 soundings is analyzed in section 5. Finally, section 6 summarizes our main findings.
2. OSSE system setup
The nature run used in the OSSE system was generated by the European Centre for Medium-Range Weather Forecasts (ECMWF), using their operational forecast model version c31r1, and runs from 1 May 2005 to 31 May 2006 at T511 (~39-km horizontal resolution) and with 91 vertical levels (Masutani et al. 2006; Andersson and Matsutani 2010). From this 1-yr period, we selected the time frame of 1 July to 30 August 2005 to conduct our study.
The data assimilation system used in all the experiments was the 2012-year version of the National Centers for Environmental Prediction’s (NCEP) operational suite. The Global Forecast System (GFS) model ran at a lower horizontal resolution of T382 (~52 km) as a result of the lower resolution of the nature run, but with the same number of vertical levels (64) as in the operational configuration. All the experiments used the nonhybrid version of NCEP’s Gridpoint Statistical Interpolation analysis system (GSI) at T382. This choice was made so we could run a large variety of experiments relatively quickly, and without making use of a significant amount of computer resources. Although the results of this study cannot be directly extrapolated to the results that we would obtain with a more modern data assimilation system (e.g., with an ensemble-based background error covariance matrix), this simplified OSSE setup enables a detailed and quantitative investigation of how the characteristics of the RO measurements impact weather forecasts.
We conducted two sets of experiments. The first set investigates the benefits of adding COSMIC-2 soundings to a poor baseline experiment configuration that assimilates surface pressure observations only. These experiments provide insight into the information content in the COSMIC-2 observations, isolated from most other observing systems.
The second set of experiments uses a control experiment that includes conventional and satellite observations (see Table 1), and represents a simplified version of an operational data assimilation system. Here, we investigate the added value of COSMIC-2 beyond the conventional and satellite data routinely assimilated at most operational weather centers. Both sets of experiments consider the impact of the COSMIC-2A (dense equatorial coverage) and COSMIC-2B (global coverage) components of COSMIC-2, independently and combined. In addition, by controlling the time window for accepting RO data into the analysis, we examine the sensitivity of the results to the number of RO soundings assimilated in both sets of experiments. To quantify the impacts of COSMIC-2 alone, none of the experiments that assimilate COSMIC-2 include other RO observations. A summary of the different experiments is listed in Table 2.
All the observations used in the experiments (conventional, satellite, and COSMIC-2) were simulated from the nature run for July–August 2005 by the NOAA OSSE team, using geographical locations from July to August 2012 for the conventional and satellite data to enable simulation of satellite sensors launched after 2005. Realistic orbits were used to simulate COSMIC-2 soundings of refractivity with the U.S. global positioning system (GPS) and the Russian Global Navigation Satellite System (GLONASS) constellations, and using the forward operator described in Cucurull (2010). No systematic or random errors were added to the observations, except for satellite radiances. The use of perfect observations complicates the application of the results to reality, but allows for a more straightforward interpretation of the results, which otherwise would be a mix of impacts from observations and their error characteristics. Since satellite radiances are bias corrected in modern operational data assimilation systems, and RO plays a key role in the bias-correction schemes by preventing the model from drifting toward its own biased climate (Dee 2005; Healy and Thépaut 2006; Poli et al. 2010; Bauer et al. 2014; Cucurull et al. 2014; Bonavita 2014), biases obtained from an experiment that used the 2012-year NCEP operational configuration with real observations at a lower horizontal resolution of T382 were added to the perfect satellite radiances. We should emphasize that the magnitude of these biases was very small, and running a control experiment with perfect and bias-added radiances produced very similar levels of forecast skill. Seven-day (168 h) experimental forecasts begin at 0000 UTC from 20 July to 30 August 2005. The first 19 days of the experiments from 1 to 19 July are used for model spinup.
3. Information content of COSMIC-2 soundings
We first conducted a series of experiments to investigate the thermodynamic information associated with the COSMIC-2 soundings. For this purpose, a poor baseline (BASE) experiment that assimilated only surface pressure observations with a 6-h assimilation time window (i.e., ±3 h from the analysis time) was run. Soundings of refractivity from COSMIC-2A were assimilated in BC2EQ (“EQ” for Equatorial launch), and soundings of refractivity from COSMIC-2B were assimilated in BC2PO (“PO” for polar launch). Finally, all COSMIC-2 profiles (i.e., COSMIC-2A and COSMIC-2B combined) were assimilated in BC2. In all three cases, the assimilation of RO observations used a 6-h assimilation time window (±3 h from the analysis time).
The sensitivity of our results to the number of RO observations was investigated by repeating the experiments with a shorter assimilation time window, from 6 to 1 h (i.e., ±30 min from the analysis time). COSMIC-2A was assimilated in BC2EQ_30, COSMIC-2B in BC2PO_30, and COSMIC-2 in BC2_30. (The assimilation time window for the surface pressure observations was kept fixed at 6 h in all experiments.) To compare the forecast skill of our experiments against the skill of a “perfect” run, we conducted an additional run that assimilated perfect observations of temperature, moisture, and wind at multiple pressure levels. These observations were simulated from the nature run at every grid point and ingested into the data assimilation system. This reference run experiment, called NR, represents the best possible forecast skill in the OSSE system and provides a benchmark for the maximum improvement that can be achieved by any single experiment.
The spatial distributions of the COSMIC-2 observations for 6- and 1-h assimilation time windows are shown in Figs. 1a and 1d, respectively. From the figures, it is evident that COSMIC-2A (~6000 profiles per day for a 6-h assimilation time window) provides denser coverage in the tropical latitudes, although the geographical location of the observations extends to 45°N and 45°S. On the other hand, COSMIC-2B (~6000 profiles per day for a 6-h assimilation time window) provides the global spatial distribution of the observations, extending beyond the midextratropics. Results of the experiments are evaluated in terms of the mass, humidity, and wind fields.
a. Mass and moisture fields
Figures 2a and 2b show the anomaly correlation (AC) skill for the 500-mb (1 mb = 1 hPa) geopotential heights using a 6-h assimilation time window (i.e., using the full amount of RO data) for the northern extratropics (NH; 20°–80°N) and southern extratropics (SH; 20°–80°S), respectively. In the NH, the assimilation of COSMIC-2 observations in BC2EQ increases the skill as compared to the BASE experiment (difference of 0.16 in AC at day 5), but the largest improvement comes from the assimilation of observations in BC2PO (difference of 0.68 in AC at day 5). Note that adding observations from the full COSMIC-2 mission in BC2 does not result in an improvement in skill as compared to BC2PO. This seems to indicate that, for this model horizontal resolution and assimilation scheme, and for this metric, saturation with COSMIC-2 data might already occur with the number and distribution of observations available in BC2PO and, by increasing the number of observations in the tropical latitudes, the skill of the model does not improve. In all cases, there is still room for improvement, as the NR, which also assimilated wind information, shows larger skill than any other of the experiments (difference of 0.82 in AC with respect to BASE at day 5). It is remarkable that a significant percentage of the skill in the NR in the extratropics is already achieved with the assimilation of COSMIC-2B and surface pressure observations in BC2PO (85.4% at day 5).
In general, and for all of the experiments, the AC skill is found to be slightly larger in the SH (winter; Fig. 2b) than in the NH (summer; Fig. 2a). Experiment BC2EQ shows the largest difference (SH > NH by ~0.1 in AC for all the lead times). The only exception is BASE, which shows larger AC skill in the NH than in the SH. The reason for this is that the density of the surface observations assimilated in BASE is much higher in the NH than in the SH, resulting in the slightly better performance of BASE in the NH. However, since the difference between any of the experiments and BASE is larger in the SH, and BASE performs worse in the SH, the benefits from assimilating RO are much greater in the SH than in the NH. This result is in agreement with earlier OSE studies with existing RO profiles that used modern data assimilation configurations and included all the observations available for operational weather forecasting (e.g., Cucurull 2010). Note that NR also shows slightly higher AC skill in the SH than in NH, indicating that predictability might be higher in the SH winter than in the NH summer. As it is found in the NH, BC2PO and BC2 show very similar AC skill levels, and the assimilation of RO observations alone in BC2PO accounts for 86.1% of the skill of the NR in the SH at day 5. Different from the results found in the NH, the denser tropical RO data in BC2EQ account for a larger amount of the skill of BC2 (49.5% at day 5), while this percentage is lower in the NH (34.1% at day 5). The room for improvement (i.e., difference between BC2 and NR) is similar in both latitudinal ranges (~0.13 in AC at day 5). All the differences between the experiments and BASE are statistically significant at the 95% confidence level for both latitudinal ranges.
The impact of reducing the number of RO soundings assimilated is addressed in Figs. 2c and 2d for the NH and SH, respectively. Differences in the AC skill of geopotential heights at 500 mb between the NH and SH are slightly larger than when a wider data acceptance time window was used, with the differences now extending to all forecast lead times. Although the difference between BC2PO_30 and BC2_30 is still insignificant in the SH, the AC skill in the NH slightly benefits from the assimilation of additional tropical RO observations (by ~0.04 in AC at day 5). This seems to indicate that the number of RO observations in the tropical latitudes in BC2PO_30 is high enough to saturate the forecast skill in terms of AC in the SH, but not in the NH, which benefits from additional tropical observations. This might be caused by the lower predictability seen in the NR in the NH and, thus, the need for a larger number of observations than in the SH in order to obtain similar weather forecast skill. A significant percentage of the skill of the NR is already achieved with the assimilation of RO observations alone in BC2PO_30 (61.8% at day 5 in NH, and 81.4% at day 5 in SH). All the differences between the experiments and BASE are statistically significant at the 95% confidence level.
The AC skill for all the experiments that assimilate RO observations is shown combined in Figs. 3a and 3b. As shown in Fig. 2, there is no benefit in the extratropics from increasing the number of RO observations in the tropical latitudes, as the differences between BC2EQ and BC2EQ_30, and BC2PO and BC2, are very small. (There is, however, a slight improvement in AC skill in the NH by adding observations in the tropics when the reduced time window is used, since BC2_30 is slightly better than BC2PO_30). On the other hand, there is benefit in the extratropics from increasing the number of RO soundings when the observations are globally distributed: BC2PO and BC2 show more skill overall than BC2PO_30 and BC2_30.
Vertical cross sections of the systematic differences between the experiments and the nature run are shown for the temperature analysis field in Fig. 4. Results are averaged over the verification period of the study. As expected, the assimilation of RO observations in BC2EQ reduces the bias in the analyses primarily within the tropical latitudes. On the other hand, the assimilation of globally distributed observations (BC2PO and BC2) improves the analyses globally. Once again, little improvement is gained from the assimilation of additional observations in the tropics (TR, 20°S–20°N) in BC2. A reduction of the assimilation window in BC2EQ_30, BC2PO_30, and BC2_30 results in similar findings, although the impact of the observations in reducing the biases is now slightly lower.
There is also a small improvement in specific humidity from adding new RO observations in both the 6- and the 1-h assimilation time window experiments (Fig. 5). The bias in specific humidity is lower for the experiments that use the 6-h assimilation time window, and the lowest value is found for the experiment that uses the largest number of RO profiles (BC2). The largest improvements are found in the mid- and lower troposphere, where the sensitivity of RO to moisture is larger.
b. Global winds
The 3-day root-mean-square (rms) errors for the lower- (850 mb) and upper-level (200 mb) winds are shown in Fig. 6 for all of the experiments. As expected, the largest errors for the lower-level winds are found in BASE and the smallest in NR. As was found for the geopotential heights, the winds in BASE are worse in the SH than in the NH by ~6 m s−1. Differences in the NR are significantly smaller (~1 m s−1) between the two latitudinal bands. The addition of RO observations in BC2EQ improves winds globally, and further improvement is achieved from assimilating RO observations in BC2PO (total error reductions over BASE of 47.9% in NH, 35.3% in TR, and 57.3% in the SH). There is no further improvement when additional tropical RO observations are added in BC2, including in the tropics. (The differences between BC2PO and BC2 are not statistically significant.) This corroborates the results of the previous section: at this model resolution and for this assimilation technique, saturation with COSMIC-2 data might be already achieved with the global distribution of BC2PO. When the assimilation time window is reduced to 30 min, global winds rms errors are slightly reduced when additional observations are ingested in BC2_30, as compared to BC2PO_30, by ~0.15 m s−1 on average, although these differences are not statistically significant. From all the experiments that assimilate RO observations, BC2PO has the lowest wind rms errors.
Similar results are found for the 200-mb upper-level winds: improvement from BASE to BC2EQ and BC2EQ_30, neutral impact from BC2PO to BC2, and from BC2PO_30 to BC2_30 (differences between BC2PO and BC2, and BC2PO_30 and BC2_30, are not statistically significant).
4. Added value of COSMIC-2 observations
We run a second set of experiments with perfect observations to quantify the additional benefits of COSMIC-2 within the context of a control (CTL) experiment that ingested all the conventional and satellite data routinely assimilated by most operational weather centers. Observations of COSMIC-2A were assimilated in C2EQ, COSMIC-2B in C2PO, and profiles from COSMIC-2A and COSMIC-2B combined were assimilated in C2. In all three cases, the assimilation of all the observations, including RO, used a 6-h assimilation time window. Identical experiments using a reduced assimilation time window of 1 h for the RO observations were run in C2EQ_30, C2PO_30, and C2_30, respectively.
a. Mass and humidity fields
AC skill results for NH and SH using a 6-h assimilation time window are shown in Figs. 7a and 7b, respectively, for the 500-mb geopotential heights as a function of the forecast hour. As expected, the CTL performs much better than BASE and the impact of assimilating RO observations is now less significant. In the NH (Fig. 7a), differences between CTL and C2EQ are not statistically significant, indicating that the impact in the NH from the assimilation of COSMIC-2 observations in the primarily tropical latitudes is neutral. A slight benefit over the CTL is obtained when the spatial distribution of the observations is more uniform among latitudinal ranges in C2PO and C2 (0.01 in AC at day 5). Note that, as was found in the baseline experiment configuration, there is no benefit from assimilating additional observations in the tropical latitudes in C2 over C2PO. (Experiments C2PO and C2 show very similar levels of skill.) Differences between C2PO (and C2) and CTL are statistically significant until day 4. Experiment C2 contains 94.7% of the AC skill of the NR at day 5, with 93.6% already contained in CTL and the rest attributed to the RO observations.
As discussed in Cucurull and Anthes (2015), it is important to emphasize that, although the differences in AC skill might look small, these small differences are considered significant within the scientific community. The improvement in the AC score at 500 mb over the past three decades has been ~1.5% yr−1 (English et al. 2013).
The contribution of COSMIC-2 to improving the weather forecast skill is larger in the SH (Fig. 7b). This result is an agreement with earlier OSE studies with real COSMIC observations. Differences between all the experiments that assimilate COSMIC-2 observations and CTL are now statistically significant until day 6. Although the largest benefits come from the assimilation of COSMIC-2 observations in C2, there is already a positive contribution from the assimilation of RO observations in the tropical latitudes (C2EQ). Note that in the SH there is some benefit from assimilating additional RO observations in the tropical region, as C2 is slightly better than C2PO. As expected, the differences between the NR and the CTL are larger in the SH than in the NH (by 0.03 in AC at day 5), as a result of a slightly worse CTL and a better NR, indicating that there is more room for improvement in the SH from the assimilation of additional observations. (The lower number of conventional observations available in the SH, as compared to the NH, results in a worse CTL.) In the SH, experiment CTL contains 90.3% of the skill of the NR at day 5, while the addition of RO observations in C2 increases this value to 92.7%.
In the previous section it was found that, at day 5, the assimilation of COSMIC-2 observations alone accounted for 73.6% of the skill of the NR in the NH, and for 86.1% of the NR in the SH, whereas the CTL assimilation accounts for 93.6% of the NR skill in the NH and 90.3% in the SH. By comparison, experiment C2 contains 94.7% of the AC skill of the NR in the NH and 92.7% in the SH. Thus, the use of all the conventional and satellite radiance observations in CTL makes a larger contribution toward the skill of the NR as compared to the COSMIC-2 observations alone, by 10% in the NH and by 4.2% in the SH. COSMIC-2 only increases the skill of the CTL by 1.1% in the NH and 2.4% in the SH. Thus, one must conclude from this result that RO observations cannot replace most of the current observing system. Of course, it should be noted that these results are based on only one metric and there are undoubtedly many individual high-impact weather events where different observing systems make critical differences.
In the SH, the assimilation of RO observations also alleviates some dropouts (i.e., a quick drop in AC skill at a given time) present in the CTL experiment during the period under study (Fig. 8a). In Fig. 8a, two dropouts in the 5-day AC skill for the 500-mb geopotential heights are evident in the CTL experiment on 8 and 23 August 2005. This drop in AC skill is less intense in C2EQ and further reduction is found with the assimilation of RO in C2PO and C2 for the 8 August case. (The improvement in skill is more similar for all the experiments that assimilate RO for the 23 August case.) In both cases, there is still room for improvement when compared to the NR, which also shows a small dropout at these two verification times.
Differences between the experiments that assimilate COSMIC-2 and the CTL become insignificant in the NH when the assimilation time window for RO observations is reduced to 1 h (Fig. 7c), suggesting that the number of RO profiles in C2EQ_30, C2PO_30, and C2_30 is too low to have a positive impact on weather forecast skill. However, the number of RO soundings is enough to improve the AC skill in the SH (Fig. 7d), where the lower number of conventional observations allows for a new observing system to make a larger positive impact, and with the largest improvement found in C2PO_30 and C2_30. Also, notice that C2PO_30 and C2_30 show very similar levels of skill, thus adding a few more profiles in the tropics does not result in an improvement in forecast skill. On the other hand, many more profiles in the tropics results in an improvement since C2 is better than C2PO (Fig. 7b). Differences in the SH are statistically significant until day 6. A smaller number of RO profiles is not capable of mitigating the dropout found in the CTL on 8 August in any of the experiments, and just partially for the 23 August dropout case (Fig. 8b).
All the experiments are shown combined in Figs. 3c and 3d for the NH and SH, respectively. The largest benefits in both latitudinal bands come from the assimilation of RO in C2PO and C2, although differences between these two runs and C2EQ_30 are not statistically significant beyond day 4.
Temperature differences between the experiment analyses and the NR, averaged over the verification period, are shown in Fig. 4. As expected, the bias in CTL is lower than in BASE, and some benefits from the assimilation of COSMIC-2 observations exist. The largest reduction in bias is found in C2, where the direct benefits of assimilating almost nonbiased RO observations by “anchoring” the model are combined with the indirect benefits of improving the bias correction of the satellite radiances. It is interesting that BC2 seems to be slightly better than CTL. This seems to suggest that the direct benefits of assimilating COSMIC-2 observations in reducing the bias of the model are slightly larger than the impact of the bias correction schemes applied to the satellite radiance data, without the assimilation of RO observations, in reducing the bias of the model. However, there is room for improvement in all experiments, as the temperature bias in the NR is lower than in any other experiment.
Similar results, although of less magnitude, are found when a 1-h assimilation time window is used instead. With the use of either assimilation time window, these results suggest that observations with global distribution are needed in order to reduce the bias globally.
The addition of COSMIC-2 observations also results in a reduction of moisture systematic differences in both the 6- and the 1-h assimilation time window experiments (Fig. 5), although the benefits are larger with the extended assimilation time window. Once again, the lowest value in bias is found for the experiment that uses the largest number of RO profiles (C2). As in the case of the temperature field, the reduction of the moisture bias in the mid- and lower troposphere is slightly larger in BC2 than in CTL, reinforcing the benefits of assimilating almost nonbiased observations in reducing biases in the analysis.
b. Global winds
Figure 9 shows the 3-day rms errors for the lower- (850 mb) and upper-level (200 mb) winds in the NH, TR, and SH latitudinal ranges. As expected, all the rms errors in CTL are much lower than in BASE (Fig. 6) and, consequently, there is now less room for improvement from assimilating additional observations.
Lower-level rms error winds are globally slightly reduced by adding RO observations in C2EQ, and further reduction is obtained when globally distributed RO observations are assimilated in C2PO. Additional RO profiles in the tropical latitudes available in C2 slightly improve the winds in the extratropics, but they slightly degrade the skill in the TR. Overall, a neutral or slightly positive impact is found with a 1-h assimilation time window when the number of profiles increases from C2EQ_30 to C2PO_30 and to C2_30. The smallest rms error winds are found when a wider assimilation time window is used, except in the TR, where C2 is slightly worse than C2_30. However, differences between experiments are small and not statistically significant.
For the upper-level winds, there is a slight improvement in C2EQ over CTL, and in C2PO over C2EQ, and the impact is neutral from C2 to C2PO, reaffirming a possible saturation in forecast skill with the number of COSMIC-2 observations already available in C2PO. Results are more mixed when the shorter assimilation time window is used: a progressive improvement in the SH from C2EQ_30 to C2PO_30 and from C2PO_30 to C2_30; a progressive improvement in the NH from C2EQ_30 to C2PO_30, but with a slight degradation from C2PO_30 to C2_30; and a degradation in the TR from C2EQ_30 to C2PO_30, but with an improvement from C2PO_30 to C2_30. However, and as for the lower-level winds, the differences between experiments are small and not statistically significant.
5. Sensitivity of weather forecast skill to COSMIC-2 observation density
To investigate the sensitivity of the model forecast skill to the number of COSMIC-2 soundings assimilated in the model, we run two additional experiments: C2_120 assimilated COSMIC-2 observations (both equatorial and polar components) with a 4-h (±2 h from the analysis time) assimilation time window, and C2_60 assimilated COSMIC-2 observations with a 2-h (±1 h from the analysis time) assimilation time window. The geographical locations of the RO soundings for these two additional assimilation time windows are shown in Figs. 1b and 1c, respectively.
Die-off curves for the AC of the 500-mb geopotential heights are represented for the different assimilation time windows in Figs. 10a and 10b for the NH and SH, respectively. The comparison between the several experiments illustrates the difference in forecast skill obtained by increasing the number of COSMIC-2 soundings. As indicated in Fig. 1, the coverage of the observations significantly decreases as the data acceptance time window shortens from 6 to 1 h.
The spatial distribution of the observations in Fig. 1b (C2_120, 4-h assimilation time window) provides the highest AC score in the NH. Differences between C2_120 and CTL are statistically significant until day 5. As found in the previous section, there is no gain in skill over CTL when the scarcer sampling in Fig. 1d is used. Results are different in the SH, where C2_60, C2_120, and C2 result in a similar improvement in skill over CTL, with differences statistically significant until day 6. Note that C2_30 also improves skill, although less so than when a denser RO data sampling is used. Thus, the COSMIC-2 data distribution from Fig. 1b results in the best forecast skill (globally) for the resolution and data assimilation configuration used in our study. The lowest sampling distribution (Fig. 1d) would result in an improvement in skill in the SH, but would have a neutral impact in the NH. In both latitudinal ranges, there is room for improvement as the NR performs better than any of the experiments that assimilate COSMIC-2 observations.
We conducted a series of simplified OSSEs to evaluate the impact of COSMIC-2 on the skill of global weather forecasts, including both the equatorial and polar components of the mission. Because our study uses an earlier version of the NCEP GFS model and GSI 3DVAR analysis with perfect observations, results cannot directly be extrapolated to what we would obtain if a state-of-the-art operational forecast model and data assimilation system were used instead, and/or realistic errors were added to the observations. For example, improvements in the variational bias correction applied to the satellite radiance observations and the use of a hybrid ensemble variational analysis that exist in NOAA’s operational system, rather than the deterministic version of the GSI analysis adopted in our experiments, might alter the results of this study by either increasing or decreasing the impact of COSMIC-2 soundings in weather forecast skill. Therefore, results of this study cannot be used to support funding decisions. Our investigation provides scientific information on the characteristics of the RO technology in general, and COSMIC-2 in particular, and its potential impact on global weather forecast skill.
Overall, the benefits from assimilating COSMIC-2 observations were found to be more significant in the SH than in the NH, consistent with earlier studies using real RO observations. Further improvement in forecast skill might be obtained with additional RO profiles with global spatial coverage that is different than that of COSMIC-2 and other types of observations. Our results suggest that a global distribution of RO observations is needed to reduce biases globally and we found that, in order to improve terrestrial weather forecasting, the impact of having a more uniform distribution of observations as provided by COSMIC-2B is more important than increasing the density of data over the tropical latitudes as achieved by COSMIC-2A.
For the data assimilation system and the metrics used in this study, it appears that the optimal assimilation time window for COSMIC-2 is 4 h. Assimilating more observations from the COSMIC-2 constellation by extending the assimilation time window to 6 h does not result in an increase in weather forecast skill. However, a more modern data assimilation system with higher horizontal resolution would likely benefit from the assimilation of a larger number of RO observations.
When a poor baseline was used, results showed significant improvement in global weather forecast skill with COSMIC-2A but the largest benefit came from the COSMIC-2B assimilation. No additional benefit was found from assimilating COSMIC-2A data in addition to COSMIC-2B. With the use of a more realistic OSSE configuration, which includes the current observing system but incorporating perfect simulated observations, we found the impact of COSMIC-2A to be neutral in the NH, and only a slight positive impact was found with COSMIC-2B. Again, no benefits over COSMIC-2B were found when COSMIC-2A data were assimilated on top of the COSMIC-2B data in the NH. This seems to indicate that, for the data assimilation configuration and model horizontal resolution used in this study, and neglecting the error structures associated with the observations that exist in the real world, the second launch of COSMIC-2 is more important than the first launch in order to improve skill in the NH. However, once again, it is possible that forecast skill would benefit from a denser pattern of RO coverage over the tropical latitudes if a more modern data assimilation system was used. For example, denser observing networks might be beneficial for the higher horizontal resolution of current data assimilation systems. Future work will address this question by performing OSSEs with a more state-of-the-art data assimilation system and the use or error-added observations. Although the largest benefits still come from the assimilation of COSMIC-2B observations and not from COSMIC-2A in the SH, a positive impact from COSMIC-2A already exists, and the assimilation of both COSMIC-2A and COSMIC-2B combined provides the best weather forecast skill in the SH.
The fact that our findings are consistent with earlier studies that used real RO observations, gives us confidence in the value of the results of this study. A new OSSE system with a more state-of-the-art nature run generated by NASA’s Global Modeling and Assimilation Office (Putman et al. 2015) is being developed at NOAA in collaboration with other entities (Boukabara et al. 2016), and more rigorous OSSEs with RO and other observing systems will be reported upon in the near future. Finally, our study has only evaluated the impact of COSMIC-2 in terrestrial weather prediction and we have not addressed the benefits that these data are expected to provide for space weather applications.
The authors thank Dr. Robert Atlas for valuable discussions and Dr. Sean Casey for helping to set up the experiments. We are also thankful to Jack Woollen for providing the templates to simulate the COSMIC-2 observations and for simulating the conventional observations, to Dr. Hongli Wang to help simulate observations for the nature run experiment, to Dr. Michiko Matsutani for generating the simulated satellite radiances, and to Dr. Zoltan Toth for careful proofreading to improve the manuscript. Dr. Richard Anthes and three anonymous reviewers provided good comments that helped to improve the manuscript. This work was funded by the Establishment of a NOAA Laboratory Activity for Observing System Simulation Experiments project, under the Funding Opportunity NOAA Weather Satellite Data Mitigation Gap Reserve Fund of the Sandy Supplemental. This project funded Mrs. Li and Dr. Peevey through the NOAA grant NA14OAR4830105. The University Corporation for Atmospheric Research (UCAR) was funded by NOAA to generate the files with the geophysical coordinates for the COSMIC-2 observations. Finally, this research was conducted under the auspices of NOAA’s Quantitative Observing System Assessment Program (QOSAP).