• Aumann, H., and et al. , 2003: AIRS/AMSU/HSB on the Aqua mission: Design, science objectives, data products, and processing systems. IEEE Trans. Geosci. Remote Sens., 41, 253264, doi:10.1109/TGRS.2002.808356.

    • Search Google Scholar
    • Export Citation
  • Baker, N. L., , T. F. Hogan, , W. F. Campbell, , R. L. Pauley, , and S. D. Swadley, 2005: The impact of AMSU-A radiance assimilation in the U.S. Navy’s Operational Global Atmospheric Prediction System (NOGAPS). Naval Research Laboratory Rep. NRL/MR/7530–05-8836, 22 pp.

  • Bauer, P., , G. Radnóti, , S. B. Healy, , and C. Cardinali, 2014: GNSS radio occultation constellation observing system experiments. Mon. Wea. Rev., 142, 555572, doi:10.1175/MWR-D-13-00130.1.

    • Search Google Scholar
    • Export Citation
  • Cameron, J., , J. Cotton, , and R. Marriott, 2013: Initial assessment of the impact of MetOp-B IASI. Forecasting Research Tech. Rep. 579, Met Office, 13 pp. [Available online at http://www.metoffice.gov.uk/media/pdf/n/9/FRTR579.pdf.]

  • Collard, A. D., , and A. P. McNally, 2009: The assimilation of Infrared Atmospheric Sounding Interferometer radiances at ECMWF. Quart. J. Roy. Meteor. Soc., 135, 10441058, doi:10.1002/qj.410.

    • Search Google Scholar
    • Export Citation
  • Cook, K., , P. Wilczynski, , M. Wenkel, , C.-J. Fong, , and V. Chu, 2014: The future of GNSS-RO for global weather monitoring and prediction—A COSMIC-2/FORMOSAT-7 program status update. Eighth FORMOSAT-3/COSMIC Data Users’ Workshop, Boulder, CO. [Available online at http://www.cosmic.ucar.edu/workshop_2014/presentations/Session8/cook-session8.pdf.]

  • Cucurull, L., , and J. C. Derber, 2008: Operational implementation of COSMIC observations into the NCEP’s Global Data Assimilation System. Wea. Forecasting, 23, 702711, doi:10.1175/2008WAF2007070.1.

    • Search Google Scholar
    • Export Citation
  • Cucurull, L., , and R. A. Anthes, 2015: Impact of loss of U.S. microwave and radio occultation observations in operational Numerical Weather Prediction in support of the U.S. data gap mitigation activities. Wea. Forecasting, 30, 255269, doi:10.1175/WAF-D-14-00077.1.

    • Search Google Scholar
    • Export Citation
  • De Chiara, G., , P. Janssen, , H. Hersback, , and N. Bormann, 2012: Assimilation of scatterometer winds at ECMWF. ECMWF, Reading, United Kingdom, 8 pp. [Available online at https://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=PDF_CONF_P60_S4_05_DECHIARA_V&RevisionSelectionMethod=LatestReleased&Rendition=Web.]

  • Han, Y., , P. van Delst, , Q. Liu, , F. Weng, , B. Yan, , R. Treadon, , and J. Derber, 2006: JCSDA Community Radiative Transfer Model (CRTM)—Version 1. NOAA Tech. Rep. NESDIS 122, Rep. CRTM-V1, 1 p. [Available online at https://cimss.ssec.wisc.edu/itwg/itsc/itsc15/posters/session_b/b19_han.pdf.]

  • Han, Y., and et al. , 2013: Suomi NPP CrIS measurements, sensor data record algorithm, calibration and validation activities, and record data quality. J. Geophys. Res. Atmos., 118, 12 73412 748, doi:10.1002/2013JD020344.

    • Search Google Scholar
    • Export Citation
  • Healy, S. B., , A. M. Jupp, , and C. Marquardt, 2005: Forecast impact experiment with GPS radio occultation measurements. Geophys. Res. Lett., 32, L03804, doi:10.1029/2004GL020806.

    • Search Google Scholar
    • Export Citation
  • Healy, S. B., , S. English, , T. McNally, , E. Di Tomaso, , and G. De Chiara, 2013: Impact of the Metop satellites in the ECMWF system. ECMWF Newsletter, No. 137, ECMWF, Reading, United Kingdom, 9–10. [Available online at http://www.ecmwf.int/sites/default/files/elibrary/2013/14579-newsletter-no137-autumn-2013.pdf.]

  • Jung, J. A., , T. H. Zapotocny, , J. F. Le Marshall, , and R. E. Treadon, 2008: A two-season impact study on NOAA polar-orbiting satellites in the NCEP Global Data Assimilation System. Wea. Forecasting, 23, 854877, doi:10.1175/2008WAF2007065.1.

    • Search Google Scholar
    • Export Citation
  • Karbou, F., , E. Gérard, , and F. Rabier, 2010: Global 4DVAR assimilation and forecast experiments using AMSU observations over land. Part I: Impacts of various land surface emissivity parameterizations. Wea. Forecasting, 25, 519, doi:10.1175/2009WAF2222243.1.

    • Search Google Scholar
    • Export Citation
  • Kim, E., , C.-H. J. Lyu, , K. Anderson, , R. V. Leslie, , and W. J. Blackwell, 2014: S-NPP ATMS instrument prelaunch and on-orbit performance evaluation. J. Geophys. Res. Atmos., 119, 56535670, doi:10.1002/2013JD020483.

    • Search Google Scholar
    • Export Citation
  • Le Marshall, J., , J. Jung, , T. Zapotocny, , C. Redder, , M. Dunn, , J. Daniels, , and L. P. Riishojgaard, 2008: Impact of MODIS atmospheric motion vectors on a global NWP system. Aust. Meteor. Mag., 57, 4551.

    • Search Google Scholar
    • Export Citation
  • Lord, S., , G. Gayno, , and F. Zhang, 2016: Analysis of an observing system experiment for the Joint Polar Satellite System. Bull. Amer. Meteor. Soc., doi:10.1175/BAMS-D-14-00207.1, in press.

    • Search Google Scholar
    • Export Citation
  • McClung, T., 2014: NOAA technical implementation notice 14-46. NOAA/NWS, Rep. 14-46.[Available online at http://www.nws.noaa.gov/os/notification/tin14-46gfs_cca.htm.]

  • McNally, A. P., , J. C. Derber, , W. Wu, , and B. B. Katz, 2000: The use of TOVS level-1b radiances in the NCEP SSI analysis system. Quart. J. Roy. Meteor. Soc., 126, 689724, doi:10.1002/qj.49712656315.

    • Search Google Scholar
    • Export Citation
  • Mo, T, 2007: Calibration of the Advanced Microwave Sounding Unit-A for NOAA-N′. NOAA Tech. Rep. NESDIS 124, 46 pp.

  • Pavelin, E. G., , and B. Candy, 2014: Assimilation of surface-sensitive infrared radiances over land: Estimation of land surface temperature and emissivity. Quart. J. Roy. Meteor. Soc., 140, 11981208, doi:10.1002/qj.2218.

    • Search Google Scholar
    • Export Citation
  • Rawlins, F., , S. P. Ballard, , K. J. Bovis, , A. M. Clayton, , D. Li, , G. W. Inverarity, , A. C. Lorenc, , and T. J. Payne, 2007: The Met Office global four-dimensional variational data assimilation scheme. Quart. J. Roy. Meteor. Soc., 133, 347362, doi:10.1002/qj.32.

    • Search Google Scholar
    • Export Citation
  • Robel, J., and et al. , 2014: Description of the NOAA KLM sensor package. NOAA KLM user’s guide with NOAA-N, N Prime, and MetOp supplements, NOAA/NESDIS/NCDC, 2530 pp. [Available online at http://www1.ncdc.noaa.gov/pub/data/satellite/publications/podguides/N-15%20thru%20N-19/pdf/0.0%20NOAA%20KLM%20Users%20Guide.pdf.]

  • Swadley, S., , G. Poe, , N. Baker, , B. Ruston, , W. Bell, , D. Kunkee, , and D. Boucher, 2010: SSMIS radiance assimilation, calibration anomaly mitigation and assimilation results from F18. 17th Int. TOVS Study Conf., Naval Research Laboratory, Monterey, CA, ITSC-17. [Available online at http://library.ssec.wisc.edu/research_Resources/publications/pdfs/ITSC17/swadley01_ITSC17_2010.pdf.]

  • Wang, X., , D. Parrish, , D. Kleist, , and J. Whitaker, 2013: GSI 3DVar-based ensemble-variational hybrid data assimilation for NCEP Global Forecast System: Single-resolution experiments. Mon. Wea. Rev., 141, 40984117, doi:10.1175/MWR-D-12-00141.1.

    • Search Google Scholar
    • Export Citation
  • View in gallery

    Polar-orbiting satellite equatorial crossing times for the ascending orbit as of July 2015.

  • View in gallery

    The number of GPSRO profiles provided per day for current and planned missions from 2015 to 2026.

  • View in gallery

    Maps of the polar-orbiting constellation coverage from one GDAS cycle for (a) the control experiment, (b) the 3polar configuration, and (c) the 2polar configuration. The 3polar coverage in (b) is also used in the 3pgps experiment.

  • View in gallery

    Map of the GPSRO constellation coverage for one GDAS cycle for (a) the control experiment and (b) the 3pgps experiment. The Satellite for Scientific Applications-D (SAC-D), C/NOFS, MetOp-A, and COSMIC observations are removed poleward of ±24° in (b).

  • View in gallery

    (top) The averaged 500-hPa geopotential height anomaly correlation (AC) of day-1–7 forecasts over the Northern Hemisphere (NH) for the experimental period 25 May–7 Aug 2014 for cntrl, 3polar, 2polar, and 3pgps verified against the cntrl analysis. (bottom) The corresponding AC differences of 3polar (red), 2polar (green), and 3pgps (blue) with respect to the cntrl AC score. The scores outside of the boxes in the bottom panel are statistically significant at the 95% confidence level.

  • View in gallery

    As in Fig. 5, but for the Southern Hemisphere (SH).

  • View in gallery

    (top) The averaged 500-hPa geopotential height root-mean-square error (RMSE) of day-1–7 forecasts over the Northern Hemisphere (NH) for the experimental period 25 May–7 Aug 2014 for cntrl, 3polar, 2polar, and 3pgps verified against the cntrl analysis. (bottom) The corresponding RMSE differences of 3polar (red), 2polar (green), and 3pgps (blue) with respect to the cntrl RMSE score. The scores outside of the boxes in the bottom panel are statistically significant at the 95% confidence level.

  • View in gallery

    As in Fig. 7, but for the Southern Hemisphere (SH).

  • View in gallery

    Verification of 48-h temperature forecast over North America showing (left) bias and (right) RMSE using radiosonde as reference for cntrl, 3polar, 3pgps, and 2polar experiments.

  • View in gallery

    As in Fig. 9, but for specific humidity forecast verification.

  • View in gallery

    Quantitative precipitation forecast skill scores computed over the CONUS showing (left) equitable threat score and (right) bias score as a function of observed 24-h accumulated precipitation. (bottom) The differences in OSE scores with respect to the cntrl forecast. Number of events for each threshold amount is labeled in blue between the panels.

  • View in gallery

    Observing system experiment hurricane track error as a function of forecast hour for all Atlantic and eastern Pacific basin cases between 22 May and 7 Aug 2014. Error bars represent ±1 standard deviation in track error.

  • View in gallery

    The normalized global overall anomaly correlation (OAC) scores of height, temperature, and vector wind over different pressure levels and forecast time for the experimental period 25 May–7 Aug 2014 of cntrl, 3polar, 2polar, and 3pgps experiments verified against the cntrl analysis.

  • View in gallery

    The normalized global overall root-mean-square error (ORMSE) scores of height, temperature, and vector wind over different pressure levels and forecast time for the experimental period 25 May–7 Aug 2014 of cntrl, 3polar, 2polar, and 3pgps experiments verified against the cntrl analysis.

  • View in gallery

    The normalized global overall forecast score (OFS) scores (the sum of the weighted average between OAC and ORMSE) of height, temperature, and vector wind over different pressure levels and forecast time for the experimental period 25 May–7 Aug 2014 of cntrl, 3polar, 2polar, and 3pgps experiments verified against the cntrl analysis.

  • View in gallery

    Computed NWP index score for the cntrl, 3polar, 3pgps, and 2polar experiments.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 96 96 22
PDF Downloads 36 36 8

Potential Gaps in the Satellite Observing System Coverage: Assessment of Impact on NOAA’s Numerical Weather Prediction Overall Skills

View More View Less
  • 1 NOAA/NESDIS/STAR, College Park, Maryland
  • | 2 RTi at NOAA/NESDIS/STAR, College Park, Maryland
© Get Permissions
Full access

Abstract

The current constellation of environmental satellites is at risk of degrading due to several factors. This includes the following: 1) loss of secondary polar-orbiting satellites due to reaching their nominal lifetimes, 2) decrease in the density of extratropical radio-occultation (RO) observations due to a likely delayed launch of the Constellation Observing System for Meteorology, Ionosphere and Climate-2 (COSMIC-2) high inclination orbit constellation, and 3) the risk of losing afternoon polar-orbiting satellite coverage due to potential launch delays in the Joint Polar Satellite System (JPSS) programs. In this study, the impacts from these scenarios on the National Oceanic and Atmospheric Administration (NOAA) Global Forecast System skill are quantified. Performances for several metrics are assessed, but to encapsulate the results the authors introduce an overall forecast score combining metrics for all parameters, atmospheric levels, and forecast lead times. The first result suggests that removing secondary satellites results in significant degradation of the forecast. This is unexpected since it is generally assumed that secondary sensors contribute to system’s robustness but not necessarily to forecast performance. Second, losing the afternoon orbit on top of losing secondary satellites further degrades forecast performances by a significant margin. Finally, losing extratropical RO observations on top of losing secondary satellites also negatively impacts the forecast performances, but to a lesser degree. These results provide a benchmark that will allow for the assessment of the added value of projects being implemented at NOAA in support of mitigation strategies designed to alleviate the negative impacts associated with these data gaps, and additionally help NOAA to define requirements of the future global observing system architecture.

Corresponding author address: Sid-Ahmed Boukabara, National Oceanic and Atmospheric Administration, NESDIS/STAR/JCSDA, NCWCP E/RA 5830 University Research Ct., 2nd Floor, Office 2617, College Park, MD 20740-3818. E-mail: sid.boukabara@noaa.gov

Abstract

The current constellation of environmental satellites is at risk of degrading due to several factors. This includes the following: 1) loss of secondary polar-orbiting satellites due to reaching their nominal lifetimes, 2) decrease in the density of extratropical radio-occultation (RO) observations due to a likely delayed launch of the Constellation Observing System for Meteorology, Ionosphere and Climate-2 (COSMIC-2) high inclination orbit constellation, and 3) the risk of losing afternoon polar-orbiting satellite coverage due to potential launch delays in the Joint Polar Satellite System (JPSS) programs. In this study, the impacts from these scenarios on the National Oceanic and Atmospheric Administration (NOAA) Global Forecast System skill are quantified. Performances for several metrics are assessed, but to encapsulate the results the authors introduce an overall forecast score combining metrics for all parameters, atmospheric levels, and forecast lead times. The first result suggests that removing secondary satellites results in significant degradation of the forecast. This is unexpected since it is generally assumed that secondary sensors contribute to system’s robustness but not necessarily to forecast performance. Second, losing the afternoon orbit on top of losing secondary satellites further degrades forecast performances by a significant margin. Finally, losing extratropical RO observations on top of losing secondary satellites also negatively impacts the forecast performances, but to a lesser degree. These results provide a benchmark that will allow for the assessment of the added value of projects being implemented at NOAA in support of mitigation strategies designed to alleviate the negative impacts associated with these data gaps, and additionally help NOAA to define requirements of the future global observing system architecture.

Corresponding author address: Sid-Ahmed Boukabara, National Oceanic and Atmospheric Administration, NESDIS/STAR/JCSDA, NCWCP E/RA 5830 University Research Ct., 2nd Floor, Office 2617, College Park, MD 20740-3818. E-mail: sid.boukabara@noaa.gov

1. Introduction

The current environmental satellite global observing system (GOS) consists of a complex arrangement of geostationary and low-earth-orbiting platforms, providing a multitude of spaceborne sensors capable of remotely measuring quantities of Earth’s atmosphere and surface across the visible, infrared, and microwave electromagnetic spectrum. These observations are important for many environmental applications at various time scales from near-real-time (NRT) atmosphere and surface monitoring, nowcasting, short- to medium-range numerical weather prediction (NWP), to longer-term climate monitoring. For short- to medium-range weather forecast applications, satellite data are primarily used to improve the initialization of regional and global forecast models through the process of data assimilation. The types of observations assimilated in NWP are either direct satellite radiances, or derived products, both of which describe the state of the atmosphere and surface such as temperature structure, water vapor distribution, and mass transport (wind), and depends on the information content from the specific satellite sensors in the observing system. The improvements to NWP medium-range forecast skill due to the inclusion of satellite data are well known (Baker et al. 2005; Collard and McNally 2009; McNally et al. 2000). This improvement is largely due to the near-global observations provided by satellites, especially in areas where conventional (e.g., in situ) observations are sparse or nonexistent.

Currently, the polar-orbiting environmental satellite constellation provides near-global coverage for NWP data assimilation application from three primary orbits, defined by the satellite equatorial crossing time: early morning, midmorning, and afternoon. The current early morning coverage is provided primarily by passive microwave (PMW) observations from the U.S. Department of Defense, Defense Meteorological Satellite Program (DMSP) Special Sensor Microwave Imager/Sounder (SSMIS) (Swadley et al. 2010). The current midmorning coverage is provided by the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT) program Meteorological Operational (MetOp) series satellites, which manifest the PMW sensors Advanced Microwave Sounding Unit/Microwave Humidity Sounder (AMSU/MHS), the Infrared Atmospheric Sounding Interferometer (IASI) hyperspectral infrared (IR) sounder, and the Advanced Scatterometer (ASCAT) sensor (Cameron et al. 2013; De Chiara et al. 2012). The afternoon orbit coverage is provided by the heritage National Oceanic and Atmospheric Administration (NOAA) Polar-orbiting Operational Environmental Satellite (POES) series satellites containing PMW sensors AMSU/MHS, and the Advanced Very High Resolution Radiometer (AVHRR) IR sounder (Mo 2007; Robel et al. 2014). The follow-on mission to POES, and predecessor to the next generation of afternoon-orbiting satellites known as the Joint Polar Satellite System (JPSS), is currently providing observations to operational NWP. Known as the Suomi–National Polar-orbiting Partnership (SNPP), the current-generation polar platform includes the PMW sensor Advanced Technology Microwave Sounder (ATMS), the Cross-track Infrared Sounder (CrIS) hyperspectral IR sounder, and the Visible Infrared Imaging Radiometer Suite (VIIRS) (Han et al. 2013, Kim et al. 2014).

Since the nominal design life of the satellite sensors typically does not exceed five years, and requirements for operational continuity of earth-system observations exists across the various operational satellite programs, multiple platforms exist in each of the three primary orbits to create a quasi redundancy in spatial coverage: F17, F18, and F19 in the early morning; MetOp-A and MetOp-B in the midmorning orbit; and NOAA-15, NOAA-18, NOAA-19, and SNPP in the afternoon orbit. Figure 1 illustrates the distribution of current polar-orbiting satellites by ascending orbit equatorial crossing time as of July 2015. It should be noted that some satellites have drifted from their original orbit, including NOAA-15 and NOAA-18, which have drifted from an afternoon orbit toward early morning coverage. Additionally, research satellite platforms including the NASA Aqua, which provides observations from AMSU, the Atmospheric Infrared Sounder (AIRS), and the Moderate Resolution Imaging Spectroradiometer (MODIS), also contribute to the afternoon coverage (Aumann et al. 2003; Le Marshall et al. 2008). Since observations from a 6-h window (e.g., ±3 h from the synoptic times of 0000, 0600, 1200, and 1800 UTC) around the data assimilation cycle are typically used in global NWP, temporal coverage is complete even with the absence of crossing times close to midnight (0000) and noon (1200) local time.

Fig. 1.
Fig. 1.

Polar-orbiting satellite equatorial crossing times for the ascending orbit as of July 2015.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

In addition to polar-orbiting PMW and IR observations, low-earth orbiting satellites also provide global positioning system radio occultation (GPSRO) observations, specifically the bending angle, for direct assimilation in NWP. GPSRO observations contain information on atmospheric temperature and humidity, and have demonstrated positive impact on forecast skill (Cucurull and Derber 2008; Healy et al. 2005). Currently, GPSRO provide point soundings globally from the Constellation Observing System for Meteorology, Ionosphere and Climate (COSMIC), as well as GPSRO instruments on board other polar-orbiting satellites including MetOp-A/B, the Communications/Navigation Outage Forecast System (C/NOFS), the Gravity Recovery and Climate Experiment (GRACE) satellite, and TerraSAR-X.

As satellite programs transition to the next generation of sensors, and launch schedules or mission continuity requirements are changed due to fiscal constraints, there is a risk of degradation in coverage from the satellite GOS. The first risk is the loss of secondary or backup satellite platforms from each of the primary orbits that add unique observations to the GOS as well as robustness. A second risk is the complete loss of observations from afternoon polar-orbiting satellites, since the JPSS-1 satellite may not launch before the failure of all other current POES, SNPP, and Aqua satellites. Currently, the launch is scheduled for early 2017, beyond the design life of current afternoon polar-orbiting satellites. A third risk is a decrease in the density of extratropical GSPRO observations. The follow-on mission to COSMIC, named COSMIC-2, is scheduled to launch six satellites into a low-inclination orbit in 2016 and another six satellites into a higher inclination orbit in 2018, each providing nearly 6000 soundings per day (Cook et al. 2014). However, because of uncertainty in funding for the COSMIC-2 high inclination constellation and its targeted launch after the end of the COSMIC nominal lifetime, the possibility exists to lose some coverage from GPSRO soundings for NWP in the extratropical latitudes (poleward of approximately ±24° latitude), although the coverage in the tropics is expected to become denser with the first phase of COSMIC-2 low inclination constellation. Figure 2 illustrates the number of soundings provided by GPSRO from 2015 to 2026, based on the nominal lifetime of current and future planned missions. The number of available GPSRO soundings is expected to jump with COSMIC-2 (providing roughly 12 000 per day with the full constellation). However, it should also be noted that some GPSRO missions, including C/NOFS, GRACE, and TerraSAR-X/TanDEM-X, are not operationally funded. As missions of opportunity, there is no guarantee that observations from these platforms will be available to operational NWP in the future.

Fig. 2.
Fig. 2.

The number of GPSRO profiles provided per day for current and planned missions from 2015 to 2026.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

In support of NOAA data gap mitigation activities, the U.S. Joint Center for Satellite Data Assimilation (JCSDA) performed a number of observing system experiments (OSEs) over part of the boreal summer 2014 season to 1) assess the impact on NWP forecast skill from a series of degraded global satellite constellation scenarios, and 2) establish a baseline of forecast skill to gauge impact from other mitigation activities designed to increase forecast skill.

Previous OSEs have been performed to assess impacts of removing various parts of the observing system (Cucurull and Anthes 2015; Bauer et al. 2014; Jung et al. 2008), or adding in specific types of observations. Lord et al. (2016) showed large degradation in forecast skill from removing afternoon polar orbit data using the 2012 operational NOAA Global Data Assimilation System/Global Forecast System (GDAS/GFS) with 27-km horizontal resolution. Typically however, the data denials, or data additions, were performed on top of the current observing system and do not consider the evolution of the observing system. In section 2, we describe the configuration of the satellite observing system for the various OSEs performed, along with the data assimilation systems and models used. In section 3, the impact on global statistical forecasts is shown. Section 4 provides a brief impact assessment on tropical cyclone track error, and section 5 shows overall forecast skill score impacts. Finally, the conclusions of this study are stated in section 6.

2. Description of the experiments

a. Numerical weather prediction system description

To test the potential impact of the degraded polar-orbiting satellite constellation as well as the evolving GPSRO constellation, we use a version of the NOAA GDAS/GFS that closely mirrors the version implemented into NOAA operations in January 2015. The data assimilation component is the hybrid 3DVAR/ensemble Kalman filter (EnKF) method employed in the Gridpoint Statistical Interpolation analysis system (GSI) (Wang et al. 2013). The GSI analysis, along with the 80 ensemble analyses and forecasts members used to generate the ensemble portion of the background error covariance, are run at the T570L64 resolution (~27-km horizontal resolution, 64 vertical levels). The full GFS forecast (0–168 h) is run at the T1534 resolution (~13-km horizontal resolution), and uses the semi-Lagrangian dynamics scheme (McClung 2014). The GDAS is cycled for all four synoptic times at 0000, 0600, 1200, and 1800 UTC, while the GFS 168-h forecast is only run at the 0000 UTC cycle as is similarly done in other OSE studies at NCEP (Lord et al. 2016) primarily because of computational resource constraints, and is used to assess the impact on medium-range forecast skill.

The GSI is unmodified from the January 2015 operational implementation. This includes the capabilities to assimilate the various satellite radiance datasets, atmospheric motion vectors (AMVs), GPRSO, conventional observations, and the quality control (QC) mechanisms implemented for those data. It also includes the specifications used to assimilate the various satellite and conventional datasets, including observation error specification for each observation type and radiance bias correction schemes. The Community Radiative Transfer Model (CRTM), version 2.1.3, is used in the GSI for both forward and tangent linear calculations (Han et al. 2006).

b. OSE satellite data configuration

The time period of the OSE covers part of the boreal summer 2014 season, with the experiments initialized at the 1800 UTC 14 May 2014 GDAS cycle. The GDAS/GFS was then run through to 7 August 2014. Several aspects of an evolving polar-orbiting environmental satellite observing system and their impacts on global NWP forecast skill, relative to a baseline control run, were explored.

The control run (cntrl) was performed using all conventional and satellite observations available to the GDAS/GFS in the January 2015 implementation. Three OSEs (data denial experiments) were then executed: the first including a configuration where all of the secondary polar-orbiting platforms were removed leaving only one satellite in each primary orbit (3polar); the second is similar to the 3polar experiment but removing further the afternoon coverage provided by SNPP, therefore leaving only observations in the early and midmorning (2polar); the third is similar to the 3polar experiment but altering the GPSRO observations to assess the impact of the future coverage during the COSMIC-2 era (3pgps). The 3pgps experiment assesses the impact if only the COSMIC-2 low inclination constellation is launched, and therefore only excludes GPSRO observation poleward from ±24° latitude from the current COSMIC constellation along with other platforms which provide GPSRO but would be beyond their design life in 2018.

Table 1 summarizes the satellite constellation configuration for each experiment. For the 3polar experiment, only the F18, MetOp-B, and SNPP platforms are assimilated from the polar satellite constellation. The exception here is that the MODIS IR winds are also assimilated, as a proxy for the VIIRS IR winds that have yet to be implemented in the operational system. For the 2polar experiment, only F18 and MetOp-B are assimilated. The spatial coverage for one GDAS cycle from the polar orbiters assimilated in each experiment is shown in Fig. 3. It should be noted that the quasi redundancy removed in the 3polar, and removal of afternoon data in 2polar, is not extended to GPSRO observations (i.e., GPSRO provided by GRAS on MetOp-A are not removed in the 3polar or 2polar, in order to isolate impacts from the polar MW and IR sounders). This aspect of the quasi redundancy is only removed in the 3pgps experiment poleward of ±24° latitude since it is expected that the COSMIC-2 low inclination constellation will provide more dense observations in the tropics. It is not possible to reflect this increased tropical coverage through synthesizing observations, so the best option is to maintain as many observations at those latitudes as possible. Figure 4 shows the current global GPSRO coverage for one GDAS cycle (Fig. 4a) and the modified coverage for the 3pgs experiment (Fig. 4b), which illustrates the loss of coverage at extratropical latitudes.

Table 1.

Satellite data assimilated in the cntrl (all sensors assimilated), 3polar, 2polar, and 3pgps experiments. Cells marked with “○” indicate sensors not assimilated, and cells marked with “●” indicate sensors assimilated, for the corresponding experiment. GPSRO data in 3pgps experiment only denied outside of ±24° latitude.

Table 1.
Fig. 3.
Fig. 3.

Maps of the polar-orbiting constellation coverage from one GDAS cycle for (a) the control experiment, (b) the 3polar configuration, and (c) the 2polar configuration. The 3polar coverage in (b) is also used in the 3pgps experiment.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

Fig. 4.
Fig. 4.

Map of the GPSRO constellation coverage for one GDAS cycle for (a) the control experiment and (b) the 3pgps experiment. The Satellite for Scientific Applications-D (SAC-D), C/NOFS, MetOp-A, and COSMIC observations are removed poleward of ±24° in (b).

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

3. Statistical forecast impact assessment

a. Methodology

The verification methods for the OSEs are performed with NOAA’s standard operational verification statistics database (VSDB) system using the day-1–7 forecast files from the cntrl, 3polar, 2polar, and 3pgps experiments for the period 25 May–7 August 2014. As mentioned in section 2, all of the OSE experiments included full cycling of the GDAS analysis with the 9-h forecast and the 80-member EnKF hybrid system. The GFS analysis and the subsequent 7-day GFS forecasts for all the OSE experiments were run using only the 0000 UTC cycle, which will be assessed here. The day-1–7 forecasts from the cntrl, 3polar, 2polar, and 3pgps experiments are all verified against both the cntrl analysis and also the independent European Centre for Medium-Range Weather Forecasts (ECMWF) analysis. The first 10 days of the analysis period 15–24 May 2014 were considered for the model spinup, as initial conditions were taken from the operational GDAS run at NOAA Central Operations (NCO). Routine model forecast statistical metrics for various parameters, such as anomaly correlation (AC), root-mean-square error (RMSE), and bias, are first computed and saved in the VSDB format and verification maps and figures are then generated to compare statistics among the different OSEs. Forecasts of temperature and water vapor profiles are also compared to radiosonde observations, and 6-h quantitative precipitation forecasts are compared with NCEP radar/rain gauge precipitation analyses. An assessment of the Atlantic and eastern Pacific basin hurricanes track error during the experimental period in comparison with the NOAA/National Hurricane Center (NHC) “best track” is also performed and will be covered in section 4.

b. Forecast impact assessment versus analysis

To assess the impact of a degraded satellite constellation and altered GPSRO coverage, the 3polar, 2polar, and 3pgps experiments are first compared with the cntrl analysis, with differences in statistics relative to the performance of the cntrl forecast. We illustrate the results of mean AC and RMSE scores of the synoptic and global scale 500-hPa geopotential height (Z500) over the Northern and Southern Hemispheres (NH and SH) and 200-hPa wind speed over the tropics. The averaged Z500 AC and RMSE scores from the day-5 forecast over NH and SH for the cntrl, 3polar, 2polar, and 3pgps are shown in Tables 2 and 3, respectively. As one would expect, the cntrl run assimilating all the conventional and satellite observations operationally available to the GDAS/GFS clearly shows both the highest AC score and lowest RMSE score for both NH and SH. The 3polar and 3pgps experiments show lower AC and higher RMSE compared to cntrl, with the 3pgps experiment exhibiting slightly lower skill than 3polar caused by the further removal of RO observations. Compared with the cntrl forecast, the 2polar configuration shows the largest reduction in AC score and largest increase in RMSE for both NH and SH, illustrating that removal of the afternoon polar-orbiting data has a larger impact than removal of extratropical GPSRO observations on top of the 3polar configuration, for the day-5 forecast.

Table 2.

Mean day-5 forecast anomaly correlation (AC) scores of 500-hPa geopotential height for the period 25 May–7 Aug 2014 for cntrl, 3polar, 2polar, and 3pgps experiments verified against cntrl analysis for the Northern (NH) and Southern Hemispheres (SH).

Table 2.
Table 3.

As in Table 2, but for the mean RMSE scores (m).

Table 3.

The significance of the degradation relative to the cntrl forecast at day 5, and at all other lead times from days 1–7, is illustrated in Figs. 5 and 6, for NH and SH, respectively. The top panels show the average Z500 AC score as a function for forecast hour for the different experiments verifying from 25 May to 7 August 2014. The bottom panel shows the corresponding AC score differences with respect to the cntrl AC score. The difference curves outside of the boxed area in the bottom panel indicate statistical significance at the 95% confidence interval. Assessment of the 3polar AC forecast skill shows that removal of secondary satellite observations has a negative impact on height forecast skill. The degradation is slightly significant up to day 4 in NH, and mostly neutral/not significant beyond day 4 and also in SH at all forecast hours. Removal of some RO observations poleward shows more significant degradation for 3pgps, especially in NH where it is statistically significant at all forecast hours and is ~0.014 worse than the cntrl forecast AC score at day 5. However, removal of the afternoon polar data in the 2polar experiment shows the largest degradation, with statistical significance in both hemispheres at all forecast hours, reaching 0.02 worse than the cntrl at day 5 in NH. The synoptic- and large-scale predictability represented by the Z500 forecast is clearly more negatively affected for the 2polar experiment compared to 3polar and 3pgps experiments. The Z500 RMSE as a function of forecast hour for NH and SH is shown in the top panels of Figs. 7 and 8, with differences of RMSE with respect to the cntrl forecast shown in the bottom panels. The results are similar to the AC scores, with 3polar impact in the NH showing significance up to day 4 but limited impact in the SH. The 3pgps shows significant degradation for NH and SH, reaching 1.5 m worse than cntrl at day 5, but is less negative than the 2polar, which reaches 2 m worse than cntrl at day 5 in NH.

Fig. 5.
Fig. 5.

(top) The averaged 500-hPa geopotential height anomaly correlation (AC) of day-1–7 forecasts over the Northern Hemisphere (NH) for the experimental period 25 May–7 Aug 2014 for cntrl, 3polar, 2polar, and 3pgps verified against the cntrl analysis. (bottom) The corresponding AC differences of 3polar (red), 2polar (green), and 3pgps (blue) with respect to the cntrl AC score. The scores outside of the boxes in the bottom panel are statistically significant at the 95% confidence level.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

Fig. 6.
Fig. 6.

As in Fig. 5, but for the Southern Hemisphere (SH).

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

Fig. 7.
Fig. 7.

(top) The averaged 500-hPa geopotential height root-mean-square error (RMSE) of day-1–7 forecasts over the Northern Hemisphere (NH) for the experimental period 25 May–7 Aug 2014 for cntrl, 3polar, 2polar, and 3pgps verified against the cntrl analysis. (bottom) The corresponding RMSE differences of 3polar (red), 2polar (green), and 3pgps (blue) with respect to the cntrl RMSE score. The scores outside of the boxes in the bottom panel are statistically significant at the 95% confidence level.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

Fig. 8.
Fig. 8.

As in Fig. 7, but for the Southern Hemisphere (SH).

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

Table 4 shows the day-1 and day-5 global relative humidity RMSE at 100 and 850 hPa for each experiment. The degradation in the RH forecast is very large at day 1 for both levels, with the 2polar experiment showing the largest negative impact in the lower troposphere (1.64% higher RMSE than cntrl), and the 3pgps showing the largest negative impact from the tropopause and above (3.67% higher RMSE than cntrl). This is to be expected as the 3polar and 2polar experiments deny instruments with sensitivities to tropospheric water vapor (e.g., MHS, ATMS), while the 3pgps experiment denies the RO observations, which have their largest impact in the upper troposphere/lower stratosphere region. The pattern of degradation is similar for day 5, but the magnitude of difference with respect to the cntrl RMSE is less for all of the experiments.

Table 4.

Mean day-1 and day-5 forecast RMSE scores (%) for 850- and 100-hPa relative humidity (RH) for the period 25 May–7 Aug 2014 for cntrl, 3polar, 2polar, and 3pgps experiments verified against the cntrl analysis globally.

Table 4.

Table 5 shows the day-1 and day-3 tropical wind speed RMSE at 200 hPa for each experiment. The 3polar shows an increase of the RMSE for day 1 from 4.52 m s−1 (cntrl RMSE) to 4.84 m s−1, and the 2polar a further increase to 4.94 m s−1. However, because no tropical GPSRO observations were removed in the 3pgps experiment, the day-1 RMSE is close to the 3polar score at 4.82 m s−1. A similar pattern of degradation is shown for the day-3 RMSE. When assessing extratropical wind scores, removal of the GPSRO observations does result in degradation relative to the 3polar experiment (not shown).

Table 5.

Mean day-1 and day-3 forecast RMSE scores (m s−1) of 200-hPa wind for the period 25 May–7 Aug 2014 for cntrl, 3polar, 2polar,and 3pgps experiments verified against the cntrl analysis over the tropics.

Table 5.

An assessment of the quality of the cntrl forecast as well as the impact from the degraded satellite constellation experiments of 3polar, 3pgps, and 2polar against the operational ECMWF analysis was also performed. Verification AC and RMSE scores for NH and SH Z500, along with other metrics (e.g., tropical winds) are consistent with the results when the cntrl analysis is used as a reference and therefore are not shown.

c. Forecast impact assessment versus ground truth

The use of analyses for forecast verification is beneficial because of the global nature of both the forecast and analysis fields. However, it is also prudent to assess forecast quality using ground truth/in situ data as an alternative reference, even if the spatial coverage is more limited. Here we assess the impact for the cntrl, 3polar, 2polar, and 3pgps experiments versus radiosonde for temperature and humidity forecasts, and rain gauge/precipitation radar for quantitative precipitation forecasts.

Figure 9 shows the forecast verification of temperature over North America using all radiosonde between 25 May and 7 August 2014. The statistical differences in bias (left) and RMSE (right) between the experiments is very small, since there are no dedicated radiosonde used only for verification (i.e., all radiosonde are assimilated, so even 48-h forecasts will be highly correlated). However, some minor differences are apparent and could be of importance. These differences are mainly seen in the temperature bias, where the 3pgps forecast shows larger negative bias near 50 hPa and the 2polar forecast shows larger positive bias below 850 hPa. The larger negative bias with the 3pgps forecast could be due to the indirect impact of assimilating GPSRO through anchoring the bias correction of passive microwave sounder brightness temperatures (Bauer et al. 2014). It is unclear what causes the bias increase near the surface in the 2polar forecasts, due to the fact that surface-sensitive channels, like those on SNPP ATMS and CrIS, are not well assimilated over land in NWP (Karbou et al. 2010; Pavelin and Candy 2014). Figure 10 shows the verification of specific humidity forecasts for each experiment versus radiosonde from 1000 to 500 hPa. As seen with the temperature verification, most differences are seen in the bias assessment, but in this case degradation in the humidity forecasts are seen with the 2polar experiment peaking around 900 hPa, but with a decrease in bias for the 3pgps experiment.

Fig. 9.
Fig. 9.

Verification of 48-h temperature forecast over North America showing (left) bias and (right) RMSE using radiosonde as reference for cntrl, 3polar, 3pgps, and 2polar experiments.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

Fig. 10.
Fig. 10.

As in Fig. 9, but for specific humidity forecast verification.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

Figure 11 shows verification of the 24-h accumulated precipitation forecast from 36- to 60-h forecasts in terms of the computed equitable threat score (ETS) and bias score, using the NOAA/NCEP radar/rain gauge analysis over CONUS as reference. For the ETS, a value of 1 is considered a perfect forecast, while a value of 0 is a useless forecast. Although differences exist in the ETS, mainly with the 3polar forecast, the bottom panels do not show any significant difference in skill with respect to the cntrl forecast, with perhaps the exception of very light precipitation events at 0.2 mm (24 h)−1. The bias score metric can determine whether an event was overpredicted or underpredicted (in terms of precipitation intensity). For 36–60-h forecasts of precipitation, there is further underprediction of moderate precipitation events for all experiments, but not statistically significant.

Fig. 11.
Fig. 11.

Quantitative precipitation forecast skill scores computed over the CONUS showing (left) equitable threat score and (right) bias score as a function of observed 24-h accumulated precipitation. (bottom) The differences in OSE scores with respect to the cntrl forecast. Number of events for each threshold amount is labeled in blue between the panels.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

4. Hurricane track forecast impact assessment

The 2014 eastern Pacific basin hurricane season was above average while the Atlantic basin was below average through 7 August. However both basins experienced high-impact tropical cyclone events during that time period, with Tropical Storm Iselle (formerly a category 4 hurricane) making landfall on the Big Island of Hawaii on 7 August, and Hurricane Arthur making landfall on the Outer Banks, North Carolina, on 3 July, at category 2 strength. There are a total of 13 named storms between the two basins during our assessment period.

In addition to the statistical assessment of global height and wind forecast skill, we assess the impact on tropical cyclone track forecast from the degraded polar-orbiting and altered GPSRO constellations. Figure 12 shows the average track forecast error as a function of forecast time for all tropical cyclone cases between 22 May and 7 August 2014 in both the Atlantic and eastern Pacific basins. The number of cases for each forecast hour is shown on the top axis. There is no noticeable difference in track error out to 24 h, but beyond from 36 to 72 h there is a departure in average track error between the cntrl, which provides the lowest (best) track error, and the 2polar experiment, which provides the highest (worst) track error. At 72 h, the 2polar track error is 18 nautical miles (n mi, 1 n mi = 1.852 km) higher than cntrl. The difference between the 3polar and 3pgps experiments with the cntrl experiment reaches only about 6 n mi at 72 h. Although there is separation between the cntrl and 2polar track errors beyond 36 h, the number of cases available is reduced to only 25 at 72 h. In addition, the error bars that represent ±1 standard deviation of track error are also plotted. As shown, the average track error is well within the error bounds for each experiment. Therefore, many more cases are needed to gain any statistical significance in the assessment on tropical cyclone track forecasts. This prevents us from drawing definite conclusions regarding the impacts on hurricane track forecasts.

Fig. 12.
Fig. 12.

Observing system experiment hurricane track error as a function of forecast hour for all Atlantic and eastern Pacific basin cases between 22 May and 7 Aug 2014. Error bars represent ±1 standard deviation in track error.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

5. Overall forecast scores

The forecast verification against the cntrl as well as ECMWF analyses using VSDB statistics for various fields and pressure levels as described in section 3 yields detailed and comprehensive verification products and metrics that are often intricate to interpret in a coherent fashion to assess the impacts of all experiments. In other words, the scores might be mixed with varying degrees of positive, neutral, and negative impacts. To get an overall picture, an objective evaluation of forecast impacts of various experiments against the cntrl is synthesized, applying a normalized score referred to as overall forecast score (OFS). This score is computed by combining the primary forecast verification metrics, namely the AC and RMSE of key parameters encompassing the lower-, mid-, and upper-tropospheric pressure levels, for forecast hours between 0 and 168 at 24-h intervals. The AC and RMSE calculations are performed using height, temperature, and vector wind at 850-, 700-, 500-, and 250-hPa pressure levels. We compute components of the OFS for the AC (OAC), for each experiment, using
eq1
and RMSE (ORMSE) by
eq2
The normalization is performed for every parameter (i = 1 to np) separately prior to computing the overall score on unitless scores. The normalization accounts for the natural behavior, by having parameter-specific normalization scales that depend on each pressure layer (j = 1 to nlev) and individual forecast lead time (k = 1 to nhr) as well. This implies that all parameters, layers, and lead times have their own scaling range. The average and are computed by summing the individual, normalized AC, and RMSE scores from each day (m = 1 to nday) and then dividing by the total number of iterations in the summation, n (np × nlev × nhr × nday). The absolute minimum and maximum ranges are determined for every parameter, at each level and for individual forecast lead times from the cntrl, 3polar, 2polar, and 3pgps experiments. This is performed with the objective of comparing scores from experiment to experiment, meaning that the minimum and maximum ranges are obtained from all experiments in order to determine their values in an absolute sense. The OFS will provide, for each experiment, the overall forecast quality based on multiple forecast parameters and forecast verification metrics by combining the and for each experiment (e = 1 to nexp) with appropriate weights for the overall AC and overall RMSE as given by
eq3
For simplicity, the weights α and β are set to 0.5.

The normalized global OAC scores are shown in Fig. 13 for the experimental period 25 May–7 August 2014 with all experiments verified against the cntrl analysis, indicating that removal of quasi-redundant polar data in 3polar results in a reduction of OAC compared to the cntrl experiment. The 3pgps experiment, which is the removal of quasi-redundant polar data in conjunction with loss of extratropical GPSRO data, further degrades the OAC compared to 3polar. The 2polar, which signifies removal of quasi-redundant polar data and additional loss of afternoon polar data, results in more significant degradation of OAC than loss of GPSRO. The normalized ORMSE global scores for all experiments shown in Figure 14 essentially follow similar behavior as described for the global OAC. The combined OFS is shown in Fig. 15 indicates that the loss of secondary polar satellites results in a significant degradation of overall forecast quality. The 3polar experiment skill is about 20% less than the reference cntrl. The reduced GPSRO coverage shows 28% loss in skill for the 3pgps experiment compared to the cntrl. The loss of the primary afternoon polar orbit results in a 35% loss in skill for the 2polar experiment compared to the cntrl. It should be noted that the computed skill from the OFS is statistical and does not necessarily represent the skill or degradation of forecasts from the various OSEs for specific weather events.

Fig. 13.
Fig. 13.

The normalized global overall anomaly correlation (OAC) scores of height, temperature, and vector wind over different pressure levels and forecast time for the experimental period 25 May–7 Aug 2014 of cntrl, 3polar, 2polar, and 3pgps experiments verified against the cntrl analysis.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

Fig. 14.
Fig. 14.

The normalized global overall root-mean-square error (ORMSE) scores of height, temperature, and vector wind over different pressure levels and forecast time for the experimental period 25 May–7 Aug 2014 of cntrl, 3polar, 2polar, and 3pgps experiments verified against the cntrl analysis.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

Fig. 15.
Fig. 15.

The normalized global overall forecast score (OFS) scores (the sum of the weighted average between OAC and ORMSE) of height, temperature, and vector wind over different pressure levels and forecast time for the experimental period 25 May–7 Aug 2014 of cntrl, 3polar, 2polar, and 3pgps experiments verified against the cntrl analysis.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

To assess its validity, we compare the OFS score against another independent index; in this case the NWP index implemented at the Met Office is used. The NWP index measures the relative skill between forecasts compared to the skill of the persistence forecast up to a 120-h lead time. The index is based on forecasts of mean sea level pressure, 500-hPa height, and 850-hPa wind in both the NH and SH, and the 850- and 250-hPa winds in the tropics (Rawlins et al. 2007).

The results for the OSEs are shown in Fig. 16, which shows the computed NWP index for the cntrl, 3polar, 2polar, and 3pgps forecasts. The index score for the cntrl is exactly 100, since its own analysis is used as the verification for the persistence forecasts. For forecasts degraded from the cntrl forecast the score is lower than 100, and for forecasts better than the cntrl, the score is greater than 100. The results for the OSEs obtained using the NWP index are consistent with the OFS, showing the highest index score for 3polar (97.4), followed by a degraded score for the 3pgps forecast (96.7), and the worst score for the 2polar forecast (95.1), but all being degraded from the cntrl. The main difference between the NWP index and the OFS is that the parameters chosen by the Met Office are qualified as the most important assessment parameters by their customers, whereas the OFS includes multiple parameters at many atmospheric levels and forecast hours, and considers the AC as well as the RMSE. The fact that the OFS and NWP index are consistent signifies that the neutrality or degradation illustrated in the OSEs is overall pervasive across all forecast parameters and at all forecast lead times.

Fig. 16.
Fig. 16.

Computed NWP index score for the cntrl, 3polar, 3pgps, and 2polar experiments.

Citation: Monthly Weather Review 144, 7; 10.1175/MWR-D-16-0013.1

6. Conclusions

We have assessed several aspects of forecast impacts from various scenarios of degraded satellite constellations on NOAA NWP: the 3polar configuration, which removes the secondary satellites from the polar-orbiting observing system; the 2polar configuration, which removes the afternoon orbit coverage on top of removing the secondary sensors to simulate the potential JPSS data gap; and the 3pgps configuration, which removes some GPSRO coverage in the extratropics to simulate a polar data gap in the COSMIC-2 high-inclination constellation. Each forecast has been compared to the cntrl analysis, ECMWF analysis, as well as ground truth data. The cntrl forecast uses all satellite observations currently implemented in the GDAS/GFS. Performance of the 3polar experiment shows that even removal of the secondary polar-orbit satellite coverage has a negative impact on forecast quality, reaching statistical significance at some forecast hours. This suggests that these secondary satellites are not redundant and offer added information to the forecast, and is supported by previous studies (Healy et al. 2013). Additional removal of the afternoon polar orbit, and exclusion of some GPSRO observations outside of tropical latitudes, also shows large degradation in forecast skill compared with the cntrl forecast. The 2polar forecast shows the largest degradation by removing the afternoon coverage provided by PMW and IR sounders, with a statistically significant decrease in skill for almost all parameters at all forecast hours. As expected, removal of GPSRO data has the largest tropospheric impacts outside of tropical latitudes, but the impacts are not as significant as those resulting from losing the afternoon polar coverage.

The overall forecast quality is represented by the overall forecast score. The OFS results are consistent with the examples of forecast skill shown using the 500-hPa height AC and RMSE, along with the tropical wind speed RMSE at 200 hPa, and supports the fact that the trend in forecast degradation is pervasive throughout all parameters in a consistent fashion among the OSEs. A similar result is yielded using an alternate bulk metric in the NWP index, confirming the use of the OFS as a way to synthesize the overall forecast impacts.

It should finally be noted that our assessment was statistical in nature. The forecast accuracy, on a case-by-case basis could lead to mixed results, but on average it may be more likely to have a poorer forecast if the current global satellite observing system is degraded by losing the secondary satellite coverage, further degraded by losing coverage from GPSRO, and even more degraded by losing the entire afternoon primary orbit.

Acknowledgments

The authors would like to extend thanks to NOAA/NCEP for access to the data assimilation and forecast systems as well as verification tools including the VSDB. We would also like to thank Dr. Ross Hoffman for his review and comments of the manuscript. Funding for this work was provided through NOAA’s satellite data gap mitigation project, as well as the U.S. Joint Center for Satellite Data Assimilation.

REFERENCES

  • Aumann, H., and et al. , 2003: AIRS/AMSU/HSB on the Aqua mission: Design, science objectives, data products, and processing systems. IEEE Trans. Geosci. Remote Sens., 41, 253264, doi:10.1109/TGRS.2002.808356.

    • Search Google Scholar
    • Export Citation
  • Baker, N. L., , T. F. Hogan, , W. F. Campbell, , R. L. Pauley, , and S. D. Swadley, 2005: The impact of AMSU-A radiance assimilation in the U.S. Navy’s Operational Global Atmospheric Prediction System (NOGAPS). Naval Research Laboratory Rep. NRL/MR/7530–05-8836, 22 pp.

  • Bauer, P., , G. Radnóti, , S. B. Healy, , and C. Cardinali, 2014: GNSS radio occultation constellation observing system experiments. Mon. Wea. Rev., 142, 555572, doi:10.1175/MWR-D-13-00130.1.

    • Search Google Scholar
    • Export Citation
  • Cameron, J., , J. Cotton, , and R. Marriott, 2013: Initial assessment of the impact of MetOp-B IASI. Forecasting Research Tech. Rep. 579, Met Office, 13 pp. [Available online at http://www.metoffice.gov.uk/media/pdf/n/9/FRTR579.pdf.]

  • Collard, A. D., , and A. P. McNally, 2009: The assimilation of Infrared Atmospheric Sounding Interferometer radiances at ECMWF. Quart. J. Roy. Meteor. Soc., 135, 10441058, doi:10.1002/qj.410.

    • Search Google Scholar
    • Export Citation
  • Cook, K., , P. Wilczynski, , M. Wenkel, , C.-J. Fong, , and V. Chu, 2014: The future of GNSS-RO for global weather monitoring and prediction—A COSMIC-2/FORMOSAT-7 program status update. Eighth FORMOSAT-3/COSMIC Data Users’ Workshop, Boulder, CO. [Available online at http://www.cosmic.ucar.edu/workshop_2014/presentations/Session8/cook-session8.pdf.]

  • Cucurull, L., , and J. C. Derber, 2008: Operational implementation of COSMIC observations into the NCEP’s Global Data Assimilation System. Wea. Forecasting, 23, 702711, doi:10.1175/2008WAF2007070.1.

    • Search Google Scholar
    • Export Citation
  • Cucurull, L., , and R. A. Anthes, 2015: Impact of loss of U.S. microwave and radio occultation observations in operational Numerical Weather Prediction in support of the U.S. data gap mitigation activities. Wea. Forecasting, 30, 255269, doi:10.1175/WAF-D-14-00077.1.

    • Search Google Scholar
    • Export Citation
  • De Chiara, G., , P. Janssen, , H. Hersback, , and N. Bormann, 2012: Assimilation of scatterometer winds at ECMWF. ECMWF, Reading, United Kingdom, 8 pp. [Available online at https://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=PDF_CONF_P60_S4_05_DECHIARA_V&RevisionSelectionMethod=LatestReleased&Rendition=Web.]

  • Han, Y., , P. van Delst, , Q. Liu, , F. Weng, , B. Yan, , R. Treadon, , and J. Derber, 2006: JCSDA Community Radiative Transfer Model (CRTM)—Version 1. NOAA Tech. Rep. NESDIS 122, Rep. CRTM-V1, 1 p. [Available online at https://cimss.ssec.wisc.edu/itwg/itsc/itsc15/posters/session_b/b19_han.pdf.]

  • Han, Y., and et al. , 2013: Suomi NPP CrIS measurements, sensor data record algorithm, calibration and validation activities, and record data quality. J. Geophys. Res. Atmos., 118, 12 73412 748, doi:10.1002/2013JD020344.

    • Search Google Scholar
    • Export Citation
  • Healy, S. B., , A. M. Jupp, , and C. Marquardt, 2005: Forecast impact experiment with GPS radio occultation measurements. Geophys. Res. Lett., 32, L03804, doi:10.1029/2004GL020806.

    • Search Google Scholar
    • Export Citation
  • Healy, S. B., , S. English, , T. McNally, , E. Di Tomaso, , and G. De Chiara, 2013: Impact of the Metop satellites in the ECMWF system. ECMWF Newsletter, No. 137, ECMWF, Reading, United Kingdom, 9–10. [Available online at http://www.ecmwf.int/sites/default/files/elibrary/2013/14579-newsletter-no137-autumn-2013.pdf.]

  • Jung, J. A., , T. H. Zapotocny, , J. F. Le Marshall, , and R. E. Treadon, 2008: A two-season impact study on NOAA polar-orbiting satellites in the NCEP Global Data Assimilation System. Wea. Forecasting, 23, 854877, doi:10.1175/2008WAF2007065.1.

    • Search Google Scholar
    • Export Citation
  • Karbou, F., , E. Gérard, , and F. Rabier, 2010: Global 4DVAR assimilation and forecast experiments using AMSU observations over land. Part I: Impacts of various land surface emissivity parameterizations. Wea. Forecasting, 25, 519, doi:10.1175/2009WAF2222243.1.

    • Search Google Scholar
    • Export Citation
  • Kim, E., , C.-H. J. Lyu, , K. Anderson, , R. V. Leslie, , and W. J. Blackwell, 2014: S-NPP ATMS instrument prelaunch and on-orbit performance evaluation. J. Geophys. Res. Atmos., 119, 56535670, doi:10.1002/2013JD020483.

    • Search Google Scholar
    • Export Citation
  • Le Marshall, J., , J. Jung, , T. Zapotocny, , C. Redder, , M. Dunn, , J. Daniels, , and L. P. Riishojgaard, 2008: Impact of MODIS atmospheric motion vectors on a global NWP system. Aust. Meteor. Mag., 57, 4551.

    • Search Google Scholar
    • Export Citation
  • Lord, S., , G. Gayno, , and F. Zhang, 2016: Analysis of an observing system experiment for the Joint Polar Satellite System. Bull. Amer. Meteor. Soc., doi:10.1175/BAMS-D-14-00207.1, in press.

    • Search Google Scholar
    • Export Citation
  • McClung, T., 2014: NOAA technical implementation notice 14-46. NOAA/NWS, Rep. 14-46.[Available online at http://www.nws.noaa.gov/os/notification/tin14-46gfs_cca.htm.]

  • McNally, A. P., , J. C. Derber, , W. Wu, , and B. B. Katz, 2000: The use of TOVS level-1b radiances in the NCEP SSI analysis system. Quart. J. Roy. Meteor. Soc., 126, 689724, doi:10.1002/qj.49712656315.

    • Search Google Scholar
    • Export Citation
  • Mo, T, 2007: Calibration of the Advanced Microwave Sounding Unit-A for NOAA-N′. NOAA Tech. Rep. NESDIS 124, 46 pp.

  • Pavelin, E. G., , and B. Candy, 2014: Assimilation of surface-sensitive infrared radiances over land: Estimation of land surface temperature and emissivity. Quart. J. Roy. Meteor. Soc., 140, 11981208, doi:10.1002/qj.2218.

    • Search Google Scholar
    • Export Citation
  • Rawlins, F., , S. P. Ballard, , K. J. Bovis, , A. M. Clayton, , D. Li, , G. W. Inverarity, , A. C. Lorenc, , and T. J. Payne, 2007: The Met Office global four-dimensional variational data assimilation scheme. Quart. J. Roy. Meteor. Soc., 133, 347362, doi:10.1002/qj.32.

    • Search Google Scholar
    • Export Citation
  • Robel, J., and et al. , 2014: Description of the NOAA KLM sensor package. NOAA KLM user’s guide with NOAA-N, N Prime, and MetOp supplements, NOAA/NESDIS/NCDC, 2530 pp. [Available online at http://www1.ncdc.noaa.gov/pub/data/satellite/publications/podguides/N-15%20thru%20N-19/pdf/0.0%20NOAA%20KLM%20Users%20Guide.pdf.]

  • Swadley, S., , G. Poe, , N. Baker, , B. Ruston, , W. Bell, , D. Kunkee, , and D. Boucher, 2010: SSMIS radiance assimilation, calibration anomaly mitigation and assimilation results from F18. 17th Int. TOVS Study Conf., Naval Research Laboratory, Monterey, CA, ITSC-17. [Available online at http://library.ssec.wisc.edu/research_Resources/publications/pdfs/ITSC17/swadley01_ITSC17_2010.pdf.]

  • Wang, X., , D. Parrish, , D. Kleist, , and J. Whitaker, 2013: GSI 3DVar-based ensemble-variational hybrid data assimilation for NCEP Global Forecast System: Single-resolution experiments. Mon. Wea. Rev., 141, 40984117, doi:10.1175/MWR-D-12-00141.1.

    • Search Google Scholar
    • Export Citation
Save