Observing system experiments (OSEs) during two seasons are used to quantify the important contributions made to forecast quality from the use of the National Oceanic and Atmospheric Administration’s (NOAA) polar-orbiting satellites. The impact is measured by comparing the analysis and forecast results from an assimilation–forecast system using one NOAA polar-orbiting satellite with results from using two and three polar-orbiting satellites in complementary orbits.
The assimilation–forecast system used for these experiments is the National Centers for Environmental Prediction (NCEP) Global Data Assimilation System–Global Forecast System (GDAS–GFS). The case studies chosen consist of periods during January–February and August–September 2003. Differences between the forecasts are accumulated over the two seasons and are analyzed to demonstrate the impact of these satellites.
Anomaly correlations (ACs) and geographical forecasts (FIs) are evaluated for all experimental runs during both seasons. The anomaly correlations are generated using the standard NCEP verification software suite and cover the polar regions (60°–90°) and midlatitudes (20°–80°) of each hemisphere. The rms error for 850- and 200-hPa wind vector differences are shown for the tropical region (20°N–20°S). The geographical distribution of forecast impact on geopotential heights, relative humidity, precipitable water, and the u component of wind are also examined.
The results demonstrate that the successive addition of each NOAA polar-orbiting satellite increases forecast quality. The use of three NOAA polar-orbiting satellites generally provides the largest improvement to the anomaly correlation scores in the polar and midlatitude regions. Improvements to the anomaly correlation scores are also realized from the use of two NOAA polar-orbiting satellites over only one. The forecast improvements from two satellites are generally smaller than if using three satellites, consistent with the increase in areal coverage obtained with the third satellite.
The NOAA suite of operational polar satellites have formed a key part of the global and regional observation database for operational weather centers around the globe. A diagnostic evaluation of the impact of the three NOAA polar-orbiting satellites being used by the National Centers for Environmental Prediction (NCEP) operational Global Data Assimilation System (GDAS) is described in this study. Observing system experiments (OSEs) document the impact of some of the numerous data sources available today. This type of study enables quantitative assessment of the impact of the numerous and disparate data sources available for meteorological analysis and prediction.
Similar OSEs have been conducted by Kelly (1997) with the European Centre for Medium-Range Weather Forecasts (ECMWF) global model and by Zapotocny et al. (2000, 2002, 2005a, b) who explored forecast impacts from satellite and in situ data with the NCEP regional model. Complementary to this work, Zapotocny et al. (2007) showed the relative importance of satellite data in the NCEP GDAS, and Zapotocny et al. (2008) showed the relative importance that the Advanced Microwave Sounding Unit (AMSU), High Resolution Infrared Radiation Sounder (HIRS), Quick Scatterometer (QuikSCAT), geostationary atmospheric motion vector data (from satellites), and rawinsondes had in the GDAS.
This work investigates the influence the NOAA polar-orbiting satellites have in the GDAS by examining the quality of the analyses and forecasts from having one polar-orbiting satellite, compared to having two polar-orbiting satellites, or the present complement of three polar-orbiting satellites. The primary goal is to determine the potential gain in weather forecast quality realized from having two or three polar-orbiting satellites versus only one. The baseline experiment, 1_NOAA, uses the AMSU and the HIRS data from NOAA-17 along with the NCEP operational complement of conventional and satellite data but excludes all data from NOAA-15 and NOAA-16 sensors. The 2_NOAA experiment adds NOAA-16 AMSU and HIRS data to the baseline experiment; the 3_NOAA experiment adds NOAA-15 AMSU data and NOAA-16 AMSU and HIRS data to the baseline experiment.
A unique aspect of this work, afforded by the Joint Center for Satellite Data Assimilation (JCSDA) and NCEP, was the ability to conduct impact studies at the operational resolution of the time. Until recently, limited computational resources required that studies covering several seasons be completed at reduced spatial and vertical resolutions. This limitation restricted the conclusions that could be reached about the impact of data types at the operational resolutions.
The paper is structured as follows: Section 2 briefly outlines the GDAS Global Forecast System (GFS) version that was used for this study. Section 3 discusses the diagnostics used to evaluate the anomaly correlation and forecast impacts. In section 4 the forecasts are investigated from zero through seven days. The anomaly correlation results from the tropical, midlatitude, and polar latitudes along with the geographic distributions of forecast impact during both seasons are presented. The results are summarized in section 5.
2. The model and assimilation systems
Most of the details concerning the assimilation system and forecast model used in this work are available in section 2 of Zapotocny et al. (2007). Consequently, only a very brief description is provided here.
The assimilation and forecast methodology used here are consistent with the NCEP operational methodology as explained in Zapotocny et al. (2007). This work examines the forecasts at 0000 UTC out to 168 h, whereas NCEP Operations runs 384-h forecasts at 0000, 0600, 1200 and 1800 UTC. The reduction in horizontal and vertical resolutions in this study from T254L64 initially to T170L42 at 84 h and T126L28 at 180 h is consistent with NCEP operations of the time.
The versions of the Global Spectral Model (Kanamitsu et al. 1991) and assimilation system (Derber et al. 1991; Parrish and Derber 1992) used in this work are described in sections 2a and 2b of Zapotocny et al. (2007). The experimental design, including the time periods used, diagnostics, and methodology for display, is also described in section 3 of Zapotocny et al. (2007). For completeness, a log of changes to the Global Spectral Model since 1991 is available online (http://www.emc.ncep.noaa.gov/gmb/STATS/html/model_changes. html). Likewise, a log of changes to the assimilation system is available online (http://www.emc.ncep.noaa.gov/gmb/gdas/). Finally, the results of this study and the previous work were computed and archived at NCEP on the research and development machine of the time.
For these experiments, the full operational database of conventional and satellite data was used, including the real-time data cutoff constraints for the early and late assimilation cycles produced at NCEP. The assimilated satellite data used in this work are shown in Table 1 and include operational Advanced Television Infrared Observation Satellite (TIROS-N) Operational Vertical Sounder (TOVS) (Smith et al. 1979) radiances from HIRS, the Microwave Sounding Unit (MSU) (Spencer and Christy 1992), and the AMSU-A and AMSU-B sensors (NOAA 2005); ozone information from the Solar Backscatter Ultraviolet (SBUV) sensors (Miller et al. 1997); Defense Meteorological Satellite Program (DMSP) Special Sensor Microwave Imager (SSM/I) surface wind speed (Alishouse et al. 1990); derived surface winds from QuikSCAT (Yu and McPherson 1984); and atmospheric motion vectors from geostationary satellites (Velden et al. 1997; Menzel et al. 1998).
The assimilated in situ data used in this work are listed in Table 2 and include rawinsonde temperatures, specific humidity, and wind components; aircraft observations of wind and temperature; land surface reports of surface pressure; and oceanic reports of surface pressure, temperature, horizontal wind, and specific humidity. Keyser (2006a, b, 2007) provide an excellent overview of the data types provided to NCEP on a daily basis and used operationally for the experiments of this study.
The control and both experiments were started from identical initial conditions. All of the surface and upper-air fields, topography, satellite bias correction, observation error, etc. files were taken from the operational database. The satellite bias correction files were allowed to adjust to the information available in each experiment after each assimilation cycle, similar to Derber and Wu (1998), which is consistent with the way these files are updated at NCEP Operations.
3. Experimental design
Diagnostics presented here include statistics commonly used at NCEP and other numerical weather prediction (NWP) centers. The computation of all anomaly correlations for forecasts produced by the GFS were completed using code developed and maintained at NCEP. NCEP (NWS 2007) provides a description of the method of computation, Krishnamurti et al. (2003) discuss the use of anomaly correlation of geopotential height, and Murphy (1990) discuss the effect of climatology on anomaly correlations. The fields being evaluated are examined in both the polar regions and middle latitudes of each hemisphere along with the tropical region.
The NCEP–National Center for Atmospheric Research (NCAR) reanalysis fields (Kistler et al. 2001) are used for the climate component of the anomaly correlations. This reanalysis was run at a resolution of T62L28 with the output grids reduced to 2.5° × 2.5° horizontal resolution and to rawinsonde mandatory levels. To calculate anomaly correlations the output grids from the three experiments were reduced to this 2.5° × 2.5° horizontal resolution using the NCEP postprocessor. The anomaly correlations are then truncated to include only spectral wavenumbers 1–20. This is the method of verification recommended by the WMO (1999) and traditionally used at NCEP for evaluation of their anomaly correlations.
Another diagnostic used here is to evaluate the forecast impact (FI), as described in Zapotocny et al. (2005a, 2007). For this study, a series of two-dimenional FI results are presented as the positive/negative impact provided by the use of data from a particular satellite. The geographic distributions of FI shown in section 4b for a specific mandatory pressure levels are evaluated using
The variable A is the 0-h GDAS 1_NOAA analysis valid at the same time as the forecasts; N is the number of diagnostic days. The variable C represents the 1_NOAA forecast while D represents the 2_NOAA or 3_NOAA forecast. All FI diagnostics were computed from the 1° × 1° grids generated by NCEP’s postprocessing package. The first term enclosed by parentheses in (1) can be considered the error in the 1_NOAA forecast. The second term on the right enclosed by parentheses in (1) can be considered the error in the 2_ NOAA or 3_NOAA experiment. Dividing by the error of the 1_NOAA forecast normalizes the results and multiplying by 100 provides a percent improvement/degradation with respect to the rms error of the 1_NOAA forecast. A positive forecast impact means the 2_NOAA or 3_NOAA forecast compares more favorably to its corresponding analysis than the 1_NOAA forecast does. Occasionally forecast impacts greater than 100% can be derived from (1) if the 1_NOAA forecast and analysis fields are very similar. This results from dividing by a small denominator in (1).
All diagnostics exclude the first 14 days of each seasonal time period. This delay in evaluating the verification statistics allows for the impact of the denied data to be removed from the assimilation system analyses and forecasts. Excluding the first 14 days reduces the two seasonal windows to 32 and 37 days for the Northern Hemisphere winter and summer seasons, respectively. The forecast diagnostics for this paper were conducted out to 168 h to concentrate on the first week of forecast impacts. The forecasts for the 1_NOAA, 2_NOAA, and 3_NOAA experiment’s anomaly correlations have been verified against their own analyses. The 2_NOAA and 3_NOAA experiments are compared to the 1_NOAA experiment. The results have been examined using the usual practice at NCEP of investigating the impact over two seasons for more than 30 days in each season.
Impact of the three operational NOAA satellites on the quality of forecasts made by the GFS for two time periods are explored in detail. The first time period covers 15 January–15 February 2003; the second time period covers 15 August–20 September 2003. The selection of these time periods enables the diagnostics to capture both summer and winter seasons in each hemisphere. As noted, examination of forecasts for at least 30 days during these two seasons is an integral part of the procedures required by NCEP in determining operational utility of satellite data. The fields examined in this study include geopotential heights, wind vectors, relative humidity, and precipitable water. Underground grid points on isobaric surfaces are not included in the evaluations.
Figure 1 shows the global coverage of each of the three NOAA satellites on 25 January 2003 for the 0000 UTC forecast cycle. This diagram takes into account the early data cutoff required by the GFS and the delay in transmitting data to NCEP. With this orbit scenario there is very little satellite overlap except in the polar regions; however, there are still some data void regions remaining near 30°E in the Northern Hemisphere and near the date line in the Southern Hemisphere (Fig. 1c).
a. Anomaly correlations
Figure 2 presents some of the day 5 anomaly correlations from the 1_NOAA, 2_NOAA, and 3_NOAA experiments during January–February (Figs. 2a and 2b) and August–September 2003 (Figs. 2c and 2d). The fields shown are the 1000- and 500-hPa geopotential height anomaly correlations for midlatitudes in both hemispheres and time periods (Figs. 2a and 2c) and the 500-, 700-, and 850-hPa geopotential height anomaly correlations for the polar regions (Figs. 2b and 2d). In Figs. 2b and 2d, 700 hPa is used in the Southern Hemisphere rather than 850 hPa to ensure that more of the diagnostic domain remains above ground over Antarctica.
The 3_NOAA simulation in general has the highest anomaly correlations, and anomaly correlations for the 2_NOAA and 1_NOAA experiments are, in general, progressively lower. Overall this indicates that the additional information from two and three satellites improves forecast quality; this can be seen for most cases. In this study, the January–February Southern Hemisphere polar region (Fig. 2b) provides an exception. In the polar region experiments for January–February, the 2_NOAA and 3_NOAA Southern Hemisphere polar anomaly correlations are always better than the 1_NOAA polar anomaly correlations; however, the 2_NOAA experiment has higher scores than the 3_NOAA experiment. This may be influenced by the considerable overlap of the satellite orbits near the poles. Consistent with the results from Zapotocny et al. (2008), all of the tropical anomaly correlations (not shown) were found to be approximately 0.15–0.20 better in January–February than they were in August–September. This may be related to the broader expanse of deep tropical convection in August–September than January–February.
Figure 3 compares the 20°–80° Northern and Southern Hemisphere 500-hPa geopotential height day 0–7 anomaly correlation die-off curves from the 1_NOAA simulation to the 2_NOAA and 3_NOAA simulations. The comparisons are for both January–February (Figs. 3a, b, e, f) and August–September 2003 (Figs. 3c, d, g, h). The dark blue line is the 1_NOAA simulation while the magenta line is either the 2_NOAA or 3_NOAA simulation anomaly correlation. In these experiments, the larger the separation between the 1_NOAA anomaly correlation and the 2_NOAA or 3_NOAA anomaly correlation, the greater is the importance of the additional satellite(s) to the quality of the simulation. For 2_NOAA and 3_NOAA experiments the 500-hPa midlatitude anomaly correlation scores are consistently better throughout the 7-day forecasts than for the 1_NOAA experiment. This indicates that the information from the additional satellite(s) is improving these forecasts. It is generally accepted that an anomaly correlation of 0.5 is the equivalent of a climatology forecast and 0.6 or greater indicates a useful measure of forecast skill, implying that some skill is evident in each hemisphere from nearly all of the simulations presented here out to 7 days.
Figure 4 presents a series of wind vector rms difference diagrams comparing the 1_NOAA experiment to the 2_NOAA or 3_NOAA experiments in the tropical region for days 0–7. Figures 4a–d depict the 200-hPa wind vector rms differences while Figs. 4e–h depict the 850-hPa wind vector rms differences. The left column in Fig. 4 presents the January–February 2003 results, while the right column presents the August–September 2003 results. The wind vector rms difference was chosen as the tropical diagnostic because the variance of the anomaly correlation for 500-hPa geopotential height is small in this region. The error growth for both the 200-hPa vector rms difference (Figs. 4a–d) and 850-hPa vector rms difference (Figs. 4e–f) is greatest during the first day. The error growth then slows as the integration proceeds to 7 days. This error characteristic in the tropics is common in the GFS and other models, as explained by Surgi (1989) and Surgi et al. (1998). The error growth was also noted in the Zapotocny et al. (2007) and Zapotocny et al. (2008) results.
The tropical comparison of wind vector rms differences identifies that both the 2_NOAA and 3_NOAA experiments show improved vector rms statistics by a small amount at 200 hPa compared to the use of 1_NOAA satellite data. The wind vector rms differences are almost neutral with degraded rms scores in some places at 850 hPa for both January–February and August–September.
Figure 5 presents the polar region 500-hPa geopotential height days 0–7 anomaly correlation die-off curves for January–February and August–September 2003. The 3_NOAA anomaly correlation die-off curves are consistently better than the 1_NOAA anomaly correlations for January–February and August–September for both polar regions. The 2_NOAA experiment anomaly correlation scores are higher than the 1_NOAA anomaly correlation scores in the Southern Hemisphere, but in the Northern Hemisphere the 2_NOAA scores in places are closer to neutral.
b. Geographic distributions of forecast impact
Figures 6 and 7 are the average rms differences between the analyses of 1_NOAA versus 2_NOAA and 1_NOAA versus 3_NOAA for 500-hPa geopotential heights, 850-hPa relative humidity, and the 850- and 200-hPa u component of wind. They present the geographical distribution of average rms differences of the model analyses during both the January–February and August–September 2003 time periods, respectively. The differences between analyses do not show any relationship with the satellite(s) orbits for any of the fields investigated. The greater differences in the 500-hPa heights in the polar regions and in the 200-hPa winds in the tropics are consistent in magnitude and location when comparing analyses between NWP centers (G. White 2007, personal communication). In general, the differences between the experiment analyses are consistent with what is generally expected in the GDAS analysis (G. White 2007, personal communication). These results are expected since even a single polar-orbiting satellite will sample the globe two times per day.
Figures 8 –17 present the geographical distributions of forecast impacts during both the January–February and August–September 2003 time periods. Figures 8 –17 are derived using Eq. (1) and present time-averaged forecast impacts for forecast periods 12, 24, 48 and 72 h. The 2_NOAA results are in the left column and the 3_NOAA results in the right column for each of these figures. In Figs. 8 –17, negative forecast impacts have magenta shading, neutral or nearly neutral forecast impacts are not shaded, and positive forecast impacts shadings proceed from blue to red. Regions below the surface are shaded black for easy identification. In comparing the area-weighted mean rms from 1_NOAA to 2_NOAA and 1_NOAA to 3_NOAA, the improvements in the forecast impact through 72 h are significant with a confidence level greater than 99% after accounting for serial correlation (Seaman 1992).
Of the four forecast fields presented in Figs. 8 –17, inspection reveals that the amplitude and coverage of the impacts are considerably larger in the 3_NOAA experiments. This can be determined by comparing the four left columns with the four right columns through 72 h. This is common to both January–February and August–September time periods.
Upon examining the 12–72-h forecast impacts (panels a–d and e–h of Figs. 8 –17) one notices a steady decrease in magnitude with time of the forecast impacts. In fact, by 72 h the largest forecast impacts are generally less than 100% with large regions of the globe being covered with neutral impacts (white regions). This decrease in forecast impact is consistent with the satellite denial results presented previously in Zapotocny et al. (2007) where the satellite and conventional data also had a steady decrease through 72 h. Similar results were also found by Zapotocny et al. (2008) where the various sensor types showed a steady decrease in forecast impact through 72 h.
Figures 8 and 9 present the 500-hPa geopotential height 2_NOAA and 3_NOAA forecast impact results for forecast hours 12–72 during January–February (Fig. 8) and August–September 2003 (Fig. 9). Comparing these results shows that the 3_NOAA impacts are generally considerably larger than the 2_NOAA impacts. (Geopotential heights are not a good measure of forecast skill in the tropics.) Even though there are large regions of negative impact, the area-weighted mean rms for the 2_NOAA and 3_NOAA experiments are still smaller than for the 1_NOAA experiment and are statistically significant with a confidence level of greater than 99%. The 3_NOAA experiment area-weighted mean rms is also statistically smaller than for the 2_NOAA experiment with a confidence level of greater than 99%. This indicates significant improvement have been realized in the short-term forecasts by adding the third satellite. It is also shown that the forecast impact is generally smaller in the Northern Hemisphere polar region during August–September compared to January–February 2003. Overall, by 72 h most of the globe is covered by neutral forecast impacts, with only scattered regions of small positive/negative impacts. These forecast impact results are consistent with the anomaly correlation results shown in Fig. 2.
In the tropics, the winds are usually considered a better measure of forecast skill than heights. The 850-hPa u components of wind forecast impacts for the 2_NOAA and 3_NOAA experiments for January–February and August–September 2003 are shown in Figs. 10 and 11. The 200-hPa u component of wind forecast impacts for the 2_NOAA and 3_NOAA experiments for January–February and August–September 2003 are shown in Figs. 12 and 13. The winds, both at 850 and 200 hPa, generally show consistently positive forecast impacts in the tropics, particularly in the first few days. The 3_NOAA experiment has the largest positive forecast impact. Both the 2_NOAA and 3_NOAA experiment areas of positive forecast impacts decrease with time.
The 850-hPa relative humidity impact plots for January–February and August–September 2003 are shown in Figs. 14 and 15. The 3_NOAA experiment has the greatest positive impact for both seasons and for all four forecast periods displayed here. The August–September time period overall shows greater impact than for the January–February time period at 12 h; however, it decays rapidly to small differences by 72 h.
The precipitable water impact plots (Figs. 16 and 17) are quite similar to the 850-hPa relative humidity impacts shown in Figs. 10 and 11. The 3_NOAA experiment again has the greatest positive impact for both time periods. Both the 850-hPa relative humidity forecast impacts, Figs. 14 and 15, and the precipitable water forecast impacts presented here, display a seasonal switch, with largest forecast impacts diagnosed in the summer hemisphere for each field. This seasonal switch is most apparent at 12 and 24 h for each field.
Observing system experiments like these are important because the NOAA series of polar-orbiting satellites are the most significant source of satellite sounding data for operational numerical weather prediction. These experiments examined the improvements in forecasts out to 168 h in the NCEP Global Data Assimilation–Global Forecast System during January–February and August–September 2003. The work described here has quantified the contributions to the forecast made by increasing the number of operational NOAA satellites used in the analysis to two and, then, to three.
The results indicate that the use of three NOAA polar-orbiting satellites, in complementary orbits, generally provides the largest improvement to the anomaly correlation scores in the polar, midlatitude, and tropical regions. Improvements to the anomaly correlation scores are realized from the use of two polar-orbiting satellites, but are generally smaller than when using three polar-orbiting satellites. These results are consistent with the benefits expected from the increase in areal coverage obtained from adding additional satellites.
The geographic distributions of the forecast impacts shown in Figs. 8 –17 indicate that using three polar-orbiting satellites provides the largest positive impact. In general the largest positive impacts are found in the polar regions of both hemispheres, possibly due to the frequent satellite overpasses and the lack of conventional data. Both 2_NOAA and 3_NOAA experiments display a decrease in forecast impact globally as the forecasts proceed from 12 to 72 h. The 3_NOAA experiment retains the greatest forecast impact out to 72 h.
In terms of future observing system experiments, operational or preoperational data types to be studied include the Moderate Resolution Imaging Spectroradiometer (MODIS) atmospheric motion vectors, ocean surface wind vector measurements from space using WINDSAT. In addition, the effective exploitation of the new hyperspectral data, which have or will become available from the Atmospheric Infrared Sounder (AIRS), the Infrared Atmospheric Sounding Interferometer (IASI), the Cross-track Infrared Sounder (CrIS), and Geostationary Operational Environmental Satellite-R (GOES-R) instruments, will be studied.
The authors wish to thank Stephen Lord, Dennis Keyser, Stacie Bender, and John Derber of NCEP for providing the appropriate hardware–software support and guidance. The authors also wish to thank Timothy J. Schmit of NOAA/NESDIS/ORA for his enlightening scientific input and Todd K. Schaack for his help with graphics generation using IDL. The study was undertaken within the Joint Center for Satellite Data Assimilation (JCSDA) and supported under NOAA Grant NA07EC0676, which supports joint center activities.
The contents of this paper are solely the opinions of the authors and do not constitute a statement of policy, decisions, or position on behalf of NOAA or the U.S. government.
Corresponding author address: James A. Jung, NOAA Science Center, 5200 Auth Rd., Camp Springs, MD 20746-4304. Email: firstname.lastname@example.org