• Atkinson, N., and McLellan S. , 1998: Initial evaluation of AMSU-B in-orbit data. Models and Retrieval Techniques, T. Hayasaka et al., Eds., International Society for Optical Engineering (SPIE Proceedings, Vol. 3503), 276–287.

    • Search Google Scholar
    • Export Citation
  • Atkinson, N., Brunel P. , Marguinaud P. , and Labrot T. , 2008: AAPP developments and experiences with processing MetOp data. Proc. 16th Int. TOVS Study Conf., Angra Dos Reis, Brazil, Int. TOVS Working Group, 1.3.

    • Search Google Scholar
    • Export Citation
  • Auligné, T., McNally A. P. , and Dee D. P. , 2007: Adaptive bias correction for satellite data in a numerical weather prediction system. Quart. J. Roy. Meteor. Soc., 133 , 631642.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bell, W., and Coauthors, 2008: The assimilation of SSMIS radiances in numerical weather prediction models. IEEE Trans. Geosci. Remote Sens., 46 , 884900.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Challon, G., Cayla F. , and Diebel D. , 2001: IASI: An advanced sounder for operational meteorology. Proc. 52nd Congress of IAF, Toulouse, France, Int. Astronautical Federation, 1–9.

    • Search Google Scholar
    • Export Citation
  • Dando, M. L., Thorpe A. J. , and Eyre J. R. , 2007: The optimal density of atmospheric sounder observations in the Met Office NWP system. Quart. J. Roy. Meteor. Soc., 133 , 19331943.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • English, S. J., Saunders R. , Candy B. , Forsythe M. , and Collard A. , 2004: Met Office satellite data OSEs. Proc. Third WMO Workshop on the Impact of Various Observing Systems on NWP, Alpbach, Austria, WMO, 146–156.

    • Search Google Scholar
    • Export Citation
  • Eyre, J. R., and Lafeuille J. , 2008: Evolution of the Global Observing System: A vision for 2025. Proc. 16th Int. TOVS Study Conf., Angra Dos Reis, Brazil, Int. TOVS Working Group, 1–38.

    • Search Google Scholar
    • Export Citation
  • Goodrum, G., Kidwell K. B. , and Winston W. , 2000: NOAA KLM user’s guide. NOAA Satellite and Information Service. [Available online at http://www.ncdc.noaa.gov/oa/pod-guide/ncdc/docs/klm/index.htm].

    • Search Google Scholar
    • Export Citation
  • Harris, B. A., and Kelly G. , 2001: A satellite radiance bias correction scheme for data assimilation. Quart. J. Roy. Meteor. Soc., 127 , 14531468.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kelly, G., Thépaut J-N. , Buizza R. , and Cardinali C. , 2007: The value of targeted observations—Part I: Data denial experiments for the Atlantic and the Pacific. ECMWF Tech. Memo. 511, 29 pp.

    • Search Google Scholar
    • Export Citation
  • Le Marshall, J., and Coauthors, 2006: Improving global analysis and forecasting with AIRS. Bull. Amer. Meteor. Soc., 87 , 891894.

  • McMillin, L., and Divakarla M. G. , 1999: Effects of possible scan geometries on the accuracy of satellite measurements of water vapor. J. Atmos. Oceanic Technol., 16 , 17101720.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Muth, C., Webb W. , Atwood W. , and Lee P. , 2005: Advanced technology microwave sounder on the National Polar-Orbiting Operational Environmental Satellite System. Proc. Int. Geoscience and Remote Sensing Symp., Seoul, South Korea, IEEE, 99–102.

    • Search Google Scholar
    • Export Citation
  • Rawlins, F., Ballard S. P. , Bovis K. J. , Clayton A. M. , Li D. , Inverarity G. W. , Lorenc A. C. , and Payne T. J. , 2007: The Met Office global four-dimensional variational assimilation scheme. Quart. J. Roy. Meteor. Soc., 133 , 347362.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rosenkranz, P. W., Hutchison K. D. , Hardy K. R. , and Davis M. S. , 1997: An assessment of the impact of satellite microwave sounder incidence angle and scan geometry on the accuracy of atmospheric temperature profile retrievals. J. Atmos. Oceanic Technol., 14 , 488494.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Swadley, S., Poe G. A. , Bell W. , Hong Y. , Kunkee D. B. , McDermid I. S. , and Leblanc T. , 2008: Analysis and characterization of the SSMIS upper atmospheric sounding channel radiances. IEEE Trans. Geosci. Remote Sens., 46 , 962983.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Met Office data-denial OSEs in which MWS data were withdrawn from an otherwise full observing system that contained advanced IR sounder data (AIRS and IASI) in 2007 but not in 2003.

  • View in gallery

    First-guess departures for AMSU-A channels 5–14 for normal (black) and noisy (gray) data used in the ECMWF OSEs.

  • View in gallery

    The mean difference in analysis temperature fields at 200, 500 and 850 hPa between normal and noisy AMSU experiments.

  • View in gallery

    Estimated analysis accuracy, expressed as a zonal-mean RMSE, for temperature at (top)–(bottom) 10, 50, 100, 200, 500 and 850 hPa for normal (dotted line) and noisy (solid line) AMSU data OSEs.

  • View in gallery

    Met Office OSEs: the impact on forecast quality, in terms of RMSE reductions for all verification measures listed in Table 4, resulting from the addition of normal and noisy AMSU-A data to a baseline experiment (CNTRL-UK) in which all microwave sounding data have been withdrawn. The x and y coordinates of each point represent the change in forecast error (RMSE) relative to a no-MWS baseline experiment (CNTRL-EC) for noisy and normal AMSU experiments, respectively. For example, points in the lower-left quadrant indicate that both normal and noisy experiments reduce forecast errors, but points below the dotted 45° line indicate the noisy data reduce errors less than the normal data.

  • View in gallery

    ECMWF OSEs: the impact on forecast quality, in terms of RMSE reductions for all verification measures listed in Table 4, resulting from the addition of normal and noisy AMSU-A data to a baseline experiment (CNTRL-EC) in which all satellite data (except AMV data) have been withdrawn.

  • View in gallery

    The effect of adding normal and noisy AMSU data on geopotential height forecast accuracy (RMSE) at 850, 500, and 200 hPa for forecast ranges 12–144 h in the SH for the ECMWF OSEs. The error bars represent the standard error on the mean (at 1σ) forecast error change over the 32-day experiment.

  • View in gallery

    As in Fig. 7, but at 100, 50, and 10 hPa.

  • View in gallery

    As in Fig. 7, but on relative humidity forecast accuracy (RMSE).

  • View in gallery

    The effect of adding normal and noisy AMSU data on vector wind forecast accuracy (RMSE) at 850, 500, and 200 hPa, verified using radiosondes, for analysis time (T + 0) and forecast ranges from 24 to 144 h in the tropics for the ECMWF OSEs. Error bars as in Fig. 7.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 103 91 14
PDF Downloads 77 69 10

The Radiometric Sensitivity Requirements for Satellite Microwave Temperature Sounding Instruments for Numerical Weather Prediction

View More View Less
  • 1 ECMWF, Reading, United Kingdom
  • | 2 Met Office, Exeter, United Kingdom
  • | 3 Sula Systems, Wotton-under-Edge, Gloucestershire, United Kingdom
© Get Permissions
Full access

Abstract

The sensitivity of NWP forecast accuracy with respect to the radiometric performance of microwave sounders is assessed through a series of observing system experiments at the Met Office and ECMWF. The observing system experiments compare the impact of normal data from a single Advanced Microwave Sounding Unit (AMSU) with that from an AMSU where synthetic noise has been added. The results show a measurable reduction in forecast improvement in the Southern Hemisphere, with improvements reduced by 11% for relatively small increases in radiometric noise [noise-equivalent brightness temperature (NEΔT) increased from 0.1 to 0.2 K for remapped data]. The impact of microwave sounding data is shown to be significantly less than was the case prior to the use of advanced infrared sounder data [Atmospheric Infrared Sounder (AIRS) and Infrared Atmospheric Sounding Interferometer (IASI)], with microwave sounding data now reducing Southern Hemisphere forecast errors by approximately 10% compared to 40% in the pre-AIRS/IASI period.

Corresponding author address: William Bell, European Centre for Medium-Range Weather Forecasts, Shinfield Park, Reading RG2 9AX, United Kingdom. Email: william.bell@ecmwf.int

Abstract

The sensitivity of NWP forecast accuracy with respect to the radiometric performance of microwave sounders is assessed through a series of observing system experiments at the Met Office and ECMWF. The observing system experiments compare the impact of normal data from a single Advanced Microwave Sounding Unit (AMSU) with that from an AMSU where synthetic noise has been added. The results show a measurable reduction in forecast improvement in the Southern Hemisphere, with improvements reduced by 11% for relatively small increases in radiometric noise [noise-equivalent brightness temperature (NEΔT) increased from 0.1 to 0.2 K for remapped data]. The impact of microwave sounding data is shown to be significantly less than was the case prior to the use of advanced infrared sounder data [Atmospheric Infrared Sounder (AIRS) and Infrared Atmospheric Sounding Interferometer (IASI)], with microwave sounding data now reducing Southern Hemisphere forecast errors by approximately 10% compared to 40% in the pre-AIRS/IASI period.

Corresponding author address: William Bell, European Centre for Medium-Range Weather Forecasts, Shinfield Park, Reading RG2 9AX, United Kingdom. Email: william.bell@ecmwf.int

1. Introduction

Microwave sounding data from polar orbiting satellites are an important component of operational numerical weather prediction (NWP) systems. Of particular importance are measurements in the 50–60-GHz spectral range, covering absorption and emission from the O2 rotational band, which contains information on atmospheric temperature. The Advanced Microwave Sounding Unit (AMSU) is a low-noise cross-track scanning radiometer (Goodrum et al. 2000), and AMSUs on board U.S. and European meteorological satellites have provided continuous observations since 1998. These observations are directly assimilated at most NWP centers in the form of radiances.

Specification of the next generation of microwave sounders (MWS) is currently underway. In Europe, the preparations for the follow-on mission to the (European Organisation for the Exploitation of Meteorological Satellites) EUMETSAT Polar System (EPS) are well under way and this mission (hereafter post-EPS) is expected to become operational around 2020. Within the U.S. National Polar Orbiting Environmental Satellite System (NPOESS), with the first preparatory platform due for launch in 2010, MWS capability will be delivered by both cross-track and conically scanning microwave radiometers. The design of the cross-track instrument [the Advanced Technology Microwave Sounder (ATMS); see Muth et al. 2005] has been finalized; however, the conical instrument [the Microwave Imager Sounder (MIS)] is currently being specified. The radiometric performance of these instruments is a key factor in determining their impact in NWP systems. The purpose of this paper is to assess the sensitivity of NWP forecast accuracy to the radiometric performance of the 50-GHz temperature sounding channels to assist in the specification of future operational instruments. This has been tackled through a series of observing system experiments (OSEs) using both normal AMSU data and synthetically noise-degraded AMSU data within two operational NWP systems.

Recent OSEs have shown that MWS data have a very large positive impact on NWP forecast accuracy: for example, forecast errors in sea level pressure are reduced by approximately 40% for forecast days 1–4 in the Southern Hemisphere (SH; English et al. 2004) through the introduction of microwave temperature sounding data from AMSU. In the time since these studies were carried out in 2003, the global satellite observing system has evolved, most notably with the successful launch of two advanced infrared (IR) sounding missions: the Atmospheric Infrared Sounder (AIRS; Le Marshall et al. 2006) and the Infrared Atmospheric Sounding Interferometer (IASI; Challon et al. 2001). It is likely that the future global satellite observing system will include two or three advanced IR sounders in complementary orbits (see Eyre and Lafeuille 2008). To estimate the likely impact of MWS data in the post-EPS era a secondary aim of this work has been to assess the current impact of microwave sounding data in observing systems which include both AIRS and IASI.

The specification of radiometric performance influences both the choice of scan geometry as well as the detailed design choices to be made for any given scan geometry. To date, two scan geometries have been used for operational microwave sounders: cross track and conical scanning. Studies by Rosenkranz et al. (1997) and McMillin and Divakarla (1999) have investigated the relative performance for temperature and humidity sounding, respectively, of both scan geometries using simulation studies. More recently, the launch of the Special Sensor Microwave Imager (SSM/I) Sounder (SSMIS) has permitted an assessment of the performance of conical scanners for NWP (Swadley et al. 2008; Bell et al. 2008) using on-orbit data. These studies showed that calibration problems can cause complex systematic errors in conical scanners that limit the impact of the temperature sounding data in NWP systems relative to that from AMSU-A, although the data still provide benefit. Another significant difference is that conical scanners, employing constant reflector rotation rates, typically have shorter integration times relative to cross-track scanners, where the smaller reflectors can be accelerated between earth scenes and calibration views, hence increasing earth scene integration time. Consequently, conical scanners typically have higher radiometric noise levels, which are normally expressed as noise-equivalent brightness temperature (NEΔT, with units of K), than cross-track configurations. There are design options that could mitigate this limitation (e.g., the use of multiple 50-GHz feed horns), but these options are untested on orbit. The significance here is that, if radiometric performance is critical, then a cross-track design, with lower noise and better calibration performance, is a more optimal choice based on current evidence. Related to this, if a conical design is to be used for temperature sounding, then particular attention has to be paid to the instrument design to ensure calibration stability and to optimize NEΔT as far as possible.

Section 2 describes the current use of AMSU data at the Met Office (UKMO) and European Centre for Medium-Range Weather Forecasts (ECMWF) and the impact derived from this data in the Met Office global NWP model in both the pre- and postadvanced IR sounder eras. Section 3 describes the OSEs carried out at both centers to assess the sensitivity of forecast accuracy to the radiometric performance for microwave sounders. Conclusions are drawn in section 4, which includes some caveats on the interpretation of these results.

2. Current use and impact of microwave sounding data in NWP

a. Use of microwave sounding data

1) Met Office

Data from AMSUs on board the National Oceanic and Atmospheric Administration (NOAA) Polar Orbiting Environmental System (POES) have been assimilated directly as brightness temperatures since 1999 and AMSU data from the first EUMETSAT polar platform (MetOp-A) have been assimilated since 2007. The Met Office currently uses data from AMSUs on board NOAA-16, NOAA-18, and MetOp-A, as well as data from F-16 SSMIS. Additional AMSU-A data are currently available from NOAA-15 and Earth Observing System (EOS) Aqua; however, previous OSEs at the Met Office showed that the effect of successive additions diminishes so that the measured impact of a third microwave sounder is relatively small, reducing SH forecast errors by 1% or less. The value of additional sounders therefore lies in adding increased robustness to the satellite observing system.

Prior to assimilation, AMSU data are preprocessed by the Advanced Operational Vertical Sounder (ATOVS) and Advanced Very High Resolution Radiometer (AVHRR) Preprocessing Package (AAPP; Atkinson et al. 2008). As part of this step, the AMSU-A data are remapped to the grid of the High Resolution Infrared Radiation Sounder (HIRS) using bilinear interpolation. This has the effect of reducing the effective noise of the AMSU data by approximately 32%. Remapping of this kind can be viewed as a special case of averaging over a spatial domain containing a number of observations; in this case, the four points define the observation grid box, which contains the remapped location. Errors in the values at corners of the grid box add as random variables in the interpolation, which is, in effect, a weighted summation. For example, in the special case of a remapped location at the center of a square grid box, the weights given to the values at the four corners are equal and the effective noise is reduced by a factor of 2. The NEΔT figures for the remapped data are shown in Table 1 for the tropospheric sounding channels (4–8). The noise for the remapped AMSU-A data is in the range 0.08–0.12 K for these channels. NEΔTs are derived from ensembles of measurements of the warm calibration load, as described in Atkinson and McLellan (1998).

Residual biases between observations and radiances modeled from forecast model fields are minimized using an offline bias correction scheme (Harris and Kelly 2001). These biases arise from a number of sources, including forecast model error, radiative transfer modeling error, and systematic errors in the radiometric calibration of the measured radiances. Bias corrections perform well when dealing with simple errors that show a high correlation with auxiliary data: for example, scan angle, latitude, or air mass. The widespread use and success of bias correction schemes at NWP centers partially alleviate the need for highly accurate absolute calibration, although this issue remains important for climate monitoring applications.

Data are thinned to one observation every 154 km, an empirically tuned thinning distance aimed at minimizing the detrimental effect of spatially correlated observation errors (Dando et al. 2007). The observation errors assumed for AMSU-A sounding channels, which determine the weight given to the observations in the four-dimensional variational data assimilation (4DVAR) analysis (Rawlins et al. 2007), are shown in Table 1. For channels 5–8, with weighting function peaks that span the troposphere and have the largest impact on forecast accuracy, the assumed observation errors are 0.25 K. These errors are empirically determined as the values that optimize forecast skill. The values are generally larger than the real measurement error. This inflation partly compensates for the effect of correlated observation errors, which are not explicitly treated in the Met Office 4DVAR system. The inflation also compensates for erroneously large assumed background errors; because the analysis state is determined by the relative weights of observation and background errors, any overestimation of the background error needs to be matched by observation error inflation to obtain an optimal analysis state.

The specified observation errors for the Met Office and ECMWF OSEs were derived independently. In the Met Office approach, the assumption was made that the largest component of the observation error is that resulting from the finite NEΔT (inflated to compensate for an overestimation of the background error). Thus, the approximate doubling of the NEΔT for the key AMSU-A channels led to a choice of doubled assumed observation errors.

2) ECMWF

AMSU data from the NOAA POES platforms have been assimilated, in the form of brightness temperatures, since 1998. ECMWF currently uses data from AMSUs on board NOAA-15, NOAA-16, NOAA-18, MetOp-A, and Aqua. F-16 SSMIS temperature sounding data are not currently used in the ECMWF system because of the persistence of local biases in the data, which remain after preprocessing and bias correction.

The data are not preprocessed; consequently, the effective noise levels in the AMSU data are higher than those shown for the Met Office remapped data in Table 1. The noise levels, first-guess departure (O-FG) statistics, and assumed observation errors for the ECMWF AMSU data are shown in Table 2. The data are thinned to one observation every 120 km. For channels 5–8, the assumed observation errors for normal AMSU-A data (channels 5–8) are 0.35 K. As described previously, these observation errors are tuned empirically to optimize forecast skill. For the noisy experiments, the observation errors were modified by adding the additional NEΔT noise, in quadrature, to the normal assumed observation errors.

Bias correction is carried out using a variational bias correction scheme (Auligné et al. 2007). In this approach, a predictor–corrector scheme is used similar to that described previously; however, the bias coefficients in the correction scheme are part of the control variable in the variational analysis and are dynamically updated each analysis cycle.

b. Impact of microwave sounding data

The impact of all microwave sounding data in the Met Office NWP system, as it stood in 2007, was assessed by a data-denial OSE in which all microwave sounding data were withdrawn from an otherwise full operational data assimilation system. This full system included conventional data types (from sondes, surface stations, buoys, and aircraft) as well as a range of satellite data types, including advanced IR sounder data from AIRS and IASI, data from a constellation of global positioning system radio occultation (GPSRO) sensors, scatterometer data from Quick Scatterometer (QuikSCAT) and Earth Resource Satellite-2 (ERS-2), SSM/I ocean surface wind speeds as well as atmospheric motion vectors (AMVs) from geostationary satellites.

Forecasts were verified relative to observations from sondes and surface observing stations. Typically, data from 20 000–30 000 global surface observations and 500–1000 sonde launches are used in the verification of each forecast day of the 32-day experiment, which covered the period 24 May–24 June 2007. Figure 1 shows the impact of removing microwave sounding data from NOAA-16, NOAA-18, MetOp-A, and F-16 SSMIS. For comparison, Fig. 1 shows the impacts from a previous experiment carried out in 2003, before AIRS, IASI, and GPSRO data were introduced.

The withdrawal of microwave sounding data increased forecast errors in the Southern Hemisphere by around 40% in 2003. In 2007, the impact is smaller, although it is still significant and important at around 10%–15%. The change is principally due to the introduction of AIRS and IASI data in the 2007 experiment. These results supported the inclusion of advanced IR sounding data in the Met Office control experiments for the OSEs aimed at establishing the sensitivity of forecast accuracy to radiometric performance to represent the global observing system as it is likely to stand in 2020.

3. Observing system experiments

a. Met Office

A series of OSEs was carried out to meet the primary aim of this work: to determine the sensitivity of forecast performance to the radiometric sensitivity of microwave sounding data. Ideally, a number of OSEs should be run for various levels of noise and the forecast performance should be assessed to fully determine the relation between the two; however, the computational expense of running OSEs for near-full operational resolutions using 4DVAR makes this impractical. The experiments were run for both a single AMSU and a single AMSU with degraded noise performance.

The Met Office OSEs covered the period 24 May–24 June 2007 and are summarized in Table 3. The data-denial experiment, described earlier, in which all MWS data were withdrawn from an otherwise full operational system was used as the control (CNTRL-UK), against which the other experiments were verified. This control experiment included data from four COSMIC GPSRO satellites giving 250–350 occultations per 6-h assimilation cycle. In the first experiment (EXPT1-UK), data from MetOp-A AMSU (normal AMSU hereafter) were added on top of the control experiment. In the second experiment (EXPT2-UK), data from MetOp-A AMSU were used that had synthetic, unbiased, and uncorrelated Gaussian noise added (hereafter, noisy AMSU). During the 32-day experiment, the total number of AMSU-A observations (footprints) used was 1.4 million. The data used are channel dependent and ranged from 0.6 million for channel 5 to 1.4 million for channel 8.

In determining the amplitude of the noise to be added, several factors were borne in mind. First, NEΔT figures were available from preliminary post-EPS instrument designs for both conical and cross-track scanners. These were in the range 0.1–0.6 K for the 50-GHz channels, depending on channel bandwidth and scan geometry. Initially, this supported the choice of a high value for the effective radiometric noise level (NEΔT′) for the noisy experiments (e.g., 0.5 K). Inspection of the first-guess departure (also known as innovation) statistics (Fig. 2), however, shows that AMSU is normally correcting relatively small (∼0.1 K) errors in the background field. For most of the time, innovations for AMSU channels 4–8 are in the range 0.1–0.2 K. Errors of 0.3 K or more are relatively infrequent. Adding synthetic noise to give NEΔT′ of 0.5 K would make these errors difficult to correct. Indeed, over several assimilation cycles, the quality of the background field would deteriorate. Anticipating that introducing synthetic noise to give NEΔT′ of 0.5 K would have very dramatic negative effects on forecast quality, it was decided to add noise to achieve NEΔT′ of 0.2 K, as a test of a realistic relaxation of the radiometric specifications of a microwave sounder, relative to current AMSU-A performance. Synthetic noise was generated such that the quadrature addition of the new noise and the noise of the remapped AMSU data was approximately 0.2 K for the key 50-GHz channels (5–9). This amounted to adding noise with a standard deviation of 0.17 K. This noise was added to all AMSU channels.

b. ECMWF

The ECMWF observing system experiments also covered the period 24 May–24 June 2007. Following previous practice at ECMWF, the control experiment (CNTRL-ECMWF) used conventional observations and a very limited set of satellite data (including only AMVs; Kelly et al. 2007). Notably, no data from AIRS and IASI were used in the control experiment. Following the convention described, the first experiment (EXPT1-ECMWF) used data from a single AMSU-A (from NOAA-18), and the second experiment (EXPT2-ECMWF) data from a noise-degraded NOAA-18 AMSU were introduced. No GPSRO measurements were included in the ECMWF experiments.

For approximate consistency with the Met Office experiments, the noise amplitude was such that the resulting effective noise (NEΔT′), if the data were remapped to the HIRS grid, would be 0.2 K. For the ECMWF data, this meant achieving NEΔT′ ≈ 0.29 K for the unmapped data, necessitating the addition of random noise with a standard deviation of 0.26 K. The bias-corrected innovations for normal and noise-degraded AMSU-A data in channels 1–15 are shown in Fig. 2. For the 32-day experiment, 0.7 million AMSU-A observations were used, ranging from 0.5 million for channel 5 to 0.7 million for channel 8.

c. Results: Analysis impacts

It was anticipated that the addition of unbiased random noise to the measured radiances would not result in any systematic difference in the resulting analysis compared to the analysis where normal radiances were used. Figure 3 shows the mean difference in analysis fields for temperature at 200, 500, and 850 hPa between normal and noisy experiments at ECMWF (EXPT1-EC and EXPT2-EC, respectively). The differences are generally below 0.3 K. There are several areas where systematic differences over large areas are evident. In the polar regions, differences of 0.2–0.25 K are evident. Systematic differences are also evident in the northeast Pacific and in the southern Atlantic. Elsewhere, the differences between normal and noisy experiments appear small scale and random.

The addition of noise would, however, be expected to make the resulting analysis more noisy. The increase in analysis noise was estimated by evaluating root-mean-square errors (RMSEs) for the normal and noisy analysis, using the analysis produced from the full system (EXPT3-EC) as a proxy for truth. Zonal-mean RMS errors in temperature at six pressure levels in the range 10–850 hPa, spanning the middle stratosphere to lower troposphere, are shown in Fig. 4 for normal and noisy experiments. At all levels, the analysis degradation is most evident in the Southern Hemisphere. The difference in analysis error between normal and noisy experiments increases southward of 20°, reaching maximums of 50, 20, and 50 mK at the 200-, 500-, and 850-hPa levels in the latitude range 60°–80°S, where analysis errors for the normal experiment are 0.5, 0.4, and 0.65 K, respectively. Analysis errors in the Southern Hemisphere are typically twice the magnitude of the analysis errors in the Northern Hemisphere (NH) for the tropospheric levels. Analysis errors are larger in the stratosphere and the interhemispheric differences are larger, with Southern Hemisphere errors typically 2–4 times larger than the Northern Hemisphere values. Analysis degradations resulting from noise addition are smaller in magnitude in the tropics (TR) and Northern Hemisphere, but they are detectable.

The analysis errors estimated in this way are necessarily an underestimate of the true errors, because the proxy for truth (the analyses from the full system) itself has nonzero errors. The analysis errors reported here are therefore lower limits to the true errors. Any approximate estimates of forecast sensitivity to analysis errors derived from this study would therefore represent upper limits to true sensitivity.

d. Results: Forecast impacts

For the Met Office OSEs, forecast fields were verified against radiosonde observations for temperature, relative humidity, geopotential height, and winds and are relative to surface based observations for mean sea level pressure. As summarized in Table 4, the verification was carried out at forecast ranges from day 1 to day 6 and for a range of pressure levels. The RMS observed minus forecast differences were then used to compute changes in the mean RMSEs for each experiment relative to the specified control experiment. The ECMWF OSEs were verified against analysis fields from a reference experiment, which included a full set of observations, sampled on a 2.5° grid. The verification measures used matched those of the Met Office, with a few minor exceptions: (i) for technical reasons, 300-hPa pressure level scores were used instead of the 250-hPa scores used by the Met Office and (ii) relative humidity scores were also verified for the 300-, 100- and 50-hPa pressure levels and for days 4–6. Neither of these differences is expected to significantly change the results reported here. Verification statistics were computed for NH, the tropics, and SH.

1) Met Office

Figure 5 shows the forecast improvement for NH, the tropics, and SH for the Met Office OSEs for both normal and noisy AMSU experiments. Each circle represents a verification measure averaged over the period of the OSEs. The impact of the AMSU data is greatest for the mass-related fields (temperature, geopotential height, and mean sea level pressure) and is smaller for winds and humidity. The impacts are also largest in the SH, for which the analysis is constrained by a relatively sparse network of conventional observations.

The changes in mean forecast RMSEs, averaged over all 123 verification measures listed in Table 4, are summarized in Table 5. The impacts of the data are small in the NH, generally less than 5% (with an overall mean RMSE reduction of 1.13% for the normal data), because of the abundance of conventional observations available for the analysis, as well as the presence of both AIRS and IASI data. The impact of the data in the tropics is similarly small, at less than 5% (overall mean RMSE reduction of 0.03%), which has been observed in previous OSEs and is attributed to tropical meteorology being governed less by geostrophic balance (which benefits from accurate analyses of mass fields from temperature sounders) and more by convection (less well analyzed by microwave sounding measurements). The relative change in the Met Office TR scores are bracketed, because these results are misleading as a result of the change being expressed relative to a near-zero impact for the normal experiment. In the SH, the impact of the data is significant with mean forecast errors, averaged over all verification measures, reduced by 5.4% for the normal AMSU data. The mean reduction in forecast error for the noisy AMSU experiment was 4.8%, representing a relative reduction in forecast improvement of 11%.

2) ECMWF

Figure 6 shows the forecast improvement for the ECMWF OSEs for both normal and noisy AMSU experiments. The impacts in the NH and the tropics are smaller than in the SH, but they are still statistically significant (at the 95% confidence level) at −4.6% and −4.8%, respectively, for the normal AMSU data, reducing to −3.3% and −3.6%, respectively, for the noisy data. The impact in the SH is larger. For the normal AMSU experiments, mean forecast errors are reduced by 22.3%. The noisy AMSU experiment shows a reduction of 19.7%, representing a relative reduction in forecast accuracy of 11%. This value is close to the value obtained in the Met Office experiments and indicates the robustness of the results. The ECMWF OSEs show a significant degradation of forecast quality in all three regions, at a similar level of significance. Comparison (ECMWF versus Met Office) of the relative change in forecast performance in the tropics is difficult because of the very small impact of the AMSU-A data in the Met Office experiment. This is probably as a result of the influence of AIRS and IASI in the experiments reducing a normally small impact to near zero. In the Northern Hemisphere, the derived sensitivity for the ECMWF experiment (−27.9% ± 10.1%) appears higher than for the Met Office experiment (−10.6% ± 16.0%), although the uncertainties overlap. It might be expected that, as the impact of the normal AMSU data approaches zero, in this case because of the strong influence of an otherwise full observing system in the Met Office experiment, the sensitivity of analyses and forecasts to increased noise reduces. It is worth emphasizing here that the larger absolute impacts on forecasts in the ECMWF experiments are due to the absence of AIRS/IASI from the control experiment.

3) General points

Figures 5 and 6 show the same qualitative behavior in that the largest forecast improvements are found for mass fields (temperature, geopotential height, and mean sea level pressure) with less impact on wind fields and even less impact for humidity fields. This holds true for both tropics and extratropics but with the largest impacts found for the SH.

Figures 5 and 6 represent overall summary plots of the impact of noisy radiance data on a range of verification measures and address the central aim of this study. Examining the verification data in more detail yields useful insights, which may aid the design of similar experiments in future. Figure 7 shows the verification for geopotential height for 200, 500, and 850 hPa for forecast ranges from T + 12 h to T + 6 days in the Southern Hemisphere, in this case from the ECMWF OSEs. The error bars represent the standard error on the mean RMSE reduction (at 1σ) for both normal and noisy experiments and indicate the significance of departures from the 45° line. Figure 7 shows that the largest forecast impacts are obtained at shortest range and that the impact decreases monotonically to the longest forecast range. Absolute forecast impacts are still significant at T + 6 days. The significance of the differences in forecast impact becomes marginal beyond T + 4 days, because the statistical uncertainties in the verification measures becomes larger at longer range. Very similar results are obtained for temperature and wind.

Forecast verification for geopotential height at 10, 50, and 100 hPa is shown in Fig. 8. Error bars are not shown if they lie within the marker circle. Consistent with the results obtained for the tropospheric levels, the forecast impact is largest (and larger than for the tropospheric levels) for the shortest range, decreasing monotonically with forecast range. The uncertainties in the verification are smaller than for the tropospheric levels, enabling the detection of significant degradation at all forecast ranges to T + 6 days.

The impact on tropospheric relative humidity in the Southern Hemisphere is shown in Fig. 9. The impact of the MWS on humidity scores is smaller, at 20% for the normal data at T + 12 h. The absolute impacts decrease with time for both normal and noisy experiments but are still significant at T + 6 days. The degradation in the impact resulting from the noisy data is most evident at 500 hPa, where the degradation is significant to forecast day 4. Significant degradations are evident to day 3 at 850 hPa, but the degradations at 200 hPa are more marginal in significance.

Figure 10 shows the impact of normal and noisy data on the analyses and forecast fields for vector winds in the tropics (20°N–20°S). In this case, the analyses and forecast fields have been verified against radiosonde observations. The impact of the MWS at 850 hPa is small, at around 1% or less, and of marginal significance. Forecast impacts are larger at 500 and 200 hPa at up to 3% for both normal and noisy data, with the largest impacts at short range (T + 24 and T + 48 h). There is some indication overall that the noisy data reduce the forecast improvement relative to the noisy data, with the most significant results at T + 24 h for 200 and 500 hPa.

The difference in absolute SH impacts of a single AMSU in the Met Office (5.4%) and ECMWF (22.3%) OSEs is explained by the presence of AIRS and IASI in the Met Office control experiment, and it is broadly consistent with the Met Office pre- and postadvanced IR sounder MWS data-denial experiments, which show a factor of 4 reduction in the relative importance of MWS data. This is a significant result that highlights the high value of the infrared data in NWP.

4. Summary and conclusions

This work was prompted by the requirement to specify the radiometric performance of microwave sounders for future meteorological satellite missions. The approach involved assessing the sensitivity of NWP forecast accuracy to the noise level (NEΔT) for AMSU data through a series of OSEs at the Met Office and ECMWF. Normal AMSU data and noisy AMSU data, in which synthetic noise was added, were introduced into global 4DVAR NWP systems at both the Met Office and ECMWF.

This study has two main conclusions. First, it has been confirmed that forecast improvements are measurably reduced (by ∼11%) for relatively small degradations (increasing NEΔT from 0.1 to 0.2 K for remapped AMSU-A data) in the radiometric performance of microwave temperature sounding data. Second, the impact of MWS data in the postadvanced IR sounder era, although still very significant at around 10% in most SH forecast scores, is significantly less than the impact during the preadvanced IR sounder era, when the impact was around 40%.

Therefore, it would appear that, if the continued steady improvement in NWP forecast accuracy is to be maintained, then any degradation in forecast performance resulting from the choice of a lower-specification MWS instrument, relative to the current operational baseline, would have to be offset by improvements elsewhere in the system. This statement, however, needs to be qualified and the limitations of this work should be made clear.

This study aimed at establishing forecast sensitivity to radiometric sensitivity in experiments that used AMSU data in the same way the data are currently exploited at operational NWP centers. In particular, the data are not averaged (other than the averaging achieved through remapping) and are spatially thinned. No account has been taken of the possibilities for spatially averaging high noise raw data to achieve acceptable radiometric noise levels. Experience in spatially averaging microwave sounding data is limited. In the operational exploitation of SSMIS data, the Met Office and the Naval Research Laboratory spatially averaged the radiance data to achieve noise levels below 0.1 K for the 50-GHz channels, but no systematic study was undertaken to compare the impacts of averaged data versus unaveraged data. Data from ATMS, due for first launch in 2009, are oversampled at relatively high noise (NEΔT = 0.75 K at 54.4 GHz), and spatial averaging will be a necessary step in the preprocessing of the data for NWP applications (Atkinson et al. 2008).

A second qualification relates to other potential improvements in the exploitation of satellite data, which could modify the sensitivities derived here. Likely developments by 2020 include a greater use of sounding data over land, a greater use of data from cloudy and precipitating regions, improvements in the definition of observation and background errors (including better treatment of observation error correlations), and the implementation of more intelligent thinning schemes.

Regarding the design of similar observing system experiments in the future, significant differences in the tropospheric geopotential height verification scores resulting from the addition of noise are only detectable to forecast day 4 of a 32-day sample period. The expense of future experiments could therefore be reduced by reducing the forecast range to 4 days.

Finally, this study assesses the cost of degrading radiometric performance of an MWS mission. It should also be possible in the future to assess the potential benefits to be achieved through improving radiometric noise performance by averaging data and increasing the weight given to observations in the analysis.

Acknowledgments

This work was supported by the European Space Agency (Contract ESTEC 20711/07/NL/HE), and the authors thank the project team from ESA and EUMETSAT for their keen interest in the project. The authors would also like to thank Jean-Noël Thépaut for his help in formulating the project and for constructive comments on the paper and Alan Geer for the verification software used in the study.

REFERENCES

  • Atkinson, N., and McLellan S. , 1998: Initial evaluation of AMSU-B in-orbit data. Models and Retrieval Techniques, T. Hayasaka et al., Eds., International Society for Optical Engineering (SPIE Proceedings, Vol. 3503), 276–287.

    • Search Google Scholar
    • Export Citation
  • Atkinson, N., Brunel P. , Marguinaud P. , and Labrot T. , 2008: AAPP developments and experiences with processing MetOp data. Proc. 16th Int. TOVS Study Conf., Angra Dos Reis, Brazil, Int. TOVS Working Group, 1.3.

    • Search Google Scholar
    • Export Citation
  • Auligné, T., McNally A. P. , and Dee D. P. , 2007: Adaptive bias correction for satellite data in a numerical weather prediction system. Quart. J. Roy. Meteor. Soc., 133 , 631642.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bell, W., and Coauthors, 2008: The assimilation of SSMIS radiances in numerical weather prediction models. IEEE Trans. Geosci. Remote Sens., 46 , 884900.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Challon, G., Cayla F. , and Diebel D. , 2001: IASI: An advanced sounder for operational meteorology. Proc. 52nd Congress of IAF, Toulouse, France, Int. Astronautical Federation, 1–9.

    • Search Google Scholar
    • Export Citation
  • Dando, M. L., Thorpe A. J. , and Eyre J. R. , 2007: The optimal density of atmospheric sounder observations in the Met Office NWP system. Quart. J. Roy. Meteor. Soc., 133 , 19331943.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • English, S. J., Saunders R. , Candy B. , Forsythe M. , and Collard A. , 2004: Met Office satellite data OSEs. Proc. Third WMO Workshop on the Impact of Various Observing Systems on NWP, Alpbach, Austria, WMO, 146–156.

    • Search Google Scholar
    • Export Citation
  • Eyre, J. R., and Lafeuille J. , 2008: Evolution of the Global Observing System: A vision for 2025. Proc. 16th Int. TOVS Study Conf., Angra Dos Reis, Brazil, Int. TOVS Working Group, 1–38.

    • Search Google Scholar
    • Export Citation
  • Goodrum, G., Kidwell K. B. , and Winston W. , 2000: NOAA KLM user’s guide. NOAA Satellite and Information Service. [Available online at http://www.ncdc.noaa.gov/oa/pod-guide/ncdc/docs/klm/index.htm].

    • Search Google Scholar
    • Export Citation
  • Harris, B. A., and Kelly G. , 2001: A satellite radiance bias correction scheme for data assimilation. Quart. J. Roy. Meteor. Soc., 127 , 14531468.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kelly, G., Thépaut J-N. , Buizza R. , and Cardinali C. , 2007: The value of targeted observations—Part I: Data denial experiments for the Atlantic and the Pacific. ECMWF Tech. Memo. 511, 29 pp.

    • Search Google Scholar
    • Export Citation
  • Le Marshall, J., and Coauthors, 2006: Improving global analysis and forecasting with AIRS. Bull. Amer. Meteor. Soc., 87 , 891894.

  • McMillin, L., and Divakarla M. G. , 1999: Effects of possible scan geometries on the accuracy of satellite measurements of water vapor. J. Atmos. Oceanic Technol., 16 , 17101720.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Muth, C., Webb W. , Atwood W. , and Lee P. , 2005: Advanced technology microwave sounder on the National Polar-Orbiting Operational Environmental Satellite System. Proc. Int. Geoscience and Remote Sensing Symp., Seoul, South Korea, IEEE, 99–102.

    • Search Google Scholar
    • Export Citation
  • Rawlins, F., Ballard S. P. , Bovis K. J. , Clayton A. M. , Li D. , Inverarity G. W. , Lorenc A. C. , and Payne T. J. , 2007: The Met Office global four-dimensional variational assimilation scheme. Quart. J. Roy. Meteor. Soc., 133 , 347362.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rosenkranz, P. W., Hutchison K. D. , Hardy K. R. , and Davis M. S. , 1997: An assessment of the impact of satellite microwave sounder incidence angle and scan geometry on the accuracy of atmospheric temperature profile retrievals. J. Atmos. Oceanic Technol., 14 , 488494.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Swadley, S., Poe G. A. , Bell W. , Hong Y. , Kunkee D. B. , McDermid I. S. , and Leblanc T. , 2008: Analysis and characterization of the SSMIS upper atmospheric sounding channel radiances. IEEE Trans. Geosci. Remote Sens., 46 , 962983.

    • Crossref
    • Search Google Scholar
    • Export Citation

Fig. 1.
Fig. 1.

Met Office data-denial OSEs in which MWS data were withdrawn from an otherwise full observing system that contained advanced IR sounder data (AIRS and IASI) in 2007 but not in 2003.

Citation: Journal of Atmospheric and Oceanic Technology 27, 3; 10.1175/2009JTECHA1293.1

Fig. 2.
Fig. 2.

First-guess departures for AMSU-A channels 5–14 for normal (black) and noisy (gray) data used in the ECMWF OSEs.

Citation: Journal of Atmospheric and Oceanic Technology 27, 3; 10.1175/2009JTECHA1293.1

Fig. 3.
Fig. 3.

The mean difference in analysis temperature fields at 200, 500 and 850 hPa between normal and noisy AMSU experiments.

Citation: Journal of Atmospheric and Oceanic Technology 27, 3; 10.1175/2009JTECHA1293.1

Fig. 4.
Fig. 4.

Estimated analysis accuracy, expressed as a zonal-mean RMSE, for temperature at (top)–(bottom) 10, 50, 100, 200, 500 and 850 hPa for normal (dotted line) and noisy (solid line) AMSU data OSEs.

Citation: Journal of Atmospheric and Oceanic Technology 27, 3; 10.1175/2009JTECHA1293.1

Fig. 5.
Fig. 5.

Met Office OSEs: the impact on forecast quality, in terms of RMSE reductions for all verification measures listed in Table 4, resulting from the addition of normal and noisy AMSU-A data to a baseline experiment (CNTRL-UK) in which all microwave sounding data have been withdrawn. The x and y coordinates of each point represent the change in forecast error (RMSE) relative to a no-MWS baseline experiment (CNTRL-EC) for noisy and normal AMSU experiments, respectively. For example, points in the lower-left quadrant indicate that both normal and noisy experiments reduce forecast errors, but points below the dotted 45° line indicate the noisy data reduce errors less than the normal data.

Citation: Journal of Atmospheric and Oceanic Technology 27, 3; 10.1175/2009JTECHA1293.1

Fig. 6.
Fig. 6.

ECMWF OSEs: the impact on forecast quality, in terms of RMSE reductions for all verification measures listed in Table 4, resulting from the addition of normal and noisy AMSU-A data to a baseline experiment (CNTRL-EC) in which all satellite data (except AMV data) have been withdrawn.

Citation: Journal of Atmospheric and Oceanic Technology 27, 3; 10.1175/2009JTECHA1293.1

Fig. 7.
Fig. 7.

The effect of adding normal and noisy AMSU data on geopotential height forecast accuracy (RMSE) at 850, 500, and 200 hPa for forecast ranges 12–144 h in the SH for the ECMWF OSEs. The error bars represent the standard error on the mean (at 1σ) forecast error change over the 32-day experiment.

Citation: Journal of Atmospheric and Oceanic Technology 27, 3; 10.1175/2009JTECHA1293.1

Fig. 8.
Fig. 8.

As in Fig. 7, but at 100, 50, and 10 hPa.

Citation: Journal of Atmospheric and Oceanic Technology 27, 3; 10.1175/2009JTECHA1293.1

Fig. 9.
Fig. 9.

As in Fig. 7, but on relative humidity forecast accuracy (RMSE).

Citation: Journal of Atmospheric and Oceanic Technology 27, 3; 10.1175/2009JTECHA1293.1

Fig. 10.
Fig. 10.

The effect of adding normal and noisy AMSU data on vector wind forecast accuracy (RMSE) at 850, 500, and 200 hPa, verified using radiosondes, for analysis time (T + 0) and forecast ranges from 24 to 144 h in the tropics for the ECMWF OSEs. Error bars as in Fig. 7.

Citation: Journal of Atmospheric and Oceanic Technology 27, 3; 10.1175/2009JTECHA1293.1

Table 1.

Met Office: AMSU-A channel characteristics, O-FG statistics, and assumed observation errors (R). The departure statistics were derived from data processed during a single 6-h assimilation cycle using MetOp-A AMSU-A. The final column shows the number of observation used in generating the statistics.

Table 1.
Table 2.

ECMWF: AMSU-A channel characteristics, O-FG statistics, and assumed observation errors (R). The departure statistics were derived from data processed during a 2-week period using NOAA-18 AMSU-A. The final column shows the number of observations used in generating the statistics.

Table 2.
Table 3.

OSEs at the Met Office and ECMWF.

Table 3.
Table 4.

Verification measures used to assess forecast accuracy.

Table 4.
Table 5.

A summary of the impact of increased MWS radiometric noise on NWP forecast accuracy.

Table 5.
Save