• Andersson, E., and H. Jarvinen, 1999: Variational quality control. Quart. J. Roy. Meteor. Soc., 125, 697722, doi:10.1002/qj.49712555416.

  • Andersson, E., C. Cardinali, B. Truscott, and T. Hovberg, 2005: High frequency AMDAR data—A European aircraft data collection trial and impact assessment. ECMWF Tech. Memo. 457, 15 pp. [Available online at https://www.ecmwf.int/en/elibrary/7765-high-frequency-amdar-data-european-aircraft-data-collection-trial-and-impact.]

  • Benjamin, S. G., B. D. Jamison, W. R. Moninger, S. R. Sahm, B. E. Schwartz, and T. W. Schlatter, 2010: Relative short-range forecast impact from aircraft, profiler, radiosonde, VAD, GPS-PW, METAR, and mesonet observations via the RUC hourly assimilation cycle. Mon. Wea. Rev., 138, 13191343, doi:10.1175/2009MWR3097.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cardinali, C., 2009: Monitoring the observation impact on the short-range forecast. Quart. J. Roy. Meteor. Soc., 135, 239250, doi:10.1002/qj.366.

  • Duan, J., and et al. , 1996: GPS meteorology: Direct estimation of the absolute value of precipitable water. J. Appl. Meteor., 35, 830838, doi:10.1175/1520-0450(1996)035<0830:GMDEOT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dworak, R., and R. Petersen, 2013: The validation of GOES-LI and AIRS total precipitable water retrievals using ground based measurements. Joint EUMETSAT Meteorological Satellite Conf./19th Conf. on Satellite Meteorology, Oceanography, and Climatology, Vienna, Austria, EUMETSAT–Amer. Meteor. Soc. [Available online at https://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=PDF_CONF_P_S7_03_DWORAKRevisionSelectionMethod=LatestReleased&Rendition=Web.]

  • Fleming, R. J., 1996: The use of commercial aircraft as platforms for environmental measurements. Bull. Amer. Meteor. Soc., 77, 22292242, doi:10.1175/1520-0477(1996)077<2229:TUOCAA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fleming, R. J., 1998: A note on temperature and relative humidity corrections for humidity sensors. J. Atmos. Oceanic Technol., 15, 15111515, doi:10.1175/1520-0426(1998)015<1511:ANOTAR>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gelaro, R., R. H. Langland, S. Pellerin, and R. Todling, 2010: The THORPEX Observation Impact Intercomparison Experiment. Mon. Wea. Rev., 138, 40094025, doi:10.1175/2010MWR3393.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gutman, S. I., S. R. Sahm, S. G. Benjamin, B. E. Schwartz, K. L. Holub, J. Q. Stewart, and T. L. Smith, 2004: Rapid retrieval and assimilation of ground based GPS precipitable water observations at the NOAA Forecast Systems Laboratory: Impact on weather forecasts. J. Meteor. Soc. Japan, 82, 351360, doi:10.2151/jmsj.2004.351.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hammersley, J. M., and D. C. Handscomb, 1975: Monte Carlo Methods. Methuen and Co., 178 pp.

  • Heideman, K. F., and J. M. Fritsch, 1988: Forcing mechanisms and other characteristics of significant summertime precipitation. Wea. Forecasting, 3, 115130, doi:10.1175/1520-0434(1988)003<0115:FMAOCO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Helms, D., A. Hoff, H. G. J. Smit, S. Taylor, S. Carlberg, and M. Berechree, 2010: Advancements in the AMDAR humidity sensing. Tech. Conf. on Meteorological and Environmental Instruments and Methods of Observation, Helsinki, Finland, WMO, 2.1. [Available online at http://www.wmo.int/pages/prog/www/IMOP/publications/IOM-104_TECO-2010/2_1_Helms_USA.doc.]

  • Hogan, T. F., and et al. , 2014: The Navy Global Environmental Model. Oceanography, 27, 116125, doi:10.5670/oceanog.2014.73.

  • Kleist, D. T., and K. Ide, 2015: An OSSE-based evaluation of hybrid variational–ensemble data assimilation for the NCEP GFS. Part I: System description and 3D-hybrid results. Mon. Wea. Rev., 143, 433451, doi:10.1175/MWR-D-13-00351.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kleist, D. T., D. F. Parrish, J. C. Derber, R. Treadon, W.-S. Wu, and S. Lord, 2009: Introduction of the GSI into the NCEP Global Data Assimilation System. Wea. Forecasting, 24, 16911705, doi:10.1175/2009WAF2222201.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Langland, R. H., and N. L. Baker, 2004: Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus, 56A, 189201, doi:10.3402/tellusa.v56i3.14413.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leblanc, T., and et al. , 2011: Measurements of Humidity in the Atmosphere and Validation Experiments (MOHAVE)-2009: Overview of campaign operations and results. Atmos. Meas. Tech., 4, 25792605, doi:10.5194/amt-4-2579-2011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moninger, W. R., R. D. Mamrosh, and P. M. Pauley, 2003: Automated meteorological reports from commercial aircraft. Bull. Amer. Meteor. Soc., 84, 203216, doi:10.1175/BAMS-84-2-203.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moninger, W. R., S. G. Benjamin, B. D. Jamison, T. W. Schlatter, T. L. Smith, and E. J. Szoke, 2010: Evaluation of regional aircraft observations using TAMDAR. Wea. Forecasting, 25, 627645, doi:10.1175/2009WAF2222321.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Olson, D. A., N. W. Junker, and B. Korty, 1995: Evaluation of 33 years of quantitative precipitation forecasting at NMC. Wea. Forecasting, 10, 498511, doi:10.1175/1520-0434(1995)010<0498:EOYOQP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ota, Y., J. C. Derber, E. Kalnay, and T. Miyoshi, 2013: Ensemble-based observation impact estimates using the NCEP GFS. Tellus, 65A, 20038, doi:10.3402/tellusa.v65i0.20038.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Petersen, R. A., 2016: On the impact and future benefits of AMDAR observations in operational forecasts. Part I: A review of the impact of automated aircraft wind and temperature reports. Bull. Amer. Meteor. Soc., 97, 585602, doi:10.1175/BAMS-D-14-00055.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Petersen, R. A., C. Dey, R. C. Martin, R. D. Londot, and G. T. Ligler, 1992: The Meteorological Data Collection and Reporting System (MDCRS): System overview and benefits. Proc. National Weather Service Aviation Workshop, Kansas City, MO, NWS, 251255.

  • Petersen, R. A., L. Cronce, R. Mamrosh, and R. Baker, 2015: Impact and benefits of AMDAR temperature, wind, and moisture observations in operational weather forecasting. WMO Tech. Rep. 2015-01, 93 pp. [Available online at http://library.wmo.int/pmb_ged/wigos-tr_2015-01_en.pdf.]

  • Petersen, R. A., L. Cronce, R. Mamrosh, R. Baker, and P. Pauley, 2016: On the impact and future benefits of AMDAR observations in operational forecasting. Part II: Water vapor observations Bull. Amer. Meteor. Soc., 97, 21172133, doi:10.1175/BAMS-D-14-00211.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schwartz, B., and S. G. Benjamin, 1995: A comparison of temperature and wind measurements from ACARS-equipped aircraft and rawinsondes. Wea. Forecasting, 10, 528544, doi:10.1175/1520-0434(1995)010<0528:ACOTAW>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, T. L., S. G. Benjamin, and S. I. Gutman, 2007: Short-range forecast impact from assimilation of GPS-IPW observations into the Rapid Update Cycle. Mon. Wea. Rev., 135, 29142930, doi:10.1175/MWR3436.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, X., D. Parrish, D. Kleist, and J. Whitaker, 2013: GSI 3DVar-based ensemble–variational hybrid data assimilation for NCEP Global Forecast System: Single-resolution experiments. Mon. Wea. Rev., 141, 40984117, doi:10.1175/MWR-D-12-00141.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D., 1995: Statistical Methods in the Atmospheric Sciences: An Introduction. Academic Press, 467 pp.

  • Zhu, Y., J. C. Derber, R. J. Purser, B. A. Ballish, and J. Whiting, 2015: Variational correction of aircraft temperature bias in the NCEP’s GSI analysis system. Mon. Wea. Rev., 143, 37743803, doi:10.1175/MWR-D-14-00235.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Profiles of observation characteristics for AMDAR and rawinsonde moisture observations. Mean profiles of specific humidity OMB (representing systemic bias in the 6-h moisture forecast), and RMSE for the (a) warm- and (b) cold-season experiments at rawinsonde launch sites. The solid (dashed) blue profile is the mean (RMSE) rawinsonde moisture OMB when AMDAR moisture observations are not assimilated. The red solid (dashed) profile is the mean (RMSE) rawinsonde moisture OMB when AMDAR moisture observations are assimilated. The green solid (dashed) profile is the mean (RMSE) AMDAR moisture OMB for assimilated AMDAR observations in the experiment. The shading around each mean profile represents the 5% and 95% confidence limits around the mean, and pressure levels where the mean rawinsonde OMB changes to statistical significance are highlighted with black squares along the ordinate. (c) Observation error (% RH) profiles for rawinsonde (blue) and AMDAR (red dashed) moisture observations.

  • View in gallery

    Precipitation skill and bias scores of 12–36-h forecasts over the continental United States for (a) warm- and (b) cold-season experiments. (left) ETS for precipitation binned by precipitation amounts [mm (24 h)−1]. (right) The precipitation bias score in the same bins. The black curve is for the control simulation, and the red curve is for the experiment. (bottom) The differences between the experiment and control, with bars indicating the minimum value necessary for 95% statistical significance.

  • View in gallery

    Error in forecast fit-to-TPW observations from GPS for (top) the April–May 2014 simulation and (bottom) December 2014–January 2015 experiment. Statistics for the control simulation are provided in blue, and statistics for the experiment are provided in red. Error is computed as (a),(d) bias of error, calculated as the mean of the difference between the forecast and observation; (b),(e) random error, calculated as the standard deviation of the error; and (c),(f) absolute error, calculated as the mean of the total (bias + random) error. Thick contours represent the sample mean or standard deviation, and the shading represents the 5% and 95% confidence limits on the mean or standard deviation. Dots are placed along the red contour for all times when the difference between the experiment and control is statistically significant based on a Student’s t test (for bias of error) or a chi-squared test on variance (for random error). RMSE is also plotted (dashed) in (b) and (e) for reference.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 161 160 21
PDF Downloads 102 102 22

Forecast Impact of Assimilating Aircraft WVSS-II Water Vapor Mixing Ratio Observations in the Global Data Assimilation System (GDAS)

View More View Less
  • 1 Cooperative Institute for Meteorological Satellite Studies, Space Science and Engineering Center, University of Wisconsin–Madison, Madison, Wisconsin
  • | 2 I. M. Systems Group, NOAA/NCEP/EMC, College Park, Maryland
© Get Permissions
Full access

Abstract

Automated aircraft observations of wind and temperature have demonstrated positive impact on numerical weather prediction since the mid-1980s. With the advent of the Water Vapor Sensing System (WVSS-II) humidity sensor, the expanding fleet of commercial aircraft with onboard automated sensors is also capable of delivering high quality moisture observations, providing vertical profiles of moisture as aircraft ascend out of and descend into airports across the continental United States. Observations from the WVSS-II have to date only been monitored within the Global Data Assimilation System (GDAS) without being assimilated. In this study, aircraft moisture observations from the WVSS-II are assimilated into the GDAS, and their impact is assessed in the Global Forecast System (GFS). A two-season study is performed, demonstrating a statistically significant positive impact on both the moisture forecast and the precipitation forecast at short range (12–36 h) during the warm season. No statistically significant impact is observed during the cold season.

Denotes content that is immediately available upon publication as open access.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Brett Hoover, brett.hoover@ssec.wisc.edu

Abstract

Automated aircraft observations of wind and temperature have demonstrated positive impact on numerical weather prediction since the mid-1980s. With the advent of the Water Vapor Sensing System (WVSS-II) humidity sensor, the expanding fleet of commercial aircraft with onboard automated sensors is also capable of delivering high quality moisture observations, providing vertical profiles of moisture as aircraft ascend out of and descend into airports across the continental United States. Observations from the WVSS-II have to date only been monitored within the Global Data Assimilation System (GDAS) without being assimilated. In this study, aircraft moisture observations from the WVSS-II are assimilated into the GDAS, and their impact is assessed in the Global Forecast System (GFS). A two-season study is performed, demonstrating a statistically significant positive impact on both the moisture forecast and the precipitation forecast at short range (12–36 h) during the warm season. No statistically significant impact is observed during the cold season.

Denotes content that is immediately available upon publication as open access.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Brett Hoover, brett.hoover@ssec.wisc.edu

1. Introduction

Automated observations of wind and temperature from commercial aircraft have become a significant source of observations, especially since the establishment of the Meteorological Data Collection and Reporting System (MDCRS; Petersen et al. 1992). Today, 39 participating airlines deploy more than 3500 aircraft under the World Meteorological Organization’s broader Aircraft Meteorological Data Relay (AMDAR) program, delivering more than 680 000 wind and temperature reports daily (Petersen et al. 2015).

Aircraft wind and temperature observations have demonstrated positive impact on numerical weather prediction since the mid-1980s when aircraft data became available in significant numbers (Moninger et al. 2003). Data-denial experiments in the Rapid Update Cycle model [RUC, replaced by the Rapid Refresh (RAP) model in 2012; Benjamin et al. (2010)] demonstrate that aircraft data are the most important source of information over the continental United States for 3–6-h RUC forecasts as well as 12-h forecasts of upper-tropospheric winds. Assimilation of wind, temperature, and moisture observations from tropospheric AMDAR (TAMDAR) observations in the RUC demonstrate the positive impact on wind, temperature, and moisture fields for the 3-h forecast (Moninger et al. 2010). Impact tests in the European Centre for Medium-Range Weather Forecasts (ECMWF) global forecast system demonstrate a positive impact at 48 h over the North Pacific, North America, the North Atlantic, and Europe when assimilating aircraft wind and temperature observations (Andersson et al. 2005). An experimental ensemble-based observation impact system used with the NCEP GFS demonstrated that aircraft wind and temperature observations supply the largest per observation impact on the 24-h forecast error of any in situ observation type, even surpassing rawinsonde (raob) observations (Ota et al. 2013). While it should be noted that some ensemble-based observation impact results differ significantly from observation impact metrics employing a model adjoint technique used at other centers (e.g., Gelaro et al. 2010), aircraft observations have demonstrated significant impact on the 24-h forecast error in adjoint-based investigations as well (Langland and Baker 2004; Cardinali 2009).

Early attempts to derive automated moisture observations from aircraft sensors included the Water Vapor Sensing System (WVSS), which used a thin-film capacitor to measure relative humidity (RH; Fleming 1996). Tests of the device indicated a wet bias at high RH values and a dry bias at low RH values (Fleming 1998). In addition, biases in AMDAR temperature reports made it difficult to retrieve precise values of moisture variables, such as specific humidity. The WVSS-II sensor was redesigned to use a tunable diode laser to measure water vapor content via infrared absorption spectroscopy, determining the water vapor content of sampled air from the measured transmittance of the laser across the air tube (Helms et al. 2010). Version 3 of the WVSS-II was developed in 2008 and performed well under most test conditions, eliminating technical issues with seals and thermal control that plagued the earlier versions of the design. The WVSS-II (v3) is the device currently on board over 100 aircraft, routinely producing approximately 65 000 moisture observations daily over the continental United States (Petersen 2016).

WVSS-II moisture observations from AMDAR have been assimilated into the North American Mesoscale Forecast System (NAM) Data Assimilation System (NDAS) since the 18 October 2011 upgrade (http://www.emc.ncep.noaa.gov/mmb/mmbpll/eric.html#TAB4), which included substantial modification of the model grid, model physics, and data assimilation. However, these observations have only been monitored within the GDAS, passing through the data assimilation system and being assigned an interpolated model background moisture value, but not actually being assimilated. These data are also used in the operational U.S. Navy global analysis and forecasts system [the Navy Global Environmental Model (NAVGEM), Hogan et al. (2014)] and have shown positive impact over the United States (Petersen et al. 2016). It is the goal of this study to assimilate these moisture observations in the GDAS and evaluate their impact on the GFS forecast. Section 2 outlines the model setup and experiment design, section 3 describes the methodology for assessing forecast impacts, the results are presented in section 4, and conclusions are provided in section 5.

2. Model setup and experiment design

Observations are assimilated using the hybrid ensemble–three-dimensional variational data assimilation (3DVAR) formulation of the GDAS to produce 6-hourly analyses. Analyses are produced at T670 resolution while using a set of 80 ensemble members at T254 resolution to define flow-dependent covariance (e.g., Wang et al. 2013). A 168-h forecast is initiated from every 0000 UTC analysis for the purposes of assessing the impact on the short- to medium-range forecasts. This experiment uses an early release version of the 2014 fiscal year (FY14) operational GDAS with the semi-Lagrangian dynamics core,1 processing all routinely assimilated observations in the NCEP prepBUFr files made available directly on EMC computing platforms, using all operational bias corrections as of FY14, and using the Gridpoint Statistical Interpolation (GSI) analysis system (Kleist et al. 2009). The model was cycled using the hybrid ensemble–3DVAR technique described in Kleist and Ide (2015).

The GDAS was cycled for a warm season (1 April–29 May 2014) and a cold season (1 December 2014–11 January 2015) to examine the impact of assimilated aircraft moisture observations across seasons, and statistics were collected following a 1-week spinup period from operational ensemble initial conditions reduced to T254 resolution, courtesy of EMC. Moisture observations from AMDAR were switched from a monitoring mode to an assimilation mode at the script level, with an observation error profile copied from the NDAS (see Fig. 1c). No change to the quality control procedure was made to account for the new moisture observations, allowing the GDAS to apply its existing quality control algorithm to these observations, which includes gross error checks and a comparison to the model background but does not include variational quality control (e.g., Andersson and Jarvinen 1999). Aircraft moisture observations were assimilated in specific humidity space, rather than as a relative humidity measurement, which can be subject to significant effects from known biases in AMDAR temperature reports (Zhu et al. 2015). Each seasonal experiment was compared with a control experiment that assimilated all observations except aircraft moisture observations. AMDAR wind and temperature observations are assimilated in both the experiment and control runs. Moisture observations from TAMDAR are not assimilated into the control simulation or other experiments.

Fig. 1.
Fig. 1.

Profiles of observation characteristics for AMDAR and rawinsonde moisture observations. Mean profiles of specific humidity OMB (representing systemic bias in the 6-h moisture forecast), and RMSE for the (a) warm- and (b) cold-season experiments at rawinsonde launch sites. The solid (dashed) blue profile is the mean (RMSE) rawinsonde moisture OMB when AMDAR moisture observations are not assimilated. The red solid (dashed) profile is the mean (RMSE) rawinsonde moisture OMB when AMDAR moisture observations are assimilated. The green solid (dashed) profile is the mean (RMSE) AMDAR moisture OMB for assimilated AMDAR observations in the experiment. The shading around each mean profile represents the 5% and 95% confidence limits around the mean, and pressure levels where the mean rawinsonde OMB changes to statistical significance are highlighted with black squares along the ordinate. (c) Observation error (% RH) profiles for rawinsonde (blue) and AMDAR (red dashed) moisture observations.

Citation: Weather and Forecasting 32, 4; 10.1175/WAF-D-16-0202.1

3. Methodology

Forecast impact was investigated in several ways. Observation-minus-background (OMB) statistics of assimilated moisture observations from both rawinsondes and nearby aircraft observations were compared to assess the data quality of the aircraft moisture observations relative to rawinsondes, an approach that is similar to that used in previous aircraft/rawinsonde collocation studies (e.g., Schwartz and Benjamin 1995). The OMB statistics provide an evaluation of the possible bias that may exist in the model background moisture field, as well as a measure of 6-h forecast improvement.

Forecast performance in the shorter-range (1–2 days) is evaluated by examining the impact of assimilated observations on the equitable threat score (ETS) and the bias score2 for precipitation in the 12–36-h forecasts. Forecast improvement or degradation is evaluated for statistical significance based on Monte Carlo resampling using 10 000 realizations (Hammersley and Handscomb 1975). Although precipitation statistics are available for the 36–60- and 60–84-h forecast ranges, focus is maintained on the 12–36-h forecast, because this is the time period over which aircraft moisture observations had the greatest impact on the forecast.

Forecast performance at longer range (2–3 days) is evaluated by comparing forecast total-column precipitable water (TPW) against TPW observed by global positioning satellite (GPS) signals (e.g., Duan et al. 1996) produced at GPS-Met ground stations and made available at the time of this study by the Earth System Research Laboratory (ESRL). Unlike precipitation statistics that focus on the impact of moisture observations as the model approaches saturation, TPW comparisons provide a good means of evaluating the vertically integrated effect of AMDAR moisture reports throughout the full range of humidity. Errors from GPS-TPW have been shown to be less than 1 mm when compared with ground-based microwave radiometer observations during the Measurements of Humidity in the Atmosphere and Validation Experiment (Leblanc et al. 2011) in California and at the Atmospheric Radiation Measurement (ARM) program facility (Dworak and Petersen 2013) in Oklahoma. Furthermore, positive impacts on RUC forecasts out to 12 h have been observed with the assimilation of GPS-TPW data (Gutman et al. 2004; Smith et al. 2007). Forecast errors relative to GPS observations were computed in this study for every 6-h forecast period out to 72 h, and deviations from the error in the control forecast are evaluated for statistical significance using a Student’s t test for mean error and bias, and a chi-squared test for random error (see section 4b). These TPW observations are an independent observation dataset used for verification purposes only.

4. Results

a. Impact of AMDAR moisture observations on rawinsonde moisture assimilation

A particular interest in this study is to investigate the relationship between rawinsonde moisture observations and AMDAR moisture observations during assimilation, when both are available at the same location. It is desirable to know, for example, how rawinsonde and AMDAR moisture observations compare to the model background derived from the 6-h forecast – a closer fit of observations to the 6-h forecast following assimilation can indicate that the observations are high quality and improve the initial (analysis) state. Likewise, an improved fit of rawinsonde observations to the 6-h forecast as a result of assimilating AMDAR observations can indicate better model performance, as this can be equivalently expressed as a closer fit of the 6-h forecast to trusted observations. Lower OMB in general implies greater consistency of the observations with other observation data sources. as well as with information from observations assimilated previously, contributing to the model background. Mean profiles of OMB represent systemic moist or dry biases in the difference between observations and the model background, while the magnitude of the error in the model background relative to the observations is described by the root-mean-square error (RMSE).

Mean profiles of OMB of specific humidity are produced for rawinsonde observations without AMDAR moisture assimilation (from the control), rawinsonde observations with AMDAR moisture assimilation (from the assimilation experiment), and for AMDAR observations when they are assimilated (Fig. 1). Profiles are produced at each rawinsonde site, averaging rawinsonde OMB scores within 25 equally spaced pressure layers between the highest recorded pressure and 300 hPa, which is the highest level where moisture observations are assimilated. AMDAR OMB scores are likewise averaged within these pressure layers using all AMDAR moisture observations within 1 h and 0.5° of the rawinsonde location, representing a radius of 66–77 km, depending on the latitude of the rawinsonde site. These profiles are then averaged across all rawinsonde sites in the continental United States. Similarly, RMSEs are calculated at these same rawinsonde sites for rawinsonde moisture observations with or without assimilated AMDAR moisture observations, as well as for AMDAR moisture observations.

Profiles indicate that rawinsonde observations fit 6-h forecasts better when AMDAR observations are assimilated during the warm-season experiment with smaller OMB bias (Fig. 1a, red versus blue solid lines) and lower RMSEs (Fig. 1a, red versus blue dashed lines). No clear change is observed in the OMB bias of rawinsondes in the cold-season experiment when AMDAR observations are assimilated (Fig. 1b, red versus blue solid lines); however, a small reduction in the RMSEs of rawinsonde observations is observed at the lowest levels when AMDAR observations are assimilated (Fig. 1b, red versus blue dashed lines). These comparisons of rawinsonde OMB statistics with and without assimilated AMDAR observations indicate that the assimilation of AMDAR observations draws the 6-h forecast closer to rawinsonde observations, which implies improved performance.

One can also compare the OMB statistics of AMDAR observations with those of rawinsonde observations as an indicator of relative observation quality. While the OMB bias of AMDAR observations is lower than that of rawinsondes throughout the troposphere in the warm season (Fig. 1a, red versus green solid lines), AMDAR observations in the cold season only express a smaller bias than rawinsonde observations between roughly 900 and 1000 hPa (Fig. 1b, red versus green solid lines). However, the RMSE of the AMDAR observations is lower than the RMSE of the rawinsonde moisture observations throughout the troposphere in both seasonal experiments (red versus green dashed lines).

The combination of low AMDAR OMB bias and RMSE compared to rawinsondes, combined with the demonstrated reduction in rawinsonde OMB bias and RMSE when AMDAR observations are assimilated, indicates that AMDAR moisture observations are of high quality, even in comparison with rawinsonde observations. Rawinsonde observations demonstrate a large positive bias with respect to the model background at the lowest levels, and while AMDAR moisture observations express less bias, the bias in rawinsonde observations does not appear to be mitigated by the assimilation of AMDAR observations at these low levels. The large positive bias could conceivably be the result of a real model (or even measurement) bias, or be caused by the relatively low sampling by both rawinsondes and AMDAR observations at the lowest levels. The profiles are sampled by significantly more observations at pressures of less than 1000 hPa, where this positive bias disappears.

During the cold season when specific humidity patterns are more strongly organized by synoptic-scale weather systems (Heideman and Fritsch 1988) and values are smaller as a result of colder temperatures, rawinsonde observations appear to have statistically insignificant bias characteristics whether AMDAR observations are assimilated or not (Fig. 1b, red versus blue solid lines). During the warm season, the OMB bias for rawinsondes is reduced to statistical significance within the lower troposphere down to just above the surface when AMDAR moisture observations are assimilated (Fig. 1a, red versus blue solid lines). The difference in OMB bias/RMSE impact between the warm and cold seasons may be a by-product of the increased presence of smaller-scale moisture structures during the warm season, or the use of specific humidity–space verification rather than RH space, though it is stressed that this interpretation of the results is conjecture.

b. Impact of AMDAR moisture observations on precipitation and TPW forecasts

To quantify the effect of the analysis changes on the forecast due to the inclusion of AMDAR moisture observations, precipitation forecast skill was determined using the ETS and bias score (Wilks 1995) over the continental United States, binned by precipitation thresholds per 24 h. The assimilation of AMDAR moisture observations improved the mean ETS to statistical significance for 12–36-h precipitation forecasts of below 5 mm day−1 during the warm-season experiment (Fig. 2a). The bias is slightly improved for these categories as well. There is statistically significant ETS degradation for only the 10 mm day−1 category of the 60–84-h forecast (not shown), while the ETS and bias are not significantly changed for any other category at any forecast lead time. The cold-season experiment expresses no statistically significant improvement in ETS or bias for any category or forecast lead time (Fig. 2b), with the exception of a degradation in bias for very high precipitation (50–75 mm day−1) in 60–84-h forecasts (not shown); these higher precipitation categories have very few observations from which to derive statistics, and are dominated by a single event, making the statistics unreliable. Since the GFS precipitation forecast is more accurate during the cold season, as a result of more organized precipitation from synoptic-scale forcing (Olson et al. 1995), the improvement in the cold-season precipitation forecast is expected to be smaller than the improvement in the warm-season forecast.

Fig. 2.
Fig. 2.

Precipitation skill and bias scores of 12–36-h forecasts over the continental United States for (a) warm- and (b) cold-season experiments. (left) ETS for precipitation binned by precipitation amounts [mm (24 h)−1]. (right) The precipitation bias score in the same bins. The black curve is for the control simulation, and the red curve is for the experiment. (bottom) The differences between the experiment and control, with bars indicating the minimum value necessary for 95% statistical significance.

Citation: Weather and Forecasting 32, 4; 10.1175/WAF-D-16-0202.1

These precipitation statistics demonstrate improvement to short-range (12–36 h) precipitation forecasts by the assimilation of AMDAR moisture observations. An additional measure of forecast skill can be observed by computing the forecast fit to observations using GPS total-column precipitable water from GPS-Met ground stations across the continental United States. For each 6-h forecast period from the analysis time to 72h, the forecast TPW fields were interpolated to a database of GPS observations, and the error was computed (Fig. 3). The error is provided both as an RMSE and absolute error as well as an error divided into two components: the bias of the error, represented by the mean difference between the observations and the forecast field, and the random error, represented by the standard deviation of the difference between the observations and the forecast field. The bias component of the error represents the systemic error of the forecast, while the random component of the error represents the distribution of the forecast error about the forecast mean error, or about zero (if the bias component is removed). For any given observation, these two components simply sum to the total error that would be used to compute the RMSE or the absolute error. In general, the bias of the error is typically 10%–20% of the magnitude of the random error, indicating that the random error is responsible for the majority of the total error.

Fig. 3.
Fig. 3.

Error in forecast fit-to-TPW observations from GPS for (top) the April–May 2014 simulation and (bottom) December 2014–January 2015 experiment. Statistics for the control simulation are provided in blue, and statistics for the experiment are provided in red. Error is computed as (a),(d) bias of error, calculated as the mean of the difference between the forecast and observation; (b),(e) random error, calculated as the standard deviation of the error; and (c),(f) absolute error, calculated as the mean of the total (bias + random) error. Thick contours represent the sample mean or standard deviation, and the shading represents the 5% and 95% confidence limits on the mean or standard deviation. Dots are placed along the red contour for all times when the difference between the experiment and control is statistically significant based on a Student’s t test (for bias of error) or a chi-squared test on variance (for random error). RMSE is also plotted (dashed) in (b) and (e) for reference.

Citation: Weather and Forecasting 32, 4; 10.1175/WAF-D-16-0202.1

While bias in the error is slightly increased in the first 18 h of the forecast in the warm-season experiment (Fig. 3a), the total error is reduced, with random error improved to statistical significance from 0 to 36 h into the forecast, with additional statistically significant improvement at 60–66 h (Fig. 3b, solid lines), while the RMSE is reduced in a similar fashion (Fig. 3b, dashed lines). The mean absolute error, which is a combination of both the bias and the random error, is reduced to statistical significance from 0 to 18 h into the forecast (Fig. 3c). The impact of AMDAR moisture observations on the cold-season experiment is less substantial, with no statistically significant change in the bias of the error (Fig. 3d) and a statistically significant reduction in random error only in the first 0–6 h of the forecast (Fig. 3e, solid lines); the RMSE is likewise reduced only for the first 12 h (Fig. 3e, dashed lines). The mean absolute error is only reduced to statistical significance at the analysis time (Fig. 3f). The difference in impact between the warm- and cold-season experiments may be due to differences in the precipitation regimes between the two periods with warm-season precipitation dominated by small-scale features under weak synoptic forcing and cold-season precipitation dominated by large-scale features with strong synoptic forcing, yielding greater forecast accuracy during the cold season (Olson et al. 1995). It is stressed that this interpretation of the results is conjecture.

Two differences between precipitation skill scores and TPW fit-to-observation scores must be considered. First, the GPS observations are more numerous than the precipitation observations, which allow every forecast to be tested for accuracy at more locations than with the more sparse precipitation data. For example, the 12–36-h forecast period over which the precipitation skill scores are presented is binned by precipitation amount, with the largest number of precipitation observations in the lowest-value bin. For the warm-season experiment, there are at least 42 057 data points used to determine the ETS and bias score. By contrast, in the forecast fit-to-observations test, 69 071 observations were tested over the same forecast period, a 64% increase in the number of available observations. Second, TPW observations can exist where precipitation observations do not, allowing for sampling across the full spectrum of moisture values. There are over 400 active GPS-Met stations at any given observation period ±15 min from the hour, with high geographic density in California.

5. Conclusions

The impact of assimilated AMDAR moisture observations from the WVSS-II was evaluated in the GDAS/GFS analysis–forecast system. Cycled experiments were carried out for a warm season (April–May 2014) and a cold season (December 2014–January 2015). The warm-season experiment demonstrated positive impacts on the ETS and bias score for low-precipitation categories in the 12–36-h forecast. Assimilation of AMDAR moisture observations in the warm season produced smaller OMB bias/RMSE relative to the rawinsonde observations, implying that the 6-h moisture forecast was improved and the OMB bias/RMSE was even lower relative to the AMDAR moisture observations taken near rawinsonde launches. The OMB RMSE relative to the rawinsonde observations was lowered only in the 100-hPa layer nearest to the surface in the cold-season experiment.

When the total-column precipitable water forecast was compared with observations from GPS-Met ground stations, the assimilation of AMDAR moisture observations during the warm season improved the random error in the forecast as far out as 66 h. By contrast, the cold-season experiment only demonstrated a statistically significant positive impact on random error out to 6 h. It should be noted that these experiments were performed at less than operational resolution, and implementation at operational resolution could affect the results. Based on this study, the decision was made to implement the assimilation of aircraft moisture observations in the operational GDAS, as part of NCEP’s 2016 upgrade.

Acknowledgments

The authors would like to acknowledge Dr. Steve Lord (NOAA) and Dr. John Derber (NOAA) for their advice and expertise in guiding this project, Dr. Jim Jung (CIMSS) and Kate Howard (NOAA) for their advice and help with running the GDAS, and Jim Nelson (CIMSS) for his help with software and data for the GPS fit-to-observation tests. This study was funded by NOAA through Grant NA13NWS4830022.

REFERENCES

  • Andersson, E., and H. Jarvinen, 1999: Variational quality control. Quart. J. Roy. Meteor. Soc., 125, 697722, doi:10.1002/qj.49712555416.

  • Andersson, E., C. Cardinali, B. Truscott, and T. Hovberg, 2005: High frequency AMDAR data—A European aircraft data collection trial and impact assessment. ECMWF Tech. Memo. 457, 15 pp. [Available online at https://www.ecmwf.int/en/elibrary/7765-high-frequency-amdar-data-european-aircraft-data-collection-trial-and-impact.]

  • Benjamin, S. G., B. D. Jamison, W. R. Moninger, S. R. Sahm, B. E. Schwartz, and T. W. Schlatter, 2010: Relative short-range forecast impact from aircraft, profiler, radiosonde, VAD, GPS-PW, METAR, and mesonet observations via the RUC hourly assimilation cycle. Mon. Wea. Rev., 138, 13191343, doi:10.1175/2009MWR3097.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cardinali, C., 2009: Monitoring the observation impact on the short-range forecast. Quart. J. Roy. Meteor. Soc., 135, 239250, doi:10.1002/qj.366.

  • Duan, J., and et al. , 1996: GPS meteorology: Direct estimation of the absolute value of precipitable water. J. Appl. Meteor., 35, 830838, doi:10.1175/1520-0450(1996)035<0830:GMDEOT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dworak, R., and R. Petersen, 2013: The validation of GOES-LI and AIRS total precipitable water retrievals using ground based measurements. Joint EUMETSAT Meteorological Satellite Conf./19th Conf. on Satellite Meteorology, Oceanography, and Climatology, Vienna, Austria, EUMETSAT–Amer. Meteor. Soc. [Available online at https://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=PDF_CONF_P_S7_03_DWORAKRevisionSelectionMethod=LatestReleased&Rendition=Web.]

  • Fleming, R. J., 1996: The use of commercial aircraft as platforms for environmental measurements. Bull. Amer. Meteor. Soc., 77, 22292242, doi:10.1175/1520-0477(1996)077<2229:TUOCAA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fleming, R. J., 1998: A note on temperature and relative humidity corrections for humidity sensors. J. Atmos. Oceanic Technol., 15, 15111515, doi:10.1175/1520-0426(1998)015<1511:ANOTAR>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gelaro, R., R. H. Langland, S. Pellerin, and R. Todling, 2010: The THORPEX Observation Impact Intercomparison Experiment. Mon. Wea. Rev., 138, 40094025, doi:10.1175/2010MWR3393.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gutman, S. I., S. R. Sahm, S. G. Benjamin, B. E. Schwartz, K. L. Holub, J. Q. Stewart, and T. L. Smith, 2004: Rapid retrieval and assimilation of ground based GPS precipitable water observations at the NOAA Forecast Systems Laboratory: Impact on weather forecasts. J. Meteor. Soc. Japan, 82, 351360, doi:10.2151/jmsj.2004.351.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hammersley, J. M., and D. C. Handscomb, 1975: Monte Carlo Methods. Methuen and Co., 178 pp.

  • Heideman, K. F., and J. M. Fritsch, 1988: Forcing mechanisms and other characteristics of significant summertime precipitation. Wea. Forecasting, 3, 115130, doi:10.1175/1520-0434(1988)003<0115:FMAOCO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Helms, D., A. Hoff, H. G. J. Smit, S. Taylor, S. Carlberg, and M. Berechree, 2010: Advancements in the AMDAR humidity sensing. Tech. Conf. on Meteorological and Environmental Instruments and Methods of Observation, Helsinki, Finland, WMO, 2.1. [Available online at http://www.wmo.int/pages/prog/www/IMOP/publications/IOM-104_TECO-2010/2_1_Helms_USA.doc.]

  • Hogan, T. F., and et al. , 2014: The Navy Global Environmental Model. Oceanography, 27, 116125, doi:10.5670/oceanog.2014.73.

  • Kleist, D. T., and K. Ide, 2015: An OSSE-based evaluation of hybrid variational–ensemble data assimilation for the NCEP GFS. Part I: System description and 3D-hybrid results. Mon. Wea. Rev., 143, 433451, doi:10.1175/MWR-D-13-00351.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kleist, D. T., D. F. Parrish, J. C. Derber, R. Treadon, W.-S. Wu, and S. Lord, 2009: Introduction of the GSI into the NCEP Global Data Assimilation System. Wea. Forecasting, 24, 16911705, doi:10.1175/2009WAF2222201.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Langland, R. H., and N. L. Baker, 2004: Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus, 56A, 189201, doi:10.3402/tellusa.v56i3.14413.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leblanc, T., and et al. , 2011: Measurements of Humidity in the Atmosphere and Validation Experiments (MOHAVE)-2009: Overview of campaign operations and results. Atmos. Meas. Tech., 4, 25792605, doi:10.5194/amt-4-2579-2011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moninger, W. R., R. D. Mamrosh, and P. M. Pauley, 2003: Automated meteorological reports from commercial aircraft. Bull. Amer. Meteor. Soc., 84, 203216, doi:10.1175/BAMS-84-2-203.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moninger, W. R., S. G. Benjamin, B. D. Jamison, T. W. Schlatter, T. L. Smith, and E. J. Szoke, 2010: Evaluation of regional aircraft observations using TAMDAR. Wea. Forecasting, 25, 627645, doi:10.1175/2009WAF2222321.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Olson, D. A., N. W. Junker, and B. Korty, 1995: Evaluation of 33 years of quantitative precipitation forecasting at NMC. Wea. Forecasting, 10, 498511, doi:10.1175/1520-0434(1995)010<0498:EOYOQP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ota, Y., J. C. Derber, E. Kalnay, and T. Miyoshi, 2013: Ensemble-based observation impact estimates using the NCEP GFS. Tellus, 65A, 20038, doi:10.3402/tellusa.v65i0.20038.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Petersen, R. A., 2016: On the impact and future benefits of AMDAR observations in operational forecasts. Part I: A review of the impact of automated aircraft wind and temperature reports. Bull. Amer. Meteor. Soc., 97, 585602, doi:10.1175/BAMS-D-14-00055.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Petersen, R. A., C. Dey, R. C. Martin, R. D. Londot, and G. T. Ligler, 1992: The Meteorological Data Collection and Reporting System (MDCRS): System overview and benefits. Proc. National Weather Service Aviation Workshop, Kansas City, MO, NWS, 251255.

  • Petersen, R. A., L. Cronce, R. Mamrosh, and R. Baker, 2015: Impact and benefits of AMDAR temperature, wind, and moisture observations in operational weather forecasting. WMO Tech. Rep. 2015-01, 93 pp. [Available online at http://library.wmo.int/pmb_ged/wigos-tr_2015-01_en.pdf.]

  • Petersen, R. A., L. Cronce, R. Mamrosh, R. Baker, and P. Pauley, 2016: On the impact and future benefits of AMDAR observations in operational forecasting. Part II: Water vapor observations Bull. Amer. Meteor. Soc., 97, 21172133, doi:10.1175/BAMS-D-14-00211.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schwartz, B., and S. G. Benjamin, 1995: A comparison of temperature and wind measurements from ACARS-equipped aircraft and rawinsondes. Wea. Forecasting, 10, 528544, doi:10.1175/1520-0434(1995)010<0528:ACOTAW>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, T. L., S. G. Benjamin, and S. I. Gutman, 2007: Short-range forecast impact from assimilation of GPS-IPW observations into the Rapid Update Cycle. Mon. Wea. Rev., 135, 29142930, doi:10.1175/MWR3436.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, X., D. Parrish, D. Kleist, and J. Whitaker, 2013: GSI 3DVar-based ensemble–variational hybrid data assimilation for NCEP Global Forecast System: Single-resolution experiments. Mon. Wea. Rev., 141, 40984117, doi:10.1175/MWR-D-12-00141.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D., 1995: Statistical Methods in the Atmospheric Sciences: An Introduction. Academic Press, 467 pp.

  • Zhu, Y., J. C. Derber, R. J. Purser, B. A. Ballish, and J. Whiting, 2015: Variational correction of aircraft temperature bias in the NCEP’s GSI analysis system. Mon. Wea. Rev., 143, 37743803, doi:10.1175/MWR-D-14-00235.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
1

The upgrade to the semi-Lagrangian core was officially implemented in December 2014; information is available online (http://www.nws.noaa.gov/om/notification/tin14-46gfs_cca.htm).

2

Bias in the NCEP precipitation statistics is calculated as the ratio of the number of verification grid boxes that are forecast to have precipitation in a given range (mm day−1) to the number of grid boxes where that amount of precipitation actually occurred.

Save