• AMS, 2002: Enhancing weather information with probability forecasts. Bull. Amer. Meteor. Soc., 83 , 450452.

  • AMS, cited. 2007: Ad hoc committee on uncertainty in forecasts charter. [Available online at http://www.ametsoc.org/boardpges/cwce/docs/BEC/ACUF/Charter.pdf.].

  • Appenzeller, C., and Davies H. C. , 1992: Structure of stratospheric intrusions into the troposphere. Nature, 358 , 570572.

  • Arribas, A., Robertson K. B. , and Mylne K. R. , 2005: Test of a poor man’s ensemble prediction system for short-range probability forecasting. Mon. Wea. Rev., 133 , 18251839.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baars, J. A., and Mass C. F. , 2005: Performance of National Weather Service forecasts compared to operational, consensus, and weighted model output statistics. Wea. Forecasting, 20 , 10341047.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bosart, L. F., 2003: Whither the weather analysis and forecasting process? Wea. Forecasting, 18 , 520529.

  • Bowler, N. E., Arribas A. , Mylne K. R. , Robertson K. B. , and Beare S. E. , 2008: The MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 134 , 703722.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bright, D. R., and Nutter P. A. , 2004: On the challenges of identifying the “best” ensemble member in operational forecasting. Preprints, 16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., J11.3. [Available online at http://ams.confex.com/ams/pdfpapers/69092.pdf.].

  • Broad, K., Leiserowitz A. , Weinkle J. , and Steketee M. , 2007: Misinterpretations of the “cone of uncertainty” in Florida during the 2004 hurricane season. Bull. Amer. Meteor. Soc., 88 , 651667.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brooks, H. E., Tracton M. S. , Stensrud D. J. , DiMego G. , and Toth Z. , 1995: Short-range ensemble forecasting: Report from a workshop, 25–27 July 1995. Bull. Amer. Meteor. Soc., 76 , 16171624.

    • Search Google Scholar
    • Export Citation
  • Bua, W. R., 2005: COMET ensemble prediction training. Preprints, 21st Conf. on Weather Analysis and Forecasting/17th Conf. on Numerical Weather Prediction, Washington, DC, Amer. Meteor. Soc., J2.7. [Available online at http://ams.confex.com/ams/pdfpapers/94545.pdf.].

  • Buizza, R., 1997: Potential forecast skill of ensemble prediction and spread and skill distributions of the ECMWF ensemble prediction system. Mon. Wea. Rev., 125 , 99119.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buizza, R., Houtekamer P. L. , Toth Z. , Pellerin G. , Wei M. , and Zhu Y. , 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133 , 10761097.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Carroll, E. B., and Hewson T. D. , 2005: NWP grid editing at the Met Office. Wea. Forecasting, 20 , 10211033.

  • Cheng, W. Y. Y., and Steenburgh W. J. , 2007: Strengths and weaknesses of MOS, running-mean bias removal, and Kalman filter techniques for improving model forecasts over the western United States. Wea. Forecasting, 22 , 13041318.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Colle, B. A., and Charles M. , 2007: Verification of extratropical cyclones within NCEP forecast models using an automated tracking algorithm. Preprints, 22nd Conf. on Weather Analysis and Forecasting/18th Conf. on Numerical Weather Prediction, Park City, UT, Amer. Meteor. Soc., 6B.3. [Available online at http://ams.confex.com/ams/pdfpapers/124612.pdf.].

  • Davis, C. A., Brown B. , and Bullock R. , 2006a: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134 , 17721784.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Davis, C. A., Brown B. , and Bullock R. , 2006b: Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Wea. Rev., 134 , 17851795.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Demirtas, M., and Thorpe A. J. , 1999: Sensitivity of short-range weather forecasts to local potential vorticity modifications. Mon. Wea. Rev., 127 , 922939.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., 2004: Weather forecasting by humans—heuristics and decision making. Wea. Forecasting, 19 , 11151126.

  • Du, J., DiMego G. , Tracton M. S. , and Zhou B. , 2003: NCEP short-range ensemble forecasting (SREF) system: Multi-IC, multi-model and multi-physics approach. Research activities in atmospheric and oceanic modeling, J. Cote, Ed., CAS/JSC Working Group Numerical Experimentation Rep. 23, WMO/TD 1161, 5.09–5.10.

  • Durante, A. V., Hart R. E. , Watson A. , Grumm R. H. , and Drag W. , 2005: The development of forecast confidence measures using NCEP ensembles and their real-time implementation within NWS web-based graphical forecasts. Preprints, 18th Conf. on Probability and Statistics in the Atmospheric Sciences, Atlanta, GA, Amer. Meteor. Soc., 5.1. [Available online at http://ams.confex.com/ams/pdfpapers/100178.pdf.].

  • Ebert, E. E., 2001: Ability of a poor man’s ensemble to predict the probability and distribution of precipitation. Mon. Wea. Rev., 129 , 24612479.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and McBride J. L. , 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239 , 179202.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Eckel, F. A., and Mass C. F. , 2005: Aspects of effective short-range ensemble forecasting. Wea. Forecasting, 20 , 328350.

  • Friday E. W. Jr., , 1994: The modernization and associated restructuring of the National Weather Service: An overview. Bull. Amer. Meteor. Soc., 75 , 4352.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fowle, M. A., and Roebber P. J. , 2003: Short-range (0–48 h) numerical prediction of convective occurrence, mode, and location. Wea. Forecasting, 18 , 782794.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Glahn, H. R., 2003: Comments on “IFPS and the future of the National Weather Service”. Wea. Forecasting, 18 , 12991304.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Glahn, H. R., and Lowry D. A. , 1972: The use of model output statistics (MOS) in objective weather forecasting. J. Appl. Meteor., 11 , 12031211.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Glahn, H. R., and Ruth D. P. , 2003: The new digital forecast database of the National Weather Service. Bull. Amer. Meteor. Soc., 84 , 195201.

  • Grumm, R. H., and Hart R. , 2001: Standardized anomalies applied to significant cold season weather events: Preliminary findings. Wea. Forecasting, 16 , 736754.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guerin, R., Desroziers G. , and Arbogast P. , 2006: 4D-Var analysis of potential vorticity pseudo-observations. Quart. J. Roy. Meteor. Soc., 132 , 12831298.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and Colucci S. J. , 1997: Verification of Eta–RSM short-range ensemble forecasts. Mon. Wea. Rev., 125 , 13121327.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and Whitaker J. S. , 2006: Probabilistic quantitative precipitation forecasts based on reforecast analogs: Theory and application. Mon. Wea. Rev., 134 , 32093229.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., Whitaker J. S. , and Wei X. , 2004: Ensemble reforecasting: Improving medium-range forecast skill using retrospective forecasts. Mon. Wea. Rev., 132 , 14341447.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hewson, T. D., 2002: A comparison of cyclone spectra in forecasts from the operational (G1) and new dynamics trial (NT) versions of the unified model—Aug to Dec 2001. Forecasting Research Tech. Rep. 376, Met Office, 15 pp. [Available online at http://www.metoffice.gov.uk/research/nwp/publications/papers/technical_reports/2002/FRTR376/FRTR376.pdf.].

  • Homar, V., Stensrud D. J. , Levit J. J. , and Bright D. R. , 2006: Value of human-generated perturbations in short-range ensemble forecasts of severe weather. Wea. Forecasting, 21 , 347363.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hoskins, B. J., McIntyre M. E. , and Robertson A. Q. , 1985: On the use and significance of isentropic potential vorticity maps. Quart. J. Roy. Meteor. Soc., 111 , 877946.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and Derome J. , 1995: Methods for ensemble prediction. Mon. Wea. Rev., 123 , 21812196.

  • Houtekamer, P. L., Derome J. , Ritchie H. , and Mitchell H. L. , 1996: A system simulation approach to ensemble prediction. Mon. Wea. Rev., 124 , 12251242.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jones, M. S., and Colle B. A. , 2007: Evaluation of a mesoscale short-range ensemble forecast system over the northeast United States. Wea. Forecasting, 22 , 3655.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joslyn, S., Pak K. , Jones D. , Pyles J. , and Hunt E. , 2007: The effect of probabilistic information on threshold forecasts. Wea. Forecasting, 22 , 804812.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J. S., Baldwin M. E. , Janish P. R. , Weiss S. J. , Kay M. P. , and Carbin G. W. , 2003a: Subjective verification of numerical models as a component of a broader interaction between research and operations. Wea. Forecasting, 18 , 847860.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J. S., Janish P. R. , Weiss S. J. , Baldwin M. E. , Schneider R. , and Brooks H. E. , 2003b: Collaboration between forecasters and research scientists at the NSSL and SPC: The Spring Program. Bull. Amer. Meteor. Soc., 84 , 17971806.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation, and Predictability. Cambridge University Press, 341 pp.

  • Katz, R. W., and Murphy A. H. , 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 238 pp.

  • Keith, R., 2003: Optimization of value of aerodrome forecasts. Wea. Forecasting, 18 , 808824.

  • Lawson, J. D., Tuell J. P. , Griffith F. P. , and Henry R. K. , 2007: Overview of the AWIPS SOA. Preprints, 23rd Int. Conf. on Interactive Information Processing Systems for Meteorology, Oceanography, and Hydrology, San Antonio, TX, Amer. Meteor. Soc., P2.2.

  • Legg, T. P., and Mylne K. R. , 2004: Early warnings of severe weather from ensemble forecast information. Wea. Forecasting, 19 , 891906. Corrigendum, 22, 216–219.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leith, C. E., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102 , 409418.

  • Leutbecher, M., 2005: On ensemble prediction using singular vectors started from forecasts. Mon. Wea. Rev., 133 , 30383046.

  • Magsig, M. A., and Page E. M. , 2003: Weather event simulator implementation and future development. Preprints, 20th Int. Conf. on Interactive Information Processing Systems for Meteorology, Oceanography, and Hydrology, Long Beach, CA, Amer. Meteor. Soc., 12.9.

  • Marsigli, C., Boccanera F. , Montani A. , and Paccagnella T. , 2005: The COSMO-LEPS mesoscale ensemble system: Validation of the methodology and verification. Nonlinear Processes Geophys., 12 , 527536.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mass, C. F., 2003: IFPS and the future of the National Weather Service. Wea. Forecasting, 18 , 7579.

  • Mass, C. F., and Coauthors, 2003: Regional environmental prediction over the Pacific Northwest. Bull. Amer. Meteor. Soc., 84 , 13531366.

  • Matthieu, P., and Arbogast P. , 2005: A cyclogenesis evolving into two distinct scenarios and its implications for short-term ensemble forecasting. Mon. Wea. Rev., 133 , 20162029.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McEnery, J., Ingram J. , Duan Q. , Adams T. , and Anderson L. , 2005: NOAA’s advanced hydrologic prediction service: Building pathways for better science in water forecasting. Bull. Amer. Meteor. Soc., 86 , 375385.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., and Ralph F. M. , 2007: Use of information by National Weather Service forecasters and emergency managers during CALJET and PACJET-2001. Wea. Forecasting, 22 , 539555.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., Demuth J. L. , and Lazo J. K. , 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23 , 974991.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Molteni, F., Buizza R. , Palmer T. N. , and Petroliagis T. , 1996: The ECMWF Ensemble Prediction System: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122 , 73119.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • NRC, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. National Academies Press, 124 pp.

    • Search Google Scholar
    • Export Citation
  • Palmer, T., and Hagedorn R. , 2006: Predictability of Weather and Climate. Cambridge University Press, 718 pp.

  • Payne, K. A., Elsberry R. L. , and Boothe M. A. , 2007: Assessment of western North Pacific 96- and 120-h track guidance and present forecastability. Wea. Forecasting, 22 , 10031015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pielke R. A. Jr., , 1999: Who decides? Forecasts and responsibilities in the 1997 Red River floods. Appl. Behav. Sci. Rev., 7 , 83101.

  • Raftery, A. E., Gneiting T. , Balabdaoui F. , and Polakowski M. , 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133 , 11551174.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roebber, P. J., Schultz D. M. , and Romero R. , 2002: Synoptic regulation of the 3 May 1999 tornado outbreak. Wea. Forecasting, 17 , 399429.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roebber, P. J., Schultz D. M. , Colle B. A. , and Stensrud D. J. , 2004: Toward improved prediction: High-resolution and ensemble modeling systems in operations. Wea. Forecasting, 19 , 936949.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roulston, M. S., and Kaplan T. R. , 2008: A laboratory-based study of understanding of uncertainty in 5-day site-specific temperature forecasts. Meteor. Appl., in press.

    • Search Google Scholar
    • Export Citation
  • Rutledge, G. K., Alpert J. , and Ebisuzaki W. , 2006: NOMADS: A climate and weather model archive at the National Oceanic and Atmospheric Administration. Bull. Amer. Meteor. Soc., 87 , 327341.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Santurette, P., and Georgiev C. G. , 2005: Weather Analysis and Forecasting. Academic Press, 200 pp.

  • Sivillo, J. K., Ahlquist J. E. , and Toth Z. , 1997: An ensemble forecasting primer. Wea. Forecasting, 12 , 809818.

  • Stensrud, D. J., and Yussouf N. , 2003: Short-range ensemble predictions of 2-m temperature and dewpoint temperature over New England. Mon. Wea. Rev., 131 , 25102524.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stuart, N. A., and Grumm R. H. , 2006: Using wind anomalies to forecast East Coast winter storms. Wea. Forecasting, 21 , 952968.

  • Stuart, N. A., and Coauthors, 2006: The future of humans in an increasingly automated forecast process. Bull. Amer. Meteor. Soc., 87 , 14971502.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stuart, N. A., Schultz D. M. , and Klein G. , 2007: Maintaining the role of humans in the forecast process: Analyzing the psyche of expert forecasters. Bull. Amer. Meteor. Soc., 88 , 18931898.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toth, Z., and Kalnay E. , 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74 , 23172330.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toth, Z., and Majumdar S. J. , 2007: NOAA THORPEX science workshop. Bull. Amer. Meteor. Soc., 88 , 717719.

  • Walser, A., Arpagaus M. , Appenzeller C. , and Leutbecher M. , 2006: The impact of moist singular vectors and horizontal resolution on short-range limited-area ensemble forecasts for two European winter storms. Mon. Wea. Rev., 134 , 28772887.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Watkin, H., and Hewson T. , 2006: The development of feature-based diagnostics to assess TIGGE handling of high-impact extratropical cyclones. Preprints, Second THORPEX Int. Science Symp., Landshut, Germany, WMO, 258–259.

  • Weiss, S. J., Bright D. R. , Kain J. S. , Levit J. J. , Pyle M. E. , Janjic Z. I. , Ferrier B. S. , and Du J. , 2006: Complementary use of short-range ensemble and 4.5 km WRF-NMM model guidance for severe weather forecasting at the Storm Prediction Center. Preprints, 23rd Conf. on Severe Local Storms, St. Louis, MO, Amer. Meteor. Soc., 8.5. [Available online at http://ams.confex.com/ams/pdfpapers/115386.pdf.].

  • WMO, 2003: 21st status report on the implementation of the World Weather Watch—2003. WMO 957, 61 pp.

  • WMO, cited. 2007: The Observing System Research and Predictability Experiment (THORPEX) programme. [Available online at http://www.wmo.ch/pages/prog/arep/thorpex/].

  • Woodcock, F., and Engel C. , 2005: Operational consensus forecasts. Wea. Forecasting, 20 , 101111.

  • Yuan, H., Gao X. , Mullen S. , Sorooshian S. , Du J. , and Juang H-M. H. , 2007: Calibration of probabilistic quantitative precipitation forecasts with an artificial neural network. Wea. Forecasting, 22 , 12871303.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yussouf, N., and Stensrud D. J. , 2007: Bias-corrected short-range ensemble forecasts of near-surface variables during the 2005/06 cool season. Wea. Forecasting, 22 , 12741286.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yussouf, N., Stensrud D. J. , and Lakshmivarahan S. , 2004: Cluster analysis of multimodel ensemble data over New England. Mon. Wea. Rev., 132 , 24522462.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 290 151 0
PDF Downloads 135 62 0

Operational Forecaster Uncertainty Needs and Future Roles

View More View Less
  • 1 NOAA/National Weather Service, Eastern Region Headquarters, Scientific Services Division, Bohemia, New York
  • | 2 NOAA/National Weather Service, Storm Prediction Center, Norman, Oklahoma
  • | 3 NOAA/National Weather Service, Hydrometeorological Prediction Center, Camp Springs, Maryland
Restricted access

Abstract

Key results of a comprehensive survey of U.S. National Weather Service operational forecast managers concerning the assessment and communication of forecast uncertainty are presented and discussed. The survey results revealed that forecasters are using uncertainty guidance to assess uncertainty, but that limited data access and ensemble underdispersion and biases are barriers to more effective use. Some respondents expressed skepticism as to the added value of formal ensemble guidance relative to simpler approaches of estimating uncertainty, and related the desire for feature-specific ensemble verification to address this skepticism. Respondents reported receiving requests for uncertainty information primarily from sophisticated users such as emergency managers, and most often during high-impact events. The largest request for additional training material called for simulator-based case studies that demonstrate how uncertainty information should be interpreted and communicated.

Respondents were in consensus that forecasters should be significantly involved in the communication of uncertainty forecasts; however, there was disagreement regarding if and how forecasters should adjust objective ensemble guidance. It is contended that whether forecasters directly modify objective ensemble guidance will ultimately depend on how the weather enterprise views ensemble output (as the final forecast or as a guidance supporting conceptual understanding), the enterprise’s commitment to provide the necessary supporting forecast infrastructure, and how rapidly ensemble weaknesses such as underdispersion, biases, and resolution are addressed.

The survey results illustrate that forecasters’ operational uncertainty needs are intimately tied to the end products and services they produce. Thus, it is critical that the process to develop uncertainty information in existing or new products or services be a sustained collaborative effort between ensemble developers, forecasters, academic partners, and users. As the weather enterprise strives to provide uncertainty information to users, it is asserted that addressing the forecaster needs identified in this survey will be a prerequisite to achieve this goal.

Corresponding author address: David R. Novak, NOAA/NWS, Eastern Region Headquarters, Suite 202, 630 Johnson Ave., Bohemia, NY 11716. Email: david.novak@noaa.gov

Abstract

Key results of a comprehensive survey of U.S. National Weather Service operational forecast managers concerning the assessment and communication of forecast uncertainty are presented and discussed. The survey results revealed that forecasters are using uncertainty guidance to assess uncertainty, but that limited data access and ensemble underdispersion and biases are barriers to more effective use. Some respondents expressed skepticism as to the added value of formal ensemble guidance relative to simpler approaches of estimating uncertainty, and related the desire for feature-specific ensemble verification to address this skepticism. Respondents reported receiving requests for uncertainty information primarily from sophisticated users such as emergency managers, and most often during high-impact events. The largest request for additional training material called for simulator-based case studies that demonstrate how uncertainty information should be interpreted and communicated.

Respondents were in consensus that forecasters should be significantly involved in the communication of uncertainty forecasts; however, there was disagreement regarding if and how forecasters should adjust objective ensemble guidance. It is contended that whether forecasters directly modify objective ensemble guidance will ultimately depend on how the weather enterprise views ensemble output (as the final forecast or as a guidance supporting conceptual understanding), the enterprise’s commitment to provide the necessary supporting forecast infrastructure, and how rapidly ensemble weaknesses such as underdispersion, biases, and resolution are addressed.

The survey results illustrate that forecasters’ operational uncertainty needs are intimately tied to the end products and services they produce. Thus, it is critical that the process to develop uncertainty information in existing or new products or services be a sustained collaborative effort between ensemble developers, forecasters, academic partners, and users. As the weather enterprise strives to provide uncertainty information to users, it is asserted that addressing the forecaster needs identified in this survey will be a prerequisite to achieve this goal.

Corresponding author address: David R. Novak, NOAA/NWS, Eastern Region Headquarters, Suite 202, 630 Johnson Ave., Bohemia, NY 11716. Email: david.novak@noaa.gov

Save