• Bakhshaii, A., and Stull R. , 2009: Deterministic ensemble forecasts using gene-expression programming. Wea. Forecasting, 24, 14311451, doi:10.1175/2009WAF2222192.1.

    • Search Google Scholar
    • Export Citation
  • Baldwin, M. E., Lakshmivarahan S. , and Kain J. S. , 2002: Development of an “events oriented” approach to forecast verification. Preprints, 19th Conf. on Weather Analysis and Forecasting/15th Conf. on Numerical Weather Prediction, San Antonio, TX, Amer. Meteor. Soc., 7B.3. [Available online at https://ams.confex.com/ams/SLS_WAF_NWP/techprogram/paper_47738.htm.]

  • Bélair, S., Roch M. , Leduc A.-M. , Vaillancourt P. A. , Laroche S. , and Mailhot J. , 2009: Medium-range quantitative precipitation forecasts from Canada’s new 33-km deterministic global operational system. Wea. Forecasting, 24, 690708, doi:10.1175/2008WAF2222175.1.

    • Search Google Scholar
    • Export Citation
  • Bosart, L. F., 2003: Whither the weather analysis and forecasting process? Wea. Forecasting, 18, 520529, doi:10.1175/1520-0434(2003)18<520:WTWAAF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Brill, K. F., 2009: A general analytic method for assessing sensitivity to bias of performance measures for dichotomous forecasts. Wea. Forecasting, 24, 307318, doi:10.1175/2008WAF2222144.1.

    • Search Google Scholar
    • Export Citation
  • Brill, K. F., and Mesinger F. , 2009: Applying a general analytic method for assessing bias sensitivity to bias-adjusted threat and equitable threat scores. Wea. Forecasting, 24, 17481754, doi:10.1175/2009WAF2222272.1.

    • Search Google Scholar
    • Export Citation
  • Brown, J. D., and Seo D.-J. , 2010: A nonparametric postprocessor for bias correction of hydrometeorological and hydrologic ensemble forecasts. J. Hydrometeor., 11, 642665, doi:10.1175/2009JHM1188.1.

    • Search Google Scholar
    • Export Citation
  • Caplan, P., Derber J. , Gemmill W. , Hong S.-Y. , Pan H.-L. , and Parrish D. , 1997: Changes to the 1995 NCEP operational Medium-Range Forecast Model Analysis–Forecast System. Wea. Forecasting, 12, 581594, doi:10.1175/1520-0434(1997)012<0581:CTTNOM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Clark, A. J., Gallus W. A. Jr., Xue M. , and Kong F. , 2009: A comparison of precipitation forecast skill between small convection-allowing and large convection-parameterizing ensembles. Wea. Forecasting, 24, 11211140, doi:10.1175/2009WAF2222222.1.

    • Search Google Scholar
    • Export Citation
  • Cui, B., Toth Z. , Zhu Y. , and Hou D. , 2012: Bias correction for global ensemble forecast. Wea. Forecasting, 27, 396410, doi:10.1175/WAF-D-11-00011.1.

    • Search Google Scholar
    • Export Citation
  • Daly, C., Neilson R. P. , and Phillips D. L. , 1994: A statistical-topographic model for mapping climatological precipitation over mountainous terrain. J. Appl. Meteor., 33, 140158, doi:10.1175/1520-0450(1994)033<0140:ASTMFM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Daly, C., Halbleib M. , Smith J. I. , Gibson W. P. , Doggett M. K. , Taylor G. H. , Curtis J. , and Pasteris P. A. , 2008: Physiographically sensitive mapping of climatological temperature and precipitation across the conterminous United States. Int. J. Climatol., 28, 20312064, doi:10.1002/joc.1688.

    • Search Google Scholar
    • Export Citation
  • De Pondeca, M. S. F. V., and Coauthors, 2011: The Real-Time Mesoscale Analysis at NOAA’s National Centers for Environmental Prediction: Current status and development. Wea. Forecasting, 26, 593612, doi:10.1175/WAF-D-10-05037.1.

    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., III, 2004: Weather forecasting by humans—Heuristics and decision making. Wea. Forecasting, 19, 11151126, doi:10.1175/WAF-821.1.

    • Search Google Scholar
    • Export Citation
  • Du, J., and Coauthors, 2006: New dimension of NCEP Short-Range Ensemble Forecasting (SREF) system: Inclusion of WRF members. Preprints, WMO Expert Team Meeting on the Ensemble Prediction System, Exeter, United Kingdom, World Meteorological Organization. [Available online at http://www.emc.ncep.noaa.gov/mmb/SREF/reference.html.]

  • Ebert, E. E., 2001: Ability of a poor man’s ensemble to predict the probability and distribution of precipitation. Mon. Wea. Rev., 129, 24612480, doi:10.1175/1520-0493(2001)129<2461:AOAPMS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Etherton, B. J., 2007: Preemptive forecasts using an ensemble Kalman filter. Mon. Wea. Rev., 135, 34843495, doi:10.1175/MWR3480.1.

  • Fritsch, J. M., and Carbone R. E. , 2004: Improving quantitative precipitation forecasts in the warm season: A USWRP research and development strategy. Bull. Amer. Meteor. Soc., 85, 955965, doi:10.1175/BAMS-85-7-955.

    • Search Google Scholar
    • Export Citation
  • Giorgi, F., 1991: Sensitivity of simulated summertime precipitation over the western United States to different physics parameterizations. Mon. Wea. Rev., 119, 28702888, doi:10.1175/1520-0493(1991)119<2870:SOSSPO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Glahn, H. R., and Ruth D. P. , 2003: The new digital forecast database of the National Weather Service. Bull. Amer. Meteor. Soc., 84, 195201, doi:10.1175/BAMS-84-2-195.

    • Search Google Scholar
    • Export Citation
  • Glahn, H. R., Gilbert K. , Cosgrove R. , Ruth D. P. , and Sheets K. , 2009: The gridding of MOS. Wea. Forecasting, 24, 520529, doi:10.1175/2008WAF2007080.1.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 1999: Hypothesis tests for evaluating numerical precipitation forecasts. Wea. Forecasting, 14, 155167, doi:10.1175/1520-0434(1999)014<0155:HTFENP>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Henkel, A., and Peterson C. , 1996: Can deterministic quantitative precipitation forecasts in mountainous regions be specified in a rapid, climatologically consistent manner with Mountain Mapper functioning as the tool for mechanical specification, quality control, and verification? Extended Abstracts, Fifth National Heavy Precipitation Workshop, State College, PA, NWS/NOAA, 31 pp. [Available from Office of Climate, Water, and Weather Services, W/OS, 1325 East–West Hwy., Silver Spring, MD 20910.]

  • Higgins, R. W., Janowiak J. E. , and Yao Y.-P. , 1996: A gridded hourly precipitation data base for the United States (1963–1993). NCEP/Climate Prediction Center Atlas 1, NOAA/NWS, 47 pp.

  • Homar, V., Stensrud D. J. , Levit J. J. , and Bright D. R. , 2006: Value of human-generated perturbations in short-range ensemble forecasts of severe weather. Wea. Forecasting, 21, 347363, doi:10.1175/WAF920.1.

    • Search Google Scholar
    • Export Citation
  • Janjić, Z. I., 2003: A nonhydrostatic model based on a new approach. Meteor. Atmos. Phys.,82, 271–285, doi:10.1007/s00703-001-0587-6.

  • Klein, W. H., and Glahn H. R. , 1974: Forecasting local weather by means of model output statistics. Bull. Amer. Meteor. Soc., 55, 12171227, doi:10.1175/1520-0477(1974)055<1217:FLWBMO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Lin, Y., and Mitchell K. , 2005: The NCEP stage II/IV hourly precipitation analyses: Development and applications. Preprints, 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor. Soc., 1.2. [Available online at https://ams.confex.com/ams/pdfpapers/83847.pdf.]

  • Magnusson, L., and Kallen E. , 2013: Factors influencing skill improvements in the ECMWF forecast system. Mon. Wea. Rev., 141, 31423153, doi:10.1175/MWR-D-12-00318.1.

    • Search Google Scholar
    • Export Citation
  • Mass, C. F., 2003: IFPS and the future of the National Weather Service. Wea. Forecasting, 18, 7579, doi:10.1175/1520-0434(2003)018<0075:IATFOT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • McCarthy, P. J., Ball D. , and Purcell W. , 2007: Project Phoenix: Optimizing the machine–person mix in high-impact weather forecasting. Preprints, 22nd Conf. on Weather Analysis and Forecasting/18th Conf. on Numerical Weather Prediction, Park City, UT, Amer. Meteor. Soc., 6A.5. [Available online at https://ams.confex.com/ams/pdfpapers/122657.pdf.]

  • Murphy, A. H., 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8, 281293, doi:10.1175/1520-0434(1993)008<0281:WIAGFA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Novak, D. R., Bright D. R. , and Brennan M. J. , 2008: Operational forecaster uncertainty needs and future roles. Wea. Forecasting, 23, 10691084, doi:10.1175/2008WAF2222142.1.

    • Search Google Scholar
    • Export Citation
  • Olson, D. A., Junker N. W. , and Korty B. , 1995: Evaluation of 33 years of quantitative precipitation forecasting at the NMC. Wea. Forecasting, 10, 498511, doi:10.1175/1520-0434(1995)010<0498:EOYOQP>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Reynolds, D., 2003: Value-added quantitative precipitation forecasts: How valuable is the forecaster? Bull. Amer. Meteor. Soc., 84, 876878, doi:10.1175/BAMS-84-7-876.

    • Search Google Scholar
    • Export Citation
  • Roebber, P. J., 2010: Seeking consensus: A new approach. Mon. Wea. Rev., 138, 44024415, doi:10.1175/2010MWR3508.1.

  • Roebber, P. J., Schultz D. M. , Colle B. A. , and Stensrud D. J. , 2004: Toward improved prediction: High-resolution and ensemble modeling systems in operations. Wea. Forecasting, 19, 936949, doi:10.1175/1520-0434(2004)019<0936:TIPHAE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Roebber, P. J., Westendorf M. , and Meadows G. R. , 2010: Innovative weather: A new strategy for student, university, and community relationships. Bull. Amer. Meteor. Soc., 91, 877888, doi:10.1175/2010BAMS2854.1.

    • Search Google Scholar
    • Export Citation
  • Ruth, D. P., Glahn B. , Dagostaro V. , and Gilbert K. , 2009: The performance of MOS in the digital age. Wea. Forecasting, 24, 504519, doi:10.1175/2008WAF2222158.1.

    • Search Google Scholar
    • Export Citation
  • Stuart, N. A., and Coauthors, 2006: The future of humans in an increasingly automated forecast process. Bull. Amer. Meteor. Soc., 87, 14971501, doi:10.1175/BAMS-87-11-1497.

    • Search Google Scholar
    • Export Citation
  • Stuart, N. A., Schultz D. M. , and Klein G. , 2007: Maintaining the role of humans in the forecast process: Analyzing the psyche of expert forecasters. Bull. Amer. Meteor. Soc., 88, 18931898, doi:10.1175/BAMS-88-12-1893.

    • Search Google Scholar
    • Export Citation
  • Voisin, N., Schaake J. C. , and Lettenmaier D. P. , 2010: Calibration and downscaling methods for quantitative ensemble precipitation forecasts. Wea. Forecasting, 25, 16031627, doi:10.1175/2010WAF2222367.1.

    • Search Google Scholar
    • Export Citation
  • Yussouf, N., and Stensrud D. J. , 2006: Prediction of near-surface variables at independent locations from a bias-corrected ensemble forecasting system. Mon. Wea. Rev., 134, 34153424, doi:10.1175/MWR3258.1.

    • Search Google Scholar
    • Export Citation
  • Zheng, M., Chang E. K. M. , and Colle B. A. , 2013: Ensemble sensitivity tools for assessing extratropical cyclone intensity and track predictability. Wea. Forecasting, 28, 1133–1156.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 2779 2482 63
PDF Downloads 971 671 33

Precipitation and Temperature Forecast Performance at the Weather Prediction Center

View More View Less
  • 1 NOAA/NWS/NCEP/Weather Prediction Center, College Park, Maryland
© Get Permissions Rent on DeepDyve
Restricted access

Abstract

The role of the human forecaster in improving upon the accuracy of numerical weather prediction is explored using multiyear verification of human-generated short-range precipitation forecasts and medium-range maximum temperature forecasts from the Weather Prediction Center (WPC). Results show that human-generated forecasts improve over raw deterministic model guidance. Over the past two decades, WPC human forecasters achieved a 20%–40% improvement over the North American Mesoscale (NAM) model and the Global Forecast System (GFS) for the 1 in. (25.4 mm) (24 h)−1 threshold for day 1 precipitation forecasts, with a smaller, but statistically significant, 5%–15% improvement over the deterministic ECMWF model. Medium-range maximum temperature forecasts also exhibit statistically significant improvement over GFS model output statistics (MOS), and the improvement has been increasing over the past 5 yr. The quality added by humans for forecasts of high-impact events varies by element and forecast projection, with generally large improvements when the forecaster makes changes ≥8°F (4.4°C) to MOS temperatures. Human improvement over guidance for extreme rainfall events [3 in. (76.2 mm) (24 h)−1] is largest in the short-range forecast. However, human-generated forecasts failed to outperform the most skillful downscaled, bias-corrected ensemble guidance for precipitation and maximum temperature available near the same time as the human-modified forecasts. Thus, as additional downscaled and bias-corrected sensible weather element guidance becomes operationally available, and with the support of near-real-time verification, forecaster training, and tools to guide forecaster interventions, a key test is whether forecasters can learn to make statistically significant improvements over the most skillful of this guidance. Such a test can inform to what degree, and just how quickly, the role of the forecaster changes.

Corresponding author address: David R. Novak, NOAA/NWS/Weather Prediction Center, 5830 University Research Ct., Rm. 4633, College Park, MD 20740. E-mail: david.novak@noaa.gov

Abstract

The role of the human forecaster in improving upon the accuracy of numerical weather prediction is explored using multiyear verification of human-generated short-range precipitation forecasts and medium-range maximum temperature forecasts from the Weather Prediction Center (WPC). Results show that human-generated forecasts improve over raw deterministic model guidance. Over the past two decades, WPC human forecasters achieved a 20%–40% improvement over the North American Mesoscale (NAM) model and the Global Forecast System (GFS) for the 1 in. (25.4 mm) (24 h)−1 threshold for day 1 precipitation forecasts, with a smaller, but statistically significant, 5%–15% improvement over the deterministic ECMWF model. Medium-range maximum temperature forecasts also exhibit statistically significant improvement over GFS model output statistics (MOS), and the improvement has been increasing over the past 5 yr. The quality added by humans for forecasts of high-impact events varies by element and forecast projection, with generally large improvements when the forecaster makes changes ≥8°F (4.4°C) to MOS temperatures. Human improvement over guidance for extreme rainfall events [3 in. (76.2 mm) (24 h)−1] is largest in the short-range forecast. However, human-generated forecasts failed to outperform the most skillful downscaled, bias-corrected ensemble guidance for precipitation and maximum temperature available near the same time as the human-modified forecasts. Thus, as additional downscaled and bias-corrected sensible weather element guidance becomes operationally available, and with the support of near-real-time verification, forecaster training, and tools to guide forecaster interventions, a key test is whether forecasters can learn to make statistically significant improvements over the most skillful of this guidance. Such a test can inform to what degree, and just how quickly, the role of the forecaster changes.

Corresponding author address: David R. Novak, NOAA/NWS/Weather Prediction Center, 5830 University Research Ct., Rm. 4633, College Park, MD 20740. E-mail: david.novak@noaa.gov
Save