Editorial Type:
Article
Article Type:
Research Article

Comparison of Methodologies for Probabilistic Quantitative Precipitation Forecasting

Scott Applequist Geophysical Fluid Dynamics Institute and Meteorology Department, The Florida State University, Tallahassee, Florida

Search for other papers by Scott Applequist in
Current site
Google Scholar
PubMed Close
,
Gregory E. Gahrs Geophysical Fluid Dynamics Institute and Meteorology Department, The Florida State University, Tallahassee, Florida

Search for other papers by Gregory E. Gahrs in
Current site
Google Scholar
PubMed Close
,
Richard L. Pfeffer Geophysical Fluid Dynamics Institute and Meteorology Department, The Florida State University, Tallahassee, Florida

Search for other papers by Richard L. Pfeffer in
Current site
Google Scholar
PubMed Close
, and
Xu-Feng Niu Department of Statistics, The Florida State University, Tallahassee, Florida

Search for other papers by Xu-Feng Niu in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Aug 2002
DOI:
https://doi.org/10.1175/1520-0434(2002)017<0783:COMFPQ>2.0.CO;2
Page(s):
783–799
Restricted access

Abstract

Twenty-four-hour probabilistic quantitative precipitation forecasts (PQPFs) for accumulations exceeding thresholds of 0.01, 0.05, and 0.10 in. are produced for 154 meteorological stations over the eastern and central regions of the United States. Comparisons of skill are made among forecasts generated using five different linear and nonlinear statistical methodologies, namely, linear regression, discriminant analysis, logistic regression, neural networks, and a classifier system. The predictors for the different statistical models were selected from a large pool of analyzed and predicted variables generated by the Nested Grid Model (NGM) during the four cool seasons (December–March) from 1992/93 to 1995/96. Because linear regression is the current method used by the National Weather Service, it is chosen as the benchmark by which the other methodologies are compared. The results indicate that logistic regression performs best among all methodologies. Most notable is that it performs significantly better at the 99% confidence limits than linear regression, attaining Brier skill scores of 0.413, 0.480, and 0.478 versus 0.378, 0.440, and 0.457 for linear regression, at thresholds of 0.01, 0.05, and 0.10 in., respectively. Attributes diagrams reveal that linear regression gives a greater number of forecast probabilities closer to climatology than does logistic regression at all three thresholds. Moreover, these forecasts are more biased toward lower-than-observed probabilities and are further from the “perfect reliability” line in almost all probability categories than are the forecasts made by logistic regression. For the other methodologies, the classifier system also showed significantly greater skill than did linear regression, and discriminant analysis and neural networks gave mixed results.

Current affiliation: Air Force Weather Agency, Offutt Air Force Base, Nebraska

Corresponding author address: Gregory E. Gahrs, Geophysical Fluid Dynamics Institute, The Florida State University, 18 Keen Bldg., Tallahassee, FL 32306-4360. Email: gahrs@gfdi.fsu.edu

Abstract

Twenty-four-hour probabilistic quantitative precipitation forecasts (PQPFs) for accumulations exceeding thresholds of 0.01, 0.05, and 0.10 in. are produced for 154 meteorological stations over the eastern and central regions of the United States. Comparisons of skill are made among forecasts generated using five different linear and nonlinear statistical methodologies, namely, linear regression, discriminant analysis, logistic regression, neural networks, and a classifier system. The predictors for the different statistical models were selected from a large pool of analyzed and predicted variables generated by the Nested Grid Model (NGM) during the four cool seasons (December–March) from 1992/93 to 1995/96. Because linear regression is the current method used by the National Weather Service, it is chosen as the benchmark by which the other methodologies are compared. The results indicate that logistic regression performs best among all methodologies. Most notable is that it performs significantly better at the 99% confidence limits than linear regression, attaining Brier skill scores of 0.413, 0.480, and 0.478 versus 0.378, 0.440, and 0.457 for linear regression, at thresholds of 0.01, 0.05, and 0.10 in., respectively. Attributes diagrams reveal that linear regression gives a greater number of forecast probabilities closer to climatology than does logistic regression at all three thresholds. Moreover, these forecasts are more biased toward lower-than-observed probabilities and are further from the “perfect reliability” line in almost all probability categories than are the forecasts made by logistic regression. For the other methodologies, the classifier system also showed significantly greater skill than did linear regression, and discriminant analysis and neural networks gave mixed results.

Current affiliation: Air Force Weather Agency, Offutt Air Force Base, Nebraska

Corresponding author address: Gregory E. Gahrs, Geophysical Fluid Dynamics Institute, The Florida State University, 18 Keen Bldg., Tallahassee, FL 32306-4360. Email: gahrs@gfdi.fsu.edu

Save
Cover Weather and Forecasting
Weather and Forecasting

  • Antolik, M. S., 1995: NGM-based quantitative precipitation forecast guidance: Performance tests and practical applications. Preprints, 14th Conf. on Weather Analysis and Forecasting, Dallas, TX, Amer. Meteor. Soc., 182–187.

    • Search Google Scholar
    • Export Citation

  • Antolik, M. S., . 1997: NGM-based statistical quantitative precipitation forecast guidance for the contiguous United States and Alaska. NWS Tech. Procedures Bull. 461, 28 pp.

    • Search Google Scholar
    • Export Citation

  • Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78 , 1–3.

  • Brunet, N., Verret R. , and Yacowar N. , 1988: An objective comparison of model output statistics and “perfect prog” systems in producing numerical weather element forecasts. Wea. Forecasting, 3 , 273–283.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Carter, M. M., and Elsner J. B. , 1996: Convective rainfall regions of Puerto Rico. Int. J. Climatol., 16 , 1033–1043.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Charba, J. P., 1998: The LAMP QPF products. Part I: Model development. Wea. Forecasting, 13 , 934–962.

  • Dagostaro, V. J., and Dallavalle J. P. , 1997: AFOS–ERA verification of guidance and local aviation/public weather forecasts—No. 23 (October 1994–March 1995). TDL Office Note 97-3, 52 pp.

    • Search Google Scholar
    • Export Citation

  • Davis, L., 1987: Genetic Algorithms and Simulated Annealing. Pitman Publishing, 216 pp.

  • Draper, N. R., and Smith H. , 1966: Applied Regression Analysis. John Wiley and Sons, 709 pp.

  • Elsner, J. B., and Schmertmann C. P. , 1994: Assessing forecast skill through cross validation. Wea. Forecasting, 9 , 619–624.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Funk, T. W., 1991: Forecasting techniques utilized by the Forecast Branch of the National Meteorological Center during a major convective rainfall event. Wea. Forecasting, 6 , 548–564.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Glahn, H. R., and Lowry D. A. , 1972: Use of model output statistics (MOS) in objective weather forecasting. J. Appl. Meteor., 11 , 1203–1211.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Goldberg, D. E., 1989: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, 412 pp.

  • Hall, T., Brooks H. E. , and Doswell C. A. , 1999: Precipitation forecasting using a neural network. Wea. Forecasting, 14 , 338–345.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Hoel, P. G., 1962: Introduction to Mathematical Statistics. John Wiley and Sons, 427 pp.

  • Hsu, W-R., and Murphy A. H. , 1986: The attributes diagram: A geometrical framework for assessing the quality of probability forecasts. Int. J. Forecasting, 2 , 285–293.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Junker, N. W., Hoke J. E. , Sullivan B. E. , Brill K. F. , and Hughes F. J. , 1992: Seasonal and geographic variations in the quantitative precipitation prediction by NMC's Nested-Grid Model and the Medium-Range Forecast Model. Wea. Forecasting, 7 , 410–429.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Koizumi, K., 1999: An objective method to modify numerical model forecasts with newly given weather data using an artificial neural network. Wea. Forecasting, 14 , 109–118.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Krzysztofowicz, R., Drzal W. J. , Drake T. R. , Weyman J. C. , and Giordano L. A. , 1993: Probabilistic quantitative precipitation forecasts for river basins. Wea. Forecasting, 8 , 424–439.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lawrence, J., 1991: Introduction to Neural Networks. California Scientific Software, 213 pp.

  • Lindner, A. J., and Krein A. S. , 1993: A neural network for forecasting heavy precipitation. Preprints, 13th Conf. on Weather Analysis and Forecasting, Vienna, VA, Amer. Meteor. Soc., 612–615.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E. N., 1977: An experiment in nonlinear statistical weather forecasting. Mon. Wea. Rev., 105 , 590–602.

  • MathSoft, 1999: S-Plus 2000 Guide to Statistics. Vol. 2. Data Analysis Products Division, MathSoft, Inc., Seattle, WA, 582 pp.

  • McCann, D. W., 1992: A neural network short term forecast of significant thunderstorms. Wea. Forecasting, 7 , 525–534.

  • Miller, R. G., 1962: Statistical Prediction by Discriminant Analysis. Meteor. Monogr., No. 25, Amer. Meteor. Soc., 53 pp.

  • Murphy, A. H., 1973: A new vector partition of the probability score. J. Appl. Meteor., 12 , 595–600.

  • Murphy, A. H., and Winkler R. L. , 1987: A general framework for forecast verification. Mon. Wea. Rev., 115 , 1330–1338.

  • Olson, D. A., Junker N. W. , and Korty B. , 1995: Evaluation of 33 years of quantitative precipitation forecasting at NMC. Wea. Forecasting, 10 , 498–511.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Schaefer, J. T., 1990: Critical success index as an indicator of warning skill. Wea. Forecasting, 5 , 570–575.

  • Su, J. C., 1993: NGM-based MOS guidance for the probability of precipitation (PoP). NWS Tech. Procedures Bull. 409, 14 pp.

  • Vislocky, R. L., and Young G. Y. , 1989: The use of perfect prog forecasts to improve model output statistics forecasts of precipitation probability. Wea. Forecasting, 4 , 202–209.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Vislocky, R. L., and Fritsch J. M. , 1995a: Generalized additive models versus linear regression in generating probabilistic MOS forecasts of aviation weather parameters. Wea. Forecasting, 10 , 669–680.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Vislocky, R. L., . 1995b: Improved model output statistics forecasts through model consensus. Bull. Amer. Meteor. Soc., 76 , 1157–1164.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Walker, S. H., and Duncan D. B. , 1967: Estimation of the probability of an event as a function of several independent variables. Biometrika, 54 , 167–179.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Weiss, N. A., and Hassett M. J. , 1991: Introductory Statistics. Addison-Wesley, 834 pp.

  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 467 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1832 1208 294
PDF Downloads 990 564 6