• Ahijevych, D., , Gilleland E. , , Brown B. , , and Ebert E. , 2009: Application of spatial verification methods to idealized and NWP-gridded precipitation forecasts. Wea. Forecasting, 24 , 14851497.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ament, F., , Bähler T. , , Weusthoff T. , , Buzzi M. , , and Damrath U. , 2008: Results of fuzzy verification methods with COSMO over Switzerland and Germany. 30th European Weather Group for Limited Area Modelling and 15th Short-Range Numerical Weather Prediction Network Meeting, Madrid, Spain, EUMETNET. [Available online at http://srnwp.met.hu/Annual_Meetings/2008/download/oct7/afternoon/fuzzy-verification_arpagaus.pdf].

    • Search Google Scholar
    • Export Citation
  • Atger, F., 2001: Verification of intense precipitation forecasts from single models and ensemble prediction systems. Nonlinear Processes Geophys., 8 , 401417.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baldwin, M. E., , and Kain J. S. , 2006: Sensitivity of several performance measures to displacement error, bias, and event frequency. Wea. Forecasting, 21 , 636648.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barnes, L. R., , Gruntfest E. C. , , Hayden M. H. , , Schultz D. M. , , and Benight C. , 2007: False alarms and close calls: A conceptual model of warning accuracy. Wea. Forecasting, 22 , 11401147.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brooks, H. E., , Kay M. , , and Hart J. A. , 1998: Objective limits on forecasting skill of rare events. Preprints, 19th Conf. on Severe Local Storms, Minneapolis, MN, Amer. Meteor. Soc., 552–555.

    • Search Google Scholar
    • Export Citation
  • Casati, B., and Coauthors, 2008: Forecast verification: Current status and future directions. Meteor. Appl., 15 , 318.

  • Damrath, U., 2004: Verification against precipitation observations of a high density network—What did we learn? Int. Verification Methods Workshop, Montreal, QC, Canada, World Weather Research Programme. [Available online at http://www.bom.gov.au/bmrc/wefor/staff/eee/verif/Workshop2004/MeetingProgram.html].

    • Search Google Scholar
    • Export Citation
  • Davis, C., , Brown B. , , and Bullock R. , 2006: Object-based verification of precipitation forecasts. Part I: Methods and application to mesoscale rain areas. Mon. Wea. Rev., 134 , 17721784.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Davis, C., , Brown B. , , Bullock R. , , and Halley-Gotway J. , 2009: The method for object-based diagnostic evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC Spring Program. Wea. Forecasting, 24 , 12521267.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., 2008: Fuzzy verification of high resolution gridded forecasts: A review and proposed framework. Meteor. Appl., 15 , 5164.

  • Ebert, E. E., , and McBride J. L. , 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239 , 179202.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., , and Gallus W. A. Jr., 2009: Toward better understanding of the contiguous rain area (CRA) method for spatial forecast verification. Wea. Forecasting, 24 , 14011415.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Germann, U., , and Zawadzki I. , 2004: Scale dependence of the predictability of precipitation from continental radar images. Part II: Probability forecasts. J. Appl. Meteor., 43 , 7489.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gilbert, G. K., 1884: Finley’s tornado predictions. Amer. Meteor. J., 1 , 166172.

  • Gilleland, E., , Ahijevych D. , , Brown B. , , and Ebert E. , 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24 , 14161430.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., , and Stephenson D. B. , 2003: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. Wiley and Sons, 240 pp.

    • Search Google Scholar
    • Export Citation
  • JWGV, cited. 2009: Forecast verification: Issues, methods, and FAQ. Joint Working Group on Forecast Verification Research, WWRP/WGNE. [Available online at http://www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html].

    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and Coauthors, 2008: Some practical considerations regarding horizontal resolution in the first generation of operational convection-allowing NWP. Wea. Forecasting, 23 , 931952.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, Y-L., , and Mitchell K. , 2005: The NCEP stage II/IV hourly precipitation analyses: development and applications. Preprints, 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor. Soc., 1.2. [Available online at http://ams.confex.com/ams/pdfpapers/83847.pdf].

    • Search Google Scholar
    • Export Citation
  • Marsigli, C., , Boccanera F. , , Montani A. , , and Paccagnella T. , 2005: The COSMO-LEPS mesoscale ensemble system: Validation of the methodology and verification. Nonlinear Processes Geophys., 12 , 527536.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mittermaier, M., , and Roberts N. , 2010: Intercomparison of spatial forecast verification methods: Identifying skillful spatial scales using the fractions skill score. Wea. Forecasting, in press.

    • Search Google Scholar
    • Export Citation
  • Rezacova, D., , Sokol Z. , , and Pesice P. , 2007: A radar-based verification of precipitation forecast for local convective storms. Atmos. Res., 83 , 211224.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roberts, N. M., , and Lean H. W. , 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136 , 7897.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Segawa, T., , and Honda Y. , 2007: The verification of high-resolution precipitation forecasts of the operational JMA mesoscale model. Third Int. Verification Methods, Workshop, Reading, United Kingdom, World Weather Research Programme. [Available online at http://www.ecmwf.int/newsevents/meetings/workshops/2007/jwgv/workshop_presentations/index.html].

    • Search Google Scholar
    • Export Citation
  • Theis, S. E., , Hense A. , , and Damrath U. , 2005: Probabilistic precipitation forecasts from a deterministic model: A pragmatic approach. Meteor. Appl., 12 , 257268.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tustison, B., , Harris D. , , and Foufoula-Georgiou E. , 2001: Scale issues in verification of precipitation forecasts. J. Geophys. Res., 106 , 1177511784.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weygandt, S. S., , Loughe A. F. , , Benjamin S. G. , , and Mahoney J. L. , 2004: Scale sensitivities in model precipitation skill scores during IHOP. Preprints, 22nd Conf. on Severe Local Storms, Hyannis, MA, Amer. Meteor. Soc., 16A.8. [Available online at http://ams.confex.com/ams/pdfpapers/81986.pdf].

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Elsevier, 627 pp.

  • Yates, E., , Anquetin S. , , Ducrocq V. , , Creutin J-D. , , Ricard D. , , and Chancibault K. , 2006: Point and areal validation of forecast precipitation fields. Meteor. Appl., 13 , 120.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zepeda-Arce, J., , Foufoula-Georgiou E. , , and Droegemeier K. K. , 2000: Space–time rainfall organization and its role in validating quantitative precipitation forecasts. J. Geophys. Res., 105 , 1012910146.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 170 170 23
PDF Downloads 1170 1170 25

Neighborhood Verification: A Strategy for Rewarding Close Forecasts

View More View Less
  • 1 Centre for Australian Weather and Climate Research, Bureau of Meteorology, Melbourne, Victoria, Australia
© Get Permissions
Restricted access

Abstract

High-resolution forecasts may be quite useful even when they do not match the observations exactly. Neighborhood verification is a strategy for evaluating the “closeness” of the forecast to the observations within space–time neighborhoods rather than at the grid scale. Various properties of the forecast within a neighborhood can be assessed for similarity to the observations, including the mean value, fractional coverage, occurrence of a forecast event sufficiently near an observed event, and so on. By varying the sizes of the neighborhoods, it is possible to determine the scales for which the forecast has sufficient skill for a particular application. Several neighborhood verification methods have been proposed in the literature in the last decade. This paper examines four such methods in detail for idealized and real high-resolution precipitation forecasts, highlighting what can be learned from each of the methods. When applied to idealized and real precipitation forecasts from the Spatial Verification Methods Intercomparison Project, all four methods showed improved forecast performance for neighborhood sizes larger than grid scale, with the optimal scale for each method varying as a function of rainfall intensity.

Corresponding author address: Dr. Elizabeth E. Ebert, Centre for Australian Weather and Climate Research, Bureau of Meteorology, GPO Box 1289, Melbourne, VIC 3001, Australia. Email: e.ebert@bom.gov.au

This article included in the Spatial Forecast Verification Methods Inter-Comparison Project (ICP) special collection.

Abstract

High-resolution forecasts may be quite useful even when they do not match the observations exactly. Neighborhood verification is a strategy for evaluating the “closeness” of the forecast to the observations within space–time neighborhoods rather than at the grid scale. Various properties of the forecast within a neighborhood can be assessed for similarity to the observations, including the mean value, fractional coverage, occurrence of a forecast event sufficiently near an observed event, and so on. By varying the sizes of the neighborhoods, it is possible to determine the scales for which the forecast has sufficient skill for a particular application. Several neighborhood verification methods have been proposed in the literature in the last decade. This paper examines four such methods in detail for idealized and real high-resolution precipitation forecasts, highlighting what can be learned from each of the methods. When applied to idealized and real precipitation forecasts from the Spatial Verification Methods Intercomparison Project, all four methods showed improved forecast performance for neighborhood sizes larger than grid scale, with the optimal scale for each method varying as a function of rainfall intensity.

Corresponding author address: Dr. Elizabeth E. Ebert, Centre for Australian Weather and Climate Research, Bureau of Meteorology, GPO Box 1289, Melbourne, VIC 3001, Australia. Email: e.ebert@bom.gov.au

This article included in the Spatial Forecast Verification Methods Inter-Comparison Project (ICP) special collection.

Save