Editorial Type:
Article
Article Type:
Research Article

Analyzing the Image Warp Forecast Verification Method on Precipitation Fields from the ICP

Eric Gilleland Research Applications Laboratory, National Center for Atmospheric Research, * Boulder, Colorado

Search for other papers by Eric Gilleland in
Current site
Google Scholar
PubMed Close
,
Johan Lindström Mathematical Statistics, Centre for Mathematical Sciences, Lund University, Lund, Sweden, and Department of Statistics, University of Washington, Seattle, Washington

Search for other papers by Johan Lindström in
Current site
Google Scholar
PubMed Close
, and
Finn Lindgren Mathematical Statistics, Centre for Mathematical Sciences, Lund University, Lund, Sweden

Search for other papers by Finn Lindgren in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Aug 2010
Collections:
Spatial Forecast Verification Methods Inter-Comparison Project (ICP)
DOI:
https://doi.org/10.1175/2010WAF2222365.1
Page(s):
1249–1262
Restricted access

We are aware of a technical issue preventing figures and tables from showing in some newly published articles in the full-text HTML view.
While we are resolving the problem, please use the online PDF version of these articles to view figures and tables.

Abstract

Image warping for spatial forecast verification is applied to the test cases employed by the Spatial Forecast Verification Intercomparison Project (ICP), which includes both real and contrived cases. A larger set of cases is also used to investigate aggregating results for summarizing forecast performance over a long record of forecasts. The technique handles the geometric and perturbed cases with nearly exact precision, as would be expected. A statistic, dubbed here the IWS for image warp statistic, is proposed for ranking multiple forecasts and tested on the perturbed cases. IWS rankings for perturbed and real test cases are found to be sensible and physically interpretable. A powerful result of this study is that the image warp can be employed using a relatively sparse, preset regular grid without having to first identify features.

Corresponding author address: Eric Gilleland, Research Applications Laboratory, National Center for Atmospheric Research, P.O. Box 3000, Boulder, CO 80301-3000. Email: ericg@ucar.edu

This article included in the Spatial Forecast Verification Methods Inter-Comparison Project (ICP) special collection.

Abstract

Image warping for spatial forecast verification is applied to the test cases employed by the Spatial Forecast Verification Intercomparison Project (ICP), which includes both real and contrived cases. A larger set of cases is also used to investigate aggregating results for summarizing forecast performance over a long record of forecasts. The technique handles the geometric and perturbed cases with nearly exact precision, as would be expected. A statistic, dubbed here the IWS for image warp statistic, is proposed for ranking multiple forecasts and tested on the perturbed cases. IWS rankings for perturbed and real test cases are found to be sensible and physically interpretable. A powerful result of this study is that the image warp can be employed using a relatively sparse, preset regular grid without having to first identify features.

Corresponding author address: Eric Gilleland, Research Applications Laboratory, National Center for Atmospheric Research, P.O. Box 3000, Boulder, CO 80301-3000. Email: ericg@ucar.edu

This article included in the Spatial Forecast Verification Methods Inter-Comparison Project (ICP) special collection.

Save
Cover Weather and Forecasting
Weather and Forecasting

  • Åberg, S., Lindgren F. , Malmberg A. , Holst J. , and Holst U. , 2005: An image warping approach to spatio–temporal modelling. Environmetrics, 16 , 833–848.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ahijevych, D., Gilleland E. , Brown B. , and Ebert E. , 2009: Application of spatial verification methods to idealized and NWP-gridded precipitation forecasts. Wea. Forecasting, 24 , 1485–1497.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Alexander, G., Weinman J. , Karyampudi V. , Olson W. , and Lee A. , 1999: The effect of assimilating rain rates derived from satellites and lightning on forecasts on the 1993 Superstorm. Mon. Wea. Rev., 127 , 1433–1457.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Baldwin, M., and Elmore K. , 2005: Objective verification of high-resolution WRF forecasts during 2005 NSSL/SPC Spring Program. Preprints, 21st Conf. on Weather Analysis and Forecasting/17th Conf. on Numerical Weather Prediction, Washington, DC, Amer. Meteor. Soc., 11B.4. [Available online at http://ams.confex.com/ams/pdfpapers/95172.pdf].

    • Search Google Scholar
    • Export Citation

  • Briggs, W., and Levine R. , 1997: Wavelets and field forecast verification. Mon. Wea. Rev., 125 , 1329–1341.

  • Brill, K., and Mesinger F. , 2009: Applying a general analytic method for assessing bias sensitivity to bias-adjusted threat and equitable threat scores. Wea. Forecasting, 24 , 1748–1754.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Casati, B., 2010: New developments of the intensity-scale technique within the Spatial Verification Methods Intercomparison Project. Wea. Forecasting, 25 , 113–143.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Casati, B., Ross G. , and Stephenson D. , 2004: A new intensity-scale approach for the verification of spatial precipitation forecasts. Meteor. Appl., 11 , 141–154.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Davis, C., Brown B. , and Bullock R. , 2006: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134 , 1772–1784.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Davis, C., Brown B. , Bullock R. , and Halley Gotway J. , 2009: The method for object-based diagnostic evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC Spring Program. Wea. Forecasting, 24 , 1252–1267.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Dryden, I., and Mardia K. , 1998: Statistical Shape Analysis. J. Wiley, 347 pp.

  • Ebert, E., 2008: Fuzzy verification of high resolution gridded forecasts: A review and proposed framework. Meteor. Appl., 15 , 51–64. doi:10.1002/met.25.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ebert, E., 2009: Neighborhood verification: A strategy for rewarding close forecasts. Wea. Forecasting, 24 , 1498–1510.

  • Ebert, E., and McBride J. , 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239 , 179–202.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ebert, E., and Gallus W. Jr., 2009: Toward better understanding of the contiguous rain area (CRA) method for spatial forecast verification. Wea. Forecasting, 24 , 1401–1415.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Fletcher, R., 1987: Practical Methods of Optimization. 2nd ed. J. Wiley, 450 pp.

  • Gilleland, E., Ahijevych D. , Casati B. , and Ebert B. , 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24 , 1416–1430.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Glasbey, C., and Nevison I. , 1997: Rainfall modelling using a latent Gaussian variable. Lect. Notes Stat., 122 , 233–242.

  • Glasbey, C., and Mardia K. , 1998: A review of image warping methods. J. Appl. Stat., 25 , 155–171.

  • Glasbey, C., and Mardia K. , 2001: A penalized likelihood approach to image warping. J. Roy. Stat. Soc., 63B , 465–514.

  • Goshtasby, A., 1987: Piecewise cubic mapping functions for image registration. Pattern Recognit., 20 , 523–533.

  • Harris, D., Foufoula-Georgiou E. , Droegemeier K. , and Levit J. , 2001: Multiscale statistical properties of a high-resolution precipitation forecast. J. Hydrometeor., 2 , 406–418.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Hoffman, R., Liu Z. , Louis J. , and Grassotti C. , 1995: Distortion representation of forecast errors. Mon. Wea. Rev., 123 , 2758–2770.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Jolliffe, I., and Stephenson D. , 2003: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. J. Wiley and Sons, 254 pp.

    • Search Google Scholar
    • Export Citation

  • Kain, J., Weiss S. , Bright D. , Baldwin M. , and Levit J. , 2008: Some practical considerations regarding horizontal resolution in the first generation of operational convection-allowing NWP. Wea. Forecasting, 23 , 931–952.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Keil, C., and Craig G. , 2007: A displacement-based error measure applied in a regional ensemble forecasting system. Mon. Wea. Rev., 135 , 3248–3259.

  • Keil, C., and Craig G. , 2009: A displacement and amplitude score employing an optical flow technique. Wea. Forecasting, 24 , 1297–1308.

  • Lack, S., Limpert G. , and Fox N. , 2010: An object-oriented multiscale verification scheme. Wea. Forecasting, 25 , 79–92.

  • Lakshmanan, V., and Kain J. , 2010: A Gaussian mixture model approach to forecast verification. Wea. Forecasting, 25 , 908–920.

  • Lee, S., Wolberg G. , and Shin S. , 1997: Scattered data interpolation with multilevel b-splines. IEEE Trans. Vis. Comput. Graph., 3 , 228–244.

  • Marzban, C., and Sandgathe S. , 2006: Cluster analysis for verification of precipitation fields. Wea. Forecasting, 21 , 824–838.

  • Marzban, C., and Sandgathe S. , 2008: Cluster analysis for object-oriented verification of fields: A variation. Mon. Wea. Rev., 136 , 1013–1025.

  • Marzban, C., and Sandgathe S. , 2009: Verification with variograms. Wea. Forecasting, 24 , 1102–1120.

  • Marzban, C., Sandgathe S. , Lyons H. , and Lederer N. , 2009: Three spatial verification techniques: Cluster analysis, variogram, and optical flow. Wea. Forecasting, 24 , 1457–1471.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Mesinger, F., 2008: Bias adjusted precipitation threat scores. Adv. Geosci., 16 , 137–143.

  • Micheas, A., Fox N. , Lack S. , and Wikle C. , 2007: Cell identification and verification of QPF ensembles using shape analysis techniques. J. Hydrol., 343 , 105–116.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Mittermaier, M., and Roberts N. , 2010: Intercomparison of spatial forecast verification methods: Identifying skillful spatial scales using the fractions skill score. Wea. Forecasting, 25 , 343–354.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Murphy, A., and Winkler R. , 1987: A general framework for forecast verification. Mon. Wea. Rev., 115 , 1330–1338.

  • Nachamkin, J., 2004: Mesoscale verification using meteorological composites. Mon. Wea. Rev., 132 , 941–955.

  • Nachamkin, J., 2009: Application of the composite method to the Spatial Forecast Verification Methods Intercomparison Dataset. Wea. Forecasting, 24 , 1390–1400.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Nehrkorn, T., Hoffman R. , Grassotti C. , and Louis J-F. , 2003: Feature calibration and alignment to represent model forecast errors: Empirical regularization. Quart. J. Roy. Meteor. Soc., 129 , 195–218.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Reilly, C., Price P. , Gelman A. , and Sandgathe S. , 2004: Using image and curve registration for measuring the goodness of fit of spatial and temporal predictions. Biometrics, 60 , 954–964.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Sampson, P. D., and Guttorp P. , 1992: Nonparametric estimation of nonstationary spatial covariance structure. J. Amer. Stat. Assoc., 87 , 108–119.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Sampson, P. D., and Guttorp P. , 1999: Operational evaluation of air quality models. NRCSE-TRS 018, National Research Center for Statistics and the Environment, Seattle, WA, 22 pp. [Available online at http://www.nrcse.washington.edu/pdf/trs18_aqmodels.pdf].

    • Search Google Scholar
    • Export Citation

  • Skamarock, W. C., Klemp J. , Dudhia J. , Gill D. , Barker D. , Wang W. , and Powers J. , 2005: A description of the Advanced Research WRF version 2. NCAR Tech. Note TN-468+STR, 88 pp.

    • Search Google Scholar
    • Export Citation

  • Venugopal, V., Basu S. , and Foufoula-Georgiou E. , 2005: A new metric for comparing precipitation patterns with an application to ensemble forecasts. J. Geophys. Res., 110 , D08111. doi:10.1029/2004JD005395.

    • Search Google Scholar
    • Export Citation

  • Wernli, H., Paulat M. , Hagen M. , and Frei C. , 2008: SAL—A novel quality measure for the verification of quantitative precipitation forecasts. Mon. Wea. Rev., 136 , 4470–4487.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Wernli, H., Hofmann C. , and Zimmer M. , 2009: Spatial Forecast Verification Methods Intercomparison Project application of the SAL technique. Wea. Forecasting, 24 , 1472–1484.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Wilks, D., 2006: Statistical Methods in the Atmospheric Sciences. An Introduction. 2nd ed. Academic Press, 627 pp.

  • Zepeda-Arce, J., Foufoula-Georgiou E. , and Droegemeier K. , 2000: Space–time rainfall organization and its role in validating quantitative precipitation forecasts. J. Geophys. Res., 105 , 10129–10146.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 3833 2688 635
PDF Downloads 322 106 2