• Ahijevych, D., , Gilleland E. , , Brown B. G. , , and Ebert E. E. , 2009: Application of spatial verification methods to idealized and NWP-gridded precipitation forecasts. Wea. Forecasting, 24 , 14851497.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baldwin, M. E., , Lakshmivarahan S. , , and Kain J. S. , 2002: Development of an “events-oriented” approach to forecast verification. Preprints, 15th Conf. on Numerical Weather Prediction, San Antonio, TX, Amer. Meteor. Soc., 7B.3. [Available online at http://ams.confex.com/ams/pdfpapers/47738.pdf].

    • Search Google Scholar
    • Export Citation
  • Bowler, N. E. H., , Pierce C. E. , , and Seed A. , 2004: Development of a precipitation nowcasting algorithm based on optical flow techniques. J. Hydrol., 288 , 7491.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brown, B. G., and Coauthors, 2004: New verification approaches for convective weather forecasts. Preprints, 11th Conf. on Aviation, Range, and Aerospace, Hyannis, MA, Amer. Meteor. Soc., 9.4. [Available online at http://ams.confex.com/ams/pdfpapers/82068.pdf].

    • Search Google Scholar
    • Export Citation
  • Bullock, R., , Brown B. G. , , Davis C. A. , , Manning K. W. , , and Chapman M. , 2004: An object-oriented approach to quantitative precipitation forecasts: Part I—Methodology. Preprints, 17th Conf. on Probability and Statistics in the Atmospheric Sciences, Seattle, WA, Amer. Meteor. Soc., J12.4. [Available online at http://ams.confex.com/ams/pdfpapers/71819.pdf].

    • Search Google Scholar
    • Export Citation
  • Chapman, M., , Bullock R. , , Brown B. G. , , Davis C. A. , , Manning K. W. , , Morss R. , , and Takacs A. , 2004: An object-oriented approach to the verification of quantitative precipitation forecasts: Part II—Examples. Preprints, 17th Conf. on Probability and Statistics in the Atmospheric Sciences, Seattle, WA, Amer. Meteor. Soc., J12.5. [Available online at http://ams.confex.com/ams/pdfpapers/70881.pdf].

    • Search Google Scholar
    • Export Citation
  • Cressie, N. A. C., 1993: Statistics for Spatial Data. Wiley Series in Probability and Mathematical Statistics, John Wiley and Sons, 900 pp.

    • Search Google Scholar
    • Export Citation
  • Davis, C. A., , Brown B. , , and Bullock R. , 2006a: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134 , 17721784.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Davis, C. A., , Brown B. , , and Bullock R. , 2006b: Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Wea. Rev., 134 , 17851795.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Davis, C. A., , Brown B. , , Bullock R. , , and Halley-Gotway J. , 2009: The method for object-based diagnostic evaluation (MODE) applied to WRF forecasts from the 2005 SPC Spring Program. Wea. Forecasting, 24 , 12521267.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Du, J., , and Mullen S. L. , 2000: Removal of distortion error from an ensemble forecast. Mon. Wea. Rev., 128 , 33473351.

  • Ebert, E. E., , and McBride J. L. , 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239 , 179202.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., , and Gallus W. A. Jr., 2009: Toward better understanding of the contiguous rain area (CRA) method for spatial forecast verification. Wea. Forecasting, 24 , 14011415.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Everitt, B. S., 1980: Cluster Analysis. 2nd ed. Heinemann Educational Books, 136 pp.

  • Gebremichael, M., , Krajewski W. F. , , and Ciach G. , 2004: Assessment of the statistical characterization of small-scale rainfall variability from radar: Analysis of TRMM ground validation datasets. Sixth Int. Symp. on Hydrological Applications of Weather Radar, Melbourne, VIC, Australia, Australian Bureau of Meteorology, 9 pp. [Available online at http://www.cawcr.gov.au/bmrc/basic/old_events/hawr6/hyrometeorology/GEBREMICHAEL_assess.pdf].

    • Search Google Scholar
    • Export Citation
  • Germann, U., , and Joss J. , 2001: Variograms of radar reflectivity to describe the spatial continuity of Alpine precipitation. J. Appl. Meteor., 40 , 10421059.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Germann, U., , and Zawadzki I. , 2002: Scale dependence of the predictability of precipitation from continental radar images. Part I: Description of the methodology. Mon. Wea. Rev., 130 , 28592873.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gilleland, E., , Ahijevych D. , , Brown B. G. , , Casati B. , , and Ebert E. E. , 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24 , 14161430.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harris, D., , Foufoula-Georgiou E. , , Droegemeier K. K. , , and Levit J. J. , 2001: Multiscale statistical properties of a high-resolution precipitation forecast. J. Hydrometeor., 2 , 406418.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hoffman, R. N., , Liu Z. , , Louis J-F. , , and Grassotti C. , 1995: Distortion representation of forecast errors. Mon. Wea. Rev., 123 , 27582770.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Keil, C., , and Craig G. C. , 2007: A displacement-based error measure applied in a regional ensemble forecasting system. Mon. Wea. Rev., 135 , 32483259.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Keil, C., , and Craig G. C. , 2009: A displacement and amplitude score employing an optical flow technique. Wea. Forecasting, 24 , 12971308.

  • Lucas, B. D., , and Kanade T. , 1981: An iterative image registration technique with an application to stereo vision. Proc. Imaging Understanding Workshop, Pittsburg, PA, Carnegie-Mellon, University, 121–130.

    • Search Google Scholar
    • Export Citation
  • Marzban, C., 1998: Scalar measures of performance in rare-event situations. Wea. Forecasting, 13 , 753763.

  • Marzban, C., , and Sandgathe S. , 2006: Cluster analysis for verification of precipitation fields. Wea. Forecasting, 21 , 824838.

  • Marzban, C., , and Sandgathe S. , 2008: Cluster analysis for object-oriented verification of fields: A variation. Mon. Wea. Rev., 136 , 10131025.

  • Marzban, C., , and Sandgathe S. , 2009a: Verification with variograms. Wea. Forecasting, 24 , 11021120.

  • Marzban, C., , Sandgathe S. , , and Lyons H. , 2008: An object-oriented verification of three NWP model formulations via cluster analysis: An objective and a subjective analysis. Mon. Wea. Rev., 136 , 33923407.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Politis, D. N., , and Romano J. P. , 1994: The stationary bootstrap. J. Amer. Stat. Assoc., 89 , 13031313.

  • Politis, D. N., , Romano J. P. , , and Wolf M. , 1999: Subsampling. Springer, 347 pp.

  • Zepeda-Arce, J., , Foufoula-Georgiou E. , , and Droegemeier K. , 2000: Space–time rainfall organization and its role in validating quantitative precipitation forecasts. J. Geophys. Res., 105 , 1012910146.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 54 54 10
PDF Downloads 52 52 12

Three Spatial Verification Techniques: Cluster Analysis, Variogram, and Optical Flow

View More View Less
  • 1 Applied Physics Laboratory, and Department of Statistics, University of Washington, Seattle, Washington
  • | 2 Applied Physics Laboratory, University of Washington, Seattle, Washington, and College of Oceanic and Atmospheric Sciences, Oregon State University, Corvallis, Oregon
  • | 3 Department of Statistics, University of Washington, Seattle, Washington
  • | 4 Applied Physics Laboratory, University of Washington, Seattle, Washington
© Get Permissions
Restricted access

Abstract

Three spatial verification techniques are applied to three datasets. The datasets consist of a mixture of real and artificial forecasts, and corresponding observations, designed to aid in better understanding the effects of global (i.e., across the entire field) displacement and intensity errors. The three verification techniques, each based on well-known statistical methods, have little in common and, so, present different facets of forecast quality. It is shown that a verification method based on cluster analysis can identify “objects” in a forecast and an observation field, thereby allowing for object-oriented verification in the sense that it considers displacement, missed forecasts, and false alarms. A second method compares the observed and forecast fields, not in terms of the objects within them, but in terms of the covariance structure of the fields, as summarized by their variogram. The last method addresses the agreement between the two fields by inferring the function that maps one to the other. The map—generally called optical flow—provides a (visual) summary of the “difference” between the two fields. A further summary measure of that map is found to yield useful information on the distortion error in the forecasts.

Corresponding author address: Caren Marzban, Dept. of Statistics, University of Washington, Box 354322, Seattle, WA 98195-4322. Email: marzban@stat.washington.edu

This article included in the Spatial Forecast Verification Methods Inter-Comparison Project (ICP) special collection.

Abstract

Three spatial verification techniques are applied to three datasets. The datasets consist of a mixture of real and artificial forecasts, and corresponding observations, designed to aid in better understanding the effects of global (i.e., across the entire field) displacement and intensity errors. The three verification techniques, each based on well-known statistical methods, have little in common and, so, present different facets of forecast quality. It is shown that a verification method based on cluster analysis can identify “objects” in a forecast and an observation field, thereby allowing for object-oriented verification in the sense that it considers displacement, missed forecasts, and false alarms. A second method compares the observed and forecast fields, not in terms of the objects within them, but in terms of the covariance structure of the fields, as summarized by their variogram. The last method addresses the agreement between the two fields by inferring the function that maps one to the other. The map—generally called optical flow—provides a (visual) summary of the “difference” between the two fields. A further summary measure of that map is found to yield useful information on the distortion error in the forecasts.

Corresponding author address: Caren Marzban, Dept. of Statistics, University of Washington, Box 354322, Seattle, WA 98195-4322. Email: marzban@stat.washington.edu

This article included in the Spatial Forecast Verification Methods Inter-Comparison Project (ICP) special collection.

Save