A Displacement-Based Error Measure Applied in a Regional Ensemble Forecasting System

Christian Keil Institut für Physik der Atmosphäre, DLR Oberpfaffenhofen, Wessling, Germany

Search for other papers by Christian Keil in
Current site
Google Scholar
PubMed
Close
and
George C. Craig Institut für Physik der Atmosphäre, DLR Oberpfaffenhofen, Wessling, Germany

Search for other papers by George C. Craig in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Errors in regional forecasts often take the form of phase errors, where a forecasted weather system is displaced in space or time. For such errors, a direct measure of the displacement is likely to be more valuable than traditional measures. A novel forecast quality measure is proposed that is based on a comparison of observed and forecast satellite imagery from the Meteosat-7 geostationary satellite. The measure combines the magnitude of a displacement vector calculated with a pyramid matching algorithm and the local squared difference of observed and morphed forecast brightness temperature fields. Following the description of the method and its application for a simplified case, the measure is applied to regional ensemble forecasts for an episode of prefrontal summertime convection in Bavaria. It is shown that this new method provides a plausible measure of forecast error, which is consistent with a subjective ranking of ensemble members for a sample forecast. The measure is then applied to hourly images over a 36-h forecast period and compared with the bias and equitable threat score. The two conventional measures fail to provide any systematic distinction between different ensemble members, while the new measure identifies ensemble members of differing skill levels with a strong degree of temporal consistency. Using the displacement-based error measure, individual ensemble members are found to compare better with observations than either a short-term deterministic forecast or the ensemble mean throughout the convective period.

Corresponding author address: Christian Keil, Institut für Physik der Atmosphäre, DLR Oberpfaffenhofen, D-82234 Wessling, Germany. Email: christian.keil@dlr.de

Abstract

Errors in regional forecasts often take the form of phase errors, where a forecasted weather system is displaced in space or time. For such errors, a direct measure of the displacement is likely to be more valuable than traditional measures. A novel forecast quality measure is proposed that is based on a comparison of observed and forecast satellite imagery from the Meteosat-7 geostationary satellite. The measure combines the magnitude of a displacement vector calculated with a pyramid matching algorithm and the local squared difference of observed and morphed forecast brightness temperature fields. Following the description of the method and its application for a simplified case, the measure is applied to regional ensemble forecasts for an episode of prefrontal summertime convection in Bavaria. It is shown that this new method provides a plausible measure of forecast error, which is consistent with a subjective ranking of ensemble members for a sample forecast. The measure is then applied to hourly images over a 36-h forecast period and compared with the bias and equitable threat score. The two conventional measures fail to provide any systematic distinction between different ensemble members, while the new measure identifies ensemble members of differing skill levels with a strong degree of temporal consistency. Using the displacement-based error measure, individual ensemble members are found to compare better with observations than either a short-term deterministic forecast or the ensemble mean throughout the convective period.

Corresponding author address: Christian Keil, Institut für Physik der Atmosphäre, DLR Oberpfaffenhofen, D-82234 Wessling, Germany. Email: christian.keil@dlr.de

Save
  • Baldauf, M., and J-P. Schulz, 2004: Prognostic precipitation in the Lokal-Modell of DWD. COSMO Newsletter, No. 4, Consortium for Small Scale Modeling, 177–180. [Available online at http://cosmo-model.cscs.ch/public/various/newsLetters/newsLetter04/chp9-7.pdf.].

  • Brewster, K. A., 2003a: Phase-correcting data assimilation and application to storm-scale numerical weather prediction. Part I: Method description and simulation testing. Mon. Wea. Rev., 131 , 480492.

    • Search Google Scholar
    • Export Citation
  • Brewster, K. A., 2003b: Phase-correcting data assimilation and application to storm-scale numerical weather prediction. Part II: Application to a severe storm outbreak. Mon. Wea. Rev., 131 , 493507.

    • Search Google Scholar
    • Export Citation
  • Bright, D. R., and S. L. Mullen, 2002: Short-range ensemble forecasts of precipitation during the southwest monsoon. Wea. Forecasting, 17 , 10801100.

    • Search Google Scholar
    • Export Citation
  • Casati, B., G. Ross, and D. B. Stephenson, 2004: A new intensity-scale approach for the verification of spatial precipitation forecasts. Meteor. Appl., 11 , 141154.

    • Search Google Scholar
    • Export Citation
  • Davis, C. D., B. Brown, and R. Bullock, 2006a: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134 , 17721784.

    • Search Google Scholar
    • Export Citation
  • Davis, C. D., B. Brown, and R. Bullock, 2006b: Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Wea. Rev., 134 , 17851795.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and J. L. McBride, 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239 , 179202.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., U. Damrath, W. Wergen, and M. E. Baldwin, 2003: The WGNE assessment of short-term quantitative precipitation forecasts. Bull. Amer. Meteor. Soc., 84 , 481492.

    • Search Google Scholar
    • Export Citation
  • Gatzen, C., 2004: A derecho in Europe: Berlin, 10 July 2002. Wea. Forecasting, 19 , 639645.

  • Hagen, M., H-H. Schiesser, and M. Dorninger, 2000: Monitoring of mesoscale precipitation systems in the Alps and the northern Alpine foreland by radar and rain gauges. Meteor. Atmos. Phys., 72 , 87100.

    • Search Google Scholar
    • Export Citation
  • Hoffman, R. N., and C. Grassotti, 1996: A technique for assimilating SSM/I observations of marine atmospheric storms: Tests with ECMWF analyses. J. Appl. Meteor., 35 , 11771188.

    • Search Google Scholar
    • Export Citation
  • Keil, C., A. Tafferner, and T. Reinhardt, 2006: Synthetic satellite imagery in the Lokal-Modell. Atmos. Res., 82 , 1925.

  • Köpken, C., 2001: Monitoring of METEOSAT WV radiances and solar stray light effects. Research Rep. 10, EUMETSAT/ECMWF Fellowship Programme, 46 pp.

  • Leuenberger, D., and A. Rossa, 2007: Revisiting the latent heat nudging scheme for the rainfall assimilation of a simulated convective storm. Meteor. Atmos. Phys., doi:10.1007/s00703-007-0260-9.

    • Search Google Scholar
    • Export Citation
  • Mannstein, H., H. Huntrieser, and S. Wimmer, 2002: Determination of the mass flux in convective cells over Europe. Proc. 2002 EUMETSAT Meteorological Satellite Conf., Dublin, Ireland, EUMETSAT, 264–269.

  • Marsigli, C., A. Montani, F. Nerozzi, T. Paccagnella, S. Tibaldi, F. Molteni, and R. Buizza, 2001: A strategy for high-resolution ensemble prediction. Part II: Limited-area experiments in four Alpine flood events. Quart. J. Roy. Meteor. Soc., 127 , 20952115.

    • Search Google Scholar
    • Export Citation
  • Marsigli, C., F. Boccanera, A. Montani, and T. Paccagnella, 2005: The COSMO-LEPS mesoscale ensemble system: Validation of the methodology and verification. Nonlinear Proc. Geophys., 12 , 527536.

    • Search Google Scholar
    • Export Citation
  • Mesinger, F., 1996: Improvements in quantitative precipitation forecasts with the Eta regional model at the National Centers for Environmental Prediction: The 48-km upgrade. Bull. Amer. Meteor. Soc., 77 , 26372649.

    • Search Google Scholar
    • Export Citation
  • Molteni, F., R. Buizza, C. Marsigli, A. Montani, F. Nerozzi, and T. Paccagnella, 2001: A strategy for high-resolution ensemble prediction. Part I: Definition of representative members and global-model experiments. Quart. J. Roy. Meteor. Soc., 127 , 20692094.

    • Search Google Scholar
    • Export Citation
  • Muller, J-P., M-A. Denis, R. D. Dundas, K. L. Mitchell, C. Naud, and H. Mannstein, 2007: Stereo cloud-top heights and cloud amount retrieval from ATSR2. Int. J. Remote Sens., 28 , 19211938.

    • Search Google Scholar
    • Export Citation
  • Saunders, R., M. Matricardi, and P. Brunel, 1999: An improved radiative transfer model for assimilation of satellite radiance observations. Quart. J. Roy. Meteor. Soc., 125 , 14071425.

    • Search Google Scholar
    • Export Citation
  • Steppeler, J., G. Doms, U. Schättler, H. W. Bitzer, A. Gassmann, U. Damrath, and G. Gregoric, 2003: Meso-gamma scale forecasts using the nonhydrostatic model LM. Meteor. Atmos. Phys., 82 , 7596.

    • Search Google Scholar
    • Export Citation
  • Venugopal, V., S. Basu, and E. Foufoula-Georgiou, 2005: A new metric for comparing precipitation patterns with an application to ensemble forecasts. J. Geophys. Res., 110 .D08111, doi:10.1029/2004JD005395.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 467 pp.

  • Wulfmeyer, V., A. Behrendt, C. Kottmeier, and U. Corsmeier, cited. 2005: COPS: Convective and Orographically-induced Precipitation Study. [Available online at http://www.uni-hohenheim.de/spp-iop/documents/051109_COPS_SOD_final.pdf.].

  • Yuan, H., S. L. Mullen, X. Gao, S. Sorooshian, J. Du, and H-M. H. Juang, 2005: Verification of probabilistic quantitative precipitation forecasts over the southwest United States during winter 2002/03 by the RSM ensemble system. Mon. Wea. Rev., 133 , 279294.

    • Search Google Scholar
    • Export Citation
  • Zepeda-Arce, J., and E. Foufoula-Georgiou, 2000: Space–time rainfall organization and its role in validating quantitative precipitation forecasts. J. Geophys. Res., 105 , 1012910146.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 728 148 3
PDF Downloads 357 122 5