Distribution-Oriented Verification of Limited-Area Model Forecasts in a Perfect-Model Framework

Ramón de Elía Département des Sciences de la Terre et de l'Atmosphère, Université du Québec à Montréal, Montreal, Quebec, Canada

Search for other papers by Ramón de Elía in
Current site
Google Scholar
PubMed
Close
and
René Laprise Département des Sciences de la Terre et de l'Atmosphère, Université du Québec à Montréal, Montreal, Quebec, Canada

Search for other papers by René Laprise in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Nested limited-area models (LAMs) have been used by the scientific community for a long time, with the implicit assumption that they are able to generate meaningful small-scale features that were absent in the lateral boundary conditions and sometimes even in the initial conditions. This hypothesis has never been seriously challenged in spite of reservations expressed by part of the scientific community. In order to study this hypothesis, a perfect-model approach is followed. A high-resolution LAM driven by global analyses is used over a large domain to generate a “reference run.” These fields are filtered afterward to remove small scales in order to mimic low-resolution nesting data. The same high-resolution LAM, but over a small domain, is nested with these filtered fields and run for several days. The ability of the LAM to regenerate the small scales that were absent in the initial and lateral boundary conditions is estimated by comparing both runs over the same region.

The simulations are analyzed for several variables using a distribution-oriented approach, which provides an estimation of the forecasting ability as a function of the value of the variable. It is found that variables with steep spectra, such as geopotential and temperature, display good forecasting skills for the entire range of values but improve little the forecast skill of a low-resolution perfect model. For noisier variables with flatter spectra, such as vorticity and precipitation, the high-resolution forecast provides a more realistic and extended range of forecast values for the variables, but rather low skill for extreme events. The probability of a successful forecast for these extreme cases, however, is much higher than that of a random model. When errors in the phase in the weather systems are not penalized, forecasting skill increases considerably. This suggests that, despite the inability to perform as pointwise deterministic forecasts, useful information may be generated by LAMs if considered in a probabilistic way.

Corresponding author address: Ramón de Elía, Groupe UQAM, Ouranos, 550 West Sherbrooke St., 19th Floor, Montreal, QC H3A 1B9, Canada. Email: relia@sca.uqam.ca

Abstract

Nested limited-area models (LAMs) have been used by the scientific community for a long time, with the implicit assumption that they are able to generate meaningful small-scale features that were absent in the lateral boundary conditions and sometimes even in the initial conditions. This hypothesis has never been seriously challenged in spite of reservations expressed by part of the scientific community. In order to study this hypothesis, a perfect-model approach is followed. A high-resolution LAM driven by global analyses is used over a large domain to generate a “reference run.” These fields are filtered afterward to remove small scales in order to mimic low-resolution nesting data. The same high-resolution LAM, but over a small domain, is nested with these filtered fields and run for several days. The ability of the LAM to regenerate the small scales that were absent in the initial and lateral boundary conditions is estimated by comparing both runs over the same region.

The simulations are analyzed for several variables using a distribution-oriented approach, which provides an estimation of the forecasting ability as a function of the value of the variable. It is found that variables with steep spectra, such as geopotential and temperature, display good forecasting skills for the entire range of values but improve little the forecast skill of a low-resolution perfect model. For noisier variables with flatter spectra, such as vorticity and precipitation, the high-resolution forecast provides a more realistic and extended range of forecast values for the variables, but rather low skill for extreme events. The probability of a successful forecast for these extreme cases, however, is much higher than that of a random model. When errors in the phase in the weather systems are not penalized, forecasting skill increases considerably. This suggests that, despite the inability to perform as pointwise deterministic forecasts, useful information may be generated by LAMs if considered in a probabilistic way.

Corresponding author address: Ramón de Elía, Groupe UQAM, Ouranos, 550 West Sherbrooke St., 19th Floor, Montreal, QC H3A 1B9, Canada. Email: relia@sca.uqam.ca

Save
  • Anthes, R. A., Y. H. Kuo, D. P. Baumhefner, R. M. Errico, and T. W. Bettge, 1985: Predictability of mesoscale motions. Advances in Geophysics, Vol. 288, Academic Press, 159–202.

    • Search Google Scholar
    • Export Citation
  • Atger, F., 2001: Verification of intense precipitation forecasts from single models and ensemble predictions. Nonlinear Processes Geophys., 8 , 401417.

    • Search Google Scholar
    • Export Citation
  • Brooks, H. E., and C. A. Doswell III, 1996: A comparison of measure-oriented and distributions-oriented approaches to forecast verification. Wea. Forecasting, 11 , 288303.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., 1997: Potential forecast skill of ensemble prediction and skill distributions of the ECMWF ensemble prediction system. Mon. Wea. Rev., 125 , 99119.

    • Search Google Scholar
    • Export Citation
  • Caya, D., and R. Laprise, 1999: A semi-implicit semi-Lagrangian regional climate model: The Canadian RCM. Mon. Wea. Rev., 127 , 341362.

    • Search Google Scholar
    • Export Citation
  • Davis, C., and F. Carr, 2000: Summary of the 1998 workshop on mesoscale model verification. Bull. Amer. Meteor. Soc., 81 , 809819.

  • de Elía, R., R. Laprise, and B. Denis, 2002: Forecasting skill limits of nested, limited-area models: A perfect model approach. Mon. Wea. Rev., 130 , 20062023.

    • Search Google Scholar
    • Export Citation
  • Denis, B., J. Côté, and R. Laprise, 2002a: Spectral decomposition of two-dimensional atmospheric fields on limited-area domains using the discrete cosine transform (DCT). Mon. Wea. Rev., 130 , 18121829.

    • Search Google Scholar
    • Export Citation
  • Denis, B., R. Laprise, J. Côté, and D. Caya, 2002b: Downscaling ability of one-way nested Regional Climate Models: The Big-Brother experiment. Climate Dyn., 18 , 627646.

    • Search Google Scholar
    • Export Citation
  • Denis, B., R. Laprise, and D. Caya, 2003: Sensitivity of a regional climate model to the resolution of the lateral boundary conditions. Climate Dyn., 20 , 107126.

    • Search Google Scholar
    • Export Citation
  • Gall, R., and M. Shapiro, 2000: The influence of Carl Gustaf Rossby on mesoscale weather prediction and an outlook for the future. Bull. Amer. Meteor. Soc., 81 , 15071523.

    • Search Google Scholar
    • Export Citation
  • Katz, R. W., and A. H. Murphy, 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.

  • Laprise, R., M. R. Varma, B. Denis, D. Caya, and I. Zawadzki, 2000: Predictability of a nested limited-area model. Mon. Wea. Rev., 128 , 41494154.

    • Search Google Scholar
    • Export Citation
  • Marsigli, C., A. Montani, F. Nerozzi, T. Paccagnella, S. Tibaldi, F. Molteni, and R. Buizza, 2001: A strategy for high-resolution ensemble prediction. Part II: Limited-area experiments in four Alpine flood events. Quart. J. Roy. Meteor. Soc., 127 , 20952116.

    • Search Google Scholar
    • Export Citation
  • Mass, C. F., D. Owens, K. Westrick, and B. A. Colle, 2002: Does increasing horizontal resolution produce more skillful forecasts? Bull. Amer. Meteor. Soc., 83 , 407430.

    • Search Google Scholar
    • Export Citation
  • Mesinger, F., K. Brill, H. Chuang, G. Di Mego, and E. Rogers, 2002: Limited Area Predictability: Can “upscaling” also take place? Research Activities in Atmospheric and Oceanic Modelling, H. Ritchie, Ed., World Meteorological Organization, WMO/TD-No. 1105, 5.30–5.31.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1991: Probabilities, odds, and forecasts of rare events. Wea. Forecasting, 6 , 302307.

  • Murphy, A. H., 1995: A coherent method of stratification within a general framework for forecast verification. Mon. Wea. Rev., 123 , 15821588.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and R. L. Winkler, 1987: A general framework for forecast verification. Mon. Wea. Rev., 115 , 13301338.

  • Murphy, A. H., B. G. Brown, and Y. S. Chen, 1989: Diagnostic verification of temperature forecasts. Wea. Forecasting, 4 , 485501.

  • Murphy, J., 1999: An evaluation of statistical and dynamical techniques for downscaling local climate. J. Climate, 12 , 22562284.

  • Paegle, J., Q. Yang, and M. Wang, 1997: Predictability in limited area and global models. Meteor. Atmos. Phys., 63 , 5369.

  • Papoulis, A., 1991: Probabilities, Random Variables and Stochastic Processes. 3d ed. McGraw-Hill, 666 pp.

  • Tracton, M. S., and J. Du, 2001: Application of the NCEP/EMC Short-Range Ensemble Forecasting system (SREF) to predicting extreme precipitation events. Preprints, Symp. on Precipitation Extremes: Predictions, Impacts, and Responses, Albuquerque, NM, Amer. Meteor. Soc., 64–65.

    • Search Google Scholar
    • Export Citation
  • van Tuyl, A. H., and R. M. Errico, 1989: Scale interaction and predictability in a mesoscale model. Mon. Wea. Rev., 117 , 495517.

  • White, B. G., J. Paegle, W. J. Steerburgh, J. D. Worel, R. T. Swanson, L. K. Cook, D. J. Onton, and J. G. Myles, 1999: Short-term forecast validation of six models. Wea. Forecasting, 14 , 84108.

    • Search Google Scholar
    • Export Citation
  • Wilks, D., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 464 pp.

  • Wilson, L. J., W. R. Burrows, and A. Lanzinger, 1999: A strategy for verification of weather elements forecast from an ensemble prediction system. Mon. Wea. Rev., 127 , 956970.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 149 45 8
PDF Downloads 76 24 9