• Baldwin, M. E., and Kain J. S. , 2006: Sensitivity of several performance measures to displacement error, bias, and event frequency. Wea. Forecasting, 21 , 636648.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Balsamo, G., Mahfouf J-F. , Bélair S. , and Deblonde G. , 2006: A global root-zone soil moisture analysis using simulated L-band brightness temperature in preparation for the Hydros satellite mission. J. Hydrometeor., 7 , 11261146.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Balsamo, G., Mahfouf J-F. , Bélair S. , and Deblonde G. , 2007: A land data assimilation system for soil moisture and temperature: An information content study. J. Hydrometeor., 8 , 12251242.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bechtold, P., Bazile E. , Guichard F. , Mascart P. , and Richard E. , 2001: A mass-flux convection scheme for regional and global models. Quart. J. Roy. Meteor. Soc., 127 , 869886.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., and Mailhot J. , 2001: Impact of horizontal resolution on the numerical simulation of a midlatitude squall line: Implicit and explicit condensation. Mon. Wea. Rev., 129 , 23622376.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Zhang D-L. , and Mailhot J. , 1994: Numerical prediction of the 10–11 June 1985 squall line with the Canadian regional finite-element model. Wea. Forecasting, 9 , 157172.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Mailhot J. , Strapp J. W. , and MacPherson J. I. , 1999: An examination of local versus nonlocal aspects of a TKE-based boundary layer scheme in clear convective conditions. J. Appl. Meteor., 38 , 14991518.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Méthot A. , Mailhot J. , Bilodeau B. , Patoine A. , Pellerin G. , and Côté J. , 2000: Operational implementation of the Fritsch–Chappell convective scheme in the 24-km Canadian regional model. Wea. Forecasting, 15 , 257274.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Crevier L-P. , Mailhot J. , Bilodeau B. , and Delage Y. , 2003a: Operational implementation of the ISBA land surface scheme in the Canadian regional weather forecast model. Part I: Warm season results. J. Hydrometeor., 4 , 352370.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Brown R. , Mailhot J. , Bilodeau B. , and Crevier L-P. , 2003b: Operational implementation of the ISBA land surface scheme in the Canadian regional weather forecast model. Part II: Cold season results. J. Hydrometeor., 4 , 371386.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Mailhot J. , Girard C. , and Vaillancourt P. , 2005: Boundary layer and shallow cumulus clouds in a medium-range forecast of large-scale weather system. Mon. Wea. Rev., 133 , 19381960.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Benoit, R., Côté J. , and Mailhot J. , 1989: Inclusion of a TKE boundary layer parameterization in the Canadian regional finite-element model. Mon. Wea. Rev., 117 , 17261750.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bougeault, P., 2008: Our changing forecast model. ECMWF Newsletter, No. 114, ECMWF, Reading, United Kingdom, 1.

  • Bougeault, P., and Lacarrère P. , 1989: Parameterization of orography-induced turbulence in a mesobeta-scale model. Mon. Wea. Rev., 117 , 18721890.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buizza, R., Houtekamer P. L. , Toth Z. , Pellerin G. , Wei M. , and Zhu Y. , 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133 , 10761097.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Casati, B., Ross G. , and Stephenson D. B. , 2004: A new intensity-scale approach for the verification of spatial precipitation forecasts. Meteor. Appl., 11 , 141154.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Côté, J., Desmarais J-G. , Gravel S. , Méthot A. , Patoine A. , Roch M. , and Staniforth A. , 1998a: The operational CMC–MRB Global Environmental Multiscale (GEM) model. Part II: Results. Mon. Wea. Rev., 126 , 13971418.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Côté, J., Gravel S. , Méthot A. , Patoine A. , Roch M. , and Staniforth A. , 1998b: The operational CMC–MRB Global Environmental Multiscale (GEM) model. Part I: Design considerations and formulation. Mon. Wea. Rev., 126 , 13731395.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Daly, C., Gibson W. P. , Taylor G. H. , Doggett M. K. , and Smith J. I. , 2007: Observer bias in daily precipitation measurements at United States cooperative network stations. Bull. Amer. Meteor. Soc., 88 , 899912.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Donaldson, R. J., Dyer R. M. , and Kraus R. M. , 1975: An objective evaluator of techniques for predicting severe weather events. Preprints, Ninth Conf. on Severe Local Storms, Norman, OK, Amer. Meteor. Soc., 321–326.

    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., Davies-Jones R. , and Keller D. L. , 1990: On summary measures of skill in rare event forecasting based on contingency tables. Wea. Forecasting, 5 , 576585.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drusch, M., and Viterbo P. , 2007: Assimilation of screen-level variables in ECMWF’s Integrated Forecast System: A study on the impact on the forecast quality and analyzed soil moisture. Mon. Wea. Rev., 135 , 300314.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and McBride J. L. , 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239 , 179202.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., Damrath U. , Wergen W. , and Baldwin M. E. , 2003: The WGNE assessment of short-term quantitative precipitation forecasts. Bull. Amer. Meteor. Soc., 84 , 481492.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Errico, R. M., Bauer P. , and Mahfouf J-M. , 2007: Issues regarding the assimilation of cloud and precipitation data. J. Atmos. Sci., 64 , 37853798.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fortin, V., Therrien C. , and Anctil F. , 2008: Correcting wind-induced bias in solid precipitation measurements in case of limited and uncertain data. Hydrol. Processes, 22 , 33933402. doi:10.1002/hyp.6959.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fritsch, J. M., and Chappell C. F. , 1980: Numerical prediction of convectively driven mesoscale pressure systems. Part I: Convective parameterization. J. Atmos. Sci., 37 , 17221733.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gallus W. A. Jr., , and Segal M. , 2001: Impact of improved initialization of mesoscale features on convective system rainfall in 10-km Eta simulations. Wea. Forecasting, 16 , 680696.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gauthier, P., Tanguay M. , Laroche S. , Pellerin S. , and Morneau J. , 2007: Extension of 3DVAR to 4DVAR: Implementation of 4DVAR at the Meteorological Service of Canada. Mon. Wea. Rev., 135 , 23392354.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Geleyn, J-F., 1987: Use of a modified Richardson number for parameterizing the effect of shallow convection. Short- and Medium-range Numerical Weather Prediction: Collection of Papers Presented at the WMO/IUGG NWP Symposium, Tokyo, Japan, WMO, 141–149.

    • Search Google Scholar
    • Export Citation
  • Goodison, B. E., Louie P. Y. T. , and Yang D. , 1998: WMO solid precipitation measurement intercomparison final report. WMO/TD 872, World Meteorological Organization, Geneva, Switzerland, 212 pp.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 1999: Hypothesis tests for evaluating numerical precipitation forecasts. Wea. Forecasting, 14 , 155167.

  • Hines, C. O., 1997a: Doppler-spread parameterization of gravity-wave momentum deposition in the middle atmosphere. Part I: Basic formulation. J. Atmos. Sol. Terr. Phys., 59 , 371386.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hines, C. O., 1997b: Doppler-spread parameterization of gravity-wave momentum deposition in the middle atmosphere. Part II: Broad and quasi monochromatic spectra, and implementation. J. Atmos. Sol. Terr. Phys., 59 , 387400.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and Mitchell H. L. , 2005: Ensemble Kalman filtering. Quart. J. Roy. Meteor. Soc., 131 , 32693289.

  • Jung, T., Gulev S. K. , Rudeva I. , and Soloviov V. , 2006: Sensitivity of extratropical cyclone characteristics to horizontal resolution in the ECMWF model. Quart. J. Roy. Meteor. Soc., 132 , 18391857.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J., and Fritsch J. M. , 1990: A one-dimensional entraining/detraining plume model and its application in convective parameterization. J. Atmos. Sci., 47 , 27842802.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J., and Fritsch J. M. , 1993: Convective parameterization for mesoscale models: The Kain–Fritsch scheme. The Representation of Cumulus Convection in Numerical Models, Meteror. Monogr., No. 46, Amer. Meteor. Soc., 165–170.

    • Search Google Scholar
    • Export Citation
  • Katz, R. W., and Murphy A. H. , Eds. 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.

  • Kuo, H. L., 1974: Further studies of the parameterization of the influence of cumulus convection on large-scale flow. J. Atmos. Sci., 31 , 12321240.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Laroche, S., Gauthier P. , St-James J. , and Morneau J. , 1999: Implementation of a 3D variational data assimilation system at the Canadian Meteorological Centre. Part II: The regional analysis. Atmos.–Ocean, 37 , 281307.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Laroche, S., Gauthier P. , Tanguay M. , Pellerin S. , and Morneau J. , 2007: Impact of the different components of 4DVAR on the Global Forecast System of the Meteorological Service of Canada. Mon. Wea. Rev., 135 , 23552364.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, J., and Barker H. W. , 2005: A radiation algorithm with correlated-k distribution. Part I: Local thermal equilibrium. J. Atmos. Sci., 62 , 286309.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ma, L., 2000: On the parameterization of slantwise convection in general circulation models. Ph.D. thesis, McGill University, Montreal, QC, Canada, 175 pp.

    • Search Google Scholar
    • Export Citation
  • Mailhot, J., and Bélair S. , 2002: An examination of a unified cloudiness–turbulence scheme with various types of cloudy boundary layers. Preprints, 15th Symp. on Boundary Layer and Turbulence, Wageningen, Netherlands, Amer. Meteor. Soc., 215–218.

    • Search Google Scholar
    • Export Citation
  • Mailhot, J., Chouinard C. , Benoit R. , Roch M. , Verner G. , Côté J. , and Pudykiewicz J. , 1989: Numerical forecasting of winter coastal storms during CASP: Evaluation of the regional finite-element model. Atmos.–Ocean, 27 , 2458.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mailhot, J., and Coauthors, 1998: Scientific description of RPN physics library: Version 3.6. Recherche en Prévision Numérique, 188 pp. [Available from RPN, 2121 Trans-Canada, Dorval, PQ H9P 1J3, Canada].

    • Search Google Scholar
    • Export Citation
  • Mailhot, J., and Coauthors, 2006: The 15-km version of the Canadian regional forecast system. Atmos.–Ocean, 44 , 133149.

  • Mason, I., 1989: Dependence of the critical success index on sample climate and threshold probability. Aust. Meteor. Mag., 37 , 7581.

  • Mesinger, F., and Brill K. , 2004: Bias normalized precipitation scores. Preprints, 17th Conf. on Probability and Statistics in the Atmospheric Sciences/20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., J12.6. [Available online at http://ams.confex.com/ams/pdfpapers/69561.pdf].

    • Search Google Scholar
    • Export Citation
  • Molinari, J., and Dudek M. , 1992: Parameterization of convective precipitation in mesoscale numerical models: A critical review. Mon. Wea. Rev., 120 , 326344.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morcrette, C. J., and Browning K. A. , 2006: Formation and release of symmetric instability following delta-M adjustment. Quart. J. Roy. Meteor. Soc., 132 , 10731089.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morcrette, J-J., Barker H. W. , Iacono M. J. , Mozdzynski G. , Pincus R. , Salmond D. , and Serrar S. , 2007: A new radiation package: McRad. ECMWF Newsletter, No. 112, ECMWF, Reading, United Kingdom, 22–32.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and Ehrendorfer M. , 1987: On the relationship between the accuracy and value of forecasts in the cost–loss ratio situation. Wea. Forecasting, 2 , 243251.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • NOAA, 1996: Standard hydrometeorological exchange format. Weather Service Hydrology Handbook 1, National Oceanic and Atmospheric Administration, 61 pp. [Available from Office of Hydrology, National Weather Service, Silver Spring, MD 20910].

    • Search Google Scholar
    • Export Citation
  • Noilhan, J., and Planton S. , 1989: A simple parameterization of land surface processes for meteorological models. Mon. Wea. Rev., 117 , 536549.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pudykiewicz, J., Benoit R. , and Mailhot J. , 1992: Inclusion and verification of a predictive cloud water scheme in a regional weather prediction model. Mon. Wea. Rev., 120 , 612626.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Richardson, D. S., 2000: Skill and relative economic value of the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc., 126 , 649667.

  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5 , 570575.

  • Stephenson, D. B., 2000: Use of the “odds ratio” for diagnosing forecast skill. Wea. Forecasting, 15 , 221232.

  • Sundqvist, H., 1978: A parameterization scheme for non-convective condensation including prediction of cloud water content. Quart. J. Roy. Meteor. Soc., 104 , 677690.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tremblay, A., 2005: The stratiform and convective components of surface precipitation. J. Atmos. Sci., 62 , 15131528.

  • Tustison, B., Harris D. , and Foufoula-Georgiou E. , 2001: Scale issues in verification of precipitation forecasts. J. Geophys. Res., 106 , (D11). 1177511784.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Verseghy, D. L., 1991: CLASS—A Canadian Land Surface Scheme for GCMs. I. Soil model. Int. J. Climatol., 11 , 111133.

  • Verseghy, D. L., McFarlane N. A. , and Lazare M. , 1993: CLASS—A Canadian Land Surface Scheme for GCMs. II. Vegetation model and coupled runs. Int. J. Climatol., 13 , 347370.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., and Hamill T. M. , 1995: Potential economic value of ensemble forecasts. Mon. Wea. Rev., 123 , 35653575.

  • Zhang, D-L., Kain J. S. , Fritsch M. , and Gao K. , 1994: Comments on “Parameterization of convective precipitation in mesoscale numerical models: A critical review.”. Mon. Wea. Rev., 122 , 22222231.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhu, Y., Toth Z. , Wobus R. , Richardson D. , and Mylne K. , 2002: The economic value of ensemble-based weather forecasts. Bull. Amer. Meteor. Soc., 83 , 7383.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Objective evaluation of 96-h forecasts against radiosondes for the GLBOLD (dotted lines) and GLBNEW (dash lines) global forecast systems for (a) geopotential height (dam), (b) temperature (K), (c) wind speed (m s−1), and (d) dewpoint depression (K). RMSE (black lines) and bias (gray lines) are shown. Verification is done over North America, for a set of 122 integrations initialized from 4DVAR analyses between 31 Aug and 31 Oct 2006.

  • View in gallery

    SYNOP surface observation network over North America. Red exes show the locations of stations where manual measurements are taken. Blue exes show the locations of automatic stations.

  • View in gallery

    Histograms showing the number of observed and predicted events with daily precipitation accumulations between 0 and 0.2 mm day−1, 0.2 and 5 mm day−1, etc. The histogram for the observations is shown in black, whereas histograms for the forecast systems are shown in yellow (GLBOLD), red (GLBNEW), and blue (REG).

  • View in gallery

    Total precipitation (m) for each daily accumulations category (i.e., between 0 and 0.2 mm day−1, 0.2 and 5 mm day−1, etc). Totals for the observations are given by the thick full line, whereas totals for the GLBOLD, GLBNEW, and REG forecast systems are indicated by the dotted, dashed, and dashed–dotted lines. The total precipitation (including all categories) is shown on the right.

  • View in gallery

    Threshold biases for the GLBOLD (dotted line), GLBNEW (dashed line), and REG (dashed–dotted line) forecast systems vs precipitation intensity.

  • View in gallery

    ETS as a function of categories of precipitation intensity (mm day−1) for the GLBOLD (yellow), GLBNEW (red), and REG (blue) configurations. (top) The 0–24-, (middle) 48–72-, and (bottom) 96–120-h accumulations.

  • View in gallery

    ETS as a function of thresholds for precipitation intensity (mm day−1) for the GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines) forecast systems. (top) The 0–24-, (middle) 48–72-, and (bottom) 96–120-h predictions.

  • View in gallery

    ETS as a function of lead time for the GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines) forecast systems. (top) The 0.2, (middle) 10, and (bottom) 30 mm day−1 threshold results. Values at a certain lead time represents the ETS for the preceding 24-h period (e.g., values at 24 h are for the day 1 ETS).

  • View in gallery

    Relative economic values of the predicted daily precipitation as a function of the cost–loss ratio (C/L) for three lead times (days 1, 3, and 5) and three thresholds of precipitation intensity (0.2, 10, and 30 mm day−1) for GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines).

  • View in gallery

    Area between zero and the positive portion of the relative economic value curves given in Fig. 9, as a function of lead time. Results for GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines) are given for thresholds of precipitation intensity of 0.2, 10, and 30 mm day−1.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 132 69 2
PDF Downloads 84 47 0

Medium-Range Quantitative Precipitation Forecasts from Canada’s New 33-km Deterministic Global Operational System

View More View Less
  • 1 Science and Technology Branch, Environment Canada, Dorval, Quebec, Canada
  • | 2 Meteorological Service of Canada, Environment Canada, Dorval, Quebec, Canada
  • | 3 Science and Technology Branch, Environment Canada, Dorval, Quebec, Canada
Full access

Abstract

The Meteorological Service of Canada (MSC) recently implemented a 33-km version of the Global Environmental Multiscale (GEM) model, with improved physics, for medium-range weather forecasts. Quantitative precipitation forecasts (QPFs) from this new system were compared with those from the previous global operational system (100-km grid size) and with those from MSC’s short-range (48 h) regional system (15-km grid size). The evaluation is based on performance measures that evaluate bias, accuracy, and the value of the QPFs.

Results presented in this article consistently show, for these three aspects of the evaluation, that the new global forecast system (GLBNEW) agrees more closely with observations, relative to the performance of the previous global system (GLBOLD). The biases are noticeably smaller with GLBNEW compared with GLBOLD, which severely overpredicts (underpredicts) the frequencies and total amounts associated with weak (strong) precipitation intensities. The accuracy and value scores reveal gains of at least 12 h and even up to 72 h for medium-range QPFs (i.e., day 3 to day 5 predictions). The new global system even performs slightly better than MSC’s operational regional 15-km system for short-range QPFs.

In a more absolute manner, results suggest that QPFs from the new global system may still have accuracy and value even at the medium range. This seems to be true at least for the smallest precipitation threshold, related to precipitation occurrence, for which the positive area under curves of relative economic value remains important, even for day 5 QPFs.

Corresponding author address: Stéphane Bélair, Numerical Prediction Research Section, Science and Technology Branch, Environment Canada, Rm. 500, 2121 Trans-Canada Highway, Dorval, QC H9P 1J3, Canada. Email: stephane.belair@ec.gc.ca

Abstract

The Meteorological Service of Canada (MSC) recently implemented a 33-km version of the Global Environmental Multiscale (GEM) model, with improved physics, for medium-range weather forecasts. Quantitative precipitation forecasts (QPFs) from this new system were compared with those from the previous global operational system (100-km grid size) and with those from MSC’s short-range (48 h) regional system (15-km grid size). The evaluation is based on performance measures that evaluate bias, accuracy, and the value of the QPFs.

Results presented in this article consistently show, for these three aspects of the evaluation, that the new global forecast system (GLBNEW) agrees more closely with observations, relative to the performance of the previous global system (GLBOLD). The biases are noticeably smaller with GLBNEW compared with GLBOLD, which severely overpredicts (underpredicts) the frequencies and total amounts associated with weak (strong) precipitation intensities. The accuracy and value scores reveal gains of at least 12 h and even up to 72 h for medium-range QPFs (i.e., day 3 to day 5 predictions). The new global system even performs slightly better than MSC’s operational regional 15-km system for short-range QPFs.

In a more absolute manner, results suggest that QPFs from the new global system may still have accuracy and value even at the medium range. This seems to be true at least for the smallest precipitation threshold, related to precipitation occurrence, for which the positive area under curves of relative economic value remains important, even for day 5 QPFs.

Corresponding author address: Stéphane Bélair, Numerical Prediction Research Section, Science and Technology Branch, Environment Canada, Rm. 500, 2121 Trans-Canada Highway, Dorval, QC H9P 1J3, Canada. Email: stephane.belair@ec.gc.ca

1. Introduction

Even when considering the sizeable progress recently achieved with global ensemble prediction systems (EPSs; e.g., Buizza et al. 2005), deterministic global models still constitute a very important tool for medium-range numerical weather prediction (NWP) at most national centers. Over the last decade, the horizontal and vertical resolutions of deterministic models have greatly increased, along with significant improvements in their representation of physical processes. For example, the current Global Spectral Model (GSM) used at the Japan Meteorological Agency (JMA) now has a spectral resolution of TL959 (about 20-km grid size) and includes recent improvements to convection and radiation. At the European Centre for Medium-Range Weather Forecasts (ECMWF), the spectral resolution of the Integrated Forecast System (IFS) is T799 (about 25 km), with improved physics for convection, condensation, radiation, and surface drag (see Bougeault 2008). At the U.S. National Centers for Environment Prediction (NCEP), the horizontal spectral resolution is now T382 (estimated to be approximately equivalent to a grid size of 35 km), with many new improvements to the surface processes.

All of these efforts have contributed to a reduction in the errors for medium-range predictions of mass and wind fields in the Northern and Southern Hemispheres (e.g., Jung et al. 2006; Drusch and Viterbo 2007; Morcrette et al. 2007). But their impact on medium-range quantitative precipitation forecasts (QPFs) is less well documented, if at all. Obviously, improving medium-range QPFs is of primary importance, not only in meteorology, with all its impact on the general population, transport, as well as on industrial and touristic activities, but also for other applications such as hydrology (river flow management, flood prediction) and agriculture (soil wetness).

Several years ago, within the context of short-range NWP, it was realized that increasing horizontal resolution alone is not sufficient to improve QPFs, unless such an increase is combined with physical schemes that are more appropriate for the new model grid size or, in this particular case, for meso-β-scale models [i.e., 10–40-km grid size; Bélair et al. (1994), (2000)]. This “combined” strategy has been adopted in Canada, where a 33-km version of the Global Environmental Multiscale (GEM; Côté et al. 1998a,b) model has been implemented at the Meteorological Service of Canada (MSC) for medium-range NWP, together with a physical package that is very similar to that used in the 15-km regional short-range NWP system (Mailhot et al. 2006). As discussed in Bélair et al. (2005), this physical configuration is particularly well suited for the prediction of clouds and precipitation in meso-β-scale models.

Our main objective in this study is thus to examine the impact on medium-range QPFs resulting from the operational implementation of this 33-km global model with improved “mesoscale” physics.

Details of the new 33-km global forecast system are given in the next section. The experimental setup is described in section 3. An objective evaluation of QPFs resulting from the new global system is presented in section 4, along with similar evaluations for the lower-resolution global system previously operational at MSC and for the higher-resolution regional short-range forecast system. Analysis and discussion follow in section 5, and the final section offers an outlook on future developments at MSC related to medium-range QPFs.

2. Canada’s 33-km global forecast system

The new GEM version for global medium-range forecasting was implemented at MSC on 31 October 2006. In this implementation, both the horizontal and vertical resolutions of the global forecast system were significantly increased. The horizontal grid size was reduced from 100 to 33 km, and the number of levels in the vertical was increased from 28 to 58. As shown in Table 1, the differences between MSC’s new and old global systems are broader and more general than just increasing the resolution, and involve important modifications to GEM’s numerics–dynamics and physics. In Table 1, GLBOLD refers to the global system previously operational at MSC, GLBNEW refers to the new global system, and REG refers to MSC’s short-range regional (i.e., continental) system (Mailhot et al. 2006).

For GLBNEW’s numerical–dynamical configuration, the time step was decreased from 45 to 15 min and the sponge layer at the top of the model (to minimize the negative impact of spurious waves reflecting from the model top) was increased from one to four levels. It should also be noted that the latitudinal and longitudinal grid lengths of the computational grid are not the same with the new global system (i.e., 0.45° grid size in the longitudinal direction of the grid and 0.3° in the latitudinal direction), which translates into equal spatial scales for both directions at about 49° north and south (grid size ∼33 km). This is in contrast with GLBOLD, in which the latitudinal and longitudinal grid sizes are the same, which leads to isotropic spatial resolution at the computational grid’s equator (grid size ∼100 km).

The modifications that were done to the physical package are as compelling, as they relate to many important physical processes. One major objective with this new global model was to represent condensation and precipitation in a way similar to what is done in MSC’s REG system (Bélair et al. 2000; Mailhot et al. 2006). To achieve this, the Kuo scheme (see Kuo 1974; Mailhot et al. 1989) for deep convection was replaced by the Kain and Fritsch (1990, 1993) scheme. And a new scheme for shallow convection [Kuo transient, based on the work of Girard and colleagues; see Bélair et al. (2005)] was included, together with a modified version of the grid-scale stable condensation scheme (Sundqvist 1978; Pudykiewicz et al. 1992). Changes in this last scheme are related to the improved representation of evaporation below the cloud base. In the turbulent vertical diffusion scheme, the Bougeault and Lacarrère (1989) mixing length is now used (see also Bélair et al. 1999). The Interactions between Surface, Biosphere, and Atmosphere (ISBA; see Noilhan and Planton 1989) land surface scheme is now replacing the previous force–restore scheme (Benoit et al. 1989), with initial conditions provided by a sequential data assimilation based on optimal interpolation (see Bélair et al. 2003a,b for details).

As can be seen in Table 1, GLBNEW’s configuration is indeed similar to that of REG, but is not identical. Because REG’s main concern is with short forecast ranges (up to 48 h), its numerical–dynamical configuration is quite different from that of GLBNEW. Even though its computational grid is still global, REG’s horizontal resolution is variable with a uniform-resolution domain over North America (see Fig. 1 in Bélair et al. 2000). Also, REG’s horizontal grid size is smaller compared with GLBNEW (approximately 15 km over North America) and the time step is shorter (7.5 min).

There are also a few differences in the vertical diffusion schemes for turbulence. The regional system includes a newly implemented statistical treatment of boundary layer clouds called MoisTKE (see Mailhot and Bélair 2002; Bélair et al. 2005), compared with the global model in which the effect of boundary layer clouds is represented by a simple modification of the Richardson number (based on Geleyn 1987). Also, REG’s turbulent mixing length is given by an asymptotic value (see Mailhot et al. 1998), and not the Bougeault and Lacarrère (1989) length. It should also be mentioned that a four-dimensional variational data assimilation (4DVAR) approach is used for upper-air data assimilation in the global forecast system (Gauthier et al. 2007; Laroche et al. 2007), whereas REG still operates with a three-dimensional variational data assimilation system (3DVAR; Laroche et al. 1999).

3. Parallel run and experimental setup

Prior to its operational implementation, the new 33-km global forecast system was thoroughly evaluated, both subjectively and objectively. As a last and crucial part of this evaluation, GLBNEW was run in parallel with MSC’s previous global system (i.e., GLBOLD). This parallel run lasted 2 months, starting on 31 August 2006 and ending with GLBNEW’s operational implementation on 31 October 2006. It produced a total of 122 medium-range (10 day) forecasts for each of the two systems (61 days with two integrations per day, at 0000 and 1200 UTC).

Objective evaluation of this parallel run against upper- air analyses and radiosondes observations revealed better performance at the short and medium ranges for GLBNEW. As an example, Fig. 1 shows biases and root-mean-square errors (RMSEs) obtained from comparing GLBOLD’s and GLBNEW’s 96-h forecasts against observations from North American radiosondes. At this medium range, a reduction in the RMSEs is evident for geopotential height and temperature (black lines in the top panels of Fig. 1), whereas the RMSEs are similar for winds and slightly increased for humidity (represented by the dewpoint depression, i.e., the difference between the temperature and the dewpoint temperature). Although beyond the scope of this article, it is worth mentioning that further investigation of these last results indicates that comparison of the two global systems’ RMSEs may not be fair for variables such as winds and humidity, due to their much larger variances in the higher-resolution GLBNEW system.

In general, biases are also improved with GLBNEW. As shown in Fig. 1b, GLBOLD’s warm bias below 600 hPa is reduced with the new system, as well as the cold bias above 500 hPa. This leads to a bias for geopotential height that is more constant with height (Fig. 1a), even though it is slightly larger then that of GLBOLD for most of the troposphere. It is worth mentioning that wind speeds are increased with GLBNEW, with smaller biases for most of the troposphere (Fig. 1c). For humidity, the dewpoint depression biases are more positive in the upper troposphere (i.e., 300–700 hPa), indicating that GLBNEW is generally drier for this layer (Fig. 1d).

It is these 122 parallel-run forecasts that are used in this study to evaluate the QPF improvement resulting from implementing GLBNEW. Two different observational surface networks are commonly used at MSC to objectively evaluate QPFs over North America: the conventional surface synoptic observations (SYNOPs) and the U.S. Cooperative Observer Program (COOP) available in standard hydrometeorological exchange format (SHEF; NOAA 1996). The spatial density of the COOP network is much greater than that of the SYNOPs network. But for several reasons, including the facts that the COOP network does not cover Canada, that observations from this network suffer from observer bias (see Daly et al. 2007), and that only one observation per day is available for most COOP stations (daily accumulations valid at 1200 UTC), only evaluation results using SYNOPs are presented in this study.

It should be mentioned, however, that objective evaluations of QPFs were performed using both networks, and were found to lead to equivalent conclusions. In fact, the statistical performance measures presented in this study are relatively stable due to the large number of cases. But confidence intervals (which could have been computed using resampling techniques such as the bootstrap method) are not plotted in the figures below.

As can be seen in Fig. 2, the spatial coverage of SYNOPs is relatively dense over the southern portion of Canada, where most of the country’s population resides. This is particularly the case over southern Alberta, Saskatchewan, and Manitoba, and in the southern Ontario–St. Lawrence River corridor. Interestingly, a greater number of U.S. SYNOP stations are manual (red), in contrast with Canada where more stations are automatic (blue).

Other relevant information concerning the objective evaluation of QPFs is as follows: (i) the evaluation period is in the fall season, that is, preceding the first snowfall for most of the stations shown in Fig. 2, thus avoiding biases that could be induced by severe underestimation of measured snowfall from automatic precipitation gauges (Goodison et al. 1998; Fortin et al. 2008); (ii) objective evaluation scores of QPFs for the 15-km short-range forecast system (REG) are shown and compared with those of the two global systems (GLBNEW and GLBOLD), but only for forecast ranges less than or equal to 48 h; (iii) the observation–prediction pairs used for the objective evaluation of QPFs are directly obtained by horizontally interpolating model outputs to surface stations; as discussed in Tustison et al. (2001), this process does not account for scale-dependent errors associated with the interpolation, referred to as representativeness errors; and (iv) otherwise, the comparison between GLBNEW and GLBOLD is clean, in the sense that the initial conditions for both systems are produced using 4DVAR, with identical cutoff times for the observations (i.e., 3-h delay after the initial forecast times at 0000 and 1200 UTC); this is not so for the comparison with REG, for which the assimilation is less advanced (i.e., 3DVAR) and the cutoff time is shorter (i.e., 1-h 40-min delay).

4. Objective evaluation of QPFs

a. Contingency tables

All the performance measures presented in this study are based on contingency tables, which are often used to analyze the statistical relationship between two categorical variables. In this study the variables are the observed and predicted daily precipitation accumulations (referred to in the text below as daily accumulations or precipitation intensities). The performance measures are not obtained from averaging scores that have been computed separately at each station or at each date, but are rather computed from summary contingency tables. These tables have the form as shown in Table 2, where the numbers 1, 2, … , K − 1, and K on the left and top of Table 2 refer to specific categories of daily accumulations (e.g., between 5 and 10 mm day−1): n(Fi,Oj) is the number of events with predicted (or forecast) accumulations in the ith category and observed accumulations in the jth category, N(Fi) is the marginal total of predicted events in the ith category, N(Oj) is the marginal total of observed events in the jth category, and N is the grand total of all the events.

Many of the performance measures discussed in this section are based on simpler 2 × 2 contingency tables (see Table 3) in which H is for the hits, that is, the number of events both predicted and observed; FA is for the false alarms, that is, the number of events predicted but not observed; M is for the misses, that is, the number of events observed but not predicted; and CN is for the correct negatives, that is, the number of events not predicted and not observed.

To calculate some of the performance measures discussed in the rest of this section, the more complete multicategory tables are collapsed into the simpler 2 × 2 tables in the following two manners. For multicategory scores, which evaluate the systems’ ability to predict daily accumulations in a specific category (1, 2, … to K in the above table), the 2 × 2 tables are obtained by setting
i1520-0434-24-3-690-e1
i1520-0434-24-3-690-e2
i1520-0434-24-3-690-e3
i1520-0434-24-3-690-e4
For scores that measure the systems’ ability to predict accumulations greater than the prespecified thresholds, referred to below as threshold scores (e.g., threshold bias, threshold equitable threat score), the 2 × 2 tables are obtained in the following way:
i1520-0434-24-3-690-e5
i1520-0434-24-3-690-e6
i1520-0434-24-3-690-e7
i1520-0434-24-3-690-e8

b. Marginal totals analysis and biases

A first approach to evaluating the performance of each forecast system (GLBOLD, GLBNEW, and REG) is to compare the distributions of marginal totals for predictions and observations, that is, N(Fi) versus N(Oi). In Fig. 3, the total number of events is shown for each category of daily accumulations (0–0.2 mm day−1, 0.2–5 mm day−1, …). Because the REG system produces forecasts up to 48 h, only results from the first 2 days of integration are used (statistics from 0–24-h accumulations and from 24–48-h accumulations are combined) to properly compare distributions and biases from the global configurations (GLBOLD and GLBNEW) with those from the regional system (REG). This does not influence the interpretation of Fig. 3’s histograms for the two global systems since these are relatively insensitive to model lead time (not shown). It is noteworthy that a logarithmic scale is used for the ordinate axis, due to the exponential nature of precipitation intensity distributions (Tremblay 2005).

Results displayed in Fig. 3 indicate that the old global system (GLBOLD, in yellow) underestimates the number of events in the no-precipitation category (0–0.2 mm day−1) and overestimates the number of events for small daily accumulations (0.2–5 mm day−1). There is also an important underestimation with this configuration for events with substantial accumulations (greater than 30 mm day−1). The new global system (GLBNEW, in red) better represents the marginal totals distribution, even though it slightly overestimates the number of events for accumulations between 0.2 and 30 mm day−1, and the opposite for accumulations greater than 40 mm day−1. These results are similar to those of the regional forecast system (REG, in blue), except for the number of events, which is larger with the regional system for all the categories (i.e., accumulations greater than 0.2 mm day−1) and which compares better for larger daily accumulations (i.e., greater than 40 mm day−1).

Information on the marginal totals can be displayed differently by multiplying the number of events with the mean precipitation for each of the daily precipitation categories. In this approach, the mean precipitation of each specific category is obtained by assuming an exponential form of the marginal totals distribution (see Fig. 3). The resulting plot displays the total precipitation accumulations (in m) for each category, referred to below as the total precipitation per category.

These totals, shown in Fig. 4, emphasize the differences pointed out from the frequency distributions shown in Fig. 3. For instance, total precipitation per category with GLBOLD is greatly overestimated for weak precipitation intensities (0.2–5 mm day−1) and severely underestimated for larger intensities (greater than 20 mm day−1). The other two systems, GLBNEW and REG, have totals of precipitation per category that compare more favorably with the observations, but they still overestimate this quantity for daily accumulations between 0.2 and 30 mm day−1. And as expected, the total precipitation per category from the REG system is larger than that from GLBNEW for all the categories.

An interesting quantity that can be derived from this figure is the total precipitation (including all categories) for observations and for the three forecast systems. The summation results, given by the horizontal lines in the right portion of Fig. 4, indicate that GLBOLD performs best for this quantity, in spite of its poor representation of the frequency and precipitation distributions. It appears that GLBOLD’s overestimation for weak precipitation intensities is canceled by its underestimation for greater intensities. The new global forecast system, GLBNEW, has total precipitation that is slightly larger than GLBOLD, but is much better than the REG system, which overestimates the total precipitation by about 20%. (This difference between GLBNEW and REG is linked to the parameters of the Kuo transient shallow convective scheme that are not the same for the two model configurations; these parameters are mainly related to the treatment of precipitation under the cloud base and will be harmonized in upcoming implementations at MSC.) This objective evaluation of the total precipitation is important for hydrological applications. For instance, the overestimation of the total water with the REG system has been found to have a negative impact on soil moisture assimilation and on river flow modeling systems currently under development at Environment Canada.

These conclusions about the three forecast systems evaluated in this study are summarized in Fig. 5, which shows the threshold biases (B) for day 1 and day 2 precipitation accumulations (statistics combined). This bias is calculated using
i1520-0434-24-3-690-e9
and essentially represents the ratio of the number of predicted events (H + FA) over the number of observed events (H + M). A perfect value for this score is equal to 1.

Figure 5 confirms that GLBOLD markedly overestimates the number of events with daily accumulations larger than small quantities (i.e., with thresholds smaller than about 5 mm day−1) and severely underestimates the number of events with large daily accumulations. With this system, the bias is about 0.2 for daily accumulations greater than 40 mm day−1. Overestimation of the number of predicted events with small daily accumulations is diminished with GLBNEW, with a crossover (B = 1) at daily accumulations of about 20 mm day−1. The bias is approximately 0.6 for daily accumulations greater than 50 mm day−1. REG, on the other hand, mainly overestimates the number of precipitation events for thresholds smaller than 50 mm day−1. This large (and positive) bias for REG has been a permanent characteristics of this system for many years (see Bélair et al. 2000; Mailhot et al. 2006).

c. Equitable threat scores

The equitable threat score (ETS) is an accuracy measurement commonly used at national weather centers (see Bélair et al. 2000; Gallus and Segal 2001; Ebert et al. 2003; Mailhot et al. 2006) to evaluate the ability of operational systems to predict precipitation. Given by
i1520-0434-24-3-690-e10
in which Hr is the number of hits associated with random chance,
i1520-0434-24-3-690-e11
the ETS is particularly suited for precipitation since it is only concerned with meaningful successes, that is, the hits H, and does not consider correct negatives, which dominate in the case of precipitation. Moreover, the ETS equally penalizes misses (M) and false alarms (FAs).

Proposed by Schaefer (1990), who called it the Gilbert skill score, the ETS is considered to be an improvement over the threat score [also called the critical success index; see Donaldson et al. (1975)], which was shown to depend on bias (Mason 1989; Baldwin and Kain 2006). Yet, the ETS is also sensitive to bias and somehow benefits from bias values above one (e.g., Hamill 1999). This deficiency led to further variants of this score proposed by Mesinger and Brill (2004) and Baldwin and Kain (2006). But as argued in this last study, these new scores as well as other scores such as the true skill statistic (Doswell et al. 1990) and the odds ratio skill score (Stephenson 2000) are all susceptible to hedging in one form or another. The ETS has been chosen in the present study because of its wide use in the operational community, including at MSC where it has been utilized for more than a decade.

Figure 6 displays the ETSs for several categories of precipitation intensity for the three forecast systems. It is clear from this figure that MSC’s new global system (GLBNEW) generally improves the ETS compared to GLBOLD. In particular, the ETS for the 0–0.2 mm day−1 category is noticeably increased, showing the new model’s improved accuracy in predicting the occurrence (or nonoccurrence) of precipitation. Remarkably, the performance difference between the two systems remains noticeable for this category even for day 5 predictions (96–120 h). For the other categories, GLBNEW’s greatest improvements over GLBOLD are for the largest precipitation intensities. This improvement, though, is reduced with lead time so that the day 5 ETSs from both GLBNEW and GLBOLD are similar for most categories of precipitation intensity.

In Fig. 7, the ETS is presented as a function of the precipitation intensity thresholds instead of the accumulation categories. The same type of separation between the two global systems is evidenced in Fig. 7, with larger ETS values from GLBNEW compared with GLBOLD for all thresholds of precipitation intensity. As observed for the categorical score shown in Fig. 6, the differences between GLBNEW’s and GLBOLD’s ETS values tend to decrease in the medium range. As an example, for the 30 mm day−1 threshold, the ETS differences between the two global systems go from 0.10 for day 1 to 0.08 for day 3 and to 0.04 for day 5. Both Figs. 6 and 7 indicate, as expected, that the day 1 ETS for GLBNEW is comparable with the ETS from the REG system, for all categories.

The ETS drop with lead time is evidenced in Fig. 8 for three different precipitation intensities: 0.2, 10, and 30 mm day−1. For each of these thresholds, GLBNEW’s ETS values are much larger than those of GLBOLD, and similar to those of REG. The ETS comparison for the two global systems obviously depends on precipitation intensity. For prediction of precipitation occurrence versus nonoccurrence, that is, for the 0.2 mm day−1 threshold, the gain in predictability at the medium range is of the order of 60 h. That is, GLBNEW’s ETS at 84 h is as good as GLBOLD’s ETS at 24 h. The gain in predictability is on the order of 24 h for the 10 mm day−1 threshold, and between 36 and 48 h for the 30 mm day−1 threshold.

The convergence of GLBNEW and GLBOLD ETSs at day 5 coincides with a general lack of accuracy for precipitation predicted by the two systems. This behavior is more evident for the 10 and 30 mm day−1 thresholds, as GLBOLD’s ETS seems to stabilize toward an asymptotic value that appears to be on the order of 0.1. Unfortunately, the objective evaluation of QPFs conducted prior to GLBNEW’s operational implementation was not done for predictions longer than 5 days; this would have allowed for a more precise documentation of the asymptotic behavior of the ETS.

Implicitly, the ETS should tend toward zero as lead time increases, that is, as the model loses accuracy and the number of hits H tends toward the number of hits associated with random chance [i.e., Hr in (11)]. But because precipitation may be organized in response to stationary forcing from the surface, such as precipitation forced by coastal orography, atmospheric models may preserve some residual or climatological accuracy to predict QPFs, which would lead to asymptotic values for ETS that would be greater than zero. This nonzero asymptotic value, which may be different for GLBOLD and GLBNEW, due to their different representations of the physical processes and different resolutions, could be used as a reference for new accuracy measures that would indicate the atmospheric models’ performance relative to a “residual accuracy” state.

d. Relative economic value

The performance measures discussed up to this point in this section provide information on the accuracy of the forecast systems, but not on their potential value to users. Several authors have investigated the link between the accuracy and value of predicting systems (e.g., Murphy and Ehrendorfer 1987; Wilks and Hamill 1995; Katz and Murphy 1997). In the present study, the cost–loss decision model described in Richardson (2000) and Zhu et al. (2002) is used to evaluate the relative economic value of QPFs from the GLBNEW, GLBOLD, and REG forecast systems. This score, also called the potential economic value or the value score, is given by
i1520-0434-24-3-690-e12
where E stands for the expected expense of users making decisions based on a specific system; therefore, Eclimate is the expense when using climate information, Eforecast is the expense when using the forecast, and Eperfect is the minimum expense of a user, which occurs when predictions are always correct. This score thus indicates the relative value (or added value) of using forecasts compared with using climatology. The optimal value is then V = 1 when Eforecast = Eperfect. Values of V > 0 indicate that users benefit from using the forecast (again, compared to using climatology).
Based on Zhu et al. (2002), the expected expenses can be expressed as functions of the H, FA, M, and CN elements of the contingency tables. Following their approach, the relative economic value is given by
i1520-0434-24-3-690-e13
in which h, fa, and m are the relative frequencies of hits, false alarms, and misses (i.e., H, FA, and M divided by the total number of entries in the contingency tables); o is the climatological frequency of the event, as observed in the 61-day sample examined in this study (i.e., h + m); and α = C/L is the so-called cost–loss ratio, that is, the cost (C) of taking preventive action in response to an event divided by the loss (L) associated with an event for which no preventive measures were taken.

The relative economic values of the QPFs from GLBOLD, GLBNEW, and REG are shown in Fig. 9 for day 1, day 3, and day 5 predictions, and for intensity thresholds of 0.2, 10, and 30 mm day−1. It should first be noted that the position (cost–loss ratio) of the maximum relative economic value does not depend on the forecast system, but is rather equal to the climatological occurrence frequency of the event (Richardson 2000). Peaks of the economic value thus occur at similar cost–loss ratios for a particular precipitation intensity, insensitive of lead time, and at smaller cost–loss ratios for more intense precipitation. For instance, the maximum relative economic values are found at C/L ∼ 0.4 for the 0.2 mm day−1 intensity, at C/L ∼ 0.08 for the 10 mm day−1 intensity, and at C/L ∼ 0.02 for the 30 mm day−1 intensity.

The range of cost–loss ratios with positive economic values (hereafter referred to as the “beneficial” range) is centered near the C/L ∼ 0.4 ratio for the 0.2 mm day−1 precipitation intensity (i.e., occurrence versus nonoccurrence; see the top panels in Fig. 9). For this low-intensity threshold, the maximum economic value is greater with GLBNEW, with a wider beneficial range of cost–loss ratios and a slight shift toward larger ratios. Although slightly diminished, this advantage of GLBNEW over GLBOLD is also found for day 3 precipitation. The two systems mostly have the same economic values for day 5 precipitation, both with narrow beneficial ranges.

For more intense precipitation (10 and 30 mm day−1, middle and bottom panels in Fig. 9), the beneficial range is shifted toward smaller ratios for the three forecast systems, due to the lower frequency of these events. This transition indicates that predictions of large precipitation accumulations are profitable to users for which the loss associated with an unpredicted event greatly exceeds the cost of preventive actions. Such users are less sensitive to false alarms, which explains the positive relative economic values over this range. For these precipitation intensities, the economic value of GLBNEW’s day 1 and day 3 QPFs is greater than those for GLBOLD. This improvement is more important for the 30 mm day−1 intensity. The beneficial range is similar for the two global forecast systems.

Interestingly, GLBNEW’s economic values compare well with those of the higher-resolution regional system REG. Although the maximum economic value of GLBNEW is either the same or slightly below that of REG, its beneficial range is marginally wider.

An overall measure of the economic value is the positive area below the curves given in Fig. 9, for all useful cost–loss ratios (between 0 and 1). Expressed mathematically, this metric is given by
i1520-0434-24-3-690-e14
and is shown in Fig. 10 as a function of lead time for the three forecast systems and for intensity thresholds of 0.2, 10, and 30 mm day−1. The optimal value of this score is Varea = 1 (i.e., V = 1 for all cost–loss ratios between 0 and 1), and its lowest value is Varea = 0 (V ≤ 0 for the same range).

According to this measure, GLBNEW’s QPFs have more value to users than those from the previous low-resolution global system GLBOLD, and similar value to those from the high-resolution short-range continental system REG. The greatest impact of the new system is found for the 0.2 mm day−1 precipitation threshold. For this precipitation intensity, the integrated economic value of GLBNEW’s day 3 (72 h) QPFs is as large as the value for GLBOLD’s day 1 (24 h) QPFs, that is, a gain of 48 h. This difference between the two global systems shrinks with lead time. Day 4 (96 h) QPFs from GLBNEW have as much value as GLBOLD’s day 2 (48 h) QPFs, a gain of more than 40 h. For day 5 QPFs, GLBNEW’s edge is on the order of 20 h.

For the larger intensity thresholds (10 and 30 mm day−1), the integrated economic values of GLBNEW’s QPFs also surpass that of GLBOLD, but to a lesser degree. For the 10 mm day−1 threshold, GLBNEW’s gain compared with GLBOLD is on the order of 18–24 h, except for day 5 predictions, which have similar values for the two systems. For the 30 mm day−1 intensity, GLBNEW’s QPFs still have more value, but this value remains low even for short-range forecasts.

5. Analysis and discussion

To provide a broad and general assessment of the quality and improvements of QPFs resulting from Canada’s implementation of a new 33-km global deterministic forecast system, several types of performance measures were examined in the previous section. These metrics essentially focus on three aspects of the evaluation of QPFs: bias, accuracy, and user value.

Evidence presented in the preceding section is consistent for these three aspects: it essentially indicates that the new global forecast system (GLBNEW) agrees considerably better with observations, relative to the performance of the previous global system (GLBOLD). The biases, for instance, are considerably less with GLBNEW, compared with GLBOLD, which severely overpredicts (underpredicts) the frequencies and total amounts associated with weak (strong) precipitation intensities. The accuracy and value scores, on the other hand, reveal gains of at least 12 h and even up to 72 h for medium-range QPFs (i.e., day 3–5 predictions).

Several factors are responsible for this substantial improvement in medium-range QPFs. But because this study’s experimental setup is intimately linked with GLBNEW’s operational implementation at MSC, the impact of each individual component of this implementation (e.g., vertical and horizontal resolutions, physical processes) could not be precisely quantified. Our objective in this study is rather to describe the overall impact of the new system, including all changes to the model numerics–dynamics, physics, and assimilation. Quantifying the impact that each of these changes has on QPFs is beyond the scope of the present article and would, in fact, require the publication of several articles.

Certainly, modifications to GEM’s dynamical and numerical configuration (see Table 1) have a positive impact on medium-range weather predictions. A significant portion of the positive results for upper-air evaluation (shown in Fig. 1) can be directly attributed to the increase in the horizontal and vertical resolutions, although changes to the representation of physical processes (e.g., condensation–precipitation, turbulence, and surface) also contribute in an important manner.

Increasing horizontal resolution also has an impact on QPFs’ objective evaluation. It is not surprising, for instance, that biases and precipitation distributions shown in Figs. 3 –5 are notably improved with the high-resolution GLBNEW. As is often observed, the number of events with small (large) accumulations is decreased (increased) with the higher-resolution system because of its ability to represent smaller-scale circulations and more intense precipitation events. For QPFs’ accuracy measures, on the other hand, increasing horizontal resolution may have a negative impact due to the larger spatial and temporal variances of the QPFs outputs, which normally translate into a deterioration of accuracy scores (e.g., ETS, RMSE). The important improvement displayed in this study for ETS and relative economic value is thus more likely related to a better treatment of the physical processes in GLBNEW (for clouds, precipitation, turbulence, and surface processes) than to horizontal resolution.

Obviously, QPFs are sensitive to GEM’s physical schemes for clouds, condensation, and precipitation. Bélair et al. (1994, 2000) have shown that deep convection schemes such as Kain–Fritsch or Fritsch and Chappell (1980), based on convective inhibition (CIN) and the release of convective available potential energy (CAPE), allow for more realistic interactions between implicit (subgrid scale) and explicit (grid scale) cloud processes. This partition between implicit and explicit processes is of critical importance for QPFs, as discussed in Molinari and Dudek (1992), Zhang et al. (1994), and Bélair and Mailhot (2001). When using the Kain–Fritsch convective scheme, the grid-scale condensation scheme (i.e., Sundqvist) is responsible for a larger portion of the total precipitation. This is in comparison with GLBOLD’s Kuo scheme, based on an equilibrium between the convective precipitation and the large-scale production of low-level convergence, which is overly active and dominates for all types of precipitation, producing weak precipitation for very large areas (Bélair et al. 1994, 2000). This explains the major adjustment of QPFs’ biases with the new system, for which the number of weak (strong) precipitation events is diminished (increased) (Fig. 5).

In addition to the Kain–Fritsch scheme for deep convection, the Kuo transient scheme for shallow cumulus activity also influences the balance between implicit and explicit cloud processes, by more realistically parameterizing the effects of low-level cloud activity that otherwise would have been captured by the grid-scale condensation scheme. As shown in Bélair et al. (2005), the Kuo transient scheme has a stabilizing effect (in the lower troposphere) that differs from the increased vertical transport typically associated with boundary layer cloud schemes. This vertical adjustment by the Kuo transient was found in Bélair et al. (2005) to reduce the amount of nonconvective cloud in a large-scale system over the Pacific Ocean, in better agreement with satellite observations. Of greater importance for the present study, it was also found to reduce the precipitation in the rear portion of this system, which could be another reason explaining the decrease of QPFs’ biases with GLBNEW.

Other components of GLBNEW’s implementation also have beneficial effects on QPFs. For instance, the implementation of the ISBA land surface scheme, together with a sequential assimilation for soil moisture and surface temperature, was found to have a noticeable positive impact on medium-range QPFs—even larger than results presented in Bélair et al. (2003a) for short-range prediction. Surface fluxes of heat and humidity strongly influence the diurnal evolution and vertical structure of the planetary boundary layer. Since precipitation and condensation processes are quite sensitive to low-level heat and humidity contents, there is a direct link between an improved representation of the daytime well-mixed layer (in large part driven by surface fluxes from the new land surface scheme) and QPFs.

The improvement of the QPFs resulting from GLBNEW’s implementation (compared with GLBOLD’s QPFs) decreases with lead time as the two global systems tend to perform similarly at longer range. (For 5 day QPFs, the two systems exhibit similar scores.) The asymptotic behavior observed for some of the performance measures could be used as an indicator of QPFs’ lack of skill and value for longer-range predictions. This is certainly obvious with GLBOLD’s forecasts, for which QPFs’ scores tend to plateau around days 4 or 5. This stabilization of the scores depends on the precipitation intensity; it occurs earlier for larger intensities.

For GLBNEW, however, the performance measures tend to decrease almost linearly with time without reaching this asymptotic state, even for the 30 mm day−1 precipitation threshold. Presumably, GLBNEW’s performance measures for QPFs would get to such a plateau if the evaluation was performed over longer integrations. The fact that this does not happen in the first 5 days of integration adds to the difficulty of attributing skill and value in an absolute manner to GLBNEW’s QPFs. Indeed, asymptotic values of ETS or of the area under economic value curves could provide additional information regarding the absolute performance of the prediction system, that is, not relative to another prediction system. In this respect, all we can say from the results presented in this study is that the asymptotic values are reached later in the integrations for the GLBNEW system, indicating that the skill and value in GLBNEW’s QPFs for day 5 predictions are still greater than that of climatology. Moreover, the area under the relative economic value curves for the 0.2 and 10 mm day−1 thresholds remains important for day 3 QPFs, and is still positive for day 5 QPFs (although small).

It is worth mentioning that the objective evaluation methods used in this study could be refined to consider the scale-dependent representativeness errors discussed in Tustison et al. (2001). Even though it is unlikely that this uncertainty would considerably influence the conclusions drawn in the present work (due to the sizeable differences in performance between GLBOLD and GLBNEW), it may be necessary in future implementations to examine more sophisticated verification approaches, such as the composite method proposed in Tustison et al. (2001). It may also be helpful to examine techniques such as the intensity-scale approach proposed by Casati et al. (2004) and the object-oriented approach described in Ebert and McBride (2000), among others.

6. Outlook

Improving QPFs at the medium-range should remain a priority for many years to come. In Canada, efforts to achieve this objective include the continued development of probabilistic products from the global EPS system, the improvement of the global deterministic model’s numerics and physics, the improvement of initial conditions by increasing the amount of data assimilated in the 4DVAR and in the ensemble Kalman filter used for the global EPS (Houtekamer and Mitchell 2005), and the use of more sophisticated approaches to evaluate the quality of medium-range QPFs.

Several modifications to MSC’s global deterministic system are expected for operational transfer in the near future. An upgrade of the global system with the model lid at 0.1 hPa (as opposed to 10 hPa in GLBNEW) is in its final phase of acceptance at MSC Operations. In this model, the number of computational levels has been increased from 58 to 80, a nonorographic gravity wave drag based on Hines (1997a,b) has been implemented, and a new radiation scheme from Li and Barker (2005) using a correlated-k distribution for gaseous transmission is included. Other improvements to the model physics, related to the representation of deep convection [e.g., inclusion of the Bechtold et al. (2001) scheme], of slantwise convection (see Ma 2000; Morcrette and Browning 2006), and of land surface processes (inclusion of the Canadian Land Surface Scheme, CLASS; Verseghy 1991; Verseghy et al. 1993) are currently being explored.

Improvements to medium-range QPFs are also expected from better initial conditions, both in the atmosphere and at the surface. For upper-air data assimilation, positive impact should come from substantial increases in the volume of space-based observations that are expected for the 4DVAR data assimilation system. The assimilation of cloud and precipitation data may also be an important source of improvement (see Errico et al. 2007). For the surface, a new Canadian Land Data Assimilation System (Balsamo et al. 2006, 2007), including space-based information on soil moisture, terrestrial snow, and vegetation, is expected to be available for operation in 3–5 yr. Progress toward these achievements is anticipated in the next few years.

Acknowledgments

The operational implementation of Canada’s new 33-km global system for medium-range weather forecast required several years of research and development, and included contributions from many scientists, research assistants, operational meteorologists, computer experts, and managers. Although the list of these people is too long to be detailed here, we thank all of them. More specifically, we would like to highlight the contributions of Jean-Marc Bélanger and François Lemay, who had the task of running the final preimplementation tests prior to submission to MSC’s decisional committee. The precious help from Rochdi Lahlou in the production of the QPF scores is acknowledged. Thanks are also due to senior managers both in the Science and Technology Branch (Gilbert Brunet) and at MSC (Pierre Dubreuil, André Méthot, and Louis Lefaivre) for their support throughout this project.

REFERENCES

  • Baldwin, M. E., and Kain J. S. , 2006: Sensitivity of several performance measures to displacement error, bias, and event frequency. Wea. Forecasting, 21 , 636648.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Balsamo, G., Mahfouf J-F. , Bélair S. , and Deblonde G. , 2006: A global root-zone soil moisture analysis using simulated L-band brightness temperature in preparation for the Hydros satellite mission. J. Hydrometeor., 7 , 11261146.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Balsamo, G., Mahfouf J-F. , Bélair S. , and Deblonde G. , 2007: A land data assimilation system for soil moisture and temperature: An information content study. J. Hydrometeor., 8 , 12251242.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bechtold, P., Bazile E. , Guichard F. , Mascart P. , and Richard E. , 2001: A mass-flux convection scheme for regional and global models. Quart. J. Roy. Meteor. Soc., 127 , 869886.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., and Mailhot J. , 2001: Impact of horizontal resolution on the numerical simulation of a midlatitude squall line: Implicit and explicit condensation. Mon. Wea. Rev., 129 , 23622376.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Zhang D-L. , and Mailhot J. , 1994: Numerical prediction of the 10–11 June 1985 squall line with the Canadian regional finite-element model. Wea. Forecasting, 9 , 157172.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Mailhot J. , Strapp J. W. , and MacPherson J. I. , 1999: An examination of local versus nonlocal aspects of a TKE-based boundary layer scheme in clear convective conditions. J. Appl. Meteor., 38 , 14991518.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Méthot A. , Mailhot J. , Bilodeau B. , Patoine A. , Pellerin G. , and Côté J. , 2000: Operational implementation of the Fritsch–Chappell convective scheme in the 24-km Canadian regional model. Wea. Forecasting, 15 , 257274.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Crevier L-P. , Mailhot J. , Bilodeau B. , and Delage Y. , 2003a: Operational implementation of the ISBA land surface scheme in the Canadian regional weather forecast model. Part I: Warm season results. J. Hydrometeor., 4 , 352370.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Brown R. , Mailhot J. , Bilodeau B. , and Crevier L-P. , 2003b: Operational implementation of the ISBA land surface scheme in the Canadian regional weather forecast model. Part II: Cold season results. J. Hydrometeor., 4 , 371386.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bélair, S., Mailhot J. , Girard C. , and Vaillancourt P. , 2005: Boundary layer and shallow cumulus clouds in a medium-range forecast of large-scale weather system. Mon. Wea. Rev., 133 , 19381960.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Benoit, R., Côté J. , and Mailhot J. , 1989: Inclusion of a TKE boundary layer parameterization in the Canadian regional finite-element model. Mon. Wea. Rev., 117 , 17261750.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bougeault, P., 2008: Our changing forecast model. ECMWF Newsletter, No. 114, ECMWF, Reading, United Kingdom, 1.

  • Bougeault, P., and Lacarrère P. , 1989: Parameterization of orography-induced turbulence in a mesobeta-scale model. Mon. Wea. Rev., 117 , 18721890.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buizza, R., Houtekamer P. L. , Toth Z. , Pellerin G. , Wei M. , and Zhu Y. , 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133 , 10761097.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Casati, B., Ross G. , and Stephenson D. B. , 2004: A new intensity-scale approach for the verification of spatial precipitation forecasts. Meteor. Appl., 11 , 141154.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Côté, J., Desmarais J-G. , Gravel S. , Méthot A. , Patoine A. , Roch M. , and Staniforth A. , 1998a: The operational CMC–MRB Global Environmental Multiscale (GEM) model. Part II: Results. Mon. Wea. Rev., 126 , 13971418.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Côté, J., Gravel S. , Méthot A. , Patoine A. , Roch M. , and Staniforth A. , 1998b: The operational CMC–MRB Global Environmental Multiscale (GEM) model. Part I: Design considerations and formulation. Mon. Wea. Rev., 126 , 13731395.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Daly, C., Gibson W. P. , Taylor G. H. , Doggett M. K. , and Smith J. I. , 2007: Observer bias in daily precipitation measurements at United States cooperative network stations. Bull. Amer. Meteor. Soc., 88 , 899912.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Donaldson, R. J., Dyer R. M. , and Kraus R. M. , 1975: An objective evaluator of techniques for predicting severe weather events. Preprints, Ninth Conf. on Severe Local Storms, Norman, OK, Amer. Meteor. Soc., 321–326.

    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., Davies-Jones R. , and Keller D. L. , 1990: On summary measures of skill in rare event forecasting based on contingency tables. Wea. Forecasting, 5 , 576585.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drusch, M., and Viterbo P. , 2007: Assimilation of screen-level variables in ECMWF’s Integrated Forecast System: A study on the impact on the forecast quality and analyzed soil moisture. Mon. Wea. Rev., 135 , 300314.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and McBride J. L. , 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239 , 179202.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., Damrath U. , Wergen W. , and Baldwin M. E. , 2003: The WGNE assessment of short-term quantitative precipitation forecasts. Bull. Amer. Meteor. Soc., 84 , 481492.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Errico, R. M., Bauer P. , and Mahfouf J-M. , 2007: Issues regarding the assimilation of cloud and precipitation data. J. Atmos. Sci., 64 , 37853798.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fortin, V., Therrien C. , and Anctil F. , 2008: Correcting wind-induced bias in solid precipitation measurements in case of limited and uncertain data. Hydrol. Processes, 22 , 33933402. doi:10.1002/hyp.6959.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fritsch, J. M., and Chappell C. F. , 1980: Numerical prediction of convectively driven mesoscale pressure systems. Part I: Convective parameterization. J. Atmos. Sci., 37 , 17221733.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gallus W. A. Jr., , and Segal M. , 2001: Impact of improved initialization of mesoscale features on convective system rainfall in 10-km Eta simulations. Wea. Forecasting, 16 , 680696.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gauthier, P., Tanguay M. , Laroche S. , Pellerin S. , and Morneau J. , 2007: Extension of 3DVAR to 4DVAR: Implementation of 4DVAR at the Meteorological Service of Canada. Mon. Wea. Rev., 135 , 23392354.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Geleyn, J-F., 1987: Use of a modified Richardson number for parameterizing the effect of shallow convection. Short- and Medium-range Numerical Weather Prediction: Collection of Papers Presented at the WMO/IUGG NWP Symposium, Tokyo, Japan, WMO, 141–149.

    • Search Google Scholar
    • Export Citation
  • Goodison, B. E., Louie P. Y. T. , and Yang D. , 1998: WMO solid precipitation measurement intercomparison final report. WMO/TD 872, World Meteorological Organization, Geneva, Switzerland, 212 pp.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 1999: Hypothesis tests for evaluating numerical precipitation forecasts. Wea. Forecasting, 14 , 155167.

  • Hines, C. O., 1997a: Doppler-spread parameterization of gravity-wave momentum deposition in the middle atmosphere. Part I: Basic formulation. J. Atmos. Sol. Terr. Phys., 59 , 371386.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hines, C. O., 1997b: Doppler-spread parameterization of gravity-wave momentum deposition in the middle atmosphere. Part II: Broad and quasi monochromatic spectra, and implementation. J. Atmos. Sol. Terr. Phys., 59 , 387400.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and Mitchell H. L. , 2005: Ensemble Kalman filtering. Quart. J. Roy. Meteor. Soc., 131 , 32693289.

  • Jung, T., Gulev S. K. , Rudeva I. , and Soloviov V. , 2006: Sensitivity of extratropical cyclone characteristics to horizontal resolution in the ECMWF model. Quart. J. Roy. Meteor. Soc., 132 , 18391857.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J., and Fritsch J. M. , 1990: A one-dimensional entraining/detraining plume model and its application in convective parameterization. J. Atmos. Sci., 47 , 27842802.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J., and Fritsch J. M. , 1993: Convective parameterization for mesoscale models: The Kain–Fritsch scheme. The Representation of Cumulus Convection in Numerical Models, Meteror. Monogr., No. 46, Amer. Meteor. Soc., 165–170.

    • Search Google Scholar
    • Export Citation
  • Katz, R. W., and Murphy A. H. , Eds. 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.

  • Kuo, H. L., 1974: Further studies of the parameterization of the influence of cumulus convection on large-scale flow. J. Atmos. Sci., 31 , 12321240.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Laroche, S., Gauthier P. , St-James J. , and Morneau J. , 1999: Implementation of a 3D variational data assimilation system at the Canadian Meteorological Centre. Part II: The regional analysis. Atmos.–Ocean, 37 , 281307.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Laroche, S., Gauthier P. , Tanguay M. , Pellerin S. , and Morneau J. , 2007: Impact of the different components of 4DVAR on the Global Forecast System of the Meteorological Service of Canada. Mon. Wea. Rev., 135 , 23552364.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, J., and Barker H. W. , 2005: A radiation algorithm with correlated-k distribution. Part I: Local thermal equilibrium. J. Atmos. Sci., 62 , 286309.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ma, L., 2000: On the parameterization of slantwise convection in general circulation models. Ph.D. thesis, McGill University, Montreal, QC, Canada, 175 pp.

    • Search Google Scholar
    • Export Citation
  • Mailhot, J., and Bélair S. , 2002: An examination of a unified cloudiness–turbulence scheme with various types of cloudy boundary layers. Preprints, 15th Symp. on Boundary Layer and Turbulence, Wageningen, Netherlands, Amer. Meteor. Soc., 215–218.

    • Search Google Scholar
    • Export Citation
  • Mailhot, J., Chouinard C. , Benoit R. , Roch M. , Verner G. , Côté J. , and Pudykiewicz J. , 1989: Numerical forecasting of winter coastal storms during CASP: Evaluation of the regional finite-element model. Atmos.–Ocean, 27 , 2458.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mailhot, J., and Coauthors, 1998: Scientific description of RPN physics library: Version 3.6. Recherche en Prévision Numérique, 188 pp. [Available from RPN, 2121 Trans-Canada, Dorval, PQ H9P 1J3, Canada].

    • Search Google Scholar
    • Export Citation
  • Mailhot, J., and Coauthors, 2006: The 15-km version of the Canadian regional forecast system. Atmos.–Ocean, 44 , 133149.

  • Mason, I., 1989: Dependence of the critical success index on sample climate and threshold probability. Aust. Meteor. Mag., 37 , 7581.

  • Mesinger, F., and Brill K. , 2004: Bias normalized precipitation scores. Preprints, 17th Conf. on Probability and Statistics in the Atmospheric Sciences/20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., J12.6. [Available online at http://ams.confex.com/ams/pdfpapers/69561.pdf].

    • Search Google Scholar
    • Export Citation
  • Molinari, J., and Dudek M. , 1992: Parameterization of convective precipitation in mesoscale numerical models: A critical review. Mon. Wea. Rev., 120 , 326344.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morcrette, C. J., and Browning K. A. , 2006: Formation and release of symmetric instability following delta-M adjustment. Quart. J. Roy. Meteor. Soc., 132 , 10731089.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morcrette, J-J., Barker H. W. , Iacono M. J. , Mozdzynski G. , Pincus R. , Salmond D. , and Serrar S. , 2007: A new radiation package: McRad. ECMWF Newsletter, No. 112, ECMWF, Reading, United Kingdom, 22–32.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and Ehrendorfer M. , 1987: On the relationship between the accuracy and value of forecasts in the cost–loss ratio situation. Wea. Forecasting, 2 , 243251.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • NOAA, 1996: Standard hydrometeorological exchange format. Weather Service Hydrology Handbook 1, National Oceanic and Atmospheric Administration, 61 pp. [Available from Office of Hydrology, National Weather Service, Silver Spring, MD 20910].

    • Search Google Scholar
    • Export Citation
  • Noilhan, J., and Planton S. , 1989: A simple parameterization of land surface processes for meteorological models. Mon. Wea. Rev., 117 , 536549.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pudykiewicz, J., Benoit R. , and Mailhot J. , 1992: Inclusion and verification of a predictive cloud water scheme in a regional weather prediction model. Mon. Wea. Rev., 120 , 612626.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Richardson, D. S., 2000: Skill and relative economic value of the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc., 126 , 649667.

  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5 , 570575.

  • Stephenson, D. B., 2000: Use of the “odds ratio” for diagnosing forecast skill. Wea. Forecasting, 15 , 221232.

  • Sundqvist, H., 1978: A parameterization scheme for non-convective condensation including prediction of cloud water content. Quart. J. Roy. Meteor. Soc., 104 , 677690.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tremblay, A., 2005: The stratiform and convective components of surface precipitation. J. Atmos. Sci., 62 , 15131528.

  • Tustison, B., Harris D. , and Foufoula-Georgiou E. , 2001: Scale issues in verification of precipitation forecasts. J. Geophys. Res., 106 , (D11). 1177511784.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Verseghy, D. L., 1991: CLASS—A Canadian Land Surface Scheme for GCMs. I. Soil model. Int. J. Climatol., 11 , 111133.

  • Verseghy, D. L., McFarlane N. A. , and Lazare M. , 1993: CLASS—A Canadian Land Surface Scheme for GCMs. II. Vegetation model and coupled runs. Int. J. Climatol., 13 , 347370.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., and Hamill T. M. , 1995: Potential economic value of ensemble forecasts. Mon. Wea. Rev., 123 , 35653575.

  • Zhang, D-L., Kain J. S. , Fritsch M. , and Gao K. , 1994: Comments on “Parameterization of convective precipitation in mesoscale numerical models: A critical review.”. Mon. Wea. Rev., 122 , 22222231.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhu, Y., Toth Z. , Wobus R. , Richardson D. , and Mylne K. , 2002: The economic value of ensemble-based weather forecasts. Bull. Amer. Meteor. Soc., 83 , 7383.

    • Crossref
    • Search Google Scholar
    • Export Citation

Fig. 1.
Fig. 1.

Objective evaluation of 96-h forecasts against radiosondes for the GLBOLD (dotted lines) and GLBNEW (dash lines) global forecast systems for (a) geopotential height (dam), (b) temperature (K), (c) wind speed (m s−1), and (d) dewpoint depression (K). RMSE (black lines) and bias (gray lines) are shown. Verification is done over North America, for a set of 122 integrations initialized from 4DVAR analyses between 31 Aug and 31 Oct 2006.

Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1

Fig. 2.
Fig. 2.

SYNOP surface observation network over North America. Red exes show the locations of stations where manual measurements are taken. Blue exes show the locations of automatic stations.

Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1

Fig. 3.
Fig. 3.

Histograms showing the number of observed and predicted events with daily precipitation accumulations between 0 and 0.2 mm day−1, 0.2 and 5 mm day−1, etc. The histogram for the observations is shown in black, whereas histograms for the forecast systems are shown in yellow (GLBOLD), red (GLBNEW), and blue (REG).

Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1

Fig. 4.
Fig. 4.

Total precipitation (m) for each daily accumulations category (i.e., between 0 and 0.2 mm day−1, 0.2 and 5 mm day−1, etc). Totals for the observations are given by the thick full line, whereas totals for the GLBOLD, GLBNEW, and REG forecast systems are indicated by the dotted, dashed, and dashed–dotted lines. The total precipitation (including all categories) is shown on the right.

Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1

Fig. 5.
Fig. 5.

Threshold biases for the GLBOLD (dotted line), GLBNEW (dashed line), and REG (dashed–dotted line) forecast systems vs precipitation intensity.

Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1

Fig. 6.
Fig. 6.

ETS as a function of categories of precipitation intensity (mm day−1) for the GLBOLD (yellow), GLBNEW (red), and REG (blue) configurations. (top) The 0–24-, (middle) 48–72-, and (bottom) 96–120-h accumulations.

Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1

Fig. 7.
Fig. 7.

ETS as a function of thresholds for precipitation intensity (mm day−1) for the GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines) forecast systems. (top) The 0–24-, (middle) 48–72-, and (bottom) 96–120-h predictions.

Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1

Fig. 8.
Fig. 8.

ETS as a function of lead time for the GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines) forecast systems. (top) The 0.2, (middle) 10, and (bottom) 30 mm day−1 threshold results. Values at a certain lead time represents the ETS for the preceding 24-h period (e.g., values at 24 h are for the day 1 ETS).

Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1

Fig. 9.
Fig. 9.

Relative economic values of the predicted daily precipitation as a function of the cost–loss ratio (C/L) for three lead times (days 1, 3, and 5) and three thresholds of precipitation intensity (0.2, 10, and 30 mm day−1) for GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines).

Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1

Fig. 10.
Fig. 10.

Area between zero and the positive portion of the relative economic value curves given in Fig. 9, as a function of lead time. Results for GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines) are given for thresholds of precipitation intensity of 0.2, 10, and 30 mm day−1.

Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1

Table 1.

Configurations of GLBNEW, GLBOLD, and REG forecast systems.

Table 1.
Table 2.

Structure of summary contingency tables.

Table 2.
Table 3.

Structure of 2 × 2 contingency tables.

Table 3.
Save