Comparison of Methods Used to Generate Probabilistic Quantitative Precipitation Forecasts over South America

Juan Ruiz Centro de Investigaciones del Mar y la Atmósfera–CONICET/University of Buenos Aires, and Departamento de Ciencias de la Atmósfera y los Océanos, Facultad de Ciencias Exactas y Naturales, University of Buenos Aires, Buenos Aires, Argentina

Search for other papers by Juan Ruiz in
Current site
Google Scholar
PubMed
Close
,
Celeste Saulo Centro de Investigaciones del Mar y la Atmósfera–CONICET/University of Buenos Aires, and Departamento de Ciencias de la Atmósfera y los Océanos, Facultad de Ciencias Exactas y Naturales, University of Buenos Aires, Buenos Aires, Argentina

Search for other papers by Celeste Saulo in
Current site
Google Scholar
PubMed
Close
, and
Eugenia Kalnay Department of Atmospheric and Oceanic Sciences, University of Maryland, College Park, College Park, Maryland

Search for other papers by Eugenia Kalnay in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

In this work, the quality of several probabilistic quantitative precipitation forecasts (PQPFs) is examined. The analysis is focused over South America during a 2-month period in the warm season. Several ways of generating and calibrating the PQPFs have been tested, using different ensemble systems and single-model runs. Two alternative calibration techniques (static and dynamic) have been tested.

To take into account different precipitation regimes, PQPF performance has been evaluated over two regions: the northern part of South America, characterized by a tropical regime, and the southern part, where synoptic-scale forcing is stronger. The results support the adoption of such area separation, since differences in the precipitation regimes produce significant differences in PQPF performance.

The more skillful PQPFs are the ones obtained after calibration. PQPFs derived from the ensemble mean also show higher skill and better reliability than those derived from the single ensemble members. The performance of the PQPFs derived from both ensemble systems is similar over the southern part of the region; however, over the northern part the superensemble approach seems to achieve better results in both reliability and skill.

Finally, the impact of using Climate Prediction Center morphing technique (CMORPH) estimates to calibrate the precipitation forecast has been explored since the more extensive coverage of this dataset would allow its use over areas where the rain gauge coverage is insufficient. Results suggest that systematic biases present in the CMORPH estimates produce only a slight degradation of the resulting PQPF.

Corresponding author address: Juan Ruiz, Centro de Investigaciones del Mar y Atmósfera, Facultad de Ciencias Exactas y Naturales, Ciudad Universitaria, Pabellón II, 2do Piso, Buenos Aires 1428, Argentina. Email: jruiz@cima.fcen.uba.ar

Abstract

In this work, the quality of several probabilistic quantitative precipitation forecasts (PQPFs) is examined. The analysis is focused over South America during a 2-month period in the warm season. Several ways of generating and calibrating the PQPFs have been tested, using different ensemble systems and single-model runs. Two alternative calibration techniques (static and dynamic) have been tested.

To take into account different precipitation regimes, PQPF performance has been evaluated over two regions: the northern part of South America, characterized by a tropical regime, and the southern part, where synoptic-scale forcing is stronger. The results support the adoption of such area separation, since differences in the precipitation regimes produce significant differences in PQPF performance.

The more skillful PQPFs are the ones obtained after calibration. PQPFs derived from the ensemble mean also show higher skill and better reliability than those derived from the single ensemble members. The performance of the PQPFs derived from both ensemble systems is similar over the southern part of the region; however, over the northern part the superensemble approach seems to achieve better results in both reliability and skill.

Finally, the impact of using Climate Prediction Center morphing technique (CMORPH) estimates to calibrate the precipitation forecast has been explored since the more extensive coverage of this dataset would allow its use over areas where the rain gauge coverage is insufficient. Results suggest that systematic biases present in the CMORPH estimates produce only a slight degradation of the resulting PQPF.

Corresponding author address: Juan Ruiz, Centro de Investigaciones del Mar y Atmósfera, Facultad de Ciencias Exactas y Naturales, Ciudad Universitaria, Pabellón II, 2do Piso, Buenos Aires 1428, Argentina. Email: jruiz@cima.fcen.uba.ar

Save
  • Atger, F., 2003: Spatial and interannual variability of the reliability of ensemble-based probabilistic forecasts: Consequences for calibration. Mon. Wea. Rev., 131 , 15091523.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78 , 13.

  • Buizza, R., Houtekamer P. L. , Toth Z. , Pellerin G. , Wei M. , and Zhu Y. , 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133 , 10761097.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, F., and Dudhia J. , 2001: Coupling an advanced land surface–hydrology model with the Penn State–NCAR MM5 modeling system. Part I: Model description and implementation. Mon. Wea. Rev., 129 , 569585.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dalcher, A., Kalnay E. , and Hoffman R. N. , 1988: Medium range lagged average forecasts. Mon. Wea. Rev., 116 , 402416.

  • Du, J., Mullen S. L. , and Sanders F. , 1997: Short-range ensemble forecasting of quantitative precipitation. Mon. Wea. Rev., 125 , 24272459.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dudhia, J., 1989: Numerical study of convection observed during the Winter Monsoon Experiment using a mesoscale two-dimensional model. J. Atmos. Sci., 46 , 30773107.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., 2001: Ability of a poor man’s ensemble to predict the probability and distribution of precipitation. Mon. Wea. Rev., 129 , 24612480.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., Janowiak J. E. , and Kidd C. , 2007: Comparison of near-real-time precipitation estimates from satellite observations and numerical models. Bull. Amer. Meteor. Soc., 88 , 4764.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebisuzaki, W., and Kalnay E. , 1991: Ensemble experiments with a new lagged average forecasting scheme. WMO Research Activities in Atmospheric and Oceanic Modeling Rep. 15, 308 pp. [Available from WMO, C.P. No 2300, CH1211, Geneva, Switzerland.].

    • Search Google Scholar
    • Export Citation
  • Eckel, F. A., and Walters M. K. , 1998: Calibrated probabilistic quantitative precipitation forecasts based on the MRF ensemble. Wea. Forecasting, 13 , 11321147.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Freitas, S. R., Longo K. , Silva Dias M. , Silva Dias P. , Chatfield R. , Fazenda A. , and Rodrigues L. F. , 2006: The coupled aerosol and tracer transport model to the Brazilian developments on the Regional Atmospheric Modeling System: Validation using direct and remote sensing observations. Proc. Eighth Int. Conf. on Southern Hemisphere Meteorology and Oceanography, Foz do Iguazu, Brazil, INPE, 101–107.

  • Gallus W. A. Jr., , and Segal M. , 2004: Does increased predicted warm-season rainfall indicate enhanced likelihood of rain occurrence? Wea. Forecasting, 19 , 11271135.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Grell, G. A., and Devenyi D. , 2002: A generalized approach to parameterizing convection combining ensemble and data assimilation techniques. Geophys. Res. Lett., 29 , 1639. doi:10.1029/2002GL015311.

    • Search Google Scholar
    • Export Citation
  • Hamill, T., 1999: Hypothesis tests for evaluating numerical precipitation forecasts. Wea. Forecasting, 14 , 155167.

  • Hamill, T., and Colucci S. J. , 1997: Verification of Eta–RSM short-range ensemble forecasts. Mon. Wea. Rev., 125 , 13121327.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T., and Colucci S. J. , 1998: Evaluation of Eta–RSM ensemble probabilistic precipitation forecasts. Mon. Wea. Rev., 126 , 711724.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T., and Juras J. , 2006: Measuring forecast skill: Is it real skill or is it the varying climatology? Quart. J. Roy. Meteor. Soc., 132 , 29052923.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hou, D., Kalnay E. , and Droegemeier K. K. , 2001: Objective verification of the SAMEX’98 ensemble forecast. Mon. Wea. Rev., 129 , 7391.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joyce, R. J., Janowiak J. E. , Arkin P. A. , and Xie P. , 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5 , 487503.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kalnay, E., Kanamitsu M. , and Baker W. E. , 1990: Global numerical weather prediction at the National Meteorological Center. Bull. Amer. Meteor. Soc., 71 , 14101428.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kinter J. L. III, , and Coauthors, 1997: The COLA Atmosphere–Biosphere general circulation model. Volume 1: Formulation. COLA Tech. Rep. 51, Center for Ocean–Land–Atmosphere Studies, Calverton, MD, 46 pp.

  • Kousky, V. E., Janowiak J. E. , and Joyce R. J. , 2006: The diurnal cycle of precipitation over South America based on CMORPH. Proc. Eighth Int. Conf. on Southern Hemisphere Meteorology and Oceanography, Foz do Iguaçu, Brazil, INPE, 1113–1116.

  • Krishnamurti, T. N., Kishtawal C. M. , LaRow T. E. , Bachiochi D. R. , Zhang Z. , Willford C. E. , Gadgil S. , and Surendran S. , 1999: Improved weather and seasonal climate forecast from multi-model superensemble. Science, 285 , 15481550.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McLean, J., Raftery A. , Gneiting T. , and Fraley C. , 2007: Probabilistic quantitative precipitation forecasting using Bayesian model averaging. Mon. Wea. Rev., 135 , 32093220.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mellor, G. L., and Yamada T. , 1982: Development of a turbulence closure model for geophysical fluid problems. Rev. Geophys. Space Phys., 20 , 851875.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mendoça, A. M., and Bonatti J. P. , 2006: Experiments with EOF-based perturbation methods to ensemble weather forecasting in middle latitudes. Proc. Eighth Int. Conf. on Southern Hemisphere Meteorology and Oceanography, Foz do Iguazu, Brazil, Amer. Meteor. Soc. and INEP, 1829–1832.

  • Mesinger, F., Janjić Z. I. , Nickovic S. , Gavrilov D. , and Deaven D. G. , 1988: The step-mountain coordinate: Model description and performance for cases of Alpine lee cyclogenesis and for a case of Appalachian redevelopment. Mon. Wea. Rev., 116 , 14931518.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mlawer, E. J., Taubman S. J. , Brown P. D. , Iacono M. J. , and Clough S. A. , 1997: Radiative transfer for inhomogeneous atmosphere: RRTM, a validated correlated-k model for the long wave. J. Geophys. Res., 102 , (D14). 1666316682.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Molteni, F., Buizza R. , Palmer T. N. , and Petroliagis T. , 1996: The ECMWF Ensemble Prediction System: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122 , 73119.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8 , 281293.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Raftery, A., Gneiting T. , Balabdaoui F. , and Polakowski M. , 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133 , 11551174.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ruiz, J. J., Saulo A. C. , and Kalnay E. , 2006a: A regional ensemble forecast system for southeastern South America: Preliminary assessment. Proc. Eighth Int. Conf. on Southern Hemisphere Meteorology and Oceanography, Foz do Iguazu, Brazil, INPE, 1977–1984.

  • Ruiz, J. J., Saulo A. C. , García Skabar Y. , and Salio P. , 2006b: Representation of a Mesoscale Convective System using the RAMS model (in Spanish). Meteorologica, 31 , 1336.

    • Search Google Scholar
    • Export Citation
  • Ruiz, J. J., Ferreira L. , and Saulo A. C. , 2007: WRF-ARW sensitivity to different planetary boundary layer parameterization over South America. Research Activities in Atmospheric and Oceanic Modelling: CAS/JSC Working Group in Numerical Experimentation (WEGNE) Blue Book, WMO 11–12.

    • Search Google Scholar
    • Export Citation
  • Saulo, A. C., and Ferreira L. , 2003: Evaluation of quantitative precipitation forecasts over southern South America. Aust. Meteor. Mag., 52 , 8193.

    • Search Google Scholar
    • Export Citation
  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5 , 570575.

  • Silva Dias, P. L., Soares Moreira D. , and Neto G. D. , 2006: The MASTER Model Ensemble System (MSMES). Proc. Eighth Int. Conf. on Southern Hemisphere Meteorology and Oceanography, Foz do Iguaçu, Brazil, INPE, 1751–1757.

  • Skamarock, W. C., Klemp J. B. , Dudhia J. , Gill D. O. , Barker D. M. , Wang W. , and Powers J. G. , 2005: A description of the Advanced Research WRF Version 2. NCAR Tech. Note NCAR/TN-468+STR, 88 pp.

  • Stensrud, D., and Yussouf N. , 2007: Reliable probabilistic quantitative precipitation forecasts from a short-range ensemble forecasting system. Wea. Forecasting, 22 , 317.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toth, Z., and Kalnay E. , 1993: Ensemble forecasting at NMC: The generation of pertubations. Bull. Amer. Meteor. Soc., 74 , 23172330.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toth, Z., and Kalnay E. , 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125 , 32973319.

  • Zhang, F., Snyder C. , and Rotunno R. , 2002: Mesoscale predictability of the “surprise” snowstorm of 24–25 January 2000. Mon. Wea. Rev., 130 , 16171632.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, F., Odins M. , and Nielsen-Gammon J. W. , 2006: Mesoscale predictability of an extreme warm-season precipitation event. Wea. Forecasting, 21 , 149166.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhu, Y., Toth Z. , Wobus R. , Richardson D. , and Mylne K. , 2002: The economic value of ensemble-based weather forecasts. Bull. Amer. Meteor. Soc., 83 , 7383.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 120 57 7
PDF Downloads 63 25 3