Verification of 24-h Quantitative Precipitation Forecasts over the Pacific Northwest from a High-Resolution Ensemble Kalman Filter System

Phillipa Cookson-Hills Department of Atmospheric and Oceanic Sciences, McGill University, Montréal, Québec, Canada

Search for other papers by Phillipa Cookson-Hills in
Current site
Google Scholar
PubMed
Close
,
Daniel J. Kirshbaum Department of Atmospheric and Oceanic Sciences, McGill University, Montréal, Québec, Canada

Search for other papers by Daniel J. Kirshbaum in
Current site
Google Scholar
PubMed
Close
,
Madalina Surcel Department of Atmospheric and Oceanic Sciences, McGill University, Montréal, Québec, Canada

Search for other papers by Madalina Surcel in
Current site
Google Scholar
PubMed
Close
,
Jonathan G. Doyle Department of Atmospheric and Oceanic Sciences, McGill University, Montréal, Québec, Canada

Search for other papers by Jonathan G. Doyle in
Current site
Google Scholar
PubMed
Close
,
Luc Fillion Data Assimilation and Satellite Meteorology Research Section, Environment and Climate Change Canada, Dorval, Québec, Canada

Search for other papers by Luc Fillion in
Current site
Google Scholar
PubMed
Close
,
Dominik Jacques Data Assimilation and Satellite Meteorology Research Section, Environment and Climate Change Canada, Dorval, Québec, Canada

Search for other papers by Dominik Jacques in
Current site
Google Scholar
PubMed
Close
, and
Seung-Jong Baek Data Assimilation and Satellite Meteorology Research Section, Environment and Climate Change Canada, Dorval, Québec, Canada

Search for other papers by Seung-Jong Baek in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Environment and Climate Change Canada (ECCC) has recently developed an experimental high-resolution EnKF (HREnKF) regional ensemble prediction system, which it tested over the Pacific Northwest of North America for the first half of February 2011. The HREnKF has 2.5-km horizontal grid spacing and assimilates surface and upper-air observations every hour. To determine the benefits of the HREnKF over less expensive alternatives, its 24-h quantitative precipitation forecasts are compared with those from a lower-resolution (15 km) regional ensemble Kalman filter (REnKF) system and to ensembles directly downscaled from the REnKF using the same grid as the HREnKF but with no additional data assimilation (DS). The forecasts are verified against rain gauge observations and gridded precipitation analyses, the latter of which are characterized by uncertainties of comparable magnitude to the model forecast errors. Nonetheless, both deterministic and probabilistic verification indicates robust improvements in forecast skill owing to the finer grids of the HREnKF and DS. The HREnKF exhibits a further improvement in performance over the DS in the first few forecast hours, suggesting a modest positive impact of data assimilation. However, this improvement is not statistically significant and may be attributable to other factors.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Daniel Kirshbaum, daniel.kirshbaum@mcgill.ca

Abstract

Environment and Climate Change Canada (ECCC) has recently developed an experimental high-resolution EnKF (HREnKF) regional ensemble prediction system, which it tested over the Pacific Northwest of North America for the first half of February 2011. The HREnKF has 2.5-km horizontal grid spacing and assimilates surface and upper-air observations every hour. To determine the benefits of the HREnKF over less expensive alternatives, its 24-h quantitative precipitation forecasts are compared with those from a lower-resolution (15 km) regional ensemble Kalman filter (REnKF) system and to ensembles directly downscaled from the REnKF using the same grid as the HREnKF but with no additional data assimilation (DS). The forecasts are verified against rain gauge observations and gridded precipitation analyses, the latter of which are characterized by uncertainties of comparable magnitude to the model forecast errors. Nonetheless, both deterministic and probabilistic verification indicates robust improvements in forecast skill owing to the finer grids of the HREnKF and DS. The HREnKF exhibits a further improvement in performance over the DS in the first few forecast hours, suggesting a modest positive impact of data assimilation. However, this improvement is not statistically significant and may be attributable to other factors.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Daniel Kirshbaum, daniel.kirshbaum@mcgill.ca
Save
  • Austin, P. M., 1987: Relation between measured radar reflectivity and surface rainfall. Mon. Wea. Rev., 115, 10531070, doi:10.1175/1520-0493(1987)115<1053:RBMRRA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baek, S.-J., L. Fillion, and P. Houtekamer, 2012: Environment Canada’s regional ensemble Kalman filter: Some preliminary results. Fifth EnKF Workshop, Rensselaerville, NY. [Available online at http://hfip.psu.edu/fuz4/EnKF2012/Baek.pdf.]

  • Barrett, A. I., S. L. Gray, D. J. Kirshbaum, N. M. Roberts, D. M. Schultz, and J. G. Fairman Jr., 2016: The utility of convection-permitting ensembles for the prediction of stationary convective bands. Mon. Wea. Rev., 144, 10931114, doi:10.1175/MWR-D-15-0148.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bouallgue, Z., and S. Theis, 2014: Spatial techniques applied to precipitation ensemble forecasts: From verification results to probabilistic products. Meteor. Appl., 21, 992929, doi:10.1002/met.1435.

    • Search Google Scholar
    • Export Citation
  • Casati, B., and Coauthors, 2008: Forecast verification: Current status and future directions. Meteor. Appl., 15, 38, doi:10.1002/met.52.

  • Clark, A. J., and Coauthors, 2012: An overview of the 2010 Hazardous Weather Testbed Experimental Forecast Program Spring Experiment. Bull. Amer. Meteor. Soc., 93, 5574, doi:10.1175/BAMS-D-11-00040.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clark, P., N. Roberts, H. Lean, S. P. Ballard, and C. Charlton-Perez, 2016: Convection-permitting models: A step-change in rainfall forecasting. Meteor. Appl., 23, 165181, doi:10.1002/met.1538.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Colle, B. A., and C. F. Mass, 2000: The 5–9 February 1996 flooding event over the Pacific Northwest: Sensitivity studies and evaluation of the MM5 precipitation forecasts. Mon. Wea. Rev., 128, 593617, doi:10.1175/1520-0493(2000)128<0593:TFFEOT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Colle, B. A., C. F. Mass, and K. J. Westrick, 2000: MM5 precipitation verification over the Pacific Northwest during the 1997–99 cool seasons. Wea. Forecasting, 15, 730744, doi:10.1175/1520-0434(2000)015<0730:MPVOTP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dey, S., G. Leoncini, N. Roberts, R. Plant, and S. Migliorini, 2014: A spatial view of ensemble spread in convection permitting ensembles. Mon. Wea. Rev., 142, 40914107, doi:10.1175/MWR-D-14-00172.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E., 2009: A framework for neighbourhood verification of high resolution spatial forecasts. 18th World IMACS/MODSIM Congress, Cairns, QLD, Australia, Modelling and Simulation Society of Australia and New Zealand. [Available online at www.mssanz.org.au/modsim09/J1/ebert.pdf.]

  • Ehrendorfer, M., and J. J. Tribbia, 1997: Optimal prediction of forecast error covariances through singular vectors. J. Atmos. Sci., 54, 286313, doi:10.1175/1520-0469(1997)054<0286:OPOFEC>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fabry, F., and I. Zawadzki, 1995: Long-term radar observations of the melting layer of precipitation and their interpretation. J. Atmos. Sci., 52, 838851, doi:10.1175/1520-0469(1995)052<0838:LTROOT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fortin, V., and G. Roy, 2011: The Regional Deterministic Precipitation Analysis (RDPA). Meteorological Service of Canada and Meteorological Research Division Tech. Note, Environment Canada, 7 pp. [Available online at http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/lib/op_systems/doc_opchanges/technote_rdpa_e_20110406.pdf.]

  • Fortin, V., G. Roy, and A. Giguère, 2012: Improvements to the Regional Deterministic Precipitation Analysis (RDPA 2.3.0). Meteorological Service of Canada and Meteorological Research Division Tech. Note, Environment Canada, 56 pp. [Available online at http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/lib/op_systems/doc_opchanges/technote_rdpa_20121018_e.pdf.]

  • Fulton, R., 2002: Quantitative precipitation estimation in the National Weather Service. Hydrology Laboratory Office of Hydrologic Development Workshop, Silver Spring, MD, National Weather Service. [Available online at http://www.nws.noaa.gov/oh/hrl/papers/wsr88d/MPE_workshop_NWSTC_lecture1_121305.pdf.]

  • Gilleland, E., D. Ahijevych, B. Brown, B. Casati, and E. Ebert, 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24, 14161430, doi:10.1175/2009WAF2222269.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Girard, C., and Coauthors, 2014: Staggered vertical discretization of the Canadian Environmental Multiscale (GEM) model using a coordinate of the log-hydrostatic-pressure type. Mon. Wea. Rev., 142, 11831196, doi:10.1175/MWR-D-13-00255.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goudenhoofdt, E., and L. Delobbe, 2009: Evaluation of radar–gauge merging methods for quantitative precipitation estimates. Hydrol. Earth Syst. Sci., 13, 195203, doi:10.5194/hess-13-195-2009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Grubišić, V., R. Vellore, and A. Huggins, 2005: Quantitative precipitation forecasting of wintertime storms in the Sierra Nevada: Sensitivity to the microphysical parameterization and horizontal resolution. Mon. Wea. Rev., 133, 28342859, doi:10.1175/MWR3004.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 1999: Hypothesis tests for evaluating numerical precipitation forecasts. Wea. Forecasting, 14, 155167, doi:10.1175/1520-0434(1999)014<0155:HTFENP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550560, doi:10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and S. J. Colucci, 1997: Verification of Eta–RSM short-range ensemble forecasts. Mon. Wea. Rev., 125, 13121327, doi:10.1175/1520-0493(1997)125<1312:VOERSR>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., R. Hagedorn, and J. S. Whitaker, 2008: Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part II: Precipitation. Mon. Wea. Rev., 136, 26202632, doi:10.1175/2007MWR2411.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., H. L. M. G. Pellerin, M. Buehner, M. C. L. Spacek, and B. Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea. Rev., 133, 604620, doi:10.1175/MWR-2864.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., X. Deng, H. L. Mitchell, S.-J. Baek, and N. Gagnon, 2014: Higher resolution in an operational ensemble Kalman filter. Mon. Wea. Rev., 142, 11431162, doi:10.1175/MWR-D-13-00138.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houze, R. A., Jr., and S. Medina, 2005: Turbulence as a mechanism for orographic precipitation enhancement. J. Atmos. Sci., 62, 35993623, doi:10.1175/JAS3555.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jacques, D., W. Chang, S.-J. Baek, and L. Fillion, 2017: Developing a convective-scale EnKF data assimilation system for the Canadian MEOPAR project. Mon. Wea. Rev., 145, 14731494, doi:10.1175/MWR-D-16-0135.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jewell, S., and N. Gaussiat, 2015: An assessment of kriging-based rain-gauge–radar merging techniques. Quart. J. Roy. Meteor. Soc., 141, 23002313, doi:10.1002/qj.2522.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joss, J., and R. Lee, 1995: The application of radar–gauge comparisons to operational precipitation profile corrections. J. Appl. Meteor., 34, 26122630, doi:10.1175/1520-0450(1995)034<2612:TAORCT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and J. Fritsch, 1993: Convective parameterization for mesoscale models: The Kain-Fritsch scheme. The Representation of Cumulus Convection in Numerical Models, Meteor. Monogr., No. 46, Amer. Meteor. Soc., 165–170.

    • Crossref
    • Export Citation
  • Kain, J. S., and Coauthors, 2010: Assessing advances in the assimilation of radar data and other mesoscale observations within a collaborative forecasting–research environment. Wea. Forecasting, 25, 15101521, doi:10.1175/2010WAF2222405.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, 342 pp.

    • Crossref
    • Export Citation
  • Kirshbaum, D. J., G. H. Bryan, and R. Rotunno, 2007: The spacing of orographic rainbands triggered by small-scale topography. J. Atmos. Sci., 64, 42224245, doi:10.1175/2007JAS2335.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lakshmanan, V., T. Smith, K. Hondl, G. Stumpf, and A. Witt, 2006: A real-time, three-dimensional, rapidly updating, heterogeneous radar merger technique for reflectivity, velocity, and derived products. Wea. Forecasting, 21, 802823, doi:10.1175/WAF942.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lean, H. W., and K. A. Browning, 2013: Quantification of the importance of wind drift to the surface distribution of orographic rain on the occasion of the extreme Cockermouth flood in Cumbria. Quart. J. Roy. Meteor. Soc., 139, 13421353, doi:10.1002/qj.2024.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, X., D. A. Randall, and L. D. Fowler, 2000: Diurnal variability of the hydrologic cycle and radiative fluxes: Comparisons between observations and a GCM. J. Climate, 13, 41594179, doi:10.1175/1520-0442(2000)013<4159:DVOTHC>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, Y., and K. E. Mitchell, 2005: The NCEP stage II/IV hourly precipitation analyses: Development and applications. 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor, 1.2. [Available online at https://ams.confex.com/ams/Annual2005/techprogram/paper_83847.htm.]

  • Mahfouf, J., B. Brasnett, and S. Gagnon, 2007: A Canadian Precipitation Analysis (CaPA) project: Description and preliminary results. Atmos.–Ocean, 45, 117, doi:10.3137/ao.v450101.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Martner, B., S. Y. White, A. White, S. Y. Matrosov, D. E. Kingsmill, and F. M. Ralph, 2008: Raindrop size distributions and rain characteristics in California coastal rainfall for periods with and without a radar bright band. J. Hydrometeor., 9, 408425, doi:10.1175/2007JHM924.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mass, C. F., and G. K. Ferber, 1990: Surface pressure perturbations produced by an isolated mesoscale topographic barrier. Part I: General characteristics and dynamics. Mon. Wea. Rev., 118, 25792596, doi:10.1175/1520-0493(1990)118<2579:SPPPBA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mass, C. F., D. Ovens, K. Westrick, and B. Colle, 2002: Does increasing horizontal resolution produce more skillful forecasts? The results of two years of real-time numerical weather prediction over the Pacific Northwest. Bull. Amer. Meteor. Soc., 83, 407430, doi:10.1175/1520-0477(2002)083<0407:DIHRPM>2.3.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Milbrandt, J., and M. Yau, 2005: A multimoment bulk microphysics parameterization. Part I: Analysis of the role of the spectral shape parameter. J. Atmos. Sci., 62, 30513064, doi:10.1175/JAS3534.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Minder, J., D. Durran, G. Roe, and A. Anders, 2008: The climatology of small-scale orographic precipitation over the Olympic Mountains: Patterns and processes. Quart. J. Roy. Meteor. Soc., 134, 817839, doi:10.1002/qj.258.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mote, P. W., and Coauthors, 2003: Preparing for climatic change: The water, salmon, and forests of the Pacific Northwest. Climatic Change, 61, 4588, doi:10.1023/A:1026302914358.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roberts, N., 2008: Assessing the spatial and temporal variation in the skill of precipitation forecasts from an NWP model. Meteor. Appl., 15, 163169, doi:10.1002/met.57.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roberts, N., and H. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 7897, doi:10.1175/2007MWR2123.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5, 570575, doi:10.1175/1520-0434(1990)005<0570:TCSIAA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schwitalla, T., H. Bauer, V. Wulfmeyer, and G. Zängl, 2008: Systematic errors of QPF in low-mountain regions as revealed by MM5 simulations. Meteor. Z., 17, 903919, doi:10.1127/0941-2948/2008/0338.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, C., 2007: Correcting the wind bias in snowfall measurements made with a Geonor T-200B precipitation gauge and alter wind shield. 14th Symp. on Meteorological Observation and Instrumentation, San Antonio, TX, Amer. Meteor. Soc., 1.5. [Available online at https://ams.confex.com/ams/87ANNUAL/techprogram/paper_118544.htm.]

  • Sundqvist, H., 1978: A parameterization scheme for non-convective condensation including prediction of cloud water content. Quart. J. Roy. Meteor. Soc., 104, 677690, doi:10.1002/qj.49710444110.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Surcel, M., I. Zawadzki, and M. Yau, 2014: On the filtering properties of ensemble averaging for storm-scale precipitation forecasts. Mon. Wea. Rev., 142, 10931105, doi:10.1175/MWR-D-13-00134.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Surcel, M., I. Zawadzki, and M. K. Yau, 2015: A study on the scale dependence of the predictability of precipitation patterns. J. Atmos. Sci., 72, 216235, doi:10.1175/JAS-D-14-0071.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Talagrand, O., R. Vautard, and B. Strauss, 1997: Evaluation of probabilistic prediction systems. Proc. Workshop on Predictability, Reading, United Kingdom, ECMWF, 1–26.

  • Taylor, K. E., 2001: Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res., 106, 71837192, doi:10.1029/2000JD900719.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vié, B., O. Nuissier, and V. Ducrocq, 2011: Cloud-resolving ensemble simulations of Mediterranean heavy precipitating events: Uncertainty on initial conditions and lateral boundary conditions. Mon. Wea. Rev., 139, 403423, doi:10.1175/2010MWR3487.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Westrick, K., C. Mass, and B. Colle, 1999: The limitations of the WSR-88D radar network for quantitative precipitation measurement over the coastal western United States. Bull. Amer. Meteor. Soc., 80, 22892298, doi:10.1175/1520-0477(1999)080<2289:TLOTWR>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Whitaker, J. S., T. M. Hamill, X. Wei, Y. Song, and Z. Toth, 2008: Ensemble data assimilation with the NCEP Global Forecast System. Mon. Wea. Rev., 136, 463482, doi:10.1175/2007MWR2018.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D., 1995: Statistical Methods in the Atmospheric Sciences: An Introduction. Academic Press, 467 pp.

  • Wilks, D., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Elsevier, 676 pp.

    • Crossref
    • Export Citation
  • Wood, S., D. Jones, and R. Moore, 2000: Accuracy of rainfall measurement for scales of hydrological interest. Hydrol. Earth Syst. Sci., 4, 531543, doi:10.5194/hess-4-531-2000.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yang, D., B. Goodison, J. Metcalfe, V. Golubev, R. Bates, T. Pangburn, and C. Hanson, 1995: Accuracy of NWS 8” standard non-recording precipitation gauge: Result of WMO intercomparison. Preprints, Ninth Conf. on Applied Climatology, Dallas, TX, Amer. Meteor. Soc., 29–34.

  • Yang, D., and Coauthors, 1999: Wind-induced precipitation undercatch of the Hellmann gauges. Hydrol. Res., 30, 5780.

  • Yang, S.-C., E. Kalnay, and B. Hunt, 2012: Handling nonlinearity in an ensemble Kalman filter: Experiments with the three-variable Lorenz model. Mon. Wea. Rev., 140, 26282646, doi:10.1175/MWR-D-11-00313.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, M., F. Zhang, X. Huang, and X. Zhang, 2011: Intercomparison of an ensemble Kalman filter with three- and four-dimensional variational data assimilation methods in a limited-area model over the month of June 2003. Mon. Wea. Rev., 139, 566572, doi:10.1175/2010MWR3610.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 434 74 1
PDF Downloads 313 70 8