Comparative Evaluation of Three Schaake Shuffle Schemes in Postprocessing GEFS Precipitation Ensemble Forecasts

Limin Wu Lynker Technologies, and Office of Water Prediction, Silver Spring, Maryland

Search for other papers by Limin Wu in
Current site
Google Scholar
PubMed
Close
,
Yu Zhang The University of Texas at Arlington, Arlington, Texas

Search for other papers by Yu Zhang in
Current site
Google Scholar
PubMed
Close
,
Thomas Adams TerraPredictions, Blacksburg, Virginia

Search for other papers by Thomas Adams in
Current site
Google Scholar
PubMed
Close
,
Haksu Lee LEN Technologies, and Office of Water Prediction, Silver Spring, Maryland

Search for other papers by Haksu Lee in
Current site
Google Scholar
PubMed
Close
,
Yuqiong Liu LEN Technologies, and Office of Water Prediction, Silver Spring, Maryland

Search for other papers by Yuqiong Liu in
Current site
Google Scholar
PubMed
Close
, and
John Schaake Consultant, Annapolis, Maryland

Search for other papers by John Schaake in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Natural weather systems possess certain spatiotemporal variability and correlations. Preserving these spatiotemporal properties is a significant challenge in postprocessing ensemble weather forecasts. To address this challenge, several rank-based methods, the Schaake Shuffle and its variants, have been developed in recent years. This paper presents an extensive assessment of the Schaake Shuffle and its two variants. These schemes differ in how the reference multivariate rank structure is established. The first scheme (SS-CLM), an implementation of the original Schaake Shuffle method, relies on climatological observations to construct rank structures. The second scheme (SS-ANA) utilizes precipitation event analogs obtained from a historical archive of observations. The third scheme (SS-ENS) employs ensemble members from the Global Ensemble Forecast System (GEFS). Each of the three schemes is applied to postprocess precipitation ensemble forecasts from the GEFS for its first three forecast days over the mid-Atlantic region of the United States. In general, the effectiveness of these schemes depends on several factors, including the season (or precipitation pattern) and the level of gridcell aggregation. It is found that 1) the SS-CLM and SS-ANA behave similarly in spatial and temporal correlations; 2) by a measure for capturing spatial variability, the SS-ENS outperforms the SS-ANA, which in turn outperforms the SS-CLM; and 3), overall, the SS-ANA performs better than the SS-CLM. The study also reveals that it is important to choose a proper size for the postprocessed ensembles in order to capture extreme precipitation events.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Limin Wu, limin.wu@noaa.gov

Abstract

Natural weather systems possess certain spatiotemporal variability and correlations. Preserving these spatiotemporal properties is a significant challenge in postprocessing ensemble weather forecasts. To address this challenge, several rank-based methods, the Schaake Shuffle and its variants, have been developed in recent years. This paper presents an extensive assessment of the Schaake Shuffle and its two variants. These schemes differ in how the reference multivariate rank structure is established. The first scheme (SS-CLM), an implementation of the original Schaake Shuffle method, relies on climatological observations to construct rank structures. The second scheme (SS-ANA) utilizes precipitation event analogs obtained from a historical archive of observations. The third scheme (SS-ENS) employs ensemble members from the Global Ensemble Forecast System (GEFS). Each of the three schemes is applied to postprocess precipitation ensemble forecasts from the GEFS for its first three forecast days over the mid-Atlantic region of the United States. In general, the effectiveness of these schemes depends on several factors, including the season (or precipitation pattern) and the level of gridcell aggregation. It is found that 1) the SS-CLM and SS-ANA behave similarly in spatial and temporal correlations; 2) by a measure for capturing spatial variability, the SS-ENS outperforms the SS-ANA, which in turn outperforms the SS-CLM; and 3), overall, the SS-ANA performs better than the SS-CLM. The study also reveals that it is important to choose a proper size for the postprocessed ensembles in order to capture extreme precipitation events.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Limin Wu, limin.wu@noaa.gov
Save
  • Ajami, N. K., G. M. Hornberger, and D. L. Sunding, 2008: Sustainable water resource management under hydrological uncertainty. Water Resour. Res., 44, W11406, https://doi.org/10.1029/2007WR006736.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baddeley, A., 1992: Errors in binary images and an Lp version of the Hausdorff metric. Nieuw Arch. Wiskunde, 10, 157183.

  • Berrocal, V. J., and A. Raftery, 2008: Probabilistic quantitative precipitation field forecasting using a two-stage spatial model. Ann. Appl. Stat., 2, 11701193, https://doi.org/10.1214/08-AOAS203.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berrocal, V. J., A. Raftery, and T. Gneiting, 2007: Combining spatial statistical and ensemble information in probabilistic weather forecasts. Mon. Wea. Rev., 135, 13861402, https://doi.org/10.1175/MWR3341.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brown, J. D., 2015: An evaluation of the minimum requirements for meteorological reforecasts from the Global Ensemble Forecast System (GEFS) of the U.S. National Weather Service (NWS) in support of the calibration and validation of the NWS Hydrologic Ensemble Forecast Service (HEFS). Tech. Rep., 120 pp., http://www.nws.noaa.gov/oh/hrl/hsmb/docs/hep/publications_presentations/HSL_LYNT_DG133W-13-CQ-0042_SubK_2013_1003_Task_3_Deliverable_04_report_FINAL.pdf.

  • Brown, J. D., L. Wu, M. He, S. Regonda, H. Lee, and D.-J. Seo, 2014a: Verification of temperature, precipitation, and streamflow forecasts from the NOAA/NWS Hydrologic Ensemble Forecast Service (HEFS): 1. Experimental design and forcing verification. J. Hydrol., 519, 28692889, https://doi.org/10.1016/j.jhydrol.2014.05.028.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brown, J. D., M. He, S. Regonda, L. Wu, H. Lee, and D.-J. Seo, 2014b: Verification of temperature, precipitation, and streamflow forecasts from the NOAA/NWS Hydrologic Ensemble Forecast Service (HEFS): 2. Streamflow verification. J. Hydrol., 519, 28472868, https://doi.org/10.1016/j.jhydrol.2014.05.030.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chernick, M. R., 2008: Bootstrap Methods: A Guide for Practitioners and Researchers. 2nd ed., Wiley, 400 pp.

  • Clark, A. J., W. A. Gallus, and M. L. Weisman, 2010: Neighborhood-based verification of precipitation forecasts from convection-allowing NCAR WRF model simulations and the operational NAM. Wea. Forecasting, 25, 14951509, https://doi.org/10.1175/2010WAF2222404.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clark, M., S. Gangopadhyay, L. Hay, B. Rajagopalan, and R. Wilby, 2004: The Schaake Shuffle: A method for reconstructing space-time variability in forecasted precipitation and temperature fields. J. Hydrometeor., 5, 243262, https://doi.org/10.1175/1525-7541(2004)005<0243:TSSAMF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cui, B., Z. Toth, Y. Zhu, and D. Hou, 2012: Bias correction for global ensemble forecast. Wea. Forecasting, 27, 396410, https://doi.org/10.1175/WAF-D-11-00011.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Daly, C., W. P. Gibson, M. Doggett, J. Smith, and G. Taylor, 2004: Up-to-date monthly climate maps for the conterminous United States. 14th Conf. on Applied Climatology, Seattle, WA, Amer. Meteor. Soc., P5.1, https://ams.confex.com/ams/84Annual/techprogram/paper_71444.htm.

  • Demargne, J., and Coauthors, 2014: The science of NOAA’s operational hydrologic ensemble forecast service. Bull. Amer. Meteor. Soc., 95, 7998, https://doi.org/10.1175/BAMS-D-12-00081.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., 2008: Fuzzy verification of high-resolution gridded forecasts: A review and proposed framework. Meteor. Appl., 15, 5164, https://doi.org/10.1002/met.25.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., 2009: Neighborhood verification: A strategy for rewarding close forecasts. Wea. Forecasting, 24, 14981510, https://doi.org/10.1175/2009WAF2222251.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Eldardiry, H., E. Habib, Y. Zhang, and J. Graschel, 2015: Artifacts in Stage IV NWS real-time multisensor precipitation estimates and impacts on identification of maximum series. J. Hydrol. Eng., 22, https://doi.org/10.1061/(ASCE)HE.1943-5584.0001291.

    • Search Google Scholar
    • Export Citation
  • Gel, Y., A. E. Raftery, and T. Gneiting, 2004: Calibrated probabilistic mesoscale weather field forecasting: The geostatistical output perturbation method. J. Amer. Stat. Assoc., 99, 575583, https://doi.org/10.1198/016214504000000872.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Georgakakos, A. P., H. Yao, M. Mullusky, and K. P. Georgakakos, 1998: Impacts of climate variability on the operational forecast and management of the Upper Des Moines River basin. Water Resour. Res., 34, 799821, https://doi.org/10.1029/97WR03135.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gilleland, E., 2011: Spatial forecast verification: Baddeley’s delta metric applied to the ICP test cases. Wea. Forecasting, 26, 409415, https://doi.org/10.1175/WAF-D-10-05061.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Greene, D., and M. Hudlow, 1982: Hydrometeorologic grid mapping procedures. Proc. Int. Symp. on Hydrometeorology, Denver, CO, AWRA, 20 pp.

  • Hamill, T. M., and J. S. Whitaker, 2006: Probability quantitative precipitation forecasts based on reforecast analogs: Theory and application. Mon. Wea. Rev., 134, 32093229, https://doi.org/10.1175/MWR3237.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., G. Bates, J. Whitaker, D. Murray, M. Fiorino, T. Galarneau, Y. Zhu, and W. Lapenta, 2013: NOAA’s second generation global medium-range ensemble reforecast dataset. Bull. Amer. Meteor. Soc., 94, 15531565, https://doi.org/10.1175/BAMS-D-12-00014.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Herr, H. D., and R. Krzysztofowicz, 2005: Generic probability distribution of rainfall in space: The bivariate model. J. Hydrol., 306, 234263, https://doi.org/10.1016/j.jhydrol.2004.09.011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15, 559570, https://doi.org/10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., and D. B. Stephenson, 2011: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. 2nd ed. Wiley, 292 pp.

    • Crossref
    • Export Citation
  • Kelly, K. S., and R. Krzysztofowicz, 1997: A bivariate meta-Gaussian density for use in hydrology. Stoch. Hydrol. Hydraul., 11, 1731, https://doi.org/10.1007/BF02428423.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kitzmiller, D., and Coauthors, 2011: Evolving multisensor precipitation estimation methods: Their impacts on flow prediction using a distributed hydrologic model. J. Hydrometeor., 12, 14141431, https://doi.org/10.1175/JHM-D-10-05038.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Maglaras, G. J., J. S. Waldstreicher, P. J. Kocin, A. F. Gigi, and R. A. Marine, 1995: Winter weather forecasting throughout the eastern United States. Part 1: An overview. Wea. Forecasting, 10, 520, https://doi.org/10.1175/1520-0434(1995)010<0005:WWFTTE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marzban, C., and S. Sandgathe, 2010: Optical flow for verification. Wea. Forecasting, 25, 14791494, https://doi.org/10.1175/2010WAF2222351.1.

  • Pinson, P., 2012: Adaptive calibration of (u,v)-wind ensemble forecasts. Quart. J. Roy. Meteor. Soc., 138, 12731284, https://doi.org/10.1002/qj.1873.

  • Raftery, A. E., T. Gneiting, F. Balabdaoui, and M. Polakowski, 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133, 11551174, https://doi.org/10.1175/MWR2906.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • R Core Team, 2016: R: A language and environment for statistical computing. R Foundation for Statistical Computing, https://www.R-project.org.

  • Robertson, D. E., D. L. Shrestha, and Q. J. Wang, 2013: Postprocessing rainfall forecasts from numerical weather prediction models for short-term streamflow forecasting. Hydrol. Earth Syst. Sci., 17, 35873603, https://doi.org/10.5194/hess-17-3587-2013.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schaake, J., and Coauthors, 2007: Precipitation and temperature ensemble forecasts from single-value forecasts. Hydrol. Earth Syst. Sci. Discuss., 4, 655717, https://doi.org/10.5194/hessd-4-655-2007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schefzik, R., 2016: A similarity-based implementation of the Schaake Shuffle. Mon. Wea. Rev., 144, 19091921, https://doi.org/10.1175/MWR-D-15-0227.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schefzik, R., T. L. Thorarinsdottir, and T. Gneiting, 2013: Uncertainty quantification in complex simulation models using ensemble copula coupling. Stat. Sci., 28, 616640, https://doi.org/10.1214/13-STS443.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Scheuerer, M., and T. M. Hamill, 2015: Statistical postprocessing of ensemble precipitation forecasts by fitting censored, shifted gamma distributions. Mon. Wea. Rev., 143, 45784596, https://doi.org/10.1175/MWR-D-15-0061.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Scheuerer, M., T. M. Hamill, B. Whitin, M. He, and A. Henkel, 2017: A method for preferential selection of dates in the Schaake shuffle approach to constructing spatiotemporal forecast fields of temperature and precipitation. Water Resour. Res., 53, 30293046, https://doi.org/10.1002/2016WR020133.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schuhen, N., T. T. Thorarinsdottir, and T. Gneiting, 2012: Ensemble model output statistics for wind vectors. Mon. Wea. Rev., 140, 32043219, https://doi.org/10.1175/MWR-D-12-00028.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schwedler, B. R. J., and M. E. Baldwin, 2011: Diagnosing the sensitivity of binary image measures to bias, location, and event frequency within a forecast verification framework. Wea. Forecasting, 26, 10321044, https://doi.org/10.1175/WAF-D-11-00032.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Seo, D.-J., A. Seed, and G. Delrieu, 2011: Radar and multisensor rainfall estimation for hydrologic applications. Rainfall: State of the Science, Geophys. Monogr., Vol. 191, Amer. Geophys. Union, 79–104.

    • Crossref
    • Export Citation
  • Sklar, A., 1973: Random variables, joint distribution functions, and copulas. Kybernetika, 9 (6), 449460.

  • Sonia, C.-G., L.-A. Emilio, and E.-M. M. Dolores, 2012: Selection of a plotting position for a normal Q-Q plot. R script. J. Commun. Comput., 9, 243250.

    • Search Google Scholar
    • Export Citation
  • Venugopal, V., S. Basu, and E. Foufoula-Georgiou, 2005: A new metric for comparing precipitation patterns with an application to ensemble forecasts. J. Geophys. Res., 110, D08111, https://doi.org/10.1029/2004JD005395.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Voisin, N., J. C. Schaake, and D. P. Lettenmeier, 2010: Calibration and downscaling methods for quantitative ensemble precipitation forecasts. Wea. Forecasting, 25, 16031627, https://doi.org/10.1175/2010WAF2222367.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vrac, M., and P. Friederichs, 2015: Multivariate—intervariable, spatial, and temporal—bias correction. J. Climate, 28, 218237, https://doi.org/10.1175/JCLI-D-14-00059.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2015: Multivariate ensemble Model Output Statistics using empirical copulas. Quart. J. Roy. Meteor. Soc., 141, 945952, https://doi.org/10.1002/qj.2414.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wu, L., D.-J. Seo, J. Demargne, S. Cong, J. D. Brown, and J. Schaake, 2011: Generation of ensemble precipitation forecast from single-valued quantitative precipitation forecast for hydrologic ensemble prediction. J. Hydrol., 399, 281298, https://doi.org/10.1016/j.jhydrol.2011.01.013.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yang, X., S. Sharma, R. Siddique, S. J. Greybush, and A. Mejia, 2017: Postprocessing of GEFS precipitation ensemble reforecasts over the U.S. Mid-Atlantic region. Mon. Wea. Rev., 145, 16411658, https://doi.org/10.1175/MWR-D-16-0251.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zepeda-Arce, J., E. Foufoula-Georgiou, and K. K. Droegemeier, 2000: Space-time rainfall organization and its role in validating quantitative precipitation forecasts. J. Geophys. Res., 105, 10 12910 146, https://doi.org/10.1029/1999JD901087.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, Y., S. Reed, and D. Kitzmiller, 2011: Effects of retrospective gauge-based readjustment of multisensor precipitation estimates on hydrologic simulations. J. Hydrometeor., 12, 429443, https://doi.org/10.1175/2010JHM1200.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 715 205 15
PDF Downloads 367 89 4