Evaluation of ECMWF Precipitation Predictions in China during 2015–18

Cui Liu aKey Laboratory of Cloud-Precipitation Physics and Severe Storms (LACS), Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, China
bUniversity of the Chinese Academy of Sciences, Beijing, China

Search for other papers by Cui Liu in
Current site
Google Scholar
PubMed
Close
,
Jianhua Sun aKey Laboratory of Cloud-Precipitation Physics and Severe Storms (LACS), Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, China
bUniversity of the Chinese Academy of Sciences, Beijing, China
cSouthern Marine Science and Engineering Guangdong Laboratory (Zhuhai), Zhuhai, China

Search for other papers by Jianhua Sun in
Current site
Google Scholar
PubMed
Close
,
Xinlin Yang dKey Laboratory of Regional Climate-Environment for Temperate East Asia, Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, China

Search for other papers by Xinlin Yang in
Current site
Google Scholar
PubMed
Close
,
Shuanglong Jin eState Key Laboratory of Operation and Control of Renewable Energy and Storage Systems, China Electric Power Research Institute, Beijing, China
fElectric Power Meteorology State Grid Corporation of China Joint Laboratory, Beijing, China

Search for other papers by Shuanglong Jin in
Current site
Google Scholar
PubMed
Close
, and
Shenming Fu gInternational Center for Climate and Environment Sciences, Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, China

Search for other papers by Shenming Fu in
Current site
Google Scholar
PubMed
Close
Restricted access

We are aware of a technical issue preventing figures and tables from showing in some newly published articles in the full-text HTML view.
While we are resolving the problem, please use the online PDF version of these articles to view figures and tables.

Abstract

Precipitation forecasts from the ECMWF model from March to September during 2015–18 were evaluated using observed precipitation at 2411 stations from the China Meteorological Administration. To eliminate the influence of varying climatology in different regions in China, the stable equitable error in probability space method was used to obtain criteria for 3- and 6-h accumulated precipitation at each station and classified precipitation into light, medium, and heavy precipitation. The model was evaluated for these categories using categorical and continuous methods. The threat score and the equitable threat score showed that the model’s forecasts of rainfall were generally more accurate at shorter lead times, and the best performance occurred in the middle and lower reaches of the Yangtze River basin. The miss ratio for heavy precipitation was higher in the northern region than in the southern region, while heavy precipitation false alarms were more frequent in southwestern China. Overall, the miss ratio and false alarm ratio for heavy precipitation were highest in northern China and western China, respectively. For light and medium precipitation, the model performed best in the middle and lower reaches of the Yangtze River basin. The model predicted too much light and medium precipitation, but too little heavy precipitation. Heavy precipitation was generally underestimated over all of China, especially in the western region of China, South China, and the Yungui Plateau. Heavy precipitation was systematically underestimated because of the resolution and the related parameterization of convection.

Significance Statement

Quantitative precipitation forecast is an important reference for operational weather forecasting. Verification of model-forecast precipitation in China is complicated because of its complex climatology. To reveal the spatiotemporal performance of the ECMWF model for 3- and 6-h precipitation forecasts in different regions of China, we defined thresholds for different rainfall categories from the cumulative precipitation at each station and evaluated the model after eliminating the influence of climatology. These verification results will help numerical model developers to improve their models and will also help forecasters have a better understanding of model predictions. Future research will focus on the accuracy of the model’s predictions under different circulation situations.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Jianhua Sun, sjh@mail.iap.ac.cn

Abstract

Precipitation forecasts from the ECMWF model from March to September during 2015–18 were evaluated using observed precipitation at 2411 stations from the China Meteorological Administration. To eliminate the influence of varying climatology in different regions in China, the stable equitable error in probability space method was used to obtain criteria for 3- and 6-h accumulated precipitation at each station and classified precipitation into light, medium, and heavy precipitation. The model was evaluated for these categories using categorical and continuous methods. The threat score and the equitable threat score showed that the model’s forecasts of rainfall were generally more accurate at shorter lead times, and the best performance occurred in the middle and lower reaches of the Yangtze River basin. The miss ratio for heavy precipitation was higher in the northern region than in the southern region, while heavy precipitation false alarms were more frequent in southwestern China. Overall, the miss ratio and false alarm ratio for heavy precipitation were highest in northern China and western China, respectively. For light and medium precipitation, the model performed best in the middle and lower reaches of the Yangtze River basin. The model predicted too much light and medium precipitation, but too little heavy precipitation. Heavy precipitation was generally underestimated over all of China, especially in the western region of China, South China, and the Yungui Plateau. Heavy precipitation was systematically underestimated because of the resolution and the related parameterization of convection.

Significance Statement

Quantitative precipitation forecast is an important reference for operational weather forecasting. Verification of model-forecast precipitation in China is complicated because of its complex climatology. To reveal the spatiotemporal performance of the ECMWF model for 3- and 6-h precipitation forecasts in different regions of China, we defined thresholds for different rainfall categories from the cumulative precipitation at each station and evaluated the model after eliminating the influence of climatology. These verification results will help numerical model developers to improve their models and will also help forecasters have a better understanding of model predictions. Future research will focus on the accuracy of the model’s predictions under different circulation situations.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Jianhua Sun, sjh@mail.iap.ac.cn
Save
  • Accadia, C., S. Mariani, M. Casaioli, M. A. Lavagnini, and A. Speranza, 2005: Verification of precipitation forecasts from two limited-area models over Italy and comparison with ECMWF forecasts using a resampling technique. Wea. Forecasting, 20, 276300, https://doi.org/10.1175/WAF854.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baldwin, M. E., and J. S. Kain, 2006: Sensitivity of several performance measures to displacement error, bias, and event frequency. Wea. Forecasting, 21, 636648, https://doi.org/10.1175/WAF933.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bao, X., and F. Zhang, 2013: Impacts of the mountain-plains solenoid and cold pool dynamics on the diurnal variation of precipitation over Northern China. Atmos. Chem. Phys., 13, 69656982, https://doi.org/10.5194/acp-13-6965-2013.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Benjamin, S. G., J. M. Brown, G. Brunet, P. Lynch, K. Saito, and T. W. Schlatter, 2019: 100 years of progress in forecasting and NWP applications. A Century of Progress in Atmospheric and Related Sciences: Celebrating the American Meteorological Society Centennial, Meteor. Monogr., No. 59, Amer. Meteor. Soc., https://doi.org/10.1175/AMSMONOGRAPHS-D-18-0020.1.

    • Crossref
    • Export Citation
  • Bi, B. G., K. Dai, Y. Wang, J. L. Fu, Y. Cao, and C. H. Liu, 2016: Advances in techniques of quantitative precipitation forecast (in Chinese). Yingyong Qixiang Xuebao, 27, 534549.

    • Search Google Scholar
    • Export Citation
  • Brill, K. F., and F. Mesinger, 2009: Applying a general analytic method for assessing bias sensitivity to bias-adjusted threat and equitable threat scores. Wea. Forecasting, 24, 17481754, https://doi.org/10.1175/2009WAF2222272.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cao, Y., L. N. Zhao, Y. F. Gong, D. B. Xu, and Y. J. Gao, 2019: Evaluation and error analysis of precipitation forecast capability of the ECMWF high-resolution mode (in Chinese). Torrential Rain Disasters, 38, 249258.

    • Search Google Scholar
    • Export Citation
  • Casati, B., G. Ross, and D. Stephenson, 2004: A new intensity-scale approach for the verification of spatial precipitation forecasts. Meteor. Appl., 11, 141154, https://doi.org/10.1017/S1350482704001239.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, M. X., Y. C. Wang, F. Gao, and X. Xiao, 2012: Diurnal variation in convective storm activity over contiguous North China during the warm-season based on radar mosaic climatology. J. Geophys. Res., 117, D20115, https://doi.org/10.1029/2012JD018158.

    • Search Google Scholar
    • Export Citation
  • Chien, F. C., Y. C. Liu, and B. J. D. Jou, 2006: MM5 ensemble mean forecasts in the Taiwan area for the 2003 mei-yu season. Wea. Forecasting, 21, 10061023, https://doi.org/10.1175/WAF960.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dai, K., Y. Cao, Q. F. Qian, S. Gao, S. G. Zhao, Y. Chen, and C. H. Qian, 2016: Situation and tendency of operational technologies in short- and medium-range weather forecast (in Chinese). Meteor. Monogr., 42, 14451455.

    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., R. Davies-Jones, and D. L. Keller, 1990: On summary measures of skill in rare event forecasting based on contingency tables. Wea. Forecasting, 5, 576585, https://doi.org/10.1175/1520-0434(1990)005<0576:OSMOSI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., U. Damrath, W. Wergen, and M. E. Baldwin, 2003: The WGNE assessment of short-term quantitative precipitation forecasts. Bull. Amer. Meteor. Soc., 84, 481492, https://doi.org/10.1175/BAMS-84-4-Ebert.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fu, J. L., Z. P. Zong, K. Dai, F. H. Zhang, and D. B. Gao, 2014: Application of a verification method on bias analysis of quantitative precipitation forecasts (in Chinese). Meteor. Monogr., 40, 796805.

    • Search Google Scholar
    • Export Citation
  • Golding, B. W., 2000: Quantitative precipitation forecasting in the UK. J. Hydrol., 239, 286305, https://doi.org/10.1016/S0022-1694(00)00354-1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Haiden, T. M., M. J. Rodwell, and D. S. Richardson, 2012: Intercomparison of global model precipitation forecast skill in 2010/11 using the SEEPS score. Mon. Wea. Rev., 140, 27202733, https://doi.org/10.1175/MWR-D-11-00301.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harris, D., E. Foufoulageorgiou, K. K. Droegemeier, and J. J. Levit, 2001: Multiscale statistical properties of a high-resolution precipitation forecast. J. Hydrometeor., 2, 406418, https://doi.org/10.1175/1525-7541(2001)002<0406:MSPOAH>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • He, H., and F. Zhang, 2010: Diurnal variations of warm-season precipitation over Northern China. Mon. Wea. Rev., 138, 10171025, https://doi.org/10.1175/2010MWR3356.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hong, J. S., 2003: Evaluation of the high-resolution model forecasts over the Taiwan area during GIMEX. Wea. Forecasting, 18, 836846, https://doi.org/10.1175/1520-0434(2003)018<0836:EOTHMF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huang, L., and Y. Luo, 2017: Evaluation of quantitative precipitation forecasts by TIGGE ensembles for south China during the presummer rainy season. J. Geophys. Res. Atmos., 122, 84948516, https://doi.org/10.1002/2017JD026512.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kniffka, A., and Coauthors, 2020: An evaluation of operational and research weather forecasts for southern West Africa using observations from the DACCIWA field campaign in June–July 2016. Quart. J. Roy. Meteor. Soc., 146, 11211148, https://doi.org/10.1002/qj.3729.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kober, K., G. C. Craig, C. Keil, and A. Dornbrack, 2012: Blending a probabilistic nowcasting method with a high-resolution numerical weather prediction ensemble for convective precipitation forecasts. Quart. J. Roy. Meteor. Soc., 138, 755768, https://doi.org/10.1002/qj.939.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, D. S., J. H. Sun, S. M. Fu, J. Wei, D. Wang, and F. Y. Tian, 2015: Spatiotemporal characteristics of hourly precipitation over central eastern China during the warm season of 1982–2012. Int. J. Climatol., 36, 31483160, https://doi.org/10.1002/joc.4543.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Y., J. Brown, J. Demargne, and D. Seo, 2011: A wavelet-based approach to assessing timing errors in hydrologic predictions. J. Hydrol., 397, 210224, https://doi.org/10.1016/j.jhydrol.2010.11.040.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lu, X. N., S. Z. Niu, C. F. Yuan, and X. C. Yuan, 2013: Verification and error analysis of quantitative precipitation estimation and forecast products in SWAN (in Chinese). Torrential Rain Disasters, 32, 142150.

    • Search Google Scholar
    • Export Citation
  • Luo, Y., and Coauthors, 2017: The Southern China Monsoon Rainfall Experiment (SCMREX). Bull. Amer. Meteor. Soc., 98, 9991013, https://doi.org/10.1175/BAMS-D-15-00235.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mao, M., J. H. Dai, B. P. Li, and X. Zhang, 2016: Object-based verification and evaluation for different types of severe convection forecasting products (in Chinese). Meteor. Monogr., 42, 389397.

    • Search Google Scholar
    • Export Citation
  • Moore, B. J., K. M. Mahoney, E. M. Sukovich, R. Cifelli, and T. M. Hamill, 2015: Climatology and environmental characteristics of extreme precipitation events in the southeastern United States. Mon. Wea. Rev., 143, 718741, https://doi.org/10.1175/MWR-D-14-00065.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mullen, S. L., and R. Buizza, 2001: Quantitative precipitation forecasts over the United States by the ECMWF ensemble prediction system. Mon. Wea. Rev., 129, 638663, https://doi.org/10.1175/1520-0493(2001)129<0638:QPFOTU>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • North, R., M. Trueman, M. Mittermaier, and M. J. Rodwell, 2013: An assessment of the SEEPS and SEDI metrics for the verification of 6 h forecast precipitation accumulations. Meteor. Appl., 20, 164175, https://doi.org/10.1002/met.1405.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pan, L. J., C. F. Xue, H. F. Zhang, J. P. Wang, and J. Yao, 2017a: Comparison of three verification methods for high-resolution grid precipitation forecast (in Chinese). Climatic Environ. Res., 22, 4558.

    • Search Google Scholar
    • Export Citation
  • Pan, L. J., C. F. Xue, H. F. Zhang, X. T. Chen, L. W. Qu, and Y. Yuan, 2017b: Evaluation of precipitation probability forecasts of ECMWF ensemble prediction system in central China (in Chinese). Plateau Meteor., 36, 138147.

    • Search Google Scholar
    • Export Citation
  • Piriou, J., J. Redelsperger, J. Geleyn, J. Lafore, and F. Guichard, 2007: An approach for convective parameterization with memory: Separating microphysics and transport in grid-scale equations. J. Atmos. Sci., 64, 41274139, https://doi.org/10.1175/2007JAS2144.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Qian, W., and X. Lin, 2005: Regional trends in recent precipitation indices in China. Meteor. Atmos. Phys., 90, 193207, https://doi.org/10.1007/s00703-004-0101-z.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roberts, N. M., and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 7897, https://doi.org/10.1175/2007MWR2123.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rodwell, M. J., D. S. Richardson, T. D. Hewson, and T. Haiden, 2010: A new equitable score suitable for verifying precipitation in numerical weather prediction. Quart. J. Roy. Meteor. Soc., 136, 13441363, https://doi.org/10.1002/qj.656.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rossa, A., P. Nurmi, and E. Ebert, 2008: Overview of methods for the verification of quantitative precipitation forecasts. Precipitation: Advances in Measurement, Estimation and Prediction, S. Michaelides, Ed., Springer, 419–452, https://doi.org/10.1007/978-3-540-77655-0_16.

    • Crossref
    • Export Citation
  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5, 570575, https://doi.org/10.1175/1520-0434(1990)005<0570:TCSIAA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stephenson, D. B., 2000: Use of the “odds ratio” for diagnosing forecast skill. Wea. Forecasting, 15, 221232, https://doi.org/10.1175/1520-0434(2000)015<0221:UOTORF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stoelinga, M. T., and Coauthors, 2003: Improvement of microphysical parameterization through observational verification experiment. Bull. Amer. Meteor. Soc., 84, 18071826, https://doi.org/10.1175/BAMS-84-12-1807.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Su, X., H. Yuan, Y. Zhu, Y. Luo, and Y. Wang, 2014: Evaluation of TIGGE ensemble predictions of Northern Hemisphere summer precipitation during 2008–2012. J. Geophys. Res. Atmos., 119, 72927310, https://doi.org/10.1002/2014JD021733.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, J. H., and F. Q. Zhang, 2012: Impacts of mountain–plains solenoid on diurnal variations of rainfalls along the mei-yu front over the East China plains. Mon. Wea. Rev., 140, 379397, https://doi.org/10.1175/MWR-D-11-00041.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tao, S. Y., 1980: Heavy Rainfalls in China (in Chinese). Science Press, 225 pp.

  • Tao, S. Y., and L. X. Chen, 1987: A review of recent research on the East Asian summer monsoon in China. Monsoon Meteorology, C. P. Chang and T. N. Krishnamurti, Ed., Oxford University Press, 60–92.

  • Tiziana, C., A. Ghelli, and F. Lalaurette, 2002: Verification of precipitation forecasts over the Alpine region using a high-density observing network. Wea. Forecasting, 17, 238249, https://doi.org/10.1175/1520-0434(2002)017<0238:VOPFOT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, B., and H. Lin, 2002: Rainy season of the Asian–Pacific summer monsoon. J. Climate, 15, 386398, https://doi.org/10.1175/1520-0442(2002)015<0386:RSOTAP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, N., and C. Lu, 2010: Two-dimensional continuous wavelet analysis and its application to meteorological data. J. Atmos. Oceanic Technol., 27, 652666, https://doi.org/10.1175/2009JTECHA1338.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, Y., and Z. H. Yan, 2007: Effect of different verification schemes on precipitation verification and assessment conclusion (in Chinese). Meteor. Monogr., 33, 5361.

    • Search Google Scholar
    • Export Citation
  • Xie, P., and P. A. Arkin, 1997: Global precipitation: A 17-year monthly analysis based on gauge observations, satellite estimates, and numerical model outputs. Bull. Amer. Meteor. Soc., 78, 25392558, https://doi.org/10.1175/1520-0477(1997)078<2539:GPAYMA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xiong, Q. F., 2011: Verification of GRAPES_meso precipitation forecasts based on fine-mesh and station datasets (in Chinese). Meteor. Monogr., 37, 185193.

    • Search Google Scholar
    • Export Citation
  • Yano, J., P. Bechtold, and J. Redelsperger, 2003: “Renormalization” approach for subgrid-scale representations. J. Atmos. Sci., 60, 20292038, https://doi.org/10.1175/1520-0469(2003)060<2029:RAFSR>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yu, R., T. Zhou, A. Xiong, Y. Zhu, and J. Li, 2007: Diurnal variations of summer precipitation over contiguous China. Geophys. Res. Lett., 34, L01704, https://doi.org/10.1029/2006GL028129.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yu, R., J. Li, W. Yuan, and H. Chen, 2010: Changes in characteristics of late-summer precipitation over eastern China in the past 40 years revealed by hourly precipitation data. J. Climate, 23, 33903396, https://doi.org/10.1175/2010JCLI3454.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, Y. F., L. J. Pan, and X. Yang, 2014: Comparative analysis of precipitation forecasting capabilities of ECWMF and Japan high-resolution models (in Chinese). Meteor. Monogr., 40, 424432.

    • Search Google Scholar
    • Export Citation
  • Zhang, Y. F., L. J. Pan, S. Lu, and X. X. Ju, 2017: Performance analysis on deterministic precipitation forecasting in surrounding areas of Qinling Mountains by ECMWF ensemble prediction system (in Chinese). Climatic Environ. Res., 22, 551562.

    • Search Google Scholar
    • Export Citation
  • Zheng, Y. G., M. Xue, B. Li, J. Chen, and Z. Y. Tao, 2016: Spatial characteristics of extreme rainfall over China with hourly through 24-hour accumulation periods based on national-level hourly rain gauge data. Adv. Atmos. Sci., 33, 12181232, https://doi.org/10.1007/s00376-016-6128-5.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zong, Z. P., T. J. Chen, and Y. Guan, 2013: Analysis and forecast verification of two southwest vortex torrential rain events in Sichuan Basin in early autumn of 2012 (in Chinese). Meteor. Monogr., 39, 567576.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 332 0 0
Full Text Views 1753 1097 393
PDF Downloads 1137 371 12