• Aksoy, A., 2015: Parameter estimation. Encyclopedia of Atmospheric Sciences, 2nd ed. G. R. North, J. Pyle, and F. Zhang, Eds., Academic Press, 181–186, doi:10.1016/B978-0-12-382225-3.00494-1.

    • Crossref
    • Export Citation
  • Bellprat, O., S. Kotlarski, D. Lüthi, and C. Schär, 2012: Objective calibration of regional climate models. J. Geophys. Res., 117, D23115, doi:10.1029/2012JD018262.

    • Search Google Scholar
    • Export Citation
  • Bennett, A. F., B. S. Chua, and L. M. Leslie, 1996: Generalized inversion of a global numerical weather prediction model. Meteor. Atmos. Phys., 60, 165178, doi:10.1007/BF01029793.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, F., and J. Dudhia, 2001: Coupling an advanced land surface–hydrology model with the Penn State–NCAR MM5 modeling system. Part I: Model implementation and sensitivity. Mon. Wea. Rev., 129, 569585, doi:10.1175/1520-0493(2001)129<0569:CAALSH>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Di, Z., and et al. , 2015: Assessing WRF model parameter sensitivity: A case study with 5-day summer precipitation forecasting in the greater Beijing area. Geophys. Res. Lett., 42, 579587, doi:10.1002/2014GL061623.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Duan, Q., and et al. , 2006: Model Parameter Estimation Experiment (MOPEX): An overview of science strategy and major results of the second and third workshops. J. Hydrol., 320, 317, doi:10.1016/j.jhydrol.2005.07.031.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Duane, G. S., and J. P. Hacker, 2007: Automatic parameter estimation in a mesoscale model without ensembles. Nonlinear Time Series Analysis in Geophysics, R. V. Donner and S. M. Barbosa, Eds., Lecture Notes in Earth Sciences, Vol. 112, Springer, 81–95.

    • Crossref
    • Export Citation
  • Dudhia, J., 1989: Numerical study of convection observed during the winter monsoon experiment using a mesoscale two-dimensional model. J. Atmos. Sci., 46, 30773107, doi:10.1175/1520-0469(1989)046<3077:NSOCOD>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dudhia, J., D. Gill, K. Manning, W. Wang, and C. Bruyere, 2005: PSU/NCAR Mesoscale Modeling System tutorial class notes and user’s guide: MM5 Modeling System version 3. NCAR Rep., 390 pp.

  • Evensen, G., D. P. Dee, and J. Schröter, 1998: Parameter estimation in dynamical models. Ocean Modeling and Parameterization, E.O. Chassignet and J. Verron, Eds., NATO Science Series, Vol. 516, Springer, 373–398.

    • Crossref
    • Export Citation
  • Gan, Y., and et al. , 2014: A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model. Environ. Modell. Software, 51, 269285, doi:10.1016/j.envsoft.2013.09.031.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gong, J., G. Wahba, D. R. Johnson, and J. Tribbia, 1998: Adaptive tuning of numerical weather prediction models: Simultaneous estimation of weighting, smoothing, and physical parameters. Mon. Wea. Rev., 126, 210231, doi:10.1175/1520-0493(1998)126<0210:ATONWP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gong, W., Q. Duan, J. Li, C. Wang, Z. Di, Y. Dai, A. Ye, and C. Miao, 2015: Multi-objective parameter optimization of common land model using adaptive surrogate modelling. Hydrol. Earth Syst. Sci., 19, 24092425, doi:10.5194/hess-19-2409-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gong, W., Q. Duan, J. Li, and Y. Dai, 2016: Multiobjective adaptive surrogate modeling-based optimization for parameter estimation of large, complex geophysical models. Water Resour. Res., 52, 19842008, doi:10.1002/2015WR018230.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guo, Z., and et al. , 2014: A sensitivity analysis of cloud properties to CLUBB parameters in the single-column Community Atmosphere Model (SCAM5). J. Adv. Model. Earth Syst., 6, 829858, doi:10.1002/2014MS000315.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hong, S. Y., and J. J. Lim, 2006: The WRF single moment 6 class microphysics scheme (WSM6). J. Korean Meteor. Soc., 42, 129151, doi:10.1155/2010/707253.

    • Search Google Scholar
    • Export Citation
  • Hong, S. Y., Y. Noh, and J. Dudhia, 2006: A new vertical diffusion package with an explicit treatment of entrainment processes. Mon. Wea. Rev., 134, 23182341, doi:10.1175/MWR3199.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ihshaish, H., A. Cortes, and M. A. Senar, 2012: Parallel multi-level genetic ensemble for numerical weather prediction enhancement. Proc. Comput. Sci., 9, 276285, doi:10.1016/j.procs.2012.04.029.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jackson, C., M. Sen, and P. Stoffa, 2004: An efficient stochastic Bayesian approach to optimal parameter and uncertainty estimation for climate model predictions. J. Climate, 17, 28282841, doi:10.1175/1520-0442(2004)017<2828:AESBAT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Johnson, J. S., Z. Cui, L. A. Lee, J. P. Gosling, A. M. Blyth, and K. S. Carslaw, 2015: Evaluating uncertainty in convective cloud microphysics using statistical emulation. J. Adv. Model. Earth Syst., 7, 162187, doi:10.1002/2014MS000383.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J. S., 2004: The Kain–Fritsch convective parameterization: An update. J. Appl. Meteor., 43, 170181, doi:10.1175/1520-0450(2004)043<0170:TKCPAU>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, 368 pp.

    • Crossref
    • Export Citation
  • Kistler, R., and et al. , 2001: The NCEP–NCAR 50-Year Reanalysis: Monthly means CD-ROM and documentation. Bull. Amer. Meteor. Soc., 82, 247267, doi:10.1175/1520-0477(2001)082<0247:TNNYRM>2.3.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krishnamurti, T. N., C. M. Kishtawal, T. LaRow, D. Bachiochi, Z. Zhang, C. E. Williford, S. Gadgil, and S. Surendran, 1999: Improved skill of weather and seasonal climate forecasts from multimodel superensemble. Science, 285, 15481550, doi:10.1126/science.285.5433.1548.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, J., and et al. , 2013: Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis. Hydrol. Earth Syst. Sci., 17, 32793293, doi:10.5194/hess-17-3279-2013.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Y., H. V. Gupta, S. Sorooshian, L. A. Bastidas, and W. J. Shuttleworth, 2004: Exploring parameter sensitivities of the land surface using a locally coupled land-atmosphere model. J. Geophys. Res., 109, D21101, doi:10.1029/2004JD004730.

    • Search Google Scholar
    • Export Citation
  • Miettinen, K., 1999: Nonlinear Multiobjective Optimization. International Series in Operations Research & Management Science, Vol. 12, Springer, 298 pp., doi:10.1007/978-1-4615-5563-6.

    • Crossref
    • Export Citation
  • Mlawer, E. J., S. J. Taubman, P. D. Brown, M. J. Iacono, and S. A. Clough, 1997: Radiative transfer for inhomogeneous atmospheres: RRTM, a validated correlated-k model for the longwave. J. Geophys. Res., 102, 16 66316 682, doi:10.1029/97JD00237.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mu, M., W. Duan, and J. Wang, 2002: The predictability problems in numerical weather and climate prediction. Adv. Atmos. Sci., 19, 191204, doi:10.1007/s00376-002-0016-x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Neelin, J. D., A. Bracco, H. Luo, J. C. McWilliams, and J. E. Meyerson, 2010: Considerations for parameter optimization and sensitivity in climate models. Proc. Natl. Acad. Sci. USA, 107, 21 34921 354, doi:10.1073/pnas.1015473107.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Quan, J., and et al. , 2016: An evaluation of parametric sensitivities of different meteorological variables simulated by the WRF model. Quart. J. Roy. Meteor. Soc., 142, 29252934, doi:10.1002/qj.2885.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ruiz, J., M. Pulido, and T. Miyoshi, 2013: Estimating model parameters with ensemble-based data assimilation: A review. J. Meteor. Soc. Japan, 91, 7999, doi:10.2151/jmsj.2013-201.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Santitissadeekorn, N., and C. Jones, 2015: Two-stage filtering for joint state-parameter estimation. Mon. Wea. Rev., 143, 20282042, doi:10.1175/MWR-D-14-00176.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Severijns, C. A., and W. Hazeleger, 2005: Optimizing parameters in an atmospheric general circulation model. J. Climate, 18, 35273535, doi:10.1175/JCLI3430.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stensrud, D. J., 2007: Parameterization Schemes: Keys to Understanding Numerical Weather Prediction Models. Cambridge University Press, 480 pp.

    • Crossref
    • Export Citation
  • Vrugt, J. A., C. G. H. Diks, H. V. Gupta, W. Bouten, and J. M. Verstraten, 2005: Improved treatment of uncertainty in hydrologic modeling: Combining the strengths of global optimization and data assimilation. Water Resour. Res., 41, W01017, doi:10.1029/2004WR003059.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, C., Q. Duan, W. Gong, A. Ye, Z. Di, and C. Miao, 2014: An evaluation of adaptive surrogate modeling based optimization with two benchmark problems. Environ. Modell. Software, 60, 167179, doi:10.1016/j.envsoft.2014.05.026.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, C., Q. Duan, C. H. Tong, and W. Gong, 2016: A GUI platform for uncertainty quantification of complex dynamical models. Environ. Modell. Software, 76, 112, doi:10.1016/j.envsoft.2015.11.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wernli, H., M. Paulat, M. Hagen, and C. Frei, 2008: SAL—A novel quality measure for the verification of quantitative precipitation forecasts. Mon. Wea. Rev., 136, 44704487, doi:10.1175/2008MWR2415.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yu, X., S. K. Park, and Y. H. Lee, 2013: Parameter estimation using an evolutionary algorithm for QPF in a tropical cyclone. Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications, Vol. II, S. K. Park and X. Liang, Eds., Springer, 707–715, doi:10.1007/978-3-642-35088-7_27.

    • Crossref
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 229 229 28
PDF Downloads 329 329 24

Automatic Model Calibration: A New Way to Improve Numerical Weather Forecasting

View More View Less
  • 1 State Key Laboratory of Earth Surface Processes and Resource Ecology, Faculty of Geographical Science, Beijing Normal University, Beijing, China
  • | 2 State Key Laboratory of Severe Weather, Chinese Academy of Meteorological Sciences, Beijing, China
  • | 3 State Key Laboratory of Earth Surface Processes and Resource Ecology, Faculty of Geographical Science, Beijing Normal University, Beijing, China
  • | 4 Institute of Urban Meteorology, China Meteorological Administration, Beijing, China
  • | 5 State Key Laboratory of Severe Weather, Chinese Academy of Meteorological Sciences, Beijing, China
  • | 6 Institute of Urban Meteorology, China Meteorological Administration, Beijing, China
© Get Permissions
Restricted access

Abstract

Weather forecasting skill has been improved over recent years owing to advances in the representation of physical processes by numerical weather prediction (NWP) models, observational systems, data assimilation and postprocessing, new computational capability, and effective communications and training. There is an area that has received less attention so far but can bring significant improvement to weather forecasting—the calibration of NWP models, a process in which model parameters are tuned using certain mathematical methods to minimize the difference between predictions and observations. Model calibration of the NWP models is difficult because 1) there are a formidable number of model parameters and meteorological variables to tune, and 2) a typical NWP model is very expensive to run, and conventional model calibration methods require many model runs (up to tens of thousands) or cannot handle the high dimensionality of NWP models. This study demonstrates that a newly developed automatic model calibration platform can overcome these difficulties and improve weather forecasting through parameter optimization. We illustrate how this is done with a case study involving 5-day weather forecasting during the summer monsoon in the greater Beijing region using the Weather Research and Forecasting Model. The keys to automatic model calibration are to use global sensitivity analysis to screen out the most important parameters influencing model performance and to employ surrogate models to reduce the need for a large number of model runs. Through several optimization and validation studies, we have shown that automatic model calibration can improve precipitation and temperature forecasting significantly according to a number of performance measures.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

CORRESPONDING AUTHOR: Qingyun Duan, qyduan@bnu.edu.cn

A supplement to this article is available online (10.1175/BAMS-D-15-00104.2)

Abstract

Weather forecasting skill has been improved over recent years owing to advances in the representation of physical processes by numerical weather prediction (NWP) models, observational systems, data assimilation and postprocessing, new computational capability, and effective communications and training. There is an area that has received less attention so far but can bring significant improvement to weather forecasting—the calibration of NWP models, a process in which model parameters are tuned using certain mathematical methods to minimize the difference between predictions and observations. Model calibration of the NWP models is difficult because 1) there are a formidable number of model parameters and meteorological variables to tune, and 2) a typical NWP model is very expensive to run, and conventional model calibration methods require many model runs (up to tens of thousands) or cannot handle the high dimensionality of NWP models. This study demonstrates that a newly developed automatic model calibration platform can overcome these difficulties and improve weather forecasting through parameter optimization. We illustrate how this is done with a case study involving 5-day weather forecasting during the summer monsoon in the greater Beijing region using the Weather Research and Forecasting Model. The keys to automatic model calibration are to use global sensitivity analysis to screen out the most important parameters influencing model performance and to employ surrogate models to reduce the need for a large number of model runs. Through several optimization and validation studies, we have shown that automatic model calibration can improve precipitation and temperature forecasting significantly according to a number of performance measures.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

CORRESPONDING AUTHOR: Qingyun Duan, qyduan@bnu.edu.cn

A supplement to this article is available online (10.1175/BAMS-D-15-00104.2)

Supplementary Materials

    • Supplemental Materials (PDF 2.52 MB)
Save