• Abramowitz, G., 2012: Towards a public, standardized, diagnostic benchmarking system for land surface models. Geosci. Model Dev., 5, 819827, doi:10.5194/gmd-5-819-2012.

    • Search Google Scholar
    • Export Citation
  • Abramowitz, G., Leuning R. , Clark M. , and Pitman A. , 2008: Evaluating the performance of land surface models. J. Climate, 21, 54685481, doi:10.1175/2008JCLI2378.1.

    • Search Google Scholar
    • Export Citation
  • Best, M. J., and Coauthors, 2011: The Joint UK Land Environment Simulator (JULES), model description—Part 1: Energy and water flux. Geosci. Model Dev., 4, 677699, doi:10.5194/gmd-4-677-2011.

    • Search Google Scholar
    • Export Citation
  • Best, M. J., and Coauthors, 2015: The plumbing of land surface models: Benchmarking model performance. J. Hydrometeor., 16, 14251442, doi:10.1175/JHM-D-14-0158.1.

    • Search Google Scholar
    • Export Citation
  • Blyth, E., Gash J. , Lloyd A. , Pryor M. , Weedon G. , and Shuttleworth J. , 2010: Evaluating the JULES land surface model energy fluxes using FLUXNET data. J. Hydrometeor., 11, 509519, doi:10.1175/2009JHM1183.1.

    • Search Google Scholar
    • Export Citation
  • Blyth, E., Clark D. B. , Ellis R. , Huntingford C. , Los S. , Pryor M. , Best M. , and Sitch S. , 2011: A comprehensive set of benchmark tests for a land surface model of simultaneous fluxes of water and carbon at both the global and seasonal scale. Geosci. Model Dev., 4, 255269, doi:10.5194/gmd-4-255-2011.

    • Search Google Scholar
    • Export Citation
  • Bowling, L., and Coauthors, 2003: Simulation of high-latitude hydrological processes in the Torne–Kalix basin: PILPS phase 2(e): 1: Experiment description and summary intercomparisons. Global Planet. Change, 38, 130, doi:10.1016/S0921-8181(03)00003-1.

    • Search Google Scholar
    • Export Citation
  • Clapp, R. B., and Hornberger G. M. , 1978: Empirical equations for some soil hydraulic properties. Water Resour. Res., 14, 601604, doi:10.1029/WR014i004p00601.

    • Search Google Scholar
    • Export Citation
  • Clark, D. B., and Coauthors, 2011: The Joint UK Land Environment Simulator (JULES), model description—Part 2: Carbon fluxes and vegetation. Geosci. Model Dev., 4, 701722, doi:10.5194/gmd-4-701-2011.

    • Search Google Scholar
    • Export Citation
  • Clark, M. P., Slater A. G. , Rupp D. E. , Woods R. A. , Vrugt J. A. , Gupta H. V. , Wagener T. , and Hay L. E. , 2008: Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resour. Res., 44, W00B02, doi:10.1029/2007WR006735.

    • Search Google Scholar
    • Export Citation
  • Clark, M. P., Kavetski D. , and Fenicia F. , 2011: Pursuing the method of multiple working hypotheses for hydrological modeling. Water Resour. Res., 47, W09301, doi:10.1029/2010WR009827.

    • Search Google Scholar
    • Export Citation
  • Cosby, B. J., Hornberger G. M. , Clapp R. B. , and Ginn T. R. , 1984: A statistical exploration of the relationships of soil-moisture characteristics to the physical properties of soils. Water Resour. Res., 20, 682690, doi:10.1029/WR020i006p00682.

    • Search Google Scholar
    • Export Citation
  • Deardorff, J., 1978: Efficient prediction of ground surface temperature and moisture, with inclusion of a layer of vegetation. J. Geophys. Res., 83, 18891903, doi:10.1029/JC083iC04p01889.

    • Search Google Scholar
    • Export Citation
  • Decker, M., Brunke M. A. , Wang Z. , Sakaguchi K. , Zeng X. , and Bosilovich M. G. , 2012: Evaluation of the reanalysis products from GSFC, NCEP, and ECMWF using flux tower observations. J. Climate, 25, 19161944, doi:10.1175/JCLI-D-11-00004.1.

    • Search Google Scholar
    • Export Citation
  • Dee, D. P., and Coauthors, 2011: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553597, doi:10.1002/qj.828.

    • Search Google Scholar
    • Export Citation
  • Déqué, M., and Coauthors, 2007: An intercomparison of regional climate simulations for Europe: Assessing uncertainties in model projections. Climatic Change, 81, 5370, doi:10.1007/s10584-006-9228-x.

    • Search Google Scholar
    • Export Citation
  • Dorigo, W. A., Scipal K. , Parinussa R. M. , Liu Y. Y. , Wagner W. , de Jeu R. A. M. , and Naeimi V. , 2010: Error characterisation of global active and passive microwave soil moisture datasets. Hydrol. Earth Syst. Sci., 14, 26052616, doi:10.5194/hess-14-2605-2010.

    • Search Google Scholar
    • Export Citation
  • Essery, R., 2013: Large-scale simulations of snow albedo masking by forests. Geophys. Res. Lett., 40, 55215525, doi:10.1002/grl.51008.

    • Search Google Scholar
    • Export Citation
  • Essery, R., Pomeroy J. , Parviainen J. , and Storck P. , 2003: Sublimation of snow from coniferous forests in a climate model. J. Climate, 16, 18551864, doi:10.1175/1520-0442(2003)016<1855:SOSFCF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Essery, R., and Coauthors, 2009: An evaluation of forest snow process simulations. Bull. Amer. Meteor. Soc., 90, 11201135, doi:10.1175/2009BAMS2629.1.

    • Search Google Scholar
    • Export Citation
  • Essery, R., Morin S. , Lejeune Y. , and Ménard C. B. , 2013: A comparison of 1701 snow models using observations from an alpine site. Adv. Water Resour., 55, 131148, doi:10.1016/j.advwatres.2012.07.013.

    • Search Google Scholar
    • Export Citation
  • Etchevers, P., and Coauthors, 2004: Validation of the energy budget of an alpine snowpack simulated by several snow models (SnowMIP project). Ann. Glaciol., 38, 150158, doi:10.3189/172756404781814825.

    • Search Google Scholar
    • Export Citation
  • European Environment Agency, 2006: CORINE land cover, CLC2006-maankytt/maanpeite (25m) -paikkatietoaineisto (rasteri). Accessed 20 May 2014. [Available online at http://wwwd3.ymparisto.fi/d3/Static_rs/spesific/corinelandcover.html.]

  • Flato, G., and Coauthors, 2013: Evaluation of climate models. Climate Change 2013: The Physical Science Basis, T. F. Stocker et al., Eds., Cambridge University Press, 741–866.

  • Gleckler, P. J., Taylor K. E. , and Doutriaux C. , 2008: Performance metrics for climate models. J. Geophys. Res., 113, D06104, doi:10.1029/2007JD008972.

    • Search Google Scholar
    • Export Citation
  • Hancock, S., Huntley B. , Ellis R. , and Baxter B. , 2014: Biases in reanalysis snowfall found by comparing the JULES land surface model to GlobSnow. J. Climate, 27, 624632, doi:10.1175/JCLI-D-13-00382.1.

    • Search Google Scholar
    • Export Citation
  • Houldcroft, C. J., Grey W. M. , Barnsley M. , Taylor C. M. , Los S. O. , and North P. R. J. , 2009: New vegetation albedo parameters and global fields of soil background albedo derived from MODIS for use in a climate model. J. Hydrometeor., 10, 183198, doi:10.1175/2008JHM1021.1.

    • Search Google Scholar
    • Export Citation
  • JCHMR, 2013: Joint UK Land Environment Simulator User Guide. Accessed 2 May 2014. [Available online at http://www.jchmr.org/jules/documentation/user_guide/vn3.4.1/.]

  • Jolliff, J. K., Kindle J. C. , Shulman I. , Penta B. , Friedrichs M. A. M. , Helber R. , and Arnone R. A. , 2009: Summary diagrams for coupled hydrodynamic-ecosystem model skill assessment. J. Mar. Syst., 76, 6482, doi:10.1016/j.jmarsys.2008.05.014.

    • Search Google Scholar
    • Export Citation
  • Lawrence, D., and Slater A. , 2010: The contribution of snow condition trends to future ground climate. Climate Dyn., 34, 969981, doi:10.1007/s00382-009-0537-4.

    • Search Google Scholar
    • Export Citation
  • Loew, A., and Schlenz F. , 2011: A dynamic approach for evaluating coarse scale satellite soil moisture products. Hydrol. Earth Syst. Sci., 15, 7590, doi:10.5194/hess-15-75-2011.

    • Search Google Scholar
    • Export Citation
  • Luo, Y. Q., and Coauthors, 2012: A framework for benchmarking land models. Biogeosciences, 9, 3857–3874, doi:10.5194/bg-9-3857-2012.

  • Manninen, T., Korhonen L. , Voipio P. , Lahtinen P. , and Stenberg P. , 2012: Airborne estimation of boreal forest LAI in winter conditions: A test using summer and winter ground truth. IEEE Trans. Geosci. Remote Sens., 50, 6874, doi:10.1109/TGRS.2011.2173939.

    • Search Google Scholar
    • Export Citation
  • McGuire, A., and Coauthors, 2001: Carbon balance of the terrestrial biosphere in the twentieth century: Analyses of CO2, climate and land use effects with four process-based ecosystem models. Global Biogeochem. Cycles, 15, 183206, doi:10.1029/2000GB001298.

    • Search Google Scholar
    • Export Citation
  • Moriasi, D. N., Arnold J. G. , Van Liew M. W. , Bingner R. L. , Harmel R. D. , and Veith T. L. , 2007: Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Trans. ASABE, 50, 885900, doi:10.13031/2013.23153.

    • Search Google Scholar
    • Export Citation
  • Pitman, A. J., 2003: The evolution of, and revolution in, land surface schemes designed for climate models. Int. J. Climatol., 23, 479510, doi:10.1002/joc.893.

    • Search Google Scholar
    • Export Citation
  • Pomeroy, J. W., Gray D. M. , Brown T. , Hedstrom N. R. , Quinton W. L. , Granger R. J. , and Carey S. K. , 2007: The cold regions hydrological process representation and model: A platform for basing model structure on physical evidence. Hydrol. Processes, 21, 26502667, doi:10.1002/hyp.6787.

    • Search Google Scholar
    • Export Citation
  • Räisänen, J., 2008: Warmer climate: Less or more snow? Climate Dyn., 30, 307319, doi:10.1007/s00382-007-0289-y.

  • Reid, T., Essery R. , Rutter N. , and King M. , 2013: Data-driven modelling of shortwave radiation transfer to snow through boreal birch and conifer canopies. Hydrol. Processes, 28, 2987–3007, doi:10.1002/hyp.9849.

    • Search Google Scholar
    • Export Citation
  • Renard, B., Kavetski D. , Kuczera G. , Thyer M. , and Franks S. W. , 2010: Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors. Water Resour. Res., 46, W05521, doi:10.1029/2009WR008328.

    • Search Google Scholar
    • Export Citation
  • Saha, S., and Coauthors, 2010: The NCEP Climate Forecast System Reanalysis. Bull. Amer. Meteor. Soc., 91, 10151057, doi:10.1175/2010BAMS3001.1.

    • Search Google Scholar
    • Export Citation
  • Saha, S., and Coauthors, 2014: The NCEP Climate Forecast System version 2. J. Climate, 27, 21852208, doi:10.1175/JCLI-D-12-00823.1.

  • Sheng, J., and Zwiers F. , 1998: An improved scheme for time-dependent boundary conditions in atmospheric general circulation models. Climate Dyn., 14, 609613, doi:10.1007/s003820050244.

    • Search Google Scholar
    • Export Citation
  • Slater, A., and Coauthors, 2001: The representation of snow in land surface schemes: Results from PILPS2(d). J. Hydrometeor., 2, 725, doi:10.1175/1525-7541(2001)002<0007:TROSIL>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Slevin, D., Tett S. , and Williams M. , 2015: Multi-site evaluation of the JULES land surface model using global and local data. Geosci. Model Dev., 8, 295316, doi:10.5194/gmd-8-295-2015.

    • Search Google Scholar
    • Export Citation
  • Takala, M., Luojus K. , Pulliainen J. , Derksen C. , Lemmetyinen J. , Kärnä J.-P. , Koskinen J. , and Bojkov B. , 2011: Estimating Northern Hemisphere snow water equivalent for climate research through assimilation of space-borne radiometer data and ground-based measurements. Remote Sens. Environ., 115, 35173529, doi:10.1016/j.rse.2011.08.014.

    • Search Google Scholar
    • Export Citation
  • Taylor, K., 2001: Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res., 106, 71837192, doi:10.1029/2000JD900719.

    • Search Google Scholar
    • Export Citation
  • Vincent, L. A., Zhang X. , Bonsal B. R. , and Hogg W. D. , 2002: Homogenization of daily temperatures over Canada. J. Climate, 15, 13221334, doi:10.1175/1520-0442(2002)015<1322:HODTOC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Weedon, G. P., Balsamo G. , Bellouin N. , Gomes S. , Best M. J. , and Viterbo P. , 2014: The WFDEI meteorological forcing data set: WATCH Forcing Data methodology applied to ERA-Interim reanalysis data. Water Resour. Res., 50, 75057514, doi:10.1002/2014WR015638.

    • Search Google Scholar
    • Export Citation
  • Williams, M., and Coauthors, 2009: Improving land surface models with FLUXNET data. Biogeosciences, 6, 13411359, doi:10.5194/bg-6-1341-2009.

    • Search Google Scholar
    • Export Citation
  • Wood, E., and Coauthors, 1998: The Project for Intercomparison of Land-Surface Parameterization Schemes (PILPS) Phase 2(c) Red–Arkansas River basin experiment: 1. Experiment description and summary intercomparisons. Global Planet. Change, 19, 115135, doi:10.1016/S0921-8181(98)00044-7.

    • Search Google Scholar
    • Export Citation
  • Wunderlich, T., 1972: Heat and mass transfer between a water surface and the atmosphere. Water Resources Research Laboratory Rep. 0-6803 14, Tennessee Valley Authority, 270 pp.

  • Yang, J., Jiang L. , Ménard C. B. , Luojus K. , Lemmetyinen J. , and Pulliainen J. , 2015: Evaluation of snow products over the Tibetan Plateau. Hydrol. Processes, 29, 32473260, doi:10.1002/hyp.10427.

    • Search Google Scholar
    • Export Citation
  • Zhao, M., Running S. W. , and Nemani R. R. , 2006: Sensitivity of Moderate Resolution Imaging Spectroradiometer (MODIS) terrestrial primary production to the accuracy of meteorological reanalyses. J. Geophys. Res., 111, G01002, doi:10.1029/2004JG000004.

    • Search Google Scholar
    • Export Citation
  • Zhao, Y., and Coauthors, 2012: How errors on meteorological variables impact simulated ecosystem fluxes: A case study for six French sites. Biogeosciences, 9, 25372564, doi:10.5194/bg-9-2537-2012.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 196 88 5
PDF Downloads 148 55 3

Effects of Meteorological and Ancillary Data, Temporal Averaging, and Evaluation Methods on Model Performance and Uncertainty in a Land Surface Model

View More View Less
  • 1 Arctic Research Centre, Finnish Meteorological Institute, Helsinki, Finland
  • | 2 Research and Development, Finnish Meteorological Institute, Helsinki, Finland
  • | 3 Arctic Research Centre, Finnish Meteorological Institute, Helsinki, Finland
Restricted access

Abstract

A single-model 16-member ensemble is used to investigate how external model factors can affect model performance. Ensemble members are constructed with the land surface model (LSM) Joint UK Land Environment Simulator (JULES), with different choices of meteorological forcing [in situ, NCEP Climate Forecast System Reanalysis (CFSR)/CFSv2, or Water and Global Change (WATCH) Forcing Data ERA-Interim (WFDEI)] and ancillary datasets (in situ or remotely sensed), and with four time step modes. Effects of temporal averaging are investigated by comparing the hourly, daily, monthly, and seasonal ensemble performance against snow depth and water equivalent, soil temperature and moisture, and latent and sensible heat fluxes from one forest site and one clearing in the boreal ecozone of Finnish Lapland. Results show that meteorological data are the largest source of uncertainty; differences in ancillary data have little effect on model results. Although generally informative and representative, aggregated performance metrics fail to identify “right results for the wrong reasons”; to do so, scrutinizing of time series and of interactions between variables is necessary. Temporal averaging over longer intervals improves metrics—with the notable exception of bias, which increases—by reducing the effects of internal data and model variability on model response. Model evaluation during shoulder seasons (fall minus spring) identifies weaknesses in the reanalyses datasets that conventional seasonal performance (winter minus summer) neglects. In view of the importance of snow on the range of results obtained with the same model, let alone identical simulations using different temporal averaging, it is recommended that systematic evaluation, quantification of errors, and uncertainties in snow-covered regions be incorporated in future efforts to standardize evaluation methods of LSMs.

Corresponding author address: Cécile Ménard, Finnish Meteorological Institute, Erik Palménin aukio 1, Helsinki 00530, Finland. E-mail: cecile.menard@ed-alumni.net

Abstract

A single-model 16-member ensemble is used to investigate how external model factors can affect model performance. Ensemble members are constructed with the land surface model (LSM) Joint UK Land Environment Simulator (JULES), with different choices of meteorological forcing [in situ, NCEP Climate Forecast System Reanalysis (CFSR)/CFSv2, or Water and Global Change (WATCH) Forcing Data ERA-Interim (WFDEI)] and ancillary datasets (in situ or remotely sensed), and with four time step modes. Effects of temporal averaging are investigated by comparing the hourly, daily, monthly, and seasonal ensemble performance against snow depth and water equivalent, soil temperature and moisture, and latent and sensible heat fluxes from one forest site and one clearing in the boreal ecozone of Finnish Lapland. Results show that meteorological data are the largest source of uncertainty; differences in ancillary data have little effect on model results. Although generally informative and representative, aggregated performance metrics fail to identify “right results for the wrong reasons”; to do so, scrutinizing of time series and of interactions between variables is necessary. Temporal averaging over longer intervals improves metrics—with the notable exception of bias, which increases—by reducing the effects of internal data and model variability on model response. Model evaluation during shoulder seasons (fall minus spring) identifies weaknesses in the reanalyses datasets that conventional seasonal performance (winter minus summer) neglects. In view of the importance of snow on the range of results obtained with the same model, let alone identical simulations using different temporal averaging, it is recommended that systematic evaluation, quantification of errors, and uncertainties in snow-covered regions be incorporated in future efforts to standardize evaluation methods of LSMs.

Corresponding author address: Cécile Ménard, Finnish Meteorological Institute, Erik Palménin aukio 1, Helsinki 00530, Finland. E-mail: cecile.menard@ed-alumni.net
Save