• Abramowitz, G., 2005: Towards a benchmark for land surface models. Geophys. Res. Lett., 32, L22702, doi:10.1029/2005GL024419.

  • Abramowitz, G., 2012: Towards a public, standardized, diagnostic benchmarking system for land surface models. Geosci. Model Dev., 5, 819827, doi:10.5194/gmd-5-819-2012.

    • Search Google Scholar
    • Export Citation
  • Abramowitz, G., , Pouyanné L. , , and Ajami H. , 2012: On the information content of surface meteorology for downward atmospheric long-wave radiation synthesis. Geophys. Res. Lett., 39, L04808, doi:10.1029/2011GL050726.

    • Search Google Scholar
    • Export Citation
  • Allen, R. G., , Pereira L. S. , , Raes D. , , and Smith M. , 1998: FAO Penman–Monteith equation. Crop evapotranspiration: Guidelines for computing crop water requirements, FAO Irrigation and Drainage Paper 56, 17–28. [Available online at www.fao.org/docrep/X0490E/X0490E00.htm.]

  • Balsamo, G., , Viterbo P. , , Beljaars A. , , van den Hurk B. J. J. M. , , Betts A. , , and Scipal K. , 2009: A revised hydrology for the ECMWF model: Verification from field site to terrestrial water storage and impact in the Integrated Forecast System. J. Hydrometeor., 10, 623643, doi:10.1175/2008JHM1068.1.

    • Search Google Scholar
    • Export Citation
  • Best, M. J., , and Grimmond C. S. B. , 2013: Analysis of the seasonal cycle within the first international urban land surface model comparison. Bound.-Layer Meteor., 146, 421446, doi:10.1007/s10546-012-9769-7.

    • Search Google Scholar
    • Export Citation
  • Best, M. J., , and Grimmond C. S. B. , 2014: Importance of initial state and atmospheric conditions for urban land surface models’ performance. Urban Climate, 10, 387–406, doi:10.1016/j.uclim.2013.10.006.

    • Search Google Scholar
    • Export Citation
  • Best, M. J., and et al. , 2011: The Joint UK Land Environment Simulator (JULES), Model description—Part 1: Energy and water fluxes. Geosci. Model Dev., 4, 677699, doi:10.5194/gmd-4-677-2011.

    • Search Google Scholar
    • Export Citation
  • Boone, A., , Calvet J.-C. , , and Noilhan J. , 1999: Inclusion of a third soil layer in a land-surface scheme using the force–restore method. J. Appl. Meteor., 38, 16111630, doi:10.1175/1520-0450(1999)038<1611:IOATSL>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Boone, A., and et al. , 2009: The AMMA Land Surface Model Intercomparison Project (ALMIP). Bull. Amer. Meteor. Soc., 90, 18651880, doi:10.1175/2009BAMS2786.1.

    • Search Google Scholar
    • Export Citation
  • Boussetta, S., and et al. , 2013: Natural land carbon dioxide exchanges in the ECMWF integrated forecasting system: Implementation and offline validation. J. Geophys. Res., 118, 59235946, doi:10.1002/jgrd.50488.

    • Search Google Scholar
    • Export Citation
  • Bowling, L. C., and et al. , 2003: Simulation of high latitude hydrological processes in the Torne–Kalix basin: PILPS Phase 2(e): 1: Experiment description and summary intercomparisons. Global Planet. Change, 38, 130, doi:10.1016/S0921-8181(03)00003-1.

    • Search Google Scholar
    • Export Citation
  • Chen, T. H., and et al. , 1997: Cabauw experimental results from the Project for Intercomparison of Land-Surface Parameterization Schemes. J. Climate, 10, 11941215, doi:10.1175/1520-0442(1997)010<1194:CERFTP>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Decharme, B., , Boone A. , , Delire C. , , and Noilhan J. , 2011: Local evaluation of the Interaction between Soil Biosphere Atmosphere soil multilayer diffusion scheme using four pedotransfer functions. J. Geophys. Res., 116, D20126, doi:10.1029/2011JD016002.

    • Search Google Scholar
    • Export Citation
  • Dirmeyer, P. A., 2011: A history and review of the Global Soil Wetness Project (GSWP). J. Hydrometeor., 12, 729749, doi:10.1175/JHM-D-10-05010.1.

    • Search Google Scholar
    • Export Citation
  • Dirmeyer, P. A., , and Zeng F. J. , 1999: An update to the distribution and treatment of vegetation and soil properties in SSiB. COLA Tech. Rep. 78, Center for Ocean–Land–Atmosphere Studies, Calverton, MD, 27 pp.

  • Dirmeyer, P. A., , Dolman A. J. , , and Sato N. , 1999: The pilot phase of the Global Soil Wetness Project. Bull. Amer. Meteor. Soc., 80, 851878, doi:10.1175/1520-0477(1999)080<0851:TPPOTG>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Ek, M. B., , Mitchell K. E. , , Lin Y. , , Rogers E. , , Grummann P. , , Koren V. , , Gayno G. , , and Tarpley J. D. , 2003: Implementation of Noah land surface model advances in the National Centers for Environmental Prediction operational mesoscale Eta model. J. Geophys. Res., 108, 8851, doi:10.1029/2002JD003296.

    • Search Google Scholar
    • Export Citation
  • Gong, W., , Gupta H. V. , , Yang D. , , Sricharan K. , , and Hero A. O. III, 2013: Estimating epistemic and aleatory uncertainties during hydrologic modeling: An information theoretic approach. Water Resour. Res., 49, 22532273, doi:10.1002/wrcr.20161.

    • Search Google Scholar
    • Export Citation
  • Grimmond, C. S. B., and et al. , 2010: The international urban energy balance models comparison project: First results from phase 1. J. Appl. Meteor. Climatol., 49, 12681292, doi:10.1175/2010JAMC2354.1.

    • Search Google Scholar
    • Export Citation
  • Grimmond, C. S. B., and et al. , 2011: Initial results from phase 2 of the international urban energy balance model comparison. Int. J. Climatol., 31, 244272, doi:10.1002/joc.2227.

    • Search Google Scholar
    • Export Citation
  • Guo, Z., , and Dirmeyer P. A. , 2013: Interannual variability of land–atmosphere coupling strength. J. Hydrometeor., 14, 16361646, doi:10.1175/JHM-D-12-0171.1.

    • Search Google Scholar
    • Export Citation
  • Guo, Z., and et al. , 2006: GLACE: The Global Land–Atmosphere Coupling Experiment. Part II: Analysis. J. Hydrometeor., 7, 611625, doi:10.1175/JHM511.1.

    • Search Google Scholar
    • Export Citation
  • Haverd, V., , and Cuntz M. , 2010: Soil–Litter–Iso: A one-dimensional model for coupled transport of heat, water and stable isotopes in soil with a litter layer and root extraction. J. Hydrol., 388, 438455, doi:10.1016/j.jhydrol.2010.05.029.

    • Search Google Scholar
    • Export Citation
  • Henderson-Sellers, A., , Yang Z.-L. , , and Dickinson R. E. , 1993: The Project for Intercomparison of Land-Surface Schemes. Bull. Amer. Meteor. Soc., 74, 1335–1349, doi:10.1175/1520-0477(1993)074<1335:TPFIOL>2.0.CO;2.

  • Henderson-Sellers, A., , Pitman A. J. , , Henderson-Sellers B. , , and Verner J. , 1995a: Applying software engineering metrics to land surface parameterization schemes. J. Climate, 8, 10431059, doi:10.1175/1520-0442(1995)008<1043:ASEMTL>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Henderson-Sellers, A., , Pitman A. J. , , Love P. K. , , Irannejad P. , , and Chen T. , 1995b: The Project for Intercomparison of Land Surface Parameterization Schemes (PILPS): Phases 2 and 3. Bull. Amer. Meteor. Soc., 76, 489503, doi:10.1175/1520-0477(1995)076<0489:TPFIOL>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hirschi, M., and et al. , 2011: Observational evidence for soil-moisture impact on hot extremes in southeastern Europe. Nat. Geosci., 4, 1721, doi:10.1038/ngeo1032.

    • Search Google Scholar
    • Export Citation
  • Kidston, J., , Brümmer C. , , Black T. A. , , Morgenstern K. , , Nesic Z. , , McCaughey J. H. , , and Barr A. G. , 2010: Energy balance closure using eddy covariance above two different land surfaces and implications for CO2 flux measurements. Bound.-Layer Meteor., 136, 193218, doi:10.1007/s10546-010-9507-y.

    • Search Google Scholar
    • Export Citation
  • Koster, R. D., , and Suarez M. J. , 1992: Modeling the land surface boundary in climate models as a composite of independent vegetation stands. J. Geophys. Res., 97, 26972715, doi:10.1029/91JD01696.

    • Search Google Scholar
    • Export Citation
  • Koster, R. D., , and Suarez M. J. , 1994: The components of a ‘SVAT’ scheme and their effects on a GCM’s hydrological cycle. Adv. Water Resour., 17, 6178, doi:10.1016/0309-1708(94)90024-8.

    • Search Google Scholar
    • Export Citation
  • Koster, R. D., , and Mahanama S. P. P. , 2012: Land surface controls on hydroclimatic means and variability. J. Hydrometeor., 13, 16041620, doi:10.1175/JHM-D-12-050.1.

    • Search Google Scholar
    • Export Citation
  • Koster, R. D., and et al. , 2004: Regions of coupling between soil moisture and precipitation. Science, 305, 11381140, doi:10.1126/science.1100217.

    • Search Google Scholar
    • Export Citation
  • Koster, R. D., and et al. , 2006: GLACE: The Global Land–Atmosphere Coupling Experiment. Part I: Overview. J. Hydrometeor., 7, 590610, doi:10.1175/JHM510.1.

    • Search Google Scholar
    • Export Citation
  • Kowalczyk, E. A., , Wang Y. P. , , Law R. M. , , Davies H. L. , , McGregor J. L. , , and Abramowitz G. S. , 2006: The CSIRO Atmosphere Biosphere Land Exchange (CABLE) model for use in climate models and as an offline model. CSIRO Marine and Atmospheric Research Paper 013, 43 pp. [Available online at www.cawcr.gov.au/projects/access/cable/cable_technical_description.pdf.]

  • Krinner, G., and et al. , 2005: A dynamic vegetation model for studies of the coupled atmosphere–biosphere system. Global Biogeochem. Cycles, 19, GB1015, doi:10.1029/2003GB002199.

    • Search Google Scholar
    • Export Citation
  • Luo, Y. Q., and et al. , 2012: A framework for benchmarking land models. Biogeosciences, 9, 38573874, doi:10.5194/bg-9-3857-2012.

  • Manabe, S., 1969: Climate and the ocean circulation: I. The atmospheric circulation and the hydrology of the earth’s surface. Mon. Wea. Rev., 97, 739805, doi:10.1175/1520-0493(1969)097<0739:CATOC>2.3.CO;2.

    • Search Google Scholar
    • Export Citation
  • Masson, V., and et al. , 2013: The SURFEXv7.2 externalized platform for the simulation of Earth surface variables and fluxes. Geosci. Model Dev., 6, 929960, doi:10.5194/gmd-6-929-2013.

    • Search Google Scholar
    • Export Citation
  • Monteith, J. L., , and Unsworth M. H. , 1990: Principals of Environmental Physics. 2nd ed. Edward Arnold, 241 pp.

  • Perkins, S. E., , Pitman A. J. , , Holbrook N. J. , , and McAneney J. , 2007: Evaluation of the AR4 climate models’ simulated daily maximum temperature, minimum temperature and precipitation over Australia using probability density functions. J. Climate, 20, 43564376, doi:10.1175/JCLI4253.1.

    • Search Google Scholar
    • Export Citation
  • Pitman, A. J., 2003: The evolution of, and revolution in, land surface schemes designed for climate models. Int. J. Climatol., 23, 479510, doi:10.1002/joc.893.

    • Search Google Scholar
    • Export Citation
  • Pitman, A. J., and et al. , 1999: Key results and implications from phase 1(c) of the Project for Intercomparison of Land-Surface Parameterization Schemes. Climate Dyn., 15, 673684, doi:10.1007/s003820050309.

    • Search Google Scholar
    • Export Citation
  • Rodell, M., , Houser P. R. , , Berg A. A. , , and Famiglietti J. S. , 2005: Evaluation of 10 methods for initializing a land surface model. J. Hydrometeor., 6, 146155, doi:10.1175/JHM414.1.

    • Search Google Scholar
    • Export Citation
  • Schlosser, C. A., and et al. , 2000: Simulations of a boreal grassland hydrology at Valdai, Russia: PILPS phase 2(d). Mon. Wea. Rev., 128, 301321, doi:10.1175/1520-0493(2000)128<0301:SOABGH>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Seneviratne, S. I., , Corti T. , , Davin E. L. , , Hirschi M. , , Jaeger E. B. , , Lehner I. , , Orlowsky B. , , and Teuling A. J. , 2010: Investigating soil moisture–climate interactions in a changing climate: A review. Earth Sci. Rev., 99, 125161, doi:10.1016/j.earscirev.2010.02.004.

    • Search Google Scholar
    • Export Citation
  • Seneviratne, S. I., and et al. , 2013: Impact of soil moisture–climate feedbacks on CMIP5 projections: First results from the GLACE-CMIP5 experiment. Geophys. Res. Lett., 40, 52125217, doi:10.1002/grl.50956.

    • Search Google Scholar
    • Export Citation
  • van den Hurk, B., , Best M. , , Dirmeyer P. , , Pitman A. , , Polcher J. , , and Santanello J. , 2011: Acceleration of land surface model development over a decade of glass. Bull. Amer. Meteor. Soc.,92, 1593–1600, doi:10.1175/BAMS-D-11-00007.1.

  • Wang, Y. P., , Kowalczyk E. , , Leuning R. , , Abramowitz G. , , Raupach M. R. , , Pak B. , , van Gorsel E. , , and Luhar A. , 2011: Diagnosing errors in a land surface model (CABLE) in the time and frequency domains. J. Geophys. Res., 116, G01034, doi:10.1029/2010JG001385.

    • Search Google Scholar
    • Export Citation
  • Wilson, K., and et al. , 2002: Energy balance closure at FLUXNET sites. Agric. For. Meteor., 113, 223243, doi:10.1016/S0168-1923(02)00109-0.

    • Search Google Scholar
    • Export Citation
  • Wood, E. F., and et al. , 1998: The Project for Intercomparison of Land-Surface Parameterization Schemes (PILPS) phase 2(c) Red–Arkansas River basin experiment: 1. Experiment description and summary intercomparison. Global Planet. Change, 19, 115135, doi:10.1016/S0921-8181(98)00044-7.

    • Search Google Scholar
    • Export Citation
  • Yang, Z.-L., , Dickinson R. E. , , Henderson-Sellers A. , , and Pitman A. J. , 1995: Preliminary study of spin-up processes in land surface models with the first stage data of Project for Intercomparison of Land Surface Parameterization Schemes phase 1(a). J. Geophys. Res., 100, 16 55316 578, doi:10.1029/95JD01076.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 591 591 153
PDF Downloads 390 390 92

The Plumbing of Land Surface Models: Benchmarking Model Performance

View More View Less
  • 1 Met Office, Exeter, United Kingdom
  • | 2 ARC Centre of Excellence for Climate System Science, University of New South Wales, Sydney, New South Wales, Australia
  • | 3 ECMWF, Reading, United Kingdom
  • | 4 CNRM-GAME, Météo-France, Toulouse, France
  • | 5 Helmholtz Centre for Environmental Research–UFZ, Leipzig, Germany
  • | 6 Center for Ocean–Land–Atmosphere Studies, George Mason University, Fairfax, Virginia
  • | 7 NOAA/NCEP/EMC, College Park, Maryland
  • | 8 Oceans and Atmosphere Flagship, CSIRO, Canberra, Australian Capital Territory, Australia
  • | 9 KNMI, De Bilt, Netherlands
  • | 10 Hydrological Sciences Laboratory, NASA GSFC, Greenbelt, Maryland
  • | 11 Oceans and Atmosphere Flagship, CSIRO, Aspendale, Victoria, Australia
  • | 12 Laboratoire des Sciences du Climat et de l'Environnement, UMR 8212, IPSL-LSCE, CEA-CNRS-UVSQ, Gif-sur-Yvette, France
© Get Permissions
Restricted access

Abstract

The Protocol for the Analysis of Land Surface Models (PALS) Land Surface Model Benchmarking Evaluation Project (PLUMBER) was designed to be a land surface model (LSM) benchmarking intercomparison. Unlike the traditional methods of LSM evaluation or comparison, benchmarking uses a fundamentally different approach in that it sets expectations of performance in a range of metrics a priori—before model simulations are performed. This can lead to very different conclusions about LSM performance. For this study, both simple physically based models and empirical relationships were used as the benchmarks. Simulations were performed with 13 LSMs using atmospheric forcing for 20 sites, and then model performance relative to these benchmarks was examined. Results show that even for commonly used statistical metrics, the LSMs’ performance varies considerably when compared to the different benchmarks. All models outperform the simple physically based benchmarks, but for sensible heat flux the LSMs are themselves outperformed by an out-of-sample linear regression against downward shortwave radiation. While moisture information is clearly central to latent heat flux prediction, the LSMs are still outperformed by a three-variable nonlinear regression that uses instantaneous atmospheric humidity and temperature in addition to downward shortwave radiation. These results highlight the limitations of the prevailing paradigm of LSM evaluation that simply compares an LSM to observations and to other LSMs without a mechanism to objectively quantify the expectations of performance. The authors conclude that their results challenge the conceptual view of energy partitioning at the land surface.

Corresponding author address: Martin Best, Met Office, Fitzroy Road, Exeter, Devon EX1 3PB, United Kingdom. E-mail: martin.best@metoffice.gov.uk

Abstract

The Protocol for the Analysis of Land Surface Models (PALS) Land Surface Model Benchmarking Evaluation Project (PLUMBER) was designed to be a land surface model (LSM) benchmarking intercomparison. Unlike the traditional methods of LSM evaluation or comparison, benchmarking uses a fundamentally different approach in that it sets expectations of performance in a range of metrics a priori—before model simulations are performed. This can lead to very different conclusions about LSM performance. For this study, both simple physically based models and empirical relationships were used as the benchmarks. Simulations were performed with 13 LSMs using atmospheric forcing for 20 sites, and then model performance relative to these benchmarks was examined. Results show that even for commonly used statistical metrics, the LSMs’ performance varies considerably when compared to the different benchmarks. All models outperform the simple physically based benchmarks, but for sensible heat flux the LSMs are themselves outperformed by an out-of-sample linear regression against downward shortwave radiation. While moisture information is clearly central to latent heat flux prediction, the LSMs are still outperformed by a three-variable nonlinear regression that uses instantaneous atmospheric humidity and temperature in addition to downward shortwave radiation. These results highlight the limitations of the prevailing paradigm of LSM evaluation that simply compares an LSM to observations and to other LSMs without a mechanism to objectively quantify the expectations of performance. The authors conclude that their results challenge the conceptual view of energy partitioning at the land surface.

Corresponding author address: Martin Best, Met Office, Fitzroy Road, Exeter, Devon EX1 3PB, United Kingdom. E-mail: martin.best@metoffice.gov.uk
Save