Evaluating the Performance of Land Surface Models

Gab Abramowitz Climate Change Research Centre, University of New South Wales, Sydney, New South Wales, Australia

Search for other papers by Gab Abramowitz in
Current site
Google Scholar
PubMed
Close
,
Ray Leuning CSIRO Marine and Atmospheric Research, Canberra, Australia

Search for other papers by Ray Leuning in
Current site
Google Scholar
PubMed
Close
,
Martyn Clark National Institute for Water and Atmospheric Research, Christchurch, Australia

Search for other papers by Martyn Clark in
Current site
Google Scholar
PubMed
Close
, and
Andy Pitman Climate Change Research Centre, University of New South Wales, Sydney, New South Wales, Australia

Search for other papers by Andy Pitman in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

This paper presents a set of analytical tools to evaluate the performance of three land surface models (LSMs) that are used in global climate models (GCMs). Predictions of the fluxes of sensible heat, latent heat, and net CO2 exchange obtained using process-based LSMs are benchmarked against two statistical models that only use incoming solar radiation, air temperature, and specific humidity as inputs to predict the fluxes. Both are then compared to measured fluxes at several flux stations located on three continents. Parameter sets used for the LSMs include default values used in GCMs for the plant functional type and soil type surrounding each flux station, locally calibrated values, and ensemble sets encompassing combinations of parameters within their respective uncertainty ranges. Performance of the LSMs is found to be generally inferior to that of the statistical models across a wide variety of performance metrics, suggesting that the LSMs underutilize the meteorological information used in their inputs and that model complexity may be hindering accurate prediction. The authors show that model evaluation is purpose specific; good performance in one metric does not guarantee good performance in others. Self-organizing maps are used to divide meteorological “‘forcing space” into distinct regions as a mechanism to identify the conditions under which model bias is greatest. These new techniques will aid modelers to identify the areas of model structure responsible for poor performance.

Corresponding author address: Gab Abramowitz, Climate Change Research Centre, University of New South Wales, Sydney, NSW 2052, Australia. Email: gabsun@gmail.com

Abstract

This paper presents a set of analytical tools to evaluate the performance of three land surface models (LSMs) that are used in global climate models (GCMs). Predictions of the fluxes of sensible heat, latent heat, and net CO2 exchange obtained using process-based LSMs are benchmarked against two statistical models that only use incoming solar radiation, air temperature, and specific humidity as inputs to predict the fluxes. Both are then compared to measured fluxes at several flux stations located on three continents. Parameter sets used for the LSMs include default values used in GCMs for the plant functional type and soil type surrounding each flux station, locally calibrated values, and ensemble sets encompassing combinations of parameters within their respective uncertainty ranges. Performance of the LSMs is found to be generally inferior to that of the statistical models across a wide variety of performance metrics, suggesting that the LSMs underutilize the meteorological information used in their inputs and that model complexity may be hindering accurate prediction. The authors show that model evaluation is purpose specific; good performance in one metric does not guarantee good performance in others. Self-organizing maps are used to divide meteorological “‘forcing space” into distinct regions as a mechanism to identify the conditions under which model bias is greatest. These new techniques will aid modelers to identify the areas of model structure responsible for poor performance.

Corresponding author address: Gab Abramowitz, Climate Change Research Centre, University of New South Wales, Sydney, NSW 2052, Australia. Email: gabsun@gmail.com

Save
  • Abramowitz, G., 2005: Towards a benchmark for land surface models. Geophys. Res. Lett., 32 .L22702, doi:10.1029/2005GL024419.

  • Baldocchi, D., and Coauthors, 2001: FLUXNET: A new tool to study the temporal and spatial variability of ecosystem-scale carbon dioxide, water vapor, and energy flux densities. Bull. Amer. Meteor. Soc., 82 , 24152434.

    • Search Google Scholar
    • Export Citation
  • Bernhofer, C., and Coauthors, 2003: Spruce forests (Norway and Sitka Spruce, including Douglas Fir): Carbon and water fluxes, balances, ecological and ecophysiological determinants. Fluxes of Carbon, Water and Energy of European Forests, R. Valentini, Ed., Springer, 99–123.

    • Search Google Scholar
    • Export Citation
  • Beven, K., and J. Freer, 2001: Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the glue methodology. J. Hydrol., 249 , 1129.

    • Search Google Scholar
    • Export Citation
  • Brutsaert, W., 1975: On a derivable formula for long-wave radiation from clear skies. Water Resour. Res., 11 , 742744.

  • Chen, T. H., and Coauthors, 1997: Cabauw experimental results from the Project for Intercomparison of Land-Surface Parameterization Schemes (PILPS). J. Climate, 10 , 11941215.

    • Search Google Scholar
    • Export Citation
  • Dai, Y., and Coauthors, 2003: The Common Land Model. Bull. Amer. Meteor. Soc., 84 , 10131023.

  • Deardorff, J. W., 1978: Efficient prediction of ground surface temperature and moisture, with inclusion of a layer of vegetation. J. Geophys. Res., 83 , C4. 18891903.

    • Search Google Scholar
    • Export Citation
  • Gerstberger, P., T. Foken, and K. Kalbitz, 2004: The Lehstenbach and Steinkreuz catchments in NE Bavaria, Germany. Ecol. Stud., 172 , 1544.

    • Search Google Scholar
    • Export Citation
  • Ginzburg, L. R., and C. X. J. Jensen, 2004: Rules of thumb for judging ecological theories. Trends Ecol. Evol., 19 , 121126.

  • Haverd, V., M. Cuntz, R. Leuning, and H. Keith, 2007: Air and biomass heat storage fluxes in a forest canopy: Calculation within a soil vegetation atmosphere transfer model. Agric. For. Meteor., 147 , 3–4. 125139.

    • Search Google Scholar
    • Export Citation
  • Henderson-Sellers, A., K. McGuffie, and A. J. Pitman, 1996: The Project for Intercomparison of Land-surface Parameterization Schemes (PILPS): 1992 to 1995. Climate Dyn., 12 , 849859.

    • Search Google Scholar
    • Export Citation
  • Hsu, K-l, H. V. Gupta, X. Gao, S. Sorooshian, and B. Imam, 2002: Self-organizing linear output map (SOLO): An artificial neural network suitable for hydrological modeling and analysis. Water Resour. Res., 38 .1302, doi:10.1029/2001WR000795.

    • Search Google Scholar
    • Export Citation
  • Kohonen, T., 1989: Self-Organization and Associative Memory. Springer-Verlag, 312 pp.

  • Koster, R. D., and Coauthors, 2004: Regions of strong coupling between soil moisture and precipitation. Science, 305 , 11381140.

  • Kowalczyk, E. A., Y. P. Wang, R. M. Law, H. L. Davies, J. L. McGregor, and G. Abramowitz, 2006: The CSIRO Atmosphere Biosphere Land Exchange (CABLE) model for use in climate models and as an offline model. CSIRO Marine and Atmospheric Research Paper 013, 42 pp. [Available online at http://www.cmar.csiro.au/e-print/open/kowalczykea_2006a.pdf.].

  • Krinner, G., and Coauthors, 2005: A dynamic global vegetation model for studies of the coupled atmosphere-biosphere system. Global Biogeochem. Cycles, 19 .GB1015, doi:10.1029/2003GB002199.

    • Search Google Scholar
    • Export Citation
  • Law, B. E., M. G. Ryan, and P. M. Anthoni, 1999: Seasonal and annual respiration of a ponderosa pine ecosystem. Global Change Biol., 5 , 169182.

    • Search Google Scholar
    • Export Citation
  • Leuning, R., H. A. Cleugh, S. J. Zegelin, and D. Hughes, 2005: Carbon and water fluxes over a temperate Eucalyptus forest and a tropical wet/dry savanna in Australia: Measurements and comparison with MODIS remote sensing estimates. Agric. For. Meteor., 129 , 151173.

    • Search Google Scholar
    • Export Citation
  • Levis, S., G. Bonan, M. Vertenstein, and K. Oleson, 2004: The community land model’s dynamic global vegetation model (CLM-DGVM): Technical description and user’s guide. NCAR Tech. Rep. TN-459+IA, 50 pp.

  • Medlyn, B. E., A. P. Robinson, R. Clement, and R. E. McMurtrie, 2005: On the validation of models of forest CO2 exchange using eddy covariance data: Some perils and pitfalls. Tree Physiol., 25 , 839857.

    • Search Google Scholar
    • Export Citation
  • Meyers, T., 2001: A comparison of summertime water and CO2 fluxes over rangeland for well watered and drought conditions. Agric. For. Meteor., 106 , 205214.

    • Search Google Scholar
    • Export Citation
  • Meyers, T., and S. E. Hollinger, 2004: An assessment of storage terms in the surface energy balance of maize and soybean. Agric. For. Meteor., 125 , 105115.

    • Search Google Scholar
    • Export Citation
  • Oleson, K., and Coauthors, 2004: Technical description of the community land model (CLM). NCAR Tech. Rep. TN-461+STR, 183 pp.

  • Potter, C. S., J. T. Randerson, C. B. Field, P. A. Matson, P. M. Vitousek, H. A. Mooney, and S. A. Klooster, 1993: Terrestrial ecosystem production: A process model based on global satellite and surface data. Global Biogeochem. Cycles, 7 , 811841.

    • Search Google Scholar
    • Export Citation
  • Sellers, P. J., and J. L. Dorman, 1987: Testing the Simple Biosphere model (SiB) using point micrometeorological and biophysical data. J. Climate Appl. Meteor., 26 , 622651.

    • Search Google Scholar
    • Export Citation
  • Swinbank, W. C., 1963: Longwave radiation from clear skies. Quart. J. Roy. Meteor. Soc., 89 , 339348.

  • van Gorsel, E., R. Leuning, H. A. Cleugh, H. Keith, M. U. F. Kirschbaum, and T. Suni, 2008: Application of an alternative method to derive reliable estimates of nighttime respiration from eddy covariance measurements in moderately complex topography. Agric. For. Meteor., 148 , 11741180.

    • Search Google Scholar
    • Export Citation
  • Wagener, T., N. McIntyre, M. J. Lees, H. S. Wheater, and H. V. Gupta, 2003: Towards reduced uncertainty in conceptual rainfall-runoff modelling: Dynamic identifiability analysis. Hydrol. Processes, 17 , 455476.

    • Search Google Scholar
    • Export Citation
  • Wang, Y-P., R. Leuning, H. Cleugh, and P. Coppin, 2001: Parameter estimation in surface exchange models using nonlinear inversion: How many parameters can we estimate and which measurements are most useful? Global Change Biol., 7 , 495510.

    • Search Google Scholar
    • Export Citation
  • Wilson, K. B., and Coauthors, 2002: Energy balance closure at FLUXNET sites. Agric. For. Meteor., 113 , 223243.

  • Zobler, L., 1988: A world soil file for global climate modeling. NASA Goddard Institute for Space Studies Tech. Rep. 87802, 32 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1650 593 76
PDF Downloads 1060 241 15