Benchmarking NLDAS-2 Soil Moisture and Evapotranspiration to Separate Uncertainty Contributions

Grey S. Nearing Hydrological Sciences Laboratory, NASA GSFC, Greenbelt, Maryland, and Science Applications International Corporation, McLean, Virginia

Search for other papers by Grey S. Nearing in
Current site
Google Scholar
PubMed
Close
,
David M. Mocko Hydrological Sciences Laboratory, NASA GSFC, Greenbelt, Maryland, and Science Applications International Corporation, McLean, Virginia

Search for other papers by David M. Mocko in
Current site
Google Scholar
PubMed
Close
,
Christa D. Peters-Lidard Hydrological Sciences Laboratory, NASA GSFC, Greenbelt, Maryland

Search for other papers by Christa D. Peters-Lidard in
Current site
Google Scholar
PubMed
Close
,
Sujay V. Kumar Hydrological Sciences Laboratory, NASA GSFC, Greenbelt, Maryland, and Science Applications International Corporation, McLean, Virginia

Search for other papers by Sujay V. Kumar in
Current site
Google Scholar
PubMed
Close
, and
Youlong Xia NOAA/NCEP/Environmental Modeling Center, College Park, and I. M. Systems Group, Rockville, Maryland

Search for other papers by Youlong Xia in
Current site
Google Scholar
PubMed
Close
Restricted access

We are aware of a technical issue preventing figures and tables from showing in some newly published articles in the full-text HTML view.
While we are resolving the problem, please use the online PDF version of these articles to view figures and tables.

Abstract

Model benchmarking allows us to separate uncertainty in model predictions caused by model inputs from uncertainty due to model structural error. This method is extended with a “large sample” approach (using data from multiple field sites) to measure prediction uncertainty caused by errors in 1) forcing data, 2) model parameters, and 3) model structure, and use it to compare the efficiency of soil moisture state and evapotranspiration flux predictions made by the four land surface models in phase 2 of the North American Land Data Assimilation System (NLDAS-2). Parameters dominated uncertainty in soil moisture estimates and forcing data dominated uncertainty in evapotranspiration estimates; however, the models themselves used only a fraction of the information available to them. This means that there is significant potential to improve all three components of NLDAS-2. In particular, continued work toward refining the parameter maps and lookup tables, the forcing data measurement and processing, and also the land surface models themselves, has potential to result in improved estimates of surface mass and energy balances.

Corresponding author address: Grey S. Nearing, Hydrological Sciences Laboratory, NASA GSFC, 8800 Greenbelt Rd., Code 617, Bldg. 33, Rm. G205, Greenbelt, MD 20771. E-mail: grey.s.nearing@nasa.gov

Abstract

Model benchmarking allows us to separate uncertainty in model predictions caused by model inputs from uncertainty due to model structural error. This method is extended with a “large sample” approach (using data from multiple field sites) to measure prediction uncertainty caused by errors in 1) forcing data, 2) model parameters, and 3) model structure, and use it to compare the efficiency of soil moisture state and evapotranspiration flux predictions made by the four land surface models in phase 2 of the North American Land Data Assimilation System (NLDAS-2). Parameters dominated uncertainty in soil moisture estimates and forcing data dominated uncertainty in evapotranspiration estimates; however, the models themselves used only a fraction of the information available to them. This means that there is significant potential to improve all three components of NLDAS-2. In particular, continued work toward refining the parameter maps and lookup tables, the forcing data measurement and processing, and also the land surface models themselves, has potential to result in improved estimates of surface mass and energy balances.

Corresponding author address: Grey S. Nearing, Hydrological Sciences Laboratory, NASA GSFC, 8800 Greenbelt Rd., Code 617, Bldg. 33, Rm. G205, Greenbelt, MD 20771. E-mail: grey.s.nearing@nasa.gov
Save
  • Abramowitz, G., 2005: Towards a benchmark for land surface models. Geophys. Res. Lett., 32, L22702, doi:10.1029/2005GL024419.

  • Abramowitz, G., 2012: Towards a public, standardized, diagnostic benchmarking system for land surface models. Geosci. Model Dev., 5, 819827, doi:10.5194/gmd-5-819-2012.

    • Search Google Scholar
    • Export Citation
  • Abramowitz, G., Leuning R. , Clark M. , and Pitman A. , 2008: Evaluating the performance of land surface models. J. Climate, 21, 54685481, doi:10.1175/2008JCLI2378.1.

    • Search Google Scholar
    • Export Citation
  • Baldocchi, D., and Coauthors, 2001: FLUXNET: A new tool to study the temporal and spatial variability of ecosystem-scale carbon dioxide, water vapor, and energy flux densities. Bull. Amer. Meteor. Soc., 82, 24152434, doi:10.1175/1520-0477(2001)082<2415:FANTTS>2.3.CO;2.

    • Search Google Scholar
    • Export Citation
  • Beck, M. B., and Coauthors, 2009: Grand challenges of the future for environmental modeling. White Paper, National Science Foundation, Arlington, VA, 135 pp. [Available online at http://www.ewp.rpi.edu/hartford/~ernesto/S2013/MMEES/Papers/ENVIRONMENT/1EnvironmentalSystemsModeling/Beck2009-nsfwhitepaper.pdf.]

  • Best, M. J., and Coauthors, 2011: The Joint UK Land Environment Simulator (JULES), model description—Part 1: Energy and water fluxes. Geosci. Model Dev., 4, 677699, doi:10.5194/gmd-4-677-2011.

    • Search Google Scholar
    • Export Citation
  • Best, M. J., and Coauthors, 2015: The plumbing of land surface models: Benchmarking model performance. J. Hydrometeor., 16, 14251442, doi:10.1175/JHM-D-14-0158.1.

    • Search Google Scholar
    • Export Citation
  • Beven, K. J., and Young P. , 2013: A guide to good practice in modeling semantics for authors and referees. Water Resour. Res., 49, 5092–5098, doi:10.1002/wrcr.20393.

    • Search Google Scholar
    • Export Citation
  • Blöschl, G., and Sivapalan M. , 1995: Scale issues in hydrological modelling: A review. Hydrol. Processes, 9, 251290, doi:10.1002/hyp.3360090305.

    • Search Google Scholar
    • Export Citation
  • Clark, M. P., Kavetski D. , and Fenicia F. , 2011: Pursuing the method of multiple working hypotheses for hydrological modeling. Water Resour. Res., 47, W09301, doi:10.1029/2010WR009827.

    • Search Google Scholar
    • Export Citation
  • Clark, M. P., and Coauthors, 2015: A unified approach for process-based hydrologic modeling: 1. Modeling concept. Water Resour. Res., 51, 24982514, doi:10.1002/2015WR017198.

    • Search Google Scholar
    • Export Citation
  • Cover, T. M., and Thomas J. A. , 1991: Elements of Information Theory. Wiley-Interscience, 726 pp.

  • Cybenko, G., 1989: Approximation by superpositions of a sigmoidal function. Math. Control Signal, 2, 303314, doi:10.1007/BF02551274.

  • Draper, D., 1995: Assessment and propagation of model uncertainty. J. Roy. Stat. Soc., 57B, 4597.

  • Edwards, A. F. W., 1984: Likelihood. Cambridge University Press, 243 pp.

  • Gong, W., Gupta H. V. , Yang D. , Sricharan K. , and Hero A. O. , 2013: Estimating epistemic and aleatory uncertainties during hydrologic modeling: An information theoretic approach. Water Resour. Res., 49, 22532273, doi:10.1002/wrcr.20161.

    • Search Google Scholar
    • Export Citation
  • Gupta, H. V., and Nearing G. S. , 2014: Using models and data to learn: A systems theoretic perspective on the future of hydrological science. Water Resour. Res., 50, 53515359, doi:10.1002/2013WR015096.

    • Search Google Scholar
    • Export Citation
  • Gupta, H. V., Clark M. P. , Vrugt J. A. , Abramowitz G. , and Ye M. , 2012: Towards a comprehensive assessment of model structural adequacy. Water Resour. Res., 48, W08301, doi:10.1029/2011WR011044.

    • Search Google Scholar
    • Export Citation
  • Gupta, H. V., Perrin C. , Kumar R. , Blöschl G. , Clark M. , Montanari A. , and Andréassian V. , 2014: Large-sample hydrology: A need to balance depth with breadth. Hydrol. Earth Syst. Sci., 18, 463477, doi:10.5194/hess-18-463-2014.

    • Search Google Scholar
    • Export Citation
  • Hansen, M. C., DeFries R. S. , Townshend J. R. G. , and Sohlberg R. , 2000: Global land cover classification at 1 km spatial resolution using a classification tree approach. Int. J. Remote Sens., 21, 13311364, doi:10.1080/014311600210209.

    • Search Google Scholar
    • Export Citation
  • Jaynes, E. T., 2003: Probability Theory: The Logic of Science. Cambridge University Press, 727 pp.

  • Jung, M., Reichstein M. , and Bondeau A. , 2009: Towards global empirical upscaling of FLUXNET eddy covariance observations: Validation of a model tree ensemble approach using a biosphere model. Biogeosciences, 6, 20012013, doi:10.5194/bg-6-2001-2009.

    • Search Google Scholar
    • Export Citation
  • Kavetski, D., Kuczera G. , and Franks S. W. , 2006: Bayesian analysis of input uncertainty in hydrological modeling: 2. Application. Water Resour. Res., 42, W03408, doi:10.1029/2005WR004376.

    • Search Google Scholar
    • Export Citation
  • Keenan, T. F., Davidson E. , Moffat A. M. , Munger W. , and Richardson A. D. , 2012: Using model–data fusion to interpret past trends, and quantify uncertainties in future projections, of terrestrial ecosystem carbon cycling. Global Change Biol., 18, 25552569, doi:10.1111/j.1365-2486.2012.02684.x.

    • Search Google Scholar
    • Export Citation
  • Kumar, S. V., and Coauthors, 2014: Assimilation of remotely sensed soil moisture and snow depth retrievals for drought estimation. J. Hydrometeor., 15, 24462469, doi:10.1175/JHM-D-13-0132.1.

    • Search Google Scholar
    • Export Citation
  • Liu, Y. Q., and Gupta H. V. , 2007: Uncertainty in hydrologic modeling: Toward an integrated data assimilation framework. Water Resour. Res., 43, W07401, doi:10.1029/2006WR005756.

    • Search Google Scholar
    • Export Citation
  • Liu, Y. Q., and Coauthors, 2011: The contributions of precipitation and soil moisture observations to the skill of soil moisture estimates in a land data assimilation system. J. Hydrometeor., 12, 750765, doi:10.1175/JHM-D-10-05000.1.

    • Search Google Scholar
    • Export Citation
  • Luo, Y. Q., and Coauthors, 2012: A framework for benchmarking land models. Biogeosciences, 9, 38573874, doi:10.5194/bg-9-3857-2012.

  • Mo, K. C., Long L. N. , Xia Y. , Yang S. K. , Schemm J. E. , and Ek M. , 2011: Drought indices based on the climate forecast system reanalysis and ensemble NLDAS. J. Hydrometeor., 12, 181205, doi:10.1175/2010JHM1310.1.

    • Search Google Scholar
    • Export Citation
  • Montanari, A., and Koutsoyiannis D. , 2012: A blueprint for process-based modeling of uncertain hydrological systems. Water Resour. Res., 48, W09555, doi:10.1029/2011WR011412.

    • Search Google Scholar
    • Export Citation
  • Neal, R. M., 1993: Probabilistic inference using Markov chain Monte Carlo methods. Tech. Rep. CRG-TR-93-1, Dept. of Computer Science, University of Toronto, 144 pp. [Available online at http://www.cs.toronto.edu/~radford/ftp/review.pdf.]

  • Nearing, G. S., and Gupta H. V. , 2015: The quantity and quality of information in hydrologic models. Water Resour. Res., 51, 524538, doi:10.1002/2014WR015895.

    • Search Google Scholar
    • Export Citation
  • Nearing, G. S., Gupta H. V. , and Crow W. T. , 2013: Information loss in approximately Bayesian estimation techniques: A comparison of generative and discriminative approaches to estimating agricultural productivity. J. Hydrol., 507, 163173, doi:10.1016/j.jhydrol.2013.10.029.

    • Search Google Scholar
    • Export Citation
  • Oberkampf, W. L., DeLand S. M. , Rutherford B. M. , Diegert K. V. , and Alvin K. F. , 2002: Error and uncertainty in modeling and simulation. Reliab. Eng. Syst. Saf., 75, 333357, doi:10.1016/S0951-8320(01)00120-X.

    • Search Google Scholar
    • Export Citation
  • Paninski, L., 2003: Estimation of entropy and mutual information. Neural Comput., 15, 11911253, doi:10.1162/089976603321780272.

  • Peters-Lidard, C. D., Kumar S. V. , Mocko D. M. , and Tian Y. , 2011: Estimating evapotranspiration with land data assimilation systems. Hydrol. Processes, 25, 39793992, doi:10.1002/hyp.8387.

    • Search Google Scholar
    • Export Citation
  • Poulin, A., Brissette F. , Leconte R. , Arsenault R. , and Malo J.-S. , 2011: Uncertainty of hydrological modelling in climate change impact studies in a Canadian, snow-dominated river basin. J. Hydrol., 409, 626636, doi:10.1016/j.jhydrol.2011.08.057.

    • Search Google Scholar
    • Export Citation
  • Rasmussen, C., and Williams C. , 2006: Gaussian Processes for Machine Learning. MIT Press, 248 pp.

  • Schöniger, A., Wöhling T. , and Nowak W. , 2015: A statistical concept to assess the uncertainty in Bayesian model weights and its impact on model ranking. Water Resour. Res., 51, 75247546, doi:10.1002/2015WR016918.

    • Search Google Scholar
    • Export Citation
  • Shannon, C. E., 1948: A mathematical theory of communication. Bell Syst. Tech. J., 27, 379423, doi:10.1002/j.1538-7305.1948.tb01338.x.

    • Search Google Scholar
    • Export Citation
  • Snelson, E., and Ghahramani Z. , 2006: Sparse Gaussian processes using pseudo-inputs. Advances in Neural Information Processing Systems 18, Y. Weiss, B. Schölkopf, and J. C. Platt, Eds., Neural Information Processing Systems, 1257–1264. [Available online at http://papers.nips.cc/paper/2857-sparse-gaussian-processes-using-pseudo-inputs.]

  • van den Hurk, B., Best M. , Dirmeyer P. , Pitman A. , Polcher J. , and Santanello J. , 2011: Acceleration of land surface model development over a decade of GLASS. Bull. Amer. Meteor. Soc., 92, 15931600, doi:10.1175/BAMS-D-11-00007.1.

    • Search Google Scholar
    • Export Citation
  • Wand, M. P., and Jones M. C. , 1994: Kernel Smoothing. CRC Press, 212 pp.

  • Weijs, S. V., Schoups G. , and Giesen N. , 2010: Why hydrological predictions should be evaluated using information theory. Hydrol. Earth Syst. Sci., 14, 25452558, doi:10.5194/hess-14-2545-2010.

    • Search Google Scholar
    • Export Citation
  • Wilby, R. L., and Harris I. , 2006: A framework for assessing uncertainties in climate change impacts: Low-flow scenarios for the River Thames, UK. Water Resour. Res., 42, W02419, doi:10.1029/2005WR004065.

    • Search Google Scholar
    • Export Citation
  • Xia, Y., and Coauthors, 2012a: Continental-scale water and energy flux analysis and validation for the North American Land Data Assimilation System project phase 2 (NLDAS-2): 1. Intercomparison and application of model products. J. Geophys. Res., 117, D03109, doi:10.1029/2011JD016051.

    • Search Google Scholar
    • Export Citation
  • Xia, Y., and Coauthors, 2012b: Continental-scale water and energy flux analysis and validation for North American Land Data Assimilation System project phase 2 (NLDAS-2): 2. Validation of model-simulated streamflow. J. Geophys. Res., 117, D03110, doi:10.1029/2011JD016051.

    • Search Google Scholar
    • Export Citation
  • Xia, Y., Sheffield J. , Ek M. B. , Dong J. , Chaney N. , Wei H. , Meng J. , and Wood E. F. , 2014: Evaluation of multi-model simulated soil moisture in NLDAS-2. J. Hydrol., 512, 107125, doi:10.1016/j.jhydrol.2014.02.027.

    • Search Google Scholar
    • Export Citation
  • Xia, Y., Hobbins M. T. , Mu Q. , and Ek M. B. , 2015: Evaluation of NLDAS-2 evapotranspiration against tower flux site observations. Hydrol. Processes, 29, 17571771, doi:10.1002/hyp.10299.

    • Search Google Scholar
    • Export Citation
  • Ziv, J., and Zakai M. , 1973: On functionals satisfying a data-processing theorem. IEEE Trans. Inf. Theory, 19, 275283, doi:10.1109/TIT.1973.1055015.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1424 474 113
PDF Downloads 803 186 10