Performance Metrics for Soil Moisture Retrievals and Application Requirements

Dara Entekhabi Ralph M. Parsons Laboratory for Environmental Science and Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts

Search for other papers by Dara Entekhabi in
Current site
Google Scholar
PubMed
Close
,
Rolf H. Reichle Global Modeling and Assimilation Office, NASA Goddard Space Flight Center, Greenbelt, Maryland

Search for other papers by Rolf H. Reichle in
Current site
Google Scholar
PubMed
Close
,
Randal D. Koster Global Modeling and Assimilation Office, NASA Goddard Space Flight Center, Greenbelt, Maryland

Search for other papers by Randal D. Koster in
Current site
Google Scholar
PubMed
Close
, and
Wade T. Crow USDA/ARS Hydrology and Remote Sensing Laboratory, Beltsville, Maryland

Search for other papers by Wade T. Crow in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Quadratic performance metrics such as root-mean-square error (RMSE) and time series correlation are often used to assess the accuracy of geophysical retrievals (satellite measurements) with respect to true fields. These metrics are related; nevertheless, each has advantages and disadvantages. In this study the authors explore the relation between the RMSE and correlation metrics in the presence of biases in the mean as well as in the amplitude of fluctuations (standard deviation) between estimated and true fields. Such biases are common, for example, in satellite retrievals of soil moisture and impose constraints on achievable and meaningful RMSE targets. Last, an approach is introduced for converting a requirement in an application’s product into a corresponding requirement for soil moisture accuracy. The approach can help with the formulation of soil moisture measurement requirements. It can also help determine the utility of a given retrieval product for applications.

Corresponding author address: Dara Entekhabi, 48-216G, MIT, Cambridge, MA 02139. Email: darae@mit.edu

Abstract

Quadratic performance metrics such as root-mean-square error (RMSE) and time series correlation are often used to assess the accuracy of geophysical retrievals (satellite measurements) with respect to true fields. These metrics are related; nevertheless, each has advantages and disadvantages. In this study the authors explore the relation between the RMSE and correlation metrics in the presence of biases in the mean as well as in the amplitude of fluctuations (standard deviation) between estimated and true fields. Such biases are common, for example, in satellite retrievals of soil moisture and impose constraints on achievable and meaningful RMSE targets. Last, an approach is introduced for converting a requirement in an application’s product into a corresponding requirement for soil moisture accuracy. The approach can help with the formulation of soil moisture measurement requirements. It can also help determine the utility of a given retrieval product for applications.

Corresponding author address: Dara Entekhabi, 48-216G, MIT, Cambridge, MA 02139. Email: darae@mit.edu

Save
  • Albertson, J. D., and Kiely G. , 2001: On the structure of soil moisture time series in the context of land surface models. J. Hydrol., 243 , 101119.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barnston, A. G., 1992: Correspondence among the correlation, RMSE, and Heidke forecast verification measures: Refinement of the Heidke score. Wea. Forecasting, 7 , 699709.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Budyko, M. I., 1974: Climate and Life. Academic Press, 508 pp.

  • Cosh, M. H., Jackson T. J. , Starks P. J. , and Heathman G. , 2006: Temporal stability of surface soil moisture in the Little Washita River Watershed and its applications in satellite soil moisture product validation. J. Hydrol., 323 , 168177.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crow, W. T., 2007: A novel method for quantifying value in spaceborne soil moisture retrievals. J. Hydrometeor., 8 , 5667.

  • Crow, W. T., and Wood E. F. , 1999: Multi-scale dynamics of soil moisture variability observed during SGP’97. Geophys. Res. Lett., 26 , 34853488.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drusch, M., Wood E. F. , and Gao H. , 2005: Observation operators for the direct assimilation of TRMM microwave imager retrieved soil moisture. Geophys. Res. Lett., 32 , L15403. doi:10.1029/2005GL023623.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Eagleson, P. S., 1978: Climate, soil and vegetation 4. The expected value of annual evapotranspiration. Water Resour. Res., 14 , 731739.

  • Entin, J. K., Robock A. , Vinnikov K. Y. , Hollinger S. E. , Liu S. , and Namkhai A. , 2000: Temporal and spatial scales of observed soil moisture variations in the extratropics. J. Geophys. Res., 105 , (D9). 1186511877.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Famiglietti, J. S., and Coauthors, 1999: Ground-based investigation of soil moisture variability within remote sensing footprints during the Southern Great Plains 1997 (SGP97) hydrology experiment. Water Resour. Res., 35 , 18391851.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Famiglietti, J. S., Ryu D. , Berg A. A. , Rodell M. , and Jackson T. J. , 2008: Field observations of soil moisture variability across scales. Water Resour. Res., 44 , W01423. doi:10.1029/2006WR005804.

    • Search Google Scholar
    • Export Citation
  • Gupta, H. V., Kling H. , Yilmaz K. K. , and Martinez G. F. , 2009: Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. J. Hydrol., 377 , 8091. doi:10.1016/j.jhydrol.2009.08.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hu, Z., Islam S. , and Cheng Y. , 1997: Statistical characterization of remotely sensed soil moisture images. Remote Sens. Environ., 61 , 310318.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koster, R. D., and Milly P. C. D. , 1997: The interplay between transpiration and runoff formulations in land surface schemes used with atmospheric models. J. Climate, 10 , 15781591.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koster, R. D., Guo Z. , Yang R. , Dirmeyer P. A. , Mitchell K. , and Puma M. J. , 2009: On the nature of soil moisture in land surface models. J. Climate, 22 , 43224335.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Martínez-Fernández, J., and Ceballos A. , 2003: Temporal stability of soil moisture in a large-field experiment in Spain. Soil Sci. Soc. Amer. J., 67 , 16471656.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mohanty, B. P., and Skaggs T. H. , 2001: Spatio-temporal evolution and time-stable characteristics of soil moisture within remote sensing footprints with varying soil, slope, and vegetation. Adv. Water Resour., 24 , 10511067.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1995: The coefficients of correlation and determination as measures of performance in forecast verification. Wea. Forecasting, 10 , 681688.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reichle, R. H., Koster R. D. , Dong J. , and Berg A. A. , 2004: Global soil moisture from satellite observations, land surface models, and ground data: Implications for data assimilation. J. Hydrometeor., 5 , 430442.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reichle, R. H., Koster R. D. , Liu P. , Mahanama S. P. P. , Njoku E. G. , and Owe M. , 2007: Comparison and assimilation of global soil moisture retrievals from the Advanced Microwave Scanning Radiometer for the Earth Observing System (AMSR-E) and the Scanning Multichannel Microwave Radiometer (SMMR). J. Geophys. Res., 112 , D09108. doi:10.1029/2006JD008033.

    • Search Google Scholar
    • Export Citation
  • Rodriguez-Iturbe, I., Vogel G. K. , Rigon R. , Entekhabi D. , Castelli F. , and Rinaldo A. , 1995: On the spatial organization of soil moisture fields. Geophys. Res. Lett., 22 , 27572760.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rodriguez-Iturbe, I., Porporato A. , Ridolfi L. , Isham V. , and Cox D. R. , 1999: Probabilistic modelling of water balance at a point: The role of climate, soil and vegetation. Proc. Roy. Soc. London, 455A , 37893805.

    • Search Google Scholar
    • Export Citation
  • Ryu, D., and Famiglietti J. S. , 2005: Characterization of footprint-scale surface soil moisture variability using Gaussian and beta distribution functions during the Southern Great Plains 1997 (SGP97) hydrology experiment. Water Resour. Res., 41 , W12433. doi:10.1029/2004WR003835.

    • Search Google Scholar
    • Export Citation
  • Scipal, K., Holmes T. , de Jeu R. A. M. , Naeimi V. , and Wagner W. , 2008: A possible solution for the problem of estimating the error structure of global soil moisture data sets. Geophys. Res. Lett., 35 , L24403. doi:10.1029/2008GL035599.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sheffield, J., Goteti G. , and Wood E. F. , 2006: Development of a 50-year high-resolution global dataset of meteorological forcings for land surface modeling. J. Climate, 19 , 30883111.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stanski, H. R., Wilson L. J. , and Burrows W. R. , 1989: Survey of common verification methods in meteorology. WMO World Weather Watch Tech. Rep. 8, WMO/TD 358, 114 pp.

    • Search Google Scholar
    • Export Citation
  • Teuling, A. J., and Troch P. A. , 2005: Improved understanding of soil moisture variability dynamics. Geophys. Res. Lett., 32 , L05404. doi:10.1029/2004GL021935.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1899 565 63
PDF Downloads 1399 396 41