1. Introduction
Introduced by Stoffelen (1998), triple collocation analysis (TCA) is an error magnitude estimation methodology that intercompares geophysical products obtained via three or more independent estimation/observation techniques. The approach was originally developed for ocean wind studies but has been increasingly being applied in land surface hydrology (Scipal et al. 2008; Crow and van den Berg 2010; Dorigo et al. 2010; Miralles et al. 2010; Hain et al. 2011; Parinussa et al. 2011; Anderson et al. 2012; Yilmaz et al. 2012; Yilmaz and Crow 2013; Zwieback et al. 2013; Draper et al. 2013). The majority of TCA land surface applications have focused on quantifying errors in satellite-based surface soil moisture products retrieved from instruments like the Advanced Microwave Scanning Radiometer for Earth Observing System (AMSR-E) sensor aboard the National Aeronautics and Space Administration’s (NASA) Aqua satellite. In addition, ongoing Soil Moisture Ocean Salinity (SMOS) validation efforts have included TCA to attribute the sources of error magnitudes to different ground conditions like forest cover, texture, and radio frequency interference (RFI; Leroux et al. 2013). Likewise, current Soil Moisture Active Passive (SMAP) calibration and validation plans call for the use of TCA along with other methodologies to estimate retrieval errors over sparse networks using point observations (T. J. Jackson 2013, personal communication). In addition to error magnitude comparison studies, the error variance estimates obtained from TCA can also be used to provide observation error covariance information required by data assimilation (Crow and van den Berg 2010; Crow and Yilmaz 2014) and least squares merging approaches (Yilmaz et al. 2012). Finally, TCA can also be used as stand-alone rescaling methodology to remove systematic differences between the signal variance (true variability) component of observations in data assimilation studies (Yilmaz and Crow 2013).
Using three products that observe the same geophysical variable, TCA is able to find relative error estimates for these products under certain assumptions, namely, the independence of product errors with respect to the truth (i.e., error orthogonality) and the relative independence of errors in each product (i.e., zero error cross correlation; Stoffelen 1998). These assumptions are generally required to reduce the overdetermined TCA system of equations into a determined one. Using three datasets, for example, the TCA system has seven unknown parameters with only three available equations (Yilmaz et al. 2012). Assuming one of the datasets as reference for rescaling purposes and further assuming error orthogonality and zero error cross correlation, the number of unknown parameters drops to three and the system is solvable. These assumptions can be eased if the magnitude of error orthogonality is known (Stoffelen 1998; Portabella and Stoffelen 2009) or more than three products are available for cross comparison (Zwieback et al. 2012); however, the majority of soil moisture–based triple collocation studies use only three datasets and are therefore forced to rely on these assumptions to ensure a bias-free TCA.
Here we investigate the impact of the error orthogonality and zero cross-correlation assumptions on TCA-based soil moisture error estimates by 1) analytically deriving the specific terms in TCA that must vanish in order for TCA to produce an unbiased error variance estimate and 2) numerically evaluating these terms over a series of watershed sites where high-quality ground-based soil moisture observations are obtainable. Section 2 presents the analytical basis of the study, section 3 presents results, and section 4 summarizes our key findings.
2. Methodology
a. Triple collocation–based errors




































































b. Implications of triple collocation assumptions


































In order for
c. Evaluation using ground-based data
Here, we attempt to quantify error nonorthogonality and cross-correlation statistics and the manner in which these statistics are combined (with each other and rescaling factors) to yield terms (16)–(18) contributing as bias to TCA error variance estimates. To this end, we use ground-based station data (




















































Sampling errors in TCA-based error variance (9)–(11), TCA components (16)–(19), and error nonorthogonality variance and cross-covariance (21) and (22) estimates are calculated using a bootstrapping approach where 1000 separate time series replicates are randomly sampled (with replacement) from the original time series (Efron and Tibshirani 1993). Plotted 95% sampling confidence intervals are calculated as twice the sampled standard deviation of these replicates.
Because the basic premise behind rescaling in TCA is to eliminate the signal variance components of products by multiplying products proportional to their signal variances [see (8)], rescaling factors themselves may be an indicator of relative accuracy of products (i.e., higher rescaling factors are associated with products with lower signal variances) when products with equal total variances are compared. To illustrate the impact of reference dataset selection on rescaling factors, rescaling factors are estimated for the cases where both
d. Data and study locations
TCA is performed over the four U.S. Department of Agriculture (USDA) Agricultural Research Service (ARS) watersheds (Little Washita, Little River, Reynolds Creek, and Walnut Gulch) currently used for the validation of AMSR-E and SMOS surface soil moisture products (Jackson et al. 2010, 2012; Leroux et al. 2014). These watersheds have dominant grassland, forest/agriculture, mountainous, and semiarid land cover (respectively) with surface (0–5 cm) soil moisture sensors collecting data at 20–60-min intervals at 16–29 separate spatial locations (Jackson et al. 2010). They extend over areas ranging between 150 and 610 km2 and have, on average, one soil moisture observation station per 16 km2 area (Jackson et al. 2010). Watershed-scale spatial averages (corresponding to
In addition to this ground-based data, remotely sensed surface soil moisture estimates (roughly corresponding to the top 1–3 cm of the soil column) are retrieved from the Advanced Scatterometer (ASCAT) and AMSR-E satellite sensors over all four watershed sites. ASCAT soil moisture values are obtained from the Vienna University of Technology using the algorithm described in Wagner et al. (1999) and Naeimi et al. (2009). AMSR-E retrievals are acquired from the VU University Amsterdam using the Land Parameter Retrieval Model (LPRM) described in Owe et al. (2001, 2008). Note that because of the active microwave basis of the ASCAT retrievals and the passive microwave nature of the AMSR-E/LPRM retrievals, these two products are commonly assumed to have independent errors.
Land surface model–based predictions of surface soil moisture are based on the top soil layer (0–10 cm) predictions acquired from version 2.7 of the Noah model. Forcing data for these Noah simulations are based on Global Land Data Assimilation System (GLDAS; Rodell et al. 2004) meteorological data distributed by the Goddard Earth Sciences (GES) Data and Information Services Center (DISC). Soil parameters are based on the dataset of Reynolds et al. (2000) and land cover maps/parameters produced from the University of Maryland global land cover product (Hansen et al. 2000). More information about the Noah model can be found in Ek et al. (2003).
All model- and satellite-based products are obtained at 0.25° spatial resolution while the study is performed between January 2007 and September 2011 at daily time steps. These soil moisture products have high mutual cross correlation (Brocca et al. 2011), which is consistent with the linear relationships assumed in (1)–(3). Datasets used in TCA (
Error estimates (9)–(11), (16)–(19), and (23), error nonorthogonality (21), and error cross covariances (22) are obtained over each watershed (total of four) and for each datasets (total of three). To condense these results, error estimates for all four watersheds are averaged into a single value for each product.
3. Results
All results are from selecting Noah-based soil moisture products as the reference dataset for TCA because it demonstrates the highest cross correlation with ground data (average cross correlations with station data are 0.60, 0.58, and 0.55 for Noah, LPRM, and ASCAT, respectively). When averaged over all four watersheds, all three soil moisture products (ASCAT, LPRM, and Noah) demonstrate significant levels of error nonorthogonality and cross covariance (Fig. 1). This implies that the neglect of error nonorthogonality and cross covariance is not justified in standard soil moisture TCA. Figure 2 examines this issue directly by plotting the TCA bias terms derived in (16)–(18). Results demonstrate that the

Error nonorthogonality variance (21) and error cross covariances (22) averaged across all four watershed sites for all three soil moisture products. Error bars represent two std dev of sampling errors estimated using a bootstrapping approach.
Citation: Journal of Hydrometeorology 15, 3; 10.1175/JHM-D-13-0158.1

Error nonorthogonality variance (21) and error cross covariances (22) averaged across all four watershed sites for all three soil moisture products. Error bars represent two std dev of sampling errors estimated using a bootstrapping approach.
Citation: Journal of Hydrometeorology 15, 3; 10.1175/JHM-D-13-0158.1
Error nonorthogonality variance (21) and error cross covariances (22) averaged across all four watershed sites for all three soil moisture products. Error bars represent two std dev of sampling errors estimated using a bootstrapping approach.
Citation: Journal of Hydrometeorology 15, 3; 10.1175/JHM-D-13-0158.1

TCA-based error variances (9)–(11) and station-based error variances
Citation: Journal of Hydrometeorology 15, 3; 10.1175/JHM-D-13-0158.1

TCA-based error variances (9)–(11) and station-based error variances
Citation: Journal of Hydrometeorology 15, 3; 10.1175/JHM-D-13-0158.1
TCA-based error variances (9)–(11) and station-based error variances
Citation: Journal of Hydrometeorology 15, 3; 10.1175/JHM-D-13-0158.1

Rescaling factors averaged over all four watershed sites. Error bars represent two std dev of sampling errors estimated using a bootstrapping approach.
Citation: Journal of Hydrometeorology 15, 3; 10.1175/JHM-D-13-0158.1

Rescaling factors averaged over all four watershed sites. Error bars represent two std dev of sampling errors estimated using a bootstrapping approach.
Citation: Journal of Hydrometeorology 15, 3; 10.1175/JHM-D-13-0158.1
Rescaling factors averaged over all four watershed sites. Error bars represent two std dev of sampling errors estimated using a bootstrapping approach.
Citation: Journal of Hydrometeorology 15, 3; 10.1175/JHM-D-13-0158.1
In contrast to the impact of error cross correlation, Fig. 2 reveals that the impact of error nonorthogonality and nonoptimal rescaling is dampened significantly when aggregated to form the TCA bias terms
The summation of the (nonnegligible) negative
TCA-based error estimates can be validated using station-based error estimates; however it is often not clear how the representativeness errors of station data and TCA assumptions impact these comparisons. Appendix A, section b, analytically investigates the difference between station- and TCA-based error variances (Fig. 2). This difference is positively biased because of representativeness errors in the ground station data. On the other hand, this bias is reduced by error cross-covariance terms (A11). Representativeness errors impact the estimation of station-based error variances (21)–(23) too. However, the sign of the bias of these estimates is difficult to predict (see concluding remarks in appendix A, section a). This is also supported by Fig. 2: the differences between
Figure 3 shows how rescaling factors may change depending on the reference dataset selection. In general, the rescaling factor is expected to be higher than one when the reference dataset has higher signal variance than the rescaled dataset. For products with the same total variance (e.g., in this study the total variance of each product is one), higher signal variance implies a more skillful product (i.e., a smaller error variance and signal-to-noise ratio). Rescaling factors are consistently higher when station data, rather than Noah model predictions, are used as the reference dataset. This implies that the signal variance of the station data is higher than that of other datasets, or alternatively, that the representativeness errors of watershed-averaged station data used in this study are less than estimation errors present in other datasets.
4. Conclusions
Here, we evaluate the appropriateness of the error orthogonality
Results suggest that required TCA assumptions of error orthogonality and zero error cross covariance do not generally hold for typical surface soil moisture data products (Fig. 1). However, error nonorthogonality does not contribute significantly to TCA bias [via the
Here, we obtained error cross-covariance information using ground datasets as truth. In locations lacking such datasets, the detection of error cross covariance using the available model and satellite products alone (i.e., no dataset available that can be assumed as truth) will present a challenge. However, as demonstrated in Draper et al. (2013) and Zwieback et al. (2012), utilizing more than three (i.e., four) datasets in TCA provides an opportunity to detect the presence of error cross covariance in the absence of any collaborating ground data observations.
TCA results presented here are all verified using comparison against station-based soil moisture observations. However, such data is just another estimate with its own characteristic (representativeness) errors. Therefore, significant bias introduced by error cross variance also results in TCA-error variances being negatively biased when compared to station-based error variances. Here this is demonstrated both numerically (Fig. 2) and analytically (appendix A, section b). However, it is not possible to predict the sign of the bias between station-based errors—derived via (23)—and
Acknowledgments
We thank Michael Cosh of the U.S. Department of Agriculture for the USDA ARS watershed soil moisture datasets, Robert Parinussa of Vrije Universiteit Amsterdam for LPRM datasets, Wouter Dorigo of Technische Universität Wien for ASCAT datasets, and NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC) for Noah datasets. Research was funded by Wade Crow’s membership in the NASA Soil Moisture Active Passive (SMAP) Science Definition Team. M. Tugrul Yilmaz's work is partially supported by Fund 2232 given by Turkish Scientific and Technical Research Council (TUBITAK).
APPENDIX A
Station- versus TCA-Based Error Estimates
In this section, the difference between
a. Station-based error variances

















In addition to true random error
b. Station- and TCA-based error variance comparison










































In summary, three out of the four terms that appear in
APPENDIX B
Impacts of Error Nonorthogonality and Cross Covariance on TCA Bias Terms
Numerical results in Fig. 2 point to a fundamental asymmetry in the impact of nonzero error orthogonality versus nonzero error cross covariance on TCA. Because (7) describes our rescaling approach, inserting (7) into (16)–(19) analytically explains this asymmetry. This combination yields a single (complex) expression that describes the combined impact of 1) error orthogonality, 2) error cross covariance, and 3) the impact of error orthogonality and error cross covariance on rescaling results.



























Taken as a whole, this analysis demonstrates that the
REFERENCES
Anderson, W. B., Zaitchik B. F. , Hain C. R. , Anderson M. C. , Yilmaz M. T. , Mecikalski J. , and Schultz L. , 2012: Towards an integrated soil moisture drought monitor for East Africa. Hydrol. Earth Syst. Sci., 9, 4587–4631, doi:10.5194/hessd-9-4587-2012.
Brocca, L., and Coauthors, 2011: Soil moisture estimation through ASCAT and AMSR-E sensors: An intercomparison and validation study across Europe. Remote Sens. Environ., 115, 3390–3408, doi:10.1016/j.rse.2011.08.003.
Cosh, M. H., Jackson T. J. , Starksb P. , and Heathm G. , 2006: Temporal stability of surface soil moisture in the Little Washita River watershed and its applications in satellite soil moisture product validation. J. Hydrol., 323, 168–177, doi:10.1016/j.jhydrol.2005.08.020.
Cosh, M. H., Jackson T. J. , Moran S. , and Bindlish R. , 2008: Temporal persistence and stability of surface soil moisture in a semi-arid watershed. Remote Sens. Environ., 112, 304–313, doi:10.1016/j.rse.2007.07.001.
Crow, W. T., and van den Berg M. J. , 2010: An improved approach for estimating observation and model error parameters in soil moisture data assimilation. Water Resour. Res., 46, W12519, doi:10.1029/2010WR009402.
Crow, W. T., and Yilmaz M. T. , 2014: The Auto-Tuned Land Data Assimilation System (ATLAS). Water Resour. Res., 50, 371–385, doi:10.1002/2013WR014550.
Dorigo, W. A., Scipal K. , Parinussa R. , Liu Y. , Wagner W. , de Jeu R. , and Naeimi V. , 2010: Error characterisation of global active and passive microwave soil moisture datasets. Hydrol. Earth Syst. Sci., 14, 2605–2616, doi:10.5194/hess-14-2605-2010.
Draper, C., de Jeu Reichle R. , Naeimi V. , Parinussa R. , and Wagner W. , 2013: Estimating root mean square errors in remotely sensed soil moisture over continental scale domains. Remote Sens. Environ., 137, 288–298, doi:10.1016/j.rse.2013.06.013.
Efron, B., and Tibshirani R. J. , 1993: An Introduction to the Bootstrap. Chapman and Hall, 436 pp.
Ek, M. B., Mitchell K. E. , Lin Y. , Rogers E. , Grunmann P. , Koren V. , Gayand G. , and Tarpley J. D. , 2003: Implementation of Noah land surface model advances in the national centers for environmental prediction operational mesoscale eta model. J. Geophys. Res., 108, 8851, doi:10.1029/2002JD003296.
Hain, C. R., Crow W. T. , Mecikalski J. R. , Anderson M. C. , and Holmes T. , 2011: An intercomparison of available soil moisture estimates from thermal infrared and passive microwave remote sensing and land surface modeling. J. Geophys. Res., 116, D15107, doi:10.1029/2011JD015633.
Hansen, M. C., DeFries R. S. , Townshend J. R. G. , and Sohlberg R. , 2000: Global land cover classification at 1 km spatial resolution using a classification tree approach. Int. J. Remote Sens., 21, 1331–1364, doi:10.1080/014311600210209.
Jackson, T. J., and Coauthors, 2010: Validation of advanced microwave scanning radiometer soil moisture products. IEEE Trans. Geosci. Remote Sens., 48, 4256–4272, doi:10.1109/TGRS.2010.2051035.
Jackson, T. J., and Coauthors, 2012: Validation of Soil Moisture and Ocean Salinity (SMOS) soil moisture over watershed networks in the U.S. IEEE Trans. Geosci. Remote Sens., 50, 1530–1543, doi:10.1109/TGRS.2011.2168533.
Leroux, D. J., Kerr Y. , Richaume P. , and Fieuzal R. , 2013: Spatial distribution and possible sources of SMOS errors at the global scale. Remote Sens. Environ., 133, 240–250, doi:10.1016/j.rse.2013.02.017.
Leroux, D. J., Kerr Y. , Bitar A. , Bindlish R. , Jackson T. , Berthelot B. , and Portet G. , 2014: Comparison between SMOS, VUA, ASCAT, and ECMWF soil moisture products over four watersheds in U.S. IEEE Trans. Geosci. Remote Sens., 52, 1562–1571, doi:10.1109/TGRS.2013.2252468.
Miralles, D. G., Crow W. T. , and Cosh M. H. , 2010: Estimating spatial sampling errors in coarse-scale soil moisture estimates derived from point-scale observations. J. Hydrometeor., 11, 1423–1429, doi:10.1175/2010JHM1285.1.
Naeimi, V., Scipal K. , Bartalis Z. , Hasenauer S. , and Wagner W. , 2009: An improved soil moisture retrieval algorithm for ERS and METOP scatterometer observations. IEEE Trans. Geosci. Remote Sens., 47, 1999–2013, doi:10.1109/TGRS.2008.2011617.
Owe, M., de Jeu R. , and Walker J. P. , 2001: A methodology for surface soil moisture and vegetation optical depth retrieval using the microwave polarization difference index. IEEE Trans. Geosci. Remote Sens., 39, 1643–1654, doi:10.1109/36.942542.
Owe, M., de Jeu R. , and Holmes T. , 2008: Multisensor historical climatology of satellite-derived global land surface moisture. J. Geophys. Res., 113, F01002, doi:10.1029/2007JF000769.
Parinussa, R. M., Holmes T. R. H. , Yilmaz M. T. , and Crow W. T. , 2011: The impact of land surface temperature on soil moisture anomaly detection from passive microwave observations. Hydrol. Earth Syst. Sci., 15, 3135–3151, doi:10.5194/hess-15-3135-2011.
Portabella, M., and Stoffelen A. , 2009: On scatterometer ocean stress. J. Atmos. Oceanic Technol., 26, 368–382, doi:10.1175/2008JTECHO578.1.
Reynolds, C. A., Jackson T. J. , and Rawls W. J. , 2000: Estimating soil water-holding capacities by linking the Food and Agriculture Organization soil map of the world with global pedon databases and continuous pedotransfer functions. Water Resour. Res., 36, 3653–3662, doi:10.1029/2000WR900130.
Rodell, M., and Coauthors, 2004: The Global Land Data Assimilation System. Bull. Amer. Meteor. Soc., 85, 381–394, doi:10.1175/BAMS-85-3-381.
Scipal, K., Holmes T. , de Jeu R. , Naeimi V. , and Wagner W. , 2008: A possible solution for the problem of estimating the error structure of global soil moisture data sets. Geophys. Res. Lett., 35, L24403, doi:10.1029/2008GL035599.
Stoffelen, A., 1998: Toward the true near-surface wind speed: Error modeling and calibration using triple collocation. J. Geophys. Res., 103, 7755–7766, doi:10.1029/97JC03180.
Wagner, W., Lemoine G. , Borgeaud M. , and Rott H. , 1999: A study of vegetation cover effects on ERS scatterometer data. IEEE Trans. Geosci. Remote Sens., 37, 938–948, doi:10.1109/36.752212.
Yilmaz, M. T., and Crow W. T. , 2013: The optimality of potential rescaling approaches in land data assimilation. J. Hydrometeor., 14, 650–660, doi:10.1175/JHM-D-12-052.1.
Yilmaz, M. T., Crow W. T. , Anderson M. C. , and Hain C. , 2012: An objective methodology for merging satellite- and model-based soil moisture products. Water Resour. Res., 48, W11502, doi:10.1029/2011WR011682.
Zwieback, S., Scipal K. , Dorigo W. , and Wagner W. , 2012: Structural and statistical properties of the collocation technique for error characterization. Nonlinear Processes Geophys., 19, 69–80, doi:10.5194/npg-19-69-2012.
Zwieback, S., Dorigo W. , and Wagner W. , 2013: Estimation of the temporal autocorrelation structure by the collocation technique with emphasis on soil moisture studies. Hydrol. Sci. J., 58, 1729–1747, doi:10.1080/02626667.2013.839876.