a. The ensemble Kalman filter

The ensemble Kalman filter is based on the generation of an ensemble of model predictions to estimate the state forecast covariance information required by the standard Kalman filter for the updating of model predictions with observations (Evensen 1994; Burgers et al. 1998; Reichle et al. 2002a). The EnKF can be generalized using a state space representation of model prediction and observation operators. Take Y(t) to be a vector of land surface state variables at time t. The equation describing the evolution of these states, as determined by a land surface model F, is given by

View Expanded

where w relates errors in model physics, parameterization, and/or forcing data and is taken to be mean zero with a covariance C_w. In this analysis, Y will contain all the soil water and energy storage (i.e., soil moisture and soil temperature) states of the hydrologic model. The goal of the filtering problem is to constrain these forecasts using a set of observations that are related to the model states contained in Y. Let the operator M represent the observation process that relates Y to the actual measurements taken at time t_k:

View Expanded

where v_k represents Gaussian measurement error with covariance C_υ and Z_k is an m-dimensional vector containing a set of observations. Here, both streamflow observations and surface soil moisture retrievals (ostensibly from a remote sensing source) will be considered for Z. The EnKF is initialized by the introduction of synthetic Gaussian error into initial conditions and generating an ensemble of model predictions using Eq. (1). At the time of measurement, predictions made by the ith model replicate are referred to as the state forecast Yⁱ₋. If F and M are linear and all errors are additive, independent, and Gaussian, the optimal updating of Yⁱ₋ by the measurement Z_k is given by

View Expanded

and

View Expanded

where C_M is the covariance matrix of the forecasted observations M_k(Yⁱ₋) and C_YM is the cross-covariance matrix linking the forecasted observations with the state variables contained in Yⁱ₋. Both covariances are statistically estimated from all individual ensemble realizations and calculated around the ensemble mean. Here Yⁱ₋ signifies the updated or analysis state representation. Filter predictions are obtained by averaging analysis representations across the model ensemble. Of particular interest here are modeling errors represented by w in (1) and the impact of making inaccurate assumptions concerning their source and magnitude. One potential diagnostic tool is the filter innovation (ν_k), defined as

View Expanded

where the brackets represent ensemble averaging. If all the assumptions underlying the optimality of the EnKF filter (e.g., fully linear models and additive Gaussian error) are met and w is perfectly represented in a statistical sense, then ν_k should be mean zero, temporally uncorrelated, and have a variance of m⁻¹(C_M + C_υ). Or, alternatively the variable α_k, defined as

View Expanded

should be mean one and temporally uncorrelated (Dee 1995). Normalized innovations have value as a diagnostic tool since various statistical choices for modeling errors used to generate the forecast ensemble can be evaluated based on how closely the resulting innovation time series statistics match these values. Being essentially the normalized second moment of ν_k, the mean of α_k over time provides an integrated measure of the deviation of innovation mean and variance quantities from theoretical expectations. For instance, overestimation (underestimation) of model error will generally lead to a time series of α with a mean less than (greater than) one. Adaptive tuning strategies exploit the operational availability of innovation statistics by tuning assumed levels of model error such that the statistical properties of α_k are optimized (Mitchell and Houtekamer 1999).

b. TOPLATS hydrologic modeling

Land surface modeling is based on TOPMODEL-based Land Atmosphere Transfer Scheme (TOPLATS; Famiglietti and Wood 1994) predictions. Baseline algorithms for this particular version of the TOPLATS are found in Peters-Lidard et al. (1997). TOPLATS representation of cold season processes is described in Pauwels and Wood (1999). The expansion of the TOPLATS soil water balance approach from a two- to four-layer model can be found in Crow (2001, 2003. The top three soil layers are TOPLATS surface-zone soil moisture (θ_sz) predictions are based on the linear combination of two dynamic model state variables: the fraction of the land surface saturated from below by the water table ( f_w) multiplied by the soil’s saturation capacity (θ_sat) and the surface soil water content in nonsaturated portions of the basin (θ_unsat):

View Expanded

The calculation of surface-zone soil moisture using a linear combination of a nonsaturated surface soil moisture state and a dynamic fraction representing surface saturation is a common modeling feature for many land data assimilation studies (see, e.g., Walker and Houser 2001; Pauwels et al. 2001; Reichle et al. 2002b; Crow 2003; Crow and Wood 2003; Reichle and Koster 2005). In TOPLATS, f_w is calculated by assuming subbasin-scale spatial variability in water table depth z is driven solely by variations in the local soils-topographic index (STI) defined as

View Expanded

where a is area drained, T is soil transmissivity, and β is local slope. Variations in STI are related to variations in local water table depth (z) by

View Expanded

where the overbar in (8) and (9) represents spatial averaging of quantities within the watershed, and f is the vertical decay of saturated hydrologic conductivity. Changes in basin-averaged water table depth z are predicted through an areally lumped balance of deep soil drainage, upward diffusion from the water table, direct transpiration from the water table, and base flow (Peters-Lidard et al. 1997). Drainage and diffusion fluxes in unsaturated portions of the basin are calculated using a finite-difference numerical approximation to the Richards equation.

Since z < = 0 in (9) indicates surface saturation, the saturated fraction of the basin can be calculated as

View Expanded

where F is the cumulative density function for STI, typically determined from high-resolution soil and topographic maps. Runoff is modeled as a combination of saturation excess runoff (the product of rainfall intensity and f_w) and a separate parameterization of infiltration excess runoff. Base flow Q occurs solely from the saturated zone and is modeled as

View Expanded

Here, the statistical version of TOPLATS is used, which invokes a type of hydrologic similarity and assumes that all pixels with the same STI will have the same hydrologic response to atmospheric forcings (i.e., precipitation and potential evaporation). Subcatchment-scale variations in rainfall and micrometeorological forcings are neglected and separate TOPLATS calculations are run on discrete intervals of the STI distribution derived from high-resolution soil texture maps and a U.S. Geological Survey (USGS) digital elevation model (DEM). Basin-averaged hydrologic responses are calculated through the weighted averaging of results for each discrete STI interval.

c. Site location and model evaluation

All hydrologic modeling was based on application of TOPLATS to the Big Cabin Creek watershed in northeastern Oklahoma from 1 October 1996 to 30 September 1999. The watershed has very little hydrologic regulation or diversion and is located in a region that has seen extensive fieldwork aimed at the development of remote sensing retrieval algorithms for surface soil moisture. Upstream of the USGS gauging station near Big Cabin, Oklahoma, the watershed has an area of approximately 1170 km², which is roughly equivalent to the spatial resolution defined through the 3-dB antenna gain (Drusch et al. 1999) of current and next-generation spaceborne radiometers designed to retrieve surface soil moisture (Entekhabi et al. 2004). TOPLATS predictions within the basin were forced by meteorological observations (i.e., rainfall, air temperature, incoming radiation, wind speed, and relative humidity) from the Vinita, Oklahoma mesonet site located near the center of the basin. Soil hydraulic parameters were based on the dominant soil texture found with the basin (loam) and the soil texture/hydraulic parameter lookup table presented in Rawls et al. (1982). However, values of θ_sat and surface saturated hydrologic conductivity (K_sat) were tuned slightly (from 0.462 to 0.420 and 3.67 × 10⁻⁶ to 1.60 × 10⁻⁶ m s⁻¹, respectively) in order to match local soil moisture observations. Vegetation characteristics were varied on a monthly basis according the typical seasonality of the dominant land-cover type in the basin (grassland). Values for f and Q_o ( f = 3 and Q_o = 100 m³ s⁻¹) were obtained through manual calibration of TOPLATS against USGS streamflow observations.

Comparisons between calibrated TOPLATS results and observed streamflow and surface soil moisture are shown in Fig. 2a. Streamflow observations and predictions plotted in Fig. 2a are 30-day moving averages of daily values. TOPLATS soil moisture predictions plotted in Fig. 2b are for the interval of the STI that corresponds to local topography near the Vinita, Oklahoma mesonet site. Since Oklahoma mesonet soil moisture observations are not available prior to 2002, soil moisture intercomparisons are based on simulations conducted during a later time period (1 October 2002 to 30 September 2003). A good calibrated fit was achieved to both streamflow and surface soil moisture except for soil moisture predictions between December 2002 and April 2003. During this time period, TOPLATS systematically under predicts surface soil moisture by up to 0.10 cm³cm⁻³. This error is likely due to the misrepresentation of the basin-averaged water table depth (z) by TOPLATS and provides a good example of the impact on surface soil moisture dynamics of inaccurately modeling saturation-zone dynamics.

d. Modeling error approach

As illustrated in (7), surface soil moisture predictions in TOPLATS are driven by two key dynamic state variables: θ_unsat and f_w. Short-term fluctuations in θ_unsat are due to rainfall flux at the top of the soil column and the cumulative effects of surface evaporation, drainage, and diffusive fluxes between the surface zone and deeper soil moisture states. In contrast, f_w is based on deeper saturation-zone dynamics and the lateral redistribution of water. Consequently, random errors in precipitation, evaporation, infiltration, and saturation-zone recharge impact TOPLATS soil moisture predictions via contrasting processes acting at the top and bottom of the soil column. Here, three different error sources are considered. Model error at the bottom of the soil column is represented by adding mean-zero additive Gaussian noise to the catchment-averaged water table depth:

View Expanded

representing the net impact of model errors in base flow, upward diffusive fluxes, and recharge to the water table. Synthetically perturbed rainfall forcing is generated by multiplying actual rainfall by a mean-unity lognormal random variable:

View Expanded

Surface soil moisture errors between rainfall events are captured by the addition of random Gaussian error directly to θ_unsat:

View Expanded

which represent the net impact on soil moisture of modeling errors in bare soil evaporation, surface drainage, and the upward diffusive flux of soil water. Representing model error via some combination of noise applied to soil moisture states and rainfall model forcing is a common practice when applying the EnKF to the case of assimilating real observations into a hydrological model (e.g., Crow 2003; Reichle and Koster 2005). For simplicity all three perturbations (γ_z, γ_p, γ_θunsat) are modeled as mutually independent and temporally uncorrelated in time.

The experimental methodology (see section 2 and Fig. 1) is based on designating a single, unperturbed model realization as truth. A statistical representation of actual model error is specified by selecting “actual” values for the statistical parameters describing its components (w_z, w_p, and w_θunsat). Using these error parameters and (12)–(14), a TOPLATS open-loop simulation is generated that represents the uncorrected impact of actual model error on the accuracy of hydrologic predictions (see “modeled states” in Fig. 1). Errors present in the open-loop simulations are then filtered via the implementation of the EnKF to assimilate (on a daily basis) values of “actual soil moisture observations” generated from the original truth simulation (Fig. 1). The EnKF filtering methodology is based on an assumed statistical representation of actual modeling errors. Assumed values of w_z, w_p, and w_θunsat are used with (12)–(14) to randomly perturb θ_unsat, z, and rainfall and create an ensemble of model predictions that, in turn, is sampled to derive the state forecast error covariance information required by the Kalman filter update Eq. (3). As noted above, the statistical properties of assumed model error driving the ensemble generation may or may not accurately represent actual errors used to perturb the original truth simulations. The focus of the analysis will be on instances in which they do not.

a. Impact of incorrect model error assumptions

Adaptive tuning using becomes problematic if an incorrect assumption is made concerning the ultimate source of modeling error. Figure 5 displays the case where model error is actually caused by random perturbations to surface soil moisture θ_unsat (w_θunsat = 0.01 cm³ cm⁻³ h⁻¹) but, for the purposes of generating the EnKF for assimilating surface soil moisture, error is assumed to be caused by fluctuations in mean water table depth. Because the source of model error is misspecified when assimilating soil moisture, normalized rmse values for EnKF root-zone soil moisture predictions rise as assumed error in z is increased. However, observed levels decrease (toward unity) as assumed error in z increases. Consequently, adaptive tuning would increase w_z in an ill-advised attempt to lower toward unity (solid line in Fig. 5b). Such misguided tuning will severely reduce the rmse accuracy of 40-cm soil moisture predictions (solid line in Fig. 5a) produced via the assimilation of surface soil moisture.

Soil moisture assimilation results in Fig. 5 illustrate the impact of representing model error using random perturbations in z when, in reality, modeling error originates from the inaccurate representation of processes acting along the top of the soil column. Results in Fig. 5 are replotted as the solid line in Fig. 6a by graphing the y axis of Fig. 5b against the y axis of Fig. 5a for the entire range of assumed model error (the x axis in Fig. 5). Results in Fig. 6a are for the case of actual error in z (w_z = 0.025 m h⁻¹) and a range of assumed error magnitudes in z, θ_unsat, and rainfall. For simplicity, both actual and assumed model error are limited to a single source.

Figures 6b and 6c are analogous except for actual error in θ_unsat (w_θunsat = 0.01 cm³ cm⁻³ h⁻¹) and rainfall (w_p = 1), respectively. The vertical line indicates the location of the = 1 line that an adaptive filter would attempt to converge on via iterative modifications to assumed error parameters. Crosses mark the closest possible approach for the adaptive filter to this optimum value. Figure 6 highlights two potential obstacles to the successful implementation of an adaptive filter.

First, there exists the possibility of a spurious local minimum where, because of an incorrect assumption concerning the source of modeling error, the tuning of the wrong error type to move toward unity leads to progressively worse filter predictions. See, for example, the case of assumed error in z and rainfall in Fig. 6a where actual error is in θ_unsat. In both cases, tuning the wrong error source to produce better innovation statistics leads to a steady reduction in the accuracy of EnKF root-zone soil moisture predictions. The application of an adaptive filter in such cases will actually worsen root-zone soil moisture predictions relative to what is possible without data assimilation (i.e., a normalized error of one on the y axis in Fig. 6). This interpretation is supported by Table 1, which plots the rmse for optimal innovation statistics—the closest approach of curves in Fig. 6 to = 1 (indicated by crosses). If θ_unsat is correctly identified as the source of modeling error, adaptive tuning will lead to a relative reduction of 25% in modeling error. However, incorrectly choosing to tune innovations via adjustments to either z or rainfall error parameters results leads to EnKF predictions that are less accurate than their respective open-loop cases.

A potential solution to this problem is to perform a broad enough optimization during adaptive filtering such that the possibility of all three sources of model error is accounted for. That is, ensuring that the adaptive filter is able to find the model error type associated with globally optimal innovation statistics. However, even if a powerful enough optimization scheme could efficiently locate such a global minimum, a second undesirable possibility for adaptive filtering is that the best innovative statistics (for all three possible error-type assumptions) is associated with optimization of the wrong error type. In this case the best obtainable innovation statistics would not reflect the correct error representation. For instance, in Fig. 6b, the best 40-cm soil moisture rmse results are realized by tuning rainfall error parameters—the correct error source. However, better innovation statistics (but reduced root-zone soil moisture accuracies) arise when either z or θ_unsat error parameters are tuned. The parenthetical values in Table 1 represent the closest approach of to unity obtainable for each particular assumption concerning the source of modeling error. While the lowest rmse for 40-cm soil moisture is always obtained by assuming the correct error source, an incorrect error assumption can lead to as good, or better, innovation statistics.

b. Sensitivity to error magnitude

One concern is the potential sensitivity of results to rather arbitrary choices made concerning the magnitude of actual observing and modeling errors. To examine sensitivity to assumptions concerning the magnitude of observation and modeling errors, results were also generated (but not shown) for the cases of doubled error (from 2% to 4% volumetric) in daily surface-zone soil moisture retrievals and for the cases of baseline 2% soil moisture measurement error and both doubled and tripled values of modeling errors (i.e., w_θunsat, w_z, and w_p). As expected, increased observation errors tended to move normalized rmse for EnKF predictions toward unity as increased weight is shifted to model open-loop predictions. However, there was no qualitative change in the appearance or interpretation of results. Baseline choices for modeling errors used in this analysis led to rms open-loop errors in root-zone soil moisture predictions on the order of 0.5% to 3% volumetric (Fig. 3). Doubled and tripled modeling error led to slightly less sensitivity to the selection of assumed error source (i.e., column to column variations along a given row of Table 1 are reduced) but did not qualitatively change results.

c. Impact of diffuse error sources

Results in Figs. 3 –6 are all based on synthetic simulations where actual (and assumed) modeling error is intentionally restricted to a single source. In reality, land surface modeling errors will likely arise from a broad range of sources. Figure 7 plots results for the case where actual model error arises simultaneously from additive noise in z, θ_unsat, and multiplicative rainfall noise (w_z = 0.025 m h⁻¹, w_θunsat=0.01 cm³ cm⁻³ h⁻¹, and w_p = 1.0), while assumed error is restricted, in turn, to only one of these three error sources and tuned using . This is done to mimic the likely case in operational settings where assumed model error does not fully characterize the true complexity of actual modeling error(s). Calibration results in Fig. 7 are well behaved in the sense that adjusting error magnitudes to force mean normalized innovations closer to unity generally leads to lower normalized error. That is, the spurious local minimum problem noted in Fig. 6 is largely avoided. However, in a global sense, there remains the problem that using innovation statistics to choose the error type (and magnitude) that leads to the best innovation statistics does not always ensure the best 40-cm soil moisture predictions. Note that the error source whose tuning produces the best innovation statistics (surface soil moisture) actually leads to the poorest filter output (i.e., highest normalized rmse), while the error source whose optimized innovation mean is furthest from one (rainfall) produces the best 40-cm soil moisture results.

d. Innovation temporal correlation

In addition to the sampled mean of α_k, the temporal correlation coefficient of ν_k (ρ_ν) provides a diagnostic variable on which to base adjustments to assumed levels of model error. If the EnKF is operating in accordance with its underlying assumptions, including an accurate representation of model error, then the ν_k time series should be temporally uncorrelated (ρ_ν = 0). A correlated time series can be taken as evidence that model errors are being improperly represented in the filter. To determine if some of the adaptive filtering difficulties encountered in Fig. 6 and Table 1 can be addressed using this additional diagnostic, results in Table 1 were regenerated in Table 2 by tuning the EnKF so that the absolute value of ρ_ν was minimized. Intercomparison of results in Tables 1 and 2 reveals that calibrations based on ρ_ν (Table 2) and (Table 1) yield very similar results. In particular, it does not appear the ρ_ν-based calibration avoids the adaptive filtering problems noted in Table 1. For instance, as in the case of optimizing , tuning z errors (when the actual error source is noise in θ_unsat predictions) to minimize ρ_ν leads to root-zone soil moisture errors that exceed the open-loop case (i.e., normalized errors greater than one). In addition, for the case of actual error in rainfall, better ρ_ν statistics are obtainable via calibration of the wrong error parameters, w_z and w_θunsat, versus tuning of the rainfall error parameter. As a result, even a perfect global optimizing algorithm will be unable to converge on the error source associated with the best root-zone soil moisture filtering results. Overall, the close correspondence between results in Tables 1 and 2 implies that ρ_ν is too closely tied to to serve as a valuable independent diagnostic for accurately constraining hydrologic modeling errors.

e. Value of ancillary runoff observations

One potential solution for adaptive filtering problems is constraining EnKF results with additional observations. In addition to the assimilation of surface soil moisture, Fig. 5 also examines the case of assimilating runoff observations. Runoff magnitudes are assumed known from streamflow observations within a relative accuracy of 20% and the basin is assumed small enough such that the time lag between runoff generation and streamflow observations can be safely neglected. Actual error is assumed to be due to random noise in θ_unsat predictions and assumed error to variations in z. While increasing the magnitude of error in z improves the statistics for soil moisture innovations, it makes runoff innovations worse (i.e., moves the dashed line away from one in Fig. 5b). If streamflow observations are available, this incapability will provide a critical diagnostic that allows data assimilation systems to avoid the pitfall associated with adaptively tuning the wrong error type within a land surface model (section 3a). By strongly fluctuating z, the filter can (wrongly) induce sufficient variability in θ_sz such that θ_sz innovations will approach a mean of one. However, such excessive variations in z will also induce excessive background spread in the model-predicted runoff ensemble. This excessive spread is detectible in filter innovations if runoff observations are available and jointly considered.

The most natural way to consider runoff observations is to include them in the observation vector and jointly assimilate both runoff and surface soil moisture observations. Figure 8 replots Fig. 6 for the case of assimilating both surface soil moisture and surface runoff. As in Fig. 6, actual and assumed error is assumed to be restricted to a single error source, and plotted lines demonstrate the relationship between mean normalized innovations and normalized error for a range of assumed model error. In contrast to Fig. 6, the dual assimilation of both surface soil moisture and runoff appears to correct the false global minimum problem. That is, the best innovation statistics (i.e., the closest approach of to one) are associated with correct assumption concerning the type of model error. This is reiterated in Table 3. Note how, in contrast to Tables 1 and 2, the best-tuned innovation statistics are associated with the correct error assumption (and thus the lowest error in EnKF results).