1. Introduction
The need for accurate estimates of soil moisture conditions is well established, as it is important for a variety of science and applications. Soil moisture influences the partitioning of heat and moisture at the land–atmosphere interface (Cohen and Entekhabi 1999; Koster et al. 2004; Seneviratne et al. 2006) and in the redistribution of rainfall into infiltration and runoff. Root-zone soil moisture has been shown to influence subseasonal prediction of precipitation because of its persistent memory over longer time scales (Dirmeyer 2003). As a result, numerical weather prediction (NWP) and seasonal climate prediction models require accurate specification of soil moisture conditions for forecast initialization. In addition, estimates of moisture conditions are also required for supporting a variety of societal applications ranging from water resources, agricultural, and natural hazards management to military mobility and famine warning assessments (Engman 1991; Norbiato et al. 1996; Beck et al. 2000; Sheffield et al. 2004; Bolten et al. 2010; Entekhabi et al. 2010).
Because of the high spatial and temporal variability of soil moisture, long-term, consistent measurements of soil moisture are not typically available. Passive microwave radiometry has been used to generate estimates of near-surface soil moisture from a number of sensors in the past 30 years (Jackson 1993; Njoku and Entekhabi 1996), including the Scanning Multichannel Microwave Radiometer (SMMR; 1978–87), the Special Sensor Microwave Imager (SSM/I; since 1987), the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI; since 1997), the Advanced Microwave Scanning Radiometer for the Earth Observing System (AMSR-E; 2002–11), scatterometer-based products from European Remote Sensing Satellites 1 and 2 (ERS-1 and ERS-2; 1991–2006), and the Advanced Scatterometer (ASCAT; since 2007). These sensors provided estimates of soil moisture from X-band (~10 GHz) and C-band (~6 GHz) microwave radiometers. However, none of these sensors were specifically designed to measure soil moisture until the launch of the Soil Moisture Ocean Salinity (SMOS; since late 2009) from the European Space Agency (ESA), which provides global observations for soil moisture and salinity from an L-band radiometer. Compared to the X and C bands, the L-band-based measurements have reduced attenuation of the signal under moderate vegetation conditions and increased penetration depth for retrievals. The upcoming Soil Moisture Active Passive (SMAP) mission (Entekhabi et al. 2010) follows a similar approach by integrating an L-band radar and an L-band radiometer as a single observation system by combining the strengths of active and passive remote sensing. This approach enables the accuracy of radiometer-only retrievals, at a higher resolution facilitated by the radar. The SMAP L-band radiometers are expected to provide data products with a 1–3-day global revisit time at spatial resolutions of about 35 km.
A common approach used to objectively assess the potential benefit of satellite observations is through observing system simulation experiments (OSSEs). Given the great resources required to implement Earth Observing System (EOS) missions (NRC 2010), the National Aeronautics and Space Administration (NASA) and other agencies conduct OSSEs to estimate the value of proposed missions (Arnold and Dey 1986), and such experiments have been reported in preparation for SMAP (Crow et al. 2009; Konings 2009; Piles et al. 2009). A typical OSSE includes the following components: 1) a “nature” or a “truth” run, which is a free-running simulation of the physical model with high-quality inputs and without data assimilation; 2) simulated observations, which are generated from the truth run after incorporating realistic errors and limitations of the observing system; 3) an open-loop (OL) simulation that employs a set of lower-quality inputs with a different physical model and without data assimilation; and 4) a data assimilation (DA) integration that assimilates the simulated observations in the OL configuration. The DA and OL integrations are then compared against the known truth from the first component to evaluate the relative impact of simulated observations. Though a number of studies have examined the use of such “classic” OSSEs for assessing the worth of soil moisture measurements (Crow et al. 2001, 2005; Reichle et al. 2008; Kumar et al. 2009), they have focused primarily on quantifying the improvements to state variables such as root-zone soil moisture that are directly connected to surface soil moisture measurements. Few studies have focused on quantifying the downstream improvements in fluxes, runoff, or streamflow, or on coupled hydrometeorological prediction from soil moisture measurements, and fewer still have attempted to fully quantify economic benefit (Arnold and Dey 1986; Fritz et al. 2008). Nevertheless, as the justification of missions is often made based on broader societal applications, it is imperative that the OSSEs also capture the potential benefits of observations on actual end-use applications. Here, the end-use applications considered are droughts and floods.
The first contribution of this article is the development of an L-band OSSE to measure improvement in the estimation of the risk of drought and floods. Drought and floods are arguably the two most societally important hydrologic applications, impacting famine conditions, water availability, diseases, and wildfires, among others. Here, the impact of simulated L-band radiometer brightness temperature observations on drought and flood risk estimation is quantified. This work builds off of previous work making L-band OSSEs more meaningful to hydrology and hydrology-related applications. As L-band measurements are most sensitive to surface soil moisture, L-band OSSEs (e.g., Crow et al. 2001, 2005) initially were confined to surface soil moisture. With the assimilation of L-band measurements into land surface models (LSMs; e.g., Balsamo et al. 2006; Reichle et al. 2008), OSSEs then were extended to root-zone soil moisture that is of greater relevance to many applications. Here, we extend L-band OSSEs further by translating improvements obtained in soil moisture and runoff estimates to improvements in drought and flood risk estimation.
The second contribution of the article is the application and demonstration of decision-theory-based OSSE metrics that address weaknesses of convention metrics. The commonly reported metrics, which include RMSE and anomaly correlation, may not be suited for capturing impact to hydrological applications. For example, implicit in the reporting of RMSE is an equal penalty for overestimation and underestimation (Berger 1985). But for floods (and likely droughts), the loss associated with underestimating severity is not equivalent to that of overestimation (Krzysztofowicz 1998, 2010). Moreover, in applying RMSE, there is no tangible connection with drought or flood decision-making that is easy to communicate with mission decision makers. To address this, we demonstrate the application of a simple value-of-information (VOI) metric that draws from statistical decision theory. The VOI metric considers a range of important (albeit simplified) information, including the cost of actions to protect against drought and flood events, losses should the events occur, and their probability of occurrence. Here, a straightforward cost–loss model is applied (Murphy 1977; Katz and Murphy 1997), which is generic enough to use for a range of applications beyond droughts and floods. The basic approach demonstrated could also be tailored to more specific drought and flood decision-making contexts (e.g., water transfers and reservoir management), as demonstrated by Thornes and Stephenson (2001), Mullen and Buizza (2002), and Yuan et al. (2005) for weather forecasting; Pagowski and Grell (2006) for air quality forecasting; Roulin (2007) for flood forecasting; and McCollor and Stull (2008) for reservoir management. Such “decision theoretic” approaches have been explored for water resources management applications (James and Freeze 1993; Hobbs 1997; Harrison 2007; Wang and Harrison 2013).
The article is organized as follows. A brief description of the modeling system, the drought and flood risk assessment methodology, and the decision-theory-based economic model is presented in section 2. This is followed by a description of the experimental setup of the OSSE in section 3. Section 4 presents the results, and the major conclusions are presented in section 5.
2. Background
a. Land Information System
The OSSE is conducted using the capabilities of the NASA Land Information System (LIS), which is an earth science observation-driven hydrological modeling and data assimilation framework. LIS provides the modeling and computational capabilities to merge observations and model forecasts to generate spatially and temporally coherent estimates of land surface conditions. A schematic of the LIS framework is shown in Fig. 1, which includes a comprehensive suite of subsystems to support land data assimilation. The central part of the system is the LIS-LSM subsystem that includes several community LSMs and supports their application over user-specified domains and resolutions supported by the background data. The data assimilation (LIS-DA) subsystem supports multiple data assimilation algorithms that are focused on generating improved estimates of hydrologic model states (Kumar et al. 2008). More recently, the LIS-DA subsystem was enhanced through the incorporation of a suite of radiative transfer models (LIS-RTM) that enables the direct use of raw satellite observations for data assimilation. The optimization (LIS-OPT; Kumar et al. 2012) and uncertainty estimation (LIS-UE; Harrison et al. 2012) subsystems help in improving the representation of model parameters and for quantifying uncertainty in model predictions. Finally, during the period of LIS development, the ability to directly couple LSMs to a number of end-use application models has also been developed (LIS-APP) to enable the estimation of landslide forecasts, food security, mobility assessment, floods, and droughts, among others. The integrated modeling, multiscale resolution, ensemble capabilities, and algorithms for exploiting space-based observations make LIS an ideal platform for conducting OSSEs for hydrology missions. In this article, we employ these capabilities toward an end-to-end OSSE that connects the raw L-band soil moisture observations to applications of droughts and floods through various LIS subsystems (LIS-RTM, LIS-DA, LIS-LSM, and LIS-APP).
Illustration of the key LIS subsystems that enable the OSSE. LIS-LSM represents the land surface modeling subsystem; LIS-RTM represents the suite of radiative transfer models; LIS-DA represents the data assimilation infrastructure; LIS-OPT and LIS-UE represent the optimization and uncertainty estimation subsystems, respectively; and LIS-APP represents the collection of end-use application models.
Citation: Journal of Hydrometeorology 15, 6; 10.1175/JHM-D-13-0204.1
b. Forward microwave emission modeling
The rough surface reflectivity is derived from the smooth surface reflectivity rs, as a function of the roughness parameter h. Inputs of soil moisture, soil temperature, vegetation water content, and air temperature are provided from the LSM, and the default values provided in CMEM are used for the RTM parameters. As many RTM parameters in CMEM are defined as a function of vegetation [using the University of Maryland land cover map classification (Hansen et al. 2000)], we use the same land cover map in the LSM simulations to ensure consistency across the LSM and the RTM simulations.
c. Assessment of droughts and flood risks
Droughts are typically quantified through normalized indices that capture deficits of the water cycle variable of interest (e.g., precipitation, soil moisture, and runoff) from average conditions (Keyantash and Dracup 2002). Root-zone soil moisture percentile–based drought indices are often used to monitor agricultural drought (Mo 2008), as done in the North American Land Data Assimilation System (NLDAS) experimental drought monitor (Xia et al. 2014; Sheffield et al. 2012). Though the standard practice is to use such indices to measure droughts, they can also be used to quantify wetter-than-normal conditions (Seiler et al. 2002; Zhang et al. 2009) that lead to flood situations. In this article, we use root-zone soil moisture–based percentiles to quantify droughts and total runoff–based percentiles to quantify flood risks of varying severity.
There are numerous studies, using both synthetic and real soil moisture retrievals, that have shown benefits from soil moisture data assimilation (e.g., Drusch et al. 2005; Reichle et al. 2007; Kumar et al. 2008; Liu et al. 2011; Peters-Lidard et al. 2011; Draper et al. 2012). These studies demonstrate improvements not only in near-surface soil moisture fields, but also in fields that are connected to the observations through modeled processes, such as root-zone soil moisture (Kumar et al. 2009). All these studies, however, are focused on quantifying the improvements to the mean soil moisture fields from DA. As drought and flood risk assessments are derived based on the tails of the soil moisture and runoff distribution, the contribution of the soil moisture retrievals for improving the extremes of the distribution must be explicitly quantified, which is difficult to do in real DA systems because of the lack of sufficient, verifiable measurements of drought and flood risk assessments. In a more recent study, Kumar et al. (2014) examined the contribution of passive microwave soil moisture assimilation for improving drought estimation in the NLDAS system. This study employed an indirect assessment of the improvements in drought estimates from DA by comparing them against drought-area estimates from the U.S. Drought Monitor (USDM; Svoboda et al. 2002) and demonstrated that assimilation was effective in providing improvements in drought estimation at short time scales. The synthetic OSSE setup, on the other hand, provides the opportunity for direct quantification of the contribution of soil moisture retrievals toward improving the extremes of the soil moisture distribution. In the current study, we use the OSSE framework to examine the improvements in drought and flood risk assessments from the assimilation of L-band measurements.
d. Decision-theory model for economic assessment
A 2 × 2 contingency table used in the cost–loss model where C and L represent the costs and losses for a single event for different decisions. The number of events is given for TP (true positives), FP (false positives), FN (false negatives), and TN (true negatives).
Decision-theoretic metrics do not suffer from the weaknesses of conventional accuracy metrics that are typically reported in OSSEs. RMSE, for example, implicitly assumes a symmetric loss function (quadratic), for which the losses of underestimation are the same as those of overestimation (Berger 1985). However, in hydrologic applications, the loss function is generally believed to be asymmetric (Krzysztofowicz 2010), and this is particularly true for extreme events such as droughts and floods: viewing the events as binary in nature, the cost of a false dismissal far exceeds that of a false alarm. The application of the cost–loss model captures this asymmetry. Though cost–loss models have been applied in assessments of the value of probabilistic hydrometeorological forecasts (McCollor and Stull 2008; Palmer 2002; Richardson 2001), they have not been incorporated into remote sensing OSSEs to our knowledge.
The cost–loss model used here reflects the simplest of decision models, representing the decision as binary—mitigating against a protectable loss (or not) at a cost C—and the event as binary; that is, the event occurs (or not) at some probability p, incurring loss L (see Fig. 3). The cost incurred for every event is either C or L depending on whether the model integration correctly predicted the event or not. The cost–loss model could be applied in two different ways: 1) in a “deterministic” manner, where the ensemble-mean estimates from the OL and DA integrations can be used in a binary decision tree model (where p is always either 0 or 1), and 2) in a “probabilistic” model, where the p is diagnosed from the OL or DA ensemble. A separate estimate of drought and flood risk assessment is made based on each ensemble member, and the probability of a drought or flood-risk event is diagnosed by looking across the ensemble. The incurred cost is computed as the minimum of C and pL. In other words, the strategy will be to mitigate (and incur the cost of C) if p > C/L and to incur the loss if p < C/L. The total costs (CostOL and CostDA) are then computed by summing the incurred costs across all drought or flood risk events, and the value of information metric is computed as in Eq. (4).
3. Experimental setup
In this study, the model simulations are conducted on a domain over the continental United States (CONUS) at 35-km spatial resolution, which is the approximate resolution of the L-band observations from the SMAP radiometer. The structure of the OSSE (shown in Fig. 2) is as follows.
A soil moisture simulation is conducted with the Mosaic LSM, using the NLDAS-2 forcing data (Xia et al. 2012) as meteorological inputs to generate the assumed “true” state of the land surface. The forward τ–ω model described in section 2 is then used to generate truth L-band Tb,p for p = [H, V] values.
L-band Tb,p observations are then generated from this simulated truth by introducing realistic retrieval errors.
OL and DA integrations are conducted using the Noah LSM forced with Modern-Era Retrospective Analysis for Research and Applications (MERRA) land (Reichle et al. 2011) data over the same domain. The MERRA-Land data were time shifted by 7 days to purposely degrade the skill of the OL integration, similar to the strategy used in Reichle et al. (2008). In the DA integration, the Tb,p observations generated in step 2 are assimilated into the Noah LSM, using the OL configuration.
Flow diagram for the OSSE.
Citation: Journal of Hydrometeorology 15, 6; 10.1175/JHM-D-13-0204.1
The data assimilation integration employs a one-dimensional ensemble Kalman filter (EnKF) algorithm, which is a widely accepted technique for the sequential assimilation of hydrologic variables (Reichle et al. 2002; Reichle 2008) and has been used for the assimilation of soil moisture, skin temperature, and snow observations (Crow and Wood 2003; Reichle et al. 2007; Kumar et al. 2009; Reichle et al. 2010; De Lannoy et al. 2012; Liu et al. 2013). An ensemble size of 100 is used in the simulations, with perturbations applied to both meteorological fields and model prognostics fields to simulate uncertainty in the model estimates. The parameters used for the perturbations are listed in Table 2. Zero-mean, normally distributed additive perturbations are applied to the downward longwave radiation forcing, and lognormal multiplicative perturbations with a mean value of 1 are applied to the precipitation and downward shortwave radiation fields. The surface soil moisture layer in Noah is also perturbed with additive noise, as shown in Table 2. Note that most DA studies in literature use ensemble size on the order of 10–20 (Kumar et al. 2008), and here we use a larger ensemble size to ensure adequate sampling density in the probabilistic cost–loss model computations.
Parameters for perturbations to meteorological forcings and model prognostic variables in the EnKF assimilation experiments.
A set of preprocessing steps is applied to the synthetic retrievals generated from the Mosaic LSM and τ–ω forward model integration. To account for difficulties in retrieving soil moisture products from microwave sensors, the synthetic observations are masked out when the green vegetation fraction values exceed 0.7 and when snow or precipitation is present. Random Gaussian noise with an error standard deviation of 1.3 K is added to the Tb,p values to mimic measurement uncertainties. This error standard deviation is chosen as an estimate of the expected error level in the Tb retrievals from the upcoming SMAP spaceborne L-band radiometer (Chan et al. 2012).
Data assimilation methods including EnKF are only designed to correct random errors in the model background and assume that the model and observations are climatologically unbiased relative to each other. A standard practice used in soil moisture data assimilation studies is to scale the observations into the model climatology prior to DA. Here we adopt the a priori scaling method of Reichle and Koster (2004), where the observations are scaled to the model’s climatology so that the cumulative distribution functions (CDFs) of the observations and the model match for each grid point. The model CDF is generated from the simulated Tb values from the τ–ω model based on the Noah OL integration. The scaling is performed for the Tb values separately for each polarization.
To generate drought and flood risk estimates, percentile-based indices are generated using the root-zone soil moisture and total runoff values from the model integrations (Fig. 2). Root-zone soil moisture in this article is defined as the soil moisture content in the top 1 m of the soil column, derived as a suitably weighted vertical average over the model layers that are within the top 1 m of the soil column, and the total runoff is defined as the sum of the surface and baseflow runoff components. The percentiles are calculated as follows. The climatology of the variable (root-zone soil moisture or total runoff) for the calculation of percentiles is generated first, for each grid point. For each calendar day, the daily-averaged variable values across all 33 years are assembled. To improve the sampling density, a moving window of 5 days is employed. For example, 3 January climatology is assembled by using all the values from 1 to 5 January, across all years (leading to 5 × 33 = 165 values for each calendar day). Once the climatology for each day is assembled, the daily percentile values are computed by ranking each day’s root-zone soil moisture or total runoff estimate against the corresponding climatology.
From the time series of percentiles, estimates of drought and flood risk conditions of different intensities are generated, similar to the convention used by the National Drought Mitigation Center (NDMC) to produce drought estimates in the USDM (Svoboda et al. 2002). Drought intensity in the USDM is classified into five categories: D0 (abnormally dry, root-zone soil moisture percentile ≤ 30%), D1 (moderate drought, percentile ≤20%), D2 (severe drought, percentile ≤10%), D3 (extreme drought, percentile ≤5%), and D4 (exceptional drought, percentile ≤2%). In addition, we extend this definition to generate a similar set of estimates for flood risk conditions of five different intensities based on total runoff–based percentiles: F0 (total runoff percentile ≥70%), F1 (percentile ≥80%), F2 (percentile ≥90%), F3 (percentile ≥95%),percentile and F4 (percentile ≥98%). The OL and DA integration–based drought (D0–D4) and flood risk (F0–F4) estimates are compared against those from the nature run to quantify the impacts of L-band observations toward drought and flood risk assessments.
Finally, the cost–loss model described in section 2 is applied to compute the associated incurred costs from the truth run and OL and DA integration (Fig. 2). The value of information metric is then computed using Eq. (4), using both deterministic and probabilistic approaches. In the deterministic approach, the associated costs are computed based on the drought and flood risk assessments from the root-zone soil moisture percentiles generated from the ensemble-mean root soil moisture values at each grid point. In the probabilistic approach, the percentiles (and the drought and flood risk estimates) are computed separately for each ensemble member within a grid cell. The probability of drought or flood risk p is then computed by comparing each ensemble member within a grid cell to the corresponding truth estimate (Fig. 3). The incurred costs are then computed as described in section 2.
The decision tree used in the cost–loss model.
Citation: Journal of Hydrometeorology 15, 6; 10.1175/JHM-D-13-0204.1
4. Results and discussion
The results presented in this section focus first on the evaluation of the soil moisture and runoff fields from data assimilation. The comparison of the drought and flood risk percentiles is presented next, followed by a discussion of the value of information from L-band observations quantified through the cost–loss model.
a. Evaluation of soil moisture and runoff fields from data assimilation
The improvements in the surface soil moisture (SFSM), root-zone soil moisture (RZSM), and total runoff (TRF) fields are evaluated by comparing them against the corresponding fields from the nature run generated using the Mosaic LSM. Since the soil moisture and runoff climatologies of the two models differ, the anomaly time series correlation coefficient (instead of RMSE) is used to quantify the skill of the estimates. The anomaly time series is generated (for each grid point) first by subtracting the monthly-mean climatology of each dataset from the corresponding daily-average data, so that the anomalies represent the daily deviations from the mean seasonal cycle. The anomaly time series correlation coefficient (anomaly R) is then computed as the time series correlation coefficient between the daily anomaly estimates and the corresponding anomalies of the truth data.
Figure 4 presents the improvements in the SFSM, RZSM, and TRF fields from data assimilation. The improvements are computed by subtracting the anomaly-R values of the OL integration from the anomaly-R values of the DA integration, at each grid point. Thus, positive values of the anomaly-R differences indicate improvements from DA and negative values indicate degradations. Note that no temporal data masking is applied in the computation of the anomaly-R values. Figure 4 indicates that there are consistent improvements in the skill values SFSM, RZSM, and TRF fields with data assimilation, with more prominent improvements obtained in parts of the domain where sufficient retrievals are available. Generally, the improvements are more prominent in the soil moisture fields compared to those obtained in the runoff estimates. For surface soil moisture, the domain-averaged anomaly R for the OL integration is 0.42, and it improves to 0.51 with DA. Similarly, the domain-averaged root-zone anomaly-R values for the root-zone soil moisture improve from 0.50 in the OL integration to 0.56 in the DA integration. For total runoff, the domain-averaged anomaly-R values improve marginally to 0.110 (in the DA integration) from 0.095 (in the OL integration). The improvements in SFSM fields are generally higher than those obtained in the RZSM and TRF fields, consistent with the results in prior synthetic fraternal-twin experiment studies (Kumar et al. 2009, 2012).
Improvements in (top) SFSM, (middle) RZSM, and (bottom) TRF in terms of anomaly time series correlation coefficients. The improvements are computed by subtracting the anomaly-R values of the OL integration from that of the DA integration.
Citation: Journal of Hydrometeorology 15, 6; 10.1175/JHM-D-13-0204.1
b. Evaluation of drought and flood risk estimates
The root-zone soil moisture– and total runoff–based percentiles generated from the OL and DA integrations are compared against the corresponding nature-run-based percentiles, for the five different drought (D0–D4) and flood risk (F0–F4) categories. Note that the percentiles from the OL and DA integrations in these comparisons are generated using the ensemble-mean estimates. Figure 5 shows the domain-averaged RMSE, bias, and R for the D0–D4 and F0–F4 categories from the OL and DA integrations. The figure also shows the 95% confidence interval for each metric. In all comparisons, the DA-based estimates show systematic improvements over the OL-based estimates. The improvements are statistically significant in all cases, except in the R comparison for the D4, F2, F3, and F4 categories. The OL simulation overestimates the drought risk percentiles and underestimates the flood risk percentiles and DA reduces the bias errors in both cases. The skills of the flood risk assessments in terms of R are low compared to the corresponding drought assessments, presumably because we limit the use of L-band observations in the data assimilation system during large precipitation events. The skill of the model simulations (in terms of R) reduces for more severe drought and flood risk categories, but DA helps in improving the skills across all categories. The evaluation shown in Fig. 4 shows the skill improvement due to DA across the full dynamic range of modeled soil moisture and runoff, whereas the comparison shown in Fig. 5 is effectively an evaluation of the influence of DA toward improving the tails of the soil moisture and runoff distributions. The results show that DA is effective in producing improvements in the extremes of the soil moisture and runoff distribution, which is important for applications such as drought and flood risk estimation.
Comparison of domain-averaged RMSE, bias, and R for the percentiles of (left) drought and (right) flood risk from the OL and DA integrations for five different categories (D0–D4 and F0–F4, respectively). Error bars indicate the 95% confidence intervals for each metric.
Citation: Journal of Hydrometeorology 15, 6; 10.1175/JHM-D-13-0204.1
Figure 6 provides an evaluation of the influence of observations toward improving the representation of drought extent, duration, and onset over the modeling domain. Figure 6 (top) shows the RMSE of the percentage area under drought from the OL and DA against the truth run for each drought category. The errors in percentage area estimates are systematically reduced through data assimilation, although the improvements also reduce for more severe drought categories. The duration of a drought (in terms of number of days) is calculated for drought events that last for more than 1, 3, and 6 months in each drought category. For each drought event, the fraction of the duration that is correctly predicted in the OL and DA integrations is then computed. The average fraction of correctly predicted duration is shown in Fig. 6 (middle). The results show that DA contributes to systematically improving the estimates of drought duration, both for short-term and longer-term droughts. The improvements once again reduce for more severe drought categories. Finally, the improvements in estimating the onset of drought is computed by estimating the number of times the OL–DA integration is in agreement with the truth estimate of drought onset, and the comparison is shown in Fig. 6 (bottom). This metric essentially shows the probability of detection of drought onset from OL and DA integrations compared to the truth estimate. As the figure shows, assimilation systematically improves the probability of detection for the onset of drought, across the drought categories, and for droughts of different durations.
Comparison of the (top) drought extent (represented by percentage area under drought), (middle) drought duration (represented by the fraction of “true” drought duration correctly simulated), and (bottom) drought onset (represented by the probability of detection of drought onset).
Citation: Journal of Hydrometeorology 15, 6; 10.1175/JHM-D-13-0204.1
A comparison of the spatial distribution of drought and flood risk intensities is presented in Fig. 7 for three representative cases in years 1989, 2003, and 2011. In the August 1989 case, the OL-based estimate underestimates the intensity of drought over areas of the Midwest whereas DA improves these representations. In the July 2003 case, however, the OL overestimates the severity of drought in the High Plains and the Southeast, and DA helps to correct these high biases. Similarly, in the May 2011 case, the low biases in the OL drought estimates over Texas are improved by DA. Similar patterns of improvements can also be observed in the flood risk assessments, though they are generally smaller compared to the patterns of improvements in the drought estimates. This is consistent with the trends in Fig. 5, which show smaller improvements from DA over OL, as the flood risk assessments are based on the total runoff, whereas the drought estimates are based on root-zone soil moisture, a variable more directly connected to the surface soil moisture measurements being assimilated. In the August 1989 comparisons, OL-based flood risk assessments show underestimations over the lower Mississippi basin, and DA helps to reduce these errors. Generally, no major improvements in the flood risk assessments in the East and Southeast are seen (July 2003, for example) as the retrievals are often excluded in the data assimilation system over these areas. In all these cases, outside CONUS, the representations of drought and flood risks are improved over Mexico in the DA, but no added impact is observed over Canada. These artifacts are again directly related to the availability of L-band observations over these areas (as evident from the patterns of skill improvements seen in Fig. 4) and similar to prior studies (Reichle et al. 2007; Kumar et al. 2009, 2012).
Comparison of the drought and flood risk percentile maps from the (left) nature run, (middle) OL, and (right) DA for three representative cases.
Citation: Journal of Hydrometeorology 15, 6; 10.1175/JHM-D-13-0204.1
c. Analysis of the VOI of observation through the cost–loss model
The VOI metric [Eq. (4)] is computed using both deterministic and probabilistic approaches for different drought (D0–D4) and flood risk (F0–F4) categories. In each computation, the cost–loss (C/L) ratios are varied from 0.01 to 0.9 to determine the trade-off in VOI as a function of C/L ratio. Figure 8 (top) shows VOI as a function of C/L ratio from the deterministic cost–loss model. The behavior of VOI is similar for both applications, with the added value of observations reducing with increasing severity of drought (from D0 to D4) and flood risks (from F0 to F4). For example, in the deterministic drought example (Fig. 8, top left), DA leads to approximately 20% “cost savings” (VOI) for the D0 category for a C/L ratio of 0.01. The VOI reduces to approximately 8% for a C/L ratio of 0.9. The flood risk example (Fig. 8, top right) also shows a similar behavior, with the cost savings varying from approximately 9% to 4%.
VOI for the (left) drought and (right) flood risk assessments as a function of the C/L ratio, computed using the (top) ensemble mean and (bottom) full ensemble.
Citation: Journal of Hydrometeorology 15, 6; 10.1175/JHM-D-13-0204.1
Note that in all four plots of Fig. 8, the VOI values are always greater than 0, indicating that DA provides systematic improvements over OL in all drought and flood risk cases. The sensitivity of VOI to the C/L ratio is small, for any given drought or flood risk category, as indicated by the relatively flat lines in Fig. 8 (top). Generally, VOI reduces with increasing C/L ratio. Note that in this deterministic case, the differences in accumulated cost (for a particular drought or flood risk category) for different C/L ratios are purely from the reduction of NFN as a result of DA. Reduction of NFN events have a greater effect in overall cost savings for the lower C/L ratios since the losses are much higher than mitigation costs (for low C/L values).
Figure 8 (bottom) shows a similar trade-off in VOI to C/L ratio, but using the probabilistic model, where the probability of drought or flood risk is assessed from the ensemble. Generally, the VOI estimates are larger than those obtained from the deterministic model, indicating that the use of the ensemble-mean estimates (used in the deterministic model) leads to loss of valuable information that is especially important for capturing extreme events such as droughts and floods. Compared to the ensemble-mean-based VOI estimates, Fig. 8 (bottom) indicates that there is a trade-off in VOI to C/L ratio. The value of information is high for low values of C/L ratio, whereas the value of information tends to be low for the middle C/L ratio range. For the low C/L values, there is a greater reduction in the total costs from increasing NTP and reducing NFP through DA. For the high C/L ratios, the primary contributions to the cost savings are from reducing the number of NFN events through DA. These artifacts cannot be represented in the deterministic model where the NTP, NFP, and NFN are constant for all C/L ratio values for a given category, whereas the probabilistic model helps in improving this binary representation. The VOI trade-off behavior is similar for both applications, with the flood risk maps showing slightly increased VOI for low C/L ratios compared to the drought cases. These artifacts are again related to how the DA helps in improving the respective application (drought or flood risk) representation through improving the probability of detection and reducing the false alarms of various events.
5. Summary
OSSEs, typically conducted to assess the worth of existing data and data yet to be collected from proposed new missions, are often focused on quantifying the impact of observations on model states alone. To realize the significant real-world benefit and full potential of the missions, it is important that the OSSEs also capture the potential benefit of these observations on end-use applications. In addition, the results from the OSSEs must be evaluated with a variety of metrics that capture the value, uncertainty, and cost benefit of the observations while factoring in both science and societal impacts. In this article, we present a soil moisture OSSE that employs simulated L-band brightness temperature measurements toward improving drought and flood risk estimation. A decision-theory-based analysis is used to provide an assessment of the economic utility of the L-band observations toward these applications.
The OSSE presented in this manuscript uses a fraternal-twin experiment setup where two different LSMs are used to conduct the nature run (Mosaic LSM) and the OL and DA experiments (Noah LSM). A first-order τ–ω forward microwave emission model is used to simulate L-band brightness temperature observations. These observations are assimilated into a Noah LSM configuration using a 1D EnKF algorithm. The modeling domain roughly covers the continental United States for the time period of 1980–2012. Based on the simulated soil moisture and runoff fields, estimates of drought and flood risk conditions are generated using percentile-based indices. The associated incurred costs for the OL and DA integrations for the drought and flood risk assessments are then estimated using a cost–loss model, and a value of information metric is computed that estimates the contribution of L-band observations toward reducing the OL costs.
The results clearly demonstrate that the assimilation of L-band observations provides systematic improvements in the surface soil moisture, root-zone soil moisture, and total runoff estimates, though the improvements are larger in the surface and root-zone soil moisture estimates, as these variables are more directly connected to the surface soil moisture observations being assimilated. The improvements in root-zone soil moisture and total runoff also translate to systematic improvements in the drought and flood risk assessments, which are derived using the extremes of the root-zone soil moisture and total runoff distribution, respectively. The skill of the simulations reduce for more severe drought and flood risk categories, as their frequency is lower and timing more difficult to capture. Nevertheless, DA provides improvements in all drought and flood risk categories.
An assessment of the trade-off in VOI for various cost–loss ratios is estimated using deterministic and probabilistic cost–loss models. In the deterministic model, the ensemble-mean values based on the OL and DA integrations are used to estimate the drought and flood risk percentiles and to compute the associated incurred costs. In the probabilistic model, the full ensemble information from these integrations is used to estimate an ensemble of drought and flood risk assessments. The cost–loss model then computes a probability of drought or flood risk from this ensemble and uses it to compute the associated costs. The results indicate that the use of the ensemble mean (in the deterministic model) leads to loss of valuable information and underestimation of the contribution of observations. The probabilistic model also helps in capturing the improvements obtained because of increased probability of detection and reduced false alarms of drought and flood events through DA, some of which cannot be represented through the binary deterministic model. Note that the cost–loss model used here is very simple, with spatially and temporally homogeneous assumptions of cost and loss values. Given more knowledge of the practices for particular applications, the model could be improved.
Computer systems used for the development of OSSEs are evolving into more formal OSSE computational environments. Such systems are in various stages of development for atmospheric and oceanic OSSEs (Lee et al. 2010; Tanelli et al. 2012; Halliwell et al. 2014). Similarly the NASA LIS has evolved into a computational platform for terrestrial hydrology OSSEs. In addition to supporting the interoperable use of community land surface models in high-performance-computing environments, the more recent advancements in LIS have added a wide range of data assimilation algorithms (state, parameter, and Bayesian-based uncertainty estimation), forward radiative transfer models, and coupling to atmospheric and application models. The experiment presented in this article demonstrates the end-to-end capability of the LIS OSSE environment, linking simulated satellite observations to soil moisture–based drought and flood risk estimates, and, further with a simple decision model, to drought and flood risk decision-making under uncertainty.
Acknowledgments
We gratefully acknowledge the financial support from the NASA Earth Science Technology Office (ESTO) [Advanced Information Systems Technology (AIST) program Award AIST-11-0013]. Computing was supported by the resources at the NASA Center for Climate Simulation (NCCS). The NLDAS-2 forcing data used in this effort were acquired as part of the activities of NASA’s Science Mission Directorate and are archived and distributed by the Goddard Earth Sciences (GES) Data and Information Services Center (DISC). We are grateful to Dr. Gabrielle De Lannoy for assistance with the radiative transfer modeling components of this work.
REFERENCES
Arnold, C., and Dey C. , 1986: Observing-systems simulation experiments: Past, present, and future. Bull. Amer. Meteor. Soc., 67, 687–695, doi:10.1175/1520-0477(1986)067<0687:OSSEPP>2.0.CO;2.
Balsamo, G., Mahfouf J.-F. , and Deblonde G. , 2006: A global root-zone soil moisture analysis using simulated L-band brightness temperature in preparation for the Hydros satellite mission. J. Hydrometeor., 7, 1126–1146, doi:10.1175/JHM525.1.
Beck, L., Lobitz B. , and Wood B. , 2000: Remote sensing and human health: New sensors and new opportunities. Emerging Infect. Dis., 6 (3), 217–227. [Available online at http://wwwnc.cdc.gov/eid/article/6/3/00-0301_article.]
Berger, J., 1985: Statistical Decision Theory and Bayesian Analysis. 2nd ed. Springer-Verlag, 617 pp.
Bolten, J., Crow W. , Zhan X. , Jackson T. , and Reynolds C. , 2010: Evaluating the utility of remotely sensed soil moisture retrievals for operational agricultural drought monitoring. IEEE J. Sel. Topics Appl. Earth Obs. Remote Sens.,3, 57–66, doi:10.1109/JSTARS.2009.2037163.
Chan, S., Njoku E. , and Colliander A. , 2012: Level 1C radiometer data product (L1C_TB). Algorithm Theoretical Basis Doc., Jet Propulsion Laboratory, Pasadena, CA, 21 pp.
Cohen, J., and Entekhabi D. , 1999: Eurasian snow cover variability and Northern Hemisphere climate predictability. Geophys. Res. Lett., 26, 345–348, doi:10.1029/1998GL900321.
Crow, W., and Wood E. , 2003: The assimilation of remotely sensed soil brightness temperature imagery into a land surface model using ensemble Kalman filtering: A case study based on ESTAR measurements during SGP97. Adv. Water Resour., 26, 137–149, doi:10.1016/S0309-1708(02)00088-X.
Crow, W., Drusch M. , and Wood E. , 2001: An observation system simulation experiment for the impact of land surface heterogeneity on AMSR-E soil moisture retrieval. IEEE Trans. Geosci. Remote Sens., 39, 1622–1632, doi:10.1109/36.942540.
Crow, W., and Coauthors, 2005: An observing system simulation experiment for Hydros radiometer-only soil moisture products. IEEE Trans. Geosci. Remote Sens., 43, 1289–1303, doi:10.1109/TGRS.2005.845645.
Crow, W., Reichle R. , Koster R. , and Kimball J. , 2009: Land data assimilation activities in preparation of the NASA Soil Moisture Active Passive (SMAP) mission. Extended Abstracts, Fifth WMO Symp. on Data Assimilation, Melbourne, Australia, WMO, 16–17. [Available online at https://www.wmo.int/pages/prog/arep/wwrp/new/documents/WWRP_5_WMO_TD_No_1549.pdf.]
De Lannoy, G., Reichle R. , Arsenault K. , Houser P. , Kumar S. , Verhoest N. , and Pauwels V. , 2012: Multiscale assimilation of Advanced Microwave Scanning Radiometer–EOS snow water equivalent and Moderate Resolution Imaging Spectroradiometer snow cover fraction observations in northern Colorado. Water Resour. Res., 48, W01522, doi:10.1029/2011WR010588.
Dirmeyer, P., 2003: The role of the land surface background state in climate predictability. J. Hydrometeor., 4, 599–610, doi:10.1175/1525-7541(2003)004<0599:TROTLS>2.0.CO;2.
Draper, C., Reichle R. , De Lannoy G. , and Liu Q. , 2012: Assimilation of passive and active microwave soil moisture retrievals. Geophys. Res. Lett., 39, L04401, doi:10.1029/2011GL050655.
Drusch, M., Wood E. , and Gao H. , 2005: Observation operators for the direct assimilation of TRMM microwave imager retrieved soil moisture. Geophys. Res. Lett., 32, L15403, doi:10.1029/2005GL023623.
Drusch, M., Holmes T. , de Rosnay P. , and Balsamo G. , 2009: Comparing the ERA-40-based L-band brightness temperatures with Skylab observations: A calibration/validation study using the community microwave emission model. J. Hydrometeor., 10, 213–226, doi:10.1175/2008JHM964.1.
Engman, E., 1991: Applications of microwave remote sensing of soil moisture for water resources and agriculture. Remote Sens. Environ., 35, 213–226, doi:10.1016/0034-4257(91)90013-V.
Entekhabi, D., and Coauthors, 2010: The Soil Moisture Active Passive (SMAP) mission. Proc. IEEE, 98, 704–716, doi:10.1109/JPROC.2010.2043918.
Fritz, S., Scholes R. , Obsersteiner M. , Bouma J. , and Reyes B. , 2008: A conceptual framework for assessing the benefits of a global Earth observation system of systems. IEEE Syst. J., 2, 338–348, doi:10.1109/JSYST.2008.926688.
Halliwell, G. R., Jr., Srinivasan A. , Kourafalou V. , Yang H. , Willey D. , Le Hénaff M. , and Atlas R. , 2014: Rigorous evaluation of a fraternal twin ocean OSSE system for the open Gulf of Mexico. J. Atmos. Oceanic Technol., 31, 105–130, doi:10.1175/JTECH-D-13-00011.1.
Hansen, M., DeFries R. , Townshend J. , and Sohlberg R. , 2000: Global land cover classification at 1 km spatial resolution using a classification tree approach. Int. J. Remote Sens., 21, 1331–1364, doi:10.1080/014311600210209.
Harrison, K., 2007: Two-stage decision making under uncertainty and stochasticity. Adv. Water Resour., 30, 641–664, doi:10.1016/j.advwatres.2006.03.006.
Harrison, K., Kumar S. , Peters-Lidard C. , and Santanello J. , 2012: Quantifying the change in soil moisture modeling uncertainty from remote sensing observations using Bayesian inference techniques. Water Resour. Res., 48, W11514, doi:10.1029/2012WR012337.
Hobbs, B., 1997: Bayesian methods for analyzing climate change and water resources uncertainties. J. Environ. Manage., 49, 53–72, doi:10.1006/jema.1996.0116.
Holmes, T., Drusch M. , Wigneron J. , and Jeu R. , 2008: A global simulation of microwave emission: Error structures based on output from ECMWF’s operational integrated forecast system. IEEE Trans. Geosci. Remote Sens., 46, 846–856, doi:10.1109/TGRS.2007.914798.
Jackson, T. J., 1993: Measuring surface soil moisture using passive microwave remote sensing. Hydrol. Processes, 7, 139–152, doi:10.1002/hyp.3360070205.
James, B., and Freeze R. , 1993: The worth of data in predicting aquitard continuity in hydrogeological design. Water Resour. Res., 29, 2049–2065, doi:10.1029/93WR00547.
Katz, R., and Murphy A. , 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 240 pp.
Keyantash, J., and Dracup J. , 2002: The quantification of drought: An evaluation of drought indices. Bull. Amer. Meteor. Soc., 83, 1167–1180.
Konings, A., 2009: An observing system simulation experiment for soil moisture measurements from the SMAP radiometer. Ph.D. thesis, Massachusetts Institute of Technology, 61 pp. [Available online at http://hdl.handle.net/1721.1/70758.]
Koster, R., and Coauthors, 2004: Realistic initialization of land surface states: Impacts on subseasonal forecast skill. J. Hydrometeor., 5, 1049–1063, doi:10.1175/JHM-387.1.
Krzysztofowicz, R., 1998: Probabilistic hydrometeorological forecasts: Toward a new era in operational forecasting. Bull. Amer. Meteor. Soc., 79, 243–251, doi:10.1175/1520-0477(1998)079<0243:PHFTAN>2.0.CO;2.
Krzysztofowicz, R., 2010: Decision criteria, data fusion and prediction calibration: A Bayesian approach. Hydrol. Sci. J., 55, 1033–1050, doi:10.1080/02626667.2010.505894.
Kumar, S., Reichle R. , Peters-Lidard C. , Koster R. , Zhan X. , Crow W. , Eylander J. , and Houser P. , 2008: A land surface data assimilation framework using the Land Information System: Description and applications. Adv. Water Resour., 31, 1419–1432, doi:10.1016/j.advwatres.2008.01.013.
Kumar, S., Reichle R. , Koster R. , Crow W. , and Peters-Lidard C. , 2009: Role of subsurface physics in the assimilation of surface soil moisture observations. J. Hydrometeor., 10, 1534–1547, doi:10.1175/2009JHM1134.1.
Kumar, S., Reichle R. , Harrison K. , Peters-Lidard C. , Yatheendradas S. , and Santanello J. , 2012: A comparison of methods for a priori bias correction soil moisture data assimilation. Water Resour. Res., 48, W03515, doi:10.1029/2010WR010261.
Kumar, S., and Coauthors, 2014: Assimilation of remotely sensed soil moisture and snow depth retrievals for drought estimation. J. Hydrometeor., doi:10.1175/JHM-D-13-0132.1, in press.
Lee, M., Weidner R. , Qu Z. , Bowman K. , and Eldering A. , 2010: An atmospheric science Observing System Simulation Experiment (OSSE) environment. Earth Science Technology Forum, Arlington, VA, NASA, 7 pp. [Available online at http://trs-new.jpl.nasa.gov/dspace/bitstream/2014/41601/1/10-1804.pdf.]
Liu, Q., and Coauthors, 2011: The contributions of precipitation and soil moisture observations to the skill of soil moisture estimates in a land data assimilation system. J. Hydrometeor., 12, 750–765, doi:10.1175/JHM-D-10-05000.1.
Liu, Y., Peters-Lidard C. , Kumar S. , Foster J. , Shaw M. , Tian Y. , and Fall G. , 2013: Assimilating satellite-based snow depth and snow cover products for improving snow predictions in Alaska. Adv. Water Resour., 54, 208–227, doi:10.1016/j.advwatres.2013.02.005.
McCollor, D., and Stull R. , 2008: Hydrometeorological short-range ensemble forecasts in complex terrain. Part II: Economic evaluation. Wea. Forecasting, 23, 533–556, doi:10.1175/2008WAF2007063.1.
Mo, K., 2008: Model-based drought indices over the United States. J. Hydrometeor., 9, 1212–1230, doi:10.1175/2008JHM1002.1.
Mullen, S., and Buizza R. , 2002: The impact of horizontal resolution and ensemble size on probabilistic forecasts of precipitation by the ECMWF ensemble prediction system. Wea. Forecasting, 17, 173–191, doi:10.1175/1520-0434(2002)017<0173:TIOHRA>2.0.CO;2.
Murphy, A., 1977: The value of climatological, categorical and probabilistic forecasts in the cost–loss ratio situation. Mon. Wea. Rev., 105, 803–816, doi:10.1175/1520-0493(1977)105<0803:TVOCCA>2.0.CO;2.
Njoku, E. G., and Entekhabi D. , 1996: Passive microwave remote sensing of soil moisture. J. Hydrol., 184, 101–130, doi:10.1016/0022-1694(95)02970-2.
Norbiato, D., Borga M. , Esposti S. , Gaume E. , and Anquetin S. , 1996: Flash flood warning based on rainfall thresholds and soil moisture conditions: An assessment for gauged and ungauged basins. J. Hydrol., 362, 274–290, doi:10.1016/j.jhydrol.2008.08.023.
NRC, 2010: Controlling Cost Growth of NASA Earth and Space Science Missions. National Academies Press, 64 pp.
Pagowski, M., and Grell G. , 2006: Ensemble-based ozone forecasts: Skill and economic value. J. Geophys. Res.,111, D23S30, doi:10.1029/2006JD007124.
Palmer, T., 2002: The economic value of ensemble forecasts as a tool for risk assessment: From days to decades. Quart. J. Roy. Meteor. Soc., 128, 747–774, doi:10.1256/0035900021643593.
Peters-Lidard, C., Kumar S. , Mocko D. , and Tian Y. , 2011: Estimating evapotranspiration with land data assimilation systems. Hydrol. Processes, 25, 3979–3992, doi:10.1002/hyp.8387.
Piles, M., Entekhabi D. , and Camps A. , 2009: A change detection algorithm for retrieving high-resolution soil moisture from SMAP radar and radiometer observations. IEEE Trans. Geosci. Remote Sens., 47, 4125–4131, doi:10.1109/TGRS.2009.2022088.
Reichle, R., 2008: Data assimilation methods in the earth sciences. Adv. Water Resour., 31, 1411–1418, doi:10.1016/j.advwatres.2008.01.001.
Reichle, R., and Koster R. , 2004: Bias reduction in short records of satellite soil moisture. Geophys. Res. Lett.,31, L19501, doi:10.1029/2004GL020938.
Reichle, R., McLaughlin D. , and Entekhabi D. , 2002: Hydrologic data assimilation with the ensemble Kalman filter. Mon. Wea. Rev., 130, 103–114, doi:10.1175/1520-0493(2002)130<0103:HDAWTE>2.0.CO;2.
Reichle, R., Koster R. , Liu P. , Mahanama S. P. P. , Njoku E. G. , and Owe M. , 2007: Comparison and assimilation of global soil moisture retrievals from the Advanced Microwave Scanning Radiometer for the Earth Observing System (AMSR-E) and the Scanning Multichannel Microwave Radiometer (SMMR). J. Geophys. Res., 112, D09108, doi:10.1029/2006JD008033.
Reichle, R., Crow W. , Koster R. , Sharif H. , and Mahanama S. , 2008: Contribution of soil moisture retrievals to land data assimilation products. Geophys. Res. Lett.,35, L01404, doi:10.1029/2007GL031986.
Reichle, R., Kumar S. , Mahanama S. , Koster R. , and Liu Q. , 2010: Assimilation of satellite-derived skin temperature observations into land surface models. J. Hydrometeor., 11, 1103–1122, doi:10.1175/2010JHM1262.1.
Reichle, R., Koster R. , De Lannoy G. , Forman B. , Liu Q. , Mahanama S. , and Toure A. , 2011: Assessment and enhancement of MERRA land surface hydrology estimates. J. Climate, 24, 6322–6338, doi:10.1175/JCLI-D-10-05033.1.
Richardson, D., 2001: Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble size. Quart. J. Roy. Meteor. Soc., 127, 2473–2489, doi:10.1002/qj.49712757715.
Richardson, D., 2011: Economic value and skill. Forecast Verification: A Practitioner’s Guide in Atmospheric Science, 2nd ed. I. T. Jolliffe and D. B. Stephenson, Eds., John Wiley & Sons, 167–184, doi:10.1002/9781119960003.ch9.
Roulin, E., 2007: Skill and relative economic value of medium range hydrological ensemble predictions. Hydrol. Earth Syst. Sci., 11, 725–737, doi:10.5194/hess-11-725-2007.
Seiler, R., Hayes M. , and Bressan L. , 2002: Using the standardized precipitation index for flood risk monitoring. Int. J. Climatol., 22, 1365–1376, doi:10.1002/joc.799.
Seneviratne, S., Lüthi D. , Litschi M. , and Schär C. , 2006: Land–atmosphere coupling and climate change in Europe. Nature, 443, 205–209, doi:10.1038/nature05095.
Sheffield, J., Goteti G. , Wen F. , and Wood E. , 2004: A simulated soil moisture based drought analysis for the United States. J. Geophys. Res., 109, D24108, doi:10.1029/2004JD005182.
Sheffield, J., Xia Y. , Luo L. , Wood E. , Ek M. , and Mitchell K. , 2012: North American Land Data Assimilation System (NLDAS): A framework for merging model and satellite data for improved drought monitoring. Remote Sensing of Drought: Innovative Monitoring Approaches, B. Wardlow, M. Anderson, and J. Verdin, Eds., Taylor and Francis, 227–260.
Svoboda, M., and Coauthors, 2002: The Drought Monitor. Bull. Amer. Meteor. Soc., 83, 1181–1190.
Tanelli, S., and Coauthors, 2012: Integrated instrument simulator suites for earth science. Extended Abstracts, 2012 Asia–Pacific Remote Sensing, Kyoto, Japan, International Society for Optics and Photonics, Abstract 8529-13, 91. [Available online at http://spie.org/Documents/ConferencesExhibitions/AE12-Abstracts.pdf.]
Thornes, J., and Stephenson D. , 2001: How to judge the quality and value of weather forecast products. Meteor. Appl., 8, 307–314, doi:10.1017/S1350482701003061.
Wang, H., and Harrison K. , 2013: Bayesian approach to contaminant source characterization in water distribution systems: Adaptive sampling framework. Stochastic Environ. Res. Risk Assess., 27, 1921–1928, doi:10.1007/s00477-013-0727-9.
Xia, Y., Ek M. , Wei H. , and Meng J. , 2012: Comparative analysis of relationships between NLDAS-2 forcings and model outputs. Hydrol. Processes, 26, 467–474, doi:10.1002/hyp.8240.
Xia, Y., Ek M. , Peters-Lidard C. , Mocko D. , Svoboda M. , Sheffield J. , and Wood E. , 2014: Application of USDM statistics in NLDAS-2: Optimal blended NLDAS drought index over the continental United States. J. Geophys. Res. Atmos., 119, 2947–2965, doi:10.1002/2013JD020994.
Yuan, H., Mullen S. , Gao X. , Sorooshian S. , Du J. , and Juang H.-M. , 2005: Verification of probabilistic quantitative precipitation forecasts over the southwest United States during winter 2002/03 by the RSM ensemble system. Mon. Wea. Rev., 133, 279–294, doi:10.1175/MWR-2858.1.
Zhang, Q., Xu C.-Y. , and Zhang Z. , 2009: Observed changes of drought/wetness episodes in the Pearl River basin, China, using the standardized precipitation index and aridity index. Theor. Appl. Climatol., 98, 89–99, doi:10.1007/s00704-008-0095-4.