1. Introduction
Hydrologic models are used to represent the watershed processes that involve complex interactions between atmospheric, land surface, and subsurface components of the water cycle (Moradkhani et al. 2005). Hydrologic model parameters often represent physical characteristics of a watershed that are spatially heterogeneous and are difficult to measure in the field. Thus, parameter values need to be estimated through calibration against observed data.
While great efforts have been devoted to the development of calibration methods that aim to find the best set of parameter values over the past decade (Gill et al. 2006; Fenicia et al. 2007; Moussu et al. 2011; He et al. 2012; Kollat et al. 2012; Singh and Bárdossy 2012; Razavi and Tolson 2013; Asadzadeh et al. 2014; Assumaning and Chang 2014; Chang and Sayemuzzaman 2014; Ma et al. 2014; Li et al. 2015; Rahmani and Zarghami 2015; Wang et al. 2015c), uncertainty assessment of hydrologic model parameters has gained increasing attention in recent years. In fact, no hydrologist would rely on a single optimum combination of parameter values, since conceptual models make use of an empirical combination of mathematical equations to describe the main features of an idealized hydrologic cycle (Kuczera and Parent 1998). A variety of uncertainties are involved in modeling the rainfall–runoff transformation, and even a slightly different assumption of the measurement errors of the data would give a different optimum parameter set. It is thus necessary to assess parameter uncertainty resulting from calibration studies.
Tremendous efforts have been made in optimization and uncertainty analysis of hydrologic model parameters. For example, Muleta and Nicklow (2005) used a genetic algorithm coupled with the generalized likelihood uncertainty estimation (GLUE) method for estimating hydrologic model parameters and analyzing underlying uncertainties. Feyen et al. (2008) used an automatic Bayesian parameter inference algorithm based on the Markov chain Monte Carlo method to infer the posterior parameter distributions for the LISFLOOD model. Vrugt et al. (2008) proposed a differential evolution adaptive Metropolis algorithm for efficiently estimating the posterior probability density functions (PDFs) of hydrologic model parameters in high-dimensional and complex search problems. Juston et al. (2009) conducted a multiobjective calibration of a conceptual hydrologic model within the framework of an uncertainty analysis by using the Monte Carlo (MC)-based GLUE method. Sadegh and Vrugt (2014) proposed an approximate Bayesian computation coupled with the Markov chain Monte Carlo simulation method for efficiently exploring the parameter space and rapidly locating posterior solutions. These probabilistic methods are recognized as powerful tools to assess uncertainties in model parameters and predictions.
While much attention has been devoted to developing probabilistic approaches for parameter estimation and uncertainty assessment, relatively little attention has been given to the nonprobabilistic approaches in hydrologic studies. The nonprobabilistic approaches should be taken into account because much human reasoning about hydrologic systems is possibilistic rather than strictly probabilistic (Montanari 2007). Possibility theory, first introduced by Zadeh (1978) as an extension of fuzzy set theory, has been widely used to represent vague linguistic information (Guyonnet et al. 2003; Zhang et al. 2009a,b; Alvisi and Franchini 2011; Khan et al. 2013; Khan and Valeo 2015). Possibility theory describes partial belief in terms of possibility and necessity measures that can be seen as consonant plausibility and belief measures, respectively, as defined in the Dempster–Shafer theory of evidence (Shafer 1976; Jacquin 2010; Ferrero et al. 2013). Combining the strengths of possibility and probability theories would allow hydrologists to benefit from different types of information, either possibilistic or probabilistic (Langley 2000). Moreover, Seibert and McDonnell (2002, 2014) argued that the necessary dialog should occur between the experimentalist and the modeler to enable a more realistic process representation of catchment hydrology in conceptual rainfall–runoff models. The experimentalist is able to specify realistic parameter ranges based on his/her knowledge of catchment behavior in the model calibration process. However, the qualitative knowledge from the experimentalist cannot be characterized by exact numbers but it can be made useful through soft (fuzzy) measures. It is thus necessary to incorporate possibilistic (expert) knowledge into the parameter adjustment procedure for enhancing the understanding of the nature of the calibration problem, enabling more realistic simulations of catchment behavior.
In addition, few studies have been conducted to explore parametric interactions under multiple types of uncertainties and to quantify their contributions to the achievement of the optimum probability distributions through searching the parameter space. Furthermore, multiple hydrological metrics are of concern in many cases, and they are often correlated to some extent. For example, the Nash–Sutcliffe efficiency (NSE) and the predictive accuracy of peak flows are hydrological metrics that are closely correlated with each other. It is thus necessary to simultaneously examine inherent interactions among model parameters and potential correlations among hydrological metrics.
To address the aforementioned issues, the objective of this study is to propose a factorial possibilistic–probabilistic inference (FPI) framework for estimation of hydrologic model parameters and characterization of interactive uncertainties. A Monte Carlo–based fractional fuzzy–factorial analysis (MFA) method will also be introduced to identify the best parameter set and the underlying PDFs in a fuzzy probability space. A series of F tests coupled with their multivariate extensions will be conducted to characterize potential interactions among model parameters and among hydrological metrics in a systematic manner. The proposed methodology will be applied to a real-world problem to reveal mechanisms embedded within a number of hydrological complexities.
This paper is organized as follows. Section 2 introduces the proposed FPI framework for estimation of model parameters and characterization of interactive uncertainties. In section 3, the Xiangxi River watershed is selected to demonstrate the performance of FPI. Section 4 presents a detailed discussion on the proposed methodology. Finally, conclusions are drawn in section 5.
2. FPI framework
The proposed FPI framework takes into account both human reasoning and objective inference for uncertainty assessment of hydrologic model parameters and piecewise characterization of parametric interactions. A general overview of the steps involved in the FPI framework is provided as follows: 1) fuzzy random representation of hydrologic model parameters, 2) computation of fuzzy hydrologic parameters by using the α-cut technique, 3) construction of a multilevel factorial design matrix with various combinations of model parameters with varying PDFs, 4) generation of probabilistic hydrologic ensemble predictions, 5) identification of the best parameter set by robustly searching the parameter space, 6) examination of parametric interactions and their contributions to variability of hydrological metrics through factorial analysis of variance (ANOVA), and 7) detection of correlations among multiple hydrological metrics and their joint response to parameter changes through multivariate F tests. These steps involved in the FPI framework can be categorized into three parts: fuzzy random representation of model parameters, MFA, and multivariate probabilistic inference.
a. Fuzzy random representation of model parameters
Fuzzy set theory is a powerful means of capturing linguistic and ambiguous features of human knowledge. A fuzzy set 
When human reasoning and objective inference are taken into account simultaneously for parameter estimation, the theory of fuzzy random variables, first introduced by Kwakernaak (1978), can be used to deal with fuzziness and randomness. Randomness reveals the inherent stochastic nature of hydrologic parameters, while fuzziness represents imprecision or vagueness in subjective estimates. Fuzzy random variables can thus be interpreted as random variables with fuzzy probabilities (Wang et al. 2015b). For example, the mean and standard deviation of a cumulative distribution function (CDF) can be represented by fuzzy numbers instead of crisp numbers, leading to multiple uncertainties. Fuzzy numbers can serve as prior (expert) knowledge in determining the parameters of a CDF in the sense of Bayesian approach. Such a fuzzy random representation is useful for characterizing the stochastic phenomenon that is disturbed by imprecision and vagueness in realizations (Fu and Kapelan 2011; Wang et al. 2012).
b. MC-based fractional fuzzy–factorial analysis
To perform parameter estimation and uncertainty assessment under fuzziness and randomness, an MFA method is proposed to deal with fuzzy random variables. The fuzzy α-cut technique has been widely used for computing functions of fuzzy variables. The α-cut (α-level set) of a fuzzy set 
The α-cut levels are chosen subjectively, and they may vary while applying to different fuzzy variables. The larger the specified value of α, the smaller the uncertainty. For instance, certain fuzzy sets may be tackled with the α-cut level of 0.5, while the others are processed with the α-cut level of 1.0. Such a complexity needs to be addressed when dealing with fuzzy variables. Therefore, a factorial design is useful for taking into account various combinations of α-cut levels applied to fuzzy variables, revealing the potential correlations among fuzzy variables and enhancing the flexibility in practical problems.
The factorial design is a powerful statistical technique to measure model outputs by systematically varying model parameters (factors) with each having a discrete set of levels (Montgomery 2000; Wang and Huang 2015). In a factorial design, an experimental run is performed at every combination of levels of all factors. Generally, a multilevel factorial design has a sample size of sk, where k is the number of factors and s (s > 2) is the number of levels for each factor. By integrating the concept of the factorial design with the fuzzy α-cut technique, various combinations of α-cut levels can be taken into account while discretizing fuzzy sets. In this paper, we adopted the 32k factorial design that consisted of 2k fuzzy variables with each at three levels (three discrete values). This is because each of the k random variables is associated with a fuzzy mean and a fuzzy standard deviation, leading to 2k fuzzy variables. Moreover, two different α-cut levels of 0.2 and 1.0 were employed to discretize the triangular-shaped fuzzy set into three deterministic values.
Since each combination of factor levels in the 32k factorial design consists of means and standard deviations of the k random variables, the MC simulations can be conducted to generate the probability distributions of model outputs. The minimum and maximum values in terms of means and standard deviations of probability distributions can then be identified as interval solutions. The MC-based fuzzy factorial analysis is able not only to deal with fuzzy random variables by generating probabilistic model outputs in terms of mean and standard deviation with a degree of uncertainty (difference between maximum and minimum values), but also to examine interactions among fuzzy random variables and their effects on hydrological metrics. Nevertheless, the computational effort required for conducting the 32k factorial experiment would increase exponentially with the number of parameters. To reduce the computational effort, a (⅓)p fraction of the 32k design can thus be constructed with 2k factors in 32k−p simulation runs, where p < 2k. Such a design is called a 32k−p fractional factorial design, which is useful for examining main and low-order interaction effects of factors by using a fraction of the effort of a full factorial design (Wu and Hamada 2009; Wei et al. 2013). In this paper, a fractional factorial design was used in MFA to address fuzzy random variables and explore their interactions in a computationally efficient way.
c. Multivariate probabilistic inference
The F approximation is distributed with g(2q + g + 1) and g(2r + g + 1) degrees of freedom. Compared with the Wilks’s lambda, the Pillai’s trace is a more robust statistical criterion because it is less vulnerable to violations of the assumptions, such as adequate sample size or approximately equal cell sizes. The factorial ANOVA coupled with the multivariate probabilistic inference is useful for examining interactions among model parameters and among hydrological metrics in a systematic manner.
A flowchart of the proposed FPI framework is provided in Fig. 1, which summarizes the steps involved in the implementation of FPI.
Flowchart of the proposed FPI framework.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Flowchart of the proposed FPI framework.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Flowchart of the proposed FPI framework.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
3. Application
a. Description of the study system
The proposed methodology is applied to the Xiangxi River watershed (31°04′–31°34′N, 110°25′–111°06′E), which is the largest tributary of the Three Gorges Reservoir (TGR) in Hubei Province, China (Fig. 2). The Xiangxi River watershed originates in the Shennongjia Nature Reserve with a mainstream length of 94 km and a total area of 3099 km2. The watershed lies in the subtropical region and experiences a typical continental monsoon climate with substantial temperature variations in spring and concentrated rainfalls in summer; the weather is rainy in autumn and snowy in winter. Temperature and precipitation are also influenced by the mountainous topography of the watershed and vary significantly with altitude. The Xiangxi River watershed is characterized by a large difference in elevation. The highest elevation with up to 3088 m can be found near the source of the watershed in the Shennongjia mountains; the elevation reaches down to the value as low as 67 m at the outlet of the watershed, where the Xiangxi River discharges into the Yangtze River.
Geographical location and topographic characteristics of the Xiangxi River watershed.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Geographical location and topographic characteristics of the Xiangxi River watershed.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Geographical location and topographic characteristics of the Xiangxi River watershed.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Exploitation and construction in the Xiangxi River watershed are constrained by the Chinese government in order to provide the best conservation of natural resources. Thus, the distribution of land use and soil types in the watershed remains relatively stable over time. The land use can be categorized into five classes: forest, grassland, urban, farmland, and water body, among which forest is the most dominant land-cover type in the watershed. Limestone soils predominate in headwater areas whereas brown and yellow-brown soils widely occur in lowlands (Xu et al. 2011). The Xiangxi River is one of the most representative watersheds in the TGR region in terms of topographic properties, runoff volumes, and economic conditions (Han et al. 2014).
There are four weather stations located in the Xiangxi River watershed, including Zhengjiaping, Zhangguandian, Shuiyuesi, and Zhaojun. Meteorological observations from the four weather stations were obtained from the Meteorological Bureau of Xingshan County, including air temperature, precipitation, sunshine duration, wind speed, and relative humidity. There is one hydrometric station, Xingshan, located in the Xiangxi River watershed. Daily streamflow records at Xingshan gauging station were provided by the Hydrological Bureau of Xingshan County. In this study, a total of 4 years of data from January 1993 to December 1996 were used in hydrologic simulations such that the first year was used as a spinup period to reduce sensitivity to state-value initialization and the remaining 3 years were used as the calibration period. To test the credibility of the calibrated model, another 2 years of data from January 1997 to December 1998 were used as an independent set of samples for model validation.
The only source of uncertainty considered for this research is associated with the estimates of model parameters. Five parameters (Sm, Vdeg, β, Ts, and Tq) are thus taken into account and need to be calibrated and validated through observed data. For a conceptual rainfall–runoff model, the parameters that characterize the hydrological processes are often highly uncertain since they are impossible to be measured directly. The quality of uncertain information may not be satisfactory enough for representing hydrologic parameters in a single form, such as probability distributions or fuzzy sets. Thus, integration of the objective estimation of PDFs and the subjective judgment of fuzzy sets is desired to reflect the multilayer uncertain information for the estimation of hydrologic parameters. As shown in Table 1, the five model parameters of Sm, Vdeg, β, Ts, and Tq are considered to be random variables with fuzzy mean and fuzzy standard deviation, and their values are determined based on prior (expert) knowledge.
Random parameters with fuzzy mean and fuzzy standard deviation as well as the corresponding values under different α-cut levels.
b. Parameter estimation in a fuzzy probability space
To obtain the best parameter estimates under randomness and fuzziness, an MFA method within the FPI framework is first proposed to address fuzzy random variables, generating a number of daily streamflow time series in terms of mean and standard deviation with degrees of uncertainty for the Xiangxi River watershed. The degree of uncertainty is obtained by calculating the difference between maximum and minimum values of daily streamflows. MFA is then able to identify the best set of parameter values and its underlying probability distributions with the maximum value of NSE.
Since the five model parameters of Sm, Vdeg, β, Ts, and Tq were random, with each having a fuzzy mean and a fuzzy standard deviation, there were 10 factors to be taken into account in this case. These factors consisted of fuzzy means and fuzzy standard deviations of the five random variables; they were denoted by factors A–J for the purpose of illustration. The fuzzy sets with triangular membership functions were used, which could be discretized into three deterministic values under the α-cut levels of 0.2 and 1.0. To deal with the 10 fuzzy numbers, 310 combinations of means and standard deviations need to be addressed, resulting in a large computational effort. Thus, a 3(10−5) fractional factorial design was constructed in this study by using a fraction of all combinations of means and standard deviations. Since the 3(10−5) fractional factorial experiment was performed with one replication (repetition) in order to estimate the experimental error, a total of 486 ensembles of probabilistic streamflow time series were generated, with each composed with 200 members due to random sampling under each combination of model parameters with varying PDFs.
As a result, a cluster of 200 NSE values were obtained for each combination of PDFs in the 3(10−5) fractional factorial design, and a total of 486 clusters of NSE values were generated through exploring the parameter space. A maximum value of NSE can be obtained within each cluster, and then the highest value of NSE in the parameter space can be identified between clusters. Figure 3 presents the 486 clusters of NSE values, with each of them having 200 samples. The highest value of NSE is 0.7412, which is obtained from the 173th sample within the 257th cluster. Thus, the best set of parameter values and the associated probability distributions can be estimated based on the location of the highest NSE value. The statistical properties of probability distributions as estimated in a fuzzy environment are shown in Table 2. The results indicate the best combination of means and standard deviations of model parameters with different α-cut levels, which reveals that small deviations of PDFs can be obtained from MFA. This is because the search of parameter values by calibration is constrained by the specification of feasible parameter ranges based on expert knowledge of catchment behavior, leading to a reduction of parameter uncertainty. The best parameter set in the probability space is shown as Sm = 191.96 mm, Vdeg = 2.77, β = 0.57, Ts = 0.05 day−1, and Tq = 0.76 day−1. Figure 4 presents a comparison between simulated (by using the best parameter set) and observed daily streamflows in the Xiangxi River watershed for model calibration over a period of 3 years from January 1994 to December 1996 and for model validation over another period of 2 years from January 1997 to December 1998. The NSE coefficients obtained for calibration and validation are 0.7412 and 0.5373, respectively. The results show that most of the time the model is able to well characterize the hydrologic behavior; however, the peak flows are often not captured well by the model. It is thus necessary to explore potential interactions among model parameters and reveal their influences on hydrologic predictions.
Evolution of NSE values derived from 486 different factorial combinations (clusters) of PDFs; within each of them are 200 random samples. The color stays the same along the vertical axis because the range of variation of NSE is small within clusters, and thus NSE values all fall into the same color class. Variability across clusters is much larger than variability within clusters because different combinations of PDFs lead to a considerable variation in NSE values. The sample value at the intersection of two lines is the max NSE value obtained from the 173th sample within the 257th cluster.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Evolution of NSE values derived from 486 different factorial combinations (clusters) of PDFs; within each of them are 200 random samples. The color stays the same along the vertical axis because the range of variation of NSE is small within clusters, and thus NSE values all fall into the same color class. Variability across clusters is much larger than variability within clusters because different combinations of PDFs lead to a considerable variation in NSE values. The sample value at the intersection of two lines is the max NSE value obtained from the 173th sample within the 257th cluster.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Evolution of NSE values derived from 486 different factorial combinations (clusters) of PDFs; within each of them are 200 random samples. The color stays the same along the vertical axis because the range of variation of NSE is small within clusters, and thus NSE values all fall into the same color class. Variability across clusters is much larger than variability within clusters because different combinations of PDFs lead to a considerable variation in NSE values. The sample value at the intersection of two lines is the max NSE value obtained from the 173th sample within the 257th cluster.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Statistical properties of probability distributions of model parameters as estimated in a fuzzy environment. The value in the parenthesis represents the fuzzy α-cut level.
Comparison between simulated and observed daily streamflow time series for (a) model calibration over a period of 3 years from 1 Jan 1994 to 31 Dec 1996 and (b) model validation over a period of 2 years from 1 Jan 1997 to 31 Dec 1998.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Comparison between simulated and observed daily streamflow time series for (a) model calibration over a period of 3 years from 1 Jan 1994 to 31 Dec 1996 and (b) model validation over a period of 2 years from 1 Jan 1997 to 31 Dec 1998.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Comparison between simulated and observed daily streamflow time series for (a) model calibration over a period of 3 years from 1 Jan 1994 to 31 Dec 1996 and (b) model validation over a period of 2 years from 1 Jan 1997 to 31 Dec 1998.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
c. Piecewise characterization of interactive uncertainties
Variations in the values of model parameters can cause a difference in the generated streamflow time series, resulting in varied NSE coefficients. In addition to parameter estimation, MFA is also able to explore parametric interactions and to quantify their contributions to the variability of NSE. A multiway ANOVA F test was conducted to reveal the statistical significance of model parameters affecting NSE based on the results obtained from the 3(10−5) fractional factorial experiment. Figure 5 presents a normal probability plot of residuals for assessing how closely a set of residuals follows a normal distribution. This plot produces an approximately straight line, validating the assumption that residuals or error terms in ANOVA are normally distributed. The Shapiro–Wilks test was also performed as a rigorous statistical checking for normality of residuals. The result with the P value being greater than 0.05 verifies that residuals are normally distributed.
Normal probability plot of raw residuals.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Normal probability plot of raw residuals.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Normal probability plot of raw residuals.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
The 3(10−5) fractional factorial design is capable of making a clear test of individual factors and two-factor interactions. Figure 6 shows the standardized effect estimates (t values) for statistically significant model parameters and pairwise interactions. The I(L) is identified as the most significant factor affecting NSE, indicating that the mean of the random parameter Tq (fraction of water that flows through quick-flow reservoirs) has the strongest linear effect on NSE. The linear effect represents the difference of NSE between the low and high levels of the factor. The I(Q) is also one of the statistically significant factors. It reveals that the mean of Tq also has a large quadratic effect on NSE. The quadratic effect represents the difference of NSE between the medium level and the average of low and high levels of the factor. Any change in the mean of Tq would cause a considerable variation in NSE. This is because Tq reflects the residence time for water in the quick-flow reservoirs, which plays a dominant role in predicting the high streamflows when heavy rainfall occurs and the accuracy of high streamflow predictions are closely related to NSE. As for pairwise interactions, the E × I interaction has the largest contribution to the variability of NSE, revealing that the mean of the random parameter β (factor distributing flow to the quick-flow reservoir) is highly correlated with the mean of Tq, and their interaction has a significant impact on NSE.
Standardized effect estimates for statistically significant model parameters and pairwise interactions. On the y axis, L and Q denote linear and quadratic effects, respectively.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Standardized effect estimates for statistically significant model parameters and pairwise interactions. On the y axis, L and Q denote linear and quadratic effects, respectively.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Standardized effect estimates for statistically significant model parameters and pairwise interactions. On the y axis, L and Q denote linear and quadratic effects, respectively.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
To visualize the relative magnitude of the effects of model parameters on NSE, Fig. 7 presents their individual effects. It can be seen that the mean of Tq has the largest positive effect on NSE with a significant increase across low, medium, and high levels of Tq. This implies that increasing the mean of Tq causes the shorter residence time for water in the quick-flow reservoirs and thus a more immediate effect of precipitation, leading to the larger NSE coefficient. In comparison, the standard deviation of Tq has significantly less contribution to the variation of NSE. The F test was also conducted to quantify the linear and quadratic effects of model parameters. Since E (mean of β), G (mean of Ts), and I (mean of Tq) are identified as the three most significant factors affecting NSE, it is necessary to further explore their potential interactions for finding more meaningful information. As shown in Fig. 8, the marginal means of NSE are estimated to represent the interaction patterns between factors E, G, and I. For example, Fig. 8a reveals a dramatic difference in the variation of NSE associated with the three levels of factor I over the levels of factor E, collapsed across the levels of factor G, implying that the E × I interaction effects on NSE vary depending on the levels of factor G. The maximum NSE value would be obtained when factors E and G are at their medium levels and factor I is at its high level. As shown in Fig. 8b, when the level of factor E is high, factor G has a positive effect on NSE at the low and medium levels of factor I, while its effect becomes negative when factor I is at its high level. The findings indicate that the parametric interactions in a fuzzy stochastic environment are complex, and the magnitude and direction of interaction effects vary in different regions of the parameter space.
Main effects of model parameters with the significance level of 0.05 used in hypothesis testing.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Main effects of model parameters with the significance level of 0.05 used in hypothesis testing.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Main effects of model parameters with the significance level of 0.05 used in hypothesis testing.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Marginal means of NSE with 95% confidence intervals for factors E [β (mean)], G [Ts (mean)], and I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Marginal means of NSE with 95% confidence intervals for factors E [β (mean)], G [Ts (mean)], and I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Marginal means of NSE with 95% confidence intervals for factors E [β (mean)], G [Ts (mean)], and I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Figure 9 represents the curvature effects on NSE for the pairwise interactions between the means of random parameters β, Ts, and Tq, which provide a general picture of the relationships between settings of correlated parameters and the resulting NSE. It can be seen that NSE rises while simultaneously increasing the means of β and Tq or Ts and Tq, indicating that the interactions between the flow partitioning factor and the fraction of water flowing through quick-flow reservoirs as well as between the fraction of water flowing from the slow-flow reservoir into the river and the fraction of water flowing through quick-flow reservoirs have a positive influence on NSE. As for the interaction between the means of β and Ts, there is an upward trend in NSE with a decreasing mean of β and an increasing mean of Ts. Quantification of the relationship between parametric interactions and the resulting NSE plays an important role in locating the regions of attraction in the parameter space. The proposed approach would be applied to more parameterized and complex hydrologic models in future studies, which is particularly useful for systematically exploring high-dimensional space.
Fitted surfaces of NSE for pairwise interactions between factors E [β (mean)], G [Ts (mean)], and I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Fitted surfaces of NSE for pairwise interactions between factors E [β (mean)], G [Ts (mean)], and I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Fitted surfaces of NSE for pairwise interactions between factors E [β (mean)], G [Ts (mean)], and I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
d. Hypothesis testing on multiple hydrological metrics
Order-of-magnitude differences often exist between simulated and observed streamflows since complex hydrologic systems cannot be perfectly modeled with mathematical equations, especially for the peak flows that are often not captured well by the model. It is thus necessary to examine the statistical significance of model parameters influencing the differences between simulated and observed peak flows (i.e., prediction errors of peak flows). For the purpose of illustration, one year of data was used to conduct the analysis of errors in predicting peak flows. Figure 10 presents the pattern of prediction errors of peak flows that occur on days 185 (3 July 1996), 186 (4 July 1996), 263 (19 September 1996), and 312 (7 November 1996) under 486 combinations of parameter settings based on the 3(10−5) factorial design with one replication. Generally, the model overpredicts the peak flows on days 185 and 186 in the summertime, while it underpredicts the peak flows on days 263 and 312. Figure 11 reveals that the quadratic effect of the mean of Tq, denoted by I(Q), has the most significant contribution to the variability in prediction errors of peak flows that occur on days 185 and 186, while its linear effect, denoted by I(L), contributes most to the variability in prediction errors of peak flows that occur on days 263 and 312. Figure 12 shows the temporal variation in individual effects of the most sensitive factor I across its low, medium, and high levels. As for parameter correlations, Fig. 13 shows the fitted surfaces of errors in predicting peak flows for the most significant interactions identified based on estimated standardized effects. It can be seen that there is an upward trend in the prediction errors of peak flows that occur on days 185 and 186 while simultaneously increasing or decreasing the means of Ts and Tq. For days 263 and 312, the prediction errors rise along with an increasing mean of β and a decreasing mean of Tq. These findings indicate that the dominant interaction effects on the predictive accuracy of peak flows vary temporally because of the dynamic behavior of hydrologic systems.
Pattern of differences between simulated and observed peak flows under different combinations of parameter settings.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Pattern of differences between simulated and observed peak flows under different combinations of parameter settings.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Pattern of differences between simulated and observed peak flows under different combinations of parameter settings.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Probability plots of the eight most significant effects on prediction errors of peak flows.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Probability plots of the eight most significant effects on prediction errors of peak flows.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Probability plots of the eight most significant effects on prediction errors of peak flows.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Temporal variation in single effects of the most sensitive factor I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Temporal variation in single effects of the most sensitive factor I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Temporal variation in single effects of the most sensitive factor I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Fitted surfaces of errors in predicting peak flows for the most significant interactions between factors E [β (mean)], G [Ts (mean)], and I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Fitted surfaces of errors in predicting peak flows for the most significant interactions between factors E [β (mean)], G [Ts (mean)], and I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Fitted surfaces of errors in predicting peak flows for the most significant interactions between factors E [β (mean)], G [Ts (mean)], and I [Tq (mean)].
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Figure 14 presents a scatterplot matrix that depicts pairwise relations between NSE and prediction errors of peak flows. The pattern of the correlation between NSE and the prediction errors that occur on day 185 seems to be similar to that of the correlation between NSE and the prediction errors on day 186, while the correlations between NSE and the prediction errors on day 263 and between NSE and the prediction errors on day 312 show similar patterns. In general, NSE rises while decreasing the prediction errors of peak flows until NSE reaches the highest value. It can also be seen that there are positive correlations between the prediction errors that occur on days 185 and 186 as well as between the prediction errors on days 263 and 312, indicating that a strong positive correlation exists among the prediction errors of peak flows that occur successively within a short period of time; such a positive correlation would be weakened over time. These findings verify that certain correlations exist between NSE and the predictive accuracy of peak flows. It is thus desired to combine the correlated hydrological metrics into a single variate, and then assess the impacts of model parameters on not only individual hydrological metrics but also multiple hydrological metrics collectively.
Assessment of relationships between NSE and prediction errors of peak flows. All variable names are shown in the diagonal of the scatterplot matrix, and each block is a scatterplot of the data obtained from the variable in the column by the variable in the row.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Assessment of relationships between NSE and prediction errors of peak flows. All variable names are shown in the diagonal of the scatterplot matrix, and each block is a scatterplot of the data obtained from the variable in the column by the variable in the row.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Assessment of relationships between NSE and prediction errors of peak flows. All variable names are shown in the diagonal of the scatterplot matrix, and each block is a scatterplot of the data obtained from the variable in the column by the variable in the row.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
To perform an overall analysis of multiple hydrological metrics, the multivariate F tests of Wilks’s lambda and Pillai’s trace were conducted to quantify the effects of model parameters on the composite of NSE and prediction errors of peak flows. As shown in Fig. 15, the most influential parameters and their interactions are identified according to the results of the Wilks’s lambda and the Pillai’s trace tests. The mean of Tq and its interaction with the mean of β have the most significant single and joint effects, respectively. These findings are in good agreement with those obtained through performing F tests. Nevertheless, there would be a difference between the statistical significance of several interaction effects estimated through testing individual hydrological metrics and those from an overall test across multiple hydrological metrics, although the multivariate F tests identify the same set of sensitive parameters as separate F tests. For example, the C × E interaction contributes more than the A × C interaction to the variability of NSE according to the results of the F test (see Fig. 6), while the effect of the A × C interaction with the Wilks’s lambda of 0.37 is more significant than the effect of the C × E interaction with the Wilks’s lambda of 0.46 on the combination of NSE and prediction errors of peak flows.
Identification of the five most influential model parameters and pairwise interactions affecting the combination of NSE and prediction errors of peak flows.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Identification of the five most influential model parameters and pairwise interactions affecting the combination of NSE and prediction errors of peak flows.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
Identification of the five most influential model parameters and pairwise interactions affecting the combination of NSE and prediction errors of peak flows.
Citation: Journal of Hydrometeorology 17, 4; 10.1175/JHM-D-15-0131.1
4. Discussion
The multiuncertainty characterization used in the proposed FPI framework is significantly distinct from the classic probabilistic approaches for parameter estimation. There are two layers of uncertainty in the form of a PDF with fuzzy mean and fuzzy standard deviation. When the degree of precision in the probabilistic approach is difficult to justify for characterizing uncertainty in model parameters, the possibilistic approach can be used to reflect the variations in PDF parameters. The imprecision inherent in expert knowledge can be represented by fuzzy sets, which can capture the linguistic and ambiguous features of human knowledge. Thus, the simultaneous consideration of randomness and fuzziness enables the objective inference and the subjective judgment for a realistic detection on parameter estimation. It also facilitates a thorough exploration of the reasonable parameter space with the aid of the MFA method, generating a number of clusters of NSE values. The highest value of NSE can then be identified in an effective and efficient way.
MFA is also capable of quantifying parametric interactions and their contributions to the variabilities of NSE and prediction errors of peak flows. The statistical significance for each of their linear, nonlinear, and interaction effects was revealed through performing the multiway ANOVA F tests, which provides hydrologists with meaningful information about the regions of attraction in the parameter space. Nevertheless, when multiple hydrological metrics are evaluated through a series of separate F tests, the type I error rate (probability of rejecting the null hypothesis when it should be accepted) rises quickly with an increase of the number of hypothesis tests. For example, assuming that five hydrological metrics are evaluated by performing separate F tests, each time 0.05 is used as the significance level. The probability of the type I error lies somewhere between 5% across the five separate tests if all hydrological metrics are perfectly correlated and 23% (i.e., 1 − 0.955) if all hydrological metrics are uncorrelated. Thus, the multivariate probabilistic inference should be conducted to provide a single overall hypothesis test when some degree of correlation is present among hydrological metrics, which is effective in controlling the inflation of the type I error rate. Consequently, this study examined the correlations between NSE and prediction errors of peak flows. The multivariate F tests of Wilks’s lambda and Pillai’s trace were conducted to quantify the effects of model parameters on the combination of NSE and prediction errors of peak flows. The ANOVA F tests coupled with the multivariate probabilistic inference are useful for quantitatively characterizing interactions among model parameters and among hydrological metrics in a systematic manner.
The proposed FPI framework is also applicable to more parameterized and complex models. However, the computational complexity would greatly increase with an increasing number of fuzzy random variables. The concept of fractional factorial design introduced in this paper is useful to dramatically reduce the computational effort required for parameter estimation and interaction detection. The proposed MFA is one of many alternatives to address fuzzy random variables, and thus future studies need to be undertaken to develop more computationally efficient approaches for estimation of model parameters and examination of parametric interactions under fuzziness and randomness.
The FPI framework was developed based on the traditional assumption that model errors follow a Gaussian distribution with zero mean and a constant variance. However, such an assumption may be violated in hydrologic applications because hydrologic model errors are always correlated, heteroscedastic, and non-Gaussian (Schoups and Vrugt 2010; Pianosi and Raso 2012), resulting in a limitation of the proposed approach. It is thus necessary to remove the traditional assumption of Gaussian errors in future studies.
5. Conclusions
An FPI framework was proposed in this paper for robustly estimating model parameters and characterizing interactive uncertainties. To address multiple uncertainties in the form of fuzzy random variables, an MFA method was introduced to generate probabilistic streamflow time series for the Xiangxi River watershed, China. Since fuzzy numbers served as prior (expert) knowledge in determining the parameters of a CDF in the sense of Bayesian approach, such a fuzzy random representation was useful for characterizing the stochastic phenomenon that was disturbed by imprecision and vagueness in realizations. MFA was then able to identify the best combination of parameter values, together with the underlying probability distributions.
The proposed methodology was applied to the Xiangxi River watershed by using the HYMOD conceptual hydrologic model to demonstrate its validity and applicability. The results reveal that small deviations of PDFs can be derived from MFA since the search of parameter values is constrained by the specification of feasible parameter ranges based on expert knowledge of catchment behavior, leading to a reduction of parameter uncertainty. Human reasoning should be taken into account in model calibration to achieve realistic simulations of catchment behavior. MFA is capable of deriving the probability distributions of model parameters under different fuzzy α-cut levels, which captures both possibilistic and probabilistic information in the calibration process. A series of multiway ANOVA F tests were also conducted to reveal the statistical significance of parametric interactions affecting NSE and the predictive accuracy of peak flows. The findings indicate that the parametric interactions affecting NSE are complex in a fuzzy stochastic environment, and the magnitude and directions of interaction effects vary in different regions of the parameter space. As for the predictive accuracy of peak flows, the findings reveal that the significant interaction effects vary temporally because of the dynamic behavior of hydrologic systems. Quantification of the relationships between parametric interactions, the predictive accuracy of peak flows, and the resulting NSE is useful for efficiently exploring the regions of attraction in the parameter space.
Since there was a direct correlation between NSE and the predictive accuracy of peak flows, the multivariate F tests of Wilks’s lambda and Pillai’s trace were also conducted to examine their correlations and quantify the effects of parametric interactions on the combination of NSE and the predictive accuracy of peak flows. The findings indicate that there would be a difference between the statistical significance of several interaction effects estimated through separate F tests and those obtained from the multivariate F tests. The F tests coupled with their multivariate extensions are useful for quantitatively characterizing interactions among model parameters and among hydrological metrics in a systematic manner.
Acknowledgments
This research was supported by the Major Project Program of the Natural Sciences Foundation (51190095), the National Natural Science Foundation (51225904), and the Natural Sciences and Engineering Research Council of Canada. The authors would like to express thanks to the editor and three anonymous reviewers for their constructive comments and suggestions.
REFERENCES
Alvisi, S., and Franchini M. , 2011: Fuzzy neural networks for water level and discharge forecasting with uncertainty. Environ. Modell. Software, 26, 523–537, doi:10.1016/j.envsoft.2010.10.016.
Asadzadeh, M., Tolson B. A. , and Burn D. H. , 2014: A new selection metric for multiobjective hydrologic model calibration. Water Resour. Res., 50, 7082–7099, doi:10.1002/2013WR014970.
Assumaning, G. A., and Chang S. Y. , 2014: State and parameter estimation in three-dimensional subsurface contaminant transport modeling using Kalman filter coupled with Monte Carlo sampling. J. Environ. Inf., 24, 80–89, doi:10.3808/jei.201400280.
Bulygina, N., and Gupta H. , 2011: Correcting the mathematical structure of a hydrological model via Bayesian data assimilation. Water Resour. Res., 47, W05514, doi:10.1029/2010WR009614.
Chang, S. Y., and Sayemuzzaman M. , 2014: Using unscented Kalman filter in subsurface contaminant transport models. J. Environ. Inf., 23, 14–22, doi:10.3808/jei.201400253.
Fenicia, F., Savenije H. H. G. , Matgen P. , and Pfister L. , 2007: A comparison of alternative multiobjective calibration strategies for hydrological modeling. Water Resour. Res., 43, W03434, doi:10.1029/2006WR005098.
Ferrero, A., Prioli M. , and Salicone S. , 2013: Processing dependent systematic contributions to measurement uncertainty. IEEE Trans. Instrum. Meas., 62, 720–731, doi:10.1109/TIM.2013.2240097.
Feyen, L., Kalas M. , and Vrugt J. A. , 2008: Semi-distributed parameter optimization and uncertainty assessment for large-scale streamflow simulation using global optimization. Hydrol. Sci. J., 53, 293–308, doi:10.1623/hysj.53.2.293.
Fu, G., and Kapelan Z. , 2011: Fuzzy probabilistic design of water distribution networks. Water Resour. Res., 47, W05538, doi:10.1029/2010WR009739.
Gill, M. K., Kaheil Y. H. , Khalil A. , McKee M. , and Bastidas L. , 2006: Multiobjective particle swarm optimization for parameter estimation in hydrology. Water Resour. Res., 42, W07417, doi:10.1029/2005WR004528.
Guyonnet, D., Bourgine B. , Dubois D. , Fargier H. , Côme B. , and Chilès J. , 2003: Hybrid approach for addressing uncertainty in risk assessments. J. Environ. Eng., 129, 68–78, doi:10.1061/(ASCE)0733-9372(2003)129:1(68).
Han, J., Huang G. , Zhang H. , Li Z. , and Li Y. , 2014: Heterogeneous precipitation and streamflow trends in the Xiangxi River watershed, 1961–2010. J. Hydrol. Eng., 19, 1247–1258, doi:10.1061/(ASCE)HE.1943-5584.0000898.
He, L., Huang G. , Lu H. , Wang S. , and Xu Y. , 2012: Quasi-Monte Carlo based global uncertainty and sensitivity analysis in modeling free product migration and recovery from petroleum-contaminated aquifers. J. Hazard. Mater., 219–220, 133–140, doi:10.1016/j.jhazmat.2012.03.067.
Huey-Kuo, C., Wen-Ko H. , and Wei-Ling C. , 1998: A comparison of vertex method with JHE method. Fuzzy Sets Syst., 95, 201–214, doi:10.1016/S0165-0114(96)00307-7.
Jacquin, A. P., 2010: Possibilistic uncertainty analysis of a conceptual model of snowmelt runoff. Hydrol. Earth Syst. Sci., 14, 1681–1695, doi:10.5194/hess-14-1681-2010.
Juston, J., Seibert J. , and Johansson P. O. , 2009: Temporal sampling strategies and uncertainty in calibrating a conceptual hydrological model for a small boreal catchment. Hydrol. Processes, 23, 3093–3109, doi:10.1002/hyp.7421.
Khan, U. T., and Valeo C. , 2015: A new fuzzy linear regression approach for dissolved oxygen prediction. Hydrol. Sci. J., 60, 1096–1119, doi:10.1080/02626667.2014.900558.
Khan, U. T., Valeo C. , and He J. , 2013: Non-linear fuzzy-set based uncertainty propagation for improved DO prediction using multiple-linear regression. Stochastic Environ. Res. Risk Assess., 27, 599–616, doi:10.1007/s00477-012-0626-5.
Kollat, J. B., Reed P. M. , and Wagener T. , 2012: When are multiobjective calibration trade-offs in hydrologic models meaningful? Water Resour. Res., 48, W03520, doi:10.1029/2011WR011534.
Kuczera, G., and Parent E. , 1998: Monte Carlo assessment of parameter uncertainty in conceptual catchment models: The Metropolis algorithm. J. Hydrol., 211, 69–85, doi:10.1016/S0022-1694(98)00198-X.
Kwakernaak, H., 1978: Fuzzy random variables—I. Definitions and theorems. Inf. Sci., 15, 1–29, doi:10.1016/0020-0255(78)90019-1.
Langley, R. S., 2000: Unified approach to probabilistic and possibilistic analysis of uncertain systems. J. Eng. Mech., 126, 1163–1172, doi:10.1061/(ASCE)0733-9399(2000)126:11(1163).
Li, Z., Huang G. H. , Fan Y. R. , and Xu J. L. , 2015: Hydrologic risk analysis for nonstationary streamflow records under uncertainty. J. Environ. Inform., 26, 41–51, doi:10.3808/jei.201500321.
Ma, Z. Z., Wang Z. J. , Xia T. , Gippel C. J. , and Speed R. , 2014: Hydrograph-based hydrologic alteration assessment and its application to the Yellow River. J. Environ. Inform., 23, 1–13, doi:10.3808/jei.201400252.
Montanari, A., 2007: What do we mean by ‘uncertainty’? The need for a consistent wording about uncertainty assessment in hydrology. Hydrol. Processes, 21, 841–845, doi:10.1002/hyp.6623.
Montgomery, D. C., 2000: Design and Analysis of Experiments. 5th ed. John Wiley & Sons, 752 pp.
Montgomery, D. C., and Runger G. C. , 2013: Applied Statistics and Probability for Engineers. 6th ed. John Wiley & Sons, 832 pp.
Moore, R. J., 2007: The PDM rainfall–runoff model. Hydrol. Earth Syst. Sci., 11, 483–499, doi:10.5194/hess-11-483-2007.
Moradkhani, H., Hsu K.-L. , Gupta H. , and Sorooshian S. , 2005: Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter. Water Resour. Res., 41, W05012, doi:10.1029/2004WR003604.
Moussu, F., Oudin L. , Plagnes V. , Mangin A. , and Bendjoudi H. , 2011: A multi-objective calibration framework for rainfall–discharge models applied to karst systems. J. Hydrol., 400, 364–376, doi:10.1016/j.jhydrol.2011.01.047.
Muleta, M. K., and Nicklow J. W. , 2005: Sensitivity and uncertainty analysis coupled with automatic calibration for a distributed watershed model. J. Hydrol., 306, 127–145, doi:10.1016/j.jhydrol.2004.09.005.
Nash, J. E., and Sutcliffe J. V. , 1970: River flow forecasting through conceptual models part I—A discussion of principles. J. Hydrol., 10, 282–290, doi:10.1016/0022-1694(70)90255-6.
Pianosi, F., and Raso L. , 2012: Dynamic modeling of predictive uncertainty by regression on absolute errors. Water Resour. Res., 48, W03516, doi:10.1029/2011WR010603.
Rahmani, M. A., and Zarghami M. , 2015: The use of statistical weather generator, hybrid data driven and system dynamics models for water resources management under climate change. J. Environ. Inform., 25, 23–35, doi:10.3808/jei.201400285.
Rao, C. R., 1952: Advanced Statistical Methods in Biometric Research. John Wiley, 390 pp.
Razavi, S., and Tolson B. A. , 2013: An efficient framework for hydrologic model calibration on long data periods. Water Resour. Res., 49, 8418–8431, doi:10.1002/2012WR013442.
Sadegh, M., and Vrugt J. A. , 2014: Approximate Bayesian Computation using Markov Chain Monte Carlo simulation: DREAM(ABC). Water Resour. Res., 50, 6767–6787, doi:10.1002/2014WR015386.
Schoups, G., and Vrugt J. A. , 2010: A formal likelihood function for parameter and predictive inference of hydrologic models with correlated, heteroscedastic, and non-Gaussian errors. Water Resour. Res., 46, W10531, doi:10.1029/2009WR008933.
Seibert, J., and McDonnell J. J. , 2002: On the dialog between experimentalist and modeler in catchment hydrology: Use of soft data for multicriteria model calibration. Water Resour. Res., 38, 1241, doi:10.1029/2001WR000978.
Seibert, J., and McDonnell J. J. , 2014: Gauging the ungauged basin: The relative value of soft and hard data. J. Hydrol. Eng., 20, A4014004, doi:10.1061/(ASCE)HE.1943-5584.0000861.
Shafer, G., 1976: A Mathematical Theory of Evidence. Princeton University Press, 314 pp.
Singh, S. K., and Bárdossy A. , 2012: Calibration of hydrological models on hydrologically unusual events. Adv. Water Resour., 38, 81–91, doi:10.1016/j.advwatres.2011.12.006.
Sorooshian, S., and Arfi F. , 1982: Response surface parameter sensitivity analysis methods for postcalibration studies. Water Resour. Res., 18, 1531–1538, doi:10.1029/WR018i005p01531.
Tabachnick, B. G., and Fidell L. S. , 2006: Using Multivariate Statistics. 5th ed. Pearson Education, 1008 pp.
Vrugt, J. A., ter Braak C. J. F. , Clark M. P. , Hyman J. M. , and Robinson B. A. , 2008: Treatment of input uncertainty in hydrologic modeling: Doing hydrology backward with Markov chain Monte Carlo simulation. Water Resour. Res., 44, W00B09, doi:10.1029/2007WR006720.
Wang, S., and Huang G. H. , 2015: A multi-level Taguchi-factorial two-stage stochastic programming approach for characterization of parameter uncertainties and their interactions: An application to water resources management. Eur. J. Oper. Res., 240, 572–581, doi:10.1016/j.ejor.2014.07.011.
Wang, S., Huang G. H. , and He L. , 2012: Development of a clusterwise-linear-regression-based forecasting system for characterizing DNAPL dissolution behaviors in porous media. Sci. Total Environ., 433, 141–150, doi:10.1016/j.scitotenv.2012.06.045.
Wang, S., Huang G. H. , and Baetz B. W. , 2015a: An inexact probabilistic–possibilistic optimization framework for flood management in a hybrid uncertain environment. IEEE Trans. Fuzzy Syst., 23, 897–908, doi:10.1109/TFUZZ.2014.2333094.
Wang, S., Huang G. H. , Baetz B. W. , and Huang W. , 2015b: A polynomial chaos ensemble hydrologic prediction system for efficient parameter inference and robust uncertainty assessment. J. Hydrol., 530, 716–733, doi:10.1016/j.jhydrol.2015.10.021.
Wang, S., Huang G. H. , Huang W. , Fan Y. R. , and Li Z. , 2015c: A fractional factorial probabilistic collocation method for uncertainty propagation of hydrologic model parameters in a reduced dimensional space. J. Hydrol., 529, 1129–1146, doi:10.1016/j.jhydrol.2015.09.034.
Wei, J., Huang G. H. , Wang S. , Zhao S. , and Yao Y. , 2013: Improved solubilities of PAHs by multi-component Gemini surfactant systems with different spacer lengths. Colloids Surf., 423A, 50–57, doi:10.1016/j.colsurfa.2013.01.049.
Wu, C. F. J., and Hamada M. S. , 2009: Experiments: Planning, Analysis, and Optimization. 2nd ed. John Wiley & Sons, 760 pp.
Xu, H., Taylor R. G. , and Xu Y. , 2011: Quantifying uncertainty in the impacts of climate change on river discharge in sub-catchments of the Yangtze and Yellow River basins, China. Hydrol. Earth Syst. Sci., 15, 333–344, doi:10.5194/hess-15-333-2011.
Young, P. C., 2013: Hypothetico-inductive data-based mechanistic modeling of hydrological systems. Water Resour. Res., 49, 915–935, doi:10.1002/wrcr.20068.
Zadeh, L. A., 1965: Fuzzy sets. Inf. Control, 8, 338–353, doi:10.1016/S0019-9958(65)90241-X.
Zadeh, L. A., 1978: Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets Syst., 1, 3–28, doi:10.1016/0165-0114(78)90029-5.
Zhang, K., Achari G. , and Li H. , 2009a: A comparison of numerical solutions of partial differential equations with probabilistic and possibilistic parameters for the quantification of uncertainty in subsurface solute transport. J. Contam. Hydrol., 110, 45–59, doi:10.1016/j.jconhyd.2009.08.005.
Zhang, K., Li H. , and Achari G. , 2009b: Fuzzy-stochastic characterization of site uncertainty and variability in groundwater flow and contaminant transport through a heterogeneous aquifer. J. Contam. Hydrol., 106, 73–82, doi:10.1016/j.jconhyd.2009.01.003.
Zimmermann, H. J., 2001: Fuzzy Set Theory—and Its Applications. 4th ed. Kluwer Academic, 514 pp.