Entropy–Copula in Hydrology and Climatology

The entropy theory has been widely applied in hydrology for probability inference based on incomplete information and the principle of maximum entropy. Meanwhile, copulas have been extensively used for multivariate analysis and modeling the dependence structure between hydrologic and climatic variables. The underlying assumption of the principle of maximum entropy is that the entropy variables are mutually independent from each other. The principle of maximum entropy can be combined with the copula concept for describing the probability distribution function of multiple dependent variables and their dependence structure.Recently,effortshave beenmade to integratetheentropyand copula concepts(hereafter,entropy– copula) in various forms to take advantage of the strengths of both methods. Combining the two concepts provides new insight into the probability inference; however, limited studies have utilized the entropy–copula methods in hydrology and climatology. In this paper, the currently available entropy–copula models are reviewed and categorized into three main groups based on their model structures. Then, a simple numerical example is used to illustrate the formulation and implementation of each type of the entropy–copula model. The potential applications of entropy–copula models in hydrology and climatology are discussed. Finally, an example application to ﬂood frequency analysis is presented.


Introduction
Understanding the global water cycle and climatic phenomena often requires investigating the interdependence of hydrologic and climatic variables (e.g., drought and heat wave, flood peak and volume, and drought severity and duration).For example, a deficit in precipitation over a certain period of time could lead to a deficit in soil moisture and runoff (Kao and Govindaraju 2008).Multivariate distribution functions have been widely used in the literature for modeling two or more dependent hydrological variables and their dependence structure (Salvadori and De Michele 2007;Hao and AghaKouchak 2013;Nazemi and Elshorbagy 2012).A number of methods to construct a joint distribution have been proposed and applied, including the kernel density estimation, entropy method, and copulas, to name a few (Kapur 1989;Kotz et al. 2000;Grimaldi and Serinaldi 2006;Nelsen 2006;Serinaldi and Grimaldi 2007;Balakrishnan and Lai 2009;AghaKouchak et al. 2010a,b).
In the past decade, application of copulas in hydrology has grown rapidly, owing to the fact that using copulas were efficient for describing the dependence among multiple hydrologic variables (Salvadori and De Michele 2007;Favre et al. 2004).The copula theory offers a flexible way to construct a joint distribution independent from the marginal distributions (Sklar 1959).Copulas are used primarily to model the dependence structure between two or more variables [e.g., precipitation and soil moisture (Hao and AghaKouchak 2014) or flood peak and volume (Salvadori et al. 2007)].A variety of copula families have been developed and can be used to model different dependence structures (Joe 1997;Trivedi and Zimmer 2005;Nelsen 2006).The main advantage of this approach is that construction of a joint distribution through a copula is independent of the marginal distributions of the individual variables.For an introduction to the copula theory, the reader is directed to Joe (1997) and Nelsen (2006).A detailed review of the development and applications of copulas in hydrology is provided in Genest and Favre (2007) and Salvadori and De Michele (2007).
Univariate and multivariate statistical analysis in hydrology and climatology typically involves inference of probability distribution, for example, flood frequency analysis, rainfall simulation, extreme value analysis, and geostatistical interpolation (Singh 1992;Stedinger 1993;Bárdossy 2006).A univariate (multivariate) probability distribution assigns probabilities to univariate (multivariate) hydrologic or climatic events.If there is no reason for one event to occur more frequently than other events, then the probabilities of all events will be equal.This is known as the principle of insufficient reason (Singh 2011).However, if there is some knowledge or partial information on the nonuniformity of the events (or outcomes of a certain process), the principle of maximum entropy allows for the inclusion of the available information in assignment of probabilities to different events or outcomes of a process (Singh 2011).In other words, the principle of maximum entropy offers a methodology for deriving a probability distribution from limited information (Zhao and Lin 2011).The entropy theory was first formulated by Shannon (1948) as a measure of information (or uncertainty).The principle of maximum entropy was developed for probability inference on the basis of partial information, given a set of constraints (Jaynes 1957a,b).The approach leads to a target distribution with the maximum entropy that satisfies a set of constraints.Entropy offers the opportunity to account for more moments, beyond just the second moment, which describes the deviation around the mean (Zhao and Lin 2011).The concept of maximum entropy for probability density inference has been applied extensively in a variety of areas, including physics, chemistry, economics, and hydrology (Kapur 1989;Kesavan and Kapur 1992;Brunsell 2010;Hao and Singh 2013).For a comprehensive review of the development and applications of the entropy theory in hydrology, the interested reader is referred to Singh (1997) and Singh (2011).
The underlying assumption of the principle of maximum entropy is that the entropy variables are mutually independent from each other (Jaynes 1957a).Copulas, on the other hand, are used to describe the dependence between multiple variables.Consequently, the principle of entropy can be combined with the concept of copula to handle mutually dependent variables (i.e., describing probability distribution function of multiple dependent variables and their dependence structure).Recently, efforts have been made to integrate the entropy and copula concepts (hereafter, entropy-copula) in various forms to take advantage of the strengths of both methods.Two important properties of a copula density function are uniformly distributed margins and a dependence structure that describes the degree of association between the margins.With these properties as the partial knowledge of a target copula, new copulas can be derived based on the entropy theory.The most common approach to derive an entropy-copula method is to maximize the entropy of the copula density function using the two properties as constraints (in discrete, continuous, or mixed forms) (Chu and Satchell 2005;Dempster et al. 2007;Piantadosi et al. 2007;Pougaza et al. 2010;Chu 2011;Pougaza and Mohammad-Djafari 2011;Ané and Kharoubi 2003;Piantadosi et al. 2012a,b;Pougaza and Mohammad-Djafari 2012;Hao and Singh 2012).The entropy-copula methods generally retain the advantage of the commonly used parametric copula: the joint distribution construction will be independent of the marginal distributions.While the entropy-copula methods seem to be powerful and promising techniques, few studies have utilized them for modeling hydrological and climatic variables.
In this study, the recently developed entropy-copula methods are reviewed in detail.The currently available models are then categorized into three main groups based on their model structures.Then, a simple numerical example is used to illustrate the formulation and implementation of each type of the entropy-copula model.A discussion of broader potential applications in hydrology and climatology is provided, followed by an example application of the entropy-copula concept to flood frequency analysis.
The remainder of this paper is outlined as follows.The entropy and copula concepts are introduced in sections 2 and 3, respectively.The recently developed entropycopula methods are reviewed and categorized in section 4, followed by a numerical example in section 5. Section 6 provides a discussion on the potential applications to hydrology and climatology and an example application to flood frequency analysis.The last section summarizes the results and conclusions.

Entropy theory
For a random variable X with probability density function (PDF) f(x) defined on the interval [a, b], the Shannon entropy can be defined as (Shannon 1948;Jaynes 1957a,b) (1) The maximum entropy concept consists of inferring the probability distribution that maximizes information entropy given a set of constraints.For the realizations x i (i 5 1, 2, . . ., n) of the random variable X, the general form of the constraints can be expressed as (Kapur 1989) where g r (x) denotes a function of x, with g r (x) being the expectation of g r (x).In this equation, m is the number of constraints, and the rth expected value of g r (x) can be obtained from observations (e.g., mean and variance; see Singh 2011).For r 5 0, the constraint given in Eq. ( 2) can be described as ð b a f (x) dx 5 1. (3) This constraint indicates that the PDF must satisfy the so-called total probability theorem, also termed as the normalization condition.In other words, this constraint ensures that the integration of the PDF f(x) over the entire interval equates unity.While there may be a variety of distributions that satisfy the above constraints shown in Eq. ( 2), according to the principle of maximum entropy, the most suitable distribution function is the one that leads to the maximum entropy (Jaynes 1957a).The maximum entropy PDF can be obtained by maximizing the entropy in Eq. ( 1) subject to Eq. ( 2) with the Lagrange multipliers method as (Kapur 1989) where l i , i 5 0, 1, 2, are the Lagrange multipliers that can be estimated with the Newton-Raphson algorithm (Kapur 1989).The Lagrange multipliers method allows maximizing (or minimizing) a function subject to some constraints (Vapnyarskii 2001).By integrating Eq. ( 4), one can obtain the cumulative distribution function (CDF).Kullback and Leibler (1951) developed the principle of minimum cross (or relative) entropy as a way to infer the probability based on the prior information.Suppose that q(x) is an unknown density and p(x) is the prior estimate of q(x).The cross entropy H 2 can be defined as (5) Minimizing the cross entropy in Eq. ( 5) subject to the constraints in Eq. ( 2) yields (Kapur 1989) The minimum cross entropy measures the distance of one distribution to another.When the prior distribution p(x) is the uniform distribution, the minimum crossentropy distribution q(x) reduces to the maximum entropy distribution shown in Eq. (4).

Copula theory
The copula theory has been commonly used to construct the joint distribution of multiple variables.For the bivariate case, denote the marginal distributions for the continuous random variables X and Y as F(x) (or U) and G(y) (or V), respectively.The joint CDFs C(u, y) or F(x, y) can be constructed with the copula C in the form (Sklar 1959) The copula C links the marginal distributions by mapping the two univariate marginal distributions into a joint distribution.If the marginal distribution is continuous, the copula function is unique.The copula function C(u, y) is a function from [0, 1] 2 to [0, 1] with the following properties (Nelsen 2006): For u 1 , u 2 , y 1 , and y 2 in [0, 1], such that u 1 # u 2 and y 1 # y 2 , Having the copula distribution C(u, y), the copula density function c(u, y) can be defined as (Nelsen 2006): c(u, y) 5 ›C 2 (u, y)/›u›y .(10)

Entropy-copula models
In this paper, the currently available entropy-copula models are broadly classified into three groups based on their model structures: 1) continuous maximum entropy copula (CMEC), 2) mixed maximum entropy copula (MMEC), and 3) discrete density maximum entropy copula (DDMEC).In this section, three recently developed entropy-copula methods (one from each category) are reviewed in depth.For the sake of simplicity, the formulations of the entropy-copula method for deriving the copula density function are introduced in the bivariate case [i.e., c(u, y)]; however, extension of the presented equations to higher dimensions is straightforward (see Piantadosi et al. 2012b;Hao and Singh 2013).

a. CMEC
The most entropic canonical copula (MECC) is derived by maximizing the entropy of the copula density function c(u, y) subject to constraints in the form of continuous functions of marginal probabilities (U and V; Chu and Satchell 2005;Chu 2011).The MECC is also termed as the empirical copula in Chui and Wu (2009).The constraints to derive the copula are imposed through a continuous function of the marginal, and thus, this method is termed as the continuous maximum entropy copula.The entropy of the copula density function c(u, y) can be expressed as The condition that the integration of the copula density function c(u, y) over the space [0, 1] 2 equates unity can be satisfied by specifying the following constraint: The condition that the marginal probability is uniformly distributed on [0, 1] can be fulfilled by specifying the following constraint: The joint behavior (or dependence structure) of the marginal probability U and V can be modeled by specifying the constraint as where h(u, y) is a function of the marginal u and y, which can be related to a certain dependence structure, and E[h(u, y)] is the expectation of the function h(u, y).There are several dependence measures that can be linked to Eq. ( 14) (Chu 2011).For example, when h(u, y) 5 uy, E[h(u, y)] can be expressed as a linear form of the Spearman rank correlation coefficient (r), which is a nonparametric statistical dependence measure: The Spearman rank correlation coefficient assesses the degree of association between two variables and depends solely on the choice of copula and not the marginal distribution of the underlying variables (Joe 1997).The problem of inferring the copula density function c(u, y) can then be formalized as maximizing the entropy subject to the constraints given in Eqs. ( 12)-( 14).However, there are infinite combinations of constraints (continuums of integrals of continuous random variables) in Eq. ( 13), thus making it difficult to solve the optimization problem of maximizing the entropy.An alternative method is to use Carleman's condition to transform these constraints into the moment constraints (Durrett 2005;Chu 2011).In this approach, the constraints in Eq. ( 13) can be replaced as where r 5 1, 2, . . ., m, and m is the maximum order of the moment.Denote the constraints as g k (u, y) 5 [u, . . ., u r , y, . . ., y r , h(u, y)], k 5 1, . . ., N, where N is the number of constraints.The copula density function c(u, y) can then be derived by maximizing the entropy in Eq. ( 11), subject to the constraints in Eqs. ( 12), ( 14), and ( 16) as (Chu 2011) where l 1 , . . ., l m ; g 1 , . . ., g m ; and t are the parameters and l 0 can be expressed as (l r u r 1 g r y r ) 2 th(u, y) du dy . ( It has been shown that the model parameters can be estimated through minimizing a convex function G expressed as (Mead and Papanicolaou 1984;Kapur 1989) There is no analytical solution to the above optimization (minimization) problem, and hence, a numerical method such as the Newton-Raphson iteration can be used to estimate the parameters.Similar concepts have been applied to derive a multivariate distribution from the entropy theory based on moments of the data (Zhu et al. 1997;Phillips et al. 2006;Hao and Singh 2011).An essential procedure in implementing the maximum entropy distribution is selecting appropriate constraints to avoid/control overfitting (Phillips et al. 2006).

b. MMEC
Instead of matching the moments of the marginal distribution, the constraints can be specified as the cumulative probability at a set of points to satisfy the uniformly distributed marginal (Dempster et al. 2007;Friedman and Huang 2010).This type of model is categorized as the mixed maximum entropy copula in this study.The socalled relative entropy copula (REC), proposed by Dempster et al. (2007), belongs to this category.The REC allows for the driving of a copula family by minimizing the entropy relative to a specific copula subject to a set of constraints.In this method, the issue of infinite constraints in Eq. ( 13) is addressed by selecting a finite number of points p i and q j , where i, j 5 1, 2, . . ., N, and by conditioning the constraints to only hold for those points, but not for all, within the interval [0, 1] (Dempster et al. 2007).Thus, the constraints in Eq. ( 13) can be replaced with and where 0 # p 0 . . . .# p k . . .# p N #1 and 0 # q 0 . . . .# q k . . .# q N #1 are a set of sequences of marginal probabilities at discrete points.Various functions of the marginal distributions can be used to model data properties (e.g., dependence structure).Here, the pairwise product of the marginal in Eq. ( 15) (the Spearman rank correlation coefficient) can be used as the constraint to derive the REC.
Assuming the prior copula to be p(u, y), the relative entropy of the copula density function c(u, y) can be expressed as The REC can be obtained by minimizing the relative entropy in Eq. ( 22), subject to the constraints given in Eqs. ( 20), ( 21), and (15) as c(u, y) 5 p(u, y) exp where and l are unknown parameters to be estimated; and l 0 can be expressed as a function of the unknown parameters The parameters can be estimated using the procedure discussed in section 4a.The REC enables incorporation of a prior distribution, which can be a prior choice of target copula, to derive the target copula density function (Dempster et al. 2007).Theoretically, this prior copula can also be introduced within the framework of the MECC by minimizing the relative entropy with respect to the choice of prior copula.

c. DDMEC
The uniformly distributed marginal property and certain types of dependence structures can be approximated by discrete forms of constraints to derive a copula with maximum entropy.These types of entropy-copula models are classified as the discrete density maximum entropy copula.The recently proposed maximum entropy checkerboard copula (MECBC), for example, belongs to this category (Piantadosi et al. 2007(Piantadosi et al. , 2012a,b),b).A d-dimensional checkerboard copula is a distribution with the density function defined by a step function on a d-uniform subdivision of the hypercube [0, 1] d that uniformly approximates a continuous copula (Piantadosi et al. 2012b).In addition, the grade correlation is imposed as a constraint to preserve the dependence structure in the original data.For the bivariate case, a two-dimensional checkerboard copula can be constructed, with n equal subintervals within [0, 1].Then, a copula probability density function can be defined in the form of the step function h(u, y) on the space [0, 1] 3 [0, 1] as (Piantadosi et al. 2007) h(u, y) 5 nh ij , (u, y) 2 (u i , u i11 ) 3 (y j , y j11 ) , (25) where h ij is the element of an n 3 n matrix H on the partition of the space [0, 1] 3 [0, 1].To satisfy the condition of uniform marginal, the marginal constraints can be specified as Through mathematical manipulation, the grade correlation coefficient can be expressed through the step function as (Piantadosi et al. 2007) The entropy of the copula probability density function [or the step function h(u, y)] can be defined as (Piantadosi et al. 2007) h 5 (21) The objective is to select an n 3 n matrix H 5[h ij ] to match the known Spearman rank correlation coefficient r and also the uniformly distributed marginal, which can be achieved by maximizing the entropy subject to the constraints given in Eqs. ( 26) and ( 27).Here, constraints can be expressed in the matrix form of AX 5 b, where X is an n 2 3 1 vector of the unknown parameters u, and A is the 2n 3 n 2 constant matrix with element a ij .Using the Fenchel duality theory, this optimization problem can be reformulated to an unconstrained optimization problem, which is easier to solve (Piantadosi et al. 2012b).The theory of Fenchel duality relies on the theory of convex analysis, which focuses on properties of convex functions and convex sets (Borwein and Lewis 2010).This theory can be used to find a numerical solution for a complex mathematical problem.Piantadosi et al. (2012a) showed that the optimization problem can then be expressed as ( The Newton iteration can be used to solve the above equation: where and J(Á) is the derivation of the function Q r .The vector of the matrix element h 5 [h 1 , h 2 , . . ., h 2 n ] can be obtained as where A T is the transpose of the matrix A.
One important feature of this method is that the condition of the copula in Eq. ( 26) can be approximated by discretizing the uniform interval [0, 1], which is similar to the concept of the discrete copula (Mayor et al. 2005;Kolesárová et al. 2006).A similar approach that also belongs to this type of maximum entropy copula is the minimum information (or entropy) copula (MIC), which is introduced by Meeuwissen and Bedford (1997) to derive the joint distribution of two random variables with respect to a background distribution (uniform distribution) with constraints of the uniform margins and a given rank correlation r (the Spearman rank correlation coefficient).Because the prior (or background) distribution is a uniform distribution, the formulation of the minimum information copula reduces to the maximum entropy copula.This bivariate minimum information copula has been used for deriving high-dimensional distributions such as the vine copula through graphic models (Bedford and Cooke 2002).In addition, other similar modeling frameworks for estimating a copula density function have been developed recently, although the models are not formally introduced as maximum entropy copula or entropy-copula (e.g., Qu et al. 2011;Qu and Yin 2012).

Illustration using a numerical example
In this section, a simple numerical example is presented to illustrate the construction of the density function c(u, y) using the MECC, REC, and MECBC entropy-copula models, discussed previously.For all of these methods, the derivation of the joint distribution is separated from marginal distributions.Thus, marginal distributions are not discussed, and the paper focuses on the construction of the joint distribution to model a dependence structure represented by the Spearman rank correlation coefficient.In this numerical example, the Spearman rank correlation coefficient is assumed to be 0.6 and is regarded as the partial knowledge about the target copula, which will be used as the constraint to derive the copula density using the entropy theory.

a. MECC
In the MECC method, the uniform marginal can be modeled through marginal constraints specified as moments of the marginal probability, and the dependence structure can be specified as continuous functions of the marginal probability (e.g., the pairwise product uy).In this example, the first two moments are used as the marginal constraints (m 5 2) to ensure that the marginal distribution is uniformly distributed, and the pairwise product of the marginal probability is used to model the dependence structure [h(u, y) 5 uy].The copula density function c(u, y) can then be obtained as After estimating the parameters l 1 , . . ., l 5 using the method discussed in section 4a, the copula density function can be expressed as c(u, y) 5 1 0:298 35 exp(21:9530u 2 3:2252u 2 2 1:9530y The estimated marginal probability [Eq.( 34)] is an approximation, and hence, it should be validated first.To validate the marginal property, the theoretical and estimated marginal probabilities from the MECC method are displayed in Fig. 1a.As shown, the theoretical and approximated marginal probabilities are markedly similar, indicating a very good agreement between the two.To quantify the error in the approximation of the marginal probability, the bias of the estimated probability u (or y) is defined as where ũ is the marginal probability for different entropycopula methods.The bias measures how close the estimated marginal property is to the theoretical value.The MECC's bias values for this numerical example are presented in Fig. 2a.The bias values are less than 1 3 10 23 , indicating that the marginal properties are well satisfied.The resulting copula density function is shown in Fig. 3a.
The theoretical and estimated marginal probabilities with four discrete points from the REC method (denoted as REC1) are shown in Fig. 1b, and the corresponding bias is shown in Fig. 2b.As shown, at the specified discrete points, the probability fits the marginal probability well.For example, for u of 0.7 (one of the discrete points specified in X and Y), the copula probability is estimated as C(0.7, 1) 5 0.700, indicating a perfect correspondence.However, the marginal distribution does not fit as well at other (unspecified) points.For example, for u 5 0.3, the probability is computed as C(0.3, 1) 5 0.2449.This indicates that more data points may be needed to approximate the marginal distribution adequately.
To improve the approximation of the marginal property, the moment constraints (first two) of the marginal are added instead of increasing the number of discrete points for each variable to derive the joint distribution (denoted as REC2).In other words, in addition to the constraints of the discrete points, the same constraints used in the MECC method are considered as well.The  ,c).For example, for u 5 0.3, the copula probability is obtained as C(0.3, 1) 5 0.3005.Because of imposing more constraints at the discrete points in REC2, the bias of the marginal probability is reduced significantly compared with that of the MECC method.The density functions from REC1 and REC2 are shown in Figs.3b and 3c, respectively.Given the high bias of REC1, one can conclude that the REC1 density function may not be reliable.It is noted that the REC2 density function is consistent with that of the MECC given in Fig. 3a.

c. MECBC
The MECBC method is based on matching the Spearman rank correlation coefficient 0.6 and the uniformly distributed marginal property in the discrete form.In the following bivariate (m 5 2) example, n 5 3 (see section 4c) and the matrix H is defined on the space [0, 1] 3 [0, 1] as The n 3 m (here, 6) constraints can be written explicitly as The last constraint in the above equation is redundant because it is automatically ensured by other constraints.Thus, the total number of effective constraints is n 3 m 2 1 (here, 5).Furthermore, the constraint for the Spearman rank correlation coefficient can be expressed as 1 9 [h 11 1 3h 12 1 5h 13 1 3h 21 1 9h 22 1 15h 23 1 5h 31 The above constraints can be written in the matrix form AX 5 b as FIG. 2. Biases of the marginal distributions for the (from top to bottom) MECC, REC1, and REC2 entropy-copula methods.
In this method, the probability density h ij is only available for the points defining the partition on [0, 1] 3 [0, 1] (here, 0, 1 /3, 2 /3, 1).Practically, a much finer partition would be necessary to ensure that the marginal probability is approximated appropriately.As an example, for n 5 20, Fig. 3d displays the density function that is consistent with those of MECC and REC2.

Potential applications to hydrology and climatology
The application of copulas in hydrology, including frequency analysis, simulation, and geostatistical interpolation, has been reviewed by Salvadori and De Michele (2007).A review of the use of copula models in climatology, including ensemble climate forecast evaluation, bias correction and error estimation, and downscaling, is provided in Schoelzel and Friederichs (2008).However, application of entropy-copula methods in hydrology, in particular, climatology, is relatively new, and a brief review of the literature is provided in section 1.This section provides a broader discussion on the potential applications of entropy-copula methods in hydrology and climatology, followed by an example application.

a. Climate change detection and analysis
Numerous studies have shown/argued that climate extremes have been changing in the past and may change in the future as well (Alexander et al. 2006;Hao et al. 2013;Estrella and Menzel 2013;Damberg and AghaKouchak 2014;Mehran et al. 2014;Madani and Lund 2010;Hansen et al. 2010;Easterling et al. 2000;AghaKouchak et al. 2012).Several empirical statistical indices have been developed that can be used to quantify changes in weather and climate extremes, including annual temperature maxima and the number of dry days (Schefzik 2011;Zhang et al. 2011) or categorical and volumetric measures of climatic variables (AghaKouchak and Mehran 2013).The minimum cross-entropy method could potentially be used for detecting changes in climatic and hydrologic extremes.As mentioned earlier, the minimum cross-entropy method quantifies the distance between (divergence of) one distribution and another.This model can be used to quantify whether the distribution of temperature or streamflow data has changed over time.Using entropy-copula models reviewed in this study, one can also investigate changes to the distributions (or extremes of the distribution) of hydrologic-climatic variables in both space (copula) and time (entropy).On the other hand, climate variables are interdependent, and changes in one variable may lead to changes in other variables as well.In a recent study, Hao et al. (2013) assessed changes in concurrent precipitation and temperature extremes and reported substantial changes in concurrent extremes.Given the properties of entropy-copula models, they can be used to assess climate change and variability based on multiple variables and their joint distribution.

b. Uncertainty and error analysis
In the past three decades, the development of satellite precipitation datasets and weather radar systems has provided remotely sensed gridded data at the global-scale variables to the community (Stedinger et al. 1985;Sorooshian et al. 2011).However, remotely sensed precipitation datasets are subject to retrieval errors and uncertainties (Lee 2012;Mehran and AghaKouchak 2014;Ebtehaj and Foufoula-Georgiou 2013;Tian et al. 2009).In addition to remote sensing techniques, global climate models have been widely used to simulate the climate of the past and future.However, climate simulations are also highly uncertain, and the biases and errors associated with climate data have been discussed in numerous publications (Westra et al. 2007;Serinaldi 2009;Mehran et al. 2014;Liu et al. 2014).Thus far, a number of uncertainty models have been developed for satellite and radar data, as well as climate model simulations (Stedinger and Vogel 1984;Koutsoyiannis 1994;AghaKouchak et al. 2010b,c;Möller et al. 2012), from which limited models account for both space-time properties of errors and uncertainties.The introduced entropy-copula method could potentially be used for developing error models for remotely sensed data and climate model simulations upon availability of ground-based point observations of error (i.e., gridded remotely sensed or climate model-based precipitation data and reference gauge observations).The error at the observation locations can be used to build the dependence structure of the error using the introduced entropy-copula models.One can then simulate stochastic models of space-time error in order to derive an ensemble of remotely sensed or model-based simulations as a measure of uncertainty.

c. Bias correction and downscaling
In addition to developing error models, the entropycopula models can be utilized for bias correction and downscaling of dependent variables.Bias correction or downscaling of coarse climate data are often carried out before using climate model outputs in climate change impact studies (Chen et al. 2011;Hagemann et al. 2011).Traditionally, the bias correction and also downscaling are conducted for individual variables (e.g., precipitation and temperature) independently, ignoring the relationship that may exist between climatic variables.In a recent study, Piani and Haerter (2012) proposed that the empirical copula be used for simultaneous bias correction of temperature and precipitation data.This approach was a major breakthrough in multivariate bias correction.While the methodology outlined in Piani and Haerter (2012) considers the relationship between precipitation and temperature for bias correction, it does not necessarily preserve the rank correlation before and after bias adjustment.Entropy-copula models can be used for multivariate bias correction while preserving the rank correlation, because the observed rank correlation can be used as a constraint in the model.Similar models can also be developed for downscaling and disaggregation of multiple dependent climate variables.

d. High-dimensional problems
In many applications (e.g., spatial error analysis and downscaling), the problem in hand involves multiple variables or the many vectors of the same variable across a large region (e.g., pixel-based precipitation data).While there are a number of copula families that can be used in high-dimensional studies [e.g., vine copula (Bedford and Cooke 2002) and Plackett family copula (Kao and Govindaraju 2008)], copula families that have been widely used in hydrology (i.e., Archimedean copulas) are mainly suitable for bivariate or trivariate problems.In general, extension of Archimedean copulas for highdimensional problems is not straightforward, and hence, their use for studies that involve a large number of variables is restricted to certain conditions (Joe 1997;McNeil and Ne slehová 2009;McNeil 2008).Given the properties of entropy-copula models (Hao and Singh 2013), discussed earlier, they offer a unique opportunity for statistical analysis of high-dimensional problems (see, e.g., Friedman and Huang 2010).

e. Frequency analysis
Entropy-copula models can be used for multivariate frequency analysis.This requires modeling the dependence structure of the variables involved (e.g., such as flood volume and peak) and can be achieved by fitting a multivariate parametric distribution (Favre et al. 2004;Shiau et al. 2006;Genest et al. 2007;Sadri and Burn 2012).As an example application, the entropy-copula concept is applied to flood frequency analysis to illustrate construction of the density function c(u, y) using the entropy-copula.Here, only the MECC is employed because its joint density function is continuous and hence is more appropriate for flood frequency analysis.The flood peak flow (Q) and volume (V) data extracted from the daily streamflow data from 1963 to 1995 from the Saguenay region in Quebec, Canada, are used for bivariate frequency analysis (Yue et al. 1999;Chebana and Ouarda 2011;Zhang and Singh 2007).In this case study, the maximum entropy copula is employed to model the dependence structure of the flood peak and volume, and the Gumbel distribution is used to model the marginal distribution.For the maximum entropy copula, the first three moments up to order 3 and the pairwise product of the marginal probability are used.The joint return period in which either the flood volume or flood peak or both exceed the threshold level can be expressed as .
The flood peak and volume based on the maximum entropy copula is shown in Fig. 4. The figure provides multivariate return period information based on both flood peak and volume.
The potential applications of the entropy-copula models are not limited to the above examples.In general, these methods could be applied to problems that involve space-time dependence, such as geostatistical interpolation, downscaling, evaluation of ensemble climate forecasts, regional extreme value analysis, etc.In the future, more studies are expected to use entropy-copula methods to address challenges in hydrology and climate data analysis.

Conclusions and remarks
The entropy and copula concepts have been shown to be powerful tools in various applications in hydrology, climatology, and other areas.In recent years, new methods have been proposed for developing copulas based on the maximum entropy theory (entropy-copula).These copulas provide the opportunity to derive probability distribution function of multiple dependent variables and their dependence structure.This study reviews the recent developments in entropy-copula and broadly classifies them into three main groups based on their model structures: 1) continuous maximum entropy copula (CMEC), 2) mixed maximum entropy copula (MMEC), and 3) discrete density maximum entropy copula (DDMEC).The three categories of the entropycopula models differ in the type of constraints (i.e., uniformly distributed marginal and dependence structure) used to derive the copula density within the maximum entropy framework.After a detailed discussion on different entropy-copula modeling concepts, a simple numerical example is used for illustrating different methods, followed by an example application to bivariate flood frequency analysis.
The CMEC method uses the continuous functions of the marginal probabilities as the constraints to derive the copula density function.The numerical example provided here, shows that the marginal probabilities and the dependence structure can be preserved reasonably well.The MMEC method uses the discrete points of the marginal probabilities and continuous functions of the marginal probabilities (e.g., pairwise product) as constraints to derive the copula density.The relative entropy copula, which belongs to the MMEC category, offers an attractive property in which the prior distribution can be incorporated into the derivation of the copula family.The results show that, by increasing the number of discrete points, modeling of the marginal probability and dependence structure will be improved.In the provided numerical example, the bias significantly reduced after increasing the number of discrete points.This indicates that, in a practical application, one can improve the model performance by increasing the number of discrete points.The DDMEC method discretizes both the marginal and joint constraints to derive the copula density function.The marginal probability is only available for the points defining the partition on the univariate interval, and thus, in practice, a fine partition is required.It should be noted that while DDMEC's joint density is not continuous, the resulting joint distribution itself is continuous.
The entropy-copula methods introduced in this study have different properties and characteristics.Selecting an appropriate entropy-copula method is problemspecific, and one cannot simply generalize the best choice of entropy-copula method.For any specific application, various entropy-copula models should be tested and validated to identify a representative model for the problem at hand.For example, when dealing with a multidimensional problem (say, dimension .3), the CMEC will be less computationally demanding than the other methods because the marginal property is approximated with the moments of the marginal probabilities instead of the probabilities at discrete points.
While entropy-copula models provide opportunities to advance multivariate statistical modeling, they have their own limitations.For many types of entropy-copula models there is no analytical solution for optimization and parameter estimation.Therefore, numerical methods or sampling approaches should be used to estimate the parameters.For high-dimensional problems, as the number of parameters grows, parameter estimation becomes more computationally demanding.This review of the current maximum entropy-copula methods sheds light on the potential applications in hydrology and climatology.However, these models are relatively new to hydrology and climatology, and more research efforts in this direction are required to fully explore their potential applications.Interested readers can request the source code of the models discussed in this paper from the author.