In a recent paper Krzysztofowicz (1999, hereafter K99) has analyzed relations between probability π of occurrence of rain within a period T and marginal probabilities π₁ and π₂ of rain within nonoverlapping subperiods T₁ and T₂ such that T₁ ∪ T₂ = T. The most nontrivial part of the study is devoted to estimating π in the case when π₁ and π₂ are known from a forecast and two approaches were put forward. The heart of them is estimating stochastic dependence between rains within subperiods.

Of course, the probability of π conditional on any event of nonzero probability may be calculated. However, unless we have knowledge that this event has occurred, any inference drawn from that conditional probability has nothing in common with the Bayesian analysis, which is based on what actually did happen but not on what might have happen but did not (Jaynes 1986). Using another partitioning of Δ and such an inference procedure, one can get any point within Δ as an estimate of π.

In the other approach proposed in K99, a priori information on correlations between rain occurrence within T₁ and T₂ was employed. It is convenient for further discussion to introduce the following notation. Let p₁₁, p₂₂, p₁₂, and p₂₁ be probabilities of four elementary events: nonoccurrence and occurrence of rain within the whole period T, rain at T₁ and no rain at T₂, and, at last, no rain at T₁ and rain at T₂, correspondingly. Then

View Expanded

K99 proposed to calculate the Pearson correlation coefficient α(π₁, π₂) between rains within the subperiods from historical data and observations of forecast probabilities π_i and to choose π in such a way that the resulting distribution of p_ij would describe the same correlations.

When only partial information about the probability distribution of the elementary events is given, a natural and probably the only possible way of reconstructing the whole statistics is to invoke prior knowledge. In this sense, the second approach seems preferential because it mobilizes more information about the phenomenon of interest. However, following this scheme, one cannot use the whole dataset but only those meteorological observations for which a prediction has been made. Further, even working with this reduced dataset, the forecaster does not extract all information contained in the observations but only the correlation coefficient. Though the data provide us with sampling probabilities μ_ij of occurrence of rain and no rain within T₁ and T₂, the approach proposed cannot accumulate these data and simply reject them.

The probabilities provided by historical records and forecasting are not and have to be not identical, otherwise there is no need in any weather prediction. Thus, combining these different types of information in a unique solution is a very delicate task and setting characteristics of the forecast statistics equal to those derived from observations is at least questionable. A point that was not recognized in K99 is fundamental difference between probabilities μ_ij and p_ij. The former express our a priori knowledge, which comes before forecasting. The objective of weather prediction is to update that a priori information. Consequently, no property of μ_ij can generally survive.

A procedure that enables one to take into account the total information contained in the meteorological data and forecasting in a consistent way and to solve much more general problems is the principle of maximum entropy (PME). Among other approaches, it has probably the longest and most spectacular history. The concept of entropy originated about a century ago within the context of a fundamental but specific physical problem; it has recently found widespread application in dynamical systems theory, ergodicity theory, inverse problems, communication theory, numerical analysis, theory of functions, decision theory etc., to say, to problems straying far from the classical thermodynamics. Probably, the application closest to weather forecasting was recently presented by Urban (1996).

All these were made possible only after recognizing, thanks to Shannon (1948), the fact that entropy has much wider content than had been thought. To be concise, Shannon proved that any measure of uncertainty in a discrete probability distribution p_k, k = 1, . . . , n, is proportional to the Gibbs entropy,

View Expanded

if that measure is (i) a continuous function of probabilities, (ii) a decreasing function of the number of possible events in the case when they are equally probable, and (iii) consistent, that is, if there is more than one way of evaluating out its value, every possible way yields the same result.

Kullback (1959) generalized (6) and introduced the cross-entropy

View Expanded

Here μ_k is another set of probabilities. The cross-entropy (7) measures the relative uncertainty in p_k with respect to μ_k and coincides with (6) for the uniform distribution μ_k.

The heart of the PME is choosing those p_k’s that maximize (7) subject to certain constraints we wish them to satisfy. This approach provides us with a probability distribution that is as much uncertain as allowed by the constraints imposed. In other words, the maximum entropy distribution is free from any information additional to that which we use as input. It might be helpful to point out that the classical equilibrium statistical mechanics is completely deduced from the PME if Liouville’s measure is chosen as a prior probability distribution over the phase space. On the other hand, failures in certain predictions pointed not to failure of the PME but, as it was realized further, to inadequate prescription of the prior. The rise of quantum theory made it possible to resolve those difficulties by assigning a new prior over a deeper hypothesis space (Jaynes 1957).

Application of the PME to the problem considered in K99 is straightforward. To chose π, the forecaster maximizes (7) subject to (3), (4), and normalizability

View Expanded

Here μ_ij are prior probabilities derived from meteorological records. Since p_ij appear in (7) as arguments of the logarithmic function, nonnegativity of all p_ij is guaranteed.

In this particular example, p_ij may be expressed in terms of π:

View Expanded

and (7) becomes

View Expanded

Then, if there is an extreme point of H(π) inside Δ, it is a solution to the equation

View Expanded

Equation (11) is reduced to a quadratic equation and the final expression for π depends on particular values of input parameters (π_i, μ_ik). After some algebra, one arrives at the following solution. At first, it can be seen that (11) has no solution if

Dμ₁₂μ₂₁dπ₁π₂²dπ₁π₂μ₁₁μ₂₂(12)

Here

dμ₁₁μ₂₂μ₁₂μ₂₁(13)

In this case, (dH/dπ) < 0 and the maximum value of H is reached when

ππ₁π₂(14)

Next, if d = 0, then (11) has a unique solution:

ππ₁π₂π₁π₂(15)

If D ≥ 0 and d ≠ 0, Eq. (11) has two solutions and only one of them

View Expanded

may in principle belong to Δ. Whether it is the case or not depends on particular values of μ_ij. If the right-hand side of (16) is greater (less) than the upper (lower) bound of Δ, then (dH/dπ) > 0 when d > 0 (d < 0). Consequently, H is maximum at

ππ₁π₂(17)

On the other side, if d > 0 (d < 0) and the right-hand side of (16) is less (greater) than the lower (upper) bound of Δ, (dH/dπ) < 0 and H is maximum at

ππ₁π₂(18)

A final point that seems worth noting is the meaning of the probabilities p_ij. These are better thought not as relative frequencies of rain or no rain at a particular period but as the forecaster’s degree of belief in those events. In this sense, p_ij are “subjective” quantities rather than attributes of underlying stochastic atmospheric processes. However, those p_ij that were calculated by means of the PME are the most “objective” among others since they contain only information we do have and nothing more. Any other choice of p_ij rests on an extremely insecure foundation built with the use of assumptions about the probability distribution that were not drawn from the data. On the other hand, if we have experimental proof that our predictions based on the PME are incorrect, it will give evidence that the inputs (π_i, μ_ik) are wrong. Modern physics has been evolving just in this way.