Search Results

You are looking at 11 - 20 of 87 items for

  • Author or Editor: ALLAN H. MURPHY x
  • Refine by Access: All Content x
Clear All Modify Search
Allan H. Murphy

Abstract

Full access
Allan H. Murphy

Abstract

Full access
Allan H. Murphy

This paper briefly examines the nature of hedging and its role in the formulation of categorical and probabilistic forecasts. Hedging is defined in terms of the difference between a forecaster's judgment and his forecast. It is then argued that a judgment cannot accurately reflect the forecaster's true state of knowledge unless the uncertainties inherent in the formulation of this judgment are described in a qualitative and/or quantitative manner. Since categorical forecasting does not provide the forecaster with a means of making his forecasts correspond to such judgments, a categorical forecast is generally a hedge. Probabilistic forecasting, on the other hand, presents the forecaster with an opportunity to eliminate hedging by making his (probabilistic) forecasts correspond exactly to his judgments. Thus, contrary to popular belief, the desire to eliminate hedging should encourage forecasters to express more rather than fewer forecasts in probabilistic terms.

Full access
Allan H. Murphy

Abstract

Two fundamental characteristics of forecast verification problems—complexity and dimensionality—are described. To develop quantitative definitions of these characteristics, a general framework for the problem of absolute verification (AV) is extended to the problem of comparative verification (CV). Absolute verification focuses on the performance of individual forecasting systems (or forecasters), and it is based on the bivariate distribution of forecasts and observations and its two possible factorizations into conditional and marginal distributions.

Comparative verification compares the performance of two or more forecasting systems, which may produce forecasts under 1) identical conditions or 2) different conditions. The first type of CV is matched comparative verification, and it is based on a 3-yariable distribution with possible factorizations. The second and more complicated type of CV is unmatched comparative verification, and it is based on a 4-variable distribution with 24 possible factorizations.

Complexity can be defined in terms of the number of factorizations, the number of basic factors (conditional and marginal distributions) in each factorization, or the total number of basic factors associated with the respective frameworks. These definitions provide quantitative insight into basic differences in complexity among AV and CV problems. Verification problems involving probabilistic and nonprobabilistic forecasts are of equal complexity.

Dimensionality is defined as the number of probabilities that must be specified to reconstruct the basic distribution of forecasts and observations. It is one less than the total number of distinct combinations of forecasts and observations. Thus, CV problems are of higher dimensionality than AV problems, and problems involving probabilistic forecasts or multivalued nonprobabilistic forecasts exhibit particularly high dimensionality.

Issues related to the implications of these concepts for verification procedures and practices are discussed, including the reduction of complexity and/or dimensionality. Comparative verification problems can be reduced in complexity by making forecasts under identical conditions or by assuming conditional or unconditional independence when warranted. Dimensionality can be reduced by parametric statistical modeling of the distributions of forecasts and/or observations.

Failure to take account of the complexity and dimensionality of verification problems may lead to an incomplete and inefficient body of verification methodology and, thereby, to erroneous conclusions regarding the absolute and relative quality and/or value of forecasting systems.

Full access
Allan H. Murphy

Abstract

Several skill scores are defined, based on the mean-square-error measure of accuracy and alternative climatological standards of reference. Decompositions of these skill scores are formulated, each of which is shown to possess terms involving 1) the coefficient of correlation between the forecasts and observations, 2) a measure of the nonsystematic (i.e., conditional) bias in the forecast, and 3) a measure of the systematic (i.e., unconditional) bias in the forecasts. Depending on the choice of standard of reference, a particular decomposition may also contain terms relating to the degree of association between the reference forecasts and the observations. These decompositions yield analytical relationships between the respective skill scores and the correlation coefficient, document fundamental deficiencies in the correlation coefficient as a measure of performance, and provide additional insight into basic characteristics of forecasting performance. Samples of operational precipitation probability and minimum temperature forecasts are used to investigate the typical magnitudes of the terms in the decompositions. Some implications of the results for the practice of forecast verification are discussed.

Full access
Allan H. Murphy

Abstract

Situations sometimes arise in which it is necessary to evaluate and compare the performance of categorical and probabilistic forecasts. The traditional approach to this problem involves the transformation of the probabilistic forecasts into categorical forecasts and the comparison of the two sets of forecasts in a categorical framework. This approach suffers from several serious deficiencies. Alternative approaches are proposed here that consist in (i) treating the categorical forecasts as probabilistic forecasts or (ii) replacing the categorical forecasts with primitive probabilistic forecasts. These approaches permit the sets of forecasts to be compared in a probabilistic framework and offer several important advantages vis-a-vis the traditional approach. The proposed approaches are compared and some issues related to these approaches and the overall problem itself are discussed.

Full access
Allan H. Murphy

Abstract

A new decomposition of the Brier score is described. This decomposition is based on conditional distributions of forecast probabilities given observed events, and, as a result, it differs in a fundamental way from most previous partitions of quadratic verification measures. The new decomposition consists of 1) a term involving the variances of the conditional distributions and 2) a term related to the mean errors in the forecasts, which involves the squared differences between the means of the conditional distributions and the respective mean observations (the latter are necessarily either zero or one). Decrease in these variances and/or mean errors generally lead to improvements in the Brier score. The decomposition may be useful in verification studies, since it appears to provide additional insight into the quality of probabilistic forecasts.

Full access
Allan H. Murphy

Abstract

Meteorologists have devoted considerable attention to studies of the use and value of forecasts in a simple two-action, two-event decision-making problem generally referred to as the cost-loss ratio situation, An N-action, N-event generalization of the standard cost-loss ratio situation is described here, and the expected value of different types of forecasts in this situation is investigated. Specifically, expressions are developed for the expected expenses associated with the use of climatological, imperfect, and perfect information, and these expressions are employed to derive formulas for the expected value of imperfect and perfect forecasts. The three-action, three-event situation is used to illustrate the generalized model and the value-information results, by considering examples based on specific numerical values of the relevant parameters. Some possible extensions of this model are briefly discussed.

Full access
Allan H. Murphy

Abstract

No abstract available.

Full access
ALLAN H. MURPHY

Abstract

In this paper, we compare the ranked probability score (RPS) and the probability score (PS) and examine the nature of the sensitivity of the RPS to distance. First, we briefly describe the nature of and the relationship between the frameworks within which the RPS and the PS were formulated. Second, we consider certain properties of the RPS and the PS including their range, their values for categorical and uniform forecasts, and their “proper” nature. Third, we describe the RPS and the PS in a manner that reveals the structure of and the relationship between these scoring rules. Fourth, we considered the RPS with reference to two definitions of distance and examine the nature of the sensitivity of the RPS to distance. The comparison of the RPS and the PS suggests that the RPS rather than the PS should be used to evaluate probability forecats, at least in those situations in which the variable of concern is ordered.

Full access