Search Results

You are looking at 21 - 30 of 87 items for

  • Author or Editor: ALLAN H. MURPHY x
  • Refine by Access: All Content x
Clear All Modify Search
Allan H. Murphy

Abstract

Full access
Allan H. Murphy

Abstract

Scalar and vector partitions of the probability score (PS) in N-state (N > 2) situations are described and compared. In N-state, as well as in two-state (N = 2), situations these partitions provide similar, but not equivalent (i.e., linearly related), measures of the reliability and resolution of probability forecasts. Specifically, the vector partition, when compared to the scalar partition, decreases the reliability and increases the resolution of the forecasts. A sample collection of forecasts is used to illustrate the differences between these partitions in N-state situations.

Several questions related to the use of scalar and vector partitions of the PS in N-state situations are discussed, including the relative merits of these partitions and the effect upon sample size when forecasts are considered to be vectors rather than scalars. The discussions indicate that the vector partition appears to be more appropriate, in general, than the scalar partition, and that when the forecasts in a collection of forecasts are considered to be vectors rather than scalars the sample size of the collection may be substantially reduced.

Full access
Allan H. Murphy

Abstract

An individual skill score (SS) and a collective skill score (CSS) are examined to determine whether these scoring or improper. The SS and the CSS are both standardized versions of the Brier, or probability, score (PS) and have been used to measure the “skill” of probability forecasts. The SS is defined in terms of individual forecasts, while the CSS is defined in terms of collections of forecasts. The SS and the CSS are shown to be improper scoring rules, and, as a result, both the SS and the CSS encourage hedging on the part of forecasters.

The results of a preliminary, investigation of the nature of the hedging produced by. the SS and the CSS indicate that, while the SS may encourage a considerable amount of hedging, the CSS, in general, encourages only a modest amount of hedging, and even this hedging decreases as the sample size K of the collection forecasts increases. In fact, the CSS is approximately strictly Proper for large collections of forecasts (K ≥ 100).

Finally, we briefly consider two questions related to the standardization of scoring rules: 1) the use of different scoring rules in the assessment and evaluation tasks, and 2) the transformation of strictly proper scoring rules. With regard to the latter, we identify standardized versions of the PS which are strictly proper scoring rules and which, as a result, appear to be appropriate scoring rules to use to measure the “skill” of probability forecasts.

Full access
Allan H. Murphy

Abstract

A new vector partition of the probability, or Brier, score (PS) is formulated and the nature and properties of this partition are described. The relationships between the terms in this partition and the terms in the original vector partition of the PS are indicated. The new partition consists of three terms: 1) a measure of the uncertainty inherent in the events, or states, on the occasions of concern (namely, the PS for the sample relative frequencies); 2) a measure of the reliability of the forecasts; and 3) a new measure of the resolution of the forecasts. These measures of reliability and resolution are and are not, respectively, equivalent (i.e., linearly related) to the measures of reliability and resolution provided by the original partition. Two sample collections of probability forecasts are used to illustrate the differences and relationships between these partitions. Finally, the two partitions are compared, with particular reference to the attributes of the forecasts with which the partitions are concerned, the interpretation of the partitions in geometric terms, and the use of the partitions as the bases for the formulation of measures to evaluate probability forecasts. The results of these comparisons indicate that the new partition offers certain advantages vis-à-vis the original partition.

Full access
Allan H. Murphy

Abstract

Scalar and vector partitions of the probability score (PS) in the two-state situation are described and compared. These partitions, which are based upon expressions for the PS in which probability forecasts are considered to be scalars and vectors, respectively, provide similar, but not equivalent (i.e., linearly related), measures of the reliability and resolution of the forecasts. Specifically, the reliability (resolution) of the forecasts according to the scalar partition is, in general, greater (less) than their reliability (resolution) according to the vector partition. A sample collection of forecasts is used to illustrate the differences between these partitions.

Several questions related to the use of scalar and vector partitions of the PS in the two-state situation are discussed, including the interpretation of the results of previous forecast evaluation studies and the relative merits of these partitions. The discussions indicate that the partition most often used in such studies has been a special “scalar” partition, a partition which is equivalent to the vector partition in the two-state situation, and that the vector partition is more appropriate than the scalar partition.

Full access
Allan H. Murphy

Abstract

Full access
Allan H. Murphy

Abstract

Comparative operational evaluation of probabilistic prediction procedures in cost-loss ratio decision situations in which the evaluator's knowledge of the cost-loss ratio is expressed in probabilistic terms is considered. First, the cost-loss ratio decision situation is described in a utility framework and, then, measures of the expected-utility of probabilistic predictions are formulated. Second, a class of expected-utility measures, the beta measures, in which the evaluator's knowledge of the cost-loss ratio is expressed in terms of a beta distribution, are described. Third, the beta measures are utilized to compare two prediction procedures on the basis of a small sample of predictions. The results indicate the importance, for comparative operational evaluation, of utilizing measures which provide a suitable description of the evaluator's knowledge. In particular, the use of the probability score, a measure equivalent to the uniform measure (which is a special beta measure), in decision situations in which the uniform distribution does not provide a suitable description of the evaluator's knowledge, may yield misleading results. Finally, the results are placed in proper perspective by describing several possible extensions to this study and by indicating the importance of undertaking such studies in actual operational situations.

Full access
Allan H. Murphy

Abstract

Full access
Allan H. Murphy

Abstract

Full access
Allan H. Murphy

This paper briefly examines the nature of hedging and its role in the formulation of categorical and probabilistic forecasts. Hedging is defined in terms of the difference between a forecaster's judgment and his forecast. It is then argued that a judgment cannot accurately reflect the forecaster's true state of knowledge unless the uncertainties inherent in the formulation of this judgment are described in a qualitative and/or quantitative manner. Since categorical forecasting does not provide the forecaster with a means of making his forecasts correspond to such judgments, a categorical forecast is generally a hedge. Probabilistic forecasting, on the other hand, presents the forecaster with an opportunity to eliminate hedging by making his (probabilistic) forecasts correspond exactly to his judgments. Thus, contrary to popular belief, the desire to eliminate hedging should encourage forecasters to express more rather than fewer forecasts in probabilistic terms.

Full access