Search Results

You are looking at 81 - 87 of 87 items for

  • Author or Editor: ALLAN H. MURPHY x
  • Refine by Access: All Content x
Clear All Modify Search
Carl-Axel S. Staël von Holstein
and
Allan H. Murphy

Abstract

A family of quadratic scoring rules (QSR's) is defined. Some properties of these scoring rules are described, and it is demonstrated that QSR's are strictly proper. The probability (or Brier) score and the ranked probability score are shown to be special cases of the general QSR.

A geometrical framework for the representation of QSR's is presented. This framework facilitates formulation of QSR's and provides insight into the properties of these scoring rules, including the sensitive-to-distance property. The relationships between QSR's and measures of the value of (probability) forecasts are briefly discussed.

The richness of the family of QSR's provides the evaluator with considerable flexibility in choosing a scoring rule that is particularly suited to the situation at hand.

Full access
Allan H. Murphy
and
Carl-Axel S. Staël von Holstein

Abstract

A geometrical framework for the representation of cumulative forecasts and observations is described. The ranked probability score is shown to be the square of the distance between the points in this framework which represent a cumulative forecast and the relevant cumulative observation. The relationship between this framework and the geometrical framework for the probability score is indicated.

Full access
Allan H. Murphy
,
Barbara G. Brown
, and
Yin-Sheng Chen

Abstract

A diagnostic approach to forecast verification is described and illustrated. This approach is based on a general framework for forecast verification. It is “diagnostic” in the sense that it focuses on the fundamental characteristics of the forecasts, the corresponding observations, and their relationship.

Three classes of diagnostic verification methods are identified: 1) the joint distribution of forecasts and observations and conditional and marginal distributions associated with factorizations of this joint distribution; 2) summary measures of these joint, conditional, and marginal distributions; and 3) performance measures and their decompositions. Linear regression models that can be used to describe the relationship between forecasts and observations are also presented. Graphical displays are advanced as a means of enhancing the utility of this body of diagnostic verification methodology.

A sample of National Weather Service maximum temperature forecasts (and observations) for Minneapolis, Minnesota, is analyzed to illustrate the use of this methodology. Graphical displays of the basic distributions and various summary measures are employed to obtain insights into distributional characteristics such as central tendency, variability, and asymmetry. The displays also facilitate the comparison of these characteristics among distributions–for example, between distributions involving forecasts and observations, among distributions involving different types of forecasts, and among distributions involving forecasts for different seasons or lead times. Performance measures and their decompositions are shown to provide quantitative information regarding basic dimensions of forecast quality such as bias, accuracy, calibration (or reliability), discrimination, and skill. Information regarding both distributional and performance characteristics is needed by modelers and forecasters concerned with improving forecast quality. Some implications of these diagnostic methods for verification procedures and practices are discussed.

Full access
Allan H. Murphy
,
Sarah Lichtenstein
,
Baruch Fischhoff
, and
Robert L. Winkler

Previous studies have suggested that the general public misinterprets probability of precipitation (PoP) forecasts, leading some meteorologists to argue that probabilities should not be included in public weather forecasts. Upon closer examination, however, these studies prove to be ambiguous with regard to the nature of the misunderstanding. Is the public confused about the meaning of the probabilities or about the definition of the event to which the probabilities refer? If event misinterpretation is the source of the confusion, then elimination of the probabilities would not reduce the level of misunderstanding.

The present paper summarizes a study of 79 residents of Eugene, Oreg., who completed a questionnaire designed to investigate their understanding of and attitude toward precipitation probability forecasts. Results indicate that the event in question frequently is misunderstood, with both traditional precipitation forecasts and PoP forecasts producing similar levels of event misinterpretation. On the other hand, the probabilities themselves are well understood. Moreover, most respondents revealed a preference for the use of probabilities to express the uncertainty inherent in precipitation forecasts. Although the sample size was limited, the results of this study strongly support the inclusion of probabilities in public forecasts of precipitation occurrence. The paper concludes with a brief discussion of some implications of these results for operational weather forecasting.

Full access
Allan H. Murphy
,
Wu-ron Hsu
,
Robert L. Winkler
, and
Daniel S. Wilks

Abstract

This paper summarizes the results of an experiment in which National Weather Service forecasters formulated probabilistic quantitative precipitation forecasts (QPFs) during a 17-month period in 1981–82. These forecasts expressed the likelihood that certain threshold amounts of precipitation would be equaled or exceeded in 12-hour periods at four locations in Texas. The forecasters had no previous experience in quantifying the uncertainty in such forecasts, but they did receive feedback regarding their collective performance at the end of the first year of the experiment. In the evaluation of the experimental results, particular attention is focused on three issues: 1) the reliability and skill of the subjective QPFs; 2) the effects of feedback and experience on the quality of these forecasts; and 3) the relative performance of the subjective probabilistic QPFs and objective probabilistic QPFs produced by the model output statistics system.

The subjective probabilistic QPFs possess positive skill, although they exhibit considerable overforecasting for larger precipitation amounts. Moreover, the feedback provided to the forecasters evidently contributed to modest increases in the reliability and skill of their forecasts. In this regard, the quality of the subjective and objective QPFs is generally comparable in the first year of the experiment. However, after the receipt of the feedback, the skill of the subjective forecasts exceeded the skill of the objective forecasts. These results are considered to be encouraging regarding the ability of forecasters to formulate reliable and skillful probabilistic QPFS, but more extensive experiments should be undertaken to investigate this and related issues in greater detail.

Full access
Allan H. Murphy
,
Richard W. Katz
,
Robert L. Winkler
, and
Wu-Ron Hsu

Abstract

The purposes of this paper are to describe a dynamic model for repetitive decision‐making in the cost–loss ratio situation and to present some theoretical and numerical results related to the optimal use and economic value of weather forecasts within the framework of the model. This model involves the same actions and events as the standard (i.e., static) cost–loss ratio situation, but the former (unlike the latter) is dynamic in the sense that it possesses characteristics (e.g., decisions, events) that are related over time. We assume that the decision maker wants to choose the sequence of actions over an n‐occasion time period that minimizes the total expected expense. A computational technique known as stochastic dynamic programming is employed to determine this optimal policy and the total expected expense.

Three types of weather information are considered in studying the value of forecasts in this context: 1) climatological information; 2) perfect information; and 3) imperfect forecasts. Climatological and perfect information represent lower and upper bounds, respectively, on the quality of all imperfect forecasts, with the latter considered here to be categorical forecasts properly calibrated according to their past performance. Theoretical results are presented regarding the form of the optimal policy and the relationship among the total expected expenses for these three types of information. In addition, quality/value relationships for imperfect forecasts are described.

Numerical results are derived from the dynamic model for specific values of the model parameters. These results include the optimal policy and the economic value of perfect and imperfect forecasts for various time horizons, climatological probabilities, and values of the cost–loss ratio. The relationship between the accuracy and value of imperfect forecasts also is examined.

Several possible extensions of this dynamic model are briefly discussed, including decision‐making problems involving more actions and/or events, more complex structures of the costs and losses, and more general forms of imperfect forecasts (e.g., probability forecasts).

Full access
P. W. Thorne
,
R. J. Allan
,
L. Ashcroft
,
P. Brohan
,
R. J. H Dunn
,
M. J. Menne
,
P. R. Pearce
,
J. Picas
,
K. M. Willett
,
M. Benoy
,
S. Bronnimann
,
P. O. Canziani
,
J. Coll
,
R. Crouthamel
,
G. P. Compo
,
D. Cuppett
,
M. Curley
,
C. Duffy
,
I. Gillespie
,
J. Guijarro
,
S. Jourdain
,
E. C. Kent
,
H. Kubota
,
T. P. Legg
,
Q. Li
,
J. Matsumoto
,
C. Murphy
,
N. A. Rayner
,
J. J. Rennie
,
E. Rustemeier
,
L. C. Slivinski
,
V. Slonosky
,
A. Squintu
,
B. Tinz
,
M. A. Valente
,
S. Walsh
,
X. L. Wang
,
N. Westcott
,
K. Wood
,
S. D. Woodruff
, and
S. J. Worley

Abstract

Observations are the foundation for understanding the climate system. Yet, currently available land meteorological data are highly fractured into various global, regional, and national holdings for different variables and time scales, from a variety of sources, and in a mixture of formats. Added to this, many data are still inaccessible for analysis and usage. To meet modern scientific and societal demands as well as emerging needs such as the provision of climate services, it is essential that we improve the management and curation of available land-based meteorological holdings. We need a comprehensive global set of data holdings, of known provenance, that is truly integrated both across essential climate variables (ECVs) and across time scales to meet the broad range of stakeholder needs. These holdings must be easily discoverable, made available in accessible formats, and backed up by multitiered user support. The present paper provides a high-level overview, based upon broad community input, of the steps that are required to bring about this integration. The significant challenge is to find a sustained means to realize this vision. This requires a long-term international program. The database that results will transform our collective ability to provide societally relevant research, analysis, and predictions in many weather- and climate-related application areas across much of the globe.

Open access