Search Results

You are looking at 1 - 10 of 16 items for :

  • Author or Editor: ALLAN H. MURPHY x
  • Weather and Forecasting x
  • Refine by Access: All Content x
Clear All Modify Search
Allan H. Murphy

Abstract

Heretofore it has been widely accepted that the contributions of W. E. Cooke in 1906 represented the first works related to the explicit treatment of uncertainty in weather forecasts. Recently, however, it has come to light that at least some aspects of the rationale for quantifying the uncertainty in forecasts were discussed prior to 1900 and that probabilities and odds were included in some weather forecasts formulated more than 200 years ago. An effort to summarize these new historical insights, as well as to clarify the precise nature of the contributions made by various individuals to early developments is this area, appears warranted.

The overall purpose of this paper is to extend and clarify the early history of probability forecasts. Highlights of the historical review include 1) various examples of the use of qualitative and quantitative probabilities or odds in forecasts during the eighteenth and nineteenth centuries, 2) a brief discussion in 1890 of the economic component of the rationale for quantifying the uncertainty in forecasts, 3) further refinement of the rationale for probability forecasts and the presentation of the results of experiments involving the formulation of quasi-probabilistic and probabilistic forecasts during the period 1900–25 (in reviewing developments during this early twentieth century period, the noteworthy contributions made by W. E. Cooke, C. Hallenbeck, and A. K. Ångström are described and clarified), and 4) a very concise overview of activities and developments in this area since 1925.

The early treatment of some basic issues related to probability forecasts is discussed and, in some cases, compared to their treatment in more recent times. These issues include 1) the underlying rationale for probability forecasts, 2) the feasibility of making probability forecasts, and 3) alternative interpretations of probability in the context of weather forecasts. A brief examination of factors related to the acceptance of—and resistance to—probability forecasts in the meteorological and user communities is also included.

Full access
Allan H. Murphy

Abstract

Several issues related to the mode of expression of forecasts of rare events (RSEs) are addressed in this paper. These issues include the correspondence between forecasters' judgments and their forecasts, the problem of overforecasting, and the use of forecasts as a basis for rational decision making. Neither forecasters nor users are well served by current practices, according to which operational forecasts of RSEs are generally expressed in a categorical format.

It is argued here that sound scientific and economic reasons exist for expressing forecasts of RSEs in terms of probabilities. Although quantification of uncertainty in forecasts of RSEs–-and the communication of such information to users–-presents some special problems, evidence accumulated from a multitude of operational and experimental probabilistic weather forecasting programs suggests that these problems involve no insurmountable difficulties. Moreover, when a probabilistic format is employed, forecasts of RSEs can correspond to forecasters’ true judgments, the forecasting and decision-making tasks can be disentangled, the rationale for overforecasting RSEs is eliminated, and the needs of all users can be met in an optimal manner.

Since the probabilities of RSEs seldom achieve high values, it might be desirable to provide users with information concerning the likelihood of such events relative to their climatological likelihood. Alternatively, the relative odds–-that is, the ratio of an event's forecast odds to its climatological odds–-could be reported. This supplemental information should help to focus users’ attention on those occasions on which the probability of RSEs is relatively high.

Full access
Allan H. Murphy

Abstract

Skill scores measure the accuracy of the forecasts Of interest relative to the accuracy Of forecasts based on naive forecasting methods, with either climatology or persistence usually playing the role of the naive method. In formulating skill scores, it is generally agreed that the naive method that produces the most accurate forecasts should be chosen as the standard of reference. The conditions under which climatological forecasts are more accurate than persistence forecasts—and vice versa—were first described in the meteorological literature more than 30 years ago. At about the same time, it was also shown that a linear combination of climatology and persistence produces more accurate forecasts than either of these standards of reference alone. Surprisingly, these results have had relatively little if any impact on the practice of forecast verification in general and the choice of a standard of reference in formulating skill scorn in particular.

The purposes of this paper are to describe these results and discuss their implications for the practice of forecast verification. Expressions for the mean-square errors of forecasts based on climatology, persistence, and an optimal linear combination of climatology and persistence—as well as expressions for the respective skill scores—are presented and compared. These pairwise comparisons identify the conditions under which each naive method is superior as a standard of reference. Since the optimal linear combination produces more accurate forecasts than either climatology or persistence alone, it leads to lower skill scores than the other two naive forecasting methods. Decreases in the values of the skill scores associated with many types of operational weather forecasts can be anticipated if the optimal linear combination of climatology and persistence is used as a standard of reference. The conditions under which this practice might lead to substantial decreases in such skill scores are identified.

Full access
Allan H. Murphy

Abstract

Differences of opinion exist among forecasters—and between forecasters and users—regarding the meaning of the phrase “good (bad) weather forecasts.” These differences of opinion are fueled by a lack of clarity and/or understanding concerning the nature of goodness in weather forecasting. This lack of clarity and understanding complicates the processes of formulating and evaluating weather forecasts and undermines their ultimate usefulness.

Three distinct types of goodness are identified in this paper: 1) the correspondence between forecasters’ judgments and their forecasts (type 1 goodness, or consistency), 2) the correspondence between the forecasts and the matching observations (type 2 goodness, or quality), and 3) the incremental economic and/or other benefits realized by decision makers through the use of the forecasts (type 3 goodness, or value). Each type of goodness is defined and described in some detail. In addition, issues related to the measurement of consistency, quality, and value are discussed.

Relationships among the three types of goodness are also considered. It is shown by example that the level of consistency directly impacts the levels of both quality and value. Moreover, recent studies of quality/value relationships have revealed that these relationships are inherently nonlinear and may not be monotonic unless the multifaceted nature of quality is respected. Some implications of these considerations for various practices related to operational forecasting are discussed. Changes in these practices that could enhance the goodness of weather forecasts in one or more respects are identified.

Full access
Allan H. Murphy

Abstract

This paper is concerned with the use of the coefficient of correlation (CoC) and the coefficient of determination (CoD) as performance measures in forecast verification. Aspects of forecasting performance that are measured—and not measured (i.e., ignored)—by these coefficients are identified. Decompositions of familiar quadratic measures of accuracy and skill are used to explore differences between these quadratic measures and the coefficients of correlation and determination. A linear regression model, in which forecasts are regressed on observations, is introduced to provide insight into the interpretations of the CoC and the CoD in this context.

Issues related to the use of these coefficients as verification measures are discussed, including the deficiencies inherent in one-dimensional measures of overall performance, the pros and cons of quadratic measures of accuracy and skill vis-à-vis the coefficients of correlation and determination, and the relative merits of the CoC and the CoD. These coefficients by themselves do not provide an adequate basis for drawing firm conclusions regarding absolute or relative forecasting performance.

Full access
Allan H. Murphy

Abstract

In 1884 a paper by J.P. Finley appeared in the American Meteorological Journal describing the results of an experimental tornado forecasting program in the central and eastern United States. Finley's paper reported “percentages of verifications” exceeding 95%, where this index of performance was defined as the percentage of correct tornado/no-tornado forecasts. Within six months, three papers had appeared that identified deficiencies in Finley's method of verification and/or proposed alternative measures of forecasting performance in the context of this 2×2 verification problem. During the period from 1885 to 1893, several other authors in the United States and Europe, in most cases stimulated either by Finley's paper or by the three early responses, made noteworthy contributions to methods-oriented and practices-oriented discussions of issues related to forecast verification in general and verification of tornado forecasts in particular.

The burst of verification-related activities during the period 1884–1893 is referred to here as the “Finley affair.” It marked the beginning of substantive conceptual and methodological developments and discussions in the important subdiscipline of forecast verification. This paper describes the events that constitute the Finley affair in some detail and attempts to place this affair in proper historical context from the perspective of the mid-1990s. Whatever their individual strengths and weaknesses, the measures introduced during the period from 1884 to 1893 have withstood important tests of time—for example, these measures have been rediscovered on one or more occasions and they are still widely used today (generally under names assigned since 1900). Moreover, many of the issues vis-à-vis forecast verification that were first raised during the Finley affair remain issues of considerable importance more than 100 years later.

Full access
Allan H. Murphy

Abstract

No abstract available

Full access
Allan H. Murphy

Abstract

No abstract available

Full access
Allan H. Murphy
and
Thomas E. Sabin

Abstract

This paper describes the results of a study of trends in the quality of National Weather Service (NWS) forecasts from 1967 to 1985. Primary attention is focused on forecasts of precipitation probabilities, maximum temperatures, and minimum temperatures A skill score based on the Brier score is used to verify the precipitation probability forecasts, whereas the temperature forecasts are evaluated using the mean absolute error and percentage of errors greater than 10°F. For each element, trends are examined for objective forecasts produced by numerical-statistical models and for subjective forecasts formulated by NWS forecasters. In addition to weather element, type of forecast, and verification measure, results are stratified by season (cool and warm), lead time (three or four periods), and NWS region (four regions and all regions combined).

At the national level, the forecasts for these three weather elements exhibit positive and highly significant trends in quality for almost all of the various stratifications. Exceptions to this general result are associated solely with the minimum temperature forecasts, primarily for the 60 h lead time. These national trends are generally stronger for the objective forecasts than for the subjective forecasts and for the cool season than for the warm season. Regionally, the trends in quality are almost always positive and are statistically significant in a majority of the cases. However, nonsignificant trends occur more frequently at the regional level than at the national level. As a result of the positive trends in performance, current levels of forecast quality for these weather elements are markedly higher than the levels that existed 15–20 years ago.

Full access
Robert T. Clemen
and
Allan H. Murphy

Abstract

This paper addresses two specific questions related to the interrelationships between objective and subjective probability of precipitation (PoP) forecasts: Do the subjective forecasts contain information not included in the objective forecasts? Do the subjective forecasts make full use of the objective forecasts? With respect to the first question, an analysis of more than 11 years of data indicates that the subjective PoP forecasts add information above and beyond that contained in the objective PoP forecasts for all combinations of geographical area, lead time, and season investigated in this study. For longer lead times, this conclusion appears to contradict the results of earlier studies in which the two types of PoP forecasts were compared using aggregate skill scores. With regard to the second question, the statistical results demonstrate that the subjective forecasts generally do not make full use of the objective forecasts. However, these latter results are not as strong, in a statistical sense, as the results related to the first question; moreover, they indicate that it is primarily in the vicinity of the climatological probability (i.e., 0.10 to 0.40) that better use could be made of the objective forecasts. This conclusion suggests that it may be possible to combine the objective and subjective forecasts to produce a PoP forecast with even greater information content.

Full access