Search Results
You are looking at 1 - 3 of 3 items for
- Author or Editor: Cynthia Lusk x
- Refine by Access: All Content x
In support of the National Weather Service (NWS) modernization, the Forecast Systems Laboratory (FSL) has been developing prototype hydrometeorological workstations for many years. The FSL Evaluation Team (E-Team) has developed a process of evaluating these prototypes to provide feedback to FSL developers and NWS management. This paper outlines the evaluation process with an emphasis on the methods, which consist of product usage logs, questionnaires, evaluation logs, structured tasks, observations, and interviews. Examples from E-Team evaluations illustrate how results from these different methods have led to improved workstation design, implementation, and training.
In support of the National Weather Service (NWS) modernization, the Forecast Systems Laboratory (FSL) has been developing prototype hydrometeorological workstations for many years. The FSL Evaluation Team (E-Team) has developed a process of evaluating these prototypes to provide feedback to FSL developers and NWS management. This paper outlines the evaluation process with an emphasis on the methods, which consist of product usage logs, questionnaires, evaluation logs, structured tasks, observations, and interviews. Examples from E-Team evaluations illustrate how results from these different methods have led to improved workstation design, implementation, and training.
Abstract
A variety of measures are used to judge the skill and accuracy with which forecasters predict the weather and to verify forecasts: Such measures can confound accuracy with decision strategy and sometimes give conflicting indications of performance. Signal detection theory (SDT) provides a theoretical framework for describing forecasting behavior and minimizing these problems. We illustrate the utility of signal detection theory in this context, show how it can be used to understand the effects of time pressure created by frequent weather activity on forecasting judgments, and illustrate how to achieve a specific social policy.
Abstract
A variety of measures are used to judge the skill and accuracy with which forecasters predict the weather and to verify forecasts: Such measures can confound accuracy with decision strategy and sometimes give conflicting indications of performance. Signal detection theory (SDT) provides a theoretical framework for describing forecasting behavior and minimizing these problems. We illustrate the utility of signal detection theory in this context, show how it can be used to understand the effects of time pressure created by frequent weather activity on forecasting judgments, and illustrate how to achieve a specific social policy.
Abstract
Two studies of microburst forecasting were conducted in order to demonstrate the utility of applying theoretical and methodological concepts from judgment and decision making to meteorology. A hierarchical model of the judgment process is outlined in which a precursor identification phase is separated from the prediction phase. In the first study, forecasters were provided with specific, unambiguous precursor values and were asked to provide judgments regarding the probability of a microburst. Results indicated that the meteorologists' forecast were adequately predicted by a linear model. Modest agreement was observed among the forecasters’ judgments. In the second study, forecasters viewed storms under dynamic conditions representative of their usual operational setting. They made judgments regarding precursor values, as well as of the probability of a microburst occurring. The forecasters’ agreement regarding microburst predictions was found to be lower than in the first study. Surprisingly, agreement regarding the (subjectively) most important precursor value was near zero. These results indicate that there are indeed practical advantages to be gained from a better understanding of the precursor identification and prediction phases of the forecasting process.
Abstract
Two studies of microburst forecasting were conducted in order to demonstrate the utility of applying theoretical and methodological concepts from judgment and decision making to meteorology. A hierarchical model of the judgment process is outlined in which a precursor identification phase is separated from the prediction phase. In the first study, forecasters were provided with specific, unambiguous precursor values and were asked to provide judgments regarding the probability of a microburst. Results indicated that the meteorologists' forecast were adequately predicted by a linear model. Modest agreement was observed among the forecasters’ judgments. In the second study, forecasters viewed storms under dynamic conditions representative of their usual operational setting. They made judgments regarding precursor values, as well as of the probability of a microburst occurring. The forecasters’ agreement regarding microburst predictions was found to be lower than in the first study. Surprisingly, agreement regarding the (subjectively) most important precursor value was near zero. These results indicate that there are indeed practical advantages to be gained from a better understanding of the precursor identification and prediction phases of the forecasting process.
