An Experiment to Measure the Value of Statistical Probability Forecasts for Airports

Ross Keith Bureau of Meteorology, and James Cook University, Townsville, Queensland, Australia

Search for other papers by Ross Keith in
Current site
Google Scholar
PubMed
Close
and
Stephen M. Leyton Center for Analysis and Prediction of Storms, University of Oklahoma, Norman, Oklahoma

Search for other papers by Stephen M. Leyton in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

The economic value of weather forecasts for airports for commercial aviation is investigated by introducing financial data into the decision-making process for fuel carriage by aircraft. Using specific operating costs for a given flight, an optimal decision probability threshold can be calculated that identifies whether that flight should carry extra fuel, in case of adverse weather conditions and subsequent diversion. Forecasts of these adverse conditions can then be applied to a critical threshold to make a real-time decision regarding the carriage of additional fuel. This study focuses on forecasts of low ceiling and/or reduced visibility and their corresponding impact on forecast value for flights arriving at three major airports in the United States. Eighteen daily flights by American Airlines were examined during a 14-month period, a total of approximately 7500 flights. Using operating cost data from this period, a critical decision threshold was derived for each daily flight. Two sets of forecasts, statistically derived probabilistic forecasts and National Weather Service terminal aerodrome forecasts (TAFs), were then applied to each flight’s fuel carriage decision-making process. The probabilistic forecasts, which utilize regional surface observations, were generated for the destination airport with a lead time appropriate to the airline’s flight planning time. If the forecast probability of adverse weather was greater than the critical decision threshold for a given flight, then additional fuel was deemed necessary for that flight. The categorical TAFs that corresponded timewise to the developed probabilistic forecasts were obtained for each location. For this study, a categorical “yes” forecast denotes the expectation that the visibility and/or cloud ceiling conditions are such that extra fuel is required, while a categorical “no” forecast does not require extra fuel. The analysis presented herein indicates that by using statistical, probabilistic forecasts rather than categorical forecasts, a significant saving is made in operating costs. This is probably because of a more optimal balance between false alarms and misses for each flight, rather than more “accurate” forecasts per se. This is the mechanism by which probabilistic forecasts create value, rather than increasing the number of hits and correct rejections and/or decreasing the number of false alarms and misses. For each of the flights investigated in this study, the total cost of using probabilistic forecasts was less than that of using TAFs. An average of $23,000 is saved per flight during this 14-month period. Projecting these figures over all American Airlines flights, a potential annual savings of approximately $50 million in operating costs would be realized by using probabilistic forecasts of adverse landing weather conditions instead of the traditional TAF.

Corresponding author address: Ross Keith, 14 Lawson St., Townsville 4812, QLD, Australia. Email: keiths14@bigpond.net.au

Abstract

The economic value of weather forecasts for airports for commercial aviation is investigated by introducing financial data into the decision-making process for fuel carriage by aircraft. Using specific operating costs for a given flight, an optimal decision probability threshold can be calculated that identifies whether that flight should carry extra fuel, in case of adverse weather conditions and subsequent diversion. Forecasts of these adverse conditions can then be applied to a critical threshold to make a real-time decision regarding the carriage of additional fuel. This study focuses on forecasts of low ceiling and/or reduced visibility and their corresponding impact on forecast value for flights arriving at three major airports in the United States. Eighteen daily flights by American Airlines were examined during a 14-month period, a total of approximately 7500 flights. Using operating cost data from this period, a critical decision threshold was derived for each daily flight. Two sets of forecasts, statistically derived probabilistic forecasts and National Weather Service terminal aerodrome forecasts (TAFs), were then applied to each flight’s fuel carriage decision-making process. The probabilistic forecasts, which utilize regional surface observations, were generated for the destination airport with a lead time appropriate to the airline’s flight planning time. If the forecast probability of adverse weather was greater than the critical decision threshold for a given flight, then additional fuel was deemed necessary for that flight. The categorical TAFs that corresponded timewise to the developed probabilistic forecasts were obtained for each location. For this study, a categorical “yes” forecast denotes the expectation that the visibility and/or cloud ceiling conditions are such that extra fuel is required, while a categorical “no” forecast does not require extra fuel. The analysis presented herein indicates that by using statistical, probabilistic forecasts rather than categorical forecasts, a significant saving is made in operating costs. This is probably because of a more optimal balance between false alarms and misses for each flight, rather than more “accurate” forecasts per se. This is the mechanism by which probabilistic forecasts create value, rather than increasing the number of hits and correct rejections and/or decreasing the number of false alarms and misses. For each of the flights investigated in this study, the total cost of using probabilistic forecasts was less than that of using TAFs. An average of $23,000 is saved per flight during this 14-month period. Projecting these figures over all American Airlines flights, a potential annual savings of approximately $50 million in operating costs would be realized by using probabilistic forecasts of adverse landing weather conditions instead of the traditional TAF.

Corresponding author address: Ross Keith, 14 Lawson St., Townsville 4812, QLD, Australia. Email: keiths14@bigpond.net.au

Save
  • Dallavalle, J. P., and Dagostaro V. J. , 1995: The accuracy of ceiling and visibility forecasts produced by the National Weather Service. Preprints, Sixth Conf. on Probability and Statistics in the Atmospheric Sciences, Toronto, ON, Canada, Amer. Meteor. Soc., 201–206.

  • Harvey L. O. Jr., , Hammond K. R. , Lusk C. M. , and Mross E. F. , 1992: The application of signal detection theory to weather forecasting behavior. Mon. Wea. Rev., 120 , 863883.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Keith, R., 2003: Optimization of value of aerodrome forecasts. Wea. Forecasting, 18 , 808824.

  • Leyton, S. M., and Fritsch J. M. , 2003: Short-term probabilistic forecasts of ceiling and visibility utilizing high-density surface weather observations. Wea. Forecasting, 18 , 891902.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leyton, S. M., and Fritsch J. M. , 2004: The impact of high-frequency surface weather observations on short-term probabilistic forecasts of ceilings and visibility. J. Appl. Meteor., 43 , 145156.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mason, I., 2004: The cost of uncertainty in weather prediction: Modeling quality-value relationships for yes/no forecasts. Aust. Meteor. Mag., 30 , 111122.

    • Search Google Scholar
    • Export Citation
  • Stephenson, D. B., 2000: Use of the “odds ratio” for diagnosing forecast skill. Wea. Forecasting, 15 , 221232.

  • Vislocky, R. L., and Fritsch J. M. , 1997: An automated, observations-based system for short-term prediction of ceiling and visibility. Wea. Forecasting, 12 , 3143.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 301 99 4
PDF Downloads 210 78 4