The economic value of weather forecasts for airports for commercial aviation is investigated by introducing financial data into the decision-making process for fuel carriage by aircraft. Using specific operating costs for a given flight, an optimal decision probability threshold can be calculated that identifies whether that flight should carry extra fuel, in case of adverse weather conditions and subsequent diversion. Forecasts of these adverse conditions can then be applied to a critical threshold to make a real-time decision regarding the carriage of additional fuel. This study focuses on forecasts of low ceiling and/or reduced visibility and their corresponding impact on forecast value for flights arriving at three major airports in the United States. Eighteen daily flights by American Airlines were examined during a 14-month period, a total of approximately 7500 flights. Using operating cost data from this period, a critical decision threshold was derived for each daily flight. Two sets of forecasts, statistically derived probabilistic forecasts and National Weather Service terminal aerodrome forecasts (TAFs), were then applied to each flight’s fuel carriage decision-making process. The probabilistic forecasts, which utilize regional surface observations, were generated for the destination airport with a lead time appropriate to the airline’s flight planning time. If the forecast probability of adverse weather was greater than the critical decision threshold for a given flight, then additional fuel was deemed necessary for that flight. The categorical TAFs that corresponded timewise to the developed probabilistic forecasts were obtained for each location. For this study, a categorical “yes” forecast denotes the expectation that the visibility and/or cloud ceiling conditions are such that extra fuel is required, while a categorical “no” forecast does not require extra fuel. The analysis presented herein indicates that by using statistical, probabilistic forecasts rather than categorical forecasts, a significant saving is made in operating costs. This is probably because of a more optimal balance between false alarms and misses for each flight, rather than more “accurate” forecasts per se. This is the mechanism by which probabilistic forecasts create value, rather than increasing the number of hits and correct rejections and/or decreasing the number of false alarms and misses. For each of the flights investigated in this study, the total cost of using probabilistic forecasts was less than that of using TAFs. An average of $23,000 is saved per flight during this 14-month period. Projecting these figures over all American Airlines flights, a potential annual savings of approximately $50 million in operating costs would be realized by using probabilistic forecasts of adverse landing weather conditions instead of the traditional TAF.
Commercial aircraft taking off and landing around the country regularly have to deal with the possible occurrences of adverse weather conditions that require decisions to be made that could be different from those made under ideal flying conditions. Perhaps visibility is so poor that airports that normally use side-by-side dual-runway approaches must reduce to single-runway approaches, cutting the flow rate by 50%. Maybe thunderstorms are expected to cross over an airport and planes may be forced into a holding pattern until the weather improves. In these instances and many others, decisions have to be made either prior to departure or en route that have significant economic impacts.
Consider a destination airport in which adverse weather is a possibility. First, the airline dispatch must decide whether the situation warrants the carriage of additional fuel in case a deviation from the existing flight plan is necessary. Next, the plane departs but conditions may or may not be suitable for landing as the plane approaches its destination. If conditions at the destination airport are or become unfavorable for landing, depending on whether additional fuel is being carried, the pilot can decide to divert immediately to a predetermined alternate airport, go into a holding pattern for a period of time hoping that weather conditions improve, or continue to the destination airport, attempt to land one or more times and then either safely land or divert to the alternate airport. If conditions at the destination are favorable for landing, regardless of whether additional fuel is being carried, the plane continues to its destination and safely lands. Obviously, the amount of fuel being carried by the aircraft affects the range of choices that can be made in a given situation.
Because each additional gallon of fuel has an inherent weight, the choice to load additional fuel requires even more additional fuel (i.e., carriage fuel) just to carry the extra weight. If additional fuel is carried but not utilized, then the carriage fuel is burned unnecessarily, resulting in an excessive fuel expense. If additional fuel is not carried but needed, then the airline must bear the very expensive cost of a diversion. It is important to note that even with the extra fuel, weather conditions may be so poor that a holding pattern or diversion may still be warranted. Because each gallon of fuel has an inherent cost, it is in the best interest of the airline to balance the cost of carrying additional fuel when it may not be necessary with the cost of holding patterns and diversions. To optimize this decision and minimize excessive costs, it is imperative that a rational decision rule be developed to determine whether a given flight should carry the additional fuel. This decision rule must be unique to each flight and would be represented by a “critical” threshold probability of adverse weather conditions occurring.
A cost-based decision-making model is an important step in reducing costs but in order to utilize this system, weather forecasts must be expressed as a probability of occurrence for a given condition. The current system, terminal aerodrome forecasts (TAFs), contains deterministic forecasts (i.e., “yes” representing 100% probability or “no” representing 0% probability) of low ceiling and visibility. However, an alternate forecasting method, using recent research findings, would be to develop short-term probabilistic forecasts of low ceiling and visibility that range from 0% to 100%, inclusive. These probabilistic forecasts could be used to trigger the decision, with a probability less than the “critical” threshold resulting in no additional fuel added and a probability greater than the “critical” threshold resulting in additional fuel being added.
This manuscript brings these ideas together in a case study of 18 daily flights over a span of 14 months by a major U.S. commercial airline in which actual flight costs are identified and a decision rule is created for each flight, which is, in turn, triggered by the probability of adverse weather conditions occurring. The estimated costs incurred by utilizing the current deterministic forecasts to make fuel carriage decisions are then compared to the estimated costs when utilizing probabilistic forecasts. It is found that for each flight route examined, the use of probabilistic forecasts results in a greater cost savings. In fact, extrapolation of these costs savings to a major carrier’s entire fleet yields an annual savings of as much as $50 million.
2. The cost-based decision rule for fuel carriage
Federal Aviation Administration (FAA) regulations require airlines to plan sufficient fuel for an aircraft to proceed to an alternate destination airport if a weather forecast predicts a cloud ceiling of ≤2000 ft and/or visibility < 3 statutory miles (SM). This is referred to as the alternate minimum (sometimes referred to as “alt. min.”) threshold. The decision whether to carry additional fuel is complicated by the fact that most flights can still land at an airport even when the ceiling is at or below 2000 ft and/or the visibility is less than 3 SM. The very existence of the alternate minimum threshold is as a mandated safety buffer to account for errors in the categorical forecast.
a. Derivation of critical decision thresholds
The critical fuel carriage threshold for a given flight can be derived using a basic contingency table, as shown in Fig. 1. First, the airline dispatch must decide whether a given plane should add additional fuel to its prescheduled fuel load. Next, the flight may or may not encounter alternate minimum conditions at its destination. This decision tree outlines these four possible outcomes and the costs incurred by each.
If one defines p as the probability that there will be below alternate minimum conditions, then (1 − p) is the probability of conditions being above the alternate minimum threshold. So for the “additional fuel” option, the expected cost, EC1 is defined as
Similarly, for the “no additional fuel” option, the expected cost (EC0) is
The logical decision then is to carry extra fuel if the expected cost from carrying the fuel (EC1) is less than that from not carrying the fuel (EC0), for example, EC1 < EC0. Therefore, to calculate the critical probability threshold of weather below the alternate minimum, we set EC1 = EC0:
By definition, the false alarm cost (FAC) is the cost of a false alarm over and above the cost of a correct rejection; that is, FAC = C10 − C00. The miss cost (MC) is similarly defined as the cost of a miss over and above the cost of a hit (i.e., the correct forecast of an event); that is, MC = C01 − C11. Note that both of the definitions specify the avoidable cost of the weather forecast, the cost relative to a correct forecast.
Solving for p yields
Further, if we define the ratio (FAC/MC) as the cost ratio (CR), then
As shown by the derivation above, the critical fuel carriage threshold is driven completely by the cost parameters of each flight. It is important to realize that most of the cost of a weather event is still incurred even if the event is correctly forecast. Usually, if a weather event occurs at an airport that has an impact on incoming flights, only a small portion of the impact is avoidable even when the forecast is correct. A perfect forecast is useless to an airline if no corrective action can be taken based on the forecast. Therefore, it is this avoidable component that must be studied when considering financial impact.
b. Cost data
For each flight, the following costs are relevant for calculating the critical fuel carriage threshold:
(a) the cost to carry the weight of extra fuel unnecessarily,
(b) the cost of an en route diversion, and
(c) the cost of a diversion from a destination after a missed approach.
These costs summarize the economic impacts of airport forecasts. Scenario a is the cost of a false alarm, whereby the forecast causes extra fuel to be carried unnecessarily. The cost is the extra fuel used to carry that extra weight, usually a fairly modest figure of the order of $100. Scenario b occurs if no extra fuel is carried, and during the flight the weather at the destination is such that a landing may not be possible, and a decision is made en route to divert. Scenario c happens if extra fuel is carried and the weather at the destination is bad. According to American Airlines, a pilot will almost always make an attempt to land given that he or she has adequate fuel to return to cruising altitude and proceed to another airport. This leads to the seemingly incongruous fact that, on a small number of occasions, the biggest cost actually occurs following a correct forecast. There are undoubtedly more subscenarios than these three, but in order to simplify the issue of operational forecast use for this study, we have settled on these three scenarios. Discussion with American Airlines staff suggested that these handle the great majority of situations.
c. Calculation of critical thresholds
To calculate a critical threshold for each flight, one must accurately calculate the cost of a “hit” above standard operating costs as well as the cost of a “miss” above standard operating costs. In addition, one must be able to account for the impact of forecast errors. The only viable way of making allowances for this is to follow the same methodology as Keith (2003). Keith calculated the climatological rate of low ceiling and reduced visibility events in which landing was impossible and expressed this as a ratio of the climatological rate that conditions were below the alternate minimum. This ratio is calculated for each month and different times of the day.
Not only did we have to consider the FAA-mandated alternate minimum ceiling and visibility conditions for fuel carriage, but we had also to consider the landing minimum, which represents the values of ceiling and visibility that would preclude an aircraft from landing. As opposed to the fixed values of ceiling and visibility that represent alternate minimum conditions, the landing minima vary according to a number of factors, including aircraft type, standard of airport landing instrumentation, and training level of the pilot. That said, we learned that American Airlines most commonly implements landing minima of ceiling ≤ 200 ft and visibility < ¾ mi (W. Qualley 2005, personal communication). Therefore, we decided it was reasonable to apply these landing minima throughout our study. While we were interested in utilizing the landing minima, an investigation of climatology at the arrival airports revealed that these conditions occur quite rarely, with an occurrence rate of <1%. Because of this, the ratio of “below landing minimum” events to “below alternate minimum” events would be too low to develop stable probabilistic forecast equations. Therefore, we had to focus solely on alternate minimum conditions in this study, other than for the purpose of calculating flight operating costs.
Some difficulty arises in calculating the cost of a “hit,” or correct forecast of weather conditions being below the alternate minimum. If the TAF correctly forecast conditions below the alternate minimum, advice from American Airlines is that the pilot will usually attempt a landing, but if conditions are such as to preclude landing, he or she will then climb out and proceed to another airport. If the climatological ratio of the number of occasions of below landing minimum conditions to the number of occasions of below alternate minimum conditions is r, then
Recall that for a hit, alternate fuel is being carried. Therefore, r is usually a small value, but is multiplied by a large dollar value (i.e., the diversion cost), whereas (1 − r) is typically close to a value of 1 but is multiplied by a smaller number (i.e., the cost of carrying additional fuel). In practice both terms of the equation are significant in the calculation of the hit cost.
The cost of a missed event is also complex. Consider the situation where an airline dispatcher plans minimum fuel due to a forecast of above alternate minimum conditions. Occasionally, after the flight departs, the weather at the destination airport deteriorates to below alternate minimum. This is almost always followed by an amended TAF, and on some occasions an amended observations-based probabilistic forecast. Sometimes the dispatcher, in consultation with the pilot, can agree upon an alternate airport that is nearby the actual destination while en route. This allows the flight to continue along its original flight path and then make a decision about whether to proceed when the flight is closer to the planned destination. However, this is not always possible and the flight diverts to another airport, in order to land and refuel. We were not able to access detailed information for each flight showing how often in-flight deterioration of weather caused en route diversions. Therefore, for the purpose of the study, we considered three scenarios: that en route weather deteriorations caused diversions 20%, 60%, and 100% of the times that extra fuel was not carried.
3. The deterministic aviation weather forecast
TAFs, created by the National Weather Service (NWS), are airport-specific forecasts of visibility and ceiling at a microscale level and have lead times of up to 24 h (though only the first 12 h are typically used operationally). Numerical models are unable to adequately define the fine scales required to analyze and then predict the detailed evolution of these parameters, which is required by the aviation industry. So, forecasters are required to produce these forecasts by primarily reverting to climatology and their own experiences. Thus, it can be said with a fair degree of validity that TAFs are still at a stage that synoptic-scale forecasting was before the advancement of numerical weather prediction. Various verification studies, for example, Dallavalle and Dagostaro (1995), Keith (2003), and Harvey et al. (1992) have shown that, in the first 3–6 h of the TAF, raw persistence usually demonstrates greater skill.
Once these TAFs are created, FAA guidelines dictate to the airlines whether additional fuel must be carried. If the forecasted ceiling or visibility level at the destination airport is less than a specified limit (e.g., the alternate minimum), then the aircraft is required to carry extra fuel. This FAA rule is not obviously related to costs, and is certainly not sensitive to differences in costs between different flights. Thus, the airline experiences significantly excessive fuel costs simply by adhering to FAA requirements.
For the purpose of the case study described below, an archive of TAFs, spanning from April 2002 to May 2003, was obtained in order to examine how these forecasts would compare to the overall costs resulting from the cost-based decision rule system. These TAFs were produced by the National Weather Service office that is responsible for each of the three airports listed in the case study discussion. Of interest to this study is whether the TAF predicts adverse weather such that flight dispatchers must specify that a given flight should add extra fuel.
4. The probabilistic aviation weather forecast
There have been some attempts to investigate alternatives to traditional numerical model prediction of those parameters that are crucial for landing aircraft. In particular, observations-based statistical techniques have shown great promise for improving short lead time forecasts of these surface weather conditions. Vislocky and Fritsch (1997) demonstrated that such a statistical forecasting system has superior skill compared to numerical models for forecasting ceiling and visibility at these short lead times. This system considered a network of surface observations surrounding an observing site (often an airport) to produce (via regression techniques) probabilistic forecasts of ceiling and visibility for that observing site. The results indicated that such a system has greater skill than model output statistics (MOS) derived forecasts, as well as persistence climatology, out to a lead time of 6 h. Leyton and Fritsch (2003, 2004) extended this work by demonstrating a further increase in skill when utilizing higher-density and higher-frequency surface observations for these short lead time forecasts.
While observations-based statistical systems have shown greater forecast skill over traditional techniques, little has been done to investigate whether the economic value of the forecast has been improved as well. Keith (2003) demonstrated how human forecasters’ subjective airport forecasts, when produced in a probabilistic format, should increase the value of that forecast to an airline compared to the traditional categorical airport forecast. This study looks to further examine the impact of probabilistic forecasts on forecast value by utilizing actual financial data.
Two datasets were utilized to develop the statistical forecasts equations. The first contained hourly surface weather observations [i.e., Surface Airways Observations (SAOs)] for the period of January 1982–December 1996. The second dataset contained hourly observations [in the METAR (routine aviation weather report) format] spanning from January 1997 to July 2003. Each dataset included hourly observations of temperature, dewpoint, wind speed and direction, cloud cover (from which ceiling can be derived), visibility, and present weather at over 1500 automated weather-observing sites around the United States. To calculate climatological values of various weather parameters, the datasets were combined to produce values that represented weather conditions over 22 yr. In addition, the latter dataset is utilized for statistical forecast development. Leyton and Fritsch (2003, 2004) provide a detailed explanation of the data processing procedures.
b. Development of probabilistic forecasts
Statistical forecast equations were developed in the same manner as described by Leyton and Fritsch (2003, 2004). To replicate the decision-making process involved in fuel carriage for a flight, careful consideration was given to the initialization and valid times for the probabilistic forecasts. This is mainly because decisions regarding the loading of jet fuel typically are made during the dispatcher’s flight planning, which usually occurs 1–2 h prior to the flight’s departure. Another complicating factor is due to the nature of observations-based probabilistic forecasts. As described by Leyton and Fritsch (2003, 2004), these forecasts can only be initialized and verified at the top of the hour. However, scheduled flights arrive and depart throughout a given hour. Therefore, to create the most accurate representation of the actual decision-making process, all of these factors were carefully considered.
It is important to note that in this study, adverse weather conditions were limited to low ceiling and/or reduced visibility. Thus, no specific attempt was made to statistically forecast the occurrence of thunderstorms at a given location. Operationally, the forecast of a thunderstorm is considered as being below alternate minimum. Of the total number of occasions of below alternate minimum conditions, less than 10% were as a result of thunderstorms, and of this number over half were captured by the 2000-ft and/or 3-mi criteria. While thunderstorms are the major consideration for in-flight weather, low ceiling and visibility at terminal airports are by far the more important issue for initial fuel allocation. The omission of thunderstorm forecasts is therefore not expected to significantly affect the validity of the result of the comparison.
5. A case study of cost-based decision rule system
A case study was performed in which the cost-related impact of low-ceiling and/or visibility forecasts was examined for 18 daily flights arriving at three American Airlines “hub” airports: Dallas–Fort Worth International Airport (DFW), John F. Kennedy International Airport in New York City (JFK), and Lambert–St. Louis International Airport (STL). A database of operating costs for several American Airlines flights arriving into each airport was obtained; this dataset contains the cost per flight information for each of the 18 daily flights, in 2003 dollars. It is important to note that the flights used in this study were predetermined in consultation with the airline and represent varying distances and departure–arrival times during the day. Thus, this study is not focused simply on a specific type of flight but rather a broad representation of an airline’s typical daily flights. These operating costs represent typical costs of various scenarios for each flight during the April 2002–May 2003 period; approximately 7500 flights in all.
For the purpose of calculating the hit cost, values of r were calculated for DFW, JFK, and STL for each month during the 14-month period, in 3-h segments, and then applied to each flight as appropriate. Thus, the expected hit cost was calculated and subtracted from the cost of a missed event to arrive at the miss cost. Using the cost-based decision rule system derived above, the critical fuel carriage threshold is driven completely by the cost parameters of each flight. Critical thresholds have been identified as high as 0.27 for some long international flights, and as low as 0.02 for shorter domestic flights, as shown in Table 1.
Once the static critical fuel carriage threshold was calculated for each daily flight, forecasts of below alternate minimum conditions could then be utilized to assess whether a given flight should carry additional fuel. Two sets of independent forecasts were used in this trial study: statistical probabilistic forecasts of low ceiling and/or reduced visibility (ranging from 0% to 100%, inclusive) and TAFs from the airport’s local NWS office (either 0% or 100%). The purpose of utilizing two forecasts was to not only identify whether this technique worked but also whether one forecast type delivers superior value compared to the other. NWS TAFs are the current source of aviation weather forecasts. However, as noted previously, recent studies have demonstrated the superior skill of probabilistic forecasts, especially when considering short lead times.
Table 2 shows the hit rate and false alarm rate for each flight, with 95% confidence intervals. The 95% confidence intervals have been calculated using the formula quoted by Stephenson (2000). It can be seen that the probabilistic forecasts are driven by the cost calculations to a much more conservative tactic (higher hit rate and false alarm rate) than the TAFs. Perusal of the data showed that the TAFs sometimes missed expensive events, while the probabilistic forecasts had more false alarms than the TAFs. The tight confidence intervals give one some faith in the robustness of the value calculations. The dollar values are calculated directly (i.e., no inference is involved) from the probabilistic forecasts and the TAFs. So the significant differences in value are likely to be caused by the different tactic or decision threshold.
As mentioned above, we were not able to access information that showed how often in-flight deterioration of weather caused en route diversions. Thus, the analysis has been performed using three different assumptions: that en route weather deteriorations caused diversions 20%, 60%, and 100% of the times that extra fuel was not carried. Table 3 shows a comparison of total costs per flight using the traditional TAF and the observations-based probabilistic forecasts, with consideration given to the different assumptions regarding weather deterioration. These costs, in 2003 dollars, represent the total costs from April 2002 to May 2003. It is important to note that for each of the 18 flights, the use of observations-based probabilistic forecasts resulted in lower total costs than the traditional TAFs. Extrapolating these results for 18 flights across the whole American Airlines domestic fleet (approximately 2200 flights per day), the airline would have saved between $15 million and $50 million annually if the initial flight planning had been done using the observations-based probabilistic airport forecasts rather than the traditional TAF. By comparison, in 2003, the annual fuel expense of a major U.S. commercial airline was $1 billion or more. Thus, by utilizing this simple decision rule system along with the probabilistic forecasts, total annual fuel expenses could be reduced by as much as 2.5%. (Note: in 2005, the total annual fuel expense of American Airlines was near $4 billion, while the total annual operating expense was over $18 billion.)
6. Summary and discussion
The sole purpose of this study was to examine whether significant savings in fuel expenses could be realized by switching from the current decision-making process to one that utilizes cost-based decision rules and probabilistic weather forecasts of adverse weather conditions. The current system utilizes TAFs in conjunction with FAA policy that allows airline dispatchers to make fuel carriage decisions arbitrary of the costs involved. We explore the development of a cost-based decision system that identifies a critical probability of adverse weather conditions for each flight. Finally, we examine two different types of probabilistic forecasts of low ceiling and/or visibility to apply to the cost-based decision rule: converting the TAF deterministic forecasts into probabilistic forecasts (100% to represent “yes” and 0% to represent “no”) and creating statistically derived probabilistic forecasts (0%–100%, inclusive). We feel that a cost-based decision system is preferable in order to minimize fuel costs and we find that utilizing statistically derived probabilistic forecasts to determine fuel carriage results in a significant cost savings compared to the deterministic TAF forecasts. The savings found in this study suggest an annual fuel cost reduction of $15–$50 million for a major commercial airline.
We do not claim to have precisely represented all the operational intricacies of forecast use by airlines. To quantify this more exactly it would be necessary to work directly with an airline, to more adequately specify the details of the financial impact of weather and weather forecasts on airline operations. This impact will undoubtedly be complex some of the time, and will require extension of our current algorithm. However, the gross differences in the dollar results from use of the two different types of forecasts certainly justify further investigation of the use of observations-based probabilistic airport forecasts as a more valuable product than categorical TAFs.
We must express our gratitude to Warren Qualley for sharing his expertise in commercial aviation operations and weather forecasting. Kelvin Droegemeier is to be thanked for his continued support and insight during this effort. We also thank American Airlines for providing the operating cost data for the flights as well as NCAR for the weather data utilized this study.
Corresponding author address: Ross Keith, 14 Lawson St., Townsville 4812, QLD, Australia. Email: email@example.com