1. Introduction
Current numerical weather analysis (NWA) systems are able to provide hourly meteorological data on horizontal scales as small as 10 km. In principle, these high-resolution NWAs, including the 20-km Rapid Update Cycle (RUC; see Benjamin et al. 2004a,b) and the University of Oklahoma Center for Analysis and Prediction of Storms (CAPS) Advanced Regional Prediction System (ARPS; see Xue et al. 2003), can provide the meteorological information necessary to diagnose contrail formation. Unfortunately, the straightforward prediction of contrail-induced cloud cover from these analyses is hindered by systematic and random measurement errors. Duda and Minnis (2009, hereinafter Part I) show that logistic regression modeling can provide a method to deal with these errors and to diagnose contrail occurrence accurately based on NWA-derived atmospheric variables including temperature, relative humidity, and vertical velocity.
Some probabilistic forecast models of contrail occurrence based on logistic regression have already been developed. Travis et al. (1997) used a combination of rawinsonde temperature and geostationary satellite water vapor absorption data to develop a logistic model of the occurrence of widespread persistent contrail coverage. Jackson et al. (2001) created a contrail prediction model using surface observations of contrails and rawinsonde measurements of temperature, humidity, and winds. In this study, we use contrail observations from both the Global Learning and Observations to Benefit the Environment (GLOBE) program (available online at http://www.globe.gov) and geosynchronous satellite imagery along with numerical weather analyses and forecasts to create forecast models for the prediction of persistent contrail formation. These models allow predictions of widespread contrail occurrences on either a real-time basis or for long-term time scales. More developed versions of the forecast models could eventually be used in aviation for the prevention of persistent contrail production, whereas long-term studies could focus on estimating the radiative impact of contrails on regional or global climate.
Despite the success of the probabilistic forecast models described in Travis et al. (1997) and Jackson et al. (2001), several questions remain about the usefulness of these models. The former study used only a limited number of observations, whereas the latter only considered contrail observations within limited geographic (New England states) and temporal (two weeks in September) domains. Neither study attempted to use numerical weather forecast data to predict contrail occurrence. The use of prognostic meteorological data within the logistic models would allow for longer forecast lead times than logistic models developed from observations only. Such longer lead times would be helpful if contrail mitigation efforts are considered.
In this paper, we assess the ability of logistic models to provide a valuable and accurate diagnosis/prediction of persistent contrail occurrence via numerical weather models. Specifically, we evaluate a sample of logistic contrail forecasts based on RUC and ARPS data and observations of contrail occurrence. The value of the contrail prediction models is then discussed in the context of a forecast evaluation theory.
The next section describes the meteorological data and contrail occurrence observations used to develop the statistical contrail occurrence models, and section 3 presents and evaluates some examples of logistic models. The final two sections briefly summarize and discuss the overall value of the logistic forecasts.
2. Data and methodology
a. Meteorological data
To provide atmospheric predictors for the logistic models, we use nearly 15 months (April 2004–27 June 2005) of meteorological data from two high-resolution, operational numerical weather analyses. Profiles of temperature, humidity, horizontal wind speed and direction, and vertical velocity were derived by using hourly analyses from the 20-km resolution RUC model and from the 27-km resolution ARPS analyses in 25-hPa intervals from 400 to 150 hPa. [After 1200 UTC 28 June 2005, the 13-km resolution version of the RUC model became operational, with significant differences in upper-tropospheric humidity (UTH).] Because of limitations in computational resources, both the RUC and ARPS data were stored at approximately 1° × 1° horizontal resolution. In addition to the RUC and ARPS analyses, ARPS 1-day, 2-day, and 3-day forecasts were also used to build logistic models.
The meteorological data were downloaded each day to a local computer. The data are subject to interruptions including computer and power failures, full disks, operator errors, lack of data availability, and other problems. Thus, approximately 77% of the hourly ARPS and 99.7% of the RUC data were collected during the time period. Two large gaps (between 20 August and 28 September 2004 and between 21 January and 21 February 2005) accounted for nearly 85% of the ARPS data loss. The ARPS forecasts had a slightly larger loss rate than the ARPS analyses because sometimes the forecasts were not available even though the analyses were available.
Both the RUC and the ARPS have been built for the prediction of storms and precipitation, and the accurate prediction of UTH is of secondary importance. Both models contain, at most, only slight ice supersaturations, which appear incidentally as the result of numerical issues. The RUC analyses do not allow relative humidity with respect to ice (RHI) to exceed 100% by more than a few percent at pressures below 300 hPa, whereas the RHI values in the ARPS analyses rarely exceed 112%. No ice supersaturation occurs in the ARPS forecasts; the maximum RHI is only 100%. The ARPS forecasts included in this study use a bulk three-phase ice microphysics scheme (Lin et al. 1983; Tao et al. 1989) and do not have a separate cirrus or contrail parameterization. Thus, methods like logistic regression are necessary to deal with these limitations to predict contrail occurrence using RUC or ARPS data.
b. Satellite data
Visual inspection of multispectral satellite data was used to detect persistent contrail occurrence for some of the logistic models. We inspected infrared (10.8 μm) and water vapor (6.5 μm) channel data from the Geostationary Operational Environmental Satellite-12 (GOES-12) and infrared (10.8 μm) minus split window (12.0 μm) brightness temperature difference (BTD) data from National Oceanic and Atmospheric Administration (NOAA) Advanced Very High Resolution Radiometer (AVHRR) imagers. The BTD data are especially sensitive to the presence of contrails (Lee 1989).
c. Surface data
Persistent contrail occurrence was also determined from a set of surface observations. The GLOBE program collects observations of contrail occurrence from primary and secondary schools across the conterminous United States (CONUS). In May 2003, GLOBE initiated a contrail observation protocol to gather and classify contrail observations. A primary goal of the GLOBE program is to use detailed written protocols to enable students to provide scientifically valuable measurements of environmental parameters (Brooks and Mims 2001). Over 18 500 observations of cloud coverage and contrail occurrence were reported over the CONUS between April 2004 and June 2005. Observations of spreading, persistent contrail observations are classified as observations of contrails that remain in the sky after the aircraft has flown out of view of the observer that are wider than the width of a finger held at arm’s length. This width corresponds to a contrail at least 350 m wide, based on a contrail altitude of 10 km (O’Shea 1991), which is the minimum width expected to be detectable in AVHRR imagery.
The GLOBE contrail dataset contains observations from 417 schools. The schools are mostly located in highly populated regions with substantial air traffic at cruise altitudes above 7.6 km (Duda et al. 2009). Unlike the Jackson et al. (2001) study, no flight track information was available to determine the altitude of the observed persistent contrails. Nearly all schools reported only one observation per day, but only 123 of the schools reported more than 30 observations during the 15-month period. Approximately 92% of all observations were between 1430 and 2030 UTC and nearly 58% of the total were between 1630 and 1830 UTC. To improve the ability to detect contrails, this study only used observations from a selected group of 11 schools that were taken under mostly clear skies (noncontrail cloud coverage less than 25%).
d. Data processing
Before deriving the logistic models, the meteorological data were checked for missing data and matched in time and location with the surface and satellite observations of contrails. No contrail observations with missing meteorological data were used in the statistical forecast models.
To match the RUC data with the contrail occurrence observations, meteorological variables from the RUC analyses closest in time with the contrail observations are linearly interpolated to the location of each contrail observation. An observation is not used if the time difference between the observation and the RUC analysis was greater than 2 h (nearly all pairs were matched to within 1 h). A similar procedure is used to match the ARPS analysis data with the contrail observations. For the ARPS forecast data, the meteorological data from the forecast time matching to within 1 h of the observation were used. Because the ARPS forecasts begin at 0000 UTC and all of the contrail observations from the 11 schools used in this study occurred between 16 and 20 UTC, the 1-day forecasts refer to the 16–20-h forecast model time, the 2-day forecasts refer to the 40–44-h forecast model time, and the 3-day forecasts refer to the 64–68-h forecast time.
For convenience, atmospheric humidity in both meteorological datasets was usually expressed in the form of the maximum RHI between 150 and 400 hPa. For the ARPS data, the RHI was computed from the ARPS fields of potential temperature and specific humidity at the 25-hPa intervals to determine the level of maximum upper-tropospheric humidity. Because it is expected that persistent contrails are most likely to form where relative humidity is greatest, for each contrail observation, the pressure level between 400 and 150 hPa with the maximum RHI that had a temperature less than or equal to −40°C was identified. Although the observed contrails may have formed at other levels, this level was chosen to represent the most probable level for contrail formation in the absence of contrail altitude information and to provide a consistent representation of humidity at typical commercial aircraft flight levels. The temperature constraint was added to eliminate areas where the atmosphere is likely to be too warm to form contrails (Appleman 1953).
e. Statistical technique
The maximum likelihood method was used to estimate the unknown coefficients of βi and to fit the logistic regression model to the data. The chi-square statistic χ2 was employed to assess the goodness of fit of each logistic model to the meteorological data. A stepwise regression technique was used to reduce the number of predictors to an optimal number of variables. In each step of the technique, a new predictor is added to the logistic model and the chi-square statistic is compared with the previous model. The new predictor that produces the largest improvement in model fit (i.e., the largest increase in χ2) is added to the model. To avoid overfitting the model, the stepwise regression technique is allowed to add predictors to the model until the test for statistical significance reaches a significance level (i.e., p value) of approximately 0.05.
f. Predictors
Table 1 includes all potential predictors considered for this study. A total of 61 potential predictors from the numerical weather models were used to develop the logistic regression models. All of the variables are expected to influence the formation (or the spreading rate) of persistent contrails. From this set of potential predictors, the stepwise regression method was used to reduce the number of predictors to approximately six. Several of the variables, including the temperature, vertical velocity, wind speed, and wind direction, were computed at the level of maximum RHI, and the vertical shear of the horizontal wind and temperature lapse rate (a measure of atmospheric stability) are computed for the 25-hPa layer below the level of maximum RHI. The logistic models developed from the ARPS analyses and forecasts do not include precipitable water, tropopause temperature, tropopause pressure, or any other variable formed by the combination of those parameters because they are not included in the ARPS analyses. Each of these variables is indicated by asterisks in Table 1.
Several other meteorological variables were also considered as possible predictors in the logistic models. In addition to determining the variables at the level of maximum RHI, the 200–300-hPa layer averages of several variables were computed, as well as regional mean variables, which are the mean (of the 200–300-hPa means) of all model grid points within 200 km of the contrail observation location. Most commercial air traffic over the CONUS cruises between 200 and 300 hPa (Garber et al. 2005). The regional mean was developed to account for some of the uncertainty in the meteorological fields forecast by the ARPS model. Finally, “upstream” means of temperature and RHI were also computed. The upstream mean is defined as the 200–300-hPa layer mean average of the variable located 2 h upstream from the contrail observation location. The upstream point is determined by computing a 2-h backward trajectory using the 200–300-hPa mean wind from the original observation point. The upstream variables were included because most persistent contrails seen within GOES infrared imagery require 1–2 h before they become wide and thick enough to be visible in the satellite imagery (Duda et al. 2004).
g. Equation development
Two groups of logistic models were created by using the meteorological data and observations of contrail occurrence. The first group of models is based on surface-based contrail observations supplemented with satellite observations of contrail occurrence. These models were designed to relate the general occurrence of persistent contrails with the meteorological conditions, and are hereinafter referred to as the SURFACE models. The second group of models is similar to the work of Travis et al. (1997) where a selected subgroup of observations within the presence of widespread persistent contrails [called both here and in Travis et al. (1997) as “outbreaks”] is used to build the logistic models. They are called the OUTBREAK models in this study.
The contrail observations were separated into a dependent (from which the statistical models were created) and an independent (on which the models were tested) dataset. Two-thirds of the data were randomly selected to build the dependent dataset, whereas the independent dataset comprises the remaining one-third of the data.
To determine the accuracy of the contrail models, two statistical measures used in Part I were employed. The contrail formation forecasts are separated into four categories based on the forecast and its outcome: a is the number of hits, b is the number of false alarms, c is the number of misses, and d is the number of correct rejections. The first measure is the percent correct (PC), and equals (a + d)/(a + b + c + d). PC is defined as the ratio of the correct forecasts to the total number of forecasts. The second variable is known as the Hanssen–Kuipers discriminant (HKD) or the true skill statistic (Wilks 1995). The HKD is calculated as (ad − bc)/[(a + c)(b + d)]. This measure of forecasting skill measures the skill of the “yes forecasts” and “no forecasts” of contrail occurrence equally, regardless of the relative numbers of each forecast. Gandin and Murphy (1992) show that HKD is the only equitable skill score for a two-event (i.e., yes or no) forecast and thus accounts for the tendency of a no-contrail-occurrence forecast being more likely to be correct than a yes, because persistent contrail occurrence is relatively rare.
3. Logistic models based on numerical weather analyses
a. SURFACE models
A subset of 11 GLOBE reporting locations with at least 50 contrail observations under mostly clear skies (noncontrail cloud coverage less than 25%) were chosen for building the SURFACE logistic regression models. Because these schools provided multiple observations throughout the 15-month period, we expect that these locations would be more likely to provide high-quality contrail observations among the GLOBE participants. Table 2 lists the location of each GLOBE school, and the locations of the GLOBE schools are also shown in Fig. 1. All schools, with the exception of Box Elder, Montana, are located in regions with substantial commercial air traffic at cruise altitudes above 7.6 km during the observation period (Duda et al. 2009). (We note that other areas of substantial air traffic in the United States, including the coastal Pacific Northwest and the Great Plains, are not sampled within this study.) From this group of locations, a set of 379 observations was selected that could be “verified” by visual inspecting time series of GOES imagery for contrail occurrence–nonoccurrence. This verification is somewhat subjective as the surface observer under mostly clear skies can detect much narrower and thinner persistent contrails than automated or visual analysis of the 4-km resolution satellite imagery. Of the 379 observations, the surface and satellite results matched nearly 75% of the time. In about 18% of the observations, the surface observer reported persistent contrails, whereas none was apparent in the satellite imagery; for the remaining 7% of the observations, contrail occurrence was detected by satellite but not reported by the GLOBE observer. DeGrand et al. (2000) and Duda et al. (2009) have previously noted differences in the detection of contrails between collocated surface- and satellite-based observations. Surface observers often miss contrails forming above lower cloudiness (although, by choosing only mostly clear observations, this type of error should be minimal here), misidentify linear cloud features as contrails (or contrails as cloud streaks), or record the observation incorrectly in the contrail report. Cloud cover and the misidentification of cloud streets as contrails also hamper visual detection of persistent contrails in the GOES imagery loops.
Two sets of probabilistic models were developed from the GLOBE surface observations and the numerical weather analysis data. The first set (denoted as Build 1) used the GLOBE contrail occurrence observations, whereas the second set (denoted as Build 2) used the GOES observations of contrail occurrence. As discussed above, the observations were randomly separated into dependent and independent datasets to create and to test the model, respectively. For convenience, the stepwise regression was stopped after 6 predictors were chosen, because additional predictors rarely improved the skill scores of the logistic models significantly. Because of the large number of potential predictors (some closely related to each other), many combinations of predictors produced chi-square statistics nearly equal to the best-fitting model. Therefore, the logistic models were evaluated by averaging the skill scores from the five models with the highest chi-square statistic, thus producing a mean skill score from each group of meteorological data.
Table 3 presents the mean PC and HKD skill scores for both builds of the logistic model for each group of meteorological data. For simplicity, the critical threshold for determining contrail occurrence was 0.5 for all cases. The logistic models from the first two rows of Table 3 are built from analysis data and therefore diagnose contrail occurrence from the analysis data. The remaining models are true forecasts evaluated by using forecast data but are developed from either analysis or forecast data. In every case except one (the HKD score for the ARPS 3-day forecast), the skill scores of Build 2 were higher than the skill scores of Build 1. Also, the differences between the Build 2 and Build 1 scores were largest when analysis data were used and smallest when 3-day forecast data were used. The accuracy of the models generally decreased as the length of the forecast increased. When the independent datasets (the remaining third of the observations not used in the development of the models) were used to evaluate the skill of the forecast models, the PC ranged from 0.69 to 0.86 for the Build 2 models and the HKD varied from about 0.18 to 0.59. These scores are worse than the results from Jackson et al. (2001). They developed a regional contrail formation model (including nonpersistent contrails) based on a network of surface observers across New England coordinated with air traffic control information such that the observers knew exactly when to expect flights. The observations were collected during a two-week period in September. Jackson et al. (2001) used nonsynoptic radiosonde launches to gather humidity information and report a PC around 0.85 and an HKD near 0.66.
The most common predictors for the Build 1 SURFACE models tend to be related to temperature when the models are derived from RUC/ARPS analysis data. No specific kind of variable is favored when the Build 1 models are derived from ARPS forecast data. The most common predictors for the Build 2 SURFACE models tend to be related to temperature, relative humidity, and wind direction when the models are generated using RUC or ARPS analyses and to vertical velocity and the product of temperature and relative humidity with respect to ice when the models are developed from ARPS forecasts.
b. OUTBREAK models
The logistic models created using GOES observations of contrail outbreaks are similar to the model created by Travis et al. (1997), who derived the meteorological data from a select set of atmospheric conditions. The water vapor channel data from noncontrail locations were taken from either completely clear or completely cloudy pixels close to the contrail outbreak regions, whereas contrail observations were taken at locations where contrails were wide enough to fill the entire satellite pixel. Cloudy pixels may contain contrails also, but in those cases they would have been obscured by the clouds. In this study, the contrail observation locations were chosen from only two sets of locations: either from a point in the clear skies near the contrail outbreak or from a point in the center of the contrail outbreak (thus, two observations were used from each outbreak). This method allows for a sharp distinction in the meteorological data between the contrail and noncontrail areas. As a result, logistic models of high accuracy can be produced.
Visual inspection of AVHRR images displaying the BTDs between the 10.8- and 12.0-μm channels and the BTDs of loops of GOES infrared and water vapor imagery were used to identify approximately 50 examples of contrail outbreaks—areas of distinct, line-shaped contrails covering at least 100 000 km2 at various locations around the CONUS between August 2004 and June 2005 (see Fig. 2 for an example of a typical contrail outbreak). Because the horizontal resolution of the GOES infrared imagery is 4 km, the contrail outbreaks are likely to be composed of extremely wide, well-developed spreading persistent contrails (or perhaps composed of several narrower contrails in close proximity to each other). Figure 1 shows the central locations of the contrail outbreaks.
A total of 104 satellite measurements in and around large contrail outbreaks was used to make the independent and dependent datasets. The stepwise regression was stopped after four predictors were chosen, and the skill scores from the five models with the highest chi-square statistic were averaged to produce a mean skill score for each group of meteorological data. The mean PC and HKD skill scores for the top five models produced from each group of meteorological data are presented in Table 4 (once again, the critical threshold for determining contrail occurrence was 0.5 for all cases). The skill scores in Table 4 are similar to the results from Travis et al. (1997). They reported PCs near 0.90 and an HKD around 0.85.
The skill scores of the logistic models developed from the OUTBREAK data generally decreased as the length of the forecast increased. The differences between the skill scores of the logistic models created from the ARPS analyses and those from the models developed from the ARPS forecasts are larger in Table 4 than in Table 3. The OUTBREAK models are developed from a smaller set of observations than the SURFACE models; because the OUTBREAK models are tuned to sharply defined meteorological conditions, they are more sensitive to the errors in the meteorological fields that are present in forecast models. Thus, this group of logistic models highlights the effects of forecast errors in the logistic model results. The most common predictors for the OUTBREAK models tend to be wind direction, atmospheric lapse rate (dT/dz), temperature, RHI, and the product of temperature and RHI.
c. Effect of random error on logistic models
Part I investigated the effects of random measurement errors of relative humidity, temperature, and vertical velocity on logistic models developed from a set of simulated meteorological measurements. A simple set of physical assumptions based on those three meteorological variables was used to determine the contrail occurrence associated with the simulated measurements. Actual contrail observations are not likely to be so strongly coupled to such measurements, because other meteorological variables probably affect the occurrence of spreading persistent contrails and errors in the determination of contrail occurrence within the observations themselves would also affect the link between these variables and contrail occurrence.
To investigate the effects of random measurement error on logistic models developed using actual contrail occurrence observations, three scenarios were created that introduce different levels of random error into the OUTBREAK models developed from the RUC and ARPS analyses. The errors are the same as those introduced in the Part I scenarios B2k, B2l, and B2m. Case 1 uses normally distributed errors in temperature, RHI, and vertical velocity with standard deviations of 1 K, 5%, and 1 cm s−1, respectively, whereas the standard deviations of the corresponding errors in cases 2 and 3 are 2 K, 10%, and 2 cm s−1 and 3 K, 15%, and 3 cm s−1, respectively. The errors were added to the meteorological data used in both the dependent and independent data, and the logistic models were created in the same manner as above. Because of the small sample sizes used in the OUTBREAK models, cases 1, 2 and 3 were run 1000 times using 1000 different realizations of random error to determine the variability of the effect of random error on the models. The results of the simulations are presented in Table 5, where the skill scores for all models were evaluated by using the independent data only.
In addition to the results from the modified OUTBREAK models in Table 5, the synthetic data from Part I were used to simulate the OUTBREAK model results. We selected 105 of the observations from the first synthetic dataset, choosing contrail formation conditions for approximately half of the observations (similar to the observations used in the OUTBREAK models). Two-thirds of the observations were randomly selected to build the models (using scenario B2) and the remaining third was used to evaluate the models and determine skill scores. The skill scores from the top five best-fitted four-predictor models were averaged together and are presented in Table 5. Once again, 1000 realizations of random errors in the three meteorological variables were then added as in cases 1, 2 and 3, and the resulting skill scores are shown in Table 5 as simulation N. We note that the sensitivity of the real observations to the addition of random error is less than that of the synthetic observations. Part of this difference may be due to the real contrail occurrence observations being subject to variables other than temperature, humidity, and vertical velocity, but the logistic models developed from the real observations have also been developed with meteorological data that already have some nonzero but unknown inherent random errors. Another simulation using the synthetic data is presented in Table 5 (simulation E). Simulation E is identical to simulation N except for one difference. The meteorological variables in the original 105 observations have been modified by adding the type of random errors in case 3, which is similar to what may be expected in the real observations. The simulation E results show a similar insensitivity to the addition of random error evident in the results derived from the real contrail occurrence observations.
4. Discussion
A comparison of results among all of the models presented here gives some insight into the overall quality of the logistic models, under which conditions they perform well, and where further improvement is necessary. Table 3 shows that the Build 2 SURFACE models are consistently better than the Build 1 models. Although the difference in model performance could be easily explained by postulating the superior quality of satellite-based contrail observations compared to the observations of primary and secondary school students with little training, it is important to note some differences between surface- and satellite-based observations. Surface observers often miss contrails forming above lower cloudiness (although, by choosing only mostly clear observations, this type of error should be minimal here) or record the observation incorrectly in another category (some GLOBE observations are suspected to suffer from such a clerical error). Both the manual and the automated methods for detecting persistent contrails in satellite imagery are also hampered by cloud cover and by the misidentification of cloud streets as contrails, or contrails as cloud streets (Mannstein et al. 1999). Surface observers, however, can detect much narrower and probably optically thinner contrails than those seen in the 4-km resolution satellite imagery of this study. If the students are detecting relatively thin but persistent contrails within thin layers of supersaturation in the upper troposphere, then a weaker correlation between the numerical weather model variables and the occurrence of persistent contrails would be expected.
As noted earlier, the differences between the Build 2 and Build 1 skill scores tended to decrease as the length of the forecast increased from analysis time to the 3-day forecast. It is likely that, in both sets of models, the increasing errors in the forecasted variables over time tended to obscure their differences between the two builds.
The accuracy of the SURFACE and OUTBREAK models is less than the accuracy of the test case models in Part I created from synthetic observations. Part of the reason for the lower accuracy is that factors other than temperature, relative humidity, and vertical velocity affect the development of spreading persistent contrails. The results from Part I show that the addition of vertical velocity to the determination of contrail formation resulted in slightly less accurate models, even when all factors were known and accounted for in the logistic model. The most common predictors chosen in the SURFACE and OUTBREAK models tended to be related to temperature and humidity, but other variables including vertical velocity, wind direction and speed, and atmospheric lapse rate were frequently chosen as predictors. Previous studies of contrail occurrence suggest that high contrail incidence is associated with areas of baroclinicity and thus with areas where wind speed, vertical velocity, and lapse rate may have significant departures from mean conditions (DeGrand et al. 2000). The results from Carleton et al. (2008) suggest that atmospheric variables lower in the atmosphere that were not included in this study may also be valuable predictors. The list of meteorological variables in Table 1 is not exhaustive, and other combinations of variables not presented here may be better predictors of contrail occurrence.
The differences between the accuracy of the dependent and independent results in Tables 3 and 4 indicate whether an adequate number of observations have been used to build the logistic models. The dependent results are often better than the independent results, which suggests that the logistic models sometimes are too finely tuned to the dependent sets and that more observations are needed to build the logistic models. The results from the OUTBREAK models, which were developed using only a subsample of possible atmospheric conditions, highlight this problem. The results for the OUTBREAK models show differences between the dependent and independent results that are 3–4 times larger than those for the SURFACE models. The OUTBREAK models are designed to have high accuracy when using the dependent data. However, the models do much worse when evaluated with the independent data, which are not used in the construction of the logistic models. The large differences between the dependent and independent data skill scores are especially apparent in the OUTBREAK models developed from ARPS forecast data.
One advantage of developing logistic models for forecasts using the analysis data is that they have the most accurate meteorological data and allow for more accurate short-term forecasts than models built with forecast data. Tables 3 and 4 present the skill scores of several prognostic models developed from both ARPS analysis data and ARPS forecast data. The skill scores of the forecasts derived from models developed from the analyses are always higher than the forecasts developed from the forecast data (when assessed using the independent data). The meteorological data in the ARPS forecasts have so much error that the logistic models respond to that error and tend to choose predictors that fortuitously correlate with contrail occurrence within that particular dependent dataset.
The accuracy of the models tended to decrease as the length of the forecast increased. The atmospheric conditions for the formation of persistent contrails in the absence of natural cirrus tend to occur at the edges of areas of high humidity and lower temperatures in the upper troposphere, and the exact location and timing of these regions are not always represented well in numerical weather models. Most of the variables chosen as possible predictors in the logistic models are based on meteorological quantities at the point of interest. Even if the general synoptic features of the forecast are accurate, relatively small errors in the motion or size of high humidity areas could reduce the accuracy of the contrail prediction models substantially. Model errors may be mitigated if more regionally or temporally averaged variables were used in the creation of the logistic models.
This study attempted to build a universal contrail model suitable for all times across the CONUS, whereas both seasonal and regional differences in contrail occurrence are common (DeGrand et al. 2000; Carleton et al. 2008). It appears that a universal model for the entire CONUS may not allow for the highest accuracy; as in probabilistic precipitation forecasts, local forecasts for a specific location or region or for a specific season may allow for more accurate models. The superior results from the spatially and temporally limited study of Jackson et al. (2001) support this idea to some extent, but they rely on having enhanced direct observations of the meteorological fields, not degraded NWA fields.
An important quality of the contrail forecast model is its overall value. The value of a forecast is defined here following Murphy and Ehrendorfer (1987), such that forecasts are of positive value only if they can lead to different actions than those that the decision maker would have taken in the absence of the forecasts. If persistent contrail forecasts are to be of any value, for example, in the case of diverting flights to reduce persistent contrail cloud cover, then the cost C of diverting the flights must be weighed against the losses L that may result as a consequence of the additional cloud cover produced by the contrails. In the absence of a contrail forecast, the decision maker would have to compare this cost–loss ratio C/L to the climatological occurrence of persistent contrails pc. If C/L > pc, then no flights would ever be diverted, whereas all flights would be diverted if C/L < pc. In general, it is expected that any valuable forecast must be accurate enough such that the percentage of forecast misses pm [defined here as c/(c + d)] must be less than or equal to the climatological frequency (and the cost–loss ratio), and the percentage of forecast hits ph [defined here as a/(a + b)] must be greater than or equal to pc (and C/L). Murphy and Ehrendorfer (1987) show that, if 0 ≤ C/L ≤ pm ≤ pc ≤ ph ≤ 1 or 0 ≤ pm ≤ pc ≤ ph ≤ C/L ≤ 1, then the problem of diverting or not diverting flights becomes trivial. In the former case, the cost of diverting flights is so inexpensive that all flights should be diverted to avoid making contrails, whereas in the latter case the cost of diverting flights is so expensive that no flights should be diverted despite the loss incurred from the production of contrails. The potentially wide range between pm and ph in the test case models from Part I suggests that logistic models would be able to produce valuable persistent contrail occurrence forecasts for a variety of cost–loss situations. The results from Part I derived from synthetic observations show that pm = 0.028 when the climatological frequency is used as the probability threshold and ph = 0.506, even in scenario B1m where the random error is maximized. If 0.5 is used as the probability threshold, then pm = 0.077 and ph = 0.726 in scenario B2m. For comparison, the Build 2 SURFACE models built from (the dependent) and evaluated with (the independent) ARPS analysis data have pm = 0.169 and ph = 0.573. Considering that the pc measured from surface observers was 0.170 in Duda et al. (2009), and 0.152 in Minnis et al. (2003), the models presented here have marginal value because pm approximately equals pc. Logistic models built from a larger number of observations, however, may have positive value because the pm and ph for the Build 2 SURFACE models built from and evaluated with the same ARPS analysis data (dependent data) are 0.094 and 0.810, respectively.
It is important to note that the conclusions of Murphy and Ehrendorfer (1987) apply to a simple two-parameter (occurrence versus nonoccurrence) system. Much more complicated cost–loss relationships could be possible if the full capability of a probabilistic forecasting system was used. For example, given a forecast probability p in a forecast region, a fraction pd of all flights within that region could be diverted. Also, reliable probabilistic forecasts inherently have extra value to users compared to categorical (simple yes or no occurrence) forecasts because users can take advantage of cost–loss analyses better with probabilistic forecasts (Keith 2003).
5. Summary and concluding remarks
Probabilistic models of persistent contrail occurrence within the CONUS were developed from high-resolution numerical weather analyses and forecasts. Meteorological data from the 20-km Rapid Update Cycle and the Advanced Regional Prediction System were combined with observations of persistent contrail occurrence from surface reports and visual inspection of satellite imagery. Two groups of logistic models were created. The first group of models (SURFACE) is based on surface-based contrail observations supplemented with satellite observations of contrail occurrence. The most common predictors selected for the SURFACE models tend to be related to temperature, relative humidity, and wind direction when the models are generated using RUC or ARPS analyses. The second group of models (OUTBREAK) is derived from a selected subgroup of satellite-based observations of widespread persistent contrails. The most common predictors for the OUTBREAK models tend to be wind direction, atmospheric lapse rate, temperature, relative humidity, and the product of temperature and humidity.
Some unanswered issues about the effectiveness of the logistic model require future study. Aircraft may not fly at all times through some regions where persistent contrails are possible, although this is not expected to be a major problem for this study because much of the CONUS is nearly continually traveled by jet aircraft throughout the day. Also, persistent contrails are unlikely in regions where adverse weather conditions (such as convection, turbulence, and icing) are expected to occur and aircraft are likely to avoid. These errors in the accurate determination of contrail occurrence should be quantified and their impact on the logistic model should be addressed.
More work is needed to realize the potential of logistic contrail forecasts. The most direct way to make the logistic models better is to reduce the errors within the meteorological data used to build the models. Reductions in the uncertainties of meteorological variables to a point where acceptable contrail forecasts are produced would be a good goal for NWA modelers. As mentioned earlier, meteorological errors directly affect the regressions developed in the logistic model; if the errors are large enough, they may cause the model to choose less pertinent predictors, further reducing model accuracy. Meteorological analyses could be improved by using several methods including onboard measurements from commercial aircraft, humidity-corrected rawinsondes, and measurements from the Atmospheric Infrared Sounder on the Aqua satellite to supplement the temperature and relative humidity data in numerical weather models. Methods to reduce errors in the determination of contrail occurrence could also be pursued. Additional studies are needed to determine if other regionally or temporally averaged variables would increase the accuracy of logistic models based on numerical weather forecasts and if other atmospheric variables may be relevant. Regional and seasonal models of contrail occurrence may help improve the overall performance of this type of persistent contrail prediction model.
Improvements to the logistic models are also possible if reliable information regarding contrail altitude were available. No information regarding flight altitude is available in the GLOBE dataset, but specific knowledge about the flight level of each contrail is difficult to obtain even when flight track data are available, especially when air traffic density reaches levels seen in most US flight track corridor. The persistent contrails observed both from the surface and by satellite are often several minutes to hours old, and it is often not possible to conclusively determine which aircraft produced which contrail. Nevertheless, some contrail altitude information is now possible with active sensing systems such as the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) on the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) platform (Winker et al. 2004). Contrail altitude information would be critical if the models were developed in the future for contrail mitigation forecasts, because it would generally be more economical to divert aircraft vertically to avoid persistent contrail production than to reroute the aircraft horizontally. However, such contrail mitigation prediction models would also require accurate details of the vertical distribution of moisture in the upper troposphere.
Logistic models of contrail occurrence provide an additional advantage that has not been used here. Because logistic models compute a probability of occurrence, they could be useful in global circulation model (GCM) simulations of contrail coverage (Ponater et al. 2002; Marquart et al. 2003) to determine the impact of contrail radiative forcing on global climate. Such models use a simple analytical formula based on relative humidity and cirrus cloud coverage to determine contrail coverage. The logistic models could be easily used within the GCM to determine an appropriate contrail coverage fraction for a region based upon the product of the air traffic and the computed probability. Because the logistic model can be developed by comparing GCM simulations with actual contrail observations, it may provide more accurate simulations of contrail coverage than current methods.
Finally, some current numerical weather prediction models such as the European Centre for Medium-Range Weather Forecasts Integrated Forecast System now include supersaturation over the ice phase explicitly (Tompkins et al. 2007), and it is encouraging that the latest versions of both the RUC and ARPS at the time of this writing are now producing significantly greater levels of ice supersaturation than were available for comparisons with the observations used in this study. It may therefore be possible to forecast potential contrail coverage directly from forecast models that allow realistic ice supersaturations in the upper troposphere.
Acknowledgments
This material is based upon work supported by the NASA Earth Science Enterprise Radiation Sciences Division, the NASA Modeling, Analysis, and Prediction Program, NASA Contracts NAG1-02044 and NCCI-02043 NIA-2579, and by the National Science Foundation under Grant 0222623.
REFERENCES
Appleman, H., 1953: The formation of exhaust condensation trails by jet aircraft. Bull. Amer. Meteor. Soc., 34 , 14–20.
Benjamin, S. G., G. A. Grell, J. M. Brown, T. G. Smirnova, and R. Bleck, 2004a: Mesoscale weather prediction with the RUC hybrid isentropic-terrain-following coordinate model. Mon. Wea. Rev., 132 , 473–494.
Benjamin, S. G., and Coauthors, 2004b: An hourly assimilation-forecast cycle: The RUC. Mon. Wea. Rev., 132 , 495–518.
Brooks, D. R., and F. M. Mims III, 2001: Development of an inexpensive handheld LED-based Sun photometer for the GLOBE program. J. Geophys. Res., 106 , (D5). 4733–4740.
Carleton, A. M., D. J. Travis, K. Master, and S. Vezhapparambu, 2008: Composite atmospheric environments of jet contrail outbreaks for the United States. J. Appl. Meteor. Climatol., 47 , 641–667.
DeGrand, J. Q., A. M. Carleton, D. J. Travis, and P. J. Lamb, 2000: A satellite-based climatic description of jet aircraft contrails and associations with atmospheric conditions, 1977–79. J. Appl. Meteor., 39 , 1434–1459.
Duda, D. P., and P. Minnis, 2009: Basic diagnosis and prediction of persistent contrail occurrence using high-resolution numerical weather analyses/forecasts and logistic regression. Part I: Effects of random error. J. Appl. Meteor. Climatol., 48 , 1780–1789.
Duda, D. P., P. Minnis, L. Nguyen, and R. Palikonda, 2004: A case study of the development of contrail clusters over the Great Lakes. J. Atmos. Sci., 61 , 1132–1146.
Duda, D. P., R. Palikonda, and P. Minnis, 2009: Relating observations of contrail persistence to numerical weather analysis output. Atmos. Chem. Phys., 9 , 1357–1364.
Gandin, L. S., and A. H. Murphy, 1992: Equitable skill scores for categorical forecasts. Mon. Wea. Rev., 120 , 361–370.
Garber, D. P., P. Minnis, and P. K. Costulis, 2005: A commercial flight track database for upper tropospheric aircraft emission studies over the USA and southern Canada. Meteor. Z., 14 , 445–452.
Hosmer, D. W., and S. Lemeshow, 1989: Applied Logistic Regression. John Wiley & Sons, 307 pp.
Jackson, A., B. Newton, D. Hahn, and A. Bussey, 2001: Statistical contrail forecasting. J. Appl. Meteor., 40 , 269–279.
Keith, R., 2003: Optimization of value of aerodrome forecasts. Wea. Forecasting, 18 , 808–824.
Lee, T. F., 1989: Jet contrail identification using the AVHRR infrared split window. J. Appl. Meteor., 28 , 993–995.
Lin, Y-L., R. D. Farley, and H. D. Orville, 1983: Bulk parameterization of the snow field in a cloud model. J. Climate Appl. Meteor., 22 , 1065–1092.
Mannstein, H., R. Meyer, and P. Wendling, 1999: Operational detection of contrails from NOAA-AVHRR data. Int. J. Remote Sens., 20 , 1641–1660.
Marquart, S., M. Ponater, F. Mager, and R. Sausen, 2003: Future development of contrail cover, optical depth, and radiative forcing: Impacts of increasing air traffic and climate change. J. Climate, 16 , 2890–2904.
Minnis, P., J. K. Ayers, M. L. Nordeen, and S. P. Weaver, 2003: Contrail frequency over the United States from surface observations. J. Climate, 16 , 3447–3462.
Murphy, A. H., and M. Ehrendorfer, 1987: On the relationship between the accuracy and value of forecasts in the cost-loss ratio situation. Wea. Forecasting, 2 , 243–251.
O’Shea, R. P., 1991: Thumb’s rule tested: Visual angle of thumb’s width is about 2 deg. Perception, 20 (3) 415–418. doi:10.1068/p200415.
Ponater, M., S. Marquart, and R. Sausen, 2002: Contrails in a comprehensive global climate model: Parameterization and radiative forcing results. J. Geophys. Res., 107 , (D13). 4164. doi:10.1029/2001JD000429.
Tao, W-K., J. Simpson, and M. McCumber, 1989: An ice-water saturation adjustment. Mon. Wea. Rev., 117 , 231–235.
Tompkins, A. M., K. Gierens, and G. Rädel, 2007: Ice supersaturation in the ECMWF Integrated Forecast System. Quart. J. Roy. Meteor. Soc., 133 , 53–63.
Travis, D. J., A. M. Carleton, and S. A. Changnon, 1997: An empirical model to predict widespread occurrences of contrails. J. Appl. Meteor., 36 , 1211–1220.
Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 467 pp.
Winker, D. M., W. H. Hunt, and C. A. Hostetler, 2004: Status and performance of the CALIOP lidar. Laser Radar Techniques for Atmospheric Sensing, U. N. Singh, Ed., International Society for Optical Engineering (SPIE Proceedings, Vol. 5575), 8–15.
Xue, M., D-H. Wang, J-D. Gao, K. Brewster, and K. K. Droegemeier, 2003: The Advanced Regional Prediction System (ARPS), storm-scale numerical weather prediction and data assimilation. Meteor. Atmos. Phys., 82 , 139–170.
The centers of persistent contrail outbreaks identified from GOES imagery for the OUTBREAK models (crosses), and GLOBE schools reporting persistent contrail coverage for the SURFACE models (squares).
Citation: Journal of Applied Meteorology and Climatology 48, 9; 10.1175/2009JAMC2057.1
Persistent contrails are highlighted in a channel 4 (10.8 μm) minus channel 5 (12.0 μm) BTD image from the NOAA-17 overpass at 1531 UTC 18 Apr 2005 over the northeastern United States.
Citation: Journal of Applied Meteorology and Climatology 48, 9; 10.1175/2009JAMC2057.1
Potential parameters used in logistic regression models.
Locations of the GLOBE schools used in the development of the SURFACE models.
PC and HKD for several versions of RUC and ARPS analyses and forecasts for the SURFACE models. The percentage correct/HKD scores from the models evaluated with the independent data are presented in the left-hand column under each build, and the scores resulting from the evaluation with the dependent data are presented in parenthesis in the right-hand column under each build. The notation “Analysis eval. w/” represents the logistic models developed from ARPS analysis data that are evaluated using the ARPS 1-day, 2-day, or 3-day forecasts.
PC and HKD for OUTBREAK models developed from the RUC and ARPS analyses and from two simulations of the OUTBREAK models using the synthetic data from Part I. The mean PC/HKD of the 1000 realizations for cases 1, 2 and 3 are presented with the standard deviation of the scores in parenthesis.
