• Brown, B. G., Thompson G. , Bruintjes R. T. , Bullock R. , and Kane T. , 1997: Intercomparison of in-flight icing algorithms. Part II: Statistical verification results. Wea. Forecasting, 12 , 890914.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cober, S. G., Isaac G. A. , Korolev A. V. , and Strapp J. W. , 2001a: Assessing cloud-phase conditions. J. Appl. Meteor., 40 , 19671983.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cober, S. G., Isaac G. A. , and Strapp J. W. , 2001b: Characterizations of aircraft icing environments that include supercooled large drops. J. Appl. Meteor., 40 , 19842002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cortinas, J. J., 2000: A climatology of freezing rain in the Great Lakes region of North America. Mon. Wea. Rev., 128 , 35743588.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Côté, J., Desmarais J. , Gravel S. , Méthot A. , Patoine A. , Roch M. , and Staniforth A. , 1998a: The operational CMC–MRB Global Environmental Multiscale (GEM) model: Part II: Results. Mon. Wea. Rev., 126 , 13971418.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Côté, J., Gravel S. , Méthot A. , Patoine A. , Roch M. , and Staniforth A. , 1998b: The operational CMC–MRB Global Environmental Multiscale (GEM) model. Part I: Design considerations and formulation. Mon. Wea. Rev., 126 , 13731395.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fritsch, J. M., and Chappell C. F. , 1980: Numerical prediction of convectively driven mesoscale pressure systems. Part I: Convective parameterization. J. Atmos. Sci., 37 , 17221733.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guan, H., Cober S. G. , and Isaac G. A. , 2001: Verification of supercooled water forecasts with in situ aircraft measurements. Wea. Forecasting, 16 , 145155.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Isaac, G. A., Cober S. G. , Strapp J. W. , Korolev A. V. , Tremblay A. , and Marcotte D. L. , 2001: Recent Canadian research on aircraft in-flight icing. Can. Aeronaut. Space J., 47 , 213221.

    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and Fritsch J. M. , 1990: A one-dimensional entraining/detraining plume model and its application in convective parameterization. J. Atmos. Sci., 47 , 27842802.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and Fritsch J. M. , 1993: Convective parameterization for mesoscale models: The Kain–Fritsch scheme. The Representation of Cumulus Convection in Numerical Models of the Atmosphere, Meteor. Monogr., No. 46, Amer. Meteor. Soc., 165–170.

    • Search Google Scholar
    • Export Citation
  • Kong, F-Y., and Yau M. K. , 1997: An explicit approach to microphysics in MC2. Atmos.–Ocean, 35 , 257291.

  • Mailhot, J., and Coauthors. 1997: Scientific description of RPN physics library—Version 3.6. 197 pp. [Available online at http://gfx.weatheroffice.ec.gc.ca/cmc_library/index_e.html.].

    • Search Google Scholar
    • Export Citation
  • Rauber, R. M., Olthoff L. S. , Ramanmurthy M. K. , and Kunkel K. E. , 2000: The relative importance of warm rain and melting processes in freezing precipitation events. J. Appl. Meteor., 39 , 11851195.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stanski, H. R., Wilson L. J. , and Burrows W. R. , 1989: Survey of common verification methods in meteorology. World Weather Watch Tech. Rep. 8, WMO/TD. No. 358, 114 pp.

    • Search Google Scholar
    • Export Citation
  • Stuart, R. A., and Isaac G. A. , 1994: Archived weather data is providing new insights into ground-based icing. ICAO J., 49 (8) 57.

  • Stuart, R. A., and Isaac G. A. , 1999: Freezing precipitation in Canada. Atmos.–Ocean, 37 , 87102.

  • Sundqvist, H., Berge E. , and Kristjansson J. E. , 1989: Condensation and cloud parameterization studies with a mesoscale numerical weather prediction model. Mon. Wea. Rev., 117 , 16411657.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tremblay, A., and Glazer A. , 2000: An improved modeling scheme for freezing precipitation forecasts. Mon. Wea. Rev., 128 , 12891308.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tremblay, A., Glazer A. , Szyrmer W. , Isaac G. A. , and Zawadzki I. , 1995: Forecasting of supercooled clouds. Mon. Wea. Rev., 123 , 20982113.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tremblay, A., Glazer A. , Yu W. , and Benoit R. , 1996: An explicit cloud scheme based on a single prognostic equation. Tellus, 48A , 483500.

    • Search Google Scholar
    • Export Citation
  • Tremblay, A., Glazer A. , and Garand L. , 2001: Comparison of three cloud schemes in winter storm forecasts. Mon. Wea. Rev., 129 , 29232938.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaillancourt, P., Tremblay A. , Cober S. G. , and Isaac G. A. , 2000: Evaluation of a mixed-phase cloud scheme's ability at forecasting supercooled liquid water in clouds. Proc. 13th Int. Conf. on Cloud and Precipitation, Reno, NV, 1114–1117.

    • Search Google Scholar
    • Export Citation
  • View in gallery

    The 10-h forecast of sea level pressure (contoured at each 4 mb) superimposed with 850-mb TWC, valid 1600 UTC 18 Mar 1995. The flight track is shown as a solid black line

  • View in gallery

    Time histories of model (gray) and aircraft (black) observations of (a) temperature, (b) TWC, and (c) SLWC following the aircraft trajectory for a flight on 18 Mar 1995. The model data are based on the MIX cloud scheme

  • View in gallery

    Histograms of the HR, FAR, and TSS for the three cloud schemes, for CFDE I, CFDE III, AIRS, and the total dataset (total).

  • View in gallery

    Histograms of the number of observations and forecasts of clouds as a function of pressure, in 100-mb intervals for the total dataset

  • View in gallery

    Comparisons of HR, FAR, and TSS between a precise vertical height (black) and an altitude range of ±300 m (white) for the three cloud schemes and for the total dataset

  • View in gallery

    Comparisons of the measured and forecast TWC for each cloud scheme for cases where the measured and forecast TWC > 0.03 g m−3. The solid curves represent a 1:1 correlation. The rmse and CORRCOEF values for each scatterplot, with respect to the 1:1 correlation, are also shown

  • View in gallery

    Histograms of the number of observations of clouds with different phases. The first group (total) represents the total number of in-cloud observations including all phases. Glaciated cloud includes only clouds that are assessed to be entirely glaciated. Warm cloud represents clouds at temperatures >0°C, where the cloud is assumed to be liquid phase. Supercooled cloud includes cloud cases that are assessed to have SLWC, including mixed-phase cases

  • View in gallery

    Histograms of HR, FAR, and TSS for SLW forecasts for the MIX and KY schemes. The numbers of observed cloud cases with SLW and with no SLW were 1316 and 2586, respectively

  • View in gallery

    Comparison of measured and forecast SLWC and IWC for the MIX and KY schemes

  • View in gallery

    Histograms of the number of observed and forecast cloud conditions for liquid-, mixed-, and glaciated-phase cases at T < 0°C.

  • View in gallery

    Histogram of the mean accumulated precipitation values for each cloud scheme, for cloud regions along the aircraft flight tracks

  • View in gallery

    Verification statistics for the MIX cloud scheme as a function of forecast leadtime, based on the 12 CFDE I flights

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 138 121 1
PDF Downloads 31 22 1

Comparison of Three Cloud Forecast Schemes with In Situ Aircraft Measurements

View More View Less
  • 1 Cloud Physics Research Division, Meteorological Service of Canada, Downsview, Ontario, Canada
© Get Permissions
Full access

Abstract

In situ aircraft measurements, collected during three research projects, are used to compare forecasts from three explicit cloud schemes. These schemes include the Canadian operational Sundqvist (SUND) scheme, the Tremblay mixed-phase (MIX) cloud scheme, and the Kong and Yau (KY) cloud scheme. The supercooled liquid water forecast accuracy is also determined for the MIX and KY schemes. For the entire in situ dataset, the three cloud forecast schemes show a similar skill in detecting the presence of clouds, with a true skill statistic ranging between 0.27 and 0.34. Quantitative comparisons of total cloud water content (TWC), supercooled liquid water content (SLWC), and ice water content (IWC) suggest that adjustments for autoconversion thresholds for precipitation formation within the different cloud microphysical schemes would improve forecasts of SLWC, IWC, and TWC.

Corresponding author address: Hong Guan, Cloud Physics Research Division, Meteorological Service of Canada, 4905 Dufferin St., Downsview, ON M3H 5T4, Canada. Email: guan@clio.arc.nasa.gov

Abstract

In situ aircraft measurements, collected during three research projects, are used to compare forecasts from three explicit cloud schemes. These schemes include the Canadian operational Sundqvist (SUND) scheme, the Tremblay mixed-phase (MIX) cloud scheme, and the Kong and Yau (KY) cloud scheme. The supercooled liquid water forecast accuracy is also determined for the MIX and KY schemes. For the entire in situ dataset, the three cloud forecast schemes show a similar skill in detecting the presence of clouds, with a true skill statistic ranging between 0.27 and 0.34. Quantitative comparisons of total cloud water content (TWC), supercooled liquid water content (SLWC), and ice water content (IWC) suggest that adjustments for autoconversion thresholds for precipitation formation within the different cloud microphysical schemes would improve forecasts of SLWC, IWC, and TWC.

Corresponding author address: Hong Guan, Cloud Physics Research Division, Meteorological Service of Canada, 4905 Dufferin St., Downsview, ON M3H 5T4, Canada. Email: guan@clio.arc.nasa.gov

1. Introduction

Since 1992, the Meteorological Service of Canada (MSC) has conducted a series of winter field projects in the Newfoundland and the Great Lakes areas. These included the Second Canadian Atlantic Storm Program (CASP II) in 1992; the First, Second, and Third Canadian Freezing Drizzle Experiments (CFDE I, CFDE II, and CFDE III, respectively) between 1995 and 1998; and the Alliance Icing Research Study (AIRS) in 1999–2000. An objective of each project was the development of improved cloud and aircraft icing forecasts in winter storms (Isaac et al. 2001).

Comparative verification of forecasting systems and algorithms is a fundamental aspect of the development and improvement process (Brown et al. 1997). For example, based on an idea suggested by Tremblay et al. (1995), Guan et al. (2001) projected model data on research aircraft trajectories in order to directly compare the model results with in situ aircraft measurements. This procedure allows verifications of numerical weather prediction model parameters such as temperature, dewpoint depression, horizontal wind, and cloud total water content (TWC).

Using data collected during CASP II, CFDE I, and CFDE III, Guan et al. (2001) also estimated the accuracy of the operational Canadian cloud (Sundqvist et al. 1989) and icing (Tremblay et al. 1995) forecast schemes. These results suggested that the cloud forecasts were relatively poor and underestimated total water content (ice and liquid phase particles). To improve the icing forecast accuracy, Tremblay et al. (1995) developed a diagnostic icing algorithm that produced a binary yes–no forecast for icing events at each model grid point. Guan et al. (2001) found that the Tremblay icing algorithm had a rather low skillfulness that was heavily influenced by the poor cloud forecasts of the Sundqvist (SUND) scheme.

A number of advanced schemes are available within the RPN/CMC (Recherche en Prévision Numérique/Centre Météorologique Canadien) physics library, although they are not yet used for operational cloud and icing forecasts. These schemes generally employ more sophisticated microphysics to predict cloud and precipitation, as well as providing additional information about icing intensity and other weather hazards, such as freezing rain/drizzle, mixed precipitation, fog, or blowing snow. Considering the high frequency of aviation hazards during Canadian winters (Stuart and Isaac 1994), implementations of advanced microphysical schemes in the Canadian operational forecast system are desirable, provided that higher levels of forecast accuracy can be reached.

In this paper in situ aircraft measurements made during CFDE I, CFDE III, and AIRS are used to compare various aspects of these schemes, and pertinent statistical estimators are computed to provide insights about the strengths and weaknesses of the different cloud and icing forecast schemes.

2. Numerical weather prediction model and cloud schemes

The numerical weather prediction model used in this study is the High Resolution Model Application Project (HIMAP) version of the Canadian Global Environmental Multiscale (GEM) model (Côté et al. 1998a, b). The version used has approximately 15-km horizontal spacing and 35 eta levels. The time step is 7.5 min. Two uniform-resolution subdomains, centered on the Quebec–Windsor corridor and St. John's, Newfoundland, respectively, were used for the simulations. Each simulation was started at 0006 UTC, which is similar to the CMC operational GEM run (HIMAP version). The CMC objective analysis data were used to initialize the model and 6–24-h forecasts were produced for each case of interest.

The HIMAP version is fully coupled with the RPN/CMC physics library. The Fritsch–Chappell (Fritsch and Chappell 1980) cumulus parameterization scheme is the current operational CMC scheme, and was used for all simulations. The Kain–Fritsch (Kain and Fritsch 1990, 1993) scheme was also tested for several cases. The results (not shown) only indicate a minor difference from the Fritsch–Chappell scheme. This is reasonable since most winter storms contain primarily stratiform clouds (Tremblay et al. 2001).

The SUND scheme is currently the operational cloud forecast scheme at CMC. It has only one prognostic variable, cloud (ice or liquid water), and there is no distinction between liquid and solid phases at temperatures below 0°C. Therefore, it does not forecast supercooled liquid water content (SLWC). The scheme assumes that a grid cell may be partially filled with hydrometeors and a subgrid-scale cloud fraction was parameterized as a function of the relative humidity. Precipitation particles are assumed to fall out within each model time step. The autoconversion threshold value (mrF) is parameterized as a function of temperature (T) and the rate of total incoming precipitation from above (P). The formulation for mrF is described in Mailhot et al. (1998).

The MIX scheme developed by Tremblay et al. (1996) represents an improvement to a previous icing scheme (Tremblay et al. 1995). In MIX, TWC is explicitly forecasted and diagnosed into solid and liquid phases at temperatures below 0°C. Six explicit microphysical processes were considered in the MIX scheme including condensation and evaporation of cloud droplets, evaporation of rain, ice nucleation, deposition and sublimation of ice particles, sedimentation, and melting. The autoconversion thresholds for liquid- (kl) and solid- (ks) phase precipitation are specified as constant values. Detailed descriptions of each microphysical process are given in Tremblay et al. (1996) and Tremblay and Glazer (2000).

The Kong and Yau (1997) scheme (KY) considers three prognostic equations for cloud water, rainwater, and ice phase (ice and snow). The microphysical processes included in these prognostic equations are condensation, evaporation, autoconversion, accretion, nucleation, deposition, sublimation, freezing, melting, riming, aggregation, and sedimentation. Crystals are assumed to fall instantly as ice-forming nuclei are activated. Therefore there is no autoconversion threshold for the sedimentation of snow. The autoconversion threshold (qc0) for liquid-phase precipitation is much larger than that in the MIX scheme. Explicit cloud water and rainwater equations allow the KY scheme to forecast supercooled liquid water (SLW). Some features of the three cloud schemes are summarized in Table 1.

3. Aircraft data and verification methodology

The CFDE I, CFDE III, and AIRS field projects were conducted in the Newfoundland and the Great Lakes regions because these are the areas with the highest frequency of surface freezing precipitation in North America (Stuart and Isaac 1999; Cortinas 2000). For this study, aircraft data collected during 51 flights from CFDE I (12 cases), CFDE III (26 cases), and AIRS (13 cases) were used to compare with the cloud and icing forecast schemes. These projects are described in Isaac et al. (2001). The research aircraft employed during the three field programs was the National Research Council Convair-580, which was fully equipped for cloud microphysical measurements. Similar microphysical parameters were measured during all 51 flights and a description of the instrumentation and measurements can be found in Cober et al. (2001a, b).

The aircraft data were initially averaged over 30-s intervals (approximately 3 km of horizontal distance), with liquid water content (LWC), ice water content (IWC), and total water content (TWC) being measured to an accuracy of 15% by hot-wire-type probes (Cober et al. 2001b). The cloud phases were assessed to be liquid, mixed, or glaciated following Cober et al. (2001a). The verification methodology for cloud and SLW forecasts was similar to the one described in Guan et al. (2001). The model fields at 15-km and 1-h resolution were numerically interpolated along the 3D–time aircraft trajectories. The time series were then compared with 150-s (15 km) averaged aircraft measurements. It was previously demonstrated that the model–aircraft comparisons for horizontal resolutions of 1.5 and 15 km gave very similar results (Guan et al. 2001) because the cloud regions measured with the aircraft were generally stratiform in nature and covered a wide horizontal region.

Following Guan et al. (2001), the measured and forecast cloud and SLW were assessed as observed or not observed and forecast or not forecast based on certain thresholds. For the present study, a threshold of 0.03 g m−3 was selected for IWC, LWC, and TWC. Sensitivity tests (Guan et al. 2001) indicated that the use of SLW thresholds between 0.03 and 0.1 g m−3 produced only small changes in the verification statistics. For each 15-km data point with coincident model and aircraft data, two sets of yes/no observations for the SLW and cloud events were assessed. A yes observation for cloud–SLW in a 15-km grid point represents one cloud–SLW event. Table 2 incorporates two sets of yes/no assessments for the SLW and cloud observations into the basic 2 × 2 contingency table. Following Stanski et al. (1989), statistical estimators including the hit rate (HR), false-alarm rate (FAR), and true skill statistics (TSS) have been calculated based on the following equations:
i1520-0434-17-6-1226-e1
Here, HR (FAR) can be interpreted as the proportion of observed (not observed) events that were correctly (incorrectly) forecast. A perfect forecast is described by HR = 1, FAR = 0, and TSS = 1.

It should be pointed out that on the scale of a model grid point, the in situ aircraft measurements are based on samples of only a few cubic meters as compared with the model grid volume of approximately 1010 m3. Regardless, in situ aircraft observations are the only existing source of direct cloud microphysical measurements and, hence, are a valuable tool for directly comparing with the results of cloud models. Future satellite-based observation platforms that will incorporate radars and lidars may allow large-scale observations of cloud microphysical properties, albeit with reduced accuracy.

Figures 1 and 2 provide an example of model data compared with aircraft data. The CFDE I case depicted in Fig. 1 reveals a cloud band that is associated with a low pressure system located to the southeast of Nova Scotia. The aircraft took off from Stephenville (1457 UTC), flew south toward the low pressure region, and landed in St. John's, Newfoundland (1847 UTC). The model-projected time series of temperature, TWC, and SLWC along the aircraft trajectory are compared with aircraft measurements in Fig. 2 for the MIX scheme. The data in Fig. 2 represent time histories of 80 distinct, 15-km data points. The simulated temperature compares well with the observed temperature during the aircraft measurements. The average difference between the two curves is about 0.6°C, which is less than the instrument error (±1°C). Consistent with the aircraft observations, the simulated clouds were present throughout most of the aircraft trajectory. The aircraft encountered two significant supercooled water regions (marked with A and B) during the flight and both were predicted, although the extent of the region of SLW was different. Region A was overpredicted, while region B was underestimated.

4. Verification results

a. Verifications of clouds

The HR, FAR, and TSS of cloud forecasts for CFDE I, CFDE III, AIRS, and the total dataset for each scheme are shown in Fig. 3. The number of data points of observed cloud (no cloud) cases for CFDE I, CFDE III, AIRS, and total are 540 (553), 1116 (1193), 401 (713), and 2057 (2459), respectively. The HR, FAR, and TSS values for the SUND scheme for CFDE I and CFDE III are similar to the results shown in Guan et al. (2001), although a larger dataset for CFDE III and a lower spatial resolution are used in the current study. For CFDE I the HR, FAR, and TSS values for KY and MIX are very similar, with the TSS ranging between 0.35 and 0.38. Conversely, the SUND scheme has a lower TSS of 0.21, which is primarily associated with a low HR value. For CFDE III and AIRS, the differences in TSS values between the three cloud schemes were relatively small, ranging from 0.21 to 0.33. Overall, the TSS is similar for each microphysical algorithm, suggesting that other aspects of the model dominate these results. As demonstrated in Guan et al. (2001) and Côté et al. (1998a), the root-mean-square error (rmse) for dewpoint depression is significantly larger than that for temperature. They suggested that the model humidity field needs to be better initialized and simulated before marked improvement in the forecast skill for cloud would be obtained.

The vertical distributions of the number of observed and simulated cloud observations for the total dataset are displayed in Fig. 4. Clouds were more frequently observed below 700 mb. Between 900 and 1000 mb, clouds were underestimated by all schemes, particularly SUND. The low forecast accuracy for low-level cloud for the SUND scheme is consistent with the results of Guan et al. (2001). The lack of a sedimentation process in the SUND scheme may cause a smaller number of cloud forecasts in the lower atmosphere because precipitation particles were forced to fall to the surface. This may artificially reduce the cloud amount in the lower atmosphere. All schemes except the KY scheme also underestimated cloud between 800 and 900 mb. All schemes demonstrated similar performances between 500 and 800 mb and were in better agreement with the observations.

To test the effect of errors in the vertical position of forecast clouds, the model cloud fields were assessed over a greater altitude range. Model fields within 300 m above and below the aircraft altitude were compared to the aircraft measurements. If a forecast value at any of the model levels was yes, then a yes forecast was assigned to the model field. If all of the model level forecasts were no, the model field was assigned a no forecast. The HR, FAR, and TSS for the three cloud schemes are compared with those for precise vertical altitudes in Fig. 5. For all schemes, the HR and FAR obtained over an altitude range are larger than those obtained using the precise vertical coincidence of model and observed clouds. Conversely, the TSS values only display small changes for all schemes, indicating that the forecast skill is not significantly affected by an error of ±300 m in the vertical position of forecast clouds.

To gain insight into the accuracy of the cloud microphysical forecasts, comparisons between the model and aircraft TWC measurements were made. Scatterplots of model and aircraft TWC for all schemes are displayed in Fig. 6, with the corresponding rmse and correlation coefficient (CORRCOEF). Only data with a TWC > 0.03 g m−3 (both aircraft and model) are shown in Fig. 6. In general, the measured TWC was less than 0.4 g m−3. The SUND scheme tends to underestimate TWC, while the two other schemes generally overestimate TWC. This partly explains the lower HR and TSS for the SUND scheme, in comparison with the other schemes. The KY scheme has numerous cases with TWC > 0.4 g m−3, which was contrary to the aircraft measurements. Better model estimates of TWC could be obtained by adjusting the autoconversion thresholds for precipitation formation.

Following Guan et al. (2001), model data were linearly interpolated between 1-h outputs to match the aircraft data at a frequency of 150 s. This methodology might fail to capture some rapidly changing TWC values. To test the accuracy of the 1-h interpolation methodology, the model data were output at a frequency of 150 s for a limited subset of the data (AIRS). The differences in HR, FAR, TSS, and scatterplots between the two comparison methodologies (not shown) are small, demonstrating that the 1-h interpolation methodology was an acceptable technique for the cases considered. However, the 150-s model output did include several TWC values >0.4 g m−3 that were missed by the 1-h model. A more elaborate procedure for model data processing, such as the one suggested by Vaillancourt et al. (2000), would minimize the interpolation of model data and would more accurately capture rapidly changing TWC conditions in the model output.

b. Verification of cloud phases

To qualitatively assess the different cloud phase forecasts, the total number of observations of cloud, glaciated cloud, warm cloud at temperatures >0°C, and supercooled cloud from all cloud schemes and aircraft measurements are displayed in Fig. 7. The total cloud numbers forecast from all schemes are very similar to the aircraft measurements. Also, all schemes predicted essentially the same number of warm clouds, which demonstrates that the model temperature is fairly reliable.

Based on aircraft observations depicted in Fig. 7, clouds with supercooled liquid water accounted for 71% of in-cloud observations, while glaciated conditions accounted for 16%. Since the SUND scheme cannot predict supercooled clouds, the clouds at temperature below 0°C are considered as glaciated cloud, and so a comparison to the aircraft observations is misleading. The MIX and KY schemes explicitly predict supercooled clouds, but overestimated the number of glaciated cases, and underestimated the number of supercooled cases. The number of glaciated clouds and supercooled clouds from KY is in better agreement with the aircraft observations, as compared with MIX. The KY scheme underestimated the number of supercooled clouds by 14%, while the number of supercooled clouds in MIX is 58% smaller than observed. The numbers of glaciated clouds from KY and MIX were 100% and 200%, respectively, higher than the number observed with the aircraft.

c. Verification of SLW forecasts

The verifications of SLW for the MIX and KY schemes are shown in Fig. 8. MIX performed better in CFDE I, while KY performed better for CFDE III and AIRS. The TSS values for MIX display significant differences from project to project, while a relative constant TSS was found for KY. These results are consistent with those previously presented for cloud forecasts. For the total dataset, the HR, FAR, and TSS values for the KY scheme are approximately a factor of 2 higher than those for the MIX scheme. The total dataset is weighted more heavily to the CFDE III and AIRS (continental) results, because of the relative numbers of data points in each project. This favors the KY because continental clouds tend to have more mixed-phase and fewer glaciated-phase conditions than maritime clouds. The lower number of SLW observations for the MIX scheme, as compared with the KY scheme, as shown in Fig. 7, is consistent with the results of Fig. 8. These results are explained by the fact that the two schemes have different thresholds for the onset of precipitation. In KY, the autoconversion threshold is 0.5 g kg−1 while in MIX a value of 0.1 g m−3 has been selected for the liquid-phase threshold. The MIX scheme can generate more SLW clouds by increasing the liquid-phase threshold for sedimentation. Decreasing the autoconversion threshold in KY can reduce the amount of SLW clouds.

The TSS scores for the MIX and KY schemes are similar to those demonstrated in other validation studies. The TSS for the MIX scheme is similar to those attained by Guan et al. (2001) who validated the combination of the SUND cloud and Tremblay diagnostic SLW forecasting schemes (Tremblay et al. 1995). Brown et al. (1997) compared pilot reports over the continental United States with SLW forecasts, for the time period 25 January–25 March 1994. Brown et al. (1997) also computed the probability of detection of yes (POD) and no (PODNo) observations for two different numerical weather prediction models (the Eta Model and the Mesoscale Analysis and Prediction System) and three different icing algorithms (the National Aviation Weather Advisory Unit, the National Center for Atmospheric Research's Research Applications Program, and the U.S. Air Force). The TSS scores, as based on POD and PODNo of Brown et al. (1997), vary between 0.12 and 0.26, and are similar to the present study.

Scatterplots of model observations and aircraft measurements of SLWC and IWC are shown in Fig. 9, for the MIX and KY schemes. There is generally a poor correlation between the model and aircraft observations, and the results are similar to the TWC comparisons shown in Fig. 6. In general, the MIX scheme underestimates SLWC, while the KY scheme tends to overestimate SLWC. There are very few aircraft measurements with SLWC of more than 0.4 g m−3. Conversely, the KY scheme produces numerous cases with SLWC > 0.4 g m−3. Only a few points with SLWC > 0.15 g m−3 are observed for the MIX scheme. As discussed above, the opposite behavior for the MIX and KY scheme likely arises from the different rain autoconversion thresholds used in the models. These thresholds are 0.1 g m−3 and 0.5 g kg−1, for the MIX and KY schemes, respectively, which correspond approximately with the observed SLWC limits of 0.15 and 0.6 g m−3. This suggests that matching the autoconversion threshold with in situ observations may provide quantitative improvements in the forecast accuracies for both schemes. The IWC observations for the KY and MIX schemes are similar, with both schemes generally overestimating IWC in comparison to the aircraft measurements.

The numbers of observations of liquid, mixed, and glaciated cloud conditions observed with each scheme are shown in Fig. 10. These are compared with the aircraft measurements, which were determined following Cober et al. (2001a). For CFDE I, both schemes forecast similar numbers of mixed-, liquid-, and glaciated-phase cloud cases, although the schemes overestimate the number of glaciated cases and underestimate the number of liquid-phase cases. For CFDE III and AIRS, there are greater differences between the two schemes and the measurements. The KY scheme more closely parallels the aircraft observations, although it overestimates (underestimates) the number of glaciated- and liquid- (mixed-) phase clouds. The MIX scheme significantly underestimates the number of liquid-phase clouds and significantly overestimates the number of glaciated-phase clouds.

The supercooled warm rain process in a precipitating cloud is not included in the MIX scheme (Tremblay and Glazer 2000). Rauber et al. (2000) analyzed 972 soundings taken during freezing precipitation, with the soundings covering regions of the United States east of the Rockies for the period 1970–94. They found that 47% of freezing precipitation events could be attributed to a condensation and collision–coalescence process (nonclassical mechanism). Cober et al. (2001b) showed that most of the supercooled large drops observed during CFDE I and CFDE III were formed through the nonclassical mechanism. Obviously, neglecting the supercooled warm rain process in the MIX scheme will lead to an underestimation in the frequency of liquid-phase clouds. Project-specific differences in the performances of the MIX and KY schemes may be related to the cloud temperatures observed. The CFDE III and AIRS projects were conducted in a colder environment than the CFDE I project. The mean cloud temperatures for regions where SLW was observed in CFDE I, CFDE III, and AIRS were −4.4°, −6.2°, and −8.4°C, respectively. In the MIX scheme, the relative amount of water vapor available for forming droplets is temperature dependent, and is greater at warmer temperatures. At colder temperatures, the MIX scheme puts more vapor into ice crystal growth, and cannot generate essentially liquid-phase clouds. Since the MIX scheme in general overestimates the fraction of glaciated clouds, it is suggested that there is too much emphasis on ice crystal growth, which would be exaggerated at colder temperatures. Hence the poorer performance of this scheme in CFDE III and AIRS, versus CFDE I. It is likely that this could be corrected by adjusting the ice crystal growth algorithms using in situ measurements.

d. Comparison of surface precipitation

To illustrate the effect of the autoconversion thresholds on precipitation, the mean accumulated surface precipitation along the horizontal projection of the aircraft trajectories is shown in Fig. 11 for the total dataset. It can be seen that KY forecasts precipitation values approximately 50% lower than the other schemes, which is likely related to the use of a higher autoconversion threshold (0.5 g kg−1) as compared with MIX (0.1 g m−3). This also explains why the KY scheme predicts more cases of supercooled cloud and higher values of SLWC, than the MIX scheme, since more hydrometeor mass remains in the form of SLW in the KY scheme.

e. Forecast lead time

Model performances are dependent on forecast lead times. The majority of the aircraft observations were made between 1500 and 2100 UTC, implying that the average aircraft–model intercomparison was for the 12-h forecast. To test if the cloud verification results changed significantly with forecast lead time, the GEM model was run for lead times of 12, 18, 24, and 30 h for the 12 CFDE I cases. The results for HR, FAR, and TSS are shown in Fig. 12 for the MIX cloud scheme. There are no strong changes of HR or FAR with lead time. The maximum TSS value of 0.38 was obtained with a 12-h lead time, and the TSS remained approximately constant between 0.29 and 0.32, for times between 18 and 30 h.

5. Conclusions

Following a methodology outlined in Guan et al. (2001), cloud and icing forecasts generated with three microphysical schemes were compared with in situ aircraft measurements. The 15-km HIMAP version of the Canadian GEM model was used to produce these forecasts. The aircraft data were collected during 51 flights from three research field programs and were averaged over a 150-s time interval (≈15 km of horizontal distance) for comparison with the model gridpoint data.

The verification results suggest a nearly equivalent ability for forecasting the presence or absence of clouds for all algorithms. The TSS values ranged between 0.27 and 0.34, which demonstrates a level of skill well above that associated with an entirely unskilled forecast. However, there is room for considerable improvement, even for the most complex prognostic scheme of KY. The similarity of TSS values suggests that other aspects of the model also need to be improved. Côté et al. (1998a) and Guan et al. (2001) have found that the forecast of the humidity field in the GEM model was poor, implying that the humidity of the model needs to be better initialized and simulated before marked improvement in the forecast skill for clouds would be obtained. A quantitative comparison of the forecast and observed TWC indicates that the two more detailed schemes (KY and MIX) generally overestimated the TWC, while SUND underestimated it. Different settings for adjustable parameters incorporated in KY and MIX, such as autoconversion thresholds, would likely provide improved forecasts of the microphysical properties of the clouds, suggesting that an optimization of the cloud models based on observations is desirable.

The present study suggests that KY has a superior skill in forecasting supercooled clouds as compared with MIX, mostly explained by a different selection of parameters incorporated in the two cloud microphysical parameterizations. Comparisons with the number of aircraft measurements of liquid-, mixed-, and glaciated-phase clouds, demonstrate that the MIX and KY schemes over- (under-) estimates the number of glaciated- (mixed-) phase clouds. The KY scheme also overestimates liquid-phase clouds, while the MIX scheme underestimates liquid-phase clouds. Even if the KY scheme incorporates more detailed microphysics than the MIX scheme, it cannot be said, based on the present study, that there is a substantial benefit in increasing the complexity of the microphysical parameterization. This is because none of the schemes discussed in this investigation have been optimized on the basis of atmospheric observations. This work suggests that such an optimization is feasible and would lead to improved cloud and icing forecasts.

Acknowledgments

The Canadian National Search and Rescue Secretariat, Environment Canada, the National Research Council of Canada, and Transport Canada provided funding for this research. The authors would like to acknowledge Judy St-James for assistance with the generation of the geophysical fields.

REFERENCES

  • Brown, B. G., Thompson G. , Bruintjes R. T. , Bullock R. , and Kane T. , 1997: Intercomparison of in-flight icing algorithms. Part II: Statistical verification results. Wea. Forecasting, 12 , 890914.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cober, S. G., Isaac G. A. , Korolev A. V. , and Strapp J. W. , 2001a: Assessing cloud-phase conditions. J. Appl. Meteor., 40 , 19671983.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cober, S. G., Isaac G. A. , and Strapp J. W. , 2001b: Characterizations of aircraft icing environments that include supercooled large drops. J. Appl. Meteor., 40 , 19842002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cortinas, J. J., 2000: A climatology of freezing rain in the Great Lakes region of North America. Mon. Wea. Rev., 128 , 35743588.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Côté, J., Desmarais J. , Gravel S. , Méthot A. , Patoine A. , Roch M. , and Staniforth A. , 1998a: The operational CMC–MRB Global Environmental Multiscale (GEM) model: Part II: Results. Mon. Wea. Rev., 126 , 13971418.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Côté, J., Gravel S. , Méthot A. , Patoine A. , Roch M. , and Staniforth A. , 1998b: The operational CMC–MRB Global Environmental Multiscale (GEM) model. Part I: Design considerations and formulation. Mon. Wea. Rev., 126 , 13731395.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fritsch, J. M., and Chappell C. F. , 1980: Numerical prediction of convectively driven mesoscale pressure systems. Part I: Convective parameterization. J. Atmos. Sci., 37 , 17221733.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guan, H., Cober S. G. , and Isaac G. A. , 2001: Verification of supercooled water forecasts with in situ aircraft measurements. Wea. Forecasting, 16 , 145155.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Isaac, G. A., Cober S. G. , Strapp J. W. , Korolev A. V. , Tremblay A. , and Marcotte D. L. , 2001: Recent Canadian research on aircraft in-flight icing. Can. Aeronaut. Space J., 47 , 213221.

    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and Fritsch J. M. , 1990: A one-dimensional entraining/detraining plume model and its application in convective parameterization. J. Atmos. Sci., 47 , 27842802.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and Fritsch J. M. , 1993: Convective parameterization for mesoscale models: The Kain–Fritsch scheme. The Representation of Cumulus Convection in Numerical Models of the Atmosphere, Meteor. Monogr., No. 46, Amer. Meteor. Soc., 165–170.

    • Search Google Scholar
    • Export Citation
  • Kong, F-Y., and Yau M. K. , 1997: An explicit approach to microphysics in MC2. Atmos.–Ocean, 35 , 257291.

  • Mailhot, J., and Coauthors. 1997: Scientific description of RPN physics library—Version 3.6. 197 pp. [Available online at http://gfx.weatheroffice.ec.gc.ca/cmc_library/index_e.html.].

    • Search Google Scholar
    • Export Citation
  • Rauber, R. M., Olthoff L. S. , Ramanmurthy M. K. , and Kunkel K. E. , 2000: The relative importance of warm rain and melting processes in freezing precipitation events. J. Appl. Meteor., 39 , 11851195.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stanski, H. R., Wilson L. J. , and Burrows W. R. , 1989: Survey of common verification methods in meteorology. World Weather Watch Tech. Rep. 8, WMO/TD. No. 358, 114 pp.

    • Search Google Scholar
    • Export Citation
  • Stuart, R. A., and Isaac G. A. , 1994: Archived weather data is providing new insights into ground-based icing. ICAO J., 49 (8) 57.

  • Stuart, R. A., and Isaac G. A. , 1999: Freezing precipitation in Canada. Atmos.–Ocean, 37 , 87102.

  • Sundqvist, H., Berge E. , and Kristjansson J. E. , 1989: Condensation and cloud parameterization studies with a mesoscale numerical weather prediction model. Mon. Wea. Rev., 117 , 16411657.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tremblay, A., and Glazer A. , 2000: An improved modeling scheme for freezing precipitation forecasts. Mon. Wea. Rev., 128 , 12891308.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tremblay, A., Glazer A. , Szyrmer W. , Isaac G. A. , and Zawadzki I. , 1995: Forecasting of supercooled clouds. Mon. Wea. Rev., 123 , 20982113.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tremblay, A., Glazer A. , Yu W. , and Benoit R. , 1996: An explicit cloud scheme based on a single prognostic equation. Tellus, 48A , 483500.

    • Search Google Scholar
    • Export Citation
  • Tremblay, A., Glazer A. , and Garand L. , 2001: Comparison of three cloud schemes in winter storm forecasts. Mon. Wea. Rev., 129 , 29232938.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaillancourt, P., Tremblay A. , Cober S. G. , and Isaac G. A. , 2000: Evaluation of a mixed-phase cloud scheme's ability at forecasting supercooled liquid water in clouds. Proc. 13th Int. Conf. on Cloud and Precipitation, Reno, NV, 1114–1117.

    • Search Google Scholar
    • Export Citation

Fig. 1.
Fig. 1.

The 10-h forecast of sea level pressure (contoured at each 4 mb) superimposed with 850-mb TWC, valid 1600 UTC 18 Mar 1995. The flight track is shown as a solid black line

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 2.
Fig. 2.

Time histories of model (gray) and aircraft (black) observations of (a) temperature, (b) TWC, and (c) SLWC following the aircraft trajectory for a flight on 18 Mar 1995. The model data are based on the MIX cloud scheme

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 3.
Fig. 3.

Histograms of the HR, FAR, and TSS for the three cloud schemes, for CFDE I, CFDE III, AIRS, and the total dataset (total).

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 4.
Fig. 4.

Histograms of the number of observations and forecasts of clouds as a function of pressure, in 100-mb intervals for the total dataset

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 5.
Fig. 5.

Comparisons of HR, FAR, and TSS between a precise vertical height (black) and an altitude range of ±300 m (white) for the three cloud schemes and for the total dataset

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 6.
Fig. 6.

Comparisons of the measured and forecast TWC for each cloud scheme for cases where the measured and forecast TWC > 0.03 g m−3. The solid curves represent a 1:1 correlation. The rmse and CORRCOEF values for each scatterplot, with respect to the 1:1 correlation, are also shown

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 7.
Fig. 7.

Histograms of the number of observations of clouds with different phases. The first group (total) represents the total number of in-cloud observations including all phases. Glaciated cloud includes only clouds that are assessed to be entirely glaciated. Warm cloud represents clouds at temperatures >0°C, where the cloud is assumed to be liquid phase. Supercooled cloud includes cloud cases that are assessed to have SLWC, including mixed-phase cases

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 8.
Fig. 8.

Histograms of HR, FAR, and TSS for SLW forecasts for the MIX and KY schemes. The numbers of observed cloud cases with SLW and with no SLW were 1316 and 2586, respectively

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 9.
Fig. 9.

Comparison of measured and forecast SLWC and IWC for the MIX and KY schemes

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 10.
Fig. 10.

Histograms of the number of observed and forecast cloud conditions for liquid-, mixed-, and glaciated-phase cases at T < 0°C.

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 11.
Fig. 11.

Histogram of the mean accumulated precipitation values for each cloud scheme, for cloud regions along the aircraft flight tracks

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Fig. 12.
Fig. 12.

Verification statistics for the MIX cloud scheme as a function of forecast leadtime, based on the 12 CFDE I flights

Citation: Weather and Forecasting 17, 6; 10.1175/1520-0434(2002)017<1226:COTCFS>2.0.CO;2

Table 1.

Features of the three cloud schemes used for the intercomparisons

Table 1.
Table 2.

The contingency table used to evaluate forecasts of cloud and SLW

Table 2.
Save