A Verification of Numerical Model Forecasts for Sounding-Derived Indices above Udine, Northeast Italy

Agostino Manzato Osservatorio Meteorologico Regionale (OSMER) dell’ARPA, Friuli Venezia Giulia, Visco, Udine, Italy

Search for other papers by Agostino Manzato in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

In this work, 40 different indices derived from real soundings and the corresponding ECMWF model forecasts for the same location (near Udine, northeast Italy) are compared. This comparison is repeated for more than 500 days, from June 2004 to December 2005. The comparison is performed in terms of linear correlation, bias, root-mean-square error, and relative absolute error. The results show that the quality of agreement varies considerably among the different indices and lead times.

It is found that only 15 indices have strong correlations (R2 always larger than 0.5) that hold up until the fourth forecast day (+96 h). The indices having the best agreement are usually representative of the thermal fields while those having the worst agreement are usually representative of the wind fields, particularly in the low levels. It is also found that the correlations for all 40 of the indices change for the 4 times of the day (one sounding every 6 h), with the 1200 UTC comparison being usually the best one. Also, at 1800 and 0000 UTC the potential instability is strongly underestimated by the model.

To test if these errors depend on the vertical resolutions, a similar comparison has been done between the observed sounding reduced in the World Meteorological Organization TEMP format (as assimilated by the NWP model) and the originals at high vertical resolution. It is found that the indices derived from the TEMP format vary from those with the highest resolution, in particular the indices that depend on the details of the low-level profile, such as the level of free convection and the initial height of the lifted parcel. Nevertheless, only the shear has a very low correlation (R2 = 0.15) with the corresponding index derived from the high-resolution soundings, because it is very sensitive to the number of vertical levels used.

Corresponding author address: Agostino Manzato, Osservatorio Meteorologico Regionale dell’ARPA, Via Oberdan 18/a, Friuli Venezia Giulia, I-33040 Visco, Udine, Italy. Email: agostino.manzato@osmer.fvg.it

Abstract

In this work, 40 different indices derived from real soundings and the corresponding ECMWF model forecasts for the same location (near Udine, northeast Italy) are compared. This comparison is repeated for more than 500 days, from June 2004 to December 2005. The comparison is performed in terms of linear correlation, bias, root-mean-square error, and relative absolute error. The results show that the quality of agreement varies considerably among the different indices and lead times.

It is found that only 15 indices have strong correlations (R2 always larger than 0.5) that hold up until the fourth forecast day (+96 h). The indices having the best agreement are usually representative of the thermal fields while those having the worst agreement are usually representative of the wind fields, particularly in the low levels. It is also found that the correlations for all 40 of the indices change for the 4 times of the day (one sounding every 6 h), with the 1200 UTC comparison being usually the best one. Also, at 1800 and 0000 UTC the potential instability is strongly underestimated by the model.

To test if these errors depend on the vertical resolutions, a similar comparison has been done between the observed sounding reduced in the World Meteorological Organization TEMP format (as assimilated by the NWP model) and the originals at high vertical resolution. It is found that the indices derived from the TEMP format vary from those with the highest resolution, in particular the indices that depend on the details of the low-level profile, such as the level of free convection and the initial height of the lifted parcel. Nevertheless, only the shear has a very low correlation (R2 = 0.15) with the corresponding index derived from the high-resolution soundings, because it is very sensitive to the number of vertical levels used.

Corresponding author address: Agostino Manzato, Osservatorio Meteorologico Regionale dell’ARPA, Via Oberdan 18/a, Friuli Venezia Giulia, I-33040 Visco, Udine, Italy. Email: agostino.manzato@osmer.fvg.it

Save
  • Black, T. L., 1994: The New NMC mesoescale Eta Model: Description and forecast examples. Wea. Forecasting, 9 , 256278.

  • Bouttier, F., 2001: The development of 12-hourly 4D-Var. ECMWF Tech. Memo. 348, Reading, United Kingdom, 21 pp.

  • Elmore, K. L., Stensrud D. J. , and Crawford K. C. , 2002: Ensemble cloud model applications to thunderstorm forecasting. J. Appl. Meteor., 41 , 363383.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Galway, J. G., 1956: The lifted index as a predictor of latent instability. Bull. Amer. Meteor. Soc., 37 , 528529.

  • Gandin, L. S., Morone L. L. , and Collins W. G. , 1993: Two years of operational comprehensive hydrostatic quality control at the National Meteorological Center. Wea. Forecasting, 8 , 5772.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • George, J. J., 1960: Weather Forecasting for Aeronautics. Academic Press, 673 pp.

  • Haklander, A. J., and Van Delden A. , 2003: Thunderstorm predictors and their forecast skill for the Netherlands. Atmos. Res., 67–68 , 273299.

    • Search Google Scholar
    • Export Citation
  • Hart, R. E., Forbes G. S. , and Grumm R. H. , 1998: The use of hourly model-generated soundings to forecast mesoscale phenomena. Part I: Forecasting warm-season phenomena. Wea. Forecasting, 13 , 11651185.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huntrieser, H., Schiesser H. H. , Schmid W. , and Waldvogel A. , 1997: Comparison of traditional and newly developed thunderstorm indices for Switzerland. Wea. Forecasting, 12 , 108125.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., and Stephenson D. B. , 2003: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. J. Wiley and Sons, 240 pp.

    • Search Google Scholar
    • Export Citation
  • Klinker, E., Rabier F. , Kelly G. , and Mahfouf J-F. , 2000: The ECMWF operational implementation of four-dimensional variational assimilation. Part III: Experimental results and diagnostics with operational configuration. Quart. J. Roy. Meteor. Soc., 126 , 11911215.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lavagnini, A., Sempreviva A. M. , Transerici C. , Accadia C. , Casaioli M. , Mariani S. , and Speranza A. , 2006: Offshore wind climatology over the Mediterranean basin. Wind Energy, 9 , 251266.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lesht, B. M., Miloshevich L. M. , and Liljegren J. C. , 2004: Comparison of ECMWF model analyses with the observed upper-air temperature and relative humidity climatology at the ARM NSA, SGP, and TWP Climate Research Facility Sites. Proc. 14th Atmospheric Radiation Measurement (ARM) Science Team Meeting, Albuquerque, NM, U.S. Department of Energy.

  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20 , 130141.

  • Manzato, A., 2003: A climatology of instability indices derived from Friuli Venezia Giulia soundings, using three different methods. Atmos. Res., 67–68 , 417454.

    • Search Google Scholar
    • Export Citation
  • Manzato, A., 2005a: The use of sounding-derived indices for a neural network short-term thunderstorm forecast. Wea. Forecasting, 20 , 896917.

  • Manzato, A., 2005b: An odds ratio parameterization for ROC diagram and skill score indices. Wea. Forecasting, 20 , 918930.

  • Manzato, A., 2007a: The 6 hours climatology of thunderstorms and rainfalls in the Friuli Venezia Giulia plain. Atmos. Res., 83 , 336348.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Manzato, A., 2007b: Sounding-derived indices for neural network based short-term thunderstorm and rainfall forecasts. Atmos. Res., 83 , 349365.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Manzato, A., and Morgan G. M. , 2003: Evaluating the sounding instability with the lifted parcel theory. Atmos. Res., 67–68 , 455473.

    • Search Google Scholar
    • Export Citation
  • Mesinger, F., Janjić Z. I. , Nivckovic S. , Gavrilov D. , and Deaven D. G. , 1988: The step-mountain coordinate: Model description and performance for cases of Alpine lee cyclogenesis and for a case of an Appalachian redevelopment. Mon. Wea. Rev., 116 , 14931518.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morgan G. M. Jr., , 1992: ThetaPlot, an equivalent potential temperature diagram. Meteor. Atmos. Phys., 47 , 259265.

  • Pailleux, J., and Ed., 1997: Impact of various observing systems on numerical weather prediction: Proceedings of a CGC/WMO Workshop. WWW Tech. Rep. 18, WMO/TD 868, 198 pp.

  • Rabier, F., Jarvinen H. , Klinker E. , Mahfouf J-F. , and Simmons A. , 2000: The ECMWF operational implementation of four-dimensional variational assimilation. Part I: Experimental results with simplified physics. Quart. J. Roy. Meteor. Soc., 126 , 11431170.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Showalter, A. K., 1953: A stability index for thunderstorm forecasting. Bull. Amer. Meteor. Soc., 34 , 250252.

  • Simmons, A. J., Burridge D. M. , Jarraud M. , Girard C. , and Wergen W. , 1989: The ECMWF medium-range prediction models. Development of the numerical formulations and the impact of increased resolution. Meteor. Atmos. Phys., 40 , 2860.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Teixeira, J., 1999: The impact of increased boundary layer vertical resolution on the ECMWF forecast system. ECMWF Tech. Memo. 268, Reading, United Kingdom, 55 pp.

  • Thompson, R. L., Edwards R. , Hart J. A. , Elmore K. L. , and Markowsky P. , 2003: Close proximity soundings within supercell environments obtained from the Rapid Update Cycle. Wea. Forecasting, 18 , 12431261.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Undén, P., Kelly G. , Le Meur D. , and Isaksen L. , 1997: Observing system experiments with the 3D-Var assimilation system. ECMWF Tech. Memo. 244, Reading, United Kingdom, 32 pp.

  • Willmott, C. J., and Matsuura K. , 2005: Advantages of the mean absolute error (MAE) over the root mean squared error (RMSE) in assessing average model performance. Climate Res., 30 , 7982.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • World Meteorological Organization, 1988: Global Aspects. Vol. 1, Manual on Codes, WMO 306, World Meteorological Organization. [Available online at http://www.wmo.int/pages/prog/www/WMOCodes/ManualCodesGuides.html.].

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 187 54 7
PDF Downloads 122 32 6