Validation of Automatic Cb Observations for METAR Messages without Ground Truth

Otto Hyvärinen Finnish Meteorological Institute, Helsinki, Finland

Search for other papers by Otto Hyvärinen in
Current site
Google Scholar
PubMed
Close
,
Elena Saltikoff Finnish Meteorological Institute, Helsinki, Finland

Search for other papers by Elena Saltikoff in
Current site
Google Scholar
PubMed
Close
, and
Harri Hohti Finnish Meteorological Institute, Helsinki, Finland

Search for other papers by Harri Hohti in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

In aviation meteorology, METAR messages are used to disseminate the existence of cumulonimbus (Cb) clouds. METAR messages are traditionally constructed manually from human observations, but there is a growing trend toward automation of this process. At the Finnish Meteorological Institute (FMI), METAR messages incorporate an operational automatic detection of Cb based solely on weather radar data, when manual observations are not available. However, the verification of this automatic Cb detection is challenging, as good ground truth data are not often available; even human observations are not perfect as Cb clouds can be obscured by other clouds, for example. Therefore, statistical estimation of the relevant verification measures from imperfect observations using latent class analysis (LCA) was explored. In addition to radar-based products and human observations, the convective rainfall rate from EUMETSAT’s Nowcasting Satellite Application Facility and lightning products from the Finnish lightning network were used for determining the existence of Cb clouds. Results suggest that LCA gives reasonable estimates of verification measures and, based on these estimates, the Cb detection system at FMI gives results comparable to human observations.

Corresponding author address: Otto Hyvärinen, Finnish Meteorological Institute, Erik Palménin aukio 1, P.O. Box 503, FI-00101 Helsinki, Finland. E-mail: otto.hyvarinen@fmi.fi

Abstract

In aviation meteorology, METAR messages are used to disseminate the existence of cumulonimbus (Cb) clouds. METAR messages are traditionally constructed manually from human observations, but there is a growing trend toward automation of this process. At the Finnish Meteorological Institute (FMI), METAR messages incorporate an operational automatic detection of Cb based solely on weather radar data, when manual observations are not available. However, the verification of this automatic Cb detection is challenging, as good ground truth data are not often available; even human observations are not perfect as Cb clouds can be obscured by other clouds, for example. Therefore, statistical estimation of the relevant verification measures from imperfect observations using latent class analysis (LCA) was explored. In addition to radar-based products and human observations, the convective rainfall rate from EUMETSAT’s Nowcasting Satellite Application Facility and lightning products from the Finnish lightning network were used for determining the existence of Cb clouds. Results suggest that LCA gives reasonable estimates of verification measures and, based on these estimates, the Cb detection system at FMI gives results comparable to human observations.

Corresponding author address: Otto Hyvärinen, Finnish Meteorological Institute, Erik Palménin aukio 1, P.O. Box 503, FI-00101 Helsinki, Finland. E-mail: otto.hyvarinen@fmi.fi
Save
  • Branscum, A. J., I. A. Gardner, and W. O. Johnson, 2005: Estimation of diagnostic-test sensitivity and specificity through Bayesian modeling. Prev. Vet. Med., 68 (2–4), 145163, doi:10.1016/j.prevetmed.2004.12.005.

    • Search Google Scholar
    • Export Citation
  • Dempster, A. P., N. M. Laird, and D. B. Rubin, 1977: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc., 39B, 1–39.

  • de Valk, P., and R. van Westhrenen, 2013: Automated Cb/Tcu METAR based on radar and satellite data. Proc. 2013 EUMETSAT Meteorological Satellite Conf., Vienna, Austria, EUMETSAT. [Available online at http://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=PDF_CONF_P_S3_03_DEVALK_V&RevisionSelectionMethod=LatestReleased&Rendition=Web.]

  • Efron, B., and R. Tibshirani, 1994: An Introduction to the Bootstrap.Chapman and Hall/CRC, 456 pp.

  • Georgiadis, M. P., W. O. Johnson, I. A. Gardner, and R. Singh, 2003: Correlation-adjusted estimation of sensitivity and specificity of two diagnostic tests. J. Roy. Stat. Soc., 52C, 6376, doi:10.1111/1467-9876.00389.

    • Search Google Scholar
    • Export Citation
  • Hagenaars, J. A., and A. L. McCutcheon, 2002: Applied Latent Class Analysis.Cambridge University Press, 454 pp.

  • Henken, C. C., M. J. Schmeits, H. Deneke, and R. A. Roebeling, 2011: Using MSG-SEVIRI cloud physical properties and weather radar observations for the detection of Cb/TCu clouds. J. Appl. Meteor. Climatol., 50, 15871600, doi:10.1175/2011JAMC2601.1.

    • Search Google Scholar
    • Export Citation
  • Hui, S. L., and S. D. Walter, 1980: Estimating the error rates of diagnostic tests. Biometrics, 36, 167171, doi:10.2307/2530508.

  • ICAO, 2013: Meteorological service for international air navigation. Annex 3, Convention on International Civil Aviation, ICAO, 208 pp. [Available online at http://www.wmo.int/pages/prog/www/ISS/Meetings/CT-MTDCF-ET-DRC_Geneva2008/Annex3_16ed.pdf.]

  • Jolliffe, I. T., and D. B. Stephenson, 2012: Forecast Verification: A Practitioner’s Guide in Atmospheric Science.2nd ed. Wiley, 292 pp.

  • Leroy, M., 2006: Status of the automatic observation on aerodrome and ongoing improvements in France. Proc. TECO-2006WMO Tech. Conf. on Meteorological and Environmental Instruments and Methods of Observation, Geneva, Switzerland, World Meteorological Organization, 8 pp. [Available online at http://www.wmo.int/pages/prog/www/IMOP/publications/IOM-94-TECO2006/1(3)_Leroy_France.pdf.]

  • Linzer, D., and J. Lewis, 2011: poLCA: An R package for polytomous variable latent class analysis. J. Stat. Software, 42 (10), 1–29. [Available online at http://www.jstatsoft.org/v42/i10/paper.]

    • Search Google Scholar
    • Export Citation
  • Magaldi, A. V., J. Bech, and J. Lorente, 2009: A multisource scheme based on NWP and MSG data to correct non-precipitating weather radar echoes. Meteor. Atmos. Phys., 105 (3–4), 121132, doi:10.1007/s00703-009-0043-6.

    • Search Google Scholar
    • Export Citation
  • Mäkelä, A., 2011: Lightning observations in Finland, 2011. Finnish Meteorological Institute Rep. 2011:7, 50 pp. [Available online at https://helda.helsinki.fi/bitstream/handle/10138/28496/2011nro7.pdf.]

  • Marcos, C., and A. Rodríguez, 2013: Validation report for “convective rainfall rate” (CRR-PGE05 v4.0). SAF/NWC/CDOP2/INM/SCI/VR/12, 21 pp. [Available online at http://www.nwcsaf.org/scidocs/Documentation/SAF-NWC-CDOP2-INM-SCI-VR-12_v1.0.pdf.]

  • OFCM, 2005: Surface weather observations and reports (FCM-H1-2005). Federal Meteorological Handbook 1, 104 pp. [Available online at http://www.ofcm.gov/fmh-1/pdf/FMH1.pdf.]

  • Patton, A., D. N. Politis, and H. White, 2009: Correction to “Automatic block-length selection for the dependent bootstrap” by D. Politis and H. White. Econom. Rev., 28, 372–375, doi:10.1080/07474930802459016.

    • Search Google Scholar
    • Export Citation
  • Pepe, M. S., 2003: The Statistical Evaluation of Medical Tests for Classification and Prediction.Oxford University Press, 302 pp.

  • Rodríguez, A., and C. Marcos, 2013: Algorithm theoretical basis document for “convective rainfall rate” (CRR-PGE05 v4.0). SAF/NWC/CDOP2/INM/SCI/ATBD/05, 36 pp. [Available online at http://www.nwcsaf.org/scidocs/Documentation/SAF-NWC-CDOP2-INM-SCI-ATBD-05_v4.0.pdf.]

  • Rutjes, A., J. Reitsma, A. Coomarasamy, K. Khan, and P. Bossuyt, 2007: Evaluation of diagnostic tests when there is no gold standard: A review of methods. Health Technol. Assess., 11 (50), doi:10.3310/hta11500.

    • Search Google Scholar
    • Export Citation
  • Saltikoff, E., A. Huuskonen, H. Hohti, J. Koistinen, and H. Järvinen, 2010: Quality assurance in the FMI Doppler weather radar network. Boreal Environ. Res, 15, 579594. [Available online at http://www.borenv.net/BER/pdfs/ber15/ber15-579.pdf.]

    • Search Google Scholar
    • Export Citation
  • Skrondal, A., and S. Rabe-Hesketh, 2004: Generalized Latent Variable Models: Multilevel, Longitudinal, and Structural Equation Models.Chapman and Hall/CRC, 512 pp.

  • Tuomi, T. J., and A. Mäkelä, 2008: Thunderstorm climate of Finland 1998–2007. Geophysica, 44 (1–2), 6780. [Available online at http://www.geophysica.fi/pdf/geophysica_2008_44_1-2_067_tuomi.pdf.]

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences.3rd ed. Elsevier, 676 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 415 152 4
PDF Downloads 339 98 2