Advanced Deep Learning–Based Supervised Classification of Multi-Angle Snowflake Camera Images

C. Key aDepartment of Electrical and Computer Engineering, Colorado State University, Fort Collins, Colorado

Search for other papers by C. Key in
Current site
Google Scholar
PubMed
Close
,
A. Hicks aDepartment of Electrical and Computer Engineering, Colorado State University, Fort Collins, Colorado

Search for other papers by A. Hicks in
Current site
Google Scholar
PubMed
Close
, and
B. M. Notaroš aDepartment of Electrical and Computer Engineering, Colorado State University, Fort Collins, Colorado

Search for other papers by B. M. Notaroš in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

We present improvements over our previous approach to automatic winter hydrometeor classification by means of convolutional neural networks (CNNs), using more data and improved training techniques to achieve higher accuracy on a more complicated dataset than we had previously demonstrated. As an advancement of our previous proof of concept study, this work demonstrates broader usefulness of deep CNNs by using a substantially larger and more diverse dataset, which we make publicly available, from many more snow events. We describe the collection, processing, and sorting of this dataset of over 25 000 high-quality Multi-Angle Snowflake Camera (MASC) image chips split nearly evenly between five geometric classes: aggregate, columnar crystal, planar crystal, graupel, and small particle. Raw images were collected over 32 snowfall events between November 2014 and May 2016 near Greeley, Colorado, and were processed with an automated cropping and normalization algorithm to yield 224 × 224 pixel images containing possible hydrometeors. From the bulk set of over 8 400 000 extracted images, a smaller dataset of 14 793 images was sorted by image quality and recognizability (Q&R) using manual inspection. A presorting network trained on the Q&R dataset was applied to all 8 400 000+ images to automatically collect a subset of 283 351 good snowflake images. Roughly 5000 representative examples were then collected from this subset manually for each of the five geometric classes. With a higher emphasis on in-class variety than our previous work, the final dataset yields trained networks that better capture the imperfect cases and diverse forms that occur within the broad categories studied to achieve an accuracy of 96.2% on a vastly more challenging dataset.

Significance Statement

Classification of precipitation, namely, deciding to which of the several typical classes of winter hydrometeors the observed particles belong, can enrich our understanding of polarimetric radar signatures of snow, as well as ice cloud processes and the resulting precipitation production. The high-resolution photographs of snowflakes collected by the Multi-Angle Snowflake Camera (MASC) are especially suitable for snowflake classification. However, classifying particle types from MASC photographs by visual inspection is not practical given the typical amount of MASC data. We present advanced automatic deep machine learning–based classification of MASC images using convolutional neural networks. This study demonstrates broad usefulness of our approach yielding trained networks that achieve extremely high classification accuracy on a large and diverse dataset from many snow events.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Branislav M. Notaroš, notaros@colostate.edu

Abstract

We present improvements over our previous approach to automatic winter hydrometeor classification by means of convolutional neural networks (CNNs), using more data and improved training techniques to achieve higher accuracy on a more complicated dataset than we had previously demonstrated. As an advancement of our previous proof of concept study, this work demonstrates broader usefulness of deep CNNs by using a substantially larger and more diverse dataset, which we make publicly available, from many more snow events. We describe the collection, processing, and sorting of this dataset of over 25 000 high-quality Multi-Angle Snowflake Camera (MASC) image chips split nearly evenly between five geometric classes: aggregate, columnar crystal, planar crystal, graupel, and small particle. Raw images were collected over 32 snowfall events between November 2014 and May 2016 near Greeley, Colorado, and were processed with an automated cropping and normalization algorithm to yield 224 × 224 pixel images containing possible hydrometeors. From the bulk set of over 8 400 000 extracted images, a smaller dataset of 14 793 images was sorted by image quality and recognizability (Q&R) using manual inspection. A presorting network trained on the Q&R dataset was applied to all 8 400 000+ images to automatically collect a subset of 283 351 good snowflake images. Roughly 5000 representative examples were then collected from this subset manually for each of the five geometric classes. With a higher emphasis on in-class variety than our previous work, the final dataset yields trained networks that better capture the imperfect cases and diverse forms that occur within the broad categories studied to achieve an accuracy of 96.2% on a vastly more challenging dataset.

Significance Statement

Classification of precipitation, namely, deciding to which of the several typical classes of winter hydrometeors the observed particles belong, can enrich our understanding of polarimetric radar signatures of snow, as well as ice cloud processes and the resulting precipitation production. The high-resolution photographs of snowflakes collected by the Multi-Angle Snowflake Camera (MASC) are especially suitable for snowflake classification. However, classifying particle types from MASC photographs by visual inspection is not practical given the typical amount of MASC data. We present advanced automatic deep machine learning–based classification of MASC images using convolutional neural networks. This study demonstrates broad usefulness of our approach yielding trained networks that achieve extremely high classification accuracy on a large and diverse dataset from many snow events.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Branislav M. Notaroš, notaros@colostate.edu
Save
  • Bringi, V. N., P. C. Kennedy, G.-J. Huang, C. Kleinkort, M. Thurai, and B. M. Notaroš, 2017: Dual-polarized radar and surface observations of a winter graupel shower with negative Zdr column. J. Appl. Meteor. Climatol., 56, 455470, https://doi.org/10.1175/JAMC-D-16-0197.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Grazioli, J., D. Tuia, S. Monhart, M. Schneebeli, T. Raupach, and A. Berne, 2014: Hydrometeor classification from two-dimensional video disdrometer data. Atmos. Meas. Tech., 7, 28692882, https://doi.org/10.5194/amt-7-2869-2014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • He, K., X. Zhang, S. Ren, and J. Sun, 2016: Deep residual learning for image recognition. 2016 IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, NV, IEEE, 770–778, https://doi.org/10.1109/CVPR.2016.90.

    • Crossref
    • Export Citation
  • Hicks, A., and B. M. Notaroš, 2019: Method for classification of snowflakes based on images by a Multi-Angle Snowflake Camera using convolutional neural networks. J. Atmos. Oceanic Technol., 36, 22672282, https://doi.org/10.1175/JTECH-D-19-0055.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kennedy, P., M. Thurai, C. Praz, V. N. Bringi, A. Berne, and B. M. Notaroš, 2018: Variations in snow crystal riming and ZDR: A case analysis. J. Appl. Meteor. Climatol., 57, 695707, https://doi.org/10.1175/JAMC-D-17-0068.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Key, C., A. Hicks, and B. Notaroš, 2021: Colorado State University geometric snowflake classification dataset, version 1.0. Zenodo, accessed 5 March 2021, https://doi.org/10.5281/zenodo.4584200.

    • Crossref
    • Export Citation
  • Kleinkort, C., G.-J. Huang, V. N. Bringi, and B. M. Notaroš, 2017: Visual hull method for realistic 3D particle shape reconstruction based on high-resolution photographs of snowflakes in free fall from multiple views. J. Atmos. Oceanic Technol., 34, 679702, https://doi.org/10.1175/JTECH-D-16-0099.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Korolev, A., and B. Sussman, 2000: A technique for habit classification of cloud particles. J. Atmos. Oceanic Technol., 17, 10481057, https://doi.org/10.1175/1520-0426(2000)017<1048:ATFHCO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leinonen, J., and A. Berne, 2020: Unsupervised classification of snowflake images using a general adversarial network and K-medoids classification. Atmos. Meas. Tech., 13, 29492964, https://doi.org/10.5194/amt-13-2949-2020.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Libbrecht, K. G., 2017: Physical dynamics of ice crystal growth. Annu. Rev. Mat. Res., 47, 271295, https://doi.org/10.1146/annurev-matsci-070616-124135.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lindqvist, H., K. Muinonen, T. Nousiainen, J. Um, G. McFarquhar, P. Haapanala, R. Makkonen, and H. Hakkarainen, 2012: Ice-cloud particle habit classification using principal components. J. Geophys. Res., 117, D16206, https://doi.org/10.1029/2012JD017573.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Magono, C., and C. W. Lee, 1966: Meteorological classification of natural snow crystals. J. Fac. Sci. Hokkaido Univ. Ser. 7, 2, 321335.

    • Search Google Scholar
    • Export Citation
  • Nakaya, U., and Y. Sekido, 1936: General classification of snow crystals and their frequency of occurrence. J. Fac. Sci. Hokkaido Univ. Ser. 2, 1, 243264.

    • Search Google Scholar
    • Export Citation
  • Newman, A. J., P. A. Kucera, and L. F. Bliven, 2009: Presenting the Snowflake Video Imager (SVI). J. Atmos. Oceanic Technol., 26, 167179, https://doi.org/10.1175/2008JTECHA1148.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Notaroš, B. M., and Coauthors, 2016: Accurate characterization of winter precipitation using Multi-Angle Snowflake Camera, visual hull, advanced scattering methods and polarimetric radar. Atmosphere, 7, 81111, https://doi.org/10.3390/atmos7060081.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Praz, C., R. Yves-Alain, and A. Berne, 2017: Solid hydrometeor classification and riming degree estimation from pictures collected with a Multi-Angle Snowflake Camera. Atmos. Meas. Tech., 10, 13351357, https://doi.org/10.5194/amt-10-1335-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Russakovsky, O., and Coauthors, 2015: ImageNet Large Scale Visual Recognition Challenge. Int. J. Comput. Vision, 115, 211–252, https://doi.org/10.1007/s11263-015-0816-y.

    • Crossref
    • Export Citation
  • Schönhuber, M., G. Lammer, and W. Randeu, 2008: The 2D video disdrometer. Precipitation: Advances in Measurement, Estimation and Prediction, S. Michaelides, Ed., Springer, 3–31.

    • Crossref
    • Export Citation
  • Simonyan, K., and A. Zisserman, 2015: Very deep convolutional networks for large-scale image recognition. Int. Conf. on Learning Representations, San Diego, CA, ICLR.

  • Straka, J., D. S. Zrnić, and A. V. Ryzhkov, 2000: Bulk hydrometeor classification and quantification using polarimetric radar data: Synthesis of relations. J. Appl. Meteor., 39, 13411372, https://doi.org/10.1175/1520-0450(2000)039<1341:BHCAQU>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Szegedy, C., and Coauthors, 2015: Going deeper with convolutions. 2015 IEEE Conf. on Computer Vision and Pattern Recognition, Boston, MA, IEEE, https://doi.org/10.1109/CVPR.2015.7298594.

    • Crossref
    • Export Citation
  • Vazquez-Martin, S., T. Kuhn, and S. Eliasson, 2020: Shape dependence of falling snow crystals’ microphysical properties using an updated shape classification. Appl. Sci., 10, 1163, https://doi.org/10.3390/app10031163.

    • Crossref
    • Export Citation
  • Zeiler, M. D., and R. Fergus, 2014: Visualizing and understanding convolutional neural networks. European Conf. on Computer Vision, Zurich, Switzerland, ECCV, 818–833, https://doi.org/10.1007/978-3-319-10590-1_53.

    • Crossref
    • Export Citation
  • Zhang, G., S. Luchs, A. Ryzhkov, M. Xue, L. Ryzhkova, and Q. Cao, 2011: Winter precipitation microphysics characterized by polarimetric radar and video disdrometer observations in Central Oklahoma. J. Appl. Meteor. Climatol., 50, 15581570, https://doi.org/10.1175/2011JAMC2343.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 279 0 0
Full Text Views 430 175 11
PDF Downloads 437 154 14