Editorial Type:
Article
Article Type:
Research Article

mCLOUD: A Multiview Visual Feature Extraction Mechanism for Ground-Based Cloud Image Categorization

Yang Xiao National Key Laboratory of Science and Technology on Multi-Spectral Information Processing, School of Automation, Huazhong University of Science and Technology, Wuhan, China

Search for other papers by Yang Xiao in
Current site
Google Scholar
PubMed Close
,
Zhiguo Cao National Key Laboratory of Science and Technology on Multi-Spectral Information Processing, School of Automation, Huazhong University of Science and Technology, Wuhan, China

Search for other papers by Zhiguo Cao in
Current site
Google Scholar
PubMed Close
,
Wen Zhuo School of Electrical Engineering, Southwest Jiaotong University, Emeishan, China

Search for other papers by Wen Zhuo in
Current site
Google Scholar
PubMed Close
,
Liang Ye National Key Laboratory of Science and Technology on Multi-Spectral Information Processing, School of Automation, Huazhong University of Science and Technology, Wuhan, China

Search for other papers by Liang Ye in
Current site
Google Scholar
PubMed Close
, and
Lei Zhu National Key Laboratory of Science and Technology on Multi-Spectral Information Processing, School of Automation, Huazhong University of Science and Technology, Wuhan, China

Search for other papers by Lei Zhu in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Apr 2016
DOI:
https://doi.org/10.1175/JTECH-D-15-0015.1
Page(s):
789–801
Restricted access

Abstract

In this paper, a novel Multiview CLOUD (mCLOUD) visual feature extraction mechanism is proposed for the task of categorizing clouds based on ground-based images. To completely characterize the different types of clouds, mCLOUD first extracts the raw visual descriptors from the views of texture, structure, and color simultaneously, in a densely sampled way—specifically, the scale invariant feature transform (SIFT), the census transform histogram (CENTRIST), and the statistical color features are extracted, respectively. To obtain a more descriptive cloud representation, the feature encoding of the raw descriptors is realized by using the Fisher vector. This is followed by the feature aggregation procedure. A linear support vector machine (SVM) is employed as the classifier to yield the final cloud image categorization result. The experiments on a challenging cloud dataset termed the six-class Huazhong University of Science and Technology (HUST) cloud demonstrate that mCLOUD consistently outperforms the state-of-the-art cloud classification approaches by large margins (at least 6.9%) under all the different experimental settings. It has also been verified that, compared to the single view, the multiview cloud representation generally enhances the performance.

Corresponding author address: Zhiguo Cao, National Key Laboratory of Science and Technology on Multi-Spectral Information Processing, School of Automation, Huazhong University of Science and Technology, 1037 Luoyu Road, Wuhan 430074, China. E-mail: zgcao@hust.edu.cn

Abstract

In this paper, a novel Multiview CLOUD (mCLOUD) visual feature extraction mechanism is proposed for the task of categorizing clouds based on ground-based images. To completely characterize the different types of clouds, mCLOUD first extracts the raw visual descriptors from the views of texture, structure, and color simultaneously, in a densely sampled way—specifically, the scale invariant feature transform (SIFT), the census transform histogram (CENTRIST), and the statistical color features are extracted, respectively. To obtain a more descriptive cloud representation, the feature encoding of the raw descriptors is realized by using the Fisher vector. This is followed by the feature aggregation procedure. A linear support vector machine (SVM) is employed as the classifier to yield the final cloud image categorization result. The experiments on a challenging cloud dataset termed the six-class Huazhong University of Science and Technology (HUST) cloud demonstrate that mCLOUD consistently outperforms the state-of-the-art cloud classification approaches by large margins (at least 6.9%) under all the different experimental settings. It has also been verified that, compared to the single view, the multiview cloud representation generally enhances the performance.

Corresponding author address: Zhiguo Cao, National Key Laboratory of Science and Technology on Multi-Spectral Information Processing, School of Automation, Huazhong University of Science and Technology, 1037 Luoyu Road, Wuhan 430074, China. E-mail: zgcao@hust.edu.cn
Save
Cover Journal of Atmospheric and Oceanic Technology
Journal of Atmospheric and Oceanic Technology

  • Allmen, M. C., and Kegelmeyer W. P. Jr., 1996: The computation of cloud-base height from paired whole-sky imaging cameras. J. Atmos. Oceanic Technol., 13, 97–113, doi:10.1175/1520-0426(1996)013<0097:TCOCBH>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Buch, K. A, Jr., Sun C.-H. , and Thorne L. , 1995: Cloud classification using whole-sky imager data. Proc. Fifth Atmospheric Radiation Measurement Science Team Meeting, San Diego, CA, ARM, 35–39. [Available online at http://www.arm.gov/publications/proceedings/conf05/extended_abs/buch_ka.pdf.]

  • Calbó, J., and Sabburg J. , 2008: Feature extraction from whole-sky ground-based images for cloud-type recognition. J. Atmos. Oceanic Technol., 25, 3–14, doi:10.1175/2007JTECHA959.1.

    • Search Google Scholar
    • Export Citation

  • Chang, C.-C., and Lin C.-J. , 2011: LIBSVM—A library for support vector machines. ACM Trans. Intell. Syst. Technol., 2, 27, doi:10.1145/1961189.1961199.

    • Search Google Scholar
    • Export Citation

  • CMA, 2003: Clouds. Specification for Ground Meteorological Observation, China Meteorological Press, 7–12.

  • Cortes, C., and Vapnik V. , 1995: Support-vector networks. Mach. Learn., 20, 273–297, doi:10.1023/A:1022627411411.

  • Cover, T., and Hart P. , 1967: Nearest neighbor pattern classification. IEEE Trans. Inf. Theory, 13, 21–27, doi:10.1109/TIT.1967.1053964.

    • Search Google Scholar
    • Export Citation

  • Csurka, G., Dance C. , Fan L. , Willamowski J. , and Bray C. , 2004: Visual categorization with bags of keypoints. Eighth European Conf. on Computer Vision, Prague, Czech Republic, Czech Technical University, 16 pp. [Available online at http://www3.xrce.xerox.com/content/download/16582/118437/file/2004_010.pdf.]

  • Dempster, A. P., Laird N. M. , and Rubin D. B. , 1977: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc., 39B, 1–38.

    • Search Google Scholar
    • Export Citation

  • Heinle, A., Macke A. , and Srivastav A. , 2010: Automatic cloud classification of whole sky images. Atmos. Meas. Tech., 3, 557–567, doi:10.5194/amt-3-557-2010.

    • Search Google Scholar
    • Export Citation

  • Isosalo, A., Turtinen M. , and Pietikäinen M. , 2007: Cloud characterization using local texture information. Proc. Finnish Signal Processing Symp. 2007, Oulu, Finland, Oulu University, 6 pp. [Available online at http://www.ee.oulu.fi/research/imag/finsig07/papers/s6p2.pdf.]

  • Jaakkola, T., and Haussler D. , 1999: Exploiting generative models in discriminative classifiers. Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems 11, M. S. Kearns, S. A. Solla, and D. A. Cohn, Eds., Advances in Neural Information Processing Systems, Vol. 11, MIT Press, 487–493.

  • Kalisch, J., and Macke A. , 2008: Estimation of the total cloud cover with high temporal resolution and parametrization of short-term fluctuations of sea surface insolation. Meteor. Z., 17, 603–611, doi:10.1127/0941-2948/2008/0321.

    • Search Google Scholar
    • Export Citation

  • Kassianov, E., Long C. N. , and Christy J. , 2005: Cloud-base-height estimation from paired ground-based hemispherical observations. J. Appl. Meteor., 44, 1221–1233, doi:10.1175/JAM2277.1.

    • Search Google Scholar
    • Export Citation

  • Li, L.-J., and Fei-Fei L. , 2007: What, where and who? Classifying events by scene and object recognition. ICCV 2007: IEEE 11th International Conference on Computer Vision, 2007, IEEE, 1–8, doi:10.1109/ICCV.2007.4408872.

  • Liu, L., Sun X. , Chen F. , Zhao S. , and Gao T. , 2011: Cloud classification based on structure features of infrared images. J. Atmos. Oceanic Technol., 28, 410–417, doi:10.1175/2010JTECHA1385.1.

    • Search Google Scholar
    • Export Citation

  • Long, C. N., Sabburg J. M. , Calbó J. , and Pagès D. , 2006: Retrieving cloud characteristics from ground-based daytime color all-sky images. J. Atmos. Oceanic Technol., 23, 633–652, doi:10.1175/JTECH1875.1.

    • Search Google Scholar
    • Export Citation

  • Lowe, D. G., 2004: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 60, 91–110, doi:10.1023/B:VISI.0000029664.99615.94.

    • Search Google Scholar
    • Export Citation

  • Nixon, M., and Aguado A. S. , 2008: Feature Extraction and Image Processing. 2nd ed. Academic Press, 405 pp.

  • O’Mahony, M., 1986: Sensory Evaluation of Food: Statistical Methods and Procedures, Food Science and Technology, Vol. 16, CRC Press, 510 pp.

  • Perronnin, F., and Dance C. , 2007: Fisher kernels on visual vocabularies for image categorization. CVPR’07: IEEE Conference on Computer Vision and Pattern Recognition, 2007, IEEE, 1–8, doi:10.1109/CVPR.2007.383266.

  • Perronnin, F., Sánchez J. , and Mensink T. , 2010: Improving the Fisher kernel for large-scale image classification. Computer Vision—ECCV 2010: 11th European Conference on Computer Vision, K. Daniilidis, P. Maragos, and N. Paragios, Eds., Lecture Notes in Computer Science, Vol. 6314, Springer, 143–156.

  • Pfister, G., McKenzie R. , Liley J. , Thomas A. , Forgan B. , and Long C. N. , 2003: Cloud coverage based on all-sky imaging and its impact on surface solar irradiance. J. Appl. Meteor., 42, 1421–1434, doi:10.1175/1520-0450(2003)042<1421:CCBOAI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Sánchez, J., Perronnin F. , Mensink T. , and Verbeek J. , 2013: Image classification with the Fisher vector: Theory and practice. Int. J. Comput. Vision, 105, 222–245, doi:10.1007/s11263-013-0636-x.

    • Search Google Scholar
    • Export Citation

  • Seiz, G., Baltsavias E. P. , and Gruen A. , 2002: Cloud mapping from the ground: Use of photogrammetric methods. Photogramm. Eng. Remote Sens., 68, 941–951.

    • Search Google Scholar
    • Export Citation

  • Singh, M., and Glennen M. , 2005: Automated ground-based cloud recognition. Pattern Anal. Appl., 8, 258–271, doi:10.1007/s10044-005-0007-5.

    • Search Google Scholar
    • Export Citation

  • Vedaldi, A., and Fulkerson B. , 2010: VLFeat: An open and portable library of computer vision algorithms. MM’10: Proceedings of the 18th ACM International Conference on Multimedia, ACM, 1469–1472, doi:10.1145/1873951.1874249.

  • Vinyals, O., Jia Y. , Deng L. , and Darrell T. , 2012: Learning with recursive perceptual representations. Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012, F. Pereira et al., Eds., Advances in Neural Information Processing Systems, Vol. 25, 2825–2833.

  • Wang, J., Yang J. , Yu K. , Lv F. , Huang T. , and Gong Y. , 2010: Locality-constrained linear coding for image classification. 2010 Conference on Computer Vision and Pattern Recognition, IEEE, 3360–3367, doi:10.1109/CVPR.2010.5540018.

  • Weng, C.-C., Chen H. , and Fuh C.-S. , 2005: A novel automatic white balance method for digital still cameras. Proceedings of IEEE International Symposium on Circuits and Systems, 2005, IEEE, 3801–3804, doi:10.1109/ISCAS.2005.1465458.

  • Wu, J., and Rehg J. M. , 2011: CENTRIST: A visual descriptor for scene categorization. IEEE Trans. Pattern Anal. Mach. Intell., 33, 1489–1501, doi:10.1109/TPAMI.2010.224.

    • Search Google Scholar
    • Export Citation

  • Xiao, Y., Wu J. , and Yuan J. , 2014: mCENTRIST: A multi-channel feature generation mechanism for scene categorization. IEEE Trans. Image Process., 23, 823–836, doi:10.1109/TIP.2013.2295756.

    • Search Google Scholar
    • Export Citation

  • Zabih, R., and Woodfill J. , 1994: Non-parametric local transforms for computing visual correpondence. Computer Vision—ECCV’94: Third European Conference on Computer Vision, J.-O. Eklundh, Ed., Lecture Notes in Computer Science, Vol. 801, Springer, 151–158.

  • Zhuo, W., Cao Z. , and Xiao Y. , 2014: Cloud classification of ground-based images using texture–structure features. J. Atmos. Oceanic Technol., 31, 79–92, doi:10.1175/JTECH-D-13-00048.1.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1363 618 95
PDF Downloads 511 92 6