Application of Cluster Analysis to Climate Model Performance Metrics

Satoru Yokoi Atmosphere and Ocean Research Institute, The University of Tokyo, Kashiwa, Japan

Search for other papers by Satoru Yokoi in
Current site
Google Scholar
PubMed
Close
,
Yukari N. Takayabu Atmosphere and Ocean Research Institute, The University of Tokyo, Kashiwa, Japan

Search for other papers by Yukari N. Takayabu in
Current site
Google Scholar
PubMed
Close
,
Kazuaki Nishii Graduate School of Science, The University of Tokyo, Tokyo, Japan

Search for other papers by Kazuaki Nishii in
Current site
Google Scholar
PubMed
Close
,
Hisashi Nakamura Graduate School of Science, The University of Tokyo, Tokyo, Japan
Research Institute for Global Change, Japan Agency for Marine-Earth Science and Technology, Yokohama, Japan

Search for other papers by Hisashi Nakamura in
Current site
Google Scholar
PubMed
Close
,
Hirokazu Endo Meteorological Research Institute, Tsukuba, Japan

Search for other papers by Hirokazu Endo in
Current site
Google Scholar
PubMed
Close
,
Hiroki Ichikawa Graduate School of Environmental Studies, Nagoya University, Nagoya, Japan

Search for other papers by Hiroki Ichikawa in
Current site
Google Scholar
PubMed
Close
,
Tomoshige Inoue Graduate School of Life and Environmental Sciences, University of Tsukuba, Tsukuba, Japan

Search for other papers by Tomoshige Inoue in
Current site
Google Scholar
PubMed
Close
,
Masahide Kimoto Atmosphere and Ocean Research Institute, The University of Tokyo, Kashiwa, Japan

Search for other papers by Masahide Kimoto in
Current site
Google Scholar
PubMed
Close
,
Yu Kosaka Graduate School of Science, The University of Tokyo, Tokyo, Japan

Search for other papers by Yu Kosaka in
Current site
Google Scholar
PubMed
Close
,
Takafumi Miyasaka Graduate School of Science, The University of Tokyo, Tokyo, Japan

Search for other papers by Takafumi Miyasaka in
Current site
Google Scholar
PubMed
Close
,
Kazuhiro Oshima Faculty of Environmental Earth Science, Hokkaido University, Sapporo, Japan

Search for other papers by Kazuhiro Oshima in
Current site
Google Scholar
PubMed
Close
,
Naoki Sato Tokyo Gakugei University, Tokyo, Japan
Research Institute for Global Change, Japan Agency for Marine-Earth Science and Technology, Yokohama, Japan

Search for other papers by Naoki Sato in
Current site
Google Scholar
PubMed
Close
,
Yoko Tsushima Research Institute for Global Change, Japan Agency for Marine-Earth Science and Technology, Yokohama, Japan

Search for other papers by Yoko Tsushima in
Current site
Google Scholar
PubMed
Close
, and
Masahiro Watanabe Atmosphere and Ocean Research Institute, The University of Tokyo, Kashiwa, Japan

Search for other papers by Masahiro Watanabe in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

The overall performance of general circulation models is often investigated on the basis of the synthesis of a number of scalar performance metrics of individual models that measure the reproducibility of diverse aspects of the climate. Because of physical and dynamic constraints governing the climate, a model’s performance in simulating a certain aspect of the climate is sometimes related closely to that in simulating another aspect, which results in significant intermodel correlation between performance metrics. Numerous metrics and intermodel correlations may cause a problem in understanding the evaluation and synthesizing the metrics. One possible way to alleviate this problem is to group the correlated metrics beforehand. This study attempts to use simple cluster analysis to group 43 performance metrics. Two clustering methods, the K-means and the Ward methods, yield considerably similar clustering results, and several aspects of the results are found to be physically and dynamically reasonable. Furthermore, the intermodel correlation between the cluster averages is considerably lower than that between the metrics. These results suggest that the cluster analysis is helpful in obtaining the appropriate grouping. Applications of the clustering results are also discussed.

Corresponding author address: Satoru Yokoi, Atmosphere and Ocean Research Institute, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8568, Japan. E-mail: yokoi@aori.u-tokyo.ac.jp

Abstract

The overall performance of general circulation models is often investigated on the basis of the synthesis of a number of scalar performance metrics of individual models that measure the reproducibility of diverse aspects of the climate. Because of physical and dynamic constraints governing the climate, a model’s performance in simulating a certain aspect of the climate is sometimes related closely to that in simulating another aspect, which results in significant intermodel correlation between performance metrics. Numerous metrics and intermodel correlations may cause a problem in understanding the evaluation and synthesizing the metrics. One possible way to alleviate this problem is to group the correlated metrics beforehand. This study attempts to use simple cluster analysis to group 43 performance metrics. Two clustering methods, the K-means and the Ward methods, yield considerably similar clustering results, and several aspects of the results are found to be physically and dynamically reasonable. Furthermore, the intermodel correlation between the cluster averages is considerably lower than that between the metrics. These results suggest that the cluster analysis is helpful in obtaining the appropriate grouping. Applications of the clustering results are also discussed.

Corresponding author address: Satoru Yokoi, Atmosphere and Ocean Research Institute, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8568, Japan. E-mail: yokoi@aori.u-tokyo.ac.jp
Save
  • Adler, R. F., and Coauthors, 2003: The version-2 Global Precipitation Climatology Project (GPCP) monthly precipitation analysis (1979–present). J. Hydrometeor., 4, 11471167.

    • Search Google Scholar
    • Export Citation
  • Forgy, E. W., 1965: Cluster analysis of multivariate data: Efficiency vs. interpretability of classifications. Biometrics, 21, 768769.

    • Search Google Scholar
    • Export Citation
  • Fovell, R. G., and M.-Y. C. Fovell, 1993: Climate zones of the conterminous United States defined using cluster analysis. J. Climate, 6, 21032135.

    • Search Google Scholar
    • Export Citation
  • Gleckler, P. J., K. E. Taylor, and C. Doutriaux, 2008: Performance metrics for climate models. J. Geophys. Res., 113, D06104, doi:10.1029/2007JD008972.

    • Search Google Scholar
    • Export Citation
  • Iizumi, T., M. Nishimori, and M. Yokozawa, 2010: Diagnostics of climate model biases in summer temperature and warm-season insolation for the simulation of regional paddy rice yield in Japan. J. Appl. Meteor. Climatol., 49, 574591.

    • Search Google Scholar
    • Export Citation
  • Josey, S. A., E. C. Kent, and P. K. Taylor, 1999: New insights into the ocean heat budget closure problem from analysis of the SOC air–sea flux climatology. J. Climate, 12, 28562880.

    • Search Google Scholar
    • Export Citation
  • K-1 Model Developers, 2004: K-1 coupled model (MIROC) description. K-1 Tech. Rep., H. Hasumi and S. Emori, Eds., 34 pp. [Available online at http://www.ccsr.u-tokyo.ac.jp/~agcmadm/.]

    • Search Google Scholar
    • Export Citation
  • Kawase, H., T. Yoshikane, M. Hara, F. Kimura, T. Yasunari, B. Ailikun, H. Ueda, and T. Inoue, 2009: Intermodel variability of future changes in the Baiu rainband estimated by the pseudo global warming downscaling method. J. Geophys. Res., 114, D24110, doi:10.1029/2009JD011803.

    • Search Google Scholar
    • Export Citation
  • Knutti, R., R. Furrer, C. Tebaldi, J. Cermak, and G. A. Meehl, 2010: Challenges in combining projections from multiple climate models. J. Climate, 23, 27392758.

    • Search Google Scholar
    • Export Citation
  • Meehl, G. A., C. Covey, T. Delworth, M. Latif, B. McAvaney, J. F. B. Mitchell, R. J. Stoufeer, and K. E. Taylor, 2007a: The WCRP CMIP3 multimodel dataset. Bull. Amer. Meteor. Soc., 88, 13831394.

    • Search Google Scholar
    • Export Citation
  • Meehl, G. A., and Coauthors, 2007b: Global climate projections. Climate Change 2007: The Physical Science Basis, S. Solomon et al., Eds., Cambridge University Press, 747–846.

    • Search Google Scholar
    • Export Citation
  • Murphy, J. M., D. M. H. Sexton, D. N. Barnett, G. S. Jones, M. J. Webb, M. Collins, and D. A. Stainforth, 2004: Quantification of modeling uncertainties in a large ensemble of climate change simulations. Nature, 430, 768772.

    • Search Google Scholar
    • Export Citation
  • Pincus, R., C. P. Batstone, R. J. P. Hofmann, K. E. Taylor, and P. J. Gleckler, 2008: Evaluating the present-day simulation of clouds, precipitation, and radiation in climate models. J. Geophys. Res., 113, D14209, doi:10.1029/2007JD009334.

    • Search Google Scholar
    • Export Citation
  • Rayner, N. A., P. Brohan, D. E. Parker, C. K. Folland, J. J. Kennedy, M. Vanicek, T. Ansell, and S. F. B. Tett, 2006: Improved analyses of changes and uncertainties in sea surface temperature measured in situ since the mid-nineteenth century: The HadSST2 dataset. J. Climate, 19, 446469.

    • Search Google Scholar
    • Export Citation
  • Reichler, T., and J. Kim, 2008: How well do coupled models simulate today’s climate? Bull. Amer. Meteor. Soc., 89, 303311.

  • Rossow, W. B., and R. A. Schiffer, 1999: Advances in understanding clouds from ISCCP. Bull. Amer. Meteor. Soc., 80, 22612287.

  • Santer, B. D., and Coauthors, 2009: Incorporating model quality information in climate change detection and attribution studies. Proc. Natl. Acad. Sci. USA, 106, 14 77814 783.

    • Search Google Scholar
    • Export Citation
  • Uppala, S. M., and Coauthors, 2005: The ERA-40 Re-Analysis. Quart. J. Roy. Meteor. Soc., 131, 29613012.

  • Ward, J. H. Jr., 1963: Hierarchical grouping to optimize an objective function. J. Amer. Stat. Assoc., 58, 236244.

  • Watanabe, M., and Coauthors, 2010: Improved climate simulation by MIROC5: Mean states, variability, and climate sensitivity. J. Climate, 23, 63126335.

    • Search Google Scholar
    • Export Citation
  • Wielicki, B. A., B. R. Barkstrom, E. F. Harrison, R. B. Lee III, G. L. Smith , and J. E. Cooper, 1996: Clouds and the Earth’s Radiant Energy System (CERES): An Earth Observing System experiment. Bull. Amer. Meteor. Soc., 77, 853868.

    • Search Google Scholar
    • Export Citation
  • Williams, K. D., and G. Tselioudis, 2007: GCM intercomparison of global cloud regimes: Present-day evaluation and climate change response. Climate Dyn., 29, 231250.

    • Search Google Scholar
    • Export Citation
  • Williams, K. D., and M. J. Webb, 2009: A quantitative performance assessment of cloud regimes in climate models. Climate Dyn., 33, 141157.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 645 247 15
PDF Downloads 487 128 6