A Combination of Cluster Analysis and Kappa Statistic for the Evaluation of Climate Model Results

Martin Kücken Potsdam Institute for Climate Impact Research, Potsdam, Germany

Search for other papers by Martin Kücken in
Current site
Google Scholar
PubMed
Close
,
Friedrich-Wilhelm Gerstengarbe Potsdam Institute for Climate Impact Research, Potsdam, Germany

Search for other papers by Friedrich-Wilhelm Gerstengarbe in
Current site
Google Scholar
PubMed
Close
, and
Boris Orlowsky Swiss Federal Institute of Technology, Zurich Institute for Atmospheric and Climate Science, Zurich, Switzerland

Search for other papers by Boris Orlowsky in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

The authors present a combination of different statistical methods for the validation of climate simulation data with respect to observational data of the same spatial and temporal coverage. It is assumed that simulated data and observed data are both given as time series at locations such as grid cells or station locations. The aim of this approach is to quantify the agreement between the two spatial structures of observed and simulated data. These spatial structures consist of the spatial distributions of clusters (obtained from a cluster analysis) that contain climatologically similar locations. If the spatial distribution of clusters were identical for the observed and the simulated data, the simulation would describe the spatial structure of the observations perfectly. Differences from this ideal situation can be objectively quantified using the κ statistic. If the simulation data have shortcomings, the different κ variants can be used to diagnose where these are located. The method is demonstrated using simulation data from the Statistical Regional Model (STAR) for Germany. The combination of cluster analysis and κ statistic proves to be an excellent tool for quantifying the spatial correctness of climate models that can be extended to multimodel comparisons. It can thereby serve as a standard measure for climate model evaluation.

Corresponding author address: Martin Kücken, Potsdam Institute for Climate Impact Research, Telegraphenberg A 31, Potsdam 14473, Germany. Email: kuecken@pik-potsdam.de

Abstract

The authors present a combination of different statistical methods for the validation of climate simulation data with respect to observational data of the same spatial and temporal coverage. It is assumed that simulated data and observed data are both given as time series at locations such as grid cells or station locations. The aim of this approach is to quantify the agreement between the two spatial structures of observed and simulated data. These spatial structures consist of the spatial distributions of clusters (obtained from a cluster analysis) that contain climatologically similar locations. If the spatial distribution of clusters were identical for the observed and the simulated data, the simulation would describe the spatial structure of the observations perfectly. Differences from this ideal situation can be objectively quantified using the κ statistic. If the simulation data have shortcomings, the different κ variants can be used to diagnose where these are located. The method is demonstrated using simulation data from the Statistical Regional Model (STAR) for Germany. The combination of cluster analysis and κ statistic proves to be an excellent tool for quantifying the spatial correctness of climate models that can be extended to multimodel comparisons. It can thereby serve as a standard measure for climate model evaluation.

Corresponding author address: Martin Kücken, Potsdam Institute for Climate Impact Research, Telegraphenberg A 31, Potsdam 14473, Germany. Email: kuecken@pik-potsdam.de

Save
  • Altman, D. G., 1991: Practical Statistics for Medical Research. Chapman and Hall, 611 pp.

  • Benestad, R., 2004: Tentative probabilistic temperature scenarios for northern Europe. Tellus, 56A , 89101.

  • Bonfils, C., N. de Noblet-Ducoudre, J. Guiot, and P. Bartlein, 2004: Some mechanisms of mid-Holocene climate change in Europe, inferred from comparing PMIP models to data. Climate Dyn., 23 , 7998.

    • Search Google Scholar
    • Export Citation
  • Cohen, J. A., 1960: A coefficient of agreement for nominal scale. Educ. Psychol. Meas., 20 , 3746.

  • Déqué, M., 2004: Uncertainties in PRUDENCE simulations: Global high resolution models. DMI Tech. Rep., 58 pp. [Available online at http://prudence.dmi.dk/].

    • Search Google Scholar
    • Export Citation
  • Gerstengarbe, F-W., and P. C. Werner, 2003: Klimaänderungen zwischen 1901 und 2000 (Climate changes between 1901 and 2000). Nationalatlas Bundesrepublik Deutschland, J. Deiters, P. Gräf, and G. Löffler, Eds., Spektrum Akademischer Verlag, 58–59.

    • Search Google Scholar
    • Export Citation
  • Gerstengarbe, F-W., M. Kücken, and P. C. Werner, 2005: A new validation scheme for the evaluation of multiparameter fields. Tellus, 57A , 3542.

    • Search Google Scholar
    • Export Citation
  • Giorgi, F., X. Bi, and J. Pal, 2004: Mean, interannual variability and trends in a regional climate change experiment over Europe. I. Present-day climate (1961–1990). Climate Dyn., 22 , 733756.

    • Search Google Scholar
    • Export Citation
  • Goodison, B. E., P. Y. T. Louie, and D. Yang, 1998: WMO solid precipitation measurement intercomparison: Final report. Instruments and Observing Methods Rep. 67, WMO, 211 pp.

    • Search Google Scholar
    • Export Citation
  • Hagen, A., 2002: Comparison of maps containing nominal data. Research Institute for Knowledge Systems Tech. Rep., 46 pp.

  • Hagen-Zanker, A., 2008: Measuring the performance of geosimulation models by map comparison. University of Maastricht Tech. Rep., 182 pp.

    • Search Google Scholar
    • Export Citation
  • Hattermann, F., F. Wattenbach, V. Krysanova, and F. Wechsung, 2005: Runoff simulations on the macroscale with the ecohydrological model SWIM in the Elbe catchment—Validation and uncertainty analysis. Hydrol. Process., 19 , 693714.

    • Search Google Scholar
    • Export Citation
  • Hattermann, F., J. Post, V. Krysanova, T. Conradt, and F. Wechsung, 2008: Assessment of water availability in a central European river basin (Elbe) under climate change. Adv. Climate Change Res., 4 , (Suppl.). 4250.

    • Search Google Scholar
    • Export Citation
  • Kotlarski, S., A. Block, U. Böhm, D. Jacob, K. Keuler, R. Knoche, D. Rechid, and A. Walter, 2005: Regional climate model simulations as input for hydrological applications: Evaluation of uncertainties. Adv. Geosci., 5 , 119125.

    • Search Google Scholar
    • Export Citation
  • Kücken, M., F-W. Gerstengarbe, and P. C. Werner, 2002: Cluster analysis results of regional climate model simulations in the PIDCAP period. Boreal Environ. Res., 7 , 219223.

    • Search Google Scholar
    • Export Citation
  • Monserud, R., and R. Leemans, 1992: The comparison of global vegetation maps. Ecol. Modell., 62 , 275293.

  • Monserud, R., N. Tchebakova, and R. Leemans, 1993: Global vegetation change predicted by the modified Budyko model. Climatic Change, 25 , 5983.

    • Search Google Scholar
    • Export Citation
  • NAG, 2002: NAG C Library Manual (Mark 7). Numerical Algorithms Group, 28 pp. [Available online at http://www.nag.com/numeric/CL/manual/html/CLlibrarymanual.asp].

    • Search Google Scholar
    • Export Citation
  • Orlowsky, B., F-W. Gerstengarbe, and P. C. Werner, 2008: A resampling scheme for regional climate simulations and its performance compared to a dynamical RCM. Theor. Appl. Climatol., 92 , 209223.

    • Search Google Scholar
    • Export Citation
  • Österle, H., P. C. Werner, and F-W. Gerstengarbe, 2006: Ein neuer meteorologischer Datensatz für Deutschland, 1951–2003 (A new meteorological data record for Germany, 1951–2003). Extended Abstracts, Seventh German Climatic Conf.—Climatic Trends: Past and Future, Munich, Germany, Meteorologisches Institut der Universität München, 3 pp. [Available online at http://www.meteo.physik.uni-muenchen.de/dkt/abstracts/klimaparameter/oesterle_p.pdf].

    • Search Google Scholar
    • Export Citation
  • Pontius, R. G., 2000: Quantification error versus location error in comparison of categorical maps. Photogramm. Eng. Remote Sens., 66 , 10111016.

    • Search Google Scholar
    • Export Citation
  • Sachs, L., 1992: Angewandte Statistik: Anwendung Statistischer Methoden (Applied Statistics: Application of Statistical Methods). Springer-Verlag, 846 pp.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 338 129 21
PDF Downloads 227 23 1