Potential Value of Expert Elicitation for Determining Differential Credibility of Regional Climate Change Simulations: An Exercise with the NARCCAP co-PIs for the Southwest Monsoon Region of North America

Linda O. Mearns National Center for Atmospheric Research, Boulder, Colorado

Search for other papers by Linda O. Mearns in
Current site
Google Scholar
PubMed
Close
,
Melissa S. Bukovsky National Center for Atmospheric Research, Boulder, Colorado

Search for other papers by Melissa S. Bukovsky in
Current site
Google Scholar
PubMed
Close
, and
Vanessa J. Schweizer National Center for Atmospheric Research, Boulder, Colorado

Search for other papers by Vanessa J. Schweizer in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

In this brief article, we report the initial results of an expert elicitation with the co-PIs (regional climate modelers) of the North American Regional Climate Change Assessment Program regarding their evaluation of the relative quality of regional climate model simulations focusing on the subregion dominated by the North American monsoon (NAM). We assumed that an expert elicitation framework might reveal interesting beliefs and understanding that would be different from what would be obtained from calculating quantitative metrics associated with model quality.

The simulations considered were of six regional climate models (RCMs) that used NCEP Reanalysis 2 as boundary conditions for the years 1980–2004. The domain covers most of North America and adjacent oceans. The seven participating regional modelers were asked to complete surveys on their background beliefs about model credibility and their judgments regarding the quality of the six models based on a series of plots of variables related to the NAM (e.g., temperature, winds, humidity, moisture flux, precipitation). The specific RCMs were not identified.

We also compared the results of the expert elicitation with those obtained from using a series of metrics developed to evaluate a European collection of climate model simulations. The results proved to be quite different in the two cases.

The results of this exercise proved very enlightening regarding regional modelers’ perceptions of model quality and their beliefs about how this information should or should not be used. Based on these pilot study results, we believe a more complete study is warranted.

CORRESPONDING AUTHOR E-MAIL: Linda Mearns, lindam@ucar.edu

A supplement to this article is available online (10.1175/BAMS-D-15-00019.2)

Abstract

In this brief article, we report the initial results of an expert elicitation with the co-PIs (regional climate modelers) of the North American Regional Climate Change Assessment Program regarding their evaluation of the relative quality of regional climate model simulations focusing on the subregion dominated by the North American monsoon (NAM). We assumed that an expert elicitation framework might reveal interesting beliefs and understanding that would be different from what would be obtained from calculating quantitative metrics associated with model quality.

The simulations considered were of six regional climate models (RCMs) that used NCEP Reanalysis 2 as boundary conditions for the years 1980–2004. The domain covers most of North America and adjacent oceans. The seven participating regional modelers were asked to complete surveys on their background beliefs about model credibility and their judgments regarding the quality of the six models based on a series of plots of variables related to the NAM (e.g., temperature, winds, humidity, moisture flux, precipitation). The specific RCMs were not identified.

We also compared the results of the expert elicitation with those obtained from using a series of metrics developed to evaluate a European collection of climate model simulations. The results proved to be quite different in the two cases.

The results of this exercise proved very enlightening regarding regional modelers’ perceptions of model quality and their beliefs about how this information should or should not be used. Based on these pilot study results, we believe a more complete study is warranted.

CORRESPONDING AUTHOR E-MAIL: Linda Mearns, lindam@ucar.edu

A supplement to this article is available online (10.1175/BAMS-D-15-00019.2)

Supplementary Materials

    • Supplemental Materials (PDF 10.41 MB)
Save
  • Adams, D. K., and A. C. Comrie, 1997: The North American Monsoon. Bull. Amer. Meteor. Soc., 78, 21972213, doi:10.1175/1520-0477(1997)078<2197:TNAM>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Aspinall, W., 2010: A route to more tractable expert advice. Nature, 463, 294295, doi:10.1038/463294a.

  • Bukovsky, M., D. Gochis, and L. O. Mearns, 2013: Towards establishing NARCCAP regional climate model credibility for the North American Monsoon: Current Simulations. J. Climate, 26, 88028826, doi:10.1175/JCLI-D-12-00538.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bukovsky, M., D., J. Thompson, and L. O. Mearns, 2013: The effect of weighting on the NARCCAP ensemble mean. 25th Conf. on Climate Variability and Change, Austin, TX, P106.

  • Bukovsky, M., D., J. Thompson, and L. O. Mearns, 2016: The effect of weighting on the NARCCAP ensemble mean: Does it make a difference? Climate Res., to be submitted.

    • Search Google Scholar
    • Export Citation
  • Christensen, J. H., E. Kjellström, F. Giorgi, G. Lenderink, and M. Rummukainen, 2010: Weight assessment in regional climate models. Climate Res., 44, 179194, doi:10.3354/cr00916.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Flato, G., and Coauthors, 2013. Evaluation of climate models. Climate Change 2013: The Physical Science Basis, T. F. Stocker et al., Eds., Cambridge University Press, 741–866.

  • Mearns, L. O., and Coauthors, 2012: The North American Regional Climate Change Assessment Program: Overview of phase I results. Bull. Amer. Meteor. Soc., 93, 13371362, doi:10.1175/BAMS-D-11-00223.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mearns, L. O., M. Bukovsky, S. Pryor, and V. Magaña, 2014: Downscaling of climate information. Climate Change in North America, G. Ohring, Ed., Springer, 201–250.

    • Crossref
    • Export Citation
  • Morgan, M. G., 2014: Use (and abuse) of expert elicitation in support of decision making for public policy. Proc. Natl. Acad. Sci. USA, 111, 71767184, doi:10.1073/pnas.1319946111.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 403 93 10
PDF Downloads 243 41 1