Climate Model Evaluation in the Presence of Observational Uncertainty: Precipitation Indices over the Contiguous United States

Peter B. Gibson Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California

Search for other papers by Peter B. Gibson in
Current site
Google Scholar
PubMed
Close
,
Duane E. Waliser Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California

Search for other papers by Duane E. Waliser in
Current site
Google Scholar
PubMed
Close
,
Huikyo Lee Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California

Search for other papers by Huikyo Lee in
Current site
Google Scholar
PubMed
Close
,
Baijun Tian Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California

Search for other papers by Baijun Tian in
Current site
Google Scholar
PubMed
Close
, and
Elias Massoud Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California

Search for other papers by Elias Massoud in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Climate model evaluation is complicated by the presence of observational uncertainty. In this study we analyze daily precipitation indices and compare multiple gridded observational and reanalysis products with regional climate models (RCMs) from the North American component of the Coordinated Regional Climate Downscaling Experiment (NA-CORDEX) multimodel ensemble. In the context of model evaluation, observational product differences across the contiguous United States (CONUS) are also deemed nontrivial for some indices, especially for annual counts of consecutive wet days and for heavy precipitation indices. Multidimensional scaling (MDS) is used to directly include this observational spread into the model evaluation procedure, enabling visualization and interpretation of model differences relative to a “cloud” of observational uncertainty. Applying MDS to the evaluation of NA-CORDEX RCMs reveals situations of added value from dynamical downscaling, situations of degraded performance from dynamical downscaling, and the sensitivity of model performance to model resolution. On precipitation days, higher-resolution RCMs typically simulate higher mean and extreme precipitation rates than their lower-resolution pairs, sometimes improving model fidelity with observations. These results document the model spread and biases in daily precipitation extremes across the full NA-CORDEX model ensemble. The often-large divergence between in situ observations, satellite data, and reanalysis, shown here for CONUS, is especially relevant for data-sparse regions of the globe where satellite and reanalysis products are extensively relied upon. This highlights the need to carefully consider multiple observational products when evaluating climate models.

Supplemental information related to this paper is available at the Journals Online website: https://doi.org/10.1175/JHM-D-18-0230.s1.

© 2019 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Peter B. Gibson, peter.gibson@jpl.nasa.gov

Abstract

Climate model evaluation is complicated by the presence of observational uncertainty. In this study we analyze daily precipitation indices and compare multiple gridded observational and reanalysis products with regional climate models (RCMs) from the North American component of the Coordinated Regional Climate Downscaling Experiment (NA-CORDEX) multimodel ensemble. In the context of model evaluation, observational product differences across the contiguous United States (CONUS) are also deemed nontrivial for some indices, especially for annual counts of consecutive wet days and for heavy precipitation indices. Multidimensional scaling (MDS) is used to directly include this observational spread into the model evaluation procedure, enabling visualization and interpretation of model differences relative to a “cloud” of observational uncertainty. Applying MDS to the evaluation of NA-CORDEX RCMs reveals situations of added value from dynamical downscaling, situations of degraded performance from dynamical downscaling, and the sensitivity of model performance to model resolution. On precipitation days, higher-resolution RCMs typically simulate higher mean and extreme precipitation rates than their lower-resolution pairs, sometimes improving model fidelity with observations. These results document the model spread and biases in daily precipitation extremes across the full NA-CORDEX model ensemble. The often-large divergence between in situ observations, satellite data, and reanalysis, shown here for CONUS, is especially relevant for data-sparse regions of the globe where satellite and reanalysis products are extensively relied upon. This highlights the need to carefully consider multiple observational products when evaluating climate models.

Supplemental information related to this paper is available at the Journals Online website: https://doi.org/10.1175/JHM-D-18-0230.s1.

© 2019 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Peter B. Gibson, peter.gibson@jpl.nasa.gov

Supplementary Materials

    • Supplemental Materials (PDF 4.19 MB)
Save
  • AghaKouchak, A., A. Behrangi, S. Sorooshian, K. Hsu, and E. Amitai, 2011: Evaluation of satellite-retrieved extreme precipitation rates across the central United States. J. Geophys. Res., 116, D02115, https://doi.org/10.1029/2010JD014741.

    • Search Google Scholar
    • Export Citation
  • AghaKouchak, A., A. Mehran, H. Norouzi, and A. Behrangi, 2012: Systematic and random error components in satellite precipitation data sets. Geophys. Res. Lett., 39, L09406, https://doi.org/10.1029/2012GL051592.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Alexander, L. V., and J. M. Arblaster, 2017: Historical and projected trends in temperature and precipitation extremes in Australia in observations and CMIP5. Wea. Climate Extremes, 15, 3456, https://doi.org/10.1016/j.wace.2017.02.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Angélil, O., and Coauthors, 2016: Comparing regional precipitation and temperature extremes in climate model and reanalysis products. Wea. Climate Extremes, 13, 3543, https://doi.org/10.1016/j.wace.2016.07.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ashouri, H., K.-L. Hsu, S. Sorooshian, D. K. Braithwaite, K. R. Knapp, L. D. Cecil, B. R. Nelson, and O. P. Prat, 2015: PERSIANN-CDR: Daily precipitation climate data record from multisatellite observations for hydrological and climate studies. Bull. Amer. Meteor. Soc., 96, 6983, https://doi.org/10.1175/BAMS-D-13-00068.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Avila, F. B., S. Dong, K. P. Menang, J. Rajczak, M. Renom, M. G. Donat, and L. V. Alexander, 2015: Systematic investigation of gridding-related scaling effects on annual statistics of daily temperature and precipitation maxima: A case study for south-east Australia. Wea. Climate Extremes, 9, 616, https://doi.org/10.1016/j.wace.2015.06.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bacmeister, J. T., M. F. Wehner, R. B. Neale, A. Gettelman, C. Hannay, P. H. Lauritzen, J. M. Caron, and J. E. Truesdale, 2014: Exploratory high-resolution climate simulations using the Community Atmosphere Model (CAM). J. Climate, 27, 30733099, https://doi.org/10.1175/JCLI-D-13-00387.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Behrangi, A., K. Andreadis, J. B. Fisher, F. J. Turk, S. Granger, T. Painter, and N. Das, 2014: Satellite-based precipitation estimation and its application for streamflow prediction over mountainous western U.S. basins. J. Appl. Meteor. Climatol., 53, 28232842, https://doi.org/10.1175/JAMC-D-14-0056.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Borg, I., P. J. Groenen, and P. Mair, 2017: Applied Multidimensional Scaling and Unfolding. Springer, 122 pp.

    • Crossref
    • Export Citation
  • Bukovsky, M. S., and D. J. Karoly, 2007: A brief evaluation of precipitation from the North American Regional Reanalysis. J. Hydrometeor., 8, 837846, https://doi.org/10.1175/JHM595.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ciach, G. J., 2003: Local random errors in tipping-bucket rain gauge measurements. J. Atmos. Oceanic Technol., 20, 752759, https://doi.org/10.1175/1520-0426(2003)20<752:LREITB>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Contractor, S., L. V. Alexander, M. G. Donat, and N. Herold, 2015: How well do gridded datasets of observed daily precipitation compare over Australia? Adv. Meteor., 2015, 325718, https://doi.org/10.1155/2015/325718.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Daly, C., M. Halbleib, J. I. Smith, W. P. Gibson, M. K. Doggett, G. H. Taylor, J. Curtis, and P. P. Pasteris, 2008: Physiographically sensitive mapping of climatological temperature and precipitation across the conterminous United States. Int. J. Climatol., 28, 20312064, https://doi.org/10.1002/joc.1688.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dee, D. P., and Coauthors, 2011: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553597, https://doi.org/10.1002/qj.828.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • de Leeuw, J., and P. Mair, 2009: Multidimensional scaling using majorization: SMACOF in R. J. Stat. Software, 31 (3), https://doi.org/10.18637/jss.v031.i03.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Diaconescu, E. P., P. Gachon, and R. Laprise, 2015: On the remapping procedure of daily precipitation statistics and indices used in regional climate model evaluation. J. Hydrometeor., 16, 23012310, https://doi.org/10.1175/JHM-D-15-0025.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Diaconescu, E. P., P. Gachon, R. Laprise, and J. F. Scinocca, 2016: Evaluation of precipitation indices over North America from various configurations of regional climate models. Atmos.–Ocean, 54, 418439, https://doi.org/10.1080/07055900.2016.1185005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Donat, M. G., J. Sillmann, S. Wild, L. V. Alexander, T. Lippmann, and F. W. Zwiers, 2014: Consistency of temperature and precipitation extremes across various global gridded in situ and reanalysis datasets. J. Climate, 27, 50195035, https://doi.org/10.1175/JCLI-D-13-00405.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gelaro, R., and Coauthors, 2017: The Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2). J. Climate, 30, 54195454, https://doi.org/10.1175/JCLI-D-16-0758.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gibson, P. B., S. E. Perkins-Kirkpatrick, L. V. Alexander, and E. M. Fischer, 2017: Comparing Australian heat waves in the CMIP5 models through cluster analysis. J. Geophys. Res. Atmos., 122, 32663281, https://doi.org/10.1002/2016JD025878.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Giorgi, F., and W. J. Gutowski Jr., 2015: Regional dynamical downscaling and the CORDEX initiative. Annu. Rev. Environ. Resour., 40, 467490, https://doi.org/10.1146/annurev-environ-102014-021217.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guttman, L., 1968: A general nonmetric technique for finding the smallest coordinate space for a configuration of points. Psychometrika, 33, 469506, https://doi.org/10.1007/BF02290164.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Henn, B., A. J. Newman, B. Livneh, C. Daly, and J. D. Lundquist, 2018: An assessment of differences in gridded precipitation datasets in complex terrain. J. Hydrol., 556, 12051219, https://doi.org/10.1016/j.jhydrol.2017.03.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Herger, N., G. Abramowitz, R. Knutti, O. Angélil, K. Lehmann, and B. M. Sanderson, 2018: Selecting a climate model subset to optimise key ensemble properties. Earth Syst. Dyn., 9, 135151, https://doi.org/10.5194/esd-9-135-2018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Herold, N., L. V. Alexander, M. G. Donat, S. Contractor, and A. Becker, 2016: How much does it rain over land? Geophys. Res. Lett., 43, 341348, https://doi.org/10.1002/2015GL066615.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Higgins, R., W. Shi, E. Yarosh, and R. Joyce, 2000: Improved United States precipitation quality control system and analysis. NCEP/Climate Prediction Center ATLAS 7, https://www.cpc.ncep.noaa.gov/research_papers/ncep_cpc_atlas/7/toc.html.

  • Hout, M. C., M. H. Papesh, and S. D. Goldinger, 2013: Multidimensional scaling. Wiley Interdiscip. Rev.: Cognit. Sci., 4, 93103, https://doi.org/10.1002/wcs.1203.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huffman, G. J., R. F. Adler, M. M. Morrissey, D. T. Bolvin, S. Curtis, R. Joyce, B. McGavock, and J. Susskind, 2001: Global precipitation at one-degree daily resolution from multisatellite observations. J. Hydrometeor., 2, 3650, https://doi.org/10.1175/1525-7541(2001)002<0036:GPAODD>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huffman, G. J., and Coauthors, 2007: The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-global, multiyear, combined-sensor precipitation estimates at fine scales. J. Hydrometeor., 8, 3855, https://doi.org/10.1175/JHM560.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kay, J. E., and Coauthors, 2015: The Community Earth System Model (CESM) Large Ensemble Project: A community resource for studying climate change in the presence of internal climate variability. Bull. Amer. Meteor. Soc., 96, 13331349, https://doi.org/10.1175/BAMS-D-13-00255.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kendon, E. J., and Coauthors, 2017: Do convection-permitting regional climate models improve projections of future precipitation change? Bull. Amer. Meteor. Soc., 98, 7993, https://doi.org/10.1175/BAMS-D-15-0004.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kim, J., J. Sanjay, C. Mattmann, M. Boustani, M. Ramarao, R. Krishnan, and D. Waliser, 2015: Uncertainties in estimating spatial and interannual variations in precipitation climatology in the India–Tibet region from multiple gridded precipitation datasets. Int. J. Climatol., 35, 45574573, https://doi.org/10.1002/joc.4306.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lee, H., A. Goodman, L. McGibbney, D. Waliser, J. Kim, P. Loikith, P. Gibson, and E. Massoud, 2018: Regional Climate Model Evaluation System. Geosci. Model Dev., 11, 44354449, https://doi.org/10.5194/gmd-11-4435-2018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lorenz, R., A. Pitman, M. Donat, A. Hirsch, J. Kala, E. Kowalczyk, R. Law, and J. Srbinovsky, 2014: Representation of climate extreme indices in the ACCESS1. 3b coupled atmosphere–land surface model. Geosci. Model Dev., 7, 545567, https://doi.org/10.5194/gmd-7-545-2014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lu, C., Y. Sun, and X. Zhang, 2018: Multimodel detection and attribution of changes in warm and cold spell durations. Environ. Res. Lett., 13, 074013, https://doi.org/10.1088/1748-9326/aacb3e.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lucas-Picher, P., R. Laprise, and K. Winger, 2017: Evidence of added value in North American regional climate model hindcast simulations using ever-increasing horizontal resolutions. Climate Dyn., 48, 26112633, https://doi.org/10.1007/s00382-016-3227-z.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mearns, L. M., and Coauthors, 2017: The NA-CORDEX dataset, version 1.0. NCAR Climate Data Gateway, accessed 18 January 2018, https://doi.org/10.5065/D6SJ1JCH.

    • Crossref
    • Export Citation
  • Mesinger, F., and Coauthors, 2006: North American Regional Reanalysis. Bull. Amer. Meteor. Soc., 87, 343360, https://doi.org/10.1175/BAMS-87-3-343.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Newman, A. J., and Coauthors, 2015: Gridded ensemble precipitation and temperature estimates for the contiguous United States. J. Hydrometeor., 16, 24812500, https://doi.org/10.1175/JHM-D-15-0026.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Prein, A. F., and A. Gobiet, 2017: Impacts of uncertainties in European gridded precipitation observations on regional climate analysis. Int. J. Climatol., 37, 305327, https://doi.org/10.1002/joc.4706.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Prein, A. F., and Coauthors, 2015: A review on regional convection-permitting climate modeling: Demonstrations, prospects, and challenges. Rev. Geophys., 53, 323361, https://doi.org/10.1002/2014RG000475.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Prein, A. F., and Coauthors, 2016: Precipitation in the EURO-CORDEX 0.11° and 0.44° simulations: High resolution, high benefits? Climate Dyn., 46, 383412, https://doi.org/10.1007/s00382-015-2589-y.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rienecker, M. M., and Coauthors, 2011: MERRA: NASA’s Modern-Era Retrospective Analysis for Research and Applications. J. Climate, 24, 36243648, https://doi.org/10.1175/JCLI-D-11-00015.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sanderson, B. M., R. Knutti, and P. Caldwell, 2015: Addressing interdependency in a multimodel ensemble by interpolation of model properties. J. Climate, 28, 51505170, https://doi.org/10.1175/JCLI-D-14-00361.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Screen, J. A., C. Deser, and L. Sun, 2015: Projected changes in regional climate extremes arising from Arctic sea ice loss. Environ. Res. Lett., 10, 084006, https://doi.org/10.1088/1748-9326/10/8/084006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sillmann, J., V. V. Kharin, X. Zhang, F. W. Zwiers, and D. Bronaugh, 2013: Climate extremes indices in the CMIP5 multimodel ensemble: Part 1. Model evaluation in the present climate. J. Geophys. Res. Atmos., 118, 17161733, https://doi.org/10.1002/jgrd.50203.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Skofronick-Jackson, G., and Coauthors, 2017: The Global Precipitation Measurement (GPM) mission for science and society. Bull. Amer. Meteor. Soc., 98, 16791695, https://doi.org/10.1175/BAMS-D-15-00306.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stephens, G. L., and Coauthors, 2010: Dreary state of precipitation in global models. J. Geophys. Res., 115, D24211, https://doi.org/10.1029/2010JD014532.

    • Search Google Scholar
    • Export Citation
  • Tang, G., A. Behrangi, D. Long, C. Li, and Y. Hong, 2018: Accounting for spatiotemporal errors of gauges: A critical step to evaluate gridded precipitation products. J. Hydrol., 559, 294306, https://doi.org/10.1016/j.jhydrol.2018.02.057.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Taylor, K. E., 2001: Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res., 106, 71837192, https://doi.org/10.1029/2000JD900719.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thornton, P. E., S. W. Running, and M. A. White, 1997: Generating surfaces of daily meteorological variables over large regions of complex terrain. J. Hydrol., 190, 214251, https://doi.org/10.1016/S0022-1694(96)03128-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thornton, P. E., M. M. Thornton, B. W. Mayer, Y. Wei, R. Devarakonda, R. S. Vose, and R. B. Cook, 2017: Daymet: Daily surface weather data on a 1-km grid for North America, version 3. ORNL Distributed Active Archive Center, accessed 1 February 2018, https://doi.org/10.3334/ORNLDAAC/1328.

    • Crossref
    • Export Citation
  • Tian, B., and Coauthors, 2017: Development of a model performance metric and its application to assess summer precipitation over the U.S. Great Plains in downscaled climate simulations. J. Hydrometeor., 18, 27812799, https://doi.org/10.1175/JHM-D-17-0045.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Torgerson, W. S., 1958: Theory and Methods of Scaling. Wiley, 460 pp.

  • Wehner, M. F., 2013: Very extreme seasonal precipitation in the NARCCAP ensemble: model performance and projections. Climate Dyn., 40, 5980, https://doi.org/10.1007/s00382-012-1393-1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wehner, M. F., R. L. Smith, G. Bala, and P. Duffy, 2010: The effect of horizontal resolution on simulation of very extreme US precipitation events in a global atmosphere model. Climate Dyn., 34, 241247, https://doi.org/10.1007/s00382-009-0656-y.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Whan, K., and F. Zwiers, 2017: The impact of ENSO and the NAO on extreme winter precipitation in North America in observations and regional climate models. Climate Dyn., 48, 14011411, https://doi.org/10.1007/s00382-016-3148-x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, X., L. Alexander, G. C. Hegerl, P. Jones, A. K. Tank, T. C. Peterson, B. Trewin, and F. W. Zwiers, 2011: Indices for monitoring changes in extremes based on daily temperature and precipitation data. Wiley Interdiscip. Rev.: Climate Change, 2, 851870, https://doi.org/10.1002/wcc.147.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 2287 998 118
PDF Downloads 1358 248 17