• Agel, L., M. Barlow, J.-H. Qian, F. Colby, E. Douglas, and T. Eichler, 2015: Climatology of daily precipitation and extreme precipitation events in the northeast United States. J. Hydrometeor., 16, 25372557, https://doi.org/10.1175/JHM-D-14-0147.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Agel, L., M. Barlow, S. B. Feldstein, and W. J. Gutowski, 2018: Identification of large-scale meteorological patterns associated with extreme precipitation in the US Northeast. Climate Dyn., 50, 18191839, https://doi.org/10.1007/s00382-017-3724-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Agel, L., M. Barlow, F. Colby, H. Binder, J. L. Catto, A. Hoell, and J. Cohen, 2019: Dynamical analysis of extreme precipitation in the US Northeast based on large-scale meteorological patterns. Climate Dyn., 52, 17391760, https://doi.org/10.1007/s00382-018-4223-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bador, M., M. G. Donat, O. Geoffroy, and L. V. Alexander, 2018: Assessing the robustness of future extreme precipitation intensification in the CMIP5 ensemble. J. Climate, 31, 65056525, https://doi.org/10.1175/JCLI-D-17-0683.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barlow, M., and Coauthors, 2019: North American extreme precipitation events and related large-scale meteorological patterns: A review of statistical methods, dynamics, modeling, and trends. Climate Dyn., 53, 68356875, https://doi.org/10.1007/s00382-019-04958-z.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, C.-T., and T. Knutson, 2008: On the verification and comparison of extreme rainfall indices from climate models. J. Climate, 21, 16051621, https://doi.org/10.1175/2007JCLI1494.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, M., and Coauthors, 2008: CPC unified gauge-based analysis of global daily precipitation. Western Pacific Geophysics Meeting, Amer. Geophys. Union, Cairns, Australia, 14 pp., ftp://ftp.cpc.ncep.noaa.gov/precip/CPC_UNI_PRCP/GAUGE_CONUS/DOCU/Chen_et_al_2008_Daily_Gauge_Anal.pdf.

  • Colle, B. A., Z. Zhang, K. A. Lombardo, E. Chang, P. Liu, and M. Zhang, 2013: Historical evaluation and future prediction of eastern North American and western Atlantic extratropical cyclones in the CMIP5 models during the cool season. J. Climate, 26, 68826903, https://doi.org/10.1175/JCLI-D-12-00498.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Collow, A. B. M., M. G. Bosilovich, and R. D. Koster, 2016: Large-scale influences on summertime extreme precipitation in the northeastern United States. J. Hydrometeor., 17, 30453061, https://doi.org/10.1175/JHM-D-16-0091.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dai, A., and K. E. Trenberth, 2004: The diurnal cycle and its depiction in the Community Climate System Model. J. Climate, 17, 930951, https://doi.org/10.1175/1520-0442(2004)017<0930:TDCAID>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Diday, E., and J. C. Simon, 1976: Clustering analysis. Digital Pattern Recognition, K. S. Fu, Ed., Springer, 47–94.

    • Crossref
    • Export Citation
  • Easterling, D. R., and Coauthors, 2017: Precipitation change in the United States. Climate Science Special Report: Fourth National Climate Assessment, D. J. Wuebbles et al., Eds., U.S. Global Change Research Program Rep., 207–230, https://doi.org/10.7930/J0H993CC.

    • Crossref
    • Export Citation
  • Eyring, V., S. Bony, G. A. Meehl, C. A. Senior, B. Stevens, R. J. Stouffer, and K. E. Taylor, 2016: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci. Model Dev., 9, 19371958, https://doi.org/10.5194/gmd-9-1937-2016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fereday, D., R. Chadwick, J. Knight, and A. A. Scaife, 2018: Atmospheric dynamics is the largest source of uncertainty in future winter European rainfall. J. Climate, 31, 963977, https://doi.org/10.1175/JCLI-D-17-0048.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Frei, A., K. E. Kunkel, and A. Matonse, 2015: The seasonal nature of extreme hydrological events in the northeastern United States. J. Hydrometeor., 16, 20652085, https://doi.org/10.1175/JHM-D-14-0237.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gelaro, R., and Coauthors, 2017: The Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2). J. Climate, 30, 54195454, https://doi.org/10.1175/JCLI-D-16-0758.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Howarth, M. E., C. D. Thorncroft, and L. F. Bosart, 2019: Changes in extreme precipitation in the northeast United States: 1979–2014. J. Hydrometeor., 20, 673689, https://doi.org/10.1175/JHM-D-18-0155.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • IPCC, 2014: Climate Change 2014: Synthesis Report. IPCC, 151 pp., https://www.ipcc.ch/site/assets/uploads/2018/05/SYR_AR5_FINAL_full_wcover.pdf.

  • Jiang, Z., W. Li, J. Xu, and L. Li, 2015: Extreme precipitation indices over China in CMIP5 models. Part I: Model evaluation. J. Climate, 28, 86038619, https://doi.org/10.1175/JCLI-D-15-0099.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Karmalkar, A. V., J. M. Thibeault, A. M. Bryan, and A. Seth, 2019: Identifying credible and diverse GCMs for regional climate change studies—Case study: Northeastern United States. Climatic Change, 154, 367386, https://doi.org/10.1007/s10584-019-02411-y.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kopparla, P., E. M. Fischer, C. Hannay, and R. Knutti, 2013: Improved simulation of extreme precipitation in a high-resolution atmosphere model. Geophys. Res. Lett., 40, 58035808, https://doi.org/10.1002/2013GL057866.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kusunoki, S., and O. Arakawa, 2015: Are CMIP5 models better than CMIP3 models in simulating precipitation over East Asia? J. Climate, 28, 56015621, https://doi.org/10.1175/JCLI-D-14-00585.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, J., H. Chen, X. Rong, J. Su, Y. Xin, K. Furtado, S. Milton, and N. Li, 2018: How well can a climate model simulate an extreme precipitation event: A case study using the transpose-AMIP experiment. J. Climate, 31, 65436556, https://doi.org/10.1175/JCLI-D-17-0801.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Menne, M. J., I. Durre, R. S. Vose, B. E. Gleason, and T. G. Houston, 2012: An overview of the Global Historical Climatology Network–Daily database. J. Atmos. Oceanic Technol., 29, 897910, https://doi.org/10.1175/JTECH-D-11-00103.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Michelangeli, P.-A., R. Vautard, and B. Legras, 1995: Weather regimes: Recurrence and quasi stationarity. J. Atmos. Sci., 52, 12371256, https://doi.org/10.1175/1520-0469(1995)052<1237:WRRAQS>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ning, L., and R. S. Bradley, 2014: Winter climate extremes over the northeastern United States and southeastern Canada and teleconnections with large-scale modes of climate variability. J. Climate, 28, 24752493, https://doi.org/10.1175/JCLI-D-13-00750.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roller, C. D., J.-H. Qian, L. Agel, M. Barlow, and V. Moron, 2016: Winter weather regimes in the northeast United States. J. Climate, 29, 29632980, https://doi.org/10.1175/JCLI-D-15-0274.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, Y., S. Solomon, A. Dai, and R. W. Portmann, 2006: How often does it rain? J. Climate, 19, 916934, https://doi.org/10.1175/JCLI3672.1.

  • Taylor, K. E., R. J. Stouffer, and G. A. Meehl, 2012: An overview of CMIP5 and the experiment design. Bull. Amer. Meteor. Soc., 93, 485498, https://doi.org/10.1175/BAMS-D-11-00094.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Haren, R., R. J. Haarsma, G. J. V. Oldenborgh, and W. Hazeleger, 2015: Resolution dependence of European precipitation in a state-of-the-art atmospheric general circulation model. J. Climate, 28, 51345149, https://doi.org/10.1175/JCLI-D-14-00279.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    (a) Typing domain for extremes located in the blue shaded region and (b) classifiability index (CI; Michelangeli et al. 1995) of k-means tests on MERRA-2 500-hPa geopotential heights on the top 1% of CPCU precipitation days, 1980–2017, shown as solid blue line for k-means clusterings from k = 1 to k = 8. Shading indicates one-sided 90% confidence interval for CI values using multiple random red-noise tests of data for similar clusterings.

  • View in gallery
    Fig. 2.

    CPCU 1980–2017 Northeast U.S. (a) grid locations, (b) top 1% wet-day threshold values at each grid location (mm; shaded), (c) grid-level monthly mean number of wet days (days month−1), intensity on wet days (mm day−1), and total precipitation (mm month−1), with 5th–95th-percentile values shaded; (d) as in (c), but for extreme precipitation days. For (c) and (d) wet days are days with precipitation of at least 0.2 mm day−1.

  • View in gallery
    Fig. 3.

    Results of k-means clustering (k = 4) of MERRA-2 500-hPa geopotential heights on CPCU 1980–2017 top 1% precipitation days, with (a) the composite patterns (contours shown as thick black lines at 6-dam intervals; anomalies shown with shading in 6-dam intervals; and mean sea level pressure shown as thin black lines at 2-hPa intervals); (b) pattern CPCU precipitation anomalies (shaded; mm), with grid-level extreme locations located by black dots (for frequencies greater than 0.15%) and gray crossbars dividing the region into four quadrants; (c) the seasonal frequency of the extreme days within each pattern, with gray shading indicating the 95% confidence interval of seasonal frequency based on random sampling of all extreme days regardless of pattern assignment (with sample size equal to the number of extreme days assigned to the pattern), and red, blue, and black bars indicating, respectively, values higher than, lower than, and within the 95% confidence interval of this background frequency; (d) histograms of spatial correlations of all pattern days to the pattern mean (in bins of 0.1); and (e) histograms of mean root-mean-square errors of all pattern days to the pattern mean (in bins of 10 m).

  • View in gallery
    Fig. 4.

    As in Fig. 2, but for CMCC-CM 1950–2005 daily precipitation. In (c) and (d) the blue line represents the model value, and the red line and the gray shading represent the observed value and the 5th–95th-percentile values for the observations, respectively, from Fig. 2.

  • View in gallery
    Fig. 5.

    As in Figs. 3a–c, but for k-means clustering of CMCC-CM 500-hPa geopotential heights on model 1950–2005 top 1% precipitation days, with (d) RMSE between each model pattern (P1–P4) and the four observational patterns (O1–O4), and (e) the spatial correlation between each model pattern and each observational pattern. Asterisks above bars indicate RMSE (correlations) significantly lower (higher) than expected due to random chance (by shuffling all model pattern dates and recalculating model pattern RMSE and correlations) at the 0.05 level.

  • View in gallery
    Fig. 6.

    As in Fig. 4, but for CRNM-CM5.

  • View in gallery
    Fig. 7.

    As in Fig. 5, but for CRNM-CM5.

  • View in gallery
    Fig. 8.

    As in Fig. 4, but for NorESM1-M.

  • View in gallery
    Fig. 9.

    As in Fig. 5, but for NorESM1-M.

  • View in gallery
    Fig. 10.

    As in Fig. 4, but for IPSL-CM5A-LR.

  • View in gallery
    Fig. 11.

    As in Fig. 5, but for IPSL-CM5A-LR.

  • View in gallery
    Fig. 12.

    Assessment of CMIP5 models based on 6 precipitation-related metrics and 12 k-means clustering-related metrics of 500-hPa geopotential heights on extreme precipitation days, as defined by Table 3. An “×” represents a substantial departure from observations, while a green dot represents an approximate correspondence to observations. The three right columns contain counts of green dots for the precipitation metrics, the pattern metrics, and all metrics, respectively.

All Time Past Year Past 30 Days
Abstract Views 286 0 0
Full Text Views 261 135 16
PDF Downloads 245 108 15

Simulation of Northeast U.S. Extreme Precipitation and Its Associated Circulation by CMIP5 Models

Laurie AgelDepartment of Environmental, Earth, and Atmospheric Sciences, University of Massachusetts Lowell, Lowell, Massachusetts

Search for other papers by Laurie Agel in
Current site
Google Scholar
PubMed
Close
,
Mathew BarlowDepartment of Environmental, Earth, and Atmospheric Sciences, and Climate Change Initiative, University of Massachusetts Lowell, Lowell, Massachusetts

Search for other papers by Mathew Barlow in
Current site
Google Scholar
PubMed
Close
,
Joseph PoloniaDepartment of Environmental, Earth, and Atmospheric Sciences, University of Massachusetts Lowell, Lowell, Massachusetts

Search for other papers by Joseph Polonia in
Current site
Google Scholar
PubMed
Close
, and
David CoeDepartment of Environmental, Earth, and Atmospheric Sciences, University of Massachusetts Lowell, Lowell, Massachusetts

Search for other papers by David Coe in
Current site
Google Scholar
PubMed
Close
Free access

Abstract

Historical simulations from 14 models participating in phase 5 of the Coupled Model Intercomparison Project (CMIP5) are evaluated for their ability to reproduce observed precipitation in the northeastern United States and its associated circulation, with particular emphasis on extreme (top 1%) precipitation. The models are compared to observations in terms of the spatial variations of extreme precipitation, seasonal cycles of precipitation and extreme precipitation frequency and intensity, and extreme precipitation circulation regimes. The circulation regimes are identified using k-means clustering of 500-hPa geopotential heights on extreme precipitation days, in both observations and in the models. While all models capture an observed northwest-to-southeast gradient of precipitation intensity (reflected in the top 1% threshold), there are substantial differences from observations in the magnitude of the gradient. These differences tend to be more substantial for lower-resolution models. However, regardless of resolution, and despite a bias toward too-frequent precipitation, many of the models capture the seasonality of observed daily precipitation intensity, and the approximate magnitude and seasonality of observed extreme precipitation intensity. Many of the simulated extreme precipitation circulation patterns are visually similar to the set of observed patterns. However, the location and magnitude of specific troughs and ridges within the patterns, as well as the seasonality of the patterns, may differ substantially from the observed corresponding patterns. A series of metrics is developed based on the observed regional characteristics to facilitate comparison between models.

This article has a companion article which can be found at http://journals.ametsoc.org/doi/abs/10.1175/JCLI-D-19-1025.1.

Corresponding author: Laurie Agel, laurie_agel@uml.edu

Abstract

Historical simulations from 14 models participating in phase 5 of the Coupled Model Intercomparison Project (CMIP5) are evaluated for their ability to reproduce observed precipitation in the northeastern United States and its associated circulation, with particular emphasis on extreme (top 1%) precipitation. The models are compared to observations in terms of the spatial variations of extreme precipitation, seasonal cycles of precipitation and extreme precipitation frequency and intensity, and extreme precipitation circulation regimes. The circulation regimes are identified using k-means clustering of 500-hPa geopotential heights on extreme precipitation days, in both observations and in the models. While all models capture an observed northwest-to-southeast gradient of precipitation intensity (reflected in the top 1% threshold), there are substantial differences from observations in the magnitude of the gradient. These differences tend to be more substantial for lower-resolution models. However, regardless of resolution, and despite a bias toward too-frequent precipitation, many of the models capture the seasonality of observed daily precipitation intensity, and the approximate magnitude and seasonality of observed extreme precipitation intensity. Many of the simulated extreme precipitation circulation patterns are visually similar to the set of observed patterns. However, the location and magnitude of specific troughs and ridges within the patterns, as well as the seasonality of the patterns, may differ substantially from the observed corresponding patterns. A series of metrics is developed based on the observed regional characteristics to facilitate comparison between models.

This article has a companion article which can be found at http://journals.ametsoc.org/doi/abs/10.1175/JCLI-D-19-1025.1.

Corresponding author: Laurie Agel, laurie_agel@uml.edu

1. Introduction

Extreme precipitation has severe societal impacts, and both observations and projections suggest that the overall intensity of extreme precipitation will likely increase in a warming planet (IPCC 2014; Easterling et al. 2017). Projections of extreme precipitation, therefore, are of great interest. Understanding the ability of climate models to simulate precipitation, particularly extreme precipitation, is critical to assessing confidence in these model projections. Two central questions are the following: 1) Do climate models generate extreme precipitation with realistic spatial and intensity distributions? 2) Are the dynamic and hydrodynamic ingredients that generate extreme precipitation in the models the same as those for observed extreme precipitation? Here, we address these questions by comparing model simulations to observations with respect to the frequency distribution of precipitation intensity, spatial variations in precipitation amount and seasonality, and extreme precipitation circulation regimes. We focus on the northeastern United States, a populous region that has experienced notable increases in extreme precipitation over recent decades (Easterling et al. 2017), and examine historical simulations from 14 models in phase 5 of the Coupled Model Intercomparison Project (CMIP5; Taylor et al. 2012), comparing to reanalysis data. This analysis will provide additional insight to previous analyses of the CMIP5 models, and will also provide a useful benchmark for assessing the next-generation CMIP phase 6 simulations (CMIP6; Eyring et al. 2016), as they become available.

Based on previous evaluations of the CMIP5 model simulations and other general circulation models, both resolution and realistic circulation have been found to be important factors within similar model families in producing realistic precipitation. Kusunoki and Arakawa (2015) found that East Asian precipitation simulation improved with better CMIP5 resolution, although most models underestimated winter precipitation and overestimated summer precipitation in this region. Kopparla et al. (2013) found that extreme precipitation was more accurately modeled at higher resolution (although with considerable biases remaining), using the CAM4 model for areas of the United States, Europe, and Australia; also, van Haren et al. (2015) found that higher-resolution (25 vs 112 km) versions of the EC-EARTH model provided better simulation of European winter precipitation, due to better storm tracks, and the resulting moisture transport and convergence. Bador et al. (2018) found that CMIP5 intermodel differences in projected changes in precipitation were not significantly different if the models shared the same atmospheric component, although this was more apparent in the tropics, where the model physics and parameterizations were more tightly coupled to circulation. Colle et al. (2013) investigated the ability of CMIP5 models to reproduce cyclone genesis, tracks, rate of development, and intensity, and found that certain models perform better than others, based largely, but not entirely, on resolution. In fact, Fereday et al. (2018) found that circulation variability between models was the leading cause of precipitation variability. Certain types of circulation features may be easier to reproduce than others. Li et al. (2018) found that a general circulation model set in weather forecast mode for an extreme event in the Yangtze River valley reproduced precipitation better when the associated circulation features were larger in spatial scale. More recently, Karmalkar et al. (2019) evaluated CMIP5 historical simulation monthly precipitation and temperature output for the U.S. Northeast using both standard and process-based seasonal metrics, and found that no one model performed well across all metrics and seasons, but identified a subset of 16 models that together generated “credible” and “diverse” simulations of precipitation and associated circulation suitable for regional impact or prediction purposes.

Although this previous work highlights the importance of model resolution and ability to reproduce appropriate circulation features associated with precipitation, only one of these studies relates specifically to the U.S. Northeast, a region that has experienced a 55% increase in 99th-percentile precipitation from 1958 to 2016 (Easterling et al. 2017). This increase includes more frequent extreme precipitation during the warm season (Frei et al. 2015) and more intense extreme precipitation during the fall (Howarth et al. 2019). Here, we concentrate on this region, as a detailed climatology of overall and extreme precipitation is available (Agel et al. 2015), and the dynamics and thermodynamics associated with extreme precipitation have been recently examined (Agel et al. 2019). Northeast precipitation has distinct seasonal cycles of daily intensity and frequency. Frequency of precipitation peaks during midwinter at inland locations, and during late spring and early winter at coastal locations; whereas intensity peaks in midsummer for inland locations, and during late spring/early summer and again in early fall for coastal locations. Extreme precipitation (top 1% of days with precipitation) frequency peaks in late summer/early fall at all locations, but extreme precipitation intensity shows less seasonal variation (~50 mm day−1 at inland locations and ~75 mm day−1 at coastal locations).

Pattern analysis has proven to be an effective tool in identifying circulation and other dynamical factors associated with extreme precipitation in this region (Ning and Bradley 2014; Roller et al. 2016; Collow et al. 2016; Agel et al. 2018). In Agel et al. (2019), six large-scale meteorological patterns (LSMPs; see Barlow et al. 2019) of tropopause heights on extreme precipitation days are identified, along with their associated dynamical mechanisms. Winter patterns of tropopause height feature strong extratropical cyclones, with the cyclone center located either along the southeastern U.S. coast, with extreme precipitation driven by quasigeostrophic upward forcing, or to the south of the Ohio Valley, with extreme precipitation driven by warm conveyor belts situated along the coastline. Summer patterns of tropopause height are associated with upper-level ridging over the Northeast, with extreme precipitation associated with shortwaves and cold fronts in the interior.

For this study, we identify midtropospheric LSMPs by performing k-means clustering on a large-scale reanalysis circulation field (500-hPa geopotential heights) for a set of observed extreme precipitation days, which are based on a high-resolution gridded precipitation dataset. A similar analysis is performed for the selected CMIP5 models, using circulation and precipitation output fields from the “historical” simulations. We choose 500-hPa heights to identify key circulation characteristics, as this field is readily available in the CMIP5 model suite, and can capture major synoptic movement and structure.

This remainder of this study is organized as follows. Data and methodology are presented in section 2, the results of the study are presented in section 3, and the results are summarized and discussed in section 4.

2. Data and methods

a. Observed extreme precipitation days

Extreme precipitation observations are defined as the top 1% wet-day (exceeding 0.2 mm) precipitation for the period 1980–2017 from the National Oceanic and Atmospheric Administration (NOAA) Climate Prediction Center (CPC) 0.25° × 0.25° Daily U.S. Unified Precipitation (CPCU; Chen et al. 2008), available at https://psl.noaa.gov/data/gridded/data.unified.html, and based on station observations from the Global Historical Climatology Network (Menne et al. 2012). The domain considered is the northeastern United States (hereinafter simply Northeast), defined as the states of Maine, New Hampshire, Vermont, Massachusetts, New York, Connecticut, Rhode Island, New Jersey, Delaware, Pennsylvania, Maryland, and West Virginia (Fig. 1a). The 0.2-mm wet-day threshold was chosen to allow better comparison to climate model precipitation, which tends to overdo “drizzle” (Sun et al. 2006; Dai and Trenberth 2004). The extreme precipitation threshold value varies from maximum values of near 60 mm day−1 along coastal locations to 30 mm day−1 at the most western inland locations. We identify 3009 days where extreme precipitation occurs within at least one grid box in the domain.

Fig. 1.
Fig. 1.

(a) Typing domain for extremes located in the blue shaded region and (b) classifiability index (CI; Michelangeli et al. 1995) of k-means tests on MERRA-2 500-hPa geopotential heights on the top 1% of CPCU precipitation days, 1980–2017, shown as solid blue line for k-means clusterings from k = 1 to k = 8. Shading indicates one-sided 90% confidence interval for CI values using multiple random red-noise tests of data for similar clusterings.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Although grid-level extreme precipitation cannot be interpreted in the same manner as extreme station observations (Chen and Knutson 2008), we find for this application that the grid-level precipitation reasonably reflects the observed regional climatology and regional variations of the station data. CPCU characteristics are compared to 31 Global Historical Climate Network (GHCN) stations (see the online supplemental information), and while we find that CPCU does not capture the full variation of observations at point sources (too-frequent precipitation and too-light intensity), the gridded dataset is effective at qualitatively capturing many of the precipitation characteristics we examine. More important, the model gridded output is likely to share these same issues, making our use of gridded observations more comparable than point sources. Sensitivity tests were run using all available grid locations in the domain, a selection of grids to mimic previously studied station locations, a random selection of grids, and areal mean precipitation. The areal mean precipitation resulted in poor duplication of extreme values, while using either all grid boxes or a subset of grid boxes provided a range of extreme precipitation that was consistent with station observations.

In addition, since we are ultimately comparing observed precipitation characteristics to modeled characteristics, we also regrid the observations to each of the CMIP5 model resolutions and note that the upscaled characteristics we examine are not substantially different from those for the original resolution. Since there is little difference in the results, we use the original resolution of the CPCU dataset as our best estimate of observed precipitation, both visually and to compare characteristics to CMIP5 results.

b. Reanalysis circulation field

We use the National Aeronautics and Space Administration (NASA) Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2; Gelaro et al. 2017) 500-hPa daily mean geopotential heights, 1980–2017, available at https://disc.gsfc.nasa.gov/datasets?keywords="MERRA-2, to represent observed circulation on extreme precipitation days. The daily mean field is converted to daily anomalies by removing the long-term daily mean at each grid point. The long-term daily mean is calculated at each grid point by taking the mean of each yearday (i.e., 1 January, 2 January, etc.) over the 38 years, and smoothing the resulting 366-day time series with a 21-day running mean.

c. CMIP5 models

Fourteen CMIP5 models were selected for this study, based on availability (at the time of access) of 500-hPa geopotential heights, total precipitation, and mean sea level pressure (MSLP) fields associated with the 1950–2005 “r1i1p1” historical simulations. Table 1 lists the models and associated information. Several of the models belong to the same model “family” but with varying resolution or underlying physics. The ACCESS1.0, ACCESS1.3, and HadGEM2-CC models share similar atmospheric components and grid resolutions, but differ in the atmospheric physics and cloud parameterizations. The CMCC-CESM, CMCC-CM, and CMCC-CMS models all share the same basic components, but differ in resolution and time steps. The IPSL-CM5A-LR and IPSL-CM5A-MR models differ only in resolution, while the IPSL-CM5B-LR model uses a different set of atmospheric physical parameterizations. The MPI-ESM-LR and MPI-ESM-MR models differ only in resolution, while the MPI-ESM-P model uses a different vegetation and land use scheme. For details regarding the atmospheric, ocean, land, and ice components, as well as the physics and moist process parameterizations for each model, several resources are available, including https://es-doc.org. The datasets are nearly complete, with the following caveats: HadGEM2-CC data are only available through November 2005, IPSL-CM5A-LR is missing six precipitation days, and MPI-ESM-LR is missing one geopotential height day. Resolution varies from coarse (3.71° × 3.75° for CMCC-CESM) to medium-high (0.75° for CMCC-CM).

Table 1.

CMIP5 models used for the study.

Table 1.

For each of the models, extreme precipitation days are identified using the same process as for the gridded observed precipitation (top 1% of wet days at each grid location), and the model geopotential height field is processed identically to the reanalysis geopotential height field. Table 2 shows the number of grid boxes used to evaluate extreme precipitation, the mean top 1% threshold for extreme precipitation, and the resulting number of extreme precipitation days for both the observations and the 14 CMIP5 models, arranged from highest to lowest resolution.

Table 2.

Additional observation and CMIP5 model information, in descending order by resolution. Included are the number of grid cells contained in the Northeast (NE) domain, the top 1% threshold value for wet days, the resulting number of extreme precipitation days, the optimum number of k-means clusters of daily 500-hPa heights as determined by CI analysis (Michelangeli et al. 1995), the mean correlation (corr) of the daily heights to those assigned cluster patterns, and the mean correlation of the same to k-means results using four clusters.

Table 2.

d. Typing methodology

The k-means clustering methodology (Diday and Simon 1976; Michelangeli et al. 1995) is a technique that separates input data into a preselected number of nonoverlapping “clusters,” where each cluster is defined by its centroid (the mean of the inputs assigned to that cluster). Data are assigned iteratively to clusters based on the nearest centroid (squared Euclidean point-to-centroid distance), after which the centroids are recalculated. This process is repeated until further iterations no longer reduce the sum of the intracluster variances. Here, the k-means methodology is based on MATLAB’s built-in kmeans function, combined with an objective technique from Michelangeli et al. (1995) to determine the optimum number of clusters k, and the most consistent partitioning for that k, for a given input field (in this case, 500-hPa geopotential heights preprocessed as detailed above on extreme precipitation days). The technique essentially searches for the k-means solution that maximizes the set of anomaly correlation coefficients (ACC) between the clusters of a trial partitioning and the clusters of all other trial partitionings. The mean of this measure over all trials (in this case, 1000) is called the classifiability index (CI). CIs can be calculated for a range of k, and compared to CIs generated using random red-noise series created from the original dataset. CIs that are above the 90% confidence interval (one-sided), based on 100 red-noise trials, indicate k values that result in consistent, easily reproduced patterns.

The typing domain (30°–50°N, 100°–60°W) is shown as the thick-lined box in Fig. 1a. The input field (MERRA-2 500-hPa geopotential height anomalies on 3009 observed extreme precipitation days) is converted to standardized temporal anomalies, arranged as a two-dimensional space–time grid, and then further reduced through empirical orthogonal function analysis, retaining 95% of the variance across the grids. The results of the CI analysis suggest an optimum k of 4 or 6 clusters (Fig. 1b). We use the k = 4 solution, as the four patterns are insensitive to grid resolution (regridding to lower resolutions, or alternatively selecting every other grid, and retyping, does not change the basic four patterns), while the k = 6 solution involves two patterns that are minor variations of the other four patterns, each of which matches the k = 4 solution. The k-means process is repeated for each of the CMIP5 model 500-hPa height fields (using extreme precipitation days as defined by the CMIP5 precipitation fields), and the k = 4 CMIP5 solution is compared to the observed/reanalysis solution. Table 2 shows the objectively determined best k using the methodology described above, and the mean correlation of individual days to their assigned patterns for both the “best k” and k = 4.

We also perform k-means typing on the MERRA-2 height field regridded to 0.5°, 1.0°, 1.5°, and 2.0° resolution (to mimic resolutions similar to those of the CMIP5 models), and note that the four-pattern solution is invariant to the resolution chosen, both in terms of visual pattern, and the majority of days assigned to each pattern.

e. Comparison of CMIP5 precipitation and circulation to observations

To facilitate comparison across the 14 different CMIP5 models, we develop a set of metrics, shown in Table 3, based in part on observed regional characteristics (Agel et al. 2015). The metrics are chosen to 1) best capture the main features of the observed precipitation and circulation patterns in this region, and 2) best capture the variations in the CMIP5 models to reproduce these features. Similar methods have been used to evaluate CMIP5 performance (Jiang et al. 2015; Kusunoki and Arakawa 2015; Karmalkar et al. 2019) for other regions, each specific to the region of interest and specific climate features. While the thresholds used are, of necessity, somewhat subjective, they are chosen to be generous in terms of assessing approximate correspondence between models and observations. That is, if a model metric is not assessed as approximately corresponding to observations, it is considered to be displaying a substantial difference from observations (in most cases, outside the 10th–90th-percentile range of the observational variation).

Table 3.

Metrics 1–6 are used to determine how well CMIP5 model precipitation simulates observed precipitation, and metrics 7–18 are used to determine how well k-means clustering of CMIP5 500-hPa geopotential heights on extreme precipitation days matches observed patterns of circulation on observed extreme precipitation days. Each metric describes the criteria for approximate correspondence to observations.

Table 3.

We use a total of 18 metrics. There are 6 metrics related to how well the models reproduce precipitation and extreme precipitation, based on the mean threshold for grid-level top 1% of wet days, regional variations in the grid-level thresholds, and seasonal means of frequency and daily intensity of both overall grid-level precipitation and extreme grid-level precipitation. There are 12 metrics related to how well each model’s extreme circulation patterns match those of observations. For most of the models, the four model patterns are qualitatively similar to the four observed patterns and lend themselves to a direct comparison to the observed patterns. The metrics assess how well the model reproduces the spatial locations of extreme precipitation within the pattern, the seasonality of extreme precipitation within the pattern, and the correlation of the pattern to the similar observed pattern. To facilitate evaluating the spatial locations of extremes within each pattern, we have divided the domain into four quadrants based on latitude 42°N and longitude 74.5°W.

f. Sensitivity to time period

To maximize our sample sizes, we use different time periods for observations (1980–2017) and the CMIP5 models (1950–2005). While potentially stronger trends may exist in the former, the particular precipitation characteristics we look at are invariant to the time period chosen. Specifically, the mean values for observed (CPCU) regional top 1% thresholds and monthly precipitation and extreme precipitation frequency and intensity are nearly identical whether using data from 1950–2005 or 1980–2017. This is also the case for CMIP5 models, in terms of using data from 1950–2005 or 1980–2005. There are only slight differences in the 10th–90th-percentile range of the underlying data for both observations and models, based on the time period chosen. In addition, the k-means patterns themselves are invariant to the time period chosen, in terms of visual appearance, and the days (common to both time periods) assigned to the patterns. Because underlying trends do not have a substantial impact on our results, we use the longer time periods for both observations and models to gain the largest sample sizes and best statistical information.

3. Results

a. Observed extreme precipitation

A summary of observed precipitation characteristics for the domain is provided in Fig. 2. The grid center locations are shown in Fig. 2a, along with the top 1% precipitation threshold at each grid location (Fig. 2b). The threshold increases from minimal values inland along the Great Lakes to maximal values along the coastline, consistent with climatology (Agel et al. 2015). The 38-yr monthly climatology of precipitation frequency, daily intensity, and total precipitation is shown in Fig. 2c. For the domain as a whole, there is more frequent precipitation of higher intensity during the warm months. A similar monthly climatology for extreme precipitation is shown in Fig. 2d, with the highest frequency and intensity of extreme precipitation occurring during late summer. The reanalysis climatology matches well with the station climatology established in Agel et al. (2015), considering that the inland and coastal locations here are combined into a single climatology. In particular the monthly values for daily intensity and daily extreme intensity are representative of the earlier climatology.

Fig. 2.
Fig. 2.

CPCU 1980–2017 Northeast U.S. (a) grid locations, (b) top 1% wet-day threshold values at each grid location (mm; shaded), (c) grid-level monthly mean number of wet days (days month−1), intensity on wet days (mm day−1), and total precipitation (mm month−1), with 5th–95th-percentile values shaded; (d) as in (c), but for extreme precipitation days. For (c) and (d) wet days are days with precipitation of at least 0.2 mm day−1.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

b. Reanalysis geopotential height patterns

There are four observed 500-hPa geopotential height patterns, or large-scale meteorological patterns (LSMPs), associated with observed extreme precipitation days in the Northeast, based on k-means typing (labeled O1–O4 in Fig. 3). Patterns O1 and O2 are characterized by more zonal than meridional flow, with O1 featuring a slight trough over the Northeast, and O2 featuring a slight ridge over the Northeast. Both feature high surface pressure to the southeast of the domain. O3 is characterized by a 500-hPa trough over the Great Lakes and a ridge over the eastern portion of the domain, and surface low pressure over the eastern Great Lakes. O4 is characterized by a deep upper-level trough over the domain, with accompanying low surface pressure centered over Maine.

Fig. 3.
Fig. 3.

Results of k-means clustering (k = 4) of MERRA-2 500-hPa geopotential heights on CPCU 1980–2017 top 1% precipitation days, with (a) the composite patterns (contours shown as thick black lines at 6-dam intervals; anomalies shown with shading in 6-dam intervals; and mean sea level pressure shown as thin black lines at 2-hPa intervals); (b) pattern CPCU precipitation anomalies (shaded; mm), with grid-level extreme locations located by black dots (for frequencies greater than 0.15%) and gray crossbars dividing the region into four quadrants; (c) the seasonal frequency of the extreme days within each pattern, with gray shading indicating the 95% confidence interval of seasonal frequency based on random sampling of all extreme days regardless of pattern assignment (with sample size equal to the number of extreme days assigned to the pattern), and red, blue, and black bars indicating, respectively, values higher than, lower than, and within the 95% confidence interval of this background frequency; (d) histograms of spatial correlations of all pattern days to the pattern mean (in bins of 0.1); and (e) histograms of mean root-mean-square errors of all pattern days to the pattern mean (in bins of 10 m).

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Figure 3b shows the locations (dots) where extreme precipitation (shaded) occurs within each pattern, and Fig. 3c shows the seasonal frequency of each pattern (DJF, MAM, JJA, and SON). Precipitation associated with O1 is not as extreme as for the other patterns, with the majority of the extremes occurring to the west of the domain, along the Great Lakes and along the Pennsylvania–West Virginia mountain chains during JJA. Anomalous precipitation in O2 is more widespread, with extremes occurring along the western and southern portions of the domain. Anomalous precipitation occurs throughout the domain in O3, with the majority of the extremes located to the southeast of the surface low. For O4, the majority of the extreme precipitation occurs in Maine. For O2–O4, extreme precipitation tends to occur less often during JJA and more often during cool seasons than expected by chance (opposite to that for O1).

In Fig. 3d, histograms of the spatial correlations of individual days to their assigned patterns are shown, while Fig. 3e shows histograms of the root-mean-square errors (RMSE) between individual days and their assigned patterns. The “best” overall pattern fit should theoretically show the steepest RMSE curve, with the mean closest to zero, since k-means typing uses Euclidean distance to fit individual days to patterns. Spatial correlation, although not directly linked to the k-means method, is also informative, and can potentially highlight the “best” overall fit of individual days to the pattern in terms of placement and magnitude of troughs and ridges. In this case, we anticipate the best fits to show the steepest correlation curve, with the mean closest to one. This is especially true for the patterns with pronounced troughs and ridges (e.g., O3 and O4), which also have the highest overall correlations. For the predominantly zonal pattern O1, however, correlations show the worst overall fit, while RMSE shows the best overall fit. This is not surprising, as small anomalies in the zonal flow can lead to large variations in spatial correlation. For this pattern, RMSE may be the better choice to assess pattern fit.

c. CMIP5 precipitation and typing results

Similar analysis of model precipitation and circulation fields is performed for each of the 14 CMIP5 models as was done for precipitation observations and reanalysis circulation fields. Each typing experiment produces an optimum k separation for the dataset, based on the Michelangeli et al. (1995) method. However, we look specifically at the k = 4 solution to facilitate direct comparison to the observed/reanalysis solution. The models range in their abilities to capture the regional variation in top 1% thresholds, the seasonal cycles of precipitation and seasonal precipitation, and the dynamical circulation on the set of extreme precipitation days, including the pattern seasonality. Here we show figures similar to Figs. 2 and 3 for two models with substantial similarities to observed precipitation and circulation on extreme days (CMCC-CM and CNRM-CM5; Figs. 47) and two models with substantial differences from observed precipitation and circulation fields (NorESM1-M and IPSL-CM5A-LR; Figs. 811), based on the metrics established in Table 3. The similarities and differences are summarized in Fig. 12 for all 14 models. A complete set of summary figures similar to Figs. 2 and 3 for all 14 models is supplied in the online supplemental information.

Fig. 4.
Fig. 4.

As in Fig. 2, but for CMCC-CM 1950–2005 daily precipitation. In (c) and (d) the blue line represents the model value, and the red line and the gray shading represent the observed value and the 5th–95th-percentile values for the observations, respectively, from Fig. 2.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Fig. 5.
Fig. 5.

As in Figs. 3a–c, but for k-means clustering of CMCC-CM 500-hPa geopotential heights on model 1950–2005 top 1% precipitation days, with (d) RMSE between each model pattern (P1–P4) and the four observational patterns (O1–O4), and (e) the spatial correlation between each model pattern and each observational pattern. Asterisks above bars indicate RMSE (correlations) significantly lower (higher) than expected due to random chance (by shuffling all model pattern dates and recalculating model pattern RMSE and correlations) at the 0.05 level.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Fig. 6.
Fig. 6.

As in Fig. 4, but for CRNM-CM5.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Fig. 7.
Fig. 7.

As in Fig. 5, but for CRNM-CM5.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Fig. 8.
Fig. 8.

As in Fig. 4, but for NorESM1-M.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Fig. 9.
Fig. 9.

As in Fig. 5, but for NorESM1-M.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Fig. 10.
Fig. 10.

As in Fig. 4, but for IPSL-CM5A-LR.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Fig. 11.
Fig. 11.

As in Fig. 5, but for IPSL-CM5A-LR.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Fig. 12.
Fig. 12.

Assessment of CMIP5 models based on 6 precipitation-related metrics and 12 k-means clustering-related metrics of 500-hPa geopotential heights on extreme precipitation days, as defined by Table 3. An “×” represents a substantial departure from observations, while a green dot represents an approximate correspondence to observations. The three right columns contain counts of green dots for the precipitation metrics, the pattern metrics, and all metrics, respectively.

Citation: Journal of Climate 33, 22; 10.1175/JCLI-D-19-0757.1

Most of the models successfully produce a top 1% mean precipitation threshold similar to observations, and qualitatively show an inland-to-coastal increase of the top 1% precipitation threshold, but few of the models are able to reproduce the actual range of thresholds (30–60 mm day−1) seen in the observed results. CMCC-CM and CNRM-CM5 (Figs. 4b and 6b) both come reasonably close to reproducing the actual range, along with HadGEM2-CC, ACCESS1.0, and ACCESS1.3 (not shown), although CMCC-CM overdoes the threshold at most of the inland locations. In contrast, both NorESM1-M and IPSL-CM5A-LR produce thresholds well below those established by observations, and without a significant range between inland and coastal locations (Figs. 8b and 10b). While low resolution may explain some of the difficulty these models have in capturing a realistic range of extreme precipitation values, we note that the observations regridded to those models’ resolutions do capture an appropriate range of values. The ability to reproduce observed precipitation, particularly at the high end, also does not appear to be entirely related to resolution, as even higher-resolution products such as CMCC-CM do not reproduce the precipitation range and thresholds as well as the medium-resolution MPI-ESM-LR and IPSL-CM5A-MR (not shown).

Only one of the models, CMCC-CM (Fig. 4c), reproduces the seasonal frequency of Northeast precipitation days well. Most models produce too many days of precipitation during the warm months (e.g., Figs. 6c, 8c, and 10c). We note that this is the case even when the observations are first regridded to the resolution of the CMIP5 model. In contrast, many of the models get the seasonal values and variations in precipitation intensity correct, ranging from 3 to 10 mm day−1, with a slight increase in the warm months. CMCC-CM and CNRM-CM5 (Figs. 4c and 6c) both get the intensity “right,” while NorESM1-M (Fig. 8c) produces well below observed intensities year-round, and IPSL-CM5A-LR (Fig. 10c) produces intensities outside the observed range in May and September.

The seasonal frequency of extreme precipitation in observations increases from the cold season to the warm season, peaking in July–September. CMCC-CM and CNRM-CM5 capture this well (Figs. 4c, 6c), while NorESM1-M produces too much cold-season extreme precipitation (Fig. 8c) and IPSL-CM5A-LR produces too much March and April extreme precipitation (Fig. 10c). For the other models, some produce too much warm-season extreme precipitation (CMCC-CESM, ACCESS1.0, ACCESS1.3) while others produce too much cold-season precipitation (MPI-ESM-LR, MPI-ESM-MR, and MPI-ESM-P). On the other hand, all but one model (NorESM1-M; Fig. 8c) capture the intensity of extreme precipitation well, in both the value and the seasonality.

The k-means typing analysis produces four 500-hPa geopotential height patterns that are visually very similar to the observed patterns for the majority of the models. Each of the models has an O1-like pattern, with predominant zonal flow and a slight anomalous trough, an O3-like pattern with a western trough and eastern ridge, and an O4-like pattern, with a deep trough over the Ohio Valley. The remaining pattern can be visually quite different from O2, but in general has ridging in the eastern part of the domain.

The four patterns (labeled P1–P4) are shown for CMCC-CM (Fig. 5a), CNRM-CM5 (Fig. 7a), NorESM1-M (Fig. 9a), and IPSL-CM5A-LR (Fig. 11a). The patterns are arranged in the same manner as for observations, with the slight Northeast trough at the top left, the slight Northeast ridge at the top right, the Great Lakes trough/surface low at the bottom left, and the deep upper-level trough with accompanying low surface pressure centered over Maine in the bottom right. We test this placement by looking at both the spatial correlation and RMSE between each model pattern and all four observation patterns (shown as grouped bar charts in Figs. 5d,e, 7d,e, 9d,e, and 11d,e). We expect the highest correlations and lowest RMSE for similar patterns (i.e. P1 → O1, P2 → O2, P3 → O3, and P4 → O4). Statistically significant values (at the 0.05 level) are indicated by asterisks above the bars. The statistical significance is evaluated by comparing these RMSE and correlation values to values obtained by randomly sampling CMIP5 daily fields (all extreme precipitation days), and using the sampled mean of the fields instead of patterns P1–P4 to compare to observed patterns O1–O4.

CMCC-CM shows good visual matching to observations, except for P1, which has slightly more zonal flow and P2, which has an enhanced ridge (Fig. 5a). P1 shows the strongest correlation to O3 but, as noted for observations, this is due to small fluctuations of relatively small anomalies, which more closely correlate to the trough and ridge of O3. The RMSE value for P1 → O1 is the smallest of the patterns, as we would expect. Likewise, the CNRM-CM5 patterns (Fig. 7a) are very similar to observations, albeit with a more enhanced ridge in P3. The patterns show high correlation and low RMSE to the matched observation pattern, and lower (or negative) correlation and larger RMSE to the other three patterns. In contrast, NorESM1-M is visually quite different from observations (Fig. 9a). For example, P2, although it correlates well to O2 (Fig. 9e), is visually quite different, with a shortwave trough over the Ohio Valley. In addition, P3 appears to be a mix of the observed O2 and O3, and P4 appears to be a mix of O3 and O4, based on spatial correlations and visual inspection. For IPSL-CM5A-LR, the P3 pattern shows a shifted-west trough and an enhanced eastern ridge (Fig. 11a). Based on RMSE (Fig. 11d) and spatial correlation (Fig. 11e), pattern P3 appears to be a mix of O2 and O3, while P4 appears to be a mix of O3 and O4. These enhanced troughs and ridges in the CMIP5 extreme patterns may be related to model dynamics that produce heaviest precipitation in conjunction with extratropical cyclones, as opposed to other dynamical processes. Depending on the model physics and parameters, this may also be related to the resolution of the model (Colle et al. 2013).

The separation of extremes is captured well by some models (CMCC-CM, MPI-ESM-LR and MPI-ESM-P) and poorly by others (IPSL-CM5B-LR). In part this is a function of grid size: the lower-resolution models may not have enough grid boxes within the quadrants defined by 42°N, 74.5°W to evaluate this feature with any skill. For CMCC-CM (Fig. 5b) we see extremes more likely to occur away from the southeast quadrant in P1, toward the west in P2, centered among the quadrants in P3, and to the northeast in P4, as for observations. For CNRM-CM5 (Fig. 7b), there are more northeast extremes in P1 and P2 than in O1 and O2, respectively. For NorESM1-M (Fig. 9b) and IPSL-CM5A-LR (Fig. 11b), patterns P1–P3 appear to have extremes distributed nearly equally across the quadrants (no grids occur in the southeast quadrant). All the models except IPSL-CM5B-LR mimic observations in favoring the northeast quadrant for extremes in pattern P4.

Finally, we evaluate the seasonality of extremes within each model pattern. Observations show that there are more extremes than expected during JJA in P1 (the near-zonal pattern with a slight trough), and conversely fewer extremes than expected during JJA in the other patterns (at the 0.05 level of significance). Both CMCC-CM and CNRM-CM5 show this same feature (Figs. 5c and 7c), as well as NorESM1-M (Fig. 9c) and IPSL-CM5A-LR (Fig. 11c). In fact, all models show the highest frequency of extremes in P1 to occur during JJA, although not all show the decreased frequency of JJA extremes in the other patterns. In addition, some models do not reproduce the observed overall seasonal distribution of extremes. For example, IPSL-CM5A-LR has the highest number of extremes during MAM, whereas NorESM1-M has the highest number of extremes during SON.

Figure 12 summarizes these results, using the precipitation and pattern metrics described in Table 3, with an “×” representing a substantial departure from observations, and a green dot representing an approximate correspondence to observations for each metric. The overall best-performing models, based on these metrics, are CMCC-CM and CNRM-CM5, with the best overall matching of both precipitation and pattern metrics. For the precipitation metrics, the higher-performing models are ACCESS1.0, CMCC-CC, CMCC-CMS, CNRM-CM5, and HadGEM2-CC, with 4 green dots each, while the lowest-performing models are IPSL-CM5A-LR and NorESM1-M, with 1 and 0 green dots, respectively. The lower-performing precipitation models in general have lower resolution.

The best-performing models in duplicating observed circulation patterns associated with extreme precipitation are CMCC-CM and MPI-ESM-LR, with counts of 10 green dots each, followed by ACCESS1.3 and CNRM-CM5, with counts of 9 green dots each. The poorest-performing models in the category are CMCC-CESM, IPSL-CM5B-LR, and MPI-ESM-MR, with 6 green dots each. One interesting result is that despite substantial differences from observations based on the precipitation metrics and being the lowest-resolution model in this set, the NorESM1-M model reproduced many of the observed pattern features, including correlation, location, and preference of JJA extremes in zonal pattern P1.

4. Summary and discussion

For the northeastern United States (hereafter simply Northeast), the ability of 14 CMIP5 models to reproduce key regional characteristics of observed overall precipitation, extreme precipitation, and extreme precipitation circulation regimes is assessed. The comparison is made in terms of regional values for extreme precipitation (top 1% of wet days), seasonal cycles of overall and extreme frequency and intensity of daily precipitation, and circulation regimes determined by cluster analysis. The key results are the following:

  • There is a wide range of model ability to reproduce observed extreme precipitation, with some models overproducing heavy precipitation and others underproducing heavy precipitation.

  • Few models capture the full seasonal cycle of both overall and extreme precipitation frequency (most models produce too-frequent precipitation), although most have considerable similarity to observations in terms of the observed seasonal range of daily intensity.

  • Model resolution may not be as important as other factors in generating realistic precipitation frequency and intensity. However, lower resolution does appear to severely curtail the ability to realistically reproduce the observed inland–coastal variations in precipitation.

  • Most of the models reproduce the main features of observed circulation patterns on extreme precipitation days (even those that were poor at reproducing observed extreme precipitation itself).

  • Many models appear to underrepresent extreme precipitation that is not related to extratropical cyclones, and thus the extreme precipitation circulation patterns tend to feature “enhanced” versions of the observed patterns of troughs and ridges.

Several of the models evaluated, in particular ACCESS1.0, CMCC-CC, CMCC-CMS, CNRM-CM5, and HadGEM2-CC, performed reasonably well with regard to the precipitation metrics, while ACCESS1.3, CMCC-CM, CNRM-CM5, and MPI-ESM-LR performed particularly well at reproducing the observed extreme-precipitation-related circulation. While no one model closely replicates all aspects of observed Northeast precipitation and extremes-related circulation considered here, there is value in understanding the limitations and strengths of each of these models in producing reasonable projections. While higher-resolution products tend to score higher based on the metrics used here, resolution alone is not a proxy for better performance, suggesting that model physics and parameterization are also playing an important role.

The metrics used here are developed to be specific to the Northeast region and are chosen to highlight the main characteristics of the regional observed precipitation and circulation, while capturing the range of abilities of the CMIP5 models to match these metrics. The intent of the study is not to define any CMIP5 model as “good” or “bad” at reproducing the characteristics of historical precipitation, but instead to provide information on the range of CMIP5 region-specific precipitation characteristics that are prominent in observations.

As CMIP6 historical runs become available, a similar analysis is planned to determine how well the CMIP6 models replicate observed Northeast precipitation and extreme precipitation, and its associated circulation. The results here will provide a useful benchmark for considering the changes from CMIP5 to CMIP6. Of particular interest is the degree to which improved resolution and physical parameterizations will relate to improvements in the simulation of extreme precipitation.

Acknowledgments

The work in this study is funded by National Science Foundation Project AGS 1623912.

Data availability statement

CPCU data are openly available from NOAA at https://psl.noaa.gov/data/gridded/data.unified.html. MERRA-2 data are openly available from NASA at https://gmao.gsfc.nasa.gov/reanalysis/MERRA-2/data. CMIP5 model data are openly available from Earth System Grid Federation sites such as https://esgf-node.llnl.gov/projects/cmip6.

REFERENCES

  • Agel, L., M. Barlow, J.-H. Qian, F. Colby, E. Douglas, and T. Eichler, 2015: Climatology of daily precipitation and extreme precipitation events in the northeast United States. J. Hydrometeor., 16, 25372557, https://doi.org/10.1175/JHM-D-14-0147.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Agel, L., M. Barlow, S. B. Feldstein, and W. J. Gutowski, 2018: Identification of large-scale meteorological patterns associated with extreme precipitation in the US Northeast. Climate Dyn., 50, 18191839, https://doi.org/10.1007/s00382-017-3724-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Agel, L., M. Barlow, F. Colby, H. Binder, J. L. Catto, A. Hoell, and J. Cohen, 2019: Dynamical analysis of extreme precipitation in the US Northeast based on large-scale meteorological patterns. Climate Dyn., 52, 17391760, https://doi.org/10.1007/s00382-018-4223-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bador, M., M. G. Donat, O. Geoffroy, and L. V. Alexander, 2018: Assessing the robustness of future extreme precipitation intensification in the CMIP5 ensemble. J. Climate, 31, 65056525, https://doi.org/10.1175/JCLI-D-17-0683.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barlow, M., and Coauthors, 2019: North American extreme precipitation events and related large-scale meteorological patterns: A review of statistical methods, dynamics, modeling, and trends. Climate Dyn., 53, 68356875, https://doi.org/10.1007/s00382-019-04958-z.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, C.-T., and T. Knutson, 2008: On the verification and comparison of extreme rainfall indices from climate models. J. Climate, 21, 16051621, https://doi.org/10.1175/2007JCLI1494.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, M., and Coauthors, 2008: CPC unified gauge-based analysis of global daily precipitation. Western Pacific Geophysics Meeting, Amer. Geophys. Union, Cairns, Australia, 14 pp., ftp://ftp.cpc.ncep.noaa.gov/precip/CPC_UNI_PRCP/GAUGE_CONUS/DOCU/Chen_et_al_2008_Daily_Gauge_Anal.pdf.

  • Colle, B. A., Z. Zhang, K. A. Lombardo, E. Chang, P. Liu, and M. Zhang, 2013: Historical evaluation and future prediction of eastern North American and western Atlantic extratropical cyclones in the CMIP5 models during the cool season. J. Climate, 26, 68826903, https://doi.org/10.1175/JCLI-D-12-00498.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Collow, A. B. M., M. G. Bosilovich, and R. D. Koster, 2016: Large-scale influences on summertime extreme precipitation in the northeastern United States. J. Hydrometeor., 17, 30453061, https://doi.org/10.1175/JHM-D-16-0091.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dai, A., and K. E. Trenberth, 2004: The diurnal cycle and its depiction in the Community Climate System Model. J. Climate, 17, 930951, https://doi.org/10.1175/1520-0442(2004)017<0930:TDCAID>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Diday, E., and J. C. Simon, 1976: Clustering analysis. Digital Pattern Recognition, K. S. Fu, Ed., Springer, 47–94.

    • Crossref
    • Export Citation
  • Easterling, D. R., and Coauthors, 2017: Precipitation change in the United States. Climate Science Special Report: Fourth National Climate Assessment, D. J. Wuebbles et al., Eds., U.S. Global Change Research Program Rep., 207–230, https://doi.org/10.7930/J0H993CC.

    • Crossref
    • Export Citation
  • Eyring, V., S. Bony, G. A. Meehl, C. A. Senior, B. Stevens, R. J. Stouffer, and K. E. Taylor, 2016: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci. Model Dev., 9, 19371958, https://doi.org/10.5194/gmd-9-1937-2016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fereday, D., R. Chadwick, J. Knight, and A. A. Scaife, 2018: Atmospheric dynamics is the largest source of uncertainty in future winter European rainfall. J. Climate, 31, 963977, https://doi.org/10.1175/JCLI-D-17-0048.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Frei, A., K. E. Kunkel, and A. Matonse, 2015: The seasonal nature of extreme hydrological events in the northeastern United States. J. Hydrometeor., 16, 20652085, https://doi.org/10.1175/JHM-D-14-0237.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gelaro, R., and Coauthors, 2017: The Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2). J. Climate, 30, 54195454, https://doi.org/10.1175/JCLI-D-16-0758.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Howarth, M. E., C. D. Thorncroft, and L. F. Bosart, 2019: Changes in extreme precipitation in the northeast United States: 1979–2014. J. Hydrometeor., 20, 673689, https://doi.org/10.1175/JHM-D-18-0155.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • IPCC, 2014: Climate Change 2014: Synthesis Report. IPCC, 151 pp., https://www.ipcc.ch/site/assets/uploads/2018/05/SYR_AR5_FINAL_full_wcover.pdf.

  • Jiang, Z., W. Li, J. Xu, and L. Li, 2015: Extreme precipitation indices over China in CMIP5 models. Part I: Model evaluation. J. Climate, 28, 86038619, https://doi.org/10.1175/JCLI-D-15-0099.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Karmalkar, A. V., J. M. Thibeault, A. M. Bryan, and A. Seth, 2019: Identifying credible and diverse GCMs for regional climate change studies—Case study: Northeastern United States. Climatic Change, 154, 367386, https://doi.org/10.1007/s10584-019-02411-y.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kopparla, P., E. M. Fischer, C. Hannay, and R. Knutti, 2013: Improved simulation of extreme precipitation in a high-resolution atmosphere model. Geophys. Res. Lett., 40, 58035808, https://doi.org/10.1002/2013GL057866.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kusunoki, S., and O. Arakawa, 2015: Are CMIP5 models better than CMIP3 models in simulating precipitation over East Asia? J. Climate, 28, 56015621, https://doi.org/10.1175/JCLI-D-14-00585.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, J., H. Chen, X. Rong, J. Su, Y. Xin, K. Furtado, S. Milton, and N. Li, 2018: How well can a climate model simulate an extreme precipitation event: A case study using the transpose-AMIP experiment. J. Climate, 31, 65436556, https://doi.org/10.1175/JCLI-D-17-0801.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Menne, M. J., I. Durre, R. S. Vose, B. E. Gleason, and T. G. Houston, 2012: An overview of the Global Historical Climatology Network–Daily database. J. Atmos. Oceanic Technol., 29, 897910, https://doi.org/10.1175/JTECH-D-11-00103.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Michelangeli, P.-A., R. Vautard, and B. Legras, 1995: Weather regimes: Recurrence and quasi stationarity. J. Atmos. Sci., 52, 12371256, https://doi.org/10.1175/1520-0469(1995)052<1237:WRRAQS>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ning, L., and R. S. Bradley, 2014: Winter climate extremes over the northeastern United States and southeastern Canada and teleconnections with large-scale modes of climate variability. J. Climate, 28, 24752493, https://doi.org/10.1175/JCLI-D-13-00750.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roller, C. D., J.-H. Qian, L. Agel, M. Barlow, and V. Moron, 2016: Winter weather regimes in the northeast United States. J. Climate, 29, 29632980, https://doi.org/10.1175/JCLI-D-15-0274.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, Y., S. Solomon, A. Dai, and R. W. Portmann, 2006: How often does it rain? J. Climate, 19, 916934, https://doi.org/10.1175/JCLI3672.1.

  • Taylor, K. E., R. J. Stouffer, and G. A. Meehl, 2012: An overview of CMIP5 and the experiment design. Bull. Amer. Meteor. Soc., 93, 485498, https://doi.org/10.1175/BAMS-D-11-00094.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Haren, R., R. J. Haarsma, G. J. V. Oldenborgh, and W. Hazeleger, 2015: Resolution dependence of European precipitation in a state-of-the-art atmospheric general circulation model. J. Climate, 28, 51345149, https://doi.org/10.1175/JCLI-D-14-00279.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
Save