Abstract

Reanalyses have increasingly improved resolution and physical representation of regional climate and so may provide useful data in many regional applications. These data are not observations, however, and their limitations and uncertainties need to be closely investigated. The ability of reanalyses to reproduce the seasonal variations of precipitation and temperature over the United States during summer, when model forecasts have characteristically weak forecast skill, is assessed. Precipitation variations are reproduced well over much of the United States, especially in the Northwest, where ENSO contributes to the large-scale circulation. Some significant biases in the seasonal mean do exist. The weakest regions are the Midwest and Southeast, where land–atmosphere interactions strongly affect the physical parameterizations in the forecast model. In particular, the variance of the Modern-Era Retrospective Analysis for Research and Applications (MERRA) is too low (extreme seasonal averages are weak), and the variability of the Interim ECMWF Re-Analysis (ERA-Interim) is affected by spurious low-frequency trends. Surface temperature is generally robust among the reanalyses examined, though; reanalyses that assimilate near-surface observations have distinct advantages. Observations and forecast error from MERRA are used to assess the reanalysis uncertainty across U.S. regions. These data help to show where the reanalysis is realistically replicating physical processes, and they provide guidance on the quality of the data and needs for further development.

1. Introduction

The characterization and understanding of climate variability at regional scales are important for both research and societal applications. Atmospheric retrospective analyses (or reanalyses) integrate a variety of observing systems with numerical models to produce a temporally and spatially consistent synthesis of data for weather and climate variability studies. While reanalyses are not observations, they provide objectively analyzed fields over the globe, including regions with minimal observations, and also for fields that are rarely or never observed. Bosilovich et al. (2013) discuss the current state of reanalysis systems and the near-term challenges that the development community is addressing. One critical use of reanalyses is to provide the lateral boundary conditions for regional climate simulations, as in the Coordinated Regional Downscaling Experiment (CORDEX; Nikulin et al. 2012) because their large-scale environment reproduces the observed circulation. Here, we assess regional variability for the United States, in the National Aeronautics and Space Administration (NASA) Modern-Era Retrospective Analysis for Research and Applications (MERRA; Rienecker et al. 2011) and other reanalyses. There have been many regional process studies using reanalyses, but this assessment aims to begin to define the limitations of reanalyses in climate monitoring and regional climate applications, considering potential needs for the National Climate Assessment (NCA). Emphasis is placed on summertime precipitation because 1) it is a difficult parameter to predict in the most difficult season (Bosilovich et al. 2009) and 2) significant observational resources exist to benchmark comparisons. Likewise, the occurrences of precipitation events, or the lack thereof, and the coinciding physical mechanisms are critical when evaluating regional extremes. Of course, another societally relevant and climatologically significant indicator is the surface air temperature, for which we will compare the reanalyses' seasonal variability and regional trends (Vose et al. 2012).

Since this assessment is intended to broadly evaluate the reanalyses (with some additional focus on MERRA), we have adopted regions of the continental United States defined by the NCA with some modifications to account for local dynamics (Fig. 1). One exception is that the NCA Great Plains (GP) region has been split into northern Great Plains (NGP) and southern Great Plains (SGP) regions (at 40°N latitude). Substantial latitudinal variations across the NCA GP region could mask smaller regional signals in the data analysis. These regions generally fit the scale of atmospheric anomalies, such as El Niño–Southern Oscillation (ENSO), that contribute to climate variability. Specific events or regional climate features may exist across the bounds of any given region, however. Some attention is given to potential connections with ENSO variations (e.g., Barlow et al. 2001; Wang et al. 2012) as well as to the trends in the data (e.g., Vose et al. 2012), although the large-scale circulation influence on U.S. precipitation also relates to variability in the Pacific and Atlantic Oceans (Wang et al. 2009; Li et al. 2012). Precipitation and temperature are the initial foci, given their relative importance in societal applications, and some connections to broad climate dynamics will also be discussed. We will also begin to extend the evaluation to the lower-tropospheric forecast errors in the reanalysis and how they vary regionally. With an understanding of MERRA's fidelity in these regions, we can decide how best to use its capability in understanding the seasonal and decadal variations for the United States and also can ascertain the usefulness of reanalyses in regional climate assessment.

Fig. 1.

Continental U.S. boundaries evaluated in this project. These are broadly related to those of the NCA with the exception that their Great Plains (GP) region is divided into northern and southern regions. Here, NE is Northeast; the rest of the expansions of the region acronyms are in the text where first referenced. In some calculations, “US” will refer to the accumulation of all area in these regions.

Fig. 1.

Continental U.S. boundaries evaluated in this project. These are broadly related to those of the NCA with the exception that their Great Plains (GP) region is divided into northern and southern regions. Here, NE is Northeast; the rest of the expansions of the region acronyms are in the text where first referenced. In some calculations, “US” will refer to the accumulation of all area in these regions.

2. Data

a. Reanalyses

Reanalyses have a long track record for providing information on climate variations and for the evaluation of climate models. Although reductions in model biases, improvements in data assimilation, and increases in resolution have improved the latest reanalyses, significant issues remain, so that validation and background studies are required before accepting physical interpretation of results. Variations in the observing system itself can cause spurious variations in the reanalysis time series (Onogi et al. 2007; Saha et al. 2010; Dee et al. 2011; Bosilovich et al. 2011; Robertson et al. 2011). Such features have been noted in the three latest global reanalyses for the satellite era. Most of our assessment will focus on MERRA (Rienecker et al. 2011), but we also make some comparisons with recent satellite-data reanalyses, the European Centre for Medium-Range Weather Forecasts (ECMWF) Interim Re-Analysis (ERA-Interim; Dee et al. 2011), and the Climate Forecast System Reanalysis (CFSR; Saha et al. 2010). For this study, we have worked with the latest ERA-Interim data (at ¾° resolution), which begin in 1979 and continue forward in near–real time. At the time of this study, CFSR-processed monthly mean data are not archived beyond 2009.

Bosilovich et al. (2009) evaluated radiative fluxes and precipitation from eight operational analyses and reanalyses systems. Over the Mississippi River basin, the quality of reanalyses' seasonal precipitation degrades noticeably in summer, as has also been seen in numerical predictions. Summertime total precipitation is governed more by local convective elements and mesoscale convection than by large-scale well-resolved dynamical processes. Yet, extreme summertime climate variability often manifests through devastating precipitation extremes (e.g., drought and flood). Given the importance of summertime precipitation and its uncertainty, this assessment of MERRA, alongside other recent reanalyses, begins by evaluating the skill in summertime precipitation variability. Near-surface temperature is more robust, however, both in observational records and reanalyses, and therefore we also consider the reanalyses' representation of regional surface temperature variability (Simmons et al. 2004, 2010; Vose et al. 2012; Decker et al. 2012; Wang and Zeng 2013).

b. MERRA

The NASA Modern-Era Retrospective Analysis for Research and Applications was developed to provide a continuous record of observational analyses extending from the beginnings of the modern satellite data record through the NASA Earth Observing System (EOS) research satellite operations. In general, MERRA has some improvements in the large-scale hydrologic cycle and global precipitation, especially over tropical oceans (Bosilovich et al. 2011); there are limitations over land regions at the detail required for hydrological processes, however (Reichle et al. 2011). At the outset, the disparate nature of the observing system between the EOS period and the historical satellite data record was known to be a challenge for reanalyses, as noted in the apparent impact of Special Sensor Microwave Imager total column water on the Japanese 25-Year Reanalysis (Onogi et al. 2007; Bosilovich et al. 2008). The observations included in a reanalysis are ultimately critical to its depiction of the weather and climate. In addition, background model biases also contribute to uncertainty in the eventual reanalyses data.

Rienecker et al. (2011) provide the MERRA project overview, including discussions of the model and analysis (and radiative transfer model). There is also a listing of input observations and some discussions of the interactions of the observations with the model through the assimilation. The assimilated observations used for MERRA benefitted from previous National Centers for Environmental Prediction reanalyses and the 40-Year ECMWF Re-Analysis (ERA-40) (Kalnay et al. 1996; Uppala et al. 2005). The MERRA analysis is performed every 6 h, where a background forecast contributes to the eventual analyzed state. The same six hours are then forecast again, but with incremental updates from the analyzed states (e.g., Bloom et al. 1996), which determine the initial condition for the next background forecast (Rienecker et al. 2011). The incremental updates represent the total influence of the observations on the reanalysis, as they are incorporated through the forecast model state budgets (Bosilovich et al. 2011), and have a strong influence on the regional climatological behavior (e.g., Roads et al. 2002). In the data assimilation, however, each observation is analyzed individually, and forecast and analysis errors for each observation are produced and saved in the assimilation output. These data contain insight into the observations, model, and data assimilation. Haimberger (2007) used these data from ERA-40 to detect discontinuities in the radiosonde observation record and has determined corrections to the radiosonde data. These corrections were applied to both ERA-Interim and MERRA (Dee et al. 2011; Rienecker et al. 2011).

One way to evaluate a reanalysis is to consider the forecast error [observation minus forecast (OmF)]. To efficiently get at the assimilated observation and forecast error, we utilize a recent data product for MERRA called the Gridded Innovations and Observations (GIO). To compare multiple instruments and observing systems more easily and to simplify the data access, the assimilated observations and innovations have been binned to the native MERRA analysis grid in space and time (⅔° longitude by ½° latitude, 42 levels, and 6-hourly synoptic times). The data files include the observation, the forecast error (OmF), and analysis error [observation minus analysis (OmA)] as well as the data count and standard deviation for the bin. Each observing system is preserved in its own record, including radiance observations (by instrument and channel). These data can be used to compute the regional forecast error for a number of prognostic states. Although some detail is lost in the binning, the data volume is reduced and the uniform format of all observing systems greatly improves data accessibility.

c. Surface observations

The Climate Prediction Center (CPC) daily gauge analysis for the conterminous United States provides the precipitation benchmark (Chen et al. 2008; Xie et al. 2007). Precipitation gauges have been analyzed to ¼° resolution over the continental United States, and gaps have been filled. Most of the analysis here is done on the grid of the original data (e.g., maps), but in cases in which data are compared or differenced, CPC is interpolated to the reanalysis grid. The daily data have been averaged to seasonal means for the time series analysis. We primarily focus on the seasonal averages of the daily data, concentrating on the interannual variability. The University of East Anglia Climate Research Unit (CRU) dataset (Mitchell and Jones 2005), version 3.1, provides the baseline for interannual variability and seasonal statistics of near-surface temperature, following the method of Simmons et al. (2004). (The CRU dataset was accessed online in February of 2012 from the National Centre for Atmospheric Science British Atmospheric Data Centre at http://badc.nerc.ac.uk/view/badc.nerc.ac.uk__ATOM__dataent_1256223773328276. Additional documentation and descriptions of the dataset are also available at this site.)

3. Summertime precipitation

Figure 2a uses the time- and area-averaged precipitation for CPC gauge observations and reanalyses for the continental U.S. regions to summarize the climate and variability of each region. All considered reanalyses have high summer biases in the southeastern United States. MERRA's overestimate over the northern Great Plains is the largest bias, although the summer dry bias in the midwestern region is likely significant as well. Despite the high NGP precipitation bias, MERRA shows very high correlation with the observed seasonal anomalies, while ERA-Interim shows a markedly lower correlation (Fig. 2b). The standard deviation (Fig. 2c) is also provided to show the extent to which reanalyses can vary. The different reanalyses are not consistent in this metric, and we note in particular that MERRA standard deviation in several of the regions is smaller than observed.

Fig. 2.

(a) A comparison of regional precipitation (mm day−1) from CPC gauge observations with reanalyses, using time and area averages for each region and JJA. (b) Correlation of the 30 years of JJA seasonal area-average precipitation anomalies of the reanalyses with CPC gauge observations (where significance correlations are 0.31 ~ 90%, 0.36 ~ 95%, and 0.46 ~ 99%). (c) Standard deviation of the area-averaged time series for each region's JJA precipitation anomalies.

Fig. 2.

(a) A comparison of regional precipitation (mm day−1) from CPC gauge observations with reanalyses, using time and area averages for each region and JJA. (b) Correlation of the 30 years of JJA seasonal area-average precipitation anomalies of the reanalyses with CPC gauge observations (where significance correlations are 0.31 ~ 90%, 0.36 ~ 95%, and 0.46 ~ 99%). (c) Standard deviation of the area-averaged time series for each region's JJA precipitation anomalies.

Figure 3 contrasts the time series of precipitation anomalies in the Northwest (NW) with those in the Midwest (MW); the former with the highest temporal correlation, the latter with the lowest. In NW, all reanalyses track the observed June–August (JJA) precipitation anomalies remarkably well. The mean flow of JJA moisture is predominantly eastward from the Pacific Ocean, so that the dynamical control of the precipitation is important. During summer in the MW region, recycling ratios increase (Bosilovich and Schubert 2001), thereby increasing the dependence of precipitation on the boundary layer parameterization, the land model (through its representation of evaporation), and the past rainfall and snowmelt. In addition, MW reanalysis precipitation calculations depend more heavily on the convection parameterization of the model. In the MW region, moisture transport has played an important role in extreme anomalies. Here, we see that MERRA underestimates both the anomalies from the 1988 drought and the 1993 flooding. This is generally true of the NGP region as well (not shown) and is consistent with the weaker standard deviation in Fig. 2. The other reanalyses also struggle in this region, with either false extremes or underrepresentation of extreme events.

Fig. 3.

Contrasting time series of the JJA precipitation anomalies in the NW and MW regions of Fig. 1.

Fig. 3.

Contrasting time series of the JJA precipitation anomalies in the NW and MW regions of Fig. 1.

While the mean and correlation of the seasonal anomalies are reasonable in some regions, the reanalyses in Figs. 3 and 4 display some trends that are unrealistic when compared with observations; for example, note the ERA-Interim trend in the MW. Simmons et al. (2010) found that this trend is more related to a declining shift that starts in the early 1990s and considered that it may be related to the prescribed SSTs used in ERA-Interim. In addition, CFSR's Southeast (SE) region experiences a dramatic downward shift in precipitation in 1997–98 (not shown), which leads to a large negative value when computing a linear regression in time (Fig. 4). In the MW region, MERRA and ERA-Interim both experience systematic decreasing trends in precipitation over the period, to varying degrees. However, ERA-Interim's decreasing trends extend across the continent at magnitudes that are much greater than observed. The NW region's precipitation trends in all of the reanalyses are comparable to observations.

Fig. 4.

As in Fig. 2, but for the precipitation trend over each region [mm day−1 (10 yr)−1] computed from the seasonal area average. Statistical significance at 95% confidence is indicated by black-outlined bars.

Fig. 4.

As in Fig. 2, but for the precipitation trend over each region [mm day−1 (10 yr)−1] computed from the seasonal area average. Statistical significance at 95% confidence is indicated by black-outlined bars.

The high degree of correlation between all of the reanalyses and the CPC observational analysis in the NW is noteworthy, especially considering the lack of correlation in other regions. Figure 5 shows the spatial distribution of time correlation between MERRA seasonal precipitation and CPC. The very high correlations extend east beyond the NW region and into NGP. Many areas show positive correlation (significantly different than zero at the 99% level), with the MW region primarily having areas of very low correlation. There are systematic biases across all of the reanalyses that lead to the similarities in the time series results, and explaining the mechanisms may lead to improvements in the reanalyses.

Fig. 5.

Correlation of JJA seasonal mean precipitation from MERRA and CPC gauge observations for the period 1979–2010. The white contour delineates correlations that are significantly different from zero at the 99% level, so that values greater than 0.46 indicate high statistical significance.

Fig. 5.

Correlation of JJA seasonal mean precipitation from MERRA and CPC gauge observations for the period 1979–2010. The white contour delineates correlations that are significantly different from zero at the 99% level, so that values greater than 0.46 indicate high statistical significance.

4. ENSO variability

While summer teleconnections are not as strong over the United States as those in the winter, Barlow et al. (2001) found that the North American hydroclimate responds to several modes of variability in the Pacific Ocean, including ENSO, in that the northwestern United States receives increased precipitation during El Niño conditions. Using observations and ensembles of global model simulations, Wang et al. (2012) find that the phase of ENSO, particularly the decaying warm phase, strongly affects the Great Plains seasonal precipitation extremes. This was also reproduced in numerical models, demonstrating potential predictability of extreme precipitation by El Niño. In addition, when multidecadal records of data are evaluated Pacific decadal variability is seen to play a role in summertime precipitation (Ting and Wang 1997; Wang et al. 2009). In this section, we evaluate the ability of reanalyses to reproduce this low-frequency variability for the United States. Reanalyses are expected to implicitly include realistic ENSO variability where atmospheric observational data coverage is sufficient. On the other hand, the physical processes are only guided by the observations and rely also on model parameterizations, especially in summertime.

By following Barlow et al. (2001) in looking at the northwestern United States, we find that the observed NW region summer precipitation is positively correlated with Niño-3.4 index (e.g., Trenberth 1997), as in Fig. 6. The Niño-3.4 correlation is fairly weak when comparing JJA precipitation with JJA Niño-3.4, however. The correlation is strongest with the antecedent springtime [March–May (MAM)] values of the index (Fig. 6). The reanalyses follow the observed pattern, with the exception that their correlations are somewhat stronger than the observations, and all of the reanalyses are closer to each other than to the observations. To emphasize the teleconnections, subsequent comparisons will focus on the relationship of summertime (JJA) precipitation and springtime (MAM) Niño-3.4 index.

Fig. 6.

Time correlation of summer seasonal precipitation (as in Fig. 3) with Niño-3.4 seasonal indices. The Niño-3.4 seasons are prior to the JJA season's precipitation. Significant correlations are 0.31 ~ 90%, 0.36 ~ 95%, and 0.46 ~ 99%.

Fig. 6.

Time correlation of summer seasonal precipitation (as in Fig. 3) with Niño-3.4 seasonal indices. The Niño-3.4 seasons are prior to the JJA season's precipitation. Significant correlations are 0.31 ~ 90%, 0.36 ~ 95%, and 0.46 ~ 99%.

Figure 7 shows the spatial distributions of the correlation of summer precipitation with spring Niño-3.4 in CPC gauge observations and the three reanalyses considered here. The gauge correlations are highest in the NW and NGP regions, with apparently little significant correlation elsewhere, in agreement with the results from Barlow et al. (2001). MERRA tracks the observed pattern well, with positive and significant correlations in the NW and NGP regions. Similar to the correlations presented in Fig. 6 for the NW, the areas of positive correlations have generally larger values when compared with observations. The stronger correlations, when compared with the gauge observations, are also found in other reanalyses and generally span all of the continental U.S. subregions considered here (Fig. 8). In particular, when comparing the aggregate United States, the large-scale correlation with Niño-3.4 is apparent. MERRA is correlated with ENSO much more than are the observations (more than the 99% confidence, whereas observation correlation is less than 90%). The broad result is that summertime precipitation and attendant physical processes are too closely related to ENSO in comparison with observations. If the correlation is derived from the coarse scale of resolved circulations and the inability to explicitly simulate fine scales of convection, it would effectively act as a filter. It is possible that inadequate land–atmospheric interactions have some effect, for example, in the MW region during summer, as these can modulate the large-scale forcing in a region (e.g., Mei and Wang 2011). More analysis is needed to determine whether there is a tendency for the reanalysis systems to draw energy away from longer modes of variability, such as the North Pacific decadal oscillation, which should also have some influence (Higgins et al. 2007). The reanalyses have some spatial variability of the relationship across the Southwest (SW) region, and the size of that region likely affects the time series statistics (Figs. 7 and 8).

Fig. 7.

Spatial distribution of the correlation between summertime precipitation and spring Niño-3.4. The colors indicate level of significance of the correlation (0.31 ~ 90%, 0.36 ~ 95%, and 0.46 ~ 99%). CFSR and Interim show patterns that are similar to that of MERRA.

Fig. 7.

Spatial distribution of the correlation between summertime precipitation and spring Niño-3.4. The colors indicate level of significance of the correlation (0.31 ~ 90%, 0.36 ~ 95%, and 0.46 ~ 99%). CFSR and Interim show patterns that are similar to that of MERRA.

Fig. 8.

Correlation between summertime precipitation and springtime Niño-3.4 index for each of the reanalyses and gauge observations, in each region and the whole of the United States. Significant correlations are 0.31 ~ 90%, 0.36 ~ 95%, and 0.46 ~ 99%.

Fig. 8.

Correlation between summertime precipitation and springtime Niño-3.4 index for each of the reanalyses and gauge observations, in each region and the whole of the United States. Significant correlations are 0.31 ~ 90%, 0.36 ~ 95%, and 0.46 ~ 99%.

5. Surface temperature

Near-surface atmospheric temperature is another essential climate variable and is important for analysis of summertime extremes such as drought and heat waves. In reanalyses and global models, this quantity is still closely related to the model parameterizations, situated between the state variables of temperature at the surface and at the lowest atmospheric model level. As such, model uncertainty will play a role in the representation of climate variability in reanalyses (Willett et al. 2012). Also, mean temperatures in reanalyses represent an integration of temperature for the period, whereas at many observation stations mean temperature is determined by the average of maximum and minimum daily temperature (Wang and Zeng 2013), and this situation may lead to some uncertainty in the comparison. The representation of temperature in reanalyses generally appears to be more robust than precipitation, likely because atmospheric temperature is assimilated from both radiosonde and satellite sources regularly. As mentioned in section 2c, the CRU atmospheric temperature dataset (version 3.1) provides the baseline for interannual variability and seasonal statistics. Figure 9 shows the mean temperature for each of the U.S. regions. Many regional summertime biases are less than 1 K, and none are more than 2 K. The temperature correlations between reanalyses and observed time series are also very high relative to those of precipitation (note the difference in scales for Figs. 2 and 9).

Fig. 9.

Regional temperature comparison between CRU, version 3.1, station observations and reanalyses: (a) Temporal and areal averages (K) for each region for JJA, (b) correlation of the 30 years of JJA seasonal area-average temperature anomalies of the reanalyses with observations (all values are statistically significant at greater than 99%), and (c) the corresponding standard deviations (K).

Fig. 9.

Regional temperature comparison between CRU, version 3.1, station observations and reanalyses: (a) Temporal and areal averages (K) for each region for JJA, (b) correlation of the 30 years of JJA seasonal area-average temperature anomalies of the reanalyses with observations (all values are statistically significant at greater than 99%), and (c) the corresponding standard deviations (K).

One noticeable feature in the reanalyses' temperature is that across all of the regions the ERA-Interim air temperature correlates with CRU observed analysis at values in the high 90s. This is a direct result of the ERA system's inclusion of both near-surface atmospheric temperature and water vapor observations to constrain soil moisture (Simmons et al. 2004, 2010; Dee et al. 2011). CFSR also uses precipitation observations over land to better constrain their soil moisture. MERRA has no direct land data assimilation and so relies on the model physics (and atmospheric profile assimilation) for the representation of near-surface atmospheric temperature. To illustrate the interannual variability of mean temperature anomalies, the time series of SE and NW temperature are presented in Fig. 10. In the SE, where MERRA has the lowest regional correlation, many of the interannual extremes are still represented well. Further investigation of certain years, such as 2000, may point to correctable problems in the system.

Fig. 10.

Time series of regional temperature anomalies (K) determined from the CRU observations and reanalyses.

Fig. 10.

Time series of regional temperature anomalies (K) determined from the CRU observations and reanalyses.

The determination of temperature trends from the station observations is complicated by variations in station locations and also by the changing numbers of stations used (Vose et al. 2012; Mitchell and Jones 2005). Observed trends have been studied and quality controlled so that they are a reliable comparison for reanalyses. All of the reanalyses and observations show increasing trends, but some regions exhibit distinct differences from CRU, version 3.1. ERA-Interim reproduces the observed trend in most regions (Fig. 11), but despite the additional land assimilation it does not identically reproduce the observed trends. The ERA-Interim analysis of the surface temperature limits feedback of their persistent precipitation trend deficiency from degrading the reanalysis surface temperature (Fig. 4), compensating for the model biases with analysis tendencies. For MERRA, the unrealistic precipitation trends in MW and SE are likely a major contribution to the degraded surface temperature representations. The reanalyses generally overestimate the upward trend of temperature, but updated quality adjustments to the station records lead to increased upward trend estimates (Vose et al. 2012). Despite the apparent robustness in variability of near-surface air temperature, trends from reanalyses need to be considered at least as carefully as the observational record because of the modeling components of the system and its uncertainty. In addition, the analysis of the observational record has distinct benefit for the ERA-Interim record of the near-surface temperature but presumably does not permit the feedback from other components of the Earth system—for example, atmospheric precipitation and land fluxes. Even without surface data assimilation, MERRA seasonal variations and correlation with observations of near-surface temperature remain reasonable, although variance of seasonal temperature is overestimated.

Fig. 11.

Trends [K (10 yr)−1] for the area-average JJA seasonal temperature anomalies in the CRU observation and reanalyses. Black-outlined bars indicate statistical significance at 95% confidence.

Fig. 11.

Trends [K (10 yr)−1] for the area-average JJA seasonal temperature anomalies in the CRU observation and reanalyses. Black-outlined bars indicate statistical significance at 95% confidence.

6. Water vapor forecast error

For the most part, the previous discussion considers the verification of the reanalysis data and relationships to physical phenomena. The interaction of the analysis system with the model forecast is a crucial factor in the results, however, and is not typically addressed with reanalyses. Rienecker et al. (2011) provided evaluation of the global forecast errors from MERRA derived from the GIO ancillary dataset. This includes the observations assimilated in MERRA, the forecast error OmF used to determine the analysis increment, and the analysis error OmA. Consider also that negative OmF represents the model bias relative to each assimilated observation. The forecast and analysis included here have been interpolated in space and time to the observations coordinates; these data are then binned to the MERRA grid for ease of access and analysis. In employing these data here, we can evaluate the forecast errors in the various regions and consider the forecast errors of the analysis state fields in comparison with the model derived fields such as precipitation (and its error).

Figure 12a shows the mean JJA forecast error of 850-hPa water vapor and temperature for 1979–2009 in each of the regions. Given the precipitation comparison discussed previously, it is not surprising to see NGP and SE with some of the highest mean errors. In general, these results indicate that the forecast is systematically drier than is observed. As the reanalysis integrates forward, the analyzed observations would tend to add water, increasing the water available for precipitation generation. The MW region is somewhat different, where the precipitation is underestimated and the analysis still, in general, adds water. Given the lower correlation of the MERRA MW precipitation with observations (Fig. 2), however, there may be other factors related to the precipitation generation that require investigation (such as land–atmosphere interactions or mesoscale convective systems). It is also apparent that the mean forecast temperature biases are generally small (less than 0.1 K in magnitude in most regions) except for SGP (warm bias) and SW (cold bias).

Fig. 12.

(a) Regional mean forecast error OmF averaged for JJA and (b) the regional RMS of the forecast error for 850-hPa temperature (K) and water vapor (g kg−1). Each is computed for each season from 1979 to 2009 from the 6-hourly data count and is then averaged in time.

Fig. 12.

(a) Regional mean forecast error OmF averaged for JJA and (b) the regional RMS of the forecast error for 850-hPa temperature (K) and water vapor (g kg−1). Each is computed for each season from 1979 to 2009 from the 6-hourly data count and is then averaged in time.

The root-mean-square (RMS) of the OmF provides an estimate of uncertainty of the forecasts. Figure 12b shows the RMS of the forecast error of water vapor and temperature in each region. Somewhat noticeable is that the SW shows some larger values when compared with other regions than were seen with the forecast bias because of compensating biases across the region. Some further consideration will be given to SW later in this section. The largest water vapor forecast errors occur in SE and NGP.

The MW region has some large errors in MERRA precipitation relative to gauge observations. Figure 13a compares the MERRA seasonal differences of precipitation with the water vapor bias (where bias is simply the negative of OmF). On average, both the water vapor forecast and the precipitation in the MW region have a dry bias. The drier the water vapor bias is, the more water is added to the analysis; this increases the precipitation but does not overcome the negative bias of precipitation. Figure 13b compares the climate anomaly of water vapor with the forecast bias for the MW region. The driest forecast biases occur in years with the wettest anomalies in the region. In dry years, the forecast bias is still dry so that the analysis still acts to add water and increase the precipitation. The reduced variability of MERRA precipitation seems to relate back to a dry bias in the lower troposphere in the MW region. The results for the NGP region are similar. The large positive precipitation biases in SE did not show a relationship with the water vapor forecast bias, nor did the small NW region precipitation variations.

Fig. 13.

The MW JJA (a) seasonal precipitation bias and (b) observed seasonal anomalies of 850-hPa water vapor in comparison with 850-hPa water vapor forecast bias (−OmF). The water vapor OmF and observed anomalies are derived from radiosonde observations assimilated in MERRA. The black line indicates the linear fit of the data and is included for reference only.

Fig. 13.

The MW JJA (a) seasonal precipitation bias and (b) observed seasonal anomalies of 850-hPa water vapor in comparison with 850-hPa water vapor forecast bias (−OmF). The water vapor OmF and observed anomalies are derived from radiosonde observations assimilated in MERRA. The black line indicates the linear fit of the data and is included for reference only.

The SW region, however, appears to have a different set of issues in its uncertainty. Unlike MW and NGP, the precipitation bias is low when the 850-hPa water vapor bias is dry (the anomaly correlation is positive; Fig. 14a). So, the water vapor is dry and the analysis should add water, but the precipitation bias is greatest with smaller-magnitude forecast errors. Observed wet seasons tend to have dry model 850-hPa water vapor biases, as in the MW and NGP regions (Fig. 14b). It may be premature to read too much into the comparison, however. Although the general comparison of the seasonal MERRA SW region precipitation with gauge observations is reasonable, the region includes the California coast, mountains, and deserts, and the initiation of the North American monsoon generally occurs during JJA (Higgins et al. 1999). Further refinement of the area and seasonality of the region's water cycle is likely needed to more clearly explain the relationship of the precipitation biases to the forecast error. Even so, there appears to be a clear deficiency in the MERRA hydrologic cycle across this region. Figure 15a compares the observed seasonal anomalies for 850-hPa water vapor and precipitation, where increased precipitation accompanies increased water vapor. The MERRA forecast water vapor and precipitation provide no such clear relationship (Fig. 15b). Having available the comparable model forecast and observations enables the identification of such issues, however, and can direct subsequent research efforts to improve the model.

Fig. 14.

As in Fig. 13, but for SW.

Fig. 14.

As in Fig. 13, but for SW.

Fig. 15.

(a) The SW JJA 850-hPa observed water vapor anomaly in comparison with the observed precipitation anomaly. (b) As in (a), but for MERRA JJA SW seasonal anomalies.

Fig. 15.

(a) The SW JJA 850-hPa observed water vapor anomaly in comparison with the observed precipitation anomaly. (b) As in (a), but for MERRA JJA SW seasonal anomalies.

7. Summary and conclusions

Reanalyses offer many advantages to climate research and monitoring, but, as with observation data, uncertainties exist. In evaluating the U.S. summer regional climate from reanalyses, we characterize the ability of reanalyses to represent summertime precipitation and temperature variability throughout the modern satellite data period. While summertime precipitation is one of the most difficult physical processes to model, we do find a certain amount of interannual variability represented realistically in the reanalyses. The NW and NGP regions are best represented, owing to large-scale controls from springtime ENSO variations, whereas the large-scale influence is overly represented in the interannual variability of other regions in reanalyses. For example, all of the reanalyses overdo the correlation between ENSO and precipitation of the whole continental United States when compared with gauge measurements. This is especially true for MERRA, and it should be noted that related quantities, such as clouds and radiation, will likely exhibit similar relationship to ENSO. Given that strong connection, it is important to note that MERRA's precipitation variability is found to be weaker than is observed, leading to weaker seasonal precipitation anomalies for 1993 (pluvial) and 1988 (drought). While atmospheric moisture transport should be reasonably well represented in reanalyses (Higgins et al. 1997), local land interactions and mesoscale convective circulations may require further attention. Interannual correlations with observations of surface temperatures are more robust than for precipitation, owing to the assimilated air temperatures and the continuous nature of the field (whereas summertime precipitation can be spatially discontinuous). Since ERA-Interim assimilates near-surface temperatures, it is likewise able to reproduce the variability closely—even trends. In general, reanalysis precipitation trends have little fidelity with observations, however. As an example, MERRA's underestimate of seasonal precipitation variability and ERA-Interim's broad decreasing precipitation trend across the United States are considerable limitations for regional climate applications.

Determination of the uncertainty of one, or many, reanalyses is an outstanding research issue, especially considering that many quantities are related to the background model forecast (Kalnay et al. 1996). The uncertainty of a reanalysis can be estimated with several key components. First, independent observations, with sufficient quality, are needed to provide a benchmark. Further, several independently derived reanalyses also provide a range of comparisons. Last, reanalyses are derived from the direct comparison of model prediction with high-quality observations. Statistics and diagnostics from the data assimilation procedure, including analysis increments and background forecast error, can provide useful guides to the reanalysis quality. At this time, not all reanalyses provide these data alongside standard output in an easily accessible manner, however. In this study, the forecast errors were examined regionally for the 30-yr climatological output of MERRA. The assimilated observations and forecast error can be used to assess the processes in the model forecast segment of the reanalysis. Because reanalyses can contribute to the U.S. regional climate assessment, their uncertainties, strengths, and weaknesses need to be better quantified.

Acknowledgments

This work was supported as a NASA agency contribution to the National Climate Assessment. Partial support was also contributed through the NASA Energy and Water Cycle Studies Program (NEWS). MERRA was developed and produced through the NASA Modeling, Analysis, and Prediction (MAP) program. Arlindo da Silva developed the Gridded Innovations and Observations (GIO) data used to evaluate the MERRA forecast error. The author greatly appreciates useful discussions with Franklin R. Robertson and Siegfried D. Schubert on the results of the study. Comments and suggestions from Russel Vose and an anonymous reviewer contributed greatly to the final form of this manuscript.

REFERENCES

REFERENCES
Barlow
,
M.
,
S.
Nigam
, and
E. H.
Berbery
,
2001
:
ENSO, Pacific decadal variability, and U.S. summertime precipitation, drought, and stream flow
.
J. Climate
,
14
,
2105
2128
.
Bloom
,
S. C.
,
L. L.
Takacs
,
A. M.
da Silva
, and
D.
Ledvina
,
1996
:
Data assimilation using incremental analysis updates
.
Mon. Wea. Rev.
,
124
,
1256
1271
.
Bosilovich
,
M. G.
, and
S. D.
Schubert
,
2001
:
Precipitation recycling over the central United States as diagnosed from the GEOS1 Data Assimilation System
.
J. Hydrometeor.
,
2
,
26
35
.
Bosilovich
,
M. G.
,
J.
Chen
,
F. R.
Robertson
, and
R. F.
Adler
,
2008
:
Evaluation of global precipitation in reanalyses
.
J. Appl. Meteor. Climatol.
,
47
,
2279
2299
.
Bosilovich
,
M. G.
,
D.
Mocko
,
J. O.
Roads
, and
A.
Ruane
,
2009
:
A multimodel analysis for the Coordinated Enhanced Observing Period (CEOP)
.
J. Hydrometeor.
,
10
,
912
934
.
Bosilovich
,
M. G.
,
F. R.
Robertson
, and
J.
Chen
,
2011
:
Global energy and water budgets in MERRA
.
J. Climate
,
24
,
5721
5739
.
Bosilovich
,
M. G.
,
J.
Kennedy
,
D.
Dee
,
R.
Allan
, and
A.
O'Neill
,
2013
: On the reprocessing and reanalysis of observations for climate. Climate Science for Serving Society: Research, Modeling and Prediction Priorities, G. R. Asrar and J. W. Hurrell, Eds., Springer, 51–71.
Chen
,
M.
,
W.
Shi
,
P.
Xie
,
V. B. S.
Silva
,
V. E.
Kousky
,
R.
Wayne Higgins
, and
J. E.
Janowiak
,
2008
:
Assessing objective techniques for gauge-based analyses of global daily precipitation
.
J. Geophys. Res.
,
113
,
D04110
,
doi:10.1029/2007JD009132
.
Decker
,
M.
,
M. A.
Brunke
,
Z.
Wang
,
K.
Sakaguchi
,
X.
Zeng
,
M. G.
Bosilovich
,
2012
:
Evaluation of the reanalysis products from GSFC, NCEP, and ECMWF using flux tower observations
.
J. Climate
,
25
,
1916
1944
.
Dee
,
D.
, and
Coauthors
,
2011
:
The ERA-Interim reanalysis: Configuration and performance of the data assimilation system
.
Quart. J. Roy. Meteor. Soc.
,
137
,
553
597
.
Haimberger
,
L.
,
2007
:
Homogenization of radiosonde temperature time series using innovation statistics
.
J. Climate
,
20
,
1377
1403
.
Higgins
,
R. W.
,
Y.
Yao
,
E. S.
Yarosh
,
J. E.
Janowiak
, and
K. C.
Mo
,
1997
:
Influence of the Great Plains low-level jet on summertime precipitation and moisture transport over the central United States
.
J. Climate
,
10
,
481
507
.
Higgins
,
R. W.
,
Y.
Chen
, and
A. V.
Douglas
,
1999
:
Interannual variability of the North American warm season precipitation regime
.
J. Climate
,
12
,
653
680
.
Higgins
,
R. W.
,
V. B. S.
Silva
,
W.
Shi
, and
J.
Larson
,
2007
:
Relationships between climate variability and fluctuations in daily precipitation over the United States
.
J. Climate
,
20
,
3561
3579
.
Kalnay
,
E.
, and
Coauthors
,
1996
:
The NCEP/NCAR 40-Year Reanalysis Project
.
Bull. Amer. Meteor. Soc.
,
77
,
437
431
.
Li
,
L.
,
W.
Li
, and
Y.
Kushnir
,
2012
:
Variation of the North Atlantic subtropical high western ridge and its implication to southeastern US summer precipitation
.
Climate Dyn.
,
39
,
1401
1412
.
Mei
,
R.
, and
G.
Wang
,
2011
:
Impact of sea surface temperature and soil moisture on summer precipitation in the United States based on observational data
.
J. Hydrometeor.
,
12
,
1086
1099
.
Mitchell
,
T. D.
, and
P. D.
Jones
,
2005
:
An improved method of constructing a database of monthly climate observations and associated high-resolution grids
.
Int. J. Climatol.
,
25
,
693
712
.
Nikulin
,
G.
, and
Coauthors
,
2012
:
Precipitation climatology in an ensemble of CORDEX-Africa regional climate simulations
.
J. Climate
,
25
,
6057
6078
.
Onogi
,
K.
, and
Coauthors
,
2007
:
The JRA-25 Reanalysis
.
J. Meteor. Soc. Japan
,
85
,
369
432
.
Reichle
,
R. H.
,
R. D.
Koster
,
G. J. M.
De Lannoy
,
B. A.
Forman
,
Q.
Liu
,
S. P. P.
Mahanama
, and
A.
Touré
,
2011
:
Assessment and enhancement of MERRA land surface hydrology estimates
.
J. Climate
,
24
,
6322
6338
.
Rienecker
,
M. R.
, and
Coauthors
,
2011
:
MERRA: NASA's Modern-Era Retrospective Analysis for Research and Applications
.
J. Climate
,
24
,
3624
3648
.
Roads
,
J. O.
,
M.
Kanamitsu
, and
R.
Stewart
,
2002
:
CSE Water and Energy Budgets in the NCEP-DOE Reanalysis II
.
J. Hydrometeor.
,
3
,
227
248
.
Robertson
,
F. R.
,
M. G.
Bosilovich
,
J.
Chen
, and
T. L.
Miller
,
2011
:
The effect of satellite observing system changes on MERRA water and energy fluxes
.
J. Climate
,
24
,
5197
5217
.
Saha
,
S.
, and
Coauthors
,
2010
:
The NCEP Climate Forecast System Reanalysis
.
Bull. Amer. Meteor. Soc.
,
91
,
1015
1057
.
Simmons
,
A. J.
, and
Coauthors
,
2004
:
Comparison of trends and low-frequency variability in CRU, ERA-40, and NCEP/NCAR analyses of surface air temperature
.
J. Geophys. Res.
,
109
,
D24115
,
doi:10.1029/2004JD005306
.
Simmons
,
A. J.
,
K. M.
Willett
,
P. D.
Jones
,
P. W.
Thorne
, and
D. P.
Dee
,
2010
:
Low-frequency variations in surface atmospheric humidity, temperature, and precipitation: Inferences from reanalyses and monthly gridded observational data sets
.
J. Geophys. Res.
,
115
,
D01110
,
doi:10.1029/2009JD012442
.
Ting
,
M.
, and
H.
Wang
,
1997
:
Summertime U.S. precipitation variability and its relation to Pacific sea surface temperature
.
J. Climate
,
10
,
1853
1873
.
Trenberth
,
K. E.
,
1997
:
The definition of El Niño
.
Bull. Amer. Meteor. Soc.
,
78
,
2771
2777
.
Uppala
,
S. M.
, and
Coauthors
,
2005
:
The ERA-40 Re-Analysis
.
Quart. J. Roy. Meteor. Soc.
,
131
,
2961
3012
.
Vose
,
R. S.
,
S.
Applequist
,
M. J.
Menne
,
C. N.
Williams
Jr.
, and
P.
Thorne
,
2012
:
An intercomparison of temperature trends in the U.S. Historical Climatology Network and recent atmospheric reanalyses
.
Geophys. Res. Lett.
,
39
,
L10703
,
doi:10.1029/2012GL051387
.
Wang
,
A.
, and
X.
Zeng
,
2013
:
Development of global hourly 0.5° land surface air temperature datasets
.
J. Climate
,
in press
.
Wang
,
H.
,
S.
Schubert
,
M.
Suarez
,
J.
Chen
,
M.
Hoerling
,
A.
Kumar
,
P.
Pegion
,
2009
:
Attribution of the seasonality and regionality in climate trends over the United States during 1950–2000
.
J. Climate
,
22
,
2571
2590
.
Wang
,
H.
,
A.
Kumar
,
W.
Wang
,
B.
Jha
,
2012
:
U.S. summer precipitation and temperature patterns following the peak phase of El Niño
.
J. Climate
,
25
,
7204
7215
.
Willett
,
K. M.
,
A. J.
Dolman
,
B. D.
Hall
, and
P. W.
Thorne
,
2012
:
Global climate [in “State of the Climate in 2011”]
.
Bull. Amer. Meteor. Soc.
,
93
(
7
),
S7
S55
.
Xie
,
P.
,
A.
Yatagai
,
M.
Chen
,
T.
Hayasaka
,
Y.
Fukushima
,
C.
Liu
, and
S.
Yang
,
2007
:
A gauge-based analysis of daily precipitation over East Asia
.
J. Hydrometeor.
,
8
,
607
626
.