The authors analyze the ability of the North American Regional Climate Change Assessment Program's ensemble of climate models to simulate very heavy daily precipitation and its supporting processes, comparing simulations that used observation-based boundary conditions with observations. The analysis includes regional climate models and a time-slice global climate model that all used approximately half-degree resolution. Analysis focuses on an upper Mississippi River region for winter (December–February), when it is assumed that resolved synoptic circulation governs precipitation. All models generally reproduce the precipitation-versus-intensity spectrum seen in observations well, with a small tendency toward producing overly strong precipitation at high-intensity thresholds, such as the 95th, 99th, and 99.5th percentiles. Further analysis focuses on precipitation events exceeding the 99.5th percentile that occur simultaneously at several points in the region, yielding so-called “widespread events.” Examination of additional fields shows that the models produce very heavy precipitation events for the same physical conditions seen in the observations.
Very heavy precipitation events can cause costly and sometimes catastrophic floods in regions that may not be adequately prepared to combat them. Although details of these events may vary, such as the Midwest floods of 1993 (e.g., Kunkel et al. 1994) and 2008 (e.g., Coleman and Budikova 2010), there is no question that these events cause immense social and economic stress to those that are affected. Furthermore, very heavy precipitation events are often highly localized in time and space and can occur independently from changes in the seasonal mean, making these events difficult to predict (Gershunov 1998; Kunkel et al. 2002). Therefore, adequate simulations by climate models are vital, a need that has prompted substantial interest in the scientific community. To gain confidence in climate models' ability to simulate the environment when these very heavy precipitation events are occurring, simulations need to be compared with a variety of observed environmental fields (e.g., Gutowski et al. 2010). By using projections based on validated models, decisions and analyses with regard to future climate change can be made with greater confidence.
Here we analyze very heavy daily precipitation events as defined by Groisman et al. (2005). Part of this paper is a continuation of work done by Gutowski et al. (2008), which focused on extreme winter precipitation in the upper Mississippi River region and its potential change under enhanced global warming in one model. Here we use climate simulation produced by seven climate models for the North American Regional Climate Change Assessment Program (NARCCAP; Mearns et al. 2009, 2012). The goals of this study are to assess the ability of the NARCCAP models collectively to reproduce very heavy daily precipitation in observations, to produce very heavy precipitation for the same physical conditions as in observations, and to provide a baseline for understanding how very heavy daily precipitation and its causal processes change under enhanced greenhouse warming scenarios.
Although the study of very heavy events has increased recently, few have examined very heavy precipitation during the winter in the upper Mississippi region. This may be because of winters in this region producing less precipitation than other seasons (e.g., Dirmeyer and Kinter 2010) or a lower frequency of very heavy events compared to the rest of the year (Schumacher and Johnson 2006). However, heavy rainfall on frozen ground, with or without snow, can cause substantial flash flooding, as the surface is unable to absorb and hold moisture as effectively as in the warmer seasons (Huff and Angel 1992). Heavy rainfall can also accelerate melting of an existing snowpack, which can also contribute to flooding concerns (Changnon and Changnon 2006). In addition, heavy precipitation can fall as substantial snow, sleet, or freezing rain. As seen in this paper, some of the precipitation occurs with surface air temperatures below freezing and with amounts that could exceed 250 mm day−1 of snow in some locations. Such snowfall could cause disruptions in transportation and require substantial expenditures for snow removal, among other impacts. Any one of these winter weather events can be a source of concern to the public.
2. Observations, simulations, and analysis methods
The analysis uses the University of Washington's (UW) gridded precipitation (Maurer et al. 2002) as the primary observational data. This dataset provides observation-based precipitation on a 0.125° grid that covers all of the contiguous United States. Interpolation for the gridded dataset used the scheme of Shepard (1984) as implemented in Widmann and Bretherton (2000). The dataset also uses corrections for systematic elevation effects given by the Parameter-Elevation Regressions on Independent Slopes Model (PRISM; Daly et al. 1994). The dataset in Network Common Data Form (NetCDF) format covers the period 1950–99.
We also use the Climate Prediction Center (CPC) Unified Gauge-Based Analysis of Daily Precipitation (Higgins et al. 2000) as secondary observational data. This dataset provides observation-based precipitation on a 0.25° grid that also covers all of the contiguous United States. Interpolation for the gridded dataset used the scheme of Cressman (1959). The dataset in NetCDF format covers the period 1948–2006.
We use the UW data output as the basis for identifying days when very heavy precipitation occurs. For all other fields in the observational analysis, we used the North American Regional Reanalysis (NARR; Mesinger et al. 2006). The fields we use include 500-hPa geopotential heights, 2-m air temperature, 2-m specific humidity, and 10-m horizontal winds. These fields represent key environmental conditions during very heavy precipitation development and are also common to the output archives for all models examined here.
Model output comes from six regional climate models (RCMs) that simulated the period 1979–2003 for NARCCAP (Mearns et al. 2011): the Canadian Regional Climate Model version 4 (designated CRCM in the NARCCAP archive), the Hadley Centre Regional Model version 3 (HadRM3; HRM3 in the archive), the National Center for Atmospheric Research (NCAR) Weather Research and Forecasting Model (WRF; WRFG in the archive), the fifth-generation Pennsylvania State University–NCAR Mesoscale Model (MM5; MM5I in the archive), the International Centre for Theoretical Physics Regional Climate Model version 3 (RegCM3; RCM3 in the archive), and the Experimental Climate Prediction Center's Regional Spectral Model (ECP2 in the archive). All models used approximately 0.5° horizontal resolution. Atmospheric boundary conditions, sea surface temperatures (SSTs), and ocean ice fractions came from the reanalysis (Kanamitsu et al. 2002) produced by the National Centers for Environmental Prediction (NCEP) and the U.S. Department of Energy (DOE). Except for the northern side, the boundaries in Fig. 1 correspond roughly to the boundaries of each model's region that was interior to its outer frame where lateral boundary conditions were ingested. On the northern side, the interior region of the models extended into the northern Canadian territories. Further details of each model appear in both the NARCCAP website (http://narccap.ucar.edu) and Mearns et al. (2009, 2012).
For comparison, we also use output from a global climate model (GCM): the Geophysical Fluid Dynamics Laboratory (GFDL) model. The GFDL model examined here is a time-slice atmospheric GCM (AGCM) that simulated the period of 1968–99 using GFDL's Atmospheric Model 2.1 (AM2.1). The simulation was part of the NARCCAP program and was run in Atmospheric Model Intercomparison Project (AMIP) mode at 0.5° resolution (GFDL Global Atmospheric Model Development Team 2004), like the NARCCAP RCMs. The model used observed SST and sea ice extent from the Hadley Centre Sea Ice and Sea Surface Temperature (HadISST) dataset (Geophysical Fluid Dynamics Laboratory 2009). Use of the time-slice GCM helped to indicate differences, if any, in downscaling outcomes between a time-slice GCM and NARCCAP RCMs.
We analyzed the period 1982–99, discarding the years 1979–81 for RCM spinup and retaining years available in both observational and climate model data. Because we are working with very heavy events, we adopted a relatively conservative spin-up period to ensure that the models' water cycles were adequately spun up to achieve climate equilibrium (Christensen 1999). Our region of interest is the upper Mississippi region, defined here as the region bounded by 37°–47°N, 89°–99°W, highlighted in Fig. 1. This was the same definition used in previous analyses (Gutowski et al. 2007, 2008, 2010). Our analysis focused on the winter season [December–February (DJF)], when synoptic dynamics are more important than in the warmer months, when smaller-scale convective events may be more important (e.g., Schumacher and Johnson 2005, 2006). The assumption here is that winter events will be governed more by the resolved circulation (Gutowski et al. 2008).
In NARCCAP, the adopted “day” is 0600–0600 UTC (midnight to midnight in the upper Mississippi region). The UW observational dataset is already in daily increments, matching the NARCCAP day. However, the CPC defines a “day” as 1200–1200 UTC, a factor that may affect some of our results. We converted the original UW output to a 0.5° grid by averaging all original grid points that fell in a 0.5° box centered on the new grid point. We applied the same conversion method to the CPC data. We did this to give the datasets the same nominal resolution as the RCMs and time-slice GCM.
Analysis examining conditions other than precipitation during very heavy events focused on instantaneous data at 1800 UTC (local noon in the upper Mississippi region), which provided information on the state of the atmosphere during the day of a very heavy event. We defined a “precipitation event” as a nonzero precipitation record for 1 day at one observational or model grid point, consistent with Gutowski et al. (2007, 2008). We extracted the top 0.5% of all precipitation events as very heavy daily events. This threshold is within the “very heavy” precipitation category of Groisman et al. (2005). We then found widespread very heavy events by searching for multiple very heavy events occurring on the same day. For our analysis, we designated simultaneous very heavy events on 15 or more grid points as widespread events. We selected this threshold in order to have sufficient numbers of events to analyze while requiring enough spatial distribution that resolved synoptic dynamics could be a governing factor. We examined several atmospheric fields, listed earlier, to understand conditions conducive to very heavy events. These fields gave insight into the preferred conditions for very heavy precipitation events and became the basis for assessing simulated versus observed processes yielding very heavy precipitation. The 10-m winds were used as our primary indicator of moisture flux. Although it is not perfectly synonymous with moisture flux direction and convergence, it is a low-level circulation field available from all the models. For some of the fields, we examined anomalies. These anomalies are composites of fields on the days of widespread very heavy events minus the 18-yr time average during the winter season. Time averages are computed separately for each model and for the observations.
We also examined gradient strengths of temperature, moisture, and momentum fields on days of our very heavy events. For those days, we compute the horizontal gradients of 2-m temperature and humidity and the horizontal convergence of 10-m wind at each grid point in the domain for each model. We then pooled all values for all very heavy precipitation days for a model and extracted the 99% level's value for each field. These values serve to indicate the magnitudes of strong gradients produced by each model on very heavy precipitation event days. We assume that these gradients indicate the ability of a model to produce intense features associated with the very heavy precipitation. Strong temperature and humidity gradients indicate the strength of frontal systems during very heavy events; strong momentum convergence indicates the strength of moisture convergence.
3. Widespread very heavy precipitation
Table 1 shows the average precipitation rate and frequency of daily precipitation events in the upper Mississippi region, for the observations and for each model. The numbers in parentheses are the percentage of days with precipitation above 2.5 mm day−1. Other than WRFG, the models produce too much precipitation, with the GFDL and ECP2 models producing the most. Other than MM5I and WRFG, the models also produce too many days with precipitation, primarily due to too much light precipitation, or “drizzle.” This is evident by the number of precipitation days above 2.5 mm, for which the models tend to show closer agreement with the observations.
Figure 2 shows a histogram of normalized frequency versus intensity in the upper Mississippi region. Intensity is separated into 2.5 mm day−1 bins. Models and observations show relatively good agreement up to about 30 mm day−1. At higher intensities, observations are around the middle of the results. Except for the CRCM, the models all have more days of precipitation above 100 mm day−1 than the UW dataset. The CPC dataset shows higher intensity precipitation than the UW dataset and shows good agreement throughout the spectrum with the RCMs. The CRCM, on the other hand, agrees well with the UW dataset over the whole intensity spectrum, while MM5I agrees well with the CPC dataset. Recall, however, that the UW and CPC datasets are gridded precipitation and the gridding process may smooth very heavy events. Interpolation of both the UW and the CPC to 0.5° may have also affected the intensity of precipitation. These results show closer agreement between models and observations than seen in Gutowski et al. (2007), who diagnosed daily precipitation frequency versus intensity for the same region but using two older RCMs. That work also examined a shorter time period (1981–88) and the cold half of the year (October–March). With those differences in mind, the models in Gutowski et al. (2007) did not produce precipitation as intense as observed. Part of the difference may be because previous work used station data for observations. A comparison of Fig. 2 here with Fig. 2 of Gutowski et al. (2007) suggests that the gridding process to produce the UW and CPC datasets does tend to smooth high-intensity events.
Table 2 shows precipitation for each model and for the observations at the 95th, 99th, and 99.5th percentiles. The models and observations show fairly good agreement for each percentile. Average precipitation of all models' 95th, 99th, and 99.5th percentiles are 9%, 15%, and 13% greater than UW, respectively, so the average difference is about the same for each of these percentiles. Much closer agreement is seen between the models and the CPC dataset. Our previous studies looking at regional model performance in this region (e.g., Gutowski et al. 2003, 2007, 2008) focused on comparing one or two models to observations. These papers showed the models producing lower very heavy precipitation than observations. Table 2 shows, especially for higher percentiles, that the models' very heavy events are mostly greater than the UW datasets, and the CPC results are comparable to the RCM average.
Figure 3 shows the distribution of days with simultaneous very heavy events on a given number of grid points. The x axis indicates the minimum area possible for a multigridpoint event, thus suggesting its spatial scale. The models tend to produce very heavy events covering a wider area than the observations. In addition, CRCM and ECP2 have the largest spatial scales among the RCMs for their very heavy events. This is noteworthy because these models also used interior nudging, in which some of a model's fields are damped toward corresponding large-scale fields of the driving reanalysis (von Storch et al. 2000). An implication of the figure is that the interior nudging produces very heavy daily precipitation events that have larger spatial scales than observations or models not using the nudging.
Further analysis focuses on very heavy events occurring on at least 15 grid points on the same day. We denote these as widespread very heavy events. As discussed above, we assume that the widespread events are especially likely to be the outcome of resolved behavior in the models.
Table 3 shows the percentage of widespread very heavy events occurring on two or three consecutive days. The UW data show the highest percentage of 2- and 3-day very heavy events. This may indicate that storms in these models either move out of the domain faster or decrease in strength more rapidly during their lifespan compared to the UW dataset. The CPC dataset shows values comparable to the models, although it still has more persistence of 3-day events than any of the models. Part of the reason for the UW-CPC difference is that their sets of widespread events were not identical. The MM5I and WRFG produce very low frequencies of consecutive very heavy events compared to the rest of the models and observations. This may be because their spatial scale (Fig. 3) is smaller than other models or observations, so a relatively small location change could move the very heavy event out of the domain of interest.
Table 4 shows the distribution of widespread very heavy events by winter months. Aside from RCM3, the models and observations have the most very heavy events in December. This may be due to the warmer SST in the Gulf of Mexico during December compared to January and February. Warmer SST promotes warmer atmospheric temperatures over the Gulf and thus promotes more atmospheric moisture for transport into the upper Mississippi region (e.g., Kunkel et al. 2002; Gutowski et al. 2008, 2010).
Table 5 shows the interannual variability of very heavy events for the observations and the models as a percentage of all very heavy from each data source. The “year” is the year for January and February. The table also shows the average among all RCMs for each year. Looking at the RCM average and the observations, the winters of 1983 and 1993 have larger numbers of very heavy events than other years. The GFDL model also captures the higher very heavy precipitation frequency of 1993. We also calculated correlations between pairs of RCM, GFDL, UW, and CPC time series. The resulting correlations, 0.286 for RCM and GFDL, −0.022 for GFDL and UW, 0.455 for RCM and UW, 0.278 for GFDL and CPC, and 0.278 for RCM and CPC, show that the RCM average matches the UW better than the time-slice model or the CPC. This result differs from previous analysis in this paper, where CPC showed better agreement with the models than the UW. The CPC and UW correlation is 0.540, showing some agreement between the two observational datasets. Again, part of the reason for the UW–CPC difference is that their sets of widespread very heavy event days were not identical. Although these results used only one time-slice model, they suggest that the RCM ensemble, though not individual models, replicates the UW observed interannual variability of very heavy events when using reanalysis boundary conditions because of their lateral boundary conditions.
Figure 4 shows composite precipitation during widespread very heavy events. Models and observations show similar locations of very heavy precipitation, centered near the southeastern corner of our analysis region. Our analysis region in winter is warmest to the south. The warmer air can contain more precipitable water, so the composite very heavy precipitation occurs where there will generally be more moisture in the atmosphere. Also, the southern end of the analysis region is closest to the primary source of the region's precipitable water, the Gulf of Mexico. This analysis is consistent with Liang et al. (2004), who also showed the observed average winter precipitation gradient decreasing from the southeast to northwest over our analysis region.
4. Supporting environmental conditions
Figures 5–9 show composite fields produced by averaging over the widespread event days from each data source. Again, the anomaly fields for a given source come from subtracting the 18-yr DJF average from the composite. The NARR provided the observational results, with the days to composite determined from analysis of the UW precipitation.
a. 500-hPa geopotential heights
As suggested by Fig. 5, a key ingredient for very heavy precipitation in the upper Mississippi region is the transport of warm, moist air from the Gulf of Mexico. Composite 500-hPa heights and composite height anomalies for each model (Fig. 6) show the very heavy events occurring when a deep trough develops around the southern Rockies, promoting a more pronounced southerly flow into the region when compared with seasonal climatology. Anomaly plots also show areas of higher heights in the northeast, suggesting that the occurrences of both low heights to the west and high heights to the east are important in very heavy precipitation development.
Figure 7 shows representativeness plots for 500-hPa height anomalies. This analysis was used to determine if the signs of 500-hPa height anomalies agree between each widespread very heavy event in the observations or a model. Features in these plots are similar to those seen in Fig. 6, indicating that composite height anomalies are representative of most, if not all, of the daily very heavy events in the observations and in each model. Further inspection of the individual events shows that composites are indeed representative of the behavior in each case, except that in some individual cases, the deep trough includes a cut-off low center at 500 hPa. The 500-hPa patterns the day before and day after very heavy events (not shown) show a slowly propagating or stationary trough, with roughly the same speed of movement in the models and observations.
b. 10-m horizontal wind
Figure 8 shows the composite 10-m winds for widespread very heavy events. As with 500-hPa heights, the composites are representative of the behavior of individual events. As discussed earlier, the winds indicate the direction of moisture transport and also the location of surface pressure centers, although these winds are not perfectly synonymous with the moisture flux direction and convergence, also discussed earlier.
During the widespread very heavy events, winds decrease and turn counterclockwise behind the area of very heavy precipitation. The behavior corresponds to a surface low in the vicinity of Oklahoma accompanying the 500-hPa trough. Composite precipitation moves as the low center moves (not shown). Wendland et al. (1983), who focused on higher than average precipitation during the 1982/83 winter, also had a surface low in the vicinity of Oklahoma during strong precipitation events. In addition, the behavior shows low-level convergence. Because relatively strong winds blow from the Gulf of Mexico, the momentum convergence likely coincides with the moisture convergence, especially in the vicinity of the very heavy precipitation. Table 6 shows that strong momentum convergence on days of widespread very heavy precipitation events in NARCCAP models is approximately the same as occurs in the NARR for observed days of widespread very heavy precipitation. Note that the GFDL time-slice model has the strongest convergence, but its precipitation percentiles (Table 2) are not the highest among the models.
Winds in the Gulf of Mexico highlight the importance of surface high pressure to the east of the analysis region. Strong winds in the composites tend to start as southwesterly flow around the southern tip of Florida. Over the Gulf, the winds turn clockwise toward the northern coast. This pattern provides substantial fetch for moistening air before it enters the southern United States. Similar results were found in Brubaker et al. (2001), which emphasized the presence of anticyclonic flow around the Bermuda high, promoting moisture transport not only from the Gulf of Mexico, but also from the Caribbean and tropical Atlantic. The domains of the RCMs do not extend into the Caribbean and tropical Atlantic, but Fig. 8 does show flow possibly originating south of the Gulf. Although Brubaker et al. (2001) focused on the warm season, Fig. 8 highlights the importance of the moisture fetch during the winter season when, climatologically, Gulf of Mexico moisture does not often penetrate our upper Mississippi region, and existing terrestrial moisture supply within the region is low (Kunkel and Liang 2005; Brubaker et al. 2001). Moreover, this flow pattern passes over the Loop Current, where SST tends to be warmer because of a consistent flow of warmer Caribbean waters into the southern Gulf (Vukovich 2007). Flow over the Loop Current may supply additional moisture into the southern portion of our domain.
c. 2-m air temperature and specific humidity
We also analyzed 2-m air temperature and specific humidity from most of the models and the NARR. Figures 9 and 10 show these two fields as composite anomalies. Regions of very heavy precipitation tend to occur in regions of positive temperature and specific humidity anomalies. Plots of temperature and specific humidity 1 day before and after the widespread very heavy events (not shown) show an anomalously warmer and wetter environment during the development and propagation of these events. Also, the composite temperature in areas of very heavy precipitation is above 275K, which increases the likelihood that the precipitation type during these events is rain, not snow. However, 5% of the UW events, 21% of the GFDL events, and 11% of the RCM events have very heavy precipitation occurring in regions with surface air temperature below 273K, most likely falling as frozen precipitation. Assuming the precipitation is then snow, these areas may get more than 250 mm day−1 (10 inches) on the ground, which raises concerns of substantial societal impacts both during snowfall and when the melting snow runs off. Finally, Table 6 shows that strong temperature and humidity gradients on days of widespread very heavy precipitation events in the NARCCAP models are comparable to those in the NARR for observed events. As with momentum convergence, the GFDL time-slice model has the strongest gradients in Table 6, but again not the most intense precipitation (Table 2).
Six different RCMs and one time-slice GCM from the NARCCAP project were compared with observational data [University of Washington (UW) and Climate Prediction Center (CPC) precipitation and the North American Regional Reanalysis (NARR)] to determine the ability of models to reproduce very heavy daily precipitation events during the winter months (DJF) between 1982 and 1999 in an upper Mississippi region. Widespread very heavy precipitation was defined as the top 0.5% of all nonzero precipitation occurring on at least 15 grid points simultaneously. For these events, we analyzed 500-hPa heights, 2-m air temperature and specific humidity, and 10-m surface winds to diagnose the environment favorable for the production of very heavy precipitation.
The observations and most models have greater frequency of very heavy events in December compared to January and February, likely because of warmer SSTs in the Gulf of Mexico in December. The warmer SSTs allow more moisture to enter the atmosphere for transport into the central United States. The models, for the most part, tend to produce too much precipitation compared to observations. Also, the models tend to produce too many precipitation days, with a large portion of them having light precipitation, or “drizzle.” CRCM and ECP2, which incorporate interior nudging, have larger spatial scales for their very heavy events, indicating that interior nudging increases the spatial scale of simulated very heavy events. For precipitation at the 95th, 99th, and 99.5th percentiles, the models are consistently near or above UW amounts, while the CPC amounts show good agreement with the RCM average. Models and observations are in good agreement for frequency versus intensity of precipitation up to about 30 mm day−1. Above this value, some of the models produce several days with precipitation amounts that are higher than any in the UW dataset. The CPC has the higher-intensity precipitation, and the models show better agreement with its results throughout the entire precipitation spectrum.
For environmental features, the observations and models show similar characteristics. Composite 500-hPa heights show a predominant southwesterly flow into the upper Mississippi region, caused by a deep trough or cutoff low near the Rockies. This allows increased moisture transport into the central United States from the Gulf of Mexico, which aids the development of very heavy precipitation. Anomaly plots show areas experiencing very heavy precipitation tend to occur in areas of positive anomalies of surface air temperatures, which provide an environment capable of containing more moisture compared to climatology. Areas experiencing very heavy precipitation also tend to occur in areas of positive moisture anomalies, showing that the warmer air does indeed have greater moisture. Surface wind analysis suggests a strong transport of Gulf of Mexico moisture into the upper Mississippi region. Features of a surface low exist slightly to the west of the area of very heavy precipitation. Low-level momentum convergence of 10-m winds near very heavy events is also present, indicating moisture convergence. Very heavy events tend to occur near the southern portion of the analysis region, centered on central Missouri. This is likely because of the warmer air in the southern part of the analysis region and transport of moisture into the part of the domain that is closest to the moisture source, the Gulf of Mexico.
Analysis of strong environmental features produced by the models on days of widespread very heavy precipitation shows that the NARCCAP models produce momentum convergence and temperature and humidity gradients that are comparable to the NARR values.
The models thus appear to be capable of producing very heavy precipitation in the analysis region for the correct physical behavior. Moreover, they are capable of producing the intensity of atmospheric features that coincide with producing the observed intensity of very heavy precipitation. This capability should support using them to assess changes in very heavy precipitation events under future climate scenarios.
This work was supported by National Science Foundation Grants ATM-0633567, AGS-1125971, and BCS-1114978 and Department of Energy Grant DEFG0201ER63250. Daily gridded observed precipitation data were obtained from the Surface Water Modeling group at the University of Washington. CPC U.S. Unified Precipitation data was provided by the NOAA/OAR/ESRL PSD, Boulder, Colorado, from their website at http://www.esrl.noaa.gov/psd/. We thank the North American Regional Climate Change Assessment Program (NARCCAP) for providing the data used in this paper. NARCCAP is funded by the National Science Foundation (NSF), the U.S. Department of Energy (DOE), the National Oceanic and Atmospheric Administration (NOAA), and the U.S. Environmental Protection Agency Office of Research and Development (EPA). We thank the reviewers for their helpful comments.