1. Introduction
Meaningful comparisons of climate model output with observational data are sometimes difficult to achieve because, unlike numerical weather prediction models, climate models do not predict the specific sequence of weather that any location is expected to experience. At best a climate model simulation can be thought of as representing one possible realization of future weather. Thus comparisons of climate model output with observational data are inherently statistical. Typically, the observational data are aggregated (e.g., averaged) over a sufficiently long period of time that the influence of individual weather events becomes small relative to the average. The climate model output is then aggregated over a similar time period, or in some cases, the outputs from an ensemble of climate simulations are combined. Either way, when differences between the aggregate observations and aggregate model output are detected, it can be difficult to determine the source of the differences (what physical processes or situations are not sufficiently represented by the model) or to determine a corrective action. This is particularly true for clouds and precipitation whose occurrence and properties are complex and highly variable in space and time.
One approach to dealing with this complexity and variability is to aggregate both the observations and model output in a way that provides insight into the interaction between the atmospheric state and cloud properties. For example, several recent papers have identify cloud regimes by clustering joint histograms of cloud optical depth and cloud top pressure produced by the International Satellite Cloud Climatology Project (ISCCP) in combination with an instrument simulator to produce ISCCP-like histograms from model output (e.g., Gordon et al. 2005; Jakob et al. 2005; Rossow et al. 2005; Williams and Webb 2008). Comparing observational data and model output as a function of the large-scale midtropospheric (500 hPa) vertical pressure velocity has also been used in a variety of recent studies including Bony and Dufresne (2005), Bony et al. (2006), Lin and Zhang (2004), Norris and Weaver (2001), Tselioudis and Jakob (2002), and Wyant et al. (2006). In general, the identification of synoptic regimes using a variety of meteorological fields including surface pressure patterns and geopotential heights at various altitudes has been the focus of considerable research (see discussion in Coleman and Rogers 2007; Fereday et al. 2008; Marchand et al. 2006; Smyth et al. 1999; Zivkovic and Louis 1992), and such regimes have been used in compositing cloud properties (e.g., Marchand et al. 2006; Tselioudis et al. 2000) and evaluating model output (e.g., Jung et al. 2005). In addition to providing insight into model behavior, evaluations of GCMs based on cloud regime or atmospheric state have the potential to detect errors that cannot easily be identified in temporal averages. For example, Norris and Weaver (2001), Lin and Zhang (2004), and Jakob et al. (2005) all demonstrated places where errors in top-of-atmosphere fluxes tended to cancel out in temporal averages, reducing the apparent size of model errors.
In this paper we compare hydrometeor (cloud and precipitation) occurrence profiles observed by a vertically pointing millimeter wavelength cloud radar with similar profiles obtained from a climate model as a function of the large-scale atmospheric state. By hydrometeor occurrence profile we mean the relative frequency that clouds or other hydrometers, such as rain or snow, are detected by the cloud radar at a given altitude above ground level (or would be detected by a radar in the case of the climate simulation). The cloud radar observations are obtained from the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) program at its primary Southern Great Plains (SGP) site near Lamont, Oklahoma, while the model-simulated radar profiles are obtained using the QuickBeam radar simulator (Haynes et al. 2007; Marchand et al. 2009).
In this analysis the atmospheric state is determined by classifying (or clustering) the large-scale (synoptic) fields that are resolved by global climate models and numerical weather prediction models. This is accomplished using a neural network following the approach of Marchand et al. (2006), hereafter M06. Specifically, we used output (analysis data) from the Rapid Update Cycle (RUC) model, which is run operationally at the National Centers for Environmental Prediction (NCEP) (Benjamin et al. 1991, 1996), to train the neural network and obtain a time series of atmospheric states over the ARM SGP site. We have made a number of improvements and extensions to the approach described by M06 and these are described in detail in section 2.
The goal of the analysis is to evaluate to what degree radar profiles of hydrometeor occurrence produced by the multiscale modeling framework (MMF) climate model match those observed by the ARM program when aggregated by the objectively determined atmospheric state. The MMF is a new type of GCM in which a two-dimensional or small three-dimensional cloud resolving model (CRM) is embedded into each grid cell of a traditional GCM. The embedded CRM removes the need for most of the cloud parameterizations used in traditional GCMs; perhaps most significantly it replaces the parameterization of deep convection. This new approach is frequently called a multiscale modeling framework but is also known as a cloud resolving convection parameterization or a superparameterization (Grabowski 2001; Randall et al. 2003). In section 3 we give a brief description of the MMF model. We use the high-resolution cloud resolving model output produced by the MMF as input to the radar simulator to produce hydrometeor occurrence profiles like those observed by the ARM cloud radar.
In section 4 we present comparison results that show that, for cold-frontal and post-cold-frontal conditions, the MMF produces profiles of hydrometeor occurrence that compare favorably with radar observations, while, for warm-frontal conditions, the model tends to produce hydrometeor fractions that are too large with too much cloud (nonprecipitating hydrometeors) above 7 km and too much precipitating hydrometeor coverage below 7 km. We also find that the MMF has difficulty capturing the formation of low clouds and that, for all atmospheric states that occur during June, July, and August, the MMF produces too much high and thin cloud, especially above 10 km (a result that appears to be a common feature of the model in convective regions). Finally in section 5 we close with some additional discussion and remarks on our plans for future research.
2. Classification technique and dataset description
The idea of weather typing or weather regimes is not a new one, but has been used extensively in meteorology. In this investigation, we used a competitive neural network (Haykin 1999; Kohonen 1995) to objectively identify patterns in just over 3 yr of analysis data from the RUC model, from December 1998 through April 2002 (Benjamin et al. 1991, 1996). Analysis data are the inputs that are used to initialize NWP computer model forecasts and are obtained through a data assimilation process that combines model output with a range of observations.
The neural network is essentially a pattern recognition algorithm, into which we have put the relative humidity, temperature, and winds at seven predetermined (sigma) pressure levels along with the surface pressure for 81 grid points. The grid points are arranged in a 9 × 9 grid centered on the ARM SGP site, each representing an area of 2° × 2.5° (for a total of 2349 input variables at each time step). Both the RUC and MMF datasets have 3-hourly resolution, with a total of approximately 9400 input vectors for the 3+ years of RUC data analyzed. The neural network has two modes of operation, a training mode and an application mode. In this study we use the RUC (and as explained later in this section ARM) data to train the neural network and later apply the neural network to the MMF output. In the training mode, a set of input data is repeatedly fed into the network until it converges on a set of N patterns, which we call the atmospheric state definitions or simply the state definitions, that best represent the input space. Figure 1, for example, shows the state definition for one of the states that we will use in our analysis in the next section. To the neural network, the best representation of the input space is defined to be the set of state definitions such that the sum of the distances between each input vector (i.e., each entry in the training set) and the closest state definition is minimized. Here the distance is defined as the sum of the absolute values of the input vector elements minus the same elements in the state definition relative to the standard deviation of each element (calculated from the entire input training set).
We note the value of N must be selected at the start of the training processes. In general, trying to determine how many states there should be is an issue for most objective classification schemes and we will return to this topic later in this section. Once the atmospheric states are defined, the neural network can be applied to obtain the state number (1 to N) associated with any input vector (whether it comes from RUC output or MMF output or some other model altogether) by finding the state definition closest in distance (as defined above) to the input. While the RUC model has a resolution of about 40 km, we subsampled the RUC data (using a nearest neighbor approach) to the same 2° × 2.5° grid used by the MMF to ensure a common set of inputs. We examined the effect of averaging (rather than sampling) the RUC data onto the 2° × 2.5° grid and found it made no significant difference in the results.
The approach to this point is essentially the same as that given by M06, except using 3+ years of RUC data (rather than 17 months) and reducing the horizontal resolution to match that of the MMF. In M06, the neural network was used to classify the atmosphere as belonging to 1 of 25 possible states. Hydrometeor occurrence profiles were then created for each state by aggregating ARM millimeter wavelength cloud radar observations. The bulk of the M06 article focused on evaluating the statistical stationarity of the hydrometeor profiles. That is, to use the atmospheric states as a means to compare ARM observations with climate model output (or more generally, to view the atmospheric states as a map from synoptic-scale atmospheric patterns to smaller-scale cloud properties) the distribution of cloud properties associated with any given state should be stable such that properties of the state do not change with time. This does not mean that every time some state X occurs exactly the same set of clouds are observed. Rather, it means that every time state X occurs the observed clouds represent one possible realization drawn from a fixed distribution.
In M06 the stability of the atmospheric states was tested by creating and comparing two sets of hydrometeor occurrence profiles: one set based on radar observations only from winter 1997 and one set based on radar observation only from winter 1998. While one expects the occurrence profiles based on data from two different years to be similar, they will not be identical. Figure 2, for example, shows the hydrometeor occurrence profiles obtained by aggregating the ARM millimeter wavelength cloud radar observations according to the atmospheric state (as determined by application of the competitive neural network to the RUC analysis data) in two different years, 1999 and 2001. Results for 4 states are shown. The percentage of time occupied by each state in 1999 and 2001 is shown above each panel. It is not surprising that the percentage of time occupied by each state can be different from year to year. In general, there is considerable variability in many observed cloud properties from year to year—which greatly hampers the effort to compare GCM climate predictions with observations and is part of the motivation behind this research.
M06 developed a statistical hypothesis test based on a moving-blocks bootstrap resampling technique to determine if the difference between two radar profiles is statistically significant (i.e., unlikely to be the result of the finite sample size). The result of the profile similarity test is a p value, which is an estimate of the likelihood that the two profiles are two realizations from a common parent population. We stress that no statistical test can prove that two profiles do come from a common parent population, we can only determine when it is unlikely to be true. In Fig. 2, for example, the two states on the left-hand side of the figure have p values much larger than 0.05, meaning we cannot reject the hypothesis—the two profiles may well come from a common parent. For the upper-right panel, on the other hand, the p value is less than 0.05, meaning that we can conclude that the profiles are likely from different parent populations with a 95% level of confidence.
Unfortunately, of the 18 states identified in M06 that occur during the winter or early spring seasons, only 7 had sufficient data to make a robust comparison. And of these 7 states, one state was found to be unstable. Part of the reason that only 7 states contained sufficient data to make a comparison was the small size of the test dataset used (17 months), but part of the reason is also that some of the identified states (while meteorologically distinct) simply occur so infrequently (less than 1% of the time) that many years of ground-based data would be needed to obtain a good representation of the distribution of hydrometeor properties. When we repeated the year-to-year stability comparison with 3 yr of data, the results were qualitatively similar. A few states were found to be unstable, and even with 3 yr of data a few states did not have sufficient data points to make a robust comparison. As was suggested in M06, the finding of well-populated and stable mappings from the large-scale to local-scale cloud properties is useful for model–data comparisons, even if less than 100% of all atmospheric conditions can be mapped successfully. Nonetheless, an objective process to modify the state definitions such that all identified states are stable is desirable. In M06, the decision to define 25 states was chosen based on intuition, with a preference for having too many rather than to few states. It was hoped that by having too many states we would find only 1) stable states or 2) states with too little data such that we could then combine the data-poor states with the stables states in an additive manner. Unfortunately, as already mentioned, we found that some states do end up having sufficient data and yet are not stable. An examination of these unstable states suggested that in fact the states are likely too broad, encompassing too wide a range of atmospheric conditions.
We have therefore developed an iterative scheme to refine the state definitions. This scheme is depicted in Fig. 3. We begin by using the neural network to identify 25 states. We then analyze the stability of each state by comparing the year-to-year similarity of the hydrometeor occurrence profiles. For the current dataset, this means comparing 1999 to 2000, 1999 to 2001, and 2000 to 2001 for each state. A state is considered stable if the p values for all year-to-year comparisons are greater than 0.05. We identify the (up to) four least stable states based on the median of the p values obtained from the three year-to-year comparisons. If an unstable state comprises more than 6% of the input dataset we divided the state into two states (thus increasing the total number of states by one) by running a second clustering algorithm on only the elements in the unstable state. If the unstable state comprised less than 6% of the dataset (or had insufficient data to permit a stability comparison) we removed the state from consideration (thus reducing the total number of states by one). We then reassign every input vector to its nearest state definition. The reassignment allows new states (obtained by division) to attract data points from neighboring states and forces data points from deleted states to be assigned to neighboring states. After reassignment, the state definitions are again evaluated for stability and the process repeats until all states are stable. For the data analyzed here, we found that it only takes a few iterations (generally fewer than 15) to converge and resulted in a total number of states between about 15 and 20.
One problem with this approach is that the fewer the data points in any state, the less likely that a state that is in fact unstable will be detected as such (assuming there are a sufficient number of data points to permit a comparison of hydrometeor profiles; see M06 section 3.3c). This is because the fewer the number of data points, the larger the expected variability from year to year will become and thus real differences become harder to detect. So the technique (to this point) favors creating states that are sparsely populated. Additionally, we also noticed that many of the more sparsely populated states tended to have hydrometeor occurrence profiles that were similar to those of other states. It is also generally possible to intentionally divide a stable state into two states, both of which will pass the stability requirement and not surprisingly tend to have very similar profiles of hydrometeor occurrence.
Therefore, we extended the iteration scheme to consider the distinctiveness of the hydrometeor occurrence profiles. Once all the states are found to be stable, we compare the occurrence profiles in each state with every other state (using the same statistical comparison test) and count the number of states that appear to have similar profiles. If all states are pairwise completely distinct from one another, we are done and have a final set of states. If not, we then do a similar adjustment process as we did previously, choosing (up to) the four states with the highest similarity count and dividing or removing them based on whether they contain above or below 6% of the total number of elements. The distinctiveness criteria therefore drive the iteration process to find the minimum number of states needed to represent the hydrometeor occurrence profiles.
We ran the full iteration scheme four times. In two of the runs the iterative process converged completely (passing through both the stability and distinctiveness tests), while in the other two runs the stability test converged many times but the distinctiveness test never passed completely. In these two nonconverging runs the algorithm would remove two states with similar occurrence profiles, causing the data points from these states to be assigned to other nearby states. The nearby states were destabilized by the additional points and subsequently divided, which led to the regeneration of the states that produced the nondistinct occurrence profiles in the first place. While the two nondistinct states had similar occurrence profiles, they were sufficiently meteorologically different that removing these states did not cause the points to be merged into a single neighbor. At this point we stopped the iteration processes, which was effectively in an infinite loop.
The four runs all produced 11 or 12 states. A summary of the 12 states from one of the four runs (which we will use as a basis for analysis in section 4) is provided in Table 1. Also, a discussion on how the 25 states identified in M06 map into this new 12-state set is provided in the appendix.
The states from each run have strong one-to-one correspondences such that most of the data points associated with a given state in one run tend to map into a very similar state in a different run. Nonetheless, the state definitions are certainly not unique or perfectly reproduced given the same training set. Perhaps the most noteworthy difference between the four test runs is that the two runs that produced 11 states (rather than 12 final states) effectively combined the two summertime states with the hottest surface temperatures occurring in the runs with 12 total states into a single large state. Other differences included a tendency to split state 6 (in our analysis set), which features a post-cold-frontal conditions, into two slightly different states and combine two of the frontal states (states 1 and 3) into a single larger state. We will discuss the nonuniqueness of the states further in section 5.
We stress that, while the cloud radar–derived hydrometeor profiles are used in the iterative process to identify atmospheric states, the state definitions depend only on the large-scale pressure, temperature, relative humidity (RH), and winds. As a result, once the states are defined they can be applied to any climate model or numerical weather prediction model that produces a similar set of large-scale atmospheric fields (by determining which state definition most nearly matches any given set of large-scale fields) and no hydrometeor profiles are needed to determine the state. Later in this article we compare simulated hydrometeor profiles from the MMF climate model with ARM observed profiles. We do this because we are interested in evaluating the MMF’s ability to predict hydrometeors, not because we require the hydrometeor profiles to determine the atmospheric state in the MMF. We could have, for example, compared ARM observations of downwelling surface shortwave fluxes with MMF simulated fluxes (as a function of the atmospheric state) even though such fluxes are not used in developing the state definitions in any way. In fact, we eventually plan to expand the current comparison of ARM data and MMF output to include a variety of additional data, including distributions of surface shortwave and longwave fluxes.
3. The MMF and radar simulator
This study uses the MMF as developed by Marat Khairoutdinov (Stony Brook) and David Randall (at Colorado State University). It consists of the National Center for Atmospheric Research (NCAR) Community Atmosphere Model, version 3.0 (CAM3.0), and an embedded 2D cloud resolving model. The details of the MMF configuration are given by Khairoutdinov and Randall (2001) and Khairoutdinov et al. (2005) and are only briefly described here. CAM is the atmospheric component of the Community Climate System Model (CCSM) (Collins et al. 2006). In our version of the MMF, CAM is run with the finite-volume dynamical core and has 26 vertical layers and a horizontal resolution of 2° latitude and 2.5° longitude. The dynamical time step of the CAM is 20 min. Details of the CAM physics can be found in Collins et al. (2006). The embedded CRM within each CAM grid cell has 64 columns at 4-km spacing and 24 layers in the vertical, which coincide with the lowest 24 levels of the CAM (Khairoutdinov and Randall 2003). The CRM domain is aligned in the east–west direction with cyclic lateral boundary conditions. The CRM runs continuously with its own 15–20-s dynamical time step. Radiation calculations using the CAM radiative transfer code are performed on each CRM column every 10 min. The CRM predicts the total nonprecipitating water (vapor + liquid + ice) and total precipitating water (rain + snow + graupel). The CRM uses the same optical property parameterizations and radiation code as the CAM, although no overlap approximation is needed.
As described in section 2, the atmospheric state of the MMF output is determined by finding the state definition closest in distance (see section 2) to the MMF output. The state definition does not depend on the hydrometeor profile; however, we want to know whether the MMF produces the correct the hydrometeor profile in each state. Radar returns were therefore obtained from the model output using the QuickBeam radar simulation package (Haynes et al. 2007). QuickBeam takes vertical profiles of cloud and precipitation mixing ratios produced by a cloud resolving model and converts them into equivalent radar reflectivities as would be viewed from a satellite passing over the model domain or from a ground-based radar. Details on application of the QuickBeam simulator to the MMF output are given by Marchand et al. (2009), who compare global zonally, monthly averaged profiles for hydrometeor occurrence (as well as joint histograms of radar reflectivity with height for select regions) from the MMF with National Aeronautics and Space Administration (NASA) CloudSat (94-GHz spaceborne radar) observations. The MMF has also been the focus of several evaluation studies including Khairoutdinov and Randall (2001), Khairoutdinov et al. (2005), Ovtchinnikov et al. (2006), DeMott et al. (2007), McFarlane et al. (2007), and Zhang et al. (2008).
4. Results
In this section we discuss the 12 synoptic patterns obtained from the classification scheme described in section 2 (and summarized in Table 1) and their associated simulated and observed profiles of hydrometeor occurrence. Figures 4 and 5 show the profiles for each atmospheric state. The observations (RUC + ARM) are in blue and the simulated (MMF) profiles are in red. Figure 4 shows the occurrence profiles using a radar reflectivity threshold of −40 dBZe, while in Fig. 5 a threshold of −25 dBZe is used. That is, Fig. 4 shows how often hydrometeors with a reflectivity of −40 dBZe or larger are observed (at each altitude) and how often it is simulated. The −40-dBZe threshold is well within the sensitivity limit the ARM radar in the upper troposphere (where the radar sensitivity is poorest) and is sufficient to detect most though not all cloud condensate (Clothiaux et al. 2000). Changing the threshold from −40 dBZe to −25 dBZe shows the degree to which low-reflectivity clouds (those with both low amounts of condensate and small particles) are contributing to the total occurrence. As in Fig. 2, the percent of time each state occupies and the p value (the estimated probability that the observed and simulated profile could be two realizations from a common parent population) are given for each state. Also as in Fig. 2, the black line and asterisks in Figs. 4 and 5 denote altitudes where comparisons were possible and where the p value is greater than 5%, respectively.
To begin our description of the atmospheric states, we first group the states by the surface-wind direction and follow this with additional comments on the representation of fronts.
a. Southerlies
States 2, 4, 9, 10, and 12 all have surface southerlies. This is typical in the summertime and in return-flow situations in the transition seasons when moisture from the Gulf of Mexico is advected northward through the plains.
States 9 and 10 are summertime states, with among the hottest surface temperatures. State 9 has the hottest surface and is associated with weak 500-hPa flow and little moisture in the upper troposphere, while state 10 features more northwesterly flow (with stronger ridging to the southwest) and somewhat higher relative humidity. In Fig. 4, the ARM observations for both states 9 and 10 show low fractional occurrence that increases with altitude from near the surface until at least 10 km above ground level. In state 9, the observations show slightly less fractional coverage than in state 10 and with a peak in cloud occurrence that is slightly higher in altitude. In both states, the MMF produces profiles with a similar overall shape to the observations but with occurrence fractions that are too large, especially above 10 km. The simulated occurrence profiles in states 9 and 10 are more similar to each other than observed profiles. In state 9, the comparison test (the p value) unambiguously indicates that the MMF occurrence fraction is too large relative to the observations between 5 and 15 km. In state 10 the MMF profile is more similar to the observations, though clearly the high-cloud peak is too high. Examination of states 9 and 10 in Fig. 5 (threshold −25 dBZe) shows that much of overestimate in occurrence is due to thin cloud. In particular, the MMF and ARM profiles for state 10 compare very favorably at the −25-dBZe threshold. We also note that state 9 occurs 23% of the time in the MMF (the most of any state) compared with about 11% in the observations. We will discuss this further in the next section.
States 2, 4, and 12 are largely transition-season states. State 4 has the coolest surface temperatures and (unlike the other two states) does sometimes occur during January and February. The ARM observations in Fig. 4 show few hydrometeors below 5 km (with perhaps a rather weak low-cloud peak near 1 km) and a high-cloud peak just above 10 km. At the −40-dBZe threshold, the MMF appears to capture the overall shape well but with occurrence fractions that are too large. The difference appears to be statistically significant between 3 and 6 km. At the −25-dBZe threshold (Fig. 5), the ARM observations show the high-cloud peak is largely due to relatively thin cloud and the upper-level peak lowers to around 8 km. The MMF captures the general trend, but the upper-level peak drops too much. Figure 4 also shows that the overprediction of hydrometeor occurrence below 5 km is not due to thin cloud.
State 2 features warmer surface temperatures than state 4 with an approaching trough and southwesterly flow at 500 hPa. The ARM observations in Fig. 4 show that state 2 features a low-cloud peak near 2 km and a high-cloud peak just below 10 km. At the −40-dBZe threshold, the MMF seems to capture the upper peak reasonably well, but the low-cloud peak is poorly captured. The MMF does appear to have a low-cloud peak, but it is too weak and too low (near 1 km). The ARM observations in Fig. 4 show that in state 2 the position of the upper-cloud peak does not change markedly when the radar minimum reflectivity threshold is increased from −40 to −25 dBZe (unlike state 4). However, the peak in the MMF occurrence profile does drop in altitude.
State 12 features warmer surface temperatures than either state 2 or 4, zonal flow at 500 hPa, and somewhat lower relative humidity at 500 and 375 hPa. Figure 4 shows significant hydrometeor occurrence throughout the troposphere, with a low-cloud peak near 2 km, a weak midlevel peak just above 5 km, and a high-cloud peak just above 10 km. Figure 4 shows that much of this condensate is relatively thick cloud and precipitation, including the condensate above 10 km. The MMF does a good job of capturing the high-level hydrometer occurrence in this state, but the low- and midlevel hydrometeors are largely missing.
b. Northerlies
States 6, 7, and 8 have northern surface winds. State 6 represents post-cold-frontal conditions at the ARM site, with the cold front located in the southeast corner of the domain. This state features cold to cool surface temperatures, high surface pressure in the north of the domain, and moist conditions throughout the troposphere. Figures 4 and 5 show that state 6 has significant hydrometeor coverage starting near the surface and extending to near 10 km, with a peak in hydrometeor occurrence below 5 km. Overall the MMF appears to capture the observed profile well. While the model shows hydrometeor occurrence profiles that are larger than observed below about 3 km, somewhat surprisingly, the difference does not appear to be statistically significant at the 95% level of confidence.
States 7 and 8 both feature dry conditions throughout the troposphere (especially state 8), high surfaces pressures, and northwesterly flow at 500 hPa. While state 7 features (weak) northwesterly surface winds and state 8 northerly winds, the more salient difference between the two states is likely the surface temperature, which is much warmer in state 8. Not surprisingly, Figs. 4 and 5 show both states have low hydrometeor coverage, with state 8 having more high thin cloud. The MMF appears to capture state 8 reasonably well, although (as with most of the summertime states) there is a statistically significant tendency to overpredict the amount of high thin cloud above 10 km. State 7 is not as well simulated. While the hydrometeor occurrence is low, the model nonetheless appears to be underpredicting the amount of hydrometeor with reflectivities larger than −25 dBZe.
c. Weak surface winds
State 11 features weak surface winds and warm surface temperatures. Of the four states that occur during June, July, and August this state has the coolest surface temperatures and is the only state that does not feature a strong southerly or southwesterly winds at 875 hPa near the ARM site (that is across the center of the analysis domain; see, for example, Fig. 1). State 12 is the most similar state to 11 overall, though we note state 11 has higher relative humidity at 500 and 375 hPa than state 12. The observed profile of hydrometeor occurrence in state 11 is quite similar to that of state 12. The most notable difference between the observed profiles for these two states is that state 12 has more hydrometeor coverage above 10 km. As Figs. 4 and 5 show, the MMF produces a hydrometeor coverage profile with considerably too much thin cloud above 5 km, while at the −25-dBZe threshold the simulated profile matches the observed profile quite well—even below 5 km.
d. Fronts
States 1, 3, and 5 feature fronts. All three feature colder air to the north and warmer and moister air to the south. States 1 and 3 are likely cold season warm fronts. State 1 (shown in Fig. 1) has surface easterlies over much of the domain (including at the ARM site), strong low pressure at the surface in the southwestern part of the domain, and very moist southwesterly flow at 500 and 375 hPa. The observed cloud occurrence profiles in Figs. 4 and 5 show large fractional coverage from near the surface to about 9 km, suggesting deep ascending air. At the −40-dBZe threshold, the MMF produces cloud occurrence profiles that are larger than observed. For the most part, however, the difference does not exceed the 95% confidence level. At the −25-dBZe threshold there is a notable drop in the simulated occurrence relative to the observations above about 7 km, while the occurrence fraction below 7 km remains too large (likely because of excessive precipitation).
State 3 differs from state 1 in that the surface low pressure is located to the northwest of the ARM site rather than to the southwest. Surface winds are weaker (at the ARM site) and 500-hPa flow is much drier and more westerly. These differences are reflected in the hydrometeor profiles, which show lower overall hydrometeor fractions in state 3 than state 1 and more distinctive warm air/cloud overrunning. Like state 1, the MMF produces a hydrometeor occurrence profile with a shape that is similar to the observed profile at −40 dBZe but with hydrometeor fractions that are too large. Also like state 1, the simulated hydrometeor fraction above 7 km drops dramatically at the −25-dBZe threshold, while the hydrometeor fractions below 7 km remain too large.
State 5 shows a surface cold or stationary front, oriented southwest–northeast across the domain. Schultz (2004) presents a detailed discussion of patterns that look like this. While 500- and 375-hPa relative humidity is high in the north of the domain (60%–80%), values over most of the domain (including the ARM site) are relatively modest at about 50%. Figures 4 and 5 show hydrometer fractions of about 10% rising from near the surface to near 5 km and then dropping to near 0% at 10 km. The MMF simulates the hydrometeor profile for this state remarkably well, especially at the −25-dBZe threshold.
e. Annual cycle
The percentage of time each state in the RUC analysis and MMF output occupies is shown at the top of each panel in Figs. 4 and 5. With a couple of exceptions discussed earlier, the percentages are similar for most states. Figure 6 provides a more detailed look, showing the annual cycle of the atmospheric states found in the RUC model (Fig. 6a) and MMF (Fig. 6b). In this figure, occurrence is normalized to 1 for each month (i.e., each column). The overall pattern of the two is similar. Both models show states 9 through 12 dominating the summer months June to August (months 6 to 8), with states 1 through 8 occurring throughout much of the rest of the year. Both models also show some states occurring more in the transition seasons (e.g., states 2 and 8) and others states more in the winter (e.g., states 3 and 7). While broadly similar, the patterns are not identical, as highlighted in Fig. 6c. Throughout most of the year, the differences in state occurrence are relatively small (approximately ±10%). However, a few of the differences are almost certainly significant. In particular, state 9, the atmospheric state with the hottest surface temperatures, occurs about 23% of the time in the MMF compared with about 11% of the time in the RUC analysis and, as shown in Fig. 6, occurs an unduly large fraction of the time throughout the summer. An examination of the MMF output shows that it overestimates summertime near-surface air temperatures (relative to the RUC), and it appears likely that this bias favors the identification of states with warmer surfaces temperatures. In general, biases in either the RUC or MMF output have the potential to lead to problems in the classification. In the future, we hope to use other numerical weather prediction analysis datasets to help assess the degree to which potential biases may be influences our results. We also plan to evaluate the effect of redefining the neural network inputs such that only differences relative to individual model mean values are used in the classification.
5. Discussion and conclusions
In section 4, we used objectively identified atmospheric states as a basis to compare profiles of hydrometeor occurrence produced by the multiscale modeling framework climate model with cloud radar observations from the U.S. Department of Energy (DOE) ARM program Southern Great Plains Site. The comparison shows that
For cold-frontal and post-cold-frontal conditions (states 5 and 6), the MMF produces profiles of cloud occurrence that compare favorably with radar observations (using either a −40- or −25-dBZe threshold). There is some indication that low-level (less than 3 km) hydrometeor fractions in post-cold-frontal conditions may be overpredicted.
For warm-frontal conditions (as represented by states 1 and 3), the MMF tends to produce hydrometeor fractions that are too large below 7 km (using either a −40- or −25-dBZe reflectivity threshold). Comparison of MMF output with CloudSat observed reflectivity height joint histograms has shown that the MMF tends to produce precipitating hydrometeor coverage that is too large in the midlatitude storm tracks (Marchand et al. 2009) in agreement with this finding.
Comparisons in states 1 and 3 also indicate that the hydrometeor fractions are likely too large above 7 km using a −40-dBZe threshold and yet too small using a −25-dBZe threshold. It may be that the total amount of condensate is about right but spread out over too large a volume (perhaps because of the model’s limited vertical resolution). States 2 and 4 may also contain some warm-frontal conditions and also display too much low-reflectivity cloud relative to the total hydrometeor coverage above 7 km.
The MMF does not appear to correctly capture the formation of low clouds in those states where low-level moisture is being advected from the Gulf of Mexico over the ARM site (states 2 and 12).
In several states, including state 8 and the four states that occur during June, July, and August (states 9, 10, 11, and 12), the MMF produces too much high and thin cloud, especially above 10 km. This result appears to be a common feature of the model in convective regions (Ovtchinnikov et al. 2006; Zhang et al. 2008). In all of these states, the MMF produces hydrometeor occurrence profiles that compare more favorably with observations using a −25-dBZe threshold than a −40 dBZe-threshold.
It is not immediately clear why the MMF should represent cold-frontal conditions better than warm-frontal conditions. We speculate that the two-dimensional nature of the cloud resolving model and its east–west orientation may be a factor. While the comparison presented here provides some insight into possible sources of error in the MMF clouds and precipitation structure, in the future we hope to use the atmospheric states to directly study the cloud resolving model used in the MMF. We plan to run the CRM using the atmospheric states (or more precisely composites of the MMF output including advective tendencies) to see if we can reproduce the MMF cloud occurrence profiles. If running the CRM with composite conditions does reproduce the MMF occurrence profiles (among other observations), the composite states might well be used to test improvements in the CRM (e.g., microphysics or subgrid-scale turbulence schemes) or the model configurations (e.g., grid resolution, 2D versus 3D, etc.) without having to run the full MMF. Alternatively, the time series of states from the RUC analysis might be used to obtain representative case studies. As with all case studies, this latter approach will require constructing advective tendencies and other forcings from numerical weather prediction analysis or other data sources.
The atmospheric states developed here are complex patterns tailored for the ARM SGP site and this set cannot be applied to other areas. Naturally, this might lead one to question why we used an objective classification technique rather than, for example, using our knowledge of Oklahoma weather patterns to design a set of atmospheric states that captures characteristics that we know are important for this location. The answer is that we eventually hope to apply the classification technique to obtain a customized set of atmospheric states for many (if not every) grid cell in the GCM—a process that needs to be fully automated. Our hope is that this technique will prove sufficiently robust that a customized set can be generated for any location. However, the classification scheme may not be as effective in identifying distinct profiles of hydrometeor occurrence during convectively dominant atmospheric conditions because the synoptic fields may not contain sufficient information to effectively capture factors that influence convective initiation or efficiency. We plan to test this hypothesis by applying the classification scheme to other locations including the ARM tropical western Pacific sites and also plan eventually to test the technique using satellite observations [e.g., from the NASA CloudSat, Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO), Multiangle Imaging Spectroradiometer (MISR) sensors] rather than ground-based radar.
One of the difficulties faced by all objective classification schemes is how to determine the optimal number of states. Our solution is to initially choose too many classes and then allow the local-scale data (radar profiles of cloud occurrence in the present analysis), which are not used in the state definition, to drive the determination of what is or is not a useful class. This process ensures that the identified states are associated with statistically meaningful differences in cloud occurrence (at the local scale) and that the differences are also statistically stable and can therefore be useful for comparisons of observations with model output. Because the hydrometeor profiles are only used to help guide the identification of states and are not used in actually defining the state, once the state definition is constructed it can be applied to any model that produces the needed large-scale fields (without having to simulate profiles of hydrometeor occurrence).
While effective, this process will tend to force atmospheric states that produce similar profiles of hydrometeor occurrence to merge, even if the hydrometeors happen to be generated by different processes. Another weakness of the classification approach is that the neural network state definitions (as described in section 4) are essentially composites of all inputs assigned to a given state (during the network training phase). Individual meteorological conditions may not fit into any of the definitions very well, with the effect that these misfits increase the within-state variance of the observed or simulated data and reduce the effectiveness of comparisons. We know the distance of each input to the state definition and could potentially remove inputs that are far from any state definition in the data analysis. It is possible that such filtering could reduce the net within-state variability and thus improve comparisons with model output. It is also possible that analysis of the within-state distances could be used to help determine if a state has multiple internal clusters that might serve as additional states and thereby help address the potential problem of overmerging. We hope to research these and other technique improvements in the future.
Acknowledgments
The authors would like to express thanks to the Department of Energy (DOE) Pacific Northwest National Laboratory (PNNL) for providing the computer time and support needed to run the MMF climate model, without which this research would not have been possible. Also to Marat Khairoutdinov (Stony Brook University) and David Randall (Colorado State University) for providing us with their original MMF code and for the helpful discussions and ideas. This research was supported by the DOE Atmospheric Radiation Measurement Program (ARM), and we thank the many member of ARM responsible for gathering, processing, and disseminating cloud radar and other data.
REFERENCES
Benjamin, S. G., K. Brewster, R. Brummer, B. F. Jewett, T. W. Schlatter, T. L. Smith, and P. A. Stamus, 1991: An isentropic three hourly data assimilation system using ACARS aircraft observations. Mon. Wea. Rev., 119 , 888–906.
Benjamin, S. G., J. M. Brown, K. J. Brundage, D. Devenyi, B. E. Schwartz, T. G. Smirnova, T. L. Smith, and F-J. Wang, 1996: Recent upgrades to MAPS/RUC. NOAA Forecast System Laboratory Forum, 33 pp.
Bony, S., and J-L. Dufresne, 2005: Marine boundary layer clouds at the heart of tropical cloud feedback uncertainties in climate models. Geophys. Res. Lett., 32 , L20806. doi:10.1029/2005GL023851.
Bony, S., and Coauthors, 2006: How well do we understand and evaluate climate change feedback processes? J. Climate, 19 , 3445–3482.
Clothiaux, E. E., T. P. Ackerman, G. G. Mace, K. P. Moran, R. T. Marchand, M. A. Miller, and B. E. Martner, 2000: Objective determination of cloud heights and radar reflectivities using a combination of active remote sensors at the ARM CART sites. J. Appl. Meteor., 39 , 645–665.
Coleman, J. S. M., and J. C. Rogers, 2007: A synoptic climatology of the central United States and associations with Pacific teleconnection pattern frequency. J. Climate, 20 , 3485–3497.
Collins, W. D., and Coauthors, 2006: The formulation and atmospheric simulation of the Community Atmosphere Model Version 3 (CAM3). J. Climate, 19 , 2144–2161.
DeMott, C. A., D. A. Randall, and M. Khairoutdinov, 2007: Convective precipitation variability as a tool for general circulation model analysis. J. Climate, 20 , 91–112.
Fereday, D. R., J. R. Knight, A. A. Scalfe, and C. K. Folland, 2008: Cluster analysis of North Atlantic–European circulation types and links with tropical Pacific sea surface temperatures. J. Climate, 21 , 3687–3703.
Gordon, N. D., J. R. Norris, C. P. Weaver, and S. A. Klein, 2005: Cluster analysis of cloud regimes and characteristic dynamics of midlatitude synoptic systems in observations and a model. J. Geophys. Res., 110 , D15S17. doi:10.1029/2004JD005027.
Grabowski, W. W., 2001: Coupling cloud processes with the large-scale dynamics using the Cloud-Resolving Convection Parameterization (CRCP). J. Atmos. Sci., 58 , 978–997.
Haykin, S., 1999: Neural Networks: A Comprehensive Foundation. 2nd ed. Prentice Hall Publishers, 842 pp.
Haynes, J. M., R. T. Marchand, Z. Luo, A. Bodas-Salcedo, and G. L. Stephens, 2007: A multipurpose radar simulation package: QuickBeam. Bull. Amer. Meteor. Soc., 88 , 1723–1727.
Jakob, C., G. Tselioudis, and T. Hume, 2005: The radiative, cloud, and thermodynamic properties of the major tropical western Pacific cloud regimes. J. Climate, 18 , 1203–1215.
Jung, T., T. N. Palmer, and G. J. Shutts, 2005: Influence of a stochastic parameterization on the frequency of occurrence of North Pacific weather regimes in the ECMWF model. Geophys. Res. Lett., 32 , L23811. doi:10.1029/2005GL024248.
Khairoutdinov, M. F., and D. A. Randall, 2001: A cloud resolving model as a cloud parameterization in the NCAR community climate model: Preliminary results. Geophys. Res. Lett., 28 , 3617–3620.
Khairoutdinov, M. F., and D. A. Randall, 2003: Cloud resolving modeling of the ARM Summer 1997 IOP: Model formulation, results, uncertainties, and sensitivities. J. Atmos. Sci., 60 , 607–625.
Khairoutdinov, M. F., D. A. Randall, and C. DeMott, 2005: Simulations of the atmospheric general circulation using a cloud-resolving model as a superparameterization of physical processes. J. Atmos. Sci., 62 , 2136–2154.
Kohonen, T., 1995: Self-Organizing Maps. Springer, 501 pp.
Lin, W. Y., and M. H. Zhang, 2004: Evaluation of clouds and their radiative effects simulated by the NCAR community atmospheric model against satellite observations. J. Climate, 17 , 3302–3318.
Marchand, R. T., N. Beagley, S. Thompson, T. P. Ackerman, and D. M. Schultz, 2006: A bootstrap technique for testing the relationship between local-scale radar observations of cloud occurrence and large-scale atmospheric fields. J. Atmos. Sci., 63 , 2813–2830.
Marchand, R. T., J. Haynes, G. G. Mace, T. Ackerman, and G. Stephens, 2009: A comparison of simulated cloud radar output from the multiscale modeling framework global climate model with CloudSat cloud radar observations. J. Geophys. Res., 114 , D00A20. doi:10.1029/2008JD009790.
McFarlane, S. A., J. H. Mather, and T. P. Ackerman, 2007: Analysis of tropical radiative heating profiles: A comparison of models and observations. J. Geophys. Res., 112 , D14218. doi:10.1029/2006JD008290.
Norris, J. R., and C. P. Weaver, 2001: Improved techniques for evaluating GCM cloudiness applied to the NCAR CCM3. J. Climate, 14 , 2540–2550.
Ovtchinnikov, M., T. P. Ackerman, R. T. Marchand, and M. Khairoutdinov, 2006: Evaluation of the multiscale modeling framework using data from the atmospheric radiation measurement program. J. Climate, 19 , 1716–1729.
Randall, D. A., M. F. Khairoutdinov, A. Arakawa, and W. W. Grabowski, 2003: Breaking the cloud-parameterization deadlock. Bull. Amer. Meteor. Soc., 84 , 1547–1564.
Rossow, W. B., G. Tselioudis, A. Polak, and C. Jakob, 2005: Tropical climate described as a distribution of weather states indicated by distinct mesoscale cloud property mixtures. Geophys. Res. Lett., 32 , L21812. doi:10.1029/2005GL024584.
Schultz, D. M., 2004: Cold fronts with and without prefrontal wind shifts in the central United States. Mon. Wea. Rev., 132 , 2040–2053.
Smyth, P., K. Ide, and M. Ghil, 1999: Multiple regimes in Northern Hemisphere height fields via mixture model clustering. J. Atmos. Sci., 56 , 3704–3723.
Tselioudis, G., and C. Jakob, 2002: Evaluation of midlatitude cloud properties in a weather and a climate model: Dependence on dynamic regime and spatial resolution. J. Geophys. Res., 107 , 4781. doi:10.1029/2002JD002259.
Tselioudis, G., Y. Zhang, and W. B. Rossow, 2000: Cloud and radiation variations associated with northern midlatitude low and high sea level pressure regimes. J. Climate, 13 , 312–327.
Williams, K. D., and M. J. Webb, 2008: A quantitative climate performance assessment of cloud regimes in GCMs. Climate Dyn., 29 , 231–250. doi:10.1007/s00382-007-0232-2.
Wyant, M. C., C. S. Bretherton, J. T. Bacmeister, J. T. Kiehl, I. M. Held, M. Z. Zhao, S. A. Klein, and B. J. Soden, 2006: A comparison of tropical cloud properties and responses in GCMs using mid-tropospheric vertical velocity. Climate Dyn., 27 , 261–279. doi:10.1007/s00382-006-0138-4.
Zhang, Y., and Coauthors, 2008: On the diurnal cycle of deep convection, high-level cloud, and upper troposphere water vapor in the multiscale modeling framework. J. Geophys. Res., 113 , D16105. doi:10.1029/2008JD009905.
Zivkovic, M., and J-F. Louis, 1992: A new method for developing cloud specification schemes in general circulation models. Mon. Wea. Rev., 120 , 2928–2941.
APPENDIX
Relationship to Previous 25-State Set
Many of the states (10, 11, 12, 13, 15, 16, 19, and 25) in the original 25-state set (M06) featured north-northwesterly surface winds and were associated with cold-frontal systems or postfrontal conditions. These states are distinguished by factors such as differences in the surface temperature, strength of the surface high pressure, or the amount of upper- and midlevel moisture (see discussion in M06). While in some sense meteorologically distinct, many of the states are were poorly populated with ARM data. In the current classification, these states have been combined into states 5, 6, and 7—as listed in Table A1. This highlights that the neural networks classifier returns the “best match” to each input pattern and so state 5, whose composite shows a clear cold or stationary front at the ARM site, will naturally include radar observations from some prefrontal and postfrontal conditions. If a longer (more than 3 yr) dataset was used, it is likely that more states could be identified (with stable and distinct hydrometeor occurrence profiles), representing more varied frontal conditions.
Some of the 25 M06 states have a very close match in the new 12-state set. For example, M06 states 1 and 8 are very similar to new states 12 and 8, respectively. However, some features in the 25 states are no longer captured distinctly. For example, M06 state 7 is a summertime cool-front case (enough to relieve the heat with some cooler air, but certainly not a good blast of cold air). This state has no direct analog in the new set. The cases that would have gone in M06 state 7 are primarily split between new states 10 and 11 (neither of which is a very good fit). Similarly, several of the original states, for example, M06 states 5 and 6, featured easterly surface winds. The cases that would have gone into these states are now contained in the new states 11 (weak winds) and 12 (weak, southerly winds), respectively. Table A1 shows how the M06 states map into the new 12-state set. Those M06 states listed in Table A1 that list two destinations are states that are not well captured by any single state in the new set.
Example of an atmospheric state definition, state 1. (top left) The 1000-hPa temperature dewpoint (°C) overlaid by contour plot of 1000-hPa geopotential heights. Arrows indicate wind direction and velocity. (middle left) Surface pressure anomaly (hPa) with 500-hPa geopotential heights and winds. (bottom left) The 500-hPa relative humidity (fraction) with 500-hPa geopotential heights and winds. (top right) The 1000-hPa temperature (°C) with 1000-hPa geopotential heights and winds. (middle right) The 875-hPa temperature (°C) with 875-hPa geopotential height and winds. (bottom right) The 375-hPa relative humidity (fraction) with 375-hPa geopotential heights and winds.
Citation: Journal of Climate 22, 17; 10.1175/2009JCLI2638.1
Four examples (from starting 25-state set) comparing hydrometeor occurrence profiles observed by the ARM cloud radar for the years 1999 and 2001. The profile for 1999 is shown in blue and 2001 is shown in red. The label at the top of each plot shows the fraction of time occupied by each state, where, e.g., (R99)(0.02) means that 2% of the data from 1999 falls within this state, along with the p value from the global similarity hypothesis test. The thin black line on the right side of each plot indicates what levels have a sufficient number of samples to make a robust comparison. Individual altitudes where the profiles do not appear to be different at the 95% level of confidence are marked with an asterisk. (left) The global p value is larger than 0.05 suggesting that the difference in the profiles is not significant at the 95% level of confidence. These states are considered stable and kept in the first refinement iterations (see text section 2). (top right) A case where the differences are significant, as given by the p value. The missing asterisks highlight individual levels that are likely different (at the 95% level of confidence). (bottom right) While the profiles appear relatively similar, there are insufficient points at any altitude to make a robust comparison. Both states shown in the right column are modified during refinement.
Citation: Journal of Climate 22, 17; 10.1175/2009JCLI2638.1
Depiction of scheme used to generate objective atmospheric states.
Citation: Journal of Climate 22, 17; 10.1175/2009JCLI2638.1
Comparison of MMF simulated profiles of hydrometeor occurrence at −40 dBZe with ARM cloud radar observations at −40 dBZe. MMF shown in red, ARM (via RUC) in blue. Other lines and symbols as per Fig. 2.
Citation: Journal of Climate 22, 17; 10.1175/2009JCLI2638.1
As in Fig. 4, but using a −25-dBZe threshold.
Citation: Journal of Climate 22, 17; 10.1175/2009JCLI2638.1
Depiction of the annual cycle of states for the (a) RUC model and (b) MMF. (c) Difference, RUC − MMF. The frequency of occurrence is normalized to 1 for each month (i.e., each column).
Citation: Journal of Climate 22, 17; 10.1175/2009JCLI2638.1
Summary of atmospheric states.
Table A1. Relationship between atmospheric states in M06 and new 12-state set.
