Abstract

An object-based evaluation method is applied to the simulated orographic precipitation for the idealized experimental setups using the National Center of Atmospheric Research (NCAR) Community Atmosphere Model (CAM) with the finite volume (FV) and Eulerian spectral transform dynamical cores with varying resolutions. The method consists of the application of k-means cluster analysis to the precipitation features to determine their spatial boundaries and the calculation of the semivariograms (SVs) for the isolated features for evaluation.

The quantitative analysis revealed differences between the simulated precipitation by the FV and Eulerian spectral transform models that are not visually apparent. The simulated large-scale precipitation features of the idealized test cases provide analogs to orographic precipitation features observed in simulations of Atmospheric Model Intercomparison Project (AMIP) models. The spatial boundaries of these features (determined by k-means clustering) for Eulerian spectral T85 and T170 resolutions revealed the level of merger between the two large-scale features simulated because of each peak in the double mountain idealized setup. Both FV 1° and 0.5° resolutions were able to simulate the dryer region between the two mountains. The SVs of precipitation for the single and double mountain setups show close agreement between FV 1°, FV 0.5°, and Eulerian spectral T170 resolutions; however, Eulerian spectral T85 simulated the precipitation in lower intensity, indicating the qualitative difference in resolutions previously determined to be equivalent. Such close agreement was not observed in the more realistic idealized setup.

1. Introduction

In recent years, there has been a significant increase in the progress of global circulation models (GCMs), including improvements in dynamical cores, physical parameterizations, and computing power. Such progress leads to a demand for improved testing and validation techniques. Validation efforts in terms of identification and quantification of the biases in GCMs are crucial for improvement of future climate simulations. However, this is a challenging task for various reasons. One prominent complication is that the summary statistics using the traditional gridpoint by gridpoint calculations on climate model simulations leads to loss of local bias signature (Gilleland 2013). Identification of global versus local biases and addressing uncertainty about which model component produces the bias is important to understand how the model fails to simulate a given process correctly (Yorgun and Rood 2014). A visual or an average-based (i.e., root-mean-square error or correlation coefficient) statistical comparison of model simulation and the observation does not provide sufficient information as to what the biases are and how they should be addressed. Moreover, these methods do not account for the small biases in the location or timing of, say, small convective precipitation features and this problem is exacerbated by finer resolution (Ahijevych et al. 2009). As alternatives for these conventional techniques, we aim to develop object-based model evaluation strategies. An “object” is a coherent system with an associated set of measurable parameters (Douglass 2000). The details of our object-based approach can be found in Yorgun and Rood (2014), where we were motivated by the wintertime western U.S. orographic precipitation simulated by the finite volume (FV) and the Eulerian spectral transform dynamical core components of the National Center for Atmospheric Research (NCAR) Community Atmosphere Model (CAM) (Neale et al. 2010). The approach can be summarized as 1) characterization of the representation of orographic precipitation by GCMs, 2) linking the representation of orographic precipitation to GCM structure, and 3) identification and verification of orographic precipitation features. The results and discussion about the first two steps are given in Yorgun and Rood (2014). In this paper, we move toward a quantitative evaluation (third step).

An object-based method involves the classification of the field of interest into parts to be treated and evaluated as objects. This is a crucial step since the accuracy in defining and isolating the objects determines the overall reliability of the final results. To decide which class of object the grid point belongs to, the object classification is traditionally done by setting a threshold for a variable and evaluating the grid values relative to that threshold (i.e., if the grid value is above or below the threshold) (Davis et al. 2006; Ebert and McBride 2000; Micheas et al. 2007; Wernli et al. 2008). This has been a limiting step in object-based methods since user-defined threshold values can be insufficient in treating different types of objects that can be manifested in different times of simulations (e.g., a big threshold might miss the small features). This problem was addressed by several studies where more than one value as a threshold were employed to increase the capability of the method (Xu et al. 2005; Xu 2009; Posselt et al. 2012; Skok et al. 2013). We adopt an approach to increase the flexibility of the object identification by making dynamic threshold decisions according to the characteristics of the field of interest. In this study, we used k-means clustering (Everitt et al. 2011) for the purpose of object identification, which eliminates the user-defined threshold by setting a prespecified number of object classes. We also investigate the applicability of variography (Isaaks and Srivastava 1989) for comparison and verification of isolated objects. Section 3 gives information about both k-means cluster analysis and variography. Both methods have been used for evaluation of weather prediction and climate models (Alhamed et al. 2002; Johnson et al. 2011; Liu and George 2005; Marzban and Sandgathe 2006, 2009). We apply them to idealized test case results (Yorgun and Rood 2014) with an object-based point of view. Section 2 gives a brief summary of the results of the idealized test cases. We also apply variography to synthetic data that resemble orographic precipitation features to observe how variography quantifies important characteristics of objects (e.g., magnitude, location, size). Results of this analysis are given in section 3. Application of these methods to the idealized test cases and the discussion of the results are given in section 4, and conclusions are given in section 5.

2. Idealized test cases (qualitative analysis summary)

The first step of our object-based approach involves identification of precipitation features that are simulated differently by different models. We examined Atmospheric Model Intercomparison Project (AMIP) model simulations and the Global Precipitation Climatology Center (GPCC) observations (Rudolf et al. 2005; Schneider et al. 2013) over the California Coast Range and the Sierra Nevada mountains on the U.S. West Coast. The AMIP simulations under examination were produced by two different dynamical cores (Williamson 2007): namely, FV (Lin 2004) and spectral (Collins et al. 2006) both coupled with full physics parameterization within CAM3.0. Our analysis (Yorgun and Rood 2014) revealed significant differences in simulation of orographic precipitation by the CAM FV and CAM Eulerian spectral dynamical cores. The CAM FV model simulated two distinct precipitation bands aligned in front of the Coast Range and the Sierra Nevada with the drier region between the two mountain ranges, which resembles the GPCC observations. The CAM Eulerian spectral model simulated these two precipitation bands with lower peak intensities and merged them together creating a single feature. We picked these precipitation features as “study features” to be analyzed further and set up idealized experimental tests to reproduce them in a simplified environment.

The details of the setup and the initial conditions of the idealized experimental model runs can be found in Yorgun and Rood (2014). In summary, three experimental cases were created using Gaussian bell-shaped mountains with single mountain (case 1), double mountain (case 2), and realistic (case 3) setups. The realistic setup resembles the structure of our study domain Coast Range and Sierra Nevada in terms of the peak heights of the mountains and their separation from each other. However the fine structures of the mountains are not represented, as the synthetic Gaussian hill mountains are smooth. The specifications of the mountains for each setup are given in Table 1.

Table 1.

The mountain specifications for three experimental setups.

The mountain specifications for three experimental setups.
The mountain specifications for three experimental setups.

As can be seen in Table 1, cases 2 and 3 have two mountains in their setups and case 2 is labeled as “double mountain”; however, case 3 is labeled as “realistic” for clarity because of its attributes being selected to mimic the Coast Range and Sierra Nevada. An important result from the qualitative analysis is the identification of the different types of orographic precipitation features simulated in idealized test cases. These features are shown in a monthly mean precipitation plot for the double mountain case (Fig. 1).

Fig. 1.

The 30-day mean total precipitation (mm day−1), surface geopotential (topography) for double mountain setup of experimental test cases, simulated by (a) CAM Eulerian spectral T85, (b) CAM Eulerian spectral T170, (c) CAM FV 1°, and (d) CAM FV 0.5° dynamical cores. Surface geopotential contours are 500, 2000, 5000, and 11 000 m2 s−2, with the peak contour at the interior of the mountain.

Fig. 1.

The 30-day mean total precipitation (mm day−1), surface geopotential (topography) for double mountain setup of experimental test cases, simulated by (a) CAM Eulerian spectral T85, (b) CAM Eulerian spectral T170, (c) CAM FV 1°, and (d) CAM FV 0.5° dynamical cores. Surface geopotential contours are 500, 2000, 5000, and 11 000 m2 s−2, with the peak contour at the interior of the mountain.

The features as indicated by their corresponding numbers in the CAM Eulerian spectral T170 simulation (Fig. 1b) are

  1. large-scale features due to stable upslope ascent;

  2. small-scale features due to local evaporation and lee-side convergence; and

  3. features due to leeward baroclinic waves.

These features are different in their origins and evolutions; thus, it is important to identify and evaluate them as separate objects (third step of our object-based approach). In this paper, we focus on the large-scale stable upslope ascent features (1 in Fig. 1b) to quantify the differences between the CAM FV and CAM Eulerian spectral dynamical cores. We first apply k-means clustering to the 30-day mean precipitation field for all three topographical setups in order to define their spatial boundaries as objects and isolate them from the whole field. The isolated features are then matched with their analogs between each model simulation and variography is used to compare them and quantify their differences. The details of these methods are given in the next section.

3. Methods

a. Cluster analysis

Cluster analysis is a technique that uses a measure of similarity (or dissimilarity) to classify data into groups called clusters. Each cluster is formed in a way that its members have higher similarity to each other than to members of other clusters. Clustering has been extensively used in meteorology and climatology (Huth et al. 1993; Littmann 2000; Liu and George 2005). It has also been applied for object-based purposes including storm and cloud regime classification (Eitzen et al. 2008; Jakob and Tselioudis 2003; Lakshmanan et al. 2003; Peak and Tag 1994), verification of precipitation fields produced by weather prediction models (Marzban and Sandgathe 2006), and classification of multimodel ensemble data (Alhamed et al. 2002). These studies employed hierarchical cluster analysis methods. Hierarchical clustering makes a series of partitions, which may run from a single cluster containing all individuals, to n clusters, each containing a single individual. In this study, we use k-means clustering, which does not form hierarchical classifications but partitions the data into a prespecified number of groups. The partitioning algorithm involves calculating the centroid of each cluster via a measure of distance, where each centroid is the mean value of the points in that cluster. We use the squared Euclidian distance as the measure of distance. It should be noted that this distance is not the spatial distance between the grid points but rather the difference between the grid point values. In general, given an matrix where is the number of observations and is the number of variables, the squared Euclidian distance () is defined as

 
formula

The Euclidian distance () can be defined as the distance between two -dimensional points xi = (xi1, … . . , xip) and xj = (xj1, … . . , xjp) in Euclidian space. In our analysis, we reduce a latitude–longitude grid matrix of a given precipitation field into a single one-dimensional (1D) vector (, where = latitude longitude) to apply the k-means algorithm. In this case, the Euclidian distance () reduces to

 
formula

The sum of distances, over all clusters, from each point to its cluster centroid is then minimized iteratively. The points are moved between clusters until the sum cannot be decreased further, resulting in clusters as well separated as possible. The mathematical details of this method can be found in Spath (1985) and Everitt et al. (2011) 

Once the clusters are formed, the reduced vector is then expanded to the latitude–longitude grid to visualize the cluster boundaries. We used 2 (k = 2) and 3 (k = 3) clusters in our calculations. The k = 2 clustering is assumed to classify the whole field into nonrain and rain parts, whereas the k = 3 clustering classifies for nonrain, light rain, and heavy rain parts. This analysis serves as the identification of the orographic precipitation features by determining the boundaries of the rain areas to be analyzed as objects. These rain areas are analogous to the ones defined in Davis et al. (2006), where they are determined by convolving the precipitation field with a given shape as a smoothing process. The smoothed field is then thresholded to get the object boundaries.

b. Variography

1) Method

The objects determined by the spatial boundaries of the rain areas from cluster analysis are evaluated by applying variography. Variography is a geostatistical method, which is used to measure the spatial continuity of a variable Z. Marzban and Sandgathe (2009) applied variography as a verification method to numerical weather prediction models. In that study, variography was used to compare two fields; in this study, we apply the method to acquire information about isolated objects rather than the whole field.

The semivariogram (SV; ) is defined as the variance of the difference between two points separated by a distance . Assuming stationarity, which states that all pairs of random variables separated by a particular distance have the same joint probability distribution regardless of their locations (Isaaks and Srivastava 1989), the SV can be defined as

 
formula

where Var is the variance function and is the variable of interest (in our case precipitation), which is a function of location (Matheron 1963). Assuming the spatial variable is free of systematic surface trends yields

 
formula

where E is the expected value (mean) function. The above equation computationally yields

 
formula

where is the set containing all the neighboring pairs at distance [i.e., ] and is the total number of pairs. The SV function is then plotted against varying to observe the spatial continuity of the variable. The selection and the classification of data pairs are crucial in SV computation. Pairs might not be separated exactly by the selected distance , so distance classes are formed in order not to exclude them from the computation. The distance class L for two arbitrary points P1 and P2 is calculated as

 
formula

where is the distance between P1 and P2, denotes the largest integer ≤ for any arbitrary , and is the user-defined parameter determined depending on the case. If the points are paired in every possible direction (i.e., if a pair is determined only depending on the scalar separation distance but not a vector with an angle), the resulting SV is called an omnidirectional SV. However, points can be paired specifying an angle. If, say, we pair the points only in the 90° direction, the resulting SV is called a 90° directional SV and gives information about the spatial continuity in that particular direction. As in the case of distance specification, an angle tolerance value is defined to have enough number of pairs in the calculation. So, as an example, if we are looking at 90° direction and define an angle tolerance of 10°, we look for a pair of a point within a 90° ± 10° arc, rather than only looking at a line on 90° direction. The parameter and the angle tolerance are taken as 300 km and 10° respectively for all SV calculations presented in section 4.

2) Variography on synthetic data

Prior to application of this method to the idealized test case results, we tested the utility of variography using synthetically created data. Two Gaussian hill-shaped objects were created to represent the mountain range precipitation features. The objects were trimmed in two sides to create an elongated feature similar to a precipitation feature aligned with a mountain range. Figures 2a,b show the 3D plots of these two objects, Figs. 2c,d show their corresponding contour plots, and Fig. 2e shows the omnidirectional SV of the two objects.

Fig. 2.

3D plots of synthetic objects (a) 1 and (b) 2; (c),(d) their corresponding contour plots; and (e) their SVs. The size, length, and magnitude of the objects as well as the SV values are in arbitrary units.

Fig. 2.

3D plots of synthetic objects (a) 1 and (b) 2; (c),(d) their corresponding contour plots; and (e) their SVs. The size, length, and magnitude of the objects as well as the SV values are in arbitrary units.

The x and y distances (the horizontal domain) that the objects are created on are both 101 arbitrary units. Each object is 71 × 21 units in horizontal plane and they are created at the same orientation in the y direction: that is, 30 units away from both ends of y axis. However, the objects do not have this symmetry in the x direction. Object 1 starts 9 units away from the x axis and object 2 ends 20 units away from the end of the x axis. The maximum value (magnitude) attained by both objects is 1.

There are striking differences between the two SVs due to the dissymmetry of two objects in the x direction. When two objects are symmetric (e.g., the same unit distances from the center of the object and the end of the domain in the x direction), the resulting SVs would be identical because of the stationarity assumption. Advantageous from an object-based point of view, SVs carry information about the location of objects; however, one should be careful interpreting the results because of the sensitivity of SVs to shifts in location. The solid line SV in Fig. 2e exhibits characteristics of object 1 in a way that is similar to the synthetic data analysis done by Marzban and Sandgathe (2009). The SV has a bend at separation distance 22, which is about the size (length) of the object in the x direction. After that point, the SV steadily increases to the peak point at separation distance 42. This peak is related to the distance of the center of the object from the origin in Marzban and Sandgathe (2009), which in our case is around 53. This difference between the two synthetic data analyses is due to the difference in shapes of the objects. In our case we have a Gaussian hill with trimmed tails in the x direction; however, Marzban and Sandgathe (2009) had a bivariate Gaussian with same mean (30) and standard deviation (7.5) in both x and y. From Eq. (5), the SV value for a particular separation distance is calculated by adding the squared difference of the pairs divided by the total number of pairs separated by that distance. If we pick a separation distance longer than the size of the object on the direction parallel to the y axis (i.e., 0°) for object 1, we can pair points on the same direction both of which will remain outside the object. This means subtracting zero from zero, which does not add to the cumulative SV value. However, if we focus on separation distance approximately 42 (which is the distance where we observe our maximum SV peak in Fig. 2e) on the same direction; it can be seen that we always pair one point inside the object and one point outside: thus, the squared difference always gives a positive value. This leads to the maximum valued peak in the SV in Fig. 2e. The next peak in the SV is at separation distance 77, which is about the distance from the center of the object to the end of the domain in x axis, and this agrees with the discussion in Marzban and Sandgathe (2009) about their analogous peak.

Object 2 is in the same location as object 1 in the y direction but is pushed to the other end of the domain in the x direction. The corresponding SV exhibits one single peak at separation distance 38 and a slight bend in separation distance at around 70. The change in location of the object changed the behavior of the corresponding SV; however, we cannot observe signatures of that particular location, except the distance 70, which is the distance of the center of the object to the beginning of the domain in the x direction.

To extract detailed information about objects, SVs of particular directions should be employed. Figure 3 shows SVs of the two objects in four directions: 0°, 90°, 120°, and 135°. The 0° direction is the direction along the y axis, and the 90° direction is the direction along the x axis. The angle tolerance is 10°. In the directional SVs, we observe signatures of size and location of the objects as peaks at particular separation distances in Fig. 3. For the 90° SV, two peaks are observed for both objects, with the initial peak being at separation distance 20 and 22 and the latter peak being at separation distance 74 and 62 for objects 1 and 2, respectively. The first peak in both SVs is related to the size of the objects in the x direction and the second peak is related to the distance of the center of the object to the far end of the domain in the x direction. So, the 90° SV gives information about the attributes of both objects in the x direction. The 120° SV gives similar results, but there is a shift in peaks to longer separation distances. On the other hand, the 0° SV for both objects give information about the attributes in the y direction. There is a single peak at separation distance 40, which is both the size of the object in the y direction and the distance of the center of the object from both sides of the x axis.

Fig. 3.

Directional SVs of objects (a) 1 and (b) 2 for four different directions.

Fig. 3.

Directional SVs of objects (a) 1 and (b) 2 for four different directions.

The synthetic data analysis proves the capability of SVs on exhibiting various information about characteristics of an object on a single curve. However, the shortcoming of omnidirectional SVs is also apparent in the case of overlapping distances: for example, if an object’s size is similar to its distance from a particular end of the domain, both will be represented by the same peak or bend on an SV. To overcome this issue, directional SVs will be employed to identify the significance of each peak or bend on an omnidirectional SV. The directional SVs will not be presented in the discussion of results of idealized test cases; however, the information obtained from them will be indicated when necessary.

4. Discussion of results

a. Identification of objects by cluster analysis

K-means clustering was applied to monthly mean simulations of single mountain, double mountain, and realistic setups of idealized test cases discussed in Yorgun and Rood (2014). The object classification results for the double mountain setup (topographical setup 2) are presented in this section as a representation of the utility of clustering. The comparison and evaluation results for all three topographical setups (Table 1) will be presented in the next section. The object boundaries for k = 2 and k = 3 cluster analyses for simulations of the double mountain setup of two dynamical cores with two resolutions are given in Fig. 4.

Fig. 4.

K-means clustering results of the double mountain setup for k = 2 and k = 3 of (a),(e) CAM Eulerian spectral T85; (b),(f) CAM Eulerian spectral T170; (c),(g) CAM FV 1°; and (d),(h) CAM FV 0.5° dynamical cores.

Fig. 4.

K-means clustering results of the double mountain setup for k = 2 and k = 3 of (a),(e) CAM Eulerian spectral T85; (b),(f) CAM Eulerian spectral T170; (c),(g) CAM FV 1°; and (d),(h) CAM FV 0.5° dynamical cores.

The dark blue areas in Fig. 4 indicate the boundaries of the objects to be isolated and evaluated by variography. These areas are predominantly stable upslope precipitation (Fig. 1). For the k = 2 clustering (Figs. 4a–d) the clusters are labeled as rain areas (dark blue) and nonrain areas (light blue). The similarity of both large-scale upslope ascent features (labeled as objects 1 and 2 in Fig. 4d) between CAM FV 0.5° and CAM Eulerian spectral T170, which has been observed visually (Yorgun and Rood 2014) is apparent in Fig. 4 in terms of their spatial boundaries. The clustering reveals the level of merger between the two features simulated in the CAM Eulerian spectral T170 (shown in Fig. 4b). This merger was observed in the AMIP simulations of spectral T170, which motivated us to select these features as study features for further analysis. However, this merger was not visually apparent in the CAM Eulerian spectral T170 simulation for the double mountain setup (Fig. 1b). Unlike T170, the merger was visually apparent for CAM Eulerian spectral T85 and this is affirmed by cluster analysis, which produced a single rain area with an eastward extension (Fig. 4a).

The large-scale stable upslope features for CAM FV 1° and 0.5° show similarity in their spatial boundaries (Figs. 4c,d). The dryer region between the two mountains for both CAM FV models is included in the nonrain area cluster as an indication of the separation of two features. The cluster analysis suggests a stronger wet–dry contrast in the CAM FV core than in the CAM Eulerian spectral core. There is a stronger sensitivity to resolution in the spectral core, as well as a suggestion that at the higher resolution the two cores are more similar.

For the k = 3 clustering (Figs. 4e–h) the clusters are labeled as heavy rain areas (dark blue), light rain areas (red), and nonrain areas (light blue). With the inclusion of a light rain category the spatial pattern of the precipitation fields are more similar. The contrast between the CAM Eulerian spectral and CAM FV dynamical cores is less distinct with the k = 3 clusters.

The results of k = 3 clustering exhibits the flexible nature of cluster analysis on object boundary identification. The total area occupied by both heavy and light rain in k = 3 clustering is bigger than the area occupied by the rain areas in k = 2 clustering. As the number of clusters changes, the proximity of points between the clusters changes. As the number of predefined clusters increases, the total area of rain represented by clusters increases. This characteristic of the method is beneficial in terms of an object-based point of view, not only because it defines objects relative to values of the field of interest but also because it can divide an object into subparts, where the analysis of which can give valuable information.

Returning to the meteorological description of the rain in Fig. 1, the additional light rain category allows more distinct identification of the stable upslope precipitation with the dark blue clusters of Fig. 4e–h. Our analysis proceeds by using the object boundaries in Fig. 4 along with the precipitation values over the defined object areas. The points outside the object boundaries were assigned the value 0 and the original sizes of the field were kept the same, since SVs are sensitive to location. Then, variography is used for all simulations for comparison and evaluation. The comparison of objects with variography is given in the next section.

b. Comparison of objects by variography

In this section, we extract objects from the three idealized topographies described in Table 1. For the double mountain case, the stable upslope precipitation features are labeled as objects 1 and 2 (Fig. 4d). In the single mountain case (not shown), there is only one stable upslope feature, and it is labeled as object 1. Object 1 in both the single and double mountain cases are characterized by being on the windward, westward side of the domain. They are the features simulated where the moist air first impinges on a mountain slope.

The omnidirectional SVs of object 1 for the single mountain and double mountain cases are given in Fig. 5. These are for k = 2 clustering given in Figs. 4a–d. The merger for the CAM Eulerian spectral model simulations in k = 2 clustering (Figs. 4a,b) means, formally, that two objects are not identified for the CAM Eulerian spectral model simulations. To allow comparison of the precipitation on the westward peak, a line is drawn between two mountains at 93°W, and the westward part of the divided rain area is included in the SV calculation as object 1.

Fig. 5.

Omnidirectional SVs [(mm day−1)2] of the large-scale upslope ascent features (object 1) for (a) single mountain and (b) double mountain setups simulated by CAM Eulerian spectral T85, CAM Eulerian spectral T170, CAM FV 1°, and CAM FV 0.5° dynamical cores. The object boundaries were determined by k-means clustering where k = 2.

Fig. 5.

Omnidirectional SVs [(mm day−1)2] of the large-scale upslope ascent features (object 1) for (a) single mountain and (b) double mountain setups simulated by CAM Eulerian spectral T85, CAM Eulerian spectral T170, CAM FV 1°, and CAM FV 0.5° dynamical cores. The object boundaries were determined by k-means clustering where k = 2.

In Fig. 5, the magnitude difference of CAM Eulerian spectral T85 from other simulations is apparent in the SVs of both single and double mountain cases. This shows a quantitative measure of how CAM Eulerian spectral T85 simulates the large-scale upslope ascent feature in relatively lower intensity. Yorgun and Rood (2014) showed that the qualitative differences in CAM Eulerian spectral T85 were related to the spectral representation of topography. When fully resolved topography was used, the CAM Eulerian spectral T85 simulation was more like the other simulations. There is close agreement in CAM FV 1° and 0.5° SVs for the single mountain setup; however, they depart in the double mountain setup. The SVs of CAM FV 0.5° and CAM Eulerian spectral T170 show close agreement for the double mountain setup. As discussed in Yorgun and Rood (2014), Fig. 5 shows that the effect of resolution is much more pronounced in the case of the CAM Eulerian spectral dynamical core. In the simpler setup (single mountain), the CAM FV dynamical core does not seem to have been affected by resolution, as the SVs of both CAM FV resolutions are in agreement. In the more complex setup (double mountain), the agreement is between the higher-resolution simulations of both dynamical cores, suggesting the advantage of a higher resolution over the more complex terrain.

The shapes of the SVs of all simulations for single and double mountain setups are similar. As indicated in the discussion of the synthetic data analysis in section 3, directional SVs are important to interpret the peaks and bends of the omnidirectional SVs in Fig. 5 (which are numbered on the SV of CAM Eulerian spectral T85 in Fig. 5a). The initial bend at separation distance around 600 km (1 in Fig. 5a) observed in all four simulations of both cases is related to the width of the objects in the zonal direction; therefore, it shows up as a separate peak in the 90° directional SVs (not shown). The difference between the single and double mountain setups in Fig. 5 is where the peak SVs occur (2 in Fig. 5a). It is at around 1200 km for the single mountain and 1500 km for the double mountain setups for all simulations. This shift is due to the change in the location of the object related to the difference of the location of the windward mountain between the two topographical setups (Table 1). After the peak SV, there is another distinct bend in all four simulations at around 1700 km for the single mountain and 2000 km for the double mountain setups (3 in Fig. 5a). The 0° directional SVs show peaks at these distances for their corresponding setups, therefore this bend is indicative of the size of object 1 in the meridional direction. The large-scale stable upslope precipitation object extends to a longer distance in the case of the double mountain setup, which shows that the circulation in the leeward side (due to the second peak and in between) affects the precipitation in the windward side. Another effect of the addition of the second peak is the reduced rain amount of the windward (westward) large-scale stable upslope feature in the double mountain case. This is shown in Fig. 5 as the reduced SV values of the double mountain setup, where the total amount of precipitation is distributed to both mountains. A simple calculation of the total amount of rain shows that there is an increase in precipitation for all simulations in double mountain setup compared to the single mountain setup. This increase is 15% for CAM Eulerian spectral T85, 20% for CAM Eulerian spectral T170, 15% for CAM FV 1°, and 18% for CAM FV 0.5°. An analysis of the total amount of precipitation in double mountain setup shows that 30% of the rain is simulated over the leeward mountain for CAM Eulerian spectral T170, CAM FV 1°, and CAM FV 0.5°, whereas this percentage is 22% for CAM Eulerian spectral T85.

In the double mountain case, object 2 is associated with the eastward mountain. The SVs of object 2 defined by k = 2 clustering for all simulations is given in Fig. 6. The close agreement between CAM FV 1° and 0.5° and CAM Eulerian spectral T170 for object 1 is not observed in the SVs for object 2 of the double mountain setup. CAM FV 1° simulates the highest amount of rain in this object followed by FV 0.5° and CAM Eulerian spectral T170. Although object 2 is the manifestation of the same phenomenon as object 1 (stable upslope ascent), it is smaller in size and affected by the small-scale circulation in between the two mountain peaks. Therefore, this feature has more complex internal (unforced) dynamics and the reduction in spatial scales. The disagreement between simulations increases as the resolvable scales get smaller.

Fig. 6.

Omnidirectional SVs [(mm day−1)2] of the large-scale upslope ascent features of eastward mountain (object 2) for double mountain setup simulated by CAM Eulerian spectral T85, CAM Eulerian spectral T170, CAM FV 1°, and CAM FV 0.5° dynamical cores. The object boundaries were determined by k-means clustering where k = 2.

Fig. 6.

Omnidirectional SVs [(mm day−1)2] of the large-scale upslope ascent features of eastward mountain (object 2) for double mountain setup simulated by CAM Eulerian spectral T85, CAM Eulerian spectral T170, CAM FV 1°, and CAM FV 0.5° dynamical cores. The object boundaries were determined by k-means clustering where k = 2.

For the more realistic setup that resembles the Coast Range and the Sierra Nevada (Table 1), it is not possible to define similar objects across all of the simulations (Fig. 7). Therefore, the isolated objects are not labeled.

Fig. 7.

The 30-day mean total precipitation (mm day−1), surface geopotential for case 3 simulated by (a) CAM Eulerian spectral T85, (b) CAM Eulerian spectral T170, (c) CAM FV 1°, and (d) CAM FV 0.5°. Surface geopotential contours are 500, 2000, 5000, and 11 000 m2 s−2, with the peak contour at the interior of the mountain.

Fig. 7.

The 30-day mean total precipitation (mm day−1), surface geopotential for case 3 simulated by (a) CAM Eulerian spectral T85, (b) CAM Eulerian spectral T170, (c) CAM FV 1°, and (d) CAM FV 0.5°. Surface geopotential contours are 500, 2000, 5000, and 11 000 m2 s−2, with the peak contour at the interior of the mountain.

Both CAM FV model resolutions were able to simulate the dry region between the mountains, whereas there is a merge between two precipitation features in the cases of CAM Eulerian spectral T85 and T170. In both CAM FV simulations, a precipitation feature occurs at the peak of the first (westward) mountain, which is lower than the second mountain. On the second mountain, two separate precipitation features are observed over both the north and south ends of the Gaussian bell shape. This separation of large-scale stable upslope ascent features into two is not observed in either CAM Eulerian spectral simulation.

Figure 8 shows the SVs of the objects defined by k = 2 clustering, simulated by CAM FV 1° and 0.5° and CAM Eulerian spectral T85 and T170 for the realistic setup given in Fig. 7. This is simple separation into rain and nonrain. In this case, the sensitivity to resolution is apparent, with CAM FV 0.5° simulating higher amounts of rain followed by CAM Eulerian spectral T170 and CAM FV 1°. The difference between each simulation is much more pronounced compared to the previous cases where the scales of the features were bigger. CAM Eulerian spectral T85 produced very light rain and CAM Eulerian spectral model with both T85 and T170 resolutions produced a single feature over the mountains, whereas CAM FV model with both 1° and 0.5° resolutions were able to simulate the dry region between two peaks as seen in Fig. 7 [which was observed in the AMIP simulations discussed in Yorgun and Rood (2014)]. This qualitative difference between the simulations, the merging of the precipitation associated with the two mountains, is not obvious in the SVs in Fig. 8, as the shapes of the SVs are similar for all simulations. That is because the area between the two mountains is small in scale and in this case SV favors longer distances such as the distance of objects from the sides of the domain. (The first peak in Fig. 8 at around 1300 km is the distance of the objects to the westward end of the domain.)

Fig. 8.

Omnidirectional SVs [(mm day−1)2] of the precipitation features for the realistic setup simulated by CAM Eulerian spectral T85, CAM Eulerian spectral T170, CAM FV 1°, and CAM FV 0.5° dynamical cores. The object boundaries were determined by k-means clustering where k = 2.

Fig. 8.

Omnidirectional SVs [(mm day−1)2] of the precipitation features for the realistic setup simulated by CAM Eulerian spectral T85, CAM Eulerian spectral T170, CAM FV 1°, and CAM FV 0.5° dynamical cores. The object boundaries were determined by k-means clustering where k = 2.

As noted in Figs. 4e–h, the additional light rain category for the k = 3 cluster analysis allows more distinct identification of the stable upslope precipitation. This supports more investigation of the sensitivity to resolvable scales. Figures 9a–d show object 2, for the eastward, downwind peak of the double mountain case. The objects are for both k = 2 and k = 3 cluster analysis results. These objects are isolated with the precipitation outside of the objects set equal to zero.

Fig. 9.

3D plot of object 2 of the double mountain setup simulated by CAM FV 0.5° with (a),(b) k = 2 and k = 3 cluster analyses and (c),(d) CAM Eulerian spectral T170 with k = 2 and k = 3 cluster analyses. The omnidirectional SVs [(mm day−1)2] of CAM FV 0.5° and CAM Eulerian spectral T170 with k = 2 and k = 3 for (e) object 1 (westward peak) and (f) object 2 (eastward peak).

Fig. 9.

3D plot of object 2 of the double mountain setup simulated by CAM FV 0.5° with (a),(b) k = 2 and k = 3 cluster analyses and (c),(d) CAM Eulerian spectral T170 with k = 2 and k = 3 cluster analyses. The omnidirectional SVs [(mm day−1)2] of CAM FV 0.5° and CAM Eulerian spectral T170 with k = 2 and k = 3 for (e) object 1 (westward peak) and (f) object 2 (eastward peak).

Figures 9a,c are for the k = 2 cluster analysis. The change of the boundary from k = 2 to k = 3 clustering (Figs. 9b,d) can be viewed as how the light rain decided objectively by the cluster analysis is removed from the object. The remaining precipitation is the heavy rain, as labeled in the selection process of the number of clusters (Fig. 4). As can be seen in Fig. 9a, FV 0.5° simulates object 2 with peak intensity without much spread, which is not the case for spectral T170 (Fig. 9c). CAM Eulerian spectral T170 simulates the same object with lower intensity but higher spread. Referring back to Fig. 1, this light rain category is primarily the rain associated with small-scale dynamics and local evaporation (Yorgun and Rood 2014). This “spread out” effect of spectral dynamical core on precipitation was documented by Bala et al. (2008).

On the bottom right of Fig. 9 are the SVs for object 2, the rain on the eastward peak. These are for the higher-resolution simulations of the two dynamical cores for the k = 2 and k = 3 cluster analysis. By using k-means clustering together with variography, we are able to quantify how the distribution of rain in the heavy and light clusters differs in the two dynamical cores. As can be seen in Fig. 9f, the difference between k = 2 and k = 3 clustering for object 2 is represented by the difference in SVs with a much higher difference in the case of spectral T170 (indicated by a vertical line between the SV peaks of the CAM Eulerian spectral T170 k = 2 and k = 3). For comparison, we present the SVs for the precipitation on the westward, windward peak (object 1 in Fig. 9e). For this object the SVs for CAM FV 0.5° and CAM Eulerian spectral T170 are very similar for both k = 2 and k = 3 cluster analysis. We see that, for the westward, windward peak (object 1), the precipitation from the two dynamical cores agrees closely. The precipitation on the westward peak is largely defined by the design of the simulation experiment and is well resolved. The precipitation on the eastward, downwind peak is organized by dynamical features that develop after the interaction with the first peak. These dynamical features are of smaller scale than the flow that impinges on the westward peak. Though the dynamical characteristics of smaller-scale features are similar in all simulations, the characteristics of the precipitation are qualitatively and quantitatively different.

5. Conclusions

In this study, we have designed a set of idealized simulations to investigate the sensitivity of orographic precipitation to the dynamical core of the model. Therefore, we are investigating the sensitivity of a quantity normally associated with model physics to the model dynamics. Compared to either a comprehensive model or the atmosphere, the design of the simulations is simple. There are only two smooth mountains, and we use the simple-physics parameterization of Reed and Jablonowski (2012). These simple simulations do reproduce analogs of local biases that were observed in the AMIP simulations (Yorgun and Rood 2014). Even with this simplicity, significant complexity evolves in the simulations. Analysis of this complexity provides insights into the interactions between the dynamics and physics and the integrated outcome of those interactions, precipitation.

We applied k-means clustering to the idealized simulations for identification and isolation of orographic precipitation features as objects. Once the objects were defined, we applied variography to isolated objects and plotted their semivariograms (SVs) to compare and evaluate the features simulated by FV and spectral models in two resolutions to quantify their differences. Such quantitative analysis revealed the differences between FV and spectral dynamical cores better than a visual analysis. The merger of two precipitation features over the Coast Range and the Sierra Nevada mountains that was observed in AMIP simulations of CAM Eulerian spectral T170 was not visually apparent in the double mountain setup (Fig. 1). However, the k = 2 clustering exposed the merger for CAM Eulerian spectral T170 by identifying the area between the two mountains as “rain area,” whereas both FV 1° and 0.5° resolutions produced a dryer region between the mountains (Fig. 4).

We were able to quantify the tendency of spectral models to spread out precipitation by examining the difference between k = 2 and k = 3 clustering results. As shown in Fig. 9, the light rain area for CAM Eulerian spectral T170 for the smaller large-scale stable upslope feature (object 2) is bigger than that of CAM FV 0.5°. This difference reflected to the difference between k = 2 and k = 3 clustering SVs (Fig. 9f), which is larger than that of CAM FVs.

We also evaluated the utility of SVs applied to isolated objects rather than to a whole precipitation field. Our synthetic data analysis showed that a single SV plot carries information about an object’s magnitude, size, and location. This characteristic of SVs let us reveal the effect of the downwind mountain on the simulation of the precipitation because of the upwind mountain. The shift in the third SV peak (Fig. 5) between the single mountain and double mountain setups is a quantitative indication of the change of size of the object 1. We were also able to show how the double mountain setup simulated object 1 in lower intensity relative to the single mountain setup from the difference of the scales of the SVs for both setups (Fig. 5). This shows the utility of SVs in quantifying the differences, which are not visually apparent. One shortcoming of the omnidirectional SVs is that they tend to aggregate the objects in calculations when the objects are small and close to each other (as in the case of the realistic setup SVs shown in Fig. 8). This shortcoming can potentially be amplified with data with high temporal resolution (i.e., daily or hourly instead of monthly data) where small-scale features tend to appear more frequently. Another potential ambiguity is that, if two or more characteristics of an object (e.g., size and location) have similar values, SVs tend to show them on a single peak or bend, which makes the evaluation difficult, especially for simulations with coarse resolution. Therefore, separate directional SVs should always be employed in order to understand the different quantitative signatures related to an object.

We conclude by placing this work in context with some of the previous work on effective resolution of dynamical cores. Williamson (2008) examined the two dynamical cores used here to establish equivalent resolutions. Williamson used an aquaplanet configuration and examined zonal and global means of diagnostics relevant to momentum, energy, moist physics and the transport of those parameters. Williamson concludes that CAM FV 1° (0.5°) and CAM Eulerian spectral T85 (T170) are equivalent. With the introduction of simple topography in our study, we see significant quantitative and qualitative differences in the spatial patterns of precipitation at resolutions that Williamson (2008) determined to be equivalent. This suggests that topographic precipitation amplifies differences in the schemes that were not discernable in Williamson’s analysis.

Kent et al. (2014) explored the effective resolution of numerical algorithms used in dynamical cores of general circulation models. In most models the dynamics interact with the physics at the scale of the grid. However, at the scale of the grid, dynamical variables are far below the resolved scales of the numerical schemes. There is a range of scales between the grid size and a number of grid boxes where the dynamical features are not fully resolved. The suggestion from these results is that, as the dynamical variables span the scales from fully resolved to partially resolved and to the grid scale, the physics, which are acting at the grid scale, are not treated in the same way by the dynamical cores. Hence, the end result of the coupling between the dynamics and physics (e.g., the precipitation) takes on different characteristics for different dynamical cores. This will be explored more fully in future studies.

Acknowledgments

The authors thank Christiane Jablonowski, Derek Posselt, Kevin Reed, James Kent, and Jared Whitehead for their help in running idealized test cases and for their valuable input. We also would like to acknowledge the computing support by National Center for Atmospheric Research (NCAR) (especially Lawrence Buja) for the model runs. This study was funded by the National Aeronautics and Space Administration (NASA) (Award NNX08AF77G) and the U.S. Department of Energy (DOE) (Award DE-SC0006684), and by NOAA Climate Program Office (award number NA10OAR4310213), which supports the Great Lakes regional Integrated Sciences and Assessments Center (GLISA).

REFERENCES

REFERENCES
Ahijevych
,
D.
,
E.
Gilleland
,
B. G.
Brown
, and
E. E.
Ebert
,
2009
:
Application of spatial verification methods to idealized and NWP-gridded precipitation forecasts
.
Wea. Forecasting
,
24
,
1485
1497
, doi:.
Alhamed
,
A.
,
S.
Lakshmivarahan
, and
D. J.
Stensrud
,
2002
:
Cluster analysis of multimodel ensemble data from SAMEX
.
Mon. Wea. Rev.
,
130
,
226
256
, doi:.
Bala
,
G.
, and Coauthors
,
2008
:
Evaluation of a CCSM3 simulation with a finite volume dynamical core for the atmosphere at 1° latitude × 1.25° longitude resolution
.
J. Climate
,
21
,
1467
1486
, doi:.
Collins
,
W. D.
, and Coauthors
,
2006
:
The Community Climate System Model version 3 (CCSM3)
.
J. Climate
,
19
,
2122
2143
, doi:.
Davis
,
C.
,
B.
Brown
, and
R.
Bullock
,
2006
:
Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas
.
Mon. Wea. Rev.
,
134
,
1772
1784
, doi:.
Douglass
,
B. P.
,
2000
: Real-time UML: Developing Efficient Objects for Embedded Systems. Addison Wesley, 328 pp.
Ebert
,
E. E.
, and
J. L.
McBride
,
2000
:
Verification of precipitation in weather systems: Determination of systematic errors
.
J. Hydrol.
,
239
,
179
202
, doi:.
Eitzen
,
Z. A.
,
K. M.
Xu
, and
T.
Wong
,
2008
:
Statistical analyses of satellite cloud object data from CERES. Part V: Relationships between physical properties of marine boundary layer clouds
.
J. Climate
,
21
,
6668
6688
, doi:.
Everitt
,
B. S.
,
S.
Landau
,
M.
Leese
, and
D.
Stahl
,
2011
:
Cluster Analysis.
Wiley, 346 pp.
Gilleland
,
E.
,
2013
:
Testing competing precipitation forecasts accurately and efficiently: The spatial prediction comparison test
.
Mon. Wea. Rev.
,
141
,
340
355
, doi:.
Huth
,
R.
,
I.
Nemesova
, and
N.
Klimperova
,
1993
:
Weather categorization based on the average linkage clustering technique—An application to European midlatitudes
.
Int. J. Climatol.
,
13
,
817
835
, doi:.
Isaaks
,
E. H.
, and
R. M.
Srivastava
,
1989
:
Applied Geostatistics.
Oxford University Press, 561 pp.
Jakob
,
C.
, and
G.
Tselioudis
,
2003
:
Objective identification of cloud regimes in the tropical western Pacific
.
Geophys. Res. Lett.
, 30, 2082, doi:.
Johnson
,
A.
,
X. G.
Wang
,
F. Y.
Kong
, and
M.
Xue
,
2011
:
Hierarchical cluster analysis of a convection-allowing ensemble during the Hazardous Weather Testbed 2009 spring experiment. Part I: Development of the object-oriented cluster analysis method for precipitation fields
.
Mon. Wea. Rev.
,
139
,
3673
3693
, doi:.
Kent
,
J.
,
J. P.
Whitehead
,
C.
Jablonowski
, and
R. B.
Rood
,
2014
:
Determining the effective resolution of advection schemes. Part I: Dispersion analysis
.
J. Comput. Phys.
, 278, 485–496, doi:.
Lakshmanan
,
V.
,
R.
Rabin
, and
V.
DeBrunner
,
2003
:
Multiscale storm identification and forecast
.
Atmos. Res.
,
67–68
,
367
380
, doi:.
Lin
,
S. J.
,
2004
:
A “vertically Lagrangian” finite-volume dynamical core for global models
.
Mon. Wea. Rev.
,
132
,
2293
2307
, doi:.
Littmann
,
T.
,
2000
:
An empirical classification of weather types in the Mediterranean basin and their interrelation with rainfall
.
Theor. Appl. Climatol.
,
66
,
161
171
, doi:.
Liu
,
Z.
, and
R.
George
,
2005
: Mining weather data using fuzzy cluster analysis. Fuzzy Modeling with Spatial Information for Geographic Problems, F. E. Petry, V. B. Robinson, and M. A. Cobb, Eds., Springer Berlin Heidelberg, 105–119.
Marzban
,
C.
, and
S.
Sandgathe
,
2006
:
Cluster analysis for verification of precipitation fields
.
Wea. Forecasting
,
21
,
824
838
, doi:.
Marzban
,
C.
, and
S.
Sandgathe
,
2009
:
Verification with variograms
.
Wea. Forecasting
,
24
,
1102
1120
, doi:.
Matheron
,
G.
,
1963
:
Principles of geostatistics
.
Econ. Geol.
,
58
,
1246
1266
, doi:.
Micheas
,
A. C.
,
I. F.
Neil
,
S. A.
Lack
, and
C. K.
Wikle
,
2007
:
Cell identification and verification of QPF ensembles using shape analysis techniques
.
J. Hydrol.
,
343
,
105
116
, doi:.
Neale
,
R. B.
, and Coauthors
,
2010
: Description of the NCAR Community Atmosphere Model (CAM 5.0). NCAR Tech. Note NCAR/TN-486+STR, 289 pp.
Peak
,
J. E.
, and
P. M.
Tag
,
1994
:
Segmentation of satellite imagery using hierarchical thresholding and neural networks
.
J. Appl. Meteor.
,
33
,
605
616
, doi:.
Posselt
,
D. J.
,
A. R.
Jongeward
,
C. Y.
Hsu
, and
G. L.
Potter
,
2012
:
Object-based evaluation of MERRA cloud physical properties and radiative fluxes during the 1998 El Nino–La Nina transition
.
J. Climate
,
25
,
7313
7327
, doi:.
Reed
,
K. A.
, and
C.
Jablonowski
,
2012
: Idealized tropical cyclone simulations of intermediate complexity: A test case for AGCMs. J. Adv. Model. Earth Syst.,4, M04001, doi:.
Rudolf
,
B.
,
C.
Beck
,
J.
Grieser
, and
U.
Schneider
, cited
2005
: Global precipitation analysis products. DWD Global Precipitation Climatology Centre (GPCC) Publ., 8 pp.
Schneider
,
U.
,
A.
Becker
,
P.
Finger
,
A.
Meyer-Christoffer
,
M.
Ziese
, and
B.
Rudolf
,
2013
: GPCC's new land surface precipitation climatology based on quality-controlled in situ data and its role in quantifying the global water cycle. Theor. Appl. Climatol.,115, 15–40, doi:.
Skok
,
G.
,
J.
Bacmeister
, and
J.
Tribbia
,
2013
:
Analysis of tropical cyclone precipitation using an object-based algorithm
.
J. Climate
,
26
,
2563
2579
, doi:.
Spath
,
H.
,
1985
: Cluster Dissection and Analysis: Theory, FORTRAN Programs, Examples. Horwood, 226 pp.
Wernli
,
H.
,
M.
Paulat
,
M.
Hagen
, and
C.
Frei
,
2008
:
SAL—A novel quality measure for the verification of quantitative precipitation forecasts
.
Mon. Wea. Rev.
,
136
,
4470
4487
, doi:.
Williamson
,
D. L.
,
2007
:
The evolution of dynamical cores for global atmospheric models
.
J. Meteor. Soc. Japan
,
85B
,
241
269
, doi:.
Williamson
,
D. L.
,
2008
: Equivalent finite volume and Eulerian spectral transform horizontal resolutions established from aqua-planet simulations. Tellus,60A, 839–847, doi:.
Xu
,
K. M.
,
2009
:
Evaluation of cloud physical properties of ECMWF analysis and re-analysis (ERA) against CERES tropical deep convective cloud object observations
.
Mon. Wea. Rev.
,
137
,
207
223
, doi:.
Xu
,
K. M.
,
T. M.
Wong
,
B. A.
Wielicki
,
L.
Parker
, and
Z. A.
Eitzen
,
2005
:
Statistical analyses of satellite cloud object data from CERES. Part I: Methodology and preliminary results of the 1998 El Niño/2000 La Niña
.
J. Climate
,
18
,
2497
2514
, doi:.
Yorgun
,
M. S.
, and
R. B.
Rood
,
2014
:
An object-based approach for quantification of GCM biases of the simulation of orographic precipitation. Part I: Idealized simulations
.
J. Climate
,
27
,
9139
9154
, doi:.

Footnotes

Publisher’s Note: The online version of this article does not match the print volume due to a late addition to the acknowledgements section.