Abstract

The NCAR Community Climate Model (CCM, version 3.6) is evaluated as a numerical weather prediction model. The model was run in real-time mode at relatively high resolution (T126 or approximately 1°) to produce 10-day forecasts over a 1-yr period ending 1 March 2004. The evaluation of the performance of the CCM could be useful for both the climate modeling community as well as the operational forecast centers. For climate modelers, the higher-resolution, short-range forecasts can be used to diagnose deficiencies in the physical parameterizations in the model. While climate models may produce good mean climatologies, they may fail to simulate important higher-frequency phenomena that may be important to climate. For operational centers, the examination of an open, well-developed, and studied model could provide insights that could lead to improvement in their own models. Furthermore, the CCM could be considered a candidate as a member for a suite of models for use in an operational context. And, finally, as operational centers gradually extend their forecast range, and climate scientists are paying more attention to the subseasonal time scales, the study of a climate model in the short range becomes more appropriate.

1. Introduction

The National Center for Atmospheric Research (NCAR) Community Climate Model (CCM) has been used extensively worldwide by the climate science community to understand climate on relatively long term (decadal and longer) time scales. The model climatology has been well studied, and generally it is found that the CCM provides a very realistic simulation compared to observations (Kiehl et al. 1998; Hack et al. 1998; Hurrel et al. 1998) when examined in the mean (monthly or annual) sense. Recently there has been interest in looking at the higher-frequency characteristics of climate models, for example, daily weather, to gain a better perspective of how well the models are simulating the atmosphere and improve physical parameterizations (Phillips et al. 2004). It is now generally regarded in the scientific community that short-time-scale phenomena can have an important impact on the longer-term climate. For example, the diurnal cycle of convection is believed to play a key role in the propagation of the Madden–Julian oscillation. Hence examination of the CCM on short time scales may lead to better improvement in the physical parameterizations, and hopefully better simulation of the climate, both long (decadal) and short (seasonal) term.

In addition to improving climate simulations, there is increasing interest in exploring the gap between medium-range forecasting and climate prediction. There has been, for example, a strong push to explore predictive capability for the subseasonal (2 weeks to 2 months) time scale. Thus it is important to model developers to understand and bridge this gap in the model physics to better simulate the atmosphere for these time scales.

The evaluation of the CCM as a numerical weather prediction (NWP) model may also be of interest to operational centers. The CCM is a well-studied and developed model. It may thus potentially simulate certain aspects of the weather better than operational models, whose development is often constrained by operational requirements. If the CCM were to have better predictive capability of medium-range weather, even partially, it would presumably be of interest to the operational centers. Furthermore, operational centers are increasingly looking at multimodel ensemble forecasting methodologies. The CCM may function well as an ensemble member if its performance is comparable to the other members. While we evaluate the CCM as though it were an operational model, using standard skill scores and measures, the lack of a data assimilation system for the CCM limits the usefulness of these results for the NWP community. The data assimilation system is a major component of skill in numerical weather prediction. To assess the impact of the data assimilation on the forecasts, limited experiments using analyses from two major centers were used. However, in the absence of a data assimilation system this study may be of greater interest to the climate modeling community at present.

We have run the CCM 3.6 in an operational mode for over one year to provide high-resolution 10-day forecasts. We evaluate the performance of the model using commonly used skill measures and compare them with models from the major operational centers, primarily the Global Forecast System (GFS) operated by the National Centers for Environmental Prediction (NCEP) (Kalnay et al. 1990; Kanamitsu 1989; Kanamitsu et al. 1991; Caplan et al. 1997). We caution that such a comparison has a number of limitations, some of which we will address in the paper. For example, operational centers verify their forecasts against their own analyses, which will be impossible in our case. Also, skill measures such as anomaly correlation and root-mean-square have deficiencies that limit their usefulness in evaluating skill and identifying model problems. Nevertheless, such a comparison can provide a “first look” at the performance, and at least identify some areas where further research can be done to evaluate skill and improve model parameterizations and overall performance. The goal of this paper, then, is to provide this first look and suggest some areas for further research. In-depth analysis of why the scores are what they are is beyond the scope of this paper. Indeed, even reporting all of the basic scores that we have computed would be too lengthy to include here.

Using the CCM as an NWP model presents a number of challenges, and there are a number of pitfalls one may encounter in the process. In the next section, we discuss briefly the caveats of doing so. In section 3 we describe the experimental setup and operational parameters. We will discuss some important aspects and methodology of computing standard skill score measures in section 4. In section 5 we will discuss the results using standard operational error diagnostics and compare them with other major operational centers. Finally the conclusions are presented in section 6.

2. Caveats

When one looks at all the factors involved in comparing the CCM to an operational model, one could conclude that a fair comparison is not possible. Nevertheless, if these factors are understood and taken in proper perspective, one could potentially still learn a lot from such a comparison.

One issue for the CCM is that there is no data assimilation system. Hence the CCM must be initialized with analyses from one of the operational sources, which in our case are operational analyses provided by NCEP and a limited number of cases using European Centre for Medium-Range Weather Forecasts (ECMWF) analyses. These analyses use model-derived data to provide a first-guess field to fill in where no observational data are available. Thus operational analyses will have a built-in bias that will favor the operational model that is part of the assimilation system. This bias impacts in two ways. First, there will be a shock, or spin up, using analyses that are not in proper balance with the forecast model. We shall see this later when looking at precipitation spin up. Second, the verification of the forecast model will be done against the operational analyses, primarily from NCEP and ECMWF. Differences in model biases can lead to large apparent errors, which may not actually be real error in the sense that these biases are likely largest where there is no observational data. However, while using analyses from other centers may present problems as just described, the NCEP and ECMWF analyses are considered among the best analyses available, which can give the CCM an edge in comparison with other centers that may be using less accurate analyses.

In addition to there being no data assimilation system, there is also no readily available initialization system to initialize the model with arbitrary analyses. We have developed the initialization code to provide proper input to the CCM model. However, no normal-mode initialization or divergence filtering or such is performed. The lack of initialization could lead to spinup effects, due to both interpolation to the model grid by the initialization code, as well as the inconsistencies in model physics as mentioned above.

While operational center analyses are used to directly initialize the atmospheric model, there is no feasible way to directly initialize the land surface model. Thus, as described in the next section, the land model is spun up by forcing the land model with daily reinitialized atmospheric model runs. This is potentially a large source of error, though many operational centers lack a direct land assimilation system as well.

The CCM has been well tested and tuned for T42 resolution. We performed virtually no tuning when running the model at T126 resolution. Aside from code changes needed for efficiency and operational use, the only changes were to the fourth-order diffusion coefficient and the second-order Laplacian diffusion coefficient at the topmost level, and of course the time step.

3. Experiment

The forecast model used was the NCAR CCM version 3.6, which is described in detail in Kiehl et al. (1998). Operational forecasts were made daily from 1 March 2003 to 1 March 2004. The forecast length was 10 days. The horizontal resolution was T126 and the vertical resolution was 27 levels. The model was initialized with 1° operational Aviation [AVN; more recently referred to as GFS] analyses provided by NCEP.

The land model was spun up by an iterative process. A series of 1-day forecasts were made, where the 1-day land model forecast was used to provide the land initial condition for the subsequent day and the atmosphere was initialized daily with AVN analyses. This series of forecasts were run for 1 month, then the whole process was repeated using the land surface output from the final day of the month as the initial land condition for the subsequent monthlong series of runs. This was repeated several times until the forecasts yielded relatively small changes in temperature from the previous series. Finally the model was run for several months in the same manner as above prior and up to 1 March 2003.

The initial sea surface temperatures were updated daily using 1° final (FNL) SST analyses from the GFS. The SSTs were held fixed during the 10-day forecast period. Sea ice is determined to exist where the SST is below −1.8°C. Sea ice temperatures were not initialized to observed, but were evolved by the built-in ice model in the CCM. In retrospect, proper sea ice initialization should have been done, and the lack thereof may contribute to some of the errors that we see in the polar regions as discussed later.

We compute verification scores using skill measures commonly used by the major operational centers. We will compare our results with those of the other centers. Skill measures of the NCEP GFS and other operational models are provided in real time by P. Caplan and G. White and are available on the Web at http://wwwt.emc.ncep.noaa.gov/modelperf. Among the other operational center model skill scores that Caplan and White included are the following: ECMWF, U.K. Met Office (UKMET), Canadian Meteorological Centre (CMC), and the Fleet Numerical Oceanography Center (FNOC). Not all scores from all models are available.

To have some indication of the impact of the analysis used to initialize (and verify) the model, a limited number of hindcasts were done using ECMWF operational analyses (not reanalyses). These forecasts were done weekly starting 1 March 2003 until 12 July 2003, and then again weekly from 7 September 2003 until 22 February 2004. From mid-July to early September we did not have analyses to perform the forecasts. These forecasts will be designated CCME throughout the rest of the paper. Forecasts using AVN (GFS) analyses are designated CCMA. All verification scores and biases that we compute will be done against the analysis used to initialize the model. This will minimize biases that are due solely to biases between the analyses themselves. Verification scores provided by other operational centers are likewise computed by verifying against the center’s own analyses, as is recommended by the World Meteorological Organization (WMO; WMO 1999).

As stated earlier, no tuning of the model was done to run the model at higher resolution. Thus higher-resolution forecasts may have different biases than those at lower resolution. On the other hand, we may expect more skill from higher resolution. To understand resolution impact on skill and model biases, another limited experiment was done by running the model at T63 resolution and with ECMWF analyses. The forecasts, designated CCMT63E, cover the same dates as CCME. Forecasts were gridded to a regular 2.5° grid prior to analysis.

Land spinup for the CCME and CCMT63E cases was done similarly to the CCMA case, but using ECMWF analysis. For the approximately 50-day period during which we did not have ECMWF analyses, the AVN analyses were substituted.

4. Discussion on skill scores

A recommended set of skill scores and verifying procedures have been established by WMO (1999). The root-mean-square error (rms) and the anomaly correlation (AC) are two of the most frequently used skill measures used by operational centers. These measures may not be the most ideal for measuring the skill or usefulness of the forecast. In particular, these measures give a “double penalty” for missed forecasts, for example, if an event is slightly misplaced, say strong winds from a tropical storm, then there is a large contribution to the error both where the storm is incorrectly forecast to be, as well as where it was verified not to be. A forecast that did not even produce the event might in fact score better, though arguably it is a worse forecast in other respects. Since we aim to provide some comparison with other operational models, we will nevertheless use these skill scores.

Another difficulty with these skill measures is that they can be sensitive to smoothness (this is not entirely independent of the above problem). The WMO recommends verifying forecasts at 2.5° resolution. However, one may produce forecasts and analyses on the coarse 2.5° grid in any number of ways, such as linear, cubic or polynomial interpolation, and spectral truncation. The amount of smoothness introduced will vary significantly depending on the method used. The least smoothing is done when reducing resolution by using a subset (e.g., every other grid point) of the data. We found that reducing the grids to 2° resolution by taking every other grid point resulted in almost no change in any of the scores. Linear interpolation produces some smoothing depending on how closely the points used in the interpolation are to the target grid. Spectral truncation, on the other extreme, results in considerable smoothing as higher wavenumbers are explicitly removed. Also, smoothness may not only be introduced in postprocessing; it can also be due to the use of strong numerical diffusion in the model, for example.

To assess the impact of smoothing on the skill scores, we compute scores first on the original analysis grid, 1° × 1°, and then apply a smoothing function, a Gaussian filter, with an adjustable parameter to provide varying levels of smoothness. The filter was applied to both forecast and analysis grids (applying the filter to only either forecast or analysis produced results similar to using a reduced level of smoothing to both). The filter function, which essentially weights neighboring grid points, is given by

 
formula

Here x is the spatial distance to the neighboring grid point in degrees. We will use two particular values for the parameters: (a = 0.093, b = 10) for the case of weak smoothing and (a = 0.033, b = 3.4) for strong smoothing. The area integral of the filter is equal to 1, so the parameters are not independent. For weak smoothing, the filter has a half-width of about 1.54°, and for strong smoothing a half-width of about 2.63°. These levels of smoothing have the effect of reducing the effective resolution roughly near 2.5°. This will facilitate comparison with scores from other centers where the method of reducing the grids to 2.5° is not known or disclosed, as well as provide one estimate of the uncertainty of the scores.

For the results we present below, we will primarily present the unfiltered (no smoothing) results. However, for those cases where smoothing makes a large impact on the scores, such as the case for tropical winds, we include the smoothed results for comparison. Using no smoothing provides an upper bound on the errors, whereas strong smoothing will tend to give a lower bound. As a check, we interpolated the fields to 2.5° and verified that the scores produced intermediary results, usually close to the unsmoothed results.

In the results that follow, the monthly statistics are for the month of verification date, not the start date of the forecast period.

5. Results

a. 500-mb heights

Figure 1 shows the 5-day 500-mb height anomaly correlations for the entire year of operational runs, 1 March 2003 to 1 March 2004. The CCMA correlations are quite comparable to the other operational models and are particularly close to the CMC and FNOC models. In the Northern Hemisphere (NH) (Fig. 1a) the correlations tend to be slightly lower than all the models, except for FNOC. In the Southern Hemisphere (SH) (Fig. 1b), however, the CCMA correlations are overall competitive with the other models, except for ECMWF. In January the CCMA had the second highest correlation of the models in the SH, but was worse in the late austral winter months. While the operational models tended to have higher correlation in the Northern Hemisphere than the Southern Hemisphere, such a distinction was not as evident in the CCM. A plot (not shown) of the anomaly correlations, annually averaged, for both NH and SH as a function of forecast length were virtually identical for days 1 through 10.

Fig. 1.

Anomaly correlations of 5-day 500-mb heights for the (a) NH and (b) SH. [Scores from all models except CCM are from Caplan (2004).] Thick solid line corresponds to the CCMA experiment, and dashed line is the CCME experiment.

Fig. 1.

Anomaly correlations of 5-day 500-mb heights for the (a) NH and (b) SH. [Scores from all models except CCM are from Caplan (2004).] Thick solid line corresponds to the CCMA experiment, and dashed line is the CCME experiment.

For the CCME case (dashed line in Fig. 1), there was some improvement in the scores, especially in the SH. We caution that these scores are subject to some sampling error, as each month has only four to five forecasts for the CCME case. However, to the extent that these scores may be representative, the CCME performs near the middle for the NH for most months, and in the SH it is the first or second best model 8 out of 12 months of the year. The impact from using the high quality ECMWF analyses is clearly discernible. An interesting question remains why the improvement in greater in the SH than in the NH.

A comparison between the CCMA, CCME, and CCMT63E cases are provided in Fig. 2. Here only forecast dates that are in common with all three cases are shown. There were a few days missing in the CCMA cases due to various operational failures, particularly in September. The CCME cases have better scores than CCMA about 90% of the time. The improvement due to using higher resolution is also apparent.

Fig. 2.

Anomaly correlations of 5-day 500-mb heights for the three experiments CCMA, CCME, and CCMT63E for the (a) NH and (b) SH. CCMA is solid line, CCME is short dashed line, and CCMT63E is long dashed line.

Fig. 2.

Anomaly correlations of 5-day 500-mb heights for the three experiments CCMA, CCME, and CCMT63E for the (a) NH and (b) SH. CCMA is solid line, CCME is short dashed line, and CCMT63E is long dashed line.

A plot of the anomaly correlations for April and June 2003 for the NH and SH as a function of forecast length is shown in Fig. 3. Forecast skill by this measure is limited to about 6 to 7 days, where useful skill is defined as a correlation greater that 0.6, and is typically higher in April than in June. Generally, the CCME had the highest scores throughout the forecast period, with CCMA and CCMT63E performing comparably to each other.

Fig. 3.

Anomaly correlations of 500-mb heights as a function of forecast length for (a) April 2003 NH, (b) April 2003 SH, (c) June 2003 NH, and (d) June 2003 SH for the CCMA, CCME, and CCMT63E experiments. Solid line is CCME, short dashed line is CCMA, and long dashed line is CCMT63E.

Fig. 3.

Anomaly correlations of 500-mb heights as a function of forecast length for (a) April 2003 NH, (b) April 2003 SH, (c) June 2003 NH, and (d) June 2003 SH for the CCMA, CCME, and CCMT63E experiments. Solid line is CCME, short dashed line is CCMA, and long dashed line is CCMT63E.

Figure 4 shows the rms errors for the January and July heights as a function of forecast length for the NH and SH. The month of January had the largest height errors for the NH and lowest for the SH, whereas July was the opposite. The boreal and austral winter months, when large-scale synoptic patterns dominate the midlatitudes, had the largest and virtually the same magnitude of errors after 5 days.

Fig. 4.

Rmse (m) for January (dashed lines) and July (solid lines) as a function of forecast length and for each hemisphere.

Fig. 4.

Rmse (m) for January (dashed lines) and July (solid lines) as a function of forecast length and for each hemisphere.

The mean errors (bias) for selected months are shown in Fig. 5. The months were selected to give the most positive and most negative bias for each hemisphere, and roughly envelope the errors for all months. February has the most negative bias for NH, while November has the most positive bias for NH. In the SH, April had the largest positive bias while December had the most negative bias. In the NH, all months had a negative bias. December, January, and February had the largest negatives biases by far, and may be related to an NH wintertime warm land temperature bias, as will be discussed later, or perhaps due to blocking. The SH had relatively low bias, which did not increase during the forecast period.

Fig. 5.

Mean error (m) of 500-mb heights for select months as a function of forecast length.

Fig. 5.

Mean error (m) of 500-mb heights for select months as a function of forecast length.

b. Tropical winds

Figure 6 shows the monthly averaged anomaly correlations for the day-3 850-mb tropical winds for the entire year. The figure shows the results for no smoothing, weak smoothing (“SM1”), and strong smoothing (“SM2”) for the CCMA and CCME cases. For the CCMT63E case no smoothing was done as it was already at 2.5° resolution. The correlations, even with strong smoothing, were relatively poor, especially in the boreal summer. For comparison with other models, we only have results for February 2004. Caplan (2004) reports the anomaly correlations for other centers as follows: GFS—0.710, ECMWF—0.857, CMC—0.739, FNOC—0.721, and UKMET—0.780. Thus the CCMA (with values of 0.54 to 0.63) and CCME (with values of 0.61 to 0.7) do not compare well, though the CCME is not too far behind. However, we emphasize again that we are validating against analyses from a different model, and with relatively sparse observed data for tropical winds these analyses may have large systematic errors themselves. The fact that smoothing results in a moderate change in the scores indicates the errors are on relatively small scales. A potential source of error would be tropical convection.

Fig. 6.

Monthly averaged anomaly correlations of 3-day 850-mb tropical (20°S–20°N) winds. Solid lines correspond to the CCMA experiment and three levels of smoothing (none, SM1, and SM2). Long dashed lines are for the CCME case and same three levels of smoothing. The short dashed line is for the correlations for the CCMT63E experiment computed on the 2.5° grid.

Fig. 6.

Monthly averaged anomaly correlations of 3-day 850-mb tropical (20°S–20°N) winds. Solid lines correspond to the CCMA experiment and three levels of smoothing (none, SM1, and SM2). Long dashed lines are for the CCME case and same three levels of smoothing. The short dashed line is for the correlations for the CCMT63E experiment computed on the 2.5° grid.

Figure 7 shows the rms vector error growth as a function of forecast length for the month of February for the CCM and other operational models. The error growth for February was virtually the same as when averaged over the entire year. Again, the CCM generally scores worse than the other operational models, though when smoothed it comes very close to the GFS and CMC models. The CCME scores were nearly the same (not shown), indicating that the errors may not be analysis dependent but rather a systematic error in the model.

Fig. 7.

Root-mean-square vector error (m s−1) growth of 850-mb wind as a function of forecast length for the month of February 2004 for the CCM and other operational models. [Scores from all models except CCM are from Caplan (2004)]

Fig. 7.

Root-mean-square vector error (m s−1) growth of 850-mb wind as a function of forecast length for the month of February 2004 for the CCM and other operational models. [Scores from all models except CCM are from Caplan (2004)]

To better understand the source of apparent error in the tropical winds, we looked at the spatial vector error. The largest errors were found to be in the tropical Pacific near the equator, with the center of largest errors shifting during the year from the central and eastern Pacific to the western Pacific and Indonesia. In Fig. 8 we show the vector error averaged over the entire year for the CCMA, CCME, and CCMT63E cases. Here we have only included common forecast start dates. The annual bias reaches about 5 m s−1 in the central Pacific. In February (not shown) there are large errors just east of Indonesia with wind speed errors of up to 10–12 m s−1 as well as large errors just west of Indonesia. These regions are areas of large amounts of convective activity, and thus the displacement or overintensification of convection in the model is likely the source of these errors. All three cases show similar errors, indicating a systematic bias in the model. The similarity of the CCME and CCMT63E cases suggest this bias is not dependent on resolution. Comparing the CCMA and CCME cases, it appears that there is less bias using ECMWF analyses in most regions except for the central Pacific.

Fig. 8.

Rms vector error (m s−1) of 850-mb wind for weekly 5-day forecasts for the year 1 Mar 2003 to 29 Feb 2004 for (a) CCMA, (b) CCME, and (c) CCMT63E.

Fig. 8.

Rms vector error (m s−1) of 850-mb wind for weekly 5-day forecasts for the year 1 Mar 2003 to 29 Feb 2004 for (a) CCMA, (b) CCME, and (c) CCMT63E.

This easterly bias has been seen in the CCM when run in low-resolution climate mode. In Fig. 14 of Hurrel et al. (1998), there is a bias of up to 6 m s−1 of the u wind component when compared to NCEP reanalysis in the western Pacific.

c. Temperatures

Figure 9 shows the rms errors of 1200 UTC 850-mb temperatures for the NH, SH, and Tropics for the months of January and July. These months generally form the envelope of the errors for all months, with the winter month having the largest errors, and the summer month the least. The error growth for the Tropics was lowest, without much seasonal variation, as expected. The errors for the boreal and austral summers were similar. The boreal winter had larger errors after 5 days compared to the austral winter.

Fig. 9.

Rms error for 850-mb temperatures (°C) for NH, SH, and Tropics for January (dashed lines) and July (solid lines).

Fig. 9.

Rms error for 850-mb temperatures (°C) for NH, SH, and Tropics for January (dashed lines) and July (solid lines).

Figure 10 shows the hemispheric mean error for days 1, 3, 5, 7, and 10 for each month. The boreal summer has a cold bias, whereas the austral winter has a warm bias. The biases have a very seasonal characteristic and continue to increase as a function of forecast length even at 10 days.

Fig. 10.

Mean 850-mb temperature error (°C) for each month for days 1, 3, 5, 7, and 10.

Fig. 10.

Mean 850-mb temperature error (°C) for each month for days 1, 3, 5, 7, and 10.

The annual bias for CCMA, CCME, and CCMT63E (again only using common dates) are shown in Fig. 11. A cold bias over land is evident over all the continents. Over the ocean, however, a warm bias is clearly seen near the convergence zones as well as in the eastern Pacific off the South American continent. The annual bias for CCMA is very similar to the monthly biases (not shown), again indicating a rather systematic error. However, for the CCME case, the warm biases over ocean are considerably reduced. In the east Pacific the bias is even reversed. Understanding the differences in this east Pacific bias may aid in stratocumulus parameterization in that region, a difficult problem for many climate models. From looking at the CCMT63E case, it appears that resolution is not much of a factor in the bias. In all cases, there appear to be a cold bias over land, as well as a general cold bias across the midlatitudes, and a warm bias in the western Pacific and Indian Ocean. Near the North Pole there is a very strong warm bias in the CCMA case. We originally thought that this may be due to lack of proper sea ice temperatures, but this bias is much reduced in the CCME case. It may be that NCEP and ECMWF analyses assume different ice temperatures.

Fig. 11.

Mean error of 850-mb temperature (°C) for weekly 5-day forecasts for the entire year for (a) CCMA, (b) CCME, and (c) CCMT63E.

Fig. 11.

Mean error of 850-mb temperature (°C) for weekly 5-day forecasts for the entire year for (a) CCMA, (b) CCME, and (c) CCMT63E.

The cold bias over land appears to be consistent with that reported by Bonan (1998). In that study, the CCM had mostly a cold bias over large land areas during most seasons. In December–February, however, Bonan (1998) shows a cold bias over most regions below 40°N in the NH but a warm bias over most of the Siberian region. Our results show mostly a cold bias over Siberia, except for some small regions. This may be due to differences in snow cover due to spinup issues.

We show a zonal mean cross section of the temperature error for February 2004, in Fig. 12 for the CCMA and CCME cases. Near the surface there is about a 1–2-K cold bias, consistent with the 850-mb temperature bias as mentioned above, particularly over latitudes that contain significant landmass. This further illustrates there may be a bias in the land model or the surface fluxes. In the upper troposphere in the Tropics there is a warm bias of about 1–2 K in the CCMA case, but not much bias in the CCME case except for some midtropospheric warming. For the CCMA case we would expect excess tropical convection, which we discuss below, would be the cause of some of this warming. However, the CCME cases have approximately the same amount of precipitation in the Tropics, without as much upper-tropospheric warming. Hence more investigation is needed.

Fig. 12.

Zonal cross section of day-5 850-mb temperature error (°C) for the month of February 2004 for the (a) CCMA and (b) CCME experiment.

Fig. 12.

Zonal cross section of day-5 850-mb temperature error (°C) for the month of February 2004 for the (a) CCMA and (b) CCME experiment.

d. Moisture

During our initial analysis we noted a rather strong spinup of precipitation using AVN or ECMWF analysis that we did not see in earlier experiments with Global Data Assimilation and Prediction System (GDAPS) analyses obtained from the Korean Meteorological Administration. Figure 13 shows the area-averaged convective and large-scale precipitation over the Tropics (30°S to 30°N) as a function of forecast length for the CCMA, CCME, and CCMT63E cases for 4 January 2003 and an earlier experiment with the CCM using GDAPS analysis (CCMG) for 7 January 2001. These cases are very representative of the general trend, and it can be seen that there is a “spin down” of convective precipitation using ECMWF or AVN analyses, even at T63 resolution. This has led us to look further into the biases in the moisture field.

Fig. 13.

Single forecast area-averaged convective and large-scale precipitation in the Tropics as a function of forecast length for CCMG (open circles) using 7 Jan 2001 start date, and CCME (closed circles), CCMA (open squares), and CCMT63E (closed squares) using 4 Jan 2004 start date.

Fig. 13.

Single forecast area-averaged convective and large-scale precipitation in the Tropics as a function of forecast length for CCMG (open circles) using 7 Jan 2001 start date, and CCME (closed circles), CCMA (open squares), and CCMT63E (closed squares) using 4 Jan 2004 start date.

A vertical profile of the area-averaged change in specific humidity over the Tropics (30°S–30°N) is given in Fig. 14 for the January case mentioned above. Thus, the CCMA, CCME, and CCMT63E are drying the lower troposphere and moistening aloft, whereas the situation is reversed for CCMG. This effective pumping moisture aloft could lead to excessive precipitation as the upper levels cannot hold the amount of moisture removed from the lower levels, and may lead to saturation. Heating due to condensation can then further induce convection. Figure 13 does seem to show more large-scale precipitation for the CCMA, CCME, and to a lesser extent CCMT63E cases than CCMG in the Tropics. A more detailed study is needed to confirm this hypothesis, with particular attention paid to the first few hours of the forecast. Precipitation is discussed further in the next section.

Fig. 14.

Vertical profile of area-averaged mean error in tropical (30°S–30°N) specific humidity (g kg−1) for 5-day forecast starting 1200 UTC 4 Jan 2004 for CCMA (open squares), CCME (closed circles), CCMT63E (closed squares), and 1200 UTC 7 Jan 2001, for CCMG (open circles).

Fig. 14.

Vertical profile of area-averaged mean error in tropical (30°S–30°N) specific humidity (g kg−1) for 5-day forecast starting 1200 UTC 4 Jan 2004 for CCMA (open squares), CCME (closed circles), CCMT63E (closed squares), and 1200 UTC 7 Jan 2001, for CCMG (open circles).

Figure 15 shows the vertical profile of the mean differences in the analyses. It can be seen that the GDAPS analysis is much drier than AVN at the lower levels and more moist aloft. Hence this may explain why the CCM responds quite differently to GDAPS analyses than with AVN. It may be that the CCM planetary boundary layer–vertical diffusion parameterization tends to settle to a profile that is somewhat drier at the lower levels than the ECMWF or AVN analyses, but more moist than GDAPS. The ECMWF analysis is also somewhat drier at lower levels (and more moist around 500 mb and at surface), and may partly explain why there are less tropical biases in moisture and temperature for the CCME cases.

Fig. 15.

Vertical profile of area-averaged differences in analyses in the tropical specific humidity (g kg−1) for GDAPS minus AVN (open circles) and ECMWF minus AVN (closed circles). GDAPS analysis valid 1200 UTC 7 Jan 2001; all others valid at 1200 UTC 4 Jan 2004.

Fig. 15.

Vertical profile of area-averaged differences in analyses in the tropical specific humidity (g kg−1) for GDAPS minus AVN (open circles) and ECMWF minus AVN (closed circles). GDAPS analysis valid 1200 UTC 7 Jan 2001; all others valid at 1200 UTC 4 Jan 2004.

In Fig. 16 the annual biases in the 5-day forecasts for specific humidity are shown for the CCMA, CCME, and CCMT63E cases. It is readily apparent that there is a dry bias in all cases, particularly near the convergence zones. The CCME and CCMT63E, however, do not have a strong dry bias in the east Pacific.

Fig. 16.

Mean error of weekly 5-day forecast of 850-mb specific humidity (g kg−1) for entire year for (a) CCMA, (b) CCME, and (c) CCMT63E.

Fig. 16.

Mean error of weekly 5-day forecast of 850-mb specific humidity (g kg−1) for entire year for (a) CCMA, (b) CCME, and (c) CCMT63E.

The strong biases in the moisture prompted us to determine whether there may have been any errors in converting humidity variables or interpolating to horizontal and vertical grids. We compared the postprocessed specific humidity on pressure levels against the initial analysis, where the relative humidity was converted using an independent routine. For any level below 300 mb, there was less than 3% mean error over the Tropics for any given level. Considering that the initial fields undergo horizontal and vertical interpolation to the model Gaussian/hybrid vertical coordinate grid, and on postprocessing interpolated back to pressure levels and a linear 1° grid, this level of error seems to indicate no fundamental problem creating the initial field. The differences between the preprocessed and postprocessed moisture fields are much less than the differences between the AVN and ECMWF analyses. Temperature fields were likewise checked. It is interesting to note that the lower- (upper-) level drying (moistening) seen here also appears to occur in the CAM2 model, the successor to the CCM, when run as a weather prediction model (Phillips et al. 2004).

e. Model physics

The global average precipitation for each month is shown in Fig. 17. The annual mean is given by the thicker solid line. It is evident that there is a spin down of precipitation in the first 3 days. Even when the precipitation stabilizes, to around 3.25 mm day−1, it is still a bit higher than the generally accepted observed value of around 2.69 mm day−1 (Xie and Arkin 1996). This excessive precipitation accounts for some of the error that we noted in the temperature fields, and perhaps the tropical wind field as well.

Fig. 17.

Global area-averaged precipitation (mm day−1) as a function of forecast length.

Fig. 17.

Global area-averaged precipitation (mm day−1) as a function of forecast length.

The excessive precipitation is not due solely to resolution, as the CCM has previously been reported to have high precipitation in climate simulations at T42 resolution. For example, Kiehl et al. (1998) reports a global mean value of around 3.09 mm day−1. This high precipitation may also not be due only to the Zhang–McFarlane convection scheme. From some preliminary experiments that we have done with the Florida State University Global Spectral Model (FSUGSM) (Cocke and LaRow 2000), we find that the planetary boundary layer (PBL) and vertical diffusion schemes to be at least be partly responsible. In the FSUGSM we have implemented most of the CCM physics package, along with six different cumulus parameterization schemes. In simulations of the FSUGSM where we used the CCM PBL and vertical diffusion scheme, we obtained excessive precipitation with five cumulus schemes (Shin et al. 2003). In fact, the Zhang–McFarlane scheme produced the lowest global average precipitation of the five schemes used in that study. We did not have excessive precipitation when using the original FSU PBL and vertical diffusion scheme. However, we caution that these results are very preliminary, and further investigation is needed.

A cross section of the zonal mean precipitation is shown in Fig. 18a. There is a substantial drift in the day-1 precipitation (open circles) and days 3 and beyond. This again is just the spin down as discussed above. After the precipitation has spun down, the peak precipitation occurs near the equator. In Fig. 18b we show a similar plot for the GFS. The GFS also exhibits a shift in the peak precipitation with increasing forecast length. For 15-day forecasts, the peak precipitation for the GFS is slightly north of the equator.

Fig. 18.

Zonal cross section of precipitation (mm day−1) for 1-, 3-, 5-, and 10-day forecasts for the month of February for (a) CCM and (b) GFS (White 2004). CCM: day 1—open circles, day 3—closed circles, day 5—open squares, day 10—closed squares.

Fig. 18.

Zonal cross section of precipitation (mm day−1) for 1-, 3-, 5-, and 10-day forecasts for the month of February for (a) CCM and (b) GFS (White 2004). CCM: day 1—open circles, day 3—closed circles, day 5—open squares, day 10—closed squares.

We show the zonal mean cross section for the latent heat flux in Fig. 19a. The overall profile is very similar to that of the GFS (Fig. 19b), but with larger values in the Tropics. There are also larger values near the North Pole, which is likely due to the lack of proper sea ice temperature. Unlike precipitation, there is little drift as a function of forecast length in either model. We caution that the latent heat fluxes for the CCM shown here are for 1200 UTC, whereas it is not clear from White (2004) whether the GFS latent heat fluxes are averaged or instantaneous, and if so, which cycle.

Fig. 19.

Zonal cross section of evaporation (mm day−1) for 1-, 3-, 5-, and 10-day forecasts for the month of February for (a) CCM and (b) GFS. CCM: day 1—open circles, day 3—closed circles, day 5—open squares, day 10—closed squares.

Fig. 19.

Zonal cross section of evaporation (mm day−1) for 1-, 3-, 5-, and 10-day forecasts for the month of February for (a) CCM and (b) GFS. CCM: day 1—open circles, day 3—closed circles, day 5—open squares, day 10—closed squares.

The zonal mean cross section of the high cloud amount for February is shown in Fig. 20. The most striking difference between the CCM and GFS is the amount of high clouds in the Tropics, near the equator. The CCM has cloud amounts from 70% to 90% compared to about 50% for the GFS at the equator. The excess convection discussed above could certainly account for some of this large cloud amount. The shift in the cloud amount toward the equator for increasing forecast length corresponds to the shift in precipitation as seen in Fig. 18. There is much closer agreement in the midlatitudes.

Fig. 20.

Zonal cross section of high cloud amount for 1-, 3-, 5-, and 10-day forecasts for the month of February for (a) CCM and (b) GFS (White 2004). CCM: day 1—open circles, day 3—closed circles, day 5—open squares, day 10—closed squares.

Fig. 20.

Zonal cross section of high cloud amount for 1-, 3-, 5-, and 10-day forecasts for the month of February for (a) CCM and (b) GFS (White 2004). CCM: day 1—open circles, day 3—closed circles, day 5—open squares, day 10—closed squares.

Low-cloud-amount cross sections are shown in Fig. 21. In the midlatitudes, the CCM and GFS give very similar low cloud amounts. In the southern Tropics, the CCM is slightly higher than GFS, 30% compared to 20%, but are close in the northern Tropics with about 20% cloud amount. There is substantial difference in the northern polar region, where the CCM gives very large low cloud amounts. The presence of low clouds over the region persists over the entire year, with an annual mean of about 60%–80%. While improper sea ice temperatures could account for some of this, it is not clear if it is the sole cause. Further investigation is needed.

Fig. 21.

Zonal cross section of low cloud amount for 1-, 3-, 5-, and 10-day forecasts for the month of February for (a) CCM and (b) GFS (White 2004). CCM: day 1—open circles, day 3—closed circles, day 5—open squares, day 10—closed squares.

Fig. 21.

Zonal cross section of low cloud amount for 1-, 3-, 5-, and 10-day forecasts for the month of February for (a) CCM and (b) GFS (White 2004). CCM: day 1—open circles, day 3—closed circles, day 5—open squares, day 10—closed squares.

Finally in Fig. 22 we show the sensible heat flux. Here we find large differences between the CCM and GFS, in the northern Tropics and polar regions. As with the latent heat flux mentioned above, the sensible heat fluxes of the two models may not be at the same time during the day, and diurnal variation could be the primary reason for the difference. At 1200 UTC, the sensible heat fluxes are quite large over the northern African continent, giving most of the contribution to the zonal averaged flux between the equator and 30°N in the CCM forecasts.

Fig. 22.

Zonal cross section of sensible heat flux (W m−2) for 1-, 3-, 5-, and 10-day forecasts for the month of February for (a) CCM and (b) GFS (White 2004). Day 1—open circles, day 3—closed circles, day 5—open squares, day 10—closed squares.

Fig. 22.

Zonal cross section of sensible heat flux (W m−2) for 1-, 3-, 5-, and 10-day forecasts for the month of February for (a) CCM and (b) GFS (White 2004). Day 1—open circles, day 3—closed circles, day 5—open squares, day 10—closed squares.

6. Conclusions

The CCM performed reasonably well at predicting large-scale flow as indicated by the 500-mb height anomaly correlations and is competitive with major operational models when using high quality initial conditions provided by NCEP and ECMWF. The CCM did perform better using ECMWF analyses. This might be expected as ECMWF generally scores highest using the skill measures presented here, and their data assimilation system is largely responsible for that. Whether the CCM performed better due to the ECMWF analysis being more accurate or whether the CCM climatology is more in tune with ECMWF analyses, or a combination of both, would require a more detailed study. In any case, using two different analyses provides a lot of insight into the biases of the CCM model, which will hopefully lead to improved parameterizations.

The CCM exhibited large errors in the tropical wind near regions dominated by convective activity. These errors are probably at least partly related to the spinup of precipitation of the model, and overactive convection in general. Both AVN and ECMWF analyses lead to a large spinup of precipitation, whereas using GDAPS analysis (from an earlier study) did not. Comparing these analyses and the CCM responses to them suggest a strong tendency to dry the lower troposphere and moisten aloft.

The 850-mb temperatures indicate a warm bias in the west Pacific and Indian Ocean and a cool bias in the midlatitudes, and especially over land. The CCM using ECMWF analyses had less of a warm bias in the tropical oceans. There was a contrasting response in the east Pacific to using different analyses, with AVN leading to a rather warm bias, and a cool bias with ECMWF. The somewhat lower biases using ECMWF analyses further suggests that the CCM climatology is closer to that of ECMWF.

We found that most of the biases here are not due to resolution according to our T63 series of experiments. The difference in biases were much larger due to using different analyses rather than different resolutions. There was improvement in the 500-mb anomaly correlations by going from T63 to T126.

Many of the biases in precipitation and land temperature are similar to those reported for the CCM model when run as a low-resolution model. This suggests that examination of the CCM model at short time scales may lead to improvements in the long-term simulation as well.

A project for future work would be to compare the performance to the successor to the CCM 3.6, the Community Atmospheric Model version 2 (or 3).

Acknowledgments

Hee-Sang Lee is partially supported by the National Research Laboratory (NRL) from the Ministry of Science and Technology, Korea, and the Research and Development project on the Earthquake and Meteorology from the Korea Meteorological Administration, Korea.

REFERENCES

REFERENCES
Bonan
,
G. B.
,
1998
:
The land surface climatology of the NCAR land surface model coupled to the NCAR community climate model.
J. Climate
,
11
,
1307
1326
.
Caplan
,
P.
,
cited
.
2004
:
Pete Caplan’s RMS errors and anomaly correlation statistics. [Available online at http://wwwt.emc.ncep.noaa.gov/gmb/STATS/STATS.html.]
.
Caplan
,
P.
,
J.
Derber
,
W.
Gemmill
,
S-Y.
Hong
,
H-L.
Pan
, and
D.
Parrish
,
1997
:
Changes to the 1995 NCEP Operational Medium-Range Forecast Model.
Wea. Forecasting
,
12
,
581
594
.
Cocke
,
S.
, and
T. E.
LaRow
,
2000
:
Seasonal predictions using a regional spectral model embedded within a coupled ocean–atmosphere model.
Mon. Wea. Rev.
,
128
,
689
708
.
Hack
,
J. J.
,
J. T.
Kiehl
, and
J. W.
Hurrell
,
1998
:
The hydrological and thermodynamic characteristics of the NCAR CCM3.
J. Climate
,
11
,
1179
1206
.
Hurrel
,
J. W.
,
J. J.
Hack
,
B. A.
Boville
,
D. L.
Williamson
, and
J. T.
Kiehl
,
1998
:
The dynamical simulation of the NCAR community climate model version 3 (CCM3).
J. Climate
,
11
,
1207
1236
.
Kalnay
,
E.
,
M.
Kanamitsu
, and
W. E.
Baker
,
1990
:
Global numerical weather prediction at the National Meteorological Center.
Bull. Amer. Meteor. Soc.
,
71
,
1410
1428
.
Kanamitsu
,
M.
,
1989
:
Description of the NMC global data assimilation and forecast system.
Wea. Forecasting
,
4
,
335
342
.
Kanamitsu
,
M.
, and
Coauthors
,
1991
:
Recent changes implemented into the global forecast system at NMC.
Wea. Forecasting
,
6
,
425
435
.
Kiehl
,
J. T.
,
J. J.
Hack
,
G. B.
Bonan
,
B. A.
Boville
,
D. L.
Williamson
, and
P. J.
Rasch
,
1998
:
The National Center for Atmospheric Research Community Climate Model: CCM3.
J. Climate
,
11
,
1131
1149
.
Phillips
,
T. J.
, and
Coauthors
,
2004
:
Evaluating parameterizations in general circulation models—Climate simulations meets weather prediction.
Bull. Amer. Meteor. Soc.
,
85
,
1903
1915
.
Shin
,
D. W.
,
T. E.
LaRow
, and
S.
Cocke
,
2003
:
Convection scheme and resolution impact on seasonal precipitation forecasts.
Geophys. Res. Lett.
,
30
.
2078, doi:10.1029/2003GL018297
.
White
,
G.
,
cited
.
2004
:
Glenn White’s real-time operational MRF monthly diagnostics. [Available online at http://wwwt.emc.ncep.noaa.gov/gmb/noor/oct98op/text.html.]
.
WMO
,
1999
:
Commission for basic systems abridged final report with resolutions and recommendations. Rep. WMO 893, Geneva, Switzerland, 184 pp. [Available online at www.wmo.ch/web/www/reports/CBS-Ext98.pdf.]
.
Xie
,
P.
, and
P. A.
Arkin
,
1996
:
Analysis of global monthly precipitation using gauge observations, satellite estimates, and numerical model predictions.
J. Climate
,
9
,
840
858
.

Footnotes

Corresponding author address: Steven Cocke, Dept. of Meteorology, The Florida State University, Rm. 410, Love Bldg., Tallahassee, FL 32306. Email: scocke@mailer.fsu.edu