This study presents an approach that converts the vertical profiles of grid-averaged cloud properties from large-scale models to probability density functions (pdfs) of subgrid-cell cloud physical properties measured at satellite footprints. Cloud physical and radiative properties, rather than just cloud and precipitation occurrences, of assimilated cloud systems by the European Centre for Medium-Range Weather Forecasts (ECMWF) operational analysis (EOA) and 40-yr ECMWF Re-Analysis (ERA-40) are validated against those obtained from Earth Observing System satellite cloud object data for the January–August 1998 and March 2000 periods. These properties include the ice water path (IWP), cloud-top height and temperature, cloud optical depth, and solar and infrared radiative fluxes. Each cloud object, a contiguous region with similar cloud physical properties, is temporally and spatially matched with EOA and ERA-40 data. Results indicate that most pdfs of EOA and ERA-40 cloud physical and radiative properties agree with those of satellite observations of the tropical deep convective cloud object type for the January–August 1998 period. There are, however, significant discrepancies in selected ranges of the cloud property pdfs such as the upper range of EOA cloud-top height. A major discrepancy is that the dependence of the pdfs on the cloud object size for both EOA and ERA-40 is not as strong as in the observations. Modifications to the cloud parameterization in ECMWF that occurred in October 1999 eliminate the clouds near the tropopause but shift power of the pdf to lower cloud-top heights and greatly reduce the ranges of IWP and cloud optical depth pdfs. These features persist in ERA-40 due to the use of the same cloud parameterizations. The less sophisticated data assimilation technique and the lack of snow water content information in ERA-40, not the larger horizontal grid spacing, are also responsible for the disagreements with observed pdfs of cloud physical properties, although the detection rates of cloud object occurrence are improved for small-size categories. A possible improvement to the convective parameterization is to introduce a stronger dependence of updraft penetration heights on grid-cell dynamics.
Numerical weather forecasts are routinely validated against surface observations at short time scales; for example, at the daily time scale. But climate model simulations are usually validated against gridded monthly mean satellite and surface observations. In both types of validations, the gridded data are used. They represent averages of physical parameters over an area covering hundreds of kilometers in the horizontal direction. The gridded monthly mean satellite data may include many different types of cloud systems at each grid cell of the model, due to changes in weather conditions with time. While these gridded data are useful in validating some aspects of climate model simulations, they do not sufficiently constrain critical assumptions about the treatment of subgrid-scale processes and thus are generally not suitable to fully explore the direct cause of model deficiencies (e.g., Norris and Weaver 2001; Jakob 2003; Xu et al. 2005). There is a need for an alternative approach to satellite data analysis that moves beyond the gridded means.
For numerical weather prediction (NWP) models, the forecast scores, which have undergone rapid improvement (Simmons and Hollingsworth 2002), are usually tied to the geopotential height at 500 hPa and the amount of surface precipitation over an extended period (e.g., Zhao and Carr 1997) although forecast scores for cloud variables are still being developed (M. Miller 2008, personal communication). These scores are good at evaluating model’s ability to forecast synoptic disturbances and hydrological cycle, respectively, but do not provide a direct evaluation of the performance of their cloud forecasts and the associated radiative budget. Cloud properties forecasted by the European Centre for Medium-Range Weather Forecasts (ECMWF) have been evaluated the most often among the NWP models. One of the earliest attempts was made by Morcrette (1991), who converted model-forecasted cloudiness into window-channel brightness temperature to compare with brightness temperature observations from satellites. This “model-to-satellite” approach eliminated uncertainties associated with cloud retrievals from satellite radiance measurements (Chevallier and Bauer 2003), which inspired the more recent instrument-based “simulator” approaches (Klein and Jakob 1999; Webb et al. 2001; Haynes et al. 2007).
The rapidly increasing efforts in evaluating ECMWF cloud forecasts have resulted from both the improvement of cloud parameterizations and the availability of new observations. The parameterization of Tiedtke (1993), which replaced the diagnostic scheme of Slingo (1987), enabled predictions of cloud fraction, cloud water, and ice contents at every model level and included physically based formulations of the interactions among deep convection, anvil clouds, turbulence, and large-scale processes. For example, Tiedtke’s (1993) scheme included explicit convective sources for the prediction of cloud fraction and ice water content (IWC). These sources are tightly coupled with the mass-flux convective parameterization of Tiedtke (1989). Further improvements in cloud parameterizations were made to the model by Gregory et al. (2000) and Jakob et al. (2000), which included gravitational sedimentation of large ice particles, and recently by Bechtold et al. (2004, 2008) for convective triggering and detrainment and Köhler (2005) for boundary layer stratocumulus clouds.
Recent efforts in evaluating ECMWF forecasted cloud properties have used either surface-based field experiment data or satellite–surface data. Evaluations using field experiment data include Bretherton et al. (1995) for Northeast Atlantic stratocumulus clouds; Mace et al. (1998), Morcrette (2002), and Willen et al. (2005) for midlatitude clouds; Comstock and Jakob (2004) for tropical thin cirrus clouds; Betts and Jakob (2002) for Amazon convection; and Beesley et al. (2000) and Xie et al. (2006) for Arctic mixed-phase clouds. For example, using the Atmospheric Radiation Measurement’s (ARM; Stokes and Schwartz 1994) Southern Great Plains data, Mace et al. (1998) concluded that the ECMWF model had good overall skill in predicting the vertical profiles of the cloud and hydrometeor occurrences during a winter season, but tended to predict the onset of deep convective events too soon (also see Bechtold et al. 2004), the dissipation too slow, and vertical depth too thick during a summer season. Comstock and Jakob (2004) found that the cirrus base heights were captured well by the model, but their tops were overestimated using the ARM tropical western Pacific data.
Evaluations of the ECMWF model using satellite–surface data include Klein and Jakob (1999) and Tselioudis and Jakob (2002) for midlatitude frontal clouds with the International Satellite Cloud Climatology Project (ISCCP; Schiffer and Rossow 1983) data; Jakob (1999) for the global climatology of cloud cover of an earlier reanalysis [i.e., the 15-yr ECMWF Re-Analysis (ERA-15)] with the ISCCP data, Chevallier et al. (2005) for the global climatology of high clouds of two reanalysis products, Chevallier and Morcrette (2000) and Chevallier and Bauer (2003) for short-term forecasts of cloud, precipitation, and radiative properties; Miller et al. (1999), Palm et al. (2005), and Wilkinson et al. (2008) with active-sensor measurements; and Li et al. (2005, 2007) for upper-tropospheric IWCs with Microwave Limb Sounder (MLS) measurements. Klein and Jakob (1999) examined a composite of about 200 midlatitude baroclinic systems and found good agreement in the general positioning of cloud types (through the ISCCP simulator) relative to the low pressure centers, but the optical depths of cloud types did not agree well with observations. Jakob (1999) and Chevallier et al. (2005) found that the reanalyses captured the main aspects of the interannual variability, but there were various degrees of deficiencies for the major cloud regimes. Li et al. (2005, 2007) found that there was good agreement in the spatial distributions of IWC, but the magnitudes of IWC exhibited various degrees of disagreement at different vertical levels in the upper troposphere.
The aforementioned validations with different data sources focused upon the cloud and hydrometeor occurrences, such as the vertical profiles of cloud occurrences from field experiments and the global distributions of cloud cover from satellite observations and the frequency of occurrence of different cloud types from satellite observations (e.g., Klein and Jakob 1999). Some of these validations also relied upon the monthly mean gridded data such as those of IWC and cloud cover from satellite observations (Jakob 1999; Li et al. 2005, 2007). The present study differs from the aforementioned studies in two fundamental ways.
The probability density functions (pdfs) of multiple cloud physical and radiative properties will be examined, instead of their simple means and standard deviations or pdfs of a single property (Chevallier and Bauer 2003). This will yield more specific information on the deficiencies in the cloud parameterizations than from previous studies.
The observed cloud physical and radiative properties are obtained from Lagrangian identification of cloud systems similar to that of Klein and Jakob (1999) for midlatitude frontal cloud systems, which avoids Eulerian averaging. This study will thus provide a detailed evaluation of a specific cloud type, instead of a mixture of cloud system types in traditional gridded data analyses, which will be helpful to pinpoint parameterization deficiencies relevant to the particular cloud-system type.
A brief description of the present approach is given below. Further details for the satellite data analysis and the conversion of model data to equivalent satellite observations can be found in sections 2 and 3, respectively.
Xu et al. (2005) proposed a cloud object approach to satellite data analysis. This technique classifies Earth Observing System (Wielicki et al. 1995) satellite data into distinct cloud objects, each of which must be over a contiguous region, for several broad types of cloud systems (e.g., trade cumulus, stratocumulus, overcast, and deep convection) regardless of the Eulerian model grid mesh. The identified cloud objects are then matched with nearly simultaneous atmospheric state data from the ECMWF or model reanalyses. A large ensemble of cloud objects are then grouped into several subtypes according to different atmospheric states. The pdfs of footprint-scale characteristics of the cloud object ensemble can then be compared with those of model analysis or reanalysis by converting model data to equivalent satellite observations.
The goal of this study is to validate statistical physical properties of tropical deep convective cloud systems forecasted by the ECMWF operational analysis (EOA) and the 40-yr ECMWF Re-Analysis (ERA-40; Uppala et al. 2005) against satellite cloud object data. The model cloud fields are a 6-h forecast. The validation presented in this study is thus pertinent to the ECMWF data assimilation system due to the short forecasting duration. The following questions will be addressed in this study: 1) To what extent do various cloud physical properties of the tropical deep convective cloud type forecasted by the ECMWF agree with the observations? 2) What aspects of the forecasted cloud physical properties are improved due to changes in cloud parameterization and decreases in horizontal and vertical grid spacings? The cloud parameterization was changed in October 1999 (Gregory et al. 2000), which was between two Tropical Rainfall Measuring Mission (TRMM) Clouds and the Earth’s Radiant Energy System (CERES; Wielicki et al. 1996) data periods. The cloud parameterization used in the latter period was the same as that used in ERA-40, but ERA-40 used a three-dimensional variational data assimilation (3DVAR) technique (Uppala et al. 2005), in contrast to the more advanced four-dimensional variational data assimilation(4DVAR) technique for both EOA periods. Although some important details of the ECMWF cloud parameterization have recently been improved (Bechtold et al. 2004, 2008; Köhler 2005), its major structures and components remain the same. Therefore, some deficiencies to be identified in this study are likely relevant and important for further improvements of the ECMWF cloud parameterization.
The rest of the paper is organized as follows. Satellite cloud object analyses are described in section 2. The procedure for converting the forecasted cloud physical properties to equivalent satellite observations is outlined in section 3. Section 4 presents the results of the comparison of cloud physical properties between the ECMWF analysis/reanalysis and satellite observations. Conclusions and discussion are given in section 5.
2. Satellite cloud object data
Passive-sensor satellite observations such as CERES are adept to describe cloud physical and radiative properties of mesoscale cloud systems (rather than individual clouds) because of the large footprint sizes of satellite sensors. Since individual clouds are elements in a cloud system, a contiguous region of cloudy footprints, which is called a cloud object by Xu et al. (2005, 2007, 2008), can describe a part of a cloud system but not necessarily an entire cloud system. This is because of the limited width of the satellite swath and the selection criteria used to determine the cloud system type, both of which can break up a large cloud system into several smaller pieces. The cloud object selection criteria depend upon the cloud system type. For the tropical deep convective (DC) cloud systems examined in this study, they are composed of cloud optical depth (τ > 10), cloud top height (Ht > 10), and footprint cloud fraction (100%) and these systems are located within the latitudinal band of 25°S–25°N, because optically thick anvil cloud systems in the tropics are a major driver of tropical energetics.
The basic data with which the cloud object data are produced is a level-2 (footprint) instantaneous data product from the CERES mission. Although this data product contains a wide variety of individual parameters, only 13 parameters are saved in the cloud object data product. They are the top-of-the-atmosphere (TOA) reflected shortwave (SW) flux, outgoing longwave radiation (OLR) flux, TOA albedo, solar insolation, τ, ice water path (IWP), ice particle size, liquid water path (LWP), droplet equivalent radius, cloud-top temperature, cloud-top pressure, and sea surface temperature (SST). Some of these parameters are briefly described below. Please refer to Xu et al. (2005, 2007) for further details, including an analysis of the impact of their uncertainties on the summary pdfs for an ensemble of cloud objects.
The CERES broadband radiative fluxes are produced using the new generation of Angular Distribution Models (ADMs) derived from TRMM CERES broadband directional radiance measurements (Loeb et al. 2003). These improved ADMs significantly reduce both root-mean-square (rms) and bias TOA broadband flux errors for all scenes by a factor of 3–4, when compared with those of the previous generation Earth Radiation Budget Experiment (ERBE) data (Suttles et al. 1988; 1989), because of an increase of scene types and the stratification of scene types by both cloud fraction and τ. [Chevallier and Morcrette (2000) used the gridded CERES TOA flux data obtained from the ERBE-like algorithm to validate ECMWF radiative fluxes for two weekly periods.] The footprint data product combines instantaneous CERES radiative flux data with scene information from a higher-resolution imager, which is the Visible Infrared Scanner (VIRS) on the TRMM satellite. Scene identification (e.g., type and clear/cloudy) and cloud physical properties (e.g., cloud effective height, temperature, pressure, particle types, and equivalent diameters) are obtained from retrievals at the higher resolution of the imager. These data are then averaged over the larger CERES footprints. Details of the retrieval methods are described in the CERES algorithm theoretical basis document (Minnis et al. 1997).
A “region-growing” strategy based on imager-derived cloud physical properties is used to identify cloud objects within a single satellite swath (Wielicki and Welch 1986). Details of this algorithm can be found in Xu et al. (2005). Briefly, for all CERES footprints in a TRMM satellite orbit swath, each CERES footprint that meets the selection criteria is marked as specific cloud type. These “seed points” are grown using the Wielicki–Welch algorithm. Only footprints that are adjacent and that meet the selection criteria of a single cloud type can be joined in a cloud object. In this context, “adjacent” refers to CERES footprints that are next to each other either along the scanning direction or perpendicular to it. Cloud objects are distinct from one another if they share no adjacent footprints. Any cloud object that grows to an equivalent diameter (of a circle) of greater than 100 km (∼75 footprints) is saved in the cloud object database (available online at http://cloud-object.larc.nasa.gov/) since large cloud objects are likely assimilated well by NWP models.
Two periods of the TRMM CERES1 cloud object data are used in this study. The first data period, January–August 1998, was analyzed in Xu et al. (2007). These 8 months corresponded to the peak and dissipative phases of the 1997/98 El Niño. The total number of DC cloud objects with equivalent diameters greater than 100 km were 2257, while the total number of footprints is 1.175 million. These cloud objects were further classified according to the range of their equivalent diameters. Three size categories are considered. They are defined by the ranges of 100–150 km (small size), 150–300 km (medium size), and greater than 300 km (large size). For convenience, they are termed the S, M, and L categories, respectively. There are 858, 899, and 500 cloud objects for the S, M, and L categories, respectively. In Xu et al. (2007), these numbers were slightly smaller due to the elimination of some cloud systems that may be influenced by baroclinicity. The total footprint number was 8.3%, 26.3%, and 65.4% of the total for the S, M, and L categories, respectively. The second data period (i.e., March 2000) is much shorter. The total number of cloud objects is 142, 144, and 54 for the S, M, and L categories, respectively. Xu et al. (2005) compared cloud physical properties associated with the strong 1997/98 El Niño in March 1998 and the very weak La Niña in March 2000 and found that pdfs of the majority of cloud physical properties are similar in spite of the climatological contrast. There were slightly more L-size cloud objects (i.e., 68) observed in March 1998, related likely to much higher SSTs during the El Niño.
3. Analysis procedure
This analysis procedure involves four steps that lead to diagnoses of subgrid-cell cloud physical properties using the ECMWF meteorological data as inputs. The pdfs of these properties can then be compared with satellite cloud object observations. The four steps are the following:
Each cloud object is matched spatially and temporally with the EOA or ERA-40 data.
A cloud generator algorithm is used to generate vertical and horizontal distributions of cloud fields with cloud fraction profile information and a cloud overlap assumption for each ECMWF grid that is divided into a number of subgrid cells or subcolumns.
The Fu–Liou radiation model (Fu and Liou 1992) is used to compute the cloud optical depth and radiative fluxes for each subcolumn.
A threshold on cloud optical depth is used to determine the cloud-top height while selection criteria for DC cloud objects are used to select subcolumns for producing the pdfs of cloud physical properties.
These steps are discussed in detail below.
First, each cloud object is matched in time and in space with ECMWF data, which include forecasted profiles of cloud fraction and liquid water and cloud ice water mixing ratios, in addition to the temperature and moisture profiles. These are 6-h forecasts starting from analyses of thermodynamic and dynamic states produced by the data assimilation system, which is available at 6-h intervals. The EOA and forecasts over the tropics are available on 0.5625° × 0.5625° grid meshes while the ERA-40 data for the same region are available on 1.125° × 1.125° grid meshes. In the subset of meteorological data for individual cloud objects, there are 31 vertical levels for the 1998 period (the same number of levels as in the original data) and 40 vertical levels for the March 2000 period and the ERA-40 data (the top 20 levels in the stratosphere of the original data are excluded). The nine extra levels were located in the lower troposphere. As described in Xu et al. (2007), the spatial matching of cloud objects with the meteorological data utilizes the latitudinal and longitudinal information of the most easterly, westerly, southerly, and northerly footprints of each cloud object. This information is used to draw a rectangular box that is large enough to contain all footprints within the cloud object and its immediately adjacent surrounding areas, in combination with the grid coordinates of the ECMWF model. The ECMWF data from the closest analysis time to the observational time of a given cloud object are matched with the cloud object. The gap in the temporal matching is less than or equal to 3 h.
Second, each of the ECMWF grid columns is divided into a set of 30 subcolumns for EOA and 120 subcolumns for ERA-40 because the horizontal size of an ECMWF grid cell in the tropics is much larger than the average size of a CERES footprint, which is only 10 km × 10 km. The vertical profile of the grid cloud fraction is the only relevant input information from the ECMWF data that can be used to determine the subcolumn cloud distribution. To populate clouds within the 30/120 subcolumns horizontally and vertically, a cloud generator algorithm is needed. A method proposed by Yu et al. (1996) and modified by Klein and Jakob (1999) and Webb et al. (2001) is used in this study although more complicated methods have also been proposed (e.g., Räisänen et al. 2004; Pincus et al. 2005, 2006; Morcrette et al. 2008). The cloud fraction of each subcolumn is assigned to be 0 or 1 at every level. The maximum number of cloudy subcolumns is determined by the grid cloud fraction (A) at that level. The cloud overlap assumption used is the maximum–random M–R overlap, which was used in the ECMWF radiation module before 5 June 2007 (Morcrette et al. 2008). Two other popular overlap assumptions (i.e., maximum and random) are also tested. Another aspect of the cloud generator algorithm is the specification of which subcolumn contains cloud condensate at a given height. A random stacking is used, following that used in the ISCCP simulator (Webb et al. 2001), instead of that used in Klein and Jakob (1999), which stacks the cloud starting from the subcolumns farthest to the left and starts at the top layer of the atmosphere.
Third, the cloud optical depth and radiative properties of each subcolumn are calculated with the Fu–Liou (1992) radiative transfer model. The key inputs to radiative transfer calculation include the vertical profiles of temperature, water vapor mixing ratio, LWC, and IWC (snow, graupel, and rainwater are not available from the standard ECMWF data archives), in addition to the observational time, latitude, longitude of the cloud objects, and the climatological ozone profile. There is no information that can be used to provide the horizontal inhomogeneity of temperature, water vapor, and condensate. Therefore, the profiles of grid-cell temperature and water vapor mixing ratio are used for every subcolumn. The subcolumn LWC/IWC is obtained from the grid-cell LWC/IWC divided by the number of cloudy subcolumns, N. The lack of horizontal inhomogeneity of condensate can impact the results to be discussed in section 4. Other inputs to the radiation code are the cloud ice equivalent diameter and water droplet equivalent radius. The former is a function of IWC (see Xu 2005) and the latter is assumed to be 12 μm, which is based upon the median value of CERES retrieval data. Surface albedo is identical to that in the ECMWF data and surface emissivity is assumed to be 0.99.
Fourth, once cloud optical depth (τ) for the visible band (0.2–0.7 μm) and radiative properties for each layer are obtained for the set of subcolumns, the vertically integrated (from model top) τ is used to determine cloud-top height and IWP of DC subcolumns. The determination of IWP and τ mimics the capability of passive-sensor satellite retrieval. Integration of subcolumn τ starts from the model top and stops either where the integrated τ reaches 128 or at the 0°C temperature level (if τ < 128). A maximum τ of 128 is used because the CERES algorithm retrieves τ from observed solar reflectance, which becomes saturated when τ is greater than 128 (Minnis et al. 1997). Since the satellite-measured reflected radiance, which was an integrated effect by the atmospheric column, contributions from cloud ice, snow, and supercooled droplets are included in the determination of subcolumn τ and IWP. In this study and previous cloud-resolving model (CRM) analyses (Eitzen and Xu 2005; Luo et al. 2007), cloud-top height for each subcolumn is determined as the level where τ reaches one when integrated from the model top. The temperature at this height is treated as the cloud-top temperature. Finally, the subcolumns that satisfy the DC cloud object criteria are selected to produce pdfs of cloud physical properties, which will be extensively compared with observations in section 4.
The results are organized into three subsections. The EOA and ERA-40 physical properties are examined in sections 4a and 4b, while the detection and overestimate of frequencies of occurrence of cloud objects are shown in section 4c. Because of the large number of cloud objects observed during the January–August 1998 period, extensive comparisons will be made for both the pdfs and the frequencies of occurrence of this period. The rest of the results will be shown in terms of the comparison between two shorter periods (March 1998 and March 2000) with contrasting climatological conditions and some sensitivity analyses.
a. ECMWF operational analysis
1) January–August 1998
The observed pdfs from the January–August 1998 period were extensively discussed in Xu et al. (2007). Six pdf plots are reproduced in Figs. 1a,b; Fig. 2a, b; and 3a, b for convenience. These parameters are τ (1a), IWP (Fig. 1b), cloud-top temperature (Fig. 2a), cloud-top height (Fig. 2b), TOA reflected SW flux (Fig. 3a), and OLR (Fig. 3b). The probability density (on the vertical axes) is defined as the frequency that the cloud property value falls within a bin divided by the bin size. The cloud objects identified from the CERES footprint data are grouped into three categories (S, M, and L), as discussed earlier, according to the ranges of their equivalent diameters. The major features appearing in the observed pdfs are summarized as follows.
There are significant differences in the pdfs among the three categories for all six cloud physical and radiative properties despite the relatively small differences in their SST pdfs (see Xu et al. 2007). This result suggests that the large-scale dynamics, not SST, is the primary cause for the differences. Second, the pdfs for the larger-size category of cloud objects are more skewed toward high cloud tops, low cloud-top temperatures, large IWP, high τ, low OLR, and high reflected SW fluxes than the smaller-size category. Third, the differences in IWP and τ among the three categories appear to be smaller than those of cloud-top height, temperature, OLR, and SW reflected radiation. Statistical significance tests based on the bootstrap method, as described in Xu (2006), were performed in Xu et al. (2007) for a pair of pdfs. The tests are not necessary in this study because the differences between the observed and ECMWF pdfs are large.
How well does the EOA reproduce the observed major features described above? What deficiencies in the cloud parameterization can be identified from comparisons with the observations. These two questions can be addressed as follows.
For all six parameters shown in Figs. 1c, d; 2c, d; and 3c,d, there are good agreements with observations as far as the overall shapes of the pdfs are concerned [e.g., exponential for τ (Fig. 1c), lognormal for IWP (Fig. 1d), and nearly Gaussian with variable skewnesses for the rest of the parameters (Figs. 2c,d and 3c,d)]. This result demonstrates the suitability of the proposed methodology for producing pdfs of subgrid-cell cloud physical properties derived from model grid-cell cloud physical property profiles. An explanation for these shapes of pdf can be found in Xu et al. (2005, 2007) and Eitzen and Xu (2005).
The dependencies of the pdfs of EOA cloud physical properties on the cloud object size are weak for all six parameters, compared with those of observations mentioned above. This result may be related to the effective resolution of the numerical model (Pielke 1991), which is close to the L size. Specific aspects of this result are as follows.
For τ, there is slightly less power (i.e., lower probability density) in the 10–15 range and higher power in the 30–90 range for the L category than for the M and S categories (Fig. 1c). The IWP pdfs also somewhat mimic the observed dependency (Fig. 1d), but those of TOA SW flux do not (Fig. 3c).
For cloud-top height, cloud-top temperature, and OLR (Figs. 2c,d and 3d), the power is nearly identical at the low end (i.e., 10–12 km) of the cloud-top height pdf (i.e., the high ends of OLR and temperature pdfs) for all three categories, where observations indicate a strong decrease of power with the cloud object size. At the height range greater than 12 km, the M and L categories show nearly identical power but the S category exhibits stronger power in the 14–18-km range and less power in the 12–14-km range. These results differ strongly from observations. As discussed in Xu et al. (2007), some of the small cloud objects resulted from truncation of large cloud objects by narrow satellite swaths. When the ECMWF data were matched to these small cloud objects, the properties of large cloud objects within the matched areas excessively contribute to the high cloud-top end of the pdfs. Thus, the S-category pdfs are skewed toward high cloud-top heights, low temperatures, and low OLRs. There is also an overestimate in the footprint numbers of individual cloud objects for the S category, which will be discussed in section 4c.
There are numerous large discrepancies at selected ranges of each of the six EOA pdfs, compared with the observations, which are likely associated with deficiencies in the ECMWF cloud parameterization. Some of them may have been alleviated by recent improvement in the convective parameterization (Bechtold et al. 2008). Specific examples are as follows.
Fig. 2d shows the presence of clouds between 16 and 18 km that are not detected by passive satellite sensors (Fig. 2b). These clouds are related to the penetration of convective updrafts close to the tropopause and produced by convective detrainment at these high altitudes. This problem originated from the convective parameterization (Tiedtke 1989) and was fixed in the operational model in October 1999 (Gregory et al. 2000) by limiting the penetration height and the intensity of deep convective updrafts. A detailed description of these modifications will be given shortly. There is, however, an unintended consequence of shifting power of the pdf to slightly lower heights related to this fix, which will be discussed in detail when results from the March 2000 period are presented shortly.
Figure 2d also shows a lack of relatively low clouds near the cloud object cutoff height (10–12 km). This result can be likely due to the overestimate of cloud fractions above this height range. That is, the convective detrainment heights are preferably located at much higher altitudes, especially for the S and M categories. (Further discussion will be given in section 4b.) A possible solution for this problem is to introduce a stronger dependence of updraft penetration heights on grid-cell dynamics, which allows model grid cells with stronger ascents to produce deeper penetrating convection.
Figures 1c,d show a lack of clouds with large τ (>90) and IWP (>1700 g m−3). These clouds are presumably the large-size cumulonimbi that are treated in the model as deep convective updrafts. In the mass-flux-type convective parameterizations (e.g., Arakawa and Schubert 1974; Tiedtke 1989), the nondetrained LWC and IWC within updrafts do not contribute to the grid-cell average. The assumption that LWC/IWC is horizontally homogeneous may also contribute to the underestimate of power in the high τ and IWP ranges. Xu (2005) showed that this assumption can contribute to a 40% reduction in τ, compared to that directly calculated from CRM input without averaging.
Discrepancies in other parameters are related to those discussed above. For example, overestimates in the low ends of cloud-top temperature and OLR ranges are related to those in the high end of the cloud-top height pdf, due to the large optical thickness of the clouds (with τ > 10) examined in this study.
2) Comparison between the March 1998 and March 2000 periods
There were several major aspects of the ECMWF operational model that were changed in October 1999, which included an increase of vertical resolution (from 31 to 60 layers) and changes in physical parameterizations (Gregory et al. 2000). The additional layers were added to both the stratosphere and the lowest 3 km of the atmosphere, so their direct impact on deep convection was probably minimal. There were numerous small changes in the cloud parameterizations, but collectively they had significant impacts on the physical properties of tropical cloud systems examined in this study. A few of the changes that directly impact the results to be discussed shortly are listed here. First, convective updrafts were not allowed to penetrate into the tropical tropopause. Second, the closure for convective parameterization was changed to a convective available potential energy (CAPE) relaxation closure with a specified time scale, which also reduced the strength of penetrating convection. Third, the formulation for ice sedimentation was modified to let large ice particles fall faster and fall outside of clouds (Jakob et al. 2000). That is, ice is depleted at a much faster rate than that in the earlier parameterization. Because snow water content (SWC) was not saved as parts of IWC by the standard ECMWF archives and the retrieved IWP data included the contribution of SWC, diagnosed IWP could, as shown later, be severely underestimated, because of changes in this aspect of the cloud parameterization. A justification for all of these changes was that radiative budgets were improved (Gregory et al. 2000).
How do the changes in cloud parameterizations impact the physical properties in the operational analysis? Why are some pdfs of cloud physical properties improved while others are degraded? To address these questions, we choose two monthly periods, March 1998 (330 cloud objects) and March 2000 (340 cloud objects; see Xu et al. 2005), whose EOA cloud physical properties were produced with the earlier and modified parameterizations, respectively. As discussed in section 4a, the pdfs of EOA cloud physical properties are nearly identical among the three categories. Therefore, the EOA pdfs from the three categories are combined to increase the smoothness of the pdfs, which are compared with those of the combined observations. These results are shown in Fig. 4.
For IWP and τ (Figs. 4a,b), the main deficiency of the March 1998 pdfs is too much power in the middle ranges, which has been shifted to lower values in March 2000. That is, the March 2000 pdfs have narrower ranges, compared to those of the March 1998 period. The power for optical depths greater than 70 and IWPs greater than 1300 g m−3 is nearly zero for the March 2000 period, but there is some power occurs at optical depths up to 100 and IWPs up to 2000 g m−3 for the March 1998 period. On the other hand, the power is more greatly overestimated at the lower ranges of these two parameters in the March 2000 period (e.g., the power at IWPs around 350 g m−3 is 50% higher than that observed). These results are related mainly to the modification to ice sedimentation formulation discussed above, which remove far greater amounts of ice from clouds.
The observed pdfs of cloud-top height and cloud-top temperature vary significantly between the two periods. These variabilities are generally reproduced although their respective differences from the observations are quite large (Figs. 4c,d). The March 2000 pdfs of these two parameters are slightly improved (thin lines), compared to those of the March 1998 period (thick lines), as seen from their closer agreements with observations. For example, the clouds with tops between 16 and 18 km have been eliminated and those with tops between 10 and 12 are less underestimated for March 2000. But clouds with tops between 12.5 and 14 km are overestimated (more pronounced overestimate for clouds with tops between 13 and 15.5 km in the ERA-40 results shown later). This was called “an unintended consequence” in section 4a. A plausible explanation for this unintended consequence is that anvil detrainment levels are too strongly restricted to a narrower height range by the modified cloud parameterization than the earlier one, which shifts the power of the pdf at heights between 16 and 18 km to slightly lower heights. Neither the lack of SWC data in the EOA nor the weaker dynamic forcing in March 2000 (Luo et al. 2007) can explain this shift of cloud-top height pdf. A sensitivity test was performed where an amount of SWC proportional to the EOA IWC was added. However, this only improved the pdfs of IWP and τ, not cloud-top height (not shown).
Observations show that OLR pdfs are relatively similar between the two periods, as are those of reflected SW fluxes (Figs. 4e,f). The values of both parameters are overestimated in both periods, but the degree of the overestimate is greater in the March 2000 period (i.e., power shifts to higher values in both pdfs). The reason for the overestimated OLR is related probably to the underestimated infrared emissivity slightly above the cloud-top heights because both cloud-top temperature and cloud-top height, which are determined at τ = 1, show less pronounced disagreement with observations for March 2000. What can cause the underestimate of emissivity? In the calculation with the Fu–Liou radiation, LWC/IWC is identical for every subgrid cell within an ECMWF grid. The lack of horizontal inhomogeneity of condensate may bias the emissivity toward the median/mean values. A piece of evidence for this is that CRM simulation results (Luo et al. 2007) agree with observations much better because horizontal inhomogeneity of condensate is simulated in CRM. [Overestimate of moisture above cloud top may also cause the overestimate of OLR.] A similar explanation cannot apply to the overestimate of reflected SW fluxes because τ over a deeper layer contributes to SW and τ is underestimated. Other factors listed in Luo et al. (2007), such as the plane-parallel assumption, may be important.
The model used for ERA-40 is basically identical to that used during the March 2000 period, except for a coarser horizontal grid spacing (1.125° × 1.125°) and a 3DVAR data assimilation system is used rather than the 4DVAR data assimilation system that was implemented in the operational model on 26 November 1997 (Uppala et al. 2005). The less sophisticated data assimilation technique can impact the assimilated dynamic and thermodynamic states (and cloud physical properties) associated with tropical convective cloud systems.
Overall, the ERA-40 pdfs (Figs. 1e, f; 2e, f; and 3e,f) for the January–August 1998 period are more similar to those of the March 2000 period than the EOA counterparts of the same period. This is an expected result because physical parameterizations used in ERA-40 are identical to those of March 2000 operational analysis. Examples of this expected result include the following:
The improvement in the OLR pdfs is significant at low OLR values (Fig. 3f), compared to the EOA pdfs (Fig. 3d). This improvement was not seen in the March 2000 period (Fig. 4f), however. This may be related to the difference in the upper-tropospheric humidity.
The clouds with tops above 16 km are eliminated in the ERA-40 pdfs, but the power at heights between 12.5 and 15.5 km is greatly overestimated (Fig. 2f) at the expense of the power at heights between 10 and 12.5 km. This unintended consequence is more pronounced than that seen in the March 2000 period discussed earlier.
Last, like the EOA results, the dependence of pdfs on cloud object size is almost nonexistent. That is, most ERA-40 pdfs more strongly resemble the observed pdfs of the L category than the S and M categories. This lack of dependence is an area that the convective parameterization can be improved upon. In the revised Tiedtke (1989) convective scheme, the presence of deep convection is determined by the depth of instability; if the cloud depth exceeds 200 hPa it is deemed to be deep convection, shallow convection otherwise. An explicit dependence on large-scale dynamics is removed from the closure assumption. Observations show that cloud physical properties depend upon the large-scale dynamics, while the strength of large-scale dynamics is closely related to the size of convective systems [see Xu et al. (2007)]. Although the more recent improvement (Bechtold et al. 2008) takes into account the effect of environment humidity on convective entrainment rate and produces stronger tropical variabilities, it remains to be seen whether the dependence of pdfs on cloud object size will be reproduced with the improved parameterization currently implemented in the ECMWF operational model.
c. Sensitivity tests
One of the questions raised from the comparisons presented in the preceding sections is whether or not the differences in the pdfs of EOA and ERA-40 cloud properties are due to the difference in the effective resolution of the numerical model, which is at least 4 times the grid mesh size (Pielke 1991). To address this question, the input data from EOA of the March 2000 period are averaged over two adjacent grid meshes in both horizontal directions. The averaged input data, which have the same total number of grid cells as in ERA-40, are then used in the last three steps of the analysis procedure outlined in section 3. This test is labeled as “EOA-avg.” The EOA, EOA-avg, ERA-40, and observed pdfs of six parameters are shown in Fig. 5.
Both the pdfs of τ and IWP for EOA-avg are identical to those of EOA (Figs. 5a,b), respectively, suggesting the averaged input has no impact on these pdfs. Although IWP is more severely underestimated in ERA-40 than in EOA, the differences in the pdfs of τ between EOA and ERA-40 are negligibly small. This is because ice diameters are smaller for smaller IWCs (Xu 2005) and τ is proportional to the ratio of IWP and ice diameter. The averaged input only slightly alters the pdfs of other parameters shown in Fig. 5, resulting from slightly smaller (averaged) IWCs. The differences in the pdfs of OLR between the EOA-avg and ERA-40 are smaller than those between the EOA and ERA-40 (Figs. 5e,f). All of these results suggest that different effective resolutions are not responsible for the larger discrepancies of the ERA-40 pdfs from the observations than the EOA (except for OLR). Thus, other differences in the model must account for the large discrepancies, such as the different data assimilation techniques (3DVAR in ERA-40 versus 4DVAR in EOA), which could significantly impact the strengths and horizontal structures of convective cloud systems.
Sensitivity to cloud overlap assumptions was examined by comparing the M–R overlap, which is the default assumption in this study, to either the maximum or random overlap. Because DC clouds are basically maximally overlapped, the pdfs of all cloud properties obtained from the maximum and M–R overlaps are identical, but those from the random overlap show some small differences in the pdfs of τ, IWP, and TOA SW (not shown). That is, there is slightly more power in the middle range of τ and IWP, which impact the pdfs of TOA SW. Impacts of these different assumptions on the frequency of occurrence (discussed later) and the pdfs of non-DC cloud properties (not shown) are, however, much more significant than the pdfs of the DC cloud properties. The non-DC clouds are those with τ < 10 or Ht < 10 km.
Last, how well do the EOA and ERA-40 pdfs compare to CRM-simulated pdfs, where the horizontal inhomogeneity of condensate and thermodynamic state is resolved? In Luo et al. (2007), both the March 1998 and March 2000 periods were simulated, where every simulation of the cloud objects is driven by advective forcings from the EOA. The large category of cloud objects from March 1998 is chosen for the comparison with EOA and ERA-40. Figure 6 shows that the CRM results are similar to the EOA rather than the ERA-40. [The EOA and ERA-40 pdfs for this month are not as smooth as those shown in Figs. 1 –3.] Probability densities of CRM cloud properties associated with the low end of the height range (10–12 km) are identical to those observed while those associated with the high end of the height range (13.5–16 km) are underestimated. The underestimate of the highest clouds is likely due to the use of horizontally uniform advective forcings in CRM simulations, which inhibit strong deep convection (Luo et al. 2007). On the other hand, the high power at the 10–13.5-km range is related to the underestimate of the magnitudes of τ (Fig. 6a), due to the underestimated cloud depths. The inclusion of SWC in IWP of the CRM pdf also improves its agreement with observations, whereas the magnitudes of IWP in ERA-40 are underestimated due to the lack of SWC data. The closer agreement in OLR and TOA SW pdfs with observations can be attributed to the horizontal inhomogeneity of condensate in the CRM simulations. Another closer agreement is that the dependence of CRM-simulated pdfs on cloud object size is stronger (see Luo et al. 2007).
d. Detection and overestimate of frequency of occurrence of cloud objects
Before addressing the detection and overestimate of occurrence of cloud objects, we will address how the ratios of the total numbers of DC subcolumns to the observed total numbers of satellite footprints (hereafter, the ratios) change with the different analysis data (i.e., ERA and ERA-40) and size categories and how the ratios are impacted by the different cloud overlap assumptions. As mentioned earlier, the January–August 1998 period is chosen for this discussion because of the large numbers of observed cloud objects.
Table 1 shows the ratios for S, M, and L categories obtained with four different cloud overlap assumptions using the EOA data. As discussed in section 3, the M–R overlap is also used in the Klein–Jakob procedure except for its nonrandom stacking of clouds within each ECMWF grid. It is expected that this procedure gives the smallest ratios (i.e., <0.8). There is almost no difference between the M–R and the maximum overlaps because the DC clouds are expected to be nearly maximally overlapped. But the random overlap produces unrealistically large ratios (up to 1.3). The ratio decreases as the cloud object size increases, especially from the M to L categories, for any of the four overlap assumptions. There are two reasons for this. One is that the overestimating rate is higher for smaller cloud objects, as discussed shortly. The other is that irregularly shaped, large cloud objects can be truncated by the subset data with a fixed area. Last, Table 1 shows that the ratios for the ERA-40 are farther away from 1.0 (1.241, 0.80, and 0.652 for S, M, and L categories obtained from the M–R overlap, respectively) than the EOA, which likely results from the underestimate in τ discussed earlier and the increased spatial–temporal mismatching of cloud objects that resulted from the less sophisticated data assimilation technique.
The missing and overestimating rates are shown in Table 2 for three categories of the EOA and ERA-40 data using the M–R overlap only. Two thresholds are chosen for the missing rate, which is the percent of cloud objects that have the ratio (i.e., the number of DC subcolumns divided by the number of observed satellite footprints) for each cloud object less than 10% or 20%. Similarly, two thresholds are chosen for the overestimating rate, which is the percent of cloud objects that have the ratio greater than 150% or 200%. Regardless of the chosen thresholds, the detection rate (100% − missing rate) increases drastically with the increase of cloud object size, for example, at 95% for the L size with the 10% threshold. This is expected because these large cloud objects are better resolved spatially by the model. The temporal mismatching is the most likely cause for the high missing rates of the smaller cloud objects because of the temporal gap of up to 3 h and their shorter durations, although the spatial mismatching also likely contributes to this. The overestimating rate is also much higher for the S category, due to the fact that some of the small cloud objects result from the truncation of large cloud objects by narrow satellite swaths (see Xu et al. 2007). The matched ECMWF areas of cloud objects are not restricted by the locations of satellite swaths so that the neighboring DC subcolumns (i.e., outside the swaths) can contribute to this overestimate of DC subcolumns.
A comparison of the ERA-40 and EOA results (Table 2) shows that the missing rates are reduced for the S (nearly 10%) and M (4%) categories and the overestimating rates are reduced for the M and L (∼5%) categories for ERA-40, compared to EOA. The overestimating rate for the S category of ERA-40 stays at the same high rates as EOA. This confirms the explanation given above. The fact that the missing rate is slightly higher while the overestimating rate is reduced for the L category of ERA-40 is consistent with the overall underestimate of DC subcolumns for this category of cloud objects. The results for the detection and overestimating rates suggest that the additional satellite data used in ERA-40 help to improve the detection rate, but the change of data assimilation techniques from 4DVAR to 3DVAR may be responsible for degrading the detection rate for the large categories of cloud objects.
5. Conclusions and discussion
This study has presented an approach to validate cloud physical and radiative properties, rather than just cloud and precipitation occurrences, of assimilated cloud systems by ECMWF operational analysis and reanalysis (i.e., ERA-40) against those obtained from satellite cloud object data. Each cloud object is matched temporally and spatially with ECMWF meteorological data. This approach attempts to convert the vertical profiles of grid-averaged cloud properties from large-scale models to pdfs of cloud physical properties measured at satellite footprints. Each ECMWF grid within the matched region is divided into subgrid cells that match the size of an average satellite footprint. An overlap assumption is used to distribute the ECMWF and ERA-40 cloud fields horizontally and vertically. The broadband radiative fluxes and cloud optical properties of all subgrid cells are computed with the Fu–Liou (1992) radiation code. The subgrid cells that satisfy cloud object selection criteria are then selected to construct pdfs of cloud physical and radiative properties for a category of cloud objects. This study has demonstrated the suitability of the proposed approach for validating cloud physical and radiative properties from large-scale numerical models. This approach can be extended to other climate models because the ISCCP simulator built into these models can provide most outputs needed for constructing pdfs of cloud physical properties. However, the number of subgrid cells used by the ISCCP simulator may have to be adjusted in order to match with the average size of CERES satellite footprints.
Results presented in this study indicate that most pdfs of cloud physical and radiative properties of tropical convective systems assimilated by EOA and ERA-40 agree with those of satellite observations for the period of January–August 1998. A closer agreement with EOA is seen in the results for most cloud properties except for cloud-top temperature and at the low OLR range. There are, however, significant discrepancies in selected ranges of the cloud property pdfs such as the upper range of EOA cloud-top height and the narrower ranges of ERA-40 τ and IWP. The most significant discrepancy is that the dependence of the pdfs on the cloud object size is not as strong in both the EOA and ERA-40 as in the observations. An improvement to the convective parameterization is thus suggested to remedy this discrepancy (i.e., introducing a stronger dependence of updraft penetration heights on grid-cell dynamics). The relevance of this suggestion to the current version of the ECMWF model needs to be reexamined because of recent changes in the ECMWF convective parameterization (Bechtold et al. 2004, 2008).
Modifications to the cloud parameterization in ECMWF that occurred in October 1999 (Gregory et al. 2000; Jakob et al. 2000) eliminated the clouds near the tropopause but shifted power of the pdf to lower cloud-top heights below the tropopause. This was achieved by limiting the penetration height of convective updrafts and modifying the convective closure assumption. The modifications to the formulation of ice sedimentation greatly reduced the ranges of IWP and τ because the increased SWC was not available from the standard ECMWF data archives. This limitation prevents a more quantitative evaluation of cloud physical properties because the selection of DC subcolumns relies upon the value of τ, which is partially related to SWC. A sensitivity test shows that both the pdfs of τ and IWP agree with observations much better by assuming that SWC is proportional to IWC. This test, however, does not show any improvement in the pdfs of other cloud physical properties, suggesting that anvil detrainment levels are strongly restricted to a narrow height range by the modified convective parameterization. Because of the decreasing grid size used in models, nondetrained condensate within updrafts can greatly contribute to the grid-averaged value, which is neglected in most models including ECMWF.
The agreements and disagreements of cloud physical property pdfs with those of observations for the March 2000 EOA persist in ERA-40 because of the use of the same cloud parameterizations, except for improvements in cloud-top temperature and in the low range of OLR. The coarser horizontal grid spacing used in ERA-40 is not a major reason for the less satisfactory results, compared to EOA. Instead, modifications in the cloud parameterization adopted in October 1999, the less sophisticated data assimilation technique, and the lack of SWC data are responsible for the stronger disagreements with observations. The detection rates of cloud object occurrence are improved for small-size categories, because of the use of more satellite data in ERA-40 than in EOA. The overestimating and missing rates remain high for the smaller-size categories. It is worthwhile to confirm these conclusions with the newest reanalysis data products, such as the ERA Interim.
This evaluation procedure will be applied to evaluations of other cloud object types, such as the boundary layer cloud objects (Xu et al. 2005, 2008). We plan to evaluate cloud physical properties of the non-DC region within the DC matched area (Eitzen and Xu 2008) in order to further evaluate the performance of ECMWF cloud parameterizations used in EOA, ERA-40, and the ERA Interim for the CERES TRMM periods once satellite footprint data for the non-DC region become available. This work will be reported in a separate study shortly. Ideally, an evaluation should also be performed with data from the newest cycle of the ECMWF operational model, but the release of CERES footprint data has a time lag of more than 1 yr.
The CERES data were obtained from the Atmospheric Sciences Data Center at the NASA Langley Research Center. This research has been supported by the NASA modeling, analysis, and prediction program and interdisciplinary study program (Drs. D. Anderson and H. Maring, program managers). The author would like to thank Dr. Takmeng Wong, Mr. Lindsay Parker, and Ms. Shengtao Dong for producing the cloud object data and the matched ECMWF data. Discussions with Drs. Bruce Wielicki, Takmeng Wong, and Zachary Eitzen are very helpful. Comments from Drs. Martin Köhler and Peter Bechtold of ECMWF and two anonymous reviewers are greatly appreciated.
Corresponding author address: Dr. Kuan-Man Xu, Climate Science Branch, NASA Langley Research Center, Mail Stop 420, Hampton, VA 23681. Email: Kuan-Man.Xu@nasa.gov
The CERES instrument on board the TRMM satellite was operational only for the January–August 1998 and March 2000 periods.