Introduction

It is well known that clouds play a fundamental role in the earth’s radiation budget and that the representation of clouds in general circulation models (GCMs) is one of the major factors limiting the accuracy of future climate prediction (Arking 1991; IPCC 1995). The limited spatial resolution available to current GCMs means that in addition to the usual cloud variables of liquid and ice water content, it is necessary for them to include some estimate of the fractional cloudiness in each gridbox. Most models diagnose cloud fraction every timestep from prognostic model variables such as humidity or total water content (Slingo 1987; Smith 1990), but in the current version of the European Centre for Medium-Range Weather Forecasts (ECMWF) model, cloud fraction is itself formulated as a prognostic variable (Tiedtke 1993). Whether diagnostic or prognostic, the physical basis underlying the treatment of this parameter is somewhat uncertain, and consequently errors in model radiative fluxes are often blamed on poor model cloud fraction. GCM studies have shown that these errors can then feed through into precipitation and circulation patterns (Slingo and Slingo 1988; Randall et al. 1989).

A common way to test the representation of clouds in models is to compare upwelling top-of-atmosphere fluxes from the model with satellite measurements, using either the broadband outgoing longwave radiation (e.g., Slingo 1987) or brightness temperature in specific satellite longwave window channels (Morcrette 1991). This approach is useful to evaluate the overall radiative impact of the model clouds but is hampered by the fact that the radiative fluxes represent integrals over many parameters, such as cloud fraction, cloud water content, cloud-top height, atmospheric conditions above the cloud, and the microphysical cloud properties. A further complication is that low clouds, which have a brightness temperature close to that of the surface, are not easily detected.

A large number of cloud products are now generated routinely by the International Satellite Cloud Climatology Project (ISCCP) from both the visible and infrared channels of the various meteorological satellites. Jakob (1999) compared monthly-mean total cloud cover from the ECMWF reanalysis (in which no observational data were used to alter clouds directly) with the ISCCP values and identified a number of biases, including an underestimate of extratropical cloud cover over the ocean by 10%–15% and an overestimate of trade wind cumulus by the same amount. Similarly, Karlsson (1996) found from a 2-month comparison with Advanced Very High Resolution Radiometer data over the Scandinavian region that the ECMWF model underestimated total cloud cover by 13%. He also found that the fraction of the total cloud cover attributable to “high” clouds was greater in the model by up to a factor of 2.

The principal limitations of such passive measurements are the lack of vertical resolution and the poor penetration when multiple cloud layers are present. Furthermore, the total cloud cover calculated from model fields depends not only on the values of cloud fraction at each model level, but also on the overlap assumption;the difference in predicted total cloud cover between assuming “maximum” and “random” overlap is typically near 10% (e.g., Morcrette and Jakob 2000), of comparable magnitude to the differences found by Karlsson (1996) and Jakob (1999). There is clearly a need for high-vertical-resolution observations by active instruments to overcome these problems by measuring cloud fraction at each model level. Mace et al. (1998) compared hydrometeor occurrence from 3 months of 35-GHz radar data with the same information held in the ECMWF model and found the model to have good overall skill at predicting the mean vertical distribution of clouds, despite some problems in the timing of the onset and dissipation of deep-cloud events. The distinct advantages of active instruments were demonstrated, but in their study the radar was used only to determine whether any cloud or precipitation had occurred in each 30-min period; cloud fraction was not calculated.

In this study, 3 months of radar and lidar observations at Chilbolton, England, during the winter of 1998/99 are used to derive cloud fraction for comparison with the ECMWF model. The same dataset was used by Hogan and Illingworth (2000) to characterize the degree of cloud overlap for use in GCMs. Throughout most of the depth of the atmosphere, one of two vertically pointing cloud radars is used, but up to the freezing level we also use the cloud base reported by a lidar ceilometer. The details of the model and the observational data used in the study are outlined in section 2, and in section 3 the technique is described. In section 4 the observed cloud fraction is compared directly with the values held in the model, and in section 5 the comparison is refined to account for important differences in the treatment of both snow and very thin cirrus, allowing a more detailed analysis to be performed in section 6.

Description of the data

Instruments at Chilbolton

The primary observational data used in this study were taken by the two vertically pointing cloud radars located at Chilbolton in southern England, between 24 October 1998 and 23 January 1999. The 35-GHz “Rabelais” radar was operational from the start of the experiment until 20 November, and the 94-GHz “Galileo” was operational from 5 November until the end of January. Together they collected a near-continuous dataset of radar reflectivity factor Z, which, in the Rayleigh-scattering regime, we define as

View Expanded

where D is the particle diameter in millimeters, and |K|² is the “dielectric parameter.” The factor of 0.93 is present to make Z relative to liquid water at centimeter wavelengths.

For extra sensitivity, the raw pulses were averaged over 2 min and two range gates (where the range-gate spacing was 75 m at 35 GHz and 60 m at 94 GHz). This averaging resulted in minimum-detectable reflectivity factors at 1 km of about −50.5 dBZ at 35 GHz and −52.5 dBZ at 94 GHz. Hence, in the period when both radars were operational, the 94-GHz radar was invariably used for the comparison with the model because of its nominally higher sensitivity, and, in the end, less than one-fifth of the 3-month comparison was performed with 35-GHz data. It is our experience that radar calibration using a radio-frequency link budget can be very error prone (especially for a bistatic system such as the 94-GHz Galileo), so our approach was to compare Z values at 35 and 94 GHz with those at 3 GHz in drizzle, which Rayleigh-scatters at all three frequencies. The 3-GHz weather radar at Chilbolton was absolutely calibrated to within 0.5 dB using the redundancy of the polarization parameters in heavy rain (Goddard et al. 1994), so we believe that the calibration of the two cloud radars should be accurate to about 1.5 dB.

Below the melting level, we made use of the continuous observations of the 905-nm Vaisala, Inc., CT75K lidar ceilometer at Chilbolton, which records 30-s-averaged profiles of lidar backscatter coefficient β with a vertical resolution of 30 m. The output from this commercial instrument also includes cloud-base height as diagnosed from the β profile. In practice, liquid water clouds always give strong echos, and the sharp gradient at cloud base can easily be picked out by the lidar.

Measurements were also taken by a drop-counting rain gauge and were used to identify when attenuation by rain could potentially have been a problem for the 94-GHz radar.

Method

The principle of the technique is straightforward; daily time–height sections of radar reflectivity are divided into boxes of 1-h duration centered on the model height levels, and cloud fraction is taken to be the fraction of radar pixels in each box that register the presence of cloud. For example, at a height of 5 km where the level spacing is 600 m, the 35-GHz radar with its 2-min/150-m pixel size records 120 pixels in each box, allowing cloud fraction to be discretized to better than 0.01. Radar is relatively unattenuated by cloud, enabling it to detect multiple layers through the whole depth of the atmosphere. However, it has some limitations in the lowest 2 km that can be overcome only with the extra information provided by the lidar.

First, radar returns below about 400 m cannot be used because of leakage by the transmitted pulse into the receiver and the presence of ground clutter. Consequently, the radar cannot retrieve cloud fraction in the lowest two levels of the model. The lidar, on the other hand, is able to make measurements right down to the surface.

The second problem is that, because Z is proportional to the sixth power of particle diameter, the radar is unable to pick out the base of liquid water clouds in the presence of rain or drizzle. This is because the signal is dominated by the contribution from these much larger drops. Lidar backscatter coefficient, on the other hand, is approximately proportional to the second power of diameter, so the base of liquid water clouds can always be identified as a sharp and substantial increase in β embedded within the weaker signal from the precipitation. It is found that even stratocumulus clouds often have sufficient drizzle falling out of them that cloud-base height according to the radar alone is too low by several hundred meters. Invariably the signal from the lidar is rapidly extinguished in liquid water clouds and the radar must be used at higher levels.

A further problem often encountered in the interpretation of cloud radar data in the boundary layer is contamination of the meteorological signal by insects. At the latitude of Chilbolton during winter, insects are simply never seen; but, in principle, a lidar could easily distinguish them from clouds because, as with raindrops, the typical value of β from these very large but low-concentration targets is very small.

To illustrate the complementary nature of the two instruments, Fig. 1 depicts 9 h of observations in light precipitation associated with a warm front, overlaid by the model grid. The radar detects the full vertical extent of the cloud and precipitation, whereas the lidar records a very strong signal from the cloud base (which is not detected by the radar), above which it is rapidly extinguished. It also detects boundary layer aerosols, although they give a much weaker echo and are never mistaken for cloud. Cloud base measured by the lidar is shown superimposed on the radar image and is used to refine the cloud fraction diagnosed by the radar alone;in this case, the cloud fraction in each grid box entirely below cloud base is set to zero, and, for those that straddle cloud base, the cloud fraction can be calculated accordingly. However, the lidar is not used above the melting level (as diagnosed from the model temperature field), where we leave the original radar cloud fraction unaltered. The logic behind this approach is that there is no physically appropriate distinction between ice cloud and ice precipitation that can be exploited in the same way as that between liquid cloud and rain. In ice, the radar and lidar returns tend to be similar in appearance, with typically very good agreement in the height of the lowest echo and no sharp gradient in the lidar β profile above to mark the transition from precipitation to cloud. In any case, the lidar usually cannot even penetrate as far as the melting level because of the common occurrence of strongly attenuating stratocumulus. The issue of discriminating cloud from precipitation is discussed further in section 5a. When the lidar does detect a sharp gradient and a high value of β above the melting level, it generally corresponds to a layer of supercooled liquid water; in Fig. 1, the strong echo of the liquid cloud base extends up through the 0°C isotherm to a height of 4 km where the temperature was −10°C. It would be difficult to explain the complete extinction of the lidar signal in only a few hundred meters as being due purely to ice crystals. Supercooled clouds such as this are fairly common at the leading edge of warm fronts.

We also use the lidar cloud base to compensate for the fact that some liquid-water clouds are too low or too tenuous to be detected at all by the radar. If the lidar detects any cloud warmer than 0°C that is not seen by the radar, then cloud fraction is increased assuming that the cloud is no more than one model-level deep. In practice it is rare for a low cloud to be completely undetected by the radar, and it is usually only fog that is too low for the radar to detect.

Figure 2 demonstrates the retrieval of cloud fraction from a whole day of data. The top two panels show the raw radar and lidar echos, and the third panel shows rain rate. Because rain can occasionally strongly attenuate the 94-GHz signal, we do not use derived cloud fraction in the comparison when the rain rate at the ground exceeds 0.5 mm h⁻¹ (shown by the dot–dashed line) at any time during the hour-long accumulation period. At 94 GHz, a rain rate of 0.5 mm h⁻¹ corresponds to a two-way attenuation of about 1 dB km⁻¹ (Lhermitte 1987). Indeed, the sharp decrease in observed high cloud in the radar image at around 1800 UTC is probably due to attenuation by rain. However, less than 10% of the data were excluded in this way, and consequently, the effect on the results was small. The fourth panel shows the cloud fraction derived from the observations, and the simultaneous model values are shown in the last panel. The 0°C isotherm was at about 2.3 km on this day.

For the comparison with the model to be valid, we are essentially assuming that in 1 h the clouds sampled over a single point will be representative of the clouds in an entire 60-km grid box. A wind speed of 17 m s⁻¹ is sufficient for clouds at both sides of the grid box to be sampled within the accumulation period, and this value is only a little lower than the actual mean wind speed over the site; of course, it is still only a two-dimensional slice through a three-dimensional volume. In an attempt to match the time step in the model, an accumulation period of 20 min was also tried, but the resulting cloud fraction was found to be much noisier than the corresponding model field, probably because the sample was not sufficiently representative of the whole grid box.

Direct comparison of observed and modeled cloud fraction

Cloud fraction has been calculated from the radar and lidar observations between 24 October 1998 and 23 January 1999 at Chilbolton, and Fig. 3 shows observed and modeled cloud fraction for a 10-day period in December of 1998. The larger-scale features match up reasonably well, although there is certainly room for improvement.

Figure 4 shows mean cloud fraction as a function of height from both the model and the observations for the entire 3-month period. We see a clear tendency for the model to underestimate cloud fraction below 7 km and overestimate it above. Figure 5 shows this information split up into “frequency of occurrence” [similar to the parameter compared in the study by Mace et al. (1998)] and “amount when present,” where cloud is deemed to be “present” when the cloud fraction is greater than 0.05. The results are found to be fairly insensitive to the exact value of this threshold. Up to 9 km, the frequency of occurrence agrees remarkably well, so it would seem that the differences in mean cloud fraction found in Fig. 4 are due almost entirely to systematic errors in simulating the amount of cloud when present. Above 9 km, the model appears to overestimate frequency of occurrence.

Problems with the direct comparison

Direct comparison of the modeled and observed cloud fraction data apparently has revealed systematic differences between the two, with the model tending to underestimate the amount when present at low and midlevels and overestimate it at high levels (Fig. 5b). In this section, we explore the extent to which these discrepancies can be explained by differences in what does and what does not contribute to cloud fraction in the model and the observations, with the aim of providing a more robust diagnosis of any shortcomings of the model.

Comparison using modified model cloud fraction

The comparisons of mean cloud fraction, frequency of occurrence, and amount when present are repeated in Figs. 8 and 9 for the two modified model cloud fractions described in the last section, ECMWF1 and ECMWF2. The inclusion of snow makes a difference almost exclusively in the height range 1–6 km, whereas the exclusion of tenuous cirrus has a noticeable effect only above 7 km. Both have had the effect of bringing the model closer to the observations, and in Fig. 8 we see much better agreement in mean cloud fraction at all heights as compared with Fig. 4, although there is still too much high cloud by up to a factor of 2. The frequency of occurrence depicted in Fig. 9a still agrees very well below 9 km, because the addition of snow usually involves increasing cloud fraction where some cloud is already present. Above 9 km, the agreement is much better than before; the exclusion of model IWC data below the radar detectability threshold has effectively more than halved the frequency of occurrence. The amount when present (Fig. 9b) now agrees significantly better at midlevels, but above 7 km the previously large overestimate (Fig. 5b) now appears to be even worse. This is an artifact of crudely setting the cloud fraction of a grid box to zero when the mean IWC_ic is deemed to be too tenuous to be detected by the radar, because there are more occasions in the dataset with low rather than high cloud fraction when IWC_ic is below the radar detectability threshold. Had we assumed some kind of spatial variation of IWC within the cloudy part of each model grid box, then the removal of undetectable cloud would have involved a partial reduction of cloud fraction in a larger number of grid boxes, and the resulting amount when present would have had much more realistic values. However, such an assumption of spatial variation would have had no effect on mean cloud fraction, which depends only on the value of IWC_min, so the finding of an overestimate in high-level mean cloud fraction (Fig. 8) is still valid.

The addition of snow does not seem to have had an effect below 750 m, where there is still an apparent underestimate of the amount when present; below 300 m, the difference exceeds 0.3 (about a factor of 2.5). It is possible that the thickness of these low clouds is overestimated by the observations when it is assumed that, in the absence of any signal in the radar, they extend upward from the lidar-measured cloud base to the top of the grid box. One should remember that in the model radiation scheme it is assumed that clouds fill grid boxes vertically, whereas a cloud fraction of 0.5 in the observations could, in principle, correspond to a cloud cover of 100%, if the cloud was concentrated in the top or bottom half of the box. Thus it would seem certain that the model is underestimating the contribution to cloud cover from the clouds in the lowest 750 m and probably is also underestimating cloud fraction.

Clearly, modifying the model fields has resulted in a significantly better overall agreement with the observed 3-month means, but we still need to verify that clouds in the model occur at the right time. The ECMWF1 modified model cloud fraction (corresponding to a critical snow flux of 0.05 mm h⁻¹) produces slightly better agreement than does ECMWF2, so for the comparisons in the remainder of this section we use ECMWF1.

First we determine how much, on average, the model is in error at any given instant. Figure 10a shows the mean absolute difference in cloud fraction between the model and the observations as a function of height. Throughout most of the depth of the atmosphere, the difference is near 0.17. In Fig. 8, we see that this value is not much less than the typical mean cloud fractions of near 0.2, implying that the model is performing poorly. However, the requirement that the model should be accurate to within 1 h and one model level is overly stringent, and one should bear in mind that the model radiation scheme is not run every time step. It would perhaps be fairer to first average both fields onto a coarser grid that better reflects the spatial and temporal accuracy required.

Mean absolute difference is shown as a function of time of day in Fig. 10b, and we see that there is a slight tendency for the error to increase through the day. This is simply because the model fields were produced on a daily basis from forecasts initialized at 1200 UTC the previous day, and the accuracy of the model naturally decreases with time. It also explains the abrupt changes sometimes seen in model cloud fraction at midnight, between the last hour of one daily forecast and the first hour of the next.

So if the model is performing poorly on a pixel-by-pixel comparison and yet the mean values are accurate, is it simply simulating clouds too early or too late? Figure 11 depicts the correlation between the two fields as a function of height and time offset. We see that throughout most of the depth of the atmosphere, as should be expected, the correlation coefficient is highest (at about 0.5) when there is no time lag between the two fields. However, in the lowest kilometer, the maximum correlation occurs with a positive offset, indicating a surprising tendency for the model to simulate cloud features up to 3 h before they were observed. This result is somewhat difficult to explain. A possibility is that the diurnal cycle in the model is wrong, and, for example, the transition from a stable nocturnal boundary layer to a well-mixed stratocumulus-topped daytime boundary layer occurs too rapidly after sunrise. However, it is interesting to note that when the lagged correlation is performed on separate 6-h periods through the entire day the same tendency is always present. It should also be kept in mind that the model grid box used for the comparison is actually centered on a point 25 km to the northwest of Chilbolton, and, in winter in England, the weather is characterized by fronts that approach from the west. However, to produce a lag of 3 h, the average frontal speed would have to be about 2 m s⁻¹, which is unrealistically small.

In the diagnostic cloud scheme of Smith (1990), cloud fraction is calculated from the total water content, which is prognostic, and its standard deviation, which is estimated from other model products such as the Richardson number. It is easy to imagine that if the model had a tendency to underestimate the standard deviation of total water content then, although the mean cloud fraction might be correct, the instantaneous values would too often take the extremes of 0 or 1 (corresponding to completely clear or completely overcast skies within one model level). To test whether this is a problem for the prognostic scheme of the ECMWF model, Fig. 12 shows a comparison of the frequency distribution of cloud fraction in three different height ranges (bounded by the 800- and 450-hPa surfaces). For low- and midlevel clouds, there is a slight skew between the frequency distributions but no significant tendency for the model to significantly under- or overestimate the occurrence of partial cloudiness. It is probable that the overestimate of the frequency of completely cloudy grid boxes at midlevels is an artifact of increasing the cloud fraction when snow is present to be equal to the largest cloud fraction above. For high clouds, there does seem to be a strong tendency for the model to predict partial cloudiness too infrequently, but this is not really meaningful because, as discussed above, the process of removing tenuous cirrus from the model so that the comparison of mean cloud fraction was fair involved the disproportionate removal of partial cloudiness events.

Discussion and conclusions

A technique for deriving cloud fraction from radar and lidar data has been developed and successfully implemented in a 3-month comparison with the ECMWF model. The very different scattering properties of radar and lidar were found to complement each other well and, in particular, allowed cloud fraction to be calculated accurately even in the presence of drizzle and light rain.

Direct comparison between the model and the observations revealed that the model was excellent at simulating the frequency of cloud occurrence but that it tended to underestimate the amount present at midlevels and overestimate it at high levels. The difference at midlevels was found to be largely attributable to the exclusion of snow from the consideration of cloud fraction in the model, and it was also found that some cirrus clouds in the model were associated with ice water contents that were too low be detected by the radar. Adjustment of the model cloud fraction field to both include snow and exclude tenuous cirrus resulted in considerably better agreement with the observations. In particular it was found that

• below 750 m the model underestimated amount when present by up to 0.3;

• between 750 m and 7 km there was excellent agreement between mean cloud fraction, frequency of occurrence, and amount when present;

• above 7 km the model overestimated mean cloud fraction by up to a factor of 2, even after the IWC threshold had been applied to remove “undetectable” cirrus;

• the pixel-by-pixel correlation coefficient between the observed and modeled cloud fraction was near 0.5 at all heights; and

• in the lowest 1 km the model tended to simulate cloud features up to 3 h before they were observed.

With regard to the incorporation of snow into the model cloud fraction, we have shown that ice does exist in the form of snow in the model to make up for the underprediction of cloud fraction at midlevels and that use of a “critical snow flux” of 0.05 mm h⁻¹ leads to very good agreement in both frequency of occurrence and amount when present. However, the radiation scheme still “sees” the unmodified cloud fraction profile, despite the fact that the observations (most significantly those of the 905-nm lidar) clearly show snow often to be more optically thick than the ice cloud above it. Moreover, the observations do not show any sharp distinction between ice cloud and ice precipitation (Fig. 7) in the same way that they do between liquid water cloud and rain (Fig. 1), which suggests that it is more physically meaningful to treat ice cloud and snow together as a single variable (e.g., Wilson and Ballard 1999). The inclusion of snow in the radiation scheme would probably not significantly affect outgoing longwave radiation but could be important for the radiative heating profile at midlevels. The fact that we were able to achieve such good agreement at midlevels by a simple modification of the model field based on snow flux suggests that the model could do the same. Note that the value of critical snow flux that gave the best agreement in this study is not necessarily transferable between models; because the distinction between ice cloud and ice precipitation is somewhat arbitrary, the dividing line between the two would be expected to occur at a different point in other models.

In the case of tenuous cirrus, adjustment of the model fields was necessary to account for a deficiency of the observations rather than of the model, and, although it reduced the amount of high cirrus, it still resulted in an apparent overestimation of cloud fraction by up to a factor of 2 above 7 km. Despite the agreement with the study of Karlsson (1996), this result is unfortunately not as robust a finding as we would like because about one-third of the total cloud volume above 8 km had to be removed from the model and because there were inherent inaccuracies in our calculation of the minimum-detectable IWC. The clouds removed from the comparison typically had an IWC of less than 10⁻³ g m⁻³, which for a 1-km-thick cloud with an effective ice crystal radius of 10 μm corresponds to an infrared optical depth of only 0.08. This valve is near the radiative significance threshold proposed by Brown et al. (1995) for midlatitude cirrus. The optical depths in the ECMWF model would be even lower, because the model radiation scheme uses an effective ice particle radius (and implied particle concentration via IWC) that varies with temperature from 40 μm at −55°C to 140 μm at −20°C (Ou and Liou 1995).

One could legitimately argue that the importance of cloud fraction for the radiation budget is much less in the case of very optically thin cloud, but only observations by a more sensitive radar could establish whether the low-IWC clouds that the model is simulating really exist. Mace et al. (1998) were able to achieve a greater sensitivity principally by the use of pulse compression, but this approach usually has the disadvantage of a blind zone of at least 1 km near the surface. The change in 94-GHz attenuation as low clouds pass overhead also introduces some uncertainty in the estimation of cloud fraction at high levels, and would seem to make 35 GHz a better choice for a single-wavelength ground-based system. This is particularly true if radar reflectivity itself is to be used quantitatively in cirrus, such as for estimating IWC.

Identification of a tendency in the model to under- or overestimate cloud fraction at a certain height does not necessarily indicate which part of the model needs modification. Even if the cloud scheme were perfect, any errors in the underlying dynamics would still feed through to cloud fraction. There is plenty of scope for further analysis of these data to address this issue, and one particular approach might be to combine the observations by synoptic type to expose consistent tendencies for the model to simulate certain features badly, such as the slope of fronts or the formation and dissipation of jet-stream cirrus and anticyclonic stratocumulus. This approach would then point toward which particular parameterization was in need of attention.

Note that, because the comparison was carried out during winter at a site in the midlatitudes, convection was unimportant in generating clouds, and the study is therefore essentially a validation of the stratiform part of the cloud parameterization. It would be very interesting to apply the technique during midlatitude summer or in the Tropics where convection is much more significant. The fact that the model has been shown to be reasonably good at simulating the presence of cloud opens the way for a more detailed comparison of model cloud parameters such as water content and particle size with the values derived by active ground-based instruments. The proposed spaceborne cloud radar and lidar (scheduled for launch in 2003 as part of the National Aeronautics and Space Administration Earth System Science Pathfinder Program) would be able to extend this comparison over the whole globe. This study also highlights the possibility that such instruments could provide real-time information on clouds for assimilation into models.