1. Introduction

The representation of the vertical overlap of cloud in climate and numerical weather forecast models is key for their radiative transfer calculations (Morcrette and Fouquart 1986; Slingo and Slingo 1991; Morcrette and Jakob 2000; Chen et al. 2000; Wu and Liang 2005). Zhang et al. (2013) recently demonstrated that one of the important causes of differences between global model radiation schemes was their treatment of cloud overlap. The cloud overlap schemes in models are derived and evaluated using available ground-based or satellite observations; thus, it is important to have a clear understanding of the latter.

Cloud overlap is usually defined in terms of three basic idealized assumptions: maximum, random, or minimum. One can consider clouds in two layers to be formed from a coherent dynamical process and, thus, to be maximally overlapped (MAX). If the cloud cover in two layers is given by C_i and C_j, respectively, this assumption minimizes their combined total cloud cover . Alternatively, the clouds in the two layers may be uncorrelated and therefore randomly overlapped (RAN) (Geleyn and Hollingsworth 1979; Morcrette and Fouquart 1986). Averaged over many scenes, the mean overlap in such cases is . A third possibility is that clouds are minimally overlapped with cloud cover: . In theory this might occur if the presence of a cloud led to a lower probability of a cloud nearby through suppression processes, although there is limited evidence that this occurs (Weger et al. 1992; Zhu et al. 1992).

In nature, the observed total cloud cover resulting from the overlap of two layers in any particular cloud scene is unlikely to equal that given by one of the above assumptions; thus, it is useful and usual to express the observed total cloud cover as a linear combination of MAX and RAN using the overlap parameter:

This expression was introduced by Hogan and Illingworth (2000, hereinafter HI00), and it is apparent that maximally overlapped layers give , while in the case of random overlap. Note that observed total cloud cover of the two layers can exceed the random overlap value, as in the case of minimum overlap, for which α will assume a negative value. Barker et al. (1999) used cloud-resolving model output to show that the overlap of continuous clouds often lies between MAX and RAN for the scenes studied. This possibility was formalized in an overlap scheme suggested by HI00, who used cloud retrievals from the Chilbolton radar in the United Kingdom for its derivation. The data showed that clouds separated by clear layers (referred to as discontinuous) were approximately randomly correlated, although α was negative for large layer separations, while vertically continuous cloudy layers were maximally overlapped at small separation distances but steadily decorrelated as the separation distance between the layers increased. HI00 fitted an exponential function to the reduction in α with layer separation distance D:

calculating the decorrelation length scale for continuous clouds to be approximately 2 km. The scheme was termed the EXP-RAN scheme by Tompkins and Di Giuseppe (2007). The quality of the exponential fit was examined in Willén et al. (2005), who suggested from observations that this derived from an exponential distribution of cloud geometrical thicknesses, which Astin and Di Girolamo (2006) subsequently confirmed analytically.

The analysis of HI00 revealed several characteristics of the cloud overlap statistics that have since been reproduced in further observational studies but, to date, have not been fully explained. First, in their data analysis, HI00 translated the temporally continuous data stream of the radar observations into an effective spatial scale, assuming a constant horizontal wind speed, and then divided these data into finite segments of various lengths, ranging in size from 24 to 216 km. We will refer to this as the sampling scale. This range of sampling scales was chosen to reflect typical grid lengths used in atmospheric models employed for numerical weather predictions or climate studies. HI00 reported that the decorrelation length scale of clouds was sensitive to the sampling scale employed, with shorter decorrelation length scales diagnosed when using shorter sampling scales, indicating a greater tendency toward random overlap. A second characteristic of the data analysis was that, for large cloud separations of clouds separated by clear layers, negative values of α were found, indicating a tendency for minimum overlap.

These conclusions have been confirmed in other, more recent observation-based studies, with a significant sensitivity of α and its decorrelation length scale for continuous clouds also reported in the studies of Naud et al. (2008), Willén et al. (2005), and Oreopoulos and Norris (2011). Mace and Benson-Troth (2002) also found this behavior in three of the four Atmospheric Radiation Measurement Program (ARM) study sites. Additionally, Naud et al. (2008) found that discontinuous cloud scenes always tended toward minimal overlap in that study, with negative α values of up to −0.2. Shonk et al. (2010) also documented a (weaker) sampling-scale sensitivity.

To demonstrate these characteristics here, we show an analysis of α derived for continuous and discontinuous clouds using CloudSat–CALIPSO data (see methods section for details of the data and calculation) divided into respective sampling scales of 10, 100, and 1000 km (Fig. 1). For clouds separated by a clear layer, α is close to zero, indicating random overlap, in agreement with HI00 and subsequent work. It is mostly positive when calculated for a sampling scale of 1000 km and mostly negative for 10- and 100-km sampling scales. Although the magnitude of the negative α values is small, the negative mean value for 10- and 100-km sampling scales is significantly different from zero at the 99% level using a two-tail Student’s t test, because of the large number of scenes (exceeding 10⁶ for 100 km) that contribute to the statistic. By fitting an exponential curve to the α values of continuous clouds (see methods and the first column of Table 3), is derived. The scene sampling length also affects the value of the decorrelation length scale. When the data are divided into scene lengths that are 10 km long, the decorrelation length scale is around 2.2 km, within the 1.4–4.0-km range of Kato et al. (2010), which increases to 3.8 km when a sampling scale of 1000 km is employed.

• Download Figure
• Download figure as PowerPoint slide

Overlap parameter for continuous and discontinuous cloud scenes derived from CloudSat–CALIPSO data, sampled using a scene length of 10, 100, and 1000 km (see legend).

Citation: Journal of the Atmospheric Sciences 72, 8; 10.1175/JAS-D-14-0278.1

Previous studies, using both ground-based and satellite-based instrumentation of varying sensitivities, have thus shown a consistent picture of cloud overlap properties, confirmed here. An explanation of these overlap characteristics—in particular, the scene length sensitivity of the diagnosed α and values—is required in order to be able to transfer this understanding to cloud and radiation parameterization schemes of models with confidence. Concerning the scene length sensitivity, both HI00 and Mace and Benson-Troth (2002) offered arguments based on the consideration of a single cloud element. HI00 stated that this was a simple result considering a single isolated cloud, since, as the sampling scale increases, the cloud fraction represented by that physical element would decrease, and the overlap would tend toward maximum. Mace and Benson-Troth (2002) expressed this consideration of a single cloud element mathematically, showing that the differential of α with respect to the observed cloud cover, , is always negative; thus, α must decrease as the sampling scale decreases. However, as pointed out by Mace and Benson-Troth (2002) themselves, the above argument may not hold more generally, since, as the sampling scale increases, scenes will likely include further cloud elements.

Astin and Di Girolamo (2014) recently conducted an analytical study of the scale dependence of α, considering the case of two cloud scenes that are subsequently merged into one single scene of twice the size, suggesting as a result that α should increase with sampling scale. However, their analysis was based on the restrictive assumption that the cloud fraction of the two layers is identical in each box [derivation of their Eq. (11)]. If this assumption is relaxed to allow differing cloud fractions in the two scenes, α may increase or decrease when merging, depending on the cloud configuration of the two scenes.

The negative α values often diagnosed for discontinuous clouds have also not been justified. Naud et al. (2008) highlighted, but did not offer an explanation for, the negative α values found for discontinuous clouds. HI00 suggested that the negative values of α, indicating a tendency toward minimum overlap, were a chance outcome of the scene length used and the typical horizontal wind profile over their observation site in the southern United Kingdom.

In this paper, we attempt to explain the above observational characteristics of overlap statistics by considering how they will change as the data sampling scale changes relative to the typical cloud system scale. After introducing the data sources used in the paper in the next section, we will offer a simple heuristic argument that data truncation could potentially lead to negative biases in the diagnosed α at larger cloud separations, shorter diagnosed decorrelation length scales and negative α values for clouds that are actually randomly overlapped. We next demonstrate that this argument holds using simple cyclic cloud fields as well as realistic fractal clouds, and we offer a simple cloud filter for cloud observations that results in a scale-invariant diagnosis of the overlap statistics.

2. Method

b. Cloud data

Three sources of cloud data are used in this analysis, which are introduced in turn.

1) Idealized cyclic clouds

We consider a very simple cloud system that consists of a single cloud element of a fixed length that is repeated infinitely in the horizontal direction, with a spacing equal to (Fig. 2). The simulated retrieval resolution is such that a scene of length X consists of columns. The system is plane parallel, and the system is repeated cyclically. Thus, the cloud fraction measured at scales of nX, where , is 50% in each scene. If the layer is sampled with scenes that are noninteger multiples of X, then other cloud fraction values are obtained in each segment. The length scale X on which the cloud is periodic is be referred to as the cloud system scale. In general, we will use X to represent the cloud system scale for all cloud data sources.

• Download Figure
• Download figure as PowerPoint slide

Schematic of the overlap of two adjacent clouds that form a plane-parallel cloud system, which repeats.

Citation: Journal of the Atmospheric Sciences 72, 8; 10.1175/JAS-D-14-0278.1

The cyclic cloud is repeated in a second layer, which can be offset (shifted) with respect to the first layer. If the offset is zero pixels, then the cloud layers are maximally overlapped, while if the offset distance is , then the layers are minimally overlapped. Each possible overlap ranging from maximum to minimum is considered in turn; that is, in the first experiment the cloud layers are maximally overlapped, in the second experiment the cloud in layer 1 is shifted one column to the right, and so on until the layers are minimally overlapped, giving a total of experiments. In each experiment, the plane-parallel scene is progressively sampled at decreasing , starting at X and decreasing by one column at a time until the scene length is equal to . We refer to the ratio as the sampling resolution.

The difference in the overlap parameter sampled at and at X provides the bias in α. We may not sample the clouds at scene length exceeding X because of the issue of aliasing when using periodically repeating cloud systems. A summary of the symbols used in the following analysis is reported in Table 1.

Table 1.

Definition of symbols.

2) Idealized fractal clouds

It is also helpful to analyze clouds that are realistic in their fractal structure but are artificially generated so that the power spectra of the cloud variability can be specified. For this, a cloud generator is the obvious tool (e.g., Evans and Wiscombe 2004; Schmidt et al. 2007) and we adopt the Spectral Idealized Thermodynamically Consistent Model (SITCOM), which is introduced in detail in Di Giuseppe and Tompkins (2003). Here, we use the model in a two-dimensional plane, and no information concerning vertical thermodynamic structure is required. The spatial variability is controlled using the power law spectra of total water mixing ratio given in Di Giuseppe (2005):

where is the spatial frequency, is the total water standard deviation, and β defines the slope of the power spectra, which is set to here. The exponential term on the right sets a large wavelength cut off, essentially defining the cloud system scale. Solving for , the power peaks at a frequency of . Thus, in this case, X is related to the wavenumber peak , and is chosen to give X = 100 km. A spatially constant value for the saturation mixing ratio provides a cloud fraction of 50%, assuming no supersaturation can exist.

We generate 50 random scenes of 2000 km × 2000 km using SITCOM with a horizontal resolution of 1 km, similar to CloudSat observations, and each one is sampled with horizontal lines spaced 500 km apart to ensure the transects are uncorrelated. A sample scene is shown in Fig. 3, which also shows how the liquid water mixing ratio is converted to a simple cloud mask. As with the cyclic clouds, a two-layer system is constructed by repeating the transects in a second layer placed above the first, and a series of overlap scenarios is generated by progressively shifting one layer with respect to the other. The two cloud layers are first aligned to give maximum overlap (). Then successive experiments are generated by shifting the top cloud layer one pixel at a time. This increases the total cloud cover of the two layers combined (sampled at 2000 km) and thus reduces α below 1. As the top cloud layer is progressively shifted, the two layers will eventually become decorrelated, and as the layers are randomly overlapped. For each shift distance, we monitor the impact of scene length and data truncation by sampling at scene lengths ranging from X′ = 0.2X to X′ = 2X. The difference in α diagnosed at these scene lengths to the value obtained using the whole 2000 km (=20X) cloud tract is referred to as the α bias. In the initial case of zero shift distance and maximum overlap, the bias in α is identically zero for all scene lengths.

• Download Figure
• Download figure as PowerPoint slide

(left) Example SITCOM-generated field with horizontal lines showing sample trajectories. (top right) A zoom in of a 200 km × 200 km subscene shown to emphasize the cloud system scale of X = 100 km; (bottom right) the resulting cloud mask. This cloud mask is repeated in two vertical layers in the experiment, and then one layer is shifted with respect to the other to provide a range of overlap situations in each sampled scene (see text for details).

Citation: Journal of the Atmospheric Sciences 72, 8; 10.1175/JAS-D-14-0278.1

3) Observed clouds

CloudSat and CALIPSO data for the period January–July 2008 are used. The CloudSat level 2B Cloud Geometrical Profile (2B-GEOPROF; Marchand et al. 2008; CSU 2013, 2014, 2015), available at approximately 240-m vertical and 1.1-km horizontal resolution provides a 0–40 range for the cloud-mask value. Each profile is interpolated to a regular 250-m grid, and then cloud is assumed present for values exceeding 20 (Stein et al. 2011; Tompkins and Adebiyi 2012). This is supplemented by CALIPSO information using the 2B-GEOPROF-LIDAR product (Mace 2007; Mace et al. 2009), assuming a cloudy pixel if the lidar-identified cloud fraction exceeds 99% (Barker 2008; Di Giuseppe and Tompkins 2015). For brevity, we will refer to the final retrieved product as CloudSat data. For an example of the CloudSat scene and the processed masks that results, refer to Fig. 1 of Di Giuseppe and Tompkins (2015).

The CloudSat overpasses are divided into segments of varying scene lengths for the analysis. For any two cloud layers, the overlap can be calculated using the definition of α given in Eq. (1). Nonadjacent layers are considered to belong to a continuous cloud block if all layers found between are also classified as cloudy. The existence of any clear (cloud free) layers between the two layers in question will instead classify them as discontinuous.

For continuous cloud, the overlap is maximum (α = 1) for a separation of zero, since this is the trivial case of the overlap of a cloud with itself. The value of α will reduce with separation distance, and the decorrelation length scale is calculated using a least squares best-fit exponential decay using all data points that are contributed to by at least 100 cloud scenes to prevent the fit being biased by small samples. Some previous studies have only used contiguous layers for the calculation of (e.g., Shonk et al. 2010), but this can result in uncertainties if the layer separation distance is much smaller than the decorrelation length scale. This is the case here, where the vertical resolution of 240 m is approximately an order of magnitude smaller than the decorrelation length scale expected from previous studies. It is emphasized that the sampling is binned as a function of layer separation only; any two layers separated by 2 km are treated equivalently, whether located in the upper, mid-, or lower troposphere, for example.

3. A heuristic argument for scene length dependency

The studies of HI00, Mace and Benson-Troth (2002), Naud et al. (2008), and Oreopoulos and Norris (2011) showed a considerable sensitivity to the scene length chosen for the analysis, with decorrelation length scale increasing with scene length, which has been confirmed in the introduction. As stated in the literature review, while Mace and Benson-Troth (2002) and HI00 offered simple arguments to explain this based on a single cloud element, Mace and Benson-Troth (2002) also admitted that these arguments would likely not hold in real scenes where multiple cloud elements would need to be considered.

Here, the arguments of Mace and Benson-Troth (2002) and HI00, which only considered idealized cloud systems that were fully contained within a scene, will be generalized to include the case when the sampling scale employed is below the typical scale of cloud systems. As an example, HI00 state categorically that the degree of overlap must decrease monotonically with decreasing scene length. Consideration of the schematic in Fig. 4 shows that the assertion of HI00 does not necessarily hold once it falls below the cloud system scale, because of the issue of data truncation briefly discussed in Astin and Di Girolamo (2014). In the schematic, a simple cloud system of overlapping clouds close to maximum overlap is sampled at four sampling scales ().

• Download Figure
• Download figure as PowerPoint slide

Schematic of the overlap of two adjacent clouds (gray), which are sampled at four sampling scales (denoted A, B, C, and D). The vertical dotted lines demark the scene boundaries, and the lengths of the scenes are indicated with the double-headed arrows below the schematic for clarity.

Citation: Journal of the Atmospheric Sciences 72, 8; 10.1175/JAS-D-14-0278.1

At sampling scale , the whole cloud system is contained within a single scene. This is still the situation in example , even though the sampling scale is reduced. Comparing case A and B, the argument of HI00 is correct; the cloud cover increases as the horizontal resolution increases and the degree of overlap decreases. However, in cases C and D, the whole system is divided into two and three scenes, respectively. In case C, the division of the cloud system into two scenes results in maximal overlap in each. The situation is similar in case D, where the first and third segments are maximally overlapped, while the central segment is neglected as both layers are overcast. If either of the two layers under consideration is overcast, α is undefined, as all overlap assumptions give the same total cloud-cover diagnosis of unity, and such scenes are neglected from the analysis. This is an example of the obvious effect of reducing the sampling scale; namely, a greater proportion of the cloud layers will be overcast and rejected from the analysis [e.g., see the cloud fraction PDF derived from aircraft data over different sampling scales in Tompkins (2003)]. Following Astin and Di Girolamo (2014), we refer to this as data truncation. Thus, in this simple example, it is seen that the average value of α that would be diagnosed for this transect decreases toward random, while the cloud system is completely contained within a scene by the sampling scale chosen (A and B) but then increases to maximum overlap once the cloud system is divided into a number of separate scenes by using shorter sampling scales (C and D). In this latter case, α has a positive bias when is less than X.

It is also possible to consider a two-layer cloud, such as depicted in the example of Fig. 5, which is close to minimum overlap. For such cloud systems, in an arrangement where the extent of the cloud in each layer exceeds that of the overlapping portion and thus α is negative, it is clear that as reduces, only the scene that contains the overlapping portion will be considered. All other adjacent scenes contain cloud in only one layer and are thus ignored. Thus, the overlapping portion of cloud sampled at small length scales is minimally overlapped, leading to negative biases in this case.

• Download Figure
• Download figure as PowerPoint slide

Schematic of the overlap of two adjacent clouds that are substantially minimally overlapped, showing that as decreases below X, only the scene containing the overlapping portion will contribute to the derivation of α.

Citation: Journal of the Atmospheric Sciences 72, 8; 10.1175/JAS-D-14-0278.1

Combining the above two cases, it appears that when α is negative, its sampling bias will be negative in the case of data truncation at small sampling scales, while positive α scenes will be subject to positive biases. To determine which, if any, of the two biases may dominate, we consider the limit of α as the sampling scale for a minimally overlapped two-layer system, such as the one depicted in Fig. 5. As the sampling scale reduces, the cloud fraction in one layer will approach unity faster than the other.¹ If the cloud cover in the respective layers are and , without loss of generality it is assumed that so that as reduces and . It is further assumed that is sufficiently small such that the combined total cloud cover of the two layers is 1, and the overlap parameter derived for can thus be written as follows:

If the two cloudy layers are arranged in a configuration of minimum overlap, as , and thus . Thus, α is unbounded for negative numbers but has an upper bound of 1. For minimal overlap cloud configurations with a finite overlap between the clouds, if as , then .

As a result of the asymmetry in the α bias, one can therefore expect that if a set of cloud systems that are randomly overlapped are sampled at length scales shorter than the typical cloud system scale, the negative bias resulting from scenes that contain cloud segments that are minimally overlapped could dominate the positive bias occurring in sections that are close to being maximally overlapped, assuming each arrangement occurs equally frequently. In these cases, the mean value α will be negative as observed for discontinuous cloud layers reported in previous studies. Moreover, for continuous cloud layers, the overlap at zero separation distance is unity by definition, but, at larger separation distances, the α value diagnosed will be more negative at shorter sampling scales. This would result in shorter being diagnosed, again in agreement with previous observations and the analysis presented in the introduction.

4. Evaluation with cyclic clouds

We test the hypothesis outlined in the previous section using the controlled framework of two-layer cyclic clouds, which are sampled at a range of sampling scales, as illustrated in Fig. 2. The results of this simple idealized case are shown in Fig. 6. By definition, if is equal to X, and the sampling resolution is unity, the bias in α is identically zero. Figure 6a shows that as decreases, the bias of α can be of either sign, depending on the value of α itself. If α lies between maximal () and random () or is even slightly minimally overlapped, then its bias is positive. On the other hand, when the cloud system tends toward minimal overlap, the bias becomes negative. However, it is also seen that the bias is highly nonlinear. The magnitude of the negative bias for scenes close to minimal overlap far exceeds the maximum positive bias found, confirming the arguments provided in the previous section.

• Download Figure
• Download figure as PowerPoint slide

Results of the idealized cloud sampling test illustrated in Fig. 2. (a) Bias in α as a function of the true overlap for individual cloud combinations and sampling length scale. (b) Bias in α averaged over all cloud combinations that give .

Citation: Journal of the Atmospheric Sciences 72, 8; 10.1175/JAS-D-14-0278.1

If we now consider a set of cloud systems that, on average, are randomly overlapped, what would be the resulting sign of the bias at small sampling scales? Figure 6a would appear to show that randomly overlapped clouds would be subject to a positive bias in α when the true α is zero. However, it is recalled that the overlap for a large set of observations is an average statistical property. For example, if we were to assume that discontinuous clouds are randomly overlapped, this does not imply that the total cover of any two cloud layers separated by a clear layer will be exactly random. Instead, the overlap can take any value from maximum to minimum overlap, and the random overlap results from the averaging of many scenes. Indeed, for this idealized case of two layers of 50% cloud cover, all overlap possibilities from minimum through random to maximum would be equally likely.

Thus, it is more appropriate to examine the mean overlap bias for all overlap cases that lead to the mean , and we assume random overlap ( = 0) to be the mean for all overlap cases ranging from maximum overlap (α = 1) to minimum (α = −1), while = 0.5 is the average for individual overlap cases ranging from maximum overlap (α = 1) to random (α = 0), for example. The averaged α bias calculation (Fig. 6b) appears to support the heuristic argument above, and previous observational investigations of overlap can potentially be explained. Although the α bias is positive for a wider range of α values when is less than unity (e.g., Fig. 6a shows that for , the bias is positive for α values exceeding approximately ), the larger magnitude of the negative bias in minimally overlapped clouds can dominate. This results in negative biases for , depending on the sampling resolution. Thus, clouds that are randomly overlapped would be diagnosed with negative α values between 0 and −0.2 when using sampling resolutions between 0.5 and 1.0, based on this simple cyclic cloud system study. This range of negative α values is consistent with the values previously reported for discontinuous clouds, such as in the work of Naud et al. (2008). Here, the hypothesis is that these scenes are, in fact, randomly overlapped, and the negative overlap parameter is a result of the scene lengths used being less than the typical cloud system scale. In addition, the decorrelation length scale should reduce with higher resolution and/or shorter averaging periods, as observed previously.

5. Evaluation with idealized fractal clouds

The results in the previous section were informative but relied on a highly idealized arrangement of clouds that were cyclic, with a cloud cover of exactly 50% in each layer. In this section, we therefore repeat the idealized experiment, but using the fractal cloud scenes generated by SITCOM. In this way, individual scenes may include numerous cloud elements of various sizes with a realistic power spectrum that is known and controllable.

With a fractal cloud scene, sampling at any specific will lead to cloud scenes containing clouds in various configurations ranging from maximum to minimum overlap. From our analysis of cyclic clouds, we expect some scenes to be subject to positive α bias and some to negative α bias. The prediction is that, overall, the negative biases will dominate. Taking one sampling resolution as an example with , Fig. 7 gives the normalized probability density function of the individual biases in α. The graph shows that individual scenes are indeed subject to both positive and negative biases, both of which can exceed a magnitude of unity. A positive bias greater than unity can occur if a particular field has an overlap between random and minimum, but α is close to maximum overlap with this sampling scale. Negative biases occur more frequently, and the magnitude of the negative biases is greater, as expected because they are unbounded. The result is that the mean of the PDF is negative, as expected for this scene resolution.

• Download Figure
• Download figure as PowerPoint slide

Normalized probability density function of the bias in α for over 10⁶ separate cloud scenes of length for the resolution generated by the fractal cloud model SITCOM.

Citation: Journal of the Atmospheric Sciences 72, 8; 10.1175/JAS-D-14-0278.1

Repeating the calculation for all scene resolutions between 0.2 and 2 with the fractal clouds indicates that the mean bias in α due to scene truncation is, in fact, always negative (Fig. 8). This is a consequence of the cloud cover of 50% used for the fractal field. Using scenes of higher mean cloud fraction can result in positive α biases occurring for high α values (not shown). If the sampling scale exceeds the cloud system scale, the bias is small (less than 0.02 for and = 2), but is nonzero, since the fractal nature of the clouds implies some data truncation can still occur in these cases. At smaller , the α bias increases in magnitude, exceeding −0.2 for sampling lengths that are less than a third of the cloud system scale. The overall magnitude of the estimated bias appears to agree quite well with those diagnosed from previous observational studies. For example, Naud et al. (2008) found that discontinuous clouds had overlap parameters that varied between −0.1 and −0.2 depending on the ground-based site location. Assuming these clouds were actually randomly overlapped, this suggests that their sampling scale approximately ranged from a factor of 0.3 to a factor of 0.8 of the cloud system scale, assuming that the mean cloud fraction at the location is not too dissimilar to 50%.

• Download Figure
• Download figure as PowerPoint slide

Bias in α derived from repeatedly sampling 200 individual fractal cloud transects of 2000-km length generated by SITCOM at ranging from 0.2X to 2X, where X is the cloud system scale. The α bias is binned as a function of the true α obtained at a much larger scale of 20X.

Citation: Journal of the Atmospheric Sciences 72, 8; 10.1175/JAS-D-14-0278.1

6. A simple filter for scale-invariant overlap statistics

As a result of the above considerations, we suggest a minor modification to the analysis method previously employed to subsample cloud scenes. As stated earlier, when considering the total cloud cover resulting from the vertical overlap of two cloudy layers, if either of the two layers is overcast, then the total cloud cover is also trivially unity, and α is undefined. As a result, HI00, Naud et al. (2008), and Willén et al. (2005) only considered cloud layers in the analysis for which the layer cloud cover , effectively setting an upper threshold of cloud cover, which we will denote . However, it is possible to implement a lower cap to the cloud cover. The above considerations suggest that, as the value of decreases, the value of α should increase, since we are retaining cloud systems that are more likely to be well resolved. Thus, data will still be truncated with this filter; indeed, truncation will be stronger at shorter sampling scales than using the standard method, but the sensitivity of the results to the truncation should be reduced.

We test this new filter by analyzing the CloudSat data using sampling scales of X′ = 10, 30, 100, 300 and 1000 km and applying values of ranging from 1 (the previous default) to 0.5. The results for X′ = 100 km are shown in Fig. 9. For discontinuous clouds, it is seen that α increases as decreases, as predicted. For , discontinuous clouds tend to be randomly overlapped, but α is always positive. The mean value of α for all X′ and (Table 2) shows that using the filter, the sampling-scale sensitivity of α for discontinuous cloud is much reduced, with values ranging from 0.05 to 0.08 with , similar to that obtained with with a sampling scale of 1000 km, which reduces truncation to a minimum.

• Download Figure
• Download figure as PowerPoint slide

Overlap parameter for continuous and discontinuous cloud scenes derived from CloudSat–CALIPSO data using X′ = 100 km and a filter value ranging from 0.5 to 1.0 (see legend).

Citation: Journal of the Atmospheric Sciences 72, 8; 10.1175/JAS-D-14-0278.1

Table 2.

Mean α for discontinuous clouds observed by CloudSat as a function of and . The standard analysis of HI00 and others is to use .

For continuous scenes, α also increases as decreases; as a result, the decorrelation length scale is larger. The value of is seen to be much less sensitive to sampling scale with lower values of (Table 3). At , the decorrelation length scale is less sensitive to sampling scale, changing from 3.4 to 4.0 km over sampling scales spanning 10–1000 km.

Table 3.

Decorrelation length scale of α for continuous clouds as a function of and . The standard analysis of HI00 and others is to use .

7. Discussion

Are cloud systems likely to be underresolved by the spatial sampling scales of 50–150 km typically used in previous analyses? This will depend on the season and location. Isolated convective events may be completely contained within a single scene, even with sampling scales of 50 km, but if the cloud events are predominantly caused by frontal systems, then this is likely not to be the case. Indeed, it was explicitly stated by Willén et al. (2005) that many frontal systems were larger than the sampling scale and that only the system boundaries contribute to the calculation of overlap. Referring to the example scene given in Fig. 2 of HI00, it is clear that the cloud system scale significantly exceeds the middle sampling time scale of 1 h illustrated, which translated to a spatial sampling scale of 72 km using the assumed horizontal advection speed. In their Fig. 2, HI00 draw a box to highlight an example subscene in which a single cloud element is fully contained for the purposes of their discussion. Examining the radar retrieval of the full figure, it is clear that this example is an exception, and the vast majority of the layer combinations that would contribute to the statistics consist of cloud edges.

In the analysis of Wood and Field (2011), the mean length scale at which clouds contributed to half of the cloud cover was 200 km. This significantly exceeds the sampling scales used for overlap statistics and would result in significant α biases, although the analysis examined total cloud cover from MODIS, and the cloud lengths in individual layers of the atmosphere could be shorter. Zhang et al. (2014) found altocumulus layers, for example, were, on average, approximately 40 km, while Kiemle et al. (2015) has recently made an analysis of CALIPSO data in 2007 and shows similar cloud lengths to Wood and Field (2011).

The study of Wood and Field (2011) raises another interesting issue, as they find a strong latitudinal dependence of the typical cloud length diagnosed (see their Fig. 8), with cloud length scales increasing by more than an order of magnitude between the equator and midlatitudes for many longitudes. This implies that, for a given fixed , will decrease strongly between the equator and the poles, which would lead to shorter decorrelation length scales being diagnosed for continuous clouds at higher latitudes. This agrees with the analysis of Shonk et al. (2010) and Oreopoulos et al. (2012), who showed shorter of the α parameter for continuous clouds at higher latitudes. While this latitudinal dependency may have physical origins, because of wind shear or differing ice crystal properties, for example, the analysis indicates that it could also be an artifact of sampling diverse system sizes with a fixed sampling length scale. We test this by sampling the CloudSat data as a function of latitude with and using the strongest filter of . With the standard analysis, we reproduce the strong latitude dependency seen in Oreopoulos et al. (2012), but, applying the filter, the latitudinal dependency of is reduced by approximately half. Note that the filter actually reduces in the tropics, a consequence of the frequently occurring maximal overlap of tropical deep convection there.

Considering that sampling at scales smaller than the cloud system scale affects the diagnosed decorrelation length scale applied to continuous clouds,² which value should be applied in a radiation scheme that uses an EXP-RAN approach if the host atmospheric model employs horizontal resolutions that are smaller than typical cloud system scales? One could argue that the value obtained sampling at a scale equal to the model grid resolution should be employed, but the above analysis emphasizes that this oversimplifies the situation, since the typical cloud system scale should also be taken into account. This could be done using long-term statistics of cloud system sizes as a function of location and month or possibly interactively using information about the dynamical situation. The analysis of the present manuscript offers a first step, explaining the reason for the sampling-scale sensitivity and diagnosing the underlying decorrelation length scales.

8. Conclusions

This paper has returned to the issue of cloud overlap, as defined by the parameter α that describes the proportion of maximum and random overlap observed in a cloud scene. We have attempted to offer a simple argument to explain the commonly observed characteristics of the cloud overlap parameter derived when dividing a cloud observation source (from satellite, aircraft, or model, for example) into a set of cloud scenes of a given length scale, referred to as the cloud sampling scale . Previous studies have found that the overlap parameter of discontinuous clouds is often negative, suggesting overlap that is not exactly random but slightly minimal, and that the decorrelation length scale of α for vertically continuous cloud layers increases with sampling scale.

A simple heuristic argument was offered to explain these characteristics as a result of data truncation: that is, the effect of necessarily discarding cloud scenes that are overcast or cloud-free when compiling cloud statistics, as α is undefined in these cases. If falls below the cloud system scale X, increasing numbers of cloud scenes will be discarded, and the overlap statistics will be based on the remaining cloud fragments that will be either maximally or minimally overlapped. This results in an increased magnitude of α when diagnosed with small sampling scales: that is, negative biases in α when α is negative, and positive biases in α when α is positive. As a direct consequence, the decorrelation length scale for continuous clouds would decrease with sampling scale.

Using a simple cyclic cloud system, we confirm the basic predictions of the truncation theory: namely, that the α bias is positive for a scene that is between random and maximum overlap and negative for a scene that is considerably minimally overlapped. Because of the definition of α, the negative bias can be considerably larger in magnitude; thus, averaging over many scenes of varying configurations, the theory predicts that clouds that are randomly overlapped will be diagnosed as slightly minimally overlapped (i.e., with negative α). We subsequently attempted to overcome the drawbacks of the idealized cyclic cloud framework by repeating the analysis with realistic fractal clouds with a power spectrum and a smooth long-wavelength cutoff to give a specified cloud system scale. With this realistic system, the α bias is always negative but increases in magnitude as the sampling scale decreases. Thus, the prediction of the negative α values for randomly overlapped clouds is maintained for these realistic scenes. Assuming discontinuous clouds are randomly overlapped, the fractal cloud model assesses that the cloud systems analyzed by Naud et al. (2008) were sampled at scene lengths that ranged from 0.3 to 0.8 of the cloud system scale.

Based on the arguments that cloud truncation leads to an underestimation of cloud-scale decorrelation length, we suggested a simple filter for cloud scenes, where only cloud scenes with fractions smaller than a given threshold (lower than 100%) are retained in the analysis. With small thresholds applied to several months of CloudSat data, discontinuous clouds are diagnosed with small but positive α values, as expected. Thus, the minimal overlap is shown to be an artifact of data truncation. For continuous clouds, the smaller the threshold used in the filter, the lower the sensitivity to the sampling scale becomes, and using an upper limit of 50% results in an overlap decorrelation length scale that is close to being invariant to sampling scale. Using this filter with the CloudSat–CALIPSO data, the decorrelation length scale at 100 km is 3.6 km, similar to that obtained at a scale length of 1000 km, whereas a value of 3.0 km was found when all clouds were retained in the analysis.

In a discussion, it was emphasized that the cloud system scale is often larger than the sampling scale, and indeed the typical resolution employed in many global and regional atmospheric models; thus, the data truncation issue is relevant. Indeed, it is demonstrated that the strong latitudinal dependence of the overlap decorrelation length scale demonstrated in previous studies is partly a result of the data truncation and indicates that the value used in model parameterization schemes is not only a function of the model resolution but also of the cloud system type, which may or may not be well resolved at the resolution employed.