1. Introduction

The difference between the albedo calculated assuming horizontally homogeneous cloud properties and that obtained by averaging independent pixel calculations, termed the plane parallel albedo bias (PPH bias), has been shown in a companion paper by Oreopoulos and Davies (1998) to be very significant when evaluated from representative satellite data and applied to regions similar in size to the grid scale of general circulation models (GCMs). This confirms similar results obtained by Cahalan et al. (1994) and Barker et al. (1996), who used more limited datasets, and indicates a need to more accurately account for the radiative effects of the subgrid-scale variability of cloud liquid water. The radiation schemes of existing GCMs (Barker 1996) typically underestimate cloud water content in order to obtain realistic radiation budgets, so that a more detailed treatment of subgrid-scale variability should also yield better consistency between the radiative and hydrologic treatment of cloud water content.

The magnitude of the observed PPH bias and the implications of this for GCM parameterization motivate the search for its reduction. This paper explores two avenues that lead to a reduced bias, especially for large-scale model applications. The first is the “effective thickness approximation” (ETA) of Cahalan et al. (1994) (hereafter CRWBS). The second is the “approximate IP” method, wherein optical depth frequency distributions are fitted with analytic functions, as was done recently by Barker et al. (1996, hereafter BWP), who used only one such function and tested it on a limited Landsat dataset. Here, we compare and extend the application of the ETA and approximate independent pixel (IP) techniques, using again the large Advanced Very High Resolution Radiometer (AVHRR) dataset of Oreopoulos and Davies (1998, hereafter OD98).

2. Dataset and methodology

The dataset used in this study is the same as that used by OD98. It consists of AVHRR local area coverage (LAC) observations (1.1-km resolution at nadir) from the NOAA-11 polar orbiter, which cover the geographical region of the Atlantic bounded by the 9° and 45°N latitude circles and 19° and 58°W meridians. More details about this dataset can be found in OD98 and in Oreopoulos (1996). The AVHRR scan lines (2048 pixels) were divided to 300-pixel-long segments viewed by the radiometer at near-nadir, medium, and oblique angles (both in the forward- and backscattering directions) as explained in OD98. The names of the various segments of the scanline are“nadir”, “fsmv” (forward scattering, medium views),“fsov” (forward scattering, oblique views), “bsmv” (backward scattering, medium views), and “bsov” (backward scattering, oblique views). The total number of pixels analyzed was 1.5 × 10⁸, but most of the results that will be shown here are from the nadir segment (∼3 × 10⁷ pixels). Solar zenith angles for the nadir dataset are between 53° and 78° (a frequency distribution of solar zenith angles for the complete dataset is given in OD98). For illustration purposes, we also show some results for the six Landsat scenes used in OD98.

Detection of cloudy pixels, retrievals of optical depth (τ) distributions for both AVHRR and Landsat, and the limitations of our method are described in OD98. The cloud optical depths and albedos for AVHRR correspond to 0.63 μm and those for Landsat to 0.83 μm. The albedo bias B of a region is defined as the difference between the albedo of the approximate method (PPH, ETA, “approximate IP”) and the IP albedo, for the cloudy pixels of the region

BR_apR_ip(1)

where

View Expanded

where N is the number of cloudy pixels in the region, Rⁱ_pp is the plane parallel albedo of the ith pixel, R(τ, θ₀) is the planar albedo, θ₀ is the solar zenith angle, and p(τ) is the normalized probability density function (PDF) of optical depth. Equation (2) implies that regional albedo can be calculated by averaging plane parallel calculations over individual pixels (columns). That is, in the independent pixel approximation (IPA) pixels are assumed to be radiatively isolated from each other. When R_ap = R_pph we obtain the standard definition of albedo bias as in CRWBS and BWP. The PPH albedo is calculated from

R_pphRτθ₀(3)

where τ is the mean optical depth of the region

View Expanded

The albedo definitions for ETA and approximate IP are given later. Most of the results that are shown in the following sections are albedo biases that are averaged over many regions of a certain size. These are either straight averages given by

View Expanded

or weighted averages given by

View Expanded

where M is the number of regions with nonzero cloud fraction A_c. Sometimes the absolute bias |B_j| is inserted in Eq. (5).

For AVHRR there are two sets of albedo biases: those calculated from optical depths retrieved without accounting for atmospheric effects (“no atm”) and those calculated from optical depths retrieved with the atmospheric effects of a standard LOWTRAN 7 maritime atmosphere included (“atm”). Details are given in OD98; we should point out here, however, that all albedo biases shown hereafter refer to the cloud top and not the top of the atmosphere (TOA). No atmospheric effects were considered in the Landsat retrievals (as in Harshvardhan et al. 1994).

3. Effective thickness approximation

Figure 1 shows the albedo of the C.1 cloud model (Deirmendjian 1969) used in OD98 as a function of logτ for solar zenith angles of 0° and 60°. The range of τ for which the logarithmic curvature is close to zero depends on solar zenith angle. Barker (1996) suggested that the second derivative approaches zero when 5 ⩽ χτ/μ₀ ⩽ 15, where μ₀ is the cosine of the solar zenith angle. However, since it is the product of curvature and variance that is important, the effectiveness of the ETA can vary even among optical depth distributions with the same τ. Ultimately, of course, the desired accuracy in albedo is the main factor that determines the suitability of the ETA for the specific problem at hand.

Here, a new reduction factor, χ₀, is defined as a measure of cloud inhomogeneity:

View Expanded

This is the ratio of the optical depth that gives the IP albedo over the mean cloud optical depth. By definition, the albedo calculated at χ^τ₀ is the IP albedo. In general, the smaller χ₀, the greater the cloud inhomogeneity. This reduction factor is expected to be close to that of CRWBS when conditions permit the omission of the second derivative and higher-order terms in Eq. (6) (low logarithmic variance and/or near-zero curvature). Note that, in principle, both reduction factors can be computed only when the complete τ distribution is known, in which case the PPH bias would also be known. Thus, the reduction factors would be useful for removing the PPH bias only if they could be obtained with sufficient accuracy from more limited information through, for example, a parameterization (this is examined in section 5). However, at least for χ, such a parameterization would be helpful for albedo corrections only if ETA conditions occur frequently in observed cloud distributions. This can be examined by comparing χ with χ₀.

The average χ₀, χ, and M₂ within variable width bins of τ are plotted in Fig. 2 for (55 km)² (50 × 50 pixels) areas of the nadir no atm dataset. Note that the variance increases monotonically with optical depth and that, in agreement with theory, the two reduction factors are close for low variance and for optical depths associated with the low logarithmic curvature region of the albedo curve [R" ≈ 0 in Eq. (6)]. The observations show that these two requirements often work together—that is, the bins of intermediate optical depth also exhibit small variance. For high values of optical depth the cloud albedo becomes insensitive to optical depth and large reductions in mean optical depth (small reduction factors) are needed to match the IP albedo. The reduction factors reported here are significantly smaller than the ones reported by CRWBS, which were based exclusively on observations of single-layer marine stratocumulus clouds. CRWBS recognized, however, that smaller reduction factors are expected for most cloud types other than stratocumulus. Since the present dataset is temporally and spatially extensive, it contains a wide variety of cloud types, structures, and microphysics, so smaller reduction factors come as no surprise.

When the reduction factors of regions that satisfy the“Barker criterion” 5 ⩽ χτ/μ₀ ⩽ 15 are compared, the agreement is very good (Fig. 3). The two reduction factors keep a constant average difference of about 0.01 and correlate well (r ≈ 0.99) for all region sizes. Thus, the Barker criterion appears to be a robust criterion for applying the ETA.

The extensive agreement between the reduction factors in Fig. 2 is an indication that ETA should perform well on average for the AVHRR dataset. Indeed, the B̂ results for the nadir segment confirm this (Fig. 4). Here, B̂ for ETA maintain values smaller than 0.01 for all region sizes of both the atm and no atm datasets, and are thus much lower than the corresponding PPH biases (also plotted in Fig. 4). The absolute ETA biases (dotted curves) suggest that canceling errors somewhat enhance the performance of the ETA, but the standard deviations are only ∼0.02–0.04 (not shown), implying that the ETA albedo rarely deviates from the IP albedo by a large amount. The proximity of the averages calculated from Eq. (5a) to the weighted averages calculated from Eq. (5b) (not shown) suggests that the performance of ETA does not depend on the regional cloud amount.

The low ETA biases for the current AVHRR dataset are explained by the low logarithmic variance for the optical depths associated with nonzero logarithmic curvature—that is, the small optical depths. Moreover, the ETA apparently performs adequately for the large optical depths too, because cloud albedo is not very sensitive to τ variability (especially at the large solar zenith angles of the dataset) and because the scaled optical depths χτ frequently assume moderate values (due to the low values of χ). The lower quality of ETA for the atm case is due to the higher logarithmic variances associated with this dataset (see OD98). ETA also succeeds when applied to off-nadir data as evidenced by Fig. 5. Errors are in general less than 0.01—much smaller than the corresponding PPH biases (shown in OD98).

The above results are in disagreement with the results of BWP, who found ETA biases larger than PPH biases for broken stratocumulus and scattered cumulus. BWP, however, did not use CRWBS’s exact definition χ = 10^{logτ−logτ} of the reduction factor, but a modified definition appropriate for gamma distributions fitted to the observed PDFs. Part of the disagreement may have therefore been due to deviations of the observed PDFs from gamma PDFs. Still, the main reason for the poor ETA results in the BWP study is the apparent presence of high variances at low optical depths when this type of boundary layer cloud is observed with a very high resolution sensor. Indeed, we also found very large discrepancies between χ₀ and χ for the four Landsat scenes of OD98 containing broken clouds. On the other hand, there was excellent agreement for the two overcast stratocumulus scenes, which were very homogeneous (note their small PPH biases in OD98) with reduction factors above 0.9.

4. The “approximate IP” method

b. Application to observations

The three approximate IP albedos are calculated from the integral (2) by inserting one of the following distributions in place of p(τ):

1. the γ-distribution,

   

   

   View Expanded

   

   where α = τ/γ and γ = (τ/σ)², σ is the standard deviation of the observed PDF, and Γ is the gamma function;
2. the β-distribution,

   

   

   View Expanded

   

   where η = [(1 − x)/σ²_x][x(1 − x) − σ²_x], ξ = xη/(1 − x), and x, σ²_x are the mean and variance, respectively, of x = (τ − τ_min)/(τ_max − τ_min); and
3. the ln-distribution,

   

   

   View Expanded

   

   where μ and s are the mean and standard deviation, respectively, of lnτ.

Thus, the theoretical distributions and their corresponding approximate IP albedos can be calculated from the method of moments with knowledge of only the regional mean and variance of τ or its logarithm. Figure 8b shows typical shapes of these functions obtained from the moments of AVHRR optical depth PDFs discussed in detail later.

Figure 6 shows the PPH and approximate IP albedo biases for the clouds of the six (58 km)² Landsat scenes used in OD98. Note that the validity of the IPA for such a high-resolution (28.5 m) dataset (especially for optical depth retrievals) is questionable, as discussed in Oreopoulos (1996). The reason is that such pixels are usually optically narrower than they are deep, and the dominance of horizontal over vertical radiative transfer violates the assumption of isolated pixels. However, Chambers et al. (1997) argue that despite the significant errors for individual pixel retrievals, PDFs are not much affected. While this is a topic of ongoing research, the IPA-retrieved PDFs nevertheless appear to be well described by the theoretical distributions of Eq. (8): all approximate IP biases are much smaller than the PPH biases, with the exception of the “β IP” bias for scene 40. The good performance of the “γ IP” is in agreement with the findings of BWP.

Figure 7 shows the average biases B̂ of the three approximate IPs for the no atm dataset. The corresponding PPH bias is also shown for comparison. Both the γ IP and β IP do not perform well and underestimate the IP cloud albedo by about the same amount (which is somewhat larger than the overestimate by PPH). On the other hand, the “ln IP” approximates very well the true IP albedo. Cancellation of albedo overestimates and underestimates plays only a minor role in the success of the ln IP. This is evidenced by the average of the absolute lognormal biases, which is still ⩽0.01 for all region sizes (not shown). However, the standard deviations range from 0.02 to 0.04 (they decrease with region size) and are larger than those of ETA (especially for small region sizes).

For the atm case (not shown), the ln IP underestimated R_ip by less than 0.01 for all region sizes, but the other two theoretical distributions produced average biases very close (in absolute values) to PPH. The absolute ln IP biases were always less than 0.02 and the standard deviations less than 0.06. Very similar qualitative behavior was found for the off-nadir data: the γ IP and β IP gave biases comparable to PPH and the ln IP albedos were in excellent agreement with R_ip. Whether the analysis is carried out on an equal pixel number or equal area basis (explained in OD98) does not affect these conclusions.

The poor performance of the β and γ distributions was not anticipated, in view of their success in the Landsat scenes (Fig. 6) and several GAC scenes we examined in a preliminary pilot study. The γ IP in particular gave very good results for 45 Landsat scenes of BWP. One of the reasons the γ and β distributions fail for the current dataset is that a large number of regions contain extreme (high) optical depths, which despite representing only a small fraction of pixels, rapidly raise the optical depth variance. The high optical depths distort the shape of the theoretical distributions when the method of moments is used for parameter estimation.

An illustration of this effect is given in Fig. 8a, which shows the observed and approximate distributions (from the method of moments) for a 150 × 150 pixel array of the nadir no atm dataset. The shape of the β and γ distributions does not resemble the shape of the observed (and lognormal) distribution since the method of moments gives ξ < 1 and γ < 1 resulting in concentration of probability near zero for both distributions (Wilks 1995) and a large error in albedo (see caption). However, when the ∼0.4% of the pixels with optical depths greater than 100 are excluded from the calculations, the γ and β distributions acquire “nonzero” modes (ξ > 1 and γ > 1), as a result of the standard deviation dropping below τ, and agree more with the observed distribution (Fig. 8b). This yields γ IP and β IP albedos much closer to the true IP albedo (see caption). Both the true IP and ln IP albedos are relatively unaffected by the truncation of the optical depth distribution to values less than 100. While the neglected pixels contribute significantly to the mean optical depth, their contribution to the mean albedo is not as large since the albedo is already near saturation for τ ≈ 100. This explains the small change in true IP. The small change in ln IP albedo after truncation is due to the fact that the mean and variance of lnτ remain close to their original values.

When albedo bias calculations are repeated with all τ > 100 pixels neglected, the γ IP and β IP give much smaller biases (Fig. 9). However, the PPH biases also drop significantly from their initial values (Fig. 4). This is consistent with the conclusion of OD98, who noted that a large fraction of the PPH bias was due to extremely high optical depths inferred from a plane parallel radiative transfer model under conditions of low solar illumination. These large optical depths (representing ∼2% of the pixels for the nadir segment) significantly increase the apparent cloud field variability and render the β and γ distributions incapable of capturing the shape of the observed distributions. However, it will be shown later that, at least for the gamma distribution, this problem can be largely eliminated by avoiding the method of moments for parameter estimation.

An additional reason that makes the failure of the β and γ IPs look so conspicuous in Fig. 7 is the occasional occurrence of extreme deviations from the true IP. An alternative way of evaluating the quality of the approximate IPs is to calculate their “success ratios,” defined, for example, as the fraction of regions (in %) for which they give biases lower than PPH, or the fraction of regions for which they approach the exact IP (from above or below) within 0.01. These results are shown in Table 1.

The γ IP and β IP biases are lower than the PPH biases for a significant number of regions, but the quality of their performance drops as area size increases because of the associated increase in the likelihood of including extreme optical depths. There is no such systematic trend for the ln IP. However, even for the cases where the γ IP and β IP are better than PPH, their average biases are still quite large (not shown). For regions larger than (55 km)², these two approximations approach the IP to within 0.01 only for the few cases where the PPH bias is also small—that is, for relatively homogeneous regions. This is shown in Fig. 10 for the γ IP. When only the areas with χ₀ > 0.7 are kept, the γ IP and β IP work better than PPH at most times (see Table 1) but are still far less successful than the ln IP. Thus, the poor showing of the γ IP and β IP in Fig. 7 is only in part due to contributions from complete failures; it also reflects their frequent failure in the presence of anomalously high optical depths.

The performance of the γ IP can be improved without rejection of pixels with high optical depth by calculating the parameters of the γ distribution with the method of maximum likelihood estimates (MLE) instead of the method of moments. MLE seeks values of the distribution parameters that maximize the likelihood function (Wilks 1995). The procedure follows from the notion that the likelihood is a measure of the degree to which the data support particular values of the parameters. Thus, the maximum likelihood estimators are considered the most probable values of the parameters, given the observed data (Wilks 1995). Unfortunately, this method is impractical to apply to the β distribution (Falls 1974; Karner and Keevallik 1993; Wilks 1995), so the MLE is applied to the current dataset only for γ IP calculations.

MLE values of γ are obtained from (Wilks 1995; BWP):

View Expanded

Wilks provides the following approximate solution:

View Expanded

where y = lnτ − μ. Thus, with the MLE the γ distributions can be calculated with knowledge of only τ and μ (both calculated as before). Figure 11 shows the average biases of the γ IP when the parameters of the γ distribution are calculated from the MLE. There is a vast improvement over the γ IP from the method of moments (Fig. 7), since the relationship that gives the γ shape parameter involves only logarithmic quantities, which are much less sensitive to the presence of extreme optical depths. The γ IP still does not perform as well as the ln IP; for example, it gives substantially larger absolute biases and standard deviations (∼0.05–0.06 for no atm and ∼0.07–0.08 for atm). However, the γ IP could in some cases be chosen over the ln IP since it requires knowledge of only the mean logarithm of τ, while the latter also requires the variance. Comparable improvement in performance is also obtained for the other scan line segments (not shown).

5. A parameterization

As can been seen from Table 2, the parameterization works better for μ (r > 0.9) than s² (r > 0.78). The goodness of the fit was found to depend on region size (number of pixels) and to differ between the atm and no atm datasets (s² for atm produces worse fits than no atm). The approximate IP biases derived from the parameterized moments of lnτ are shown in Fig. 12. The curves plotted are the simple and weighted averages, determined from Eqs. (5a) and (5b), for both the atm and no atm datasets (lower set of curves). The corresponding absolute averages (upper set of curves) are also plotted. In these calculations, only the regions where the parameterization gave positive values of s² and γ, and χ values between 0 and 1 (more than 95% of the regions for all sizes) were considered.

The parameterization works quite well overall, giving biases much lower than the PPH assumption for all three methods. The performance in all three cases is comparable, with differences depending on whether the atm or no atm dataset was used, and on whether the averaging was weighted or nonweighted. The ETA does not appear very sensitive to the type of averaging and seems to perform slightly better than the other two. The ln IP and γ IP average biases do not depend significantly on the dataset used (for nonweighted averaging). The ln IP performs well considering that it requires the parameterization of both μ and s². In general, weighting the biases by cloud fraction erodes the performance (probably because for overcast regions the parameterization has only mean optical depth as the independent variable). The standard deviations (not shown) are slightly larger than the average absolute biases shown in Fig. 12; the high values of these two quantities indicate a significant cancellation of errors. Thus, the fitted versions of the three correction methods are successful only in an average sense and do not necessarily provide good local albedos at a given instant.

In conclusion, GCMs using the above type of parameterization could be able to remove a great fraction of the PPH albedo bias by using only mean optical depth and cloud fraction information. Of course, the parameterization introduced here should be further tested with more extensive satellite observations. Most likely, it can be improved by stratifying the data by solar zenith angle, cloud fraction, and optical depth. It should be underlined, however, that such an empirical parameterization of optical depth variability is only one of the possible approaches climate modelers may wish to consider. It is perhaps more physically sound for climate models to explicitly represent the processes that produce heterogeneous clouds. Results obtained by turbulence closure and large eddy simulation models can prove useful in this respect.

6. Conclusions

As discussed by Harshvardhan and Randall (1985), Barker (1996), CRWBS, and OD98, the inability of current GCMs to allow for subgrid water variability forces them to use cloud liquid water amounts lower than observed to counteract inflated TOA albedos due to the PPH assumption. This study used the same satellite observations as OD98 to investigate two methods that GCM modelers may find useful in their effort to achieve better TOA albedos with realistic cloud water amounts. The first simply applies a scaling factor to the mean cloud optical depth to yield albedos close to the IP values. It was shown that the effective thickness approximation of CRWBS provides a reduction factor (calculated from the mean logτ) that is quite successful. The second method assumes that cloud optical depth distributions can be fitted by theoretical functions when the first two moments of the optical depth PDF are known. The lognormal distribution was found to be very successful in this regard, while the gamma distribution gave good results only when its parameters were calculated with the MLE method.

The range of optical depths over which the effective thickness approximation is valid appears somewhat broader than originally expected. This is attributed to the small variance in logτ at low optical depths (where the approximation would otherwise break down), to the small values of χ at large optical depths, and possibly to some canceling of error when biases are averaged. ETA albedos were within 0.01 of the IP values at least 40% of the time for the no atm dataset, within 0.02 of IP at least 50% of the time for the atm dataset, and standard deviations ranged from ∼0.02 to ∼0.04. The exact numbers depend on region size. Due to the greater apparent variability of the atm optical depth fields (as shown in OD98), performance was better for the no atm dataset.

Of the three theoretical distributions used to fit the observed distributions of optical depth, the gamma and lognormal proved the most useful in approximating the IP albedo. The gamma distribution gave average biases ∼0.01–0.02 with standard deviations 0.05–0.08, while the lognormal had average biases of almost zero and smaller standard deviations (0.02–0.06). The beta distribution should be rejected for this task, even though it is the distribution with the widest variety of possible shapes. The reason is that parameter estimation with the method of moments can be very inaccurate in the presence of outlier optical depths, while maximum likelihood estimation requires a large computational effort. Despite the fact that the lognormal is clearly superior to the gamma distribution, implementation of the latter in a climate model may be more practical. The γ IP calculations would be computationally more efficient since the albedo can be expressed in a closed form for the generalized two-stream albedo function (as shown by Barker 1996). Also, only τ and μ need to be known (when the MLE is used) for the distribution to be defined; in contrast, the ln IP requires knowledge of both μ and s², and more time-consuming numerical integration.

Based on these results, we suggest that quantities such as μ, s², and χ₀ should be added to the list of observables for satellite sensors. Climatologies of the first two would allow estimates of χ, further comparisons among ETA, γ IP, ln IP, and exact IP albedos, and development of parameterizations in terms of known variables [as in Eq. (11)]. These comparisons should help us identify with greater confidence which method is more appropriate for the task at hand. The appropriate data resolution for building such a climatology is an issue open to debate. Too high a resolution (such as Landsat) while allowing for more accurate cloud detection pushes the limits of the IPA. On the other hand, for certain types of clouds such as fair weather cumulus, the AVHRR resolution may render the assumption of pixel homogeneity dubious and yield underestimates in horizontal cloud water variability (see OD98).

This study showed that the ln IP has the best overall performance, but the computational burden may restrict its usefulness to only satellite applications. For example, recovery of regional albedos would require storing only the first two logarithmic moments of τ, and not the entire PDF. For GCM applications the ETA and γ IP appear more attractive. The ETA would be extremely simple to apply in a climate model provided μ is available. However, further measurements of χ₀ and comparisons with χ (at appropriate solar zenith angles) are needed to identify how often and for which cloud regimes the ETA is appropriate.

There are several other considerations that affect the choice of correction method. While the primary consideration is that the average albedo be unbiased, the method should also yield a realistic spatial variability and be valid for both broadband and visible albedos. The ETA, for example, has the lowest standard deviation, suggesting that ETA albedos rarely deviate much from true IP albedos, satisfying the main requirements. It is not obvious, however, that the broadband absorptance corresponding to a scaled optical depth would be close to the IP absorptance. Further cloud and radiation modeling will also help to more accurately delineate the limits of the IPA in optical depth retrieval and albedo calculations. While the IPA corrects first-order effects of horizontal cloud variability it does not provide accurate fluxes for all types of clouds and solar geometries. The recent development of extensions of the IPA such as the “tilted” IPA (Várnai 1996) and the nonlocal IPA (Marshak et al. 1995) may provide a theoretical framework to bridge the gap between complex 3D and IP-type approaches. However, since current GCMs still have too coarse a resolution to meaningfully include 3D cloud structure, it is reasonable to consider in many situations the IPA as the benchmark of accuracy.