An approach is presented to distinguish between clouds and heavy aerosols in sun-glint regions with automated cloud classification algorithms developed for the National Polar-orbiting Operational Environmental Satellite System (NPOESS) program. The approach extends the applicability of an algorithm that has already been applied successfully in areas outside the geometric and wind-induced sun-glint areas of the earth over both land and water surfaces. The successful application of this approach to include sun-glint regions requires an accurate cloud phase analysis, which can be degraded, especially in regions of sun glint, because of poorly calibrated radiances of the National Aeronautics and Space Administration (NASA) Moderate Resolution Imaging Spectroradiometer (MODIS) sensor. Consequently, procedures have been developed to replace bad MODIS level 1B (L1B) data, which may result from saturation, dead/noisy detectors, or data dropouts, with radiometrically reliable values to create the Visible Infrared Imager Radiometer Suite (VIIRS) proxy sensor data records (SDRs). Cloud phase analyses produced by the NPOESS VIIRS cloud mask (VCM) algorithm using these modified VIIRS proxy SDRs show excellent agreement with features observed in color composites of MODIS imagery. In addition, the improved logic in the VCM algorithm provides a new capability to differentiate between clouds and heavy aerosols within the sun-glint cone. This ability to differentiate between clouds and heavy aerosols in strong sun-glint regions is demonstrated using MODIS data collected during the recent fires that burned extensive areas in southern Australia. Comparisons between heavy aerosols identified by the VCM algorithm with imagery and heritage data products show the effectiveness of the new procedures using the modified VIIRS proxy SDRs. It is concluded that it is feasible to accurately detect clouds, identify cloud phase, and distinguish between clouds and heavy aerosol using a single cloud mask algorithm, even in extensive sun-glint regions.
A new approach is demonstrated to distinguish between clouds and heavy aerosols with automated cloud classification algorithms developed for the National Polar-orbiting Operational Environmental Satellite System (NPOESS) program (Hutchison et al. 2008). This approach exploits differences in both spectral and textural signatures between clouds and aerosols to identify pixels that contain heavy aerosols but were originally classified as clouds by the Visible Infrared Imager Radiometer Suite (VIIRS) cloud mask (VCM) algorithm. The approach relies strongly upon the identification of heavy aerosol candidates using spectral tests applied to VIIRS moderate-resolution data (750 m) collected in the 2.25- and 0.65-μm bands in conjunction with the 0.412-μm band, where smoke has a maximum reflectance while dust simultaneously has a minimum reflectance (Dubovik et al. 2002; Figs. 4.8–4.12 in Hutchison and Cracknell 2006). The procedures benefit from the VIIRS design, which is dual gain in the 0.412-μm band, to avoid saturation in cloudy conditions (Hutchison and Cracknell 2006). The methodology also exploits other information available from the VIIRS cloud mask algorithm in addition to cloud confidence, including the phase of each cloudy pixel (Hutchison et al. 2005), which is critical for identifying water clouds and restricting the use of spectral tests that would misclassify ice clouds as heavy aerosols. Initial testing has shown these procedures to accurately distinguish clouds from dust, smoke, and industrial pollution over both land and ocean backgrounds in global datasets collected by the National Aeronautics and Space Administration’s (NASA) Moderate Resolution Imaging Spectroradiometer (MODIS) sensor. However, the procedures were not originally employed in sun-glint regions because 1) sun glint impacts the spectral tests used to identify heavy aerosol candidates that are further evaluated with spatial tests, and 2) missing, saturated, and/or degraded MODIS level 1B (L1B) data impact the cloud phase analyses that are needed to identify water clouds (Hutchison et al. 2008).
The procedures used to discriminate between clouds and heavy aerosols are now extended to areas within the geometric and wind-induced sun-glint areas over both land and water surfaces. The successful application of the VCM logic into regions of strong sun glint requires an accurate cloud phase analysis, which had previously been degraded by poor sensor (L1B) data records collected by MODIS. Consequently, procedures have been developed at Northrop Grumman Aerospace Systems (NGAS) to replace pixel data degraded by saturation, dead or noisy detectors, and data dropouts, with radiometrically reliable values obtained using neighboring bandpasses and/or pixels where these problems are not experienced, as discussed in the next section. In section 3, comparisons between cloud phase analyses obtained with both modified and unmodified VIIRS proxy sensor data records (SDRs) are shown in relation to the manual interpretation of features present in the MODIS imagery. Comparisons are also shown between the heavy aerosols detected using the unmodified VIIRS proxy SDRs, the modified SDRs, and those generated by the NASA MODIS cloud mask (MCM) algorithm (MOD35), using MODIS data collected as fires burned extensive areas of southern Australia in 2009. Concluding remarks are found in section 4.
2. Generation of VIIRS proxy data
The spectral characteristics of the MODIS L1B sensor data make them suitable to serve as VIIRS proxy SDRs, although the spatial characteristics and data coverage of MODIS data are less well-suited to represent VIIRS data (Hutchison and Cracknell 2006). The current MODIS-to-VIIRS band mapping used at NGAS associates each VIIRS band with a single MODIS band, except for the day–night band (DNB). This mapping is based on the best spectral match between the respective sensor bands, as shown in Table 1. (The VIIRS DNB is constructed as a weighted sum of MODIS bands 1, 2, 4, and 17, with the weights based on the transmittance of the DNB bandpass filter at each MODIS band center wavelength and the predicted signal-to-noise ratio for each band.)
Several MODIS reflective solar bands have significant problems with saturation, including bands 8 (0.412 μm), 9 (0.443 μm), and 15 (0.748 μm), which are used to emulate VIIRS bands M1 (0.412 μm), M2 (0.443 μm), and M6 (0.746 μm), respectively. Bands 8 and 9 tend to saturate over clouds, depending on optical depth, with band 9 usually saturating before band 8. MODIS band 15 not only saturates for any cloud condition, it also saturates over land. In addition, MODIS L1B sensor data may be degraded by dead or noisy detectors and data dropouts.
In our previous publication (Hutchison et al. 2008), the impact of bad pixel data collected by MODIS could be seen in the VCM analyses. Saturation in the single-gain MODIS band 8, which has a signal-to-noise ratio design of 880 to support ocean color analyses, is particularly problematic for the VCM algorithm. Data collected in this bandpass have been shown to be useful for detecting clouds over desert surfaces (Hutchison and Jackson 2003) and identifying aerosols that have been classified as clouds (Hutchison et al. 2008). The impact of these bad data is most evident in cloud phase analyses generated by the VCM algorithm, as seen in Fig. 6c in Hutchison et al. (2008). In this figure, the VCM algorithm failed to retrieve a cloud phase for many pixels located in the lower-left corner of the image. These failed retrievals appear as dark blue stripes in the cloud phase analysis, which meant that no cloud phase classification could be made for these pixels.
Consequently, procedures have been developed to replace bad MODIS L1B values in the VIIRS proxy data with radiometrically reliable data that have been shown useful to assess the performance of NPOESS algorithms. These procedures address 1) improving radiances in saturated pixels, 2) removing striping in the 1.6-μm (VIIRS M10) band, and 3) flagging pixels assigned a zero radiance value.
The accurate replacement of saturated pixels in MODIS band 8 relies upon radiance values from unsaturated neighboring bands. However, this substitution considers the physical behavior of the spectral radiance under the conditions that cause the pixels to saturate, as will be shown in examples that follow. A number of simulations were run of the top-of-the-atmosphere (TOA) spectral reflectance for cloud-covered ocean scenes. The Coupled Ocean Atmosphere Radiative Transfer (COART) model (Jin and Stamnes 1994) was used to generate the TOA spectral reflectance for wavelengths from 0.35 to 0.9 μm in 0.001-μm steps. Because the wavelengths of interest for VIIRS bands M1 and M2 are well below 0.55 μm, the effect of absorption by atmospheric gases is not significant; hence, the simulations were all performed using just the U.S. Standard Atmosphere, 1962 (US-62; Anderson et al. 1986). A maritime aerosol with an optical thickness of 0.1 at 0.55 μm was also assumed for all of simulations. The cloud phase (water or ice), optical depth (1 and 10), and cloud-top height (1–2 and 4–5 km for water clouds, and 4–5 km for ice clouds) were varied in the simulations. The droplet size distribution of the clouds was chosen such that the effective droplet radius was 10 μm.
The results of these simulations are shown in Fig. 1a. Several interesting features can be seen in this figure. First, the main factor affecting the TOA reflectance is the optical thickness of the cloud, where the results for different cloud properties other than optical depth only provide slight variations in the spectral reflectance. Another prominent feature is that for wavelengths less than approximately 0.55 μm, the logarithm of the spectral reflectance at TOA appears to decrease approximately linearly with the logarithm of the wavelength. Finally, as the optical thickness of the cloud increases, the variations of the spectral radiances (or reflectances) as a function of the changes in wavelength, become relatively small. Thus, replacing saturated values can be accomplished accurately by interpolation or extrapolation using data from a neighboring band. Figure 1b shows the errors in VIIRS M1 (0.412 μm) reflectance that result from using a linear interpolation in the log of wavelength and reflectance of data in the VIIRS M3 and M4 bands. Figure 1c shows the errors that result from a simple replacement of M1 data with M3 reflectance data. The mean error in reflectance is 1.47% and the standard deviation is 3.41% using the interpolation method, whereas values using a direct substitution were 7.62% and 2.85%, respectively. Similarly, extrapolations were made with data from the VIIRS M3 and M1 bands to replace bad data in the M2 band. An emulation of the M2 data using only the M3 band produced errors in reflectance values with a mean of 10.7% and 2.78%, respectively. When the emulation of M2 included both M3 and M1 bands, these errors had a mean of 0.03% with a standard deviation of 0.42%.
A majority of the MODIS band 6 (1.61 μm) detectors on Aqua MODIS are either anomalous or nonfunctional, and data collected in this band are needed to determine cloud confidence and cloud phase in the VCM. The MODIS band 6 samples the scene at a 500-m nadir resolution and is used in its native resolution to emulate the VIIRS I3 (375 m) imagery band, as shown in Table 1. To emulate the VIIRS M10 (750 m) radiometry band, which has the same bandpass as VIIRS I3, the MODIS 1KM L1B product is used, where the pixels for band 6 have been aggregated to produce a 1-km resolution pixel. The MODIS L1B code does not calibrate earth-view pixels for nonfunctional detectors. However, to facilitate building images with the L1B production data, the scaled integer values for the nonfunctional detectors are interpolated from the nearest live detectors and are flagged to alert users so that they can decide whether to use pixels from the nonfunctional detector gaps. On aggregating the scaled integer values from their native 500-m resolution to 1 km, those 1-km pixels containing nonfunctional detector data are given fill values denoting aggregation failure. This procedure produces the severe pixel striping shown in the color composite in Fig. 2a, which is constructed from the unmodified MODIS SDR granule collected at 0355 UTC on 8 February 2009, that is, MYDA2009.039.0355. [In this figure, dense smoke from fires burning over much of coastal southern and eastern Australia is seen mixed with clouds extending over the western Pacific Ocean region. This color composite was constructed by combining reflectance data from the MODIS 0.412- and 1.6-μm bands with the 3.7–12.0-μm brightness temperature difference (T3.7–T12) data in the red, green, and blue (RGB) assignments of an RGB display. The display equates to the VIIRS M1 and M10 reflectances and the VIIRS M12–M16 brightness temperature difference channels, respectively. In this image, smoke has a reddish hue, labeled A, because the main contribution of energy from smoke in this composite comes from the 0.412-μm band. Water clouds, labeled B, are yellow because of their high reflectivity in both the 0.412- and 1.6-μm bands. Cirrus clouds range in color from blue (optically thin, labeled C) to pink (optically thick, labeled D), showing differences in the 1.6-μm reflectance values for these two features and (T3.7–T12) thermal contrast of them.] The approach to eliminating the effects of striping on the VCM algorithm is to identify each pixel with a fill value due to aggregation failure and replace it by the average of the radiance values from the corresponding 2 × 2 imagery-resolution pixel data emulated for VIIRS band I3. The results of this process are seen by the inspection of Fig. 3a.
For nighttime granules or day/night granules with zero or very low radiance, the MODIS reflective solar bands, except band 26, which emulates VIIRS band M9 (1.375 μm), use a fill value of 65535, signifying either missing data at night or completely missing scans. When this fill value is observed, the corresponding VIIRS bands are assigned a value of zero radiance.
Finally, other procedures are used to replace saturated/fill values in MODIS L1B data, which are not used in the VCM algorithm. One procedure employs a normalized difference vegetation index (NDVI) test to replace missing data in MODIS band 15, which is the surrogate for the VIIRS M6 (0.746 μm) band. The bands surrounding VIIRS M6 are the M5 (MODIS band 1) and M7 (MODIS band 2), both of which very rarely saturate. Thus, the natural approach is to interpolate between the reflectance of bands M5 (0.672 μm) and M7 (0.865 μm) to replace saturated values for band M6 (0.746 μm). However, the way this interpolation is made depends on whether the pixel contains a cloud background or a cloud-free surface background, and if it is the latter, then the type of surface present becomes important. For clouds and most nonvegetated terrain surfaces, the TOA reflectance varies relatively smoothly between bands M5 and M7. However, if the background is vegetation dominated, a sharp rise in reflectance occurs at the chlorophyll edge at just about the location of MODIS band 15 or VIIRS band M6 (Fig. 4.8 in Hutchison and Cracknell 2006). As a result, if the NDVI is less than 0.3, the interpolation procedure is used to replace bad data in the VIIRS M6 band with MODIS data from bands 1 and 2, comparable to the VIIRS M5 and M7 bandpasses. However, if the NDVI is ≥0.3, a simple substitution is made for bad VIIRS M6 data using the M7 band.
3. Cloud versus heavy aerosols discrimination in sun-glint regions using the VCM algorithm
The VCM algorithm was designed to form the beginning of the processing chain for other NPOESS data products. For VIIRS, this chain includes seven cloud products (cloud optical thickness and effective particle size; cloud-top pressure, height, and temperature; cloud-base height; and cloud cover/layers); three ocean surface products (sea surface temperature, ocean color, and net heat flux); five land surface products (land surface temperature, normalized difference vegetation index and enhanced vegetation index, albedo, soil moisture, and surface type); three snow/ice products (a snow mask, ice surface temperature, and ice age); and three atmospheric aerosol products (optical thickness, particle size parameter, and suspended matter).
Although the VCM algorithm has its heritage in the MCM algorithm (Ackerman et al. 2002, 2006), it has been modified substantially to exploit the design of the VIIRS sensor. Both the VCM and MCM algorithms output a pixel-level cloud confidence consisting of four possible classes—that is, confidently clear, probably clear, probably cloudy, and confidently cloudy conditions—although individual cloud tests used in these algorithms have diverged in recent years. Both the VCM and MCM algorithms also generate cloud phase analyses; however, the VCM includes up to seven possible classes described by Pavolonis and Heidinger (2004). Possible cloud phase classes include water, mixed phase, cirrus, opaque ice, overlap (water and ice clouds in a single pixel), partly cloudy, and cloud free. The MCM algorithm does not differentiate between the three types of ice clouds (Menzel et al. 2006). In addition, the VCM produces a geometrically based cloud shadow to reduce undesirable artifacts in the NPOESS aerosol and surface data products (Hutchison et al. 2009), whereas the MCM algorithm uses spectral signatures in an attempt to identify cloud shadows (Ackerman et al. 2002).
Accurate procedures are needed to identify pixels that contain heavy aerosols in order to meet the derived requirements for both the NPOESS cloud and aerosol data products (Hutchison et al. 2008). The analyses of MODIS granules collected over a global range of conditions revealed that heavy aerosols can be frequently misclassified as confidently cloudy by automated cloud detection algorithms, such as the VIIRS cloud mask algorithm and the current (collection 5) MCM algorithm. The challenges of differentiating between clouds and heavy aerosols typically experienced by automated cloud detection algorithms are shown by returning to Fig. 2. Both the VCM, in Fig. 2b, and MCM, in Fig. 3c, algorithms classify most of the heavy aerosol as confidently cloudy (red). Other cloud confidences in both the VCM and MCM analyses are confidently clear (dark blue), probably clear (light blue), and probably cloudy (gold).
The impact of poor quality (unmodified) MODIS L1B data on the quality of the VCM cloud phase analysis is seen in Fig. 2d. Stripes that are evident in the MODIS imagery, shown in Fig. 2a are accentuated in Fig. 2d, where dark blue lines reflect “no analysis” was performed. Two large areas, in the lower-left center (labeled E) and upper-right center (labeled F) of the panel, show no analysis was performed for entire cloud fields as well. The inability to discriminate between ice and water clouds limited the utility of these new procedures to regions outside sun glint because 1) the phase of clouds could not be analyzed with certainty, and 2) the misclassification of ice clouds would result in them being classified as aerosols by the spatial test used in cloud versus heavy aerosol logic.
The replacement of bad or missing pixel data in the VIIRS proxy SDRs produces an excellent cloud phase analysis with the modified SDR data. Note that these results are in good agreement with features clearly evident in the MODIS imagery as seen in Fig. 3. The color composite shown in Fig. 3a is based upon the VIIRS proxy SDR data, after it was modified using procedures described in section 2. Figures 3 and 2 were produced using identical display techniques. The resulting VCM cloud confidence and cloud phase analyses are shown in Figs. 3b and 3d, respectively. The sun-glint envelope for both geometry (light blue) and wind-induced (red) effects is contained in Fig. 3c. A comparison of the cloud phase and color imagery shows the improvement in the cloud phase analysis with the higher-quality VIIRS proxy SDR. Therefore, it is possible to identify pixels that are heavy aerosol candidates in the sun-glint regions using the cloud phase analysis in a manner similar to that done previously over land, that is, all water clouds become heavy aerosol candidates in sun-glint regions. [As noted in a previous publication (Hutchison et al. 2008), smoke, dust, and industrial pollution features are typically classified as water clouds by the VCM cloud phase algorithm.] These candidates are further evaluated using a spatial test before they are declared heavy aerosols (Hutchison et al. 2008).
The results of the VCM logic to identify heavy aerosols are shown in Figs. 4a and 4b for the two VIIRS proxy SDRs shown in Figs. 2 and 3, respectively. Figure 4a shows heavy aerosols detected with the unmodified VIIRS proxy SDR, which can only be applied outside sun-glint regions, whereas Fig. 4b shows those pixels identified to contain heavy aerosols using the modified VIIRS proxy SDRs. In addition, Fig. 4c contains the obscuration (i.e., not cloudy) flag from the MODIS collection 5 MOD35 MCM.
The improved quality of the VIIRS proxy SDR, shown in Fig. 3, allows the water clouds within sun-glint regions to be identified as heavy aerosol candidates that are then evaluated using the spatial test. No changes were made to the VCM algorithm logic used to identify heavy aerosols in pixels initially classified as confidently cloudy, except for the removal of the sun-glint constraint. The result is that additional areas of heavy aerosol are accurately identified in the sun-glint region using the modified and improved MODIS SDR.
Note the near-circular cloud in the middle of the sun-glint regions as seen in the VCM cloud confidence analysis in both Figs. 2 and 3. This feature is enhanced in Fig. 5. The imagery in Fig. 5a clearly shows this not to be a circular cloud but an elongated (pencil shaped) cloud with heavy aerosol surrounding it. In Fig. 5b, it is seen that this feature is classified initially as confidently cloudy and determined to be a water cloud using the modified VIIRS proxy SDR. The results of the heavy aerosol logic in the VCM reveal that the procedures correctly identify the regions of aerosol not coincident with clouds, whereas areas containing clouds and aerosols continue to be correctly classified as confidently cloudy because no NPOESS surface products should be retrieved under cloudy conditions.
It is desirable to use a single approach to generate automated cloud analyses to meet the system-level performance requirements and minimize latency in operational weather systems, such as NPOESS. However, it becomes necessary that the approach used in generating automated cloud masks be sufficiently robust to satisfy all the derived requirements of the remaining VIIRS data products.
Previously, an approach was demonstrated to distinguish between clouds and heavy aerosols with automated cloud classification algorithms developed for the NPOESS program. However, these procedures could not be applied in sun-glint regions with current VIIRS proxy SDR data, which may be degraded by pixel saturation, dead or noisy detectors, and data dropouts in the MODIS L1B data. Therefore, it was necessary to modify these VIIRS proxy SDR data to produce radiometrically reliable values from neighboring bandpasses and/or pixels where these problems are not experienced. Cloud phase analyses produced by the NPOESS VIIRS cloud mask algorithm using these modified SDRs show excellent agreement with features observed in color composites of MODIS imagery. In addition, using these modified VIIRS proxy SDRs allow the VCM heavy aerosol logic to be applied in sun-glint regions over both land and water surfaces without any major algorithm changes. It now seems possible that heavy aerosols can be identified and differentiated from clouds within strong sun-glint regions using the VIIRS cloud mask algorithm and modified VIIRS proxy SDRs. Furthermore, it appears evident that these superior products can be created during the NPOESS era using VIIRS SDRs that employ dual-gain bands to avoid the effects of pixel saturation.
The views, opinions, and findings contained in this report are those of the authors and should not be construed as an official position of Northrop Grumman, NOAA, or the NPOESS program.
Corresponding author address: Keith Hutchison, Senior Research Fellow, The University of Texas at Austin, 3925 W. Braker Lane, Suite 200, Austin, TX 78759. Email: firstname.lastname@example.org