Advancing Maritime Transparent Cirrus Detection Using the Advanced Baseline Imager “Cirrus” Band

Theodore M. McHardy aDepartment of Hydrology and Atmospheric Sciences, The University of Arizona, Tucson, Arizona

Search for other papers by Theodore M. McHardy in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-5996-5320
,
James R. Campbell bNaval Research Laboratory, Monterey, California

Search for other papers by James R. Campbell in
Current site
Google Scholar
PubMed
Close
,
David A. Peterson bNaval Research Laboratory, Monterey, California

Search for other papers by David A. Peterson in
Current site
Google Scholar
PubMed
Close
,
Simone Lolli cCNR-IMAA, Istituto di Metodologie per l’Analisi Ambientale, Tito Scalo, Italy

Search for other papers by Simone Lolli in
Current site
Google Scholar
PubMed
Close
,
Richard L. Bankert bNaval Research Laboratory, Monterey, California

Search for other papers by Richard L. Bankert in
Current site
Google Scholar
PubMed
Close
,
Anne Garnier dScience Systems Applications, Inc., Hampton, Virginia

Search for other papers by Anne Garnier in
Current site
Google Scholar
PubMed
Close
,
Arunas P. Kuciauskas bNaval Research Laboratory, Monterey, California

Search for other papers by Arunas P. Kuciauskas in
Current site
Google Scholar
PubMed
Close
,
Melinda L. Surratt bNaval Research Laboratory, Monterey, California

Search for other papers by Melinda L. Surratt in
Current site
Google Scholar
PubMed
Close
,
Jared W. Marquis eDepartment of Atmospheric Sciences, University of North Dakota, Grand Forks, North Dakota

Search for other papers by Jared W. Marquis in
Current site
Google Scholar
PubMed
Close
,
Steven D. Miller fCooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado

Search for other papers by Steven D. Miller in
Current site
Google Scholar
PubMed
Close
,
Erica K. Dolinar bNaval Research Laboratory, Monterey, California

Search for other papers by Erica K. Dolinar in
Current site
Google Scholar
PubMed
Close
, and
Xiquan Dong aDepartment of Hydrology and Atmospheric Sciences, The University of Arizona, Tucson, Arizona

Search for other papers by Xiquan Dong in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

We describe a quantitative evaluation of maritime transparent cirrus cloud detection, which is based on Geostationary Operational Environmental Satellite 16 (GOES-16) and developed with collocated Cloud–Aerosol Lidar with Orthogonal Polarization (CALIOP) profiling. The detection algorithm is developed using one month of collocated GOES-16 Advanced Baseline Imager (ABI) channel-4 (1.378 μm) radiance and CALIOP 0.532-μm column-integrated cloud optical depth (COD). First, the relationships between the clear-sky 1.378-μm radiance, viewing/solar geometry, and precipitable water vapor (PWV) are characterized. Using machine-learning techniques, it is shown that the total atmospheric pathlength, proxied by airmass factor (AMF), is a suitable replacement for viewing zenith and solar zenith angles alone, and that PWV is not a significant problem over ocean. Detection thresholds are computed using the channel-4 radiance as a function of AMF. The algorithm detects nearly 50% of subvisual cirrus (COD < 0.03), 80% of transparent cirrus (0.03 < COD < 0.3), and 90% of opaque cirrus (COD > 0.3). Using a conservative radiance threshold results in 84% of cloudy pixels being correctly identified and 4% of clear-sky pixels being misidentified as cirrus. A semiquantitative COD retrieval is developed for GOES ABI based on the observed relationship between CALIOP COD and 1.378-μm radiance. This study lays the groundwork for a more complex, operational GOES transparent cirrus detection algorithm. Future expansion includes an overland algorithm, a more robust COD retrieval that is suitable for assimilation purposes, and downstream GOES products such as cirrus cloud microphysical property retrieval based on ABI infrared channels.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Theodore M. McHardy, tmchardy@email.arizona.edu

Abstract

We describe a quantitative evaluation of maritime transparent cirrus cloud detection, which is based on Geostationary Operational Environmental Satellite 16 (GOES-16) and developed with collocated Cloud–Aerosol Lidar with Orthogonal Polarization (CALIOP) profiling. The detection algorithm is developed using one month of collocated GOES-16 Advanced Baseline Imager (ABI) channel-4 (1.378 μm) radiance and CALIOP 0.532-μm column-integrated cloud optical depth (COD). First, the relationships between the clear-sky 1.378-μm radiance, viewing/solar geometry, and precipitable water vapor (PWV) are characterized. Using machine-learning techniques, it is shown that the total atmospheric pathlength, proxied by airmass factor (AMF), is a suitable replacement for viewing zenith and solar zenith angles alone, and that PWV is not a significant problem over ocean. Detection thresholds are computed using the channel-4 radiance as a function of AMF. The algorithm detects nearly 50% of subvisual cirrus (COD < 0.03), 80% of transparent cirrus (0.03 < COD < 0.3), and 90% of opaque cirrus (COD > 0.3). Using a conservative radiance threshold results in 84% of cloudy pixels being correctly identified and 4% of clear-sky pixels being misidentified as cirrus. A semiquantitative COD retrieval is developed for GOES ABI based on the observed relationship between CALIOP COD and 1.378-μm radiance. This study lays the groundwork for a more complex, operational GOES transparent cirrus detection algorithm. Future expansion includes an overland algorithm, a more robust COD retrieval that is suitable for assimilation purposes, and downstream GOES products such as cirrus cloud microphysical property retrieval based on ABI infrared channels.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Theodore M. McHardy, tmchardy@email.arizona.edu

1. Introduction

In 1999, the International Satellite Cloud Climatology Project (ISCCP; Rossow and Schiffer 1999), using a composite dataset composed of numerous passive radiometric sensors, estimated that the global instantaneous frequency of cirrus clouds was approximately 20%. The launch of the Cloud–Aerosol Lidar with Orthogonal Polarization (CALIOP) aboard the Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite in 2006, however, drastically changed cloud detection. Studies using data from this active sensor have suggested that cirrus clouds actually cover roughly 40% of Earth at any given time (Mace and Zhang 2014) and as much as 70% for some tropical regions (Lolli et al. 2017). This discrepancy comes from the difficulty of passive radiometric instruments to detect thin transparent clouds with optical depths of less than 0.3 (Ackerman et al. 2008; Stubenrauch et al. 2013). Campbell et al. (2015), among others, further showed that the distribution of cirrus clouds in terms of optical depth is exponentially skewed toward optically thinner clouds, meaning that passive sensors were failing to detect roughly half of cirrus in the atmosphere, a result predicted presciently by Sassen and Cho (1992) (Stubenrauch et al. 2013). An example of transparent cirrus is shown in Fig. 1a.

Fig. 1.
Fig. 1.

(a) A photograph of transparent cirrus clouds taken in Monterey, California, on 11 Jun 2020, (b) GOES-16 ABI “true color” image for a sector over the Caribbean Sea at 1800 UTC 19 Aug 2020, and (c) the GOES-16 ABI 1.378-μm channel reflectance at 1800 UTC 19 Aug 2020.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

Passive radiometric datasets and subsequent level-2 retrievals based upon them are widely used in both operational and research applications. The failure to accurately discriminate transparent clouds in satellite-based radiometric pixels therefore has significant ramifications for meteorological and oceanographic interests. For example, Marquis et al. (2017) showed that optically thin cirrus (cloud optical depths less than 0.3; Sassen and Cho 1992) contaminate 25% of infrared (IR) sea surface temperature (SST) retrievals with average per-pixel biases of up to 0.5 K. Chew et al. (2011) found that version-2, level-1.5 and level-2 aerosol optical depth measurements collected by the NASA AERONET sun photometer network were similarly contaminated at a near 25% rate, with a positive bias near 0.03 per observation. Lidar profilers are the obvious solution to correcting for thin cloudy pixel undersampling in passive radiometric composites. However, satellite-based instruments, such as CALIOP, have a very small ground footprint (~70 m) with a long revisit period (16 days; Winker et al. 2007) and ground-based lidars are difficult and expensive to deploy and are generally limited to landmasses. Therefore, removal of thin cirrus contamination from global observations for downstream retrieval and assimilation purposes has been very challenging.

The potential for passive detection of transparent cirrus using a 1.38-μm channel was first proposed over 25 years ago by Gao et al. (1993). The reasoning behind was twofold. There is a strong water absorption band centered at this wavelength that absorbs nearly all signal from Earth’s surface, given sufficient water vapor in the column. Also, cirrus clouds are typically located above most of the atmosphere’s water vapor, thus solar radiation scattered by these clouds stands out against a relatively dry upper troposphere. Therefore, these cirrus clouds could be detected by satellites with sufficient underlying contrast from which to specifically discriminate thin and transparent cloud filaments.

Early success in detecting cirrus using data from the Airborne Visible Infrared Imaging Spectrometer (AVIRIS) during field campaigns resulted in the selection of a channel near 1.38 μm for the Moderate Resolution Imaging Spectroradiometer (MODIS) instruments (Gao and Kaufman 1995). Methods for detecting transparent cirrus and even retrieving the cloud optical depth (COD; Meyer et al. 2004; Meyer and Platnick 2010) and ice cloud particle size and shape (Wang et al. 2014) followed. For example, of pixels deemed clear sky by the MODIS cloud mask, Dessler and Yang (2003) found that one-third contained transparent cirrus and Lee et al. (2009) found that over 40% were contaminated. However, operational use of such algorithms was never fully implemented due to technical issues with the MODIS 1.375-μm channel leading to more uncertainty than was considered acceptable (Dessler and Yang 2003). These issues include contamination due to light leakage at 5.2 μm, detector cross talk among bands between 1.24 and 4.5 μm, and a bandwidth of the 1.375-μm channel that, with hindsight, proved, perhaps, to be too large with respect to the background solar contamination (1.36–1.39 μm).

A 1.375-μm channel was implemented in both Visible Infrared Imaging Radiometer Suite (VIIRS) instruments aboard the Suomi National Polar-Orbiting Partnership (NPP) and NOAA-20 satellites, which launched 28 October 2011 and 18 November 2017, respectively. Some overlap between these satellites and the A-Train occurs every few days. However, the calibration and evaluation of a transparent cirrus detection algorithm with the use of reasonably well collocated CALIOP profiling is impractical due to the limited number of samples. Geostationary satellite sensors, such as the Advanced Himawari Imager (AHI) aboard the Himawari-8 and Himawari-9 satellites, provide an opportunity for collocation with CALIOP. However, a channel near 1.38 μm was omitted from AHI in favor of a green (0.51 μm) visible channel. The Advanced Baseline Imager (ABI; Schmit et al. 2017) aboard the next-generation Geostationary Operational Environmental Satellites (GOES) currently observing the Western Hemisphere (GOES-16 and GOES-17) therefore present the only opportunity to take advantage of CALIOP to provide robust algorithm calibration and evaluation. To develop a transparent cirrus detection algorithm and more accurately understand detection efficiencies for transparent cirrus clouds using CALIOP, the GOES ABI must be used.

This study will quantitatively evaluate maritime optically thin transparent cirrus (COD < ~0.3) cloud detection from GOES-16 using collocated vertical profiles from CALIOP. CALIOP provides the opportunity to estimate a baseline for thin cirrus detection in terms of COD and develop a semiquantitative retrieval for COD. We will show that the algorithm can detect cirrus clouds identified by CALIOP at rates greater than 80% for most CODs down to 0.1 and nearly 80% for CODs down to 0.05, while preserving over 96% of clear-sky pixels. Caveats, such as collocation error and the presence of aerosols, likely have a small impact on these results, but are discussed in detail. This research sets a foundation for a future comprehensive cirrus detection algorithm, including application over land and a more robust COD retrieval that is suitable for operational and research applications (level 2).

2. Data and methods

a. Study region and period

Because use of the 1.378-μm channel will become more complicated over landmasses because of ice/snow cover and reflectance of elevated terrain, we limit the focus here to maritime analysis only. August 2018 is the study period, which was chosen for lacking any significant upper-tropospheric contamination (i.e., volcanic and pyroconvective activity; e.g., Peterson et al. 2018). A month occurring during the Northern Hemisphere summer was selected to limit the presence of sea ice. Only one month of GOES ABI data are used here to limit computational demands. This sample still provides a wide range of viewing situations that we believe are sufficient for proof of concept. As this study relies on radiance and solar scattering at 1.378 μm, only data collected during daytime are used, which is broadly determined as when the solar zenith angle (SZA) is < 80°, which is near the 82° threshold commonly used to discriminate between daytime and twilight [e.g., Minnis et al. 2008; this is close to a more stringent definition applied by, for instance, Campbell et al. (2012)].

b. Datasets

1) GOES-16 ABI data

The primary dataset used in this study is the GOES-16 ABI channel-4 (1.378 μm; hereinafter “Ch. 4”) radiance. GOES-R was launched on 19 November 2016 and became GOES-16 when it was placed into the GOES-East operational position on 17 November 2017 at 75.2°W, providing a view centered between North and South America. It was declared fully operational on 18 December 2017. GOES-16 was chosen over GOES-17 because it is centered over the Atlantic basin and using both would double the computing resources needed while providing little additional value. The ABI is significantly more advanced than GOES legacy instruments. Only full disk scans are used in this study, which were available every 15 min during the study period but are now available every 10 min. With 16 channels ranging from 0.47 to 13.28 μm and nadir-viewing spatial resolutions from 0.5 to 2 km (Schmit et al. 2017), this instrument is closer to MODIS than it is to the previous GOES-15 imager. To date, ABI Ch. 4 is performing within specifications (±4%; Bartlett et al. 2018; Yu et al. 2017). Readers are directed to the GOES-R product users guide for more information. The 1.378-μm channel has a nominal nadir spatial resolution of 2 km and a bandwidth (50% full-width at half maximum) from 1.366 to 1.38 μm, which is narrower than its cousin on MODIS, reducing susceptibility to noise from the surface and lower troposphere. In addition to the Ch.-4 data, brightness temperatures from channels 14 and 15 (~11 and 12 μm, respectively) are used as part of a case study to test a traditional cirrus cloud detection method.

An example image from Ch. 4 is shown in Fig. 1c along with a simulated true-color image (Fig. 1b) for comparison. Because the ABI lacks a green channel, a green channel is estimated from the blue, red, and near-infrared adjacent bands via a lookup-table approach that has been trained on Himawari-8 AHI data. A further correction to this band to increase sensitivity to green vegetation involved a blend of ~7% of the vegetation band reflectance to the estimated green band. Atmospheric corrections for suppression of the Rayleigh scattering molecular atmosphere have also been applied. The background and application of this approach are described in Miller et al. (2012, 2016).

2) CALIOP data

This study employs 0.532-μm column-integrated COD data from the CALIOP v4.20 level-2, 5-km cloud profile data product (Vaughan et al. 2020; Winker et al. 2009). Extinction coefficients are derived using the Hybrid Extinction Retrieval Algorithm (HERA; Young and Vaughan 2009; Young et al. 2018). The profile products are reported at a uniform spatial resolution of 60 m vertically and 5 km horizontally, over a nominal altitude range from 30 to −0.5 km. For this study, we select profiles that contain single-layered “transparent cirrus” as reported in the atmospheric volume descriptor (bits 10 to 12 must be equal to 6). The cloud-type categorization follows ISCCP definitions (Rossow and Schiffer 1999) and indicates, in particular, that the ice cloud is transparent to CALIOP and has a top pressure lower than 440 hPa. Furthermore, we require the CAD score, which indicates the confidence in the cloud classification by the Cloud Aerosol Discrimination algorithm (Liu et al. 2019), is between 20 and 100 or equal to 106 to discard clouds classified with no confidence. “Deep convective” clouds are not included in this study because these clouds are opaque to CALIOP and have COD > ~3 (Young et al. 2018) and will thus be detected by standard cloud masks, which makes them redundant to the goals of this study. Cases with transparent cirrus above another cloud layer are also not included, as detectability of transparent cirrus is the focus here. Last, the cloud extinction coefficient values and COD are considered of high quality if they were retrieved with Extinction QC flags equal to 0, 1, or 2.

Precipitable water vapor (PWV) is also a major component of this study because it has a direct impact on the ABI radiance in Ch. 4. Here, we compute PWV from relative humidity profiles provided in the CALIOP cloud product, which are derived from the Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2; Gelaro et al. 2017), output, by converting the layer relative humidity into mixing ratio and integrating through the column.

3) Auxiliary data

Two other datasets are used in minor roles in this study. First, the land/sea mask from the Global Multiresolution Terrain Elevation Data (GMTED2010; Danielson and Gesh 2011) product is used to determine which GOES-16 pixels are over the ocean.

Aerosols are detected using the GOES-16 ABI L2 + Aerosol Detection Product (ADP; NOAA/NESDIS 2018). This product is composed of various band ratio and reflectance thresholds that produce a binary aerosol mask (0 = definitively no aerosol; 1 = definitively aerosol), utilizing bands primarily in the visible and thermal infrared range, with the addition of the 1.378-μm (over land only) and 3.9-μm channels. The ADP relies on the ABI Cloud Mask (ACM; Heidinger 2010) to remove errors induced by cloud cover. With respect to optically thin cirrus detection, the ACM algorithm has a simple threshold of 5% reflectance in the 1.378-μm channel. For a solar zenith angle of 0°, 5% reflectance corresponds to a radiance value of approximately 5.5 W m−2 μm−1 sr−1. It will be shown later that this value is an order of magnitude larger than the thresholds developed and tested in this study.

The ADP provides a supplemental screening that likely removes highly aerosol-contaminated pixels. Though CALIOP could have been used to screen samples for the presence of aerosols, this would only serve to complicate this study more than was deemed acceptable. The ADP was chosen as the lone aerosol screening dataset because collocation between CALIOP and GOES is imperfect and likely causes some sampling error, especially at large viewing zenith angles (VZA). CALIOP is also not available for near-real-time operational use and therefore cannot be implemented as a core component of the final detection algorithm. Even using CALIOP to merely screen for aerosols would lead to an algorithm that produces different results in this study versus operationally. Last, screening for aerosols in this case warrants more than simply removing cases based on aerosol optical depth (AOD). A thorough quantification of the effects of aerosol loading on the Ch.-4 radiance would be required to determine any AOD-based threshold. This is outside of the scope of this study and is therefore left for future work.

GOES-16 cloud-top height (CTH) and COD are included as part of a case study for further validation. These data are not a major component of this study; therefore, readers are directed to the algorithm theoretical basis documents for more information [Heidinger (2012) and Walther et al. (2013), respectively]. Note that the GOES COD and CTH products have stricter viewing/solar geometry constraints than are used in this study, but the case study shown is within these constraints. Also, the GOES COD is retrieved at 640 nm, while CALIOP COD is retrieved at 532 nm.

c. Methods

1) Collocation

To calibrate and evaluate the algorithm produced in this study, ABI and CALIOP data must be spatiotemporally collocated. This process is not straightforward because GOES-16 is geostationary and not part of the A-Train. One day of daytime CALIOP overpasses are shown in Fig. 2a superimposed over the VZA of GOES pixels in the full disk scan. Figure 2b shows the estimated GOES pixel size as a function of the VZA. The pixel size is estimated by interpolating the GOES pixel corners from the GOES pixel centers and then computing the two-dimensional area. GOES-16 pixels increase in size as VZA increases, growing from 2 km at nadir to over 10 km at the scan edge. The CALIOP L2 cloud profiles product footprint is essentially a one-dimensional curtain (~70-m width; Winker et al. 2007) broken down into 5-km “pixels” along track.

Fig. 2.
Fig. 2.

(a) One day of daytime CALIOP ground tracks (black lines) superimposed over viewing zenith angle for the GOES-16 disk (shading). (b) Estimated GOES-16 pixel size as a function of viewing zenith. Pixel size is estimated by interpolating GOES pixel centers to find the pixel corners and then computing the two-dimensional area. The blue lines in (b) and (a) represent the VZA = 80° threshold used in this study and its location in the GOES-16 disk, respectively.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

A collocated observation is defined as the intersection of a CALIOP orbit with a GOES pixel within 7.5 min of a GOES full disk scan. CALIOP 5-km “pixels” that intersect multiple GOES pixels are paired with each GOES pixel independently. Examples of this collocation near GOES nadir viewing and near the edge of the GOES disk are shown in Fig. 3. GOES pixels are shaded in gray as a visual aid. In Fig. 3b, the “missing” GOES pixels (i.e., nonshaded area that is underneath the CALIOP track) are due to a first-pass filter in the collocation that removes GOES pixels with centers that are more than 0.2° away from the CALIOP “pixel” center. Information from CALIOP therefore becomes less representative of what GOES sees as distance from nadir viewing increases. We adopt a VZA limit of < 80° throughout this study to limit the effects of the increasing size of the GOES pixels on the results.

Fig. 3.
Fig. 3.

Examples of the collocation process for (a) near-nadir viewing and (b) near the GOES-16 disk edge. The black line indicates the CALIOP ground track, with CALIOP 5-km pixel centers indicated by the circles. GOES pixels are shaded in gray as a visual aid.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

2) Filtering

Because this study relies on radiance and solar scattering at 1.378 μm, only data collected during daytime are used, which is broadly determined as when the SZA is < 80°. In addition to the full collocated sample, the temporal collocation range dt is also constrained to less than 1 min to limit cirrus cloud advection and detrainment between the GOES scan and the CALIOP overpass [hereinafter, high-quality (HQ) sample].

Two subsamples of collocated data are compiled and analyzed—cloudy and clear-sky samples. Therefore, we have four samples in total—cloudy and clear sky with full observations (dt < 7.5 min) and the HQ subset (dt < 1 min). Single-layered cirrus samples are determined as when the CALIOP 0.532-μm column-total COD is equal to the CALIOP 0.532-μm column transparent cirrus COD, the CALIOP 0.532-μm column-total COD is greater than 0, and when the ADP does not positively flag the pixel as containing aerosol (i.e., ADP ≠ 1). We do not use ADP = 0 (thus including pixels with fill values) as the criteria here because of the possibility that cirrus clouds are misidentified as dust. In the end, the ADP filter only removes roughly 1.5% of those samples that would otherwise be included in the transparent cirrus sample. For the clear-sky sample, the ocean-only, VZA, and SZA criteria are the same as in the transparent cirrus sample. The CALIOP 0.532-μm column total COD must be equal to 0, indicating cloud-free samples, and the ADP must positively identify clear sky (i.e., no aerosols, ADP = 0). Here, the ADP filter removes over roughly 15% pixels that would otherwise be included in this subsample. Table 1 shows the total number of collocated samples, as well as the clear-sky and transparent-cirrus-only sample sizes for the full and HQ samples.

Table 1.

Sample sizes of the full collocated dataset and the transparent cirrus and clear-sky cases broken down on the basis of the collocation time range.

Table 1.

3) Parallax Correction

Parallax must be accounted for because this study relies on the collocation of data from two spaceborne sensors with very different viewing geometries. This issue occurs when a high-altitude feature (e.g., a cirrus cloud) is observed by two different sensors with different (and nonnadir) viewing angles and becomes increasingly prominent as VZA and cloud altitude increase. Miller et al. (2018) suggest that the geolocation displacement due to parallax for a cloud with an altitude of 10 km and viewed with a modest angle of 50° is greater than 10 km. A large portion of the GOES disk has VZAs greater than 50° (Fig. 2a); therefore, parallax likely has a significant impact on the data samples in this study.

For a cirrus cloud observed by both CALIOP and the GOES-16 ABI, the geolocation of the cloud according to CALIOP will be the location on Earth directly beneath the cloud because CALIOP is approximately nadir viewing. However, the geolocation of the cloud according to GOES will be the location on Earth at the tip of a ray originating at the GOES-16 position and traveling through the cloud to the surface. In other words, a basic spatial collocation at the pixel level will result in cases where CALIOP observes a cloud within a column that is not within the solid angle of the GOES pixel and therefore the GOES geolocation must be corrected. The method for correcting the GOES-16 geolocation for parallax is described in detail in Wang and Huang (2014). Correcting for parallax requires a cloud-top height; therefore, it is only applied to the cases in the transparent cirrus sample.

3. Results

a. A clear-sky “correction”

The first step in developing a calibrated threshold response capable of detecting transparent cirrus clouds is isolating the signal specifically from these clouds. This is especially critical here because the signal from thin cirrus is relatively small, approaching the background noise levels. We achieve this by characterizing the contribution to the 1.378-μm radiance from the clear atmosphere. Figure 4 shows the distributions of Ch.-4 radiances from the clear-sky samples described in section 2c(2). These radiance values are, generally, very low, with approximately 90% of them begin below 1 W m−2 sr−1. In fact, they are so low that the radiometric resolution, or dynamic range, of the ABI shows up as discrete stratifications in the distribution. As the signal here only comes from diffuse radiation in the clear atmosphere, it is likely that viewing and solar geometry, which dictate the pathlength of the solar radiation, strongly modify the clear-sky radiance.

Fig. 4.
Fig. 4.

PDFs of clear-sky Ch.-4 radiance (W m−2 sr−1) for the (a) full sample (dt < 7.5 min) and (b) the HQ sample (dt < 1 min).

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

Figure 5 shows the distributions of VZA (Fig. 5a), SZA (Fig. 5b), PWV (Fig. 5c), and airmass factor (AMF; Fig. 5d) for the full clear-sky sample. The values of these parameters will vary slightly with the study period and the location of landmasses within the chosen study region. The AMF is computed as
AMF=[1cos(VZA)+1cos(SZA)]
(Palmer et al. 2001) and is analogous to a normalized pathlength of the solar radiation between a scattering event in the atmosphere and ABI. To understand the significance of relationships between the clear-sky Ch.-4 radiance and these four parameters, the radiances are first binned by VZA, SZA, and PWV, and the means and standard deviations are computed for each bin. This is shown as bar plots in Fig. 6, where the solid line indicates the mean and the error bars indicate the standard deviation. Clear-sky radiance increases with VZA for values greater than approximately 35° (Fig. 6a). While radiance does appear to increase with SZA (Fig. 6b), the relationship is not as pronounced. As expected, radiance is higher for PWV values less than 2 cm (Fig. 5c). Otherwise, a clear signal between PWV and radiance is not discernible. Figure 6 suggests that the pathlength of the solar radiation (e.g., AMF dependent on VZA and SZA) largely determines the clear-sky radiance and PWV only becomes significant when it is less than 2 cm, which is in line with the results from Gao and Kaufman (1995).
Fig. 5.
Fig. 5.

PDFs of (a) VZA, (b) SZA, (c) PWV, and (d) AMF for the full (dt <7.5 min) clear-sky samples.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

Fig. 6.
Fig. 6.

Mean Ch.-4 radiance (W m−2 sr−1) as a function of (a) VZA, (b) SZA, and (c) PWV. Bin sizes are 10, 10, and 1 for VZA, SZA, and PWV, respectively. Error bars indicate ±1 standard deviation.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

Similar to Fig. 6, Fig. 7 shows the mean and standard deviation of the clear-sky radiance when binned by AMF, with the full clear-sky sample shown in Fig. 7a and the HQ (dt < 1 min) clear-sky sample shown in Fig. 7b. Additional red error bars indicate 2 standard deviations, and the dotted lines represent linear regression lines computed from the values 1 and 2 standard deviations above the mean. These lines represent the basis for our transparent cirrus detection thresholds and will be discussed further in section 3b(1). The relationship between clear-sky radiance and AMF is the clearest yet, with radiance linearly increasing as a function of AMF. This is intuitive because AMF is a proxy for the normalized pathlength of radiation traveling through the atmosphere. So a larger AMF means more contribution from the clear atmosphere to the Ch.-4 radiance. However, a simple linear regression cannot be used without first examining the contributions of VZA, SZA, AMF, and PWV independently.

Fig. 7.
Fig. 7.

Mean Ch.-4 radiance (W m−2 sr−1) as a function of AMF for (a) the full clear-sky sample (dt < 7.5 min) and (b) the HQ clear-sky sample (dt < 1 min). Black and red error bars indicate 1and 2 standard deviations, respectively. Black and red dotted lines indicate the respective linear regression lines. The AMF bin size is 1. Note that the last AMF bin is for all values greater than 7 because of sample sizes.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

1) Precipitable water vapor

The detection of transparent cirrus using ABI Ch. 4 depends on strong water vapor absorption around 1.38 μm. Therefore, the column water vapor content must be considered when evaluating potential detection thresholds. While describing the process behind selecting a 1.375-μm channel for placement on the MODIS instrument, Gao and Kaufman (1995) examined the potential effects of column PWV on such a band in detail. Specifically, they computed the two-way transmittance in a 30-nm-width band centered on 1.375 μm as a function of altitude Z for five different atmospheric profiles approximating five different PWV conditions. These calculations were performed with a VZA of 0° and a SZA of 45°. A list of the five profiles they chose, and their corresponding PWV values, is shown in Table 2.

Table 2.

A list of the atmospheric profiles and corresponding PWV (cm) values used in Gao and Kaufman (1995).

Table 2.

For Z = 0 km (surface), they found that all profiles except the subarctic winter profile had two-way transmissions of 0%, which means that a satellite would not measure a signal from the surface under these conditions. For the subarctic winter profile, which has a PWV value of 0.42 cm, the two-way transmittance is 3% for Z = 0 km. This means that a satellite would receive some limited signal from the surface. For Z = 3 km, the two-way transmittance for the subarctic winter profile is around 20%. In this case, the two-way transmittances for the midlatitude winter and subarctic summer profiles are slightly greater than 10% and just less than 5%, respectively. Therefore, we should expect significant signal from an altitude of 3 km when the PWV value is between that of the midlatitude winter and subarctic summer profiles and contribution from the surface will become increasingly unlikely as PWV increases from there.

One factor working in our favor is that these calculations were performed with a SZA of 45° and nadir satellite viewing. Because GOES-16 is geostationary, such low VZAs only occur in the tropics (Fig. 2a), where the PWV over the ocean is often much higher than 1cm. Higher VZAs and SZAs, like those seen in the midlatitudes and subarctic, increase the pathlength of the solar radiation and decrease the chance that scattering from the surface, or any significant low cloudiness, reaches the satellite. Of course, high surface elevation and highly reflective surfaces such as snow and ice can increase the likelihood of background contamination. It is partially for that reason that we chose to restrict this study to exclusively ocean pixels and the northern midlatitude summer.

While this discussion focused on the importance of PWV below the transparent cirrus, the PWV above the cloud must also be discussed. Meyer and Platnick (2010) point out that up to 10% of the column PWV can be above the cirrus layer. This is nontrivial due to the nature of the 1.378-μm channel. They present a method for correcting the MODIS 1.38-μm reflectance for attenuation due to above-cloud water vapor. Full implementation of this method is outside of the scope of the current study. Attenuation due to above-cloud water vapor will be explored during future development of an operational transparent cirrus detection algorithm.

2) Untangling VZA, SZA, AMF, and PWV

To investigate the dependence of the 1.378-μm radiance on VZA, SZA, AMF, and PWV independently, a method similar to that in Lolli and Di Girolamo (2015) is adopted here. This is required because viewing/solar geometry and PWV are likely correlated in space. First, principal component analysis is performed to determine which of the four variables mentioned above have the largest impact on 1.378-μm radiance. The results show that AMF and PWV account for 94% of the variability in the clear-sky radiance, which confirms that AMF is a suitable replacement for VZA and SZA individually. However, because of the interdependence of AMF and PWV, typical mathematical methods may not be able to delineate their individual impact of 1.378-μm radiance. For this reason, the random forest (RF) “bagged ensemble tree” machine-learning (ML) method was applied (Breiman 1996).

ML is a platform that is used to “teach” algorithms to learn from experience without relying on a particular analytical (i.e., mathematical) model. The implemented ML technique (commercially available in MATLAB) is based on bootstrap-aggregated (bagged) decision trees to avoid overfitting. The bagged tree method, using bootstrap samples of the data, grows an ensemble of decision trees. Also, the algorithm selects a random subset of predictors (SZA, VZA, AMF, and PWV) to use at each decision split as in the RF algorithm. Before training the model, it is critical to choose a strategy to evaluate its performance once the model is trained. It is common to “hold out” a portion of the whole dataset to evaluate the final model. Nevertheless, there is not a well-established rule stating the percentage of the whole database that should be withheld for training and for validation. As a rule of thumb, at least 50% of the database should be used for training the model (Breiman 2001). Considering that we deal with a very large dataset, we find it reasonable to hold out 40% of the dataset for validation. In this application, we use a supervised RF regression technique using a minimum leaf size of 8 and 30 learners. The model was trained using 60% of the full (i.e., not restricted to HQ) clear-sky sample, while the remaining 40% of the data were reserved for validation. A detailed description of the RF model is beyond the scope of this manuscript. Further information on RF methods can be found in Baker and Ellison (2008) and Hamad et al. (2019).

Two different metrics are used to quantify the RF model performance in predicting Ch.-4 radiance values—the mean absolute error (MAE; W m−2 μm−1 sr−1) and the root-mean-square error (RMSE; W m−2 sr−1). The model has a MAE of 0.048 W m−2 μm−1 sr−1 and an RMSE of 0.078 W m−2 sr−1, and the total variability of Ch.-4 radiance explained by the model is 71% (r2 = 0.71). We consider these values to be sufficient to consider the model to be a capable predictor of Ch.-4 radiance values as a function of PWV and AMF.

Now that a model predicting the Ch.-4 radiance based on PWV and AMF has been created, we can examine their impacts independently. This is done varying PWV while holding AMF constant (we chose AMF = 5) and vice versa (choosing PWV = 3). Figure 8 shows the results of this exercise. It is clear that PWV, even at low values, does not contribute to the Ch.-4 radiance with any consistent relationship (Fig. 8a), while Ch.-4 radiance increases with increasing AMF (Fig. 8b). This suggests that, at least over the ocean, a PWV threshold is not required in the GOES detection algorithm. We recognize, however, that additional analysis will be required to facilitate thin cirrus detection over land.

Fig. 8.
Fig. 8.

Modeled Ch.-4 radiance (W m−2 sr−1) to (a) PWV (with AMF = 5) and (b) AMF (with PWV = 3 cm). Note that there are no AMF values less than ~0.2 in the full clear-sky sample used to create the model.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

b. Transparent cirrus detection algorithm

1) Differentiating clear sky and transparent cirrus

Having shown that PWV does not systematically modify the Ch.-4 radiance over ocean on the same order of magnitude as viewing/solar geometry, a threshold for differentiating clear sky from transparent cirrus based solely on AMF can be determined. Returning to Fig. 7, the dotted lines representing the regressions of the 1– and 2–standard deviation values above the mean radiance as a function of AMF are chosen as the radiance thresholds. The equations for the 1– and 2–standard deviation regressions for the full clear-sky sample (Fig. 7a) are, respectively,
Threshold=0.116537+(0.038114×AMF)and
Threshold=0.221887+(0.040561×AMF),
and the equations for the 1– and 2–standard deviation regressions for the HQ clear-sky sample (Fig. 7b) are, respectively,
Threshold=0.150679+(0.0258×AMF)and
Threshold=0.266235+(0.022984×AMF).

The process for using these thresholds to detect cloud is displayed as a flowchart in Fig. 9. First, AMFs are computed at the pixel level. A threshold for each pixel is then determined using one of Eqs. (2)(5). Last, the Ch.-4 radiance is compared with the threshold. Pixels with a radiance greater than the threshold are identified as transparent cirrus. Note that, for the purposes of this manuscript, we maintain that the overwhelming majority of such detections represent cirrus clouds, although we acknowledge that these layers could very well be lofted aerosol particles. However, as we will show, the sensitivity of the baseline detection is very high in optical depth space. The likelihood that lofted aerosols would be present in high enough densities to represent such optical depths over water, in the absence of any significant volcanic or pyroconvective events, is sufficiently low for this 2018 sample, and thus our analysis. Future application of the technique will require subsequent study, and likely additional spectral analysis of available ABI bands, to best distinguish ice from aerosol particulate matter.

Fig. 9.
Fig. 9.

A process flowchart of the GOES transparent cirrus detection algorithm and cloud optical depth estimation presented in this study.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

Naturally, any cloud at an altitude higher than the apparent Ch.-4 floor (dependent on AMF) will pass the radiance threshold and thus the clouds detected using this technique alone may not be cirrus clouds. Operational users will need to pair this technique with traditional cloud masking methods to create a full composite of the scene. Subsequent analysis in this study is performed using Eqs. (4) and (5), which were determined using the HQ samples, to limit the possibility of error due to collocation. Algorithm performance in terms of detection capability and COD is discussed in section 3c(1). First, the ability to estimate transparent cirrus COD using the 1.378-μm channel must be investigated.

2) Channel-4 radiance and cloud optical depth

With a calibrated method for detecting transparent cirrus established, we can develop a capability to derive COD. Figure 10 shows the scatter density of the CALIOP-identified single-layer “transparent cirrus” COD as a function of Ch.-4 radiance (Figs. 10a,b) and Ch.-4 radiance with the parallax correction (Figs. 10c,d) for the full cirrus sample (Figs. 10a,c) and the HQ cirrus sample (Figs. 10b,d). The sample size of the parallax-corrected sample is smaller because initially multiple GOES pixels were collocated to each CALIOP pixel. After parallax correction, these pixels were parallax corrected to the same GOES pixel, resulting in duplicates. The scatter density, which is indicated by the point coloring, is computed as the normalized number of points within ± 20% of the points COD and radiance values. This density should be treated as an estimate to be used as a visual aid. The dotted regression lines are computed using a linear least squares regression and the solid regression lines in Fig. 10 are computed using the Theil–Sen (Theil 1950; Sen 1968) slope estimator. This method, which selects the median of the slopes of the lines through each pair of points in the sample as the regression, is more robust with respect to outliers than a more widely used parametric regression such as least squares. However, the RMSEs for the regression lines are not significantly different (not shown; between ~0.615 and 0.65 in log–log space). The coefficient of determination (r2) values are higher for the parallax-corrected Ch.-4 radiance–COD relationships and the r2 for the HQ cirrus sample is the highest of the four relationships. Therefore, we recommend using this linear regression relationship for the HQ parallax-corrected cirrus sample when estimating COD; however, equations for both the full cirrus sample (Fig. 10c) and the HQ cirrus sample (Fig. 10d) are provided. As a first-order estimate, the r2 value of 0.44, which indicates that 44% of the variability in COD can be attributed to Ch.-4 radiance, suggests that there is a reasonable amount of skill in very simply using only the Ch.-4 radiance to predict the transparent cirrus COD. However, the complexity of this estimation method will need to be increased before being used quantitatively in downstream operational applications. Therefore, these COD estimates should be treated as semiquantitative in nature. A more advanced estimation method with a robust quantification of the estimate uncertainty will be developed in future work. The equations for the linear regressions in Figs. 10c and 10d are, respectively,
COD=10{0.83048+[log10(radiance)×0.710454]}and
COD=10{0.85082+[log10(radiance)×0.709307]}.
Fig. 10.
Fig. 10.

Scatter density of CALIOP 0.532-μm COD as a function of (a),(b) Ch.-4 radiance (W m−2 sr−1) and (c), (d) the parallax corrected Ch.-4 radiance (W m−2 sr−1) for (left) the full transparent cirrus sample and (right) the HQ cirrus sample. Point density is computed as the normalized number of pixels within 20% of (x, y) for each point and should be treated as a visual aid only. The solid regression line is computed using the Theil–Sen slope estimation method, and the dotted regression line is computed using a least squares multiple linear regression.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

These relationships can be used to estimate the COD of detected transparent cirrus and they are also useful to further estimate the detectability of transparent cirrus in terms of their COD. This is important because of the aforementioned cloud detection inefficiencies for thin cirrus clouds using passive radiometric sensors at CODs < 0.3 [or roughly half of all cirrus clouds according to Campbell et al. (2015)]. It should be stressed again that there is a large amount of uncertainty in these COD estimations. This is not surprising, as the estimates are made using the Ch.-4 radiance as the only predictor.

The process for estimating the transparent cirrus COD following detection is shown in Fig. 9. To estimate the minimum detectable COD, we first take Eqs. (4) and (5), which represent the radiance thresholds, to get the lowest detectable cloud radiance values. Then these radiance values are input into Eqs. (6) and (7) to convert the radiance values into COD. These CODs then represent the theoretical detection floor in COD space as a function of AMF. The results of this are shown in Fig. 11. The minimum detectable COD increases with AMF. While this is counterintuitive because a longer pathlength means more signal from the media (e.g., cirrus clouds) that the solar radiation is incident upon, in this case the path radiance, which should increase with the pathlength, is built into the threshold. Figure 11 suggests that using the COD regression from Fig. 10d [Eq. (7)] as opposed to the COD regression from Fig. 10c [Eq. (6)] has a larger impact on the minimum detectable COD than whether the 1– or 2–standard deviation radiance thresholds [Eqs. (4) and (5), respectively] are used. Therefore, it is suggested to use the 2–standard deviation radiance threshold, to conservatively preserve the most clear-sky pixels, paired with the HQ COD regression.

Fig. 11.
Fig. 11.

The theoretical transparent cirrus detection floor in COD space as a function of AMF. These values are computed by converting AMF values to radiance space using the regressions shown in Fig. 7 and then converting the radiance values to COD space using the linear regressions shown in Fig. 10. Red or black lines respectively indicate that the 2– or 1–standard deviation threshold was used in converting the AMFs to radiances. Solid or dotted lines respectively indicate that the full or HQ transparent cirrus sample regression was used to convert radiances to CODs.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

To make sure that the distributions of estimated COD are representative of a real sample (e.g., from CALIOP used as the reference) we perform the following exercise. Using real data from the full transparent cirrus sample (Fig. 10a), detection thresholds are computed on the pixel level using AMF and Eqs. (4) and (5). Then each radiance value is checked against its respective threshold value to determine if transparent cirrus is detected. Finally, CODs are estimated using the radiance values and Eq. (7). The probability density functions (PDFs) of the CALIOP CODs, the estimated CODs using Eq. (4), and the estimated CODs using Eq. (5) are shown in Fig. 12. The dark and light gray shading indicate the COD ranges for subvisual (COD < 0.03; Sassen et al. 1989) and transparent (what they term “translucent” or “optically thin”; 0.03 < COD < 0.3) cirrus, respectively. The distribution of the estimated CODs (Figs. 12b,c) are indeed representative of the CALIOP COD distribution (Fig. 12a).

Fig. 12.
Fig. 12.

PDFs of (a) CALIOP 0.532-μm column-integrated COD, (b) estimated COD using the 1–standard deviation radiance threshold, and the HQ COD regression, and (c) estimated COD using the 2–standard deviation radiance threshold. COD ranges for subvisual and transparent cirrus are shaded in dark gray and light gray, respectively.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

These PDFs are consistent with Fig. 11, which showed that the theoretical minimum detectable COD is between 0.055 and 0.06 for both the 1– and 2–standard deviation thresholds, depending on AMF. Also note that the PDF for the 1–standard deviation threshold has a larger percentage of lower COD values (e.g., 0.6–0.7), which is also consistent with Fig. 11. Now that this quantified model for estimating transparent COD from the Ch.-4 radiance has been established, the performance of the detection algorithm as a whole can be evaluated.

The method for estimating transparent cirrus COD presented here is simple in comparison with those presented in Meyer et al. (2004) and Meyer and Platnick (2010). Those studies utilize multiple channels and lookup tables built with radiative transfer model calculations. Even with the increased complexity, Meyer and Platnick (2010) still found COD retrieval uncertainties for thin cirrus clouds (COD < 1) to be as high as 90%. However, with the amount of skill the Ch.-4 radiance alone has in estimating the COD, it seems likely that improvements can be made.

c. Algorithm performance

1) Bulk validation

A means for evaluating transparent cirrus detection and determining its efficacy for being implemented operationally in the future is with a confusion matrix. Table 3 shows the total number of pixels identified by CALIOP as clear sky (full sample) and transparent cirrus (full samples, Figs. 4a and 10c, respectively), after filtering the samples as outlined in section 2c(1), and the number of pixels in each that the GOES algorithm correctly (hit) and incorrectly (miss) labeled using both Eqs. (4) and (5). The values shown in Table 3 were computed independently (i.e., clear-sky pixels mislabeled by the GOES algorithm were not counted as hits for transparent cirrus and vice versa). The goal here is for the transparent cirrus detection algorithm to correctly identify as many cloudy pixels as possible while minimizing the number of clear-sky pixels misidentified as cloudy.

Table 3.

Confusion matrix for transparent cirrus and clear-sky pixels (full sample) as determined by CALIOP for the GOES transparent cirrus detection algorithm. Counts based on radiance thresholds holds from both Eqs. (4) and (5) are shown. Note that clear-sky pixels misidentified as transparent cirrus are not included in transparent cirrus hit counts and vice versa.

Table 3.

It can be seen that, when using Eq. (4) (HQ sample, 1–standard deviation regression) to compute the radiance thresholds, over 90% of pixels determined by CALOP to contain transparent cirrus were correctly identified by the GOES algorithm. However, 8% of clear-sky pixels were mislabeled. Meanwhile, using Eq. (5) (HQ sample, 2–standard deviations regression) to compute the radiance thresholds results in 84% of cloudy pixels being correctly identified and less than 4% of clear-sky pixels being mislabeled. This suggests that the choice of Eq. (4) or Eq. (5) depends on the application. Users interested in reliable cloud detection may want to use the more aggressive radiance threshold achieved by using Eq. (4), while users wanting to avoid contaminated clear-sky pixels may want to use Eq. (5) to compute the radiance thresholds.

The detectability of transparent cirrus in terms of COD is explored in Fig. 13, which provides the PDFs of CALIOP COD for the clouds detected by the GOES algorithm using Eqs. (4) (Fig. 13a) and (5) (Fig. 13b). The red lines indicate the fraction of clouds detected by the GOES algorithm as a function of CALIOP COD, which correspond to the right-side y axes. Overall, even when using the more conservative 2–standard deviation radiance threshold (Fig. 13b), the GOES algorithm detects nearly 45% of subvisual cirrus (COD < 0.03), over 80% of transparent cirrus (0.03 < COD < 0.3), and 90% of opaque cirrus (COD > 0.3). We do not limit the discussion of results here to cirrus strictly having a COD of less than 0.3 and instead use the CALIOP-based definition of transparent cirrus, which includes some clouds with higher CODs.

Fig. 13.
Fig. 13.

PDFs of collocated CALIOP 0.532-μm column-integrated COD for GOES pixels with detected cirrus using the (a) 1–standard deviation radiance threshold and (b) 2–standard deviation radiance threshold. COD ranges for subvisual and transparent cirrus are shaded in dark gray and light gray, respectively. The red line indicates the fraction of transparent cirrus detected by the GOES algorithm with respect to CALIOP as a function of CALIOP COD.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

It is also apparent that the GOES algorithm detects cirrus clouds down to CODs of 0.01 (and likely even lower) at a rate of over 40%, even when using the more conservative radiance threshold, despite the theoretical detection floor being around 0.06. This is due to two factors. The first is that the COD regressions used to compute the theoretical detection floors for the algorithm presented here overestimate COD at the low end of the spectrum, resulting in a theoretical floor that is higher than in reality. The second is that, as seen in Fig. 10c, there are many transparent cirrus that have CODs less than 0.01 according to CALIOP but have relatively high Ch.-4 radiance values, which allows for them to be detected by the GOES algorithm. Whether these high radiance values are due to issues with collocation, the presence of dust in addition to cirrus, increased pathlength due to oblique geometry, or something else is presently unknown. Figure 10 suggests that the number of these pixels is reduced in the HQ transparent cirrus sample (Fig. 10d), which limits collocation to within 1 min of a GOES scan. However, this could also be due to the overall reduction in sample size.

2) Example application

GOES-16 ABI simulated true color, Ch.-4 radiance, transparent cirrus mask, and estimated COD images are shown for a sector encompassing the western Atlantic for 1830:49 UTC 16 August 2018 in Fig. 14. This region often contains cirrus clouds that originate from convective anvil blow off, synoptic-scale forcing, and processes in the tropical tropopause transition layer (TTL; Virts et al. 2010). Figures 14a and 14b, indicate a variety of meteorological features, cloud types, and the general presence of overlying cirrus clouds in portions of the region. The quantitative cirrus detection algorithm in Fig. 14c, however, reveals that expansive areas of transparent cirrus are actually present, especially in the central Gulf of Mexico and the southern Caribbean Sea near the coasts of Panama and Colombia. This “cirrus mask” identifies the cirrus type based on the COD estimates displayed in Fig. 14d. The clouds marked as “thin cirrus” have a COD below 0.3 and are not immediately evident in the standard true-color or raw 1.378-μm imagery over the region (Figs. 14a,b). These cirrus clouds may otherwise be undetected by traditional cloud masks that rely on other channels and significantly contaminate operational applications, such as MODIS SST retrievals (Marquis et al. 2017).

Fig. 14.
Fig. 14.

GOES-16 (a) simulated true color, (b) 1.378-μm reflectance, (c) cirrus mask, and (d) estimated cloud optical depth based on the GOES transparent cirrus detection algorithm using the 2–standard deviation radiance threshold for the scan performed at 1830:49 UTC 16 Aug 2018. The path of the CALIOP overpass is shown in red.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

Figure 15 shows the CALIOP 532 nm backscatter curtain, GOES and CALIOP CTH and COD, GOES Ch.-4 radiance, and the GOES 11–12-μm brightness temperature difference (BTD) for the corresponding CALIOP ground track in Fig. 14 (over ocean only). The BTD is a traditional cloud detection test and is shown for comparison with the Ch.-4 radiance. The magenta lines in the middle panel indicate where the transparent cirrus detection algorithm, using the 2–standard deviation threshold, indicates the presence of cloud. A couple areas of interest will be discussed. First, between 12.5° and 13° latitude (highlighted in cyan), the operational GOES products fail to detect the high clouds and incorrectly place the CTH near the surface. While the transparent cirrus algorithm also misses a large portion of this cloud, it does detect some. According to CALIOP, these clouds have COD around 0.2. Between 13.5° and 15° latitude (highlighted in yellow), the operational GOES algorithm indicates the presence of high clouds, while CALIOP does not. The transparent cirrus algorithm does not detect high clouds, except in two instances around 14° latitude where the CALIOP attenuated backscatter is enhanced locally. A tenuous layer is detected by CALIOP, but it is classified as aerosol with little confidence and therefore not shown in Fig. 15. Between 22° and 22.7° latitude, the operational GOES algorithm detects high clouds, while no features are detected by CALIOP and no high clouds are detected by our algorithm. Generally, CALIOP, the GOES operational products, and the transparent cirrus detection algorithm agree well on the presence of high clouds, and there is no an instance of the transparent cirrus detection algorithm failing in this specific case. In the example, most of the transparent clouds detected by the algorithm are high cirrus clouds according to CALIOP. The only exception is that the fairly high cloud with a top altitude of ~9.7 km at 15° latitude is a supercooled water cloud according to CALIOP. In instances where the atmosphere is relatively dry and midlevel clouds are present this may not be the case.

Fig. 15.
Fig. 15.

(top) CALIOP 2D 532-nm attenuated backscatter curtain corresponding to the CALIOP ground track shown in Fig. 14. (middle) The CALIOP cloud-top height (CTH; blue stars), CALIOP cloud optical depth (COD; green stars), GOES CTH (black dots), and GOES COD (red dots) for the same CALIOP track. (bottom) The GOES Ch.-4 radiance (red dots) and the GOES 11–12-μm brightness temperature difference (black dots). The magenta lines in the middle panel show where the transparent cirrus algorithm indicates the presence of cloud. Areas of interest are highlighted and outlined by the cyan and yellow vertical lines. Note that white areas in the top panel are over land and data are therefore not shown in any of the panels.

Citation: Journal of Atmospheric and Oceanic Technology 38, 6; 10.1175/JTECH-D-20-0130.1

The bottom panel in Fig. 15 suggests that, as compared with the Ch.-4 radiance, the BTD is not very sensitive to the presence of thin high clouds. BTD values are generally around 3 K regardless of whether there is high cloud, low cloud, or clear sky. When thicker clouds are present, the BTD either increases (between 15° and 16° latitude) or decreases (between 29° and 32° latitude) sharply. In contrast, the Ch.-4 radiance generally remains below 0.2 W m−2 μm−1 sr−1 unless high clouds are present.

d. Limitations

This algorithm is not a replacement for traditional passive-radiometric-based cloud mask algorithms that utilize numerous bands that span the visible-thermal IR spectrum. While it is likely that most, if not all, opaque clouds (COD > 0.3) in the mid- and upper troposphere would be detected by this algorithm, it cannot be forgotten that, depending on the amount of water vapor in the atmospheric column, the 1.378-μm channel often does not receive signal from below 3 km in altitude. This means that, unless there is also a layer of high clouds over it, a low-altitude cloud layer often will not be detected, especially given the locations of semipermanent stratocumulus layers.

Cases of a transparent cirrus layer over another low-altitude cloud layer were not explored here. While largely outside the objectives of establishing a simple transparent cirrus detection for the GOES-16 ABI using collocated CALIOP data and exploring potential COD estimation, these cases are still important as they are relatively common in the atmosphere. We chose to leave more complex situations such as these for future work in part due to the complexities introduced by accounting for PWV in the algorithm, such as selecting an additional PWV dataset that is readily available for operational use and covers the entire GOES disk. This is also why we limited this study to over ocean only. Over a dark surface such as the ocean, contamination via radiation reflected by the surface under low PWV conditions is likely relatively unimportant. However, over landmasses where it is more likely that PWV values are lower, surfaces are brighter, and elevations are higher, PWV must be accounted for.

As mentioned in section 2b(3), it was decided to leave more detailed study of the impacts of aerosol contamination on this algorithm to future work. The study period for this study was chosen specifically to avoid volcanic activity or large pyrocumulonimbus events. These phenomena produce thick, high-altitude aerosol plumes that would undoubtedly impact the GOES transparent cirrus detection algorithm. Therefore, most aerosol contamination is likely confined to the lower troposphere where, depending on atmospheric PWV, the signal does not reach the satellite.

4. Summary and conclusions

This study developed and evaluated a maritime transparent cirrus cloud detection algorithm using the GOES-16 Advanced Baseline Imager channel 4 radiance (1.378 μm) based on collocated CALIOP profiling. Using a random-forest machine-learning model, we showed that the clear-sky radiance is primarily a function of airmass factor. While the atmospheric precipitable water vapor content undoubtedly has an effect on the Ch.-4 radiance, this model suggested that it does not play a large role over the ocean. This is because the surface elevation is low and the ocean is a relatively unreflective surface.

A Ch.-4 radiance threshold to differentiate between clear sky and transparent cirrus was developed based on the relationship between clear-sky radiance and AMF. This formed the basis of the transparent cirrus detection algorithm. The relationship between the Ch.-4 radiance and CALIOP cloud optical depth was then used to estimate the COD. Last, a baseline for thin cirrus detection in terms of COD and a semiquantitative retrieval COD were developed, and the algorithms performance was examined. It was shown that the algorithm presented here can detect cirrus clouds identified by CALIOP at rates greater than 80% for most CODs down to 0.05, while preserving over 96% of clear-sky pixels.

The focus of this study was a proof of concept for simple transparent cirrus detection; therefore, many complications were left to be dealt with in the future. This algorithm is not a replacement for traditional passive-radiometric-based cloud mask algorithms. Because the 1.378-μm channel is located within a water vapor absorption band, low-altitude cloud layers often will not be detected and cases of a transparent cirrus layer over another low-altitude cloud layer were not explored here. Though a simple GOES ABI-based aerosol screening was performed, an in-depth quantification of the effects of aerosols on the Ch.-4 radiance and this algorithm was not. PWV was not accounted for after we showed that its impact was limited over the ocean. All of these limitations will be the focus of future work.

The simple algorithm presented here shows that passive detection of transparent cirrus is possible and can be easily improved by adding additional steps. When using the more conservative radiance threshold equation, 84% of transparent cirrus identified by CALIOP are detected at the expense of less than 4% of clear-sky pixels. Studies such as Marquis et al. (2017) and Chew et al. (2011) found significant transparent cirrus contaminations in sea surface temperature and aerosol optical depth retrievals, respectively. The algorithm presented here can be implemented to remove a large amount of transparent cirrus contamination in these and other operational applications.

This study lays the groundwork for a wide array of research opportunities. For example, future expansion of the GOES transparent cirrus detection algorithm to over land begins with the development of a more complex algorithm for overland use, accounting for surface reflectivity elevation and atmospheric PWV. This requires a more in-depth look at the effects of PWV, land surface type, and surface elevation on the GOES-16 ABI Ch.-4 radiance. A more robust transparent COD retrieval that is suitable for assimilation purposes will be extremely useful to the community, which means that the uncertainty in the COD retrieval presented needs to be carefully quantified. Following these tasks, cirrus cloud climatology can be revisited from a passive perspective and global radiation studies can begin.

Acknowledgments

This research was supported by the Office of Naval Research Grant N0001420WX00481 (R. Ferek and D. Eleuterio). We graciously acknowledge the achievements of Naval Research Laboratory colleague Dr. Bo-Cai Gao in his groundbreaking work demonstrating and advocating for the potential of the 1.378-μm band as a means for distinguishing thin cirrus clouds. We thank Dr. Will Komaromi for his photograph of the optical phenomena shown in Fig. 1.

Data availability statement

GOES-16 data (radiances and ADP) used in this study are available at https://doi.org/10.7289/V5BV7DSR. CALIOP cloud profiles data are available at https://doi.org/10.5067/CALIOP/CALIPSO/LID_L2_05KMCPRO-STANDARD-V4-20. Information on the GMTED data and access can be found at https://www.usgs.gov/land-resources/eros/coastal-changes-and-impacts/gmted2010?qt-science_support_page_related_con=0#qt-science_support_page_related_con.

REFERENCES

  • Ackerman, S. A., R. E. Holz, R. Frey, E. W. Eloranta, B. C. Maddux, and M. McGill, 2008: Cloud detection with MODIS. Part II: Validation. J. Atmos. Oceanic Technol., 25, 10731086, https://doi.org/10.1175/2007JTECHA1053.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baker, L., and D. Ellison, 2008: The wisdom of crowds—Ensembles and modules in environmental modelling. Geoderma, 147, 17, https://doi.org/10.1016/j.geoderma.2008.07.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bartlett, B., J. Casey, F. Padula, A. Pearlman, D. Pogorzala, and C. Cao, 2018: Independent validation of the Advanced Baseline Imager (ABI) on NOAA’s GOES-16: Post-launch ABI airborne science field campaign results. Earth Obs. Syst., 10764, 107640H, https://doi.org/10.1117/12.2323672.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Breiman, L., 1996: Bagging predictors. Mach. Learn., 24, 123140, https://doi.org/10.1007/BF00058655.

  • Breiman, L., 2001: Random forests. Mach. Learn., 45, 532, https://doi.org/10.1023/A:1010933404324.

  • Campbell, J. R., and Coauthors, 2012: Evaluating nighttime CALIOP 0.532 μm aerosol optical depth and extinction coefficient retrievals. Atmos. Meas. Tech., 5, 27472794, https://doi.org/10.5194/AMT-5-2143-2012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Campbell, J. R., M. A. Vaughan, M. Oo, R. E. Holz, J. R. Lewis, and J. E. Welton, 2015: Distinguishing cirrus cloud presence in autonomous lidar measurements. Atmos. Meas. Tech., 8, 435449, https://doi.org/10.5194/amt-8-435-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chew, B. N., J. R. Campbell, J. S. Reid, D. M. Giles, E. J. Welton, S. V. Salinas, and S. C. Liew, 2011: Tropical cirrus cloud contamination in sun photometer data. Atmos. Environ., 45, 67246731, https://doi.org/10.1016/j.atmosenv.2011.08.017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Danielson, J. J., and D. B. Gesch, 2011: Global multi-resolution terrain elevation data 2010 (GMTED2010). U.S. Geological Survey Rep., 26 pp., https://doi.org/10.3133/OFR20111073.

    • Crossref
    • Export Citation
  • Dessler, A. E., and P. Yang, 2003: The distribution of tropical thin cirrus clouds inferred from Terra MODIS data. J. Climate, 16, 12411247, https://doi.org/10.1175/1520-0442(2003)16<1241:TDOTTC>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gao, B. C., and Y. J. Kaufman, 1995: Selection of the 1.375-μm MODIS channel for remote sensing of cirrus clouds and stratospheric aerosols from space. J. Atmos. Sci., 52, 42314237, https://doi.org/10.1175/1520-0469(1995)052<4231:SOTMCF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gao, B. C., A. F. Goetz, and W. J. Wiscombe, 1993: Cirrus cloud detection from airborne imaging spectrometer data using the 1.38 μm water vapor band. Geophys. Res. Lett., 20, 301304, https://doi.org/10.1029/93GL00106.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gelaro, R., and Coauthors, 2017: The Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2). J. Climate, 30, 54195454, https://doi.org/10.1175/JCLI-D-16-0758.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamad, K., M. A. Khalil, and A. R. Alozi, 2019: Predicting freeway incident duration using machine learning. Int. J. Intell. Transp. Syst. Res., 18, 367380, https://doi.org/10.1007/S13177-019-00205-1.

    • Search Google Scholar
    • Export Citation
  • Heidinger, A. K., 2010: ABI cloud mask. NOAA/NESDIS/Center for Satellite Applications and Research Algorithm Theoretical Basis Doc., 93 pp.

  • Heidinger, A. K., 2012: ABI cloud height. NOAA/NESDIS/Center for Satellite Applications and Research Algorithm Theoretical Basis Doc., 79 pp.

  • Lee, J., P. Yang, A. E. Dessler, B. C. Gao, and S. Platnick, 2009: Distribution and radiative forcing of tropical thin cirrus clouds. J. Atmos. Sci., 66, 37213731, https://doi.org/10.1175/2009JAS3183.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z., and Coauthors, 2019: Discriminating between clouds and aerosols in the CALIOP version 4.1 data products. Atmos. Meas. Tech., 12, 703734, https://doi.org/10.5194/amt-12-703-2019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lolli, S., and P. Di Girolamo, 2015: Principal component analysis approach to evaluate instrument performances in developing a cost-effective reliable instrument network for atmospheric measurements. J. Atmos. Oceanic Technol., 32, 16421649, https://doi.org/10.1175/JTECH-D-15-0085.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lolli, S., and Coauthors, 2017: Daytime top-of-the-atmosphere cirrus cloud radiative forcing properties at Singapore. J. Appl. Meteor. Climatol., 56, 12491257, https://doi.org/10.1175/JAMC-D-16-0262.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mace, G. G., and Q. Zhang, 2014: The CloudSat radar-lidar geometrical profile product (RL-GeoProf): Updates, improvements, and selected results. J. Geophys. Res. Atmos., 119, 94419462, https://doi.org/10.1002/2013JD021374.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marquis, J. W., A. S. Bogdanoff, J. R. Campbell, J. A. Cummings, D. L. Westphal, N. J. Smith, and J. Zhang, 2017: Estimating infrared radiometric satellite sea surface temperature retrieval cold biases in the tropics due to unscreened optically thin cirrus clouds. J. Atmos. Oceanic Technol., 34, 355373, https://doi.org/10.1175/JTECH-D-15-0226.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meyer, K., and S. Platnick, 2010: Utilizing the MODIS 1.38 μm channel for cirrus cloud optical thickness retrievals: Algorithm and retrieval uncertainties. J. Geophys. Res., 115, D24209, https://doi.org/10.1029/2010JD014872.

    • Search Google Scholar
    • Export Citation
  • Meyer, K., P. Yang, and B. C. Gao, 2004: Optical thickness of tropical cirrus clouds derived from the MODIS 0.66 and 1.375-μm channels. IEEE Trans. Geosci. Remote Sens., 42, 833841, https://doi.org/10.1109/TGRS.2003.818939.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Miller, S. D., C. C. Schmidt, T. J. Schmit, and D. W. Hillger, 2012: A case for natural colour imagery from geostationary satellites, and an approximation for the GOES-R ABI. Int. J. Remote Sens., 33, 39994028, https://doi.org/10.1080/01431161.2011.637529.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Miller, S. D., T. J. Schmit, C. J. Seaman, D. T. Lindsey, M. M. Gunshor, R. A. Kors, Y. Sumida, and D. W. Hillger, 2016: A sight for sore eyes: The return of true color to geostationary satellites. Bull. Amer. Meteor. Soc., 97, 18031816, https://doi.org/10.1175/BAMS-D-15-00154.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Miller, S. D., M. A. Rogers, J. M. Haynes, M. Sengupta, and A. K. Heidinger, 2018: Short-term solar irradiance forecasting via satellite/model coupling. Sol. Energy, 168, 102117, https://doi.org/10.1016/j.solener.2017.11.049.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Minnis, P., and Coauthors, 2008: Cloud detection in nonpolar regions for CERES using TRMM VIRS and Terra and Aqua MODIS data. IEEE Trans. Geosci. Remote Sens., 46, 38573884, https://doi.org/10.1109/TGRS.2008.2001351.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • NOAA/NESDIS, 2018: ABI aerosol detection product. NOAA/NESDIS/Center for Satellite Applications and Research Algorithm Theoretical Basis Doc., 70 pp.

  • Palmer, P. I., and Coauthors, 2001: Air mass factor formulation for spectroscopic measurements from satellites: Application to formaldehyde retrievals from the Global Ozone Monitoring Experiment. J. Geophys. Res., 106, 14 53914 550, https://doi.org/10.1029/2000JD900772.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peterson, D. A., J. R. Campbell, E. J. Hyer, M. D. Fromm, G. P. Kablick, J. H. Cossuth, and M. T. DeLand, 2018: Wildfire-driven thunderstorms cause a volcano-like stratospheric injection of smoke. npj Climate Atmos. Sci., 1, 30, https://doi.org/10.1038/s41612-018-0039-3.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rossow, W. B., and R. A. Schiffer, 1999: Advances in understanding clouds from ISCCP. Bull. Amer. Meteor. Soc., 80, 22612288, https://doi.org/10.1175/1520-0477(1999)080<2261:AIUCFI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sassen, K., and B. S. Cho, 1992: Subvisual-thin cirrus lidar dataset for satellite verification and climatological research. J. Appl. Meteor., 31, 12751285, https://doi.org/10.1175/1520-0450(1992)031<1275:STCLDF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sassen, K., M. K. Griffin, and G. C. Dodd, 1989: Optical scattering and microphysical properties of subvisual cirrus clouds, and climatic implications. J. Appl. Meteor., 28, 9198, https://doi.org/10.1175/1520-0450(1989)028<0091:OSAMPO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schmit, T. J., P. Griffith, M. M. Gunshor, J. M. Daniels, S. J. Goodman, and W. J. Lebair, 2017: A closer look at the ABI on the GOES-R series. Bull. Amer. Meteor. Soc., 98, 681698, https://doi.org/10.1175/BAMS-D-15-00230.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sen, P. K., 1968: Estimates of the regression coefficient based on Kendall’s tau. J. Amer. Stat. Assoc., 63, 13791389, https://doi.org/10.1080/01621459.1968.10480934.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stubenrauch, C. J., and Coauthors, 2013: Assessment of global cloud datasets from satellites: Project and database initiated by the GEWEX radiation panel. Bull. Amer. Meteor. Soc., 94, 10311049, https://doi.org/10.1175/BAMS-D-12-00117.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Theil, H., 1950: A rank-invariant method of linear and polynomial regression analysis. Indagationes Math., 12, 173.

  • Vaughan, M. A., and Coauthors, 2020: Cloud–aerosol lidar infrared pathfinder satellite observations: Data management system data products catalog. NASA Doc. PC-SCI-503, 256 pp., https://www-calipso.larc.nasa.gov/products/CALIPSO_DPC_Rev4x92.pdf.

  • Virts, K. S., J. M. Wallace, Q. Fu, and T. P. Ackerman, 2010: Tropical tropopause transition layer cirrus as represented by CALIPSO lidar observations. J. Atmos. Sci., 67, 31133129, https://doi.org/10.1175/2010JAS3412.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Walther, A., W. Straka, and A. K. Heidinger, 2013: ABI algorithm theoretical basis document for daytime cloud optical and microphysical properties (DCOMP). NOAA/NESDIS/Center for Satellite Applications and Research Algorithm Theoretical Basis Doc., 66 pp., https://www.star.nesdis.noaa.gov/goesr/documents/ATBDs/Baseline/ATBD_GOES-R_Cloud_DCOMP_v3.0_Jun2013.pdf.

  • Wang, C., and X. Huang, 2014: Parallax correction in the analysis of multiple satellite data sets. IEEE Geosci. Remote Sens. Lett., 11, 965969, https://doi.org/10.1109/LGRS.2013.2283573.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, C., P. Yang, A. Dessler, B. A. Baum, and Y. Hu, 2014: Estimation of the cirrus cloud scattering phase function from satellite observations. J. Quant. Spectrosc. Radiant. Transfer, 138, 3649, https://doi.org/10.1016/j.jqsrt.2014.02.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Winker, D. M., W. H. Hunt, and M. J. McGill, 2007: Initial performance assessment of CALIOP. Geophys. Res. Lett., 34, L19803, https://doi.org/10.1029/2007GL030135.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Winker, D. M., and Coauthors, 2009: Overview of the CALIPSO mission and CALIOP data processing algorithms. J. Atmos. Oceanic Technol., 26, 23102323, https://doi.org/10.1175/2009JTECHA1281.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Young, S. A., and M. A. Vaughan, 2009: The retrieval of profiles of particulate extinction from Cloud-Aerosol Lidar Infrared Pathfinder Satellite Observations (CALIPSO) data: Algorithm description. J. Atmos. Oceanic Technol., 26, 11051119, https://doi.org/10.1175/2008JTECHA1221.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Young, S. A., M. A. Vaughan, A. Garnier, J. L. Tackett, J. D. Lambeth, and K. A. Powell, 2018: Extinction and optical depth retrievals for CALIPSO’s version 4 data release. Atmos. Meas. Tech., 11, 57015727, https://doi.org/10.5194/amt-11-5701-2018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yu, F., X. Wu, X. Shao, B. Efremova, H. Yoo, H. Qian, and B. Iacovazzi, 2017: Early radiometric calibration performances of GOES-16 Advanced Baseline Imager. Earth Obs. Syst., 10402, 104020S, https://doi.org/10.1117/12.2275195.

    • Crossref
    • Search Google Scholar
    • Export Citation
Save
  • Ackerman, S. A., R. E. Holz, R. Frey, E. W. Eloranta, B. C. Maddux, and M. McGill, 2008: Cloud detection with MODIS. Part II: Validation. J. Atmos. Oceanic Technol., 25, 10731086, https://doi.org/10.1175/2007JTECHA1053.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baker, L., and D. Ellison, 2008: The wisdom of crowds—Ensembles and modules in environmental modelling. Geoderma, 147, 17, https://doi.org/10.1016/j.geoderma.2008.07.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bartlett, B., J. Casey, F. Padula, A. Pearlman, D. Pogorzala, and C. Cao, 2018: Independent validation of the Advanced Baseline Imager (ABI) on NOAA’s GOES-16: Post-launch ABI airborne science field campaign results. Earth Obs. Syst., 10764, 107640H, https://doi.org/10.1117/12.2323672.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Breiman, L., 1996: Bagging predictors. Mach. Learn., 24, 123140, https://doi.org/10.1007/BF00058655.

  • Breiman, L., 2001: Random forests. Mach. Learn., 45, 532, https://doi.org/10.1023/A:1010933404324.

  • Campbell, J. R., and Coauthors, 2012: Evaluating nighttime CALIOP 0.532 μm aerosol optical depth and extinction coefficient retrievals. Atmos. Meas. Tech., 5, 27472794, https://doi.org/10.5194/AMT-5-2143-2012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Campbell, J. R., M. A. Vaughan, M. Oo, R. E. Holz, J. R. Lewis, and J. E. Welton, 2015: Distinguishing cirrus cloud presence in autonomous lidar measurements. Atmos. Meas. Tech., 8, 435449, https://doi.org/10.5194/amt-8-435-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chew, B. N., J. R. Campbell, J. S. Reid, D. M. Giles, E. J. Welton, S. V. Salinas, and S. C. Liew, 2011: Tropical cirrus cloud contamination in sun photometer data. Atmos. Environ., 45, 67246731, https://doi.org/10.1016/j.atmosenv.2011.08.017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Danielson, J. J., and D. B. Gesch, 2011: Global multi-resolution terrain elevation data 2010 (GMTED2010). U.S. Geological Survey Rep., 26 pp., https://doi.org/10.3133/OFR20111073.

    • Crossref
    • Export Citation
  • Dessler, A. E., and P. Yang, 2003: The distribution of tropical thin cirrus clouds inferred from Terra MODIS data. J. Climate, 16, 12411247, https://doi.org/10.1175/1520-0442(2003)16<1241:TDOTTC>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gao, B. C., and Y. J. Kaufman, 1995: Selection of the 1.375-μm MODIS channel for remote sensing of cirrus clouds and stratospheric aerosols from space. J. Atmos. Sci., 52, 42314237, https://doi.org/10.1175/1520-0469(1995)052<4231:SOTMCF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gao, B. C., A. F. Goetz, and W. J. Wiscombe, 1993: Cirrus cloud detection from airborne imaging spectrometer data using the 1.38 μm water vapor band. Geophys. Res. Lett., 20, 301304, https://doi.org/10.1029/93GL00106.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gelaro, R., and Coauthors, 2017: The Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2). J. Climate, 30, 54195454, https://doi.org/10.1175/JCLI-D-16-0758.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamad, K., M. A. Khalil, and A. R. Alozi, 2019: Predicting freeway incident duration using machine learning. Int. J. Intell. Transp. Syst. Res., 18, 367380, https://doi.org/10.1007/S13177-019-00205-1.

    • Search Google Scholar
    • Export Citation
  • Heidinger, A. K., 2010: ABI cloud mask. NOAA/NESDIS/Center for Satellite Applications and Research Algorithm Theoretical Basis Doc., 93 pp.

  • Heidinger, A. K., 2012: ABI cloud height. NOAA/NESDIS/Center for Satellite Applications and Research Algorithm Theoretical Basis Doc., 79 pp.

  • Lee, J., P. Yang, A. E. Dessler, B. C. Gao, and S. Platnick, 2009: Distribution and radiative forcing of tropical thin cirrus clouds. J. Atmos. Sci., 66, 37213731, https://doi.org/10.1175/2009JAS3183.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z., and Coauthors, 2019: Discriminating between clouds and aerosols in the CALIOP version 4.1 data products. Atmos. Meas. Tech., 12, 703734, https://doi.org/10.5194/amt-12-703-2019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lolli, S., and P. Di Girolamo, 2015: Principal component analysis approach to evaluate instrument performances in developing a cost-effective reliable instrument network for atmospheric measurements. J. Atmos. Oceanic Technol., 32, 16421649, https://doi.org/10.1175/JTECH-D-15-0085.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lolli, S., and Coauthors, 2017: Daytime top-of-the-atmosphere cirrus cloud radiative forcing properties at Singapore. J. Appl. Meteor. Climatol., 56, 12491257, https://doi.org/10.1175/JAMC-D-16-0262.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mace, G. G., and Q. Zhang, 2014: The CloudSat radar-lidar geometrical profile product (RL-GeoProf): Updates, improvements, and selected results. J. Geophys. Res. Atmos., 119, 94419462, https://doi.org/10.1002/2013JD021374.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marquis, J. W., A. S. Bogdanoff, J. R. Campbell, J. A. Cummings, D. L. Westphal, N. J. Smith, and J. Zhang, 2017: Estimating infrared radiometric satellite sea surface temperature retrieval cold biases in the tropics due to unscreened optically thin cirrus clouds. J. Atmos. Oceanic Technol., 34, 355373, https://doi.org/10.1175/JTECH-D-15-0226.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meyer, K., and S. Platnick, 2010: Utilizing the MODIS 1.38 μm channel for cirrus cloud optical thickness retrievals: Algorithm and retrieval uncertainties. J. Geophys. Res., 115, D24209, https://doi.org/10.1029/2010JD014872.

    • Search Google Scholar
    • Export Citation
  • Meyer, K., P. Yang, and B. C. Gao, 2004: Optical thickness of tropical cirrus clouds derived from the MODIS 0.66 and 1.375-μm channels. IEEE Trans. Geosci. Remote Sens., 42, 833841, https://doi.org/10.1109/TGRS.2003.818939.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Miller, S. D., C. C. Schmidt, T. J. Schmit, and D. W. Hillger, 2012: A case for natural colour imagery from geostationary satellites, and an approximation for the GOES-R ABI. Int. J. Remote Sens., 33, 39994028, https://doi.org/10.1080/01431161.2011.637529.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Miller, S. D., T. J. Schmit, C. J. Seaman, D. T. Lindsey, M. M. Gunshor, R. A. Kors, Y. Sumida, and D. W. Hillger, 2016: A sight for sore eyes: The return of true color to geostationary satellites. Bull. Amer. Meteor. Soc., 97, 18031816, https://doi.org/10.1175/BAMS-D-15-00154.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Miller, S. D., M. A. Rogers, J. M. Haynes, M. Sengupta, and A. K. Heidinger, 2018: Short-term solar irradiance forecasting via satellite/model coupling. Sol. Energy, 168, 102117, https://doi.org/10.1016/j.solener.2017.11.049.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Minnis, P., and Coauthors, 2008: Cloud detection in nonpolar regions for CERES using TRMM VIRS and Terra and Aqua MODIS data. IEEE Trans. Geosci. Remote Sens., 46, 38573884, https://doi.org/10.1109/TGRS.2008.2001351.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • NOAA/NESDIS, 2018: ABI aerosol detection product. NOAA/NESDIS/Center for Satellite Applications and Research Algorithm Theoretical Basis Doc., 70 pp.

  • Palmer, P. I., and Coauthors, 2001: Air mass factor formulation for spectroscopic measurements from satellites: Application to formaldehyde retrievals from the Global Ozone Monitoring Experiment. J. Geophys. Res., 106, 14 53914 550, https://doi.org/10.1029/2000JD900772.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peterson, D. A., J. R. Campbell, E. J. Hyer, M. D. Fromm, G. P. Kablick, J. H. Cossuth, and M. T. DeLand, 2018: Wildfire-driven thunderstorms cause a volcano-like stratospheric injection of smoke. npj Climate Atmos. Sci., 1, 30, https://doi.org/10.1038/s41612-018-0039-3.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rossow, W. B., and R. A. Schiffer, 1999: Advances in understanding clouds from ISCCP. Bull. Amer. Meteor. Soc., 80, 22612288, https://doi.org/10.1175/1520-0477(1999)080<2261:AIUCFI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sassen, K., and B. S. Cho, 1992: Subvisual-thin cirrus lidar dataset for satellite verification and climatological research. J. Appl. Meteor., 31, 12751285, https://doi.org/10.1175/1520-0450(1992)031<1275:STCLDF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sassen, K., M. K. Griffin, and G. C. Dodd, 1989: Optical scattering and microphysical properties of subvisual cirrus clouds, and climatic implications. J. Appl. Meteor., 28, 9198, https://doi.org/10.1175/1520-0450(1989)028<0091:OSAMPO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schmit, T. J., P. Griffith, M. M. Gunshor, J. M. Daniels, S. J. Goodman, and W. J. Lebair, 2017: A closer look at the ABI on the GOES-R series. Bull. Amer. Meteor. Soc., 98, 681698, https://doi.org/10.1175/BAMS-D-15-00230.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sen, P. K., 1968: Estimates of the regression coefficient based on Kendall’s tau. J. Amer. Stat. Assoc., 63, 13791389, https://doi.org/10.1080/01621459.1968.10480934.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stubenrauch, C. J., and Coauthors, 2013: Assessment of global cloud datasets from satellites: Project and database initiated by the GEWEX radiation panel. Bull. Amer. Meteor. Soc., 94, 10311049, https://doi.org/10.1175/BAMS-D-12-00117.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Theil, H., 1950: A rank-invariant method of linear and polynomial regression analysis. Indagationes Math., 12, 173.

  • Vaughan, M. A., and Coauthors, 2020: Cloud–aerosol lidar infrared pathfinder satellite observations: Data management system data products catalog. NASA Doc. PC-SCI-503, 256 pp., https://www-calipso.larc.nasa.gov/products/CALIPSO_DPC_Rev4x92.pdf.

  • Virts, K. S., J. M. Wallace, Q. Fu, and T. P. Ackerman, 2010: Tropical tropopause transition layer cirrus as represented by CALIPSO lidar observations. J. Atmos. Sci., 67, 31133129, https://doi.org/10.1175/2010JAS3412.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Walther, A., W. Straka, and A. K. Heidinger, 2013: ABI algorithm theoretical basis document for daytime cloud optical and microphysical properties (DCOMP). NOAA/NESDIS/Center for Satellite Applications and Research Algorithm Theoretical Basis Doc., 66 pp., https://www.star.nesdis.noaa.gov/goesr/documents/ATBDs/Baseline/ATBD_GOES-R_Cloud_DCOMP_v3.0_Jun2013.pdf.

  • Wang, C., and X. Huang, 2014: Parallax correction in the analysis of multiple satellite data sets. IEEE Geosci. Remote Sens. Lett., 11, 965969, https://doi.org/10.1109/LGRS.2013.2283573.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, C., P. Yang, A. Dessler, B. A. Baum, and Y. Hu, 2014: Estimation of the cirrus cloud scattering phase function from satellite observations. J. Quant. Spectrosc. Radiant. Transfer, 138, 3649, https://doi.org/10.1016/j.jqsrt.2014.02.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Winker, D. M., W. H. Hunt, and M. J. McGill, 2007: Initial performance assessment of CALIOP. Geophys. Res. Lett., 34, L19803, https://doi.org/10.1029/2007GL030135.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Winker, D. M., and Coauthors, 2009: Overview of the CALIPSO mission and CALIOP data processing algorithms. J. Atmos. Oceanic Technol., 26, 23102323, https://doi.org/10.1175/2009JTECHA1281.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Young, S. A., and M. A. Vaughan, 2009: The retrieval of profiles of particulate extinction from Cloud-Aerosol Lidar Infrared Pathfinder Satellite Observations (CALIPSO) data: Algorithm description. J. Atmos. Oceanic Technol., 26, 11051119, https://doi.org/10.1175/2008JTECHA1221.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Young, S. A., M. A. Vaughan, A. Garnier, J. L. Tackett, J. D. Lambeth, and K. A. Powell, 2018: Extinction and optical depth retrievals for CALIPSO’s version 4 data release. Atmos. Meas. Tech., 11, 57015727, https://doi.org/10.5194/amt-11-5701-2018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yu, F., X. Wu, X. Shao, B. Efremova, H. Yoo, H. Qian, and B. Iacovazzi, 2017: Early radiometric calibration performances of GOES-16 Advanced Baseline Imager. Earth Obs. Syst., 10402, 104020S, https://doi.org/10.1117/12.2275195.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fig. 1.

    (a) A photograph of transparent cirrus clouds taken in Monterey, California, on 11 Jun 2020, (b) GOES-16 ABI “true color” image for a sector over the Caribbean Sea at 1800 UTC 19 Aug 2020, and (c) the GOES-16 ABI 1.378-μm channel reflectance at 1800 UTC 19 Aug 2020.

  • Fig. 2.

    (a) One day of daytime CALIOP ground tracks (black lines) superimposed over viewing zenith angle for the GOES-16 disk (shading). (b) Estimated GOES-16 pixel size as a function of viewing zenith. Pixel size is estimated by interpolating GOES pixel centers to find the pixel corners and then computing the two-dimensional area. The blue lines in (b) and (a) represent the VZA = 80° threshold used in this study and its location in the GOES-16 disk, respectively.

  • Fig. 3.

    Examples of the collocation process for (a) near-nadir viewing and (b) near the GOES-16 disk edge. The black line indicates the CALIOP ground track, with CALIOP 5-km pixel centers indicated by the circles. GOES pixels are shaded in gray as a visual aid.

  • Fig. 4.

    PDFs of clear-sky Ch.-4 radiance (W m−2 sr−1) for the (a) full sample (dt < 7.5 min) and (b) the HQ sample (dt < 1 min).

  • Fig. 5.

    PDFs of (a) VZA, (b) SZA, (c) PWV, and (d) AMF for the full (dt <7.5 min) clear-sky samples.

  • Fig. 6.

    Mean Ch.-4 radiance (W m−2 sr−1) as a function of (a) VZA, (b) SZA, and (c) PWV. Bin sizes are 10, 10, and 1 for VZA, SZA, and PWV, respectively. Error bars indicate ±1 standard deviation.

  • Fig. 7.

    Mean Ch.-4 radiance (W m−2 sr−1) as a function of AMF for (a) the full clear-sky sample (dt < 7.5 min) and (b) the HQ clear-sky sample (dt < 1 min). Black and red error bars indicate 1and 2 standard deviations, respectively. Black and red dotted lines indicate the respective linear regression lines. The AMF bin size is 1. Note that the last AMF bin is for all values greater than 7 because of sample sizes.

  • Fig. 8.

    Modeled Ch.-4 radiance (W m−2 sr−1) to (a) PWV (with AMF = 5) and (b) AMF (with PWV = 3 cm). Note that there are no AMF values less than ~0.2 in the full clear-sky sample used to create the model.

  • Fig. 9.

    A process flowchart of the GOES transparent cirrus detection algorithm and cloud optical depth estimation presented in this study.

  • Fig. 10.

    Scatter density of CALIOP 0.532-μm COD as a function of (a),(b) Ch.-4 radiance (W m−2 sr−1) and (c), (d) the parallax corrected Ch.-4 radiance (W m−2 sr−1) for (left) the full transparent cirrus sample and (right) the HQ cirrus sample. Point density is computed as the normalized number of pixels within 20% of (x, y) for each point and should be treated as a visual aid only. The solid regression line is computed using the Theil–Sen slope estimation method, and the dotted regression line is computed using a least squares multiple linear regression.

  • Fig. 11.

    The theoretical transparent cirrus detection floor in COD space as a function of AMF. These values are computed by converting AMF values to radiance space using the regressions shown in Fig. 7 and then converting the radiance values to COD space using the linear regressions shown in Fig. 10. Red or black lines respectively indicate that the 2– or 1–standard deviation threshold was used in converting the AMFs to radiances. Solid or dotted lines respectively indicate that the full or HQ transparent cirrus sample regression was used to convert radiances to CODs.

  • Fig. 12.

    PDFs of (a) CALIOP 0.532-μm column-integrated COD, (b) estimated COD using the 1–standard deviation radiance t