The algorithm to produce the Clouds and the Earth’s Radiant Energy System (CERES) Edition 4.0 (Ed4) Energy Balanced and Filled (EBAF)-surface data product is explained. The algorithm forces computed top-of-atmosphere (TOA) irradiances to match with Ed4 EBAF-TOA irradiances by adjusting surface, cloud, and atmospheric properties. Surface irradiances are subsequently adjusted using radiative kernels. The adjustment process is composed of two parts: bias correction and Lagrange multiplier. The bias in temperature and specific humidity between 200 and 500 hPa used for the irradiance computation is corrected based on observations by Atmospheric Infrared Sounder (AIRS). Similarly, the bias in the cloud fraction is corrected based on observations by Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) and CloudSat. Remaining errors in surface, cloud, and atmospheric properties are corrected in the Lagrange multiplier process. Ed4 global annual mean (January 2005 through December 2014) surface net shortwave (SW) and longwave (LW) irradiances increase by 1.3 W m−2 and decrease by 0.2 W m−2, respectively, compared to EBAF Edition 2.8 (Ed2.8) counterparts (the previous version), resulting in an increase in net SW + LW surface irradiance of 1.1 W m−2. The uncertainty in surface irradiances over ocean, land, and polar regions at various spatial scales are estimated. The uncertainties in all-sky global annual mean upward and downward shortwave irradiance are 3 and 4 W m−2, respectively, and the uncertainties in upward and downward longwave irradiance are 3 and 6 W m−2, respectively. With an assumption of all errors being independent, the uncertainty in the global annual mean surface LW + SW net irradiance is 8 W m−2.
The energy that drives and maintains dynamics in the Earth system takes different forms while it flows through the system. Solar irradiance absorbed by Earth is energy input to the system. Longwave irradiance emitted to space is energy output by Earth. Driven by the gradient of the energy deposition, dynamics redistributes the energy. At an annual scale, the zonal top-of-atmosphere (TOA) net irradiance is the energy transported poleward by dynamics. The surface of Earth receives solar radiation (shortwave) and emission from the atmosphere (longwave). Approximately 12% of solar radiation incident on the surface is reflected and the rest is absorbed by the surface (Stephens et al. 2012). The surface emits longwave radiation proportional to the fourth power of its temperature. The downward longwave irradiance emitted by the atmosphere is primarily sensitive to near-surface temperature and the amount of water vapor as well as cloud fraction and base height in the atmosphere.
Although exact values vary depending on satellite data products used in the estimate and the method to adjust fluxes to balance energy budget (Trenberth et al. 2009; Stephens et al. 2012; Wild et al. 2013; L’Ecuyer et al. 2015), the global annual mean surface net shortwave irradiance is 165 ± 6 W m−2 and global annual mean net longwave irradiance is approximately −53 W m−2 (Stephens et al. 2012), where a positive value indicates net energy deposition to the surface. Most of the energy deposited to the surface by radiation is converted to the enthalpy flux and enters the atmosphere in the form of sensible and latent heat fluxes. Over ocean, 85% and 14% of energy deposited to the surface by radiation heats the atmosphere by, respectively, latent and sensible heat flux. Less than 1% heats the ocean at a global annual scale (Loeb et al. 2012; Palmer 2017; Wild et al. 2015). The latent heat flux from the ocean at a climate time scale is driven by net surface irradiance (Held and Soden 2006; Stephens and Ellis 2008). The spatial and temporal distribution of the difference between net surface irradiance and enthalpy flux determines the regional energy deposition to oceans. Therefore, understanding spatial and temporal distribution of shortwave, longwave, and net surface irradiances is important in understanding how energy is distributed within the Earth system.
Net atmospheric irradiance, which is the net TOA irradiance minus net surface irradiance, is negative because longwave cooling is larger than heating by shortwave absorption. At a global annual scale, the net atmospheric shortwave irradiance is 75 ± 10 W m−2 while the net atmospheric longwave irradiance is −188 ± 13 W m−2 (Stephens et al. 2012). The radiative cooling of the atmosphere is compensated by the latent heat release by precipitation (88 ± 10 W m−2) and sensible heat flux from the surface (24 ± 7 W m−2). At a regional scale, because the sensible heat flux is small compared to other components, the three large energy forms that maintain the balance in an atmospheric column are latent heat released by precipitation, radiative cooling, and divergence of dry static energy and kinetic energy by dynamics (Trenberth and Stepaniak 2003; Kato et al. 2016). Part of the energy deposited in the tropics by net radiation is transported to the midlatitude mostly by mean meridional circulation in the form of potential energy (Peixoto and Oort 1992). Energy is further transported poleward mostly by transient eddies in the form of latent heat and sensible heat fluxes (Peixoto and Oort 1992). Meridional transport of energy alters the effective emission temperature of atmosphere and surface. Longwave radiation is emitted to space according to the effective emission temperature. To quantitatively understand the poleward transport of energy by atmosphere and ocean, therefore, the energy flux in all forms needs to be estimated.
Among energy fluxes, the Clouds and the Earth’s Radiant Energy System (CERES) project provides TOA and surface irradiances at various temporal and spatial scales. TOA and surface irradiances are derived nearly independently. TOA irradiances are derived from radiance measured by CERES instruments (Su et al. 2015; Loeb et al. 2005). Surface irradiances are computed with satellite-derived cloud and aerosol properties and temperature, and specific humidity profiles from reanalysis. Surface irradiances are, therefore, more susceptible to the error in inputs used for the computations. To reduce the error in surface irradiances and increase the consistency with TOA irradiances, TOA irradiances are used to constrain surface irradiances (Kato et al. 2013). The EBAF-surface data product contains monthly 1° × 1° upward and downward shortwave and longwave surface irradiances constrained by CERES-derived TOA irradiances.
The purpose of this paper is to describe the algorithm used to produce the Edition 4.0 (Ed4) EBAF-surface data product and to evaluate EBAF-surface irradiances with surface observations. The previous version, Edition 2.8 (Ed2.8) EBAF-surface data product, has been used for the evaluation of surface irradiances of climate models and other data products. These studies identify biases and spread among surface irradiances in models and data products (e.g., Boeke and Taylor 2016; Slater 2016; Loew et al. 2017). In addition, a study by Levine and Boos (2017) shows that intermodal precipitation variation is related to intermodal surface albedo variation. The EBAF data products have also been used for analyses of regional and global mean surface and net atmospheric irradiances (e.g., Hakuba et al. 2016) and for constraining other energy fluxes (e.g., Boos and Korty 2016; Slessarev et al. 2016; DeAngelis et al. 2015). Increasing shortwave absorption in the atmosphere because of increasing water vapor reduces the precipitation increase in a warmer climate. Because the spread of the sensitivity of shortwave absorption to water vapor change is large among climate models, DeAngelis et al. (2015) show that sensitivity of shortwave absorption (derived from EBAF-surface) to precipitable water can be used to constrain the models. In addition, the product has been used for analyses of seasonal and interannual variability of surface irradiances (e.g., Mayer et al. 2016; Wild 2016).
In describing the revision of the EBAF-surface in this paper, we provide descriptions of essential elements of the algorithm in section 2, an evaluation of Ed4 surface irradiance and how surface irradiances are changed from Ed2.8 in section 3, a description of the uncertainty in surface irradiance at various temporal and spatial scales in section 4, and the application of EBAF-surface irradiance data to climate research in section 5.
Overall, the approach used in producing Edition 4.0 EBAF-surface is similar to the approach used in producing Ed2.8 EBAF-surface (Kato et al. 2013). The flow diagram of the Ed4 process is shown in Fig. 1. Ed4 synoptic 1° (SYN1deg)-Month (Rutan et al. 2015) and Ed4 EBAF-TOA (Loeb et al. 2018) are used as inputs for all-sky irradiances. In addition, SYN1deg-Hour is used for clear-sky irradiances. Clear-sky SYN1deg-Hour irradiances are computed by removing clouds and are provided every hour for all grid boxes. Ed4 SYN1deg-Month contains monthly TOA and surface irradiances at a 1° × 1° resolution computed by a radiative transfer model (Fu and Liou 1993; Rose et al. 2013). Irradiances are computed hourly with retrieved cloud properties from Moderate Resolution Imaging Spectroradiometer (MODIS) and geostationary satellites (GEOs). Minnis et al. (2011, 2010) (Sun-Mack et al. 2018; P. Minnis et al. 2017, unpublished manuscript) discuss the cloud mask algorithm and algorithm for retrieving cloud optical thickness, emissivity, cloud effective temperature and height, particle size, phase, and ice and liquid water path from MODIS radiances. Over regions between 60°N and 60°S, Ed4 cloud properties from geostationary satellites are derived hourly, improved from 3-hourly in Ed2.8. In addition, if available, up to 5 channels are used for cloud retrievals from geostationary satellites. The cloud fraction is increased in Ed4, especially for low-level clouds, compared with Ed2.8. These cloud properties are also included in CERES Ed4 SYN1deg-Month and SYN1deg-Hour data products. The Goddard Earth Observing System, version 5.4.1 (GEOS-5.4.1), reanalysis (Rienecker et al. 2008), which provides temperature, specific humidity, and ozone profiles, is used throughout the time period for Ed4 for irradiance computations. In addition, the Model of Atmospheric Transport and Chemistry (MATCH; Collins et al. 2001) that assimilates MODIS aerosol optical thickness provides hourly optical thickness, increased from daily for Ed3 SYN, and aerosol type. MODIS Collection 5 radiances are used for cloud retrievals and aerosol assimilations from March 2000 through February 2017. MODIS Collection 6 radiances and aerosol optical thickness are used from March 2017 onward. These input changes are summarized in Table 1 along with expected impacts. As discussed in Rutan et al. (2009), the surface albedo for Ed3 SYN is derived from clear-sky CERES footprints. In Ed4, partly cloudy footprints are also used to derive surface albedo.
The Ed4 EBAF-surface algorithm adjusts SYN1deg-Month surface irradiances by two processes: bias correction and Lagrange multiplier. Biases in surface irradiances caused by biases in temperature, humidity, and cloud fraction with known sign are adjusted in the bias correction process. The bias correction is needed to mitigate the error in the Lagrange multiplier process caused by incorrectly attributing TOA irradiance differences to errors in atmospheric and cloud properties.
How well TOA irradiances can constrain surface irradiances depends on the correlation between TOA and surface. For shortwave irradiances, because of energy conservation, when TOA albedo A, surface absorptance a, and atmospheric absorptance α are defined as
where F is the irradiance, subscripts TOA and sfc indicate, respectively, top-of-atmosphere and surface, and superscript arrows indicate upward or downward. When A is plotted as a function of a, over a 1° × 1° grid where the surface albedo does not change significantly with season, A and a are well correlated with a linear relationship (Pinker and Lazlo 1992; Cess et al. 1995; Li and Moreau 1996; Barker and Li 1997) even with monthly mean irradiances (Fig. 2, top). The slope of the linear relationship between A and a is related to the α change with respect to a by
The strong correlation among A, a, and α is the reason that TOA shortwave irradiance can provide a constraint to surface shortwave irradiances. The relationship for longwave is complex because surface and atmosphere emit longwave radiation. As a consequence, the constraint by TOA irradiance on surface longwave irradiance is weaker (e.g., Ellingson 1995). The range of the net surface longwave irradiance divided by the TOA downward shortwave irradiance is about twice as large as the shortwave counterpart; the root-mean-square (RMS) difference of monthly mean surface net irradiances and regression line for the left, center, and right panels of Fig. 2 is approximately 2, 4, and 7 W m−2 for shortwave (top panels), and 6, 9, and 14 W m−2 for longwave (bottom panels). These RMS differences give a range of monthly regional mean surface irradiances over specific regions if we know TOA irradiances and surface, atmospheric, and cloud properties. Because uncertainties are associated with the inputs and the relationship varies depending on region, the surface irradiance uncertainty can be larger than these RMS differences.
a. Bias correction
The bias correction considered in the Ed2.8 process is the upper-tropospheric humidity. The variables included in the Ed4 bias correction processes are: upper-tropospheric (500 to 200 hPa) temperature and specific humidity, low-level cloud fraction viewed from space, and cloud fraction viewed from the surface. Irradiances corrected by these bias correction processes are listed in Table 2. Biases are determined by comparisons of variables derived from different instruments along with the difference between computed and observed TOA irradiances. For example, GEOS-5.4.1 upper-tropospheric humidity is larger than upper-tropospheric humidity derived from the Atmospheric Infrared Sounder (AIRS) instrument (Chahine et al. 2006). A moist bias of upper-tropospheric humidity is consistent with the result using field campaign data (e.g., Wang et al. 2017). Although the difference alone does not necessarily imply that GEOS-5.4.1 upper-tropospheric humidity is biased high, reducing upper-tropospheric humidity helps reduce the difference between computed clear-sky TOA longwave irradiances (SYN1deg-Month) and CERES-derived clear-sky TOA longwave irradiances (EBAF-TOA).
We use radiative kernels (e.g., Soden et al. 2008) to convert the bias in temperature T and specific humidity q between the 200 and 500 hPa levels, low-level cloud fraction viewed from space, cloud fraction viewed from the surface, and cloud-base height to the bias in TOA and surface irradiances. All-sky and clear-sky radiative kernels are derived from Ed4 SYN1deg-Hour with a temporal resolution of 3-hourly and spatial resolution of 1° × 1° using 2008 data (Thorsen et al. 2018, manuscript submitted to J. Climate) except for the bias correction of cloud fraction viewed from space. Temperature and specific humidity kernels are derived from 2008 data since the interannual variability of these kernels was found to be negligible. Cloud base and fraction kernels are derived separately for each month and are computed separately for four different cloud types (high, mid–high, mid–low, and low). High, mid–high, mid–low, and low cloud types are defined by their cloud-top height: less than 300 hPa, 500–300 hPa, 700–500 hPa, and greater than 700 hPa, respectively. All kernels are built numerically by perturbing one variable at a time and computing TOA and surface irradiances. TOA and surface irradiance changes are computed by subtracting unperturbed values from those computed with the perturbation. We average kernels over a month but maintain the 1° × 1° spatial resolution.
1) Bias in upper-tropospheric temperature and specific humidity
The bias of T and q between 200 and 500 hPa is estimated by comparing GEOS-5.4.1 1° × 1° monthly means at 5 levels between 200 and 500 hPa with those from the AIRS level-3, version 6, product (AIRX3STM.006, TqJoint grid product). We convert the difference (GEOS-5.4.1 − AIRS) of T and q, layer by layer, to the difference of all-sky and clear-sky TOA longwave irradiances using, respectively, all-sky and clear-sky radiative kernels. In addition, the clear-sky surface downward longwave irradiance is also corrected using clear-sky radiative kernel.
AIRS observations are not available before September 2002. For this time period, we use climatological T and q differences between GEOS-5.4.1 and AIRS monthly 1° × 1° in the layer between 200 and 500 hPa derived from 14 years of data (from September 2002 through August 2015). The standard deviation of the difference of monthly mean gridded T between 200 and 500 hPa is about 0.25 K, and the standard deviation of the difference of monthly mean gridded q between 200 and 500 hPa is less than 10% of the mean q. We also use climatological clear-sky radiative kernel to convert T and q differences to TOA longwave and surface downward longwave irradiances. Therefore, TOA longwave and surface downward longwave from March 2000 through August 2002 are adjusted using their monthly 1° × 1° climatological adjustments derived from September 2002 through August 2015.
2) Bias in the cloud fraction as viewed from space
Cloud fraction derived from CALIPSO (Winker et al. 2010), CloudSat (Stephens et al. 2008), MODIS, and geostationary satellites are used for the bias correction of all-sky TOA shortwave and longwave irradiances and surface downward shortwave irradiances. We correct only low-level cloud fractions over ocean because regions with larger differences between computed and EBAF-TOA shortwave and longwave irradiances coincide with regions where low-level clouds are often present (Fig. 3).
The cloud fraction derived from CALIPSO and CloudSat is more accurate than the cloud fraction derived from passive sensors. However, because they were launched in 2007, a direct comparison with cloud fraction derived from all geostationary satellites is not possible. The cloud fraction bias is, therefore, estimated in multiple steps. We first estimate the difference between the zonal cloud fraction over ocean derived from MODIS on Aqua and CALIPSO–CloudSat (CC) using the Ed Release B1 (RelB1) CALIPSO–CloudSat–CERES–MODIS (CCCM) data product (Kato et al. 2010) that uses version 3 of CALIPSO and R-04 CloudSat products. Using 4 months of data (January, April, July, and October 2010), Ed4 MODIS cloud fraction is derived over CERES footprints that include the CALIPSO–CloudSat ground track (Kato et al. 2010). Therefore, the viewing zenith angle of MODIS is restricted to near nadir. Clouds with optical thickness less than 0.3 (Minnis et al. 2008a,b) are excluded from both cloud fractions derived from MODIS (using MODIS-derived optical thickness) and CALIPSO–CloudSat (using CALIPSO-derived extinction profile). We then derive the zonal cloud fraction relative difference (CC minus Aqua)/TAGi (blue line in Fig. 4), including all cloud types, where CC indicates the cloud fraction derived from CALIPSO–CloudSat, TAGi indicates cloud fraction derived from Terra MODIS + Aqua MODIS + geostationary satellites (GEOs), and Aqua indicates cloud fraction derived from Aqua MODIS only. When all cloud types are included, the global mean relative cloud fraction difference averaged over 4 months (CC − Aqua)/TAGi is −6.7%, that is, the Aqua MODIS–derived cloud fraction is generally larger than CC. A part of the reason is the size of the MODIS pixel. Overestimation of the cloud fraction by MODIS is consistent with the results of Zhao and Di Girolamo (2006), who compare the cloud fraction derived from MODIS and Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) that has a smaller pixel size than MODIS (15–90 m). Our further study indicates that the different instrument footprint size of MODIS and CALIPSO leads to a difference in the global mean water-cloud fraction difference of about 0.02. We include all cloud types in comparing MODIS and CALIPSO–CloudSat because, when high- and low-level clouds overlap, MODIS tends to retrieve low-level clouds since the high-level cloud tends to be optically thin, while the upper-most cloud layer derived from CALIPSO–CloudSat is a high-level cloud.
Second, we compute zonal low-level cloud fraction derived from Terra MODIS, Aqua MODIS, and GEOs using the same seasonal months (January, April, July, and October 2010). We then compute the relative zonal cloud fraction difference of Terra, Aqua, and GEOs combined (TAGi) to Terra plus Aqua by (T&A − TAGi)/TAGi (purple line in Fig. 4). The global mean relative low-level cloud fraction difference is −6.0%, that is, GEO-derived cloud fraction is generally larger than MODIS-derived cloud fraction.
Third, we simply add these two relative zonal cloud fraction differences [(CC − Aqua)/TAGi + (T&A − TAGi)/TAGi] to come up with the zonal relative cloud fraction bias correction, which is indicated by the black line in Fig. 4. The global mean relative difference is −12.7%. We multiply 1° × 1° monthly mean low-level cloud fraction over ocean by this relative zonal cloud fraction difference to compute the 1° × 1° monthly low-level cloud fraction bias. We then use 1° × 1° monthly low-level cloud fraction kernels computed with Ed4 SYN1deg-Month clouds, varying every month, to convert the cloud fraction bias to TOA shortwave and longwave irradiance and surface downward shortwave irradiance bias adjustments.
3) Bias in the cloud fraction as viewed from the surface
The cloud fraction exposed to the surface separated by cloud type (i.e., viewed from the surface) is different from the cloud fraction viewed from space. The surface downward longwave irradiance depends largely on low-level cloud fraction viewed from the surface. The bias of the surface downward longwave irradiance is, therefore, inferred from the bias in the cloud fraction viewed from the surface. In addition to four cloud types (high, mid–high, mid–low, and low), Ed4 SYN1deg considers lower-level clouds with a random cloud overlap assumption. This results in a total of 16 different single and two-layer overlapping combinations. Only the four most frequently occurring cloud-layer combinations in an hour box and in a 1° × 1° grid are used in computing surface irradiances in Ed4 SYN1deg. Using the four most frequently occurring cloud-layer combinations used in Ed4 SYN1deg, we compute zonal cloud fraction viewed from the surface over ocean and land. Similarly, we compute the zonal cloud fraction viewed from the surface from CALIPSO- and CloudSat-merged cloud profiles (Kato et al. 2010). We exclude clouds with optical thickness less than 0.3 and CALIPSO cloud–aerosol discrimination (CAD) score less than 70. We compute the monthly zonal-mean surface-view cloud fraction difference over ocean and nonocean (including land, snow and ice-covered surfaces) separately for each cloud type at a 1° resolution with 5° smoothing (to reduce nadir-view sampling noise) using data from January through December 2008. We also compute the difference of cloud-base height derived from CALIPSO–CloudSat and that estimated from passive sensors (Minnis et al. 2011; P. Minnis et al. 2017, in preparation) by cloud layers. Cloud-base heights are derived from cloud-top effective temperature and optical thickness (Minnis et al. 2010, 2011). We then convert the bias in zonal monthly mean cloud fraction viewed from the surface and cloud-base height to the bias in the monthly zonal surface downward longwave irradiance using zonal cloud-type-dependent cloud fraction and height kernels derived with Ed4 SYN1deg 2008 clouds. Radiative kernels derived with 2008 clouds are used for all years. Summing up the monthly zonal surface downward longwave irradiance adjustment computed for four cloud layers, we obtain the zonal monthly bias correction for land and ocean separately.
b. Lagrange multiplier
TOA irradiance biases discussed in the previous section are subtracted from the difference between Ed4 SYN1deg-Month and Ed4 EBAF-TOA irradiances. Figure 5 shows the bias and RMS differences between computed and CERES-derived TOA shortwave and longwave irradiances after the bias correction is applied to computed irradiances. The correction reduces the difference shown in Fig. 3 except over the tropical western Pacific and Indian Oceans. Once the known bias adjustments are made, we use Lagrange multiplier to adjust surface, atmospheric, and cloud properties by region (1° × 1°), based on their uncertainties. Lagrange multiplier finds a solution to minimize the change of surface, cloud, and atmospheric properties to eliminate the difference between computed and observed TOA shortwave and longwave irradiances (Rose et al. 2013; Kato et al. 2013). Tables 3 and 4 summarize uncertainties used in the Lagrange multiplier process. Radiative kernels used in the Lagrange multiplier process are calculated with monthly mean properties that vary every month with a spatial scale of a 1° × 1° separated for clear-sky and all-sky. Mathematical expressions of the algorithm used in the Lagrange multiplier process are presented in the appendix. After the bias correction and Lagrange multiplier processes, the mean and RMS difference between adjusted and EBAF-TOA regional monthly mean irradiances are, respectively, −0.03 and 0.24 W m−2 for shortwave and −0.06 and 0.13 W m−2 for longwave. Global maps of the regional differences are shown in Fig. 6.
c. Clear-sky surface irradiance averaging
Clear-sky surface irradiances are adjusted separately from all-sky surface irradiances using the same algorithm but with a different set of inputs. Because the number of observations used to derive mean clear-sky irradiances depends on the occurrence of clear-sky scenes, and Ed4 SYN1deg hourly clear-sky irradiances are computed by removing clouds (i.e., clear-sky irradiances are provided every hour for all grid boxes), we average the computed clear-sky irradiances in the following way to reduce the difference between computed and observed monthly mean clear-sky irradiances. We first compute monthly hourly mean clear-sky surface shortwave and longwave irradiances by averaging Ed4 SYN1deg hourly irradiances of the same hour of the day in a 1° × 1° grid box over a month weighted by the clear fraction in a 1° × 1° grid box. Second, we further average 24 monthly hourly mean clear-sky irradiances to form the monthly mean clear-sky irradiance 〈F〉, that is
where wij is the clear fraction at the ith hour on the jth day of the month, nhour = 24, and Fij is either hourly mean shortwave or longwave irradiances. Clear-sky fractions for hour boxes containing MODIS observations are derived from MODIS. Clear fractions for hour boxes with no MODIS observations are derived by interpolating MODIS-derived clear fractions. For longwave irradiances, 〈FLW〉 = 〈F〉. For shortwave irradiances, the insolation correction ratio 〈R〉 is computed by
where F0 is the solar constant and 〈FSW〉 = 〈R〉〈F〉. Similar to the all-sky process, clear-sky TOA longwave and surface downward longwave irradiance bias due to upper-tropospheric temperature and specific humidity is corrected by the process explained in section 2a. Subsequently, clear-sky surface irradiances are adjusted before the Lagrange multiplier process discussed in section 2b.
As mentioned earlier, the difference between Ed2.8 and Ed4 EBAF-surface is primarily in their inputs. The Ed4 cloud algorithm generally detects more clouds than the Ed2.8 cloud algorithm does. In addition, Ed4 nighttime cloud properties derived from GEOs are significantly improved from Ed2.8. Furthermore, Ed4 EBAF-TOA global annual mean all-sky shortwave irradiance decreases by 0.5 W−2 and longwave irradiance increases by 0.5 W m−2 compared with Ed2.8 counterparts (Loeb et al. 2018).
Figure 7 shows the net surface irradiance difference between Ed4 and Ed2.8. Ed4 net surface irradiance is larger over the tropics and smaller over the Arctic compared with Ed2.8 net surface irradiance. Table 5 summarizes the difference of global mean irradiances between EBAF-surface Ed2.8 and Ed4. The mean irradiances are computed by averaging over 10 years from July 2005 through June 2015. Ed4 all-sky net surface shortwave irradiances increase by 1.3 W m−2 while shortwave absorption by the atmosphere decreases by 0.7 W m−2 compared with Ed2.8 counterparts. All-sky net surface longwave irradiance decreases by 0.2 W m−2 while net atmospheric longwave irradiance decreases by 0.3 W m−2 (i.e., more cooling) compared with Ed2.8 counterparts. A smaller atmospheric absorption of shortwave and a slightly larger atmospheric cooling by longwave are likely due to the smaller optical thicknesses of low-level clouds. Reducing optical thickness of low-level clouds reduces shortwave absorption and, to a lesser extent, increases longwave cooling in the atmosphere (Kato 2009, their Fig. 7), resulting in a negative net atmospheric irradiance change.
For clear-sky, the net surface irradiance difference between Ed4 and Ed2.8 is larger over land and polar regions (Fig. 8) compared to all-sky differences. A part of the difference is caused by larger changes that occurred in TOA clear-sky irradiances. Global annual mean Ed4 TOA shortwave irradiance is larger by 0.8 W m−2 and Ed4 TOA longwave irradiances are larger by 2.7 W m−2 compared to Ed2.8 counterparts (Loeb et al. 2018). The 2.7 W m−2 increase of Ed4 clear-sky TOA longwave irradiance from Ed2.8 is caused by the improved Ed4 cloud algorithm that screens high thin clouds better than the Ed2.8 cloud algorithm does. The primary reason for a smaller Ed4 surface downward longwave irradiance by 2.2 W m−2 and a smaller Ed4 longwave atmospheric net irradiance by 1.3 W m−2 is likely dryer atmospheres used for Ed4 compared to the atmospheres use for Ed2.8. Because thin cirrus clouds are usually associated with higher upper-tropospheric humidity, screening out thin cirrus clouds that exclude moist atmosphere from the clear-sky sampling also leads to a dryer clear-sky upper troposphere.
a. Evaluation by surface observations over ocean and land
We use surface observations to evaluate EBAF-surface irradiances. General discussions of the use of surface observations as validation of satellite data are given in Loew et al. (2017). The geographical location of surface buoy and land-surface sites used in this study’s validation is shown in Fig. 9. Some coastal Baseline Surface Radiation Network (BSRN) sites and BSRN sites located at mountain regions are excluded in computing bias and RMS differences because irradiances measured at these sites do not represent mean irradiances over the grid box where these sites are located (Kato et al. 2012). To compute monthly irradiances from observed surface irradiances taken at these sites, we first compute hourly mean irradiances from irradiances at the native temporal resolution of the original data (e.g., 1 and 10 min) after checking quality flags, if available, provided with the original data. Once hourly irradiances are computed, we calculate a monthly mean diurnal cycle, and then use it to calculate a monthly mean. For EBAF validation we require a minimum of 85% temporal coverage for the month for a particular site and month to be included.
There is, as well, uncertainty is associated with surface observations. The uncertainty in monthly mean downward shortwave irradiances is approximately 5 W m−2 (Ohmura et al. 1998; Michalsky et al. 1999, 2003, 2006, 2007; Colbo and Weller 2009). The uncertainty in monthly mean downward longwave irradiances is about 2%, which corresponds to 2–6 W m−2 (Gröbner et al. 2014). Irradiances measured at buoys might have a larger uncertainty. For example, Foltz et al. (2013) point out that downward shortwave irradiances over the tropical Atlantic Ocean are significantly underestimated because of dust accumulation on buoys. Consequently, positive biases of downward shortwave irradiances from Ed4 EBAF-surface are apparent when the biases of individual buoy sites are plotted separately (Fig. 10). We, however, do not have a separate uncertainty estimate for irradiances measured at buoys.
Results of comparisons over ocean are shown in Fig. 11. Buoys with a bias larger than 20 W m−2 are excluded in the histogram shown in Fig. 11. The bias in Ed4 is slightly larger than Ed2.8 for both downward shortwave and longwave irradiances over ocean. However, Ed4 standard deviations are slightly smaller than Ed2.8 standard deviations for both downward shortwave and longwave irradiances. While the improvement of monthly mean irradiance appears to be small, a significant improvement is apparent when the nighttime hourly mean downward longwave irradiance is compared with observations over the eastern and western Pacific Ocean (Table 6). As mentioned earlier, all Ed4 SYN1deg products use hourly geostationary satellite data as opposed to 3-hourly data used in earlier versions. In addition, up to five channels of radiances are used in cloud property retrieval, which allow the Ed4 cloud algorithm to retrieve particle size and more accurate cloud-top height during night. These improvements lead to increasing downward longwave irradiance over ocean. Both daytime and nighttime Ed4 downward longwave irradiance are slightly positively biased, as opposed to a negative bias in Ed2.8 nighttime and a positive bias in Ed2.8 daytime downward longwave irradiance. The improvement leads to a smaller standard deviation indicated in Fig. 11.
Similar to the comparison over ocean, the difference of Ed4 downward shortwave and longwave irradiances over land is similar, but slightly better compared to the difference in Ed2.8 (Fig. 12). Results of these comparisons are used to estimate the uncertainty in Ed4 EBAF-surface irradiance in section 4.
b. Evaluation by surface observations for polar regions
The surface radiation budget over polar regions is highly variable compared to other regions. In addition, surface albedo varies spatially and seasonally. However, only four validation sites in the Arctic and four validation sites in the Antarctic exist. The bias of downward shortwave and longwave irradiances for both Arctic and Antarctic sites is less than 5 W m−2 (Table 7). However, because atmospheric and surface properties (except over ice sheets) are highly variable temporally and spatially over polar regions, the bias at one specific site can be much larger. For example, the bias of the downward longwave irradiance over the Greenland Summit site is larger than 10 W m−2 (Fig. 13). The large bias is caused by overestimation of the cloud fraction over the Greenland Summit site, especially during polar night. Overestimation of the nighttime cloud fraction affecting downward longwave irradiances, however, appears to be limited to high-altitude regions.
c. Anomaly time series
Observed anomaly time series can also be used to evaluate the Ed4 EBAF-surface irradiance anomaly time series. For the evaluation, we average observed monthly mean irradiances at all available buoys or land sites for each month. Similarly, we average surface irradiances from grid boxes containing buoys or land sites to compute monthly mean irradiances. We select EBAF irradiances to match temporal sampling of observation at each location. We then calculate deseasonalized anomalies by subtracting climatological monthly means of January, February, etc., from corresponding months. The climatological means are computed for observed and computed surface irradiance separately. The number of observation sites and grid boxes used for the climatological monthly means varies, depending on month (shown by gray bars in Figs. 14 and 15). The variability shown in Figs. 14 and 15 are, therefore, driven by both natural variability and the number of regional monthly means used in the time series. However, the contribution from the variation of the number of regional means is small, as the correlation coefficient is less than 0.5, while the correlation coefficients of computed and observed irradiance anomalies are greater than 0.8 (numbers shown on the top-right corner of Figs. 14 and 15). Similar to Ed2.8 demonstrated in Loeb et al. (2014) and Kato et al. (2015), the agreement of deseasonalized anomalies computed from EBAF and surface observations is remarkable, with correlation coefficients generally greater than 0.93. The variability of anomalies is driven by variabilities in atmospheric and cloud properties, and the good agreement suggests that these variabilities are captured in inputs used for surface irradiance computations. We have, however, less confidence in the surface irradiance variability over polar regions because of the small number of surface observation sites used for the evaluation.
While the agreement of surface downward irradiances with observations is nearly the same for Ed2.8 and Ed4 (section 3a), the Ed4 anomaly time series is improved significantly (Fig. 16). Because two different versions of GEOS reanalyses are used for temperature and humidity sources, there is a discontinuity at the beginning of 2008 in the Ed2.8 time series when the reanalysis is switched. Ed4 uses GEOS-5.4.1 for the entire record, which eliminates the discontinuity (Fig. 17). In addition, Collection 5 MODIS aerosol optical thicknesses are used from March 2000 through February 2017 for Ed4 as opposed to Collection 4 being switched to Collection 5 at the beginning of May 2006 in Ed2.8. This switch introduced a large discontinuity of aerosol optical thickness over some land regions. For clear-sky downward shortwave irradiance averaged over land, Ed4 anomalies are relatively uniform throughout the time series shown in Fig. 17 (red line), while Ed2.8 anomalies tend to be negative before May 2006 and positive after May 2006 (blue line in Fig. 17).
4. Uncertainty estimate
Results of the comparison of surface irradiances from Ed4 EBAF-surface and observations at surface sites indicate that mean biases are smaller than the uncertainty of surface observations. In this section, we estimate the uncertainty in computed surface irradiances at various temporal and spatial scales using surface observations.
a. Regional monthly mean
We estimate uncertainties in 1° × 1° monthly mean using irradiances from Ed4 EBAF-surface and surface observations and compare with the uncertainties estimated by Kato et al. (2012, 2013). Kato et al. (2012) compare surface irradiances computed with CALIPSO- and CloudSat-derived cloud profiles and with MODIS-derived cloud profiles to estimate the uncertainty associated with vertical cloud profiles. In addition, Kato et al. (2012) compare surface skin temperature derived from clear-sky scenes identified by CALIPSO and CloudSat with reanalysis skin temperature to estimate the uncertainty associated with surface skin and near-surface temperatures. In this study, we use the RMS difference of computed and observed monthly mean irradiances at surface sites as 1-sigma uncertainty, although other measures of uncertainty could also be used (e.g., Willmott et al. 2009). The RMS difference separated by region (ocean, land, Arctic, and Antarctic) is computed by
where x is either shortwave (SW) or longwave (LW), N is the number of months, M is the number of surface sites, ΔFij = Fij,comp − Fij,obs, and comp and obs indicate, respectively, computed and observed irradiances. Table 8 summarizes the RMS difference by region in the column labeled “monthly gridded.” Arctic and Antarctic regions are defined as, respectively, 60°–90°N and 60°–90°S.
When compared with uncertainties given by Kato et al. (2013), ΔFSW,RMS over ocean is slightly larger than the uncertainty given by Kato et al. (2013) (blue and cyan bars in Fig. 17). The term ΔFLW,RMS is smaller than the uncertainty given by Kato et al. (2013) for both land and ocean (blue and cyan bars in Fig. 18). In estimating the uncertainty in downward shortwave and longwave irradiances, Kato et al. (2012) mainly consider the uncertainty due to uncertainty in cloud properties. The cloud properties used in Kato et al. (2012) were derived by a cloud algorithm similar to the Ed2.8 cloud algorithm. The smaller ΔFLW,RMS found here, compared to the uncertainty of downward longwave irradiance given by Kato et al. (2012) is in part caused by the improvement of cloud properties. In addition, overlapping clouds are treated by a random cloud overlap assumption in Ed4 SYN1deg, which further reduces bias error, especially in the downward longwave irradiance.
The example of the bias at Greenland Summit compared to the bias at the other three sites illustrates the difficulty of generalizing the uncertainty in the surface irradiance over polar regions. Cloud, surface, and atmospheric properties are more variable than those at other regions, while surface and in situ observations are limited, especially during polar nights. These properties and their uncertainty also highly depend on time of year because retrieval errors depend on day, night, or surface type, which can change with season. With a caveat that the uncertainty is highly variable spatially and temporally, we use the RMS difference at four surface sites as the uncertainty in 1° × 1° monthly mean irradiances over polar regions.
For a consistency check, we also compute the uncertainty in surface irradiances from bias (Table 2) and random errors of input variables used in the correction processes (Table 4), and use the sensitivity of surface irradiance to these variables (radiative kernels) as the uncertainty (herein input perturbation approach). Zhang et al. (1995) use this approach to estimate the uncertainty in ISCCP surface irradiances. In the sensitivity approach, the uncertainty ΔFj is
where subscripts i and x indicate, respectively, different input variables and components of surface irradiances, that is, upward and downward shortwave and longwave. The first term on the right side is the uncertainty of variables from Table 4 multiplied by the radiative kernel of 1° × 1° monthly mean surface irradiances, and the second term on the right side δFj is the bias correction of the surface irradiance Fj (Table 2). An implicit assumption of Eq. (9) is that errors in different variables are independent. To compute the mean regional uncertainty from the sensitivity approach, ΔFj computed at a 1° × 1° resolution is averaged over 12 months and over 4 different regions.
When they are compared separated by regions, both ΔFSW,RMS and ΔFLW,RMS (red bars) are larger than, respectively, ΔFSW,per and ΔFLW,per (orange bars), except for downward longwave irradiance over ocean. While this may suggest that uncertainties shown in Tables 2 and 4 are either underestimated or missing the contribution from other variables or both, we expect that ΔFSW,RMS and ΔFLW,RMS are larger. The terms ΔFSW,RMS and ΔFLW,RMS include the uncertainty of surface observations discussed in section 3a. In addition, the spatial variability of surface irradiance within a 1° × 1° grid box can cause a difference greater than 10 W m−2 between the monthly mean irradiance averaged over a 1° × 1° grid and the monthly mean irradiance at a site in the grid box (Kato et al. 2012). These are a part of uncertainty elements and should be included in the uncertainty estimates. While the reason for larger differences between two estimates (cyan and green bars) for polar regions are unknown, we conclude that the agreement of uncertainties derived by the two approaches is reasonable.
Figure 18 also shows the sum of adjusted irradiances from both the bias correction and Lagrange multiplier processes, as well as the RMS difference between Ed2.8 and Ed4 EBAF downward shortwave and longwave irradiances averaged by region. It confirms that, except for the downward longwave over ocean, all adjustments and RMS differences are within the uncertainty shown by red bars.
b. Uncertainties in different spatial scales
The evaluation of uncertainty at different spatial scales using surface observations is complicated by the spatial distribution of the surface sites and the fact that surface observations are often not continuous over time, with sites being established and removed. To evaluate the uncertainty in spatially averaged irradiance at different scales, as given by Kato et al. (2013), we modify Eq. (8) by first averaging over selected K surface sites before calculating the RMS difference with the averaged irradiances from L groups of K sites. As any single grouping of sites is arbitrary, we calculate each RMS difference 100 times, each time randomly selecting sites with no repetition in the selected group. Defining and , then , where, with no duplicated sites being used, the number of L groups of averaged irradiances over K sites requires that L × K is less than or equal to the total number of sites M, that is, LK ≤ M. The RMS difference with L averaged irradiances over N months is
Figure 19 shows as a function of K. As expected, decreases as the number of sites averaged over K sites increases, indicating that a monthly mean irradiance difference contains a spatially random error. Although it may not be the only cause of the random error, the deviation of the monthly mean irradiance measured at the site from the gridded monthly mean irradiance where the site is located (Kato et al. 2012) is likely the primary reason for the random error. The vertical bars indicate the variability found as groups of sites are randomly selected 100 times and found to have relatively small effect. The downward shortwave and longwave irradiance difference over land decreases faster than the difference over ocean. This implies that the error in the downward irradiance over land tends to be more spatially random than the downward irradiance over ocean. The decrease of downward longwave difference over ocean with K is much slower, probably for the following reason. The downward longwave irradiance was measured only 20 buoys (out of 46). Two-thirds of them only have measurements though 2005. Therefore, the number of observation sites is significantly smaller. In addition, buoys are concentrated within limited geographical regions. As a consequence, the error in the downward longwave irradiance over ocean is not as spatially random as the error in the downward shortwave.
The relationship between and K is fitted by
to derive bias ΔFxbias and spatially random ΔFx,random errors. Once ΔFxbias and ΔFx,random are known, the uncertainty can be computed for any K.
The uncertainty as a function of K is plotted in Fig. 19 by solid lines. The uncertainty corresponding to the order of 100 samples approximately represents the uncertainty in a zonal monthly mean irradiance. The uncertainty corresponding to the order of 104 is considered as the uncertainty in a monthly mean irradiance averaged over the entire ocean or land. With this relationship, the uncertainty in zonal and global downward surface irradiance over land is smaller than the corresponding uncertainty over ocean even though the uncertainty in the irradiance over a 1° grid box containing a surface site might be larger. The uncertainty in the irradiance averaged over ocean or land shown in Fig. 19 is smaller than the uncertainty in monthly mean irradiances given by Kato et al. (2013). While smaller uncertainties might be, in part, caused by the difference between Ed4 and Ed2.8, the difference is largely due to different ways of estimating the uncertainty. We therefore use the average values of Fig. 19 and uncertainties given by Kato et al. (2013) as uncertainties for Ed4 downward irradiances at equal to or larger than a zonal spatial scale. If the resulting uncertainty at a larger spatial scale is larger than the uncertainty in the gridded irradiance, we use the uncertainty of the gridded irradiance for the uncertainty at that spatial scale. We use uncertainties given by Kato et al. (2013) for the uncertainty in Ed4 upward irradiances. Because Fig. 19 shows the uncertainty is nearly constant for K > 100, we also assume that the uncertainty in annual global mean irradiances is the same as the uncertainty in the monthly global mean irradiances. Uncertainties for Ed4 surface irradiances, revised from Ed2.8 uncertainties given in Kato et al. (2013), are shown in Table 8.
If we assume that errors affecting upward and downward shortwave and longwave irradiances are independent, 1σ (or k = 1) uncertainty in the global annual mean surface net irradiance is 8 W m−2 [=(52 + 32 + 42 + 32)1/2]. Therefore, the residual of surface energy balance computed with satellite data products of nearly 15W m−2 (Kato et al. 2011; Stephens et al. 2012; Loeb et al. 2014; L’Ecuyer et al. 2015) is well outside the 1σ uncertainty of the net surface irradiance. In addition, the difference of the global annual mean net irradiance derived from Ed2.8 and Ed4 EBAF-surface is only 1.1 W m−2. Given differences in inputs and algorithm used in two different versions, the small difference suggests robustness of the global annual mean net surface irradiance. As our result indicates, one reason for a small uncertainty in global mean irradiances is that a part of the error is spatially random. Averaging irradiance over a large area reduces the random errors, resulting in a small uncertainty in a global mean irradiance.
5. Applications to climate research
We expect that the Ed4 EBAF-surface data product will continue to contribute to research areas discussed in the introduction section. In addition, one of our goals is to record surface irradiance responses to increasing radiative forcing with time. According to Wild et al. (2008) the rate of the downward shortwave irradiance averaged over GEBA land sites from 1986 through 2000 is 2.2 W m−2 decade−1. The standard deviation of deseasonalized monthly anomalies of downward shortwave irradiance averaged over land is 1.35 W m−2, while the autocorrelation coefficient with a 1-month time lag is 0.23. Based on the form given by Weatherhead et al. (1998), 9 years of data is sufficient to detect the trend reported by Wild et al. (2008) at a 95% confidence level. The trend in the global mean irradiance is much smaller and requires a longer time to detect the trend [e.g., a 2.2% surface net irradiance or a 0.4 W m−2 decade−1 signal (Held and Soden 2006) with observed variability (~0.7 W m−2) and autocorrelation coefficient (0.65) would take 2–3 decades]. Given the length of the current data record, the expected contribution to future climate prediction is through providing mean-surface irradiances and their natural variability. Although the relationship between global-scale covariabilities of temperature and TOA irradiance derived from natural variability and feedback parameters is not well understood (Loeb et al. 2016), a study by Zhou et al. (2015) shows encouraging correlation. Klein and Hall (2015) list potential regional-scale natural variabilities that might be relevant to future climate prediction. Klein and Hall (2015) argue that understanding physical reasons underlying the empirical relationship between variability under current climate condition and changes that occur under radiative forcing is required to constrain future climate prediction. We believe that conservation of energy used as a constraint can be a powerful tool to understand physical reasons. Understanding how the energy balance in the atmospheric column varies with natural variability might provide insights into how the energy balance is altered under radiative forcing. A possible approach to gain insight into physical reasons of the empirical relationship is to build an energy balance model that focuses on a particular physical process responsible for the empirical relationship. Observed covariabilities of multiple variables provide insights into how variables change when they are perturbed. Using radiative forcing as a perturbation combined with feedback constructed based on observed covariability from observations used in the energy balance model, we can test whether or not the model can reproduce the feedback predicted by climate models (e.g., Boos and Korty 2016).
Inputs and algorithms to produce Ed4 EBAF-surface are revised from those for Ed2.8. The reanalysis product GEOS-5.4.1, which provides temperature, specific humidity, and ozone profiles, is used for the entire record of Ed4. In addition, MODIS Collection 5 is used from March 2000 through February 2017 and Collection 6 is used from March 2017 onward. The Ed4 cloud algorithm is used to detect clouds from MODIS and geostationary satellite observations, which produces more low-level clouds compared with the Ed2.8 cloud algorithm. The Ed4 cloud algorithm also detects more thin high-level clouds compared with the Ed2.8 cloud algorithm. In addition, using up to five channels to retrieve cloud properties from geostationary satellite observations improves nighttime cloud properties. Ed4 EBAF all-sky global annual mean TOA shortwave and longwave irradiances are, respectively, smaller by 0.5 W m−2 and larger by 0.5 W m−2 compared with Ed2.8 counterparts, while clear-sky global annual mean TOA irradiance differences are larger, especially for longwave. The algorithm to produce Ed4 EBAF-surface consists of two parts: bias correction and Lagrange multiplier. The bias correction adjusts the TOA and surface irradiance error because of the bias in 500 to 200 hPa temperature and humidity, and cloud fraction viewed from space and the surface.
Even though inputs and algorithms used in Ed4 data products and algorithms of Ed4 EBAF-surface are significantly revised, the global annual mean surface upward and downward shortwave and longwave irradiances change less than 1 W m−2 compared with Ed2.8 irradiances. The change in the global annual mean surface shortwave plus longwave net irradiance is 1.2 W m−2. The uncertainty of surface irradiances given by Kato et al. (2013) is also revised using surface observations. Differences of revised values from those given in Kato et al. (2013) are small. If we assume errors in all surface irradiance components are independent, the uncertainty in the global annual mean surface net irradiance is 8 W m−2. Downward irradiance measured at 46 buoys and 36 land sites is used to evaluate monthly mean downward irradiances. Mean biases of Ed4 EBAF-surface downward shortwave and longwave irradiances separately averaged for all buoys and land sites are less than 5 W m−2, which is within the uncertainty of surface measurements. Mean biases of downward shortwave and longwave irradiance averaged separately for four Arctic and four Antarctic sites are also smaller than 5 W m−2. However, because of a larger positive bias of cloud fraction in high altitude over polar regions during polar night, the mean bias of monthly mean downward longwave irradiance at the Greenland Summit site is 11 W m−2. This is also the result of large spatial and temporal variability of surface irradiances over polar regions that warrants caution when interpreting the validation results from only four sites in the Arctic or Antarctic.
This research has been supported by the NASA CERES project. The Ed4 CERES EBAF-surface (https://doi.org/10.5067/TERRA+AQUA/CERES/EBAF-SURFACE_L3B004.0), EBAF-TOA (https://doi.org/10.5067/TERRA+AQUA/CERES/EBAF-TOA_L3B004.0), SYN1deg-Month (https://doi.org/10.5067/TERRA+AQUA/CERES/SYN1degMonth_L3.004A), and SYN1deg-Hour (https://doi.org/10.5067/TERRA+AQUA/CERES/SYN1DEG-1HOUR_L3.004A) dataset was downloaded from https://ceres.larc.nasa.gov/order_data.php. Surface observations of downward irradiances were collected from a number of sources. For land-based observations we include 19 sites from the BSRN (Ohmura et al. 1998), available at www.bsrn.de. Eight surface sites come from NOAA’s Global Monitoring Division (GMD) and SURFRAD network at NOAA’s Air Resources Laboratory/Surface Radiation Research Branch (Augustine et al. 2000), and are available from www.esrl.noaa.gov/gmd/grad. Nine surface sites come from the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) Program and are available from http://www.arm.gov/data. Buoy observations come from two sources. The Upper Ocean Processes Group at Woods Hole Oceanographic Institution maintains the Stratus, North Tropical Atlantic Station (NTAS), and Hawaii Ocean Time-Series (HOTS) buoys providing valuable time series of observations in climatically important regions of the ocean. Data are available at http://uop.whoi.edu/. We would also like to acknowledge the Tropical Atmosphere Ocean/Triangle Trans-Ocean Buoy Network (TAO/TRITON) Project Office of NOAA’s Pacific Marine Environmental Laboratory (PMEL), where three groups of buoy data were downloaded: in the Pacific, the TAO/TRITON data; from the tropical Atlantic Ocean, the Prediction and Research Moored Array in the Tropical Atlantic (PIRATA); and the Research Moored Array for African–Asian–Australian Monsoon Analysis and Prediction (RAMA) in the Indian Ocean, as well as the Kuroshio and Papa buoy sites. All data are made available at http://www.pmel.noaa.gov/. We thank the editor Dr. Stephen Klein, Dr. Jörg Trentmann of Deutscher Wetterdienst, and two anonymous reviewers for helpful comments and suggestions.
EBAF-Surface Lagrange Multiplier Algorithm
Following Eq. (A4) of Kato et al. (2013), we minimize Z, of which components are surface, cloud, and atmospheric property changes as well as irradiance uncertainties under the constraint that the change of computed TOA shortwave and longwave irradiance matches with the difference between computed and observed TOA irradiance to within their uncertainties. (Rose et al. 2013). We define the difference of computed Fcomp and observed Fobs top-of-atmosphere (TOA) irradiances ΔF = Fcomp – Fobs, where the elements of F are all-sky TOA shortwave and longwave irradiances. The difference ΔF also contains four other elements, surface upward and downward irradiance bias corrections.
We then express Y as
the clear or cloud fraction change is δCi, atmospheric and surface property change is δV, and irradiance residual that is not taken out from this adjustment is δF. Elements of these vectors are normalized by the respective uncertainty s so that
where subscript i indicates clear and all-sky (i ≤ 2), k is the number of variables (k ≤ n, n = 9 ) such as temperature, water vapor amount, and cloud properties and j is TOA and surface shortwave and longwave irradiances (j ≤ 6). In Eq. (A1), LT = [λ1, λ2, …, λ6], Fi contains computed TOA and surface shortwave and longwave irradiances, the 6 × n matrix i contains the partial derivatives of TOA and up and downward surface irradiances with respect to cloud and atmospheric properties, λ represents Lagrange multipliers, is an n × n matrix of which diagonal terms are sυ,k, and δv and δf are, respectively, n- and 6-element vectors. Note that sizes of δv, δf, and the matrix are different from those used in Kato et al. (2013). Clear-sky irradiance constraint is similar to all-sky but i = 1 (clear-sky only) and there is no δCi.We rewrite Eq. (A1) as
Unknowns are δci, δυk, δfj, and Lagrange multipliers. We take a derivative of Y with respect to δci, δυk, δfj, and with respect to Lagrange multipliers:
Multiplying Eq. (A7) by sc,i leads to
Rearranging Eq. (A13) leads to
Eq. (A14) sets up a matrix equation
The diagonal element of is
and the off-diagonal element is
Once λx is known, summing up Eq. (A12) for all m and using