The Intergovernmental Panel on Climate Change (IPCC 2014) Fifth Assessment Report (AR5) and the three special reports of the AR6 cycle state that mankind and the biosphere face great threats due to the rapidly changing climate (IPCC 2013, 2018, 2019a,b). To support political decisions on climate change mitigation and adaptation, and to quantify the implications for economic and noneconomic loss and damage, the United Nations Framework Convention on Climate Change (UNFCCC) requires systematic monitoring of the global climate system (e.g., Doherty et al. 2009; UNFCCC Art. 4 and Art. 5, 1992; Paris Agreement 7.7c, Adaptation). In particular, systematic monitoring is important in assessing progress on the aims of the Paris Agreement (e.g., for the global stocktake). The main tools at hand to determine the extent and impacts of climate change on local to global scales and understand its causes are a combination of global and regional climate and Earth system models, reanalysis data, and systematic observations. The latter are indispensable for all Earth system domains (atmospheric, terrestrial, and oceanic) to increase the understanding of and quantify processes, budgets, and reservoirs within the global Earth cycles (carbon, energy, and water).
To promote systematic climate monitoring, the World Meteorological Organization (WMO), Intergovernmental Oceanographic Commission (IOC), United Nations Environment Programme (UNEP), and International Science Council (ISC), established in 1992 the Global Climate Observing System (GCOS). GCOS aims at sustained “provision of reliable physical, chemical, and biochemical observations and data records for the total climate system—across the atmospheric, oceanic, and terrestrial domains, including hydrological and carbon cycles and the cryosphere” (GCOS 2016). GCOS defined a set of currently 54 “essential climate variables” (ECVs) (Bojinski et al. 2014) that must be observed in a sustained and consistent manner to enable detection of climate trends and provide data suitable for climate model evaluation and climate change attribution.
Complementary to relatively sparse airborne and ground-based measurements and inventory data, satellite observations are of ever-growing importance for evaluating, initializing, and parameterizing Earth system processes represented in models. This growing importance is due to the increasing satellite global coverage and resolution (in space and time), their improved calibration accuracy, and the increasing diversity of relevant observables provided by advances in satellite sensor technologies. Satellite observations can provide a significant contribution for 21 out of the 54 GCOS ECVs. Some of these are exclusively derived from satellite measurements (e.g., the Earth radiation budget), whereas for others dedicated spaceborne sensors provide better coverage but lower accuracy or resolution than in situ measurements (e.g., aboveground biomass, column atmospheric concentration of CO2 and CH4).
Studies of the Earth system require combined analysis of datasets of many variables. Since these are derived from different sources (satellite, ground, air, and model based) and processing systems, one underlying precondition of any such analysis is that the datasets are consistent. However, despite the importance of consistency, many open questions remain, ranging from a clear definition of consistency for multiple quantities to systematically assessing consistency between the many data records used.
Possible reasons for inconsistencies include the use of different auxiliary datasets, simplifications in corrections and retrieval algorithms, calibration uncertainties, and differences in sampling and gridding. For example, a time series of a single variable built from data records obtained from different sensors may exhibit “jumps” where they are merged with each other, which may hinder any trend analysis. Another example is assigning different land cover classes (e.g., glacier, water, rock, or vegetation) to the same pixel by using different glacier masks, which may lead to highly variable budget calculations of related exchange processes.
Consistency as an issue in creating satellite-based data records was first met by operational entities like NOAA, EUMETSAT, or NASA within their near-real-time (NRT) processing chains across different satellite missions. This includes aspects such as common input datasets, gridding methodology, cloud and land/sea masking, aerosol and water vapor corrections, and the land-cover map used. The measures taken are typically documented in algorithm theoretical basis documents [e.g., consistent Ozone Monitoring Instrument (OMI)–MODIS cloud products; Siddans 2016; or merged Tropospheric Monitoring Instrument (TROPOMI)–VIIRS cloud product; NASA 2014]. However, the need for consistency across different variables, domains, and processing systems is inherent in climate studies and thus much broader than in the often independent NRT applications.
Over the past 10 years, space agencies (including ESA, EUMETSAT, NASA, and NOAA) have emphasized the generation and delivery of satellite-based CDRs. Hollmann et al. (2013) describe the efforts of ESA through its Climate Change Initiative (CCI). CCI leverages and harvests the long-term satellite archives available from European and other satellites, and enhances or expands these records with observations from other space agencies to obtain global coverage. In addition, CCI is extending its newly established CDRs with the most recent satellite instruments to guarantee continuation into the future using operational missions (e.g., Sentinel). During its first six years (2011–17), CCI implemented 14 projects, each targeting one (or two) ECVs; in 2018, CCI was expanded to include nine additional ECVs, as shown in Fig. 1. It should be noted that most of the ECVs consist of several quantities, so-called products (detailed information on the products of each CCI ECV for which CDRs have been processed in CCI is available at http://cci.esa.int). Of course, products within a particular ECV have to be consistent. A particular element within the CCI program is independent analysis of the quality of its CDRs and particularly their consistency (between different ECVs and products) in a climate modeling context by the CCI Climate Model User Group (CMUG) and several budget closure study projects.
Temporal coverage of CDRs for ECVs analyzed by CCI. Filled bars indicate CDRs available in 2019, and outlined bars indicate CDRs that are planned within the ongoing phase of the CCI program.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Temporal coverage of CDRs for ECVs analyzed by CCI. Filled bars indicate CDRs available in 2019, and outlined bars indicate CDRs that are planned within the ongoing phase of the CCI program.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Temporal coverage of CDRs for ECVs analyzed by CCI. Filled bars indicate CDRs available in 2019, and outlined bars indicate CDRs that are planned within the ongoing phase of the CCI program.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Together with the Copernicus Climate Change Service (C3S) and contributions from EUMETSAT through its Satellite Application Facilities (SAFs) such as the Climate Monitoring SAF (Schulz et al. 2009), the NOAA Climate Data Record program (www.ncdc.noaa.gov/cdr; Bates et al. 2016), and the NASA Measures program (https://earthdata.nasa.gov/measures), about 1,000 different satellite-based CDRs for GCOS ECV products and further variables are available or will become available in the near future. An overview of these CDRs is given in the ECV inventory (https://climatemonitoring.info/ecvinventory), recently established by the joint Committee on Earth Observation Satellites (CEOS)–Coordination Group for Meteorological Satellite (CGMS) Working Group on Climate, which conducts regular gap analysis to define future satellite development needs (WGClimate 2018).
This study introduces a concept developed in CCI to define and assess consistency between multiple satellite-based ECV products. It is shown that such an assessment allows remaining inconsistencies to be identified and quantified in the light of given CDR uncertainties and relevant physical principles. A key application of assessing and ensuring consistency is in closure studies where multiple CDRs are used together. A selection of topics for such closure studies is briefly discussed in this paper to illustrate the concept.
The second section discusses different kinds of inconsistencies and develops a definition of consistency, followed by a brief analysis of ECVs covered by CDRs from CCI and consistency needs in Earth system monitoring in the third section. The fourth section develops a concept for assessing the different levels of consistency and illustrates it with examples from different ECV products in CCI. The fifth section presents a discussion of the main findings and identifies remaining consistency gaps.
Consistency in Earth system monitoring
While “consistency” (e.g., between two datasets) is a concept frequently referred to in the observation community, there is, to our knowledge, no comprehensive definition specific to observation datasets of different variables. This may reflect the complexity of relations between the large set of ECVs. This study proposes such a comprehensive definition and an assessment concept for consistency. The focus is on consistency between datasets of different variables, as needed for climate studies, but also single-variable cases are included.
According to the common definition of the word “consistency” (Oxford Dictionary), it is “the quality of always behaving in the same way or of having the same opinions or standards; the quality of being consistent, i.e., 1/in agreement with something; not contradicting something, 2/happening in the same way and continuing for a period of time, 3/consistent with something in agreement with something, not contradicting something, 4/having different parts that all agree with each other.” In the observation scientific community, consistency is usually understood as “agreement,” “compatibility,” or “no contradiction.” When considering CDRs, “consistency” goes beyond “agreement” and rather refers to “compatibility.” First, agreement per se can only be tested between datasets of the same variable. A mature terminology and a comprehensive set of mathematical tools for this purpose exists, which forms the basis of most calibration, validation, and model evaluation activities. Second, there can even be cases where two datasets of the same variable agree (their bias is smaller than their combined uncertainties) but are inconsistent (e.g., if only one of two datasets shows a distinct diurnal or seasonal cycle). In contrast, regionally averaged time series of one variable can disagree (have regional biases larger than the combined uncertainties), but be consistent in their temporal behavior, as shown for multisensor aerosol optical depth (AOD) records (Sogacheva et al. 2020).
In a physical sense, consistency can be understood as fulfilling a conservation balance equation (of mass or energy) or exhibiting a correlation in time or space between two data records as expected by a physical theory. In CDR production, also simple category inconsistencies occur (e.g., for one pixel land cover assigns bare soil, while biomass gives a nonzero carbon mass to it).
Immler et al. (2010) defined consistency between measurements of the GCOS Reference Upper Air Network (GRUAN) as “when the independent measurements agree to within their individual uncertainties,” which requires knowledge of their (combined) uncertainties. This definition applies to different measurements of the same variable, but in the wider context of Earth system monitoring, a definition of consistency across multiple ECVs is also needed.
Several kinds of inconsistency between different data records of the same quantity or of different quantities can be recognized:
-
inconsistencies due to differences in auxiliary datasets;
-
temporal inhomogeneities in time series (e.g., due to calibration biases, degradation in data obtained from a sequence of different input data records, or sampling differences in terms of measurement time, frequency, or geographical coverage during gridding);
-
spatial inhomogeneities due to combining fields from different datasets (e.g., with different observing geometry or different sampling, e.g., all-sky vs clear-sky sampling).
Many of these inconsistencies are linked to the statistical properties of the raw data used to create a CDR, when for practical reasons simplifications and aggregations cannot be avoided.
To cover the wide range of aspects of consistency, it is convenient to structure it on three complementary levels:
While consistency on a technical level is easy to define and needs limited scientific insight, it is often a resource-consuming barrier hindering data use. Thus, the Earth observation community has focused on this area in recent years (e.g., by adopting common metadata standards). In particular, the CCI program has adopted existing solutions (and when needed developed new ones) that facilitate combined satellite-based CDR use. This includes a harmonized data format (netCDF, with a few exceptions where a different standard is needed for a particular community, e.g., shapefiles for glaciers) and a common metadata convention (CCI data standards; ESA 2019), which follow the climate and forecast (CF) convention (http://cfconventions.org). It covers additional cross-ECV standardized metadata attributes, using common vocabularies for index terms and harmonized variable names, as well as a harmonized/interoperable data access portal with common catalogue and data services to simplify multiquantity data search and download within the CCI portfolio (http://cci.esa.int/data). This common vocabulary also helps to reduce inconsistent nomenclature, such as labeling slightly different variables as the same retrieved quantity (e.g., due to wavelength dependencies of retrieved information). Furthermore, the underlying documentation of algorithms and datasets in CCI has been harmonized to some extent, as in other initiatives such as the SAF network or NOAA CDR program. This information helps users to quickly understand each dataset and its strengths, weaknesses, and limitations. A good example of the benefit of such harmonized climate data records on the technical level is given by the CCI toolbox (https://climatetoolbox.io), which can be used for harmonized data preprocessing, analysis and visualization of the multiple CDRs in a standardized way.
On the retrieval level, consistency aims at using the same (or a similar) observation strategy (same or similar satellite sensors, frequencies, etc.), and similar auxiliary datasets for the same variable in different retrieval algorithms. Those auxiliary datasets are either categorical datasets, so-called masks, or continuous datasets of physical variables. Typical masks used in many retrieval algorithms include, for example, a particular land cover (vegetated areas), land–water, sea ice, snow cover, and glacier masks, since many retrieval algorithms behave differently over different surface types. Other masks commonly needed across many variables are cloud masks, since many retrievals in the visible to thermal spectral range need to avoid contamination by clouds. Frequently used continuous auxiliary data fields include meteorological fields (e.g., from reanalysis) and climatologies of atmospheric variables (e.g., water vapor, aerosols, ozone) to conduct atmospheric corrections of visible bands used to retrieve land and ocean ECVs.
There is no sharp boundary between retrieval and scientific consistency. Ultimately, scientific consistency deals with the compatibility in CDR properties relevant for climate processes. All data records of a single ECV product, if obtained from different sources, need to be consistent within their uncertainties and within sampling differences. One aspect is consistency across borders in space (horizontally and vertically) and in time. Most importantly, systematic biases between datasets need to be avoided as they may lead to errors when evaluating model performance (e.g., Waugh and Eyring 2008). This applies to different combinations such as one variable based on multiple sensors, one sensor but using multiple algorithms, or combined satellite, model and in situ data. Finally, when several datasets of different variables are included in a physical model or budget equation, multivariable consistency needs to distinguish uncertainties of calculated closure budgets due to propagated input uncertainties from real physical process imbalances or net effects.
Consistency needs for CCI Earth system climate data records
In this section, linkages on the retrieval and scientific level between the different CCI ECVs (Fig. 1) are analyzed. This analysis remains at the high level of the GCOS ECVs while it is well understood that most ECVs consist of several different quantities, or so-called products (e.g., the glacier ECV in CCI consists of the three products: glacier outlines, elevation change, and velocity). In most of the analysis in this study the primary product of an ECV is considered (e.g., aerosol optical depth for aerosol properties) and the most common methodology used to retrieve it. This means that, for using a specific CDR of one ECV, there may be a need to assess in more detail its linkages if, for example, a new retrieval technique in another spectral range is considered or if another product of this ECV is assessed. Detailed information on the products of each CCI ECV for which CDRs have been processed is available online (at http://cci.esa.int).
As a first step, the needs for consistency between ECVs on the retrieval level are assessed. Retrievals of Earth system variables from satellite observations aim to produce high quality CDRs by constraining the (often underdetermined) inversion equations as good as possible. Typically, the measurements are chosen to have high sensitivity to the target variable, but they are usually subject to perturbations from other variables. In such cases, the inversion needs to either coretrieve these additional variables or use auxiliary datasets to describe their spatiotemporal distributions. Moreover, different retrieval algorithms are often optimal for use over different surface types as their reflectance or spectral characteristics are highly variable (e.g., over dark water or over bright land). The use of different approaches for obtaining the same variable in different retrieval algorithms is one possible source of inconsistency between CDRs.
After processing, all CDRs have to pass validation against external reference datasets (e.g., from ground-based stations) to quantify their accuracy. Furthermore, CCI insists for CDRs to be accompanied by proper uncertainty characterization (using error propagation or uncertainty characterization during validation) within their data files (Merchant et al. 2017), so that uncertainties can be assessed when using the datasets. However, since reference data can have temporal or spatial representativeness issues and different validation methods also have their inconsistencies, unexplored uncertainties may remain (for the retrieved values themselves and for the estimated uncertainties). Validation and error propagation implicitly quantify inconsistencies from using imperfect auxiliary datasets and retrieval simplifications to within uncertainties. However, proof of consistency needs to explicitly test together the CDRs considered.
The part of Table 1 that is above the diagonal summarizes links between ECVs generated and analyzed by CCI with regard to their retrieval consistency. A need for retrieval consistency is identified where either one or both retrievals rely on consistent coretrieved or auxiliary variables of the other ECV (links only within CCI are considered, but there are other products, algorithms, or sensors for which these may not apply).
Links between ECVs on the retrieval (above the shaded diagonal) and scientific (below the shaded diagonal) level, which need to be consistent if used together. Weak linkages are indicated in parentheses. Cycles are indicated with the following acronyms: C = carbon cycle; W = water cycle; E = energy cycle. Processes are indicated with the following acronyms: r = radiation interaction; d = deposition; e = emission/evaporation; t = transport; c = chemical transformation; mtf = melting/thawing/freezing; i = ecosystem interaction; a = air land/sea fluxes of carbon and water; m = mask.
The part of Table 1 below the diagonal summarizes the need for consistency on the scientific level based on our knowledge of how two variables are linked by Earth system processes or cycles in more detail. For this, the relevance of CCI ECVs for the energy, water and carbon cycles is briefly recalled. Figure 2 lists available or upcoming ECVs for which ESA CCI generates CDRs that contribute to the characterization of these three main cycles. For simplicity, each ECV is only attributed to the cycle in which it plays the most important role. Practically all ECVs contribute to the energy cycle, either directly through radiation interaction or through mass-attached energy transport in the water or carbon cycle. Studies of subelements of these main cycles may also be relevant (e.g., physical processes such as emission, transport, deposition, or radiation interactions, and chemical transformations; also regional limitations, such as ice-free conditions) that may only require consistency among a reduced set of ECVs. Some further details on the CCI CDRs for the three cycles are provided in the following.
The ECVs covered by ESA CCI CDRs, ordered according to the key Earth system cycle (energy, carbon, water) they help characterize. The cycles are interlinked, and most water and carbon cycle ECVs are also relevant to the energy cycle, since energy is stored and transported in water and matter, at least on transient time scales.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
The ECVs covered by ESA CCI CDRs, ordered according to the key Earth system cycle (energy, carbon, water) they help characterize. The cycles are interlinked, and most water and carbon cycle ECVs are also relevant to the energy cycle, since energy is stored and transported in water and matter, at least on transient time scales.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
The ECVs covered by ESA CCI CDRs, ordered according to the key Earth system cycle (energy, carbon, water) they help characterize. The cycles are interlinked, and most water and carbon cycle ECVs are also relevant to the energy cycle, since energy is stored and transported in water and matter, at least on transient time scales.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Carbon cycle.
CCI CDRs help quantifying the dynamics of the amount of carbon stored in the atmosphere, oceans, and terrestrial biosphere and the fluxes between these reservoirs [see overview about the carbon cycle in Le Quéré et al. (2018)]. CO2 in the atmosphere is a key measure of the anthropogenic perturbation to the carbon cycle. The air–sea CO2 flux is strongly affected by sea surface temperature (SST) and ocean photosynthetic activity (monitored using ocean-color observations). The CCI CDRs also help constraining carbon fluxes from the land biosphere (e.g., Reuter et al. 2017) including land-use change and biomass-burning emissions, together with direct estimates of aboveground biomass and burned area (Chuvieco et al. 2019). Other CCI CDRs of importance to the carbon cycle are snow cover (which affects the duration and start of photosynthetic processes in boreal forests; Pulliainen et al. 2017), similar to the impact of sea ice on marine photosynthesis in high latitudes, soil moisture (which affects land–atmosphere CO2 fluxes), permafrost (which contains frozen carbon stores with about twice the mass of atmospheric carbon), and sea surface salinity, which, together with SST, determines CO2 solubility, with important impacts in rainy regions and serves as a proxy for seawater alkalinity (Vinogradova et al. 2019).
Water cycle.
CCI helps quantifying the global water cycle over land and ocean (see overview in, e.g., Levizzani and Cattani 2019) by providing CDRs related to the reservoirs within the water cycle (lake levels, sea level, sea ice, ice sheets, glaciers, soil moisture, and snow), atmospheric water vapor content (water vapor), and clouds. From these, processes such as precipitation and runoff that transfer water between the various reservoirs may be inferred. CCI delivers additional relevant parameters such as sea surface salinity (related to precipitation, evaporation, and runoff), SST and land surface temperature (LST) (determining evaporation), land cover, and biomass (both linked to evapotranspiration).
Energy cycle.
CCI also helps constraining the global energy cycle (for an overview, see Allan 2012) by providing CDRs for SST and LST, land, and sea ice, as well as snow cover, sea level (which is affected among others by the ocean heat content and land ice melt), sea state, clouds, water vapor, ozone, greenhouse gases, and aerosols that help determine the vertical temperature structure of the atmosphere. Finally, the biosphere (biomass) may also be considered a part of the energy cycle since it converts solar energy into chemically stored energy (organic matter). In the oceans, a significant portion of the organic matter sinks out of the surface layers, exporting the energy to the deep ocean (with the photosynthesis activity being observed indirectly through ocean color).
Concept for assessing consistency on different levels
Due to the complexity of different consistency aspects, no single method can be used for assessing consistency of CDRs on various levels. Therefore, a concept employing a range of appropriate methods was developed in CCI, which is summarized here and then illustrated with short examples.
Overview: Methods to assess consistency.
All methods for assessing consistency contain several key elements. First, any method needs to be based on physical background knowledge to understand the relevance of any disagreement or incompatibility. Such background knowledge can be a simple principle (e.g., if the land cover is bare soil and the biomass product provides a high biomass value, there is an obvious inconsistency) or knowledge of the sensitivity of a target variable toward an auxiliary dataset, or a more complex physical equation or “model.” Second, any assessment needs to select an appropriate characteristic (patterns, time series, masks) tailored toward the relevant process (or cycle) and choose a suitable mathematical tool (metric). Finally, this metric needs to be evaluated against the relevant physical background knowledge while the threshold on the chosen metric for judging consistency depends on the considered process or cycle and the datasets. To make any assessment of consistency objective, a study needs to specify the threshold used. This is shown for the following examples for various metrics.
In essence, consistency then means that several datasets have been evaluated against the underlying physical background knowledge and were found “fit for purpose” for a specific application domain. This leads to cases where seemingly small values of a chosen metric (compared to its uncertainty) can mean inconsistency, while in other cases apparently large deviations mean consistency, as will be shown in the examples of this section. Table 2 lists a variety of related basic principles and methods to assess consistency on different levels used in the following examples.
Summary of assessment methods for consistency on different levels and types.
Methods to assess retrieval level consistency.
As a principle, retrieval level inconsistencies become significant if the difference of the auxiliary data used in two independent processing systems multiplied by the sensitivity of the target variable to the respective auxiliary variable is larger than the target uncertainty. This means that testing retrieval level consistency needs to assess auxiliary dataset differences in the light of target variable sensitivities or incompatibilities.
Consistency OF Categorical Auxiliary Datasets (Masks).
A first approach to assess consistency of masks used in independent retrievals lies in visual inspection of combined maps of datasets, as for example, of surface temperature composed from four independent CDRs for land surface (LST), sea surface (SST), ice surface (IST), and lake surface water (LSWT) temperatures against required pixel-level agreement of the masks. In CCI the four retrievals use a common land–sea mask (and sea ice mask), but apply different cloud mask algorithms optimized over land, sea ice, and water surfaces. As shown in Fig. 3, the reader can visually confirm the absence of any obvious scatter near the land–sea borders, which indicates that the land–sea masks used in the different processing systems are consistent. Additionally, the application of different optimal cloud masking in the retrievals for LST and SST has led to obvious discontinuities in the sampling with temperature observations at the land–sea border, which may be judged as second-order inconsistencies. Such visual inspection of a set of typical scenes can be employed for most ECVs to get an understanding of their physical consistency within one variable across borders of the same mask used in different retrievals. Additionally, Fig. 3 shows a case where a contrast in the values in the ECVs between neighboring pixels (surface temperature of ocean and water) does not mean inconsistency, but reflects physical differences arising from the different heat capacities of water and land.
Gaps in surface temperature fields (LST and SST from SLSTR on Sentinel-3A at 1038 UTC 5 Aug 2018) due to masked clouds (gray), showing the absence of scatter at land–sea borders and sampling discontinuities across some land–sea boundaries due to different cloud-clearing approaches between LST and SST processing.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Gaps in surface temperature fields (LST and SST from SLSTR on Sentinel-3A at 1038 UTC 5 Aug 2018) due to masked clouds (gray), showing the absence of scatter at land–sea borders and sampling discontinuities across some land–sea boundaries due to different cloud-clearing approaches between LST and SST processing.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Gaps in surface temperature fields (LST and SST from SLSTR on Sentinel-3A at 1038 UTC 5 Aug 2018) due to masked clouds (gray), showing the absence of scatter at land–sea borders and sampling discontinuities across some land–sea boundaries due to different cloud-clearing approaches between LST and SST processing.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
The retrieval of many ECVs needs a cloud mask to avoid cloud contamination. Also cloud properties need a cloud mask to ensure that a pixel truly represents cloud (Poulsen et al. 2012). When, for example, aerosol and cloud property retrievals for the same sensor are implemented as separate algorithms (as is usually the case), individual pixels need to be analyzed either as cloud or as aerosol; analysis of the same pixel as aerosol and as cloud under the wrong assumption (cloud-free or aerosol-free) could severely degrade the retrievals and must be minimized (e.g., Sogacheva et al. 2017; Li et al. 2009). To assess if this principle is fulfilled, independent AATSR cloud masks used in the aerosol and cloud products were analyzed for four days in September 2008 (covering difficult scenes with high aerosol loads or complicated mixtures of aerosol and clouds). Figure 4 shows a map of different combinations of cloud/no cloud assignment by the two cloud masks and a contingency matrix of those class combinations. The matrix shows, that while 21% of observations are not used for aerosol or cloud retrievals at all (losing sampling coverage but not leading to inconsistency), only 0.3% of them were found to violate the physical principle (i.e., no pixel must be double analyzed as clouds and as aerosols). Even if a very stringent threshold for this fraction of 1% is set (since cloud mask errors lead to very large AOD errors) the two cloud masks are fully consistent. The map also shows that the inconsistent cases (yellow pixels) occur only over land but in all climate zones. Together with the underlying physical principle one can use such a contingency matrix/mapping of class combinations to assess the contingency of masks and to understand where/when inconsistencies mostly occur and need to be corrected.
Consistency overview between Aerosol_cci (Swansea University) and Cloud_cci (FAME-C) AATSR cloud masks for observations of four selected days in September 2008. “No cloud/no cloud” and “cloud/cloud” situations are solely analyzed as aerosol or clouds in Aerosol_cci and Cloud_cci, respectively. “No cloud/cloud” situations are wrongly analyzed as aerosols and clouds, while “cloud/no cloud” situations are not analyzed at all.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Consistency overview between Aerosol_cci (Swansea University) and Cloud_cci (FAME-C) AATSR cloud masks for observations of four selected days in September 2008. “No cloud/no cloud” and “cloud/cloud” situations are solely analyzed as aerosol or clouds in Aerosol_cci and Cloud_cci, respectively. “No cloud/cloud” situations are wrongly analyzed as aerosols and clouds, while “cloud/no cloud” situations are not analyzed at all.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Consistency overview between Aerosol_cci (Swansea University) and Cloud_cci (FAME-C) AATSR cloud masks for observations of four selected days in September 2008. “No cloud/no cloud” and “cloud/cloud” situations are solely analyzed as aerosol or clouds in Aerosol_cci and Cloud_cci, respectively. “No cloud/cloud” situations are wrongly analyzed as aerosols and clouds, while “cloud/no cloud” situations are not analyzed at all.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Another typical aspect of multiquantity spatial consistency is the agreement of locations between the outlines of physically related quantities (different products within one ECV, between different ECVs). For example, glacier outlines are derived from high-resolution satellite imagery or aerial photography using semiautomated mapping techniques or manual on-screen digitization (Paul et al. 2015). Due to their higher spatial resolution, the location of glaciers can be used for land cover as an independent validation source for its “permanent ice and snow” classes. Furthermore, glacier maps serve as an important auxiliary dataset for clouds and LST (to choose the correct retrieval algorithm), and lakes as a reciprocal mask (these can only occur in places not covered by glaciers) for sea ice, ice sheets and permafrost. Again, contingency matrices between glacier or lake location and the other variables can be used to assess consistency in the light of the expected compatible combinations; the threshold for the acceptable fraction of inconsistent pixels needs to be set depending on the potential harm of misclassifications. A limitation for the assessment of categorical auxiliary datasets lies in the fact that mixed cases often exist, in particular for coarser spatial resolutions.
Consistency OF Continuous Auxiliary Datasets OF The Same Quantity.
Often the retrieval of a land/ocean CDR is affected by perturbations in the measured bands due to atmospheric absorption or scattering, so an atmospheric correction needs to be applied. Examples of necessary atmospheric corrections include visible or thermal retrievals impacted by aerosol, water vapor, ozone, or other trace gases (e.g., Popp 1995). A first step in algorithm development would be to assess the sensitivities of the measured reflectances to the various absorbing trace gases and to aerosol particles (e.g., Holzer-Popp et al. 2002). This provides the basis for deciding which corrections can be neglected or made with a simple parameterization and which need more precise corrections using an auxiliary dataset of distributions of the responsible agents influencing the signal. When the auxiliary datasets come from the same sensor as the target CDR, accurate spatiotemporal matching (pixel collocation) would be possible. However, in cases where the auxiliary data come from different sensors, it may be necessary to deal with spatial and temporal mismatches, introducing a requirement for assessment of the associated additional uncertainties. Figure 5 shows a gridded map of differences of aerosol optical depth between the by-products of the ocean-color atmospheric correction of Medium Resolution Imaging Spectrometer (MERIS) data (processed using a NASA algorithm) and the corresponding CCI aerosol ECV product from Advanced Along Track Scanning Radiometer (AATSR), both for 865 nm (both sensors were on board the same platform Envisat and thus exhibit zero time difference). The global average difference of AOD between both products of 0.03 is acceptable for the purpose of aerosol corrections, but the variability is larger for the aerosol ECV product than for the ocean-color product (0.10 ± 0.11 vs 0.13 ± 0.04, respectively), which has higher AOD values over the open ocean, but lower ones closer to land. Given the importance of AOD in ocean-color atmospheric correction (IOCCG 2010), the aerosol-corrected ocean-color ECV can be regarded as consistent with the aerosol ECV in its global average, but not regionally. These results merit further investigations to identify the sources of the discrepancies and to assess the potential to improve the MERIS ocean-color atmospheric correction algorithm by using concurrent auxiliary information on AOD from the main ECV aerosol product obtained from AATSR.
Mean AOD differences at 865 nm between ocean-color MERIS atmospheric correction by-product and aerosol ECV product from AATSR in May 2003 when both instruments retrieve AOD.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Mean AOD differences at 865 nm between ocean-color MERIS atmospheric correction by-product and aerosol ECV product from AATSR in May 2003 when both instruments retrieve AOD.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Mean AOD differences at 865 nm between ocean-color MERIS atmospheric correction by-product and aerosol ECV product from AATSR in May 2003 when both instruments retrieve AOD.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Methods to assess scientific consistency.
Scientific consistency includes self-consistency within one quantity (when independently retrieved pieces are integrated into a longer time series or a larger map) and mutual consistency between different quantities (different products of one ECV or multiple ECV CDRs) as a consequence of all types of retrieval inconsistencies, limitations of the retrieval algorithms or sensor calibrations, as well as sampling differences between aggregated datasets.
Self-Consistency OF A Single Quantity.
One major problem of satellite-based CDRs is that satellite instruments typically survive in orbit only for a limited time, so that a long-term record needs to be constructed from combining data from a time series of similar sensors. Plotting regional or global data records of the related parts of a time series often allows visual inspection of their consistency, where “jumps” or “breakpoints” are obvious against background knowledge of any known or absent true discontinuities. As example, column-averaged dry-air mole fractions (“vertical columns” XCO2) of carbon dioxide (Buchwitz et al. 2015) from the greenhouse gas (GHG) ECV are selected. Those CDRs serve as input data for inverse modeling schemes to improve the knowledge on natural and anthropogenic sources and sinks (e.g., Reuter et al. 2017). In creating a multisensor CDR covering a longer time period, a merging algorithm (EMMA; Reuter et al. 2013, 2020) corrects potential remaining offsets of individual datasets to avoid jumps in the merged time series. In EMMA, the ensemble members have been bias corrected and brought to common a priori CO2 profiles before being combined to obtain the merged product. Figure 6 shows in the top panel the resulting multisensor, multialgorithm monthly mean XCO2 merged record for 2003–18 for northern midlatitudes (30°–60°N) with the known nearly linear increase in time and seasonal cycle and no remaining biases, while in the bottom panel differences between individual ensemble members and the merged product before the corrections are shown to be larger than the required XCO2 uncertainties of 0.5 ppm. In this case, this threshold for the target uncertainties of the gap-corrected merged dataset is defined by the user requirement for the application of XCO2 trend analysis.
(top) Time series of monthly mean northern midlatitude XCO2 (red thick line) based on merging individual XCO2 ensemble members (black lines) from GOSAT (since 2009) and OCO-2 (since 2014). The time series (2003–18) begins with one XCO2 product from SCIAMACHY/Envisat. (bottom) XCO2 difference between ensemble members (black lines) and the multisensor/multialgorithm merged product (red line in top panel). For details, see Reuter et al. (2020).
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
(top) Time series of monthly mean northern midlatitude XCO2 (red thick line) based on merging individual XCO2 ensemble members (black lines) from GOSAT (since 2009) and OCO-2 (since 2014). The time series (2003–18) begins with one XCO2 product from SCIAMACHY/Envisat. (bottom) XCO2 difference between ensemble members (black lines) and the multisensor/multialgorithm merged product (red line in top panel). For details, see Reuter et al. (2020).
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
(top) Time series of monthly mean northern midlatitude XCO2 (red thick line) based on merging individual XCO2 ensemble members (black lines) from GOSAT (since 2009) and OCO-2 (since 2014). The time series (2003–18) begins with one XCO2 product from SCIAMACHY/Envisat. (bottom) XCO2 difference between ensemble members (black lines) and the multisensor/multialgorithm merged product (red line in top panel). For details, see Reuter et al. (2020).
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Similarly, spatial inconsistencies in one variable can often be assessed visually by looking at maps combined from independent pieces (different sensors, different overpass times of the same sensor with different observing geometries, different algorithms). In this case, inconsistencies are visible as artificial border lines or gradients that are larger than the noise in the image. Again, physical understanding is needed to decide whether a discontinuity at a physical border is real or erroneous (e.g., surface temperature often shows true differences between land and sea as shown in Fig. 3, while a dust plume should be continuous). Another example for spatial inconsistencies revealed by data overlay are glacier outlines derived from satellite images that have been orthorectified with different digital elevation models (DEMs). In steep and/or high topography geolocation shifts of several pixels (about 30–90 m) can occur, making any change assessment (trend analysis) or joint use of sensors nearly impossible (Kääb et al. 2016).
Another way of testing the consistency of independently retrieved CDRs for one variable is by comparing estimates of a derived quantity such as their trend with a physical equation. For example, within the GEWEX Water Vapor Assessment (G-VAP; see http://gewex-vap.org/ for details), intercomparisons of total-column water vapor (TCWV) trend estimates from different CDRs were made and it was concluded that the trend estimates are generally significantly different. It was then shown that several data records disagree with the physical expectation from the Clausius–Clapeyron equation using data over the global ice-free ocean (Schröder et al. 2016, 2019). After homogenization, a new analysis was applied to the trend estimates and associated results are shown in Fig. 7. While the diversity in original trend estimates (−0.15 to +0.12 kg m−2 yr−1) is several times higher than individual uncertainties, it is largely reduced after homogenization (−0.02 to +0.04 kg m−2 yr−1), but still slightly larger than the individual trend uncertainties (up to ± 0.01 kg m−2 yr−1). As a consequence, after homogenization there was a significant increase in the fraction of datasets that can be seen as consistent as indicated by agreement of trends within twice their combined uncertainties.
Trend estimates computed after (green) and before (black) homogenization for all long-term TCWV data records available from the G-VAP data archive (Schröder et al. 2018). Trend estimates are sorted in ascending order without homogenization. The gray horizontal line marks a trend of 0 kg m−2 yr−1 (updated from Schröder et al. 2019).
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Trend estimates computed after (green) and before (black) homogenization for all long-term TCWV data records available from the G-VAP data archive (Schröder et al. 2018). Trend estimates are sorted in ascending order without homogenization. The gray horizontal line marks a trend of 0 kg m−2 yr−1 (updated from Schröder et al. 2019).
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Trend estimates computed after (green) and before (black) homogenization for all long-term TCWV data records available from the G-VAP data archive (Schröder et al. 2018). Trend estimates are sorted in ascending order without homogenization. The gray horizontal line marks a trend of 0 kg m−2 yr−1 (updated from Schröder et al. 2019).
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Mutual Consistency between Different Quantities.
In testing multiple quantity consistency, the role of the underlying background knowledge becomes stronger since the physical processes connecting different ECVs need to be taken into account. One method to test the consistency of two ECVs is by looking at their correlations. For example, in the lower stratosphere, the strong physical dependency of lower-stratospheric water vapor on tropical tropopause temperatures can be exploited to test the consistency between climate data records of temperature and stratospheric water vapor as highlighted by Hegglin et al. (2014). This study proposed a new merging method that uses a chemistry–climate model as a transfer function between different satellite instrument records to create a CDR. The methodology allows the bias between instruments to be determined throughout the instrument’s lifetime and not only for the overlap period (when old instruments may show first signs of degradation), hence improving characterization of systematic differences (or biases) between datasets. By using the correlation between the newly merged stratospheric water vapor record and the zonal mean temperature from ERA-Interim, visual inspection indicated that the new merging method led to physically more consistent results than the traditional one based on bias correction of instruments during overlap periods. Figure 8 shows the visually well correlated time series of a prototype version of the stratospheric water vapor CDR merged using the new methodology in comparison with zonal mean temperatures at 100 hPa from ERA5 in the tropical region (left panel). We set the threshold for correlations to accept consistency with medium (high) confidence to 0.5 (0.7) since the covariability of time series of two different variables may also be influenced by other processes that reduce the correlation. In this case (right panel), a correlation of 0.58 suggests that the two variables are physically consistent with medium confidence; if only assessing the last 15 years (not shown) with better data quality, the correlation increases to 0.69 (consistency with high confidence).
(left) Covariation between a prototype version of the stratospheric water vapor CDR H2O (produced within the Water_Vapour_cci) and ERA5 monthly zonal mean temperatures T at 100 hPa. (right) Correlation between the two datasets.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
(left) Covariation between a prototype version of the stratospheric water vapor CDR H2O (produced within the Water_Vapour_cci) and ERA5 monthly zonal mean temperatures T at 100 hPa. (right) Correlation between the two datasets.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
(left) Covariation between a prototype version of the stratospheric water vapor CDR H2O (produced within the Water_Vapour_cci) and ERA5 monthly zonal mean temperatures T at 100 hPa. (right) Correlation between the two datasets.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Alternatively, differences of multiyear trend maps of one variable can be used to assess the consistency of two different ECV CDRs. The example here is the intercomparison between wave height (measured by satellite altimetry) and sea ice concentrations assessed in Stopa et al. (2016). Daily sea ice concentrations produced from the Special Sensor Microwave Imager (SSM/I) by Institut Français de Recherche pour l’Exploitation de la Mer (IFREMER) (Ezraty et al. 2007) are used to define open ocean versus sea ice conditions with a 15% concentration threshold at 12.5 km resolution within the Arctic Ocean. For the period 1992–2014, the SSM/I ice concentrations are used along with wind vectors from the Climate Forecast System Reanalysis to reproduce the wave field through the numerical wave model, WAVEWATCH3 (WW3; Tolman et al. 2014), which includes wave–ice interaction through an under-ice parameterization of wave dissipation. Figure 9 shows a comparison of the trends of the significant wave height (H s ) directly measured with altimetry (denoted ALT; Queffeulou and Croizé-Fillon 2016) and from the collocated model data from WW3 (denoted WW3 CoLoc) in which SSM/I ice concentrations have been used. Qualitatively, the regional patterns match between the two datasets, despite stronger trends in the altimeters (of up to 1 cm yr−1). At present the confidence interval for trends in wave heights is not known. Therefore the quantitative discrepancies between modeled and measured trends here could be due to both systematic time-varying biases in the wave height ECV, which are expected to be only a function of time and sensor, or to a trend error in the surface wind reanalysis used to drive the wave model. However, the wind trends are also constrained by sea level pressure data and sea ice drift (e.g., Spreen et al. 2011). In the future, a wider range of ECVs, combined with in situ data and models, may be used for a quantitative refinement of sea-state trends.
Trends of monthly averaged significant wave height (H s ) datasets (cm yr−1) with the Mann–Kendall test (thatched areas) from (left) satellite altimetry (ALT), and (right) collocated model WW3 hindcast (CoLoc).
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Trends of monthly averaged significant wave height (H s ) datasets (cm yr−1) with the Mann–Kendall test (thatched areas) from (left) satellite altimetry (ALT), and (right) collocated model WW3 hindcast (CoLoc).
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Trends of monthly averaged significant wave height (H s ) datasets (cm yr−1) with the Mann–Kendall test (thatched areas) from (left) satellite altimetry (ALT), and (right) collocated model WW3 hindcast (CoLoc).
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
An example of testing the (anti-) correlation of multiple regional ECV CDRs as predicted by physical theory, is the use of the El Niño–Southern Oscillation (ENSO) index for ECV anomalies in the tropical Pacific Ocean Niño-3.4 region (5°S–5°N, 170°–120°W). This natural phenomenon is an ideal candidate for investigating multiple ECV consistency due to its relatively short time scale, large amplitude, and multiple ECVs affected by it. This first attempt focuses on the main ENSO signatures at large scale. Physical or biological processes leading to spatiotemporal lags of a few months or a few degrees longitude between some variables have been neglected. This could be refined in future studies. ENSO variability is compared in several ocean (SST, SL, SSS, Chlor_a), atmosphere (CFChigh, TCWV, AOD550), and land (SM, burned area/fire) ECV products—see Table A1 for the acronyms, more detailed information on the datasets and their correlations. All variables were interpolated to a common 1° × 1° grid, deseasonalized by removing the corresponding monthly mean value and normalized by dividing by the standard deviations for their respective available time period. Figure 10 shows the index variability across the tropical Pacific Ocean for the ECVs in time–longitude anomaly cross sections. The ocean and most atmosphere ECV time series show consistent spatiotemporal covariability, as expected. Whereas SST and SL have their largest variability in the Niño-3.4 region, CFChigh and TCWV variability peak farther west (∼180°), except for the strong El Niño years 1982/83, 1997/98, and 2015/16. Moreover, SSS and Chlor_a are anticorrelated with SST, as expected from a reduced upwelling. For ECVs affected indirectly by El Niño from dry conditions and wildfires over Indonesia (fire, aerosol, and soil moisture), the highest correlations occur in their Indonesian time series (10°S–10°N, 100°–150°E). For certain El Niño-3.4 years, e.g., 1997, 2007, and 2015, there are clear indicators of covariability of them and SST (Fig. 10g). Here again the use of a correlation threshold of 0.5 (0.7) for medium (high) confidence on consistency is adopted. In conclusion, by quantifying (anti-) correlations between these nine independently derived satellite ECVs versus the scientific understanding of the ENSO phenomenon, a medium (high) confidence in their consistency can be shown for eight (four) of them.
Zonal month–longitude cross sections (averaged between 5°S and 5°N) for 150°E–280°E normalized indices of (a) sea surface temperature (SST), (b) sea level height (SL), (c) sea surface salinity (SSS), (d) chlorophyll-a (Chlor_a), (e) high-level cloud fraction (CFChigh), and (f) total-column water vapor (TCWV). All ECVs are plotted for their respective full-year availability. The black lines in the Hovmöller plots show the Niño-3.4 box. (g) Time series of Niño-3.4 SST and Indonesia soil moisture (SM), burned area (Fire), and aerosol optical depth at 550 nm (AOD550). Information on the used datasets is provided in Table A1 in the appendix.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Zonal month–longitude cross sections (averaged between 5°S and 5°N) for 150°E–280°E normalized indices of (a) sea surface temperature (SST), (b) sea level height (SL), (c) sea surface salinity (SSS), (d) chlorophyll-a (Chlor_a), (e) high-level cloud fraction (CFChigh), and (f) total-column water vapor (TCWV). All ECVs are plotted for their respective full-year availability. The black lines in the Hovmöller plots show the Niño-3.4 box. (g) Time series of Niño-3.4 SST and Indonesia soil moisture (SM), burned area (Fire), and aerosol optical depth at 550 nm (AOD550). Information on the used datasets is provided in Table A1 in the appendix.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
Zonal month–longitude cross sections (averaged between 5°S and 5°N) for 150°E–280°E normalized indices of (a) sea surface temperature (SST), (b) sea level height (SL), (c) sea surface salinity (SSS), (d) chlorophyll-a (Chlor_a), (e) high-level cloud fraction (CFChigh), and (f) total-column water vapor (TCWV). All ECVs are plotted for their respective full-year availability. The black lines in the Hovmöller plots show the Niño-3.4 box. (g) Time series of Niño-3.4 SST and Indonesia soil moisture (SM), burned area (Fire), and aerosol optical depth at 550 nm (AOD550). Information on the used datasets is provided in Table A1 in the appendix.
Citation: Bulletin of the American Meteorological Society 101, 11; 10.1175/BAMS-D-19-0127.1
State of consistency assessments for CCI ECVs.
Several examples of closure/budget studies of partial Earth system cycles demonstrate the usefulness of CCI (and several other) CDRs that are consistent at all three levels. For example, closure of the carbon budget is still an outstanding scientific challenge (Le Quéré et al. 2018) that is impacted by CDR inconsistencies. Different CCI products provide direct and indirect constraints on carbon fluxes that help to improve the consistency of carbon budgets. For example, CCI greenhouse gas products are used to inform atmospheric inversions. Top-down inversion results can be complemented by other ECVs to attribute diagnosed fluxes to different components such as biomass carbon changes (biomass CCI product), fire emissions (CCI products on burned area and fire size), and land-use change emissions (land-cover CCI products).
Another example is the regional closure of the water budget. Based on multiple satellite ECVs it has been demonstrated that the water budget can be closed within less than 10% uncertainty at a continental annual time scale, while, at monthly time scales, its residuals and uncertainty estimates are larger (about 20%; Rodell et al. 2015). These uncertainties in the water budget closure can be reduced by introducing additional constraints, e.g., by using multiple CDRs with different uncertainties of a single quantity or by additionally forcing closure of the atmosphere and ocean terms. Uncertainties in existing CDRs need to be further reduced and new CDRs of other key variables (most importantly, river discharge and irrigation water use) need to be included or developed to reach the 5% closure error targeted by GCOS (GCOS 2016).
The global mean sea level budget closure has also been assessed within the CCI program by comparing the sum of changes in ocean thermal expansion, land ice melt, and liquid water storage on continents with the total observed sea level change. The latter can be estimated globally from satellite altimetry with an accuracy of about 10% on different time scales (e.g., WCRP sea level budget group, 2018). These observations enable closure of the trend in sea level budget with an uncertainty of ±0.3 mm yr−1 over the last 25 years. The sea level budget involves additional variables from the global water budget (through land ice and liquid water components) and from the global energy budget (through thermal expansion directly related to global ocean heat content; Meyssignac et al. 2017) and thus connects the energy and water budgets. At regional scale, uncertainties in the observed components of the sea level budget are considerably larger (few tens of percent) and need to be further reduced to reach the regional GCOS target.
Finally, an assessment of the current state of affairs regarding consistency between the CDRs of the CCI program was made based on the combined scientific expertise of the CCI community; it is not meant to be exhaustive but intended as initial guidance for the use of multiple ECV CDRs or for defining priorities in further consistency analysis. Table 3 provides for each pair of CDRs the consistency status as either “no evident need to consider consistency,” “further studies needed,” “consistency explicitly ensured by shared processing or coretrieving,” or “studies already performed,” referenced to Table A2 with the underlying publication or technical report (characterized as “theoretical,” “exemplary/partial,” or “comprehensive”). As can be seen from Table 3, quite some work remains to be done where the definition and concept presented in this paper can be applied and further refined.
Consistency analysis status between pairs of CCI ECVs: intrinsically assured (*), study needed (×), study done (c = comprehensive; e = exemplary; t = theoretical)—empty fields indicate that no study is needed, this link cannot be studied (e.g., due to resolution), or the link is considered weak. Numbered references for conducted studies are provided in the appendix (Table A2).
Summary and conclusions
Climate data records of essential climate variables derived from satellite instruments provide essential information to monitor the state of the Earth system and its changing climate. A key requirement for these CDRs to be useful for Earth system science applications is that the CDRs are internally and mutually consistent. The ESA CCI program provides a set of CDRs for 21 GCOS ECVs in a common framework, and from the outset has invested heavily in establishing their consistency, as presented in this study. To our knowledge, no comprehensive definition of CDR consistency exists. Therefore, a three-level definition of consistency applicable to single- and multiple-variable cases is proposed and a concept for assessing if two or more CDRs are consistent with each other and possibly with reference data are presented. On the technical level, straightforward data access and usage, including availability of comprehensive documentation and product user guides, are needed. On the retrieval level, one needs to limit contradictions in the use of auxiliary datasets (masks or continuous fields) of the same variables in separate processing chains. On the scientific level, consistency of multiple ECV CDRs means judging their relevant correlations, patterns, periodicity, trends, etc. (as appropriate for a given variable, process, or cycle) in the light of underlying physical background knowledge (e.g., by jointly confronting them with a model). Through this link with background knowledge, “consistency” as defined in this study goes beyond “agreement” and relies rather on “compatibility.” Finding inconsistencies in one or more ECV dataset(s) (i.e., patterns whose disagreements exceed underlying uncertainties, contradict physical principles or a well-founded model) often indicates errors in a dataset or model whose resolution can lead to new scientific understanding.
This study also provides an overview of the technical consistency of CCI CDRs (common format and metadata standards, common data portal, harmonized documentation, common uncertainty reporting). An open issue in this regard is harmonization across programs and communities. Here, the CCI program has made an important step by adopting the netCDF format, with the CF and ACDD conventions (the de facto standard in the modeling community) for its gridded satellite data records. The Climate Data Store (CDS) of the Copernicus Climate Change Service (C3S) is also based largely on CCI standards. Such common standards are a prerequisite for the use of automated data services for accessing multiple data sources with little manual interaction, hence facilitating use of the data in scientific studies across multiple ECVs.
The discussion of a concept for assessing consistency and related methods on the retrieval and scientific level shows how consistency with regard to different categorical and continuous auxiliary datasets can be tested and how the assessment of single-variable self-consistency and multiple quantity mutual consistency can be conducted. In all these methods a basic understanding of “the truth” needs to be employed. A relevant characteristic of an ECV and an appropriate metric (e.g., bias, correlation, contingency matrix) for its evaluation need to be chosen. A tabular summary of different methods to assess consistency is given in Table 2. For each of the different metrics, a threshold needs to be defined to judge on consistency of two datasets. This may well differ from commonly applied thresholds for validation purposes since also other processes than consistency may affect the datasets. We suggest as a minimum requirement that each consistency study states the applied thresholds, as is done for the examples in this paper. Whereas the methods used to assess consistency rely on well-established tools for calibration and validation, placing them into the systematic context with relevance to consistency as done here, can serve as a practical guideline to consistency assessment. A brief high-level analysis of the interdependencies of CCI ECVs at the retrieval and scientific levels (Table 1) is provided to understand where consistency is needed and thus needs to be checked. Finally, a high-level assessment of the current state of affairs regarding consistency assessment between the CDRs of the CCI program (Table 3) is compiled to outline possible further research needs.
When discussing consistency, datasets from sources other than satellite data (e.g., Earth system models) are often required to comprehensively study an Earth system cycle, and their uncertainties also need to be considered, together with uncertainties in simplified or estimated budget equations. It is well understood that establishing consistency between two or more variables requires targeted analysis. Within and outside CCI much effort has been spent on quantifying the sensitivities and dependencies of the retrieved quantities. However, a lot more remains to be done in this area.
Acknowledgments
This study is based on ongoing work of, altogether, 30 projects of the ESA Climate Change Initiative (23 ECV projects, the Climate Model User Group project, cross-cutting outreach components on portal, toolbox, visualization; CCI data standards and system engineering working group). We are grateful to ESA for creating the CCI program, which has strengthened the consistency of the many research communities related to developing, processing, qualifying, and using satellite CDRs. We are grateful to the several hundred scientists building the CCI community for making a consistent Earth observation based data repository real. The “operational” part of the CCI program has been transferred to the Copernicus Climate Change Service [C3S, (re-) processing to extend the CDRs, associated quality control, user support]. We are also thankful for many other datasets from outside CCI and C3S that help cover all relevant ECVs: GOSAT level 1 data from JAXA, GOSAT level 2 data from NIES and NASA, OCO-2 level 1 and level 2 data from NASA, HOAPS data from EUMETSAT CM SAF, water vapor records from the G-VAP data archive, CAMS and ERA5 data from ECMWF/Copernicus Atmosphere and Climate Change Services, SSM/I daily sea ice concentrations from IFREMER, and wind vectors from the Climate Forecast System Reanalysis.
Appendix: Underlying sources
This appendix provides the underlying sources for the multiple regional CDR correlation test (shown in Fig. 10: datasets used) and the consistency analysis status (shown in Table 3: publications and technical reports).
Information on the datasets used for Fig. 10 : versions, DOIs, and references. The correlations between the SST Niño-3.4 region (averaged from 5°S to 5°N and from 190° to 240°E) time series and the other ECVS’s Niño-3.4 time series (and for SM, BA, and AOD time series with Indonesia (averaged from 10°S to 10°N and from 100° to 150°E) are given in the right column.
Snapshot of publications or technical reports (available from ESA CCI program) until the submission of this manuscript behind entries on done consistency studies in Table 3.
References
Allan, R. P. , 2012: The role of water vapour in Earth’s energy flows. Surv. Geophys., 33, 557–564, https://doi.org/10.1007/s10712-011-9157-8.
Andersson, A. , K. Graw , M. Schröder , K. Fennig , J. Liman , S. Bakan , R. Hollmann , and C. Klepp , 2017: Hamburg Ocean Atmosphere Parameters and Fluxes from Satellite Data—HOAPS 4.0. Satellite Application Facility on Climate Monitoring, accessed 15 July 2019, https://doi.org/10.5676/EUM_SAF_CM/HOAPS/V002.
Bates, J. J. , J. L. Privette , E. J. Kearns , W. Glance , and X. Zhao , 2016: Sustained production of multidecadal climate records: Lessons from the NOAA Climate Data Record program. Bull. Amer. Meteor. Soc., 97, 1573–1581, https://doi.org/10.1175/BAMS-D-15-00015.1.
Bevan, S. L. , P. R. J. North , S. O. Los , and W. M. F. Grey , 2012: A global dataset of atmospheric aerosol optical depth and surface reflectance from AATSR. Remote Sens. Environ., 116, 199–210, https://doi.org/10.1016/j.rse.2011.05.024.
Bojinski, S. , M. Verstraete , T. C. Peterson , C. Richter , A. Simmons , and M. Zemp , 2014: The concept of essential climate variables in support of climate research, applications, and policy. Bull. Amer. Meteor. Soc., 95, 1431–1443, https://doi.org/10.1175/BAMS-D-13-00047.1.
Boutin, J. , J.-L. Vergely , J. Koehler , F. Rouffi , and N. Reul , 2019: ESA Sea Surface Salinity Climate Change Initiative (Sea_Surface_Salinity_cci): version 1.8 data collection. Centre for Environmental Data Analysis, accessed 15 June 2019, http://doi.org/10.5285/9ef0ebf847564c2eabe62cac4899ec41.
Buchwitz, M. , and Coauthors, 2015: The Greenhouse Gas Climate Change Initiative (GHG-CCI): Comparison and quality assessment of near-surface-sensitive satellite-derived CO2 and CH4 global data sets. Remote Sens. Environ., 162, 344–362, https://doi.org/10.1016/j.rse.2013.04.024.
Chuvieco, E. , and Coauthors, 2019: Historical background and current developments for mapping burned area from satellite Earth observation. Remote Sens. Environ., 225, 45–64, https://doi.org/10.1016/j.rse.2019.02.013.
Doherty, S. J. , and Coauthors, 2009: Lessons learned from IPCC AR4: Scientific developments needed to understand, predict, and respond to climate change. Bull. Amer. Meteor. Soc., 90, 497–514, https://doi.org/10.1175/2008BAMS2643.1.
Dorigo, W. A. , and Coauthors, 2017: ESA CCI soil moisture for improved Earth system understanding: State-of-the art and future directions. Remote Sens. Environ., 203, 185–215, https://doi.org/10.1016/j.rse.2017.07.001.
ESA, 2019: CCI data standards version 2.0. ESA Climate Office Doc. CCI-PRGM-EOPS-TN-13-0009, 23 pp., http://cci.esa.int/sites/default/files/filedepot/incoming/CCIDataStandards_v2-0_CCI-PRGM-EOPS-TN-13-0009.pdf.
Ezraty, R. , F. Girard-Ardhuin , J. F. Piolle , L. Kaleschke , and G. Heygster , 2007: Arctic and Antarctic sea ice concentration and Arctic sea ice drift estimated from special sensors microwave data. User manual version 2.1, IFREMER CERSAT Doc., 22 pp.
GCOS, 2016: GCOS Implementation Plan 2016—The Global Observing System for Climate: Implementation needs. ESA, http://cci.esa.int/content/gcos-implementation-plan-2016-global-observing-system-climate-implementation-needs.
Gruber, A. , T. Scanlon , R. van der Schalie , W. Wagner , and W. Dorigo , 2019: Evolution of the ESA CCI soil moisture climate data records and their underlying merging methodology. Earth Syst. Sci. Data, 11, 717–739, https://doi.org/10.5194/essd-11-717-2019.
Hegglin, M. I. , and Coauthors, 2014: Vertical structure of stratospheric water vapour trends derived from merged satellite data. Nat. Geosci., 7, 768–776, https://doi.org/10.1038/ngeo2236.
Hollmann, R. , and Coauthors, 2013: The ESA Climate Change Initiative: Satellite data records for essential climate variables. Bull. Amer. Meteor. Soc., 94, 1541–1552, https://doi.org/10.1175/BAMS-D-11-00254.1.
Holzer-Popp, T. , and Coauthors, 2002: Das automatische Atmosphärenkorrekturverfahren DurchBlick. Fernerkundung und GIS: Neue Sensoren–Innovative Methoden, T. Blaschke , Ed., H. Wichmann Verlag, 78–87.
Immler, F. J. , J. Dykema , T. Gardiner , D. N. Whiteman , P. W. Thorne , and H. Vomel , 2010: Reference quality upper-air measurements: Guidance for developing GRUAN data products. Atmos. Meas. Tech., 3, 1217–1231, https://doi.org/10.5194/amt-3-1217-2010.
IOCCG, 2010: Atmospheric correction for remotely-sensed ocean-colour products. International Ocean-Colour Coordinating Group Rep. 10, 84 pp.
IPCC, 2013: Climate Change 2013: The Physical Science Basis. Cambridge University Press, 1535 pp., https://doi.org/10.1017/CBO9781107415324.
IPCC, 2014: Climate Change 2014: Synthesis Report. IPCC, 151 pp.
IPCC, 2018: Global Warming of 1.5°C. World Meteorological Organization, 630 pp., www.ipcc.ch/site/assets/uploads/sites/2/2019/06/SR15_Full_Report_High_Res.pdf.
IPCC, 2019a: The Ocean and Cryosphere in a Changing Climate. WMO, 1170 pp., https://report.ipcc.ch/srocc/pdf/SROCC_FinalDraft_FullReport.pdf.
IPCC, 2019b: Climate Change and Land. World Meteorological Organization, 874 pp., www.ipcc.ch/site/assets/uploads/2019/11/SRCCL-Full-Report-Compiled-191128.pdf.
Kääb, A. , S. H. Winsvold , B. Altena , C. Nuth , T. Nagler , and J. Wuite , 2016: Glacier remote sensing using Sentinel-2. Part I: Radiometric and geometric performance, and application to ice velocity. Remote Sens ., 8, 598, https://doi.org/10.3390/rs8070598.
Legeais, J.-F. , and Coauthors, 2018: An improved and homogeneous altimeter sea level record from the ESA Climate Change Initiative. Earth Syst. Sci. Data, 10, 281–301, https://doi.org/10.5194/essd-10-281-2018.
Le Quéré, C. , and Coauthors, 2018: Global carbon budget 2018. Earth Syst. Sci. Data, 10, 2141–2194, https://doi.org/10.5194/essd-10-2141-2018.
Levizzani, V. , and E. Cattani , 2019: Satellite remote sensing of precipitation and the terrestrial water cycle in a changing climate. Remote Sens ., 11, 2301, https://doi.org/10.3390/rs11192301.
Li, Z. , and Coauthors, 2009: Uncertainties in satellite remote sensing of aerosols and impact on monitoring its long-term trend: A review and perspective. Ann. Geophys., 27, 2755–2770, https://doi.org/10.5194/angeo-27-2755-2009.
Lizundia-Loiola, J. , G. Otón , R. Ramo , and E. Chuvieco , 2020: A spatio-temporal active-fire clustering approach for global burned area mapping at 250 m from MODIS data. Remote Sens. Environ., 236, 111493, https://doi.org/10.1016/j.rse.2019.111493.
Merchant, C. J. , and Coauthors, 2017: Uncertainty information in climate data records from Earth observation. Earth Syst. Sci. Data, 9, 511–527, https://doi.org/10.5194/essd-9-511-2017.
Merchant, C. J. , and Coauthors, 2019: Satellite-based time-series of sea-surface temperature since 1981 for climate applications. Sci. Data, 6, 223, https://doi.org/10.1038/s41597-019-0236-x.
Meyssignac, B. , and Coauthors, 2017: Evaluating model simulations of twentieth-century sea-level rise. Part II: Regional sea-level changes. J. Climate, 30, 8565–8593, https://doi.org/10.1175/JCLI-D-17-0112.1.
NASA, 2014: OMI/Aura and MODIS/Aqua merged cloud product. NASA, https://cmr.earthdata.nasa.gov/search/concepts/C1265734652-GES_DISC.html.
North, P. , S. Briggs , S. Plummer , and J. Settle , 1999: Retrieval of land surface bidirectional reflectance and aerosol opacity from ATSR-2 multiangle imagery. IEEE Trans. Geosci. Remote Sens., 37, 526–537, https://doi.org/10.1109/36.739106.
Paul, F. , and Coauthors, 2015: The Glaciers Climate Change Initiative: Methods for creating glacier area, elevation change and velocity products. Remote Sens. Environ., 162, 408–426, https://doi.org/10.1016/j.rse.2013.07.043.
Popp, T. , 1995: Correcting atmospheric masking to retrieve the spectral albedo of land surfaces from satellite measurements. Int. J. Remote Sens., 16, 3483–3508, https://doi.org/10.1080/01431169508954642.
Popp, T. , and Coauthors, 2016: Development, production and evaluation of aerosol climate data records from European satellite observations (Aerosol_cci). Remote Sens ., 8, 421, https://doi.org/10.3390/rs8050421.
Poulsen, C. A. , and Coauthors, 2012: Cloud retrievals from satellite data using optimal estimation: Evaluation and application to ATSR. Atmos. Meas. Tech., 5, 1889–1910, https://doi.org/10.5194/amt-5-1889-2012.
Pulliainen, J. , and Coauthors, 2017: Early snowmelt significantly enhances boreal springtime carbon uptake. Proc. Natl. Acad. Sci. USA, 114, 11 081–11 086, https://doi.org/10.1073/pnas.1707889114.
Quartly, G. D. , and Coauthors, 2017: A new phase in the production of quality-controlled sea level data. Earth Syst. Sci. Data, 9, 557–572, https://doi.org/10.5194/essd-9-557-2017.
Queffeulou, P. , and D. Croizé-Fillon , 2016: Global altimeter SWH data set. User guide. IFREMER Laboratoire d’Océanographie Spatiale Doc., 10 pp.
Reuter, M. , and Coauthors, 2013: A joint effort to deliver satellite retrieved atmospheric CO2 concentrations for surface flux inversions: The ensemble median algorithm EMMA. Atmos. Chem. Phys., 13, 1771–1780, https://doi.org/10.5194/acp-13-1771-2013.
Reuter, M. , and Coauthors, 2017: How much CO2 is taken up by the European terrestrial biosphere? Bull. Amer. Meteor. Soc., 98, 665–671, https://doi.org/10.1175/BAMS-D-15-00310.1.
Reuter, M. , and Coauthors, 2020: Ensemble-based satellite-derived carbon dioxide and methane column-averaged dry-air mole fraction data sets (2003–2018) for carbon and climate applications. Atmos. Meas. Tech., 13, 789–819, https://doi.org/10.5194/amt-13-789-2020.
Rodell, M. , and Coauthors, 2015: The observed state of the water cycle in the early twenty-first century. J. Climate, 28, 8289–8318, https://doi.org/10.1175/JCLI-D-14-00555.1.
Sathyendranath, S. , and Coauthors, 2012: Ocean Colour Climate Change Initiative—Approach and initial results. IEEE Int. Geoscience and Remote Sensing Symp ., Munich, Germany, IEEE, 2024–2027, https://doi.org/10.1109/IGARSS.2012.6350979.
Schröder, M. , M. Lockhoff , J. Forsythe , H. Cronk , T. H. Vonder Haar , R. Bennartz , 2016: The GEWEX water vapor assessment: Results from intercomparison, trend, and homogeneity analysis of total column water vapor. J. Appl. Meteor. Climatol., 55, 1633–1649, https://doi.org/10.1175/JAMC-D-15-0304.1.
Schröder, M. , and Coauthors, 2018: The GEWEX water vapor assessment archive of water vapour products from satellite observations and reanalyses. Earth Syst. Sci. Data, 10, 1093–1117, https://doi.org/10.5194/essd-10-1093-2018.
Schröder, M. , and Coauthors, 2019: The GEWEX water vapor assessment of global water vapour and temperature data records from satellites and reanalyses. Remote Sens ., 11, 251, https://doi.org/10.3390/RS11030251.
Schulz, J. , and Coauthors, 2009: Operational climate monitoring from space: The EUMETSAT Satellite Application Facility on Climate Monitoring (CM-SAF). Atmos. Chem. Phys., 9, 1687–1709, https://doi.org/10.5194/acp-9-1687-2009.
Siddans, R. , 2016: S5P-NPP Cloud Processor ATBD. EUMETSAT Doc. S5P-NPPC-RAL-ATBD-0001, 41 pp., https://sentinel.esa.int/documents/247904/2476257/Sentinel-5P-NPP-ATBD-NPP-Clouds.
Sogacheva, L. , P. Kolmonen , T. H. Virtanen , E. Rodriguez , G. Saponaro , and G. de Leeuw , 2017: Post-processing to remove residual clouds from aerosol optical depth retrieved using the Advanced Along Track Scanning Radiometer. Atmos. Meas. Tech., 10, 491–505, https://doi.org/10.5194/amt-10-491-2017.
Sogacheva, L. , and Coauthors, 2020: Merging regional and global aerosol optical depth records from major available satellite products. Atmos. Chem. Phys., 20, 2031–2056, https://doi.org/10.5194/acp-20-2031-2020.
Spreen, G. , R. Kwok , and D. Menemenlis , 2011: Trends in Arctic sea ice drift and role of wind forcing: 1992–2009. Geophys. Res. Lett., 38, L19501, https://doi.org/10.1029/2011GL048970.
Stengel, M. , and Coauthors, 2019: Cloud_cci Advanced Very High Resolution Radiometer Post Meridiem (AVHRR-PM) dataset version 3: 35-year climatology of global cloud and radiation properties. Earth Syst. Sci. Data, 12, 41–60, https://doi.org/10.5194/essd-12-41-2020.
Stopa, J. E. , F. Ardhuin , and F. Girard-Ardhuin , 2016: Wave climate in the Arctic 1992–2014: Seasonality and trends. Cryosphere, 10, 1605–1629, https://doi.org/10.5194/tc-10-1605-2016.
Tolman, H. L. , and Coauthors, 2014: User manual and system documentation of WAVEWATCH III version 4.18. NOAA Tech. Note 316, 311 pp., https://polar.ncep.noaa.gov/waves/wavewatch/manual.v4.18.pdf.
Vinogradova, N. , and Coauthors, 2019: Satellite salinity observing system: Recent discoveries and the way forward. Front. Mar. Sci., 6, 243, https://doi.org/10.3389/fmars.2019.00243.
Waugh, D. W. , and V. Eyring , 2008: Quantitative performance metrics for stratospheric-resolving chemistry-climate models. Atmos. Chem. Phys., 8, 5699–5713, https://doi.org/10.5194/acp-8-5699-2008.
WGClimate , 2018: WGClimate ECV inventory gap analysis report v1.1. WGCL/REP/18/986356, 160 pp., http://ceos.org/document_management/Working_Groups/WGClimate/Documents/WGClimate_ECV-Inventory_Gap_Analysis_Report_v1.1.pdf.