Today’s satellite data rescue activities uncover dormant records of our recent past and enable users to access, assess, and utilize early images and soundings for climate studies.
The Twenty-First Conference of the Parties (COP21) governing the United Nations Frame work Convention on Climate Change (UNFCCC) saw the adoption in Paris in December 2015 of a global climate pact by representatives from 195 nations. The Paris agreement represents a historic milestone in terms of recognition by governments and the general public of the issues at stake. The Intergovernmental Panel on Climate Change (IPCC) Fifth Assessment Report states, “Warming of the climate system is unequivocal” (IPCC 2014a, p. 2). Several island states are already seeing the effects of climate change, confirming that “sea-level rise poses one of the most widely recognized climate change threats to low-lying coastal areas on islands and atolls” (IPCC 2014b, p. 1619). Better information on these and other impacts of climate change requires accurate and continuous measurements as laid out in the Global Climate Observing System (GCOS; GCOS 2010a) monitoring principles, originally adopted at the Fifth Conference of the Parties (COP5) in 1999. To construct useful multidecadal climate data records from those measurements, they must be combined with existing time series of past observations of the recent climate.
This goal has an impact on the planning of new observing systems (e.g., Leroy et al. 2008), imposing accuracy requirements that enable meaningful change detection times given the relative magnitudes of expected changes and natural variability. Space agencies have provided a strong response to the GCOS monitoring principles and the need for sustained climate data record (CDR) generation on essential climate variables (ECVs; Hollmann et al. 2013; Bojinski et al. 2014). The Committee on Earth Observation Satellites (CEOS)/Coordination Group for Meteorological Satellites (CGMS) Working Group on Climate (WGClimate) is currently addressing the realization of an architecture for climate monitoring from space (Dowell et al. 2013). Generating and interpreting climate records requires sustained research efforts as recognized by the World Climate Research Programme (WCRP) and the Committee on Space Research (COSPAR), which prepared a roadmap for observations in support of integrated Earth system science (Simmons et al. 2016).
This forward-looking view on observing systems for climate and long-term dataset generation supplements efforts to make optimal use of the great wealth of existing instrumental data records related to climate. Many potentially valuable in situ data records collected prior to the digital era remain largely inaccessible [see, e.g., Stickler et al. (2014) for a recent update]. The first photograph taken from space in 1946 by a rocket launched from New Mexico (Reichhardt 2006) also predates the digital era. However, since 1959, satellites have been launched for the purpose of observing the Earth’s weather and climate (e.g., Suomi 1958; Kållberg et al. 2010). The early generations of satellite instruments during the 1960s and 1970s produced a wealth of data archived on magnetic tapes with high potential value for climate science. The term “data rescue” is often used to describe the laborious processes of locating, imaging, digitizing, and reformatting historical climate observations into a form that renders them useful for further analysis. Similar efforts are needed to rescue satellite data collected during the digital age. The early satellite records are especially valuable because they contain unique information about parameters and locations that have not otherwise been observed. As for historical in situ observations, the location of early satellite data records is not always known. The data may exist only on nonstandard and vulnerable media; documentation of instrument response and other metadata may be incomplete. These reasons, combined with the declining expertise about old instruments, highlight the urgency of satellite data rescue.
In the early 1990s, to realize the potential value of existing satellite records, the National Aeronautics and Space Administration (NASA) launched the Pathfinder datasets concept in response to the question of what can be done now to further global change research (Asrar and Dokken 1993). The number of datasets to be reprocessed then was limited to a few long series of measurements from the Advanced Very High-Resolution Radiometer (AVHRR), Television Infrared Observation Satellite (TIROS) Operational Vertical Sounder (TOVS), Geostationary Operational Environmental Satellite (GOES), Special Sensor Microwave Imager (SSM/I), Scanning Multispectral Microwave Radiometer (SMMR), and Land Remote Sensing Satellite (LANDSAT). The legacy of these sensors still extends to the present; some examples of the use of reprocessed Pathfinder data are presented below.
Today there are additional reasons to accelerate and expand the data rescue efforts initiated in the 1990s. Both the archive media and the pertinent expertise are fading fast. On the positive side, increasingly powerful computing facilities are now available that allow us to analyze these data faster than ever thought possible. General expertise about the use of satellite data has advanced a great deal since the launch of these early missions. In addition, new powerful tools and methods are available to assess the quality of the data. These factors taken together form the basis for the recent advances in satellite data rescue presented in this paper.
IMPROVED PRESERVATION AND DATA SERVICES FOR HISTORICAL RECORDS FROM SPACE.
Unlike the early satellite programs, which did not address data recalibration and reprocessing, most space agencies today operate comprehensive data stewardship programs that include preservation of historical satellite data and provide substantial support for climate-related activities. Some agencies have developed complementary initiatives such as the Satellite Application Facility on Climate Monitoring (CM SAF; Schulz et al. 2009) attached to the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT) and the Climate Change Initiative (CCI; Hollmann et al. 2013) sponsored by the European Space Agency (ESA). Their objectives include the generation of several fundamental CDR (FCDR) and thematic CDR (TCDR) to supplement those generated by the agencies themselves. Other examples are National Oceanic and Atmospheric Administration (NOAA)-sponsored FCDRs generated by Remote Sensing Systems (RSS; e.g., Wentz 2013) and the Center for Satellite Applications and Research (STAR; e.g., Zou and Wang 2011).
At the international level, coordination of satellite reprocessing activities is supported by the Sustained, Coordinated Processing of Environmental Satellite Data for Climate Monitoring (SCOPE-CM; Lattanzio et al. 2013). The primary objective of the Global Space-Based Intercalibration System (GSICS; Goldberg et al. 2011) is to coordinate intersatellite calibration of sensor data. Several efforts exist that attempt to harmonize practices; for example, the project Coordinating Earth Observation Data Validation for Re-Analysis for Climate Services (CORE-CLIMAX) reviewed available methodologies for validation of space-based CDR (Zeng et al. 2015).
Progress toward preservation of historical data from geostationary and polar-orbiting meteorological satellites has taken slightly different paths.
Geostationary satellites.
The situation is well advanced for geostationary satellites back to the beginning of the 1980s (Knapp 2008), with SCOPE-CM coordinating various dedicated projects. Note, however, that current plans are missing historical records from the former Soviet Union and India, who have flown geostationary satellites for decades. They also do not address some of the earliest geostationary records. For example, from the United States, before the 1980s, the Applications Technology Satellite (ATS) series followed by the Synchronous Meteorological Satellite (SMS) predate the GOES series. The SMS data are being rescued from aging tapes at the Space Science and Engineering Center (SSEC) of University of Madison–Wisconsin. Figure 1 shows a photograph of such tapes and a sample series of four 5-min images held on tapes. The data rescued are served online (Table 1).
(left) A nine-track tape, holding historical geostationary data, with the corresponding player underneath, at SSEC and (right) data extracted from the nine-track tapes: a series of four 5-min SMS-2 satellite images from 6 May 1975. Such images will help derive atmospheric motion vectors, with potential application to assimilation into global reanalyses.
Citation: Bulletin of the American Meteorological Society 98, 7; 10.1175/BAMS-D-15-00194.1
Radiance data rescued from aging media over the past few years.
In Japan, most of the archive from the Geostationary Meteorological Satellite (GMS) has also been recovered from tapes and reprocessed for the year 1979 and from March 1987 to September 2009.
In Europe, most Meteosat data have already been reprocessed using current algorithms. A redesign of the entire processing chain is underway, so that novel algorithms can be applied to future reprocessing. Some of this effort is supported by Europe’s Copernicus Climate Change Service (C3S), with a view to provide a background record for its series of Sentinel satellites that will continue the legacy of high-quality data records from space. However, satellite data rescue can still improve the Meteosat record; SSEC recently discovered a copy of the second year of data of the first Meteosat satellite (whose 2-yr original record had been declared lost because of tape decay despite several recovery attempts), and rescue work is underway in collaboration with EUMETSAT (Table 1).
Data recovery for polar-orbiting satellites.
For polar-orbiting satellites, the largest agency with the longest records is undoubtedly NASA. Its Nimbus satellite program started in 1964 and collected Earth observations for over 30 yr. NASA now operates a data preservation program at the Goddard Earth Sciences (GES) Data and Information Services Center (DISC). Using novel technology patented by John Bordynuik Inc. (JBI), aging magnetic tapes are copied to new digital media by recovering the exact, original data bits and format. Access to data archived on the modern media is significantly improved, with data archive and transfer rates several orders of magnitude above those available when the data were first created. This makes it possible to serve entire mission datasets online for users to download. Open and fast data access enables researchers and users from many fields to investigate past cases or to search for correlation patterns with data in their own interest area.
The process of extracting data files from the Nimbus tapes brings about several problems. These include deviations of the data format from documentation, extra or missing label records, false end-of-file marks contained within data files, incorrect record length markers within files, corrupted or unreadable tapes, and bad data at the front or end of the old tapes. Reconciling duplicate files (primary tape vs backup) is another challenge. In addition, unlike current practice, file level metadata were nonexistent in the original Nimbus tapes other than as a listing of the number of files on the tape and vague date ranges, often written by hand on the tapes. This requires the data recovery teams to correctly understand the data format, that is, rely on the original documentation, in order to retrieve the true temporal and spatial extent as well as other ancillary information from the individual data files in the old original tapes.
In Europe, ESA leads a long-term data preservation (LTDP) program as part of its mandatory activities. This covers not only heritage data from ESA missions but also third-party missions available to ESA. The LTDP coordinates and optimizes European efforts in a cooperative framework. A recent example is the recovery of images from the oldest synthetic aperture radar (SAR), on Seasat, and on ERS-1. Comparing these data with the latest SAR generation on Sentinel-1 shows in Fig. 2 the retreat of two large glaciers in southeast Greenland over a 36-yr period. This activity, part of the ESA LTDP, contributes to the ESA CCI. As this initiative moves forward, there is scope to improve the time series of several ECVs from such imagery data, such as continental ice sheets and sea ice.
Greenland glaciers retreat seen by three generations of radar missions. Radar scans from Seasat on 16 Aug 1978, ERS-2 on 7 Aug 1996, and Sentinel-1 on 20 Aug 2014 show glacier edges in southeast Greenland. Over the 36-yr period, the glaciers receded by about (top) 180 and (bottom) 61.5 m yr−1. Such images help create long data records of glaciers and sea ice, with application to improve forcing datasets for climate models.
(Credits: ESA and University of Rome La Sapienza.)
Citation: Bulletin of the American Meteorological Society 98, 7; 10.1175/BAMS-D-15-00194.1
Individual action.
In addition to well-structured agency programs described thus far, foresight of individuals and research agencies has sometimes played key roles in satellite data preservation. Two such examples are given here. The first example involves four datasets of the British ancestry of sounding, which was developed at the University of Oxford.
Data from pressure-modulated infrared sounders [selective chopper radiometer (SCR) and pressure modulator radiometer (PMR); Table 1] and one limb-viewing infrared sounder [stratospheric and mesospheric sounder (SAMS)] were curated by a project funded by the U.K. Natural Environment Research Council. This project was visionary in its intention to keep raw mission data, whereas the prevailing approach at the time was to preserve geophysical retrievals. The discs holding the data were located in the library of the responsible institution, and a copy of the data was recently added to mass storage at the European Centre for Medium-Range Weather Forecasts (ECMWF). The data curation project included migrating them to a hardware-independent format [American Standard Code for Information Exchange (ASCII) files of hexadecimal characters] and providing a decoding software. This approach of converting records to a new data format was prevalent in the 1990s and applied not only to satellite data but other records, such as recovered from in situ. The norm now is to retain also the original data in their native format (or images), as allowed by the JBI technology.
A second example involves data collected by two infrared sounding Spectrometer Interferometer-1 (SI-1; Kempe et al. 1980) instruments flown on former Soviet meteorological polar-orbiting satellites (Meteor-28 and Meteor-29). The SI-1 was an instrument (please see online supplement at http://dx.doi.org/10.1175/BAMS-D-15-00194.2) developed by the former Academy of Sciences of German Democratic Republic (GDR), in collaboration with the former Soviet Hydrometeorological Service (HYDROMET) and the former Meteorological Service of GDR (Spänkuch 1980). The data came to light thanks to a fortuitous meeting in Noordwijk (the Netherlands) at the end of 2013. Fortunately, the investigators who had worked on SI-1 in the late 1970s had preserved all (about 2000) original SI-1 spectra (between 6 and 25 µm). With support from EUMETSAT, the data were recovered and moved to a current data format. In addition, these SI-1 data were patiently quality controlled and georeferenced, and user documentation was drafted (Coppens et al. 2015).
RECOGNITION OF THE IMPORTANCE OF METADATA DESCRIBING HISTORICAL SATELLITE INSTRUMENTS.
High-level metadata: Inventories.
Various inventories of Earth observation satellites have been made over the years (e.g., Houghton et al. 1986). For the initial purpose of reviewing capability and avoiding duplication in future satellite missions, the World Meteorological Organization (WMO) Space Programme has set up a database of metadata on past, present, and future satellite missions: the Observing Systems Capability Analysis and Review Tool (OSCAR; www.wmo-sat.info/oscar/). This facility, developed in coordination with CGMS and CEOS, has now evolved into a core element of WMO Integrated Global Observing System (WIGOS), covering also surface-based capabilities. These inventories help users identify past missions relevant to their topic of study.
Data lost?
However, corresponding data records are not always readily available. There are even cases of data records possibly lost forever. This includes some early geostationary satellites mentioned earlier. It also includes, for ozone, data from the backscatter ultraviolet (BUV) sensor on Atmosphere Explorer-E (Bhartia et al. 2013), which would patch a gap between Nimbus-4 and Nimbus-7 ozone-sensing UV instruments. Information gathered during the course of the present work indicates that the data were lost during transition from one mass archive system to another, as all unused data were discarded. Furthermore, infrared sounding data collected by the Special Sensor-H (SSH) instrument flown on four Defense Meteorological Satellite Program (DMSP) satellites appear to have been lost. So far, all attempts to locate these data have failed, but maybe the present paper will help jog memories.
Mission metadata.
The preservation of mission metadata is somehow more challenging than the data themselves. Metadata are meant to include any knowledge or information present at the time the mission was operated. They are thus essential in order to use the data properly. Until recently, such information was not given the same attention as mission data. The situation has now improved with standards (e.g., GCOS 2010b) mandating management of algorithm theoretical basis documents, all supporting data, and data format definitions. Standards are emerging to qualify the system maturity (e.g., Bates and Privette 2012; John et al. 2014).
The knowledge accrued over the years, from laboratory measurements or field campaigns, was not always associated with the relevant mission, but their importance is now recognized to better exploit data a posteriori. Ancient instruments pose specific problems. For infrared radiometers, a key difficulty is determining precisely the instrument spectral response function. Such information, available at the time in digital form, has generally been lost and needs restoring from published charts or graphs. Recognizing this challenge, NASA and ESA have adopted specifications for preservation of data and documentation, and new systems are being developed as a result (e.g., Khayat et al. 2013; Albani et al. 2012).
ADVANCES IN THE QUALITY ASSESSMENT OF HISTORICAL SATELLITE DATA RECORDS.
Once data have been rescued from aging media, a first prerequisite before application in climate studies is a quality assessment. This can be done by comparison with similar observations at the same time. In the case of polar-orbiting satellites, instrument data can be compared from one instrument to the next by spatiotemporal matching (collocation) or using the simultaneous nadir overpass (SNO) technique as used by GSICS. As demonstrated for the High Resolution Infrared Radiometer (HIRS; Chen et al. 2013), calibration corrections obtained from present-day, highly calibrated instruments can then be propagated back in time to past instruments. This, however, requires an unbroken chain of radiometers. After 1979, the HIRS series has very few gaps between satellites [TIROS series N (TIROS-N), NOAA-6 to NOAA-19, and MetOp]. Before 1979, the vertical temperature profile radiometer (VTPR) on NOAA-2 to NOAA-5 goes back to 1972. Before 1972, there are low-resolution and pressure-modulated sounders and imagers. However, the overlaps between instrument series are sometimes short and would largely benefit from the DMSP SSH data mentioned earlier. One may also use spatiotemporally complete NWP model datasets as a transfer standard.
Such datasets can be used as a comparison basis to assess historical satellite data quality. The evaluation of climate model simulations with observations requires long observational datasets, for example, as prepared by the Observations for Model Intercomparison Projects (Obs4MIPS; Teixeira et al. 2014). However, a reverse assessment of (potentially short) observation records can use model datasets as a reference. The latter need to be at subdaily temporal resolution and provide all the variables required to simulate the observations, typically temperature and humidity at various levels of the atmosphere. For this reason, such comparisons are more easily done with global reanalyses (see the sidebar on “Global reanalyses”).
GLOBAL REANALYSES
The scope of global reanalyses has expanded tremendously in recent years. From a niche activity—initially a few decades long and atmosphere only—reanalyses now include several components of the climate system and extend back in time by 100 yr or more (Dee et al. 2014; Compo et al. 2006). Assimilation of observations in a climate model generates a physically consistent set of gap-free gridded time series, with estimates of many essential climate variables. A reanalysis thus provides access to a massive amount of information contained in millions of weather reports from locations around the world in the form of comprehensive datasets that can be compared and sliced in any dimension. These datasets have many thousands of users (Gregow et al. 2015), who also request access to observations in a usable format. Historical in situ observations recovered by projects such as the European Union (EU) European Reanalysis of Global Climate Observations (ERA-CLIM; Dee et al. 2014), facilitated by the Atmospheric Circulation Reconstructions over the Earth initiative (ACRE; Allan et al. 2011), have now been used in several century-long reanalyses, such as the NOAA twentieth century reanalysis (20CR; Compo et al. 2011) and the ERA-20C (Poli et al. 2016). An outcome of ERA-CLIM was an inventory of early satellite missions of interest to global atmospheric reanalyses (see the supplement).
Mapping the three-dimensional state of the reanalyses into the observation space requires an observation operator. For radiances, EUMETSAT supports in Europe such developments through its Satellite Application Facility on Numerical Weather Prediction (NWP SAF) by enhancing capabilities of a fast radiative transfer model used in several present-day NWP systems and reanalysis systems [Radiative TOVS (RTTOV); Saunders et al. 1999].
One challenge consists of adjusting the fast radiative transfer model for a time period when trace gas concentrations in the atmosphere differed from present-day values, in some cases significantly. This requires new line-by-line radiative transfer computations as references. For instruments that are no longer operating (e.g., SCR and PMR), the task is more complex because their sensing principle is no longer applied on any current instrument, and so no observation operator was ever developed, and most investigators of the missions have already retired.
Early infrared sounders.
An example of such quality assessment is shown here for the infrared interferometer spectrometer (IRIS) on Nimbus-4. Harries et al. (2001) already compared this data to present-day instruments, showing long-term changes in Earth’s atmosphere composition. With the additional help of instrument metadata in the newly recovered IRIS records (Table 1), advanced observation operators, and new reanalyses (all described earlier), it is now possible to produce closer comparisons between IRIS observations and models. Such comparisons also help quantify how accurately modern observation operators and reanalyses can represent the early satellite data variability in its various dimensions. For the spectral dimension, Fig. 3 shows the observed spectra of brightness temperatures, after applying the cloud detection scheme of McNally and Watts (2003) using the NWP SAF Aerosol and Cloud Detection Package for High Resolution Infrared Sounders. Applying the cloud detection restricts the comparison to clear-sky situations, as assumed in the radiative transfer calculations. In Fig. 3b, the differences with ECMWF twentieth century reanalysis (ERA-20C) represent the total sum of errors in the IRIS observations, the ERA-20C, the radiative transfer calculations, and the temporal and spatial mismatch between IRIS’s 96-km pixel resolution and ERA-20C’s 125-km horizontal and 3-hourly resolution. Note some errors may compensate and could be masked by such single-variable comparison. For the water vapor sounding channels (wavenumbers above 1200 cm−1 in Fig. 3) as well as temperature-sounding channels in the 15-µm carbon dioxide band (wavenumbers 600–750 cm−1), the negative mean differences between IRIS and ERA-20C suggest possibly suboptimal cloud detection. For the temperature-sounding channels the differences are under 1.5 K in standard deviation and about 1 K for the low-peaking and window channels. In addition to quantifying the total sum of errors, such comparisons enable a better understanding of the data quality and isolation of problematic time periods in the instrument record by comparing with mission metadata (Poli and Brunel 2016). The cloud detection could be further improved by using the imager data collected by Nimbus-4 (now recovered; Table 1).
(a) Spectra of Nimbus-4 IRIS (Apr 1970–Jan 1971) brightness temperatures, quality controlled to retain only clear channels, and (b) departures with ERA-20C.
Citation: Bulletin of the American Meteorological Society 98, 7; 10.1175/BAMS-D-15-00194.1
Another example of the data quality assessment is in the spatial domain. Maps enable identification of potential geolocation errors in the rescued satellite data (e.g., Poli et al. 2015) but also characterization of regional variations in reanalysis quality. For Nimbus-6 HIRS-1, which predates the long series started in 1979 with TIROS-N, Fig. 4 shows the percentage of variance in observations for channel 5 (peaking in the mid- to upper troposphere) explained by ERA-20C over two periods (17–31 August 1975 and 31 January–3 March 1976, the only dates rescued so far). For a fair comparison with clear-sky calculations, the scenes are filtered to retain only pixels believed to be clear. In each 5° × 5° latitude–longitude box, the variance in the observations represents synoptic weather as well as seasonal changes between August and February and instrument noise. Where ERA-20C is able to represent the variations observed by the sounder, the variance explained is high. As expected, the results indicate that ERA-20C is able to represent the tropospheric temperature seasonal changes that dominate the variance over the southern and northern high latitudes. In northern (southern) midlatitudes, where the variance results from seasonal and synoptic signals, a fair (low) percentage of variance explained is found over well-observed (poorly observed) in situ land and ocean areas. These hemispheric differences reflect geographical differences in accuracy within ERA-20C (Poli et al. 2016). The low variance explained over the tropics is likely caused by ERA-20C assimilating only surface pressure and marine wind observations and the atmospheric model forced by monthly sea surface temperatures. Such comparison of observations with prior reanalyses helps prepare for the assimilation of HIRS-1 into future reanalyses.
Maps of percentage of variance in Nimbus-6 HIRS-1 channel 5 (peaking in the mid- to upper troposphere) brightness temperature observations explained by ERA-20C, for scenes believed to be clear, by considering where observation minus ERA-20C differences for channel 8 (window) are between –1 and 2 K. Variances are computed within 5° × 5° latitude–longitude bin, for 17–31 Aug 1975 and 31 Jan–3 Mar 1976.
Citation: Bulletin of the American Meteorological Society 98, 7; 10.1175/BAMS-D-15-00194.1
Early microwave sensors for humidity.
Continued measurements of humidity by microwave sensors have been available since SMMR on Nimbus-7. Its record was reprocessed two decades ago as part of the leading NASA Pathfinder datasets, and data are available online (Njoku 2003). Today the CM SAF is working to enhance the existing SMMR Pathfinder dataset by applying Climate and Forecast (CF) metadata conventions (http://cfconventions.org/) and by intercalibrating with SSM/I DMSP F-8 for the channels that had similar characteristics. One aim is to achieve consistency with an existing FCDR of SSM/I and Special Sensor Microwave Imager/Sounder (SSM/IS) brightness temperatures (Fennig et al. 2015). After spectra and maps, another way to assess satellite data quality is to consider various types of weather conditions and reanalyses. Comparing a preliminary data sample produced by the CM SAF with ERA-20C indicates in Fig. 5 a satisfactory data quality for the 22-GHz, water vapor–sensitive SMMR channel, especially when only clear-sky scenes are considered. The match is better for the ECMWF interim reanalysis (ERA-Interim; Dee et al. 2011), a reanalysis that assimilated a number of in situ and satellite data, than for ERA-20C, a reanalysis that only assimilated surface pressure and marine wind observations. The superior “instant accuracy” of a reanalysis that assimilates many more data sources compared to one that is restricted to only surface observations suggests a virtuous improvement; such assessments become more powerful as additional observations are assimilated into reanalyses and new observations are recovered from aging media.
Density scatterplots of observed brightness temperatures for Nimbus-7 SMMR channel 21V (horizontal axis) vs clear-sky radiative transfer calculations from reanalysis (vertical axis) for 26 Oct 1978. (a),(c) Comparisons to ERA-20C. (b),(d) Comparisons to ERA-Interim. (a),(b) All scenes and (c),(d) scenes believed to be rain free.
Citation: Bulletin of the American Meteorological Society 98, 7; 10.1175/BAMS-D-15-00194.1
Other instruments have been operating in the 183-GHz band, allowing for vertical sounding of water vapor, such as the Advanced Microwave Sounding Unit-B (AMSU-B), the Microwave Humidity Sounder (MHS), and the Advanced Technology Microwave Sounder (ATMS). Preceding them, the Special Sensor Microwave Water Vapor Profiler (SSM/T-2) series covers over 15 yr but has so far remained underused to study climatic variations of humidity (John et al. 2011). Assessing the quality of instrument data, by comparison with other datasets, is preparatory work for their exploitation in climate studies. With such a long record, one can also identify which model or reanalysis dataset tracks best the observed long-term variability. Figure 6 shows a time series of brightness temperature observations from SSM/T-2. Overlaid are simulations from ERA-Interim and ERA-20C. The long-term evolution of ERA-20C with respect to SSM/T-2 is more stable than ERA-Interim when projected into brightness temperature space. Kobayashi et al. (2015) give more details about lessons learned from this quality assessment of SSM/T-2 data.
Time series of brightness temperature data from SSM/T-2, and corresponding simulations from ERA-Interim and ERA-20C. Plots show 12-month running means for profiles averaged over the tropical ocean (30°N–30°S), using clear-sky data only.
Citation: Bulletin of the American Meteorological Society 98, 7; 10.1175/BAMS-D-15-00194.1
APPLICATIONS IN CLIMATE STUDIES.
There are several applications to climate studies of the historical satellite data records recovered, tagged with metadata, and assessed by the aforementioned activities. Early observations from space, covering large regions that were largely inhabited at the time, are essential, along with recovered in situ observations, to help extend back in time our understanding of the climate and its changes with time.
Long time series of FCDR and TCDR.
These enhancements can occur in several ways. One way is comparison with climate datasets for mutual validation. For example, the comparison to SSM/T-2 FCDR presented earlier enabled us to identify which of two reanalyses presented the more stable water cycle.
Another way is the direct application to retrievals to generate TCDRs and thereby improve ECV databases. The data can also be used as forcing to climate simulations. Such forcing datasets are largely derived from observations (e.g., Taylor et al. 2012). Rescuing early satellite data offers the potential to improve several forcing datasets. For example, in order to improve knowledge of past variations in sea ice concentration, the National Snow and Ice Data Center (NSIDC) has estimated North and South Pole sea ice edges from visible images collected by cameras on Nimbus satellites since the 1960s (Gallaher and Campbell 2014).
Assimilation into reanalyses.
There is also the prospect of direct assimilation into reanalyses for all sounders that predate TIROS-N. It has been demonstrated that assimilation of a single sounder series such as VTPR can substantially improve the reanalysis quality in poorly observed areas such as the Southern Hemisphere (Uppala et al. 2005). Recovering as many additional sounders as possible prior to 1979 can potentially extend reanalysis quality further back in time.
There are additional benefits to aim for in a virtuous cycle of improvements initiated by reanalysis. Assimilation of satellite data not previously used may, at first, lead to discontinuities in reanalysis products. Usually, lessons are then learned that lead to improvements in aspects of data quality control and bias correction, assimilation methods, the atmospheric general circulation model, or the observation processing. A recent example is the mass and water balance improvement in Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2), after assimilation of humidity-sensitive satellite data that had initially caused difficulties in the previous reanalysis (Takacs et al. 2016). As reanalyses become more accurate because of greater computing power, better models, and more observations, the errors in reconstructed time series decrease so that additional observations can be assimilated to further drive progress.
Interdisciplinary benefits.
There are other applications that benefit from old satellite data thanks to intersatellite calibration bridging with continuous instrument data series. These benefits cut across several themes of Earth science because all sensors in space usually have some sensitivity to an unintended component of the Earth system (e.g., an atmospheric sensitivity for surface sensors or a surface sensitivity for atmospheric sounders).
A case in point is the satellite altimeter, as its data analysis requires wet atmospheric corrections to reduce a major source of uncertainty in resulting estimates of sea level rise (Ablain et al. 2009). The altimeters fly with radiometers in the 22-GHz frequency band in order to estimate total column water vapor. There would hence be benefits to oceanography in propagating backward the calibration of present-day, 22-GHz observations, now observed by state-of-the-art sensors of superior calibration, namely the SSMIS, ATMS, and Imaging/Sounding Microwave Radiometer–Improved (MTVZA-GY), which also observe the 183-GHz humidity sounding frequency. Furthermore, the committed future microwave imager (MWI) and microwave sounder (MWS) instruments on board the EUMETSAT Polar System Second Generation will continue these measurements until 2040. Knowing that remote sensing at these frequencies will continue for years to come and that sea level rise is one of the key factors of climate change, of potentially huge societal impact, this makes it more worthwhile to invest into producing a FCDR of 22- and 183-GHz measurements. Likewise, land surface estimates from early satellite records could benefit from improved atmospheric corrections, and sounder data could be better exploited through improved surface state knowledge via emissivity models.
Looking ahead.
Over the past two decades, satellite data rescue complemented by reprocessing and mission continuity (e.g., National Academy of Sciences Engineering and Medicine 2015) have gained importance in satellite agencies’ programs alongside new and innovative research missions. For data rescue, the difficulties reported in this paper include the loss of datasets, reconciliation of actual media contents with metadata available, deviation of the actual data format from expectations or documentation, and retiring expertise. These impediments could hinder satellite data rescue in years to come, if not dealt with urgently. The major anticipated benefit is the provision of longer “background” time series through climate simulations and reanalyses to help better exploit future series of Earth observation instruments in space and ensure continuity with records derived from past missions. We hope that this paper will encourage younger scientists to engage with the retiring generation of space scientists to pass on their knowledge and allow new insights to be gained from early satellite data.
ACKNOWLEDGMENTS
The Nimbus data used in this effort were acquired as part of the activities of NASA’s Science Mission Directorate and are archived and distributed by the Goddard Earth Sciences (GES) Data and Information Services Center (DISC). The authors thank the following persons for enabling to locate data or metadata: Anthony McNally, Bob Wells, Clive Rodgers, Peter Bauer, Johannes Schmetz, W. Paul Menzel, Jerrold Robaidek, William L. Smith, and Ronald Glumb. The work presented here was supported by the respective affiliation institutions of the authors as well as EU Grant Agreements 265229 (ERA-CLIM), 607029 (ERA-CLIM2), and 313085 (CORE-CLIMAX), and the EUMETSAT CM SAF visiting scientist program (CM_VS14_01). The authors would like to acknowledge Adrian Simmons for his comments that helped improve the manuscript.
REFERENCES
Ablain, M., A. Cazenave, G. Valladeau, and S. Guinehut, 2009: A new assessment of the error budget of global mean sea level rate estimated by satellite altimetry over 1993–2008. Ocean Sci., 5, 193–201, doi:10.5194/os-5-193-2009.
Albani, M., and Coauthors, 2012: Long term preservation of Earth observation space data: European LTDP common guidelines. LTDP Working Group Rep. GSCB-LTDP-EOPG-GD-09–0002 Issue 2.0, 48 pp. [Available online at https://earth.esa.int/documents/1656065/1681917/EuropeanLTDPCommonGuidelines_Issue2.0.pdf.]
Allan, R., P. Brohan, G. P. Compo, R. Stone, J. Luterbacher, and S. Brönnimann, 2011: The International Atmospheric Circulation Reconstructions over the Earth (ACRE) Initiative. Bull. Amer. Meteor. Soc., 92, 1421–1425, doi:10.1175/2011BAMS3218.1.
Asrar, G., and D. J. Dokken, Eds., 1993: Earth Observing System (EOS) Reference Handbook. NASA, 148 pp. [Available online at http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19930014548.pdf.]
Bates, J. J., and J. L. Privette, 2012: A maturity model for assessing the completeness of climate data records. Eos, Trans. Amer. Geophys. Union, 93, 441, doi:10.1029/2012EO440006.
Bhartia, P. K., R. D. McPeters, L. E. Flynn, S. Taylor, N. A. Kramarova, S. Frith, B. Fisher, and M. DeLand, 2013: Solar Backscatter UV (SBUV) total ozone and profile algorithm. Atmos. Meas. Tech., 6, 2533–2548, doi:10.5194/amt-6-2533-2013.
Bojinski, S., M. Verstraete, T. C. Peterson, C. Richter, A. Simmons, and M. Zemp, 2014: The concept of essential climate variables in support of climate research, applications, and policy. Bull. Amer. Meteor. Soc., 95, 1431–1443, doi:10.1175/BAMS-D-13-00047.1.
Chen, R., C. Cao, and W. P. Menzel, 2013: Intersatellite calibration of NOAA HIRS CO2 channels for climate studies. J. Geophys. Res. Atmos., 118, 5190–5203, doi:10.1002/jgrd.50447.
Compo, G. P., J. S. Whitaker, and P. D. Sardeshmukh, 2006: Feasibility of a 100-year reanalysis using only surface pressure data. Bull. Amer. Meteor. Soc., 87, 175–190, doi:10.1175/BAMS-87-2-175.
Compo, G. P., and Coauthors, 2011: The Twentieth Century Reanalysis Project. Quart. J. Roy. Meteor. Soc., 137, 1–28, doi:10.1002/qj.776.
Coppens, D., B. Theodore, W. Doehler, A. Damiano, D. Oertel, D. Klaes, J. Schmetz, and D. Spaenkuch, 2015: Exploitation of SI-1 data from Meteor-28 and 29 spacecraft for climate purposes. 2015 EUMETSAT Meteorological Satellite Conf., Toulouse, France, EUMETSAT. [Available online at https://cimss.ssec.wisc.edu/itwg/itsc/itsc20/program/PDFs/30Oct/session7b/7p_07_coppens.pdf.]
Dee, D. P., and Coauthors, 2011: The ERA-Interim Reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553–597, doi:10.1002/qj.828.
Dee, D. P., M. Balmaseda, G. Balsamo, R. Engelen, A. J. Simmons, and J.-N. Thépaut, 2014: Toward a consistent reanalysis of the climate system. Bull. Amer. Meteor. Soc., 95, 1235–1248, doi:10.1175/BAMS-D-13-00043.1.
Dowell, M., and Coauthors, 2013: Strategy towards an architecture for climate monitoring from space. CEOS, WMO, and CGMS Rep., 39 pp. [Available online at http://ceos.org/document_management/Working_Groups/WGClimate/WGClimate_Strategy-Towards-An-%20Architecture-For-Climate-Monitoring-From-Space_2013.pdf.]
Fennig, K., A. Andersson, and M. Schröder, 2015: Fundamental climate data record of SSM/I / SSMIS brightness temperatures. Satellite Application Facility on Climate Monitoring, accessed 20 Apr 2017, doi:10.5676/EUM_SAF_CM/FCDR_MWI/V002.
Gallaher, D., and G. Campbell, 2014: Nimbus ice edge points from Nimbus visible imagery L2, CSV. National Snow and Ice Data Center, Boulder, CO, digital media, doi:10.5067/NIMBUS/NmIcEdg2.
GCOS, 2010a: Implementation plan for the global observing system for climate in support of the UNFCCC (2010 update). GCOS Rep. 138, 186 pp. [Available online at www.wmo.int/pages/prog/gcos/Publications/gcos-138.pdf.]
GCOS, 2010b: Guideline for the generation of datasets and products meeting GCOS requirements. GCOS Rep. 143, 12 pp. [Available online at www.wmo.int/pages/prog/gcos/Publications/gcos-143.pdf.]
Goldberg, M., and Coauthors, 2011: The Global Space-based Inter-Calibration System. Bull. Amer. Meteor. Soc., 92, 467–475, doi:10.1175/2010BAMS2967.1.
Gregow, H., and Coauthors, 2015: Worldwide survey of awareness and needs concerning reanalyses and respondents views on climate services. Bull. Amer. Meteor. Soc., 97, 1461–1473, doi:10.1175/BAMS-D-14-00271.1.
Harries, J. E., H. E. Brindley, P. J. Sagoo, and R. J. Bantges, 2001: Increases in greenhouse forcing inferred from the outgoing longwave radiation spectra of the Earth in 1970 and 1997. Nature, 410, 355–357, doi:10.1038/35066553.
Hollmann, R., and Coauthors, 2013: The ESA climate change initiative: Satellite data records for essential climate variables. Bull. Amer. Meteor. Soc., 94, 1541–1552, doi:10.1175/BAMS-D-11-00254.1.
Houghton, J. T., F. W. Taylor, and C. D. Rodgers, 1986: Remote Sounding of Atmospheres. Cambridge University Press, 343 pp.
IPCC, 2014a: Climate Change 2014: Synthesis Report Summary for Policymakers. Cambridge University Press, 32 pp. [Available online at www.ipcc.ch/pdf/assessment-report/ar5/syr/AR5_SYR_FINAL_SPM.pdf.]
IPCC, 2014b: Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part B: Regional Aspects. Cambridge University Press, 688 pp. [Available online at www.ipcc.ch/pdf/assessment-report/ar5/wg2/WGIIAR5-PartB_FINAL.pdf.]
John, V. O., G. Holl, R. P. Allan, S. A. Buehler, D. E. Parker, and B. J. Soden, 2011: Clear-sky biases in satellite infrared estimates of upper tropospheric humidity and its trends. J. Geophys. Res., 116, D14108, doi:10.1029/2010JD015355.
John, V. O., R. Roebeling, and J. Schulz, 2014: CORE-CLIMAX system maturity matrix instruction manual. EUMETSAT Doc. CC/EUM/MAN/13/002 v4, 34 pp. [Available online at www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=PDF_CORE_CLIMAX_MANUAL&RevisionSelectionMethod=LatestReleased&Rendition=Web.]
Kållberg, P., S. Uppala, and A. Simmons, 2010: The real first weather satellite picture. Weather, 65, 211–213, doi:10.1002/wea.652.
Kempe, V., D. Oertel, R. Schuster, H. Becker-Ross, and H. Jahn, 1980: Absolute IR-spectra from the measurement of Fourier-spectrometers aboard Meteor 25 and 28. Acta Astronaut., 7, 1403–1416, doi:10.1016/0094-5765(80)90015-6.
Khayat, M., S. Kempler, B. Deshong, I. Gerasimov, J. Johnson, E. Esfandiari, and M. Berganski, 2013: Implementing an open-source document preservation system at NASA GES-DISC. 2013 Fall Meeting, San Francisco, CA, Amer. Geophys. Union, Abstract IN13A-1548. [Available online at http://disc.sci.gsfc.nasa.gov/additional/publications/year_2013/implement_open_source_preservation/.]
Knapp, K. R., 2008: Scientific data stewardship of International Satellite Cloud Climatology Project B1 global geostationary observations. J. Appl. Remote Sens., 2, 023548, doi:10.1117/1.3043461.
Kobayashi, S., P. Poli, and V. John, 2015: CM-SAF visiting scientist activity CM_VS14_01 report: Characterisation of SSM/T-2 radiances using ERA-Interim and other reanalyses. ERA Rep. 21, 37 pp. [Available online at www.ecmwf.int/sites/default/files/elibrary/2015/10518-cm-saf-visiting-scientist-activity-cmvs1401-report-characterisation-ssmt-2-radiances-using.pdf.]
Lattanzio, A., and Coauthors, 2013: Land surface albedo from geostationary satellites: A multiagency collaboration within SCOPE-CM. Bull. Amer. Meteor. Soc., 94, 205–214, doi:10.1175/BAMS-D-11-00230.1.
Leroy, S. S., J. G. Anderson, and G. Ohring, 2008: Climate signal detection times and constraints on climate benchmark accuracy requirements. J. Climate, 21, 841–846, doi:10.1175/2007JCLI1946.1.
McNally, A. P., and P. D. Watts, 2003: A cloud detection algorithm for high-spectral-resolution infrared sounders. Quart. J. Roy. Meteor. Soc., 129, 3411–3423, doi:10.1256/qj.02.208.
National Academy of Sciences, Engineering, and Medicine, 2015: Continuity of NASA Earth Observations from Space: A Value Framework. National Academies Press, 105 pp.
Njoku, E. G., 2003: Nimbus-7 SMMR Pathfinder brightness temperatures. National Snow and Ice Data Center, Boulder, CO, digital media. [Available online at http://nsidc.org/data/docs/daac/nsidc0036_smmr_pathfinder_tbs.gd.html.]
Poli, P., and P. Brunel, 2016: Meteorological satellite data rescue: Assessing radiances from Nimbus-IV IRIS (1970-1971) and Nimbus-VI HIRS (1975-1976). ERA Rep. 23, 59 pp. [Available online at www.ecmwf.int/sites/default/files/elibrary/2016/16338-meteorological-satellite-data-rescue-assessing-radiances-nimbus-iv-iris-1970-1971-and-nimbus.pdf.]
Poli, P., C. Peubey, K. Fennig, M. Schroeder, R. Roebelling, and A. Geer, 2015: Pre-assimilation feedback on a fundamental climate data record of brightness temperatures from special sensor microwave imagers: A step towards MIPs4Obs? ERA Rep. 19, 50 pp. [Available online at www.ecmwf.int/sites/default/files/elibrary/2015/11695-pre-assimilation-feedback-fundamental-climate-data-record-brightness-temperatures-special.pdf.]
Poli, P., and Coauthors, 2016: ERA-20C: An atmospheric reanalysis of the 20th century. J. Climate, 29, 4083–4097, doi:10.1175/JCLI-D-15-0556.1.
Reichhardt, T., 2006: First photo from space. Air and Space Magazine, 24 October. [Available online at www.airspacemag.com/space/the-first-photo-from-space-13721411.]
Saunders, R. W., M. Matricardi, and P. Brunel, 1999: An improved fast radiative transfer model for assimilation of satellite radiance observations. Quart. J. Roy. Meteor. Soc., 125, 1407–1425, doi:10.1002/qj.1999.49712555615.
Schulz, J., and Coauthors, 2009: Operational climate monitoring from space: The EUMETSAT Satellite Application Facility on Climate Monitoring (CM-SAF). Atmos. Chem. Phys., 9, 1687–1709, doi:10.5194/acp-9-1687-2009.
Simmons, A., and Coauthors, 2016: Observation and integrated Earth-system science: A roadmap for 2016–25. Adv. Space Res., 57, 2037–2103, doi:10.1016/j.asr.2016.03.008.
Spänkuch, D., 1980: Arbeiten des Meteorologischen Dienstes der DDR auf dem Gebiet der indirekten Sondierung. Z. Meteor., 30, 205–214.
Stickler, A., and Coauthors, 2014: ERA-CLIM: Historical surface and upper-air data for future reanalyses. Bull. Amer. Meteor. Soc., 95, 1419–1430, doi:10.1175/BAMS-D-13-00147.1.
Suomi, V. E., 1958: The radiation balance of the Earth from a satellite. Ann. IGY, I, 331–340.
Takacs, L. L., M. Suarez, and R. Todling, 2016: Maintaining atmospheric mass and water balance within reanalysis. Quart. J. Roy. Meteor. Soc., 142, 1565–1573, doi:10.1002/qj.2763.
Taylor, K., R. Stouffer, and G. Meehl, 2012: An overview of CMIP5 and the experiment design. Bull. Amer. Meteor. Soc., 93, 485–498, doi:10.1175/BAMS-D-11-00094.1.
Teixeira, J., D. Waliser, R. Ferraro, P. Gleckler, T. Lee, and G. Potter, 2014: Satellite observations for CMIP5: The genesis of Obs4MIPs. Bull. Amer. Meteor. Soc., 95, 1329–1334, doi:10.1175/BAMS-D-12-00204.1.
Uppala, S. M., and Coauthors, 2005: The ERA-40 Re-Analysis. Quart. J. Roy. Meteor. Soc., 131, 2961–3012, doi:10.1256/qj.04.176.
Wentz, F. J., 2013: SSM/I version-7 calibration report. Remote Sensing Systems Tech. Rep. 011012, 46 pp.
Zeng, Y., and Coauthors, 2015: Analysis of current validation practices in Europe for space-based climate data records of essential climate variables. Int. J. Appl. Earth Obs. Geoinf., 42, 150–161, doi:10.1016/j.jag.2015.06.006.
Zou, C. Z., and W. H. Wang, 2011: Intersatellite calibration of AMSU-A observations for weather and climate applications. J. Geophys. Res., 116, D23113, doi:10.1029/2011JD016205.
