BACKGROUND.
Global climate modeling systems are the essential tools that provide climate projections. Observations play an essential role in the development and evaluation of these climate modeling systems. In particular, observations from satellite platforms often provide a global depiction of the climate system that is uniquely suited for these purposes.
The initial goal of the Observations for Model Intercomparison Projects (Obs4MIPs), launched by NASA and the U.S. Department of Energy (DOE), is to better exploit existing satellite measurements by making them more accessible for research involving the fifth phase of the World Climate Research Programme's (WCRP) Coupled Model Intercomparison Project (CMIP5)1. CMIP5 specifies a series of standard experimental protocols that facilitate the community-based study of coupled Earth system model simulations, and has been a centralizing resource for the Intergovernmental Panel on Climate Change Working Group I contribution to the Fifth Assessment Report (IPCC WGI AR5) and Summary for Policy Makers.
In a 2012 BAMS article, Taylor and colleagues describe in detail the protocol for CMIP5, which defines the scope of simulations that were undertaken by the participating modeling groups. For several of the prescribed retrospective simulations (e.g., decadal hindcasts, AMIP, and twentieth-century coupled simulations), observational datasets can be used to evaluate and diagnose the simulation outputs.
A broad range of observational datasets is used for climate model evaluation. The Obs4MIPs project was launched making selected NASA datasets more readily accessible for CMIP5 research, and efforts have been underway to enable other agencies and data experts to contribute well-established products with demonstrated value for model evaluation (see Summary below). Enthusiastic support for the project has been expressed by the WCRP's Data Advisory Council and via recommendations of a recent international workshop targeting systematic errors in climate models (www.metoffice.gov.uk/media/pdf/h/9/WGNE_Workshop_Summary_v1p0.pdf).
APPROACH.
Given the importance of observations to the model evaluation process, along with the range and complexity of the observational datasets needed for a robust assessment, a simple framework to identify, organize, and disseminate them for CMIP5 was created by Obs4MIPs.
The CMIP5 simulation protocol is utilized as a strict guideline for deciding which observations to stage in parallel to the model simulations—in particular: which variables, and for what periods, temporal frequencies, and spatial resolutions. Figure 1 illustrates the essence of the approach: The goal is to use the CMIP5 simulation protocol, produced by the WCRP's Working Group on Coupled Modeling (WGCM), which organizes climate model intercomparisons (top path in figure) to select the satellite observations that constitute the datasets being put together in this project and create a parallel path for the observations (bottom path in figure).
Schematic showing the essence of the Obs4MIPs approach: to use the CMIP5 simulation protocol, produced by WGCM, which organizes climate model intercomparisons (top path), to select the satellite observations that are in the Obs4MIPs project and create a parallel path for the observations (bottom path).
Citation: Bulletin of the American Meteorological Society 95, 9; 10.1175/BAMS-D-12-00204.1
Schematic showing the essence of the Obs4MIPs approach: to use the CMIP5 simulation protocol, produced by WGCM, which organizes climate model intercomparisons (top path), to select the satellite observations that are in the Obs4MIPs project and create a parallel path for the observations (bottom path).
Citation: Bulletin of the American Meteorological Society 95, 9; 10.1175/BAMS-D-12-00204.1
Schematic showing the essence of the Obs4MIPs approach: to use the CMIP5 simulation protocol, produced by WGCM, which organizes climate model intercomparisons (top path), to select the satellite observations that are in the Obs4MIPs project and create a parallel path for the observations (bottom path).
Citation: Bulletin of the American Meteorological Society 95, 9; 10.1175/BAMS-D-12-00204.1
The main tasks of Obs4MIPs are to
engage with the climate modeling, observational, and analysis communities to identify potential observational datasets for model evaluation and diagnostics, strictly following the CMIP5 protocol document;
work with the observational teams to establish the necessary metadata information for the candidate observational datasets while documenting as best as possible the relative quality of the observations and their applicability for direct comparison to model quantities, and produce a technical document addressing these issues;
enable the observational science teams to facilitate production of the identified datasets, with the needed characteristics (variables, periods, resolutions) and formats [e.g., adhering to the Climate-Forecast (CF) metadata convention as applied in CMIP5]; and
organize and disseminate these datasets in a manner that closely parallels the model data archive.
DATA.
The goals for the tasks described above were achieved for the initial datasets by directly involving the NASA science teams responsible for the relevant observational datasets. A variety of satellite data products were considered. It was felt that for a successful outcome of the first phase of this project it was more important to produce a relatively small but reliable set of observational products. Essentially all of the selected products have been publicly available for some time, but have not historically been tailored for a direct comparison with climate models with respect to output statistics, format, and metadata information.
Table 1 highlights the initial Obs4MIPs datasets that were available with documentation when this paper was submitted. The initial datasets include key climate variables that are being routinely produced from space-based observational systems such as atmospheric temperature profiles from the Atmospheric Infrared Sounder (AIRS) and the Microwave Limb Sounder (MLS) instruments; specific humidity profiles from AIRS and MLS; mole-fraction of ozone from the Tropospheric Emission Spectrometer (TES); sea surface temperature from the Advanced Microwave Scanning Radiometer (AMSR-E); top-of-the-atmosphere longwave and shortwave radiation from the Clouds and the Earth's Radiant Energy System (CERES) instrument; total cloud fraction from the Moderate Resolution Imaging Spetro-radiometer (MODIS); AVISO sea surface height from the TOPEX and JASON instruments; total surface precipitation from the Tropical Rainfall Measuring Mission (TRMM); and the 10-m (above the surface) wind over the ocean from QuikSCAT. More recent additions and planned contributions to Obs4MIPs are summarized in the Summary below.
Initial set of obs4MIPs published and documented datasets (at date of submission). The datasets are 1 × 1 degree Lat-Lon monthly averages, with global coverage, unless otherwise noted. The temperature, specific humidity, and ozone datasets are also vertically stratified at the CMIP5 required pressure levels.
This initial set of satellite observations—which is expected to grow over time—is directly accessible from the Earth System Grid Federation (ESGF) supporting CMIP, providing a readily accessible and focused resource for climate model evaluation. A first set of Obs4MIPs datasets and corresponding technical documents can be obtained at http://esg-datanode.jpl.nasa.gov/esgf-web-fe/.
A particularly important component of Obs4MIPs is the production of technical documentation synthesizing the most essential information needed by the researchers that will analyze the models and the observations. These documents have been produced (often one per variable), and basically contain detailed information about the data field and data origin (e.g., “measurement-to-product” processing), validation and uncertainty estimates, considerations for model– observation comparisons (e.g., sampling biases), the instrument overview, and finally, key references and points of contact.
EVALUATING CLIMATE MODELS WITH OBSERVATIONS: A BROADER PERSPECTIVE AND DISCUSSION.
Up to this point, we have concentrated our discussion on satellite observations. A key reason is the fact that, to a good approximation, measurements made from satellite platforms are global in nature. However, it is envisioned that the strict metadata and data constraints applied to CMIP5 will be generalized to facilitate the inclusion of in situ data within Obs4MIPs. As a test case, multiyear measurements of atmospheric structure at specific fixed locations have been made available, including from the DOE's Atmospheric Radiation Measurement (ARM) program “best estimates” (ARMBE) of key observables at selected ARM sites. In fact, one of the more novel aspects of the CMIP5 output, coordinated with the Cloud Feedback Model Intercomparison Project (CFMIP), is the high-frequency archiving of selected quantities at several locations around the globe (including the ARM locations).
In a related effort, selected output from the major analysis and reanalysis products is being made available in a similar manner to Obs4MIPs. Analyses are optimal combinations of observations and complex dynamical models, which while suffering from shortcomings inherent to the dynamical model, data-assimilation method, and the quantity and quality of the observations used, are often capable of producing high-quality products associated with variables that tend to be difficult to produce directly from satellite-based measurements.
Initial reanalysis products (currently available via ESGF under the project name “Ana4MIPs”) consist of monthly averaged output provided from NASA's Modern Era Retrospective Analysis (MERRA), with plans to include NOAA's Climate Forecast System Reanalysis (CFSR) and twentieth-century Reanalysis (20CR), the European Center for Medium-range Weather Forecasts Interim reanalysis (ECMWF-Interim), and the Japanese Meteorological Agency (JMA) 25-year Reanalysis (JRA-25). These datasets are being published on ESGF in a similar way to CMIP5 and Obs4MIPs.
The evaluation and diagnostics of climate models using complex observations such as the ones produced from satellite remote sensing is a field that is growing in sophistication. The current Obs4MIPs, and companion efforts such as one by the CFMIP communi t y (CFMIPOBS; http://climserv.ipsl.polytechnique.fr/cfmip-obs/), are the initial steps in a long-term effort to bring together expertise in climate modeling and observations to improve climate projections. We anticipate that for future climate-model intercomparison endeavors, the observational community will play a larger role in helping to define the requirements for the model intercomparison output; in fact, a workshop is currently being organized with the goal of improving the use of satellite data for the next-generation model intercomparison, CMIP6. In this context, some efforts, including CFMIP-OBS, are trying to go beyond a more traditional comparison between model output and observationally derived (retrieved) geophysical variables using what is often referred to as “observation simulators.”
These efforts could be thought of as part of a more comprehensive process that is depicted in Fig. 2, where four different stages of model-observation comparison are illustrated in a simple manner. Stage I refers to the very early efforts, when model-derived quantities (e.g., temperature) could not be directly compared to satellite-observed quantities (e.g., radiances as in Fig. 2). The traditional approach, illustrated in stage II, involves the development of retrieval algorithms that attempt to solve the problem of obtaining geophysical variable values from directly measured quantities (e.g., from radiances to temperatures). As mentioned, recent efforts have moved the field to stage III, where observation simulators attempt to simulate the quantities directly measured by satellite instruments from model-derived geophysical quantities (e.g., from model temperature to model radiances). Many of these efforts have an origin in modern data-assimilation (for numerical weather prediction) systems that assimilate radiances directly by using observation simulators (also known as “forward models” or “forward operators”). A final stage IV is achieved when observation simulators and retrieval algorithms are combined on the modeling side to produce a model-derived geophysical variable that mimics as much as possible the measurement/retrieval procedure, which would help us to understand the uncertainties of models and observations in “model space.” Stage IV allows for the comparison between geophysical variables from both the modeling and the observational systems, which is often more intuitive to analyze than a comparison in “observation space.” Although the decision concerning which stage needs to be attained in a particular model–observation comparison will depend on a variety of factors (e.g., what is the specific process being investigated? Which observational system is being used?), it is clear that any future efforts in this exciting and growing field of model evaluation with satellite observations will be at one of the stages of this diagram.
The four stages of model vs satellite observations comparison.
Citation: Bulletin of the American Meteorological Society 95, 9; 10.1175/BAMS-D-12-00204.1
The four stages of model vs satellite observations comparison.
Citation: Bulletin of the American Meteorological Society 95, 9; 10.1175/BAMS-D-12-00204.1
The four stages of model vs satellite observations comparison.
Citation: Bulletin of the American Meteorological Society 95, 9; 10.1175/BAMS-D-12-00204.1
SUMMARY.
In this short paper, the Obs4MIPs project is summarized. The main goal of Obs4MIPs is to serve the climate-science community that will analyze CMIP5 simulations by facilitating the accessibility to well-established observational products, specifically those suited for model evaluation. The essence of the method devised to achieve this goal is to strictly follow the CMIP5 protocol document (Taylor et al. 2012) that specifies the output for the CMIP5 simulations (e.g., variables, statistics, metadata). By following this document, it was possible to create a fairly small (compared to the large variety of climate-related observational datasets in existence) set of satellite observations that strictly comply with the output demands of the climate-model simulations. The different mission and instrument projects responsible for these specific observations have been heavily involved in the processing of the Obs4MIPs datasets and the elaboration of the accompanying technical documents that describe the key aspects of each product. The Obs4MIPs data are available from the ESG websites accessible from PCMDI, the Jet Propulsion Laboratory (JPL), and the other ESGF gateways.
At the time of the final version of this paper (late 2013), a variety of additional datasets have been added to Obs4MIPs, including: Aerosol optical depth over land from the Multiangle Imaging SpectroRadiometer (MISR) and over ocean from MODIS, CERES surface radiation budget, Leaf Area Index (LAI) from MODIS, and a number of satellite simulator products contributed by CFMIP-OBS. The NASA Science Working Group has also recommended including the MODIS-derived snow cover product, and the NOAA National Snow & Ice Data Center (NSIDC) sea ice concentration climate data record.
One important challenge during this first phase of Obs4MIPs has been the selection of data among different observational products that may produce similar climate (geophysical) variables. In the early stages, we have relied on NASA's instrument science teams in conjunction with an informal working group to provide scientific and technical expertise in making the selection. As Obs4MIPs has grown, the inclusion of new datasets or the replacement of existing datasets has required broader oversight. NASA has established an Obs4MIPs Science Working Group to help shepherd the process forward with PCMDI/DOE and NOAA participation.
Obs4MIPs is now being fostered by WCRP, and an international task team is being established by the WCRP's Data Advisory Council (WDAC) to help shepherd the evolution of obs4MIPs and provide a governance framework as it expands to more agencies and international contributors. In the meantime, we strongly encourage other observational teams and experts to consider contributing to Obs4MIPs. More information about Obs4MIPs and how to contribute data can be found at http://obs4mips.llnl.gov.
Along with the desire to have this activity serve as a means for observations to inform model development and evaluation, it is also hoped that it will lead to more feedback from the model development and research communities into the formulation of new observational systems.
ACKNOWLEDGMENTS
This research was partly carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. Work at Lawrence Livermore National Laboratory was carried out under the auspices of the U.S. DOE under contract DE-AC52-07NA27344. Renu Joseph of DOE has been instrumental in helping to foster this interagency collaborative effort.
FOR FURTHER READING
Bodas-Salcedo, A., and Coauthors, 2011: COSP: Satellite simulation software for model assessment. Bull. Amer. Meteor. Soc., 92, 1023–1043, doi:10.1175/2011BAMS2856.1.
Gleckler, P., R. Ferraro, and D. Waliser, 2011: Improving use of satellite data in evaluating climate models. Eos, Trans. Amer. Geophys. Union, 92, 172, doi:10.1029/2011EO200005.
IPCC, 2013: Summary for policymakers. Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, T. F. Stocker et al., eds. Cambridge University Press, 3–29.
Jonkheid, B. J., R. A. Roebeling, and E. van Meijgaard, 2012: A fast SEVIRI simulator for quantifying retrieval uncertainties in the CM SAF cloud physical property algorithm. Atmos. Chem. Phys., 12, 10 957–10 969, doi:10.5194/acp-12-10957-2012.
Rienecker, M. M., and Coauthors, 2011: MERRA— NASA's Modern-Era Retrospective Analysis for Research and Applications. J. Climate, 24, 3624–3648, doi:10.1175/JCLI-D-11-00015.1.
Taylor, K. E., R. J. Stouffer, and G. A. Meehl, 2012: An Overview of CMIP5 and the experiment design. Bull. Amer. Meteor. Soc., 93, 485–498, doi:10.1175/BAMS-D-11-00094.1.
Teixeira, J., D. Waliser, R. Ferraro, P. Gleckler, and G. Potter, 2011: Satellite observations for CMIP5 simulations. CLIVAR Exchanges (Special Issue on the WCRP Coupled Model Intercomparison Project Phase 5—CMIP5), 16, 46–47.
Williams, D. N., and Coauthors, 2009: The Earth System Grid: Enabling access to multimodel climate simulation data. Bull. Amer. Meteor. Soc., 90, 195–205, doi:10.1175/2008BAMS2459.1.
