1. Introduction

Observing system simulation experiments (OSSEs) are data assimilation (DA) and forecast experiments in which simulated observations are used in place of real observations. (Acronyms are listed in Table A1.) Observations in OSSEs are simulated by using atmospheric state variables from a long numerical weather prediction (NWP) model run as input to forward operators—for example, a radiative transfer model for simulating satellite radiances—and adding errors with appropriate statistical properties. Since the truth from the NWP model run or “nature run” (NR) is perfectly known, the OSSE framework is efficient and effective for many purposes, including the evaluation of the analysis and forecast skill impact of observations from proposed or new instruments and platforms or from new data assimilation techniques. In contrast to OSSEs, in observing system experiments (OSEs)—that is, DA and forecast experiments using real observations—the true atmosphere is not perfectly known, but instead is sampled irregularly in space and time by imperfect instruments. Therefore, a major advantage of OSSEs is that results can be validated with respect to the truth. Further, in OSSEs, observations can be simulated for any proposed instruments, including entirely new types of sensors, as long as realistic forward operators and methods of specifying the locations and error characteristics of the observations are available. OSSEs are also useful to increase readiness for upcoming/new sensors—to reduce the time for these data to be used in operational DA systems and to mature the science necessary to extract the maximum information content from these observations. Note that while conceptually OSSEs can be applied to any environmental prediction model—ocean, land, space, etc.—the focus of this paper is global NWP OSSEs. Over time, OSSE systems have tended to become more realistic. For example, thanks to efforts to maintain up-to-date operations to research (O2R) test beds, global OSEs, and OSSEs are now performed with operational DA and forecast components (e.g., Boukabara et al. 2016). Also, OSSEs have been applied to other forecast systems (e.g., for hurricanes; Atlas et al. 2015a). For a more thorough description of the scientific and technological background of OSSEs and an overview of recent and anticipated progress in OSSEs, see Hoffman and Atlas (2016) and references therein.

In response to the challenges of creating, maintaining, and validating a state-of-the-art OSSE system, this article describes a collaborative effort by the Joint Center for Satellite Data Assimilation (JCSDA), the National Oceanic and Atmospheric Administration (NOAA) Atlantic Oceanographic and Meteorological Laboratory (AOML), NOAA/Earth System Research Laboratory (ESRL), and the National Aeronautics and Space Administration (NASA) Goddard Space Flight Center (GSFC) to assemble, calibrate, and validate the next-generation OSSE system using the NOAA/National Centers for Environmental Prediction (NCEP) Global DA System.

This new OSSE system is the Community Global OSSE Package (CGOP) and is the first major update since the European Centre for Medium-Range Weather Forecasts (ECMWF) T511 NR was created in July 2006 (Masutani et al. 2007). CGOP goals include fostering collaboration and making realistic OSSEs accessible to the wider community. Toward these goals, the CGOP is designed to be highly flexible, user friendly (i.e., relatively easy to install and use), well documented, and regularly synchronized with the operational versions of the global DA system, the Community Radiative Transfer Model (CRTM), and other OSSE system components that are adopted from operations and the research community, thereby enhancing the credibility of results from OSSEs made using the CGOP. The key components of the CGOP distributed by the JCSDA are illustrated in Fig. 1. Creating a full OSSE system or porting an existing one to a new computing environment is a substantial undertaking. A particular concern is that the operational systems are sustained by a complex web of dependencies, and porting all the required libraries requires painstaking attention to detail. The CGOP includes all required libraries in order to be a stand-alone package. A key feature of CGOP is its flexibility in incorporating other packages developed by different members of the research community, including the CRTM developed by JCSDA, the DA system developed by NOAA/NCEP, the error addition tool developed by NASA, and the radio occultation (RO) observation simulator developed by NOAA/Office of Oceanic and Atmospheric Research (OAR). Thus, the CGOP represents a unique consolidation into a single package, of many components of the OSSE process that in the past were distributed across many institutions and centers of expertise.

View Full Size

The CGOP components provided by the JCSDA include forward operators to simulate observations, operational DA system components maintained by an ongoing O2R process, and a set of assessment tools with a unified graphical user interface (GUI).

Citation: Journal of Atmospheric and Oceanic Technology 33, 8; 10.1175/JTECH-D-16-0012.1

• Download Figure
• Download figure as PowerPoint slide

As the CGOP is developed, NOAA is testing and validating the CGOP components in OSSEs that are currently underway to assess several different future observing system configurations. It is our goal to continually evolve this system, so that it remains state of the art, calibrated, and validated. We intend to regularly release updated versions of the OSSE package to engage the research community both inside and outside NOAA. Interested readers can obtain information to access the current release of CGOP and the NR from the authors. Improvements under current consideration are listed in section 7. For example, JCSDA plans to enhance the DA system in CGOP from the current operational NOAA hybrid three-dimensional (3D) ensemble–variational (EnVar) gridpoint statistical interpolation (GSI) to the hybrid four-dimensioinal (4D) EnVar GSI that became operational 1200 UTC 11 May 2016. As part of this development, the 4D system will be benchmarked on several different computing environments.

2. A functional description of the OSSE components

Here we describe the functions and interrelationships of the main elements of the OSSE framework. These elements are ordered in the development of CGOP and in the sections of this paper as depicted in the flowchart in Fig. 2. The basic methodology used for OSSEs is based on that implemented by Atlas et al. (1985a,b). Calibration and validation are critical to the overall structure of the development of a rigorous OSSE, but the discussion of these activities is postponed to a later study.

View Full Size

A top-level view of an OSSE system (after Atlas et al. 1985a; Atlas and Pagano 2014). Colors are orange for the NR, blue for observation simulation, green for DA, and yellow for assessments. Abbreviations conv., sim., and obs. are used for conventional, simulated, and observations, respectively. A flattened hexagon indicates an activity with a possible branch back for another iteration. Refer to the text for explanation.

Citation: Journal of Atmospheric and Oceanic Technology 33, 8; 10.1175/JTECH-D-16-0012.1

• Download Figure
• Download figure as PowerPoint slide

First, an NR or reference atmosphere, a long atmospheric model integration using a very high resolution “state of the art” numerical model, is created to provide a complete record of the “true” state of the atmosphere (section 3). Conventional and space-based observations are then simulated from the nature run for existing observing systems (section 4). The simulation process interpolates the NR to the observing location, evaluates the forward operator for the observation type, and adds errors. The CGOP has the ability to simulate a large variety of observations, including conventional surface and upper-air observations (section 4d), observations made by tracking features in imagery (section 4e), radiance observations (section 4f), RO observations (section 4g), and observations from the Tropical Cyclone Vitals Database (TCVitals; section 4h). Random measurement errors—and in some cases, explicit biases—should also be added. The DA and forecast systems for the CGOP are defined to be as similar as possible to the NOAA/NCEP global DA and forecast systems that became operational 1200 UTC 14 January 2015 (section 5). The operational system components include the cycling procedures, forecast model, the DA system, and the set of packages to evaluate analysis and forecast skill.

With the OSSE system validated and calibrated, OSSEs for proposed observing systems can be conducted. It should be noted, however, that no OSSE system is perfect. For every OSSE system, it is essential that the limitations of the system are documented and that no conclusions are drawn that are inconsistent with these limitations (Atlas et al. 1985a). Conventional or space-based observations for the new observing system are simulated (section 6) from the NR in a manner similar to that for existing observing systems. These new observations should be simulated with the expected coverage, resolution, and accuracy. In some cases, enhancements to the DA system designed to fully utilize the observations from new sensors are also tested in the OSSE environment. In each case, specific characteristics of the new observing system will usually need to be accommodated. To run the OSSE, a control experiment is run first to assimilate the observations currently used operationally. Then experimental DA cycles and forecasts are created by adding the new data to a control experiment, and the impact of the new data on the analyses and forecasts is assessed. In a similar fashion, the OSSE framework can also assess the impact of a new forecast model or new DA technique. The evaluation of an OSSE includes both 1) statistical metrics to obtain a quantitative estimate of the impact of proposed observing systems on the expected accuracies of the analysis and forecast products that incorporate the new data and 2) synoptic assessments to better understand the physical mechanisms that cause the impact on analysis and forecast skill (e.g., Atlas et al. 1982; Atlas 1982).

This paper also includes two concluding sections: Section 7 lists the potential improvements to the CGOP, and section 8 provides the summary and concluding remarks. Ultimately, CGOP will be extended to an Earth system modeling framework (ESMF) with components including land, ocean, aerosols, and more as described by Hoffman and Atlas (2016).

3. Nature run description

The current version of the CGOP includes a single NR, a 2-yr, 7-km-resolution, nonhydrostatic simulation created with the Goddard Earth Observing System Model, version 5 (GEOS-5), atmospheric general circulation model at NASA’s Global Modeling and Assimilation Office (GMAO). This GEOS-5 NR (G5NR; Putman et al. 2015) has higher resolution than the ECMWF T511 NR (Andersson and Masutani 2010) that was used in a number of OSSEs in the past (e.g., Atlas et al. 2015b, and references therein). For the G5NR run, GEOS-5 was implemented on the c1440 cubed sphere with 72 layers and an upper boundary at 1 Pa (approximately 85 km). Each face of the cube has 1440 × 1440 grid points. The GEOS-5 setup used to generate the G5NR is detailed by Putman et al. (2014). The G5NR initial conditions are interpolated from the ½° × ⅔° latitude–longitude Modern-Era Retrospective Analysis for Research and Applications (MERRA) reanalysis valid 21 UTC 15 May 2005. The G5NR sea surface temperature (SST) and sea ice are from the daily ⅝1/20° Operational Sea Surface Temperature and Sea Ice Analysis (OSTIA) product (Donlon et al. 2012). The model prognostic variables include ozone (O₃), carbon monoxide (CO), carbon dioxide (CO₂), and 15 radiatively active aerosol tracers. In addition, the Goddard Chemistry Aerosol Radiation and Transport (GOCART) module has parameterizations for chemical production of sulfate () from oxidation of sulfur dioxide (SO₂) and dimethyl sulfide (DMS). Surface fluxes of aerosols and trace gases, including emissions from volcanoes and biomass burning, as well as other biogenic and anthropogenic sources and sinks, are specified with 24-h resolution.

For ease of use, the G5NR output is saved on a (i.e., 0.0625° × 0.0625° or approximately 7 km) latitude–longitude grid every 30 min and on a ½° × ½° latitude–longitude grid every hour. The archive includes many quantities of interest to simulate observations, including 10-m neutral winds and cloud fraction and optical thickness at every model level. The G5NR archive is composed of a number of collections as described by da Silva et al. (2014). These collections may be at full resolution (i.e., on the grid) or at reduced resolution (on the ½° × ½° grid), instantaneous or time averaged, and on the 72 model vertical layers or on 42 pressure surfaces. For reference, a full-resolution 3D variable at a single time is contained in a file approximately a gigabyte in size. Note that the method of determining the archived gridpoint values from the model gridpoint values varies with collection and includes simple latitude–longitude linear interpolation, nearest-neighbor assignment, and mass-conserving interpolation. Some additional details about the G5NR data collections that are important for simulating observations are discussed in section 4. Since the entire G5NR archive is almost 4 PB in size, it is not trivial to obtain a copy. A small sample is included in the CGOP download for testing. In practice, the needed G5NR data may be downloaded using file transfer protocol (FTP) or accessed remotely via Open-Source Project for a Network Data Access Protocol (OPeNDAP; http://gmao.gsfc.nasa.gov/projects/G5NR/). The CGOP automates downloading the required fields by FTP.

4. Observation simulation

This section describes the generation of “perfect” observations and how explicit errors are specified and added. The main processes and outputs in simulating observations—illustrated in Fig. 3—are 1) interpolate the NR to the observing locations, 2) invoke the forward operators to 3) create the perfect observations, and 4) add explicit errors to 5) create the simulated observations. For a proposed sensor, the additional steps necessary to incorporate new simulated observations in the DA system are described in section 6. Currently, two forward operators are included in the package, including CRTM for simulating satellite radiances [section 4f(2)] and a forward operator for Global Navigation Satellite System/radio occultation (GNSS/RO) bending angle observations (section 4g). Note that these forward operators are used both in simulating observations and assimilating observations. Consequently, the simulated errors must account for the difference between the actual physics of the observing system and the forward model. Since CGOP is flexible, alternative forward models could be used in simulating observations and this might provide more realistic observations provided the alternative forward model is closer to reality than the one used in the DA system. Conventional data are interpolated directly from the NR—no forward calculations are required for the conventional data types such as radiosonde observation and other in situ observations.

View Full Size

The simulation of observations in the CGOP. Refer to the text for explanation.

Citation: Journal of Atmospheric and Oceanic Technology 33, 8; 10.1175/JTECH-D-16-0012.1

• Download Figure
• Download figure as PowerPoint slide

Observations are also simulated during the DA when the observation innovations (observations minus background values) are calculated by the GSI (section 5c). The two approaches are similar in many respects, but there are important differences, including the treatment of errors, time interpolation, horizontal and vertical resolution, and, in some cases, the forward operators. These differences contribute to the representativeness errors in the simulated observations. The GSI estimates of error statistics are used as preliminary estimates for simulating explicit errors, but these statistics are normally tuned during the calibration procedures (section 4c).

f. Radiance observations

Simulating satellite data requires first defining the characteristics of the observation—that is, the location and viewing geometry [section 4f(1)], and the sensor spectral response—and then modeling the radiative transfer processes that result in the measured radiance. The measured or simulated satellite radiances include three components: radiation reflected or emitted by the surface (land, including vegetation and snow cover, ocean, and sea ice), atmospheric absorption and emission, and also the radiance scattered, emitted, and/or reflected by hydrometeors, such as clouds. Since there is a source of radiation from the surface [emission or reflection; section 4f(3)] and from clouds [emission or scattering from the cloud particles and hydrometeors, section 4f(4)], these effects should be included in the forward operator [section 4f(2)]. For satellite radiances, explicit errors are added independently for each channel for each location with no correlation. Error statistics are specified by channel, sensor, and satellite separately for land and ocean.

1) Defining the satellite observations

To simulate representative satellite observations, the timing and locations of individual observations and the observing geometry must be defined. The observing geometry includes scan angle and Earth incidence angle. CGOP evaluates radiances assuming homogeneous scenes in the horizontal, so the geometry of the satellite footprint or instantaneous field of view (IFOV) is not needed. For existing observing systems, these parameters may be taken from the radiance BUFR and GSI diagnostics files created when real radiances are assimilated. In some cases, the real observations may not be available for the simulation date; thus, the locations and observing geometry can be extracted from the observations for a different date. For example, to create a template for simulating Advanced Technology Microwave Sounder (ATMS) observations, which did not exist before 2012, we change the year to 2006 in real observation files from 2014. When such information does not exist for proposed instruments, it must be either calculated from the parameters describing the spacecraft orbit and the sensor pointing with an orbit simulator or extracted from the observations of an instrument with similar geometry. If the scan angle and Earth incidence angle are not included in the template, then CGOP includes the capability to calculate these from the location of the satellite and the location of the satellite footprints on Earth’s surface. If an orbit simulator is used, the observation times should be consistent with the sensor geometry and operational implementation. For example, simulated observation times should follow the instrument timing sequence, which depends on the intervals that are used by the instrument to step from one beam position to another beam position, and/or on the integration time.

2) CRTM forward operator

The CRTM is the CGOP and GSI forward operator for infrared (IR) and microwave (MW) radiances. The CRTM has been described and validated in different settings, including by Chen et al. (2008) for clear and cloudy conditions for the Advanced Microwave Sounding Unit (AMSU), the Microwave Humidity Sounder (MHS), and the Advanced Very High Resolution Radiometer (AVHRR); by Liang et al. (2009) for SST retrievals from AVHRR; and by Ding et al. (2011) for clear and cloudy conditions for the Atmospheric Infrared Sounder (AIRS). The CRTM inputs are the atmospheric profiles of temperature, water vapor, and atmospheric gases, and cloud properties. In CGOP, the values provided by G5NR and interpolated horizontally to NRIP include all the profile information needed by CRTM—the layer-averaged temperature and humidity and the layer and level (i.e., interface) pressures—for the calculation of clear-sky radiances. Surface quantities required in the radiative transfer calculation are surface pressure, surface temperature, and surface emissivity (additional parameters describing surface reflectivity are not needed for any of the MW and IR channels currently simulated by CGOP). Since CGOP makes use of the CRTM emissivity databases and models, additional parameters that must be specified are the 10-m wind speed and the surface type. Surface pressure, surface temperature, and 10-m winds are included in the NRIP data. Surface type is “interpolated” using the type of the nearest-neighbor grid point in the external databases described below. CRTM uses fast models for the calculation of layer optical depths. All the information needed for different sensors is included in a “coefficient” file for each sensor. The sensor coefficients are determined by best fitting the fast optical depth model to values calculated by the very accurate and computationally expensive line-by-line model. Therefore, the calculated optical depths are valid for the range of profiles (of temperature, water vapor, and other gases) in the training datasets used. CRTM outputs are the channel radiances and/or brightness temperatures ().

3) Surface characteristics

The G5NR includes SST, sea ice, and snow cover, and these are used as inputs in CRTM. The situation is more complicated for land surfaces. Each of the 11 Simple Biosphere Model (SiB) land types, which were derived from the U.S. Geological Survey (USGS) Global Land Cover Characteristics (GLCC) Data Base, Version 2.0, is mapped to one of the six G5NR land types. For CRTM, we need to convert the G5NR land types to one of the land type classifications included in CRTM, such as International Geosphere–Biosphere Programme (IGBP). However, there is also some ambiguity in mapping SiB to IGBP or vice versa. Since the SiB land types and in turn the G5NR land types are based on the GLCC, it is best to start with the GLCC land types converted to IGBP and SiB and then obtain the corresponding IGBP or SiB dominant land types. Essentially, the GLCC is the truth underlying the G5NR, and the transformations to SiB, G5NR, and IGBP land types are necessary approximations/parameterizations for different purposes. The CGOP includes files that give the dominant IGBP and SiB types in each 7-km G5NR grid cell (S. P. Mahanama 2015, personal communication). The IGBP dominant land type is used in the CRTM to specify the surface emissivity for all IR channels. Since the CRTM requires the Global Forecast System [GFS; i.e., Global Spectral Model (GSM)] soil and vegetation type to specify the surface emissivity for all MW channels, the SiB dominant land type is mapped to the “GFS” soil and vegetation type according to Table 1.

Table 1.

Mapping from SiB land type index (i) to GFS vegetation types index and GFS soil type index (in CRTM terminology GFS is used to indicate GSM).

View Table

4) Treatment of clouds

In reality, the DA system thins radiances from a single sensor to approximately 145-km spacing, preferentially selecting observations close in time to the analysis time and those that are likely not contaminated by clouds or precipitation as indicated by a close match between the observed and estimated window channels radiances. We will call these observations the selected observations. Note that radiances used in DA systems are organized so that observations from all sensor channels are present at each observed location at the same time. This is a requirement of retrieval systems and of the QC systems within the GSI radiance assimilation. For some instruments, including most modern MW sensors, this requires spatial resampling. Within GSI, the CRTM simulated IR radiances are compared to the selected observations by the QC procedures to detect cloud layers. Then IR channels with contributions to the radiances of greater than 2% from below the cloud top are rejected. The observations passing QC are used in the analysis (i.e., contribute to the observation sum of squares), but the selected observations failing QC continue to be processed passively (i.e., not contributing to the observation sum of squares) and are included in the GSI diagnostic files.

Currently, CGOP radiance simulations assume cloud liquid and ice water are zero and that there are no hydrometeors. In reality, clouds affect IR radiances, precipitation affects all but the lowest-frequency MW radiances, and ice clouds affect high-frequency MW radiances. CGOP produces the correct number and representative distribution of selected radiance observations because observations are simulated only where and when observations were selected by the DA system. This approach has the drawback that clear-sky radiances are simulated in situations that were flagged as cloudy by the GSI QC and not used in the data analysis in reality. In the OSSE, these radiances will probably not be flagged by the cloud detection QC checks, and then will provide too much information (below cloud top) to the GSI.

The calculation of cloudy radiances from variables typically archived in a NR, including the G5NR, presents some challenges. In reality, cloud optical thickness (COT) in each layer depends on the observing frequency, the cloud particle shape, cloud particle size distribution (PSD), and the liquid and ice water content for various species. Not all of these variables are archived for the G5NR. While COT is archived at a representative frequency, that is not sufficient to accurately simulate radiances at other frequencies. As an example, Fig. 4 compares the all-sky observed (Fig. 4a) and simulated (Figs. 4b,c) Geostationary Operational Environmental Satellite-12 (GOES-12) Imager 10.7-μm brightness temperatures 24 h after the start of the G5NR. The representative values of COT archived in the NR are too large and result in estimated radiances that are too cold when clouds are present (Fig. 4b).

View Full Size

Observed and simulated (K) in the 10.7-μm window channel. (a) The observations are from GOES-12 at 2045 UTC 16 May 2005. Both simulations—(b) using representative COT and (c) using COT calculated as explained in the text—are based on the G5NR valid 15 min later at 2100 UTC 16 May 2005, which is 24 h after the start of the G5NR.

Citation: Journal of Atmospheric and Oceanic Technology 33, 8; 10.1175/JTECH-D-16-0012.1

• Download Figure
• Download figure as PowerPoint slide

In the absence of PSDs, Li et al. (2015) developed a linear relationship that estimates COT as a function of frequency from liquid and ice water paths, variables that are available. The training set to develop this relationship, from a Weather Research and Forecasting (WRF) Model run with the double-moment 6-class (WDM6) microphysics scheme, included the liquid and ice water paths and the COT at various frequencies estimated from the forecast hydrometeor profiles. This result is much improved, as seen in the example shown in Fig. 4c (see section 7 for further discussion of improved radiance observation simulations).

h. TCVitals observations

TCVitals are issued in near–real time but are otherwise similar to best tracks. TCVitals are used by the operational global DA system to relocate the storm and as ordinary surface pressure observations, but with special estimates of the observation standard deviation (Kleist 2011). In these procedures, only central pressure and position are needed and only these variables are defined in the simulated TCVitals observations.

The CGOP contains sets of TCVitals observations for selected study periods both with and without added errors. To create error-free TCVitals observations, storms are detected and tracked in the G5NR in two stages. The first stage detects a tropical cyclone (TC) if the values of central pressure, maximum 10-m wind speed, maximum vorticity, and temperature difference between the storm core and surroundings satisfy fairly stringent criteria (Gelaro et al. 2015, chapter 4). This automated storm detection does not capture the genesis and lysis of the detected TCs. Therefore, a second stage searches forward and backward in time to extend the TCVitals observations for each storm based on the NR surface pressure and wind speed.

To create TCVitals observations with realistic errors, the general approach of tuning observational errors by matching observation innovations in reality and the OSSE cannot be applied. First, the sample of storms is small. Second, adding errors to the observation will not make up for deficiencies in the background. Third, a large part of the observation innovation may be due to location errors, not central pressure errors. As a result, we rely on published estimates of best track errors (Landsea and Franklin 2013). As expected, uncertainties for intensity and track are smaller when aircraft observations are available. As intensity increases, intensity uncertainty increases and position uncertainty decreases. As a result, our estimates of simulated errors are stratified by the intensity and availability of aircraft observations. As a proxy, we assume aircraft observations are available for all fixes west of 60°W in the Atlantic and Caribbean (see Fig. 4 in Torn and Snyder 2012).

TCVitals simulated observations are assumed to have zero mean Gaussian errors in central pressure and eastward and northward displacements. We simulate the errors in central pressure and eastward and northward displacements as , , and , respectively, where q, u, and υ are sampled from a standard normal distribution. The values reported by Landsea and Franklin (2013) are subjective uncertainty estimates for best tracks. Although TCVital errors are expected to be somewhat larger than best track errors, we take the values from Table 2 of Landsea and Franklin (2013) as estimates of the TCVitals uncertainty. According to C. Landsea (2015, personal communication), “mean absolute error may be the best way to consider these estimates” of uncertainty. Accordingly, we converted these values into the required standard deviations, and , which are displayed in Table 2, for three different classes of storms and for two regions—the western Atlantic and Caribbean and everywhere else.

Table 2.

TCVitals simulated error standard deviations and for central pressure (hPa) and for eastward and northward position (km) for different intensities and for the western Atlantic and Caribbean west of 60°W and the rest of the globe.

View Table

5. Data assimilation and forecast systems

The forecast model and data analysis system included in CGOP are summarized in this section. The CGOP system components are based on NCEP’s current operational procedures, implemented in January 2015, as described by NWS (2014). The observation simulation includes additional operational components, some of which are also part of the GSI observation operator, such as the CRTM. These were described in section 4. The CGOP distribution also includes the Independent Assessment Tools (IAT) package, which provides a graphical user interface (GUI) to a number of postprocessing tools previously developed at various NOAA centers for operations and research to monitor, visualize, and analyze DA and forecast results (Xu et al. 2015). The IAT presently includes the Verification Statistics Data Base (VSDB), radiance monitoring, hurricane intensity and track diagnostics, and two other tools to visualize the difference between two experiments [Gridded Binary (GRIB) extremes and forecast differences]. Note that the CGOP includes conversions from the G5NR to pressure-level GRIB (PGB) files to support verification of the gridded analyses and forecasts versus the NR.

b. Forecast model

The GSM current operational GFS configuration (see NWS 2014) is a 240-h forecast at T1534 resolution using a 64-layer sigma-pressure hybrid coordinate system and a semi-Lagrangian time integration formulation, followed by a lower-resolution T574 run to 384 h. The GDAS configuration is the same, but the forecast length is only 9 h. The research version uses T670 and T254 resolution and the same 64 vertical layers. The lower horizontal resolutions (T574 and T254) are also used in the ensemble Kalman filter (EnKF) described below. [Note that the resolution of these components is given in terms of their spectral truncation; for example, T574 means triangular truncation at total wavenumber 574. For each spectral truncation referred to in this paper, Table 3 lists several estimates of the spatial resolution as defined by Laprise (1992) and Abdalla et al. (2013).] The GSM prognostic variables are vorticity, divergence, logarithm of surface pressure, specific humidity, virtual temperature, cloud condensate, and ozone. A low-pass digital filter is applied at the initial time of all GDAS and GFS forecasts. The GSM includes parameterizations for all important physical processes and representations of various surface characteristics. The GSM is coupled to the Noah land surface model (i.e., the NOAA/NCEP–Oregon State University–Air Force Research Laboratory–NOAA/Office of Hydrology land surface model) following Ek et al. (2003). The GSM orography is based on the 30-arc-s (~1 km) USGS global digital elevation model (DEM).

Table 3.

Estimates of horizontal resolution (km) for different spectral resolutions. The Laprise (1992) estimate is the spacing of the transform grid points at the equator. Note that since the transform grid is chosen to represent quadratic terms with no aliasing, this grid is more closely spaced than is needed to represent the model state. The Laprise (1992) estimate, favored by others, is half the wavelength of the shortest wave going around the equator in the truncation. Based on matching spectra of model and radar altimeter surface wind speed, Abdalla et al. (2013) show that the effective resolution is typically 4 and the smallest fully resolved scale is 8 .

View Table

c. Hybrid analysis system

The current CGOP uses the hybrid GSI–EnKF analysis system. The GSI is a 3D variational analysis that is extensible and currently includes a wide range of observation operators (see Table 4). The GSI design philosophy includes using data in a form as close as possible to the actual observation in order to reduce the impact of the prior and the complexity of the observation errors present in more processed (retrieved) data. Thus, IR and MW radiance observations and GNSS/RO bending angle observations are used. Kleist et al. (2009b) gives an overview of the original GSI implementation at NCEP. Hu et al. (2014a,b) describe many technical details of using the GSI.

Table 4.

Satellite data sources. In CGOP experiments, instruments, locations, and times match those of data actually assimilated in the operational global DA system. All of these satellite observations are assimilated as radiances or brightness temperatures, except GNSS/RO, which is assimilated as bending angle.

View Table

The GSI includes a number of advanced features. A tangent-linear normal-mode constraint (TLNMC) reduces the amplitudes of gravity waves in the analysis increments (Kleist et al. 2009a). The GSI observation function compares the observations to the background at the observation time by means of first guess at appropriate time (FGAT; Rabier et al. 1998). The variational bias correction (VarBC) estimates errors in the radiances that depend linearly on a small set of predictors, including incidence angle and airmass factors. The coefficients of these predictors are determined for each analysis time, each sensor, and each channel, but they do not depend on location, except that the Special Sensor Microwave Imager/Sounder (SSMIS) coefficients are determined for several latitude bands. The new implementation of the VarBC used in CGOP as described by Zhu et al. (2014) uses a unified approach that spins up (equilibrates) more quickly than the previous version.

The GSI–EnKF hybrid configuration used in CGOP is described by Wang et al. (2013) and Kleist and Ide (2015a). The analysis is determined for the model layers on a linear-Gaussian grid—a grid of sufficient resolution to allow error-free transforms of linear computations. The 80-member EnKF follows the implementation of Hamill et al. (2011), but now uses stochastic physics. The GSI observation innovations are reused in the EnKF analyses. After each analysis the GSI solution is used to recenter the EnKF analysis ensemble. The EnKF forecast information is used in the GSI following Wang (2010)—the EnKF estimate of the background error covariance (BEC) is linearly combined with the static GSI BEC to create a hybrid BEC for use in the GSI. The weight given to the static BEC matrix is 0.25. The forecast from the GSI solution is run at full resolution (T1534 or T670) and is then truncated to lower resolution (T574 or T254) for the next GSI analysis.

6. Adding a new observation type

Adding a new observation type may pose several challenges. These include defining the observing pattern and geometry of the new sensor; generating a new CRTM optical depth coefficient file; developing BUFR software; and specifying the simulated error statistics. Here we outline the steps required in general and refer to some of the adjustments to the general procedure that were needed for two recently generated datasets—geostationary hyperspectral sounder (Geo-HSS; e.g., Schmit et al. 2009) and geostationary MW observations (Geo-MW; e.g., Lambrigtsen 2015).

As described in section 4f(1), we generally need the location of observations, observation time, as well as the geometry of the observations—for example, Earth incidence angle—to be able to interpolate the NR atmospheric state and surface information to the observation location. In the case of new instruments, if the orbit and observing geometry match a current or previous instrument, then datasets from that instrument can be used as the template for simulating observations with the times and possibly longitudes shifted to match the new instrument. For example, we simulate Geo-HSS data assuming an Infrared Atmospheric Sounding Interferometer (IASI) sensor in geostationary Earth orbit (GEO) at five operational locations. For GOES-East, observation locations are thinned to 21-km spacing for a satellite stationed at 75°W. For the other four satellites, these locations are shifted in longitude. Since the global DA system thins radiances to approximately 145-km resolution no matter what resolution is provided, simulating data at 21-km resolution is adequate even though the nominal spatial resolution of a Geo-HSS is 4 km. In principle, full-resolution data could be assimilated, but currently the DA system is limited by computational resources and assumes radiance errors are uncorrelated. Eventually, when DA systems overcome these limitations, Geo-HSS radiances should be simulated in the OSSE system at high spatial resolution with realistic correlated errors. If a template is not available, the locations of the satellite and observation locations must be estimated, for example, with an orbit simulator. An orbit simulator is not yet included in CGOP, but we note that in an OSSE setting, some simplifications are possible. For example, if we assume the satellite has sufficient capabilities and fuel to maintain a constant altitude, then atmospheric drag can be neglected.

To apply the CRTM forward operator for a new instrument, we must first calculate the lookup tables for the absorption coefficients. The required inputs for this calculation in general are the parameters that define each of the new sensor’s channels—the wavelength or frequency and the spectral response functions. For example, a proposed Geo-MW sensor has 10 channels, 6 near the 118-GHz oxygen resonance (i.e., spectral line) for temperature sounding and 4 near the 183-GHz water vapor resonance for water vapor sounding. New CRTM coefficients were determined for this sensor. The passband of the channels is normally well defined for the proposed instruments, but the spectral response function (SRF) or instrument line shape (ILS) depends on the hardware. In reality this is not known until the instrument is calibrated. For simulation of both real and simulated observation, a simple SRF is often used. For example, a boxcar-shaped passband is used for several microwave instruments by CRTM, including AMSU, MHS, and SSMIS, as well as Geo-MW. For an imaging IR Fourier transform instrument, the SRF varies from pixel to pixel and from channel to channel in reality, but in an OSSE setting it is adequate to use a single sinc function for all observations. Note that CRTM assumes horizontal homogeneity. For example, although the Geo-MW IFOV (say, 35 km) is several times larger than the G5NR 7-km grid resolution, the CGOP bilinearly interpolates the NR to the center of the Geo-MW IFOV and assumes horizontal homogeneity.

For use in the DA system, the simulated observations must be written in a supported format, which is normally BUFR for GSI. The BUFR writer/reader software (and associated tables) may need to be developed from scratch, but in most cases the BUFR software for a related sensor may be adapted. For instance, in the case of Geo-HSS, we use the existing BUFR software for IASI on polar-orbiting satellites. However, in the case of Geo-MW, since the proposed instrument is not identical to any of the current microwave instruments, it was necessary to develop new BUFR software, based on existing BUFR software for other MW instruments.

The error calibration procedure described by Errico et al. (2013) cannot be applied to a new sensor, since there is no way to calculate real observation innovations. As an approximation, we use the simulated error statistics from the most similar existing sensor. Bounding experiments with zero errors and very large errors may be used to establish the sensitivity of the OSSE results to this approximation.

7. Planned technical improvements

The CGOP is expected to evolve incrementally with time. In addition, the community OSSE package concept may be extended beyond global NWP (see section 8). A number of planned improvements to keep CGOP up to date and to mitigate some of the current limitations and issues are listed here.

1. New NRs: New NRs with higher resolution and improved simulated clouds are desired. Two selected months of the G5NR may be rerun with 3.5-km and 15-min resolution. A new ECMWF NR is under development with T1279 horizontal, 3-h temporal, and 91-level resolutions. The finite-volume icosahedral model (FIM) or possibly the nonhydrostatic icosahedral model (NIM) could provide a NOAA NR, but such a NR would have limited applicability, since these models share the physics of the GSM. AOML is developing a Hurricane Weather Research and Forecasting Model (HWRF) high-resolution basin-scale hurricane NR covering the entire North Atlantic using the nonhydrostatic mesoscale model (NMM) core.

2. Orbit simulator: For new sensors, it will generally be necessary to calculate the satellite orbit and sensor footprint geometry from design specifications. In some cases, a new instrument will be proposed with orbit and observing geometry so similar to a current or recent sensor that the footprint locations and times can be taken from archived observations, all shifted in time and/or longitude as needed. But for the general case, a model that propagates satellite ephemeris is desired.

3. More realistic and additional ground-based and aircraft observations: The locations of observations should be consistent with the NR weather. Planned improvements include simulating the drift of radiosondes with winds, and adjusting the locations of aircraft and ships to include their typical behavior with respect to weather conditions (aircraft like tailwinds, ships dislike intense storms). As additional observation types are added to the operational system, these should be added to the OSSE system. For example, realistic hurricane hunter observations simulated from typical flight paths are under development at AOML, including in situ, dropwindsonde, Stepped Frequency Microwave Radiometer (SFMR) and tail Doppler radar (TDR) observations.

4. More realistic distribution of radiance observations: Currently locations of radiances used in reality by the GSI are used to simulate current sensors. The locations of cloud-free IR and precipitation-free MW radiances should be consistent with the NR. The NR cloudiness cannot be used directly without some adaptation because the G5NR is too cloudy and the vertical distribution of cloud is not sufficiently realistic.

5. Realistic AMVs: The simulation of AMVs is not very realistic in the CGOP and in many earlier OSSEs (with exceptions noted below). In the CGOP, the observation patterns from reality are used to sample the NR. This means simulated AMVs may not be at locations with trackable features in the NR. Also, AMVs typically have errors in height assignment, which result in horizontally correlated errors. There are two possible paths for improving the simulation of AMVs. Bormann et al. (2014) demonstrates that it is possible to determine AMVs by simulating satellite imagery and then the AMV production process. Alternatively, statistical approaches could predict the locations and height assignment errors of AMVs from the NR humidity, clouds, and hydrometeors. For example, in the global OSSEs performed by Atlas and collaborators (e.g., Atlas and Emmitt 2008; and other studies cited previously), AMVs were simulated only where there were trackable features and with height assignment errors, which introduced speed biases and horizontally correlated errors.

6. Cloudy IR radiances: The use of cloudy radiances in future DA systems will require the simulation of realistic cloudy radiances. Since the G5NR and other NRs do not save everything necessary to rigorously simulate cloudy radiances, modeling of the cloud optical depths is necessary [section 4f(4)]. Note that cloudy radiances are much more uncertain in terms of representativeness due to many factors, including small-scale variations, particle size and shape variations, and scattering effects.

7. Surface-affected MW radiances: The use of surface-affected MW radiances in future DA systems will require coupling the NR atmospheric model with a realistic land surface model, validating and calibrating the coupled models, and using the resulting surface emissivity to simulate the MW radiances. Since the G5NR model includes a land surface component, surface-affected MW radiances could be calculated within the CGOP structure.

8. Correlated simulated observation errors: A new error addition methodology described by Errico et al. (2013) will be implemented. This approach calculates correlated error fields and then samples these at the observation locations.

9. Data preparation: In reality gross QC, data selection, data thinning, and other preprocessing activities can have large impacts on DA and forecast skill. All these data preparation steps should be included in future versions of the CGOP.

10. Hybrid 4D EnVar: The operational system has transitioned to the GSI-based hybrid 4D EnVar that uses ensemble perturbations valid at multiple time periods throughout the DA window to estimate 4D error covariances during the variational minimization, avoiding the tangent linear and adjoint of the forecast model (Wang and Lei 2014; Kleist and Ide 2015b). The O2R process will enable CGOP to follow this development. No changes to observation simulation will be required for the new system, except possibly to simulate data at more frequent intervals for GEO-based sensors, which in principle would be an advantage in a 4D system.

8. Summary and conclusions

Rigorous OSSEs require a complex infrastructure. The CGOP provides all the functional components of such an infrastructure for global NWP applications. Here, we described these components: the NR, simulation of existing and new observations, and the DA system, including cycling, the forecast model, analysis procedures, and verification procedures. The CGOP calibration and validation, which will be the subject of a future study, include comparing the NR to reality, checking the observation coverage and quality control yields, tuning the simulated observation innovation statistics to match those observed in reality, and comparing forecast skill in the OSSE system and in reality.

Thus, the CGOP provides a comprehensive, calibrated, and validated system that integrates forward operators for simulating observations with DA methods, forecast models, and postprocessing tools. The CGOP encapsulates the work of many collaborators carried out over several months and in some cases several years. It is anticipated that the effort to create the CGOP will serve the community well through ongoing release cycles. All components are fully tested in several high-performance computing environments. CGOP is fully subversion controlled, open source, and easily installed. All necessary software libraries can be compiled using one Makefile, and scripts to compile and run the CGOP are provided.

OSSEs provide a controlled setting where the truth is known and can be used for a variety of experiments, including those that quantify the impact on weather analyses and forecasts of 1) future instruments, such as additional GNSS/RO receivers, 2) future data gaps, such as the loss of the afternoon polar orbiters, and mitigation strategies for such data gaps, using, for example, 3) new data types, such as Geo-HSS, and 4) new DA techniques, such as a new forward operator for cloudy radiances. In addition OSSEs can be used to evaluate designs and improvements for many other applications, such as product retrieval and satellite data bias correction. The CGOP simplifies and encourages such undertakings.

It should be stressed that the CGOP is modular and extensible and is designed with flexibility to link to new or updated components (caution: changes to CGOP should usually be accompanied by additional calibration and validation). This article is a snapshot of the OSSE package, and it is important to note that there are a number of plans to update and improve on the OSSE tool components. After calibration and validation of the updated CGOP, these improvements will appear in subsequent releases. In section 7, we listed our plans for improvement, which may also be read as a list of limitations and caveats for the current version. These areas for improvement and caveats should be carefully considered in using the current version of the OSSE package, so that conclusions are not reached that are beyond the capability of the system. In addition to the evolutionary improvements described in section 7, major enhancements that are under consideration may lead to new community OSSE packages aimed at other components or coupled components of the earth system. Of particular interest are adding or coupling Earth system component models for the land surface, ocean, space weather, atmospheric chemistry, and aerosols as described by Hoffman and Atlas (2016). In addition, different packages might be constructed to study phenomena on different scales, including studies of tropical cyclones, severe storms, the coastal ocean, and urban air quality [e.g., the hurricane OSSE system described by Atlas et al. (2015a)]. As the OSSE package is meant to be a community-supported resource, you, the user community, are invited to suggest and implement improvements and enhancements.