Advances in operational NWP will greatly challenge our ability to conduct realistic OSSEs and require us to add extensions and innovations to future OSSE systems.
Observing system simulation experiments (OSSEs) provide a rigorous, cost-effective approach to evaluating the potential impact of new observing systems and alternate deployments of existing systems and to optimizing observing strategies. They are also used to prepare for the assimilation of new types of data in order to accelerate their application to operational prediction as well as to optimize the assimilation of existing data. Examples of how OSSEs have been used are given in the sidebar “Past OSSEs and value to decision-makers.” OSSEs are an extension of observing system experiments (OSEs), which are data-denial experiments run to determine the impact of existing observing systems. Atmospheric OSSEs determine the impact of new observing systems by performing data-denial experiments that assimilate synthetic observations simulated from a realistic nature run (NR) stipulated to represent the “true” atmosphere.
PAST OSSES AND VALUE TO DECISION-MAKERS
OSSEs are extremely useful in an-swering numerous questions about the impact and optimal use of observing systems before they are developed and deployed. Since 1985 Atlas and others have conducted an extensive series of global OSSEs using the methodology of the “Basic OSSE description” section. These OSSEs evaluated quantitatively the following:
The relative impact of temperature, wind, and moisture profiles from polar-orbiting satellites (Atlas et al. 1985a,b)—these experiments showed wind data to be more effective than mass data in correcting analysis errors and indicated significant potential for space-based wind profile data to improve weather prediction. The impact on average statistical scores for the Northern Hemisphere was modest, but in approximately 10% of the cases a significant improvement in the prediction of weather systems over the United States was observed.
The relative importance of upper- and lower-level wind data—these experiments showed that upper-level wind profile data, at and above 500 hPa, provided most of the impact on numerical forecasting (Atlas et al. 2001).
The impact of a space-based DWL (e.g., Rohaly and Krishnamurti 1993; Atlas 1997; Zhang and Pu 2010)—experiments were conducted including the effect of different orbital configurations and the effect of different amounts of power available to make observations; more recently, OSSEs were used to determine the specific requirements for space-based lidar winds for the proposed Global Wind Observing System and Global Tropospheric Wind Sounder missions (Atlas and Riishojgaard 2008; Riishojgaard et al. 2012; Ma et al. 2015; Atlas et al. 2015).
The relative impact of the European Remote Sensing Satellite (ERS) and NASA Scatterometer (NSCAT) prior to their launch—this relative impact was confirmed after the launch of these instruments; in addition, OSSEs were used to develop and test an improved methodology for assimilating both passive and active microwave satellite surface wind data, which led to the first beneficial impact of scatterometer data on numerical weather prediction, as well as to the assimilation of Special Sensor Microwave Imager (SSM/I) wind speed data (Atlas et al. 1996; Atlas and Hoffman 2000; Atlas et al. 2001, 2011; Atlas 2004).
The quantitative impact of observations from the Atmospheric Infrared Sounder (AIRS) and the importance of cloud clearing, which was later confirmed with real AIRS data (Chahine et al. 2006).
For the OSSEs to produce accurate quantitative results, all of the components of the OSSE system must be realistic. This means that
the NR, which is used to represent the atmosphere, should be generated by a state-of-the-art numerical model;
there should be realistic differences between the NR model and the model used for assimilation and forecasting;
the assimilation methodology must conform to current or future practices;
observations should be simulated with realistic coverage and accurately calibrated errors; and
the entire OSSE system must be validated to ensure that the accuracy of analyses and forecasts and that the impact of existing observing systems in the OSSE are comparable to the accuracies and impacts of the same observing systems in the real world.
One motivation for our consideration of future OSSEs is the development by the authors and our collaborators of a new advanced OSSE system for the National Oceanic and Atmospheric Administration (NOAA), which is based on the recently released National Aeronautics and Space Administration (NASA) Global Modeling and Assimilation Office (GMAO) global mesoscale NR (G5NR; Putman et al. 2015). This new system replaces the OSSE capability based on the European Centre for Medium-Range Weather Forecasts (ECMWF) T511 NR (Andersson and Masutani 2010). As we have developed the new OSSE system, we see many opportunities for further improvements and anticipate that some of these potential improvements will become requirements in the future as the OSSE technique is applied in new settings—in diverse and coupled domains and with the use of increasingly advanced and sophisticated simulations of nature and observations. For example, Halliwell et al. (2014, 2015) are now conducting OSSEs for the ocean. In addition, opportunities for further improvements are also challenges to current OSSEs and in general correspond to important considerations when interpreting the experimental results of any particular OSSE in support of decision-making. Thus, the list of opportunities/challenges as well as our list of requirements for current OSSEs constitutes an “OSSE checklist,” which is provided both as a supplement to this article and as an interactive web page at www.aoml.noaa.gov/qosap/osse-checklist/. Previously, Atlas et al. (1985a,b) and Arnold and Dey (1986) provided suggestions for best practices for OSSEs.
In this paper, we will first define the OSSE method and introduce the concept of a coupled OSSE (see the next section). Since OSSEs are most advanced for global atmospheric forecast and DA systems, we draw examples from our development of a new NOAA OSSE system (see the “Development of a new OSSE system” section). Then, the future of each major OSSE system component is considered in turn in the “Future nature runs” section, the “Future forecast and data assimilation systems” section, and the “Future observation simulation methods” section. That is followed by the section discussing future observing systems of interest. OSSE validation and calibration procedures are described next, again from the viewpoint of the G5NR. A look ahead to future OSSEs and enhancements to the OSSE system is given in the final section.
OSSE SYSTEM COMPONENTS.
Basic OSSE description.
The methodology currently used for OSSEs was established in the early 1980s to increase the realism and usefulness of such experiments (Atlas 1997; Atlas and Pagano 2014). It consists of the following elements (shown schematically in Fig. 1):
A long atmospheric model integration using a very high-resolution “state of the art” numerical model to provide a complete record of the assumed “true” state of the atmosphere referred to as the “nature run” or “reference atmosphere”—for the OSSE to be meaningful, it is essential that the NR be realistic, that is, that it possesses a model climatology, patterns of storm tracks, etc., that agrees with the real atmosphere to within prespecified limits.
Simulated conventional and remotely sensed observations from the NR—all of the observations should be simulated with observed (or expected) coverage, resolution, and accuracy. In addition, systematic errors and horizontal and vertical correlations of errors with each other and with the synoptic situation should be introduced appropriately.
Control and experimental data assimilation cycles—these are as similar as possible to operational assimilation cycles. Note that a different model from that used to generate the NR is used for assimilation and forecasting. Typically this model has less accuracy and resolution than the NR model. Ideally, the differences between the assimilation and nature models should approximate the differences between an operational model and the real atmosphere.
Forecasts produced from the control and experimental assimilations—the analyses and forecasts are then verified against the NR to obtain a quantitative estimate of the impact of proposed observing systems and the expected accuracies of the analysis and forecast products that incorporate the new data.
A top-level view of an OSSE system. For regional OSSEs the NR is taken to be a high-resolution regional (or hurricane) NR, which is embedded in a global NR. Elements taken from the operational DA system are shaded in green.
Citation: Bulletin of the American Meteorological Society 97, 9; 10.1175/BAMS-D-15-00200.1
An important component of the OSSE that improves the interpretation of results is validation against a corresponding OSE. In this regard, the accuracy of analyses and forecasts and the impact of already existing observing systems in simulations is compared with the corresponding accuracies and data impacts in the real world. This ensures that the results of the OSSEs are credible and realistic.
Coupled OSSE systems.
In a coupled OSSE system, the NR is generated with a coupled Earth system model. Here, individual components of the Earth system are modeled separately, but these models are coupled in a number of ways. For example, coupling between atmosphere and ocean is performed via fluxes at the ocean surface. For a coupled atmosphere and chemistry model, the coupling could be one way with the atmosphere providing winds, temperature, and humidity to the chemistry model. Or the chemistry model could feed back species concentrations and aerosols for use in the atmospheric radiative heating calculation. In a coupled OSSE system the data simulation should make use of information from the various component models. Another form of coupling is to host a higher-resolution limited-domain model in a coarser global model. In this scenario lateral boundary conditions are passed from the global model to the embedded model (and possibly back to the global model). Observations should be simulated using the high-resolution model where possible and the low-resolution model otherwise. An example of this type of coupling—in the Hurricane Weather Research and Forecasting Model (HWRF) OSSE system—is given by Atlas et al. (2015).
DEVELOPMENT OF A NEW OSSE SYSTEM.
The new or G5NR OSSE system contains comprehensive improvements compared to the old or T511 OSSE system. Note that we name these systems for the NR used, although it is entirely possible to change out the NR and keep everything else in one of these systems. Table 1 provides a summary of the T511 and G5NR OSSE system components, which are described in the following paragraphs.
Component-by-component comparison of the T511 OSSE system and the new G5NR OSSE system. Acronyms used in the table and not defined in the text are Community Radiative Transfer Model (CRTM), Stand-Alone AIRS Radiative Transfer Algorithm (SARTA), and Verification Statistics Data Base (VSDB). Also note that the abbreviation “obs” is used for observation. (Table created by S. P. F. Casey, JCSDA.)
ECMWF T511 global nature run.
The T511 NR was generated by ECMWF. The T511 NR is described by Andersson and Masutani (2010) and is a free-running forecast from 1 May 2005 to 1 June 2006 that used observed sea surface temperature and sea ice. The T511 spectral truncation corresponds to a horizontal resolution of 26 km and there are 91 vertical levels. [Note that here and elsewhere, for a given spectral resolution, we report the equivalent L1 resolution, as defined by Laprise (1992).]
GMAO mesoscale-resolution nature run (G5NR).
The GMAO 7-km-resolution NR (known as the G5NR) is a nonhydrostatic global mesoscale simulation for the period 16 May 2005–16 June 2007 produced with the Goddard Earth Observing System (GEOS-5) atmospheric general circulation model (Putman et al. 2015). The geometry of G5NR is the cubed sphere, in which Earth is projected onto a cube. This allows great computational speed through parallelization. Each face of the cube has 1440 × 1440 grid points, approximately 2 million points per face. With six faces and 72 layers at each location, the G5NR divides the global atmosphere into almost 900 million computational volumes. The G5NR output takes up nearly 4 petabytes of storage and was produced in just over 75 days of dedicated computation on the NASA Center for Climate Simulation (NCCS) Discover cluster. Since navigating the cubed sphere is nontrivial, the G5NR output is saved onto a 7-km (1/16° × 1/16° or 0.0625° × 0.0625°) latitude–longitude grid every 30 min. The data may be downloaded or accessed remotely via the Open-source Project for a Network Data Access Protocol (OpenDAP).
Figure 2 shows a snapshot of the very realistic simulated weather in G5NR. A comprehensive validation study by GMAO (Gelaro et al. 2014) shows that the G5NR performs well as measured by a majority of metrics considered. Particular benefits derived from the 7-km resolution of G5NR include realistic representations of extreme weather events in both the tropics and extratropics. Some deficiencies of the G5NR noted by Gelaro et al. (2014) potentially have important implications for OSSEs due to, for example, the differences in cloudiness in the G5NR and in reality.
A top-down view in the visible (i.e., as seen from space) of the G5NR at 0000 UTC 11 Sep 2006. Two major hurricanes are present. One is making landfall on the Gulf Coast and the second is south of Baja California. The labels indicate that at this time they are both category 3 hurricanes with central pressures of 938 and 941 hPa, respectively. (Graphic created by W. Putnam, NASA GMAO.)
Citation: Bulletin of the American Meteorological Society 97, 9; 10.1175/BAMS-D-15-00200.1
Observation simulation.
For observation types that already exist, observations are simulated according to the types, including both conventional and satellite, and locations and times as observed in reality. By running a real data assimilation, template files are created. Then, the NR is interpolated to the time and position of the real observations and the observed quantities are simulated. The resulting NR values, with errors added, replace the corresponding values in the template files. The following data types are identified:
conventional data, principally, surface observations including ships and buoys, radiosondes, and aircraft reports;
cloud track winds (CTWs) from geostationary imagery in low to midlatitudes and from polar-orbiting instruments [e.g., Moderate Resolution Imaging Spectroradiometer (MODIS)] in high latitudes;
Global Navigation Satellite System/Radio Occultation (GNSS/RO) profiles of temperature and humidity; and
radiances for infrared (IR) and microwave (MW) operational observing systems.
The use of PrepBUFR [i.e., preprocessed and quality controlled observations in the Binary Universal Form for the Representation of meteorological data (BUFR)] files and other templates of this sort from data assimilation cycles in real life provides an efficient shortcut, but should not be used in future OSSEs. One advantage is that this strategy skips most preprocessing and data selection steps, including gross quality control (QC), observation innovation [i.e., the observation minus background (O − B)] QC, and data thinning. This ensures realistic numbers of simulated observations and, for some observing systems, realistic locations of simulated observations. The associated problem is that this strategy incorrectly locates some types of simulated observations with respect to the geophysical phenomena such as clouds and precipitation, which are typically entirely uncorrelated between reality and the NR. In recent and current experiments this issue is mitigated since the current operational DA systems do not use observations affected by clouds and precipitation. In future OSSEs, observation locations should be determined from first principles. For example, the locations of satellite observations should be determined from the parameters describing the orbit and sensor viewing geometry. Observations should be simulated including the effects of the clouds, precipitation, etc. present in the instantaneous field of view (IFOV). Then, the QC and data thinning procedures in the OSSE would identify and reject cloud- and/or precipitation-contaminated observations.
Forecast system.
The T511 OSSE system uses the T382L64 (35-km horizontal resolution, 64 vertical layer) version of the Global Spectral Model (GSM) that was operational as of 31 May 2005. The G5NR OSSE system uses the T1534L64 (9-km horizontal resolution, 64 vertical layer) semi-Lagrangian version of the GSM that became operational on 14 January 2015. Ideally, resources permitting, the NR would have considerably greater resolution than the forecast model. The physics and dynamics of the GSM are substantially different than those of the GEOS-5 model. While it is desirable to use the operational DA system for OSSEs, the G5NR OSSE system also optionally can use the T670L64 (20-km horizontal resolution, 64 vertical layer) research DA system.
Data assimilation system.
Recent and planned global OSSEs use the operational National Centers for Environmental Prediction (NCEP) Global Data Assimilation System (GDAS; Kleist et al. 2009). Currently, the GDAS system combines the Gridpoint Statistical Interpolation analysis system (GSI) and an ensemble Kalman filter (EnKF). The GSI is a three-dimensional variational analysis system that assimilates a wide range of data types, including satellite radiances. GSI includes a variational bias correction for radiances. The GSI combines a background estimate (i.e., a 6-h forecast) of the atmospheric state and a selection of recently observed data valid within 3 h of the analysis time. The stand-alone GSI uses static (or climatological) background error statistics. In contrast, the EnKF implements the traditional Kalman filter update equation but uses an ensemble of short-term forecasts to estimate background-error covariances. In the systems described here, the size of the ensemble is 80. In the hybrid GSI–EnKF, the EnKF provides an estimate of the background error covariances that depends on the synoptic weather situation. For the GSI component, the estimates from the EnKF and from the static climatological background error covariances are combined. The GSI solution is then used to recenter the EnKF ensemble before advancing to the next analysis time. In the T511 OSSE system, the GSM/GSI/EnKF systems use a spectral resolution of T382/T382/T190, and in the G5NR OSSE system, the GSM/GSI/EnKF use a spectral resolution of T1534/T574/T574 or T670/T254/T254 in the research and development version. This notation, for example, T1534/T574/T574, means that the T1534 6-h forecasts for the GSI background are spectrally truncated at T574 and that the EnKF 6-h forecasts and analyses use T574 resolution.
FUTURE NATURE RUNS.
Future global NRs will enhance the standard set by the G5NR. As a minimum, future NRs will be based on nonhydrostatic mesoscale models resolving phenomena on the kilometer scale in the horizontal and on the subkilometer scale in the vertical. The vertical resolution should be sufficient to simulate realistic boundary layer and tropopause processes. Complete interacting physical parameterizations of clouds, hydrometeors, and radiation in the atmosphere are included in current atmospheric models. For simulating observations it is especially important to have realistic NR clouds—with realistic occurrence in space and time of the naturally occurring types—in order to have a realistic coverage and accuracy of simulated infrared and visible radiances and other data types affected by clouds. In addition to higher resolution, in future NRs, exchanges of momentum, energy, and mass with the surface will be coupled with ocean and land surface models that include realistic seasonal variations of sea ice, snow cover, and vegetation. Realistic surface conditions are required to simulate surface-affected infrared and microwave radiances with correct coverage and accuracies. Still, clouds, hydrometeors, and land surfaces have variations on the scale of meters to tens of meters that have important impacts on real observing systems. We return to this issue in the section on tuning simulated observation errors.
For the purpose of demonstrating realistic differences between the NR and the forecast model, the NR can be used to initialize an OSSE model forecast (see example in the “Predictability experiments” section and Fig. 3). For verification, simple interpolation to pressure levels and a coarse horizontal grid is adequate, but for initialization, interpolation errors and imbalances should be simultaneously minimized. One approach is to assimilate evenly distributed error-free dropwindsonde-like observations until the observation innovations (O − B) statistics asymptote. An alternative is to interpolate to the forecast model grid and apply a digital filter (Lynch and Huang 1994) to remove spurious gravity waves. In an OSSE, a similar strategy could be used to obtain initial conditions for the NR model from the forecast or data assimilation system (i.e., initial conditions corresponding to an analysis or forecast). Forecasts made by the NR model and the operational forecast model could then be compared to assess the relative importance of initial state and model errors.
Global RMSE for predictability experiment temperature forecasts (K) initialized at 0000 UTC during the period 5–20 Sep (a) in 2005 in the simulation and (b) in 2013 in reality. The RMSE is plotted vs forecast time (hours, x axis) and pressure (hPa, y axis). (Graphic created by H. Wang, NOAA/ESRL.)
Citation: Bulletin of the American Meteorological Society 97, 9; 10.1175/BAMS-D-15-00200.1
FUTURE FORECAST AND DATA ASSIMILATION SYSTEMS.
DA systems are constantly evolving. To maximize relevance, OSSEs should be conducted with up-to-date versions of operational DA systems. These systems regularly increase their resolution and complexity. OSSEs will be needed for diverse analysis and forecast systems and domains, either individually or coupled. These include global, hurricane, and severe storm atmosphere domains; global, basin, and coastal ocean domains; global, continental, regional, and urban area air quality domains; as well as for land surface, sea ice, hydrological, and ecological systems in various settings. In coupled Earth systems, there will often be large differences in time scales between the components. This may require extensive spinup periods and corresponding long coupled NRs to allow equilibration in the OSSE system.
In the future, forecast systems will increasingly include ensembles of forecasts as the impetus for providing probability information increases (Hirschberg et al. 2011). This will increase the computational demands for operational systems and therefore for OSSE systems. In addition, large samples of ensemble forecasts are needed to validate probability forecasts—much larger than for deterministic forecasts. If the forecast initial ensemble can be converted into its initial conditions for the NR model (as described in the section on future nature runs), then the true probabilistic forecast skill—that of the NR—could be compared to that of the operational forecast model.
OSSE systems borrow operational forecast and DA systems. Therefore, in order to keep pace with operations, OSSE systems have demanding requirements for computational resources. An additional requirement is to port or adapt the operational computer codes to research computational systems. In addition, this generally entails adapting the suite of operational code libraries. It is now generally recognized that in support of research to operations (R2O), a complementary operations to research (O2R) process should be put in place. At the Joint Center for Satellite Data Assimilation (JCSDA), the OSSE system directly leverages the existing robust O2R process for porting the GSM and GDAS (Kumar et al. 2015).
FUTURE OBSERVATION SIMULATION METHODS.
Future OSSEs will need to accurately simulate all observations used by future DA systems. As observations with more complex error characteristics are included in the DA, the OSSE simulation procedures, including the generation of errors, will become more complicated and require more attention to detail to assure realistic results. Ideally, the OSSE simulation of errors should include phenomena in the NR not in the DA observation operator and phenomena in nature not in the NR. For example, representativeness error includes scales not included in the DA system, including those small scales both included and not included in the NR. Note that using the same observation operator in the DA and in simulating the observations should be accompanied by simulating the additional source of errors due to the forward problem (e.g., the radiative transfer model). Using different forward models does not ensure that the forward problem errors are realistically included in the OSSE, but using the same forward problem definitely leaves out these errors.
Current operational data selection avoids observations “contaminated” by phenomena that are not accurately represented in the DA system and/or are ambiguously related to the analysis variables (i.e., to the state vector). These unused observations, such as cloud- and precipitation-affected radiances, present opportunities and challenges for future research in DA. Currently, substantial progress is being made in the assimilation of all-sky microwave radiances (Bauer et al. 2010; Geer et al. 2014; Haddad et al. 2015) and all-sky IR radiances (Okamoto et al. 2014). When the DA system assimilates both cloud-free and cloud-affected infrared radiances, the error characteristics of these two subsets will be very different and may vary with cloud type, synoptic situation, and other factors. Even if the DA system greatly simplifies how the error statistics are specified, the OSSE observation simulation should aim for great fidelity.
Some of the “contaminated” observations include signals from more than one Earth system components, for example, surface-affected microwave radiances. These observations will be accurately represented and fully utilized in future coupled DA systems. For quite some time, it has been common in NWP experiments to assimilate measured satellite radiances directly rather than applying inversion techniques to retrieve modeled geophysical parameters (Kalnay 2002). This shift in NWP from retrievals to radiances has had a significant positive impact that has been well documented (Derber and Wu 1998). However, this approach has not yet been applied to the assimilation of ocean-, land-, or ice-affected satellite observations because such an approach is only possible if the entire geophysical state necessary for reconstructing a simulated observation is available at the assimilation time. A significant opportunity that arises in the advent of strongly coupled data assimilation is the ability to form a mapping that transforms the full Earth system state into a set of synthetic observations based on the same observing principles used by real-world instruments.
A practical implementation of a coupled DA system is most direct when starting from separate EnKF systems because the EnKF needs only the observation innovations. Thus, all that is needed is an observation operator that makes use of model state information from the various component models to evaluate the coupled background in observation space. The resulting observation innovations are then provided to each EnKF and each of the analyses can proceed in parallel independently. With a coupled NR, as in a coupled observation operator within a coupled DA system, the observation simulation should include the most detailed and realistic representations across the coupled models. So, one should use the land surface specification from the land model, not the atmospheric model, if both land and atmospheric model components are present. Also, the inner-most grid covering the observation location should be employed when using coupled mesoscale and global atmospheric models.
Error simulation.
Rigorous OSSEs must simulate the various types of errors that exist in the real world. These include random errors, instrumental errors, calibration errors, representativeness errors, and systematic errors. Representativeness error is also included in DA systems, and accounts for differences between what is measured by the observations and the model simulations of these observations. For simulating observations, representativeness error should include differences due to all phenomena that affect the observation in reality and how those phenomena are represented in the NR. In other words, representativeness error comes from phenomena in the real world that are not represented correctly or not represented at all in the NR model or the observation simulation procedure. Examples of phenomena contributing to representativeness error are wind gusts and turbulence; small-scale features in clouds, topography, SST, land surfaces, and vegetation; and temporal changes in vegetation and soil moisture.
To simulate realistic errors, including realistic correlations in space, time, and with respect to various phenomena in nature, two approaches are possible and should be pursued in tandem. For examples of realistic error simulation, see Atlas et al. (1985a), Arnold and Dey (1986), Errico et al. (2013), and Atlas et al. (2015). The two approaches are designed to 1) develop more accurate and probably more complex statistical models of the errors and 2) simulate the observations with greater fidelity. The statistical approach might involve extensive studies of observation innovations (O − B) collected from an operation system and employing a variety of stratification or sub-setting strategies. Such studies could also benefit the specification of error statistics in DA systems. The high-fidelity simulation approach might make use of engineering models of the instrument and high-resolution geophysical parameters (e.g., real high-resolution cloud fields, land surfaces, etc.) from situations that are analog matches between the NR and reality. Analog matches may also be useful for obtaining realistic observing patterns for observing systems that involve complex human decision-making processes that we might not want to simulate. For example, patterns of observations from commercial aircraft depend on flight paths, which in turn depend on nowcasts and short-term forecasts of winds and turbulence at flight level.
Instrument manufacturers routinely develop extremely detailed and accurate sensor simulation models that are used for proof-of-concept and trade studies. Such models are in use both for existing and, at least past the proof-of-concept phase, for proposed instruments. These sensor simulation models could in theory be used directly in the OSSE simulations; however, they are likely proprietary and very demanding of computer resources. An alternative is to work with the instrument manufacturer to develop a database of simulated observations for a set of cases spanning all possible environmental conditions. Such a database could then be used to develop a somewhat simplified physical or neural net model of the sensor error. The detailed calculations should include everything possible that might affect the observations in reality. Geophysical parameters/processes and spatial/temporal scales not included in NR should be included here. Even though these processes and scales cannot be specified directly from the NR, these should be realistic and consistent with NR.
Tuning simulated observation errors.
Errico et al. (2013) describe an iterative method of tuning the variance of the random component of the simulated observation error for an OSSE. The process aims to match the innovation variance in reality and in the OSSE. In practice so far, the mean of the innovation has been ignored, but it is possible to also tune the constant (i.e., systematic) component of the simulated observation error by matching the mean of the innovations in reality and in the OSSE. However, care must be exercised when using this procedure to estimate a constant error component since systematic errors in the observations and in the forecast model may be commingled, with the possibility of assigning forecast model errors to an unbiased observing system. The estimates of simulated observation error statistics are implicit and therefore iterative. For purely random errors, Errico et al. (2013) report essentially immediate convergence; however, in the general case, acceleration factors may be used to speed the convergence. When converged, the innovation statistic sampled from reality is equal to the sum of the perfect innovation statistic from the simulation and the simulation error statistic, where the statistic is either the mean, variance, or covariance. Here, the perfect innovations—the difference between the “perfect” simulated observations and the background in observation space—contain errors due to the process of mapping the NR and background to the observations and due to the background errors.
Potential advances in simulating observations.
Examples of some possible improvements to current methods of simulating observations are presented here. Typically, the current method involves interpolating the NR to the observation locations from reality, running the forward model and adding errors. This approach is not satisfactory in many cases for current observing systems and, often, is impractical for new observing systems. The proposed improvements are based on more realistic simulation of the observing processes that occur in reality and the use of naturally occurring analogs.
Atmospheric motion vectors (AMVs).
As an example of more realistic simulation of the processes that create observations in reality, future simulation of AMVs (including cloud-track winds) should include the operational (or near operational) systems that generate AMVs. For this purpose realistic imagery must be simulated from the NR. The simulated imagery must be of sufficiently high spatial and temporal resolution. This is a moving target as operational AMV systems evolve. Currently, imagery with 3–5-km spatial resolution every 15 min is used.
A prototype of how simulated AMVs might be generated is described by Bormann et al. (2014). In that study geostationary Spinning Enhanced Visible and Infrared Imager (SEVIRI) imagery for Meteosat-8 was generated using the RTTOV [i.e., Radiative Transfer for the Television and Infrared Observation Satellite (TIROS) Operational Vertical Sounder (TOVS)] package every 15 min from a high-resolution Weather Research and Forecasting (WRF) Model run, and infrared and water vapor AMVs were derived from sequences of three (cloudy) images. An advantage of this approach is that realistic vector wind and height assignment errors would be generated automatically (Hernandez-Carrascal and Bormann 2014).
Natural analogs as an aid to observation simulation.
There are cases where natural phenomena that are known to affect observations are not present in the NR and cannot be realistically represented given the limitations and capabilities of current forecast models. In such cases, a possible approach to include these effects in the simulated observations is to use naturally occurring analogs to select supplemental observations of the missing phenomena that are then input into the observation simulation method. By finding analog matches in a reanalysis for the NR at the observation location and time, the required natural phenomena can be sampled from reality at the matching reanalysis location and time. Given the NR and a reanalysis, the analog-matching process requires definition of the matching metric and a database of natural phenomena. For example, real radiances depend on cloud field details, such as the cloud spatial patterns within the IFOV of the sensor. For this application, the matching metric might be the difference between the temperature and humidity profiles in the NR and in the reanalysis, and the database of high-resolution cloud patterns might be MODIS imagery. In this example, only reanalysis times and locations associated with MODIS imagery should be included in the set of possible matches.
The analog match method could also be applied to cases where the location and timing of the observation is determined in part by internal decisions in the observing system that may be difficult to simulate. For example, for observations from commercial aircraft the analog match metric would be the difference between the large-scale flight-level winds between the origin and destination of a particular flight (e.g., IAD − LHR) and the observations would be evaluated from the NR along the flight path that actually occurred on the matching day. Variations on this approach might be used for reports from ships at sea. Ships of course tend to avoid storms, but real ship tracks might directly transect a storm in the NR. In the future, OSSEs may need to simulate reports from automobiles, mobile phones, social media, and other technologies that depend on human behavior, which is influenced by the weather, day of the week, holidays, and many other factors that would need to be controlled for in applying an analog technique.
FUTURE OBSERVING SYSTEMS.
Potential new or enhanced observing systems of current interest include the following:
Geostationary Hyperspectral Sounder (Geo-HSS)—proposed IR hyperspectral sounder (HSS) sensors in geostationary Earth orbit (GEO) (e.g., Smith et al. 2004; Bingham et al. 2013);
Global Navigation Satellite System/Radio Occultation (GNSS/RO)—additional and possibly improved GNSS/RO receiving satellite constellations (e.g., Cucurull et al. 2013);
Geostationary Microwave (Geo-MW)—proposed MW instruments in GEO (e.g., Lambrigtsen 2015);
Optical Autocovariance Wind Lidar (OAWL)—a proposed Doppler wind lidar (DWL) (Schwiesow and Mayor 1995); OSSEs using both the T511 NR and the HNR1 have been conducted (Atlas et al. 2015 and references therein);
unmanned aerial systems (UASs)—OSSEs are being used to compare different sampling strategies for NOAA UASs (e.g., Privé et al. 2014; Wang et al. 2015); and
Cyclone Global Navigation Satellite System (CYGNSS)—CYGNSS is a NASA mission that will launch a constellation of microsatellites that use reflected signals from existing GNSS satellites to retrieve the ocean surface wind speed. Preliminary OSSEs using the HWRF OSSE system are reported by Annane et al. (2015).
Each type of observing system has individual characteristics that should be accounted for in an OSSE. As examples, we discuss the challenges for future OSSEs, which are also caveats for current OSSEs, to fairly assess the full potential of observations from Geo-HSS and GNSS/RO.
Geo-HSS OSSEs.
There is great potential for Geo-HSS in terms of temporal, spatial, and spectral coverage. However, much of the data that Geo-HSS would observe would not be used in current DA systems. This is true for existing instruments: channels that are impacted by clouds, precipitation, or surface conditions are not used. Few channels are used that are sensitive to temperature in the boundary layer or to humidity at any level. As an example, the number of AIRS channels observed by the Aqua satellite is 2378, but only 145 channels are actually assimilated operationally in the GDAS system. Clearly, reaching the full potential of Geo-HSS will require enhancements to the operational DA systems.
An important further potential of Geo-HSS is in the possibility to leverage its high spatial and temporal resolution to determine AMVs. AMVs are an important dataset for DA, but are mostly limited to GEO imagery, and high-latitude imagery from LEO sensors in polar orbits. Designs of proposed Geo-HSS instruments would allow the generation of huge volumes of AMVs both from tracking clouds in window channels and from tracking features in water vapor channels. Achieving the potential of these data will require advances in the production and use of AMVs. Geo-HSS instruments also have potential value to monitor surface characteristics (e.g., soil moisture) and air quality (e.g., trace species), as well as in various near-cast applications (e.g., severe weather watches and warnings). (See discussion of OSSEs for warning systems in the final section of this paper.)
GNSS/RO OSSEs.
The largest contribution to improving weather forecast skill over the last decade has come from the assimilation of microwave and infrared radiances from passive nadir sounders. However, these observations contain systematic errors (i.e., errors that occur regularly in similar situations). These “biases” can be quite significant and can mask the information content of the observations themselves. Therefore, the assimilation of radiances in operational NWP requires corrections for these systematic errors. Modern DA systems include a variational bias correction (VarBC) to identify and correct systematic radiance errors (e.g., Auligné et al. 2007; Zhu et al. 2014). Zhu et al. (2015) describe applying the variational bias correction (VarBC) method to aircraft temperatures as well, and this approach is operational now at ECMWF and is expected to soon become operational at NCEP. The VarBC procedure requires some independent unbiased observations to be assimilated in the system that can act as “anchor” points, which thus prevent a drift of the analysis and forecasts to the model climatology. Since RO observations are, for practical purposes, unbiased and occur at times and in regions not well sampled by radiosondes, these data are ideal anchor points (Cucurull et al. 2014). RO data are almost insensitive to clouds and precipitation and depend ultimately on very precise measurements of the time delay of the received GNSS signals, relative to the expected travel time in a vacuum. NOAA has been assimilating GNSS/RO observations into its operational GDAS since 1 May 2007. Since then, RO observations have shown a positive impact on global numerical weather prediction worldwide, complementing infrared and microwave observations from satellites (Cucurull 2010; Anthes 2011). RO data reduce the model drift and the resulting spurious drift in the bias corrections applied to other observations, thus improving the assimilation of other observations, including radiances (WMO 2012). Thus, the use of RO observations in operational weather forecasting is beneficial because of two different effects: first, from the direct effect of the observations by providing accurate, precise, and independent information on the thermodynamic state and, second, by improving the VarBC applied to the radiance observations.
OSSE VALIDATION AND CALIBRATION: EXAMPLE OF THE G5NR.
Before OSSEs are conducted, validation and calibration studies should be undertaken to assess the accuracy of the OSSE system. First, the NR must be examined to determine in what ways it deviates from reality. Differences from reality are not necessarily showstoppers—in fact, no NR will be identical to reality in all respects. However, these differences should be documented to ascertain the applicability of results from OSSEs using the NR. In other words, a particular NR may be valid for some OSSEs, and not for others.
The OSSE system should also be validated and calibrated if necessary. Various DA and forecast statistics in the OSSE system and reality should be similar. Ideally, the differences should not be statistically significant. To some extent, differences can be removed by calibration. Typically, calibration of the simulated observation errors may be required. When we are interested in assessing the impact of an observing system on an existing DA system, we ordinarily would not change the forecast model. However, to ensure that the forecast model and NR model are not too similar, it may be necessary to make some changes to the physical parameterizations in the forecast model or even in the model used to generate the NR. [In most of the earliest OSSEs (those conducted prior to the 1980s), the forecast and NR models were often the same and this “identical twin” setup typically led to overly optimistic OSSE forecast impacts.] The OSSE results may also be calibrated empirically using a comparison between an OSE and OSSE data-denial experiment to scale the OSSE impact for a new observing system (e.g., Hoffman et al. 1990).
Examples of OSSE validation and calibration studies for the G5NR OSSE system are summarized in the following paragraphs.
Predictability experiments.
The forecast model used to generate the NR and employed in the DA system should be realistically different. To demonstrate that this is true for the G5NR and GSM, the GSM was initialized with G5NR initial conditions (Casey et al. 2015). The forecast skill of the GSM to predict the real atmosphere and the G5NR were compared. Differences exist in the sense that GSM is better at predicting the NR than the real world, but these differences in how fast forecast errors grow are small enough to conclude that there will be no twin problem. For example, Fig. 3 shows that the growth of error in temperature for these two situations is quite similar.
Reality to simulation validation.
A very sensitive validation of the entire OSSE system is obtained by running parallel OSEs and OSSEs at the start of the NR when the NR and the real atmosphere are very similar. We call this the reality to simulation validation (RSV) method. For RSV the first OSE begins well before the start of the NR and assimilates only conventional data. At the start of the NR, the OSE continues and the OSSE begins from the OSE model state. There should be a smooth hand off from OSE to OSSE at the start of the NR, after which the OSE and OSSE results should closely parallel each other. If not, it is very likely that the conventional data are not properly simulated. Additional RSV experiments add one other observing system to the conventional data. Such experiments validate the added observing system and should demonstrate that the impact of adding the observing system is nearly the same in the OSEs and the OSSEs.
For a successful RSV, all components of the OSSE system must be calibrated and perform properly. For example, it may be necessary to calibrate observation errors statistics for the conventional observations following the procedure outlined in the “Tuning simulated observation errors” section. In the validation reported by Casey et al. (2015) a single iteration to correct the random errors used in the simulated observations was sufficient to bring RMSE values to a point where the differences between real and simulated radiosonde RMS errors were not statistically significant. Figure 4 demonstrates this for radiosonde temperatures in which the right panel shows the good agreement obtained after adding random errors to the “perfect” simulated radiosondes observations.
Global mean and RMS differences for temperature innovations (O – B, plotted in red) and analysis departures [observations minus analysis (O – A), plotted in black] for the period 0000 UTC 16–0000 UTC 22 May 2005. Solid lines are results for the OSE and dotted lines are results for the OSSE. The OSE results are the same in both panels. The OSSE results are for (a) perfect simulated observations (i.e., with no added errors) and (b) simulated observations with calibrated errors added. [Graphic created by J. Woollen, NOAA/Environmental Modeling Center (EMC).]
Citation: Bulletin of the American Meteorological Society 97, 9; 10.1175/BAMS-D-15-00200.1
LOOKING AHEAD.
The new OSSE capability using the G5NR will be applied in a number of experiments to obtain results compatible with today’s operational systems. These OSSEs will quantify the impact and optimize the use of various proposed and operational observing systems, including Geo-HSS and GNSS/RO, as well as MW sounders in GEO, UAS, CYGNSS, DWL (Baker et al. 2014), and others.
Additionally, we anticipate a number of enhancements to the current OSSE system, including
New NRs, with increased resolution to 3.5 km for selected periods in the G5NR and a new NR from ECMWF;
New DA components from the O2R process to keep pace with NCEP advances, including hybrid four-dimensional DA, which is expected to soon become operational;
Increasingly realistic simulated observations, which will introduce natural and realistic representativeness errors and lessen the current dependence on statistical error models;
New applications of OSSEs to models of other components of the Earth system that interact with the atmosphere, including severe storms, ocean–wave–sea ice, land surface, ecosystems, cryosphere, air quality, atmospheric chemistry, carbon cycle, and space weather OSSEs;
New applications of OSSEs to a wide range of observing systems applicable to these Earth system components, including measurements related to soil moisture, hydrometeors, air composition, gravity, ocean-bottom pressure, sea ice freeboard and thickness, fluxes of solar particles, and others;
Coupled Earth system components in the NR, in the DA system, and in the observation simulation; and
Applications to warning systems and other systems with humans in the loop, which may necessitate the incorporation of artificial intelligence systems as a stand-in for the human.
ACKNOWLEDGMENTS
Contributions to this work, ranging from discussions to preparation of figures, come from a number of colleagues and collaborators. The authors thank all of their colleagues who have contributed to OSSEs, especially M. Halem, E. Kalnay, E. Brin, J. Terry, J. C. Jusem, S. Boukabara, J. Woollen, M. Matsutani, S. Casey, S. Murillo, B. Annane, L. Bucci, L. Cucurull, H. Wang, T. Nehrkorn, S. M. Leidner, R. Gelaro, R. Errico, W. Putnam, and N. Privé. This research was carried out in part under the auspices of the Cooperative Institute for Marine and Atmospheric Studies (CIMAS), a Cooperative Institute of the University of Miami and the National Oceanic and Atmospheric Administration (Cooperative Agreement NA10OAR4320143), with funding from Award NA14OAR4830103 (“CIMAS Contributions to OAR Disaster Recovery Act Projects”). The authors thank the manuscript reviewers for their thorough and careful reviews and helpful suggestions and comments.
REFERENCES
Andersson, E., and M. Masutani, 2010: Collaboration on observing system simulation experiments (joint OSSE). ECMWF Newsletter, No. 123, ECMWF, Reading, United Kingdom, 14–16.
Annane, B., B. McNoldy, J. Delgado, L. Bucci, R. Atlas, and S. Majumdar, 2015: CYGNSS data and surface wind analysis. 19th Conf. on Integrated Observing and Assimilation Systems for the Atmosphere, Oceans, and Land Surface (IOAS-AOLS), Phoenix, AZ, Amer. Meteor. Soc., 238. [Available online at https://ams.confex.com/ams/95Annual/webprogram/Paper251064.html.]
Anthes, R. A., 2011: Exploring Earth’s atmosphere with radio occultation: Contributions to weather, climate and space weather. Atmos. Meas. Tech., 4, 1077–1103, doi:10.5194/amt-4-1077-2011.
Arnold, C. P., and C. H. Dey, 1986: Observing-systems simulation experiments: Past, present, and future. Bull. Amer. Meteor. Soc., 67, 687–695, doi:10.1175/1520-0477(1986)067<0687:OSSEPP>2.0.CO;2.
Atlas, R., 1997: Atmospheric observations and experiments to assess their usefulness in data assimilation. J. Meteor. Soc. Japan, 75, 111–130.
Atlas, R., 2004: Impact of SeaWinds scatterometer data on numerical weather prediction. 20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., J9.1. [Available online at https://ams.confex.com/ams/pdfpapers/69405.pdf.]
Atlas, R., and R. N. Hoffman, 2000: The use of satellite surface wind data to improve weather analysis and forecasting. Satellites, Oceanography and Society, D. Halpern, Ed., Elsevier Oceanography Series 63, Elsevier, 57–78.
Atlas, R., and L. P. Riishojgaard, 2008: Application of OSSEs to observing system design. Remote Sensing System Engineering, P. E. Ardanuy, and J. J. Puschell, Eds., International Society for Optical Engineering (SPIE Proceedings, Vol. 7087), doi:10.1117/12.795344.
Atlas, R., and T. S. Pagano, 2014: Observing system simulation experiments to assess the potential impact of proposed satellite instruments on hurricane prediction. Imaging Spectrometry XIX, P. Mouroulis, and T. S. Pagano, Eds., International Society for Optical Engineering (SPIE Proceedings, Vol. 9222), 922202, doi:10.1117/12.2063648.
Atlas, R., E. Kalnay, W. E. Baker, J. Susskind, D. Reuter, and M. Halem, 1985a: Simulation studies of the impact of future observing systems on weather prediction. Preprints, Seventh Conf. on Numerical Weather Prediction, Montreal, QC, Canada, Amer. Meteor. Soc., 145–151.
Atlas, R., E. Kalnay, and M. Halem, 1985b: Impact of satellite temperature soundings and wind data on numerical weather prediction. Opt. Eng., 24, 242341, doi:10.1117/12.7973481.
Atlas, R., R. N. Hoffman, E. Brin, S. C. Bloom, and P. M. Woiceshyn, 1996: The impact of ERS-1 scatterometer data on GEOS and NCEP model forecasts. Preprints, 11th Conf. on Numerical Weather Prediction, Norfolk, VA, Amer. Meteor. Soc., 99–101.
Atlas, R., and Coauthors, 2001: The effects of marine winds from scatterometer data on weather analysis and forecasting. Bull. Amer. Meteor. Soc., 82, 1965–1990, doi:10.1175/1520-0477(2001)082<1965:TEOMWF>2.3.CO;2.
Atlas, R., R. N. Hoffman, J. Ardizzone, S. M. Leidner, J. C. Jusem, D. K. Smith, and D. Gombos, 2011: A cross-calibrated, multiplatform ocean surface wind velocity product for meteorological and oceanographic applications. Bull. Amer. Meteor. Soc., 92, 157–174, doi:10.1175/2010BAMS2946.1.
Atlas, R., and Coauthors, 2015: Observing system simulation experiments (OSSEs) to evaluate the potential impact of an optical autocovariance wind lidar (OAWL) on numerical weather prediction. J. Atmos. Oceanic Technol., 32, 1593–1613, doi:10.1175/JTECH-D-15-0038.1.
Auligné, T., A. P. McNally, and D. P. Dee, 2007: Adaptive bias correction for satellite data in a numerical weather prediction system. Quart. J. Roy. Meteor. Soc., 133, 631–642, doi:10.1002/qj.56.
Baker, W. E., and Coauthors, 2014: Lidar-measured wind profiles: The missing link in the global observing system. Bull. Amer. Meteor. Soc., 95, 543–564, doi:10.1175/BAMS-D-12-00164.1.
Bauer, P., A. J. Geer, P. Lopez, and D. Salmond, 2010: Direct 4D-Var assimilation of all-sky radiances. Part I: Implementation. Quart. J. Roy. Meteor. Soc., 136, 1868–1885, doi:10.1002/qj.659.
Bingham, G., and Coauthors, 2013: Rapidly updated hyperspectral sounding and imaging data for severe storm prediction. Infrared Remote Sensing and Instrumentation XXI, M. Strojnik Scholl and G. Páez, Eds., International Society for Optical Engineering (SPIE Proceedings, Vol. 8867), 886702, doi:10.1117/12.2030687.
Bormann, N., A. Hernandez-Carrascal, R. Borde, H.-J. Lutz, J. A. Otkin, and S. Wanzong, 2014: Atmospheric motion vectors from model simulations. Part I: Methods and characterization as single-level estimates of wind. J. Appl. Meteor. Climatol., 53, 47–64, doi:10.1175/JAMC-D-12-0336.1.
Casey, S. P. F., H. Wang, R. Atlas, R. N. Hoffman, S.-A. Boukabara, Y. Xie, Z. Toth, and J. S. Woollen, 2015: Initial validation of a new OSSE capability. 19th Conf. on Integrated Observing and Assimilation Systems for the Atmosphere, Oceans, and Land Surface (IOAS-AOLS), Phoenix, AZ, Amer. Meteor. Soc., 3.2. [Available online at https://ams.confex.com/ams/95Annual/webprogram/Paper267725.html.]
Chahine, M. T., and Coauthors, 2006: AIRS: Improving weather forecasting and providing new data on greenhouse gases. Bull. Amer. Meteor. Soc., 87, 911–926, doi:10.1175/BAMS-87-7-911.
Cucurull, L., 2010: Improvement in the use of an operational constellation of GPS radio occultation receivers in weather forecasting. Wea. Forecasting, 25, 749–767, doi:10.1175/2009WAF2222302.1.
Cucurull, L., J. C. Derber, and R. J. Purser, 2013: A bending angle forward operator for global positioning system radio occultation measurements. J. Geophys. Res. Atmos., 118, 14–28, doi:10.1029/2012JD017782.
Cucurull, L., R. A. Anthes, and L.-L. Tsao, 2014: Radio occultation observations as anchor observations in numerical weather prediction models and associated reduction of bias corrections in microwave and infrared satellite observations. J. Atmos. Oceanic Technol., 31, 20–32, doi:10.1175/JTECH-D-13-00059.1.
Derber, J. C., and W.-S. Wu, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126, 2287–2299, doi:10.1175/1520-0493(1998)126<2287:TUOTCC>2.0.CO;2.
Errico, R. M., R. Yang, N. C. Privé, K.-S. Tai, R. Todling, M. E. Sienkiewicz, and J. Guo, 2013: Development and validation of observing-system simulation experiments at NASA’s Global Modeling and Assimilation Office. Quart. J. Roy. Meteor. Soc., 139, 1162–1178, doi:10.1002/qj.2027.
Geer, A., F. Baordo, N. Bormann, and S. English, 2014: All-sky assimilation of microwave humidity sounders. ECMWF Newsletter, No. 139, ECMWF, Reading, United Kingdom,25–32.
Gelaro, R., and Coauthors, 2014: Evaluation of the 7-km GEOS-5 nature run. NASA Technical Report Series on Global Modeling and Data Assimilation, R. D. Koster, Ed., TM-2014-104606, Vol. 36, NASA Goddard Space Flight Center, 305 pp. [Available online at gmao.gsfc.nasa.gov/pubs/docs/Gelaro736.pdf.]
Haddad, Z. S., J. L. Steward, H.-C. Tseng, T. Vukićević, S.-H. Chen, and S. Hristova-Veleva, 2015: A data assimilation technique to account for the nonlinear dependence of scattering microwave observations of precipitation. J. Geophys. Res. Atmos., 120, 5548–5563, doi:10.1002/2015JD023107.
Halliwell, G. R., Jr., A. Srinivasan, V. Kourafalou, H. Yang, D. Willey, M. Le Hénaff, and R. Atlas, 2014: Rigorous evaluation of a fraternal twin ocean OSSE system for the open Gulf of Mexico. J. Atmos. Oceanic Technol., 31, 105–130, doi:10.1175/JTECH-D-13-00011.1.
Halliwell, G. R., V. Kourafalou, M. Le Hénaff, L. K. Shay, and R. Atlas, 2015: OSSE impact analysis of airborne ocean surveys for improving upper-ocean dynamical and thermodynamical forecasts in the Gulf of Mexico. Prog. Oceanogr., 130, 32–46, doi:10.1016/j.pocean.2014.09.004.
Hernandez-Carrascal, A., and N. Bormann, 2014: Atmospheric motion vectors from model simulations. Part II: Interpretation as spatial and vertical averages of wind and role of clouds. J. Appl. Meteor. Climatol., 53, 65–82, doi:10.1175/JAMC-D-12-0337.1.
Hirschberg, P. A., and Coauthors, 2011: A weather and climate enterprise strategic implementation plan for generating and communicating forecast uncertainty information. Bull. Amer. Meteor. Soc., 92, 1651–1666, doi:10.1175/BAMS-D-11-00073.1.
Hoffman, R. N., C. Grassotti, R. G. Isaacs, J.-F. Louis, and T. Nehrkorn, 1990: Assessment of the impact of simulated satellite lidar wind and retrieved 183-GHz water vapor observations on a global data assimilation system. Mon. Wea. Rev., 118, 2513–2542, doi:10.1175/1520-0493(1990)118<2513:AOTIOS>2.0.CO;2.
Kalnay, E., 2002: Atmospheric Modeling, Data Assimilation, and Predictability. Cambridge University Press, 364 pp.
Kleist, D. T., D. F. Parrish, J. C. Derber, R. Treadon, W.-S. Wu, and S. Lord, 2009: Introduction of the GSI into the NCEP Global Data Assimilation System. Wea. Forecasting, 24, 1691–1705, doi:10.1175/2009WAF2222201.1.
Kumar, K., J. A. Jung, S. A. Boukabara, S. Nolin, and J. Stroik, 2015: Overview of O2R and R2D activities at JCSDA and NESDIS. S4 and JIBB upgrade status. 20th Conf. on Satellite Meteorology and Oceanography, Phoenix, AZ, Amer. Meteor. Soc., 176. [Available online athttps://ams.confex.com/ams/95Annual/webprogram/Paper268215.html.]
Lambrigtsen, B. H., 2015: Progress in developing a geostationary microwave sounder. 11th Annual Symp. on New Generation Operational Environmental Satellite Systems, Phoenix, AZ, Amer. Meteor. Soc., J19.3. [Available online at https://ams.confex.com/ams/95Annual/webprogram/Paper267078.html.]
Laprise, R., 1992: The resolution of global spectral models. Bull. Amer. Meteor. Soc., 73, 1453–1454.
Lynch, P., and X.-Y. Huang, 1994: Diabatic initialization using recursive filters. Tellus, 46A, 583–597, doi:10.1034/j.1600-0870.1994.t01-4-00003.x.
Ma, Z., L. P. Riishojgaard, M. Masutani, J. S. Woollen, and G. D. Emmitt, 2015: Impact of different satellite wind lidar telescope configurations on NCEP GFS forecast skill in observing system simulation experiments. J. Atmos. Oceanic Technol., 32, 478–495, doi:10.1175/JTECH-D-14-00057.1.
Okamoto, K., A. P. McNally, and W. Bell, 2014: Progress towards the assimilation of all-sky infrared radiances: An evaluation of cloud effects. Quart. J. Roy. Meteor. Soc., 140, 1603–1614, doi:10.1002/qj.2242.
Privé, N. C., Y. Xie, S. Koch, R. Atlas, S. J. Majumdar, and R. Hoffman, 2014: An observation system simulation experiment for the unmanned aircraft system data impact on tropical cyclone track forecasts. Mon. Wea. Rev., 142, 4357–4363, doi:10.1175/MWR-D-14-00197.1.
Putman, W. M., A. Darmenov, A. da Silva, R. Gelaro, A. Molod, L. Ott, and M. J. Suarez, 2015: A 7-km non-hydrostatic global mesoscale simulation for OSSEs with the Goddard Earth Observing System model (GEOS-5). 19th Conf. on Integrated Observing and Assimilation Systems for the Atmosphere, Oceans, and Land Surface (IOAS-AOLS), Phoenix, AZ, Amer. Meteor. Soc., 3.1. [Available online at https://ams.confex.com/ams/95Annual/webprogram/Paper260701.html.]
Riishojgaard, L. P., Z. Ma, M. Masutani, J. S. Woollen, G. D. Emmitt, S. A. Wood, and S. Greco, 2012: Observation system simulation experiments for a global wind observing sounder. Geophys. Res. Lett., 39, L17805, doi:10.1029/2012GL051814.
Rohaly, G. D., and T. N. Krishnamurti, 1993: An observing system simulation experiment for the Laser Atmospheric Wind Sounder (LAWS). J. Appl. Meteor., 32, 1453–1471, doi:10.1175/1520-0450(1993)032<1453:AOSSEF>2.0.CO;2.
Schwiesow, R. L., and S. D. Mayor, 1995: Coherent optical signal processing for a Doppler lidar using a Michelson interferometer, coherent laser radar. Opt. Soc. Amer., Tech. Dig. Ser., 19, 212–215.
Smith, W. L., H. E. Revercomb, G. Bingham, H. L. Huang, D. K. Zhou, C. S. Velden, J. B. Miller, and D. Emmitt, 2004: GIFTS—Hyperspectral imaging and sounding from geostationary orbit. 20th Int. Conf. on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, Seattle, WA, Amer. Meteor. Soc., 14.6. [Available online at https://ams.confex.com/ams/pdfpapers/71904.pdf.]
Wang, H., Y. Xie, R. N. Hoffman, R. Atlas, and Z. Toth, 2015: Ensemble transform sensitivity and tropical cyclone targeted observation: An OSSE case study. 19th Conf. on Integrated Observing and Assimilation Systems for the Atmosphere, Oceans, and Land Surface (IOAS-AOLS), Phoenix, AZ, Amer. Meteor. Soc., 236. [Available online at https://ams.confex.com/ams/95Annual/webprogram/Paper263675.html.]
WMO, 2012: Fifth WMO workshop on the impact of various observing systems on numerical weather prediction. WMO Final Rep. [Available online at www.wmo.int/pages/prog/www/OSY/Reports/NWP-5_Sedona2012.html.]
Zhang, L., and Z. Pu, 2010: An observing system simulation experiment (OSSE) to assess the impact of Doppler wind lidar (DWL) measurements on the numerical simulation of a tropical cyclone. Adv. Meteor., 2010, 743863, doi:10.1155/2010/743863.
Zhu, Y., J. Derber, A. Collard, D. Dee, R. Treadon, G. Gayno, and J. A. Jung, 2014: Enhanced radiance bias correction in the National Centers for Environmental Prediction’s Gridpoint Statistical Interpolation data assimilation system. Quart. J. Roy. Meteor. Soc., 140, 1479–1492, doi:10.1002/qj.2233.
Zhu, Y., J. C. Derber, R. J. Purser, B. A. Ballish, and J. Whiting, 2015: Variational correction of aircraft temperature bias in the NCEP’s GSI analysis system. Mon. Wea. Rev., 143, 3774–3803, doi:10.1175/MWR-D-14-00235.1.
