An infrastructure has been established to engage the broader research community that is working on improving satellite data utilization in NOAA numerical modeling systems.
Arobust research-to-operations (R2O) environment, by which research and development results are transitioned to operational environments, is recognized to be an important component of the long-term sustainability, enhancement, relevance, and competitiveness of National Oceanic and Atmospheric Administration (NOAA)’s operational numerical modeling systems. Until recently, little attention was given to the critical mirror image of the R2O, the operations-to-research (O2R) environment, which is a process by which an operational environment is made available to scientists to undertake their research. Indeed, for efficient R2O transitions, innovative research efforts should be encouraged to be performed and implemented in an environment that is closely synchronized with the target operational system in order to maximize the chances for smooth transitions back to the targeted operational systems. Past experiences (Robinson 2012; Merceret et al. 2013) have shown that research efforts not performed within an appropriate O2R environment are, in the best case scenario, at higher risk of duplicating integration and testing in the operational environment before implementation or, in the worst and unfortunately frequent scenario, are never considered for operational implementation, resulting in wasted resources and lost opportunities.
Many independent reports and studies (Merceret et al. 2013; Bernardet et al. 2015) pointed out this O2R shortcoming in the overall R2O approach in the research community at large. To begin addressing this deficiency, a multiagency initiative between the National Aeronautics and Space Administration (NASA) and NOAA, led by the U.S. Joint Center for Satellite Data Assimilation (JCSDA), implemented an O2R test bed for the improvement of global and regional data assimilation and forecast systems. This initiative, called JCSDA in a Big Box (JIBB), focused on transitioning those JCSDA projects performed by the external research community to operations. Those projects, such as the global observing system simulation experiments (OSSEs) and data denial experiments, were considered to have the highest R2O success potential. The success of this initiative inspired a NOAA-specific initiative, the Supercomputer for Satellite Simulations and Data Assimilation Studies (S4), which is the subject of this overview. The S4 initiative, designed to have a wider outreach to the research community than that of JIBB, reaches out to both mature and higher risk, higher payoff types of projects, as long as they are relevant to the NOAA mission, regardless of funding sources.
The goal of this article is to highlight the lessons learned from this experience and to introduce, describe, and highlight the benefits of adding the O2R component to any R2O initiative. It also serves to highlight some of the achievements made on S4 and especially those aimed at maximizing the use of the major satellite programs, such as the Geostationary Operational Environmental Satellites series R (GOES-R) and Joint Polar Satellite System (JPSS). As part of this discussion, we attempt to quantitatively assess the R2O maturity index of the projects running on S4 by estimating the technical readiness level (TRL) of each project before and after using the S4.
The sections titled “The O2R concept” and “Description of the S4 supercomputer” briefly describe the O2R concept and the S4 supercomputer’s technical characteristics. The section titled “Description of the O2R/R2O environment” presents the O2R environment in more detail and highlights NOAA’s numerical prediction models targeted in the S4 system. The section titled “Areas of applications and highlights of major achievements” provides examples of current projects running on the S4, identifying outcomes and potential value added to the NOAA modeling enterprise. The sections titled “Summary” and “Discussion” summarize the findings and present some of the learned lessons and recommendations. A list of acronyms used in this article is given in the appendix, in addition to introducing the acronyms in the text itself the first time they are mentioned.
THE O2R CONCEPT.
The O2R represents a bridge that links the research to operations. It allows linking of the research from the onset to the targeted operational systems by making these systems available to researchers. After many iterations over the last several years, it became clear that the O2R environment, in order to be effective to researchers and to achieve the maximum R2O potential, should consist of the following components. All of them are important for a smooth execution of the project and, ultimately, successful R2O transitions: 1) a high-performing supercomputer, accessible to researchers, capable of executing current operational models for the operational domains at operational spatial and temporal resolutions; 2) target operational numerical modeling and data assimilation systems ported to the O2R environment and synchronized to operational upgrades (even if with a delay) as well as postprocessing tools for analyzing the results; 3) access to observational datasets, including both satellite data and conventional radiosonde, aircraft, and ship data (when no longer restricted); 4) a support team for guiding researchers in navigating the intricacies of the O2R environment and facilitating experiments for those researchers not accustomed to the complicated and sometimes intimidating components of the operational systems; and 5) a rigorous software configuration management system and a testing methodology for the needed traceability of changes for R2O implementations. The O2R offers a bridge that links the research being undertaken on it to the operational environment. Note that the S4 high-performance computer is a unique collaboration between NOAA and an academic partner. This specificity distinguishes it from other NOAA or non-NOAA sponsored high-performance platforms such as NCAR’s Yellowstone, NOAA’s research and development machine (Theia), and so on. The key features of S4 include its O2R component that accompanies account access, the funneling of the projects toward NOAA’s systems making it a funnel for research to benefit the NOAA mission, and its policy toward accessibility for the research community. It should be understood as a way to allow research to be undertaken, with some constraints admittedly under the S4 O2R environment, which will facilitate a successful R2O transition down the road.
DESCRIPTION OF THE S4 SUPERCOMPUTER.
The S4 supercomputer, operating since June 2011, was funded by NOAA and administered by the Space Science and Engineering Center (SSEC) at the University of Wisconsin–Madison (UW) to provide scientists (without access to the operational system) with an operationally relevant environment. Originally the S4 system consisted of 3,072 Advanced Micro Devices Inc. (AMD) 2.2-GHz CPU cores in 64 compute nodes (48 cores per node), equaling the computing power of 27 TFLOPs (a peak theoretical performance number), with a disk storage capacity of 520 TB. A major expansion to the S4 system under a NOAA grant was completed in late July 2014. The S4 expansion provided additional resources including 1,600 processing cores that increased the computational performance by approximately 3 times over the previous S4 for the actual numerical weather prediction model sustained performance and added another 1,200 TB of disk storage. The S4 O2R environment altogether provides nearly 4,700 computing cores (∼60 TFLOPs) and about 1,700-TB disk storage capacity. The upgraded S4 supercomputer is capable of conducting four simultaneous experiments of the high-resolution Global Data Assimilation System (GDAS)/Global Forecast System (GFS) T1534 (a semi-Lagrangian model at triangular truncation 1,534, approximately 13-km resolution) or six GFS experiments at the lower T670 resolution (25 km). The original S4 and expansion hardware operate as a single system, sharing a common gateway that connects all compute nodes and disks together.
DESCRIPTION OF THE O2R/R2O ENVIRONMENT.
A successful R2O process, a key element for NOAA to accelerate and improve the use of research and operational satellite data in weather, ocean, climate, and environmental analysis and prediction systems, requires a research-friendly environment that is functionally equivalent to the operational environment (i.e., an O2R system). The S4 supercomputer described in the previous section is a major component the O2R/R2O environment but is only one component. We mentioned earlier that other aspects have to be present for the O2R and R2O to be effective. In this section, we describe in detail the specific systems and environmental prediction models provided by the S4 environment. The O2R and R2O components on S4 are designed to facilitate accelerated research on satellite data assimilation (DA) applications and for integrating mature science into NOAA’s operational assimilation and forecast modeling suite. The base paradigm for the S4 O2R environment requires making available NOAA’s operational system configurations to researchers to undertake their science and therefore make the subsequent R2O transition much more likely (and easier). The O2R environment’s software management system is routinely synchronized to the operational versions of the various DA and modeling packages, including those versions that are targeted for future operational implementation and still under development at NOAA. Implementation and benchmarking were recently completed on the S4 for the operational T1534 GDAS/GFS (Kanamitsu 1989) that includes the Gridpoint Statistical Interpolation analysis system (GSI; Derber et al. 1991) analysis with the 80-member ensemble Kalman filter (EnKF; Wang et al. 2013) at T574 (27 km) resolution. In addition to the GDAS, the S4 infrastructure also serves as the O2R environment for research activities using the Hurricane Weather Research and Forecasting (HWRF; Tallapragada et al. 2013; Atlas et al. 2015c) Model, the regional North American Mesoscale Forecast System (NAM; Janjic 2003) model with the NAM Data Assimilation System (NDAS; Kleist et al. 2009), the Hybrid Coordinate Ocean Model (HYCOM; Bleck 2002), and the Land Information System (LIS; Kumar et al. 2008) model. It is important to mention that not all environment modeling systems are under strict software management on S4. Typically the atmospheric systems have fully been incorporated in the O2R environment that reflects it, but ocean and land systems are not yet implemented.
For global data assimilation purposes, all conventional and satellite observational data of GDAS (four cycles 0000, 0600, 1200, and 1800 UTC) and GFS (0000 UTC) are transferred to S4 from NOAA’s operational outputs. In NCEP operations, certain datasets, such as aircraft and ship data, are restricted and therefore cannot be used. On S4, numerical experiments are performed in retrospective mode; therefore, the users do not have to deal with restricted data in real time. We have made these nonrestricted datasets available to S4 users. However, there is no radar data on S4 owing to large data volumes.
A number of research tools have been developed to support users on S4, including the interactive Community Binary Universal Form for the Representation of Meteorological Data (BUFR)-Formatting Tool (CMFT) for converting satellite data into operational BUFR format, the Community Observation Assessment Tool (COAT) for analyzing observation error in the input satellite data, the Community Satellite Data Thinning and Representation Optimization Tool (CSTROT; Zhu and Boukabara 2015) for optimally thinning the data based on signal variability, and an Independent Assessment Tool (IAT) package to objectively assess both analysis quality and forecast impacts. Once projects on S4 reach maturity, these tools support the transition to operations, following R2O protocols inspired by close coordination with National Weather Service (NWS) scientists. These protocols require, for example, generating a comprehensive performance assessment report, presenting it at technical meetings in NOAA, and performing code reviews and regression testing. The performance assessment report should contain the main improvements that the projects are providing. Which metrics (improvements) are assessed typically vary and depend on the nature of the research. For global data assimilation projects, for instance, anomaly correlation improvement is closely monitored, as is the fit to independent reference observations.
To perform an R2O transition, either new datasets or capabilities are implemented and then tested by researchers on a branch of the main trunk containing the software that is scheduled for transition to operations. While the authority to “accept” changes from the branch to the trunk remains a decision of the operational partner, the chances of the changes to be implemented in operations increase when the work done in the branch is closely in sync with the operational version. This synchronization happens on a routine basis (roughly every two weeks) for the case of the NOAA/National Environmental Satellite, Data, and Information Service (NESDIS) for the GSI and GFS systems, reducing the risk of difficult to fix conflicts in the codes between the trunk and the branch. Once the trunk version has been updated with the changes needed to duplicate the research results, a thorough testing is done by the operational partner before adoption and inclusion in the parallel testing.
AREAS OF APPLICATIONS AND HIGHLIGHTS OF MAJOR ACHIEVEMENTS.
A wide variety of research activities have been conducted on, and benefited from, S4 since the establishment of the O2R environment. There are currently around 50 projects being conducted on S4. These activities include global and regional atmospheric data assimilation, ocean and land data assimilation, global and regional OSSE studies, new sensor readiness, calibration/validation activities, radiative transfer modeling, and climate applications. Table 1 lists some of the activities that are being conducted on S4, along with the leads, the project title, and major outcomes as well as the target model and operational partners. It also lists the R2O maturity indexes of the projects (current index and the one prior to the use of S4). The description of these indexes, in terms of TRL, is included in Table 2. These indexes and the differences between current and pre-S4 levels allow us to assess the added value (and maturity) of the projects since they started being performed on S4.
Summary of major projects conducted on S4.
NOAA technical readiness level.
The following provides technical details and highlights the achievements of the projects that are directly related to satellite data assimilation. It is important to note that not all projects are reported here because of page limitations. The S4 scope is wider in nature; it also includes activities such as radiative transfer modeling and calibration/validation (as listed in Table 1), which are typically funded by proving ground and risk reduction programs. In fact, this kind of research can help to improve radiative transfer modeling and therefore benefit the satellite data assimilation as well.
Global atmospheric data assimilation.
The GSI analysis system is a three-dimensional variational data assimilation (3D-Var) system for both global and regional applications used in NOAA operational systems (Kleist et al. 2009). As shown in Table 1, a number of projects aim at improving or expanding the applications of the GSI system. For example, a new GSI quality control (QC) scheme for the assimilation of the Moderate Resolution Imaging Spectroradiometer (MODIS) atmospheric motion vectors (AMVs) has been developed at the University of Wisconsin Cooperative Institute for Meteorological Satellite Studies (CIMSS). The current AMV QC scheme (Zapotocny et al. 2007) has been used in the GSI system since October 2005. The old QC technique discards winds if an AMV zonal or meridional component deviates by more than a threshold (7 m s−1 for normal cases or 6 m s−1 for slower wind cases) from the background state. This fixed threshold is too constraining for fast AMVs (e.g., jet level). The new QC scheme discards observations based on the ratio of the AMV vector departure from the background to the logarithm of the AMV speed and is called the log normalized vector departure (LNVD; Santek et al. 2014). Two 2-month seasonal experiments were run with the MODIS winds using LNVD QC. The experiments used the version r29119 of the hybrid GDAS on S4 and verified against the 0000 UTC forecasts. Generally, the impact of the LNVD was neutral, as compared to the old QC, although slight improvements were noted. For example, the 500-hPa anomaly correlation coefficient (ACC) of the Southern Hemisphere heights (Fig. 1) depicts a statistically significant difference between the LNVD and the control for day 4 and day 5 of the first season. The LNVD GSI code changes have been checked into the National Centers for Environmental Prediction (NCEP) software repository. This is a concrete and tangible example of a successful R2O that was made trivial thanks to the O2R environment provided on S4. The evaluation of LNVD was applied to the MODIS and Advanced Very High Resolution Radiometer (AVHRR) satellite-derived polar winds. Since the derivation of AMVs from the next-generation JPSS Visible Infrared Imaging Radiometer Suite (VIIRS) instrument will use an updated winds algorithm, it is expected that the LNVD with its current threshold will provide a good starting point to evaluate the forecast impact of the VIIRS AMVs in addition to MODIS and AVHRR AMVs, and additional evaluation and experiments will likely be needed in the S4 environment. This will contribute to our readiness for the next-generation, NOAA, polar-orbiting, satellite JPSS program, scheduled in 2017.
(top) Southern Hemisphere 500-hPa height ACC for 10 Sep–24 Oct 2012 for the control (black) and LNVD (red). (bottom) Variation between LNVD (red) and the control (operational QC of MODIS winds: black) based on forecast hour. All ACC differences outside of the bars are significant at the 95% confidence level. The circle represents a statistically significant improvement for the day-4 and day-5 forecasts.
Citation: Bulletin of the American Meteorological Society 97, 12; 10.1175/BAMS-D-14-00188.1
Despite the high average forecast skill, the reliability of current numerical weather prediction forecasts is somewhat tainted by occasional abrupt drops in the forecast skills. Recent studies have shown that these “forecast skill dropouts” are caused not by the model’s deficiencies but by the use of strongly detrimental observations that the operational QC system failed to filter out (Kumar et al. 2009). A new QC scheme, so-called proactive quality control (PQC), was proposed by scientists at University of Maryland to minimize the occurrences of NOAA GFS forecast skill dropout cases by filtering out detrimental observations. A diagnostic technique, called ensemble forecast sensitivity to observations (EFSO; Liu and Kalnay 2008; Li et al. 2010; Kalnay et al. 2012; Ota et al. 2013), was recently developed to detect very detrimental observations with a 6-h lead time of the analysis at only minimal additional cost to regular DA cycles. A 1-month DA experiment was performed with the NCEP operational EnKF/3D-Var hybrid GSI/GFS system at a lower resolution, and 20 cases of regional “dropouts” were identified. The exclusion of the detrimental observations identified by PQC resulted in improved 24-h forecasts for 18 out of 20 cases, and the improvements continued in 5-day forecasts, especially in the 11 significant cases. Figure 2 shows the average percentage improvement over 5-day forecasts of the 20 cases, grouped into 11 “significant” and 9 “nonsignificant” cases, depending on whether the EFSO-estimated, 6-h forecast relative error improvement in the target region was greater or smaller than 20%. The improvement for the Northern Hemisphere (NH) and Southern Hemisphere (SH) extratropics and the global average due to the withdrawal of the observations found by EFSO to be detrimental after 6 h increased throughout the 5-day forecast for the significant cases and up to 3.5 days in the nonsignificant cases. It is interesting that although in the tropics PQC initially introduced apparent errors due to the analysis change, this was greatly reduced within 6 h, and the tropical forecasts improved with time owing to the denial of detrimental observations in NH or SH extratropics. A test is ongoing to find out whether it is possible to avoid doing a new analysis by using a linear correction similar to EFSO (Hotta 2014) and whether the system could work in a cycled environment rather than in the offline approach that was used so far. If successful, this would allow the operational implementation of PQC (R2O transition). In addition to its direct benefit of improving NWP forecast skills and reliability, PQC can be used to build a database of failed observations by collecting their occurrences along with relevant metadata, which can help developers to improve their algorithms. PQC can also be used to accelerate the development of a DA system for new observing systems by enabling the estimation of each observation’s impact on the short-term forecast and then determining the optimal data selection.
Relative forecast improvements measured with moist total energy norm averaged over (a) the 11 significant cases for which the EFSO-estimated 6-h forecast improvement for the target domain exceeded 20% and (b) the 9 nonsignificant cases. The relative forecast improvements are computed for each of the NH extratropics (30°–90°N), the SH extratropics (90°–30°S), the tropics (30°S–30°N), and the whole globe.
Citation: Bulletin of the American Meteorological Society 97, 12; 10.1175/BAMS-D-14-00188.1
Regional data assimilation.
Several projects address regional DA for improving or investigating the benefit of high-resolution satellite data on the forecasts of mesoscale weather systems. Scientists from CIMSS, for instance, recently developed a near-real-time (NRT) regional Satellite Data Assimilation System for Tropical Storm Forecasts (SDAT) on the S4. The core parts of the SDAT system are the GSI and the Advanced Research Weather Research and Forecasting (WRF) Model (ARW). The objective of SDAT is to serve as a research test bed and application demonstration on the new capability of JPSS and GOES-R observations for improving high-impact weather forecasts. For example, a new cloud detection method that uses the spatially and temporally collocated high spatial resolution imager data (i.e., MODIS) for the hyperspectral IR sounder [i.e., Atmospheric Infrared Sounder (AIRS)] subpixel cloud characterization (Li et al. 2004) has been evaluated and demonstrated with SDAT. Compared to the GSI stand-alone cloud detection scheme, this cloud detection method can reduce the risk of assimilating the observed IR sounder cloudy radiances as clear ones. The forecast experiments on Hurricane Sandy (2012) indicated that both hurricane track and intensity were improved when the collocated, high spatial resolution MODIS cloud mask was used for the assimilation of AIRS radiances (Wang et al. 2014). This cloud detection method can also be applied to the Cross-Track Infrared Sounder (CrIS) or VIIRS data on board the Suomi National Polar-Orbiting Partnership (Suomi NPP) and JPSS satellites. Figure 3 shows the clear-sky data locations for CrIS channel 96 (709.37 cm−1) with the stand-alone cloud detection scheme from GSI and VIIRS cloud detection technique, respectively, overlaying on the GOES-13 IR (10.7 µm) brightness temperature image for the Hurricane Sandy (2012) case; the track errors are reduced with VIIRS cloud detection for the assimilation of CrIS radiances (Li et al. 2016). Based on the IR sounder/imager collocation and IR sounder subpixel cloud detection, the imager-based, cloud-clearing method (Li et al. 2005) is further applied to get the clear, equivalent advanced IR sounder radiances under partially cloudy regions. The assimilation of cloud-cleared advanced IR sounder radiances is an alternative way to get the thermodynamic information under partially cloudy regions. The cloud-cleared AIRS radiances’ assimilation using SDAT as a research test bed showed that the hurricane track error was reduced and the environmental atmospheric fields were improved (Wang et al. 2015). These new techniques evaluated and demonstrated by SDAT have the potential for improving the operational hyperspectral IR radiance assimilation in NWP models (i.e., HWRF/GSI). The IR sounder subpixel cloud characterization is now being integrated for imager/sounder real-time processing by the NOAA/NESDIS algorithm integration team, so that the GSI users can use it as an additional QC for hyperspectral IR sounder radiance assimilation.
The clear-sky data locations for CrIS channel 96 (709.37 cm−1) at 0600 UTC 25 Oct 2012 with stand-alone cloud detection scheme from (top) GSI and (bottom) VIIRS cloud detection technique, respectively, overlaying on the GOES-13 IR (10.7 μm) brightness temperature (K) image for Hurricane Sandy (2012) case.
Citation: Bulletin of the American Meteorological Society 97, 12; 10.1175/BAMS-D-14-00188.1
A second regional DA project on S4 aims to study the assimilation of all-sky infrared satellite radiances for the preparation of the upcoming GOES-R Advanced Baseline Imager (ABI) by Colorado State University (CSU) Cooperative Institute for Research in the Atmosphere (CIRA). An assimilation/forecasting system was developed with the following components: 1) the NOAA HWRF model, 2) the forward components of the GSI and the Community Radiative Transfer Model (CRTM; Han et al. 2006), and 3) the maximum likelihood ensemble filter (MLEF; Zupanski 2005; Zupanski et al. 2008) ensemble data assimilation system. Previous studies demonstrated that the assimilation of all-sky microwave satellite radiances has a positive impact (Zhang et al. 2013). In this study, the assimilation setup includes 32 ensemble members plus a control, and there is a 1-h assimilation window. The assimilation period covers six data assimilation cycles for Hurricane Fred from 0000 to 0600 UTC 9 September 2009. Meteosat Second-Generation (MSG) Spinning Enhanced Visible and Infrared Imager (SEVIRI) Channel 9 (Ch-9; 10.8-µm wavelength) was assimilated. Two basic experiments were conducted: 1) a control (no data assimilation) and 2) a data assimilation (with all-sky IR radiance assimilated) experiment. Figure 4 shows the total cloud condensate for the 21-h forecast after data assimilation. While both the control and experimental forecasts displace the center of the storm, the assimilation of the MSG SEVIRI all-sky IR radiances indicates improvement of near–inner core tropical cyclone cloud prediction, which implies potential benefit for tropical cyclone intensity prediction. This is seen in the separation of cloud bands in the experimental forecast (Fig. 4, center), which also suggests a potential improvement in the forecasting of hurricane rainbands. Work is continuing to confirm the statistical robustness.
The 21-h forecast of total cloud condensate (CWM, g kg−1) for Hurricane Fred (2009): (left) CWM forecast without assimilation, (middle) CWM forecast after assimilation, and (right) the verifying SEVIRI radiance observations.
Citation: Bulletin of the American Meteorological Society 97, 12; 10.1175/BAMS-D-14-00188.1
A third regional DA project, led by University of Utah scientists, involves the application of the NCEP operational GSI data assimilation system in the study of the processes that control tropical cyclone genesis and rapid intensification. The motivation of this study is to understand the processes of tropical cyclone genesis and rapid intensification with state-of-the-art numerical modeling and data assimilation systems and explore potential ways to improve hurricane forecasts. Specifically, the NASA AIRS-derived temperature profiles were assimilated by the GSI system into the ARW model for numerical simulations of Typhoon Nuri (2008). The impacts of model top pressure on satellite data assimilation and subsequent forecasts of the genesis and rapid intensification of Typhoon Nuri were examined. Results indicated that assimilation of AIRS-derived temperature profiles resulted in more accurate prediction of Nuri in terms of its genesis, rapid intensification, and track. Increasing the altitude of the WRF Model top (from 50 to 15 hPa) allowed more data to be assimilated into the model and thus had a significant positive impact on the forecasts (Fig. 5). Furthermore, it was found that the upper-level warming contributed to Nuri’s genesis and rapid intensification. Assimilation of AIRS data improved the representation of upper-level warming and circulation, notably in the regions above 400 hPa, including the outflow layer. The results also suggest that it is necessary to use the higher model top in operational regional models in order to maximize satellite data usage and improve hurricane forecasts (Pu et al. 2014).
Impacts of AIRS data assimilation on numerical simulations of Typhoon Nuri as revealed by time series of minimum central sea level pressure (simulations vs best track). NODA denotes an experiment without data assimilation. CTRL represents an experiment with assimilation of NCEP BUFR (all routine data). AIRS_BUFR_L is an experiment with assimilation of both AIRS and NCEP BUFR data but uses a lower model top (50 hPa). AIRS_BUFR_H is the same as AIRS_BUFR_L except it used a higher model top (15 hPa).
Citation: Bulletin of the American Meteorological Society 97, 12; 10.1175/BAMS-D-14-00188.1
Ocean data assimilation.
Satellite remotely sensed sea surface salinity (SSS) is a new observation capability, with the data quality only now approaching mission accuracy targets; consequently, development work is required and is underway to sufficiently characterize the data and their impact prior to operational assimilation in the NOAA operational global Real-Time Ocean Forecast System (RTOFS-Global). Scientists from NOAA (both NESDIS/STAR and NWS/NCEP) assessed the impact of SSS observations in the RTOFS-Global, which employs the HYCOM (Bleck 2002) for its computational core. Using NCEP Climate Forecast System Reanalysis atmospheric forcing (Saha et al. 2010), two experiments were performed: first, HYCOM was relaxed to Polar Science Center Hydrographic Climatology (Steele et al. 2001) monthly climatological SSS fields, and second, the HYCOM was relaxed to monthly mean experiments using European Space Agency’s (ESA) Soil Moisture Ocean Salinity (SMOS) SSS data. Figure 6 depicts preliminary results for the time evolution of mean sea surface heights for 2010–12, averaged over four selected subregions, with satellite sea surface height (SSH) anomaly observations shown for comparison. The SMOS SSS data provide some improvement with extremes of simulated SSH anomaly for the midlatitudes of the North Atlantic and North Pacific Ocean regions (Figs. 6a,b). In the equatorial regions of the North Atlantic and North Pacific Oceans, modeled SSH anomalies agree better with observations, and the impact of satellite SSS data are less clear (Figs. 6c,d). Direct benefits of this study include improved model ocean surface density (salinity and temperature); better representation of mesoscale dynamics and upper-ocean heat content for improved coupled modeling, such as NOAA’s Climate Forecast System and HWRF, and better salinity constraints for downscaling to nested regional ocean/coastal models. These experiments on S4 have served us to mature the effort in preparation for initiating an R2O transition.
HYCOM mean sea surface height anomalies for four regions: (a) North Pacific, (b) North Atlantic, (c) equatorial Pacific, and (d) equatorial Atlantic. The observations are from NESDIS/STAR SSH data (Leuliette et al. 2004).
Citation: Bulletin of the American Meteorological Society 97, 12; 10.1175/BAMS-D-14-00188.1
Land data assimilation.
The land surface research team at NOAA (NESDIS and NWS) has focused on the development and evaluation of a procedure for the dual assimilation of microwave (MW) and thermal infrared (TIR)-based soil moisture (SM) retrievals into an offline continental-scale land surface model (LSM; Walker et al. 2001; Hain et al. 2011). The TIR methods provide relatively high spatial resolution (on the order of ∼100 m to 10 km) and the potential for SM retrievals over a wider range of vegetation cover but lower temporal resolution due to the limitation of TIR-based land surface temperature (LST) retrieval to clear-sky conditions (typical repeat cycles of 2–7 days). The current operational North America Land Data Assimilation System (NLDAS) drought monitoring system uses no data assimilation of any remotely sensed geophysical quantity. A new experimental NLDAS-type system, NLDAS-DA, is currently being tested using the ensemble Kalman filter of NASA’s LIS (Kumar et al. 2006, 2008; Peters-Lidard et al. 2007) implemented on S4 resources. The new system provides an upgrade in spatial resolution (0.125° to 0.04°) and can directly assimilate both MW and TIR SM information. Figure 7 shows a comparison of total column SM anomaly for 5 August 2011 from the NLDAS-DA compared to the U.S. Drought Monitor drought map, valid for the same week.
Total column soil moisture anomaly for 5 Aug 2011 from (left) NLDAS-DA compared to the (right) U.S. Drought Monitor drought map, valid for the 5 Aug 2011 week.
Citation: Bulletin of the American Meteorological Society 97, 12; 10.1175/BAMS-D-14-00188.1
Another research task carried out by the land team is the analysis of the impact of three NRT satellite observations of land surface parameters (green vegetation fraction, albedo, and solar insolation) on SM estimates from the Noah LSM. Positive impact is observed on SM estimates for both surface and root-zone SM estimates from Noah LSM using NRT surface parameters. The research on the S4 system is currently being expanded to implement these techniques for inclusion and evaluation in the NCEP operational NWP modeling suite. A test of assimilating global soil moisture data from SMOS into GFS has demonstrated the positive impacts of satellite soil moisture observations on GFS forecasts (Zhan et al. 2014), making this project mature for an R2O transition. Using the NASA Unified Weather Research and Forecasting (NUWRF) model coupled with LIS (Peters-Lidard et al. 2015), the land team is also testing the impacts of land satellite data products (e.g., green vegetation fraction, soil moisture, land surface temperature, and snow) on regional NWP (Fang et al. 2016).
Global and regional OSSEs.
OSSEs provide an effective means to evaluate the potential impact of a proposed observing system, to determine tradeoffs in their design, and to evaluate data assimilation methodology (Atlas 1997). By their nature, OSSE efforts do not necessarily result in a transition to operations in a traditional way, but they do provide critical information to decision-makers in the satellite acquisition program. Offering the O2R environment to the OSSE projects makes the projects’ outcome credible and therefore the information provided useful for a decision-making process. By keeping the model system for these OSSEs as close to operations as possible, we are able to provide a better sense of how new instruments will impact operational weather forecasts. If these prospective instruments show significant impacts and are subsequently built and flown, the operational system can build off of the work done by the OSSE team to allow for assimilation of these new observations, therefore increasing readiness for new sensors. A series of OSSEs are being performed on S4 to allow interested scientists from JCSDA, Earth System Science Interdisciplinary Center (ESSIC)/University of Maryland (UMD), NOAA/Earth System Research Laboratory (ESRL), and NOAA/Atlantic Oceanographic and Meteorological Laboratory (AOML) to test the impacts that prospective new satellite observations may have on meteorological forecasts.
Two global OSSEs have been run on S4 by using the GSI to perform data assimilation and the GFS to conduct control and sensitivity experiments. The first tested two potential wind lidars to be flown on the International Space Station (ISS). After a control run, based on the observing system from July to August 2012, a second run added an optical autocovariance wind lidar (OAWL), and the third run added Winds from the ISS for Climate Research (WISSCR) instrument instead of the OAWL winds. Results showed that flying a wind lidar aboard the ISS could lead to significant reductions in root-mean-square error (Fig. 8) as well as large increases in anomaly correlation scores in the GFS (Ma et al. 2014; Atlas et al. 2015b). Recently, the OAWL instrument tested here was selected as part of the NASA Earth Science Technology Office (ESTO) Instrument Incubator Program (IIP), increasing the likelihood that these Doppler wind lidar observations will be assimilated operationally in the near future (Tucker 2014).
(top) Wind RMSE in GFS model for control data (black), control + OAWL (red), and control + WISSCR (green). (bottom) OAWL and WISCSR are compared directly to control. Boxes at top of (bottom) denote 95% confidence intervals (from Ma et al. 2014).
Citation: Bulletin of the American Meteorological Society 97, 12; 10.1175/BAMS-D-14-00188.1
The second experiment, currently ongoing, examines the impact of new global positioning system radio occultation (GPS-RO) measurements on the global model. The assimilation of just six GPS-RO satellites in the operational model has already shown a significant positive impact in numerical weather prediction (Cucurull 2010). The facts that GPSRO is minimally affected by clouds and precipitation, and provides equal accuracy over land as well as over oceans, are a few of the main reasons for the high value of these data. OSSEs are indeed needed for an objective evaluation of the benefits that more GPS-RO observations would bring. A control run using the 2014 observing system (with effectively 6 GPS-RO satellites) is being compared with subsequent runs adding 6, 12, 18, 30, and 42 GPS-RO instruments. As GPS-RO observations are relatively new and few in number, the question is whether the observing system will reach a saturation point, where additional observations have no significant forecast impact. Ongoing work shows that increasing the number of GPS-RO satellites being assimilated results in better weather forecast skill: 18 satellites are better than 12 satellites, and 12 satellites are better than 6 satellites.
Two regional OSSEs were also conducted on S4. In the first hurricane OSSEs, the hurricane analysis and hurricane track and intensity prediction are being studied in a collaborative effort between NOAA, NASA, Simpson Weather Associates, the University of Miami, and JCSDA. The objectives of these hurricane OSSEs are to determine 1) the potential impact of unmanned aerial systems, 2) the relative impact of alternative concepts for space-based lidar winds, 3) the potential impact of geostationary microwave sounders, and 4) the relative impact of alternative concepts for polar and geostationary hyperspectral sounders. For these experiments, the ARW model at 1-km resolution was embedded in a T511 global nature run that had previously been generated by the European Centre for Medium-Range Weather Forecasts (ECMWF). The first nature run to be generated covered a 13-day period and included tropical cyclone formation, movement, and rapid intensification (Nolan et al. 2013). Global assimilation was performed using GDAS. Regional assimilation used NOAA’s operational HRWF at a 9-km resolution and either 3D VAR or EnKF analysis schemes. Forecasts were generated using the HWRF model at 3-km resolution. The experiments that have been performed thus far have shown that global assimilation improves the boundary conditions for the HWRF regional model and has a significantly larger impact on hurricane track forecast accuracy than regional data assimilation does. In contrast, regional assimilation has a significantly larger impact on the forecast of maximum wind (figure not shown; Atlas et al. 2015a).
The second regional-scale OSSE employing an EnKF assimilation system (Anderson et al. 2009) and the WRF (Skamarock et al. 2005) Model was conducted at CIMSS/UW. The primary objective of this study was to examine the potential impact of assimilating GOES-R ABI IR brightness temperatures that would be sensitive to clouds and water vapor (WV) on the forecast accuracy of a high-impact weather event (Otkin 2012). Overall, the temperature and wind fields were most accurate when observations sensitive to WV in the upper troposphere were assimilated; however, for the cloud and WV analyses, the largest improvements occurred when brightness temperatures sensitive to WV in the lower and middle troposphere were assimilated. Compared to a control case in which only conventional observations were assimilated, short-range precipitation forecasts were more accurate when the IR observations were assimilated. These results demonstrate the benefit of using WV-sensitive IR brightness temperatures from geosynchronous satellite platforms in data assimilation systems to improve not only cloud and moisture fields, but also temperature and wind fields.
SUMMARY.
A wide variety of projects are running on the S4 with a large range of applications. Their common thread is a close coordination between researchers from NOAA and academics. Feedback from the above research projects indicate that the O2R environment provided by the S4 supercomputer is indispensable. The S4 supercomputer provides computational and disk storage resources for handling computationally expensive activities (otherwise difficult to obtain for most research institutions), such as data assimilation and impact experiments, NRT forecasts, radiance simulations, and calibration/validation activities. These projects have greatly benefited from the S4 infrastructure, specifically from having access to an operationally equivalent environment, supercomputing resources, and technical support. With NOAA’s operational numerical modeling suites (e.g., GSI, GFS, HWRF, NAM, CRTM, HYCOM, and NLDAS) available on the S4, researchers are able to eliminate the otherwise difficult and time-consuming process of porting systems and observational datasets to the development computing environment. NOAA, on the other hand, significantly increases the potential of these projects that it is investing in to get their outcomes transitioned to operational systems. Indeed, new developments or modifications to the operational suites developed in the S4 O2R environment are expected to be easily ported back to the operational suite, thus facilitating smooth R2O transitions. This is because all code changes, necessary to achieve the scientific added value, are all rigorously tracked through a software management approach, making the effort to transition these changes back to operations (R2O) a simple task.
The research projects highlighted in this paper are at various stages of development as they evolve through the R2O process. As mentioned in the previous section, some of the projects have already matured and managed to perform an R2O transition. Some others are on the verge of an R2O transition and some others require additional effort to reach maturity, but the infrastructure for R2O exists to facilitate that. Benefits of the S4 O2R environment to NOAA and society include projects successfully transitioned to operational systems and consequent forecast capability enhancement or application. Projects such as the new LNVD QC scheme for GSI AMV assimilation and the generation of a NAM continental U.S. (CONUS) nest brightness temperature simulation for NCEP’s postprocessing (listed in Table 1 but not highlighted in this section) as well as the S4 OSSE study showing positive impact by the OAWL instrument, leading to the selection of that instrument as part of NASA’s ESTO IIP, have all added value to the NOAA capability. Additional projects have already demonstrated operational benefit, but their R2O transitions are still in progress.
Table 1 summarizes the degree of maturity of the different projects. The NOAA TRL nomenclature (refer to Table 2) is used to describe this maturity. To quantify the added value of the S4 environment, the TRLs before using the S4 and the current level are listed. The increase in the TRL index gives an efficient way to assess R2O potential maturity and monitor a project’s progress in its R2O path. It is worth noting that many projects on the S4 are aimed at increasing readiness for the next generation of NOAA polar orbiting and geostationary satellites (GOES-R and JPSS), which are scheduled to be launched in late 2016 and 2017, respectively. Having this S4 environment available at the onset of these proving ground and risk reduction projects has played a major role in increasing this readiness to take full advantage of these sensors when they are launched. The project of the real-time simulation of GOES-R ABI and Advanced Himawari Imager (AHI) data stream by the CIMSS/UW and the NWS/Office of Science and Technology (OST; Greenwald et al. 2016), for example, has allowed tests of the end-to-end data flow for the $10 billion GOES-R sensors, highlighting the challenges and allowing the validation of the system performance on Advanced Weather Interactive Processing System (AWIPS II), on time for the upcoming launch.
DISCUSSION.
As evident in the highlighted projects, high potential for success exists for research-to-operations (R2O) transition when modeling and data assimilation projects are developed in a robust and well-maintained operations-to-research (O2R) environment comprising high performance computing (HPC) capability and user support. Each of these projects demonstrated the value of using the O2R’s NOAA “operational” systems for research as well as the added value from well-documented and traceable code, scripts, and other software library changes through rigorous software configuration management. Not all projects in the S4 O2R environment have been able to follow the guidelines and to perform the research under rigorous software configuration management. Other NOAA-funded projects continue to be developed outside of the S4 O2R environment either because of unawareness of the utility or lack of interest in working in a constrained environment, such as the S4 O2R environment. Consequently, a challenge remains, requiring further training and outreach, as well as S4 expansion, to convey that using one of the O2R environments (S4 or otherwise) is a means for exploiting existing operational capabilities while ensuring that the research undertaken is as relevant as possible to NOAA’s mission and to the nation as a whole.
Although sometimes listed as a reason for not using the O2R environment to undertake research, using the S4 O2R environment in no way stifles the science being performed (a point researchers are sensitive to). One of the lessons learned is that there have been cases where projects funded by non-NOAA sources showed great enthusiasm to be part of the O2R, motivated solely by the desire to make a positive impact in the operational systems. Scientists in this case believed they could have a positive impact on society at large if their scientific innovations were implemented into operations. The O2R environment taps therefore into this powerful motivator (of wanting to make a positive impact) that drives many of us researchers.
The S4 should be considered a community resource, available to the research community willing to work closely with NOAA, on subjects of common interest. We welcome requests for accounts of S4, as we see this collaboration as a way to funnel efforts in the wider research community to contribute to the NOAA mission through a robust O2R/R2O infrastructure. This, we believe, will offer a way to infuse new ideas from the research community into NOAA operational-like systems and, in return, researchers will benefit from having access to these systems and computer resources to assess their scientific effort in a larger context. It is worth noting that the funding source of these projects is not a factor in allocating resources to S4 but rather the alignment of the project objectives with those of NOAA. For more information, including how to obtain S4 access, please e-mail the NOAA S4 point of contact (the first author of this publication).
In summary, the O2R allows an alignment of interest, research, operational needs, and the desire of scientists to make a difference, making this O2R/R2O concept a win–win approach. We do expect that more outreach and training is needed to make the O2R environment more widely accessible. This will likely be accompanied by a need to expand it to other areas that support the NOAA mission and a subsequent need to expand its capacity to support more projects.
One of the lessons learned is the importance of securing adequate support and funding to 1) the O2R environment as described above, 2) the research itself with an explicit agreement to implement it on an O2R environment (or test bed), but also 3) the operational partner on the receiving end of the R2O transition. Sustained interaction between operational staff and researchers is also necessary over the lifetime of an O2R/R2O project. Early engagement of all stakeholders is indeed crucial for the success of the projects. Another critical component one must address is the synchronization of the research project execution timeline with the operational timeline. Often, there are limited open windows when operational upgrades can occur, and R2O transitions are allowed to happen. Projects should account for these open windows in their planning at the onset of the project.
Other lessons learned for a successful R2O include the need for nurturing a culture of team building between researchers and operational recipients, encouraging an environment of mutual respect for the innovation of the research community and the pragmatism of the operational community. Perhaps one of the most successful and visible aspects of this close coordination is the S4 concept itself: a success story of collaboration between NOAA and its outside research partners. Indeed, the full funding for the S4 HPC and its upgrade was borne by NOAA, as is the cost for maintaining and porting the NOAA systems on it, while the cost for IT and power/cooling is borne by the academic partner (University of Wisconsin). All projects sustained on S4 are from a wide range of NOAA and academic partners, as reflected in the list of coauthors of this study. In a sense, the mixed nature of the S4 itself (mixed NOAA–academic partnership), as opposed to a pure NOAA or a pure academic initiative, helps facilitate this O2R/R2O concept and set the right tone for the projects on it. Everyone involved has a vested interest in making the projects succeed in their R2O transition and bridge the gap between research and operations.
In conclusion, long-term and sustained O2R support for internal and external research projects is critical for the competitiveness of NOAA’s operational systems because without a robust O2R environment, NOAA’s exploitation of R2O transitions will not reach its potential for effectively and efficiently enhancing NOAA’s operational numerical analysis and forecast systems. Working within the S4 O2R environment provides researchers with a realistic sense of operational constraints and real-life limitations (e.g., actual operational HPC infrastructure and computational resources as well as firsthand experience on models that run on them) that R2O transitions must address and must allow an easy traceability of all changes needed for the R2O transitions.
ACKNOWLEDGMENTS
The views expressed in this publication are those of the authors and do not necessarily represent those of NOAA. The authors thank many programs that contributed to the funding of the projects supported on the S4 environment. These programs are diverse and numerous, and include many in NOAA and other federal agencies: JCSDA, GOES-R, JPSS, QOSAP, OWAQ, NASA Headquarters, and so on. Special thanks go to Drs. Rick Spinrad, Peter Colohan, and Christopher Moses, with whom we had fruitful discussions and who shared with us the main lessons learned from a NOAA-led R2X summit, some of which were included in this study.
Appendix. Acronyms.
| 3D-Var | Three-dimensional variational data assimilation |
| ACC | Anomaly correlation coefficient |
| ABI | Advanced Baseline Imager |
| AHI | Advanced Himawari Imager |
| AIRS | Atmospheric Infrared Sounder |
| AMV | Atmospheric motion vector |
| AOML | Atlantic Oceanographic and Meteorological Laboratory |
| ARW | Advanced Research WRF |
| AVHRR | Advanced Very High Resolution Radiometer |
| AWIPS | Advanced Weather Interactive Processing System |
| CIMSS/UW | Cooperative Institute for Meteorological Satellite Studies/University of Wisconsin |
| CIRA/CSU | Colorado State University/Cooperative Institute for Research in the Atmosphere |
| CMFT | Interactive Community BUFR-Formatting Tool |
| COAT | Community Observation Assessment Tool |
| CONUS | Continental United States |
| CRTM | Community Radiative Transfer Model |
| CSTROT | Community Satellite Data Thinning and Representation Optimization Tool |
| DA | Data assimilation |
| ECMWF | European Centre for Medium-Range Weather Forecasting |
| EFSO | Ensemble forecast sensitivity to observations |
| EMC | Environmental Modeling Center |
| EnKF | Ensemble Kalman filter |
| ESRL | Earth System Research Laboratory, NOAA |
| ESTO | Earth Science Technology Office, NASA |
| ESSIC/UMD | Earth System Science Interdisciplinary Center/University of Maryland |
| GDAS | Global Data Assimilation System |
| GFS | Global Forecast System |
| GOES | Geostationary Operational Environmental Satellite |
| GOES-R | Geostationary Operational Environmental Satellite R Series |
| GPS-RO | Global positioning system radio occultation |
| GSI | Gridpoint Statistical Interpolation analysis system |
| HWRF | Hurricane Weather Research and Forecasting Model |
| HYCOM | Hybrid Coordinate Ocean Model |
| IAT | Independent assessment tools |
| IIP | Instrument Incubator Program |
| IR | Infrared |
| ISS | International Space Station |
| JCSDA | U.S. Joint Center for Satellite Data Assimilation |
| JIBB | JCSDA In a Big Box |
| JPSS | Joint Polar Satellite System |
| LIS | Land Information System model |
| LNVD | Log normalized vector departure |
| LSM | Land Surface Model |
| MODIS | Moderate Resolution Imaging Spectroradiometer |
| MSG | Meteosat Second-Generation satellite |
| MW | Microwave |
| NASA | National Aeronautics and Space Administration |
| NCEP | National Centers for Environmental Prediction |
| NDAS | NAM Data Assimilation System |
| NESDIS/STAR | National Environmental Satellite, Data, and Information Service/Center for Satellite Applications and Research |
| NLDAS | North America Land Data Assimilation System |
| NOAA | National Oceanic and Atmospheric Administration |
| NUWRF | NASA Unified Weather Research and Forecasting |
| NWS | National Weather Service |
| O2R | Operations to research |
| OAWL | Optical autocovariance wind lidar |
| OSSE | Observation System Simulation Experiment |
| OST | Office of Science and Technology |
| PQC | Proactive quality control |
| QC | Quality control |
| R2O | Research to operations |
| RTOFS | NOAA’s operational Real-Time Ocean Forecast System |
| S4 | Supercomputer for Satellite Simulations and Data Assimilation Studies |
| SAPHIR | Sounder for Atmospheric Profiling of Humidity in the Intertropical Regions |
| SDAT | Satellite Data Assimilation system for Tropical Storm Forecasts |
| SEVIRI | Spinning Enhanced Visible and Infrared Imager |
| SM | Soil moisture |
| SMOS | European Space Agency’s Soil Moisture Ocean Salinity |
| SSA | Sea surface height |
| SSEC/UW | Space Science and Engineering Center at the University of Wisconsin–Madison |
| SSS | Sea surface Salinity |
| TIR | Thermal infrared |
| VIIRS | Visible Infrared Imaging Radiometer Suite |
REFERENCES
Anderson, J., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Avellano, 2009: The Data Assimilation Research Testbed: A community facility. Bull. Amer. Meteor. Soc., 90, 1283–1296, doi:10.1175/2009BAMS2618.1.
Atlas, R., 1997: Atmospheric observations and experiments to assess their usefulness in data assimilation. J. Meteor. Soc. Japan, 75, 111–130.
Atlas, R., L. Bucci, B. Annane, R. Hoffman, and S. Murillo, 2015a: Observing System Simulation Experiments to assess the potential impact of new observing systems on hurricane forecasting. Mar. Technol. Soc. J., 49, 140–148, doi:10.4031/MTSJ.49.6.3.
Atlas, R., and Coauthors, 2015b: Observing System Simulation Experiments (OSSEs) to evaluate the potential impact of an optical autocovariance wind lidar (OAWL) on numerical weather prediction. J. Atmos. Oceanic Technol., 32, 1593–1613, doi:10.1175/JTECH-D-15-0038.1.
Atlas, R., V. Tallapragada, and S. Gopalakrishnan, 2015c: Advances in tropical cyclone intensity forecasts. Mar. Technol. Soc. J., 49, 149–160, doi:10.4031/MTSJ.49.6.2.
Bernardet, L., and Coauthors, 2015: Community support and transition of research to operations for the hurricane weather research and forecasting model. Bull. Amer. Meteor. Soc., 96, 953–960, doi:10.1175/BAMS-D-13-00093.1.
Bleck, R., 2002: An oceanic general circulation model framed in hybrid isopycnic-Cartesian coordinates. Ocean Modell., 4, 55–88, doi:10.1016/S1463-5003(01)00012-9.
Cucurull, L., 2010: Improvement in the use of an operational constellation of GPS radio occultation receivers in weather forecasting. Wea. Forecasting, 25, 749–767, doi:10.1175/2009WAF2222302.1.
Derber, J., D. Parrish, and S. Lord, 1991: The New Global Operational Analysis System at the National Meteorological Center. Wea. Forecasting, 6, 538–547, doi:10.1175/1520-0434(1991)006<0538:TNGOAS>2.0.CO;2.
Fang, L., C. Hain, X. Zhan, J. Yin, W. Zheng, J. Dong, and M. Ek, 2016: Enhancing weather forecasts via assimilating real-time satellite-based soil moisture and green vegetation fraction. 30th Conf. on Hydrology, New Orleans, LA, Amer. Meteor. Soc., J10.2. [Available online at https://ams.confex.com/ams/96Annual/webprogram/Paper283808.html.]
Greenwald, T., and Coauthors, 2016: Real-time simulation of the GOES-R ABI for user readiness and product evaluation. Bull. Amer. Meteor. Soc., 97, 245–261, doi:10.1175/BAMS-D-14-00007.1.
Hain, C. R., W. T. Crow, J. R. Mecikalski, M. C. Anderson, and T. R. Holmes, 2011: An intercomparison of available soil moisture estimates from thermal infrared and passive microwave remote sensing and land surface modeling. J. Geophys. Res., 116, D15107, doi:10.1029/2011JD015633.
Han, Y., P. van Delst, Q. Liu, F. Weng, B. Yan, R. Treadon, and J. Derber, 2006: JCSDA Community Radiative Transfer Model (CRTM)—Version 1. NOAA Tech. Rep. NESDIS 122, 40 pp. [Available online at www.star.nesdis.noaa.gov/sod/sst/micros/pdf/CRTM_v1_NOAAtechReport-1.pdf.]
Hotta, D., 2014: Proactive quality control based on ensemble forecast sensitivity to observations. Ph.D. dissertation, University of Maryland, 248 pp.
Janjic, Z. I., 2003: A nonhydrostatic model based on a new approach. Meteor. Atmos. Phys., 82, 271–285, doi:10.1007/s00703-001-0587-6.
Kalnay, E., Y. Ota, T. Miyoshi, and J. Liu, 2012: A simpler formulation of forecast sensitivity to observations: Application to ensemble Kalman filters. Tellus, 64A, 18462, doi:10.3402/tellusa.v64i0.18462.
Kanamitsu, M., 1989: Description of the NMC global data assimilation and forecast system. Wea. Forecasting, 4, 335–342, doi:10.1175/1520-0434(1989)004<0335:DOTNGD>2.0.CO;2.
Kleist, D. T., D. F. Parrish, J. C. Derber, R. Treadon, R. M. Errico, and R. Yang, 2009: Improving incremental balance in the GSI 3DVAR analysis system. Mon. Wea. Rev., 137, 1046–1060, doi:10.1175/2008MWR2623.1.
Kumar, S. V., and Coauthors, 2006: Land information system: An interoperable framework for high resolution land surface modeling. Environ. Modell. Software, 21, 1402–1415, doi:10.1016/j.envsoft.2005.07.004.
Kumar, S. V., R. H. Reichle, C. D. Peters-Lidard, R. D. Koster, X. Zhan, W. T. Crow, J. B. Eylander, and P. R. Houser, 2008: A land surface data assimilation framework using the land information system: Description and applications. Adv. Water Resour., 31, 1419–1432, doi:10.1016/j.advwatres.2008.01.013.
Kumar, S. V., J. C. Alpert, D. L. Carlis, and B. A. Ballish, 2009: Investigation of NCEP GFS model forecast skill “dropout” characteristics using the EBI index. 23rd Conf. on Weather Analysis and Forecasting/19th Conf. on Numerical Weather Prediction, Omaha, NE, Amer. Meteor. Soc., 13A.1. [Available online at https://ams.confex.com/ams/23WAF19NWP/techprogram/paper_154282.htm.]
Leuliette, E.W., R. S. Nerem, and G. T. Mitchum, 2004: Calibration of TOPEX/Poseidon and Jason Altimeter data to construct a continuous record of mean sea level change. Mar. Geod., 27, 79–94, doi:10.1080/01490410490465193.
Li, H., J. Liu, and E. Kalnay, 2010: Correction of ‘Estimating observation impact without adjoint model in an ensemble Kalman filter.’ Quart. J. Roy. Meteor. Soc., 136, 1652–1654, doi:10.1002/qj.658.
Li, J., W. P. Menzel, F. Sun, T. J. Schmit, and J. Gurka, 2004: AIRS subpixel cloud characterization using MODIS cloud products. J. Appl. Meteor., 43, 1083–1094, doi:10.1175/1520-0450(2004)043<1083:ASCCUM>2.0.CO;2.
Li, J., C. Y. Liu, H.-L. Huang, T. J. Schmit, W. P. Menzel, and J. Gurka, 2005: Optimal cloud-clearing for AIRS radiances using MODIS. IEEE Trans. Geosci. Remote Sens., 43, 1266–1278, doi:10.1109/TGRS.2005.847795.
Li, J., P. Wang, H. Han, J. Li, and J. Zheng, 2016: On the assimilation of satellite sounder data in cloudy skies in the numerical weather prediction models. J. Meteor. Res., 30, 169–182, doi:10.1007/s13351-016-5114-2.
Liu, J., and E. Kalnay, 2008: Estimating observation impact without adjoint model in an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 134, 1327–1335, doi:10.1002/qj.280.
Ma, Z., L. P. Riishojgaard, M. Masutani, and J. Woollen, 2014: Observing system simulation experiments for space-based Doppler wind lidar observations. Second Symp. of the Joint Center for Satellite Data Assimilation, Atlanta, GA, Amer. Meteor. Soc., 779. [Available online at https://ams.confex.com/ams/94Annual/webprogram/Paper234212.html.]
Merceret, F. J., T. P. O’Brien, W. P. Roeder, L. L. Huddleston, W. H. Bauman III, and G. J. Jedlovec, 2013: Transitioning research to operations: Transforming the “Valley of Death” into a “valley of opportunity.” Space Wea., 11, 637–640, doi:10.1002/swe.20099.
Nolan, D. S., R. Atlas, K. T. Bhatia, and L. R. Bucci, 2013: Development and validation of a hurricane nature run using the joint OSSE nature run and the WRF Model. J. Adv. Model. Earth Syst., 5, 382–405, doi:10.1002/jame.20031.
Ota, Y., J. C. Derber, E. Kalnay, and T. Miyoshi, 2013: Ensemble-based observation impact estimates using the NCEP GFS. Tellus, 65A, 20038, doi:10.3402/tellusa.v65i0.20038.
Otkin, J. A., 2012: Assimilation of water vapor sensitive infrared brightness temperature observations during a high impact weather event. J. Geophys. Res., 117, D19203, doi:10.1029/2012JD017568.
Peters-Lidard, C. D., and Coauthors, 2007: High-performance Earth system modeling with NASA/GSFC’s Land Information System. Innovations Syst. Software Eng., 3, 157–165, doi:10.1007/s11334-007-0028-x.
Peters-Lidard, C. D., and Coauthors, 2015: Integrated modeling of aerosol, cloud, precipitation and land processes at satellite-resolved scales. Environ. Modell. Software, 67, 149–159, doi:10.1016/j.envsoft.2015.01.007.
Pu, Z., H. Zhang, and S. A. Boukabara, 2014: The impact of satellite temperature and moisture profiles on simulating and understanding tropical cyclone genesis. Second Symp. on the Joint Center for Satellite Data Assimilation, Atlanta, GA, Amer. Meteor. Soc., 796. [Available online at https://ams.confex.com/ams/94Annual/webprogram/Paper241713.html.]
Robinson, R. M., 2012: Research to operations: Space weather’s valley of opportunity. Space Wea., 10, doi:10.1029/2012SW000854.
Saha, S., and Coauthors, 2010: The NCEP Climate Forecast System Reanalysis. Bull. Amer. Meteor. Soc., 91, 1015–1057, doi:10.1175/2010BAMS3001.1.
Santek, D., B. Hoover, and S. Nebuda, 2014: Evaluation of a new method to quality control satellite-derived polar winds in the NCEP GDAS/GFS. Second Symp. on the Joint Center for Satellite Data Assimilation, Atlanta, GA, Amer. Meteor. Soc., J5.4. [Available online at https://ams.confex.com/ams/94Annual/webprogram/Paper241928.html.]
Skamarock, W. C., J. B. Klemp, J. Dudhia, D. O. Gill, D. M. Barker, W. Wang, and J. G. Powers, 2005: A description of the Advanced Research WRF version 2. NCAR Tech. Note NCAR/TN-468+STR, 88 pp., doi:10.5065/D6DZ069T.
Steele, M., R. Morley, and W. Ermold, 2001: PHC: A global ocean hydrography with a high quality Arctic Ocean. J. Climate, 14, 2079–2087, doi:10.1175/1520-0442(2001)014<2079:PAGOHW>2.0.CO;2.
Tallapragada, V., and Coauthors, 2013: Hurricane Weather and Research and Forecasting (HWRF) model: 2013 scientific documentation. Development Testbed Center Doc., 99 pp. [Available online at www.dtcenter.org/HurrWRF/users/docs/scientific_documents/HWRFv3.5a_ScientificDoc.pdf.]
Tucker, S. C., 2014: HSRL for aerosols, winds, and clouds using optical autocovariance wind lidar. 17th Symp. on Meteorological Observation and Instrumentation, Westminster, CO, Amer. Meteor. Soc., 6.7. [Available online at https://ams.confex.com/ams/21Applied17SMOI/webprogram/Paper247042.html.]
Walker, J. P., G. R. Willgoose, and J. D. Kalma, 2001: One-dimensional soil moisture profile retrieval by assimilation of near-surface observations: A comparison of retrieval algorithms. Adv. Water Resour., 24, 631–650, doi:10.1016/S0309-1708(00)00043-9.
Wang, P., J. Li, J. Li, Z. Li, T. J. Schmit, and W. Bai, 2014: Advanced infrared sounder subpixel cloud detection with imagers and its impact on radiance assimilation in NWP. Geophys. Res. Lett., 41, 1773–1780, doi:10.1002/2013GL059067.
Wang, P., and Coauthors, 2015: Assimilation of thermodynamic information from advanced infrared sounders under partially cloudy skies for regional NWP. J. Geophys. Res. Atmos., 120, 5469–5484, doi:10.1002/2014JD022976.
Wang, X., D. Parrish, D. Kleist, and J. Whitaker, 2013: GSI 3DVar-based ensemble–variational hybrid data assimilation for NCEP Global Forecast System: Single-resolution experiments. Mon. Wea. Rev., 141, 4098–4117, doi:10.1175/MWR-D-12-00141.1.
Zapotocny, T., J. A. Jung, and J. F. Le Marshall, 2007: Recent MODIS data assimilation experiments in the NCEP GFS. 14th Conf. on Satellite Meteorology and Oceanography, Atlanta, GA, Amer. Meteor. Soc., P5.14. [Available online at https://ams.confex.com/ams/Annual2006/webprogram/Paper103928.html.]
Zhan, X., J. Liu, W. Zheng, and M. B. Ek, 2014: Soil moisture operational product system (SMOPS) for NCEP GFS soil moisture data assimilation. Second Symp. on the Joint Center for Satellite Data Assimilation, Atlanta, GA, Amer. Meteor. Soc., J4.2. [Available online at https://ams.confex.com/ams/94Annual/webprogram/Paper233737.html.]
Zhang, M., M. Zupanski, M. Kim, and J. Knaff, 2013: Assimilating AMSU-A radiances in TC core area with NOAA operational HWRF (2011) and a hybrid data assimilation system: Danielle (2010). Mon. Wea. Rev., 141, 3889–3907, doi:10.1175/MWR-D-12-00340.1.
Zhu, T., and S. A. Boukabara, 2015: Development and impact study of community satellite data thinning and representation optimization tool. 20th Conf. on Satellite Meteorology and Oceanography, Phoenix, AZ, Amer. Meteor. Soc., 225. [Available online at https://ams.confex.com/ams/95Annual/webprogram/Paper267375.html.]
Zupanski, M., 2005: Maximum likelihood ensemble filter: Theoretical aspects. Mon. Wea. Rev., 133, 1710–1726, doi:10.1175/MWR2946.1.
Zupanski, M., I. M. Navon, and D. Zupanski, 2008: The maximum likelihood ensemble filter as a non-differentiable minimization algorithm. Quart. J. Roy. Meteor. Soc., 134, 1039–1050, doi:10.1002/qj.251.
