The existence of freely available data assimilation software is permitting the general NWP community to perform research and development in the same framework as used in a significant number of operational implementations.
The merging of meteorological observations and short-range weather forecasts, via the process of data assimilation, to provide the initial conditions for NWP is an increasingly popular area of interdisciplinary research. The end-to-end process is highly complex. Measurements from a wide range of remote sensing and in situ observing platforms, each with unique error characteristics, require thinning, bias correction, and the application of complex quality control procedures before they can be assimilated. In most modern data assimilation algorithms, short-range (typically 1–6 h) forecasts, with their own equally complex systematic and random errors, propagate previous observations to the present, thereby filling in the gaps where no current observations exist and providing a meteorologically valid “background field” as a basis for the analysis. The data assimilation procedure produces an analysis through the invocation of fundamental statistical concepts (e.g., Bayes rule relates the statistics of the analysis to the input background “prior” and observations), meteorological understanding (e.g., dynamical balance, physical parameterizations), numerical methods (e.g., minimization algorithms), and modern software engineering techniques (e.g., shared/distributed memory parallelism). Daley (1991) and Kalnay (2003) provide comprehensive overviews of the data assimilation challenge.
Modern operational data assimilation systems ingest millions of observations and must produce the analysis (with a similar number of degrees of freedom) in a time slot of between 5 and 60 min, within a few hours of the observations being taken. Improved NWP models, novel data assimilation algorithms, and new observing types emerge continuously to provide better analyses and subsequent forecasts. However, experience over the past few decades indicates that the impact of all these efforts on NWP skill has been incremental, with average forecast improvements on the order of a few percent per year (e.g., Simmons and Hollingsworth 2002). Historical trends are not always a good indicator of future performance, but the implication is that there are no short cuts to the complex, resource-intensive process outlined above if one wishes to retain a world-class data assimilation system. Consequently, the development of state-of-the-art systems has been confined to large development teams within operational weather centers (e.g., Parrish and Derber 1992; Rabier et al. 2000; Rawlins et al. 2007). External community influence has been generally confined to publishing longer-term data assimilation research using toy model systems, and short-term collaborations to assist testing relatively simple tweaks to current systems.
In the past decade, a new paradigm for collaborative community data assimilation research has emerged. An upsurge in interest in data assimilation in universities, federal agencies, and the private sector, combined with the ability to freely exchange algorithms and data over the Internet, have led to the development and use of freely available operational quality community data assimilation tools. Two recent examples include the Community Radiative Transfer Model (CRTM; Han et al. 2006) and the National Center for Atmospheric Research's (NCAR's) ensemble-based Data Assimilation Research Testbed (Anderson et al. 2009). This article provides an overview of a third community tool: the Weather Research and Forecasting (WRF) model's Community Variational/Ensemble Data Assimilation System (WRFDA), developed for and widely used by the Weather Research and Forecasting (Skamarock et al. 2008) model's international data assimilation research/operational community.
COMMUNITY DATA ASSIMILATION FOR WRF: THE WRFDA DEVELOPMENT STORY.
The multiagency WRF model effort has had a significant data assimilation component from its earliest days. Initial discussions between major partners [NCAR, National Oceanic and Atmospheric Administration (NOAA), the U.S. Air Force Weather Agency (AFWA), Oklahoma University, and the U.S. Naval Research Laboratory] in 1999–2001 resulted in a set of generic requirements for a unified community data assimilation system: accuracy, robustness, computational efficiency, portability, flexibility, support, documentation, and ease of use. A major challenge for community NWP is to satisfy the diverse requirements of the member agencies. For example, robustness and efficiency are highest priority for operations, but arguably less so for academia. In contrast, portability, support, and flexibility are vital if the wider research community is to use the operational system to research observation impacts, advanced data assimilation algorithms, etc. The WRF data assimilation working group, set up to assess the possibility of developing a common framework, initially tasked each partner to define its “essential” and “desirable” scientific and technical capabilities. A subset of essential features [conventional observations, three-dimensional variational data assimilation (3DVAR) algorithm, portable, supported, and with good documentation] was agreed as the initial basic requirement. A review of the available codes was then undertaken to assess which (if any) of the available systems could provide a suitable starting point, resulting in an agreement to base the initial WRF data assimilation system on the community fifth-generation Pennsylvania State University–National Center for Atmospheric Research Mesoscale Model (MM5) 3DVAR system (Barker et al. 2003). Practical issues rather than scientific merit dominated this choice; that is, MM5 3DVAR's relative flexibility, ease of use, dedicated WRF resources to further develop the system, and available support. The MM5 3DVAR system also had the advantage of being built directly within the WRF software framework, thus providing a direct interface to other components of the WRF modeling system (see Fig. 1).
The basic WRF modeling system, including WRF-VAR (WRFDA), from Skamarock et al. (2008).
Citation: Bulletin of the American Meteorological Society 93, 6; 10.1175/BAMS-D-11-00167.1
The first version of the WRF 3DVAR was distributed to developers in June 2003, with upgraded versions released to the general community in WRF version 2.0 (May 2004) and version 3.0 (Skamarock et al. 2008). In 2004, the inclusion of a four-dimensional variational data assimilation (4DVAR; Huang et al. 2009) capability resulted in a change of name to WRF-VAR. Similarly, in 2008 the release of a hybrid variational–ensemble algorithm (see below) led to a second renaming to WRFDA.
Support for the WRFDA community is provided through scientific/technical documentation, a user's guide, an online tutorial, and test datasets (available at www.mmm.ucar.edu/wrf/users/wrfda). A modest level of helpdesk support (25% of one scientist) is also available (wrfhelp@ucar.edu). A component of the WRF tutorial held yearly in Boulder, Colorado, is devoted to WRFDA. Additional user-requested WRFDA tutorials have also been held in numerous countries, including Taiwan, India, Vietnam, South Korea, and China.
WRFDA OBSERVATIONAL CAPABILITIES.
The WRFDA system can ingest a wide variety of observation types, in addition to the standard conventional observation types (surface, rawinsonde, aircraft, wind profiler, and atmospheric motion vectors). An overview of WRFDA's radar radial velocity and reflectivity assimilation capabilities is given in Xiao et al. (2008a), including results from the Korean Meteorological Administration's (KMA's) operational mesoscale WRF configuration. The significant potential of observations derived from GPS radio occultation measurements is investigated in Cucurull et al. (2004 in Cucurull et al. (2006), and positive signals using the GPS-derived zenith total delay have been found by Guo et al. (2005) and Faccani and Ferretti (2005a,b). Powers (2007) found significant benefit assimilating both high-resolution Antarctic surface data and Moderate Resolution Imaging Spectroradiometer (MODIS)-derived atmospheric motion vectors in the Antarctic Mesoscale Prediction System (AMPS).
Satellite radiance data assimilation.
Over the past two decades, most leading NWP centers have moved from the assimilation of derived temperature/humidity retrievals to the direct assimilation of raw satellite radiances (i.e., brightness temperatures; e.g., McNally et al. 2000). Direct radiance assimilation requires a forward radiative transfer model (RTM), which creates model-simulated satellite brightness temperatures from input NWP model atmospheric temperature and moisture profiles and surface parameters. Variational radiance data assimilation additionally requires the tangent linear and adjoint versions of the RTM. An RTM within WRFDA must be accurate yet fast enough to be utilized in operational NWP assimilation. The WRFDA system is unique in that it interfaces to the two most widely used fast RTMs: Radiative Transfer for Television and Infrared Observation Satellite (TOVS; RTTOV) developed and maintained by the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT), and the U.S. Joint Center for Satellite Data Assimilation (JCSDA) CRTM (Han et al. 2006). A flexible interface to both RTTOV and CRTM ensures that WRFDA users can assimilate radiance data from all sensors that can be simulated by either RTM, provided that corresponding data interface and quality control have been implemented.
The WRFDA system directly ingests radiances in the National Centers for Environmental Prediction (NCEP) Binary Universal Form for the Representation of Meteorological Data (BUFR) format. Thus, WRFDA can directly assimilate both NOAA/AFWA near-real-time operational feeds (ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/), as well as historical data (http://nomads.ncdc.noaa.gov/). Table 1 provides a list of radiance data sources that have been successfully assimilated in the latest WRFDA version 3.3 release (March 2011). The 20 instruments from eight platforms include both microwave [e.g., Advanced Microwave Sounding Unit (AMSU)-A, AMSU-B, and Microwave Humidity Sounder (MHS)] and infrared [e.g., High Resolution Infrared Radiation Sounder (HIRS) and Atmospheric Infrared Sounder (AIRS)] sensors. Only instruments on board polar platforms are currently supported; future work will add sensors on board geostationary platforms [e.g., Geostationary Operational Environmental Satellite (GOES)] to the list. A flexible program design allows WRFDA users to add additional instruments for their own research, with relatively small development effort. The following observation screening and quality control is an essential component of the data assimilation process and is included within WRFDA: domain check, gross error check, data thinning, data splitting into discrete time slots, bias correction, and the objective tuning of observation errors (Desroziers and Ivanov 2001).
Satellite radiance platforms/instruments used in WRFDA version 3.3. Blank cells indicate instruments that are either not present on the particular platform or failed early after launch. EOS = Earth Observing System; DMSP = Defense Meteorological Satellite Program; SSMIS = Special Sensor Microwave Imager/Sounder.
Satellite radiance measurements and RTMs are prone to systematic errors (i.e., biases) that must be corrected before radiances can be assimilated. Biases typically vary with platform, instrument, channel, scan angle, and atmospheric conditions. Estimation of bias via offline linear regression (i.e., modeling biases in past data as a weighted combination of bias predictors, e.g., scan angle and thickness) has been used successfully for many years in operational NWP (e.g., Harris and Kelly 2001). However, biases can change rapidly so they should ideally be estimated adaptively as part of the assimilation algorithm. This is achieved in WRFDA via a variational bias correction algorithm, which updates the bias coefficients within the linear regression as a part of the variational minimization (Dee 2005; Auligné et al. 2007). The evolving nature of the radiance bias is illustrated in Fig. 2a for a selection of radiance channels for the AMSUA microwave instrument aboard Meteorological Operation (MetOp)-2. Observation minus background forecast (O – B) differences are taken from a 20-month period within a continuously cycling Arctic regional reanalysis application of WRFDA. In this example, the bias corrections are estimated offline [variationally, but only minimizing the bias coefficients using the European Centre for Medium-Range Weather Forecasts (ECMWF) Re-Analysis (ERA)-Interim as a reference fields]. Results show that the bias correction clearly differs for each channel, and exhibits a clear monthly to seasonal dependence. The corresponding random component of the observation minus forecast differences is shown in Fig. 2b. The introduction of the bias correction in May 2007 reduces the standard deviation of the O – B difference over the first few months as the system spins up. Monitoring of the evolving O – B statistics is standard practice within operational NWP. The inclusion of an adaptive bias correction algorithm helps to automate this process, thus reducing the human effort required to monitor observation quality. As an example, the progressive failure of AMSU-A channel 7 from October 2008 onward (the blue curve in Fig. 2b) is also picked up by the adaptive bias correction algorithm, leading to its eventual rejection from the data assimilation system.
Time series of (a) bias correction and (b) standard deviation of observed minus CRTM-calculated brightness temperatures for MetOp-2 AMSU-A channels 5–9. Data from a regional 30-km WRFDA polar stereographic application centered on the North Pole for the 20-month period between Apr 2007 and Dec 2008.
Citation: Bulletin of the American Meteorological Society 93, 6; 10.1175/BAMS-D-11-00167.1
Radiance data assimilation in WRFDA has been applied in numerous applications to date to study different weather phenomena over various regions (e.g., Lu et al. 2010; He et al. 2011). The challenge of cloudy radiance assimilation in WRFDA has been illustrated by Liu et al. (2009), and will be a focus for future developments. Figure 3a shows AIRS channel 1519 (which is sensitive to upper-tropospheric water vapor) radiance observations for the 0600 UTC assimilation cycle on 26 August 2005, clearly indicating the presence of cloud/precipitation around Hurricane Katrina over Florida. Currently, WRFDA calculates only “clear” radiances (i.e., no cloud/precipitation information is considered in the RTM), as illustrated in Fig. 3b for the corresponding brightness temperatures from a 15-km WRF 6-h forecast. Therefore, observations “contaminated” by cloud/precipitation are currently removed as part of the quality control procedure. The development of an all-sky radiance capability is an area of active research.
(a) AIRS channel 1519 (sensitive to upper-tropospheric water vapor) radiance observations for the 0600 UTC assimilation cycle on 26 Aug 2005, clearly indicating the presence of cloud/precipitation around Hurricane Katrina over Florida. (b) The corresponding “clear” (see text) brightness temperatures from a 15-km WRF 6-h forecast are shown.
Citation: Bulletin of the American Meteorological Society 93, 6; 10.1175/BAMS-D-11-00167.1
DATA ASSIMILATION TECHNIQUES WITHIN WRFDA.
At the heart of WRFDA's core is a variational minimization of a cost function designed to optimally blend observations and prior NWP forecasts. A variety of alternative variational data assimilation techniques are available: 3DVAR (Barker et al. 2004), the more computationally intensive 4DVAR (Huang et al. 2009), and a hybrid variational/ensemble algorithm that combines the benefits of the physically based variational approach with the statistical, flow-dependent error information provided by ensemble forecasts (Wang et al. 2008a,b).
Forecast error covariance estimation.
In order to optimize the use of input observational and prior forecast data, data assimilation requires accurate estimates of observation and forecast error. The WRFDA system includes a table of observation errors for each major observation type, as used in AFWA applications of WRFDA (see below). Default synoptic-scale climatological forecast error statistics are also provided for initial setup, testing, and training runs. However, significantly enhanced performance is usually obtained using forecast error statistics calculated for the specific domain of interest. The WRFDA's “gen_be” utility estimates domain-specific climatological estimates of forecast errors based on input training data, either time series of forecast differences (Parrish and Derber 1992) or perturbations from an ensemble prediction system (Skamarock et al. 2008, chapter 9). Figure 4 illustrates the positive impact of domain-specific forecast errors during a 1-month 3DVAR cycling experiment in the AMPS (Powers et al. 2003). The ~0.5-K reduction in temperature T + 24 forecast error is one of the most significant improvements made to the real-time AMPS configuration since its inception (e.g., larger than the impact of any individual observation type). The production of domain-specific climatological forecast error covariances is clearly an essential component of any publication-quality research project, especially those that attempt to compare WRFDA performance with alternative assimilation methods such as the ensemble Kalman filter (e.g., Zhang et al. 2011) for which forecast error estimation is more automated (but also more expensive).
AMPS domain mean and root-mean-square temperature T + 24 forecast error as verified against rawinsondes through May 2004. Forecast run from 3DVAR analyses produced using default background errors supplied with the WRFDA download (green curves), and forecast skill obtained using climatological background errors calculated via gen_be specifically for the AMPS configuration (red curves) are indicated.
Citation: Bulletin of the American Meteorological Society 93, 6; 10.1175/BAMS-D-11-00167.1
Like most variational data assimilation algorithms, gen_be estimates multivariate covariances that permit observations of one variable (e.g., surface pressure) to influence the analyses of others (e.g., surface wind and temperature) via statistical regressions. A unique feature of WRFDA's gen_be utility is its ability to model forecast error covariances using data from a variety of models in addition to WRF, as demonstrated in Fig. 5 using KMA global spectral model T213 (~60 km) resolution data to train the statistical covariances. In the mid-/high latitudes there is a very strong correlation (up to 90% in the midtroposphere) between temperature and surface pressure errors and those estimates from the nondivergent wind (actually streamfunction) via geostrophic balance. In the tropics, this correlation drops to 0%–20% (as expected, WRFDA will not erroneously apply geostrophic increments at low latitudes).
Normalized correlation between (left) temperature and (right) surface pressure errors and values predicted via statistical regression of streamfunction T + 48–T + 24 forecast differences (after Wu et al. 2002). Values reaching 0.9 in the extratropical midtroposphere imply that 90% of the temperature/surface pressure error is predictable from knowledge of the nondivergent wind. Forecast data are taken from Aug 2002 and Jan 2004 KMA global model output. (right) Correlations for Aug 2002 (red curves) and Jan 2004 (blue curves) are indicated, showing little seasonal dependence in the correlation for surface pressure errors.
Citation: Bulletin of the American Meteorological Society 93, 6; 10.1175/BAMS-D-11-00167.1
Many WRF applications are at convective scale, where the usual synoptic-scale balances (e.g., hydrostatic and geostrophic) are of reduced relevance and interactions between mass, wind, and hydrometeor fields are of primary interest. The gen_be algorithm provides a test bed to investigate advanced covariance models. For example, Michel et al. (2011) found quite different relationships in forecast error between precipitating and nonprecipitating areas in a WRF 3-km ensemble. In addition, a number of significant relationships between hydrometeor and wind errors were found (e.g., the link between rain errors and vertical velocity through the unbalanced convective-scale divergence field).
Hybrid variational–ensemble data assimilation.
NWP forecast errors generally depend on the synoptic conditions at a particular time. The WRFDA's 4DVAR capability provides a degree of flow dependence through the use of the linear forecast model to evolve perturbations through a short time window (Huang et al. 2009). However, this comes at a cost; there are both computational as well as human resources required to maintain a linear forecast model and its adjoint. The ensemble Kalman filter (EnKF) is an attractive alternative to 4DVAR because it does not require a linear/adjoint model to provide this flow dependence (e.g., Houtekamer and Mitchell 1998; Hunt et al. 2007; Anderson et al. 2009). Instead, the EnKF derives error estimates from the nonlinear short-range forecasts of an ensemble prediction system (EPS). Computational cost is significantly reduced if an operational EPS is already in place (e.g., to provide probabilistic NWP products). The EnKF update step is computationally relatively cheap compared to the cost of running the ensemble of forecasts. In contrast, the cost of 4DVAR is dominated by the iterative integration of linear and adjoint models within the variational minimization. The major limitation of the EnKF approach is sampling error caused by the relatively small number of ensemble members (typically 20–100) that are affordable for operational NWP. Hybrid variational– ensemble data assimilation attempts to combine the benefits of ensemble data assimilation (flow dependence and flexibility) with those of variational systems (simultaneous treatment of observations, dynamical/physical constraints, complex quality control, treatment of nonlinearities via an outer loop, etc.). In the hybrid approach, EnKF sampling error is ameliorated through the combination of flow-dependent (but low rank) ensemble-derived covariances and full-rank (but climatological) estimates typically used within variational data assimilation (Hamill and Snyder 2000; Etherton and Bishop 2004). The WRFDA hybrid algorithm mirrors that developed at the Met Office (Barker 1999) by introducing flow dependence via additional control variables within the minimization. The WRFDA hybrid algorithm requires relatively minor modifications to the variational algorithm, and has been shown to beat pure variational and ensemble techniques in both 3DVAR (Wang et al. 2008a,b) and 4DVAR modes (Zhang et al. 2011). The coupled hybrid WRFDA–EPS algorithm is shown in Fig. 6. A significant benefit of this hybrid approach over 3DVAR is illustrated in Fig. 7 for a 1-month trial within AFWA's 15-km Caribbean theater (Demirtas et al. 2009).
Use of WRFDA within a coupled ensemble prediction system. In the forecast step, an ensemble of N WRF forecasts xnf is integrated forward to the next assimilation time window. In the update step, the ensemble mean forecast is used as background for WRFDA (3DVAR or 4DVAR), and ensemble perturbations (member minus mean) supply estimates of flow-dependent forecast errors (dashed blue line). The hybrid method mitigates ensemble sampling error by combining both climatological and flow-dependent estimates of forecast error. Observations y° are assimilated simultaneously via WRFDA and also are used within an ETKF (Bishop et al. 2001) to update the ensemble perturbations ready for the next cycle of ensemble forecasts.
Citation: Bulletin of the American Meteorological Society 93, 6; 10.1175/BAMS-D-11-00167.1
Impact of hybrid data assimilation during a 30-day trial (from 17 Aug to 15 Sep 2007) in AFWA's Caribbean theater. The T + 48 forecast error verified against radiosondes is shown for (a) u wind, (b) v wind, (c) temperature, and (d) specific humidity. Results demonstrate that hybrid 3DVAR data assimilation (red curves) significantly reduces forecast error relative to pure 3DVAR (blue curves).
Citation: Bulletin of the American Meteorological Society 93, 6; 10.1175/BAMS-D-11-00167.1
Liu et al. (2008, 2009) take the WRFDA hybrid concept one stage further via the En4DVAR algorithm, which replaces linear and adjoint models within 4DVAR with a time series of EPS perturbations. The cost of En4DVAR is then approximately the same as that of 3DVAR, but with the advantage over the pure EnKF of reduced sampling error through the use of hybrid covariances.
The WRFDA system has been applied in a variety of purely ensemble data assimilation studies. Barker (2005) used WRFDA's covariance modeling capabilities to assess the impact of ensemble size and covariance localization on the accuracy of flow-dependent multivariate forecast errors in the AMPS. WRFDA's climatological covariance model has been used to provide meteorologically consistent initial and lateral boundary perturbations for EnKF experiments (e.g., Torn et al. 2006; Meng and Zhang 2008). The WRFDA algorithm has also been used as a preprocessor to provide ensembles of model-simulated observations to the EnKF, thus bypassing the need to develop complex quality control, bias correction, and radiance observation operators within the EnKF itself (e.g., Schwartz et al. 2012).
A particular focus for WRFDA studies has been the improved prediction of tropical cyclone track and intensity in mesoscale NWP (Chen et al. 2004, 2007; Gu et al. 2005; Huang et al. 2005; Xiao et al. 2006, 2007). Xiao et al. (2005) and Xiao and Sun (2007) describe applications of WRFDA to the prediction of severe weather events in convective-scale NWP. Lee et al. (2006) present results of the positive impact of an Incremental Analysis Update (IAU; Bloom et al. 1996) initialization procedure applied to mesoscaleb NWP. Xiao et al. (2008b) demonstrated the use of a diagnostic adjoint sensitivity technique to trace back Antarctic forecast errors to their origins within the initial analysis for an Antarctic severe-weather event. A complete publication list, together with tutorials, and detailed documentation for the interested user are available (www.mmm.ucar.edu/wrf/users/wrfda/).
OPERATIONAL APPLICATIONS OF WRFDA.
The WRFDA system has from the outset been designed with operational implementation in mind. Indeed, the bulk of the funding for WRFDA development has been provided by operational centers (see the acknowledgements). A list of known real-time/operational implementations to date is shown in Table 2. The distinction between real-time and operational designations is significant and reflects the very high level of robustness required for operational status (typically <1% failure rate), as well as potentially contractually agreed upon customer service–level agreements, etc. The implementation of an early version of WRFDA with the MM5 at the Taiwanese Central Weather Bureau (CWB) in May 2002 represented the first use of a component of WRF in operations (Barker et al. 2004). Previous BAMS articles include details of WRFDA's application in the AMPS (Powers et al. 2003) and at the KMA (Xiao et al. 2008a). In this article, we document results from two additional WRFDA operational implementations that arose in very different ways.
WRFDA real-time (RT)/operational (O) implementations 2002–10. AFAD = Air Force and Air Defense, HWRF = Hurricane WRF, WRFRT = Real-time WRF, NCMRWF = National Center for Medium-Range Weather Forecasts, BMB = Beijing Meteorological Bureau, SAF = Swedish Air Force.
WRFDA in operations at AFWA.
The U.S. AFWA has been a partner in the WRF effort since its inception, and has provided continual guidance and significant support in the development of data assimilation and modeling facilities that are now freely available to the entire WRF community. The September 2002 implementation of WRFDA in AFWA operations represented the U.S. Department of Defense's first operational 3DVAR capability as well as the first implementation of a component of WRF in operations in the United States. Worldwide AFWA theaters at the time of the 2002 WRFDA implementation are shown in Fig. 8. Since 2002, AFWA has had effectively global coverage at 45-km resolution, which has recently been upgrading to 15 km.
Worldwide AFWA theaters at the time of initial WRFDA implementation with MM5 in Sep 2002; the first use of a component of WRF in operations, and the first 3DVAR system to be implemented by the U.S. Department of Defense are shown.
Citation: Bulletin of the American Meteorological Society 93, 6; 10.1175/BAMS-D-11-00167.1
Sample forecast verification statistics from preoperational WRFDA testing in AFWA's European theater during the period from 4 June to 10 July 2002 are shown in Fig. 9. Results indicate that WRFDA significantly and systematically improved the quality of MM5 height and humidity forecasts compared to the multivariate optimal interpolation (MVOI; see Daley 1991) system that it replaced (observations and model were identical in the comparison). In addition, given that WRFDA is designed to run efficiently on massively parallel supercomputers, the runtime for AFWA's data assimilation was reduced from 5 to less than 1 min. Since initial implementation in 2002, the WRFDA version used at AFWA has been updated to incorporate new capabilities as they become available.
Comparison of WRFDA (triangles) and MVOI (squares) root-mean-square forecast error verified against rawinsonde observations for the period from 4 Jun to 10 Jul 2002: analysis (red), T + 12 (green), and T + 24 (blue) forecasts in AFWA's European domain are shown. (a) Height and (b) relative humidity are also indicated. [From B. Craig, AFWA.]
Citation: Bulletin of the American Meteorological Society 93, 6; 10.1175/BAMS-D-11-00167.1
WRFDA in operations in the United Arab Emirates.
Given the very limited support available at NCAR for WRFDA use in the general community, it is vital that the system be well documented, easy to use, and robust to a range of applications, computing platforms, compilers, etc. A good example of how the community has succeeded (with very little support) in setting up a sophisticated operational NWP system, including data assimilation, is provided by the United Arab Emirates WRF system (UAE/WRF), developed as part of the United Nations Development Program by the UAE Air Force and Air Defense Meteorological Department (Ajjaji et al. 2007). The UAE/WRF configuration shown in Fig. 10 consists of triple, two-way WRF nests with observations assimilated in all three domains by cycling WRFDA. Lateral/lower boundary conditions for the outer domain are based on 6-hourly, 0.5° data available via ftp from NOAA's Global Forecast System. After a series of setup and verification tests, the model became operational on 15 August 2006.
Domains of the triple-nested 312-/6-/4-km resolution UAE/WRF system run operationally by the UAE's Air Force and Air Defense Meteorological Department.
Citation: Bulletin of the American Meteorological Society 93, 6; 10.1175/BAMS-D-11-00167.1
Running WRFDA locally allows the system to ingest local radar and classified conventional data in addition to the wide range of observations available on the Global Telecommunications System (surface, rawinsonde, aircraft, atmospheric motion vectors). Nonconventional data used include radiances, radar reflectivities, and GPS refractivities. Level 1b radiances are assimilated through WRFDA's implementation of the JCSDA's CRTM (Han et al. 2006). Typical observation numbers assimilated in a single 6-h assimilation window via the First Guess at Appropriate Time (FGAT; Lee and Barker 2005) 3DVAR method are given in Table 3. Observation errors are tuned and radiance biases are corrected. The initial analysis is updated via a second outer-loop iteration to account for nonlinear effects and to improve observation quality control. Estimates of forecast error are computed locally using WRFDA's gen_be utility to further tune 3DVAR for the UAE/WRF application.
Number of observations per report type used in the 0000, 0006, 0012, and 0018 UTC FGAT UAE/WRFDA analyses in the outer domain on 15 Jun 2010. METAR = Aviation routine weather report, AIREP = Aircraft, GEOAMV = Geostationary Atmospheric Motion Vector, SATEM = Satellite temperature retrieval, GPSREF = GPS refractivity.
RETROSPECTIVE OF THE WRFDA COMMUNITY DATA ASSIMILATION EFFORT.
A decade on from its inception, it is instructive to review the successes and failures of the WRFDA program. In terms of its initial goal to build an advanced, flexible, easy-to-use data assimilation system that is used in both research and operational environments, the WRFDA program has succeeded with over 40 publications to date, numerous unique features, and a significant number of worldwide operational implementations. Several hundred scientists have attended the Boulder-based WRFDA tutorials to date, with the tutorial also being provided at various international locations. With no single source of funding, the WRFDA program has relied on synergies and leveraging between a number of individual research and development projects to build a unified system, with an expanded range of capabilities made freely available to all in yearly public releases. There is a snowball effect—as the sophistication of the community system grows, more projects and partners see the benefits of getting involved, and in turn they provide additional capabilities, resources for testing, and new applications that improve the system further still.
The community model approach comes with significant overhead. Close communications is essential to ensure that the evolving needs of each project within scope are given due consideration in the direction of the program. The level of effort involved in testing and maintaining the system for a wide variety of applications, platforms (supercomputers, desktops, and laptops), operating systems, compilers, visualization tools, etc., grows exponentially as the program develops, and so must be efficiently organized. Reliance on development partners outside one's own organization sometimes requires commitment and good planning because conflicts naturally arise between internal project milestones and community model plans. Inevitably, individual projects end and partners' strategies evolve, sometimes to the extent that they withdraw from the effort. Despite these challenges, the potential for significant further enhancements to the WRFDA system are tremendous. The WRF community currently stands at over 15,000 users worldwide. Even a very conservative estimate that only 3% of the users' efforts lead to new capabilities within the community system, this is a nominal 450 people, larger than any NWP group in the world. Given the complexity of modern-day NWP systems, the community model paradigm will continue to positively influence the development of global NWP capabilities for many years to come.
ACKNOWLEDGMENTS.
The WRFDA program could not have been possible without the guidance and support of a large number of people and agencies, too many to list in their entirety. The U.S. AFWA, Taiwanese CWB, KMA, NASA, NSF, and NOAA have provided major support for the development of WRFDA. The WRFDA system has benefitted greatly from close interaction with the WRF development and support team in the MMM Division of NCAR: Dave Gill, Wei Wang, Cindy Bruyere, Michael Duda, Jimy Dudhia, Bill Skamarock, and Joe Klemp. Wan-Shu Wu (NOAA/NCEP) and the late Desző Dévényi (NOAA/ESRL) contributed capabilities to an early version of WRFDA. A full list of community WRFDA-related papers is maintained at www.mmm.ucar.edu/wrf/users/wrfda/pub-doc.html.
REFERENCES
Ajjaji, R., A. A. Al-Katheri, and A. Dhanhani, 2007: Tuning of WRF 3D-var data assimilation system over Middle-East and Arabian Peninsula. Extended Abstracts, Eighth WRF Users' Workshop, Boulder, CO, WRF. [Available online at www.mmm.ucar.edu/wrf/users/workshops/WS2007/abstracts/4-6_Ajjaji.pdf.]
Anderson, J., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Avellano, 2009: The Data Assimilation Research Testbed: A community facility. Bull. Amer. Meteor. Soc., 90, 1283–1296.
Auligné, T., A. P. McNally, and D. P. Dee, 2007: Adaptive bias correction for satellite data in a numerical weather prediction system. Quart. J. Roy. Meteor. Soc., 133, 631–642.
Barker, D. M., 1999: The use of synoptically-dependent background error structures in 3DVAR. Met Office VAR Scientific Documentation Paper 26, 11 pp. [Available from Met Office, Fitzroy Road, Exeter EX1 3PB, United Kingdom.]
Barker, D. M., 2005: Southern high-latitude ensemble data assimilation in the Antarctic Mesoscale Prediction System. Mon. Wea. Rev., 133, 3431–3449.
Barker, W. Huang, Y.-R. Guo, and A. Bourgeois, 2003: A three-dimensional variational (3DVAR) data assimilation system for use with MM5. NCAR Tech. Note NCAR/TN-453+STR, 68 pp. [Available online at www.mmm.ucar.edu/mm53dvar/docs/3DVARTechDoc.pdf.]
Barker, W. Huang, Y.-R. Guo, and A. Bourgeois, and Q. N. Xiao, 2004: A three-dimensional (3DVAR) data assimilation system for use with MM5: Implementation and initial results. Mon. Wea. Rev.,132, 897–914.
Bishop, C., B. J. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420–436.
Bloom, S., L. L. Takacs, A. Silva, and D. Ledvian, 1996: Data assimilation using incremental analysis updates. Mon. Wea. Rev., 124, 1256–1271.
Chen, S.-H., 2007: The impact of assimilating SSM/I and QuikSCAT satellite winds on Hurricane Isidore simulations. Mon. Wea. Rev., 135, 549–566.
Chen, S.-H., F. Vandenberghe, G. W. Petty, and J. Bresch, 2004: Application of SSM/I satellite data to a hurricane simulation. Quart. J. Roy. Meteor. Soc., 130, 801–825.
Cucurull, L., F. Vandenberghe, D. M. Barker, E. Vilaclara, and A. Rius, 2004: Three-dimensional variational data assimilation of ground-based GPS ZTD and meteorological observations during the 14 December 2001 storm event over the western Mediterranean Sea. Mon. Wea. Rev., 132, 749–763.
Cucurull, L., Y.-H. Kuo, D. M. Barker, and S. R. H. Rizvi, 2006: Assessing the impact of COSMIC GPS radio occultation data on weather analysis and short-term forecast over the Antarctic: A case study. Mon. Wea. Rev., 134, 3283–3296.
Daley, R., 1991: Atmospheric Data Analysis. Cambridge Atmospheric and Space Science Series, Cambridge University Press, 457 pp.
Dee, D. P., 2005: Bias and data assimilation. Quart. J. Roy. Meteor. Soc., 131, 3323–3343.
Demirtas, M., D. M. Barker, Y. Chen, J. Hacker, X.-Y. Huang, C. Snyder, and X. Wang, 2009: A hybrid data assimilation system (ensemble transform Kalman filter and WRF-VAR) based retrospective tests with real observations. Preprints, 19th Conf. on Numerical Weather Prediction, Omaha, NE, Amer. Meteor. Soc., 9A.5. [Available online at http://ams.confex.com/ams/pdfpapers/153340.pdf.]
Desroziers, G., and S. Ivanov, 2001: Diagnosis and adaptive tuning of observation-error parameters in a variational assimilation. Quart. J. Roy. Meteor. Soc., 127, 1433–1452.
Etherton, B., and C. Bishop, 2004: Resilience of hybrid ensemble 3DVAR analysis schemes to model error and ensemble covariance error. Mon. Wea. Rev., 132, 1065–1080.
Faccani, C., and R. Ferretti, 2005a: Data assimilation of high-density observations. I: Impact on initial conditions for the MAP/SOP IOP2b. Quart. J. Roy. Meteor. Soc., 131, 21–42.
Faccani, C., and R. Ferretti, 2005b: Data assimilation of high-density observations. II: Impact on the forecast of the precipitation for the MAP/SOP IOP2b. Quart. J. Roy. Meteor. Soc., 131, 43–61.
Gu, J., Q.-N. Xiao, Y.-H. Kuo, D. M. Barker, J. Xue, and X. Ma, 2005: A case study of Typhoon Rusa (2002) on its analysis and simulation using WRF 3DVAR and the WRF modeling system. Adv. Atmos. Sci., 22, 415–425.
Guo, Y.-R., H. Kusaka, D. M. Barker, Y.-H. Kuo, and A. Crook, 2005: Impact of ground-based GPS PW and MM5-3DVAR background error statistics on forecast of a convective case. SOLA, 1, 73–76.
Hamill, T. M., and C. Snyder, 2000: A hybrid ensemble Kalman filter–3D variational analysis scheme. Mon. Wea. Rev., 128, 2905–2919.
Han, Y., P. van Delst, Q. Liu, F. Weng, B. Yan, R. Treadon, and J. Derber, 2006: JCSDA Community Radiative Transfer Model (CRTM)—Version 1. NOAA/NESDIS/Tech. Rep. 122, 33 pp. [Available online at ftp://ftp.emc.ncep.noaa.gov/jcsda/CRTM/CRTM_v1-NOAA_Tech_Report_NESDIS122.pdf.]
Harris, B. A., and G. Kelly, 2001: A satellite radiancebias correction for data assimilation. Quart. J. Roy. Meteor. Soc., 127, 1453–1468.
He, W., Z. Liu, and H. Cheng, 2011: Influence of surface temperature and emissivity on AMSU-A assimilation over land. Acta Meteor. Sin., 25, 545–557.
Houtekamer, P. L., and Herschel L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796–811.
Huang, C.-Y., Y.-H. Kuo, S.-H. Chen, and F. Vandenberghe, 2005: Improvements in typhoon forecasts with assimilated GPS occultation refractivity. Wea. Forecasting, 20, 931–953.
Huang, X.-Y., and Coauthors, 2009: Four-dimensional variational data assimilation for WRF: Formulation and preliminary results. Mon. Wea. Rev., 137, 299–314.
Hunt, B., E. Kosterlich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble Kalman filter. Physica D, 230, 112–126.
Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, 364 pp.
Lee, M.-S., and D. M. Barker, 2005: Preliminary tests of first guess at appropriate time (FGAT) with WRF 3DVAR and WRF model. J. Korean Meteor. Soc., 41, 495–505.
Lee, M.-S., Y.-H. Kuo, and D. M. Barker, 2006: Incremental analysis updates initialization technique applied in 10-km MM5 3DVAR and model. Mon. Wea. Rev., 134, 1389–1404.
Liu, C., Q. Xiao, and B. Wang, 2008: An ensemble-based four-dimensional variational data assimilation scheme. Part I: Technical formulation and preliminary test. Mon. Wea. Rev., 136, 3363–3373.
Liu, C., Q. Xiao, and B. Wang, 2009: An ensemble-based fourdimensional variational data assimilation scheme. Part II: Observing system simulation experiments with Advanced Research WRF (ARW). Mon. Wea. Rev., 137, 1687–1704.
Liu, Z., X. Zhang, T. Auligné, and H.-C. Lin, 2009: Variational analysis of hydrometeors with satellite radiance observations: A simulated study. 10th WRF Users' Workshop, Boulder, CO, WRF, 2A.1. [Available online at www.mmm.ucar.edu/wrf/users/workshops/WS2009/abstracts/2A-01.pdf.]
Lu, Q., W. Zhang, P. Zhang, X. Wu, F. Zhang, Z. Liu, and D. M. Barker, 2010: Monitoring the 2008 cold surge and frozen disasters snowstorm in south China based on regional ATOVS data assimilation. Sci. China; Earth Sci., 53, 1216–1228.
McNally, P., J. C. Derber, W. Wu, and B. B. Katz, 2000: The use of TOVS level-1b radiances in the NCEP SSI analysis system. Quart. J. Roy. Meteor. Soc., 126, 689–724.
Meng, Z., and F. Zhang, 2008: Tests of an ensemble Kalman filter for mesoscale and regional-scale data assimilation. Part III: Comparison with 3DVAR in a real-data case study. Mon. Wea. Rev., 136, 522–540.
Michel, Y., T. Auligné, and T. Montmerle, 2011: Heterogeneous convective-scale background error covariances with the inclusion of hydrometeor variables. Mon. Wea. Rev., 139, 2994–3015.
Parrish, D. F., and J. C. Derber, 1992: The National Meteorological Center's spectral statistical interpolation analysis system. Mon. Wea. Rev., 120, 1747–1763.
Powers, J. G., 2007: Numerical prediction of an Antarctic severe wind event with the Weather Research and Forecasting (WRF) model. Mon. Wea. Rev., 135, 3134–3157.
Powers, J. G., A. J. Monaghan, A. M. Cayette, D. H. Bromwich, Y. -H. Kuo, and K. W. Manning, 2003: Real-time mesoscale modeling over Antarctica: The Antarctic Mesoscale Prediction System (AMPS). Bull Amer. Meteor. Soc., 84, 1533–1546.
Rabier, F., H. Järvinen, E. Klinker, J.-F. Mahfouf, and A. Simmons, 2000: The ECMWF operational implementation of four-dimensional variational assimilation. I: Experimental results with simplified physics. Quart. J. Roy. Meteor. Soc., 126, 1143–1170.
Rawlins, F., S. P. Ballard, K. J. Bovis, A. M. Clayton, D. Li, G. W. Inverarity, A. C. Lorenc, and T. J. Payne, 2007: The Met Office global four-dimensional variational data assimilation scheme. Quart. J. Roy. Meteor. Soc., 133, 347–362.
Schwartz, C. S., Z. Liu, Y. Chen, and X.-Y. Huang, 2012: Impact of assimilating microwave radiances with a limited-area ensemble data assimilation system on forecasts of Typhoon Morakot. Wea. Forecasting, in press.
Simmons, A. J., and A. Hollingsworth, 2002: Some aspects of the improvement in skill of numerical weather prediction. Quart. J. Roy. Meteor. Soc., 128, 647–677.
Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech Note NCAR/TN-475+STR, 113 pp. [Available online at www.mmm.ucar.edu/wrf/users/docs/arw_v3.pdf.]
Torn, R. D., G. J. Hakim, and C. Snyder, 2006: Boundary conditions for limited-area ensemble Kalman filters. Mon. Wea. Rev., 134, 2490–2502.
Wang, X., D. M. Barker, C. Snyder, and T. M. Hamill, 2008a: A hybrid ETKF-3DVAR data assimilation scheme for the WRF model. Part I: Observing system simulation experiment. Mon. Wea. Rev., 136, 5116–5131.
Wang, X., D. M. Barker, C. Snyder, and T. M. Hamill, 2008b: A hybrid ETKF- 3DVAR data assimilation scheme for the WRF model. Part II: Real observation experiments. Mon. Wea. Rev., 136, 5132–5147.
Xiao, Q., and J. Sun, 2007: Multiple-radar data assimilation and short-range quantitative precipitation forecasting of a squall line observed during IHOP_2002. Mon. Wea. Rev., 135, 3381–3404.
Xiao, Q., Y. H. Kuo, J. Sun, W. C. Lee, E. Lim, Y. R. Guo, and D. M. Barker, 2005: Assimilation of Doppler radar observations with a regional 3DVAR system: Impact of Doppler velocities on forecasts of a heavy rainfall case. J. Appl. Meteor., 44, 768–788.
Xiao, Q., Y. H. Kuo, Y. Zhang, D. M. Barker, and D.-J. Won, 2006: A tropical cyclone bogus data assimilation scheme in the MM5 3D-VAR system and numerical experiments with Typhoon Rusa (2002) near landfall. J. Meteor. Soc. Japan, 84, 671–689.
Xiao, Q., Y. H. Kuo, J. Sun, W.-C. Lee, D. M. Barker, and E. Lim, 2007: An approach of Doppler reflectivity data assimilation and its assessment with the inland QPF of Typhoon Rusa (2002) at landfall. J. Appl. Meteor. Climatol., 46, 14–22.
Xiao, Q., and Coauthors, 2008a: A successful collaboration between research institutions and operational center: Realization of Doppler radar data assimilation with WRF 3D-VAR in KMA operational forecasting. Bull. Amer. Meteor. Soc., 89, 39–43.
Xiao, Q., Y. Kuo, Z. Ma, W. Huang, X. Huang, X. Zhang, D. M. Barker, and J. Michalakes, 2008b: Development of the WRF adjoint modeling system and its application to the investigation of the May 2004 McMurdo Antarctica severe wind event. Mon. Wea. Rev., 136, 3696–3713.
Zhang, M., F. Zhang, X.-Y. Huang, and X. Zhang, 2011: Intercomparison of an ensemble Kalman filter with three- and four-dimensional variational data assimilation methods in a limited-area model over the month of June 2003. Mon. Wea. Rev., 139, 566–572.