1. Introduction
Lorenc (2003) provided a thorough comparison between ensemble Kalman filter (EnKF) and four-dimensional variational data assimilation (4DVar) and proposed a hybrid method with extended control variable formulation for regional mesoscale numerical weather prediction (NWP) systems. Kalnay et al. (2007) and Gustafsson (2007) discussed the advantages and the disadvantages of these two approaches and also concluded that a hybrid method would be beneficial to meteorological data assimilation.
The hybrid ensemble–variational [EnVar as recommended nomenclature in Lorenc (2013)] data assimilation (DA) algorithms employ both the static and ensemble-based, flow-dependent error covariances within a variational framework. For variational data assimilation (Var), full-rank static background error covariances are usually derived with assumptions of isotropy and homogeneity in space and time (Parrish and Derber 1992). On the other hand, the error amplitudes and error structures and correlations among variables with the ensemble-based background error covariances are flow dependent and fully nonlinear without the limitation of a certain shape of the structure or using a simplified projection relationship between variables.
Without using the tangent linear and the adjoint of the forecast model as in 4DVar, the 4DEnVar algorithms are used with an ensemble of nonlinear model integrations over the data assimilation window to provide model guidance (Liu et al. 2008, 2009; Liu and Xiao 2013; Buehner et al. 2010a,b; Desroziers et al. 2014; Wang and Lei 2014; Kleist and Ide 2015b). In this application, the 4D advantage is mainly through the model evolution represented in the ensemble perturbations of each ensemble member. Gustafsson and Bojarova (2014) reported that 4DEnVar for the High Resolution Limited Area Model outperformed 4DVar, its computational costs were significantly reduced in comparison with standard 4DVar, and the scalability of the algorithm was improved.
The 3DEnVar and 4DEnVar approaches have been used within both global and regional frameworks. For example, in regional applications there have been studies of various hybrid algorithms (Wang et al. 2007, 2008a,b; Liu et al. 2009; Zhang and Zhang 2012; Liu and Xiao 2013; Zhang et al. 2013; Schwartz et al. 2013; Schwartz and Liu 2014; Pan et al. 2014; Schwartz et al. 2015; Caron et al. 2015). It has also been demonstrated that 3DEnVar and 4DEnVar DA methods are suitable for operational NWP applications (Hamill et al. 2011; Buehner et al. 2013; Clayton et al. 2013; Kuhl et al. 2013; Wang et al. 2013; Gustafsson and Bojarova 2014; Buehner et al. 2015; Caron et al. 2015). Following Wang (2010) as first proposed by Lorenc (2003), the 3DEnVar was developed in the Gridpoint Statistical Interpolation (GSI) system using an extended control variable at the National Centers for Environmental Prediction (NCEP). The 3DEnVar was implemented in the operational Global Data Assimilation System (GDAS) in May 2012 and was used in studies with the Global Forecast System (GFS) system (Wang et al. 2013; Kleist and Ide 2015a,b). The same 3DEnVar scheme in the GSI is employed in this study for the North American Mesoscale Forecast System (NAM) Data Assimilation System (NDAS) at NCEP.
Xu et al. (2008) and Zhao et al. (2015) showed that the time-expanded sampling method could be used to reduce the number of forecast runs needed to produce an ensemble with the desired sample size. Gustafsson et al. (2014) also used a +2-h lagged valid time ensemble to double the size of the ensemble applied in their 4DVar hybrid experiments. It was reported in both papers that the ensemble perturbations seem to have dynamical consistency and can be used with a time shift. In this study, whether the perturbations have the dynamical consistency to be used in another forecast system will be tested. The flow-dependent error covariances derived from the global ensemble will be used to produce the initial conditions for the regional NAM system with the assumption that they can provide useful information to improve the large-scale components of the regional background error within the DA system.
Because of an improved forecast impact over 3DVar in NCEP’s regional NAM forecast system, a 3DEnVar DA using global EnKF ensemble forecasts from NCEP’s GFS was implemented into NCEP’s regional NDAS on 12 August 2014. A similar approach was later used in the Regional Deterministic Prediction System in Environment Canada (Caron et al. 2015). This paper documents the work of implementing the 3DEnVar DA and the extension to a 4DEnVar in the regional NAM system.
The regional model and DA systems, the 4DEnVar setup, the global ensemble, and the proposed trajectory correction and its associated impact are described in section 2. The comparison of the forecast impact from the 3DVar, 3DEnVar, and 4DEnVar with its final configuration in NDAS is included in section 3. The conclusions and discussion are provided in section 4.
2. Model and DA configuration
a. Forecast model NMMB and NDAS
The NAM is one of the primary forecast systems by which NCEP provides mesoscale guidance to public- and private-sector meteorologists. It is run four times daily at 0000, 0600, 1200, and 1800 UTC using the NOAA Environmental Modeling System (NEMS) version of the Nonhydrostatic Multiscale Model with Arakawa B-grid staggering (NMMB; Janjić et al. 2001). The North American parent domain, as shown in Fig. 1, has a horizontal grid spacing of 12 km and a model top at 2 hPa with 60 vertical levels. The NAM run makes forecasts in its parent domain out to 84 h. There are four fixed nested domains: the continental United States (CONUS), Alaska, Hawaii, and Puerto Rico, with grid spacings of 4, 6, 3, and 3 km, respectively, run to 60 h, which are all one-way nests inside the parent domain. One high-resolution fire weather nest runs to 36 h with grid spacing of 1.33 km if over the CONUS and 1.5 km if over Alaska. The NAM is initialized with a 12-h run of NDAS, which runs a sequence of four GSI (Wu et al. 2002; Kleist et al. 2009a,b) analyses and 3-h NEMS–NMMB forecasts using all available observations to provide a first guess to the NAM “on time” analysis. NDAS 3-h forecasts are initialized with a diabatic digital filter. Figure 2 shows a schematic diagram of NDAS/NAM partial cycling. Each forecast cycle begins with a 12-h analysis–forecast window during which analyses are conducted at 3-h intervals (TM12, TM09, etc.). TM00 refers to the forecast initialization time (e.g., 0000, 0600, 1200, or 1800 UTC). At TM12 the first guess for the atmosphere is a 6-h forecast from GDAS. The land states are cycled from the previous NAM/NDAS cycle (Rogers et al. 2009).
Map showing the parent domain (black line) with 12-km horizontal grid spacing and the nest domains of the regional NAM forecast system. The fire weather nest domain is shown by the magenta line, the continental U.S. domain in red, the AK domain in green, the HI domain in blue, and the PR domain in turquoise.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
A schematic diagram of NDAS/NAM DA cycling. Each forecast cycle begins with a 12-h analysis–forecast window during which analyses are conducted at 3-h intervals (TM12, TM09, etc.). TM00 refers to the forecast initialization time (e.g., 0000, 0600, 1200, or 1800 UTC). At TM12 the first guess for the atmosphere is a 6-h forecast from the GDAS, and the land states are cycled from the previous NAM/NDAS cycle.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
b. Hybrid DA
Vertical weighting function used in the ensemble contribution to the analysis increments of surface pressure.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
For the 3DEnVar in NDAS, Eqs. (2) and (3) are used at the center of the analysis window. With the 4DEnVar for NAM, Eqs. (2) and (3) are applied at each time bin in the assimilation window. While the analysis is available at each time bin with 4DEnVar, the four-dimensional incremental analysis update (4DIAU) with time-varying increments proposed by Buehner et al. (2015) is not used in this application. The analysis at the center of the window is chosen to initialize the NAM free forecast.
The background error covariances currently used in NDAS are derived with the National Meteorological Center (NMC, now known as NCEP) method (Parrish and Derber 1992). Forecast differences are used to estimate the balance projection, the variances, and the error structure statistics for the analysis variables. With the forecasts of the NAM parent domain in Fig. 1, spectral calculations are used in finding the streamfunction and velocity potential from the wind fields. The structure functions are estimated with an autocorrelation method that generates more stable and robust statistics than those with other methods. For the cycle at TM12 in Fig. 2, when the first guess for the atmosphere is a 6-h forecast from the GDAS, global background error statistics derived from global forecasts are employed. The statistics are interpolated to the latitude and height of the regional grid, and the differences in the resolutions are taken into account in the application of the error structure. For the other cycles, from TM09 to TM00 when regional forecasts are used as the first guess, regional background error statistics are used.
The 80-member T574 global EnKF ensemble is used in the NDAS 3DEnVar. The EnVar method is one-way coupled. The analysis increment grid is 3 times coarser (36 km) than the model grid. The analysis increments are interpolated to the original 12-km grid and added back to the deterministic background. The global T574 ensemble is of 35-km grid spacing. Since the global ensemble perturbations used in this study are of similar resolution to the regional analysis, single-resolution EnVar is chosen. Unlike Schwartz et al. (2015), no interpolation operator is used in Eqs. (2) or (3). The ensemble perturbation is interpolated to the regional analysis grid before being used in the minimization.
A slight degradation over 3DVar near the upper and lower boundaries was observed when a 3DEnVar analysis using global ensemble was first applied in NDAS. There are important differences between GFS and the regional NAM related to model resolution, physics, and numerical approximations at the top and bottom of the model. To mitigate the problem, βf and βe are made height dependent in the regional application, as shown in Fig. 4, where βe is 0.5 in the middle of the atmosphere and decreases below (above) 850 (150) hPa to 0.25 at the boundaries. The degradation near the boundaries is eliminated with the vertically varying weighting in 3DEnVar. However, the vertical variation of βf and βe is turned off in the regional 4DEnVar since no obvious degradation is observed without the variation. The reason for this discrepancy may be the stronger constraint to fit the model trajectory within the 4D application.
The weighting function for hybrid covariance; static βf (black) and ensemble βe (blue) used in NCEP’s regional 3DEnVar.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
c. 3DEnVar and 4DEnVar setup
For the 3DEnVar application, T = 1 in Eq. (1) and t = 1 in Eqs. (2) and (3) are employed. FGAT is not used in the current operational NDAS with 3DEnVar because it produces no impact on forecast quality. For each of the five analysis cycles in Fig. 2, a set of 80-member T574 global EnKF ensemble forecasts and a regional first guess valid at the analysis time are needed. The tangent-linear normal-mode constraint as described in Kleist et al. (2009a) is not utilized in the regional mode.
When Eq. (1) is applied to 4DEnVar, the static part of the solution 
Time evolution of the domain-averaged 24-h forecast RMSEs from experiments with hourly bins (green) and 3-hourly bins (red) against radiosonde observations for forecasts initialized at 0000 UTC during 8 Apr–3 May 2015 of (a) vector wind (m s−1), (b) temperature (K), (c) relative humidity (%), and (d) surface pressure (hPa). The time means are also shown in insets.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
Figure 6 shows a schematic diagram of the final 4DEnVar setup in NDAS. Each forecast cycle begins with a 9-h NMMB forecast from the previous cycle. The 3-, 6-, and 9-h forecasts and corresponding global EnKF ensemble are used at T − 3, T, and T + 3 of the assimilation window. The 4DEnVar analysis valid at the center of the assimilation window is used as the initial conditions for the 84-h NAM forecast. To run 4DEnVar, it is necessary to apply FGAT, which allows the observation innovation to be evaluated in each time bin of the assimilation window. For the regional 4DEnVar experiments, a better forecast impact was found in the experiments using the 3-, 6-, and 9-h forecasts (see Fig. 6) from the previous cycle as the first guess than those using three 3-h forecasts produced during the 3-hourly cycles (see Fig. 2) of NDAS assimilation.
The 4DEnVar DA diagram for NDAS. Each forecast cycle begins with a 9-h NMMB forecast from the previous cycle. The 3-, 6-, and 9-h forecasts and corresponding global EnKF ensemble are used at T − 3, T, and T + 3 of the 4DEnVar assimilation window. The analysis valid at the center of the assimilation window is used as the initial conditions for the 84-h NAM forecast.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
The forecast accuracies for the 3DEnVar and 4DEnVar approaches are sensitive to the scale of the horizontal localization. For 4DEnVar, horizontal localization with a Gaussian scale of 800 km is found to have the optimal impact on the NAM forecasts while 300 km is used for 3DEnVar in the operational NDAS. For vertical localization a recursive filter with a scale of five model grid spaces is used in both 3DEnVar and 4DEnVar. A different partitioning between the static and the ensemble background error covariances is used for 4DEnVar (βe = 0.75) as compared with 3DEnVar (βe = 0.5). The choices are based on the impact to the quality of the NAM forecasts.
All the observations used in the operational NDAS are used in the experiments reported upon in this study. The types of the conventional data are available online in a table showing the PREPBUFR report type for regional GSI (EMC 2015). Another table showing the current use of data not passing through PREPBUFR processing (EMC 2016) includes satellite radiances, satellite-retrieved products (sea surface temperature, sea surface wind, and atmospheric motion winds), radar, and GPS radio occultation.
d. Global ensemble
An 80-member global EnKF ensemble with T574 resolution is used in the operational NDAS with 3DEnVar. Table 1 shows the experiments designed to test the impact of various versions of the global ensemble used with regional 4DEnVar. There are the operationally generated 80-member T574 global EnKF ensemble with a 3DEnVar recentering and an experimental global EnKF ensemble with 4DEnVar recentering. The recentering refers to the global EnKF ensemble mean being replaced every cycle by the global EnVar analysis. A recentering procedure ensures that the EnKF analysis ensemble is always centered on the hybrid EnVar analysis to maintain synergy between the EnKF and hybrid EnVar analyses (Wang et al. 2013; Kleist and Ide 2015a). The control was initiated from the regional 3DEnVar. The regional 4DEnVar with a global ensemble using EnKF from the operational GDAS with 3DEnVar recentering was used in the first experiment. The second and third experiments were with T254 and T574 using EnKF in an experimental GDAS with 4DEnVar recentering. The domain-mean RMSEs compared with conventional data for the 24-h forecasts are included in Table 2. Using the EnKF ensemble with the 4DEnVar recentered from GDAS in experiments T254_4d and T574_4d produced no forecast advantage over using the ensemble with the 3DEnVar recentered in experiment T574_op. It also shows little impact from using the full-resolution ensemble in experiment T574_4d compared with experiment T254_4d. This result seems to suggest that the impact from the global ensemble on the regional 4DEnVar DA system is mostly at the larger scales of the background error covariances, which are resolved in both the truncated T254 and full-resolution T574 ensembles. The same finding was reached by Schwartz et al. (2015), that EnVar with a lower-resolution ensemble can initialize forecasts at a quality similar to EnVar with a full-resolution ensemble. The setup as in experiment T574_op is used in the following experiments.
Description of experiments with NDAS EnVar at 0000 UTC 4 Feb 2015.
The 24-h forecast domain-averaged RMSEs for the experiments in Table 1 compared with conventional observations of surface pressure, relative humidity, temperature, and vector wind.
e. Trajectory correction
In the original GSI EnVar, only the ensemble perturbations are used, and the ensemble mean is disregarded. The trajectory of the ensemble mean can be different from that of the first guess, and the information in the ensemble mean can potentially be beneficial to the EnVar DA system. For a regional system using a global ensemble, the domain sizes, model resolutions, physics parameterizations and numerical methods employed in each system can all contribute to the differences between the regional forecast used as a first guess and the global ensemble mean. Even for a “two way” coupled system, the trajectory of the low-resolution ensemble mean could also be deviated from the high-resolution deterministic forecast. For example, the recentering procedure is needed in the global EnKF/EnVar system in NCEP. To explicitly make use of the information in the ensemble mean, trajectory correction (TC) was added to GSI as an option in the regional EnVar system. When the TC option is turned on, an extra member is added to the ensemble. The perturbation for this extra member is the difference between the global ensemble mean and the regional first guess. In 3DEnVar, TC enables the solution to be drawn, locally, close to the global ensemble mean. In 4DEnVar, TC enables the solution in the assimilation window to be locally close to the trajectory of the global ensemble mean.
The α for this member is given the same weight (the same error covariance) as the α for any of the other 80 ensemble members. The influence of the extra member (the magnitude of αn) on the analysis is determined by the solution of Eq. (1). Since localization is applied within the EnVar approach, TC is also local. Within the localization distance, if the perturbations associated with this extra member fit well with the innovations throughout the assimilation window, αn for the member could be close to one. By adding the increments of this member back to the first guess, the resulting analysis would be close to the global ensemble mean. Note that this process does not forcefully correct the regional analysis toward the global ensemble mean. The correction happens automatically when Eq. (1) is solved. The addition of the extra member with TC is made just to improve the coverage of the ensemble in phase space (to enrich the ensemble).
Figure 7 shows the time evolution of the domain-averaged 24-h forecast RMSE against the radiosonde observations from experiments using 3DEnVar with TC (3DTC) and 3DEnVar without TC (3DNTC). Although the TC produced a limited impact in the regional 3DEnVar, the benefits to the wind forecasts are consistently positive in Fig. 7a. The time evolution of domain-averaged 24-h forecast RMSEs for forecasts initialized from 4DEnVar with TC (4DTC), and from 4DEnVar without TC (4DNTC), is depicted in Fig. 8. The data assimilation and forecast experiment was carried out for the period 3 January–8 February 2016. The positive impact on all the variables from TC is more accentuated for the 4DEnVar than for the 3DEnVar. Note that the RMSEs in Figs. 7 and 8 represent 3D (2D for surface pressure) domain averages, and the RMSEs have significant spatial variations. To further illustrate this, we present a vertical profile of horizontally domain-averaged 24-h forecast RMSEs of wind to radiosondes for one randomly selected case: 0000 UTC 14 January 2016 (time 14 in Fig. 8a) in Fig. 9. The result of 3DTC is also included as reference. The impact from applying 4DEnVar (4DTC) as opposed to 3DEnVar (3DTC) is positive at most pressure levels except at 950 hPa, with the maximum benefit seen between 500 and 150 hPa. The impact from using TC (4DTC versus 4DNTC) is positive at all pressure levels and is most substantial in the mid- to upper troposphere. The horizontally averaged fit to radiosonde winds is improved by 0.9 m s−1 at 275 hPa from using 4DEnVar instead of 3DEnVar. Note that the impact of adding the extra member onto the existing 80 members with TC is nearly half of that magnitude (0.4 m s−1, comparing 4DNTC and 4DTC) at that level. With the encouraging results, TC is used in NDAS parallel tests for EnVar reported in the next section.
Time evolution of the domain-averaged 24-h forecast RMSEs from the 3DTC (blue) and 3DNTC (black) experiments against radiosonde observations for forecasts initialized at 0000 UTC during 3–22 Dec 2015 of (a) vector wind (m s−1), (b) temperature (K), (c) relative humidity (%), and (d) surface pressure (hPa). The time means are shown in insets.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
As in Fig. 7, but from the 4DTC (red) and 4DNTC (black) experiments, for 3 Jan–8 Feb 2016.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
The 24-h forecast RMSEs of vector wind (m s−1) against radiosondes for the 14 Jan 2016 case. The black line represents the forecast initialized with 3DEnVar with trajectory correction, the green line 4DEnVar with trajectory correction, and the blue line 4DEnVar without trajectory correction.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
It was shown in Gustafsson et al. (2014) that the richness in spatial structures of the ensemble is more important than the scaling of the amplitude of the ensemble perturbations in experiments with HIRLAM 4DVar Hybrid assimilation. The proposed TC in this study provides a way to enrich the ensemble. Our results with NAM 3D and 4DEnVar assimilation indicate that the forecast quality is improved with TC, which confirms the importance of the richness in the spatial structure of the ensemble perturbations to the hybrid ensemble assimilations.
3. Forecast impact with 3DEnVar and 4DEnVar
The final configurations of the 3DEnVar analysis systems were tested in regional NDAS parallels. The geopotential height RMSE of the preimplementation parallel tests of 3DEnVar, averaged from 25 April to 6 June 2012, are shown in Fig. 10. The 3DEnVar has a consistent and substantial positive impact at all levels over the 3DVar in the control with regard to the 24-, 48-, and 72-h forecast quality. A similar impact was observed in all variables (not shown). Consequently, the 3DEnVar analysis system was implemented in the operational NDAS on 12 August 2014.
Cumulative RMSEs 24-h (black), 48-h (red), and 72-h (blue) forecast errors vs radiosondes for geopotential height (m) over the CONUS for 25 Apr–6 Jun 2012. Solid lines are the control with 3DVar and dashed lines are experiments with 3DEnVar.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
The regional 4DEnVar was applied in an NDAS parallel while a regional parallel with 3DEnVar served as the control. Although the overall 4DEnVar impact was consistently positive over 3DEnVar during the 5-month (15 June–15 November 2015) test, the magnitude of the improvement was more significant during the cold-season months. This might be attributed to the fact that the background error covariance from the global ensemble provides more meaningful error statistics during the winter season because of the additional baroclinicity in the extratropics. The impact from using 4DEnVar to initiate the forecasts compared with using a 3DEnVar analysis in NDAS is depicted in Fig. 11. The forecast RMSEs against radiosonde observations of geopotential height, wind, temperature, and specific humidity averaged for 2 months (from 15 September 2015 to 15 November 2015) are shown. The fit to the height in Fig. 11a is neutral at 24 h and better for 4DEnVar parallel at 48 and 72 h everywhere except near the top of the model domain when compared with the 3DEnVar. The fit to the wind observations in Fig. 11b for the forecasts initiated during the 4DEnVar analysis is better at all layers and all lead times than that from the 3DEnVar approach. The most notable benefit is realized for 48-h forecasts between 200 and 300 hPa. A positive impact on the temperature in Fig. 11c is found at all levels, and the largest improvement over 3DEnVar is realized in the middle of the troposphere. The impact on the specific humidity in Fig. 11d is also positive at all layers. Equitable threat scores and bias scores for day 3 precipitation over the continental United States for the same two months are shown in Fig. 12. In general, the experiment with 4DEnVar has better equitable threat scores and lower biases than that with 3DEnVar.
Cumulative RMSEs 24-h (black), 48-h (red), and 72-h (blue) forecast errors vs radiosondes for (a) geopotential height (m), (b) vector wind (m s−1), (c) temperature (K), and (d) specific humidity fit (g kg−1) over the CONUS for 15 Sep–15 Nov 2015. Solid lines are the control with 3DEnVar and dashed lines are experiments with 4DEnVar.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
Equitable threat scores and bias scores from the 4DEnVar (blue dashed lines) and 3DEnVar (red solid lines) experiments for day 3 precipitation over the CONUS for 15 Sep–15 Nov 2015. The box plot denotes the 95% confidence interval.
Citation: Weather and Forecasting 32, 1; 10.1175/WAF-D-16-0045.1
4. Conclusions and discussion
In this paper we describe the work undertaken to use the global EnKF ensemble in the regional NDAS with the EnVar algorithm at NCEP. Three new features are added to the existing EnVar system in the GSI, namely, the vertically varying weighting coefficients (βf and βe) for the static and the ensemble partitioning, the vertically integrated contribution of the ensemble coefficient α to the surface pressure analysis, and TC. The vertical varying βf and βe are necessary because we use the ensemble perturbations from the global system, which has very different vertical model grid spacing than that in the NAM system. However, the feature is only used in the 3DEnVar application but not in 4DEnVar. The second feature is from the physical consideration that changes in surface pressure are tied to changes in the column mass and dynamics in the free atmosphere. With TC, the third feature, both the perturbations, and the ensemble mean from the global EnKF system contribute to the regional analysis through the EnVar algorithm. The difference between the first guess and the ensemble mean is added as the perturbation of an extra member to the ensemble. TC enriches the ensemble by improving the coverage in phase space and allows the analysis with EnVar to be drawn locally close to the ensemble mean. The proposed TC is found to be beneficial, especially with 4DEnVar.
The results of parallel tests in NDAS over extended periods demonstrated that the ensemble from the global GFS/EnKF system could be used to improve the regional NAM analysis with an EnVar system. The regional EnVar with TC effectively combined contributions from both the global and the regional systems to produce initial conditions for the NAM. With a substantially positive impact on the forecasts over those initiated with 3DVar, 3DEnVar with a global EnKF ensemble was implemented in August 2014, and the parallel test results indicated a 4D extension of the EnVar system provided further improvement to the forecasts.
A possible extension of this work is to apply EnVar with TC to combine a superensemble: an ensemble consisting of forecasts from more than one forecast system to produce initial conditions. In the future, in addition to the global EnKF ensemble, both the perturbations and the ensemble means from the regional ensemble (i.e., NCEP’s Short-Range Ensemble Forecast) will be used in the regional EnVar to help define the smaller-scale part of the background error covariances.
Acknowledgments
The work was completed as part of the authors’ job responsibilities at the National Centers for Environmental Prediction. We thank the support of the National Oceanic and Atmospheric Administration/National Weather Service. The authors thank Daryl Kleist, Rahul Mahajan, Russ Treadon, and Jeff Whitaker, whose contributions made this work possible. We express our appreciation to David Groff, Rahul Mahajan, and Mary Hart, whose comments helped to improve the paper. WSW thanks Eugenia Kalnay for the idea of using ensemble perturbations in data assimilation. Qin Xu and two anonymous reviewers helped to significantly improve the manuscript.
REFERENCES
Buehner, M., Houtekamer P. L. , Charette C. , Mitchell H. L. , and He B. , 2010a: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part I: Description and single-observation experiments. Mon. Wea. Rev., 138, 1550–1566, doi:10.1175/2009MWR3157.1.
Buehner, M., Houtekamer P. L. , Charette C. , Mitchell H. L. , and He B. , 2010b: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part II: One-month experiments with real observations. Mon. Wea. Rev., 138, 1567–1586, doi:10.1175/2009MWR3158.1.
Buehner, M., Morneau J. , and Charette C. , 2013: Four-dimensional ensemble-variational data assimilation for global deterministic weather prediction. Nonlinear Processes Geophys., 20, 669–682, doi:10.5194/npg-20-669-2013.
Buehner, M., and Coauthors, 2015: Implementation of deterministic weather forecasting systems based on ensemble–variational data assimilation at Environment Canada. Part I: The global system. Mon. Wea. Rev., 143, 2532–2559, doi:10.1175/MWR-D-14-00354.1.
Caron, J.-F., Milewski T. , Buehner M. , Fillion L. , Reszka M. , Macpherson S. , and St-James J. , 2015: Implementation of deterministic weather forecasting systems based on ensemble–variational data assimilation at Environment Canada. Part II: The regional system. Mon. Wea. Rev., 143, 2560–2580, doi:10.1175/MWR-D-14-00353.1.
Clayton, A. M., Lorenc A. C. , and Barker D. M. , 2013: Operational implementation of a hybrid ensemble/4D-Var global data assimilation system at the Met Office. Quart. J. Roy. Meteor. Soc., 139, 1445–1461, doi:10.1002/qj.2054.
Desroziers, G., Camino J.-T. , and Berre L. , 2014: 4DEnVar: Link with 4D state formulation of variational assimilation and different possible implementations. Quart. J. Roy. Meteor. Soc., 140, 2097–2110, doi:10.1002/qj.2325.
EMC, 2015: Code table for PREPBUFR report types used by Regional NAM and NDAS GSI analyses. NOAA/NCEP/Environmental Prediction Center. [Available online at http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_4.htm.]
EMC, 2016: Summary of the current NCEP GFS-GSI, NAM-GSI and RAP-GSI analysis system usage of data types that do not pass through PREPBUFR processing. NOAA/NCEP/Environmental Prediction Center. [Available online at http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_18.htm.]
Gustafsson, N., 2007: Discussion on ‘4D-Var or EnKF.’ Tellus, 59A, 774–777, doi:10.1111/j.1600-0870.2007.00262.x.
Gustafsson, N., and Bojarova J. , 2014: Four-dimensional ensemble variational (4D-En-Var) data assimilation for the High Resolution Limited Area Model (HIRLAM). Nonlinear Processes Geophys., 21, 745–762, doi:10.5194/npg-21-745-2014.
Gustafsson, N., Bojarova J. , and Vignes O. , 2014: A hybrid variational ensemble data assimilation for the High Resolution Limited Area Model (HIRLAM). Nonlinear Processes Geophys., 21, 303–323, doi:10.5194/npg-21-303-2014.
Hamill, T. M., Whitaker J. S. , Kleist D. T. , Fiorino M. , and Benjamin S. G. , 2011: Predictions of 2010’s tropical cyclones using the GFS and ensemble-based data assimilation methods. Mon. Wea. Rev., 139, 3243–3247, doi:10.1175/MWR-D-11-00079.1.
Janjić, Z. I., Gerrity J. P. Jr., and Nickovic S. , 2001: An alternative approach to nonhydrostatic modeling. Mon. Wea. Rev., 129, 1164–1178, doi:10.1175/1520-0493(2001)129<1164:AAATNM>2.0.CO;2.
Kalnay, E., Li H. , Miyoshi T. , Yang S.-C. , and Ballabrera-Poy J. , 2007: 4D-Var or ensemble Kalman filter? Tellus, 59A, 758–773, doi:10.1111/j.1600-0870.2007.00261.x.
Kleist, D. T., and Ide K. , 2015a: An OSSE-based evaluation of hybrid variational–ensemble data assimilation for the NCEP GFS. Part I: System description and 3D-hybrid results. Mon. Wea. Rev., 143, 433–451, doi:10.1175/MWR-D-13-00351.1.
Kleist, D. T., and Ide K. , 2015b: An OSSE-based evaluation of hybrid variational–ensemble data assimilation for the NCEP GFS. Part II: 4D EnVar and hybrid variants. Mon. Wea. Rev., 143, 452–470, doi:10.1175/MWR-D-13-00350.1.
Kleist, D. T., Parrish D. F. , Derber J. C. , Treadon R. , Errico R. M. , and Yang R. , 2009a: Improving incremental balance in the GSI 3DVAR analysis system. Mon. Wea. Rev., 137, 1046–1060, doi:10.1175/2008MWR2623.1.
Kleist, D. T., Parrish D. F. , Derber J. C. , Treadon R. , Wu W.-S. , and Lord S. J. , 2009b: Introduction of the GSI into the NCEP Global Data Assimilation System. Wea. Forecasting, 24, 1691–1705, doi:10.1175/2009WAF2222201.1.
Kuhl, D. D., Rosmond T. E. , Bishop C. H. , McClay J. , and Baker N. L. , 2013: Comparison of hybrid ensemble/4DVar and 4DVar within the NAVDAS-AR data assimilation framework. Mon. Wea. Rev., 141, 2740–2758, doi:10.1175/MWR-D-12-00182.1.
Liu, C., and Xiao Q. , 2013: An ensemble-based four-dimensional variational data assimilation scheme. Part III: Antarctic applications with Advanced Research WRF using real data. Mon. Wea. Rev., 141, 2721–2739, doi:10.1175/MWR-D-12-00130.1.
Liu, C., Xiao Q. , and Wang B. , 2008: An ensemble-based four-dimensional variational data assimilation scheme. Part I: Technical formulation and preliminary test. Mon. Wea. Rev., 136, 3363–3373, doi:10.1175/2008MWR2312.1.
Liu, C., Xiao Q. , and Wang B. , 2009: An ensemble-based four-dimensional variational data assimilation scheme. Part II: Observing system simulation experiments with the Advanced Research WRF (ARW). Mon. Wea. Rev., 137, 1687–1704, doi:10.1175/2008MWR2699.1.
Lorenc, A. C., 2003: The potential of the ensemble Kalman filter for NWP—A comparison with 4D-VAR. Quart. J. Roy. Meteor. Soc., 129, 3183–3203, doi:10.1256/qj.02.132.
Lorenc, A. C., 2013: Recommended nomenclature for EnVar data assimilation methods. Research Activities in Atmospheric and Oceanic Modeling, A. Zadra, Ed., WGNE, 7–8. [Available online at http://www.wcrp-climate.org/WGNE/BlueBook/2013/individual-articles/01_Lorenc_Andrew_EnVar_nomenclature.pdf.]
Pan, Y., Zhu K. , Xue M. , Wang X. , Hu M. , Benjamin S. G. , Weygandt S. S. , and Whitaker J. S. , 2014: A GSI-based coupled EnSRF–En3DVar hybrid data assimilation system for the operational Rapid Refresh model: Tests at a reduced resolution. Mon. Wea. Rev., 142, 3756–3780, doi:10.1175/MWR-D-13-00242.1.
Parrish, D. F., and Derber J. C. , 1992: The National Meteorological Center’s spectral statistical-interpolation system. Mon. Wea. Rev., 120, 1747–1763, doi:10.1175/1520-0493(1992)120<1747:TNMCSS>2.0.CO;2.
Purser, R. J., Wu W.-S. , Parrish D. F. , and Roberts N. M. , 2003a: Numerical aspects of the application of recursive filters to variational statistical analysis. Part I: Spatially homogeneous and isotropic Gaussian covariances. Mon. Wea. Rev., 131, 1524–1535, doi:10.1175//1520-0493(2003)131<1524:NAOTAO>2.0.CO;2.
Purser, R. J., Wu W.-S. , Parrish D. F. , and Roberts N. M. , 2003b: Numerical aspects of the application of recursive filters to variational statistical analysis. Part II: Spatially inhomogeneous and anisotropic general covariances. Mon. Wea. Rev., 131, 1536–1548, doi:10.1175//2543.1.
Rogers, E., and Coauthors, 2009: The NCEP North American Mesoscale modeling system: Recent changes and future plans. Preprints, 23rd Conf. on Weather Analysis and Forecasting/19th Conf. on Numerical Weather Prediction, Omaha, NE, Amer. Meteor. Soc., 2A.4. [Available online at http://ams.confex.com/ams/pdfpapers/154114.pdf.]
Schwartz, C. S., and Liu Z. , 2014: Convection-permitting forecasts initialized with continuously cycling limited-area 3DVAR, ensemble Kalman filter, and “hybrid” variational–ensemble data assimilation systems. Mon. Wea. Rev., 142, 716–738, doi:10.1175/MWR-D-13-00100.1.
Schwartz, C. S., Liu Z. , Huang X.-Y. , Kuo Y.-H. , and Fong C.-T. , 2013: Comparing limited-area 3DVAR and hybrid variational-ensemble data assimilation methods for typhoon track forecasts: Sensitivity to outer loops and vortex relocation. Mon. Wea. Rev., 141, 4350–4372, doi:10.1175/MWR-D-13-00028.1.
Schwartz, C. S., Liu Z. , and Huang X.-Y. , 2015: Sensitivity of limited-area hybrid variational–ensemble analyses and forecasts to ensemble perturbation resolution. Mon. Wea. Rev., 143, 3454–3477, doi:10.1175/MWR-D-14-00259.1.
Wang, X., 2010: Incorporating ensemble covariance in the Gridpoint Statistical Interpolation (GSI) variational minimization: A mathematical framework. Mon. Wea. Rev., 138, 2990–2995, doi:10.1175/2010MWR3245.1.
Wang, X., and Lei T. , 2014: GSI-based four-dimensional ensemble–variational (4DEnsVar) data assimilation: Formulation and single-resolution experiments with real data for the NCEP Global Forecast System. Mon. Wea. Rev., 142, 3303–3325, doi:10.1175/MWR-D-13-00303.1.
Wang, X., Snyder C. , and Hamill T. M. , 2007: On the theoretical equivalence of differently proposed ensemble–3DVAR hybrid analysis schemes. Mon. Wea. Rev., 135, 222–227, doi:10.1175/MWR3282.1.
Wang, X., Barker D. M. , Snyder C. , and Hamill T. M. , 2008a: A hybrid ETKF–3DVar data assimilation scheme for the WRF model. Part I: Observing system simulation experiment. Mon. Wea. Rev., 136, 5116–5131, doi:10.1175/2008MWR2444.1.
Wang, X., Barker D. M. , Snyder C. , and Hamill T. M. , 2008b: A hybrid ETKF–3DVar data assimilation scheme for the WRF model. Part II: Real observation experiments. Mon. Wea. Rev., 136, 5132–5147, doi:10.1175/2008MWR2445.1.
Wang, X., Parrish D. , Kleist D. , and Whitaker J. , 2013: GSI 3DVar-based ensemble–variational hybrid data assimilation for NCEP Global Forecast System: Single-resolution experiments. Mon. Wea. Rev., 141, 4098–4117, doi:10.1175/MWR-D-12-00141.1.
Wu, W.-S., Parrish D. F. , and Purser R. J. , 2002: Three-dimensional variational analysis with spatially inhomogeneous covariances. Mon. Wea. Rev., 130, 2905–2916, doi:10.1175/1520-0493(2002)130<2905:TDVAWS>2.0.CO;2.
Xu, Q., Wei L. , Lu H. , Qiu C. , and Zhao Q. , 2008: Time-expanded sampling for ensemble-based filters: Assimilation experiments with a shallow-water equation model. J. Geophys. Res., 113, D02114, doi:10.1029/2007JD008624.
Zhang, F., Zhang M. , and Poterjoy J. , 2013: E3DVar: Coupling an ensemble Kalman filter with three-dimensional variational data assimilation in a limited-area weather prediction model and comparison to E4DVar. Mon. Wea. Rev., 141, 900–917, doi:10.1175/MWR-D-12-00075.1.
Zhang, M., and Zhang F. , 2012: E4DVar: Coupling an ensemble Kalman filter with four-dimensional variational data assimilation in a limited-area weather prediction model. Mon. Wea. Rev., 140, 587–600, doi:10.1175/MWR-D-11-00023.1.
Zhao, Q., Xu Q. , Jin Y. , McLay J. , and Reynolds C. , 2015: Time-expanded sampling for ensemble-based data assimilation applied to conventional and satellite observations. Wea. Forecasting, 30, 855–872, doi:10.1175/WAF-D-14-00108.1.
