1. Introduction
Gridpoint statistical interpolation (GSI; Wu et al. 2002; Kleist et al. 2009), a three-dimensional variational data assimilation method (3DVAR) has been widely used in operations and research in numerical weather prediction (NWP). GSI is capable of ingesting a large variety of atmospheric observations and has developed capabilities for data thinning, quality control, and satellite radiance bias correction. The operational GSI uses a static background error covariance, which does not reflect the flow-dependent error statistics. The anisotropic recursive filter (e.g., Purser et al. 2003a,b) provides the possibility to introduce anisotropic and inhomogeneous covariances. How to determine the parameters used in the anisotropic recursive filter to realistically represent varying background error covariance, however, still remains an issue.
A four-dimensional variational data assimilation system (4DVAR) implicitly includes a time-evolving covariance model through the evolution of initial errors under tangent linear dynamics (Lorenc 2003). However, the evolved, flow-dependent covariance model may still be limited by the usage of a static covariance model at the beginning of each 4DVAR cycle (Buehner et al. 2010a,b).
On the other hand, the ensemble Kalman filter (EnKF) has been extensively explored as an alternative data assimilation method (e.g., Houtekamer et al. 2005; Whitaker et al. 2008; Szunyogh et al. 2005; Liu et al. 2008; Torn and Hakim 2008; Zhang et al. 2009; Aksoy et al. 2009; Jung et al. 2008). The main advantage of the EnKF is that it can conveniently provide, through ensemble covariance, flow-dependent estimate of the background error covariance, and therefore the background forecast and the observations are more appropriately weighted during the assimilation.
A hybrid method was proposed (e.g., Hamill and Snyder 2000; Lorenc 2003; Zupanski 2005; Wang et al. 2007a) and has been developed and implemented recently for real NWP models (e.g., Buehner 2005; Wang et al. 2008a,b; Buehner et al. 2010a,b). The hybrid method combines the advantages of both the variational method and the EnKFs and therefore demonstrates great potential for future research and operations. In the hybrid method, the variational framework is used to conduct data assimilation within which the flow-dependent ensemble covariances are effectively incorporated to estimate the background error covariance. Studies using simple models and full NWP models have demonstrated the potential of the hybrid method as compared with a standalone variational method (VAR; e.g., Hamill and Snyder 2000; Wang et al. 2007b, 2008a,b, 2009; Buehner et al. 2010a,b). Recent studies also suggest potential advantages of the hybrid method in comparison with a stand-alone EnKF. For example, Wang et al. (2007b, 2009) suggested that for large-scale application the hybrid EnKF–3DVAR was more robust than the EnKF for small ensemble size and large model errors since the static covariance used in 3DVAR helped to reduce the sampling errors, which indicates smaller ensemble size may be needed by the hybrid than by the EnKFs to achieve similar performance. This is particularly attractive for operational forecasts for which the size of the ensemble may be constrained by the availability of the computational resources and the timing of the forecasts. Studies by Buehner et al. (2010a,b) suggested the 4DVAR-based hybrid is better than the stand-alone EnKF, likely because the temporal evolution of the background error covariance was in a space with much higher dimension by the 4DVAR-based hybrid method than the EnKF. Another advantage of the hybrid is that it is built based on the existing operational variational framework so that the established capability in VAR can be easily adopted. In addition, since the hybrid adopts the variational framework, the dynamic constraint can be conveniently added during the data assimilation (Kleist et al. 2009). In the current implementation of the hybrid method, the ensemble covariance is incorporated in the variational framework through the extended control variable method (Lorenc 2003; Buehner 2005; Wang et al. 2007a, 2008a), and the ensemble covariance localization is conducted in the model state variable space (i.e., model space localization); therefore, no assumption about the explicit position of the observation is required during this procedure of the covariance localization. For widely used EnKFs, explicit positions of the observations are needed to apply covariance localization (so-called observation space localization; Hamill et al. 2001). For satellite radiances, for which there is no explicit vertical position, such observation space localization is thus inappropriate. Results in Campbell et al. (2010) suggested that model space localization such as used in the hybrid framework is more sensible and effective in assimilating satellite radiances.
In an operational variational data assimilation system, the background error covariance and its inversion are never explicitly formed during the minimization because of the large dimension of the problem. Incorporating the ensemble covariance therefore cannot be conducted by a simple weighted sum of the static and ensemble covariances. For VAR that is preconditioned upon the square root of the background error covariance, Buehner (2005) and Wang et al. (2008a,b) described a method to modify the variational minimization to incorporate the ensemble covariance. Different from Buehner (2005) and Wang et al. (2008a,b), in the widely used GSI the minimization procedure is preconditioned upon the full background error covariance (Derber and Rosati 1989). A framework to define how to incorporate the ensemble covariance in GSI is therefore needed. The goal of this paper is to provide a mathematical derivation demonstrating this framework. Such a framework can be applied in the variational system with similar preconditioning. Given the popularity of the GSI and encouraging results from previous work on hybrid data assimilation, this paper represents the first step toward developing and investigating the hybrid data assimilation method based on the GSI.
Section 2 briefly summarizes the key components in GSI minimization, and section 3 provides mathematical details on how ensemble covariance is incorporated in the GSI minimization. Section 4 suggests using the existing GSI capabilities to conduct ensemble covariance localization and discusses ways to reduce computational cost. Section 5 summarizes the paper and provides further discussion.
2. The GSI minimization algorithm
3. Incorporating ensemble covariance in GSI minimization
The goal of this section is to provide a mathematical framework to show how the GSI minimization described in section 2 will be modified to incorporate the ensemble covariance as part of the background error covariance. The idea is to follow the same preconditioning of the original GSI. We first briefly summarize the changes needed in the GSI cost function, following the method of Wang et al. 2008a. Then the derivation on how changes need to be made to fit the GSI minimization outlined in Eqs. (2)–(4) is given.
As described by Eqs. (2)–(4), the GSI minimization is conducted through the conjugate gradient method preconditioned on the full background error covariance. Next we derive that to minimize the new cost function in Eq. (6) we can follow the same minimization procedure used in the original GSI. The key of the new derivation is that the minimization of the new cost function can be preconditioned in the same way as shown in Eqs. (2)–(4). In other words, the same conjugate gradient minimization procedure from the original GSI will be followed, except that we will just need to extend the control variable and extend the background error covariance. Note also that the second term in Eq. (6) is preconditioned on 𝗔 and not on the ensemble covariance. In other words, similar to the fact that the preconditioning for the first term in Eq. (6) is with respect to the static covariance, the added second term in Eq. (6) is preconditioned with respect to 𝗔, which plays a role of constraining the covariance of extended control variables that is similar to that of the static covariance to the original control variables.
4. Modeling of the error covariance of the extended control variable 𝗔 and thoughts to reduce computational cost
In Eq. (18), the background error covariance consists of two components, the original 𝗕1 and the error covariance 𝗔 that constrains the extended control variable. As discussed in section 3, effectively 𝗔 conducts the covariance localization on the ensemble covariance. In the original GSI, 𝗕1 is approximated by the recursive filter transform (Hayden and Purser 1995; Wu et al. 2002; Purser et al. 2003a,b). The recursive filter capability in GSI can also be used to approximate 𝗔, which is different from Buehner (2005) in which the correlation was modeled with a truncated spectral expansion. Note that 𝗔 is a block diagonal matrix with K identical submatrices. Each submatrix will be approximated by the same recursive filter transform. As discussed in Wang et al. 2008a, the parameters in the recursive filter will determine the correlation length scale in 𝗔 and therefore prescribe the covariance localization length scale for the ensemble covariance.
The number of extended control variables is equal to the dimension of the model space where the covariance localization is applied times the number of ensemble members. Because with a relatively long localization scale the extended control variables a are spatially smooth fields, the extended control variables and the recursive filter transform associated with 𝗔 can both be on a coarser grid. A large savings in computational cost is expected without loss of accuracy. A similar idea was suggested by Wang et al. (2008a) and was adopted by Buehner (2005) in which the extended control variable was in a severely truncated spectral space and by Yang et al. (2009) in the context of the local ensemble transform Kalman filter. Wang et al. (2007c) showed that the number of iterations during the minimization in the hybrid system for the Weather and Forecasting Model (WRF) in which the ensemble covariance is incorporated is less than that of the original WRF variational system in which the static background error covariance is used, suggesting a faster convergence in the minimization of the hybrid system in which ensemble covariance and static covariance were assigned zero weights than in the original VAR system with only the static covariance. They also found that the number of iterations is not sensitive to the length of the covariance localization scales. Such tests will be conducted for the GSI-based hybrid data assimilation system.
5. Summary and discussion
The popularity of GSI and the potential advantages of the hybrid ensemble–GSI relative to GSI and a pure ensemble Kalman filter suggest a need to develop the hybrid ensemble–GSI system. Different from the hybrid system in Buehner (2005) and Wang et al. (2008a,b), in the widely used GSI the minimization procedure is preconditioned upon the full background error covariance (Derber and Rosati 1989). A framework to define how to incorporate the ensemble covariance in GSI is therefore provided. Possible ways to reduce computational cost are also discussed. The method can be applied in the variational system with similar preconditioning.
The improvement in the accuracy of the analysis obtained from a hybrid method over a standard variational approach may depend on how accurately the short-range ensemble forecasts used in the hybrid estimate the flow-dependent forecast error covariance. The current operational ensemble may not optimize for this application, and alternative ensemble-generation methods such as the ensemble Kalman filter–based method may need to be explored.
Acknowledgments
The author was supported by a new-faculty startup award from the University of Oklahoma. The author also greatly acknowledges discussions with Dave Parrish and the valuable comments by Mark Buehner and an anonymous reviewer that helped to improve the manuscript.
REFERENCES
Aksoy, A., D. C. Dowell, and C. Snyder, 2009: A multicase comparative assessment of the ensemble Kalman filter for assimilation of radar observations. Part I: Storm-scale analyses. Mon. Wea. Rev., 137 , 1805–1824.
Buehner, M., 2005: Ensemble-derived stationary and flow-dependent background-error covariances: Evaluation in a quasi-operational NWP setting. Quart. J. Roy. Meteor. Soc., 131 , 1013–1043.
Buehner, M., P. L. Houtekamer, C. Charette, H. L. Mitchell, and B. He, 2010a: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part I: Description and single-observation experiments. Mon. Wea. Rev., 138 , 1885–1901.
Buehner, M., P. L. Houtekamer, C. Charette, H. L. Mitchell, and B. He, 2010b: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part II: One-month experiments with real observations. Mon. Wea. Rev., 138 , 1902–1921.
Campbell, W. F., C. H. Bishop, and D. Hodyss, 2010: Vertical covariance localization for satellite radiances in ensemble Kalman filters. Mon. Wea. Rev., 138 , 282–290.
Daryl, T. K., D. F. Parrish, J. C. Derber, R. Treaton, W. Wu, and S. Lord, 2009: Introduction of the GSI into the NCEP global data assimilation system. Wea. Forecasting, 24 , 1691–1705.
Derber, J., and A. Rosati, 1989: A global oceanic data assimilation system. J. Phys. Oceanogr., 19 , 1333–1347.
Hamill, T. M., and C. Snyder, 2000: A hybrid ensemble Kalman filter-3D variational analysis scheme. Mon. Wea. Rev., 128 , 2905–2919.
Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129 , 2776–2790.
Hayden, C. M., and R. J. Purser, 1995: Recursive filter objective analysis of meteorological fields: Applications to NESDIS operational processing. J. Appl. Meteor., 34 , 3–15.
Houtekamer, P., G. Pellerin, M. Buehner, and M. Charron, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea. Rev., 133 , 604–620.
Jung, Y., G. Zhang, and M. Xue, 2008: Assimilation of simulated polarimetric radar data for a convective storm using the ensemble Kalman filter. Part I: Observation operators for reflectivity and polarimetric variables. Mon. Wea. Rev., 136 , 2228–2245.
Kleist, D. T., D. F. Parrish, J. C. Derber, R. Treadon, W-S. Wu, and S. Lord, 2009: Introduction of the GSI into the NCEP Global Data Assimilation System. Wea. Forecasting, 24 , 1691–1705.
Liu, H., J. Anderson, Y. Kuo, C. Snyder, and A. Caya, 2008: Evaluation of a nonlocal quasi-phase observation operator in assimilation of CHAMP radio occultation refractivity with WRF. Mon. Wea. Rev., 136 , 242–256.
Lorenc, A. C., 2003: The potential of the ensemble Kalman filter for NWP—A comparison with 4D-VAR. Quart. J. Roy. Meteor. Soc., 129 , 3183–3203.
Purser, R. J., W. S. Wu, D. F. Parrish, and N. M. Roberts, 2003a: Numerical aspects of the application of recursive filters to variational statistical analysis. Part I: Spatially homogeneous and isotropic Gaussian covariances. Mon. Wea. Rev., 131 , 1524–1535.
Purser, R. J., W. S. Wu, D. F. Parrish, and N. M. Roberts, 2003b: Numerical aspects of the application of recursive filters to variational statistical analysis. Part II: Spatially inhomogeneous and anisotropic general covariances. Mon. Wea. Rev., 131 , 1536–1548.
Szunyogh, I., E. J. Kostelich, G. Gyarmati, D. J. Patil, B. R. Hunt, E. Kalnay, E. Ott, and J. A. York, 2005: Assessing a local ensemble Kalman filter: Perfect model experiments with the NCEP global model. Tellus, 57A , 528–545.
Torn, R., and G. J. Hakim, 2008: Performance characteristics of a pseudo-operational ensemble Kalman filter. Mon. Wea. Rev., 136 , 3947–3963.
Wang, X., C. Snyder, and T. M. Hamill, 2007a: On the theoretical equivalence of differently proposed ensemble/3D-Var hybrid analysis schemes. Mon. Wea. Rev., 135 , 222–227.
Wang, X., T. M. Hamill, J. S. Whitaker, and C. H. Bishop, 2007b: A comparison of hybrid ensemble transform Kalman filter-OI and ensemble square-root filter analysis schemes. Mon. Wea. Rev., 135 , 1055–1076.
Wang, X., D. Barker, C. Snyder, and T. M. Hamill, 2007c: A hybrid ensemble transform Kalman filter (ETKF)-3DVAR data assimilation scheme for WRF. Preprints, 11th Symp. on Integrated Observing and Assimilation Systems for the Atmosphere, Oceans, and Land Surface, San Antonio, TX, Amer. Meteor. Soc., P4.10.
Wang, X., D. Barker, C. Snyder, and T. M. Hamill, 2008a: A hybrid ETKF–3DVAR data assimilation scheme for the WRF model. Part I: Observing system simulation experiment. Mon. Wea. Rev., 136 , 5116–5131.
Wang, X., D. Barker, C. Snyder, and T. M. Hamill, 2008b: A hybrid ETKF–3DVAR data assimilation scheme for the WRF model. Part II: Real observation experiments. Mon. Wea. Rev., 136 , 5132–5147.
Wang, X., T. M. Hamill, J. S. Whitaker, and C. H. Bishop, 2009: A comparison of the hybrid and EnSRF analysis schemes in the presence of model errors due to unresolved scales. Mon. Wea. Rev., 137 , 3219–3232.
Whitaker, J. S., T. M. Hamill, X. Wei, Y. Song, and Z. Toth, 2008: Ensemble data assimilation with the NCEP Global Forecast System. Mon. Wea. Rev., 136 , 463–482.
Wu, W. S., R. J. Purser, and D. F. Parrish, 2002: Three-dimensional variational analysis with spatially inhomogeneous covariances. Mon. Wea. Rev., 130 , 2905–2916.
Yang, S-C., E. Kalnay, B. Hunt, and N. E. Bowler, 2009: Weight interpolation for efficient data assimilation with the local ensemble transform Kalman filter. Quart. J. Roy. Meteor. Soc., 135 , 251–262.
Zhang, F., Y. Weng, J. A. Sippel, Z. Meng, and C. H. Bishop, 2009: Cloud-resolving hurricane initialization and prediction through assimilation of Doppler radar observations with an ensemble Kalman filter. Mon. Wea. Rev., 137 , 2105–2125.
Zupanski, M., 2005: Maximum likelihood ensemble filter: Theoretical aspects. Mon. Wea. Rev., 133 , 1710–1726.
