1. Introduction
A long-standing problem in NWP has been to provide initial conditions to convective-scale1 models for the prediction of convective systems and precipitations, given the available observations (Errico et al. 2007). The assimilation of cloud- or precipitation-related observations raises several important issues, such as the choice of observable, the depiction of observation error statistics, the characterization of model error in cloud and precipitation parameterization schemes, and the nonlinearity of the model forecasts and observation operators at those spatiotemporal scales (Lopez 2007). Current operational data assimilation systems use some form of Kalman filtering suitable for Gaussian errors. Following Auligné et al. (2010), cloud processes are highly nonlinear, giving non-Gaussian errors, but quasi-linear data assimilation techniques can be kept as a starting point and progress can be made on their respective advantages by hybridizing them. Simpler and physically consistent schemes such as latent heat nudging may also be of interest, through they have shortcomings such as neglecting precipitation advection, time lag, and a strong sensitivity to tuning (Leuenberger and Rossa 2007).
Even in the “Gaussian” framework, the depiction and representation of background error covariances at the convective-scale is not straightforward. In a variational data assimilation scheme, the background error penalty term is constructed from a control variable transform (CVT), which is a sequence of operators that approximate the square root of the true term as reviewed by Bannister (2008). The calibration of these operators is carried out over a sample of background errors built from either differences of forecasts valid at the same time (Parrish and Derber 1992) or from ensemble forecasts (Pereira and Berre 2006). In the ensemble Kalman filter (EnKF; Evensen 2003), the background error covariance matrix 
Recent papers (Brousseau et al. 2008; Montmerle and Berre 2010) have described the structure of background error statistics at convective scale, but the structure of background error for hydrometeor variables is still a widely discarded subject. In the variational framework, vertical autocovariances for hydrometeor variables have been computed by Amerault and Zou (2006) on tropical cyclones cases with coarse resolutions. Xiao and Sun (2007) and Xiao et al. (2007) have highlighted some aspects of the balance between rain content and temperature, humidity, and vertical motion induced by a linearized stationary warm rain process. The horizontal and vertical structures of the covariances are, however, not depicted. In the EnKF framework, previous studies mostly focused on the ability of the algorithm to retrieve wind and temperature from reflectivity observations (Dowell et al. 2004; Aksoy et al. 2009, 2010) rather than on a tridimensional multivariate synthetic description of the obtained covariance matrices.
Hybrid ensemble–variational assimilation (Hamill and Snyder 2000; Wang et al. 2008) blends the covariances sampled on an ensemble with the static covariance model designed for representing climatological errors. Ensemble covariances are incorporated into the variational minimization by using an extended control variable. Another possibility to retrieve the information on the covariance of the meteorological situation is to design more flexible covariance operators and to calibrate them on an ensemble of the day [see Berre and Desroziers (2010) for a recent review on this idea]. This has the advantage of not extending the control variable, but the drawback of requiring more complicated transforms in the background error modeling, including inhomogeneity (Michel and Auligné 2010).
Recently, Caron and Fillion (2010) and Montmerle and Berre (2010; hereafter MB2010) compared background error statistics in precipitating and nonprecipitating areas. MB2010 used geographical masks based on vertically averaged rain content to diagnose background error statistics with a convective-scale nonhydrostatic NWP model (Brousseau et al. 2008). Strong differences were found in the error variances, in the correlation length scales, and in the cross covariances between humidity and divergence errors between precipitating and nonprecipitating areas. MB2010 also suggested a methodology, named as a “heterogeneous” variational scheme, to build a piecewise analysis based on separate background error covariances for precipitating and nonprecipitating areas. Albeit the hybrid method will naturally represent flow-dependent covariances, the heterogeneous scheme may also prove useful to impose some covariances that are typical of a mesoscale structure such as a convective cell or as fog (Montmerle et al. 2010) when they are not present or displaced in the background.
After these approaches, this paper advances a synthetic description of the background error covariances, including for hydrometeor variables, that have been obtained through 3-km resolution ensemble forecasts from the Weather Research and Forecasting (WRF) model (Skamarock et al. 2008; Dowell and Stensrud 2008). For that purpose, the technique of the geographical masks based on rain content introduced by Caron and Fillion (2010) and MB2010 is used because hydrometeor errors are likely to be highly dependent on cloud physics. Andersson et al. (2005) pointed out that forecast differences for specific humidity are easier to approximate with a Gaussian for a limited geographical region and similar values of the background field. Here, it is also shown that it is meaningful to compute hydrometeor background error covariances only for similar values of precipitations.
The paper is organized as follows. Section 2 describes the convective-scale ensemble over which the diagnostics are performed. Section 3 introduces the updated formulation of the CVT in WRF variational data assimilation (Var) devised for this study. It includes a change of control variables with respect to previous formulations and a binning based on the rain content of the perturbations. Section 4 describes and compares the background error covariances in precipitating and nonprecipitating areas. The conclusion summarizes the results and discusses some issues about the implementation of a hydrometeor control variable in variational data assimilation with the heterogeneous formulation and on the possible use of these diagnostics for EnKF localization.
2. The dataset
a. The ensemble
Surface observations are known to be of primary importance for the description of mesoscale structures (Ducrocq et al. 2000), and in particular of cold pools and inflow jets that are frequently associated with mesoscale convective systems. As part of the National Oceanic and Atmospheric Administration (NOAA) Hazardous Weather Testbed 2007 Spring Forecasting Experiment, a mesoscale short-range ensemble was designed with the WRF model to explore the value of these surface observations for the forecasts of severe events (Stensrud et al. 2009). For this purpose, hourly analyses were produced at 30-km resolution on an extended domain covering the contiguous 48 states of the United States. Routine hourly surface observations of 2-m potential temperature, 2-m dewpoint temperature, and 10-m winds were assimilated within a 30-member ensemble with the Data Assimilation Research Testbed (DART; Anderson et al. 2009) square root EnKF. The ensemble is started by creating initial and boundary perturbations consistent with the static covariance information from WRF 3D-Var, following Torn et al. (2006) and Stensrud et al. (2000) for the humidity perturbations. In addition to the initial dispersion of the ensemble and to the perturbations deduced from the EnKF, physics perturbations were introduced following the methodology of Stensrud et al. (2000), with variations of land surface, planetary boundary layer, radiation, convection, and microphysical perturbations. This ensemble was shown to well represent both initial condition and model physics uncertainties and results on case studies are examined in Stensrud et al. (2009).
This mesoscale ensemble analyses and forecasts were used by Dowell and Stensrud (2008) to provide initial and boundary conditions for an ensemble of forecasts that utilized a 3-km horizontal grid spacing with explicit convection, for several severe weather cases during spring 2007. This convective-scale ensemble was shown to provide valuable guidance about convective storm timing, location, and mode and is further exploited herein to estimate background error statistics with a focus on hydrometeor variables.
b. Meteorological situations
Two different meteorological situations are here considered: a case of high plains tornado outbreak at 0300 UTC 29 March 2007 and a multicellular storm at 0000 UTC 6 May 2007. Note that the sizes and locations of the domains are a different, but the forecast lead time is the same (6 h). Both ensembles have been run using initial and boundary conditions from the largest-scale EnKF.
Some features of these two cases are given in Fig. 1: the location where the ensemble produces strong precipitation is indicated by the ensemble-estimated probability of high reflectivity occurrence at the lowest model level (within a radius of 15 km from grid point). Following Dowell and Stensrud (2008), the cyclonic supercells in the ensemble members are detected as locations where strong value of vertical velocity and vorticity arise (ω > 0, ζ > 0, and ω × ζ > 0.1 m s−1 at 600 hPa). As shown by the ensemble-averaged 10-m wind, the storms are located in an environment of strong low-level convergence. Both cases, albeit different, look reasonably similar in terms of number of supercells and structure of the low-level flow. The situation at 0300 UTC 29 March 2007 looks weaker compared to the case at 0000 UTC 6 May 2007, which is associated with a strong probability of high reflectivities.
Probability (%) that the radar reflectivity exceeds the level 50 dBZ within 15 km of grid point over the ensemble for (a) the 0300 UTC 29 Mar 2007 case and (b) the 0000 UTC 6 May 2007 case (shadings). Supercells locations defined as grid point of joint strong and positive values of vertical velocity and vorticity (ω > 0, ζ > 0, and ω × ζ > 0.1 m s−1) are shown with dots. The 10-m ensemble mean wind is plotted with barbs.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
3. The heterogeneous 3D-Var with a rain-dependent binning and hydrometeor variables
a. Introduction to 3D-Var
b. The formulation of the control variable transform
The calibration of operators defining 
c. The rain-dependent binning
The variable 
d. The physical transform
The state vector has been extended and includes the vorticity ζ, the divergence η, the temperature T, the surface pressure Ps, the specific humidity q, the cloud content qc, and the rain content qr. Alternatively, the state vector for hydrometeors could instead admit the logarithm of cloud content 
The last two lines statistically couple hydrometeors with the other variables, with the noticeable difference that the geostrophic transform has been discarded as there is few coupling between variables that have such different scales and because the errors probably have very different sources (e.g., microphysics and clouds for hydrometeors).
e. Design of correlations
The design of the vertical correlations is based on the decomposition of EOF. The vertical covariances are computed and a diagonalization is performed for each bin, resulting in vertical correlations that depend on the geographical masks.
In the variational scheme horizontal correlations are produced by recursive filters, which produce Gaussian-like shapes. A larger class of shapes can be obtained by positive linear combinations (Purser et al. 2003a). For variables such as vorticity and divergence, negative lobes of the correlation function can be reproduced if a scaled Laplacian operator is applied (Purser et al. 2003b) or with carefully designed nonpositive linear combinations (Mirouze and Weaver 2010) that preserve the positive definite character of the covariances.
The model for the horizontal correlations decidedly depends on the reduction of the number of tunable parameters, such as, for instance, one or more length scales with associated weighting coefficients. A general statistical framework to estimate these coefficients is described by Dee and da Silva (1999). A widespread and simpler approach consists in using a diagnostic as an input for the model of the horizontal correlations. In Wu et al. (2002), a single length scale parameter was computed from the ratio of the variance of a field over the variance of the Laplacian of this field as an input for the recursive filters. This has the benefit of relying on a simple computation of variances that can easily be made bin dependent.
In this study, the single length scale parameter L is first diagnosed following Wu et al. (2002) and Michel and Auligné (2010). To broaden the range of correlation functions and especially that of fat tails, the correlation is written as a positive linear combination of three different Gaussian-shaped functions of scales L/2, L, and 2L. The nonnegative least squares algorithm designed by Lawson and Hanson (1974) is used to fit the weighting coefficients of the combination to the locally diagnosed correlation function.
f. Illustration
So far, the discussion has focused on the definition and calibration of square root operators defining the background error covariance matrices. The complete background error covariance matrix is the sum of the different background error covariance matrices weighted by the geographical masks [Eq. (8)]. If the masks are strictly taken as piecewise constant (for instance, with 0 in heavy precipitating areas and 1 otherwise for Ph), then the resulting increments will exactly vanish on the border between geographical masks. This situation is illustrated in Fig. 2a, where single observation results for temperature are shown to have shapes strongly driven by the geographical mask. This is likely to produce spurious numerical artifacts. Correlations between the precipitating and nonprecipitating regions need to be introduced by spatially smoothing the geographical masks, as suggested by MB2010.
Single observation test for temperature, using (a) raw geographical masks based on the rain content of ensemble member 1 and (b) opening and convolving filtered geographical masks on the 0000 UTC 6 May 2007 case. Contours of the correlation functions are [0.20, 0.28, 0.38, 0.49, 0.61, 0.75, 0.90]. The additional contours delineate the precipitating area.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
Sometimes the mislocation of clouds in the background will cause deficiency in the analyses (Auligné et al. 2010), because a displacement error may cause a non-Gaussian distribution for background errors (Lawson and Hansen 2005; Ravela et al. 2007). Given reliable observations of reflectivity (for instance from a dense radar network), it is possible to use the mixed bin at places where the background is not precipitating but where the observations show precipitations. This feature is quite unique to the heterogeneous 3D-Var and may help producing clouds and rain where necessary.
We shall now describe some elementary operations on geographical masks that may be of interest when using the heterogeneous 3D-Var in a realistic context. The first one, from the preceding discussion, is the ensemble union of observed and forecast precipitating areas to produce clouds and rain where necessary. Enlarging the mask with the help of observations is unlikely to be sufficient to provide good analysis. In particular, one may want to get rid of precipitating areas that are much smaller than the typical background length scale to ensure that the analysis increment do not follow exactly the geographical masks. In short, it may be desirable to smooth out the contour of the geographical masks shown in Fig. 2a. One solution for this is the use of algorithms from mathematical morphology [see, for instance, appendix A of Michel and Bouttier (2006) and references therein]. In particular, the opening of an image is obtained as successive erosion and dilatations. Intuitively, dilation expands an image object and erosion shrinks it. Opening smoothes a contour in an image, breaking narrow isthmuses and eliminating thin protrusions. The result is shown in Fig. 2b. The contours of the geographical mask have been smoothed and the tiniest cells have been removed by the algorithm. Together with convolution of the mask (as in MB2010), this ensures smooth transition for analysis increments between the different zones, as shown in Fig. 2b. When the precipitating area is small, the analysis increment however exhibits some fat tails that are a contribution from the nonprecipitating area. The amount of smoothing, both for the opening procedure and for the convolution, results from a trade-off between smoothness and representation of precipitating areas. It will need to be set with an objective criterion and is the topic of current study by Ménétrier and Montmerle (2011).
g. Gaussianity
Auligné et al. (2010) have suggested that focusing first on the covariances of hydrometeor errors may be a valuable strategy, if the displacement error is taken into account as a first step. Recent progress has been made in non-Gaussian data assimilation [see Bocquet et al. (2010) for a recent review], such as the use of Gaussian anamorphosis prior to the analysis update, but it is still not clear how to handle multivariate aspects. Section 4 will show in particular that using a lognormal transform significantly alters the multivariate aspects, and in particular the coupling with divergence of hydrometeors found in these convective situations.
Figure 3 shows the histograms of background errors projected onto EOF modes, for different bins (based on the geographical masks previously defined, not to be confused with the bins of the histogram itself). Results from Fig. 3 should be interpreted while keeping in mind that the total distributions shown are spatial averages over different grid points, each of which could have their own characteristic errors (Holm et al. 2002). Because of this spatial averaging, it is not directly possible to relate (non)Gaussianity of histogram to the (non)Gaussianity of the underlying probability density functions. Rather, this should be understood as a consistency check of whether the covariances are representative of the (averaged) probability density functions.
Normalized histograms (adjacent rectangles in solid black lines) displaying relative frequencies of the occurrence of background errors normalized by their standard deviation and projected on the first EOF for different variables and geographical masks: (a) temperature in heavy rain bin, (b) specific humidity in heavy rain bin, (c) cloud content in the mixed bin, (d) rain content in the mixed bin, (e) cloud content in the heavy rain bin, and (f) rain content in the heavy rain bin. Also shown the Gaussian (solid gray) and exponential (dashed gray) continuous distributions of same variance.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
The background errors for wind and temperature variables appear to have near-normal distributions in all bins (see Fig. 3a). For humidity, a near-normal distribution is observed in the rainy bins (Fig. 3b) but not in the nonrainy bin (not shown), which may be a result of the inhomogeneity of the humidity field (Holm et al. 2002). The background errors for hydrometeor variables look nearly Gaussian in the rainy bins (Figs. 3e,f) but not in the mixed bin (Figs. 3c,f) or the nonprecipitating one (not shown). This support the idea that the probability density functions of hydrometeor background errors may be reasonably approximated by Gaussian in cases where there is no displacement error of rainy areas. We turn now to the description of the covariances obtained in this case.
4. Diagnostics of background error covariances
a. Background error standard deviations
The vertical profiles of bin-averaged background error standard deviations σb are shown in Fig. 4 for the traditional control variables (vorticity, divergence, temperature, and specific humidity) for the March case. Overall, as in MB2010, they are bigger by a large factor for wind variables in precipitating areas with respect to nonprecipitating ones. Moreover these standard deviations depend on the amount of rain chosen as a threshold to define the geographical masks (the standard deviations in “light rain” conditions are smaller than in “heavy rain” conditions). This probably reflects an important small-scale dynamical circulation associated with convective precipitating clouds; in particular the divergence profile (Fig. 4b) shows two extremes on the vertical that could be linked to the uncertainty in cloud top and bottoms. Temperature errors (Fig. 4c) are slightly bigger in precipitating areas than in nonprecipitating ones within the clouds but of smaller amplitude at lowest levels, which may reflect some isothermal effects caused by a strong convection. The overshoot of convective towers over the tropopause could explain the large σb(T) at 200 hPa. Humidity errors also show two extrema on the vertical that could be linked with the uncertainty in cloud top and bottoms. Background error standard deviations for vorticity and divergence were similar in the May case. Figure 5 shows the vertical profiles obtained for temperature and specific humidity. The contrast between precipitating and nonprecipitating areas is even more apparent, with larger errors in precipitating areas. Contrary to the March case, the standard deviation for temperature is larger in precipitating areas than in nonprecipitating ones at all altitudes.
Vertical profiles of background error standard deviations for (a) vorticity, (b) divergence, (c) temperature, and (d) specific humidity computed over rain-dependent binnings: heavy precipitating areas (dotted–dashed line), light precipitating areas (dotted gray line), mixed situations (dashed line), and nonprecipitating areas (solid line). Case at 0300 UTC 29 Mar 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
Vertical profiles of background error standard deviations for (a) temperature and (b) specific humidity (same legend as Fig. 4). Case at 0000 UTC 6 May 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
The vertical profiles of background error standard deviations for the March case for hydrometeors (Fig. 6) share some similarities with the mean distribution of those hydrometeors themselves (not shown). As for dynamical variables, standard deviations are increasing at larger precipitation thresholds. Small σb for qc are visible in nonprecipitating areas, which shows that some nonprecipitating clouds are present in the ensemble. The standard deviation of cloud content error is monopolar shaped with a maximum at 700 hPa. The standard deviation of rain error has two local extremes, one inside the cloud (toward 450 hPa) that is probably linked to the occurrence of rain at higher altitudes in strong convective towers with broad vertical extension and one below the cloud base (toward 800 hPa). The evaporation of precipitations may induce a decrease of the variance at lower levels. Similar results have been obtained for the May case (Fig. 7), but the variance of cloud content is broader. The quantity of rain error is also larger (it reaches 1.35 g kg−1 rather than 0.95 g kg−1). The highest value for rain error is located at higher altitude than for the March case, which indicates the strong dependence of rain covariances on the environmental conditions, for example, the altitude of the freezing level or the vertical extension of convective towers. These findings are consistent with the stronger intensity mentioned in section 2.
Vertical profiles of background error standard deviations for (a) cloud content, (b) rain content, (c) logarithm of cloud content, and (d) logarithm of rain content computed over rain-dependent binnings: heavy rain areas (dotted–dashed line), light rain areas (dotted gray line), and mixed situations (dashed line). Case at 0300 UTC 29 Mar 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
Vertical profiles of background error standard deviations for (a) cloud content and (b) rain content. Same legend as in Fig. 6. Case at 0000 UTC 6 May 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
The logarithmic transform stresses relative, rather than absolute, variations: a small amount change in logqr may be written 
b. Physical transform
Figure 8 shows the scaled eigenvalues of the vertical background error covariances matrices that need to be inverted to form the regression coefficients in Eq. (13). Generally, good conditioning (~103) can be obtained for most variables except for humidity qu (~108) and hydrometeors. This problem appears for both total and unbalanced hydrometeors. Using the log transform only alleviate the problem for qc in the May case (Fig. 8). The bin-averaged vertical covariance matrices for hydrometeors in rainy areas are of reduced rank, with three null eigenvalues. This may be caused by the limited sampling or lack of statistical diversity in the ensemble. Regularization is therefore required at least for the 
Normalized eigenvalues of the vertical background error covariances matrices for unbalanced variables for (a) the 0300 UTC 29 Mar 2007 case and (b) the 0000 UTC 6 May 2007 case (heavy precipitating areas).
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
It is common to assess the goodness-of-fit of the linear least squares by comparing how much of the variation of the total variables can be reduced, or explained, by the regression onto the predictors (Bouttier et al. 1997). This is achieved through partial ratios of explained regressed variance to the total variance. Figure 9 shows the results from nonprecipitating areas for temperature (Fig. 9a) and specific humidity (Fig. 9b). The vorticity is the main contributor and explains about 50% of the variance in temperature. This value is smaller than the ones for larger-scale global models (Derber and Bouttier 1999; Ingleby 2001; Wu et al. 2002) because of the loss of geostrophy at smaller scales (Berre 2000), but still larger than those in MB2010 and also larger than the ones for the May case (not shown). Thus around 30% of the variance of q is explained; a result comparable with the description in Berre (2000), except that there is no coupling at all with divergence in this case.
Explained variance ratio of (a) temperature in nonprecipitating areas, (b) specific humidity in nonprecipitating areas, (c) temperature in heavy precipitating areas, and (d) specific humidity in heavy precipitating areas as a function of pressure level. “Total 2” is one minus the unexplained variance ratio, which should be equal to the sum of explained variance ratio “total” if no regularization. Case at 0300 UTC 29 Mar 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
The situation is very different in precipitating areas, with the divergence as the predominant statistical coupling for humidity in the midtroposphere (Fig. 9d), and the divergence and the vorticity-derived balanced mass for temperature (Fig. 9c). This confirms the interest for considering multivariate relationships for the specific humidity, especially within precipitating areas. Such a relationship may help sustain convection while assimilating humidity-related observations (Caumont et al. 2010) and Doppler winds (Montmerle and Faccani 2009).
For hydrometeors (Fig. 10), the prevalent coupling is with the unbalanced divergence, which is directly linked to the vertical motion through the continuity equation. This does not mean that couplings with temperature or humidity are small, as a significant part of temperature and humidity is explained by the unbalanced divergence. Furthermore, there is very good agreement between the sum of the explained variance ratios (“total” in Fig. 10) and one minus the unexplained variance ratio (“total 2” in Fig. 10), thus corroborating the fact that the regularization does not significantly affect the orthogonality of the successive regressions. When using the log transform, the relative order of the predictor has been found to differ dramatically: Fig. 11 shows that the prevalent coupling of the variance of the logarithm of rain content are unbalanced humidity and temperature below 800 hPa. Temperature and humidity are better predictants of the relative variations of rain content than of the absolute variations. This highlights that the use of the log transform may significantly alter the multivariate coupling with divergence in the low troposphere. It would be necessary to apply the log transform to unbalanced variables to avoid this.
Explained variance ratio of (a) cloud content and (b) rain content as a function of pressure level, computed over the heavy precipitating areas. Case at 0300 UTC 29 Mar 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
Explained variance ratio of logarithm of rain content as a function of pressure level, computed over the heavy precipitating areas. Case at 0300 UTC 29 Mar 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
The use of qc and qr variables will cause analysis increments to exhibit a complex structure in all other wind and mass variables. Additional diagnostics are required to infer physical meaning, and ascertaining of the vertical cross covariances can provide clues to the underlying structure of errors (Berre 2000; Michel and Auligné 2010). Cross covariances of q and qc with the unbalanced divergence are shown in Fig. 12. The same structure as in MB2010 is observed, with a positive humidity error linked to the low-level convergence and a higher-level divergence in the convective clouds (Fig. 12a). A similar structure exists for cloud content but is vertically displaced toward midlevels (Fig. 12b). The cross covariance of qr background error with unbalanced divergence is more complex, with a tripolar structure (Fig. 12c). A positive rain increment at the lowest levels is linked with the convergence and the divergence associated with downdrafts below 900 hPa. The cross covariance with unbalanced temperature highlights some impacts of microphysics: the influence of evaporation is noticeable by the occurrence of a negative cross covariance below 850 hPa (the “cold pool” effect, cf. Fig. 12d), and the effect of latent heat release is noticeable by the occurrence of positive cross covariances in the cloud. Similar structure is found when computing the cross covariance between q and qr, with the effect of evaporation below the cloud and the effect of saturation in the cloud (not shown).
Vertical cross covariances of background errors between (a) specific humidity and unbalanced divergence (contour interval 2 × 10−4 g kg−1 s−1), (b) cloud content and unbalanced divergence (contour interval 2 × 10−4 g kg−1 s−1), (c) rain content and unbalanced divergence (contour interval 10−4 g kg−1 s−1), and (d) rain content and unbalanced temperature (contour interval 2 × 10−2 g kg−1 K) computed over the heavy precipitating areas. The regions with large cross-covariance values are filled dotted.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
At convective scale, the notion of balance has been recently revisited by Caron and Fillion (2010). Their main finding is that the vertical motion should be in balance with physical tendencies. This result is believed to be consistent with the large coupling of hydrometeors fields with divergence found here. However, it is likely that the rain increments that will be produced by the heterogeneous scheme will be washed out at the very first steps by the physics package. If the rain increment is big, this may yield unbalance. One possibility would be to use some form of incremental analysis update (Bloom et al. 1996). The question of the balanced character of the analysis increments will be addressed with the first assimilation experiments when the full heterogeneous control variable transform will be available in models Application of Research to Operations at Mesoscale (AROME)-Var or WRF-Var.
c. Autocorrelations
The structure of vertical correlations has also been found to depend strongly on the rain-dependent binning for both total and unbalanced variables. Figure 13 highlights how the total covariances differ for temperature and humidity variables. In particular, the negative correlations that appear in nonprecipitating air between the stratosphere and the troposphere (Fig. 13a) do not exist in precipitating areas (Fig. 13b). Correlations for humidity are broader in precipitating areas and above 850 hPa, reflecting the mixing in clouds. Similar results have been found for the May case (not shown).
Average vertical autocorrelations for temperature in (a) nonprecipitating areas and (c) heavy precipitating areas, and total specific humidity in (b) nonprecipitating areas and (d) heavy precipitating areas as a function of pressure level (contour interval 0.2).
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
Horizontal length scales have been found to be smaller by a factor of 2 in precipitating areas (Fig. 14), which is consistent with the results of MB2010. The mixed bin, which is the cloud/no-cloud situation, has an intermediate length scale, which is likely to be dominated by the displacement error and by the size of convective cells. Temperature and humidity horizontal correlations in precipitating areas are tightened toward 600 hPa, which may mean a smaller horizontal mixing.
The e-folding horizontal length scale (km) as a function of pressure level for (a) temperature and (b) specific humidity for different rain-dependent binnings. Case at 0300 UTC 29 Mar 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
In practice, the horizontal length scales are diagnosed on unbalanced variables decomposed on EOF modes. This impedes the full representation of the variation of length scales with height (Ingleby 2001) but allows for a weak nonseparability of the tridimensional covariances. The EOF modes, when ranked with their variances, are more and more oscillating on the vertical (Bouttier et al. 1997; Barker et al. 2004; Michel and Auligné 2010) and thus of smaller vertical scale. As shown in Fig. 15, the horizontal length scales of unbalanced temperature (Fig. 15a) and unbalanced humidity (Fig. 15b) in nonprecipitating areas decrease with the index of the EOF mode. It ensues that there is a geostrophic scaling between horizontal and vertical scales. This is less pronounced for heavy precipitating areas, which is consistent with the findings of Caron and Fillion (2010) as precipitating areas are “less geostrophic,” because a significant part of the background error is linked to microphysical processes.
The e-folding horizontal length scale (km) as a function of EOF mode for (a) unbalanced temperature and (b) unbalanced specific humidity for different rain-dependent binnings. Case at 0300 UTC 29 Mar 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
The structure of vertical correlations for hydrometeors (shown in Figs. 16a,b) shares some similarities with the results of Amerault and Zou (2006), despite using different models, methodologies and weather phenomena. The broad vertical correlations for qc in the midtroposphere denotes the vertical mixing within precipitating clouds. Rain is vertically highly correlated under 800 hPa, which is the direct consequence of rain falling freely under this level. A similar shape is found when using the log transform, but the vertical extent of the correlations is reduced (Figs. 16c,d).
Average vertical autocorrelations for (a) cloud content and (b) rain content in heavy precipitating areas as a function of model vertical level (contour interval 0.2). Case at 0300 UTC 29 Mar 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
The horizontal length scales of cloud and rainwater contents (Fig. 17) reach about 6 km, which is 2 grid spacings and is smaller than for other variables. Similar results have been obtained using the log transform (not shown). For cloud content, the maximum displayed at 950 hPa below the cloud is probably an artifact due to poor sampling at those vertical levels, as it does not appear in the diagnosis of the May ensemble. This is not a problem in practice as length scales are prescribed for unbalanced variables on truncated EOF modes. Errors in rain content tend to be correlated over larger distances at the lowest levels (6 km at 1000 hPa but 5 km at 800 hPa), which may be due to a coalescence by collision. This result shows that hydrometeor errors tend to be less extended on the horizontal than other variables, and this discrepancy should be taken into account in localization procedures used to filter sampling noise in the EnKF. Finer diagnosis based on sample correlations (not shown) indicate that negative correlations occasionally occur at longer range in the midtroposphere, in agreement with the large statistical coupling found with divergence.
The e-folding horizontal length scale (km) as a function of model vertical level for (a) cloud content and (b) rain content for different rain-dependent binnings. Case at 0300 UTC 29 Mar 2007.
Citation: Monthly Weather Review 139, 9; 10.1175/2011MWR3632.1
5. Conclusions and discussion
Variational assimilation heavily relies on the design of the background error covariance matrix to filter and spread spatially the information brought by the observations. Because of technical and mathematical difficulties in the construction of three-dimensional covariance operators, this matrix is generally simplified and represents only some spatial and temporal average of the true matrix. In contrast, the EnKF uses a sampled background error covariance matrix that is affected by sampling noise but dependent on the meteorological context and on the quality and density of observations.
In this study, a sample from an EnKF is used to build up a background error covariance matrix for 3D-Var in which the computation of statistics considers precipitating and nonprecipitating areas separately. For traditional control variables, the two regions are found to be characterized by different statistics, including bigger variance, larger coupling of humidity, and shorter length scales in precipitating versus nonprecipitating areas. This has two implications, for both the EnKF and 3D-Var: localization can be made more restrictive in precipitating areas for the EnKF, and 3D-Var formulations at cloud scales should be able to represent these major inhomogeneities. In this way, these two algorithms could assimilate more observations in a physically consistent way in precipitating areas where the background uncertainty is known to be large.
Another caveat of background error control variable transforms is that they generally ignore hydrometeors. In this paper, using the geographical masks, the full covariance structure of background errors for liquid hydrometeors was computed, including the balance and the spatial correlations that are generally neglected. The inclusion of cloud and rainwater contents required the use of regularized regression; otherwise the same framework as for standard variables was used. Significant coupling was found with divergence, and realistic spatial correlations exist, which may help spread of the information from observations to related hydrometeors and then to other variables. It was found that the direct use of the lognormal approach was having a strong impact on the structure of the physical transform, in particular reducing the coupling with divergence.
The computation of the covariances was accomplished for similar values of the rain amount in the background. This is necessary to keep the probability density functions of hydrometeor variables close to the Gaussian case. This may be seen as a (simple) spatial dividing–localization strategy in the terminology of Bocquet et al. (2010). In practice, however, it is not clear yet if the heterogeneous scheme can handle the difficult situations of precipitating background and nonprecipitating observation, as this can happen because of displacement problems. It is possible to produce hydrometeor increment in arbitrary regions by extending and smoothing the geographical masks, but the scheme neglects the non-Gaussianity induced by this situation. This will be explored in future studies, following the ideas developed in Auligné et al. (2010).
Acknowledgments
This research has been partly supported by Météo-France, CNRM-GAME, by the Air Force Weather Agency, and by the ANR project GeoFluids under the reference ANR-09-SYSC-005. The National Center for Atmospheric Research is sponsored by the National Science Foundation. Data has been kindly provided by David Dowell, NCAR ESL, Boulder, Colorado.
REFERENCES
Aksoy, A., D. C. Dowell, and C. Snyder, 2009: A multicase comparative assessment of the ensemble Kalman filter for assimilation of radar observations. Part I: Storm-scale analyses. Mon. Wea. Rev., 137, 1805–1824.
Aksoy, A., D. C. Dowell, and C. Snyder, 2010: A multicase comparative assessment of the ensemble Kalman filter for assimilation of radar observations. Part II: Short-range ensemble forecasts. Mon. Wea. Rev., 138, 1273–1292.
Amerault, C., and X. Zou, 2006: Comparison of model-produced and observed microwave radiances and estimation of background error covariances for hydrometeor variables within hurricanes. Mon. Wea. Rev., 134, 745–758.
Anderson, J., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Avellano, 2009: The Data Assimilation Research Testbed: A community facility. Bull. Amer. Meteor. Soc., 90, 1283–1296.
Andersson, E., and Coauthors, 2005: Assimilation and modeling of the atmospheric hydrological cycle in the ECMWF forecasting system. Bull. Amer. Meteor. Soc., 86, 387–402.
Auligné, T., A. Lorenc, Y. Michel, T. Montmerle, A. Jones, M. Hu, and J. Dudhia, 2010: Toward a new cloud analysis and prediction system. Bull. Amer. Meteor. Soc., 92, 207–210.
Bannister, R. N., 2008: A review of forecast error covariance statistics in atmospheric variational data assimilation. II: Modelling the forecast error covariance statistics. Quart. J. Roy. Meteor. Soc., 134, 1971–1996.
Barker, D., 2005: Southern high-latitude ensemble data assimilation in the Antarctic Mesoscale Prediction System. Mon. Wea. Rev., 133, 3431–3449.
Barker, D., W. Huang, Y.-R. Guo, A. Bourgeois, and Q. Xiao, 2004: A three-dimensional variational data assimilation system for MM5: Implementation and initial results. Mon. Wea. Rev., 132, 897–914.
Berre, L., 2000: Estimation of synoptic and mesoscale forecast error covariances in a limited-area model. Mon. Wea. Rev., 128, 644–667.
Berre, L., and G. Desroziers, 2010: Filtering of background error variances and correlations by local spatial averaging: A review. Mon. Wea. Rev., 138, 3693–3720.
Bloom, S. C., L. L. Takacs, A. M. da Silva, and D. Ledvina, 1996: Data assimilation using incremental analysis updates. Mon. Wea. Rev., 124, 1256–1271.
Bocquet, M., C. A. Pires, and L. Wu, 2010: Beyond Gaussian statistical modeling in geophysical data assimilation. Mon. Wea. Rev., 138, 2997–3023.
Bouttier, F., J. Derber, and M. Fisher, 1997: The 1997 revision of the Jb term in 3D/4D-Var. Tech. Memo. 238, ECMWF, 55 pp.
Brousseau, P., and Coauthors, 2008: A prototype convective-scale data assimilation system for operation: The Arome-RUC. Tech. Rep., HIRLAM Tech. Rep. 68, 8 pp.
Caron, J.-F., and L. Fillion, 2010: An examination of background error correlations between mass and rotational wind over precipitation regions. Mon. Wea. Rev., 138, 563–578.
Caumont, O., V. Ducrocq, E. Wattrelot, G. Jaubert, and S. Pradier-Vabre, 2010: 1D+3DVar assimilation of radar reflectivity data: A proof of concept. Tellus, 62, 173–187.
Courtier, P., J.-N. Thépaut, and A. Hollingsworth, 1994: A strategy for operational implementation of 4D-Var, using an incremental approach. Quart. J. Roy. Meteor. Soc., 120, 1367–1387.
Dee, D., and A. da Silva, 1999: Maximum-likelihood estimation of forecast and observation error covariance parameters. Part I: Methodology. Mon. Wea. Rev., 127, 1822–1834.
Derber, J., and F. Bouttier, 1999: A reformulation of the background error covariance in the ECMWF global data assimilation system. Tellus, 51A, 195–221.
Dowell, D., and D. Stensrud, 2008: Ensemble forecasts of severe convective storms. Preprints, 24th Conf. on Severe Local Storms, Savannah, GA, Amer. Meteor. Soc., 13A.5. [Available online at http://ams.confex.com/ams/24SLS/techprogram/paper_141628.htm.]
Dowell, D., F. Zhang, L. Wicker, C. Snyder, and N. Crook, 2004: Wind and temperature retrievals in the 17 May 1981 Arcadia, Oklahoma supercell: Ensemble Kalman filter experiments. Mon. Wea. Rev., 132, 1982–2005.
Ducrocq, V., J.-P. Lapore, J.-L. Redelsperger, and F. Orain, 2000: Initialization of a fine-scale model for convective-system prediction: A case study. Quart. J. Roy. Meteor. Soc., 126, 3041–3065.
Errico, R., P. Bauer, and J.-F. Mahfouf, 2007: Issues regarding the assimilation of cloud and precipitation data. J. Atmos. Sci., 64, 3785–3798.
Evensen, G., 2003: The ensemble Kalman filter: Theoretical formulation and practical implementation. Ocean Dyn., 53, 343–367.
Fischer, C., T. Montmerle, L. Berre, L. Auger, and S. Stefanescu, 2005: An overview of the variational assimilation in the ALADIN/France numerical weather-prediction system. Quart. J. Roy. Meteor. Soc., 131, 3477–3492.
Fletcher, S., and M. Zupanski, 2006: A hybrid multivariate normal and lognormal distribution for data assimilation. Atmos. Sci. Lett., 7, 43–46.
Fletcher, S., and M. Zupanski, 2007: Implications and impacts of transforming lognormal variables into normal variables in VAR. Meteor. Z., 6, 755–765.
Gauthier, P., C. Charette, L. Fillion, P. Koclas, and S. Laroche, 1999: Implementation of a 3D variational data assimilation system at the Canadian Meteorological Centre. Part I: The global analysis. Atmos.–Ocean, 37, 103–156.
Hamill, T. M., and C. Snyder, 2000: A hybrid ensemble Kalman filter-3D variational analysis scheme. Mon. Wea. Rev., 128, 2905–2919.
Hamill, T. M., J. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 2776–2790.
Hayden, C., and R. Purser, 1995: Recursive filter objective analysis of meteorological fields: Applications to NESDIS operational processing. J. Appl. Meteor., 34, 3–15.
Holm, E., E. Andersson, A. Beljaars, P. Lopez, J.-F. Mahfouf, A. Simmons, and J.-N. Thepaut, 2002: Assimilation and modelling of the hydrological cycle: ECMWF’s status and plans. Tech. Rep. 383, ECMWF, 55 pp.
Houtekamer, P., and H. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123–137.
Ingleby, N., 2001: The statistical structure of forecast errors and its representation in The Met. Office Global 3-D Variational Data Assimilation Scheme. Quart. J. Roy. Meteor. Soc., 127, 209–231.
Kepert, J., 2009: Covariance localisation and balance in an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 135, 1157–1176.
Lawson, C., and R. Hanson, 1974: Solving Least Square Problems. Prentice Hall, 340 pp.
Lawson, W., and J. Hansen, 2005: Alignment error models and ensemble-based data assimilation. Mon. Wea. Rev., 133, 1687–1709.
Leuenberger, D., and A. Rossa, 2007: Revisiting the latent heat nudging scheme for the rainfall assimilation of a simulated convective storm. Meteor. Atmos. Phys., 98, 195–215.
Lopez, P., 2007: Cloud and precipitation parameterizations in modeling and variational data assimilation: A review. J. Atmos. Sci., 64, 3766–3784.
Lorenc, A. C., 1997: Development of an operational variational assimilation scheme. J. Meteor. Soc. Japan, 75, 339–346.
Lorenc, A. C., and Coauthors, 2000: The Met Office global 3-dimensional variational data assimilation scheme. Quart. J. Roy. Meteor. Soc., 126, 2991–3012.
Ménétrier, B., and T. Montmerle, 2011: Heterogeneous background error covariances for the analysis and forecast of fog events. Quart. J. Roy. Meteor. Soc., in press.
Michel, Y., and F. Bouttier, 2006: Automated tracking of dry intrusions on satellite water vapour imagery and model output. Quart. J. Roy. Meteor. Soc., 132, 2257–2276.
Michel, Y., and T. Auligné, 2010: Inhomogeneous background error modeling and estimation over Antarctica. Mon. Wea. Rev., 138, 2229–2252.
Mirouze, I., and A. Weaver, 2010: Representation of correlation functions in variational assimilation using an implicit diffusion operator. Quart. J. Roy. Meteor. Soc., 136, 1421–1443.
Montmerle, T., and C. Faccani, 2009: Mesoscale assimilation of radial velocities from Doppler radars in a preoperational framework. Mon. Wea. Rev., 137, 1939–1953.
Montmerle, T., and L. Berre, 2010: Diagnosis and formulation of heterogeneous background error covariances at mesoscale. Quart. J. Roy. Meteor. Soc., 136, 1408–1420.
Montmerle, T., Y. Michel, and B. Ménétrier, 2010: Modelling of background error covariances for the analysis of clouds and precipitation. Proc. ECMWF–JCSDA Workshop on Assimilating Satellite Observations of Clouds and Precipitation, Reading, United Kingdom, ECMWF–JCSDA, 10 pp.
Neumaier, A., 1998: Solving ill-conditioned and singular linear systems: A tutorial on regularization. SIAM Rev., 40, 636–666.
Orlanski, I., 1975: A rational subdivision of scales for atmospheric processes. Bull. Amer. Meteor. Soc., 56, 527–530.
Parrish, D., and J. Derber, 1992: The National Meteorological Center’s spectral statistical-interpolation analysis system. Mon. Wea. Rev., 120, 1747–1763.
Parrish, D., J. Derber, J. Purser, W. Wu, and Z. Pu, 1997: The NCEP global analysis system: Recent improvements and future plans. J. Meteor. Soc. Japan, 75, 359–365.
Pereira, M., and L. Berre, 2006: The use of an ensemble approach to study the background error covariances in a global NWP. Mon. Wea. Rev., 134, 2466–2489.
Purser, R., W. Wu, D. Parrish, and N. Roberts, 2003a: Numerical aspects of the application of recursive filters to variational statistical analysis. Part I: Spatially homogeneous and isotropic Gaussian covariances. Mon. Wea. Rev., 131, 1524–1535.
Purser, R., W. Wu, D. Parrish, and N. Roberts, 2003b: Numerical aspects of the application of recursive filters to variational statistical analysis. Part II: Spatially inhomogeneous and anisotropic general covariances. Mon. Wea. Rev., 131, 1536–1548.
Ravela, S., K. Emanuel, and D. McLaughlin, 2007: Data assimilation by field alignment. Physica D, 230, 127–145.
Skamarock, W., and Coauthors, 2008: A description of the advanced research WRF version 3. NCAR Tech. Note, 125 pp.
Snyder, C., and F. Zhang, 2003: Assimilation of simulated Doppler radar observations with an ensemble Kalman filter. Mon. Wea. Rev., 131, 1663–1677.
Stensrud, D. J., J.-W. Bao, and T. T. Warner, 2000: Using initial condition and model physics perturbations in short-range ensemble simulations of mesoscale convective systems. Mon. Wea. Rev., 128, 2077–2107.
Stensrud, D. J., N. Yussouf, D. C. Dowell, and M. C. Coniglio, 2009: Assimilating surface data into a mesoscale model ensemble: Cold pool analyses from spring 2007. Atmos. Res., 93 (1–3), 207–220.
Torn, R., G. Hakim, and C. Snyder, 2006: Boundary conditions for limited-area ensemble Kalman filters. Mon. Wea. Rev., 134, 2490–2502.
Wang, X., D. M. Barker, C. Snyder, and T. M. Hamill, 2008: A hybrid ETKF-3DVAR data assimilation scheme for the WRF model. Part I: Observing system simulation experiment. Mon. Wea. Rev., 136, 5116–5131.
Wu, W., R. Purser, and D. Parrish, 2002: Three-dimensional variational analysis with spatially inhomogeneous covariances. Mon. Wea. Rev., 130, 2905–2916.
Xiao, Q., and J. Sun, 2007: Multiple-radar data assimilation and short-range quantitative precipitation forecasting of a squall line observed during IHOP 2002. Mon. Wea. Rev., 135, 3381–3404.
Xiao, Q., Y.-H. Kuo, J. Sun, W.-C. Lee, D. Barker, and E. Lim, 2007: An approach of radar reflectivity data assimilation and its assessment with the inland QPF of Typhoon Rusa (2002) at landfall. J. Appl. Meteor. Climatol., 46, 14–22.
Zhang, F., C. Snyder, and J. Sun, 2004: Impacts of initial estimate and observations on convective-scale data assimilation with an ensemble Kalman filter. Mon. Wea. Rev., 132, 1238–1253.
Also called the meso-γ scale (Orlanski 1975).
