Evaluation of the Ensemble Transform Analysis Perturbation Scheme at NRL

Justin G. McLay NRC/Naval Research Laboratory, Monterey, California

Search for other papers by Justin G. McLay in
Current site
Google Scholar
PubMed
Close
,
Craig H. Bishop Naval Research Laboratory, Monterey, California

Search for other papers by Craig H. Bishop in
Current site
Google Scholar
PubMed
Close
, and
Carolyn A. Reynolds Naval Research Laboratory, Monterey, California

Search for other papers by Carolyn A. Reynolds in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

The ensemble transform (ET) scheme changes forecast perturbations into analysis perturbations whose amplitudes and directions are consistent with a user-provided estimate of analysis error covariance. A practical demonstration of the ET scheme was undertaken using Naval Research Laboratory (NRL) Atmospheric Variational Data Assimilation System (NAVDAS) analysis error variance estimates and the Navy Operational Global Atmospheric Prediction System (NOGAPS) numerical weather prediction (NWP) model. It was found that the ET scheme produced forecast ensembles that were comparable to or better in a variety of measures than those produced by the Fleet Numerical and Oceanography Center (FNMOC) bred-growing modes (BGM) scheme. Also, the demonstration showed that the introduction of stochastic perturbations into the ET forecast ensembles led to a substantial improvement in the agreement between the ET and NAVDAS analysis error variances. This finding is strong evidence that even a small-sized ET ensemble is capable of obtaining good agreement between the ET and NAVDAS analysis error variances, provided that NWP model deficiencies are accounted for. Last, since the NAVDAS analysis error covariance estimate is diagonal and hence ignores multivariate correlations, it was of interest to examine the ET analysis perturbations’ spatial correlation. Tests showed that the ET analysis perturbations exhibited statistically significant, realistic multivariate correlations.

Corresponding author address: Justin G. McLay, Naval Research Laboratory, 7 Grace Hopper Ave., Stop 2, Monterey, CA 93943-5502. Email: mclay@nrlmry.navy.mil

Abstract

The ensemble transform (ET) scheme changes forecast perturbations into analysis perturbations whose amplitudes and directions are consistent with a user-provided estimate of analysis error covariance. A practical demonstration of the ET scheme was undertaken using Naval Research Laboratory (NRL) Atmospheric Variational Data Assimilation System (NAVDAS) analysis error variance estimates and the Navy Operational Global Atmospheric Prediction System (NOGAPS) numerical weather prediction (NWP) model. It was found that the ET scheme produced forecast ensembles that were comparable to or better in a variety of measures than those produced by the Fleet Numerical and Oceanography Center (FNMOC) bred-growing modes (BGM) scheme. Also, the demonstration showed that the introduction of stochastic perturbations into the ET forecast ensembles led to a substantial improvement in the agreement between the ET and NAVDAS analysis error variances. This finding is strong evidence that even a small-sized ET ensemble is capable of obtaining good agreement between the ET and NAVDAS analysis error variances, provided that NWP model deficiencies are accounted for. Last, since the NAVDAS analysis error covariance estimate is diagonal and hence ignores multivariate correlations, it was of interest to examine the ET analysis perturbations’ spatial correlation. Tests showed that the ET analysis perturbations exhibited statistically significant, realistic multivariate correlations.

Corresponding author address: Justin G. McLay, Naval Research Laboratory, 7 Grace Hopper Ave., Stop 2, Monterey, CA 93943-5502. Email: mclay@nrlmry.navy.mil

1. Introduction

Ensemble prediction is a comprehensive undertaking that involves quantifying and sampling the error probability density functions (PDFs) of both the analyzed atmospheric state and a numerical weather prediction (NWP) model, evolving the samples within the NWP model to generate a forecast PDF, and postprocessing the forecast PDF. The treatment of the analysis PDF to obtain initial states for integration is the historical centerpiece of ensemble prediction and is a fundamental step in the undertaking. In this treatment, more specifically, the analysis PDF must be defined given estimates of error in observed and background fields, and then a representative but finite sample suitable for evolution must be taken from this PDF. No consensus has been reached on how to accomplish these two tasks.

Regarding the sampling of analysis error PDFs, the operational schemes include the ensemble transform (ET) scheme of the National Centers for Environmental Prediction (NCEP; Bishop and Toth 1999; Wei et al. 2006), the ensemble transform Kalman filter (ETKF) scheme of the Met Office (UKMET; Wang and Bishop 2003), the breeding of growing modes (BGM) scheme of the U.S. Navy’s Fleet Numerical Meteorology and Oceanography Center (FNMOC; Toth and Kalnay 1997), the singular vector (SV) scheme of the European Centre for Medium-Range Weather Forecasts (ECMWF; Molteni et al. 1996), and the perturbed observation (PO) scheme of the Meteorological Service of Canada (MSC; Houtekamer et al. 1996). The SV scheme imposes an explicit dynamical constraint on the sampling, and the ET, ETKF, BGM, and PO schemes also impose some dynamical constraint on the sampling as a result of operating within a cycling framework. The dynamically constrained approach specifically seeks those samples from the analysis error PDF that spawn growth relative to the ensemble-mean forecast or the forecast initiated from the control analysis. The argument for a dynamical constraint is that if the important structural features of the forecast error probability distribution are determined by growing forecast errors, as some studies suggest (e.g., Ehrendorfer and Tribbia 1997), then given a small sample of forecasts these features are best described when each of the samples is a realization of a growing error. The unconstrained sampling approach is based upon random sampling of the full analysis error PDF, not just that part of the PDF associated with growing errors (Leith 1974; Houtekamer and Derome 1995). At present this approach is not used operationally, in spite of indications from some simple model studies that it may provide ensemble forecasts that are better under certain metrics than corresponding forecasts from the dynamically constrained approach (Anderson 1997).

There are long-standing problems inherent to some of the operational dynamically constrained sampling methods. Specifically, the SV scheme suffers from great computational expense, while the BGM scheme has a tendency to produce analysis perturbations whose variance is concentrated in considerably fewer eigen-directions than there are perturbations, even when the perturbations are unpaired (Wang and Bishop 2003; Wei et al. 2006). The absence of variability in some eigen-directions is undesirable given that the ensemble perturbations are already too few to span all the directions in which analysis error variability really exists. A promising alternative to the SV and BGM sampling schemes is found in the ET technique. The ET was introduced in Bishop and Toth (1999) as a method of deciding where and when to deploy weather reconnaissance aircraft in order to reduce the error variance of forecasts likely to have a high socioeconomic impact. However, as Wei et al. (2004, 2006) first demonstrated, the ET also can be used for dynamically constrained sampling without the expense of the SV scheme and the undesirable eigen-spectrum issue of the BGM scheme. Furthermore, the ET constrains perturbation selection using estimates of analysis error covariance. A variant of the ET scheme has been in use at NCEP since May 2006. In this variant, the basic ET algorithm is followed by a postprocessing step in which a rescaling mask is applied to ensure that the ET perturbations are consistent with analysis error variance estimates. The NCEP analysis error variance estimates are derived from a climatology of differences between analyses generated from two perturbed data assimilation cycles (Toth and Kalnay 1997).

The objective of this paper is to discuss several points pertaining to the practical demonstration of the basic ET algorithm. The first point concerns the practical performance of a forecast ensemble generated from the basic ET algorithm. In this case, analysis error variance estimates are obtained directly from the Naval Research Laboratory (NRL) Atmospheric Variational Analysis System (NAVDAS) and NWP integrations are undertaken with the Navy Operational Global Atmospheric Prediction System (NOGAPS). The second point relates to how ensemble size and NWP model-error mitigation are reflected in the ET analysis-perturbation variance. The final point relates to the existence of spatial correlation within the ET analysis perturbations given small ensemble size and the use of a diagonal matrix of analysis error covariance estimates. Discussion of these points will provide insight into the practical workings of the basic ET that is not forthcoming from the ET’s theoretical properties. The paper is organized as follows: Section 2 gives a synopsis of the theoretical properties of the basic ET scheme. Section 3 describes the NAVDAS analysis error statistics, the NOGAPS NWP model, and the ensemble configuration used for the practical implementation of the basic ET. Section 4 details the aforementioned points pertaining to the practical demonstration. Conclusions follow in section 5. Comprehensive derivations of the theoretical properties of the ET are provided in the appendix.

2. The ET initial-state selection technique

a. Formulation of the ET technique

Let 𝗣ag = 𝗙𝗗ag𝗙T be an N × N matrix whose elements list the best available guess of the error covariance of the initial condition, where N is the number of variables defining the model state, 𝗙 = [f1, f2, . . . , fN] is an N × N orthogonal matrix listing the eigenvectors of 𝗣ag, and 𝗗ag = diag(d11, d22, . . . , dNN) is a diagonal matrix listing the corresponding eigenvalues of 𝗣ag. In the present study, 𝗣ag is obtained directly from NAVDAS. Let 𝗫′f = [xf1, xf2, . . . , xfK] be an N × K matrix whose ith column is given by
i1520-0493-136-3-1093-e1
and where xfi gives the ith member of the ensemble forecast and represents the mean of the ensemble forecast. Provided the perturbations xfi are “balanced” in the sense that they represent the difference between forecasts that are relatively free of spurious inertia–gravity waves and “small” in the sense that the nonlinear terms associated with the equations governing their evolution are negligible, then any linear combination
i1520-0493-136-3-1093-e2
that results in another small perturbation will also be balanced. Furthermore, if the forecast perturbations xfi are growing, orthogonal, and small, then any linear combination of the forecast perturbations will also be growing. The ET generates K initial perturbations 𝗫′a = [xa1, xa2, . . . , xaK] from K forecast perturbations via a K × K transformation matrix 𝗧 = [t1, t2, . . . , tK] using the formula
i1520-0493-136-3-1093-e3
i1520-0493-136-3-1093-e4
and where Γ = diag(γ11, γ22, . . . , γKK) is a K × K diagonal matrix to be defined below and 𝗖 = [c1, c2, . . . , cK] is a K × K orthogonal matrix containing the eigenvectors of the symmetric matrix (𝗫′fT𝗣a−1g𝗫′f/N). In other words,
i1520-0493-136-3-1093-e5
where Λ = diag(λ11, λ22, . . . , λKK) is a K × K diagonal matrix listing the eigenvalues of 𝗫′fT(𝗣ag)−1𝗫′f/N. Note that in the present study the 𝗣ag from NAVDAS is diagonal, and so the inverse is trivial to obtain. Since the sum of the forecast perturbations is equal to zero, one of these eigenvalues will be equal to zero. Consequently, provided each ensemble contains K − 1 linearly independent perturbations, Λ can be written in the form
i1520-0493-136-3-1093-e6
where 𝗚 is a (K − 1) × (K − 1) diagonal matrix whose elements are all greater than zero and is a K − 1 vector whose elements are all equal to zero. The Γ used in (4) is obtained from Λ by setting its zero diagonal value equal to 1; in other words,
i1520-0493-136-3-1093-e7
Note that while Γ has an inverse, the inverse of Λ does not exist. This adjustment of the eigenvalue matrix is permissible because it does not affect the sample covariance matrix of initial perturbations implied by (3). To see this, first note that pre- and postmultiplying (5) by the eigenvector cK corresponding to the zero eigenvalue λK = 0 shows that
i1520-0493-136-3-1093-e8
Second, note that if λii and γii denote the diagonal elements of Λ and Γ, respectively, then because of (8), (3), and (4) we may deduce that the initial perturbation sample covariance 𝗣ae is given by
i1520-0493-136-3-1093-e9
and hence is entirely independent of the difference between Λ and Γ. Throughout this discussion we will assume that every ensemble contains K − 1 linearly independent ensemble perturbations.
In understanding the properties of the perturbations 𝗫′a it is useful to realize that
i1520-0493-136-3-1093-e10
i1520-0493-136-3-1093-e11
Note that since 𝗖T𝗖 = 𝗜, the sample covariance matrix of the columns of 𝗫′aorth = [xaorth_1, xaorth_2, . . . , xaorth_K] is identical to the sample covariance of the columns of 𝗫′a given by (9), and hence properties of the columns of 𝗫′aorth are implicitly incorporated in the sample covariance of 𝗫′aorth. Note that (8) implies that the Kth column of 𝗫′aorth is the zero vector. As shown below, the columns of 𝗫′aorth are orthogonal under the norm (y; x) = yT𝗣a−1gx. For this reason, the subscript “orth” stands for orthogonal. Some specific properties of interest are as follows.

b. Properties of the ET technique

Note that proofs of the following properties are provided in the appendix.

1) Property 1

For all nonzero columns 𝗫′aorth (i.e., for all columns except the Kth column), the average value of (xaTifjfTjxai)/djj, which is the square of the projection of the ith orthogonal ensemble perturbation onto the jth eigenvector of 𝗣ag divided by the jth eigenvalue of 𝗣ag, is equal to (K − 1)/K. Since sample variance estimates are given by the average squared deviation multiplied by K/(K − 1), this variance is precisely what is required of the perturbations in order that they be consistent, on a globally averaged basis, with the variance estimates contained in 𝗣ag.

2) Property 2

Under the analysis error variance norm, the cosine of the angle between nonidentical columns of 𝗫′a is −1/(K − 1) provided that K is less than N. For the 32-member ensemble considered in this paper, the corresponding angle between each of the analysis perturbations is 91.8°. Hence, even with a relatively small ensemble the ET analysis perturbations are almost orthogonal.

3) Property 3

If analysis errors have a Gaussian distribution, the sample covariance obtained from (9) may be viewed as representing the sample covariance of a set of equally likely perturbations.

4) Property 4

If the number of independent ensemble perturbations K − 1 is equal to the number of variables N, then the sample covariance of the initial perturbations 𝗣ae generated by the ET technique will equal 𝗣ag. For smaller ensemble sizes 𝗣ae ≠ 𝗣ag and the ET 𝗣ae may be thought of as a low-rank but balanced approximation to 𝗣ag.

5) Property 5

The sum of the initial perturbations 𝗫′a is zero, as shown in Wei et al. (2006). This property is desirable because it ensures that the mean of the perturbations at the initial time is precisely equal to the best available analysis. Wang et al. (2004) referred to such ensembles as spherical simplex ensembles.

6) Property 6

If the ensemble size K = 2 and the linear equation
i1520-0493-136-3-1093-e12
describing the propagation of perturbations from one analysis time ti to the next analysis time ti+1 is time invariant, then after an infinite number of ensemble forecast/assimilation cycles, the ET initial perturbations will have a structure identical to the fastest-growing eigenvector of 𝗠.

This property suggests that when the ensemble is small and the dynamical operator does not change much from one assimilation time to the next, then there is a good chance that the ensemble perturbations will be composed of some of the most rapidly growing eigenstructures present in the system. This is a useful property to have, since infinitesimal analysis errors tend to amplify in chaotic systems such as the atmosphere and ocean. Property 6 also shows that the ET ensemble-generation scheme has some similarity with Toth and Kalnay’s (1997) simple breeding scheme in which analysis perturbations are obtained by forecast perturbations by multiplying them by a number whose magnitude is less than 1. However, properties 1 and 2 ensure that ET ensemble perturbations maintain error variance in as many directions as the smaller of K − 1 or the rank r of 𝗣ag, unlike simple breeding perturbations.

7) Property 7

If the NWP forecast perturbations, 𝗫′f, satisfy a linearized balance equation then the ET analysis perturbations 𝗫′a will also satisfy this balance equation. This property follows from the fact that each ET analysis perturbation is a simple linear combination of the global forecast perturbations. Any type of masking using coefficients that vary with location and/or variable will lead to this property being violated.

8) Property 8

The majority of the floating-point operations in the ET algorithm can be performed in parallel.

3. Details of the practical demonstration of the ET

a. NAVDAS analysis error statistics, 𝗣ag

NAVDAS employs a three-dimensional variational data assimilation (3DVAR) algorithm that yields a maximum likelihood state estimate given a forecast, observations, and accurately specified error covariance matrices. The minimization required for the analysis increment is performed in observation space. This is accomplished with the aid of block diagonal preconditioning matrices obtained by dividing the three-dimensional analysis space into prisms equally populated with observations. An extension of the block diagonal approach conveniently permits estimation of analysis error variance for locations within a given prism using information about the observations and first-guess errors within that prism. These “local” analysis error variance estimates will reflect isolated and transient observations, unlike analysis error variance estimates obtained by alternative, nonlocal methods. Figure 1 shows examples of the NAVDAS analysis error standard deviation fields on two consecutive days and the difference between them. Note in Fig. 1c the substantial difference over the central Atlantic Ocean as well as near the west coast of Mexico. This difference is due entirely to day-to-day variation in the observing network. The reader is referred to Daley and Barker (2001), Gelaro et al. (2002), and Reynolds et al. (2005) for further discussion of the NAVDAS implementation.

b. The NOGAPS model

Both the ET and BGM schemes are implemented in conjunction with the NOGAPS global model for testing. The formulation of NOGAPS (Hogan and Rosmond 1991) is spectral in the horizontal coordinates and energy conserving finite difference in the vertical coordinate. The model uses vorticity and divergence, virtual potential temperature, specific humidity, and terrain pressure as the dynamic variables, with a semi-implicit treatment of gravity wave propagation. The physical parameterizations include boundary layer turbulence (Louis et al. 1982), moist convection (Emanuel and Zivkovic-Rothman 1999), convective and stratiform clouds (Teixeira and Hogan 2002), and solar and longwave radiation (Harshvardhan et al. 1987). NOGAPS is the global NWP model of the U.S. Navy and drives several applications such as the Coupled Ocean Atmosphere Mesoscale Prediction System (COAMPS; Hodur 1997) and the Navy aerosol prediction model. For the present set of experiments, the NOGAPS is run with spectral resolution T119 and 30 vertical levels.

c. Ensemble configuration

The ET and BGM global ensembles are each configured with 32 perturbed members, where each member involves perturbations in wind, temperature, terrain pressure, and specific humidity. The ensembles are run with a 24-h cycling interval over the period 1 August 2005 to 30 September 2005. In the case of the BGM ensemble, the 32 members are obtained through the addition to and subtraction from the control analysis of 16 different perturbations. The BGM scheme used for comparison here employs a spatially invariant scaling factor to obtain the analysis perturbations from the forecast perturbations. Thus, this BGM scheme is less sophisticated than the one formerly used at NCEP, which employed a spatially variant scaling factor (Toth and Kalnay 1997). Note that the BGM ensemble configuration described here is identical to the one used operationally at FNMOC with the exception that the configuration described here has twice as many perturbed members.

4. Findings from practical demonstration of the ET

a. Performance of forecast ensemble generated using the basic ET

The favorable properties listed in section 2b do not in themselves guarantee satisfactory ET forecast-ensemble performance. The performance will be a function of many other factors including cycling interval, the size of the NWP model state vector (N), the size of the ensemble (K), NWP model errors, and the quality of 𝗣ag. Some factors, such as the NWP model errors and 𝗣ag, may be dependent upon the prediction center that is doing the ET implementation. There may be interrelationships among factors. For instance, as suggested in section 4b the NWP model errors may influence how well 𝗣ae conforms to 𝗣ag for a given ensemble size. Note also that if 𝗣ag is a poor estimate then satisfactory forecast-ensemble performance may not be obtained even if 𝗣ae conforms quite well to 𝗣ag. These points make it essential to assess the practical performance of a given ET forecast ensemble.

Here, forecasts from the ET and BGM ensembles described in section 3 are evaluated using multiple diagnostics. Neither ensemble includes any model-error remediation. The ET ensemble does not incorporate the rescaling of Wei et al. (2006). Thus, the evaluation provides a comparison between a baseline ET ensemble and a baseline BGM ensemble.

1) Root-mean-square error of ensemble mean

Figure 2 presents the rmse of the ensemble-mean 10-m wind and 500-hPa geopotential height as a function of ensemble, lead time, and region. The ET ensemble generally has a modest advantage over the BGM ensemble. Allowing for the lack of an extensive sample of ensemble-mean/verification pairs to use in calculating the diagnostic, the results suggest that the ET ensemble mean is at least as skillful as the BGM ensemble mean.

2) Ensemble-average perturbation dry total energy

Figure 3 shows the ensemble-average domain-integrated1 perturbation dry total energy as a function of ensemble, lead time, and region. This diagnostic can be thought of as describing a domain-integrated, normalized perturbation variance. Figure 3 indicates that the ET perturbations are more variant than the BGM perturbations at 0 h and that the ET perturbations continue to be more variant through 144 h. The difference in variance is most pronounced at early lead times. The larger ET variance is (in a climatological-average sense) closer to the observed forecast error variance for all lead times in the global and tropical domains and for lead times of 72 h or more in the Northern Hemisphere extratropical domain. The BGM variance could, of course, be bolstered simply by changing the scaling factor (see section 3c) so that the 0 h BGM variance is larger. However, in the absence of any such additional step, the ET perturbations have an overall more appropriate level of variance as gauged by the observed error variance.

3) Binning of squared-error/ensemble-variance pairs

Ensemble variance ought to be a good predictor of the magnitude of the error in the control forecast: large ensemble variance should be associated with larger errors in the control forecast, on average, than small ensemble variance is. Such a relationship is desired for practical purposes because it gives an assessment of flow-dependent predictability. Wang and Bishop (2003) evaluate this relationship with the following procedure: Pairs of ensemble variance and squared forecast error (forecast − verifying analysis) are ordered from smallest ensemble variance to largest ensemble variance, and then the ordered list is divided into approximately equally populated bins. Finally, the bin-averaged squared error is plotted against the bin-averaged ensemble variance. Ideally, the points of the plot would define a line that has a slope of one and that intercepts the ordinate at a value equal to the analysis error variance. In the presence of model error that is not represented by the ensemble, the points would be expected to lie above the 45° line of the plot. It is also desired that the range of the squared error encompassed by the plot points be as large as possible, because this range is directly related to the ensemble’s ability to distinguish large forecast error variance from small forecast error variance. Figure 4 presents some examples of the binning results from the experiment. It is seen that for 10-m wind in the tropics the ET ensemble points span a slightly greater range of squared error than the BGM points and have an orientation closer to the ideal, the 45° line. Thus, for 10-m wind in the tropics the ET ensemble has a more favorable relationship between variance and squared-error. For other regions and other variables, one generally finds that the ET has an advantageous relationship for lead times beyond approximately 48 to 72 h.

4) Eigenvalue spectrum of 𝗣 fe

As described in section 2b, the eigenvalue spectrum of the ET ensemble analysis error covariance matrix 𝗣ae is perfectly flat under the analysis error variance norm equation (5) whereas that of the BGM 𝗣 fe ensemble analysis error covariance matrix is quite sloped. The expectation is that the eigenvalue spectrum of a given ET ensemble forecast error covariance matrix 𝗣 fe will also be flatter than that of the BGM ensemble forecast error covariance matrix. Figure 5 shows the eigenvalue spectrum of 𝗣 fe under the dry total energy norm as a function of ensemble and lead time. The eigen-spectrum of the ET-based 𝗣 fe is indeed consistently flatter than that of the BGM-based 𝗣 fe. Not surprisingly, the difference in slope of the two eigenspectra does decrease with lead time, as the fastest-growing errors come to dominate both ensembles.

5) Brier score

An ensemble’s fundamental purpose is to provide for probabilistic prediction of events. The Brier score for an event can be interpreted as the mean-squared error of the ensemble-derived probability of the event (Wilks 2006). Figure 6 presents the Brier score for the event that the 10-m zonal-wind magnitude exceeds 5 m s−1 as a function of domain, lead time, and ensemble. To obtain the domain-averaged Brier score for each lead time in Figs. 6a–d, the Brier score was first calculated at each grid point in the domain as the temporal mean (over the 46 dates between 15 August 2005 and 30 September 2005) of the squared error of the raw ensemble-derived event probability. Then, the gridded Brier scores were spatially averaged with area weighting. Figures 6a–d indicate that the ET ensemble has better Brier scores for the given event at all lead times and for all domains. The relative improvement of the ET-ensemble Brier score is greatest in the tropical domain, and smallest in the Northern Hemisphere extratropical domain. Similar results are obtained when the event is defined using a threshold magnitude of 2.5 and 10 m s−1 (not shown). These results, though not comprehensive, suggest that the ET ensemble is more capable than the BGM ensemble at the fundamental task of approximating event probabilities.

b. Agreement between 𝗣ae and 𝗣ag

Recall from section 2a that the ET scheme incorporates a constraint [(5)] that ensures global agreement between the ET analysis perturbations and the analysis error variance estimates. Also recall from property 4 of section 2b that unless the number of independent ensemble members K − 1 equals the dimension of the state vector N or equals or exceeds the rank of 𝗣ag that 𝗣ae will not equal 𝗣ag. Property 4 means that one can expect some discrepancy between 𝗣ae and 𝗣ag owing to small ensemble size. Figure 7 indicates how this discrepancy is manifest in the ET ensemble described in section 3, in terms of a pole-to-pole profile of the ET ensemble-average perturbation dry total energy in comparison to the NAVDAS dry total energy. Quite obviously the 00-h ET ensemble variance is too large in the midlatitudes and too small in the tropics. Wei et al. (2006) independently reveal this same circumstance. The latitudinal structure exhibited by the error variance discrepancy is consistent with the global constraint imposed by (5). In particular, given that the ET analysis perturbations are underdispersive in the tropics relative to 𝗣ag, (5) requires that they be overdispersive in the extratropics relative to 𝗣ag.

Since radiation and convection play a key role in the dynamics of the tropical atmosphere, it seems plausible that the lack of ET ensemble variance in the tropics might owe primarily to the fact that no attempt was made to represent NWP model error in the experiment that produced Fig. 7. To test this hypothesis, stochastic perturbations were introduced into the ensemble in regions where the ensemble-derived forecast errors exhibited a variance deficit. To ensure that the stochastic perturbations were balanced, they were defined to lie in a vector subspace whose basis was given by a historical archive of forecast-ensemble perturbations (see McLay et al. 2007 for details). Figure 8 shows the result of adding the stochastic perturbations as a postprocessing exercise to an ET ensemble of the same configuration as the one described in section 3. The stochastic perturbations were added to the 24-h lead time ET forecast perturbations of a given ensemble-generation cycle immediately prior to transformation of those forecast perturbations for the next cycle. This was done for all cycles in the 6-week period from 31 December 2004 to 11 February 2005. Looking at Figs. 8a,b, it can be seen that the postprocessing has a pronounced effect on the variance profile of the ET analysis wind perturbations: The curves for the wind perturbations are much flatter, with greatly reduced peaks in the extratropics. A corresponding curve for the temperature perturbations exhibits improvement that is less pronounced but still considerable (not shown). These results demonstrate that a small-sized ET ensemble is capable of defining a 𝗣ae that is a good match to 𝗣ag, provided that NWP model errors are accounted for. By extension, the results also strongly suggest that the discrepancy shown in Fig. 7 is at least partly due to model errors.

c. Averaged spatial and multivariate correlation for the ET ensemble

Analysis errors are expected to have considerable spatial and multivariate correlation, owing to dynamically adjusted errors present in the background fields of the data assimilation process. Hence, ideally one should employ a 𝗣ag that includes spatial and multivariate error covariance. In practice, however, it may be expedient to use a diagonal 𝗣ag, as was the case in this study. If one insists on using a diagonal 𝗣ag, then the ET will attempt to produce analysis perturbations that have zero spatial and multivariate correlation. Also, the analysis perturbations will exhibit progressively less correlation as the ensemble size K − 1 increases, because of property 4 of section 2b. It follows that the use of a diagonal 𝗣ag may force the ET to produce analysis perturbations with unrealistic correlation, particularly when the ensemble size is very large. Note in addition that when 𝗣ag is diagonal that attempts to improve the agreement between the diagonal elements (i.e., the variances) of 𝗣ae and 𝗣ag through an increase in ensemble size may sacrifice realistic error correlation structure. This gives added value to the results of Fig. 8 in section 4b, since those results do not involve an increase in ensemble size.

Property 4 also implies that for cases in which K − 1 is less than the rank of 𝗣ag the character of the ET analysis perturbations’ spatial and multivariate correlation structure is a priori unknown. Examination of the average ensemble correlations is one way to get a general sense of this structure. Figure 9 depicts the average spatial correlation that is present within a 27-member ET ensemble with an implementation like that described in section 3. To create Fig. 9, one-point correlation maps were calculated for each of 170 different grid points within the latitude band between 20° and 60°N. Each of the 170 different grid points was separated from the others by 10° in both latitude and longitude. Each correlation map was 21° × 21° in dimension and was centered on the given grid point. The average of the 170 different correlation maps was determined, and then the significance of the average correlations was assessed using an adaptation of the Monte Carlo approach detailed by Vautard et al. (1990). Those average correlations that are significant at the 95% level or greater are plotted in Fig. 9. Clearly, there are numerous significant correlations. Furthermore, the correlation patterns are qualitatively similar to those described by the correlation models used in 3DVAR systems such as NAVDAS (not shown). Thus, the average correlations obtained using a diagonal 𝗣ag and a relatively small ensemble size are consistent with the estimates that have been derived for use in data assimilation.

5. Conclusions

The ensemble transform is a method of selecting initial states for ensemble prediction. It takes into consideration estimates of analysis error covariance and employs a dynamical sampling approach. The ET analysis perturbations for a given ensemble are equally likely under a multivariate Gaussian assumption, balanced, conditioned for growth, and asymptotically orthogonal under an analysis error covariance norm as ensemble size increases. Also, the eigenvalue spectrum of the ET analysis perturbation covariance matrix for a given ensemble is flat under an analysis error covariance norm. This property cannot be introduced by the BGM scheme, which does not control the eigenvalue spectrum of the analysis perturbations and which consequently produces perturbations with variance in fewer directions than the available number of perturbations. In addition, the ET is more economical than the singular vector scheme. Hence, the ET is in theory an attractive alternative to other initial-state selection schemes that employ dynamical sampling.

The basic ET scheme was implemented for demonstration purposes using a 24-h cycling interval, a 32-member ensemble, and analysis error variance estimates obtained from the NRL Atmospheric Variational Data Assimilation System. Numerical weather prediction model integrations were performed using the Navy Operational Global Atmospheric Prediction System. Forecast ensembles were initialized at 0000 UTC of each of the 57 days in the period 6 August 2005 to 30 September 2005.

The practical demonstration of the ET highlighted several important points surrounding the ET’s operation. The first point relates to the performance of the forecast ensemble generated using the basic ET. Multiple aspects of the forecast ensemble were evaluated, including root-mean-square error of the ensemble mean, ensemble-average spatially integrated dry total energy, ensemble forecast error covariance matrix eigenvalue spectrum, the relationship between ensemble variance and observed squared error, and the Brier score. In these measures, the basic ET scheme was found to produce forecast ensembles comparable to or better than those produced by the FNMOC BGM scheme. Thus, the theory of the ET scheme translates into useful forecast ensembles. This holds even when the ensemble size is relatively small, and even when modifications related to rescaling and stochastic perturbations are neglected. Wei et al. (2006) obtained similar results using different analysis error variance estimates and a different NWP model. Hence, the results above are also important because they confirm those in Wei et al. (2006) under independent circumstances.

Second, consistent with theory, some disparity between the ET ensemble analysis error variance and the NAVDAS analysis error variance estimates was observed. Notably, however, the introduction of tropical stochastic perturbations into the ET forecast ensembles led to substantial improvement in the agreement between the ET and NAVDAS analysis error variances. This finding suggests that the disparity between the ET and NAVDAS analysis error variances is exacerbated by NWP model deficiencies in the tropics. It also demonstrates that even a small-sized ET ensemble is capable of defining analysis error variances that agree well with those from NAVDAS, provided that the NWP model deficiencies are accounted for.

A third point is in regard to the spatial and multivariate correlation of the ET analysis perturbations. The average correlations for ET perturbations obtained using a diagonal 𝗣ag and a relatively small ensemble size were found to be statistically significant and to exhibit patterns consistent with correlation models used in data assimilation.

Acknowledgments

This research was sponsored by the Naval Research Laboratory and the Office of Naval Research under Program Element 0601153N, project number BE-033-03-4M. The DoD High Performance Computing program at NAVO MSRC and at FNMOC provided the computing resources. The authors thank Mike Sestak for producing the FNMOC BGM ensemble used in the comparison experiment, and two anonymous reviewers for their helpful comments.

REFERENCES

  • Anderson, J. L., 1997: The impact of dynamical constraints on the selection of initial conditions for ensemble predictions: Low-order perfect model results. Mon. Wea. Rev., 125 , 29692983.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., and Z. Toth, 1999: Ensemble transformation and adaptive observations. J. Atmos. Sci., 56 , 17481765.

  • Daley, R., and E. Barker, 2001: NAVDAS source book 2001. Naval Research Laboratory Rep. NRL/PU/7530–01-441, 161 pp.

  • Ehrendorfer, M., and J. J. Tribbia, 1997: Optimal prediction of forecast error covariances through singular vectors. J. Atmos. Sci., 54 , 286313.

    • Search Google Scholar
    • Export Citation
  • Emanuel, K. A., and M. Zivkovic-Rothman, 1999: Development and evaluation of a convection scheme for use in climate models. J. Atmos. Sci., 56 , 17661782.

    • Search Google Scholar
    • Export Citation
  • Gelaro, R., T. Rosmond, and R. Daley, 2002: Singular vector calculations with an analysis error variance metric. Mon. Wea. Rev., 130 , 11661186.

    • Search Google Scholar
    • Export Citation
  • Harshvardhan, R. Davies, D. A. Randall, and T. G. Corsetti, 1987: A fast radiation parameterization for atmospheric circulation models. J. Geophys. Res., 92 , 10091016.

    • Search Google Scholar
    • Export Citation
  • Hodur, R. M., 1997: The Naval Research Laboratory’s Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS). Mon. Wea. Rev., 125 , 14141430.

    • Search Google Scholar
    • Export Citation
  • Hogan, T. F., and T. E. Rosmond, 1991: The description of the Navy Operational Global Atmospheric Prediction System’s spectral forecast model. Mon. Wea. Rev., 119 , 17861815.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and J. Derome, 1995: Methods for ensemble prediction. Mon. Wea. Rev., 123 , 21812196.

  • Houtekamer, P. L., L. Lefaivre, J. Derome, H. Ritchie, and H. L. Mitchell, 1996: A system simulation approach to ensemble prediction. Mon. Wea. Rev., 124 , 12251242.

    • Search Google Scholar
    • Export Citation
  • Leith, C. E., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102 , 409418.

  • Louis, J. F., M. Tiedtke, and J. F. Geleyn, 1982: A short history of the operational PBL parameterization at ECMWF. Proc. ECMWF Workshop on Planetary Boundary Parameterization, Reading, United Kingdom, European Centre for Medium-Range Weather Forecasts, 59–79.

    • Search Google Scholar
    • Export Citation
  • McLay, J. G., C. H. Bishop, and C. A. Reynolds, 2007: The ensemble-transform scheme adapted for the generation of stochastic forecast perturbations. Quart. J. Roy. Meteor. Soc., 133 , 12571266.

    • Search Google Scholar
    • Export Citation
  • Molteni, F., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF Ensemble Prediction System: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122 , 73119.

    • Search Google Scholar
    • Export Citation
  • Reynolds, C. A., R. Gelaro, and T. E. Rosmond, 2005: A comparison of variance and total-energy singular vectors. Quart. J. Roy. Meteor. Soc., 131 , 19551973.

    • Search Google Scholar
    • Export Citation
  • Teixeira, J., and T. F. Hogan, 2002: Boundary layer clouds in a global atmospheric model: Simple cloud cover parameterizations. J. Climate, 15 , 12611276.

    • Search Google Scholar
    • Export Citation
  • Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125 , 32973319.

  • Vautard, R., K. C. Mo, and M. Ghil, 1990: Statistical significance test for transition matrices of atmospheric Markov chains. J. Atmos. Sci., 47 , 19261931.

    • Search Google Scholar
    • Export Citation
  • Wang, X., and C. H. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60 , 11401158.

    • Search Google Scholar
    • Export Citation
  • Wang, X., C. H. Bishop, and S. J. Julier, 2004: Which is better, an ensemble of positive–negative pairs or a centered spherical simplex ensemble? Mon. Wea. Rev., 132 , 15901605.

    • Search Google Scholar
    • Export Citation
  • Wei, M., Z. Toth, R. Wobus, Y. Zhu, and C. H. Bishop, 2004: Initial perturbations for the NCEP ensemble forecast system. Proc. First THORPEX Int. Science Symp., Montreal, QC, Canada, World Meteorological Organization, 227–230.

  • Wei, M., Z. Toth, R. Wobus, and Y. Zhu, 2006: Initial perturbations based on the ensemble transform (ET) technique in the NCEP global ensemble forecast system. NOAA/NCEP Office Note 453, 33 pp.

  • Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Academic Press, 627 pp.

APPENDIX

Proofs of ET Properties

Properties (1)(3) make use of the following observations. Note that these observations were originally described in slightly different form in Wei et al. (2006). First,
i1520-0493-136-3-1093-ea1
where cK is the Kth eigenvector. Its eigenvalue is zero and from (8) and the fact that it is a unit vector, it always takes the form cTK = 1T = [1, 1, . . . , 1]/K. Using this fact in (A1) gives
i1520-0493-136-3-1093-ea2

Property 1

i1520-0493-136-3-1093-ea3

Property 2

From (A2), the cosine [cos(θij)] of the angle between the ith and jth ET analysis perturbations under the analysis error covariance norm is given by
i1520-0493-136-3-1093-ea4
where ij.

Property 3

If analysis errors have a Gaussian distribution then the probability ρ of an initial condition error εa would be given by
i1520-0493-136-3-1093-ea5
Equation (A2) states that the expression in brackets is identical for all K ET analysis perturbations. Hence the K − 1 independent members are equally likely.

Property 4

If K = N + 1, then 𝗫′f is an N × (N + 1) matrix and 𝗖 is an (N + 1) × (N + 1) matrix. It will prove helpful to express 𝗖 in the form 𝗖 = [𝗨(N+1)×N, cN+1] where 𝗨(N+1)×N lists the first N columns of 𝗖. Under these conditions 𝗫′f𝗖 is an N × (N + 1) matrix. However, from (8) it is clear that the (N + 1)th column of this matrix is the zero vector. Hence,
i1520-0493-136-3-1093-ea6
Now since 𝗫′f𝗨(N+1)×(N) is an N × N matrix and since we have assumed that the ensemble contains K − 1 = N linearly independent perturbations, it follows that the inverse of 𝗫′f𝗨(N+1)×(N) exists. Equation (5) defining 𝗖 and Λ may then be manipulated to prove that, with an ensemble of this size, 𝗣ag = 𝗣ae. The algebraic argument is as follows. First, note that
i1520-0493-136-3-1093-eqa1
It follows that
i1520-0493-136-3-1093-ea7
For smaller ensemble sizes 𝗣ae ≠ 𝗣ag. This lack of equality for small ensemble sizes can be an advantage in cases where the estimate 𝗣ag is very crude and does not, for example, account for dynamical balances. In such circumstances, 𝗣ae is balanced even though 𝗣ag is not because 𝗣ae is based upon balanced analysis perturbations.

Property 5

This property was first proven by Wei et al. (2006). Let 1T = [1, 1, . . . , 1]/K be a K-vector and note that
i1520-0493-136-3-1093-ea8
Consequently, from Eqs. (5) and (A8)
i1520-0493-136-3-1093-eqa2

Property 6

Assume that K = 2, xa1 = −xa2, and xf1 = −xf2. After some analysis, it may be shown that
i1520-0493-136-3-1093-ea9
where λ0 gives the single nonzero eigenvalue associated with (5) for a two-member ensemble and the subscripts refer to the analysis time from which each forecast is initialized. Note that the update Eq. (A9) is identical to the ensemble update used in the method of ensemble generation known as “breeding” (see Toth and Kalnay 1997). Equation (A9) implies that
i1520-0493-136-3-1093-ea10
Thus, the columns of 𝗫′am have the structure of 𝗠mxa0. Assuming that 𝗠 is diagonalizable, it may be written in the eigenvector form 𝗠 = 𝗨Σ𝗨−1 where 𝗨 lists the complex eigenvectors of the nonsymmetric matrix 𝗠 and the diagonal matrix Σ lists the corresponding complex eigenvalues. Since
i1520-0493-136-3-1093-ea11
if the magnitude of the first complex eigenvalue σ1 is larger than the magnitudes of all the other eigenvalues, then as the number of assimilation/forecast cycles goes to infinity
i1520-0493-136-3-1093-ea12
because |(σi/σ1)m| = (|σi|/|σ1|)m will tend to zero as m tends to infinity, provided |σi| < |σ1|. Consequently, 𝗠mxa0 will have a structure directly proportional to u1, the fastest growing eigenvector of 𝗠.

Property 7

The linearized balance equation can be represented in operator form. Property 7 then follows from the definition of a linear operator.

Property 8

To perform an ET ensemble update, one first needs to form the K × K matrix given by (5). The symmetric matrix has (K2/2) + K/2 = K(K + 1)/2 unique terms. Each of these terms represents an inner product. The formation of each inner product requires N multiplications and N − 1 additions. Thus, the formation of the matrix in (5) requires [K(K + 1)/2](2N − 1) floating-point operations. The eigenvector decomposition required to obtain the right-hand side of (5) depends on the routine employed but is generally of order K3. The formation of the 𝗧 matrix according to (4) involves ≈2K2 floating-point operations, and the evaluation of 𝗫′f𝗧 involves ≈2KN floating-point operations. In most oceanic and atmospheric applications NO(106) while KO(102). Thus, the majority of the floating-point operations associated with the update occur during the construction of the inner products required for the formation of the matrix [(5)]. However, since each of the inner products is entirely independent of the other, all K(K + 1)/2 of these inner products could be performed on a separate processor.

Fig. 1.
Fig. 1.

(a) NAVDAS-derived standard deviation of 200-hPa zonal wind analysis error for 0000 UTC 23 Jan 2007. Shaded contours every 0.2 m s−1 as indicated by the grayscale bar. (b) As in (a) but for 0000 UTC 24 Jan 2007. (c) The difference between (b) and (a). Shaded contours every 0.1 m s−1 as indicated by the grayscale bar.

Citation: Monthly Weather Review 136, 3; 10.1175/2007MWR2010.1

Fig. 2.
Fig. 2.

(a) Area-weighted rmse of the ET and BGM ensemble-mean 10-m zonal wind averaged over the 57-day period from 6 Aug to 30 Sep 2005. Black solid (dash–dot) line indicates the rmse for the ET (BGM) ensemble in the midlatitude regions of 60° to 20°S and 20° to 60°N. Light gray solid (dash–dot) line indicates the rmse for the ET (BGM) ensemble in the tropical region of 20°S to 20°N. (b) As in (a) except that the variable is 500-hPa geopotential height.

Citation: Monthly Weather Review 136, 3; 10.1175/2007MWR2010.1

Fig. 3.
Fig. 3.

(a) Ensemble-average perturbation dry total energy for the 0000 UTC ET (BGM) ensemble horizontally and vertically integrated over the global NWP model domain and averaged over the 57-day period from 6 Aug 2005 to 30 Sep 2005, indicated by black solid line with filled dots (light gray solid line with filled dots). Error dry total energy integrated over the same domain and averaged over the same 57-day period indicated by light gray dashed line with open triangles. The error total energy for the 00-h lead time is derived from the 0000 UTC NAVDAS analysis error variance estimates. The error total energy for all lead times following the 00-h lead time is derived from the 57-day sample variance of error in the ensemble control forecast, where the error is defined as the control forecast minus the verifying analysis. (b) As in (a) except that the total energy is integrated over the Northern Hemisphere extratropical domain, defined as the region north of 20°N latitude. (c) As in (a) except that the total energy is integrated over the tropical domain, defined as the region between 20°S and 20°N latitude.

Citation: Monthly Weather Review 136, 3; 10.1175/2007MWR2010.1

Fig. 4.
Fig. 4.

(a) Scatter diagram involving bin-mean ensemble variance and bin-mean squared error for 24-h lead time 0000 UTC 10-m zonal wind in the NWP model tropical domain, defined as the region between 20°S and 20°N latitude. The binning process is based on 100 bins and makes use of all the 24-h lead time 0000 UTC 10-m wind data in the tropical domain during the 57 days in the period 6 Aug 2005 to 30 Sep 2005. The black filled circles (gray open circles) denote the scatter points for the ET (BGM) ensembles. For reference, the black (gray) solid line is a least squares fit line to the scatter points for the ET (BGM) ensemble. The 45° line of the scatter diagram is indicated by a gray dotted line. (b) As in (a) but for the 72-h lead time 0000 UTC 10-m zonal wind. (c) As in (a) but for the 120-h lead time 0000 UTC 10-m zonal wind.

Citation: Monthly Weather Review 136, 3; 10.1175/2007MWR2010.1

Fig. 5.
Fig. 5.

(a) Average eigenvalue spectrum of the 24-h lead time ensemble forecast error covariance matrix under the dry total energy norm. The eigenvalue spectrum of each 24-h lead time 0000 UTC ensemble forecast error covariance matrix in the 57-day period 6 Aug 2005 to 30 Sep 2005 is normalized by the leading eigenvalue, and the resulting normalized spectra are averaged over the 57-day period. Black closed circles (open squares) denote the average nonzero eigenvalues of the 24-h lead time 0000 UTC ET (BGM) ensemble forecast error covariance matrix. (b) As in (a) but for the 72-h lead time. (c) As in (a) but for the 120-h lead time.

Citation: Monthly Weather Review 136, 3; 10.1175/2007MWR2010.1

Fig. 6.
Fig. 6.

(a) Area-weighted average over the domain 90°S to 90°N of the Brier score for the event that the 10-m zonal wind magnitude exceeds 5 m s−1. ET (BGM) ensemble Brier score indicated by black (gray) solid line. (b) As in (a) but for the domain 20° to 90°N. (c) As in (a) but for the domain 90° to 20°S. (d) As in (a) but for the domain 20°S to 20°N. See text for details on the calculation of the Brier score.

Citation: Monthly Weather Review 136, 3; 10.1175/2007MWR2010.1

Fig. 7.
Fig. 7.

Zonally averaged, vertically integrated 0000 UTC ensemble-average ET analysis perturbation (NAVDAS analysis error) total energy averaged over the 57-day period from 6 Aug 2005 to 30 Sep 2005 indicated by black solid line (light gray solid line). Note that the total energy values are not weighted by the cosine of latitude.

Citation: Monthly Weather Review 136, 3; 10.1175/2007MWR2010.1

Fig. 8.
Fig. 8.

(a) Zonally and vertically averaged ratio of ET ensemble zonal wind analysis error variance and NAVDAS zonal wind analysis error variance for 0000 UTC 31 Dec 2004, the last date prior to start of stochastic perturbation (open-circle light gray line), for 0000 UTC 22 Jan 2005 after 3 weeks of stochastic perturbation (closed-circle black line), and for 0000 UTC 12 Feb 2005 after 6 weeks of stochastic perturbation (open-square black line). The ideal ratio of 1 is marked by the horizontal black dotted line. (b) As in (a) except that the variable is meridional wind.

Citation: Monthly Weather Review 136, 3; 10.1175/2007MWR2010.1

Fig. 9.
Fig. 9.

(a) Average one-point correlation map for the case of the uu correlation at sigma-level 10 (∼300 hPa) (i.e., the case where the correlation is between the zonal wind at sigma-level 10 at the center point of the map and the zonal wind at sigma-level 10 at each of the other points of the map). Positive (negative) contours every 0.10 (−0.10) beginning at 0.05 (−0.05) given by solid (dotted) lines. (b) As in (a) but for the case of the υu correlation at sigma-level 10. Positive (negative) contours every 0.05 (−0.05) beginning at 0.05 (−0.05). See the text for details on the construction of the correlation map.

Citation: Monthly Weather Review 136, 3; 10.1175/2007MWR2010.1

1

The domains referenced in this paper are bounded in the vertical by the surface and the 100-hPa level.

Save
  • Anderson, J. L., 1997: The impact of dynamical constraints on the selection of initial conditions for ensemble predictions: Low-order perfect model results. Mon. Wea. Rev., 125 , 29692983.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., and Z. Toth, 1999: Ensemble transformation and adaptive observations. J. Atmos. Sci., 56 , 17481765.

  • Daley, R., and E. Barker, 2001: NAVDAS source book 2001. Naval Research Laboratory Rep. NRL/PU/7530–01-441, 161 pp.

  • Ehrendorfer, M., and J. J. Tribbia, 1997: Optimal prediction of forecast error covariances through singular vectors. J. Atmos. Sci., 54 , 286313.

    • Search Google Scholar
    • Export Citation
  • Emanuel, K. A., and M. Zivkovic-Rothman, 1999: Development and evaluation of a convection scheme for use in climate models. J. Atmos. Sci., 56 , 17661782.

    • Search Google Scholar
    • Export Citation
  • Gelaro, R., T. Rosmond, and R. Daley, 2002: Singular vector calculations with an analysis error variance metric. Mon. Wea. Rev., 130 , 11661186.

    • Search Google Scholar
    • Export Citation
  • Harshvardhan, R. Davies, D. A. Randall, and T. G. Corsetti, 1987: A fast radiation parameterization for atmospheric circulation models. J. Geophys. Res., 92 , 10091016.

    • Search Google Scholar
    • Export Citation
  • Hodur, R. M., 1997: The Naval Research Laboratory’s Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS). Mon. Wea. Rev., 125 , 14141430.

    • Search Google Scholar
    • Export Citation
  • Hogan, T. F., and T. E. Rosmond, 1991: The description of the Navy Operational Global Atmospheric Prediction System’s spectral forecast model. Mon. Wea. Rev., 119 , 17861815.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and J. Derome, 1995: Methods for ensemble prediction. Mon. Wea. Rev., 123 , 21812196.

  • Houtekamer, P. L., L. Lefaivre, J. Derome, H. Ritchie, and H. L. Mitchell, 1996: A system simulation approach to ensemble prediction. Mon. Wea. Rev., 124 , 12251242.

    • Search Google Scholar
    • Export Citation
  • Leith, C. E., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102 , 409418.

  • Louis, J. F., M. Tiedtke, and J. F. Geleyn, 1982: A short history of the operational PBL parameterization at ECMWF. Proc. ECMWF Workshop on Planetary Boundary Parameterization, Reading, United Kingdom, European Centre for Medium-Range Weather Forecasts, 59–79.

    • Search Google Scholar
    • Export Citation
  • McLay, J. G., C. H. Bishop, and C. A. Reynolds, 2007: The ensemble-transform scheme adapted for the generation of stochastic forecast perturbations. Quart. J. Roy. Meteor. Soc., 133 , 12571266.

    • Search Google Scholar
    • Export Citation
  • Molteni, F., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF Ensemble Prediction System: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122 , 73119.

    • Search Google Scholar
    • Export Citation
  • Reynolds, C. A., R. Gelaro, and T. E. Rosmond, 2005: A comparison of variance and total-energy singular vectors. Quart. J. Roy. Meteor. Soc., 131 , 19551973.

    • Search Google Scholar
    • Export Citation
  • Teixeira, J., and T. F. Hogan, 2002: Boundary layer clouds in a global atmospheric model: Simple cloud cover parameterizations. J. Climate, 15 , 12611276.

    • Search Google Scholar
    • Export Citation
  • Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125 , 32973319.

  • Vautard, R., K. C. Mo, and M. Ghil, 1990: Statistical significance test for transition matrices of atmospheric Markov chains. J. Atmos. Sci., 47 , 19261931.

    • Search Google Scholar
    • Export Citation
  • Wang, X., and C. H. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60 , 11401158.

    • Search Google Scholar
    • Export Citation
  • Wang, X., C. H. Bishop, and S. J. Julier, 2004: Which is better, an ensemble of positive–negative pairs or a centered spherical simplex ensemble? Mon. Wea. Rev., 132 , 15901605.

    • Search Google Scholar
    • Export Citation
  • Wei, M., Z. Toth, R. Wobus, Y. Zhu, and C. H. Bishop, 2004: Initial perturbations for the NCEP ensemble forecast system. Proc. First THORPEX Int. Science Symp., Montreal, QC, Canada, World Meteorological Organization, 227–230.

  • Wei, M., Z. Toth, R. Wobus, and Y. Zhu, 2006: Initial perturbations based on the ensemble transform (ET) technique in the NCEP global ensemble forecast system. NOAA/NCEP Office Note 453, 33 pp.

  • Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Academic Press, 627 pp.

  • Fig. 1.

    (a) NAVDAS-derived standard deviation of 200-hPa zonal wind analysis error for 0000 UTC 23 Jan 2007. Shaded contours every 0.2 m s−1 as indicated by the grayscale bar. (b) As in (a) but for 0000 UTC 24 Jan 2007. (c) The difference between (b) and (a). Shaded contours every 0.1 m s−1 as indicated by the grayscale bar.

  • Fig. 2.

    (a) Area-weighted rmse of the ET and BGM ensemble-mean 10-m zonal wind averaged over the 57-day period from 6 Aug to 30 Sep 2005. Black solid (dash–dot) line indicates the rmse for the ET (BGM) ensemble in the midlatitude regions of 60° to 20°S and 20° to 60°N. Light gray solid (dash–dot) line indicates the rmse for the ET (BGM) ensemble in the tropical region of 20°S to 20°N. (b) As in (a) except that the variable is 500-hPa geopotential height.

  • Fig. 3.

    (a) Ensemble-average perturbation dry total energy for the 0000 UTC ET (BGM) ensemble horizontally and vertically integrated over the global NWP model domain and averaged over the 57-day period from 6 Aug 2005 to 30 Sep 2005, indicated by black solid line with filled dots (light gray solid line with filled dots). Error dry total energy integrated over the same domain and averaged over the same 57-day period indicated by light gray dashed line with open triangles. The error total energy for the 00-h lead time is derived from the 0000 UTC NAVDAS analysis error variance estimates. The error total energy for all lead times following the 00-h lead time is derived from the 57-day sample variance of error in the ensemble control forecast, where the error is defined as the control forecast minus the verifying analysis. (b) As in (a) except that the total energy is integrated over the Northern Hemisphere extratropical domain, defined as the region north of 20°N latitude. (c) As in (a) except that the total energy is integrated over the tropical domain, defined as the region between 20°S and 20°N latitude.

  • Fig. 4.

    (a) Scatter diagram involving bin-mean ensemble variance and bin-mean squared error for 24-h lead time 0000 UTC 10-m zonal wind in the NWP model tropical domain, defined as the region between 20°S and 20°N latitude. The binning process is based on 100 bins and makes use of all the 24-h lead time 0000 UTC 10-m wind data in the tropical domain during the 57 days in the period 6 Aug 2005 to 30 Sep 2005. The black filled circles (gray open circles) denote the scatter points for the ET (BGM) ensembles. For reference, the black (gray) solid line is a least squares fit line to the scatter points for the ET (BGM) ensemble. The 45° line of the scatter diagram is indicated by a gray dotted line. (b) As in (a) but for the 72-h lead time 0000 UTC 10-m zonal wind. (c) As in (a) but for the 120-h lead time 0000 UTC 10-m zonal wind.

  • Fig. 5.

    (a) Average eigenvalue spectrum of the 24-h lead time ensemble forecast error covariance matrix under the dry total energy norm. The eigenvalue spectrum of each 24-h lead time 0000 UTC ensemble forecast error covariance matrix in the 57-day period 6 Aug 2005 to 30 Sep 2005 is normalized by the leading eigenvalue, and the resulting normalized spectra are averaged over the 57-day period. Black closed circles (open squares) denote the average nonzero eigenvalues of the 24-h lead time 0000 UTC ET (BGM) ensemble forecast error covariance matrix. (b) As in (a) but for the 72-h lead time. (c) As in (a) but for the 120-h lead time.

  • Fig. 6.

    (a) Area-weighted average over the domain 90°S to 90°N of the Brier score for the event that the 10-m zonal wind magnitude exceeds 5 m s−1. ET (BGM) ensemble Brier score indicated by black (gray) solid line. (b) As in (a) but for the domain 20° to 90°N. (c) As in (a) but for the domain 90° to 20°S. (d) As in (a) but for the domain 20°S to 20°N. See text for details on the calculation of the Brier score.

  • Fig. 7.

    Zonally averaged, vertically integrated 0000 UTC ensemble-average ET analysis perturbation (NAVDAS analysis error) total energy averaged over the 57-day period from 6 Aug 2005 to 30 Sep 2005 indicated by black solid line (light gray solid line). Note that the total energy values are not weighted by the cosine of latitude.

  • Fig. 8.

    (a) Zonally and vertically averaged ratio of ET ensemble zonal wind analysis error variance and NAVDAS zonal wind analysis error variance for 0000 UTC 31 Dec 2004, the last date prior to start of stochastic perturbation (open-circle light gray line), for 0000 UTC 22 Jan 2005 after 3 weeks of stochastic perturbation (closed-circle black line), and for 0000 UTC 12 Feb 2005 after 6 weeks of stochastic perturbation (open-square black line). The ideal ratio of 1 is marked by the horizontal black dotted line. (b) As in (a) except that the variable is meridional wind.

  • Fig. 9.

    (a) Average one-point correlation map for the case of the uu correlation at sigma-level 10 (∼300 hPa) (i.e., the case where the correlation is between the zonal wind at sigma-level 10 at the center point of the map and the zonal wind at sigma-level 10 at each of the other points of the map). Positive (negative) contours every 0.10 (−0.10) beginning at 0.05 (−0.05) given by solid (dotted) lines. (b) As in (a) but for the case of the υu correlation at sigma-level 10. Positive (negative) contours every 0.05 (−0.05) beginning at 0.05 (−0.05). See the text for details on the construction of the correlation map.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 252 41 1
PDF Downloads 69 17 2