1. Introduction

The statistical analysis of innovation (observation minus forecast) vectors is currently the most accurate method for estimating observation and forecast error covariances in large-scale data assimilation. For practical purposes, the error covariances are often assumed to be describable by a small number of statistical parameters (such as the variance and the characteristic spatial scale or truncated spectral coefficients describing the gross features of the shape of the covariance function). To estimate these parameters, some advanced methods have been developed based on the maximum-likelihood criterion (Dee and da Silva 1999; Dee et al. 1999) and the Bayesian technique (Purser and Parrish 2000). These advanced methods incorporate the same statistical information about forecast and observation errors used in the data assimilation system. Thus, if the underlying assumptions on the error distributions are sufficiently accurate, they should produce more accurate parameter estimates than the traditional innovation method. The traditional method uses the least-squares technique to fit the parameterized error covariance functions to innovation covariances, so it may not be as sensitive as the advanced methods to those underlying assumptions. In this sense, the traditional method is probably still the most robust method in situations where the available innovation data are numerous and reasonably dense in space.

The traditional innovation method has been widely used for error covariance estimates (Gandin 1963; Rutherford 1972; Hollingsworth and Lönnberg 1986; Lönnberg and Hollingsworth 1986; Thiebaux et al. 1986; Bartello and Mitchell 1992; Dévényi and Schlatter 1994) in data-dense areas. In data-sparse areas the National Meteorological Center's “NMC method” (Parrish and Derber 1992; Derber and Bouttier 1999) is typically used to estimate forecast error correlation structures (and perhaps forecast error variances as well, but with additional assumptions) by analyzing the difference between two model forecasts verifying at the same time. This technique, however, cannot estimate observation error variances nor their vertical correlation structures, which must be approximated by other means. Thus, the innovation method remains preferable whenever possible. The method was recently revisited by Xu et al. (2001, henceforth referred to as Part I) with some refinement and applied to North America height innovation data from the Navy Operational Global Atmospheric Prediction System (NOGAPS; Hogan and Rosmond 1991). As a follow-up study, this paper is aimed at further improving the method for wind analyses. The improved method is applied to wind innovation data collected from NOGAPS over North America (between 25°–65°N and 60°–130°W) during the period from 1 March to 3 May 1999. The data quality control and assimilation system has been briefly described in section 2 of Part I and the detailed techniques can be found in the references cited in Part I.

The innovation vector method of Hollingsworth and Lönnberg (1986; henceforth referred to as HL), which uses truncated spectral representations of forecast error covariance functions with assumed homogeneity and isotropy in the horizontal, forms the basis for the current study. Building on this previous study, the spectral representations of wind forecast error covariance functions are reformulated in this paper. The detailed formulations are derived for the auto-covariance functions in section 2 and for the cross-covariance functions in the appendix. The basic assumptions and methodology for isolating the computation of observation and prediction errors are described in section 3. The method is illustrated for the horizontal dimension in section 4, and then extended to full space in section 5 with simplified and yet more rigorously constrained multilevel analysis. Conclusions follow in section 6.

2. Covariance and spectral formulations for wind forecast error

The wind forecast error is treated as a random vector field, denoted by v′ = (u′, υ′)^T in the pressure coordinates (x, y, p), where ( )^T denotes the transpose of ( ). This random vector field can be expressed in terms of streamfunction and velocity potential by

View Expanded

where ψ and χ represent the forecast errors in the streamfunction and velocity potential, respectively. Without loss of generality, we can assume that

uυψχ(2.2)

where 〈( )〉 denotes the ensemble mean of ( ) and will be computed as the time mean under the ergodicity assumption.

The two random scalar fields ψ and χ are assumed to be jointly homogeneous and isotropic in the horizontal; that is, their joint probability distributions are invariant with respect to a complete set of translations, rotations and mirror reflections of the system of points in (x, y). In other words, statistical moments depend on the configuration of the system of points for which they are formed, but not on the position of the (x, y). In this case, the second-order statistical moments or, say, the covariance functions of ψ and χ have the following forms:

View Expanded

where r = [(x_j − x_i)² + (y_j − y_i)²]^1/2 is the horizontal distance between the two points, (x_i, y_i) and (x_j, y_j), and p_m and p_n denote the vertical levels in the pressure coordinates (x, y, p). Here, the horizontal distance r is calculated along the great circle (see appendix A of Daley and Barker 2000) and the planar geometry of (x, y) represents a limited area centered at point (x_i, y_i) on the earth's spherical surface. As shown in (2.3), for simplicity the covariances can be sometimes written as functions of r only as long as their implicit dependence on (p_m, p_n) are well understood.

The vector wind forecast errors at points (x_i, y_i) and (x_j, y_j) can be projected onto the radial r-direction from (x_i, y_i) to (x_j, y_j) and the tangential direction (perpendicular to the r-direction with positive to the left). The resulting radial and tangential components, denoted by l′ and t′ respectively, are related to υ′ = (u′, υ′)^T by (l′, t′)^T = 𝗥υ′ where 𝗥 is the rotational matrix that rotates the x-axis to the r-direction [see (5.2.17) and Fig. 5.1 of Daley 1991]. The covariance functions in (2.3) can be related to the covariances of l′ and t′ as follows [see (5.2.23)–(5.2.24) of Daley 1991 and (A.4d) of this paper]:

View Expanded

where

View Expanded

Here, C_tl(r, p_m, p_n) = C_lt(r, p_n, p_m) in (2.5c) is derived from C_ψχ(r, p_m, p_n) = C_χψ(r, p_n, p_m) in (2.3c) by using (A4c,d).

With the assumed homogeneity and isotropy for ψ and χ, the covariance functions in (2.5) are invariant with respect to translations and rotations but not invariant with respect to mirror reflections of the system of points in (x, y). To see this, we denote by r = (x_j − x_i, y_j − y_i)^T the vector distance from point (x_i, y_i) to point (x_j, y_j). Note that r can be transformed to −r by either a 180° rotation or mirror reflection (around the middle point of r) of the system of points (x_i, y_i) and (x_j, y_j). If r is transformed to −r by the rotation, then t′(x_i, y_i, p_m) and l′(x_j, y_j, p_n) change to t′(x_j, y_j, p_m) and l′(x_i, y_i, p_n), respectively, so all the covariance functions in (2.5) remain invariant. However, if r is transformed to −r by the mirror reflection, then t′(x_i, y_i, p_m) and l′(x_j, y_j, p_n) change to −t′(x_j, y_j, p_m) and l′(x_i, y_i, p_n), respectively, so the two auto-covariance functions in (2.5) remain invariant but the cross-covariance function changes sign. This implies that the covariance functions of the radial and tangential components of υ′ are not invariant with respect to mirror reflections of the system of points in (x, y) unless it is further assumed that

View Expanded

The equivalence between (2.6a) and (2.6b) can be verified by (A5a,b) assuming that C_ψχ(r, p_m, p_n) and C_χψ(r, p_m, p_n) → 0 as r → ∞.

The condition in (2.6) is supported by the result in section 7.3 of HL and the result (Fig. 3c) of this study, so the assumed isotropy for the random vector field of the wind forecast error may be understood in terms of invariance with respect to both rotations and mirror reflections. However, by using the NMC method, the analysis of Derber and Bouttier (1999) suggests that a part of the divergent wind error (associated with χ) can be explained by the rotational wind error (associated with ψ) in the midlatitudes near the surface and near the jet stream (around the tropopause). This implies that the cross-covariance functions in (2.6) are not identically zero, so the assumed isotropy for the random vector field of the wind forecast error should be understood in terms of invariance with respect to rotations only. The analysis of Polavarapu (1995) also suggests that the cross-covariance between ψ and χ should be nonzero in the marine boundary layer, especially in the vicinity of cyclones. Thus, in general, we should consider both the auto-covariance and cross-covariance functions. Since the auto-covariance and cross-covariance functions can be analyzed separately [see (A.4)] and the cross-covariances are found to be very small in this paper, only the auto-covariance functions will be considered in the remaining part of this section, while the cross-covariance functions will be examined together with the partition of the covariance functions in the appendix.

In principle, C_ψψ(r) and C_χχ(r) can be obtained by integrating (2.4a,b) with given C_tt(r) and C_ll(r) and proper boundary conditions. In practice, since C_tt(r) and C_ll(r) will be estimated with their spectral expansions, C_ψψ(r) and C_χχ(r) can be easily derived in the spectral space. The power spectra of the two random scalar fields ψ and χ are given by the Hankel transformations of C_ψψ(r) and C_χχ(r) [see (8.9) part I of Panchev 1971], that is,

View Expanded

The inverse transformations are

View Expanded

Using (2.8a,b) and the identities (d²/dr² + r⁻¹d/dr)J₀(kr) = −k²J₀(kr) and (d²/dr² − r⁻¹d/dr)J₀(kr) = k²J₂(kr), we obtain

View Expanded

where J₂( ) is the second-order Bessel function. The inverses of (2.9a,b) are

View Expanded

Using (2.10a,b), the coupled differential equations (2.4a,b) can be transformed into the following two independent algebraic equations in the spectral space:

View Expanded

where

View Expanded

The inverses of (2.12a,b) are

View Expanded

These spectral formulations are new and their discrete forms will be used to estimate wind forecast error covariance functions over an finite radial range in the later sections.

The physical meanings of S_ll(k) and S_tt(k) can be interpreted as follows. In association with the streamfunction and velocity potential defined in (2.1), the random vector field υ′ can be partitioned into two parts: υ′ = υ^rot + υ^div where υ^rot = (−∂ψ/∂y, ∂ψ/∂x)^T defines the rotational (or nondivergent) part and υ^div = (∂χ/∂x, ∂χ/∂y)^T defines the divergent (or irrotational) part. The components of the covariance tensors 〈υ^rot(υ^rot)^T〉 and 〈υ^div(υ^div)^T〉 are not invariant with respect to a complete set of translations and rotations in (x, y), but the covariance functions of the radial and tangential components of either υ^rot or υ^div are invariant. In particular, since the cross-covariance function between the radial and tangential components of υ′ is related only to the cross-covariance between ψ and χ as shown in (A.4c)–(A.4d), the cross-covariance function between the radial and tangential components of υ^rot (or υ^div) is identically zero. This means that the covariance tensors 〈𝗥υ^rot(𝗥υ^rot)^T〉 and 〈𝗥υ^div(𝗥υ^div)^T〉 are diagonal where 𝗥 is the rotational matrix that rotates the x-axis to the r-direction. The traces of these two covariance tensors are given by [see (A.4a)–(A.4b)]

View Expanded

By substituting (2.8) into (2.14) and using (d²/dr² + r⁻¹d/dr)J₀(kr) = −k²J₀(kr) and (2.11), we obtain

View Expanded

Thus, physically, S_tt(k) and S_ll(k) are the power spectra of v^rot and v^div, respectively.

Using (2.12)–(2.13) and the identities: J₀(kr) + J₂(kr) = 2J₁(kr)/(kr) and J₀(kr) − J₂(kr) = 2J^′₁(kr), one can verify that

View Expanded

where J^′₁( ) ≡ ∂J₁( )/∂( ). Their inverses are

View Expanded

Here, (2.16)–(2.17) recover the classic results [see (8.43)–(8.44) part I of Panchev 1971]. As shown in (8.38) part I of Panchev 1971, S_ll(k) and S_tt(k) are the two principal components of the spectral tensor of υ′ in the wavenumber space. These mathematical interpretations are connected with the above physical interpretations of S_ll(k) and S_tt(k).

3. Basic assumptions and separation of observation and forecast errors

In the previous section, the vector wind forecast error υ′ = (u′, υ′) is assumed to be unbiased (or bias removed), homogeneous, and isotropic in the horizontal. In addition to the previous basic assumptions for the wind forecast error, here the wind observation error is assumed to be (i) independent of the forecast error, and (ii) not correlated between different stations. The radial and tangential components of the wind observation error are denoted by l ″ and t″, respectively, and these two components are not cross-correlated due to the assumed isotropy.

With the above assumptions, the innovation covariance formulation (4.1) of Part I for the separation of the forecast and observation errors in height can be extended to the innovation covariances of the two wind components: l^d = l ″ − l′ and t^d = t″ − t′. The assumed isotropy implies that

View Expanded

where ( )_im denotes the value of ( ) at the point (x_i, y_i, p_m) in pressure coordinates (x, y, p) and, similarly, ( )_jn denotes the value of ( ) at the point (x_j, y_j, p_n). Thus, although l^d and t^d become undetermined and the radial direction is undefined when r = 0, their variances can be computed, according to (3.1b), by

View Expanded

The innovation covariances can be coupled into 〈l^d_iml^d_jn〉 + 〈t^d_imt^d_jn〉 and 〈l^d_iml^d_jn〉 − 〈t^d_imt^d_jn〉 and then, by using (3.1a,b), partitioned into the following forms:

View Expanded

where

View Expanded

As in (3.1b), the assumed isotropy implies that C(0, p_m, p_n) = 2C_ll(0, p_m, p_n) = 2C_tt(0, p_m, p_n) and C^ob(p_m, p_n) = 2C^ob_ll(p_m, p_n) = 2C^ob_tt(p_m, p_n) = C^ob_uu(p_m, p_n) + C^ob_υυ(p_m, p_n) = 〈u^″_imu^″_in〉 + 〈υ^″_imυ^″_in〉, where u″ and υ″ denote the two components of vector wind observation error in (x, y) coordinates.

The two components, l^d and t^d, of the vector wind innovation are computed for each observation station pair [see appendix B of Daley and Barker (2000)]. Their associated covariances are calculated for each pair of observation stations and binned for each interval of Δr = 100 km over the range of 0 ≤ r ≤ D (=3000 km). The available station pairs within each bin are shown in Fig. 1 of Part I. Using (3.3a,b), the wind forecast and observation error covariances can be separated from the binned innovation covariances. This technique is used with truncated spectral formulations based on the continuous formulations derived in section 2. As explained in Part I, the use of truncated spectral expansions assumes implicitly that the background error fields are sufficiently smooth so that their covariance functions are also smooth and can be adequately represented by the truncated spectral expansions [see (4.1a,b)]. Furthermore, since the analysis will be performed over a finite range (r ≤ D = 3000 km) constrained by the limited data coverage, it is also assumed that the background error covariance function becomes sufficiently flat as r approaches the boundary of the finite range so that its horizontal derivative is negligibly small at the boundary of the finite range. This additional assumption is supported by the behavior of the binned innovation covariance data in the vicinity of the boundary of the finite range (see Figs. 1, 5, and 6 of HL and Figs. 1, 3, and 7 in this paper).

4. Single-level analysis

The following coupled spectral expansions of the forecast error covariance functions are used for least squares fitting to the binned covariance data:

View Expanded

where the summation Σ₀ is from i = 0 to i = M. These formulations are truncated discrete spectral forms of (2.13a,b) for the finite radial range of 0 ≤ r ≤ D (=3000 km). Here, k₀ = 0 corresponds to the constant term in the discrete spectral expansion, representing the effect of the large-scale components not included within the range of 0 ≤ r ≤ D. The remaining wavenumbers k_i (i = 1, 2, … M) are zeros of J^′₀(kD), where J^′₀( ) ≡ ∂J₀( )/∂( ). One can verify that k_i(i = 1, 2, …) are also zeros of J₁(kD), or zeros of kDJ^′₂(kD) + 2J₂(kD). Note that k₀ = 0 and J₂(k₀r) = 0, so i = 0 is actually excluded from the summation in (4.1b). We keep the summation in (4.1b) formally the same as in (4.1a) to facilitate the subsequent derivation [see (4.3a,b)]. As the large-scale component term is not really included in (4.1b), S_ll(k₀) − S_tt(k₀) cannot be estimated by least squares fitting of (4.1b). Thus, although the large-scale component can be estimated in terms of the kinetic energy S_ll(k₀) + S_tt(k₀) by the constant term obtained from least squares fitting of (4.1a), the partition between its divergent and rotational components [represented by S_ll(k₀) and S_tt(k₀), respectively] cannot be determined.

Note that (p_m, p_n) are implicit for the function forms in (4.1a,b). For the single-level analysis considered in this section, we have p_m = p_n. In this case, the auto-covariance functions C_ll(r) and C_tt(r) should be semidefinite positive over 0 ≤ r ≤ D. This implies that S_ll(k_i) ≥ 0 and S_tt(k_i) ≥ 0 or, equivalently,

View Expanded

These conditions are used to check the spectral coefficients obtained in (4.1a,b) from least squares fitting. The upper bound of the summations is set to M = 10. With this setting, the conditions in (4.2) can be satisfied or nearly satisfied for all i ≤ M at each pressure level. When one of the conditions is not exactly but nearly satisfied by a spectral component, a minimal adjustment is applied to this component to meet the condition marginally.

Under the constraints of (4.2), the spectral coefficients S_ll(k_i) ± S_tt(k_i) are obtained by least squares fitting of (4.1a,b) to the coupled innovation covariances [(3.3a,b) with p_m = p_n] binned for each Δr (=100 km) over the range of 0 ≤ r ≤ D. The weight used in the least squares formulation is proportional to r. As explained in Part I, this choice of weight is the simplest among three possible choices and it gives approximately the same results as the other two choices for least squares fitting of (4.1a,b). The resultant covariance functions are shown for the pressure levels of p_m = p_n = 850, 500, and 200 mb in Fig. 1, where the truncation is chosen to be M = 10 under the conditions in (4.2) to ensure S_ll(k_i) ≥ 0 and S_tt(k_i) ≥ 0 for i ≤ M. To compare with the global spectra in Fig. 2 of HL, the kinetic energy spectra S_ll(k_i) + S_tt(k_i) are multiplied by the square of J₀(k_iD) and plotted in Fig. 2 as functions of the global wavenumber K_i, where K_i is related to the local wavenumber k_i by K_i (K_i + 1) = (ak_i/D)² (for i ≥ 1) and a is radius of the earth (see Table 1 of HL). The spectra in Fig. 2 have roughly the same shapes as those in Fig. 2 of HL.

Using the identities: J₀(kr) + J₂(kr) = 2J₁(kr)/(kr) and J₀(kr) − J₂(kr) = 2J^′₁(kr), one can verify that (4.1a,b) can be combined into the following forms:

View Expanded

These are the truncated spectral forms of (2.17a,b) for the finite radial range of 0 ≤ r ≤ D. Note that J^′₁(kr) and J₁(kr)/(kr) → 1/2 as k → k₀ = 0. The large-scale spectral component terms on the right-hand sides of (4.3a) and (4.3b) both reduce to [S_ll(k₀) + S_tt(k₀)]/2. Thus, the covariance functions in (4.3a) and (4.3b) can be computed from the spectral coefficients S_ll(k_i) and S_tt(k_i) for 1 ≤ i ≤ M and S_ll(k_i) + S_tt(k_i) for i = 0, although the partition between S_ll(k₀) and S_tt(k₀) cannot be determined as explained earlier.

The computed covariance functions C_ll(r) and C_tt(r) are plotted for p_m = p_n = 850, 500, and 200 mb in Figs. 3a,b against the binned innovation covariances (symbols). As shown, the covariance functions fit closely the innovation covariances although the spectral coefficients are obtained by least squares fittings of (4.1a,b) rather than (4.3a,b). The spectra S_ll(k_i) and S_tt(k_i) should remain the same if they are obtained directly by least squares fittings of (4.3a,b). Note that S_ll(k_i) and S_tt(k_i) are coupled in (4.3a,b) and so should their least squares fittings. On the other hand, least squares fittings of (4.1a) and (4.1b) are not coupled and can be performed separately. Because of this, (4.1a,b) are used, preferably over (4.3a,b), for least squares fittings in this paper.

Binned cross-covariances 〈l^d_imt^d_jm〉 are plotted in Fig. 3c. In comparison with the covariances 〈l^d_iml^d_jm〉 and 〈t^d_imt^d_jm〉 plotted in Figs. 3a,b, these cross-covariances are very small. Since nearzero cross-covariances are obtained for all the vertical levels, C_lt(r) ≈ 0 can be assumed as in (2.6). The binned data in Figs. 3a and 3b are similar to those in Figs. 5 and 6 of HL. Note that C_ll(r) is positive for all separations of r ≥ 0, but C_tt(r) becomes negative before approaching zero at large separations. This structural difference is similar to that between Figs. 5 and 6 of HL. As explained by HL, based on the discussion of Batchelor (1953), this structural difference suggests that the wind forecast errors are largely nondivergent. The results in Figs. 3a,b, however, cannot tell how the wind forecast error is partitioned between the rotational and divergent components, because C_ll(r) ≠ C^div(r) and C_tt(r) ≠ C^rot(r) as noted from (2.4a,b) and (2.14a,b). The partition can be conveniently examined in the spectral space as shown below.

As explained in (2.15a,b), S_ll(k_i) and S_tt(k_i) are the power spectra of the rotational and divergent components of the wind forecast error, respectively. The obtained power spectra are plotted in Fig. 4 as functions of the local wavenumber index for 1 ≤ i ≤ M. As shown, the rotational component is larger than the divergent component for all the wavenumbers (1 ≤ i ≤ 10) and at all the three vertical levels. The ratio between rotational and divergent components is quite uniformly large for all the wavenumbers at 500 mb. This feature may be related to often observed weak divergence in the middle troposphere. At 200 mb, the rotational component has a sharp peak over the range of low wavenumbers (1 ≤ i ≤ 4) while the divergent component has a rather uniform distribution over the broad range of middle wavenumbers (2 ≤ i ≤ 7). These features may be related to the atmospheric flow at the tropopause (around 200 mb) where the large-scale flow is strong and largely nondivergent but the synoptic-scale flow is much less nondivergent than the large-scale flow.

The power spectra S_ll(k_i) and S_tt(k_i) were not shown in HL, but they appeared to be obtained by least squares fitting of zero order Bessel function expansions [see appendix A of Bartello and Mitchell (1992)]. By replacing J₂(k_ir) with J₀(k_ir) in (4.1b), their zero order Bessel function expansions can be similarly applied to our innovation data, but the resultant spectra (not shown) are found to be quite different from those in Fig. 4 obtained by using the spectral expansions (4.1) or (4.3). As shown in section 2, the correct spectral expansions should be (4.1) or (4.3) rather than the zero order Bessel function expansions in (A2) of Bartello and Mitchell (1992), although the streamfunction and velocity-potential spectra S_ψψ(k) and S_χχ(k) obtained by using (A3) of Bartello and Mitchell (1992) could be not very different from obtained in this paper [not shown but can be inferred from Fig. 4 by using (2.11a,b)].

The forecast error variance for the vector wind is given by [σ(p_m)]² = C(0, p_m, p_m) and can be estimated by the spectral expansion (4.1a) at the zero-distance limit as shown by the vertical interception of the fitted curve at r = 0 (see Fig. 1a). Then, according to (3.3a), the observation error variance for the vector wind, that is, [σ^ob(p_m)]² = C^ob(p_m, p_m) can be obtained by subtracting the forecast error variance from the vector wind innovation variance [σ^d(p_m)]². The resultant [σ^ob(p_m)]² is shown by the remaining part of the total variance above the interception point in Fig. 1a. The estimated forecast error and observation error are plotted together with the innovation in terms of their standard deviations, σ^d(p_m), σ^ob(p_m), and σ(p_m), as functions of the vertical levels in Fig. 5. By comparing these results with the errors estimated in Fig. 3 of HL for the European Centre for Medium-Range Weather Forecasts (ECMWF) wind data (for the period of 1 January–31 March 1983 over North America), we can see that the NOGAPS 6-h wind forecast error over North America is smaller than that of ECMWF 16 years ago by 20% or more, and the (radiosonde) wind observation error is also smaller than 16 years ago. The wind forecast error reduction may be explained by the enhanced resolution of the 1999 NOGAPS global spectral mode (T159 truncation, comparable to 0.75°, and 24 vertical levels) relative to the 1983 ECMWF global gridpoint model (1.875° resolution in the horizontal and 15 levels in the vertical). Another contributing factor to the wind forecast error reduction is that large numbers of aircraft and wind profiler data become available over North America since 1983. The enhanced resolution could also have improved the representativeness of the radiosonde observations and thus contributed indirectly to the reduction of the wind observation error in addition to the data quality control and bias correction techniques used by the 1999 NOGAPS (see section 2 of Part I).

Using (4.1a), the forecast error covariance function defined in (3.4) can be partitioned as follows:

CrC^LC^Sr(4.4)

where C^L = S_ll(k₀) + S_tt(k₀) is the large-scale part, C^S(r) = Σ₁ [S_ll(k_i) + S_tt(k_i)]J₀(k_ir) is the synoptic-scale part resolved in the range of 0 ≤ r ≤ D, and the summation Σ₁ is from i = 1 to i = M. Using the discrete forms of (2.15a,b), the synoptic-scale component can be further partitioned as follows:

C^SrC^rotrC^divr(4.5)

where C^rot(r) = Σ₁ S_tt(k_i)J₀(k_ir) and C^div(r) = Σ₁ S_ll(k_i)J₀(k_ir) are the rotational-wind component and divergent-wind component of C^S(r), respectively. In association with the above partitions, the following variances can be introduced:

View Expanded

The rms errors, defined by the square roots of the variances in (4.6), are plotted in Fig. 6. As shown by the σ^L(p_m) profile, the large-scale error is very small and nearly constant within the troposphere and stratosphere. The synoptic-scale error, shown by the σ^S(p_m) profile, is very close to the total forecast error, σ(p_m), and reaches a maximum around the tropopause. The rotational-component error, σ^rot(p_m), is about twice as large as the divergent-component error, σ^div(p_m). These profiles have roughly the same shapes as those in Figs. 8a and 10 of HL except that the σ^div(p_m) profile in Fig. 10 of HL showed two maxima (at 400 and 250 mb). The synoptic-scale error in Fig. 6 is reduced by about 20% and the large-scale error is reduced by about 50% in comparison with those in Fig. 8a of HL obtained for ECMWF 16 years ago. These partitioned error reductions are consistent with the total forecast error deduction in Fig. 5 in comparison with Fig. 3 of HL.

5. Multilevel analysis

When p_m ≠ p_n, the analysis can be done similarly to the above single-level analysis. However, since p_m ≠ p_n, the spectral coefficients S_ll(k_i) = S_ll(k_i, p_m, p_n) and S_tt(k_i) = S_tt(k_i, p_m, p_n) in (4.1a,b) are not constrained by the non-negative condition or the condition in (4.2). Instead, they are constrained by

View Expanded

These conditions are derived similarly to those in (6.1)–(6.2) of Part I and the details are omitted. With the upper bound of the summations set to M = 10, the two conditions in (5.1a,b) can be satisfied or nearly satisfied for all i ≤ M. When one of the conditions is not exactly but nearly satisfied by a spectral component, a minimal adjustment is applied to this component to meet the condition marginally. The involved adjustments are found to be small and cause no significant change but smooth the structures of the covariance functions.

The resultant C_ll(r, p_m, p_n) and C_tt(r, p_m, p_n) from the multilevel analysis are shown in Fig. 7a and Fig. 7b, respectively, for p_m = 500 mb and p_n = 700 and 400 mb. The zero-distance limit of C(r, p_m, p_n) = C_ll(r, p_m, p_n) + C_tt(r, p_m, p_n) gives the vertical forecast error covariance for the vector wind. Then, the vertical observation error covariance C^ob(p_m, p_n) is obtained by subtracting C(0, p_m, p_n) from the vector wind innovation covariance according to (3.3a). The associated correlation functions are defined by

View Expanded

The results are plotted in Figs. 8a,b as functions of p_n for fixed p_m = 200, 500, and 850 mb. The structures of R^ob(p_m, p_n) in Fig. 8b are similar to those in Fig. 4 of HL obtained by an indirect method of multilevel analysis.

The multilevel analysis in this paper is a direct method, which also estimates the three-dimensional structures of R_ll(r, p_m, p_n) ≡ C_ll(r, p_m, p_n)[σ(p_m)σ(p_n)/2]⁻¹ and R_tt(r, p_m, p_n) ≡ C_tt(r, p_m, p_n)[σ(p_m)σ(p_n)/2]⁻¹. The estimated R_ll(r, p_m, p_n) and R_tt(r, p_m, p_n) are plotted as functions of p_n for different r and fixed p_m = 500 mb in Figs. 9a,b. As shown, R_ll(r, p_m, p_n) decreases gradually and remains positive but R_tt(r, p_m, p_n) decreases rapidly and becomes negative as r increases from zero to 550 km. These features are most prominent when p_n = p_m (=500 mb) (also see Figs. 3a,b) and gradually decay and vanish as p_n shifts away from p_m = 500 mb up to 300 mb or down to 700 mb. As explained earlier, the correlation functions in Figs. 9a,b cannot tell how the wind forecast error is partitioned between the rotational and divergent components. The partitioned error correlation functions are examined in the remaining part of this section.

In association with the partitioned error covariance functions in (4.4)–(4.5) and error variances in (4.6), the partitioned error correlation functions can be defined as follows:

View Expanded

The vertical structures of these correlation functions are plotted as functions of p_n in Figs. 10a–c for p_m = 850, 500, 200 mb, respectively. The R^L(p_m, p_n) profile in Fig. 10a shows that the large-scale component error at 850 mb is negatively correlated with the errors at and above 500 mb. The R^L(p_m, p_n) profiles in Figs. 10b,c show that the large-scale component errors are positively correlated over very deep vertical layers above 850 mb. This is quite different from the profile in Fig. 9 of HL (for p_m = 500 mb), which exhibited a rapid decrease to negative at p_n = 200 mb. The R^S(0, p_m, p_n) profiles in Figs. 10a–c are very close to (slightly less positive than) their corresponding R(0, p_m, p_n) profiles in Fig. 8a. Their function values are mostly non-negative and largely positive over deep vertical layers (but not as deep as for the large-scale errors) above 850 mb. Unlike the large-scale errors, as shown in Fig. 10a, the synoptic-scale error at 850 mb has nearly zero correlation to the synoptic-scale errors at and above 500 mb. The R^rot(0, p_m, p_n) profiles are more positive than their corresponding R^S(0, p_m, p_n) profiles and the positive-correlation layers are relative deep. The R^rot(0, p_m, p_n) profiles in Fig. 10 have sharper and more symmetrically shaped peaks (at p_n = p_m) than those in Fig. 16 of HL. The R^div(0, p_m, p_n) profiles in Fig. 10 have much sharper and more symmetrically shaped peaks (at p_n = p_m) than those in Fig. 19 of HL. Away from the peaks (p_n = p_m), the R^div(0, p_m, p_n) profiles drop to negative values (in shallow layers above and below each peak layer) much more rapidly than those in Fig. 19 of HL.

The three-dimensional structures of R^rot(r, p_m, p_n) and R^div(r, p_m, p_n) are plotted as functions of p_n in Fig. 11a and Fig. 11b, respectively, for fixed p_m = 500 mb and r = 0, 350, 550, 950, and 1950 km. As shown, R^rot(r, p_m, p_n) maintains basically the same vertical structure as previously described (for r = 0) but decreases gradually to nearly zero as r increases from zero to 550 km. The vertical structure of R^div(r, p_m, p_n), however, becomes slightly broad (in the vertical) and shifts upward as r increases from zero to 550 km, and then becomes nearly zero as r increases further to 950 km. In the troposphere, the three-dimensional structure of R^rot(r, p_m, p_n), which is axisymmetric in the space of (r, p_n) for fixed p_m, can be characterized by a relatively narrow and deep positive core centered at r = 0 and p_n = p_m. The three-dimensional structure of R^div(r, p_m, p_n) can be characterized by a relatively wide and shallow positive core centered at r = 0 and p_n = p_m and coupled with two shallow negative regions above and below the positive core in the space of (r, p_n). These structures may be related to the potential vorticity (PV) dynamics of fully developed baroclinic waves in the troposphere (Hoskins et al. 1985). For example, as shown by Xu et al. (1998) and Xu and Gu (2000), the geostrophic flow inverted from PV anomalies in fully developed baroclinic waves tends to maintain the sign of relative vorticity through a deep layer between two coupled lower- and upper-level PV anomalies, while the induced secondary flow tends to change direction (from convergent to divergent or vice versa) between the coupled PV anomalies.

As in HL [see (6.8) of Part I], three horizontal length scales can be defined for the correlation functions in (5.4) [or covariance functions in (4.5)] as follows:

View Expanded

where the dependence on p_m = p_n is implicit. The vertical profiles of these three length scales are plotted in Fig. 12. As shown, L^rot(p_m) is slightly smaller than L^S(p_m), but L^div is larger than L^S(p_m) especially in the lower troposphere (below 700 mb) and upper troposphere (between 500 and 200 mb). Thus, the rotational component of wind forecast error has relatively small decorrelation lengths and the divergent component has relatively large decorrelation lengths in the lower and upper troposphere. These features are consistent with the above examined structures of R^rot(r, p_m, p_n) and R^div(r, p_m, p_n).

6. Conclusions

The method of statistical analysis of innovation vectors is further developed in two aspects: (i) improved spectral representations of wind forecast error covariance functions, and (ii) simplified and yet more rigorously constrained formulations for multilevel analysis. These spectral representations are derived in consistency with the assumed horizontal homogeneity and isotropy for the random vector field of wind forecast error (whose joint probability distributions are invariant with respect to translations and rotations but not necessarily to mirror reflections of the system of points in the horizontal as explained in section 2 and appendix). The derived spectral formulations have decoupled forms [see (4.1a,b)] and thus can be easily used with least squares fittings to binned innovation data. The multilevel analysis in this paper is a direct method that is simpler than the indirect method of HL. The method is basically the same as for the single-level analysis except that the innovation data are from two different vertical levels (instead of a single level) and the truncated spectra for the multilevel least squares fitting are constrained by the geometric mean of the two associated power spectral components [see (5.1a,b)] instead of the non-negative condition. As in Part I [see (6.4)–(6.5)], the multilevel analysis developed in this paper for wind innovation data is more rigorously constrained than that of HL.

The method is applied to wind innovation data collected from NOGAPS over North America. The results are compared with HL, showing a 20% overall reduction in wind forecast error and a slight reduction in wind observation error for the NOGAPS data in comparison with the ECMWF global model data 16 years ago. The estimated wind observation error covariances have basically the same vertical correlation structures as those in HL, and the estimated forecast error covariances show roughly the same three-dimensional correlation structures as those in HL. The forecast error power spectra (Fig. 2b) are also similar to those in HL.

The wind forecast error variance is partitioned into large-scale and synoptic-scale components and the latter is further partitioned into rotational and divergent components. In particular, the large-scale error is found to be very small and thus the synoptic-scale error is very close to the total forecast error. The rotational-component error is about twice as large as the divergent-component error. The vertical profiles of the synoptic-scale error and rotational-component error are similar to those in HL in their shapes, but the vertical profiles of the large-scale error and divergent-component error are not quite so similar. The synoptic-scale error is reduced by about 20% and the large-scale error is reduced by about 50% in comparison with those estimated by HL for ECMWF 16 years ago. These partitioned error reductions are consistent with the total forecast error deduction.

In the troposphere, the three-dimensional correlation structure for the rotational-component error can be characterized by a relatively narrow and deep positive core (centered at r = 0 and p_n = p_m), while the three-dimensional correlation structure for the divergent-component error can be characterized by a relatively wide and shallow positive core coupled with two shallow negative regions above and below. Thus, the rotational-component error has relatively small horizontal decorrelation lengths and the divergent component error has relatively large horizontal decorrelation lengths especially in the lower and upper troposphere. These features may be related to the potential vorticity dynamics of fully developed baroclinic waves in the troposphere.

As mentioned in the introduction, the innovation method studied in this paper is a traditional one, which uses the least squares technique to fit the parameterized error covariance functions (represented by truncated spectral expansions) to binned innovation covariances against separation. This is the first way of least squares fitting. Least squares fittings are also performed in two different ways (not presented in this paper): (i) fit the parameterized covariance functions directly to innovation covariance data cloud (for all station pairs without binning); (ii) fit the error correlation functions to innovation correlations (instead of covariances). The estimated error covariances are found to remain nearly the same for all three different ways. The third way is the same as used in HL, which assumes that the forecast error normalized by the innovation standard deviation at each station is homogeneous and isotropic in the horizontal and so should be the error correlations but not the error covariances. This underlying assumption is better satisfied than that in the first and second ways, because the innovation variance computed for each wind component (u or υ) at each observation station does exhibit moderate inhomogeneity from station to station [not shown in this paper, see Fig. 15b Lönnberg and Hollingsworth (1986)]. However, since the estimated error covariances are found to be insensitive to different ways of fitting (with different underlying assumptions), only the first way is presented in this paper. In light of its insensitivity, the traditional method does appear to be quite robust.