• Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129 , 28842903.

  • Anderson, J. L., 2003: A local least squares framework for ensemble filtering. Mon. Wea. Rev., 131 , 634642.

  • Anderson, J. L., 2006: Exploring the need for localization in ensemble data assimilation using an hierarchical ensemble filter. Physica D, in press.

    • Search Google Scholar
    • Export Citation
  • Beljaars, A. C. M., and A. A. M. Holtstag, 1991: Flux parameterization over land surfaces for atmospheric models. J. Appl. Meteor., 30 , 327341.

    • Search Google Scholar
    • Export Citation
  • Ek, M. B., K. E. Mitchell, Y. Lin, P. Grunmann, V. Koren, G. Gayno, and J. D. Tarpley, 2003: Implementation of Noah land surface model advances in the National Centers for Environmental Prediction operational mesoscale Eta model. J. Geophys. Res., 108 .8851, doi:10.1029/2002JD003296.

    • Search Google Scholar
    • Export Citation
  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99 , 1014310162.

    • Search Google Scholar
    • Export Citation
  • Furrer, R. H., and T. Bengtsson, 2007: Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants. J Multivar. Anal., 98 , 227255.

    • Search Google Scholar
    • Export Citation
  • Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125 , 723757.

    • Search Google Scholar
    • Export Citation
  • Gneiting, T., 1999: Correlation functions for atmospheric data analysis. Quart. J. Roy. Meteor. Soc., 125 , 24492464.

  • Hacker, J. P., and C. Snyder, 2005: Ensemble Kalman filter assimilation of fixed screen-height observations in a parameterized PBL. Mon. Wea. Rev., 133 , 32603275.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., J. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129 , 27762790.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126 , 796811.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129 , 123137.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., H. L. Mitchell, G. Pellerin, M. Buehner, M. Charron, L. Spacek, and B. Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea. Rev., 133 , 604620.

    • Search Google Scholar
    • Export Citation
  • Janjić, Z. I., 2001: Nonsingular implementation of the Mellor–Yamada level 2.5 scheme in the NCEP meso model. National Centers for Environmental Prediction Office Note Tech. Rep. 437, 61 pp.

  • Keppenne, C. L., and M. M. Rienecker, 2002: Initial testing of a massively parallel ensemble Kalman filter with the Poseidon isopycnal ocean general circulation model. Mon. Wea. Rev., 130 , 29512965.

    • Search Google Scholar
    • Export Citation
  • Mellor, G. L., and T. Yamada, 1982: Development of turbulence closure model for geophysical fluid problems. Rev. Geophys. Space Phys., 20 , 851875.

    • Search Google Scholar
    • Export Citation
  • Mitchell, H. L., P. L. Houtekamer, and G. Pellerin, 2002: Ensemble size, balance, and model-error representation in an ensemble Kalman filter. Mon. Wea. Rev., 130 , 27912808.

    • Search Google Scholar
    • Export Citation
  • Mitchell, K., and Coauthors, 2000: Recent GCIP-sponsored advancements in coupled land surface modeling and data assimilation in the NCEP Eta Mesoscale Model. Preprints, 15th Conf. on Hydrology, Long Beach, CA, Amer. Meteor. Soc., 180–183.

  • Pagowski, M., 2004: Some comments on PBL parameterizations in WRF. Preprints, Joint 5th WRF/14th MM5 Users’ Workshop, Boulder, CO, NCAR, 43–46. [Available online at http://www.mmm.ucar.edu/mm5/workshop/workshop-papers_ws04.html.].

  • Pagowski, M., J. Hacker, and J-W. Bao, 2005: Behavior of WRF PBL schemes and land surface models in 1D simulations during BAMEX. Preprints, Joint 6th WRF/15th MM5 Users’ Workshop, Boulder, CO, NCAR, 4.6. [Available online at http://www.mmm.ucar.edu/wrf/users/workshops/WS2005/WorkshopPapers.html.].

  • Skamarock, W. C., J. B. Klemp, J. Dudhia, D. O. Gill, D. M. Barker, W. Wang, and J. G. Powers, 2005: A description of the advanced research WRF version 2. National Center for Atmospheric Research Tech. Rep. TN-468, 100 pp.

  • van Leeuwen, P. J., 1999: Comment on “Data assimilation using an ensemble Kalman filter technique.”. Mon. Wea. Rev., 127 , 13741377.

    • Search Google Scholar
    • Export Citation
  • Weber, R. O., and P. Talkner, 1993: Some remarks on spatial correlation function models. Mon. Wea. Rev., 121 , 26112617.

  • Whitaker, J. S., and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130 , 19131924.

  • Whitaker, J. S., G. P. Compo, X. Wei, and T. M. Hamill, 2004: Reanalysis without radiosondes using ensemble data assimilation. Mon. Wea. Rev., 132 , 11901200.

    • Search Google Scholar
    • Export Citation
  • Zilitinkevich, S. S., 1995: Non-local turbulent transport: Pollution dispersion aspects of coherent structure of convective flows. Air Pollution Theory and Simulation, H. Power, N. Moussiopoulos, and C. A. Brebbia, Eds., Vol. I, Air Pollution III, Computational Mechanics Publications, 53–60.

    • Search Google Scholar
    • Export Citation
  • View in gallery

    Group-mean regression coefficients (dashed), regression confidence factors (dotted), and their product (solid) for the univariate case: (a) T2 observations and T profiles, (b) U10 observations and U profiles, and (c) Q2 observations and Q profiles. Results are averaged over all 100 experiments valid at 0000 UTC.

  • View in gallery

    Same as in Fig. 1 but valid at 1200 UTC.

  • View in gallery

    Ensemble-mean RMSE over the profiles for (a) T, (b) U, and (c) Q for large ensembles. Solid curves show the no-localization case (NL100), dashed curves show errors with dynamically estimated localization (HF100), and dotted curves show errors with no assimilation (NO100).

  • View in gallery

    Ensemble spread–error ratio averaged over the profiles for (a) T, (b) U, and (c) Q for large ensembles. Solid curves show the no-localization results (NL100), dashed curves show ratios with dynamically estimated localization (HF100), and dotted curves show ratios with no assimilation (NO100).

  • View in gallery

    Time-averaged leading eigenvalues of the prior T localized covariance matrices for large ensembles. The solid curve shows the no-localization case (NL100), the dashed curve shows eigenvalues with dynamically estimated localization (HF100), and the dotted curve shows eigenvalues with no assimilation (NO100).

  • View in gallery

    Ensemble-mean RMSE over the profiles for (a) T, (b) U, and (c) Q for comparing large and small ensembles. Solid curves show the large-ensemble no-localization case (NL100), dash–dotted curves show errors for the small ensemble and no localization (NL10), and dotted curves show errors with no assimilation (NO100).

  • View in gallery

    Ensemble spread–error ratio averaged over the profiles for (a) T, (b) U, and (c) Q for comparing large and small ensembles. Solid curves show the large-ensemble no-localization case (NL100), dash–dotted curves show ratios for the small ensemble and no localization (NL10), and dotted curves show ratios with no assimilation (NO100).

  • View in gallery

    MSE of the localized small-ensemble covariance estimate for T2 and T profiles. It is presented as a function of the cutoff grid point for the localization function C(zc). The vertical line denotes the size of the model in grid points (33).

  • View in gallery

    Optimal zc for the GC localization function in temperature, as a function of analysis time.

  • View in gallery

    Ensemble-mean RMSE over the profiles for (a) T, (b) U, and (c) Q for evaluating the effect of time-dependent localization on small ensembles. Dash–dot curves show the no-localization case (NL10), dashed curves show errors with localization by archived HF runs (HF10), dotted curves show errors with optimized GC localization (GC10), and widely spaced dashed curves show errors when using the GC functions of only distance (GCD10).

  • View in gallery

    Ensemble spread–error ratio averaged over the profiles for (a) T, (b) U, and (c) Q for evaluating the effect of time-dependent localization on small ensembles. Dash–dotted curves show the no-localization case (NL10), dashed curves show ratios with localization by archived HF runs (HF10), dotted curves show ratios with localization by optimized GC functions (GC10), and widely spaced dashed curves show ratios when using the GC functions of only distance (GCD10).

  • View in gallery

    Time-mean regression localization functions for observations of T2 regressing onto the moisture profiles.

  • View in gallery

    Ensemble-mean RMSE over the profiles for (a) T, (b) U, and (c) Q for comparing time-dependent and stationary localization. Dashed curves show the case with time-dependent localization by archived HF runs (HF10), double-dashed curves show errors with a time-averaged HF localization (HFS10), dotted curves show errors with time-dependent GC localization (GC10), and triple dot–dashed curves show errors with a time-averaged GC localization (GCS10).

  • View in gallery

    Ensemble spread–error ratio averaged over the profiles for (a) T, (b) U, and (c) Q for evaluating the effect of time-dependent localization on small ensembles. Dashed curves show the case with time-dependent localization by archived HF runs (HF10), double-dashed curves show ratios with a time-averaged HF localization (HFS10), dotted curves show ratios with time-dependent GC localization (GC10), and triple dot–dashed curves show ratios with a time-averaged GC localization (HFS10).

  • View in gallery

    Time-averaged leading eigenvalues of the prior (T2, Q) joint-state localized covariance matrices for large ensembles.

  • View in gallery

    Time-averaged leading eigenvalues of the prior (T2, T) joint-state localized covariance matrices for small ensembles.

  • View in gallery

    Time-averaged leading eigenvalues of the prior (T2, Q) joint-state localized covariance matrices for small ensembles.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 48 18 1
PDF Downloads 43 18 1

Improved Vertical Covariance Estimates for Ensemble-Filter Assimilation of Near-Surface Observations

View More View Less
  • 1 National Center for Atmospheric Research, * Boulder, Colorado
  • | 2 NOAA/Forecast Systems Laboratory, and Colorado State University/Cooperative Institute for Research in the Atmosphere, Boulder, Colorado
Full access

Abstract

Strategies to improve covariance estimates for ensemble-based assimilation of near-surface observations in atmospheric models are explored. It is known that localization of covariance estimates can improve conditioning of covariance matrices in the assimilation process by removing spurious elements and increasing the rank of the matrix. Vertical covariance localization is the focus of this work, and two basic approaches are compared: 1) a recently proposed hierarchical filter approach based on sampling theory and 2) a more commonly used fifth-order piecewise rational function. The hierarchical filter allows for dynamic estimates of localization functions and does not place any restrictions on their form. The rational function is optimized for every analysis time of day and for every possible observation and state variable combination. The methods are tested with a column model containing PBL and land surface parameterization schemes that are available in current mesoscale modeling systems. The results are expected to provide context for assimilation of near-surface observations in mesoscale models, which will benefit short-range mesoscale NWP applications. Results show that both the hierarchical and rational function approaches effectively improve covariance estimates from small ensembles. The hierarchical approach provides localization functions that are irregular and more closely related to PBL structure. Analysis of eigenvalue spectra show that both approaches improve the rank of the covariance matrices, but the amount of improvement is not always directly related to the assimilation performance. Results also show that specifying different localization functions for different observation and state variable combinations is more important than including time dependence.

Corresponding author address: Joshua Hacker, National Center for Atmospheric Research, P.O. Box 3000, Boulder, CO 80307. Email: hacker@ucar.edu

Abstract

Strategies to improve covariance estimates for ensemble-based assimilation of near-surface observations in atmospheric models are explored. It is known that localization of covariance estimates can improve conditioning of covariance matrices in the assimilation process by removing spurious elements and increasing the rank of the matrix. Vertical covariance localization is the focus of this work, and two basic approaches are compared: 1) a recently proposed hierarchical filter approach based on sampling theory and 2) a more commonly used fifth-order piecewise rational function. The hierarchical filter allows for dynamic estimates of localization functions and does not place any restrictions on their form. The rational function is optimized for every analysis time of day and for every possible observation and state variable combination. The methods are tested with a column model containing PBL and land surface parameterization schemes that are available in current mesoscale modeling systems. The results are expected to provide context for assimilation of near-surface observations in mesoscale models, which will benefit short-range mesoscale NWP applications. Results show that both the hierarchical and rational function approaches effectively improve covariance estimates from small ensembles. The hierarchical approach provides localization functions that are irregular and more closely related to PBL structure. Analysis of eigenvalue spectra show that both approaches improve the rank of the covariance matrices, but the amount of improvement is not always directly related to the assimilation performance. Results also show that specifying different localization functions for different observation and state variable combinations is more important than including time dependence.

Corresponding author address: Joshua Hacker, National Center for Atmospheric Research, P.O. Box 3000, Boulder, CO 80307. Email: hacker@ucar.edu

1. Introduction

Ensemble-based filters for data assimilation continue to gain popularity because of their effectiveness, ability to estimate probability density functions (pdfs), and ease of implementation. Several methods for solving the analysis equation exist, but all rely on Monte Carlo approximations to prior (background) pdfs gleaned from an ensemble of model runs (e.g., Evensen 1994). Because the ensemble is typically small compared to the number of degrees of freedom in the physical system (here a model of atmospheric evolution but more generally the atmosphere) under consideration, Monte Carlo methods usually suffer from rank-deficient background error covariance matrices. Houtekamer and Mitchell (1998) showed that applying an additional, horizontal-distance-dependent restriction (localization) on the estimated covariances reduces the negative effects of poorly estimated covariance values that are far away from an observation. Subsequent work (Houtekamer and Mitchell 2001; Hamill et al. 2001; Mitchell et al. 2002) has further demonstrated and clarified the benefits of horizontal localization on assimilation performance. The question of vertical localization has been addressed in a few specific instances (Keppenne and Rienecker 2002; Whitaker et al. 2004; Houtekamer et al. 2005) but has received much less attention in general. Here we examine the special case of vertical localization for near-surface observations of temperature, winds, and humidity.

Hacker and Snyder (2005) showed the potential for near-surface observations to accurately determine a profile in an observation system simulation experiment (OSSE) framework with an ensemble filter and a column model. Those experiments used large ensembles of 100 members to avoid any need for localization. Because it appears that large ensembles for mesoscale applications will not be feasible for some time, localization may be a necessity in practice. Here we study the effects of small ensembles on the column, and approaches to addressing them, within an OSSE framework similar to that presented in Hacker and Snyder (2005).

Small ensembles suffer from sampling error that contributes to the need for localization. Spatial localization has been shown to mitigate some of the effects of small and spurious correlations (e.g., Houtekamer and Mitchell 1998, 2001; Hamill et al. 2001; Whitaker and Hamill 2002) by eliminating them far from an observation.

Joint observation-state error covariances in the PBL are expected to be transient, anisotropic, and inhomogeneous. Unlike an integrated quantity such as pressure, other near-surface parameters have a more tenuous connection to components of the flow aloft. This may limit the effectiveness of isotropic, stationary covariance models typically used for localization. To achieve maximum benefit from surface observations in an ensemble-filter data assimilation system, the proper vertical localization may be crucial.

Additionally, localization in multivariate systems (i.e., those with different types of observation and state variables) with functions of distance alone is a questionable approach. Nonlinear relationships between variables introduce imbalances under any distance-dependent localization, and the problem is worse when the length scale is short (Mitchell et al. 2002) as it is expected to be in the PBL. The appropriate vertical localization could be O(10 m) when the surface decouples from the atmosphere in the nocturnal surface layer.

Smooth functions typically used for localization, such as the fifth-order piecewise rational function of Gaspari and Cohn [1999, their Eq. (4.10)], may also be inappropriate under a sharp transition from a well-mixed PBL to a stable layer aloft. This fact introduces an additional difficulty when considering that the localization function should be mathematically acceptable, implying that it is positive definite and the underlying correlation function is continuous (e.g., Weber and Talkner 1993; Gaspari and Cohn 1999; Gneiting 1999). This issue is also addressed in Furrer and Bengtsson (2007) but is often ignored in practice because the addition of observational uncertainty often ensures that the denominator in the gain matrix is positive definite.

Anderson (2006) recently proposed a method to estimate the appropriate localization based on sampling theory. That approach provides a way to use state-dependent localization dynamically during an assimilation cycle. Dynamic localization requires a large ensemble that avoids major rank deficiency while providing good statistics to estimate significant covariances, but it may be impractical to run in an operational setting. Alternately, localization functions can be derived from an experiment with a large ensemble and later applied to a smaller ensemble with the same model. The small ensemble will produce rank-deficient covariances that can be localized with some kind of average functions accumulated from the large experiment. A user must make decisions about whether to average in time or space, or both.

Here we use a column model containing a PBL parameterization to run OSSEs designed to test different vertical localization approaches. A large experiment with the hierarchical filter (HF; Anderson 2006) provides both average localization functions and information for tuning the Gaspari–Cohn (GC) function. We present results in terms of assimilation performance and improvement of covariance rank.

Results from this study can be synthesized to provide recommendations on vertical localization, including whether legitimate covariance functions must be used and whether multivariate localization that depends on distance alone is sufficient. The next section describes the model and experiments. Section 3 characterizes the empirically estimated localization functions. Section 4 compares the performance of the assimilation with different localization techniques. Section 5 investigates covariance rank, and section 6 summarizes the results and provides additional comments.

2. Model and experiments

a. Column model

A column model containing a suite of physical parameterization schemes is suitable for the experiments described here. We are interested in only vertical structures and relationships within and near the PBL, and a column model allows experimentation at a fraction of the cost associated with a 3D mesoscale model. Large ensembles are feasible, enabling convergence of results and experimentation with sensitivity to ensemble size.

Our model can be thought of as a simpler cousin to the Weather Research and Forecast (WRF) mesoscale model (Skamarock et al. 2005). It contains the same suite of physical parameterizations for subgrid processes associated with the soil, surface layer, and PBL. For these experiments, we chose the Mellor–Yamada–Janjić (MYJ) PBL scheme (Janjić 2001) and the Noah land surface model (LSM) for the soil (Ek et al. 2003). The vertical grid is defined as 33 vertically stretched atmospheric levels, with the first layer extending to approximately 40 m above the surface and a top at approximately 4800 m. Further details of the column model are given in Pagowski (2004) and Pagowski et al. (2005). Further details can be found in appendix A.

Initial conditions, large-scale forcing, and surface radiation are imposed by randomly sampling two forecasts from a (warm) season of WRF real-time forecasts at a column located over Oklahoma, then combining them with a uniform random coefficient between zero and one (U[0, 1]). WRF 36-h forecasts from the Bow Echo and Mesoscale Convective Vortex Experiment (BAMEX) observation period spanning 3 May through 14 July 2003 constitute the sample. Forecasts were launched at 0000 UTC every day, on a Δx = 4 km grid. More details on the sampling approach are available in Hacker and Snyder (2005). This approach permits construction of a large ensemble, containing slow time scales, with forecast error that is saturated with respect to a conditional climatology. Although the distribution of large-scale forcing is narrowed with this approach, the small-scale effects on the column are isolated and ensembles larger than the WRF sample are available.

b. Hierarchical filter

The hierarchical filter proposed by Anderson (2006) is used to estimate the appropriate localization functions. In essence, it is a method to estimate the robustness of covariance estimates from the ensemble. Our experiments use the ensemble adjustment Kalman filter (EAKF), documented in Anderson (2001). Because it is computationally inefficient to compute the background error covariance and gain matrices directly, several alternatives to computing the analysis increments have emerged. Anderson (2003) showed that analysis increments in the statistical analysis equation can be computed by first linearly regressing the ensemble of observation increments (innovations) onto the prior (background) ensemble, computing a posterior distribution of innovations, and then applying the regression coefficients to update the ensemble. When the ensemble is broken into groups, the regression can be performed for each group separately. The spread of regression coefficients among the groups is a measure of the noise in the estimates, and the magnitude of the regression coefficients themselves is a measure of the signal. Just as traditional localization diminishes the importance of far-removed covariance estimates that typically have poor signal-to-noise ratios, these estimates can be used to determine in which covariances (here codified in regression coefficients) we have less confidence. Thus, the signal-to-noise estimates provide a regression confidence factor (RCF). Further details describing how the RCF is computed can be found in appendix B.

The hierarchical filter can be used directly during assimilation with a relatively large ensemble. Alternately, an inexpensive assimilation system can be constructed by collecting RCFs from large-ensemble experiments and imposing RCF values on smaller ensembles within the straightforward EAKF implementation. The large ensemble, of size M × N, is broken into M groups of N members each, and the hierarchical filter provides estimates of robust covariances for a smaller ensemble of size N. One key question we address is whether this option is viable for PBL analysis and forecasting, where the significant covariances are expected to be highly variable in time and space.

c. Experiment setup

Perfect-model OSSEs are designed to produce robust RCFs that can be used to deduce appropriate vertical localization for assimilating near-surface observations. In all experiments, temperature and water vapor mixing ratio at z = 2 m (T2, Q2) and wind components at z = 10 m (U10, V10) are assimilated hourly. To accumulate robust statistics the experiments are repeated 100 times, each with a different evolution of the true state. The simulated observations are contaminated with unbiased, uncorrelated noise drawn from a Gaussian distribution with error variances 1.0 K2, 1.0 × 10−6 kg2 kg−2, and 2.0 m2 s−2, respectively, for T2, Q2, and (U10, V10), before assimilation. These agree with the values used in Hacker and Snyder (2005). Initial conditions and forcing for truth and the ensemble are drawn randomly from the WRF climatology, where each forecast begins at 0000 UTC. All column model simulations are 24 h, beginning at 1200 UTC. Beginning 12 h later than the WRF forecasts avoids any moisture spinup in the WRF forecasts and also any adjustment from the initial conditions to its own internal climatology (cf. Hacker and Snyder 2005). Note that time series will be presented from 12 to 36 h, which is the forecast hour relative to the forcing from the WRF, and is 0–24 h for the column model.

The first set of three experiments provides a basis for quantifying the effects of rank-deficient covariances in later experiments and for exploring the structure of the RCFs. It includes free-running ensembles (no assimilation), assimilation with no localization, and assimilation with dynamically estimated localization. For each of these experiments, a large ensemble having 100 members is used. In the third experiment, localization functions are estimated using groups of size M = 4, each with N = 25 members. For reference, Table 1 summarizes the experiments, which are further described in later sections.

3. Regression confidence factors

Here we describe the RCFs estimated from the large ensembles (M × N = 100) with M = 4 and N = 25. First, examples are shown to get an idea of the spatial and temporal variability in the estimates. Next, we evaluate some basic performance statistics with and without assimilation, and with dynamic localization provided by the hierarchical approach. The metrics for comparison are the ensemble-mean error and the spread–error ratio. Spread is defined as the standard deviation about the ensemble mean for each state variable, averaged over the profile, and error is the RMSE of the ensemble-mean profile. All results are then averaged over the 100 runs. The spread–error ratio would be 1.0 for an infinite ensemble with a perfect model, a perfect ensemble filter, and Gaussian error statistics, but error in sampling the complete error growth distribution and deviations from Gaussian statistics act to reduce it.

a. Time and space dependence

Examples prove useful for demonstrating the spatial and temporal variability of the estimated localization functions. Here we show the group-mean (among M = 4 groups) regression coefficients, RCFs, and the product of the two, averaged over all 100 experiments. The product of the regression coefficient and the RCF is the factor that quantifies the effect of the innovation on the state in computing the analysis increment. Alternatively, it is the relevant part of the localized gain matrix. Results valid at 0000 UTC (late afternoon local time) are shown in Fig. 1 for the univariate case where the observation is the same physical quantity as the profile against which the increments are regressed.

The vertical variability is evident in Fig. 1. None of the RCF profiles (dotted) are reminiscent of a typical localization function, which may resemble a step function, the GC function, a compact cosine, or a compact exponential. With the exception of the wind (which is nearly monotonic), any monotonically decreasing function of distance would erroneously reduce the impact of an observation on layers aloft, according to the sampling theory behind the hierarchical filter.

The effect of the well-mixed afternoon PBL is evident in each profile. The RCFs on the T profiles are rather large throughout the domain, indicating high confidence in coefficients well above the PBL depth (zi). The minimum at approximately z = 1600 m corresponds to variability in zi. Similarly, the larger coefficients above z = 3000 m, combined with a large RCF, suggest that information from the observations of T2 may be important aloft. The regression coefficients for U show the largest values above zi, counterintuitively suggesting that the diagnostic 10-m wind is more closely tied to winds in the free atmosphere aloft than winds in the PBL in an average sense. Corresponding RCFs suggest this relationship is robust. The Q coefficient profiles suggest that moisture is well mixed in the PBL, as expected, but uncorrelated where z > zi. The variability in the PBL depth shows up as a local minimum near z = 1700 m. Although the minima are all related to zi, their differences suggest it would be difficult to correctly localize all three state components with a single function of distance alone.

The temporal variability is evident when comparing Fig. 1 to Fig. 2, which is valid at 1200 UTC (presunrise local time). The profiles suggest an interface similar to the top of a well-mixed PBL, but this is actually the top of the residual layer from the previous day, which is still well mixed but somewhat decoupled from the surface. These suggest that memory in the system is important to covariance structures and indicate one advantage of recursive filters (filters with output dependent upon all previous input) when assimilating surface observations. The temperature near the surface is more closely correlated to the temperature aloft than it is to the temperature in the PBL, and the RCF indicates high confidence in that connection. This could be caused by horizontal advection in the inhomogeneous residual layer. The free atmosphere is more horizontally homogeneous at scales commensurate with advection during quiescent summer periods. The coefficients for U are larger than at 0000 UTC, but the RCF is diminished aloft, showing the effect of the surface decoupling from the free atmosphere. The large coefficients and RCFs in the PBL for the Q profiles suggest an extra long memory in the connection between Q2 and Q in the residual layer. The primary effect of dynamically estimating RCFs and using them to localize is examined next.

b. Effects of localization on a large ensemble

The effect of both observations and localization within the 100-member ensemble can be summarized by quantifying profile-averaged spread and skill, and by an estimate of the effect on the rank of the ensemble covariance. All results are also averaged over the 100 experiments. The experiments are denoted as follows: without any data assimilation (NO100), assimilating observations but without any localization (NL100), and using the HF to dynamically localize (HF100).

Figure 3 shows that the data assimilation effectively reduces the ensemble-mean RMSE, and that the localization has little effect with a 100-member ensemble. Without data assimilation (dotted curve), the ensemble maintains the initially saturated error levels in both T and Q. The U-wind error continues to grow through the analysis period, suggesting that the column model may have sensitivities that are not present in the 3D WRF model. The substantial error reduction occurring when near-surface observations are assimilated at hour 13 (solid curve) demonstrates the effectiveness of the assimilation.

The fact that the dynamic localization does not have much effect can suggest either one of two possibilities: that the ensemble is large enough to produce well-sampled covariances or that the localization is weak. Qualitatively, the localization appears somewhat weak (Figs. 1 and 2), but the evidence presented below suggests that an ensemble of 25 members (the size of each group) is nearly sufficient to capture the variability of the column system. The error in Q analyses is slightly higher when localization is imposed, suggesting that the localization is withholding useful information from the assimilation process. The need for localization should thus be carefully evaluated before it is applied.

Spread–skill ratios (Fig. 4) support the notion that the localization is unnecessary for a 100-member ensemble. The free-running forecast (NO100) maintains a constant averaged spread–error ratio near 1.0 for all three quantities. The ratios become noisier, and the mean values drop slightly, when observations are assimilated. Whether or not localization is imposed has little bearing on the ensemble behavior described by this metric.

The effects of localization can also be summarized by examining the variability in the prior error-covariance matrices (e.g., Hamill et al. 2001). Figure 5 shows the time- and experiment-averaged eigenvalue spectra for the 100-member ensemble-estimated temperature-error-covariance matrices. Introduction of assimilated observations has the effect of slightly reducing the variability in all modes with positive eigenvalues, corresponding to a reduction in the ensemble spread. For both experiments NO100 and NL100 the matrices are full rank, and the number of positive eigenvalues is equal to the dimension of the temperature state space (33). Localization in the full ensemble has the effect of flattening the spectrum while eliminating some trailing modes of variability. The potentially detrimental effects of the truncation cannot be seen in the assimilation performance within the relatively short cycling period of 36 h, suggesting these modes are not particularly important in this implementation.

Localization results in a flatter spectrum, similar to the behavior found in Hamill et al. (2001) and Furrer and Bengtsson (2007). This would appear to be a generally positive effect, but in this case it corresponds to a reduction in the rank of the prior covariance matrices. Variance associated with the trailing modes is removed and injected closer to the leading modes. Here we simply conclude that the flatness of the eigenvalue spectrum of error covariance may be a useful diagnostic of the overall effect of localization, but it cannot determine whether the effect on assimilation will be positive or negative. This effect is further explored in section 5.

4. Localization for degenerate ensembles

In this section, the quality of the assimilation is examined for smaller ensembles (N = 10) using different localization methods. The small size of the ensemble introduces noise to the background error covariance and results in sampling error, which is partially addressed by localization (Houtekamer and Mitchell 1998, 2001; Hamill et al. 2001; Mitchell et al. 2002). We first quantify the effects on spread and skill, and then examine the performance enhancements when localization functions are used. The HF approach is compared to the more common GC function. In section 5, the rank is estimated, and we try to reconcile the performance.

a. Small ensembles and rank deficiency

Reducing the size of the ensemble by one order of magnitude results in poor skill and inconsistency between the spread and the skill. Although this is a well-known effect of small ensembles (e.g., Houtekamer and Mitchell 1998; van Leeuwen 1999), we quantify the effect here to provide context for the experimental results.

The effect on skill for this system is shown in Fig. 6. The asymptotic temperature error increases from near 0.9 to 1.3 K when the number of ensemble members is reduced from 100 to 10 and no localization is imposed. Wind errors at the final analysis time are greater by 1.5 m s−1. The effect on the moisture profile is more severe, and the error at 36 h actually surpasses the ensemble-mean error in the free-running 100-member forecast (cf. Fig. 3c). This may be the result of spurious covariance estimates arising near the interface between the PBL and the free atmosphere aloft. The error-covariance structure in the large ensemble is not smooth (Figs. 1c and 2c), and the small ensembles may not adequately sample that structure.

The ratio between spread and skill also suffers (Fig. 7). Temperature and moisture ratios appear to asymptote to about 0.4 by 36 h. Although the magnitude of the increase in temperature error is not large, it is relatively large (≈45%), and the spread of the small ensemble cannot increase accordingly. The same is true for the moisture profiles. The effect on winds is not as great, fluctuating about 0.6 through most of the analysis period. The wind-error increase is also relatively large (≈35%), but the small ensemble appears more capable of a corresponding increase in spread in the wind. This is consistent with the sensitivity shown in the NO100 curve (Fig. 6b).

b. Comparison of localization methods

The HF provides an estimate of the sampling error associated with a particular ensemble size, which is 10 for the small ensembles here. Thus, it is inappropriate to use the 100-member (4 × 25) ensembles to find localization functions for 10-member ensembles. Instead, a separate set of 100 ensembles, with M = 4 groups of N = 10 members, is used to produce RCF “lookup tables” for experiments with the small ensembles. The RCFs are functions of the observed variable and state variable types, the time of day, the particular ensemble members in each group, and the flow. To produce one repository of localization functions, we average over the 100 experiments. This results in independent localization functions for each (observation, state variable) combination and time of day. Experiments using these functions are denoted HF10. Further averaging over time of day, to produce another repository, results in single (stationary) functions for each observed variable type. Experiments using these functions are denoted HFS10.

Additionally, GC localization functions can be tuned directly with the 100-member ensemble because we consider the covariances estimated from this ensemble to be relatively free of noise. Gaspari–Cohn functions are typically functions of distance alone, C(z), with radius determined by a distance zc, where C(zc) = 0 and shape determined by an inflection point typically placed at zc/2. Equation (4.10), which defines the GC function in Gaspari and Cohn (1999), is reproduced in appendix C for reference. Similar to Furrer and Bengtsson (2007), we choose an “optimal localization function” of the GC class by minimizing the mean squared error (MSE) of a localized estimate of the covariance matrix and a “true” covariance matrix. Using the forecast xf and linear forward operator 𝗛, define the true covariance matrix 𝗣f with the joint (𝗛xf, xf) covariances computed from the 100-member ensemble with assimilation (NL100), and an estimate of the covariances from the rank-deficient ensemble (NL10) as . The localized estimate of the covariance is then the element-wise multiplication 𝗖○, where the elements of 𝗖 are C(|zobszj|) for observation location zobs and gridpoint location zj. The MSE as a function of time and distance for all (expected observation, state variable) combinations in (𝗛xf, xf) is
i1520-0493-135-3-1021-e1
where the mean is taken over the 100 individual experiments. The resulting E as a function of zc in grid points, for observations of T2 and profiles of T, is shown in Fig. 8.

The GC localization function for each (observation, state variable) combination and analysis time is defined by choosing zc at the minimum in the error curves such as those shown in Fig. 8. These curves show that the error in the localized covariances is insensitive to zc for zc greater than some threshold. This result holds for all times and variable combinations. Despite the flat curves, minima are present at grid points 114 and 73 for 0000 and 1200 UTC, respectively, and were found for nearly all other variable combinations as well (localization was not performed when a global minimum was absent). The minimum in each curve determines zc for that time of day and (observation, state variable) combination. Experiments localizing with the resulting GC functions are denoted GC10. Many of the best-fit functions result in very weak localization, since these functions are greater than 0.5 through the entire domain when zc > 97 grid points. One of the glaring exceptions results from the covariance of temperature observations with moisture profiles, with consequences discussed below.

The diurnal nature of the optimal zc for temperature is shown in Fig. 9. This curve reflects the intuition that during the day the well-mixed PBL produces deeper covariance structures. At the peak during the middle of the day (hour 25), the localization at the top of the domain is approximately 0.68. During the night (hours 30–36) the values at the top of the domain are approximately 0.3. For each (observation, state variable) combination, the parameter zc is averaged over time of day to give stationary GC localization functions for the small-ensemble experiments. Experiments localizing with these are denoted GCS10.

It is important to note that finding the optimal localization functions for all (observation, state variable) combinations in a large system would be computationally infeasible. To quantify the importance of the full multivariate optimization for this system, an additional experiment is introduced. The functions optimized for the (T2, T) joint space are used for all (observation, state variable) combinations, making the localization a function of distance only. This is denoted GCD10.

Comparisons between HF10 and GC10 show different effects for different variables. For temperature, these localization approaches improve neither the skill nor the spread–error ratio (Figs. 10a and 11a). Both approaches improve skill and the spread–skill ratio for the winds by nearly 1.0 m s−1 and 0.2, respectively (Figs. 10b and 11b), but choosing a clear winner is difficult. Skill also improves for the moisture (Figs. 10c), where both localization approaches reduce the error by approximately 15%. GC10 appears to have an advantage here by improving the final spread–error ratio to 0.85 (Fig. 11c).

One possible explanation for the advantage of GC10 over HF10 for moisture lies in the multivariate relationships. The univariate localization (observations and state of the same type) is weak for the optimized GC functions, and one expects the assimilation should behave more like the experiment that did not impose localization (NL10). The HF generally imposes tighter localization but still permits spatially distant variables to be updated in the assimilation. The localization imposed in the moisture profiles, when T2 is assimilated, is an exception where the GC localization is tighter than the HF localization. Figure 12 shows the time-mean localization functions, where larger values for the HF10 localization permit more impact from temperature observations. The localization for experiment GCD10 shows that localization based on multivariate optimization is much different from that based on distance alone or optimized for a different variable. Figures 10, 11 and 12 suggest that spurious covariances between T2 and moisture profiles, which degrade the impact of the observations, are retained for small ensembles with weak localization. These results are consistent with the analysis in Hacker and Snyder (2005), which shows weak vertical cross correlations between expected observations and state variables of different types in a column of the full WRF modeling system.

c. Time-dependent and stationary localization

The influence of the diurnal cycle on the vertical localization of near-surface observations is clear from Figs. 1, 2 and 9. Here we measure the importance of accounting for it. For a large NWP model, tuning and maintaining a different localization function for each assimilation time and observation may prove intractable. One way to reduce the burden is to find time-averaged localization functions. These could be either optimized GC functions (experiment GCS10) or be a time average of the hierarchical estimates (experiment HFS10).

The skill and spread–error ratios suggest that time variation is not particularly important for this problem. Figure 13 shows that, in general, the difference between classes of localization function is greater than the difference between time-varying and stationary functions. Frequently the differences are small but are most noticeable in the last few hours of the assimilation period, when the skill for GC10 (HF10) and GCS10 (HFS10) are nearly identical for temperature and winds. Although this characterization is not observed in the skill of the moisture profiles, it is obvious in the spread–skill ratios (Fig. 14), where the performance difference is greatest. From these results and the results of the last subsection, we deduce that the multivariate relationships are more important than time dependence for vertical localization of near-surface observations.

5. Effect on rank-deficient covariances

As mentioned earlier, previous studies have used eigenvalue spectra as evidence that localization is achieving a desirable effect (e.g., Hamill et al. 2001; Furrer and Bengtsson 2007), because it indicates that the covariance matrix estimated from a small ensemble improves in rank (becomes less rank deficient). The argument is based on the notion that a flatter spectrum indicates that the covariance matrix better samples all the modes of variability in the system. Here we examine the joint-state (observation, state variable) covariances for evidence of rank improvements.

Comparison of eigenvalue spectra for joint-state error covariances in the large ensembles shows that localization across different physical quantities can cause a loss of useful information in a well-sampled system. Results in section 3b showed that localization has a small, but negative, effect on the skill for Q when assimilating with large ensembles. Figure 5 demonstrates that the localization truncates the eigenvalue spectrum for the (T2, T) joint-state error covariance, while inflating the variance in the leading modes that are retained. The same behavior is observed for the (Q2, Q) joint state (not shown). A different result can be seen in Fig. 15, which shows the eigenvalue spectra for the (T2, Q) joint-state error covariance estimated from the same 100-member ensembles. The hierarchical filter again truncates the spectrum, leading to some rank deficiency, but here does not produce any flattening to compensate. The same behavior is observed for the (U10, Q) joint state (not shown). In both cases, localization causes covariances in the trailing modes to be ignored in the assimilation process. For (Q2, Q), increased variability in the leading modes may compensate, but for (T2, Q) no such compensation is evident. These spectra suggest that the truncation resulting from localization in the multivariate joint states of observations with profiles of Q is throwing out useful information leading to the loss of skill shown in Fig. 3c.

Eigenvalue spectra for covariances estimated from the small ensembles, which we know are degenerate, show the expected rank improvement resulting from localization. Figure 16 shows results for temperature alone, demonstrating both a flattening of the spectrum and improved rank for both the GC and HF localization. But the metrics in section 4 (Figs. 10a and 11a) do not show any real benefit to localization in the temperature state, indicating that improved rank does not guarantee better assimilation performance.

The eigenvalue spectra of error covariances estimated for the (T2, Q) joint state are more consistent with the assimilation results. Figure 17 shows that the GC localization greatly improves the rank, but the HF localization slightly reduces the rank with no commensurate flattening of the spectrum. Rather it reduces the variability in the modes that are already well sampled in the 10-member ensembles. This can be further understood by referring to Fig. 12, which shows strong localization with the optimized GC function, but no effective tapering of the covariances with the HF. The HF localization reduces the impact of T2 on the moisture state, both near the surface and aloft, with a somewhat uniform magnitude. The GC function eliminates covariances between T2 and the moisture profile aloft, while the HF function permits them. The small ensemble apparently produces spurious covariance estimates at those distances, and a rank improvement coincides with improved assimilation performance.

Contrary to the effect on large ensembles, where localization eliminates useful information in the multivariate joint-state error covariances, the weaker HF localization retains too much information from the small ensembles. The rank of the joint-state covariance is not improved, and the reduction in covariance throughout the domain actually reduces the signal-to-noise ratio in them. Thus, it is the localization of (Q2, Q) covariances in the HF that produces the improvement over NL10, but it is not enough to result in better performance than GC10. The corresponding (Q2, Q) eigenvalue spectra (not shown) support this.

The results of this section further demonstrate that the eigenvalue spectra alone are not enough to understand the effects of localization, and that other metrics must be included in any useful analysis. Additionally, we again emphasize that the GC functions used in this study are optimized for every (observation, state variable) combination. This exercise would be impossible to complete with an NWP model and many observations. The hierarchical filter is able to more efficiently produce estimates of appropriate localization functions that result in assimilation performance only slightly below that of the optimized GC functions.

6. Summary

Within a simplified dynamic column-model system, including PBL, surface layer, and land surface parameterizations, and an ensemble-filter data assimilation system we have explored vertical localization for near-surface observations. The primary point of comparison is between localization functions gleaned from the hierarchical filter approach of Anderson (2006) and the fifth-order piecewise rational function of Gaspari and Cohn (1999), which has been used extensively for ensemble-based assimilation. The hierarchical filter allows for arbitrary localization functions for any (observation, state variable) combination, or more generally any (observation, parameter) combination when model parameters are included in the state vector. The localization functions, called regression confidence factors (RCFs), can be derived from a limited number of assimilation experiments. Examples show that the RCFs are temporally variable and are not necessarily monotonic (Figs. 1 and 2). The GC localization is a monotonically decreasing function of distance in the model coordinate system. Here, those are tuned for every type of observation and state variable combination separately, and for each analysis time of day, and can be called optimal in some sense. Such an exercise would be computationally infeasible with an advanced mesoscale modeling and data assimilation system.

Large ensembles (100 members) provide baseline performance statistics for the free-running ensembles, while assimilation experiments show the maximum positive impact of synthetic 2-m temperature and mixing ratio (T2, Q2) and 10-m winds (U10, V10). The results show that the observations provide information about the state of the column and the implied uncertainty, and also that localization does not add any value when the ensembles are sufficiently large (Figs. 3 and 4). The eigenvalue spectra of covariance matrices estimated from the ensembles show that they are full rank without any localization. The imposed localization actually truncates the spectrum, moving some of the variance in the trailing eigenvectors up the spectrum (Fig. 5), without harming assimilation performance.

Small ensembles (10 members) demonstrate the need for vertical localization and also the relative performance of the different approaches (Figs. 6, 7, 10 and 11). Both approaches effectively improve the assimilation performance for winds and moisture, though neither achieves the performance of the large ensembles. It is difficult to determine whether the RCFs or the GC functions are better, though the latter at times appears to marginally outperform the former. The slight advantage may be due to second-order sampling error that arises because of the small number of groups (4) used to estimate the RCFs (Anderson 2006). When a single GC function is used for all (observation, state variable) combinations, so that it is a function of distance alone, the performance dramatically deteriorates.

Despite the large variation in time (diurnal cycle), a stationary function appears to be sufficient for average performance metrics (Figs. 13 and 14). This somewhat surprising result, combined with the other results summarized above, suggests that localizing with different functions for different (observation, state variable) combinations is the most important factor.

The decision to use the hierarchical approach or the optimization approach depends on several considerations. In both cases, large ensembles are needed to obtain statistics. In the hierarchical case, the tuning ensemble needs to be M times the size of the ensemble that one could afford to run regularly, where M could be as small as four but may need to be larger. In the GC case, an ensemble large enough to give confidence that the covariances are sampled with little error is required. For both, enough cases or assimilation cycles would need to be run with the large ensembles to get statistics that converge. The size of the state and the number of observations are further considerations in the optimization procedure. The additional possibility of including model parameters in the localization procedure, which may prove a fruitful research topic, is an advantage inherent to the HF that may not be available when distance-dependent localization functions are specified.

The localization problem will get worse with more observations and more model columns, as exist in a typical mesoscale NWP implementation, because of the likelihood of increased noise in the covariance estimates. Here we have compiled robust statistics with a simplified modeling system. Quantitative results are only valid for this model and configuration, but they should qualitatively extend to other similar systems. Undoubtedly the results will change when observations aloft are also assimilated, and full 3D dynamics are included. When observations aloft are assimilated, we can expect a decrease in the correlation of near-surface observations with their overlying profiles, enhancing the need for localization. The full 3D dynamics also add noise to covariance estimates, and the use of real observations will impose additional challenges. The results of this work can serve as a basis for comparison and will help guide experiment design for the more expensive experiments necessary to understand the complete effects of vertical localization of near-surface observations.

Acknowledgments

The authors are grateful to the other developers of the NCAR Data Assimilation Research Testbed (DART) for providing a useful experimental platform. J. Hacker was supported by the Army Test and Evaluation Command while performing this research. J. Anderson was supported by the NCAR Data Assimilation Initiative. Conversations with Tomoko Matsuo and Reinhard Furrer were valuable in understanding the effects of localization.

REFERENCES

  • Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129 , 28842903.

  • Anderson, J. L., 2003: A local least squares framework for ensemble filtering. Mon. Wea. Rev., 131 , 634642.

  • Anderson, J. L., 2006: Exploring the need for localization in ensemble data assimilation using an hierarchical ensemble filter. Physica D, in press.

    • Search Google Scholar
    • Export Citation
  • Beljaars, A. C. M., and A. A. M. Holtstag, 1991: Flux parameterization over land surfaces for atmospheric models. J. Appl. Meteor., 30 , 327341.

    • Search Google Scholar
    • Export Citation
  • Ek, M. B., K. E. Mitchell, Y. Lin, P. Grunmann, V. Koren, G. Gayno, and J. D. Tarpley, 2003: Implementation of Noah land surface model advances in the National Centers for Environmental Prediction operational mesoscale Eta model. J. Geophys. Res., 108 .8851, doi:10.1029/2002JD003296.

    • Search Google Scholar
    • Export Citation
  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99 , 1014310162.

    • Search Google Scholar
    • Export Citation
  • Furrer, R. H., and T. Bengtsson, 2007: Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants. J Multivar. Anal., 98 , 227255.

    • Search Google Scholar
    • Export Citation
  • Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125 , 723757.

    • Search Google Scholar
    • Export Citation
  • Gneiting, T., 1999: Correlation functions for atmospheric data analysis. Quart. J. Roy. Meteor. Soc., 125 , 24492464.

  • Hacker, J. P., and C. Snyder, 2005: Ensemble Kalman filter assimilation of fixed screen-height observations in a parameterized PBL. Mon. Wea. Rev., 133 , 32603275.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., J. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129 , 27762790.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126 , 796811.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129 , 123137.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., H. L. Mitchell, G. Pellerin, M. Buehner, M. Charron, L. Spacek, and B. Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea. Rev., 133 , 604620.

    • Search Google Scholar
    • Export Citation
  • Janjić, Z. I., 2001: Nonsingular implementation of the Mellor–Yamada level 2.5 scheme in the NCEP meso model. National Centers for Environmental Prediction Office Note Tech. Rep. 437, 61 pp.

  • Keppenne, C. L., and M. M. Rienecker, 2002: Initial testing of a massively parallel ensemble Kalman filter with the Poseidon isopycnal ocean general circulation model. Mon. Wea. Rev., 130 , 29512965.

    • Search Google Scholar
    • Export Citation
  • Mellor, G. L., and T. Yamada, 1982: Development of turbulence closure model for geophysical fluid problems. Rev. Geophys. Space Phys., 20 , 851875.

    • Search Google Scholar
    • Export Citation
  • Mitchell, H. L., P. L. Houtekamer, and G. Pellerin, 2002: Ensemble size, balance, and model-error representation in an ensemble Kalman filter. Mon. Wea. Rev., 130 , 27912808.

    • Search Google Scholar
    • Export Citation
  • Mitchell, K., and Coauthors, 2000: Recent GCIP-sponsored advancements in coupled land surface modeling and data assimilation in the NCEP Eta Mesoscale Model. Preprints, 15th Conf. on Hydrology, Long Beach, CA, Amer. Meteor. Soc., 180–183.

  • Pagowski, M., 2004: Some comments on PBL parameterizations in WRF. Preprints, Joint 5th WRF/14th MM5 Users’ Workshop, Boulder, CO, NCAR, 43–46. [Available online at http://www.mmm.ucar.edu/mm5/workshop/workshop-papers_ws04.html.].

  • Pagowski, M., J. Hacker, and J-W. Bao, 2005: Behavior of WRF PBL schemes and land surface models in 1D simulations during BAMEX. Preprints, Joint 6th WRF/15th MM5 Users’ Workshop, Boulder, CO, NCAR, 4.6. [Available online at http://www.mmm.ucar.edu/wrf/users/workshops/WS2005/WorkshopPapers.html.].

  • Skamarock, W. C., J. B. Klemp, J. Dudhia, D. O. Gill, D. M. Barker, W. Wang, and J. G. Powers, 2005: A description of the advanced research WRF version 2. National Center for Atmospheric Research Tech. Rep. TN-468, 100 pp.

  • van Leeuwen, P. J., 1999: Comment on “Data assimilation using an ensemble Kalman filter technique.”. Mon. Wea. Rev., 127 , 13741377.

    • Search Google Scholar
    • Export Citation
  • Weber, R. O., and P. Talkner, 1993: Some remarks on spatial correlation function models. Mon. Wea. Rev., 121 , 26112617.

  • Whitaker, J. S., and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130 , 19131924.

  • Whitaker, J. S., G. P. Compo, X. Wei, and T. M. Hamill, 2004: Reanalysis without radiosondes using ensemble data assimilation. Mon. Wea. Rev., 132 , 11901200.

    • Search Google Scholar
    • Export Citation
  • Zilitinkevich, S. S., 1995: Non-local turbulent transport: Pollution dispersion aspects of coherent structure of convective flows. Air Pollution Theory and Simulation, H. Power, N. Moussiopoulos, and C. A. Brebbia, Eds., Vol. I, Air Pollution III, Computational Mechanics Publications, 53–60.

    • Search Google Scholar
    • Export Citation

APPENDIX A

Model Details

A 1D model similar to the WRF model is developed to study boundary layer and surfaces processes using physical parameterizations of the original 3D model. The atmospheric model consists of momentum, thermodynamic, and moisture conservation equations written as
i1520-0493-135-3-1021-ea1
i1520-0493-135-3-1021-ea2
i1520-0493-135-3-1021-ea3
i1520-0493-135-3-1021-ea4

Geostrophic wind (Ug, Vg) is obtained from the hourly 3D WRF forecasts and interpolated linearly to the current model time. Divergences of fluxes ∂〈wφ′〉/∂z, where φ is any of u, υ, θ, or q, are provided by a boundary layer scheme. Equations are solved implicitly to prevent numerical instabilities in the diffusion equation and due to the Coriolis terms. The atmospheric model is coupled with the LSM, which also uses radiative fluxes from the 3D model in calculating the surface energy balance.

In the present application turbulence is parameterized with the Eta implementation of 1.5-order closure of Mellor and Yamada (1982), by Janjić (2001), coupled with the Noah LSM (Mitchell 2000). The turbulence parameterization employs the prognostic equation for the turbulent kinetic energy (TKE) with the assumption of downgradient diffusion and pressure covariance, and a diagnostic equation for potential temperature and moisture covariance. Janjić (2001) imposed additional restrictions on the value of TKE and master length scale, revised empirical constants, and devised a novel method to solve the TKE equation. The master length scale is computed diagnostically. Surface layer parameterization follows Monin–Obukhov similarity theory extended by Beljaars and Holtstag (1991) to free convection regime with roughness scales for momentum, temperature, and moisture calculated using Zilitinkevich (1995) formulas. The Noah LSM (Mitchell 2000) is composed of a four-layer soil temperature and moisture model plus canopy moisture and snow cover parameterizations. It provides sensible and latent heat fluxes to the atmospheric model taking into account atmospheric model output and land use characteristics, such as vegetation type and soil texture.

APPENDIX B

Localization Using the Hierarchical Filter

Assume that M groups of N-member ensembles (M × N total members) are available. When using linear regression to compute the increment in a state variable x, given increments for an observation variable y, M sample values of the regression coefficient, β, are available. The regression coefficient for each group is calculated as in a standard ensemble filter: βi = σx,y/σy,y, where the numerator is the prior sample covariance of the state variable x with the observed variable y, and the denominator is the prior variance of the observed variable. Both are computed using the N members of the ith group. Without defining the “correct” regression factor (more details can be found in Anderson 2006), a regression confidence (weighting) factor α is defined to minimize the expected RMS difference between the increment in a state variable and the increment that would be used if the correct regression factor were used:
i1520-0493-135-3-1021-eb1
An additional constraint, βi ≥ 0, is applied to ensure that αmin ≥ 0. The regression is completed for each ensemble using its sample regression coefficient multiplied by αmin, equivalent to a localization function that is not necessarily a function of distance from y.

An RCF is computed for every (observation, state variable) pair. For analysis it is convenient to plot segments of the state determined by the type of physical quantity (e.g., temperature). In the text the (observation, state variable type) is referred to as an (observation, state) pair, which can be thought of as a function in state space.

APPENDIX C

Localization Using a Piecewise Rational Function

The fifth-order piecewise rational function of Gaspari and Cohn [1999, their Eq. (4.10)] has been used as both a covariance function and a localization function in data assimilation. It is a homogeneous and isotropic correlation function on R3 and is similar in shape to a compact Gaussian function. Here we repeat it for reference.

Given a constant scaling distance zc and inflection point at c = zc/2, the remaining independent variable is distance z from the observation location:
i1520-0493-135-3-1021-ec1

Fig. 1.
Fig. 1.

Group-mean regression coefficients (dashed), regression confidence factors (dotted), and their product (solid) for the univariate case: (a) T2 observations and T profiles, (b) U10 observations and U profiles, and (c) Q2 observations and Q profiles. Results are averaged over all 100 experiments valid at 0000 UTC.

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 2.
Fig. 2.

Same as in Fig. 1 but valid at 1200 UTC.

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 3.
Fig. 3.

Ensemble-mean RMSE over the profiles for (a) T, (b) U, and (c) Q for large ensembles. Solid curves show the no-localization case (NL100), dashed curves show errors with dynamically estimated localization (HF100), and dotted curves show errors with no assimilation (NO100).

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 4.
Fig. 4.

Ensemble spread–error ratio averaged over the profiles for (a) T, (b) U, and (c) Q for large ensembles. Solid curves show the no-localization results (NL100), dashed curves show ratios with dynamically estimated localization (HF100), and dotted curves show ratios with no assimilation (NO100).

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 5.
Fig. 5.

Time-averaged leading eigenvalues of the prior T localized covariance matrices for large ensembles. The solid curve shows the no-localization case (NL100), the dashed curve shows eigenvalues with dynamically estimated localization (HF100), and the dotted curve shows eigenvalues with no assimilation (NO100).

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 6.
Fig. 6.

Ensemble-mean RMSE over the profiles for (a) T, (b) U, and (c) Q for comparing large and small ensembles. Solid curves show the large-ensemble no-localization case (NL100), dash–dotted curves show errors for the small ensemble and no localization (NL10), and dotted curves show errors with no assimilation (NO100).

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 7.
Fig. 7.

Ensemble spread–error ratio averaged over the profiles for (a) T, (b) U, and (c) Q for comparing large and small ensembles. Solid curves show the large-ensemble no-localization case (NL100), dash–dotted curves show ratios for the small ensemble and no localization (NL10), and dotted curves show ratios with no assimilation (NO100).

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 8.
Fig. 8.

MSE of the localized small-ensemble covariance estimate for T2 and T profiles. It is presented as a function of the cutoff grid point for the localization function C(zc). The vertical line denotes the size of the model in grid points (33).

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 9.
Fig. 9.

Optimal zc for the GC localization function in temperature, as a function of analysis time.

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 10.
Fig. 10.

Ensemble-mean RMSE over the profiles for (a) T, (b) U, and (c) Q for evaluating the effect of time-dependent localization on small ensembles. Dash–dot curves show the no-localization case (NL10), dashed curves show errors with localization by archived HF runs (HF10), dotted curves show errors with optimized GC localization (GC10), and widely spaced dashed curves show errors when using the GC functions of only distance (GCD10).

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 11.
Fig. 11.

Ensemble spread–error ratio averaged over the profiles for (a) T, (b) U, and (c) Q for evaluating the effect of time-dependent localization on small ensembles. Dash–dotted curves show the no-localization case (NL10), dashed curves show ratios with localization by archived HF runs (HF10), dotted curves show ratios with localization by optimized GC functions (GC10), and widely spaced dashed curves show ratios when using the GC functions of only distance (GCD10).

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 12.
Fig. 12.

Time-mean regression localization functions for observations of T2 regressing onto the moisture profiles.

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 13.
Fig. 13.

Ensemble-mean RMSE over the profiles for (a) T, (b) U, and (c) Q for comparing time-dependent and stationary localization. Dashed curves show the case with time-dependent localization by archived HF runs (HF10), double-dashed curves show errors with a time-averaged HF localization (HFS10), dotted curves show errors with time-dependent GC localization (GC10), and triple dot–dashed curves show errors with a time-averaged GC localization (GCS10).

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 14.
Fig. 14.

Ensemble spread–error ratio averaged over the profiles for (a) T, (b) U, and (c) Q for evaluating the effect of time-dependent localization on small ensembles. Dashed curves show the case with time-dependent localization by archived HF runs (HF10), double-dashed curves show ratios with a time-averaged HF localization (HFS10), dotted curves show ratios with time-dependent GC localization (GC10), and triple dot–dashed curves show ratios with a time-averaged GC localization (HFS10).

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 15.
Fig. 15.

Time-averaged leading eigenvalues of the prior (T2, Q) joint-state localized covariance matrices for large ensembles.

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 16.
Fig. 16.

Time-averaged leading eigenvalues of the prior (T2, T) joint-state localized covariance matrices for small ensembles.

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Fig. 17.
Fig. 17.

Time-averaged leading eigenvalues of the prior (T2, Q) joint-state localized covariance matrices for small ensembles.

Citation: Monthly Weather Review 135, 3; 10.1175/MWR3333.1

Table 1.

Experiment summary. The subscripts denote the number of ensemble members.

Table 1.

* The National Center for Atmospheric Research is sponsored by the National Science Foundation.

Save