## 1. Introduction

Even with perfect initial conditions, numerical weather prediction models are expected to diverge from the true state of the atmosphere. Commonly known as model error, this divergence usually arises from such factors as misrepresentation of the actual physical system, numerics, resolution, boundary conditions, and parameterization of unresolved processes. Unlike imperfect initial conditions, influences of model error on the structure and propagation of forecast error covariances have received limited attention, both because many difficulties exist for assessing model-error characteristics objectively and because of the very complicated and application-dependent nature of errors associated with operational models (Dee 1995; Etherton and Bishop 2004).

One approach within the research community to account for model error is the attempt to apply data assimilation techniques to the estimation of model error. One such branch of studies deals with the estimation of systematic forecast error (bias) through the use of weak-constraint four-dimensional variational (4DVAR) assimilation schemes. Some of the previous studies that followed this path are Derber (1989), Zupanski (1993, 1997, D’Andrea and Vautard (2000), and Lee and Lee (2003). A somewhat different approach to model error acknowledges the uncertainty in the parameters of a numerical model as the partial source and attempts to minimize it by estimating the uncertain parameters online. Zou et al. (1992) used a 4DVAR scheme to estimate the parameters of an optimal nudging data assimilation scheme. This idea of estimating parameters for data assimilation was also employed by Dee (1995) in a maximum-likelihood method to tune the model-error and/or observation-error parameters. In a later and more comprehensive study, this method was further developed (Dee and da Silva 1999; Dee et al. 1999) and not only issues such as parameter identifiability, estimation accuracy, and robustness of the method were addressed but the method was also applied to a wider variety of cases and with a more complex general circulation model. Identifiability, in this context, refers to the propensity of a parameter to be estimated from given information content. Mitchell and Houtekamer (2000) developed a method of accounting for model error in an ensemble Kalman filter context by adding an ensemble of realizations of model error to the ensemble of model predictions by a three-level quasigeostrophic model. Following Dee (1995), the method involved parameterizing the model error and using innovations to estimate the model-error parameters. Mitchell et al. (2002) extended the same method to a primitive equation context where the model-error statistics were assumed to be known, eliminating the need to adaptively estimate the model error.

In the field of atmospheric sciences, studies that involve direct estimation of model parameters have been mostly limited to the application of variational techniques. Navon (1998) provides a detailed review of meteorological and oceanographic literature on parameter estimation through variational methods. One of the earliest applications of direct parameter estimation using contemporary techniques was by Rinne and Järvinen (1993) who, through the use of the adjoint of their model, attempted to estimate the Cressman term of a barotropic model, which is used to parameterize the divergence associated with long waves. Gong et al. (1998) used a simple linear model of an equivalent barotropic vorticity equation for the streamfunction on a latitude circle and its adjoint to attempt adaptive online tuning of multiple weighting, smoothing, and physical parameters. They concluded that physical parameters to which the analysis is sensitive can be tuned along with one or two weighting parameters (a parameter that controls the trade-off between fits to the data and forecast and a parameter that controls the relative strength or the weakness of the constraint) and a smoothing parameter. In another application of simultaneous optimization of parameters and initial conditions with a 4DVAR scheme, Zhu and Navon (1999) employed a much more complex global spectral model and its adjoint where an interesting conclusion was that, although the impact of optimal initial conditions dominated that of the optimal parameters at early stages of forecasts, the model soon lost the impact of the initial conditions while the impact of optimal parameters persisted beyond 72 h. Nevertheless, at this point, it is still an open question how the relative impacts of optimal initial conditions and optimal parameters would vary in a continuous data assimilation system with more frequent analysis cycles. Clearly, such an assessment would also depend on the underlying error growth mechanisms of the system.

In the present study, we aim to address the model-error problem by adopting the parameter estimation approach in an ensemble Kalman filter (EnKF) environment. The concept of parameter estimation inherently assumes parameter uncertainty and leads to the idea that parameters should be treated as variables within a numerical model, effectively making them a part of the unknown state of the system. In the context of atmospheric and oceanic numerical modeling, this point of view naturally directs the study of parameter estimation toward the active research area of data assimilation. In recent years, data assimilation has experienced an increasing interest in ensemble-based data assimilation methods, with the EnKF and its variants drawing the most intense research attention. The “standard” EnKF formulation was first proposed by Evensen (1994) and further tested by Evensen and van Leeuwen (1996) and Evensen (1997) who concluded that the scheme performed well for large-scale models. For more background on the application of the EnKF in atmospheric sciences, see, for example, Mitchell and Houtekamer (2000), Keppenne (2000), Hamill and Snyder (2000), Anderson (2001, 2003), Keppenne and Rienecker (2002), Snyder and Zhang (2003), Zhang et al. (2004, 2006), and Houtekamer et al. (2005).

In atmospheric sciences, variational data assimilation schemes have been more widely used than ensemble-based schemes for the treatment of model error through estimation of model parameters. To our knowledge, Anderson (2001) is the first study where state augmentation, concatenation of uncertain parameters and the state variables into one composite vector, are used in the context of ensemble-based data assimilation for the estimation of model parameters. In Anderson (2001), a demonstration of the idea is performed for the forcing parameter of the 40-parameter Lorenz model (Lorenz 1996) and it is concluded that, “given the extreme difficulty of tuning sets of model parameters, an investigation of the possibility that this mechanism could be used seems to be of great importance.” Several recent papers by J. D. Annan, J. C. Hargreaves, and collaborators have further investigated the applicability of the EnKF to atmospheric systems. Annan et al. (2005a) applied the concept to the estimation of parameters of a climate model of intermediate complexity. In this application, the estimation is performed to a steady-state problem, effectively reducing the dimensionality to the number of uncertain model parameters. In a separate study, Annan and Hargreaves (2004) have applied the same concept of parameter estimation to the steady-state climatology of the much more chaotic Lorenz (1963) model. The applicability of the technique used in these two studies for parameter estimation of steady-state climatology is further generalized in Annan et al. (2005b) for an intermediate complexity general circulation model (GCM) both in a perfect-model context and with real observations. Finally, Annan (2005) applied the same concept to the Lorenz (1963) model with time-varying forcing. All of these studies provide encouraging results concerning ensemble-based estimation of model parameters. However, their focus on the climatological response of the atmosphere naturally minimizes the initial-condition sensitivity of model behavior. How parametric model error can be treated in the presence of initial-condition error is conceptually a significantly different problem and is the main focus of the present study. One very recent study that considers this problem is Hacker and Snyder (2005), where the viability of the method is demonstrated for the estimation of a single planetary boundary layer (PBL) parameter, moisture availability, in a one-dimensional PBL model context. One noteworthy aspect of the approach of Hacker and Snyder (2005) is that no specific method is utilized to counteract filter divergence, while the mentioned studies by J. D. Annan, J. C. Hargreaves, and collaborators applied covariance inflation, akin to Anderson (2001), to the prior parameter distributions. Along these lines, in this study, we chose to apply a method that we coined “conditional covariance inflation” to reduce the effects of filter divergence caused by the narrowing of parameter variance through repeated cycling of the EnKF. Detailed explanation of this phenomenon and our method to compensate for it is given in section 2c.

The present study aims to thoroughly investigate ensemble-based parameter estimation by state augmentation through its application to an idealized intermediate-complexity dynamical environment. For this purpose, the sea breeze is chosen as the prototype circulation and is modeled as the nonlinear response to a specific oscillating interior heat source in two dimensions, as in the linear theory of Rotunno (1983). The details of our two-dimensional nonlinear sea-breeze model and the sequential EnKF have been documented in a previous paper (Aksoy et al. 2005) where we demonstrated how the EnKF performs in a thermally forced environment in a perfect-model setting. Furthermore, parameter variance is treated differently compared to the studies mentioned in the previous paragraph (explained in detail in the following section). Our investigation is presented in the following format: To begin with, section 2 explains the major modifications that were made to the sea-breeze model and the EnKF to accommodate imperfect-model experiments. The results of imperfect-parameter experiments and the estimation of such model parameters are presented in section 3. We conclude with a summary and conclusions in section 4.

## 2. Modifications to the model and the filter to accommodate estimation of parameters

### a. Original model and filter properties

*x*= 0) and the land located to the right of the coast. The model time step is set at 90 s. The model equations are repeated here for ease of reference:where

**u**= (

*u*′,

*w*′) is the disturbance fluid velocity,

*η*′ = ∂

*u*′/∂

*z*is the hydrostatic vorticity, so that

*u*′ is obtained by the vertical integration of

*η*′ and

*w*′ is obtained from

*u*′ and the continuity equation, while

*b*′ =

*gθ*′/

*θ*

_{0}is the Boussinesq buoyancy. Here

*g*is the acceleration of gravity and

*θ*=

*θ*

_{0}+

*θ*(

_{B}*z*) +

*θ*′(

*x*,

*z*,

*t*) is the potential temperature, where

*θ*

_{0}and

*θ*are reference and background potential temperatures, respectively, while

_{B}*N*

^{2}= (g/

*θ*

_{0})(∂

*θ*/∂

_{B}*z*) is the square of the background Brunt–Väisälä frequency. Forcing takes the form of an explicit heating function

*Q*that varies sinusoidally in time and contains additive stochastic noise in its amplitude (in all experiments, the standard deviation of the stochastic error is set at 4 × 10

^{−6}m s

^{−3}).

Simulated land surface buoyancy observations with 40-km spacing and assumed observational noise (standard deviation) of 10^{−3} m s^{−2} are assimilated every 3 h sequentially following Snyder and Zhang (2003). The limitation to land surface buoyancy observations is mainly due to our goal of mimicking a conventional observing network. A sensitivity analysis with a single buoyancy sounding is also presented in section 3. A square root EnKF is used as proposed by Whitaker and Hamill (2002) so that observations are not perturbed. Localization of the observations is achieved through the use of Gaspari and Cohn’s (1999) fifth-order compactly supported correlation function with a large radius of influence of 100 grid points both horizontally and vertically. The ensemble consists of 50 members while an additional member is used as truth to ensure perfect ensemble statistics. The method employed for ensemble initialization is coined “climatological initialization,” which selects states from a time series produced by a previous independent run of the model, giving higher selection probability (through a normal distribution) to times within the time series that coincide with the initial time of the experiments. The model initial time corresponds to a maximum heating phase, and peak sea-breeze and peak land-breeze phases occur at the 9th and 21st hours of the diurnal cycle, respectively.

In Aksoy et al. (2005), we analyzed the behavior of the “perfect” ensemble (with the stochastic forcing term being the only source of model error) both through pure-forecast and EnKF-analysis experiments. Briefly, our findings indicate that the model exhibits moderate nonlinearity, with the strongest nonlinearity occurring along the sea-breeze front at the time of peak sea-breeze phase. Both model variables contain significant large-scale spread in their initial conditions. For vorticity, this structure is advected out of the domain within about 72 h so that at later times domain-averaged vorticity spread exhibits a diurnal nature with growth coinciding with the peak sea-breeze phase. On the other hand, because of the horizontally uniform structure of buoyancy spread, advection is weak so that buoyancy spread remains dominated by the large-scale initial condition error and does not exhibit diurnal variability. In the experiments with the EnKF, the large-scale component of error for both model variables is effectively removed at the first analysis step (3 h). With the repeated 3-hourly EnKF analyses, the error structure is altered quickly within about 24 h. For buoyancy, the stochastic heating uncertainty becomes the prevailing source of mostly large-scale error that is successfully eliminated by the filter at each analysis step. The filter also performs well at reducing smaller-scale vorticity error associated with the sea-breeze front.

### b. Initialization of parameters

As there is no “climatological information” available for model parameters, a random perturbation technique was chosen for the ensemble initialization of the model parameters to be estimated. An important issue at this point is the magnitude of variance since no straightforward guidance exists for the proper range of variance of the individual parameters to be estimated, although clearly the magnitude to be chosen will have a direct and significant impact on how well the parameter in question can be estimated. Moreover, the initial spread of the parameter should ideally also be restricted by the amount of parameter error the system is likely to possess. In this study, the ensemble spread for each parameter is initialized such that the standard deviation is equal to the initial error (absolute difference between initial mean parameter value and true parameter value) to ensure reliable initial ensemble statistics.

### c. Treatment of parameter variance

Another modification to the original form of the EnKF analysis scheme is related to the treatment of parameter variance. This is a fundamental issue because, if not given special attention, it may lead to filter divergence. In filtering applications, filter divergence commonly occurs when the prior distribution becomes too narrow so that observations have progressively smaller impact (Anderson and Anderson 1999). In this regard, parameter estimation imposes a special challenge because parameters are not dynamical variables. In other words, while the variance of both model variables and parameters is reduced at an analysis step, between analysis steps the model can only contribute to the variance of model variables while parameters remain constant. As a result, parameter spread remains unchanged between analysis steps only to be reduced again during the following analysis step. This leads to a progressively decreasing parameter variance and eventually may cause filter divergence especially if the mean parameter value does not promptly converge toward the true value or if the true parameter value varies in time.

The method chosen to treat posterior parameter spread is related to the inflation technique Anderson and Anderson (1999) employed; although here, instead of multiplying the posterior standard deviation of a given parameter by a constant factor as they did, posterior standard deviation is inflated back to a minimum predefined value when necessary. In this respect, we coined our method conditional covariance inflation. In the present study, the predefined standard deviation is empirically chosen as 1/4 of the initial standard deviation. When posterior standard deviation is smaller than this value, parameter spread is adjusted back to it. In the experiments described here, the posterior standard deviation usually decreases to 1/4 of its initial value in the first few analysis cycles, and then remains constant at that predefined value throughout the remainder of the experiment. With this technique, a parameter is guaranteed to contain a minimum amount of spread, and this spread cannot grow uncontrollably through inflation. The minimum amount of spread specified through the variance inflation technique would ideally reflect the acceptable variability of a given parameter with negligible impact on the state variables, although the level chosen here is somewhat empirical and we analyze the sensitivity of the performance of the estimation process to the chosen limit in section 3.

### d. Selection of parameters to be estimated

The criterion for the selection of parameters to be estimated was motivated by the necessity to work with a parameter space that would reflect the realistic model error associated with the sea-breeze circulation. In this regard, several factors influence the circulation and, for this reason, are critical from a numerical modeling point of view (Simpson 1994): 1) diurnal variation of the ground temperature, 2) diffusion of heat, 3) spatial and diurnal variation of static stability, 4) Coriolis force, 5) diffusion of momentum, 6) topography, and 7) prevailing winds. Up to six model parameters that control many of these critical factors are subjected to estimation attempts in various experiments. In their current form, these parameters are global and not varied spatially. These parameters and their values are given in Table 1. In all cases, the initial standard deviation of each parameter is chosen to be equal to the respective initial error. Parameters that are not estimated are left unperturbed at their perfect (true) value.

## 3. Results

### a. Single-parameter results

Results from the individual estimation of the six parameters, that is, mean horizontal wind * u*, static stability

*N*

^{2}, vertical diffusion coefficients of vorticity

*κ*and buoyancy

_{η}*κ*, heating amplitude

_{b}*A*

_{0}, and heating depth

*z*

_{0}are presented in Figs. 1 and 2. Thus, in each experiment, only one such parameter is perturbed (about its imperfect mean value) and the estimation is performed through state augmentation and updating the composite state vector while the other parameters are kept unperturbed at their correct (true) value. For each parameter case, the average of five independent experiments are computed to reduce random variations, which can be quite large. Results from individual experiments are not shown because they are quite different in nature in many cases and showing results from one experiment and not others may lead to invalid generalizations. Figure 1 shows the 72-h evolution of the mean parameter values along with the true parameter values that stay constant in time. The shaded area around the estimated mean parameter value is intended to provide visual guidance for the variability: It represents the 1-

*σ*(one standard deviation) limits of the parameter spread. Specifically, these limits are computed by averaging the standard deviations obtained individually from each 50-member ensemble. When ensemble statistics are perfect, the true parameter value should be expected to be found within this interval 68% of the time. However, two factors cause these limits to often be overestimated. Firstly, parameter spread is inflated to a predetermined value to prevent filter divergence. Secondly, the choice of the initial parameter error is ad hoc. Consequently, the 68% criterion, in this application, should rather be viewed as a lower limit to the parameter variance performance and not an absolute measure of it. In addition to the mean parameter value, the evolution of the rms error of buoyancy and vorticity can be observed in Fig. 2 where the results are shown for four of the six parameters (

*,*u

*N*

^{2},

*κ*, and

_{η}*z*

_{0}) for which the strongest sensitivity in terms of rms error is exhibited. In all of the rms error panels, the rms error from the respective estimation experiment is plotted along with the rms error from “perfect-parameter” and “no-estimation” benchmark experiments. In perfect-parameter benchmark experiments, there is no model error and therefore only state estimation is performed. With comparable filter setup, the results from these experiments represent the lower limit for expected rms error (or an upper bound for assimilation quality). No-estimation benchmark results are obtained with imperfect parameters and no parameter estimation; they represent the highest acceptable rms error (lowest acceptable filter performance) with similar state estimation quality.

Comparing the evolution of the mean estimated parameter values, it can be seen that estimated values of all six parameters approach true values within the first 24 h (eight assimilation cycles) of respective experiments and subsequently the true values stay almost always within the 1-*σ* interval. Further analysis reveals that the rate of approach to the true values varies among the six parameters. As a measure of the rate of approach to truth, “approach time” is defined as the duration it takes for a true parameter value to first fall within the 1-*σ* interval about the estimated mean that corresponds to the “desired” uncertainty level of 1/4 initial error (ideally, acceptable uncertainty). It can be seen that the approach times of all parameters are ∼12–18 h. The six parameters also show similarities with respect to the variability of their mean estimated values; the respective true values remain exclusively within the 1-*σ* interval, meeting our basic criterion of parameter estimation. In general, our conclusion is that the estimation of the mean values of all six parameters individually is successful with acceptable variability of mean estimated values.

The evolution of the rms error of both model variables is also consistent with our findings on the behavior of the mean estimated parameter values. A quick visual comparison reveals that, for all parameters, the rms error of the estimation experiments is lower than that of the no-estimation benchmark experiments. For the parameters * u* and

*κ*, the rms error of the estimation experiments is indistinguishable from that of the perfect-parameter benchmark experiments. Meanwhile, for the parameters

_{η}*N*

^{2}and

*z*

_{0}, the rms error remains slightly above but nevertheless very close to its perfect-parameter counterpart. As a result, both time-averaged rms error and standard deviation of rms error demonstrate comparable magnitudes for the parameter estimation and perfect-parameter experiments (not shown).

*z*

_{0}(33%) and

*N*

^{2}(18%), while from a vorticity point of view by

*N*

^{2}(30%) and

*κ*(15%). While these findings further substantiate our conclusion about the success of the estimation of single parameters, they also hint to minor variabilities among parameters in terms of the potential impact of the EnKF for more complicated situations.

_{η}### b. Multiple-parameter results

To obtain a more systematic picture of the EnKF’s capability and limits in providing improved analyses when multiple imperfect parameters are involved, we present here the results from two different three-parameter cases and subsequently the six-parameter estimation case. Again, for each case, the averages of five independent experiments are computed to minimize random variations. The three-parameter experiments are performed with *N* ^{2}, *κ _{η}*,

*A*

_{0}and

*,*u

*N*

^{2},

*z*

_{0}as imperfect triplets. Imperfect parameters that were estimated in the six-parameter experiments are

*,*u

*N*

^{2},

*κ*,

_{η}*κ*,

_{b}*A*

_{0}, and

*z*

_{0}.

Figures 3 and 4 show the evolution of the mean parameter values from the three-parameter experiment with *N* ^{2}, *κ _{η}*,

*A*

_{0}and

*,*u

*N*

^{2},

*z*

_{0}as imperfect parameters, respectively. The performance of the simultaneous estimation of both of the parameter triplets is comparable to that of their individual estimations. Similar to single-parameter experiments, the approach time for all parameters is ∼12–18 h (Fig. 3). Once within the 1-

*σ*acceptable uncertainty range, all parameters appear to show similar variability, with respective true values staying within the 1-

*σ*interval most of the time. Furthermore, error evolution of model variables is also very good for both cases. Because of their similarity, we have only shown here the rms error evolution for the triplet

*,*u

*N*

^{2},

*z*

_{0}(Fig. 5). We see that while the vorticity MRE is 20%, for buoyancy it is 30%. In general, we conclude that the estimation with three uncertain parameters produces consistently improved results (relative to the no-estimation benchmarks) that are independent of the parameters estimated and are very similar in nature to single-parameter estimation results. Similar results are also obtained from four-parameter estimation experiments that we performed with sets

*,*u

*N*

^{2},

*κ*,

_{b}*z*

_{0}and

*,*u

*N*

^{2},

*κ*,

_{η}*z*

_{0}(not shown).

When the uncertainty is extended to the entire set of six parameters, some deterioration of the estimation performance is observed (Fig. 6). The mean values of the parameters * u*,

*κ*,

_{b}*A*

_{0}, and

*z*

_{0}(Figs. 6a,d,e,f) are still successfully estimated so that the true values remain within the respective 1-

*σ*interval most of the time, meeting our basic criterion of parameter estimation. However, this is not observed for

*N*

^{2}and, to a smaller extent, for

*κ*(Figs. 6b,c). The deterioration in the estimation of

_{η}*N*

^{2}is noteworthy as its true value stays systematically below the 1-

*σ*interval during the entire 72-h duration, although some convergence between the true and estimated mean values is still noticeable. We also see that there is a distinct increase in both the buoyancy and vorticity rms errors (Fig. 7). For the six-parameter experiments, the buoyancy MRE is computed as 60% and the vorticity MRE is 54%, both of which are larger than their respective three-parameter estimation counterparts.

Despite the worsened estimation of *N* ^{2} and *κ _{η}*, we nevertheless conclude that the overall performance of the filter in reducing the parameter-related model error is still very good. While the mean values of four out of the six uncertain parameters converge toward their respective true values, the rms error associated with the six uncertain parameters is on average also reduced by 40% and 46% (compared with the no-estimation benchmark rms error) for buoyancy and vorticity, respectively. We believe that, for the most part, deterioration of the estimation performance of

*N*

^{2}is linked to the effect that surface observations do not provide an appropriate sampling for its correlation with buoyancy, mainly because stability’s influence on buoyancy is not local but rather extends vertically in any given column of air. For the parameter

*κ*, however, the low correlation with buoyancy is believed to be a numerical phenomenon. During the process of random perturbations, some ensemble members are occasionally assigned critically low

_{η}*κ*values to force the model into a regime that it was not designed for. This divergent behavior among the ensemble members naturally causes the correlation signal between buoyancy and

_{η}*κ*to be weak compared to other parameters for which the instability issue is much less critical. The nature of the behavior of the correlation between buoyancy and such poorly identified parameters is further explored in the following sections through both a more detailed investigation of the correlation itself and sensitivity analyses.

_{η}### c. Parameter identifiability

*) between a respective parameter and the observed variable (i.e., buoyancy) where*r

*is computed as follows:Here, cor(*r

*θ*,

*b*) represents the sample correlation between the ensemble values of any (global) parameter

_{i}*θ*and the ensemble values of buoyancy

*b*at grid point

*i*. The rms computation is carried out over

*M*, which denotes any collection of grid points. By definition,

*values range between 0 and 1, thus representing an absolute measure of correlation between a global parameter and spatially varying buoyancy. In the analyses,*r

*is computed both for surface grid points over land and for the grid points in a vertical column. This permits the effects of both surface observations over land and a vertical sounding to be evaluated. It should be noted, however, that in this application the spatial nature of the correlation between a model parameter and a state variable arises entirely from the spatial variability of the state variable. This is a direct result of the global nature of the model parameters (i.e., the fact that they have no notion of physical location).*r

Figure 8 shows the time evolution of * r* from pure ensemble forecasts (no state or parameter estimation is performed). In each of these experiments, one parameter is randomly perturbed about its mean value in addition to the initial state uncertainty contained in the 50-member ensemble. The magnitude of perturbations for both the respective parameter and the state is identical to that of the previous estimation experiments. Each member is then integrated forward for 72 h in pure forecast mode with the initially assigned, perturbed parameter value remaining constant in time. For each case, the average spatial influence of an observation network is compared by separately computing

*for the entire domain, land surface, and a single vertical sounding located approximately 90 km inland from the coast. In the figure, the results for four of the six parameters (*r

*,*u

*N*

^{2},

*κ*, and

_{η}*z*

_{0}) are shown. We see that the

*values appear to range between 0.1 and 0.6 yet there is also considerable variability within and across the parameters depending on the location of observations. In terms of absolute magnitudes, the correlation signal becomes especially low for*r

*N*

^{2}over the land surface, while

*z*

_{0}exhibits lower land surface correlations than a single sounding, although the overall level of correlation is still significantly higher for

*z*

_{0}than

*N*

^{2}. Furthermore, the correlation signal is also quite low for

*κ*.

_{η}Overall, we see that the two parameters (*N* ^{2} and *κ _{η}*), for which the estimation performance deteriorated most dramatically as the number of simultaneously estimated parameters is increased to six, also appear to exhibit the weakest sensitivity in terms of their rms correlations with the observed buoyancy variable. To analyze some of the potential factors that may explain the differences between the estimation performances as the number of parameters is increased, the time evolution of the land surface

*is compared between the single- and six-parameter estimation experiments (Fig. 9) for the parameters*r

*,*u

*N*

^{2},

*κ*, and

_{η}*z*

_{0}as a rough measure of how much the signal-to-noise ratio is affected by the increasing number of parameters. For presentation purposes, the

*values at analysis steps are omitted and only the values at forecast steps are shown. The results present some hints for the differences in the identifiability of the parameters. When the time averages of the rms correlation values are computed, a small yet consistent decrease emerges from single- to six-parameter cases for all parameters (for*r

*from 0.23 to 0.18, for*u

*N*

^{2}from 0.18 to 0.15, for

*κ*from 0.19 to 0.15, and for

_{η}*z*

_{0}from 0.19 to 0.16). Such a decrease can partially be responsible for a potential loss of identifiability, especially considering the fact that the smallest six-parameter time-averaged values coincide with the two poorly estimated parameters

*N*

^{2}and

*κ*.

_{η}To further test our findings, we now attempt to isolate the effects of the two low-correlation parameters, *N* ^{2} and *κ _{η}*, in two separate experiments (Fig. 10). In the first experiment, we exclude

*κ*from the set of uncertain parameters so that only five parameters are estimated. As was the case for the previous experiments, we present here the average results from five independent experiments. The results of this five-parameter experiment produce no striking differences from the six-parameter experiment of the previous section. Most importantly, the behavior of the mean value of the critical parameter

_{η}*N*

^{2}does not change significantly (not shown) and, similar to the six-parameter case, the true value stays outside the 1-

*σ*interval at all times, although the convergence between the true and estimated mean values is more persistent than the six-parameter counterpart. Similarly, there are also no significant differences apparent in the behavior of the other parameters (not shown). As a result, the rms error performance of the five-parameter experiment (Figs. 10a,b) does not exhibit noticeable differences compared to the six-parameter experiment. The MRE values for buoyancy and vorticity are 67% and 47%, respectively, variability in both of which (compared to the six-parameter counterparts) mostly reflect the noise associated with the limited sample size of five experiments. Our conclusion is that, although being a critical parameter by itself,

*κ*has limited influence on the overall identifiability of other parameters when more critical parameters dominate the error evolution in the system.

_{η}As another test on identifiability, we again make all six parameters uncertain but simulate a 3-hourly single sounding observation (vertically sampled buoyancy with 250-m spacing) located 90 km inland instead of the land surface observations with 40-km spacing. Previous sensitivity experiments had revealed that, especially for *N* ^{2} (Fig. 8b), the overall level of correlation with the observed buoyancy variable is very sensitive to the location of observations. As was determined by Aksoy et al. (2005), the state estimation itself is not very sensitive to a sounding observation as opposed to land surface observations, so that any significant difference in filter performance can be confidently interpreted as a contribution from the parameter estimation part, exclusively. The behavior of the mean parameter values (not shown) indicates some differences from the six-parameter experiment with land surface observations. To begin with, consistent with its stronger sounding correlations, there is an improvement in the evolution of the mean value of *N* ^{2}. On the other hand, some deterioration in the mean value of * u* also occurs while

*κ*continues to exhibit a relatively weak performance. The cumulative consequence of these individual parameters is that no significant change is observed in the rms error of the model variables (Figs. 10c,d). While, compared to the six-parameter experiment with land surface observations, there is a slight decrease of the buoyancy MRE to 56%, the vorticity MRE increases to 62%. In general, we conclude that the sounding observations, as applied to the estimation of all six parameters, do not lead to a qualitative improvement in the overall performance of the filter. Although, on one hand, the improved estimation of

_{η}*N*

^{2}contributes positively to the error reduction, the slightly worsened performance of

*apparently counterbalances this improvement. On the other hand, we also note that different parameters respond to the spatial configuration of observations differently. Thus, we believe that, in principle, a data assimilation system can be optimized to take advantage of such differences in the spatial information content of observations.*u

### d. Sensitivity of parameter estimation to filter configuration

We finally turn our attention to how the filter configuration influences the performance of the parameter estimation. Filter configuration encompasses some of the numerical attributes that control the available information content and thus the identifiability of the parameters. As was the case for the previous experiments, here too, we report the average results from five independent experiments.

One of the most critical filter parameters in terms of its influence on parameter identifiability is believed to be the ensemble size. In addition to our control experiments with 50 members, additional six-parameter sensitivity experiments are performed with 20 and 100 members and the results are summarized in Fig. 11 (mean estimated parameter values of three noteworthy parameters * u*,

*N*

^{2}, and

*κ*) and Figs. 12a,b (evolution of the rms error of model variables). As anticipated, we see that the estimated mean parameter values exhibit a considerable sensitivity to ensemble size, especially the parameters

_{η}*N*

^{2}and

*κ*. This is believed to be mainly due to the deteriorated sampling quality with 20 members. This becomes critical for the two parameters

_{η}*N*

^{2}and

*κ*that exhibited the lowest overall level of correlation to buoyancy (on land surface) in the pure-forecast experiments (Figs. 8b,c). As a result, the mean values of

_{η}*N*

^{2}and

*κ*become nonresponsive to the estimation attempt and meander about their initial value. Furthermore, although the estimated mean value of

_{η}*converges toward its true value, we see that it exhibits a much more pronounced variability as a direct consequence of the deteriorated sampling quality associated with smaller ensemble size. At the higher ensemble size of 100 members, the improvement in the estimation performance is not as dramatic compared to the differences between 50 and 20 members, although especially*u

*N*

^{2}appears to be positively influenced by the increased sampling quality. Meanwhile, the differences in the performance of individual parameters as a result of the varied ensemble size are also reflected proportionally in the rms error of the model variables. We see that both buoyancy and vorticity error increase dramatically for the 20-member case while with 100 members, the decrease in the buoyancy error is much less distinct compared to the vorticity error.

Another filter parameter that is tested for sensitivity is the radius of influence. In the control experiments, the radius of influence is kept at a large 100 grid points to minimize the effects of localization. For sensitivity purposes, we performed experiments with progressively smaller values of 75 and 50 grid points and summarize the results in Fig. 13 (mean estimated parameter values of three noteworthy parameters *N* ^{2}, *κ _{b}*, and

*z*

_{0}) and Figs. 12c,d (evolution of the rms error of model variables). In general, a distinct decrease in estimation quality is observed as the radius of influence is reduced. However, the response of individual parameters to the reduction in the radius of influence varies across parameters. For instance, the estimation of

*N*

^{2}, which is generally found to be one of the most critical parameters to estimate, appears to improve with decreasing radius of influence. On the other hand, parameters

*κ*and

_{b}*z*

_{0}, that normally do not exhibit critical identifiability, become nonresponsive to the estimation attempt. Such differences in the behavior of the parameters further reinforce our conviction that the complicated relationship between global parameters and spatial observations plays an intricate and critical role in determining the identifiability of each parameter. As was the case with the difference between surface-based and sounding observations, we believe that a nonglobal, parameter-specific strategy in the application of localization could be necessary to optimize the estimation process and ensure identifiability.

Finally, we also briefly discuss the sensitivity of the parameter estimation to the limit applied to parameter variances. Obviously, the conditional covariance inflation method that we have employed is ad hoc in nature and, therefore, our findings are not as generalizable in terms of identifiability as other filter characteristics such as ensemble size or radius of influence. However, we also believe that the relatively straightforward application of the method does enable us to establish some links between the acceptable variability of a given parameter and how that may be accounted for in an estimation system. Similar to previous sensitivity experiments, we have seen varying results (not shown) for different parameters both in terms of the mean estimated parameter values and rms error. Experiments with a smaller (1/10 of initial parameter error) and a larger (1/2 of initial parameter error) variance limit indicate that the optimal limit is parameter specific. While most parameters appear to be identifiable when a systemwide limit of 1/4 initial error is applied, *κ _{η}* becomes more responsive with the smaller limit tested while

*N*

^{2}exhibits better identifiability with the larger limit. We believe that consistency in the success of parameter estimation will necessarily depend on prior knowledge of the acceptable variability of each parameter estimated and the ability to devise an estimation scheme that takes maximum advantage of this knowledge.

## 4. Summary and conclusions

This article explores the applicability of the ensemble Kalman filter (EnKF) to simultaneous state and parameter estimation and investigates the performance of the EnKF under imperfect-model conditions where the source of model error is the uncertainty in the model parameters. The model developed and used for this purpose is the two-dimensional sea-breeze model explained in more detail in Aksoy et al. (2005). Characteristics of the EnKF are also identical to those employed there. Estimation of the imperfect parameters by the EnKF is performed using state augmentation, which is the process of concatenating unknown parameters and state variables into a single vector. Up to six model parameters are subjected to estimation attempts in various experiments. To reduce random variations among experiments that were observed to be quite large, the average of five independent experiments are computed for each reported case. The parameters in question are mean horizontal wind * u*, static stability

*N*

^{2}, vertical diffusion coefficient of buoyancy

*κ*, vertical diffusion coefficient of vorticity

_{b}*κ*, heating amplitude

_{η}*A*

_{0}, and heating depth

*z*

_{0}. The estimation of single imperfect parameters with the EnKF is in general very successful resulting in rms error of model variables that is indistinguishable from the respective perfect-parameter cases. Parameters that appear to be most critical in their estimation are

*κ*,

_{η}*N*

^{2}, and

*z*

_{0}.

As might be expected, increasing the number of imperfect parameters leads to a decline in the level of improvement achieved by parameter estimation. However, the EnKF is observed to consistently perform well and produce quantitatively similar results to single-parameter estimation for up to three imperfect parameters. When the number of imperfect parameters is increased to six, which constitute a parameter space believed to control most of the uncertainty of the sea-breeze circulation, noticeable decline in the estimation power of the filter is observed, although the overall EnKF performance in terms of the error statistics is still superior to the no-estimation benchmark scenario (when no parameter estimation is performed), as is demonstrated by the consistently improved MREs of both model variables (60% for buoyancy and 54% for vorticity). At this point, parameters that appear to have reached their identifiability threshold are *N* ^{2} and *κ _{η}*. We believe that the identifiability of

*N*

^{2}is linked to the lack of information content of the land surface buoyancy observations so that a vertical profile is not appropriately sampled. On the other hand, the lack of identifiability of

*κ*is rather related to numerical instability issues and does not appear to be critical for the overall multiparameter performance of the filter as a five-parameter experiment with perfect

_{η}*κ*does not reveal any significant improvement over the six-parameter results. Similarly, when the six-parameter estimation results with land surface observations are compared to a six-parameter experiment with sounding observations of the same 3-h frequency, although an improvement is observed in the estimation of

_{η}*N*

^{2}, a significant overall rms error performance is not obtained because of the deterioration in estimation performance of other parameters.

Several aspects of the filter configuration have been found to considerably influence the identifiability of the parameters. One such significant factor is the ensemble size. Through its direct impact on sampling quality, while an increase in the ensemble size to 100 members results in the improved estimation of *κ _{η}*, experiments with a smaller ensemble size of 20 members lead to the complete loss of identifiability of the critical parameters

*N*

^{2}and

*κ*. Interesting results are obtained from the sensitivity experiments with smaller values of radius of influence (75 and 50 grid points). While the parameters

_{η}*κ*and

_{b}*z*

_{0}, which otherwise exhibit strong identifiability, become completely nonresponsive to estimation attempts with a radius of influence of 50 grid points,

*N*

^{2}actually benefits significantly from the smaller radius of influence in terms of its identifiability. As is also hinted by the comparison of land-based and sounding observations, this parameter-specific response indicates that there is a complicated relationship between the global model parameters and the spatial information content of observations. We conjecture that a spatially oriented strategy for covariance “localization” does not necessarily lead to an optimal solution from a parameter estimation point of view.

Another important result that we obtained pertains to the sensitivity of the parameter estimation to the parameter variance limit. Our results indicate that the optimal limit is parameter specific. Although the “conditional covariance inflation” method that we applied for the treatment of parameter variance is ad hoc in nature, we nevertheless conclude that such parameter-dependent sensitivity to the variance limit provides insight for the behavior of the variability of individual parameters. It should be noted that our goal here is not to investigate the nature of parameter variability in specific details. However, we believe that a better understanding of it is imperative for the optimality of the parameter estimation, which would also involve a parameter-specific treatment of variance.

An important conclusion that we reach through our estimation and sensitivity experiments with multiple uncertain parameters is that significant nonlinearities exist between the model variables and model parameters so that the errors associated with individual parameters tend to counterbalance each other in many complicated ways. As a result, attempts to improve the estimation of one parameter unavoidably influence the estimation of other parameters. Such interactions will clearly be more critical in more complicated models with many more parameters and thus greater degrees of freedom. The parameter-dependent differences in the overall level of correlations for different spatial observation configurations and the parameter-specific sensitivity to radius of influence suggest that a straightforward spatial covariance localization does not necessarily produce optimality. We therefore believe that a key to success in such complicated models will require a unified “meta-localization” approach and conjecture that a global localization in the correlation space may be a feasible alternative. Such a technique should be expected to improve the identifiability of the model parameters considerably especially for larger models with many complex parameterization schemes. Our future plans include an attempt to develop such a method, the results of which will be reported elsewhere.

As a final note, in its current condition, there are several ad hoc components of the estimation scheme that need to be addressed in the future. As there is no direct observational or physical evidence about the variability of parameter values, in this study, the magnitude of initial parameter spread has been chosen through a subjective evaluation of sensitivity of model performance to different parameter values. For applications to more complicated models, real observations, and/or operational environments, a more systematic approach would be required to establish techniques for determining an appropriate magnitude of parameter spread.

## Acknowledgments

The authors are grateful to Chris Snyder, Rich Rotunno, Jeff Anderson, Lee Panetta, and Amy Stuart for their comments on the sea-breeze model, ensemble initiation, and filter design. A thorough review of an early version of the manuscript by Jim Hansen is greatly appreciated. This research is sponsored by the GTRI/HARC Project H24-2003 and NSF Grant ATM-0205599.

## REFERENCES

Aksoy, A., , F. Zhang, , J. W. Nielsen-Gammon, , and C. Epifanio, 2005: Ensemble-based data assimilation for thermally forced circulations.

,*J. Geophys. Res.***110****.**D16105, doi:10.1029/2004JD005718.Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation.

,*Mon. Wea. Rev.***129****,**2884–2903.Anderson, J. L., 2003: A local least squares framework for ensemble filtering.

,*Mon. Wea. Rev.***131****,**634–642.Anderson, J. L., , and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts.

,*Mon. Wea. Rev.***127****,**2741–2758.Annan, J. D., 2005: Parameter estimation using chaotic time series.

,*Tellus***57A****,**709–714.Annan, J. D., , and J. C. Hargreaves, 2004: Efficient parameter estimation for a highly chaotic system.

,*Tellus***56A****,**520–526.Annan, J. D., , J. C. Hargreaves, , N. R. Edwards, , and R. Marsh, 2005a: Parameter estimation in an intermediate complexity earth system model using an ensemble Kalman filter.

,*Ocean Modell.***8****,**135–154.Annan, J. D., , D. J. Lunt, , J. C. Hargreaves, , and P. J. Valdes, 2005b: Parameter estimation in an atmospheric GCM using the ensemble Kalman filter.

,*Nonlinear Processes Geophys.***12****,**363–371.D’Andrea, F., , and R. Vautard, 2000: Reducing systematic errors by empirically correcting model errors.

,*Tellus***52A****,**21–41.Dee, D. P., 1995: On the estimation of error covariance parameters for atmospheric data assimilation.

,*Mon. Wea. Rev.***123****,**1128–1145.Dee, D. P., , and A. M. da Silva, 1999: Maximum-likelihood estimation of forecast and observation error covariance parameters. Part I: Methodology.

,*Mon. Wea. Rev.***127****,**1822–1834.Dee, D. P., , G. Gaspari, , C. Redder, , L. Rukhovets, , and A. M. da Silva, 1999: Maximum-likelihood estimation of forecast and observation error covariance parameters. Part II: Applications.

,*Mon. Wea. Rev.***127****,**1835–1849.Derber, J. C., 1989: A variational continuous assimilation technique.

,*Mon. Wea. Rev.***117****,**2437–2446.Etherton, B. J., , and C. H. Bishop, 2004: Resilience of hybrid ensemble/3DVAR analysis schemes to model error and ensemble covariance error.

,*Mon. Wea. Rev.***132****,**1065–1080.Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics.

,*J. Geophys. Res.***99****,**C5. 10143–10162.Evensen, G., 1997: Advanced data assimilation for strongly nonlinear dynamics.

,*Mon. Wea. Rev.***125****,**1342–1354.Evensen, G., , and P. J. van Leeuwen, 1996: Assimilation of Geosat altimeter data for the Agulhas Current using the ensemble Kalman filter with a quasigeostrophic model.

,*Mon. Wea. Rev.***124****,**85–96.Gaspari, G., , and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions.

,*Quart. J. Roy. Meteor. Soc.***125****,**723–757.Gong, J., , G. Wahba, , D. R. Johnson, , and J. Tribbia, 1998: Adaptive tuning of numerical weather prediction models: Simultaneous estimation of weighting, smoothing, and physical parameters.

,*Mon. Wea. Rev.***126****,**210–231.Hacker, J. P., , and C. Snyder, 2005: Ensemble Kalman filter assimilation of fixed screen-height observations in a parameterized PBL.

,*Mon. Wea. Rev.***133****,**3260–3275.Hamill, T. M., , and C. Snyder, 2000: A hybrid ensemble Kalman filter-3D variational analysis scheme.

,*Mon. Wea. Rev.***128****,**2905–2918.Houtekamer, P. L., , H. L. Mitchell, , G. Pellerin, , M. Buehner, , M. Charron, , L. Spacek, , and B. Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations.

,*Mon. Wea. Rev.***133****,**604–620.Keppenne, C., 2000: Data assimilation into a primitive-equation model with a parallel ensemble Kalman filter.

,*Mon. Wea. Rev.***128****,**1971–1981.Keppenne, C., , and M. M. Rienecker, 2002: Initial testing of a massively parallel ensemble Kalman filter with the Poseidon isopycnal ocean general circulation model.

,*Mon. Wea. Rev.***130****,**2951–2965.Lee, M-S., , and D-K. Lee, 2003: An application of a weakly constrained 4DVAR to satellite data assimilation and heavy rainfall estimation.

,*Mon. Wea. Rev.***131****,**2151–2176.Lorenz, E. N., 1963: Deterministic non-prediodic flow.

,*J. Atmos. Sci.***20****,**130–141.Lorenz, E. N., 1996: Predictability: A problem partly solved.

*Proc. ECMWF Seminar on Predictability,*Vol. I, Reading, United Kingdom, ECMWF, 1–18.Mitchell, H., , and P. L. Houtekamer, 2000: An adaptive ensemble Kalman filter.

,*Mon. Wea. Rev.***128****,**416–433.Mitchell, H., , P. L. Houtekamer, , and G. Pellerin, 2002: Ensemble size, balance, and model-error representation in an ensemble Kalman filter.

,*Mon. Wea. Rev.***130****,**2791–2808.Rinne, J., , and H. Järvinen, 1993: Estimation of the Cressman term for a barotropic model through optimization with use of the adjoint model.

,*Mon. Wea. Rev.***121****,**825–833.Rotunno, R., 1983: On the linear theory of land and sea breeze.

,*J. Atmos. Sci.***40****,**1999–2009.Simpson, J. E., 1994:

*Sea Breeze and Local Wind*. Cambridge University Press, 234 pp.Snyder, C., , and F. Zhang, 2003: Assimilation of simulated Doppler radar observations with an ensemble Kalman filter.

,*Mon. Wea. Rev.***131****,**1663–1677.Whitaker, J. S., , and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations.

,*Mon. Wea. Rev.***130****,**1913–1924.Zhang, F., , C. Snyder, , and J. Sun, 2004: Impacts of initial estimate and observations availability on convective-scale data assimilation with an ensemble Kalman filter.

,*Mon. Wea. Rev.***132****,**1238–1253.Zhang, F., , Z. Meng, , and A. Aksoy, 2006: Tests of an ensemble Kalman filter for mesoscale and regional-scale data assimilation. Part I: Perfect model experiments.

,*Mon. Wea. Rev.***134****,**722–736.Zhu, Y., , and I. M. Navon, 1999: Impact of parameter estimation on the performance of the FSU global spectral model using its full-physics adjoint.

,*Mon. Wea. Rev.***127****,**1497–1517.Zou, X., , I. M. Navon, , and F. X. LeDimet, 1992: An optimal nudging data assimilation scheme using parameter estimation.

,*Quart. J. Roy. Meteor. Soc.***118****,**1163–1186.Zupanski, D., 1997: A general weak constraint applicable to operational 4DVAR data assimilation systems.

,*Mon. Wea. Rev.***125****,**2274–2292.Zupanski, M., 1993: Regional four-dimensional variational data assimilation in a quasi-operational forecasting environment.

,*Mon. Wea. Rev.***121****,**2396–2408.

Model parameters and their values as they were used in estimation experiments.