1. Introduction
Parameter estimation using ensemble-based filters (Anderson 2001) is emerging as a promising approach to optimize parameters in a complex model (Annan and Hargreaves 2004; Hacker and Snyder 2005; Annan et al. 2005a,b; Ridgwell et al. 2007; Hacker and Snyder 2005; Aksoy et al. 2006a,b; Tong and Xue 2008a,b; Nielsen-Gammon et al. 2010; Hu et al. 2010; Zhang et al. 2012; Zhang 2011a,b; Wu et al. 2012, 2013; Liu et al. 2014, manuscript submitted to J. Climate). In parameter estimation in a complex system, such as a coupled ocean–atmosphere general circulation model (CGCM), one common issue is sampling error accumulation when a large number of observations are used to update a single-value parameter sequentially (Aksoy et al. 2006a). To address this issue, Aksoy et al. (2006a) proposed a spatial updating technique that transforms a single-value parameter into a two-dimensional field and updates the field spatially, so that localization in filtering can limit the observational error accumulation. The final model parameter after each analysis has been derived in two methods. In the first method, the globally uniform parameter value is recovered using a spatial average of the entire spatially varying parameter field (SA; Aksoy et al. 2006a,b). In the second method, the spatially varying parameters are allowed to vary spatially after each analysis, in the so-called geographically dependent parameter optimization (GPO; see Wu et al. 2012, 2013).
Here, our objective is the recovery of the spatially uniform parameter value. We propose an average method called the adaptive spatial average method (ASA). The ASA is refined from the SA method to increase the efficiency of parameter estimation. The ASA uses the ensemble spread as the criterion for selecting “good” parameter values from the spatially varying parameter estimation; these good values are then averaged to give the final posterior parameter. Liu et al. (2014, manuscript submitted to J. Climate) have recently shown some examples of successful ASA estimation in a CGCM. In this paper, we will examine in detail the ASA methodology for parameter estimation in a CGCM using ensemble-based filter. The e-folding solar penetration depth (SPD) is used as the major parameter for estimation in this study. We will show that, compared with the SA method and the GPO method, our proposed ASA produces a faster convergence rate for parameter estimation. The paper is organized as follows. Section 2 briefly describes the parameter estimation scheme and the CGCM used in this study. Section 3 shows the model sensitivity to the parameter SPD. Section 4 discusses the ASA method. The ASA method is compared with the GPO method and the SA method in section 5. A summary and further discussion are given in section 6.
2. Model and method
a. Fast Ocean Atmosphere Model
Our model, the Fast Ocean Atmosphere Model (FOAM; Jacob 1997) is a CGCM with an atmospheric component having an R15 spectral (7.5° longitude, 4° latitude, and 18 layers) resolution. The ocean component is a z-coordinate model with a resolution of 2.8° longitude, 1.4° latitude, and 24 layers. Without flux adjustment, the fully coupled model has been run for over 6000 yr with no apparent drift in tropical climate (Liu et al. 2007a). In spite of its low resolution, FOAM has a reasonable tropical climatology (Liu et al. 2003), ENSO variability (Liu et al. 2000), and Pacific decadal variability (Wu et al. 2003; Liu et al. 2007b).
b. Data assimilation scheme
We will use a particular ensemble Kalman filter (EnKF) scheme, the ensemble adjustment Kalman filter (EAKF; Anderson 2001, 2003) in this study. Model parameters will be estimated simultaneously with the state variables by augmenting state variables with model parameters (Banks 1992a,b; Anderson 2001).
The e-folding SPD is used as the major testing parameter for estimation. Solar attenuation in the ocean is a function of the amount of biomass in the upper layers of the ocean (Smith and Baker 1978; Ohlmann et al. 2000). Previous studies suggest that solar penetration can have a significant impact on the surface climate in a climate model (Schneider and Zhu 1998; Nakamoto et al. 2001; Murtugudde et al. 2002; Ballabrera-Poy et al. 2007; Anderson et al. 2007). In particular, some modeling studies found that a deeper solar attenuation leads to warming in the tropical Pacific annual mean SST, which may then reduce the cold bias in the equatorial Pacific in a coupled ocean–atmosphere model (Murtugudde et al. 2002; Ballabrera-Poy et al. 2007; Anderson et al. 2007).
In this paper, we assume the “truth” SPD has a globally uniform value of 17 m, and the truth simulation is performed with this SPD. The first guess of SPD is assumed to be 20 m with an uncertainty of 3 m (standard deviation). The observation for the assimilations are the monthly mean SST and SSS, which are generated by adding a Gaussian white noise to the corresponding truth states at each grid point. The observational error scales (standard deviation) are 1 K for SST and 1 psu for SSS. An ensemble size of 30 is used in all of our experiments. A 30-yr simulation from the control truth run is used for the initialization of the ensemble, with the restart file of 1 January of each year used as the initial condition for each ensemble member. For the state variable, the upper eight layers of ocean temperature and salinity (0–235 m) are updated by the observations. The Gaspari and Cohn (1999) covariance localization is used with an influence radius of three horizontal grid points for both state variables and the parameter SPD. To extract signal-dominant state-parameter covariance, the data assimilation scheme for enhancive parameter correction (DAEPC) is applied (Zhang et al. 2012). Before the parameter estimation is activated, the data assimilation is performed in a spinup period of 2 yr during which only the state variables are estimated.
3. Model sensitivity with respect to solar penetration depth
We first investigate the model sensitivity to the solar penetration depth. Two types of parameter sensitivities need to be considered when DAEPC is used to improve the model climate. The first type is the sensitivity of the response of the model climatology to the change of the parameter; this sensitivity shows if the final model climate can be improved by tuning this specific parameter. The ocean surface climates of FOAM are significantly different between a deeper SPD (20 m) simulation and a shallower (17 m) one, characterized by a warming of up to over 0.5 K in the tropical ocean and a cooling of up to −0.5 K in the subtropical ocean (see Fig. 1 in Liu et al. 2014, manuscript submitted to J. Climate).
The second type of sensitivity tests the model’s sensitivity to parameter uncertainty (represented, say, by the ensemble spread of the parameter) in the observational space at the observational time interval; this sensitivity examines the possibility of reducing parameter uncertainty using the observations available. Furthermore, the model response to parameter uncertainty consists of linear and nonlinear parts. Since the Kalman filter framework is derived as the optimal analysis for a linear system, some features involving nonlinear dependence may be regarded as noise for parameter estimation. Successful parameter estimation requires a signal-dominant state-parameter covariance, which is derived most favorably in a model whose state variables exhibit a strong linear dependence on model parameters (Aksoy et al. 2006a,b).
An ensemble simulation starting from the same initial condition but using different values of the parameter SPD (i.e., a perturbed ensemble of parameters) demonstrates the second type of sensitivity (Fig. 1). (Here, the parameter ensemble is constructed as a Gaussian distribution with the mean of 20 m and the standard deviation of 3 m.) Since we will use the observations of monthly SST for parameter estimation, we will examine the ensemble response of the first month SST. The ensemble spread of the first month SST (monthly mean) represents the response of the model SST to the uncertainty of SPD in the observational space; the correlation coefficient between the SPD ensemble and the first month SST quantifies the linear part of the response. Figure 1 shows an overwhelmingly negative correlation between SST and SPD, implying predominantly a colder SST with a deeper SPD. This cooling is likely to be caused by the direct effect of solar penetration. Physically, a deeper SPD allows more solar radiation to penetrate below the surface layer, leaving less shortwave radiation heating the surface layer and therefore causes surface cooling. The direct effect of solar penetration is dominant in the initial months in response to a sudden change of the SPD (Hokanson 2006). One striking feature of the sensitivity is the strong variation with season and location. The SST ensemble spread is large and exhibits negative correlations in the summer hemisphere where the mixed layer is shallow and therefore the SST is more sensitive to heat flux perturbations. Figure 1 is important for our parameter estimation because it indicates the key regions for parameter estimation. The regions with large sensitivity and high correlation represent the regions of large linear model response to SPD. These regions have high signal-to-noise ratio and therefore are the regions where the observation of SST are most effective for parameter estimation. The rest of regions, which account for more than half of the grid points at each analysis step, are unlikely to provide significant information for parameter estimation.
The model monthly SST response to 3-m SPD uncertainty at different month for (a) March, (b) June, (c) September, and (d) December. The shading represents the correlation coefficient between the SPD ensemble and the first month monthly SST response while the contours represent the magnitude of the monthly SST response (ensemble spread). A 30-member ensemble simulation starts from the same initial condition but uses different values of the parameter SPD. The SPD ensemble is constructed as a Gaussian distribution with a mean of 20 m and a standard deviation of 3 m. We integrate the model from the beginning of each month to the end of the month to obtain the monthly mean response.
Citation: Journal of Climate 27, 11; 10.1175/JCLI-D-13-00091.1
4. The adaptive spatial average scheme
The sensitivity experiments in section 3 show that the model response to the parameter SPD varies significantly in both space and time. We speculate that neither GPO nor SA is most efficient for estimating the parameter. This follows that only the regions with large model-to-parameter linear response can provide state-parameter covariance with high signal/noise ratio for parameter estimation. Figure 1 implies that the state-parameter covariance is insignificant over about half of the grid points at a time and in about half of the year at a given grid point. Therefore, for the purpose of parameter estimation, the estimations are not useful for more than half of the time at a given grid, and the estimations are not useful for more than half of the grids in the basin for a given observation time. Therefore, SA and GPO are not the most efficient methods to estimate the parameter SPD, as will be shown below.
Here we refine the SA method to the adaptive spatial average method, to increase the efficiency of parameter estimation. In SA, the final spatially uniform parameter is estimated as the average of all the spatially different posteriors, each derived at a grid point using localization. The ASA is based on the idea that a parameter estimation, which will be derived from an average of spatially different posteriors, should be more accurate if it only includes the average of those posteriors of smaller uncertainties (i.e., errors). For practical applications where the truth parameter, and therefore the parameter error, is unknown, we can consider the ensemble spread as a representation of the error, as in traditional application of ensemble filtering to state variables (e.g., Evensen 2007). (We will return to this point later.) Therefore, the ensemble spread can be considered as the indicator of the quality of each posterior parameter values and a higher-quality posterior has a smaller ensemble spread. The ASA will only retain those high-quality values for the final averaging to derive the value for the spatially uniform parameter. This average value of high-quality values should have smaller error than the average value of averaging all the values as in SA, which include the high-quality as well as low-quality values. A preliminary theoretical analysis of this point is given in the appendix.
A posterior value is good if its ensemble spread is relatively small among all the posteriors estimated at all the grid points. In practice, we use a threshold of the spread ratio between the posterior and the prior to judge the quality of the posterior and a posterior with a spread ratio below the threshold is considered a good posterior to be included for the final spatial average. (It should be noted that the ensemble spread of the prior is spatially uniform over the globe. Therefore, this spread ratio of the posterior over prior does not affect the relative magnitude of the posterior.) The speed of the decrease of the parameter uncertainty depends greatly on the magnitude of the signal. Initially, the ASA can use a small ratio as the threshold because the initial parameter uncertainty is large and the response magnitude (signal) is large. The threshold will be increased during the simulation with the decrease of the parameter uncertainty. The ASA is applied every few EnKF analysis cycles to obtain sufficient numbers of good parameter posterior values. The ASA therefore differs from the SA of Aksoy et al. (2006a), in which the spatial average is performed every EnKF analysis cycle and on all grid points. A conditional covariance inflation technique (CCI) as in Aksoy et al. (2006b) is also employed here on parameter ensemble after each ASA step to avoid the filter divergence for parameter estimation. The CCI inflates the parameter ensemble back to a predefined minimum value when necessary. The predefined minimum value is also the final uncertainty target for the estimated parameter.
5. Comparison of ASA with GPO and SA
We now compare ASA with SA and GPO schemes in FOAM. Two sets of experiments of parameter estimation are performed using observations of monthly SST and SST at every grid point. The first set of experiments (EXP-1a and EXP-1b) use the GPO scheme and confirm that the parameter ensemble spread is a good index for the parameter uncertainty (Figs. 2 and 3). The second set of experiments (EXP-2a and EXP-2b; Figs. 4 and 5) compares the parameter estimations between SA and ASA schemes. The details of the experimental settings are shown in Table 1.
Solar penetration depths estimated using DAEPC with the GPO method. The total ensemble size is 30. (a),(c) The spatial distribution of parameter error values and parameter ensemble spreads after 20 years of simulation for the perfect observation experiment. (b),(d) The parameter errors and parameter ensemble spreads after 20 years of simulation for regular observation experiments.
Citation: Journal of Climate 27, 11; 10.1175/JCLI-D-13-00091.1
The estimated SPD after 40-yr simulations using DAEPC with the GPO method. (a) The scatter diagram between SPD error values and its ensemble spreads. The red line is the regression line. (b) The blue line is the averaging value of SPD using top percent grids (with smallest ensemble spread) and the red dashed lines represent 1 standard deviation of the averaging values. The black dashed line is the “truth.” Both (a) and (b) are for EXP-1a using perfect observations. (c),(d) As in (a),(b), but for EXP-1b using regular observations.
Citation: Journal of Climate 27, 11; 10.1175/JCLI-D-13-00091.1
The estimated SPD using DAEPC with the ASA (EXP-2a) and SA (EXP-2b). (a) Temporal evolution of parameter error (thick lines) and 1 standard deviation of ensemble spread (thin lines). The red lines are for EXP-2b and the blue lines are for EXP-2a; the green lines are also for EXP-2a but with a reduced minimum parameter ensemble spread of 0.2 for years 31–47. The black solid line is the “truth” and the black dashed lines are the minimum parameter ensemble spreads (uncertainty goals) for the experiments. (b) Temporal evolution of total numbers of grids used for average in ASA.
Citation: Journal of Climate 27, 11; 10.1175/JCLI-D-13-00091.1
(a) The scatter diagram between SPD error values and its ensemble spreads for EXP-2a after the first analysis cycle of parameter updating. The red line is the regression. (b) As in (a), but for after the fifth analysis cycle. (c) The numbers of “good” grids (values) for 1–5 analysis cycles of EXP-2a using ASA with different thresholds. The blue line is for the first analysis cycle, the green line is for the second, the red line is for the third, the cyan line is for the fourth, and the magenta is for the fifth. (d) The mean SPD values of the good grids from (c) respectively.
Citation: Journal of Climate 27, 11; 10.1175/JCLI-D-13-00091.1
The experiment setting. The oceanic observations are SST and SSS; atmospheric observations are T, U, and V. EXP-1a uses the perfect observations (truth). EXP-5a and EXP-5b estimate the parameter 
a. The assimilations with the GPO scheme
Both EXP-1a and EXP-1b use the GPO scheme but with different observations. EXP-1b uses regular observations that consist of the “truth” plus noise. EXP-1a, called perfect observation experiment, uses the truth from the control as the observations but nevertheless treats it as having the same uncertainty scale as in EXP-1b. For these two GPO experiments, neither EXP-1a nor EXP-1b is able to produce good parameter estimations if only the monthly SST and SSS data are assimilated. Therefore, we are forced to also assimilate daily atmosphere wind (U, V) and temperature T with an error scale of 1 m s−1 and 1 K, respectively; the observational error scales for SST and SSS are also forced to be reduced from 1 K and 1 psu to 0.5 K and 0.5 psu, respectively. The initial SPD error is also reduced from 3 to 1 m.
As speculated, the spatial pattern of the RMSE of SPD in EXP-1a is very consistent with the ensemble spread after 20 years of simulation (Figs. 2a,b). There are some regions of low uncertainty of SPD in different ocean basins. A further study shows that the low uncertainty in the midlatitude North Pacific and North Atlantic is related to the large model sensitivity to SPD during the boreal summer (Fig. 1b) and fall (Fig. 1c); the low uncertainty in the eastern South Pacific, western equatorial Pacific, South Atlantic, and southern Indian Ocean is partly related to the large sensitivity of the model SST to SPD in the austral fall (Fig. 1a) and summer (Fig. 1d). The high positive correlation between the parameter uncertainty and its ensemble spread can be seen more clearly in the scatterplot, for example, at the simulation year of 40 (Fig. 3a). The RMSE of SPD estimation and its ensemble spread show a strong positive linear correlation with only modest spread residual. The estimate values are closer to the truth when the ensemble spread is small, except for the case of very small ensemble spread (<~0.3 m in Fig. 3a). The positive correlation between the posterior error and ensemble spread supports our speculation before that the ensemble spread can be used to represent the estimation error or uncertainty. Furthermore, it is clear that a spatial average will decrease the parameter error because the average reduces the part of parameter uncertainty that is spatially independent [see Eq. (A4)]. The error of SPD can be further reduced by using only the posterior values with smaller ensemble spread for average (Fig. 3b), as hypothesized for the ASA. The error of SPD is reduced to 0.40 m when the posterior values of SPD over all the global grid points are averaged in EXP-1a (after 40 years of assimilation), compared with the global mean RMSE of SPD of 0.6 m (first RMSE and then global average); this error is decreased to 0.2 and 0.1 m when the top 50% and 20% of grid points of smallest ensemble spread are averaged, respectively. When the ensemble spread is at its smallest values, the estimated values suffer from an overshoot (i.e., the parameter error becomes negative). This phenomenon also occurs in Liu et al. (2014, manuscript submitted to J. Climate) when the similar observation coverage is applied (i.e., U, V, and T for the atmosphere and SST and SSS for the ocean). The reason for the overshoot will be discussed in a future study.
The positive correlation between the parameter uncertainty and parameter RMSE, however, is disrupted significantly when the regular observation (“truth” plus noise) is used as in EXP-1b. Now, the spatial pattern of the parameter ensemble spread (Fig. 2d) remains similar to that in EXP-1a (Fig. 2b), but the pattern of the SPD uncertainty (Fig. 2c) become very noisy. This occurs because the parameter updating using EnKF also introduces observational errors into the SPD posterior, which is equivalent to adding random noise onto the parameter posterior of EXP-1a. This noise leads to a decrease of the consistence between the SPD uncertainty and its ensemble spread. The distortion on the correlation is seen clearly in the scatterplot Fig. 3c, where the error value of SPD and its ensemble spread of EXP-1b show a very weak linear relationship with a much enhanced residual variance. Nevertheless, this correlation is still significant at the 99% level. Furthermore, since the uncertainty associated with the observation errors is spatially independent, it can be reduced dramatically using a spatial average. Indeed, the averaging values of SPD are very similar for EXP-1a and EXP-1b (cf. Figs. 3b and 3d), although the estimated values of SPD are much noisier in EXP-1b than in EXP-1a.
Overall, the consistency between the parameter uncertainty and its ensemble spread indicates that the parameter ensemble spread can be used as a good index for the uncertainty of the parameter value and therefore can be used as the criteria for selecting good posteriors for averaging. A spatial average of those good posteriors tends to give a better final estimation.
b. Comparison between SA and ASA
As discussed regarding EXP-1a and EXP-1b, and in the appendix, the uncertainty of the parameter posterior can be reduced using spatial averages. The ASA and SA are applied in EXP-2a and EXP-2b, respectively. A predefined minimum ensemble spread value of 0.3 m for the CCI is applied in EXP-2. Unlike the GPO experiments above, now, the error of SPD is reduced dramatically in both EXP-2a and EXP-2b even only with monthly mean SST and SSS observations (Fig. 4a), implying an increased robustness of parameter estimation using spatial average.
Based on the ensemble sensitivity shown in Fig. 1, we apply the ASA every six analysis cycles (6 months) in EXP-2a with an initial threshold of 0.68. To prevent the degeneration case of too few good values, the threshold increases by 0.1 until it reaches 0.98 whenever the total number of good values is smaller than a given number, here set as 400. The ASA picks different grids at different times for averaging. The number of grid points of good values also varies temporally in the range of 400–4000, which is around 2%–40% of total ocean grids (Fig. 4b). The ensemble spread of SPD initially decreases much faster than its real uncertainty (Fig. 4a), reaching the minimum parameter ensemble spread of 0.3 m in five simulation years. Although this ensemble spread (0.3 m) is smaller than the real error in years 5–20, the SPD continues to converge to its truth. The SPD error in EXP-2a is decreased from 3 to 0.3 m (the estimating goal) in 20 years (Fig. 4a).
During the assimilation cycle, the ensemble spread still remains positively correlated with the estimation errors among different points, albeit with a substantial spread (as discussed for EXP-1b in Fig. 3b). This can be seen in the two examples of scatterplots of SPD after the first and fifth spatial updating cycles in Figs. 5a and 5b, respectively. The ASA produces a good SPD estimation by averaging only a moderate number of good values (200–2000) once the threshold (the uncertainty ratios between the posterior and prior) is selected appropriately. This can be seen in Figs. 5c and 5d, which shows the number of good values and the average of these good values respectively, as functions of the threshold in ASA for the first five assimilation cycles. For example, for the first assimilation cycle, the average SPD is 18.5 m with the threshold of 0.8 m and the number of good values of ~400; the average SPD is 17.6 m with the threshold of 0.65 m and the number of good values of ~1000. If the threshold is too small, too few values are defined as good values. This will lead to a too small sample size and large sampling error, such that ASA no longer produces good results (Figs. 5b,d).
The final estimation also depends on the minimum ensemble spread specified in CCI. The error of the estimated SPD seems to saturate at the equilibrium level of ~0.2-m error in ~30 yr in EXP-2a if the minimum parameter ensemble spread remains at 0.3 m. This minimum ensemble spread can be decreased afterward to yield more accurate estimation. The ASA estimation is repeated from year 31 to year 47 but now with the minimum parameter ensemble spread reduced from 0.3 to 0.2 m; now the SPD error further decreases from 0.2 to ~0.1 m (Fig. 4a, green lines). In this case, a reduced minimum ensemble spread further improves the final convergence of the parameter estimation.
In comparison with the ASA (in EXP-2a), the spatial average using all the grid points in SA (EXP-2b) shows a considerably slower convergence in the SPD estimation, with the SPD error barely reaching 0.3 m after 47 years of assimilation (Fig. 4a, red lines). Similar to the ASA, the ensemble spread of SPD in SA also decreases much faster than its real error scale. The CCI with the minimum parameter ensemble spread of 0.3 m prevents the filter divergence of the parameter estimation. In the meantime, the evolution of estimation SPD in SA is more stable than in ASA because more grids and in turn a bigger sample size in the former than the latter. Overall, ASA demonstrates a faster convergence rate than SA for SPD estimation because the former uses only good values for averaging.
6. Summary and discussion
Refining the spatial average scheme (SA), we proposed the adaptive spatial average scheme (ASA) to improve the efficiency of the parameter estimation in a complex system, such as a CGCM. The ASA is explored in the twin experiment framework in FOAM, where the biased parameter (SPD) is the only model error source. The e-folding scale of the solar penetrating depth is used as the biased parameter for estimation. Sensitivity experiments show that the response of the FOAM to the parameter uncertainty varies spatially and temporally. The ASA is demonstrated to increases the efficiency of parameter estimation significantly over previous assimilation techniques such as the SA (Aksoy et al. 2006a) and geographic dependent parameter optimization (GPO) (Wu et al. 2012).
The ASA uses the posterior ensemble spread as the criterion to select the “good” values from the spatial updating posterior parameter values and only use the good values for the averaging to yield the globally uniform posterior. In comparison with the SA scheme, the ASA produces a faster convergence for parameter estimation. The faster convergence of ASA than SA is robust in other settings, as seen in two additional pairs of experiments the same as EXP-2a and EXP-2b, except for the observational interval of 10 days (EXP-3a and EXP-3b) and 1 day (EXP-4a and EXP-4b), respectively (Table 1). When the observational interval is shortened, the model response to the parameter uncertainty becomes more linear. However, the response amplitude still varies spatially and temporally (not shown). Therefore, ASA is still more suitable than SA. Similar to EXP-2, both EXP-3 and EXP-4 show faster decreases of the SPD ensemble spread than its real uncertainty in the initial stage. The convergence time is also shortened for a shorter observational interval. In ASA, the SPD errors reach the objective uncertainty (0.3 m) in ~10 yr (EXP-3a; Fig. 6a) and ~5 yr (EXP-4a; Fig. 6b) of simulations, for the observational interval of 10 and 1 days, respectively, whereas in SA they take ~30 yr (EXP-3b; Fig. 6a) and ~10 yr (EXP-4b; Fig. 6b). It is noted that the estimated SPD in EXP-4 (Fig. 6b) is less stable than in EXP-2 or EXP-3 (Figs. 3a and 6a). The observational interval in EXP-4 is only 1 day, whereas the decorrelation time scale of SST is a few months. This results in the accumulation of sampling error because the model SST does not have the time to respond before another observation is added. The accumulation of sampling error causes poor parameter estimation compared to the other experiments. Furthermore, the instability of the estimated parameter in Fig. 6b could become worse as the total assimilation time increases. We could increase the assimilation time interval for parameter estimation to reduce the instability of parameter estimation.
(a) The temporal evolution of SPD (thick lines) and 1 standard deviation of ensemble spread (thin lines) for EXP-3. The red lines are for EXP-3b and the blue lines are for EXP-3a. The black solid line is the “truth” and the black dashed lines are the minimum parameter ensemble spreads (uncertainty goals) for the experiments. (b) As in (a), but for EXP-4.
Citation: Journal of Climate 27, 11; 10.1175/JCLI-D-13-00091.1
The ASA is designed to deal with the spatially and temporally varying feature of model response to parameter in CGCM. As pointed out by one reviewer, for SPD, SST shows little sensitivity to the parameter perturbation in about half of the World Ocean (Figs. 1a–d). One may speculate that our experiments for the estimation of SPD are too peculiar. The SA is inferior to ASA because the posteriors in these regions of little sensitivity are subject to too large a noise (with little response signal) and therefore contaminate the SA estimation seriously. To clarify this, it will be desirable to test the estimation for a parameter that has more spatially uniform response sensitivity. Therefore, we repeated the estimation for two other parameters md and mq (also see Liu et al. 2014, manuscript submitted to J. Climate): md and mq are artificial multipliers to the momentum and latent heat fluxes between the ocean and atmosphere, respectively, with 1 as the default truth model value. The model SST sensitivity to either parameter is more uniform than for SPD (not shown). Our experiments EXP-5a and EXP-5b and EXP-6a and EXP-6b use the same experimental setting as EXP-2a and EXP-2b except for estimating the imperfect parameters md and mq, respectively (Table 1, Fig. 7). Both EXP-5a and EXP-6a show faster decreases of the parameter errors than EXP-5b and EXP-6b. It is found that md reaches the objective uncertainty of 0.04 (set by the minimum ensemble spread specified in CCI) in ~10 yr with ASA but in more than 30 years of assimilation with SA (Fig. 7a). Similarly, mq reaches an objective uncertainty of 0.04 in ~25 yr with ASA but in more than 40 years of assimilation with SA (Fig. 7b). Therefore, the improvement of ASA over SA is valid for more general cases than the SPD.
(a) The temporal evolution of 
Citation: Journal of Climate 27, 11; 10.1175/JCLI-D-13-00091.1
The ASA has also been shown successful for the estimation of multiple parameters (Liu et al. 2014, manuscript submitted to J. Climate). Therefore, we believe that the ASA method is well suited for the estimation of those parameters with a globally uniform feature in CGCM. The estimation of a spatially varying parameter in CGCM, however, remains to be further studied.
Much further work remains. All of our experiments of parameter estimation in this study were implemented in a twin experiment framework, where the sampling error is one of the major error sources for parameter estimation. The parameter estimation using the real observational data will be much more complex than that. Aside from the parameter uncertainties, the model bias can be generated in a CGCM due to model structural errors, such as the imperfect dynamical framework and the incomplete understanding for physical processes. It remains a great challenge to identify the sources of the model bias from the candidates of the model structural deficiencies, as well as the large number of model parameters. Hu et al. (2010), in their real-data parameter estimation study, pointed out that the parameter estimation using real observations might produce the right answer for the wrong reasons. Furthermore, the uncertainty generated by the model structural errors cannot be included in a single model ensemble forecast. Therefore, the background uncertainty estimated from the ensemble perturbations usually suffers a negative deficiency when we apply parameter estimation using real observations. A negatively biased background uncertainty could cause poor filter performance or even filter divergence, and therefore cause parameter estimation failure. One has to tune the inflation factor to compromise the uncertainty deficiency using a state-of-the-art inflation schemes, such as the covariance inflation/relaxation (Zhang et al. 2004), the additive inflation (Hamill and Whitaker 2005), or the adaptive covariance inflation (Anderson 2007, 2009).
Acknowledgments
We gratefully appreciate the help of Ms. M. Kirchmeier in editing the manuscript. We also thank two anonymous reviews for their comments on an earlier version of the manuscript. We gratefully acknowledge the computing resources provided on “Fusion,” a 320-node computing cluster operated by the Laboratory Computing Resource Center at Argonne National Laboratory. This research is sponsored by NSF and Chinese MOST 2012CB955200.
APPENDIX
Preliminary Theoretical Consideration for ASA
Here, we will discuss the SA and ASA from a more quantitative perspective. When we implement the spatial updating in ensemble-based parameter estimation, we obtain a spatially varying parameter posterior field. The posterior errors at different locations are correlated because the parameter priors are identical for the entire field. To quantify the effect of spatial averaging, we can separate the posterior errors into two independent components: one linearly dependent on the parameter prior error and the other uncorrelated with the first one.
The scale (variance) of each term in Eq. (A2). The blue curve is for the scale of the first term 
Citation: Journal of Climate 27, 11; 10.1175/JCLI-D-13-00091.1
We now discuss the two terms on the right-hand side of Eq. (A4) one by one, regarding the difference between SA and ASA. The first term is linearly dependent on the parameter prior error 
The second term on the right-hand side of Eq. (A4) decreases with the increase of the average sample size of M because the 
The ASA can reduce the error related to the parameter prior error in spite of a reduced the averaging sample size because good posteriors are used, which have sufficiently large 
REFERENCES
Aksoy, A., F. Zhang, and J. W. Nielsen-Gammon, 2006a: Ensemble-based simultaneous state and parameter estimation with MM5. Geophys. Res. Lett., 33, L12801, doi:10.1029/2006GL026186.
Aksoy, A., F. Zhang, and J. W. Nielsen-Gammon, 2006b: Ensemble-based simultaneous state and parameter estimation in a two-dimensional sea-breeze model. Mon. Wea. Rev., 134, 2951–2970, doi:10.1175/MWR3224.1.
Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 2884–2903, doi:10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2.
Anderson, J. L., 2003: A local least squares framework for ensemble filtering. Mon. Wea. Rev., 131, 634–642, doi:10.1175/1520-0493(2003)131<0634:ALLSFF>2.0.CO;2.
Anderson, J. L., 2007: An adaptive covariance inflation error correction algorithm for ensemble filters. Tellus, 59A, 210–224, doi:10.1111/j.1600-0870.2006.00216.x.
Anderson, J. L., 2009: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 61A, 72–83, doi:10.1111/j.1600-0870.2008.00361.x.
Anderson, W. G., A. Gnanadesikan, R. Hallberg, J. Dunne, and B. L. Samuels, 2007: Impact of ocean color on the maintenance of the Pacific cold tongue. Geophys. Res. Lett., 34, L11609, doi:10.1029/2007GL030100.
Annan, J. D., and J. C. Hargreaves, 2004: Efficient parameter estimation for a highly chaotic system. Tellus, 56A, 520–526, doi:10.1111/j.1600-0870.2004.00073.x.
Annan, J. D., J. C. Hargreaves, N. R. Edwards, and R. Marsh, 2005a: Parameter estimation in an intermediate complexity Earth system model using an ensemble Kalman filter. Ocean Modell., 8, 135–154, doi:10.1016/j.ocemod.2003.12.004.
Annan, J. D., D. J. Lunt, J. C. Hargreaves, and P. J. Valdes, 2005b: Parameter estimation in an atmospheric GCM using the ensemble Kalman filter. Nonlinear Processes Geophys., 12, 363–371, doi:10.5194/npg-12-363-2005.
Ballabrera-Poy, J., R. Murtugudde, R. H. Zhang, and A. J. Busalacchi, 2007: Coupled ocean–atmosphere response to seasonal modulation of ocean color: Impact on interannual climate experiments in the tropical Pacific. J. Climate, 20, 353–374, doi:10.1175/JCLI3958.1.
Banks, H. T., Ed., 1992a: Control and Estimation in Distributed Parameter Systems. Frontiers in Applied Mathematics, Vol. 11, SIAM, 227 pp.
Banks, H. T., 1992b: Computational issues in parameter estimation and feedback control problems for partial differential equation systems. Physica D, 60, 226–238, doi:10.1016/0167-2789(92)90239-J.
Evensen, G., 2007: Data Assimilation: The Ensemble Kalman Filter. Springer, 187 pp.
Frouin, R., D. W. Lingner, C. Gautier, K. S. Baker, and R. C. Smith, 1989: A simple analytical formula to compute clear sky total photosynthetically available solar irradiance at the ocean surface. J. Geophys. Res., 94, 9731–9742, doi:10.1029/JC094iC07p09731.
Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723–757, doi:10.1002/qj.49712555417.
Hacker, J. P., and C. Snyder, 2005: Ensemble Kalman filter assimilation of fixed screen-height observations in a parameterized PBL. Mon. Wea. Rev., 133, 3260–3275, doi:10.1175/MWR3022.1.
Hamill, T. M., and J. S. Whitaker, 2005: Accounting for the error due to unresolved scales in ensemble data assimilation: A comparison of different approaches. Mon. Wea. Rev., 133, 3132–3147, doi:10.1175/MWR3020.1.
Hokanson, E. P., 2006: The effects of solar penetration on a coupled general circulation model. M.S. thesis, Dep. of Atmospheric and Oceanic Sciences, University of Wisconsin–Madison, 110 pp.
Hu, X.-M., F. Zhang, and J. W. Nielsen-Gammon, 2010: Ensemble-based simultaneous state and parameter estimation for treatment of mesoscale model error: A real-data study. Geophys. Res. Lett., 37, L08802, doi:10.1029/2010GL043017.
Jacob, R., 1997: Low frequency variability in a simulated atmosphere–ocean system. Ph.D dissertation, University of Wisconsin–Madison, 155 pp.
Liu, Z., J. Kutzbach, and L. Wu, 2000: Modeling climatic shift of El Niño variability in the Holocene. Geophys. Res. Lett., 27, 2265–2268, doi:10.1029/2000GL011452.
Liu, Z., B. Otto-Bliesner, J. Kutzbach, L. Li, and C. Shields, 2003: Coupled climate simulations of the evolution of global monsoons in the Holocene. J. Climate, 16, 2472–2490, doi:10.1175/1520-0442(2003)016<2472:CCSOTE>2.0.CO;2.
Liu, Z., and Coauthors, 2007a: Simulating the transient evolution and abrupt change of Northern Africa atmosphere–ocean–terrestrial ecosystem in the Holocene. Quat. Sci. Rev., 26, 1818–1837, doi:10.1016/j.quascirev.2007.03.002.
Liu, Z., Y. Liu, L. Wu, and R. Jacob, 2007b: Seasonal and long-term atmospheric responses to reemerging North Pacific Ocean variability: A combined dynamical and statistical assessment. J. Climate, 20, 955–980, doi:10.1175/JCLI4041.1.
Murtugudde, R., J. Beauchamp, C. R. McClain, M. Lewis, and A. Busalaccki, 2002: Effects of penetrative radiation on the upper tropical ocean circulation. J. Climate, 15, 470–486, doi:10.1175/1520-0442(2002)015<0470:EOPROT>2.0.CO;2.
Nakamoto, S., S. Prasanna Kumar, J. M. Oberhuber, J. Ikshizaka, K. Muneyama, and R. Frouin, 2001: Response of the equatorial Pacific to chlorophyll pigment in a mixed layer isopycnal ocean general circulation model. Geophys. Res. Lett., 28, 2021–2024, doi:10.1029/2000GL012494.
Nielsen-Gammon, J. W., X.-M. Hu, F. Zhang, and J. E. Pleim, 2010: Evaluation of planetary boundary layer scheme sensitivities for the purpose of parameter estimation. Mon. Wea. Rev., 138, 3400–3417, doi:10.1175/2010MWR3292.1.
Ohlmann, J. C., D. A. Siegel, and C. D. Mobley, 2000: Ocean radiant heating. Part I: Optical influences. J. Phys. Oceanogr., 30, 1833–1848, doi:10.1175/1520-0485(2000)030<1833:ORHPIO>2.0.CO;2.
Ridgwell, A., J. C. Hargreaves, N. R. Edwards, J. D. Annan, T. M. Lenton, R. Marsh, A. Yool, and A. Watson, 2007: Marine geochemical data assimilation in an efficient Earth system model of global biogeochemical cycling. Biogeosciences, 4, 87–104, doi:10.5194/bg-4-87-2007.
Schneider, E., and Z. Zhu, 1998: Sensitivity of the simulated annual cycle of sea surface temperature in the equatorial Pacific to sunlight penetration. J. Climate, 11, 1932–1950, doi:10.1175/1520-0442-11.8.1932.
Smith, R. C., and K. S. Baker, 1978: The bio-optical state of ocean waters and remote sensing. Limnol. Oceanogr., 23, 247–259, doi:10.4319/lo.1978.23.2.0247.
Tong, M., and M. Xue, 2008a: Simultaneous estimation of microphysical parameters and atmospheric state with simulated radar data and ensemble square root Kalman filter. Part I: Sensitivity analysis and parameter identifiability. Mon. Wea. Rev., 136, 1630–1648, doi:10.1175/2007MWR2070.1.
Tong, M., and M. Xue, 2008b: Simultaneous estimation of microphysical parameters and atmospheric state with simulated radar data and ensemble square root Kalman filter. Part II: Parameter estimation experiments. Mon. Wea. Rev., 136, 1649–1668, doi:10.1175/2007MWR2071.1.
Wu, L., Z. Liu, R. Gallimore, R. Jacob, D. Lee, and Y. Zhong, 2003: Pacific decadal variability: The tropical Pacific mode and the North Pacific mode. J. Climate, 16, 1101–1120, doi:10.1175/1520-0442(2003)16<1101:PDVTTP>2.0.CO;2.
Wu, X., S. Zhang, Z. Liu, A. Rosati, T. Delworth, and Y. Liu, 2012: Impact of geographic dependent parameter optimization on climate estimation and prediction: Simulation with an intermediate coupled model. Mon. Wea. Rev., 140, 3956–3971, doi:10.1175/MWR-D-11-00298.1.
Wu, X., S. Zhang, Z. Liu, A. Rosati, and T. Delworth, 2013: A study of impact of the geographic dependence of observing system on parameter estimation with an intermediate coupled model. Climate Dyn., 40, 1789–1798, doi:10.1007/s00382-012-1385-1.
Zhang, F., C. Snyder, and J. Sun, 2004: Impact of initial estimate and observation availability on convective-scale data assimilation with an ensemble Kalman filter. Mon. Wea. Rev., 132, 1238–1253, doi:10.1175/1520-0493(2004)132<1238:IOIEAO>2.0.CO;2.
Zhang, S., 2011a: Impact of observation-optimized model parameters on decadal predictions: Simulation with a simple pycnocline prediction model. Geophys. Res. Lett., 38, L02702, doi:10.1029/2010GL046133.
Zhang, S., 2011b: A study of impacts of coupled model initial shocks and state–parameter optimization on climate predictions using a simple pycnocline prediction model. J. Climate,24, 6210–6226, doi:10.1175/JCLI-D-10-05003.1.
Zhang, S., Z. Liu, A. Rosati, and T. Delworth, 2012: A study of enhancive parameter correction with coupled data assimilation for climate estimation and prediction using a simple coupled model. Tellus, 64A, 10963, doi:10.3402/tellusa.v64i0.10963.
