1. Introduction
Given observations of a geophysical system and a numerical model for simulating that system, data-assimilation methods are designed to produce (in most cases approximate) estimates of the distribution of the model state variables conditional upon observations of the system. This distribution is called the “analysis” in least squares parlance or the “posterior” in Bayesian frameworks. For least squares data-assimilation methods (e.g., variational methods, optimal interpolation, Kalman filtering) the error covariance associated with the “background” or “prior” distribution of the model state variables and the error variance associated with the observations are both required by the algorithms. Indeed, their accurate specification is a necessary condition for the optimality of least squares methods. Even in real-world modeling applications, wherein error covariances can never be precisely represented and optimality will never be achieved, reasonable specification of the error covariances still plays a critical role in designing a useful assimilation system (Dee 1995).
In addition to errors associated with the measurement process (including instrumental errors), observational errors also include “representativeness errors.” Representativeness error is commonly understood to account for physical processes and scales that are detectable through observation but are not resolvable by the numerical model (e.g., Cohn 1997; Fukumori et al. 1999; Oke and Sakov 2008). This category of errors can also be extended to include errors arising from the process of mapping from the discrete state space of the model to the observed field (sometimes called forward interpolation errors; e.g., Daley 1993). The statistical characteristics of representativeness errors will be model dependent, variable dependent, and geographically inhomogeneous.
The intent of this work is to present a simple framework for estimating observational error variances that include both measurement error and representativeness error. The approach introduced here was motived by design requirements for the assimilation of in situ ocean hydrographic data into a coarse-resolution global ocean model (i.e., horizontal resolution of order 100 km). Coarse-resolution global ocean models do not resolve sharp density fronts or oceanic mesoscale eddies. Thus, in frontal zones or regions where mesoscale variability is active, representativeness error can be the dominant contributor to the total observational error.1 Our a priori understanding of this motivated the need for a method that admits spatial inhomogeneity.
In this work we are only concerned with the estimation of observational error variance, not the full covariance. While it is possible to extend this framework to the estimation of the time and space covariances we do not cover this here for two reasons: (i) over most of the ocean in situ data are not sufficient to estimate spatial or temporal covariances without assuming parameterized forms (Menemenlis and Chechelnitsky 2000) and (ii) it is common practice in global ocean assimilation to treat observational errors as if they are spatially and temorally uncorrelated (e.g., Wunsch and Heimbach 2007; Karspeck et al. 2013; Derber and Rosati 1989; Behringer et al. 1998; Giese and Ray 2011; Balmaseda et al. 2013; Zhang et al. 2007). This practice, while difficult to justify when observational information is dense, is not unreasonable for hydrographic observations, wherein observations are only occasionally in close physical proximity. At any rate, whether or not it is reasonable, it is also a choice that is routinely made for algorithmic convenience.
The literature on representativeness error and the variety of ways that it can be estimated is vast, and an exhaustive review is not presented here. It is worth mentioning, however, that a great deal of the seminal work on estimating observational error emerged in the context of atmospheric data assimilation (e.g., Rutherford 1976; Lönnberg and Hollingsworth 1986; Daley 1993; Dee and DaSilva 1999; Desroziers and Ivanov 2001; Desroziers et al. 2005). Newer techniques involve the hierarchical Bayesian estimation of observational error (e.g., Li et al. 2009) and the estimation and treatment of correlated and time-varying observational errors (e.g., Waller et al. 2014a,b; Miyoshi et al. 2013; Janjić and Cohn 2006).
In the context of global-scale ocean data assimilation, the observation-based methods of Oke and Sakov (2008) and Forget and Wunsch (2007) make the assumption that representativeness error primarily stems from the limited resolution of model grids. From this perspective, the observational error variance can be diagnosed by analyzing the differences between observations and their area (Oke and Sakov 2008) or area and time (Forget and Wunsch 2007) averages. The downside of these methods is that they are tailored to the assimilation models for which they will be used only to the extent that averaging domains can be chosen for consistency with a model grid; no consideration of whether the model actually resolves the true large-scale dynamics is made.
Another approach to estimating observational error characteristics is to use observation-minus-forecast residuals from the successive cycling of a data-assimilation update and a short-term forecast. These residuals contain a combination of model-resolvable forecast error and unresolvable observation error. Disentangling the two requires that the time and/or geographic length scales of the error sources (and, hence, their covariance characteristics) can be assumed to be sufficiently different as to be robustly and mutually identifiable and that there is sufficient data to make a useful joint estimation. These challenges are well appreciated and have been highlighted in both the oceanographic and atmospheric data-assimilation literature (e.g., Rutherford 1972; Hollingsworth and Lönnberg 1986; Daley 1993; Dee 1995; Dee and DaSilva 1999). For example, in the context of ocean data assimilation, Richman et al. (2005) take the approach of defining observational errors as the component of the observation-minus-forecast residuals in a forced global ocean assimilation that is globally orthogonal to the model’s leading spatial eigenvectors—essentially imposing identifiability.
Instead of using observation-minus-forecast residuals, in this work we will adopt the approach of using observation-minus-simulation residuals from an ocean model forced with an observationally based estimate of the atmosphere. Menemenlis and Chechelnitsky (2000) and Fu et al. (1993) both take this approach when estimating observation and forecast error covariances for altimeter data. Model simulations are simply forecasts that have been extended long enough that boundary forcing and the internal variability dominate over initial condition information. This strategy allows for the estimation of observational error variance before any assimilation has been performed. (A discussion of the relative advantages of using observation-minus-simulation and observation-minus-forecast residuals in the context of the estimator that will be described in the body of this paper is included in the final section.) Note that the fundamental problem of needing to assume identifiability of the simulation and observational error covariance structures is still present when using observation-minus-simulation residuals. Both Menemenlis and Chechelnitsky (2000) and Fu et al. (1993) make some progress by choosing highly parameterized forms for their simulation error covariances, but Menemenlis and Chechelnitsky (2000) point out that for the ocean there is rarely sufficient data to infer the forecast or simulation error covariance.
A notable aspect of the forced-ocean residual approaches cited above is that they are based on a single realization of the model state that is meant to represent the expected value of the forecast or simulation distribution. However, the atmospheric state used to force an ocean model is, in actuality, uncertain. This uncertainty will naturally lead to uncertainty in the ocean simulation (e.g., Leeuwenburgh 2007). Typically, the net impact of forcing uncertainty on the ocean state is treated as “process” error (also referred to as “system” or “model” error) and must be inferred as part of the joint forecast and observational error estimation problem (Menemenlis and Chechelnitsky 2000; Fu et al. 1993). However, with the rise in popularity of ensemble data-assimilation systems, ensembles of possible atmospheric states that can be used for forcing ocean models are now available (e.g., Compo et al. 2011; Raeder et al. 2012; Karspeck et al. 2013; Yang and Giese 2013).
The problem of needing to jointly infer both the observational error characteristics and the error characteristics of the simulation from observation-minus-simulation residuals can be mitigated by treating an ensemble of ocean simulations as samples from the simulation distribution. With this additional information about the simulation distribution, it is not necessary to assume or infer a preferred structure of the simulation error in order to separate the observational error and simulation errors; the contribution of simulation error to the observation-minus-simulation residuals can simply be computed directly from the ensemble. That being said, care must be taken to construct a meaningful simulation ensemble and in section 2 we discuss process error in this context and outline some general guidelines for how to construct a suitable ensemble.
Section 3 presents the proposed estimator for observational error variance that is based on observation-minus-simulation residuals from an ensemble of forced-ocean simulations. In the appendix the theoretical mean and variance of the estimator are derived under an explicit set of assumptions and these are referenced in this section. We present the variance of the estimator (not to be confused with the estimator of the observational error variance) to develop an understanding of the factors that limit the usefulness of directly equating the estimation value with the true observational error variance. In particular, it is shown that the variance of the estimator depends on the variance and autocorrelation of the model simulations, the number of ensembles used for estimation, the number of observations available for computing the residuals, and the size of the true, underlying observational error variance. In section 4, we use Monte Carlo experiments to validate the analytical equations for the properties of the estimator. In section 5, we apply the technique to a state-of-the-art, non-eddy-resolving global ocean general circulation model, showing maps of the estimator. A discussion of the technique in the context of other approaches in the literature is presented in section 6.
2. The modeling framework
a. Ensemble of forced-ocean simulations
In this context, variations in f are the sole source of process error in the system. In its broadest definition, process error may also include the effects of model parameter uncertainty and numerical imprecision (e.g., Fu et al. 1993; Blanchet et al. 1997). However, it is important to note that there is no unique definition of what constitutes process error; it must be determined subjectively through consideration of how the model is intended to be used. For example, if the intention of the practitioner is to use a version of the model with variable parameters, then the additional process error that emerges as a result of parameter uncertainty should be included in the generation of the ensemble. On the other hand, if static parameters are to be used (as is most common in practice) then errors emerging as a result of parameter misspecification will fall into the category of either bias or unresolvable processes. Because we are focused here on the use of a global ocean model with static parameters, static topography, compiled code with fixed precision, etc., it is appropriate to assume that forcing variability defines the error subspace of the model. To the extent that it is desirable in other applications to consider additional forms of process error [parameter uncertainty likely being the next most important for non-eddy-resolving ocean general circulation models; P. Gent (2015, personal communication)] the ensemble of ocean states can be constructed to contain these uncertainties through manipulation of the numerical model M. As a general rule, if the practitioner generates an ensemble that underestimates (overestimates) the process error, then the estimate of the observational error variance will tend to be too high (too low).
b. The treatment of unresolved processes as observational representativeness error
Before progressing, we consider why errors of representation can be lumped together with measurement errors under the general term observation error. The real ocean, in contrast to the modeled ocean state, exists in the continuous time and space that we associate with reality. It includes scales and physical processes that are unresolvable by the model and/or impossible to accurately map from model state variables. Imagine that the real ocean state can be expressed as the sum of a random variable Z, which is the model-resolvable ocean state and a random variable 
The goal here is to use observations to constrain the model-resolvable vector random variable Z, not 
3. An estimator for the observational error variance
Moving forward, we focus on the special (but common) case where the observational error variance for a given data type is assumed to be time stationary over a fixed, but limited, geographic region. The choice of these regional boundaries is, naturally, application specific and in many cases may be a heuristic choice based on expert insight into the system behavior. Henceforth, we will refer to this more specific scalar parameter as 
Properties of the estimator
The estimator 
The model-resolvable true state (denoted
There is no correlation between observation errors and simulation errors (i.e., deviations from the mean).
The model solution is unbiased relative to the observations (or the bias can be estimated and removed).
Regarding condition i, keep in mind that 
observation errors are uncorrelated in space and time,
Here 
In the special case of no autocorrelation in the model simulations, 
The value of having an analytic form for the variance of 
4. A Monte Carlo verification of the properties of the estimator
In this section we use Monte Carlo sampling to demonstrate that (9) and (10) accurately reflect the first two moments of the estimator 
In any real application there is only one set of observations and collection of ensembles. Thus, although the estimator 
Figure 1a shows the Monte Carlo generated discrete probability density function of 
Monte Carlo estimates of the mean and variance of the estimator for comparison to the theoretical values. (a) The probability density function 
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
Monte Carlo estimates of the mean and variance of the estimator for comparison to the theoretical values. (a) The probability density function
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
Monte Carlo estimates of the mean and variance of the estimator for comparison to the theoretical values. (a) The probability density function
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
5. Illustration using the POP2 ocean model and in situ hydrographic data
Here we apply the observational error variance estimator to in situ temperature data that can be assimilated into the Parallel Ocean Program version 2 (POP2; Smith et al. 2010) global ocean general circulation model. POP2 is a level-coordinate model, with 60 vertical levels, configured with a nominal horizontal resolution of 1°, increasing to ¼° meridionally near the equator. POP2 is the ocean component of the Community Earth System Model (CESM; Gent et al. 2011). Further details on the model can be found in Danabasoglu et al. (2012). Note that the goal of this illustration is not to make the most precise estimation of the observational error but to simply illustrate the methodology.
An ensemble of 30 POP2 ocean simulations is produced by forcing with 30 unique samples of the atmospheric state from an ensemble reanalysis of the atmosphere (Raeder et al. 2012) produced with the nominal 2° resolution Community Atmosphere Model, version 4 (CAM4; Neale et al. 2013). This ensemble atmospheric reanalysis assimilates temperature and winds from radiosondes and aircraft- and satellite-derived drift winds. The ocean ensemble was initialized on 1 January 2004 from a climatological ensemble of POP2 ocean states. The ocean simulations were integrated through 31 December 2006, but 2004 was treated as a spinup period and only years 2005 and 2006 were used for estimating the observation error variance. In situ temperature observations from the 2009 World Ocean Database (WOD09; Johnson et al. 2009) were used to compute observation-minus-simulation residuals for use in (8). During 2005 and 2006, most of the in situ temperature data in the WOD09 comes from autonomous drifting profiling floats (Argo), bathythermographic (XBT) instruments, moored thermistors, and surface drifting buoys.
To compute residuals and ensemble sample statistics as called for by (8) the temperature field from each ocean ensemble was interpolated to the time and space location of the observations. (This interpolation is simply the application of the forward observation operator 
Number of in situ temperature observations in each 5° box used to compute the estimator. White boxes indicate regions where there were no observations available.
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
Number of in situ temperature observations in each 5° box used to compute the estimator. White boxes indicate regions where there were no observations available.
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
Number of in situ temperature observations in each 5° box used to compute the estimator. White boxes indicate regions where there were no observations available.
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
For this illustration, we assume time stationarity of the observational error variance, although for some applications there may be benefit to computing a seasonally dependent estimator. With enough data, this would amount to simply applying the estimator to seasonally differentiated subsets of the data.
The 
(left) Estimate of the standard deviation of observational error computed as the square root of the expected value of the estimator 
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
(left) Estimate of the standard deviation of observational error computed as the square root of the expected value of the estimator
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
(left) Estimate of the standard deviation of observational error computed as the square root of the expected value of the estimator
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
As in Fig. 3, but for longitude–depth transects at the equator and 35°N. In all panels the line indicates the 1°C contour.
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
As in Fig. 3, but for longitude–depth transects at the equator and 35°N. In all panels the line indicates the 1°C contour.
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
As in Fig. 3, but for longitude–depth transects at the equator and 35°N. In all panels the line indicates the 1°C contour.
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
The variance of our estimator will be a function of the simulation variance and autocorrelation at the time–space locations of the observations, the number of observations, the number of ensembles, and the underlying true observational error. Of course, excepting the number of ensembles and the number of observations, we do not know these parameters exactly. However within each 5° box approximate values can be assigned using sample estimates from the 2005 period. This is a very rough estimate based on the understanding that the years 2005 and 2006 will share simulation and observational network characteristics and that the sample statistics will be correct within an order of magnitude. Based on these estimated parameter values, Fig. 5 shows spatial maps of the approximate variance of the estimator at 100- and 1000-m depths. Note that the color scale is logarithmic; by these approximations there is enormous spatial and depth inhomogeneity in the variance of the estimator. What dominates the uncertainty in the estimator? In Fig. 6, the horizontal average of the percent contribution of each of the three terms in (10) to the total variance of 
The approximate variance of the estimator 
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
The approximate variance of the estimator
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
The approximate variance of the estimator
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
For each depth, the horizontal average of the percent contribution of each of the three terms to the total variance of 
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
For each depth, the horizontal average of the percent contribution of each of the three terms to the total variance of
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
For each depth, the horizontal average of the percent contribution of each of the three terms to the total variance of
Citation: Monthly Weather Review 144, 5; 10.1175/MWR-D-14-00336.1
6. Discussion
This paper presents a simple ensemble-based technique for estimating observational error variance for ocean data-assimilation systems. Given an ensemble of atmospheres that reflect the uncertainty in the forcing to the ocean, observation-minus-simulation residuals from the corresponding forced-ocean ensembles can be used to form an unbiased estimator for the observational error variance. We also derive an uncertainty associated with the estimator in the case where the standard assumption of uncorrelated observational error is made.
We show that the uncertainty in the estimator 
If one only considers the expected value of the estimator, it can appear that there is no benefit to choosing an ensemble that is both reliable and minimizes the variance of the simulation ensemble. On the other hand, if estimator precision is a goal, there is some benefit to decreasing the simulation ensemble spread and autocorrelation. The extent to which that benefit is significant will depend on the relative contributions of the three terms in (10).
This reduction of uncertainty in the simulation ensemble can result from the assimilation of data. If a data-assimilation update is performed optimally, with observational and background error covariances perfectly known and perfectly represented and the model unbiased, then the forecasts used in sequences of observation-minus-forecast residuals will remain reliable, and the estimator shown here will remain unbiased, but with increased precision. On the other hand, it is impossible to do optimal assimilation since we have no a priori knowledge of the observational error variance—and forecasts following a suboptimal update will always violate the reliability criteria. Desroziers et al. (2005), who developed a set of diagnostic statistics that could be applied after successive cycles of assimilation to assess whether the observation and forecast errors had been correctly specified, use analogous reasoning. They show that in the case of perfectly specified errors, the observation error variance is equal to the expected value of the sample covariance of observation-minus-forecasts residuals and observation-minus-analysis residuals. The advantage of using simulation residuals instead of forecast residuals from a sequential data-assimilation cycling is that the assumption of correct error statistics need only apply to the model for the estimator to remain unbiased. However, the cost of this less stringent assumption is that precision in the estimator will decrease.
We have focused on an ensemble of forced-ocean simulations for the estimation. But, in fact, the estimator 
We have also focused on the estimation of observational error variance and have not attempted to estimate the possible time or space covariance. As mentioned in the manuscript, it is straightforward to extend the estimator to multivariate space so that observational covariances could be computed. However, the lack of time and space proximate hydrographic observations makes the actual estimation practically impossible. Estimation of observational covariances is more sensible for densely sampled observations such as those derived from satellite remote sensors.
The framework presented here relies on relatively strict assumptions regarding the independence of simulation and observational errors and normally distributed processes. While these assumptions are rarely realized precisely, they tend to be useful approximations. For much of the global ocean, the large-scale oceanic response to atmospheric perturbations is well approximated with linear dynamics (e.g., Anderson and Gill 1975; Anderson and Killworth 1977; Malanotte-Rizzoli 1996); however, in some regions (e.g., high-latitude convective regimes) the extent to which nonlinear dynamics lead to nonnormal distributions could potentially be an issue. This is also an issue for any nonlinear forward operator. However, as mentioned in the appendix, even when errors are non-Gaussian, the estimator presented here is still unbiased but the variance of the estimator is no longer given by the formula presented in this work. The assumption of independence between the resolved and unresolved process poses another interesting challenge. This assumption goes far beyond this work; it is fundamental to the formulation of most data-assimilation methods. The development of state-dependent observational error statistics would be a fruitful line of research, as would the development of techniques for the joint estimation of resolvable and unresolvable processes.
Finally, it is worth commenting on one of the basic assumptions in this work: that of a “reliable” ensemble with which to generate our estimate of observational error. How can one verify that a model ensemble has been suitably formed? Unfortunately, when real data are used for benchmarking, there is no objective answer to the question of whether or not the ensemble is reliable because measures of reliability must account for observational errors [see Hamill (2001) and Anderson (1996)]. Thus, there is no way of knowing with certainty whether any emerging inconsistencies result from the misspecification of background errors or observational errors. It is a frustratingly circular state of affairs. A possible route forward is to attempt to establish reliability based on independent data sources that can be used to aggregate over time–space scales that can be assumed to be well resolved by the model. It might then be possible to assume that the error of representation (and measurement) is much much less than the background error—such that traditional measures of reliability such as the “uniformity of the rank histogram” (Hamill 2001) could be interpreted with less ambiguity.
Acknowledgments
Special thanks to Alexey Kaplan, Jeff Anderson, Robert Miller, and Doug Nychka for helpful discussions on this topic. Thanks also to the three anonymous reviewers for their constructive reading of the manuscript. This work was funded in part by the NOAA Climate Program Office under the Climate Variability and Predictability Program Grants NA09OAR4310163 and NA13OAR4310138 and by the NSF Collaborative Research EaSM2 Grant OCE-1243015.
APPENDIX
Deriving the First Two Moments of the Estimator
The simulation ensemble is reliable (
The deviations associated with the simulation ensembles are independent of one another and independent of deviations associated with the model-resolvable true state; that is,
The observation errors are uncorrelated with the simulation- and model-resolvable true state deviations; that is,
REFERENCES
Anderson, D., and A. Gill, 1975: Spin-up of a stratified ocean, with applications to upwelling. Deep-Sea Res. Oceanogr. Abstr., 22, 583–596, doi:10.1016/0011-7471(75)90046-7.
Anderson, D., and P. Killworth, 1977: Spin-up of a stratified ocean, with topography. Deep-Sea Res., 24, 709–732, doi:10.1016/0146-6291(77)90495-7.
Anderson, J. L., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9, 1518–1530, doi:10.1175/1520-0442(1996)009<1518:AMFPAE>2.0.CO;2.
Balmaseda, M. A., K. Mogensen, and A. T. Weaver, 2013: Evaluation of the ECMWF ocean reanalysis system ORAS4. Quart. J. Roy. Meteor. Soc., 139, 1132–1161, doi:10.1002/qj.2063.
Bao, Y., and A. Ullah, 2010: Expectation of quadratic forms in normal and nonnormal variables with applications. J. Stat. Plann. Inference, 140, 1193–1205, doi:10.1016/j.jspi.2009.11.002.
Behringer, D. W., M. Ji, and A. Leetmaa, 1998: An improved coupled model for ENSO prediction and implications for ocean initialization. Part I: The ocean data assimilation system. Mon. Wea. Rev., 126, 1013–1021, doi:10.1175/1520-0493(1998)126<1013:AICMFE>2.0.CO;2.
Blanchet, I., C. Frankignoul, and M. Cane, 1997: A comparison of adaptive Kalman filters for a tropical Pacific Ocean model. Mon. Wea. Rev., 125, 40–58, doi:10.1175/1520-0493(1997)125<0040:ACOAKF>2.0.CO;2.
Bormann, N., A. Collard, and P. Bauer, 2010: Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. II: Application to AIRS and IASI data. Quart. J. Roy. Meteor. Soc., 136, 1051–1063, doi:10.1002/qj.615.
Cohn, S., 1997: An introduction to estimation theory. J. Meteor. Soc. Japan, 75 (1B), 257–288.
Compo, G., and Coauthors, 2011: The Twentieth Century Reanalysis project. Quart. J. Roy. Meteor. Soc., 137, 1–28, doi:10.1002/qj.776.
Daley, R., 1993: Estimating observation error statistics for atmospheric data assimilation. Ann. Geophys., 11, 634–647.
Danabasoglu, G., S. Bates, B. Briegleb, S. R. Jayne, M. Jochum, W. Large, S. Peacock, and S. Yeager, 2012: The CCSM4 ocean component. J. Climate, 25, 1361–1389, doi:10.1175/JCLI-D-11-00091.1.
Dee, D. P., 1995: On-line estimation of error covariance parameters for atmospheric data assimilation. Mon. Wea. Rev., 123, 1128–1145, doi:10.1175/1520-0493(1995)123<1128:OLEOEC>2.0.CO;2.
Dee, D. P., and A. M. DaSilva, 1999: Maximum-likelihood estimation of forecast and observation error covariance parameters. Part I: Methodology. Mon. Wea. Rev., 127, 1822–1834, doi:10.1175/1520-0493(1999)127<1822:MLEOFA>2.0.CO;2.
Derber, J., and A. Rosati, 1989: A global oceanic data assimilation system. J. Phys. Oceanogr., 19, 1333–1347, doi:10.1175/1520-0485(1989)019<1333:AGODAS>2.0.CO;2.
Desroziers, G., and S. Ivanov, 2001: Diagnosis and adaptive tuning of observation-error parameters in a variational assimilation. Quart. J. Roy. Meteor. Soc., 127, 1433–1452, doi:10.1002/qj.49712757417.
Desroziers, G., L. Berre, B. Chapnik, and P. Poli, 2005: Diagnosis of observation, background and analysis-error statistics in observation space. Quart. J. Roy. Meteor. Soc., 131, 3385–3396, doi:10.1256/qj.05.108.
Forget, G., and C. Wunsch, 2007: Estimated global hydrographic variability. J. Phys. Oceanogr., 37, 1997–2008, doi:10.1175/JPO3072.1.
Fu, L.-L., I. Fukumori, and R. Miller, 1993: Fitting dynamic models to the Geosat sea level observations in the tropical Pacific Ocean. Part II: A linear, wind-driven model. J. Phys. Oceanogr., 23, 2162–2181, doi:10.1175/1520-0485(1993)023<2162:FDMTTG>2.0.CO;2.
Fukumori, I., R. Raghunath, L.-L. Fu, and Y. Chao, 1999: Assimilation of TOPEX/Poseidon altimeter data into a global ocean circulation model: How good are the results? J. Geophys. Res., 104, 25 647–25 665, doi:10.1029/1999JC900193.
Giese, B. S., and S. Ray, 2011: El Niño variability in simple ocean data assimilation (SODA), 1871–2008. J. Geophys. Res., 116, C02024, doi:10.1029/2010jc006695.
Gent, P. R., and Coauthors, 2011: The Community Climate System Model version 4. J. Climate, 24, 4973–4991, doi:10.1175/2011JCLI4083.1.
Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550–560, doi:10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2.
Hodyss, D., and N. Nichols, 2015: The error of representation: Basic understanding. Tellus, 67A, 24822, doi:10.3402/tellusa.v67.24822.
Hollingsworth, A., and R. Lönnberg, 1986: The statistical structure of the short-range forecast errors as determined from radiosonde data. Part I: The wind field. Tellus, 38A, 111–136, doi:10.1111/j.1600-0870.1986.tb00460.x.
Janjić, T., and S. E. Cohn, 2006: Treatment of observation error due to unresolved scales in atmospheric data assimilation. Mon. Wea. Rev., 134, 2900–2915, doi:10.1175/MWR3229.1.
Johnson, D., T. Boyer, H. Garcia, R. Locarnini, O. Baranova, and M. Zweng, 2009: World Ocean Database 2009 Documentation. NOAA Printing Office, 175 pp.
Karspeck, A., S. Yeager, G. Danabasoglu, T. Hoar, N. Collins, K. Raeder, J. Anderson, and J. Tribbia, 2013: An ensemble adjustment Kalman filter for the CCSM4 ocean component. J. Climate, 26, 7392–7413, doi:10.1175/JCLI-D-12-00402.1.
Leeuwenburgh, O., 2007: Validation of an EnKF system for OGCM initialization assimilating temperature, salinity, and surface height measurements. Mon. Wea. Rev., 135, 125–139, doi:10.1175/MWR3272.1.
Li, H., E. Kalnay, and T. Miyoshi, 2009: Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 135, 523–533, doi:10.1002/qj.371.
Lönnberg, R., and A. Hollingsworth, 1986: The statistical structure of the short-range forecast errors as determined from radiosonde data. Part II: The covariance of height and wind errors. Tellus, 38A, 137–161, doi:10.1111/j.1600-0870.1986.tb00461.x.
Lorenc, A. C., 1986: Analysis methods for numerical weather prediction. Quart. J. Roy. Meteor. Soc., 112, 1177–1194, doi:10.1002/qj.49711247414.
Magnus, J. R., 1978: The moments of products of quadratic forms in normal variables. Stat. Neerl., 32, 201–210, doi:10.1111/j.1467-9574.1978.tb01399.x.
Malanotte-Rizzoli, P., Ed., 1996: Modern Approaches to Data Assimilation in Ocean Modeling. Oceanography Series, Vol. 61, Elsevier, 455 pp.
Menemenlis, D., and M. Chechelnitsky, 2000: Error estimates for an ocean general circulation model from altimeter and acoustic tomography data. Mon. Wea. Rev., 128, 763–778, doi:10.1175/1520-0493(2000)128<0763:EEFAOG>2.0.CO;2.
Miyoshi, T., E. Kalnay, and H. Li, 2013: Estimating and including observation-error correlations in data assimilation. Inverse Probl. Sci. Eng., 21, 387–398, doi:10.1080/17415977.2012.712527.
Neale, R., J. Richter, S. Park, R. Lauritzen, S. Vavrus, P. Rasch, and M. Zhang, 2013: The mean climate of the Community Atmosphere Model (CAM4) in forced SST and fully coupled experiments. J. Climate, 26, 5150–5168, doi:10.1175/JCLI-D-12-00236.1.
Oke, P., and P. Sakov, 2008: Representation error of oceanic observations for data assimilation. J. Atmos. Oceanic Technol., 25, 1004–1017, doi:10.1175/2007JTECHO558.1.
Raeder, K., J. Anderson, N. Collins, T. Hoar, J. Kay, P. Lauritzen, and R. Pincus, 2012: DART/CAM: An ensemble data assimilation system for CESM atmospheric models. J. Climate, 25, 6304–6317, doi:10.1175/JCLI-D-11-00395.1.
Richman, J. G., R. N. Miller, and Y. H. Spitz, 2005: Error estimates for assimilation of satellite sea surface temperature data in ocean climate models. Geophys. Res. Lett., 32, L18608, doi:10.1029/2005GL023591.
Rutherford, I. D., 1972: Data assimilation by statistical interpolation of forecast error fields. J. Atmos. Sci., 29, 809–815, doi:10.1175/1520-0469(1972)029<0809:DABSIO>2.0.CO;2.
Rutherford, I. D., 1976: An operational three-dimensional multivariate statistical objective analysis scheme. Proc. JOC Study Group Conf. on Four Dimensional Data Assimilation, Paris, France, WMO, 98–121.
Smith, R., and Coauthors, 2010: The Parallel Ocean Program (POP) reference manual—Ocean component of the Community Climate System Model (CCSM) and Community Earth System Model (CESM). Los Alamos National Laboratory Tech. Rep LAUR-10-01853, 141 pp. [Available online at http://www.cesm.ucar.edu/models/ccsm4.0/pop/doc/sci/POPRefManual.pdf.]
Springer, M., 1979: The Algebra of Random Variables. Wiley, 470 pp.
Stewart, L., S. Dance, and N. Nichols, 2008: Correlated observation errors in data assimilation. Int. J. Numer. Methods Fluids, 56, 1521–1527, doi:10.1002/fld.1636.
Waller, J., S. Dance, A. Lawless, and N. Nichols, 2014a: Estimating correlated observation error statistics using an ensemble transform Kalman filter. Tellus, 66A, 23294, doi:10.3402/tellusa.v66.23294.
Waller, J., S. Dance, A. Lawless, N. Nichols, and J. Eyre, 2014b: Representativity error for temperature and humidity using the Met Office high-resolution model. Quart. J. Roy. Meteor. Soc., 140, 1189–1197, doi:10.1002/qj.2207.
Wunsch, C., and P. Heimbach, 2007: Practical global oceanic state estimation. Physica D, 230, 197–208, doi:10.1016/j.physd.2006.09.040.
Yang, C., and B. S. Giese, 2013: El Niño Southern Oscillation in an ensemble ocean reanalysis and coupled climate models. J. Geophys. Res. Oceans, 118, 4052–4071, doi:10.1002/jgrc.20284.
Zhang, S., M. J. Harrison, A. Rosati, and A. Wittenberg, 2007: System design and evaluation of coupled ensemble data assimilation for global oceanic climate studies. Mon. Wea. Rev., 135, 3541–3564, doi:10.1175/MWR3466.1.
For example, the results of Oke and Sakov (2008), Richman et al. (2005), and Forget and Wunsch (2007) support the ranges of 0.5°–3.0°C for the standard deviation of observed temperature error in the Gulf Stream for nominal 1° grids, while measurement error is estimated at 0.005°C (for Argo profiling floats) to 0.1°C [for expendable bathythermographs (XBTs); Johnson et al. (2009)].
Neither of these things will be strictly true for most state-of-the-art global ocean models, but it is a useful approximation at coarse resolutions.
Note that the estimator presented in the next section will still be unbiased in the case of a non-Gaussian likelihood.
Note that it is certainly possible to construct a more complex hierarchical spatial model of the observational error variance. And there may be some benefit to doing so for very data-sparse applications. But since one of the goals of this method is ease of implementation, this simpler framework of regionally independent estimation is presented here.
Five-degree boxes were used because we found that there were not enough observations in 2° boxes over 1-yr time frames to make meaningful estimates of the observational error variance.
This identity is easily derivable with basic algebraic expansion of the ensemble average operator.
