Search Results

You are looking at 1 - 10 of 27 items for

  • Author or Editor: Mark Buehner x
  • Refine by Access: All Content x
Clear All Modify Search
Mark Buehner

Abstract

Many ensemble data assimilation (DA) approaches suffer from the so-called inbreeding problem. As a consequence, there is an excessive reduction in ensemble spread by the DA procedure, causing the analysis ensemble spread to systematically underestimate the uncertainty of the ensemble mean analysis. The stochastic EnKF used for operational NWP in Canada largely avoids this problem by applying cross validation, that is, using an independent subset of ensemble members for updating each member. The goal of the present study is to evaluate two new variations of the local ensemble transform Kalman filter (LETKF) that also incorporate cross validation. In idealized numerical experiments with Gaussian-distributed background ensembles, the two new LETKF approaches are shown to produce reliable analysis ensembles such that the ensemble spread closely matches the uncertainty of the ensemble mean, without any ensemble inflation. In ensemble DA experiments with highly nonlinear idealized forecast models, the deterministic version of the LETKF with cross validation quickly diverges, but the stochastic version produces better results, nearly identical to the stochastic EnKF with cross validation. In the context of a regional NWP system, ensemble DA experiments are performed with the two new LETKF-based approaches with cross validation, the standard LETKF, and the stochastic EnKF. All approaches with cross validation produce similar ensemble spread at the first analysis time, though the amplitude of the changes to the individual members is larger with the stochastic approaches. Over the 10-day period of the experiments, the fit of the ensemble mean background state to radiosonde observations is statistically indistinguishable for all approaches evaluated.

Open access
Mark Buehner

Abstract

In this study, several approaches for estimating background-error covariances from an ensemble of error realizations are examined, including a new spatial/spectral localization approach. The new approach shares aspects of both the spatial localization and wavelet-diagonal approaches. This approach also enables the use of different spatial localization functions for the covariances associated with each of a set of overlapping horizontal wavenumber bands. The use of such scale-dependent spatial localization (more severe localization for small horizontal scales) is shown to reduce the error in spatial correlation estimates. A comparison of spatial localization, spatial/spectral localization, and wavelet-diagonal approaches shows that the approach resulting in the lowest estimation error depends on the ensemble size. For a relatively large ensemble (48 members), the spatial/spectral localization approach produces the lowest error. When using a much smaller ensemble (12 members), the wavelet-diagonal approach results in the lowest error. Qualitatively, the horizontal correlation functions resulting from spatial/spectral localization appear smoother and less noisy than those from spatial localization, but preserve more of the heterogeneous and anisotropic nature of the raw sample correlations than the wavelet-diagonal approach. The new spatial/spectral localization approach is compared with spatial localization in a set of 1-month three-dimensional variational data assimilation (3D-Var) experiments using a full set of real atmospheric observations. Preliminary results show that spatial/spectral localization provides a nearly similar forecast quality, and in some regions improved forecast quality, as spatial localization while using an ensemble of half the size (48 vs 96 members).

Full access
Mark Buehner and Ahmed Mahidjiba

Abstract

This study examines the sensitivity of global ensemble forecasts to the use of different approaches for specifying both the initial ensemble mean and perturbations. The current operational ensemble prediction system of the Meteorological Service of Canada uses the ensemble Kalman filter (EnKF) to define both the ensemble mean and perturbations. To evaluate the impact of different approaches for obtaining the initial ensemble perturbations, the operational EnKF approach is compared with using either no initial perturbations or perturbations obtained using singular vectors (SVs). The SVs are computed using the (dry) total-energy norm with a 48-h optimization time interval. Random linear combinations of 60 SVs are computed for each of three regions. Next, the impact of replacing the initial ensemble mean, currently the EnKF ensemble mean analysis, with the higher-resolution operational four-dimensional variational data assimilation (4D-Var) analysis is evaluated. For this comparison, perturbations are provided by the EnKF. All experiments are performed over two-month periods during both the boreal summer and winter using a system very similar to the global ensemble prediction system that became operational on 10 July 2007. Relative to the operational configuration that relies on the EnKF, the use of SVs to compute initial perturbations produces small, but statistically significant differences in probabilistic forecast scores in favor of the EnKF both in the tropics and, for a limited set of forecast lead times, in the summer hemisphere extratropics, whereas the results are very similar in the winter hemisphere extratropics. Both approaches lead to significantly better ensemble forecasts than with no initial perturbations, though results are quite similar in the tropics when using SVs and no perturbations. The use of an initial-time norm that does not include information on analysis uncertainty and the lack of linearized moist processes in the calculation of the SVs are two factors that limit the quality of the resulting SV-based ensemble forecasts. Relative to the operational configuration, use of the 4D-Var analysis to specify the initial ensemble mean results in improved probabilistic forecast scores during the boreal summer period in the southern extratropics and tropics, but a near-neutral impact otherwise.

Full access
Mark Buehner and Dominik Jacques

Abstract

Data assimilation (DA) approaches currently used for operational numerical weather prediction (NWP) generally assume that errors in the background state are Gaussian. At the same time, approaches that make no assumptions regarding the background state probability distribution are gaining attention in research. Most such approaches, including the particle filter, are ensemble DA methods that produce an ensemble of analysis states consistent with the background and observation distributions. The present study instead proposes a non-Gaussian deterministic (NGD) DA method for producing a single deterministic analysis state. Consequently, the usual challenge of maintaining an ensemble with sufficient spread and diversity is avoided. The NGD approach uses background ensembles generated by a standard ensemble Kalman filter. A series of noncycled DA experiments is conducted to evaluate the NGD approach for assimilating precipitation derived from North American weather radars to initialize limited-area deterministic forecasts. The resulting forecasts are compared with those produced using either a local ensemble transform Kalman filter (LETKF) deterministic analysis or latent heat nudging (LHN). The experimental results indicate that, for forecast lead times beyond 1.5 h, the NGD approach improves precipitation forecasts relative to LHN. The NGD approach also leads to better temperature and zonal wind forecasts at lead times up to 12 h when compared to those obtained with either LHN or the LETKF. For precipitation, the NGD and LETKF approaches produce forecasts that are of comparable quality. Finally, simple strategies are demonstrated that combine the NGD approach for assimilating radar-derived precipitation accumulations with the ensemble–variational approach for assimilating all other observations.

Open access
Jean-François Caron and Mark Buehner

Abstract

Scale-dependent localization (SDL) consists of applying the appropriate (i.e., different) amount of localization to different ranges of background error covariance spatial scales while simultaneously assimilating all of the available observations. The SDL method proposed by Buehner and Shlyaeva for ensemble–variational (EnVar) data assimilation was tested in a 3D-EnVar version of the Canadian operational global data assimilation system. It is shown that a horizontal-scale-dependent horizontal localization leads to implicit vertical-level-dependent, variable-dependent, and location-dependent horizontal localization. The results from data assimilation cycles show that horizontal-scale-dependent horizontal covariance localization is able to improve the forecasts up to day 5 in the Northern Hemisphere extratropical summer period and up to day 7 in the Southern Hemisphere extratropical winter period. In the tropics, use of SDL results in improvements similar to what can be obtained by increasing the uniform amount of spatial localization. An investigation of the dynamical balance in the resulting analysis increments demonstrates that SDL does not further harm the balance between the mass and the rotational wind fields, as compared to the traditional localization approach. Potential future applications for the SDL method are also discussed.

Open access
Maziar Bani Shahabadi and Mark Buehner

Abstract

The all-sky assimilation of radiances from microwave instruments is developed in the 4D-EnVar analysis system at Environment and Climate Change Canada (ECCC). Assimilation of cloud-affected radiances from Advanced Microwave Sounding Unit-A (AMSU-A) temperature sounding channels 4 and 5 for non-precipitating scenes over the ocean surface is the focus of this study. Cloud-affected radiances are discarded in the ECCC operational data assimilation system due to the limitations of forecast model physics, radiative transfer models, and the strong nonlinearity of the observation operator. In addition to using symmetric estimate of innovation standard deviation for quality control, a state-dependent observation error inflation is employed at the analysis stage. The background-state clouds are scaled by a factor of 0.5 to compensate for a systematic overestimation by the forecast model before being used in the observation operator. The changes in the fit of the background state to observations show mixed results. The number of AMSU-A channels 4 and 5 assimilated observations in the all-sky experiment is 5%–12% higher than in the operational system. The all-sky approach improves temperature analysis when verified against ECMWF operational analysis in the areas where the extra cloud-affected observations were assimilated. Statistically significant reductions in error standard deviation by 1%–4% for the analysis and forecasts of temperature, specific humidity, and horizontal wind speed up to maximum 4 days were achieved in the all-sky experiment in the lower troposphere. These improvements result mainly from the use of cloud information for computing the observation-minus-background departures. The operational implementation of all-sky assimilation is planned for the fall of 2021.

Open access
Jean-François Caron and Mark Buehner

The approach of applying different amounts of horizontal localization to different ranges of background-error covariance horizontal scales as proposed by Buehner and Shlyaeva (2015) was recently implemented in the four-dimensional ensemble-variational (4DEnVar) data assimilation scheme of the global deterministic prediction system (GDPS) at Environment and Climate Change Canada operations. To maximize the benefits from this approach to reduce the sampling noise in the ensemble-derived background-error covariances, it was necessary to adopt a new weighting between the climatological and flow-dependent covariances that increases significantly the role of the latter. Thus, in December 2021 the GDPS became the first operational global deterministic medium-range weather forecasting system to rely completely on flow-dependent covariances in the troposphere and the lower stratosphere. The experiments that led to the adoption of these two related changes and their impacts on the forecasts up to 7 days for various regions of the globe during the boreal summer of 2019 and winter of 2020 are presented here. It is also illustrated that relying more on ensemble-derived covariances amplifies the positive impacts on the GDPS when the background ensemble generation strategy is improved.

Open access
Alain Caya, Mark Buehner, and Tom Carrieres

Abstract

A three-dimensional variational data assimilation (3DVAR) system has been developed to provide analyses of the ice–ocean state and to initialize a coupled ice–ocean numerical model for forecasting sea ice conditions. This study focuses on the estimation of the background-error statistics, including the spatial and multivariate covariances, and their impact on the quality of the resulting sea ice analyses and forecasts. The covariances are assumed to be horizontally homogeneous and fixed in time. The horizontal correlations are assumed to have a Gaussian shape and are modeled by integrating a diffusion equation. A relatively simple implementation of the ensemble Kalman filter is used to produce ensembles of the ice–ocean model state that are representative of background error and from which the 3DVAR covariance parameters are estimated.

Data assimilation experiments, using various configurations of 3DVAR and simpler assimilation approaches, are conducted over a 7-month period during the winter of 2006/07 for the Canadian east coast region. The only data assimilated are the gridded daily ice charts and RADARSAT image analyses produced by the Canadian Ice Service. All of the data assimilation experiments produce significantly improved short-term forecasts as compared with persistence. When assimilating the same data, the forecast quality from the experiments employing either the 3DVAR, direct insertion, or nudging is quite similar. However, assimilation of both the daily ice charts and RADARSAT image analyses in 3DVAR results in significant improvements to the sea ice concentration forecasts. This result supports the use of a data assimilation approach, such as 3DVAR, for combining multiple sources of observational data together with a sophisticated forecast model to provide analyses and forecasts of sea ice conditions.

Full access
Louis Garand, Mark Buehner, and Nicolas Wagneur

Abstract

This paper makes use of ensemble forecasts to infer the correlation between surface skin temperature T s and air temperature T a model errors. The impact of this correlation in data assimilation is then investigated. In the process of assimilating radiances that are sensitive to the surface skin temperature, the T sT a error correlation becomes important because it allows statistically optimal corrections to the background temperature profile in the boundary layer. In converse, through this correlation, surface air temperature data can substantially influence the analysis of skin temperature. One difficulty is that the T sT a correlation depends on the local static stability conditions that link the two variables. Therefore, a correlation estimate based on spatial or temporal averages is not appropriate. Ensembles of forecasts valid at the analysis time provide a novel means to infer the correlation dynamically at each model grid point. Geostationary Operational Environmental Satellite (GOES)-8 and -10 surface-sensitive imager radiances are assimilated with and without the inferred correlations in a 3D variational analysis system. The impact of the correlation on analyses is assessed using independent radiosonde data. The impact on 6-h forecasts is also evaluated using surface synoptic reports. The influence of the correlation extends from the surface to about 1.5 km. Temperature differences in the resulting analyses on the order of 0.3–0.6 K are typical in the boundary layer and may extend over broad regions. These difference patterns persist beyond 6 h into the forecasts.

Full access
Louis Garand, Sylvain Heilliette, and Mark Buehner

Abstract

The interchannel observation error correlation (IOEC) associated with radiance observations is currently assumed to be zero in meteorological data assimilation systems. This assumption may lead to suboptimal analyses. Here, the IOEC is inferred for the Atmospheric Infrared Radiance Sounder (AIRS) hyperspectral radiance observations using a subset of 123 channels covering the spectral range of 4.1–15.3 μm. Observed minus calculated radiances are computed for a 1-week period using a 6-h forecast as atmospheric background state. A well-established technique is used to separate the observation and background error components for each individual channel and each channel pair. The large number of collocations combined with the 40-km horizontal spacing between AIRS fields of view allows robust results to be obtained. The resulting background errors are in good agreement with those inferred from the background error matrix used operationally in data assimilation at the Meteorological Service of Canada. The IOEC is in general high among the water vapor–sensing channels in the 6.2–7.2-μm region and among surface-sensitive channels. In contrast, it is negligible for channels within the main carbon dioxide absorption band (13.2–15.4 μm). The impact of incorporating the IOEC is evaluated from 1D variational retrievals at 381 clear-sky oceanic locations. Temperature increments differ on average by 0.25 K, and ln(q) increments by 0.10, where q is specific humidity. Without IOEC, the weight given to the observations appears to be too high; the assimilation attempts to fit the observations nearly perfectly. The IOEC better constrains the variational assimilation process, and the rate of convergence is systematically faster by a factor of 2.

Full access