1. Introduction
There has been impressive progress in the area of ensemble prediction since the early 1990s. Nowadays many global and regional ensemble prediction systems (EPSs) are running operationally to provide probabilistic forecasts, especially for high-impact weather (Hamill and Swinbank 2015; Belair 2015). Yet, the best practice in the ensemble design for a reliable probabilistic forecast remains a challenge in research. Specifically, while forecasts of free atmospheric variables on the global scale are quite reliable and skillful (Buizza et al. 2005; Wang et al. 2014; Tennant and Beare 2014), it is very common that the forecasts of screen-level variables, such as 2-m temperature, 10-m wind, and precipitation, are underdispersive in many EPSs (Hamill and Colucci 1997; Clark et al. 2009; Hacker et al. 2011; Romine et al. 2014; Schwartz et al. 2014). That leads to forecasts of screen-level variables, which are unreliable and overconfident. Several factors are probably contributing to this particular forecast problem near the surface; for example, inadequate representation of uncertainties in the initial conditions (ICs) and in the model, in particular in the lower atmosphere and at the land surface, and even not taking into account the observation errors in the verification (Tennant and Beare 2014). In operational practice, many EPSs do not perturb the surface ICs, and thus use the same surface ICs for each member of the ensemble system. For short-range regional ensemble forecast, the lack or inappropriate generation of surface IC perturbations could be a crucial factor for the underestimation of the spread near the surface. A number of studies (Guo et al. 2006; Koster et al. 2006; Sutton et al. 2006; Barthlott and Kalthoff 2011; Lavaysse et al. 2013) investigated the model sensitivity to soil moisture and other land surface perturbations. They showed that perturbing surface ICs, such as soil moisture, have a positive impact on the skill of short-range probabilistic forecast of screen-level variables. Tennant and Beare (2014) demonstrated that regional modeling systems can be highly sensitive to the way in which they are initialized. Therefore, accounting for surface IC perturbations becomes particularly important in generating sufficient dispersion near the surface, which is an important feature of a regional EPS.
A few ideas on quantifying the surface IC uncertainties have been proposed in the literature, in particular Du et al. (2012) suggested the use of various surface ICs from different land surface data assimilation systems in short-range ensemble forecasting (SREF; Du and Tracton 2001) at the National Centers for Environmental Prediction (NCEP). Lavaysse et al. (2013) studied the performance of perturbing the two prognostic land surface variables. They used a Markov chain process to perturb soil moisture and soil temperature with a two-dimensional random and independent function on the sphere correlated in space, with a probability density function symmetric around the mean (Li et al. 2008; Charron et al. 2010). Tennant and Beare (2014) used a similar scheme to perturb the sea surface temperature (SST) in the Met Office Global and Regional Ensemble Prediction System (MOGREPS; Bowler et al. 2008). They employed a first-order Markov process as described in Frankignoul (1985), where the spatial structure of the stochastic perturbation is computed based on the statistics of the daily mean SST. They concluded that there is a scope for improvements to regional model spread and reduced forecast error through additional IC perturbations to soil moisture and surface temperature.
One of the major concerns in the stochastic methods is that perturbations are generated randomly, and without any physical realism. Different from the stochastic schemes for perturbing surface ICs as mentioned above, Wang et al. (2010) proposed a noncycling surface breeding (NCSB) technique. The perturbations in NCSB are not randomly seeded, but dynamically constrained. The main idea is to use short-range surface forecasts driven by perturbed atmospheric forcings for generating the perturbation to surface ICs. It allows for error growth consistent with the dynamical model. Such perturbations are physically more realistic and may be beneficial for the forecast. A similar technique was applied by Deng et al. (2012) for ensemble tropical cyclone forecasts in NCEP GEFS and by Tennant and Beare (2014) for perturbing initial soil moisture in MOGREPS. Both studies showed an improvement in the ensemble dispersion. Their schemes only differ in details from NCSB. More discussions follow in section 2a.
NCSB is in principle a breeding technique for the surface ICs. It may well capture the uncertainties in the surface analyses from fast-growing forecast errors in atmospheric forcing. It is based on the interaction between atmosphere and land surface, rather than on the uncertainties from random and nongrowing observational errors (Toth and Kalnay 1993). Breeding only mimics the effects of the observation error (poor data coverage, representativeness errors, or measurement errors) and requires an external estimate of analysis uncertainty for the rescaling (Toth and Kalnay 1993; Buizza et al. 2008).
An alternative technique for simulating the analysis uncertainty is the ensemble of data assimilation (EDA). This technique is now used at the European Centre for Medium-Range Weather Forecast (ECMWF), the Meteorological Service of Canada (MSC), and Météo-France for the generation of atmospheric IC perturbations in their EPSs (Houtekamer and Mitchell 2005; Isaksen et al. 2007; Houtekamer et al. 2007; Berre et al. 2007; Buizza et al. 2008). By randomly perturbing the observations consistent with the observation error statistics, and by additionally perturbing the model physics in the data assimilation system, EDA provides an ensemble of analyses or perturbed ICs. Compared to breeding, an EDA might be better at simulating analysis uncertainties since it uses observation errors in a more realistic way by changing the observation coverage and error statistics (Buizza et al. 2008).
Following ECMWF, an ensemble of surface data assimilations (ESDA) has been recently implemented at Zentral Anstalt für Meteorologie und Geodynamik (ZAMG) aiming for a realistic surface IC perturbations in the regional EPS Aire Limitée Adaptation Dynamique Développement International-Limited Area Ensemble Forecasting (ALADIN-LAEF; Wang et al. 2011). This ensemble of surface analyses is based on the ALADIN surface data assimilation Code d’ Analyze Nécessaire à ARPEGE pour ses Rejets et son Initialization (CANARI; Mahfouf 1991; Bouttier et al. 1993a,b; Giard and Bazile 2000), which randomly perturbs observations, and was first tested by Horányi et al. (2011).
The aim of this paper is to study surface IC perturbations in a regional EPS by evaluating and comparing the two aforementioned techniques, NCSB and ESDA, and to address the question, how to best construct the surface IC perturbations. This might potentially lead to the improvement of probabilistic short-range forecasts near the surface.
The experiments are conducted by using the regional ensemble system ALADIN-LAEF, in which NCSB has been operational since 2009 and ESDA since 2012. A two-month late spring to summer period in 2011 is chosen for the statistical evaluation and comparison. In the following, section 2 introduces both surface initial perturbation methods NCSB and ESDA. Section 3 describes the ALADIN-LAEF configuration, observations, and experimental setup. Section 4 evaluates the results from the two-month trial and a summary and the conclusions follow in section 5.
2. Surface initial perturbation methodology
The section describes the two concerned techniques for perturbing surface ICs in ALADIN-LAEF: NCSB and ESDA.
a. Noncycling surface breeding technique
NCSB perturbs the surface ICs by using short-range surface forecasts driven by the perturbed free atmosphere. The perturbations are not randomly seeded, but rather downscaled from a global EPS. The NCSB implementation in ALADIN-LAEF is as follows. 1) The downscaled ECMWF-EPS analysis and forecast provide the set of perturbed ICs and lateral boundary conditions (LBCs) for ALADIN-LAEF at time t − 12 h, while the ECMWF surface ICs are replaced by the unperturbed Action de Recherche Petite Echelle Grande Echelle (ARPEGE; Déqué et al. 1994) soil and surface analysis valid at the same time. That is because ARPEGE soil and surface analysis better fits to the surface parameterization used in ALADIN-LAEF (Wang et al. 2010). 2) ALADIN-LAEF is then integrated up to 12 h to the time t. 3) These 12-h surface forecasts, valid at time t, are considered as perturbed surface ICs for the ALADIN-LAEF forecast starting at the time t. The amplitude of the perturbation can be addressed by including a rescaling step as it is done in the breeding method of Toth and Kalnay (1993). The same process is repeated every 12 h with the new perturbed atmosphere obtained from ECMWF-EPS. The model drifting problem is entirely avoided (Wang et al. 2010), since the fresh surface analysis from ARPEGE is always used in the NCSB implementation in ALADIN-LAEF. Deng et al. (2012) applied a similar technique to perturb the surface ICs, such as soil moisture and surface temperature, etc., but with the use of short-range surface forecasts from the latest ensemble cycle. Thus, there is a potential risk of systematic drifting. To keep the values of the perturbations within a reasonable range, Deng et al. scale down the maximum amplitude of the perturbations to climate reference values. Tennant and Beare (2014) perturbed soil moisture in the same way. However, they build the perturbation in a process that ensures the perturbations sum up to zero, so as not to introduce a systematic drift in the model.
b. Ensemble of surface data assimilations
The ensemble of surface analysis is generated by randomly perturbing observations after they passed the quality control in each single CANARI assimilation cycle. The application of perturbations just after the quality control ensures that the perturbed observations are not later rejected during the assimilation procedure.
The observation perturbation method (Storto and Randriamampianina 2008; Horányi et al. 2011) involves a pseudo–random number generator producing sets of Gaussian-distributed numbers with zero mean and standard deviation equal to the respective observation error. The random number generator is based on the “Zufall” code by the lagged-Fibonacci approach (Petersen 1994), and is initialized according to Burns and Pryor (1998). These numbers are added to the original unperturbed observations. The ESDA experiments have been performed with a 12-h CANARI assimilation cycle, where the 12-h short-range forecasts of the previous ALADIN-LAEF run have been used as the different model background for the ensemble of assimilation.
3. Numerical experiments setup
a. ALADIN-LAEF configuration
ALADIN-LAEF has been developed within the framework of the Regional Cooperation Limited Area modeling in Central Europe (RC LACE; http://www.rclace.eu), and has run operationally since March 2007. It is based on the hydrostatic spectral limited-area model ALADIN with a hybrid vertical coordinate, semi-implicit semi-Lagrangian two-time-level advection scheme, fourth-order implicit linear horizontal diffusion, digital filter initialization, and a set of sophisticated parameterizations of unresolved physics processes, among other things (Wang et al. 2006). Figure 1 shows the ALADIN-LAEF forecast and verification domains, together with the observation sites processed in the surface assimilation. ALADIN-LAEF runs operationally at a horizontal resolution of 11 km, with 45 levels in the vertical and it is integrated up to 72 h twice a day (initialized at 0000 and 1200 UTC).
ALADIN-LAEF forecast domain (the blue rectangle) and the verification domain covering central Europe (denoted by inner black lines). The dots are observation sites processed in CANARI, in which the red dots are the stations available in global GTS and the green dots are local national observations available for ALADIN-LAEF operations.
Citation: Monthly Weather Review 144, 9; 10.1175/MWR-D-16-0038.1
ALADIN-LAEF is constructed by 16 perturbed members that are coupled to the first 16 ECMWF global EPS members (Buizza and Palmer 1995; Leutbecher and Palmer 2008) through the perturbed LBCs. The upper-air IC perturbations are obtained by blending the perturbed ICs of ECMWF global EPS with the regional ALADIN-LAEF breeding vectors. The combination of the large-scale and small-scale signal is done via a sophisticated spectral blending by a digital filter technique (Wang et al. 2014). Model uncertainties are simulated by using different physics parameterization schemes (Wang et al. 2011).
b. Observations
CANARI surface data assimilation primarily assimilates the 2-m relative humidity and temperature observations. This includes the observations from Global Telecommunication System (GTS) and central European national local observations. An example of the available observations in the CANARI surface data assimilation is shown in Fig. 1.
c. Experiments
Two experiments (NCSB and ESDA) were designed to investigate the surface IC perturbation methods in a regional ensemble system ALADIN-LAEF. In Table 1, the setup of the experiments is summarized. For the sake of a clean comparison of the NCSB and ESDA methods, no other perturbations were applied (i.e., neither blending of IC atmospheric perturbations nor multiphysics were used in the ALADIN-LAEF configuration for this study). Both experiments include 16 perturbed ensemble members that are generated once per day starting at 1200 UTC, with the integration up to 54 h. However, to keep the 12-h update of the assimilation cycle in ESDA experiment, an additional short integration (+12 h) starting at 0000 UTC is performed. The experiments are performed from 15 May to 15 July 2011. The evaluation period is chosen in late spring to summer, during which the weather is expected to be more sensitive to the soil moisture modifications due to more pronounced evapotranspiration and weaker large-scale forcing with a tendency for increased convective activity.
Description of ALADIN-LAEF configurations regarding the related initial perturbation, lateral boundary perturbation, and the model perturbation used in the experiments.
An example of the IC perturbations from NCSB and ESDA for the forecasts started at 1200 UTC 1 July 2011 for surface temperature and surface layer (1 cm) soil moisture content are shown in Figs. 2 and 3, respectively. The IC perturbations of NCSB and ESDA have rather similar structure and magnitude, although the ESDA perturbations appear more localized and detailed, in particular over areas with a dense observation network.
Initial perturbations of surface temperature for the (left) NCSB and (right) ESDA experiments over (top) the whole ALADIN-LAEF domain and (bottom) central Europe. The IC perturbations are valid at 1200 UTC 1 Jul 2011.
Citation: Monthly Weather Review 144, 9; 10.1175/MWR-D-16-0038.1
Initial perturbations of soil moisture content in surface layer (1 cm) for the (left) NCSB and (right) ESDA experiments over (top) the whole ALADIN-LAEF domain and (bottom) central Europe. The IC perturbations are valid at 1200 UTC 1 Jul 2011.
Citation: Monthly Weather Review 144, 9; 10.1175/MWR-D-16-0038.1
4. Results
In this study, the ALADIN-LAEF is integrated from the ICs containing the perturbed surface fields generated by the two different methods: NCSB and ESDA. The forecasts of screen-level variables are verified against the observations for the period from 15 May to 15 July 2011, on a verification domain covering central Europe. For this purpose, the forecasted values are interpolated into the observation locations for smoothly varying fields, such as 2-m temperature and relative humidity. As the precipitation field is not spatially homogeneous, the observation is instead matched to the nearest grid point. To investigate the impact of the surface IC perturbation methods on the atmospheric fields, the ECMWF analysis was used for the verification of the upper-air weather variables. However, our primary focus was on the verification of the screen-level variables, using a set of standard ensemble and probabilistic forecast verification scores, in particular ensemble spread, root-mean-square error for the ensemble mean, percentage of outliers, continuous ranked probability score, among others.
The root-mean-square error (RMSE) of the ensemble mean measures the error of the model forecast in terms of the observed values. The smaller the RMSE, the better the forecast fits the observations. On the other hand, it is expected that the members of an EPS system represent the uncertainty of the forecast well. So, the standard deviation between all the ensemble members (the so-called spread) should be of the same order as the RMSE. If the RMSE is larger, the system is underdispersive, and indicates that the observed values often lie outside of the range covered by the ensemble forecast. However, if the RMSE is significantly smaller than the spread, the uncertainty represented by the ensemble members is larger than the mean error of the model forecast, which is also undesirable.
Figure 4 shows both RMSE of ensemble mean and the ensemble spread for screen-level variables (2-m temperature and relative humidity, 10-m wind speed, and 12-h accumulated precipitation). In terms of RMSE and spread, ESDA performs slightly better than NCSB. Both NCSB and ESDA are clearly underdipersive for all shown screen-level variables.
RMSE (red lines) and spread (blue lines) of the ensemble for the screen-level variables: NCSB (solid) and ESDA (dashed) experiments as a function of lead time. Displayed parameters are (top left) 2-m temperature, (top right) 2-m relative humidity, (bottom left) 10-m wind speed, and (bottom right) 12-hourly precipitation averaged over the verification domain and over the verification period from 15 May to 15 Jul 2011.
Citation: Monthly Weather Review 144, 9; 10.1175/MWR-D-16-0038.1
For the 2-m temperature forecast, ESDA is more skillful than NCSB. It has smaller RMSE error and larger spread. The clear superiority of ESDA can be observed at the initial time. It means that the surface ICs obtained by the ESDA have positively affected forecast skill. The improvement is larger during the nighttime, when the boundary layer is more decoupled from the upper troposphere and the influence on the screen-level variables from the ground is higher.
For the 2-m relative humidity, ESDA is slightly better than NCSB in terms of spread. That might be attributed to the observation perturbation in the ESDA. ESDA has also slightly smaller RMSE in the nighttime in comparison with NCSB; however, it is slightly worse during the day at noon.
The forecast performance for 10-m wind speed and 12-h accumulated precipitation of ESDA and NCSB is very similar. This is likely the consequence of perturbing only surface temperature and surface layer soil temperature, which will have considerable impact on the 2-m temperature and relative humidity, but only minor impact on 10-m wind speed and precipitation.
The percentage of outliers is the number of forecasts, where the observed or measured value is not enclosed, but lies outside (below or above) the range of all the ensemble members. However, it does not measure the amount by which an outlier, or extreme event, exceeds the range. Small values in the percentage of outliers are preferable; otherwise a forecaster relying on the ensemble would likely miss an extreme event. An optimum is reached when the percentage p is around 2/(n + 1), where n is the number of ensemble members [see also Wang et al. (2011)]. In our case of 16 ensemble members, the optimal value is 11.76%. For lower values the system shows too much dispersion and the uncertainty is overestimated. Figure 5 shows the outlier score for the screen-level variables. In both experiments the optimum p is not reached, which is in agreement with the findings of the RMSE and spread evaluation (underdispersion). Nevertheless, an improvement in the ESDA experiment compared to NCSB is evident for both 2-m temperature and 2-m relative humidity. A small improvement for 10-m wind speed can be observed as well, while there is no signal for 12-h precipitation.
Outliers for the screen-level variables: NCSB (solid) and ESDA (dashed) experiments as a function of lead time. Displayed parameters are (top left) 2-m temperature, (top right) 2-m relative humidity, (bottom left) 10-m wind speed, and (bottom right) 12-hourly precipitation averaged over the verification domain and over the verification period from 15 May to 15 Jul 2011. The faint lines of solid and dashed show the 95% and 5% confidence intervals of NCSB and ESDA experiments, respectively.
Citation: Monthly Weather Review 144, 9; 10.1175/MWR-D-16-0038.1
The percentage of outliers changes only slightly with the lead time. The maximum values for 2-m temperature and 10-m wind speed are reached during the second half of the night (+12, +18 and +36, +42 h), when the planetary boundary layer is most stable. The results of 2-m relative humidity show a strong decrease in the percentage of outliers by about 20% in the first hours of integration for both experiments after when it remains constant. The 2-m temperature is directly interpolated from the surface and atmospheric values. In the contrary, the soil moisture perturbations are more indirectly coupled to the 2-m values by evapotranspiration, such that some spinup time can be assumed till the latter adapt to the former. Therefore, at the initial time more underdispersion of the ensemble can be expected than after this adaptation period. In a less obvious extent this is also true for the 10-m wind, which is additionally perturbed, when it adapts to the soil moisture and temperature perturbations via turbulent mixing in the boundary layer.
The statistical significance test was computed for the outliers using the bootstrap method (Wilks 2006). The 95% and 5% confidence intervals for both experiments are shown in Fig. 5. The differences in the scores are statistically significant for 2-m temperature and 2-m relative humidity for most of the forecast hours, but the scores for 10-m wind speed and 12-h accumulated precipitation are statistically not significant.
Probabilistic forecast quality can be well measured by the continuous ranked probability score (CRPS), which evaluates the ensemble forecast distribution against the observations, where both entries are represented by the cumulative distribution functions. The CRPS is given by the mean squared error of the cumulative distribution of an ensemble forecast. It is reduced to the mean absolute error (MAE) in case of a single deterministic forecast. Therefore, this score can be easily interpreted as an error measure and it is also expressed in the same units as the verified variable. A perfect score is given by zero. The CRPS of an ensemble system can even be compared to the MAE of a deterministic forecast. However, here CRPS is used only to assess the probabilistic quality of the surface perturbation methods.
The CRPS scores of 2-m temperature, 2-m relative humidity, 10-m wind speed, and 12-h accumulated precipitation for ESDA and NCSB are compared in Fig. 6. The beneficial effect of perturbing the ICs by the ESDA is most obvious for the 2-m temperature. The error of ESDA is clearly reduced compared to NCSB, especially during the night hours. For 2-m relative humidity the positive impact can be observed as well, with the exception of afternoon hours. This is in accordance with the findings related to the RMSE error and its dependency on the time of the day. Neutral performance can be found for 10-m wind speed, while for 12-h accumulated precipitation the CRPS of ESDA gives even slightly worse results.
CRPS for the screen-level variables: NCSB (solid) and ESDA (dashed) experiments as a function of lead time. Displayed parameters are (top left) 2-m temperature, (top right) 2-m relative humidity, (bottom left) 10-m wind speed, and (bottom right) 12-hourly precipitation averaged over the verification domain and over the verification period from 15 May to 15 Jul 2011. The faint lines of solid and dashed show the 95% and 5% confidence intervals of NCSB and ESDA experiments, respectively.
Citation: Monthly Weather Review 144, 9; 10.1175/MWR-D-16-0038.1
The 95% and 5% confidence intervals for both experiments are shown in Fig. 6. As for the percentage of outliers, the CRPS exhibits statistical significance only for 2-m temperature and 2-m relative humidity. The scores for 10-m wind speed and 12-h accumulated precipitation are not significantly different.
NCSB and ESDA are different methods to generate the surface ICs perturbations. Therefore, it is interesting to evaluate their performance regarding the initial forecast errors. Figure 7 provides a comparison of 2-m temperature statistical scores at the initial time for ESDA and NCSB in terms of RMSE, CRPS, outliers, and spread for the whole verification period of the 62 days. A significant enhancement regarding all monitored scores (except the ensemble spread) can be clearly identified for ESDA for the whole duration of the experiment. The reduction of the initial errors can be explained by the assimilation of surface data, which makes the analysis more accurate. What is more interesting is that a reduction in the initial errors leads to a larger ensemble spread for the longer lead times, while keeping the system error low, as it was already shown earlier. This means that the ESDA has a beneficial effect on both the error growth (ensemble spread) and the system error itself. We hypothesize that the observation perturbation applied in ESDA is more optimal in estimation of IC uncertainties.
Statistical skill scores for the ensemble of 2-m temperature analysis (+0 h) for the whole 62 days of the validation period, averaged over the verification domain: (top left) RMSE, (top right) CRPS, (bottom left) outliers, and (bottom right) spread for NCSB (solid) and ESDA (dashed) experiments.
Citation: Monthly Weather Review 144, 9; 10.1175/MWR-D-16-0038.1
The impact of the two discussed surface ICs perturbation techniques on the upper-air variables has been investigated as well. Figure 8 shows the ensemble spread and RMSE of ensemble mean for temperature and wind speed forecasts of ESDA and NCSB at the 850-hPa level. The overall impact on the upper-air variables is small. No significant differences can be observed; both ESDA and NCSB experiments behave similarly in the free atmosphere. The same conclusion can be made from Figs. 9 and 10, where the outliers and CRPS of temperature and wind speed forecasts of ESDA and NCSB at the 850-hPa level are shown. Figure 10 also shows the 95% and 5% confidence intervals of ESDA and NCSB experiments and reveals that there is no significant difference in the scores at all. The overall neutral impact was observed also for the 500-hPa level (not shown).
RMSE (red lines) and spread (blue lines) of the ensemble forecast for (left) temperature and (right) wind speed at 850 hPa: NCSB (solid) and ESDA (dashed) experiments as function of lead time, averaged over the verification domain and over the verification period from 15 May to 15 Jul 2011.
Citation: Monthly Weather Review 144, 9; 10.1175/MWR-D-16-0038.1
Outliers of the ensemble forecast for (left) temperature and (right) wind speed at 850 hPa: NCSB (solid) and ESDA (dashed) experiments as a function of lead time, averaged over the verification domain and over the verification period from 15 May to 15 Jul 2011.
Citation: Monthly Weather Review 144, 9; 10.1175/MWR-D-16-0038.1
CRPS for (left) temperature and (right) wind speed at 850 hPa: NCSB (solid) and ESDA (dashed) experiments as a function of lead time, averaged over the verification domain and over the verification period from 15 May to 15 Jul 2011. The faint lines of solid and dashed show the 95% and 5% confidence intervals of NCSB and ESDA experiments, respectively.
Citation: Monthly Weather Review 144, 9; 10.1175/MWR-D-16-0038.1
5. Summary and conclusions
The forecasts of screen-level variables in many global and regional EPSs are still unsatisfactory, as their values are often over/underestimated and overconfident. The appropriate surface initial conditions (IC) uncertainties are one of the key factors of reliable ensemble forecasting. Therefore, the treatment of the surface IC perturbation remains a challenge for many global and regional EPSs.
In this paper, the two different surface IC perturbation techniques—the ensemble of surface data assimilations (ESDA) and noncycling surface breeding (NCSB), applicable in a regional EPS—have been described. ESDA and NCSB are implemented in the regional ensemble system ALADIN-LAEF, which is configured with 16 perturbed members. The performance of ESDA and NCSB in ALADIN-LAEF is thoroughly evaluated by standard probabilistic scores over central Europe for a two-month late spring to summer period. The main focus of the evaluation is on the screen-level variables like 2-m temperature, 2-m relative humidity, 10-m wind, and precipitation. Upper-air variables in the lower troposphere (e.g., at 850 hPa) are verified as well.
The NCSB technique uses short-range surface forecasts driven by perturbed global atmospheric forcing and is based on the idea of breeding. Similar methods are used in NCEP GEFS by Deng et al. (2012) for ensemble tropical cyclone forecasts and in MOGREPS by Tennant and Beare (2014) for perturbing initial soil moisture. NCSB describes possibly well the surface IC uncertainties from the fast-growing forecast errors in the atmospheric forcing or interaction between the atmosphere and land surface, but does not include the uncertainties from the observation errors (Toth and Kalnay 1993).
The ESDA technique applies an ensemble of surface data assimilations in which the screen-level observations are randomly perturbed. It uses the same principle as other ensemble of data assimilations (EDAs) widely used, for example, at ECMWF, MSC, and Météo-France. If the observation perturbation is consistent with the observation error statistics in the data assimilation system, ESDA should be better in simulating the analysis uncertainties (e.g., the impact of the observation coverage, etc.). Furthermore, in ESDA the model initial soil and surface fields are directly perturbed through the assimilation procedure. The apparent advantage of such a method is that it allows a high-resolution assimilation cycle, where the first-guess fields are already well balanced with the local model orography and, therefore, contain the important small-scale signal.
The results of this study show that ESDA is superior to NCSB. It has a positive impact on the forecast of screen-level variables, which is statistically more consistent and reliable, while at the same time it has a neutral impact on the upper-air weather variables.
It is found, that the 2-m temperature forecasts of ESDA particularly benefit from ESDA technique. It has smaller RMSE error, larger spread, lower outlier score, and better CRPS than NCSB, which indicates that the 2-m temperature forecasts of ESDA are statistically more reliable than those of NCSB. ESDA provides a clearly more accurate surface analysis and more realistic perturbations at the analysis time.
The better skill of ESDA is likely attributed to the introduction of the surface data assimilation and the observation perturbations. ESDA provides more accurate surface ICs than NCSB, with an estimate of the observation uncertainties, which is not included in NCSB.
A slightly positive impact is also found for the 2-m humidity and 10-m wind forecasts of ESDA, while there is a neutral impact on the forecast skill of the precipitation and the upper-air weather variables in the lower troposphere. These findings should be related to the fact that we perturb and assimilate only the surface temperature and soil moisture content via the increments of the 2-m temperature and relative humidity in ESDA.
The results presented in this study demonstrate that ESDA improves the forecast skill in a regional ensemble, in particular for some screen-level weather variables like 2-m temperature and 2-m relative humidity. It reduces the screen-level forecast error and increases the ensemble spread, which is of central interest in a regional ensemble. It does not deteriorate the statistical scores for the other screen-level variables, nor for the upper-air fields. We conclude that ensemble surface data assimilation is an effective method for perturbing the surface ICs in a regional ensemble system, such as in ALADIN-LAEF.
In our work, there was no observation error taken into account in the probabilistic scores in the verification. One has to know that the inclusion of observation error could have a very large impact on the verification results for variables such as 2-m temperature, 10-m wind, and surface pressure (Bowler and Mylne 2009). Similarly, the system might no longer be underdispersive and the number of outliers will change drastically.
Our results further indicate that the improvement of the precipitation forecast remains a challenge. More work and experiments have to be done in the near future (e.g., introduction of stochastic surface physics in ESDA as well as in the regional ensemble forecast). It would be very interesting to investigate the impact of the stochastic surface physics and how it interacts with ESDA. Some experiments toward this direction are ongoing at ZAMG.
Acknowledgments
We gratefully acknowledge all the RC LACE and ALADIN colleagues who have contributed to the development of ALADIN-LAEF. Special thanks to Gergely Bölöni and Alena Trojakova for the valuable discussions during the ensemble CANARI implementation. ECMWF has provided the computer facilities and technical help running ALADIN-LAEF on the ECMWF HPCF.
REFERENCES
Balsamo, G., F. Bouyssel, and J. Noilhan, 2004: A simplified bi-dimensional variational analysis of soil moisture from screen-level observations in a mesoscale numerical weather-prediction model. Quart. J. Roy. Meteor. Soc., 130, 895–915, doi:10.1256/qj.02.215.
Barthlott, C., and N. Kalthoff, 2011: A numerical sensitivity study on the impact of soil moisture on convection-related parameters and convective precipitation over complex terrain. J. Atmos. Sci., 68, 2971–2987, doi:10.1175/JAS-D-11-027.1.
Belair, S., 2015: Regional environmental prediction systems. Seamless Prediction of the Earth System: From Minutes to Months, WMO-1156, G. Brunet, S. Jones, and P. Ruti, Eds., World Meteorological Organization, 331–346.
Berre, L., O. Pannekoucke, G. Desroziers, S. E. Stefanescu, B. Chapnik, and L. Raynaud, 2007: A variational assimilation ensemble and the spatial filtering of its error covariances: Increase of sample size by local spatial averaging. Proc. ECMWF Workshop on Flow-Dependent Aspects of Data Assimilation, Reading, United Kingdom, ECMWF, 151–168.
Bouttier, F., J.-F. Mahfouf, and J. Noilhan, 1993a: Sequential assimilation of soil moisture from atmospheric low-level parameters. Part I: Sensitivity and calibration studies. J. Appl. Meteor., 32, 1335–1351, doi:10.1175/1520-0450(1993)032<1335:SAOSMF>2.0.CO;2.
Bouttier, F., J.-F. Mahfouf, and J. Noilhan, 1993b: Sequential assimilation of soil moisture from atmospheric low-level parameters. Part II: Implementation in a mesoscale model. J. Appl. Meteor., 32, 1352–1364, doi:10.1175/1520-0450(1993)032<1352:SAOSMF>2.0.CO;2.
Bowler, N. E., and K. R. Mylne, 2009: Ensemble transform Kalman filter perturbation for a regional ensemble prediction system. Quart. J. Roy. Meteor. Soc., 135, 757–766, doi:10.1002/qj.404.
Bowler, N. E., A. Arribas, K. R. Mylne, K. B. Robertson, and S. E. Beare, 2008: The MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 134, 703–722, doi:10.1002/qj.234.
Buizza, R., and T. N. Palmer, 1995: The singular vector structure of the atmospheric general circulation. J. Atmos. Sci., 52, 1434–1456, doi:10.1175/1520-0469(1995)052<1434:TSVSOT>2.0.CO;2.
Buizza, R., P. L. Houtekamer, Z. Toth, G. Pellerin, M. Wei, and Y. Zhu, 2005: A comparison of the ECMWF, MSC, and NCEP Global Ensemble Prediction Systems. Mon. Wea. Rev., 133, 1076–1097, doi:10.1175/MWR2905.1.
Buizza, R., M. Leutbecher, and L. Isaksen, 2008: Potential use of an ensemble of analyses in the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc., 134, 2051–2066, doi:10.1002/qj.346.
Burns, P. J., and D. V. Pryor, 1998: Surface radiative transport at large scale via Monte Carlo. Ann. Rev. Heat Transfer, 9, 79–158, doi:10.1615/AnnualRevHeatTransfer.v9.40.
Charron, M., G. Pellerin, L. Spacek, P. L. Houtekamer, N. Gagnon, H. L. Mitchell, and L. Michelin, 2010: Toward random sampling of model error in the Canadian ensemble prediction system. Mon. Wea. Rev., 138, 1877–1901, doi:10.1175/2009MWR3187.1.
Clark, A. J., W. A. Gallus Jr., M. Xue, and F. Kong, 2009: A comparison of precipitation forecast skill between small convection-allowing and large convection-parameterizing ensembles. Wea. Forecasting, 24, 1121–1140, doi:10.1175/2009WAF2222222.1.
Deng, Z., Y. Tang, D. Chen, and G. Wang, 2012: A hybrid ensemble generation method for EnKF for Argo data assimilation. Atmos.–Ocean, 50, 129–145, doi:10.1080/07055900.2012.719823.
Déqué, M., C. Dreveton, A. Braun, and D. Cariolle, 1994: The ARPEGE/IFS atmosphere model: A contribution to the French community climate modelling. Climate Dyn., 10, 249–266, doi:10.1007/BF00208992.
Du, H., F. J. Doblas-Reyes, J. García-Serrano, V. Guemas, Y. Soufflet, and B. Wouters, 2012: Sensitivity of decadal predictions to the initial atmospheric and oceanic perturbations. Climate Dyn., 39, 2013–2023, doi:10.1007/s00382-011-1285-9.
Du, J., and M. S. Tracton, 2001: Implementation of a real-time short-range ensemble forecasting system at NCEP: An update. Preprints, Ninth Conf. on Mesoscale Processes, Ft. Lauderdale, FL, Amer. Meteor. Soc., 355–356.
Frankignoul, C., 1985: Sea surface temperature anomalies, planetary waves, and air-sea feedback in the middle latitudes. Rev. Geophys., 23, 357–390, doi:10.1029/RG023i004p00357.
Giard, D., and E. Bazile, 2000: Implementation of a new assimilation scheme for soil and surface variables in a global NWP model. Mon. Wea. Rev., 128, 997–1015, doi:10.1175/1520-0493(2000)128<0997:IOANAS>2.0.CO;2.
Guo, Z., and Coauthors, 2006: GLACE: The Global Land–Atmosphere Coupling Experiment. Part II: Analysis. J. Hydrometeor., 7, 611–625, doi:10.1175/JHM511.1.
Hacker, J. P., and Coauthors, 2011: The U.S. Air Force Weather Agency’s mesoscale ensemble: Scientific description and performance results. Tellus, 63A, 625–641, doi:10.1111/j.1600-0870.2010.00497.x.
Hamill, T. M., and S. J. Colucci, 1997: Verification of Eta–RSM short-range ensemble forecasts. Mon. Wea. Rev., 125, 1312–1327, doi:10.1175/1520-0493(1997)125<1312:VOERSR>2.0.CO;2.
Hamill, T. M., and R. Swinbank, 2015: Stochastic forcing, ensemble prediction systems, and TIGGE. Seamless Prediction of the Earth System: From Minutes to Months, WMO-1156, G. Brunet, S. Jones, and P. Ruti, Eds., World Meteorological Organization, 187–212.
Horányi, A., M. Mile, and M. Szücs, 2011: Latest developments around the ALADIN operational short-range ensemble prediction system in Hungary. Tellus, 63A, 642–651, doi:10.1111/j.1600-0870.2011.00518.x.
Houtekamer, P. L., and H. L. Mitchell, 2005: Ensemble Kalman filtering. Quart. J. Roy. Meteor. Soc., 131, 3269–3289, doi:10.1256/qj.05.135.
Houtekamer, P. L., M. Charron, H. Mitchell, and G. Pellerin, 2007: Status of the global EPS at Environment Canada. Proc. ECMWF Workshop on Ensemble Prediction, Reading, United Kingdom, ECMWF, 57–68.
Isaksen, L., M. Fisher, and J. Berner, 2007: Use of analysis ensembles in estimating flow-dependent background error variance. Proc. ECMWF Workshop on Flow-Dependent Aspects of Data Assimilation, Reading, United Kingdom, ECMWF, 65–86.
Koster, R. D., and Coauthors, 2006: GLACE: The Global Land–Atmosphere Coupling Experiment. Part I: Overview. J. Hydrometeor., 7, 590–610, doi:10.1175/JHM510.1.
Lavaysse, C., M. Carrera, S. Bélair, N. Gagnon, R. Frenette, M. Charron, and M. Yau, 2013: Impact of surface parameter uncertainties within the Canadian Regional Ensemble Prediction System. Mon. Wea. Rev., 141, 1506–1526, doi:10.1175/MWR-D-11-00354.1.
Leutbecher, M., and T. N. Palmer, 2008: Ensemble forecasting. J. Comput. Phys., 227, 3515–3539, doi:10.1016/j.jcp.2007.02.014.
Li, X., M. Charron, L. Spacek, and G. Candille, 2008: A regional ensemble prediction system based on moist targeted singular vectors and stochastic parameter perturbation. Mon. Wea. Rev., 136, 443–462, doi:10.1175/2007MWR2109.1.
Mahfouf, J.-F., 1991: Analysis of soil moisture from near surface parameters: A feasibility study. J. Appl. Meteor., 30, 1534–1547, doi:10.1175/1520-0450(1991)030<1534:AOSMFN>2.0.CO;2.
Mahfouf, J.-F., K. Bergaoui, C. Draper, F. Bouyssel, F. Taillefer, and L. Taseva, 2009: A comparison of two off-line soil analysis schemes for assimilation of screen level observations. J. Geophys. Res., 114, D08105, doi:10.1029/2008JD011077.
Noilhan, J., and S. Planton, 1989: A simple parameterization of land surface processes for meteorological models. Mon. Wea. Rev., 117, 536–549, doi:10.1175/1520-0493(1989)117<0536:ASPOLS>2.0.CO;2.
Petersen, W. P., 1994: Lagged Fibonacci series random number generator for the NEC SX-3. Int. J. High Speed Comput., 06, 387, doi:10.1142/S0129053394000202.
Romine, G. S., C. S. Schwartz, J. Berner, K. R. Fossell, C. Snyder, J. L. Anderson, and M. L. Weisman, 2014: Representing forecast error in a convection-permitting ensemble system. Mon. Wea. Rev., 142, 4519–4541, doi:10.1175/MWR-D-14-00100.1.
Schneider, S., Y. Wang, W. Wagner, and J.-F. Mahfouf, 2014: Impact of ASCAT soil moisture assimilation on regional precipitation forecasts: A case study for Austria. Mon. Wea. Rev., 142, 1525–1540, doi:10.1175/MWR-D-12-00311.1.
Schwartz, C. S., G. S. Romine, K. R. Smith, and M. L. Weisman, 2014: Characterizing and optimizing precipitation forecasts from a convection-permitting ensemble initialized by a mesoscale ensemble Kalman filter. Wea. Forecasting, 29, 1295–1318, doi:10.1175/WAF-D-13-00145.1.
Storto, A., and R. Randriamampianina, 2008: Use of satellite data in ALADIN-HARMONIE/Norway. Proc. 16th Int. TOVS Study Conf. (ITSC-16), Angra dos Reis, Brazil, PETROBRAS, 11 pp. [Available online at https://cimss.ssec.wisc.edu/itwg/itsc/itsc16/proceedings/6_13_Randriamampianina_paper.pdf.]
Sutton, C., T. M. Hamill, and T. T. Warner, 2006: Will perturbing soil moisture improve warm-season ensemble forecasts? A proof of concept. Mon. Wea. Rev., 134, 3174–3189, doi:10.1175/MWR3248.1.
Tennant, W., and S. Beare, 2014: New schemes to perturb sea-surface temperature and soil moisture content in MOGREPS. Quart. J. Roy. Meteor. Soc., 140, 1150–1160, doi:10.1002/qj.2202.
Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 2317–2330, doi:10.1175/1520-0477(1993)074<2317:EFANTG>2.0.CO;2.
Wang, Y., T. Haiden, and A. Kann, 2006: The operational Limited Area Modelling system at ZAMG: Aladin-Austria. Österreich. Beitr. Meteor. Geophys., 37, 33 pp.
Wang, Y., A. Kann, M. Bellus, J. Pailleux, and C. Wittmann, 2010: A strategy for perturbing surface initial conditions in LAMEPS. Atmos. Sci. Lett., 11, 108–113, doi:10.1002/asl.260.
Wang, Y., and Coauthors, 2011: The Central European limited-area ensemble forecasting system: ALADIN-LAEF. Quart. J. Roy. Meteor. Soc., 137, 483–502, doi:10.1002/qj.751.
Wang, Y., M. Bellus, J.-F. Geleyn, X. Ma, W. Tian, and F. Weidle, 2014: A new method for generating initial condition perturbations in a regional ensemble prediction system: Blending. Mon. Wea. Rev., 142, 2043–2059, doi:10.1175/MWR-D-12-00354.1.
Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Academic Press, 627 pp.
