1. Introduction
Due to the chaotic nature of atmospheric dynamics, numerical weather prediction (NWP) systems are sensitive to initial and model errors. These kinds of small errors will grow with forecast lead times and finally render complete loss of forecast skill (Lorenz 1963). Ensemble forecasts are designed to address the initial and model uncertainties existing in deterministic forecasts. By integrating multiple initial conditions or adopting multiple models, ensemble forecasts can estimate the uncertainty of forecast states in the form of probability distribution (Leith 1974; Steven Tracton and Kalnay 1993; Palmer 1999; Saito et al. 2012).
Compared with a good description of model uncertainties, an accurate representation of initial uncertainties is much more important for short-range ensemble forecasts (Raynaud and Bouttier 2016). Initial uncertainties are described by initial perturbation methods. Two characteristics are desirable for the initial perturbation representation. One characteristic is that the initial perturbations are able to sample uncertainties in the analysis field, which ensures that the perturbed initial conditions can represent the distribution of true states of atmosphere (Ehrendorfer 1994). The other characteristic is that the initial perturbation can properly describe the evolution of initial errors (the predictability of the atmosphere) with time. This characteristic helps ensemble forecast members capture the fastest error growing modes that are compatible with the uncertainties (Palmer 1993; Magnusson et al. 2008). Therefore, the reasonable initial perturbation structures and growth characteristics are essential for the initial perturbation representation of ensemble forecast systems (EFS) (Molteni et al. 1996).
NWP centers have developed kinds of initial perturbation methods for representing initial state uncertainties. These methods can be classified into three categories according to their perturbation properties. The first kind contains the breeding vector (BV) method of the National Centers for Environmental Prediction (NCEP) and the singular vector (SV) technique of the European Centre for Medium-Range Weather Forecasts (ECMWF). The BV utilizes dynamically cycled forecasts to generate perturbations representing atmospheric dynamic structures (Toth and Kalnay 1993, 1997), while the singular vector method uses a mathematical calculation to achieve the maximum perturbation growth rate over a given optimization time (Lorenz 1965; Palmer 1993; Buizza and Palmer 1995; Buizza et al. 2005; Feng et al. 2014; Duan and Huo 2016). The first kind of method can capture the fastest error growth mode in phase space. However, they ignore the analysis error uncertainties modulated by assimilating observation data. The second kind, such as the perturbed observations (PO) method of the Meteorological Service of Canada (MSC), is used to represent the uncertainties of observation data. In the PO method, a new analysis field is obtained by adding random perturbations to the observations and rerunning the assimilation system (Houtekamer et al. 1996). The advantages of the PO method over SV and BV were shown by Talagrand et al. (2007). However, the PO method is time consuming due to its dependence on the data assimilation system (Magnusson et al. 2009), and it contains both fast growing and neutral/decaying perturbation patterns (Wei et al. 2008). The last kind of initial perturbation methods cannot only sample fast growing error directions, but also take the observation uncertainties into account. The ensemble transform (ET) technique, the ET with rescaling (ETR) and ET Kalman filter (ETKF) methods are included. ET and ETR are improved versions of the BV method and new NCEP techniques. ETKF has already been operational used at the Met Office (UKMO), Korea Meteorological Administration (KMA), and China Meteorological Administration (CMA). The ET and ETR techniques transform forecast perturbations to be orthogonal with respect to the analysis error variance (Bishop and Toth 1999; Wei et al. 2005, 2008), and the ETKF generates perturbations orthonormal in the observational space (Wang and Bishop 2003; Wang et al. 2004). The difference between ET(R) and ETKF lies in the fields that are used to restrain the initial perturbations. In the ET method, analysis error variances from the best data assimilation (DA) system are employed to restrain the initial perturbations. In the ETKF, short time forecasts from the last cycle and observations are used to restrain the initial perturbations instead (Wei et al. 2008).
The ETKF initial perturbation method was first introduced for adaptive observations by Bishop et al. (2001) and then used for initial perturbation generation. Wang and Bishop (2003) first applied the ETKF method to construct initial perturbations in an idealized observation framework and compared it with a masked BV method. The results showed that the ETKF and masked BV ensemble variances at the analysis time have reasonable correspondences between the variance and observational density. The ETKF maintains comparable amounts of variance in all orthogonal and uncorrelated directions spanning its ensemble perturbation subspace, while BV techniques maintain variance in a few directions. Wei et al. (2006) further extended ETKF to an operational environment with real observations. Bowler et al. (2008) and Kay and Kim (2014) studied the characteristics of the ETKF initial perturbation method and illustrated the good agreement between its initial perturbation horizontal structure and dynamical unstable areas.
Since 2008, China Meteorological Administration (CMA) has developed the GRAPES-REPS model based on the ETKF initial perturbation method (Zhang et al. 2015). Numerous studies on ETKF have been made using this system in recent decades. Wang (2008) compared two kinds of initial perturbation schemes including BGM and ETKF using the GRAPES-REPS model. The results indicated that ETKF performs better in terms of rainfall area, although both methods possess certain probabilistic forecast skills. Long et al. (2011) studied the basic properties of ETKF initial perturbation method in GRAPES-REPS. The results showed that ETKF initial perturbation distribution are reasonable, which could reflect the spatial distribution of observation stations. Wang et al. (2018b) revealed the characteristics of the initial perturbation structures and perturbation growth rates in GRAPES-REPS by evaluating some verification metrics (e.g., initial perturbation components, variance accuracy, kinetic energy spectrum, perturbation energy evolution, ensemble spread, and root-mean-square error). The results showed that the ETKF initial perturbations can maintain even distribution to the orthogonal and uncorrelated ensemble directions and the perturbation field is mainly large scale with a flow-dependent structure. In addition, the total energy of initial perturbation and ensemble spread could keep appropriate growth rates at all forecast lead times. Since 2014, the GRAPES-REPS system based on the ETKF ensemble initialization scheme has been transitioned into operational run (Zhang et al. 2014).
The ETKF initial perturbation method has matured in the GRAPES-REPS model and has already generated a fast error growing mode. However, the different model grids apply the same inflation factor in the ETKF initial perturbation formulation, so the inflated ETKF initial perturbation magnitude may be locally larger than the analysis error magnitude. This means that the ETKF is unable to form an initial perturbation distribution similar to the analysis error. In addition, Wang and Bishop (2003) showed that ETKF ensemble with a small number of members is not large enough to reliably resolve even large-scale geographical fluctuations in observational density; thus, some sorts of masking technique would have to be applied to ETKF perturbation to reasonably represent the effect of observational density fluctuations on the forecast error variance. So ETKF-generated initial perturbation should not only generate a fast error growing mode, but also be representative of the analysis uncertainties and responsive to observation distributions. Therefore, the energy of the initial perturbation needs to be redistributed. A regional rescaling mask should be imposed to the ETKF method to suppress high-amplitude perturbations in areas where the analysis uncertainties are relatively low. A regional rescaling mask can ensure that the perturbation magnitude is restrained by the analysis uncertainties, and then, a sufficient ensemble dispersion can be obtained (Toth and Kalnay 1997; Wang and Bishop 2003). Several studies (Wei et al. 2008; Ma et al. 2014) defined the rescaling factor as the ratio of the mask to the square root of the analysis perturbation kinetic energy norm, as in the NCEP ETR method. Following the setup of the masked BV method, the mask is computed using the square root of the long-term averaged analysis error variance in the kinetic energy norm at the 500-hPa pressure level obtained from the variational DA system (Szunyogh and Toth 2002; Wei et al. 2008). Magnusson et al. (2009) defined the rescaling factor as the ratio of the square root of the vertically integrated analysis error variance measured in the total energy to the square root of the vertically integrated analysis perturbation total energy. The two-dimensional mask was unable to represent the vertical structure of the analysis uncertainties, so Feng et al. (2019) constructed the ensemble transform with 3D rescaling (ET_3DR) scheme. The total energy norm derived from the ensemble analysis from the NCEP hybrid 3DVar/EnKF system (Wang et al. 2013) is used in the mask calculation. The results showed that ET_3DR could represent the analysis uncertainties well with a faster ensemble spread growth and a better forecast skill than ETR.
To address the issue that an initial perturbation distribution similar to the analysis error cannot be generated in the ETKF method of the GRAPES-REPS, a regional rescaling mask is introduced into the ETKF method following the rescaling factor of ETR and masked BV method (we call it ETKF_R). We hope that the method can effectively describe the initial perturbation structures and growing characteristics. To demonstrate how rescaling can affect the initial perturbation characteristics as well as the ensemble forecast skills, we also compare the simulation results between ETKF and ETKF_R methods. The GRAPES-REPS is used in the comparison. The model introduction and initial perturbation methods are described in section 2. In section 3 we compare the structures and growth characteristics of the initial ensemble perturbations. The qualities of ensemble forecasts (including ensemble spread and probabilistic skill) are investigated in section 4, and finally, we summarize the results in section 5.
2. Model and methodology
a. GRAPES-REPS regional ensemble prediction system
GRAPES-REPS employed in this study is a regional version of the Global and Regional Assimilation and Prediction Enhanced System (GRAPES). The system was developed at the NWP Center of the CMA (Chen et al. 2008; Wang et al. 2018a; Chen et al. 2020). The GRAPES-REPS has 15 ensemble members (1 control and 14 perturbed members) with a horizontal resolution of 10 km and 51 vertical levels (the model top is 10 hPa). The initial and boundary conditions of the control forecast are provided by the NCEP global forecast system. The initial perturbation of the GRAPES-REPS is generated from the ETKF method (Wang et al. 2018b). Model perturbation is considered by applying stochastically perturbed parameterization tendency schemes (SPPT) (Buizza et al. 1999; Yuan et al. 2016) and single physics schemes. The model physics includes Rapid Radiative Transfer Model (RRTM) longwave radiation (Mlawer et al. 1997), Dudhia shortwave radiation (Dudhia 1989), the Weather Research and Forecasting single-moment 6-class (WSM-6) microphysics (Hong and Lim 2006), Noah land surface model (Mahrt and Ek 1984), Medium-Range Forecast (MRF) planetary boundary layer (PBL) scheme (Hong and Pan 1996), Monin–Obukhov surface layer scheme (Noilhan and Planton 1989), and Kain–Fritsch (new eta) convective parameterization scheme (Kain 2004). Moreover, the boundary uncertainty is derived from the GRAPES Global Ensemble Prediction System (GRAPES-GEPS). We retrieve the perturbation fields of perturbed members from the control forecast of the GRAPES-GEPS and add the perturbations to the control forecast boundary of the GRAPES-REPS to form the boundary conditions of different members. The uncertainty of the typhoon vortex center location is described by conditional typhoon vortex relocation (CTVR) (Kurihara et al. 1995; Wu et al. 2020). A cloud analysis scheme is introduced to obtain more accurate cloud initial fields, and the scheme applies a nudging technique to assimilate cloud hydrometeors (Zhu et al. 2007). GRAPES-REPS runs twice a day, initialized at 0000 and 1200 UTC, respectively, out to a 72-h forecast lead time. The model integration step is 60 s.
b. ETKF formulation
In formula (8),
In addition, the ETKF initial perturbation method applied in the GRAPES-REPS employs a 12-h ensemble perturbation cycle scheme. That is, the 12-h forecast perturbation of last cycle is regenerated to the analysis perturbation at the present time. The calculation variables of covariance matrix
c. ETKF with rescaling method
The accurate estimation of analysis error is very important for constructing the ETKF_R initial perturbation method. The analysis error could either be provided by the DA system (Fisher and Courtier 1995) or derive from different center analysis data (Buizza et al. 2005; Swanson and Roebber 2008; Langland et al. 2008). In addition, some studies (Pena and Toth 2014; Feng et al. 2017, 2020) have proposed a new statistical algorithm to provide an independent and unbiased estimation of analysis error variance. The new method combines information from differences between forecast and analysis fields with prior knowledge regarding the time evolution of 1) forecast error variance and 2) correlation between errors in analyses and forecasts. In our study, we crudely apply the difference between the ECMWF analysis (AEC) with a horizontal resolution of 0.125° and the GRAPES regional model analysis of China (ACHN) with a horizontal resolution of 0.1° as the analysis error. We also use the spread of the multicenter analysis data (including GRAPES, ECMWF, NCEP and Japan analysis) to estimate the analysis uncertainty (Wei et al. 2010). The distribution of analysis uncertainty calculated from multiple centers is very similar to that derived from two centers. The analysis uncertainties in the sparse observation regions are high, but the analysis uncertainties in the concentrated observation coverages are relatively low. For simplicity, we only apply the difference between two high-resolution analyses as the analysis error. The vertical levels of the above two analysis fields range from 1000 to 10 hPa (30 pressure levels in total). Note that as time goes by and analysis data from more centers become available as well as the resolution of the analysis data becomes higher, the estimates of the analysis error derived from multicenter analysis data are expected to be more accurate and rational.
To construct the ETKF with rescaling method, we first use the bilinear interpolation method to interpolate the ECMWF analysis field to the horizontal grid of the GRAPES analysis. Then, we apply the cubic spline interpolation method to carry out the vertical interpolation from the pressure level to the GRAPES-REPS model level (51 levels in total). Figure 1 gives the vertical distribution of the GRAPES analysis and horizontal interpolated ECMWF analysis fields at two respective points before and after the vertical interpolations. By comparing the variable distributions between pressure and model layers, we can validate the correctness of vertical interpolations. Furthermore, Fig. 1 shows that the wind vertical distributions of the ECMWF and GRAPES analysis fields resemble with the maximum values distributed in the same vertical layer.
Meridional and zonal wind vertical distributions of the GRAPES (CHN) and ECMWF (EC) analysis fields at 0000 UTC 1 May 2019. (a),(b) At the respective point of the flat region (110°E, 35°N), and (c),(d) at the respective point of the high-altitude region (90°E, 30°N). (left) The vertical distributions of wind fields at pressure levels, and (right) the vertical distributions of wind fields at model levels.
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
Second, we calculate the meridional and zonal wind differences between the ECMWF and GRAPES analysis fields at model levels, respectively, which is expressed as DIFF = |AEC − ACHN|. Then, we compute the kinetic energy EDIFF of the wind differences. By averaging the historical data, the averaged kinetic energy
The horizontal distributions of analysis uncertainties (MASK) in May 2019 at different model levels. (a) 13th model level (approximately 850 hPa), (b) 19th model level (approximately 700 hPa), and (c) 24th model level (approximately 500 hPa).
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
If the ratio is larger than one, the rescaling factor will be set to 1.0. We apply a nine-point smoothing scheme to smooth the rescaling factor. Compared with the rescaling factor in the sparse observation regions, the rescaling factor is lower in East China, where there are traditionally more observations (figure not shown). We multiply each analysis perturbation derived from the ETKF by the rescaling factor. By applying a regional rescaling mask, the initial perturbation magnitude should vary in accordance with the analysis uncertainties.
Notably, the regional rescaling mask in the ETKF_R method is similar to that in the masked BV method. However, there are a few differences between these two methods. In the operational masked BV method, the mask is calculated by 2D analysis uncertainty measured by the kinetic energy norm obtained from multimember ensemble analysis or a DA system. However, in the ETKF_R method, a new mask is defined by 3D analysis uncertainty measured by the kinetic energy norm obtained from different analysis data of multiple NWP centers.
d. Experimental design
In this study, we introduce a regional rescaling mask into the ETKF method of the GRAPES-REPS and compare it with the operational ETKF method. The structures and growth rates of the initial perturbation, as well as the ensemble forecast verification metrics (including ensemble spread and probabilistic distribution), are evaluated in the comparison. The experiment is performed from 4 to 15 May 2019 for forecasts initialized at both 0000 and 1200 UTC (a total of 24 forecasts) over China (15°–55°N, 70°–140°E). As the ETKF method adopts the adaptive regulation processes of the inflation factor, the verification period is only from 7 to 15 May 2019. The inflation factor is stable during that time. In addition, the calculation of the mask mentioned above could use the monthly averaged MASK value in the same month during historical years. However, for simplicity and convenience, we only apply the monthly averaged MASK value in May 2019 as a mask for our experiments. Then, these verification results are averaged to obtain the average value over a time period of 9 days, which will be presented in the next section. The GRAPES 10-km gridded analysis and the precipitation data at approximately 2400 observation sites are used as truths for verifying the isobaric pressure elements and precipitation, respectively. The precipitation observations are provided by the National Meteorological Information Center of CMA.
3. Characteristics of initial ensemble perturbation
Reasonable characteristics of the initial ensemble perturbation (including initial perturbation structures and growth rates) are essential for the initial perturbation representation of EFS (Molteni et al. 1996). We compare the initial perturbation characteristics between the ETKF and ETKF_R methods. The initial perturbation horizontal distribution, kinetic energy spectrum, perturbation energy evolution and perturbation versus error correlation analysis (PECA) are considered in the comparison.
a. The structures of the initial ensemble perturbation
Figure 3 shows the horizontal distributions of the meridional and zonal wind initial perturbations of the ETKF and ETKF_R methods. In ETKF method, the meridional and zonal wind perturbations are dominated by a few local maxima (exceeding 6 m s−1), which may be caused by the same inflation factor that is applied to different model grids. The initial perturbation magnitude is obviously restrained after applying the ETKF_R method. Compared with the ETKF initial perturbation method, the initial perturbation distribution of the ETKF_R method is more similar to the analysis uncertainty (Fig. 2), indicating that the ETKF_R initial perturbation method is more representative of the analysis uncertainties. And the ETKF_R method could better reflect observation distributions with large initial perturbation magnitudes distributed in sparse observation regions. Therefore, we can conclude that the initial perturbation magnitude structures become reasonable after applying the regional rescaling procedure.
The horizontal distributions of the initial perturbations at 0000 UTC 13 May 2019 of the (a1),(b1) ETKF and (a2),(b2) ETKF_R initial perturbation methods. (top) The zonal wind U at the 19th model level of ensemble member 1 (m s−1), and (bottom) the meridional wind V at the 24th model level of ensemble member 10 (m s−1).
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
b. The growth rates of the initial ensemble perturbation
The kinetic energy spectrum analysis can effectively describe the dimension information measured in the NWP system (Wang et al. 2018b). We apply a two-dimensional discrete cosine transform (2D-DCT) (Denis et al. 2002) to implement the spectrum decomposition of the perturbation fields. Figure 4 shows the distributions of the perturbation kinetic energy spectrum at the 500-, 700-, and 850-hPa levels for the ETKF and ETKF_R methods. The peak values of the perturbation energy spectrum are concentrated at large-scale wavelengths (covered with 2000–5000 km) for both the ETKF and ETKF_R methods. The perturbation energy spectrum tends to increase with forecast lead times. The growth characteristics of the initial ensemble perturbation indicate that the ETKF initial perturbation method can capture the forecast uncertainties well. The variations of the energy spectrum with lead times at other layers and wavelengths also follow these characteristics. Due to the perturbation restraint of the rescaling factor, the perturbation spectrum energy of the ETKF_R method is lower than that of the ETKF method at the initial forecast periods (Figs. 4a1–c1). The differences between these two methods are statistically significant at the 50% (the maximum is 98%) level based on a t test at meso-α and large-scale wavelengths (250–10 000 km) and low to middle vertical levels. However, the differences between the two methods will become very small with increasing forecast lead times. To estimate the growth rates of the energy spectrum, the Lyapunov exponential growth rates of the perturbations in a chaotic system (Palmer 1993; Magnusson et al. 2008; Kay and Kim 2014) are calculated (formula not shown), and we can see that the energy spectrum growth rates of ETKF_R are larger than those of the ETKF method. We can conclude that the rescaling procedure makes the initial perturbation kinetic energy spectrum lower at the low to middle levels and large-scale wavelengths. However, the evolution of the energy spectrum with forecast lead times is faster for the ETKF_R than for the ETKF initial perturbation method. This finding indicates that ETKF_R could capture a faster error growing mode than the ETKF method.
The perturbation kinetic energy spectrum distributions (K2) and the growth rates of the perturbation kinetic energy spectrum (h−1) at (top) 500, (middle) 700, and (bottom) 850 hPa averaged from 0000 UTC 7 May 2019 to 1200 UTC 15 May 2019 (twice a day, 18 times in total). (a1),(b1),(c1) 12-; (a2),(b2),(c2) 36-; and (a3),(b3),(c3) 60-h forecast lead time. Shown are the ETKF method (blue) and the ETKF with rescaling method (red).
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
Figure 5 shows the vertical profiles of ensemble mean perturbation kinetic energy and perturbation total energy at 12- and 72-h forecast lead times for the ETKF and ETKF_R methods. We conclude that some characteristics are consistent with the results of Wang et al. (2018b) for both the ETKF and ETKF_R methods. For example, the perturbation energy at different vertical levels increases with forecast lead times, the maximum perturbation total energy is located at 300 hPa near the axis of the jet stream, the second largest perturbation total energy is distributed near the surface, and the internal perturbation energy and kinetic perturbation energy are the prevailing perturbation energy at low and high levels, respectively. Furthermore, by comparing the ETKF and ETKF_R methods, we can see that the rescaling effect makes the perturbation energy lower at the initial forecast lead times. However, with the increasing of forecast lead times, the perturbation energies between both initial perturbation methods are almost the same. It could be well explained by Descamps and Talagrand (2007), which shows that the model perturbations are more important than the initial perturbations at the later forecast lead times and the effects of the initial perturbation method on the perturbation energy are mainly reflected at the initial lead times. Compared to the ETKF method, the lower initial perturbation energy of ETKF_R method leads to higher perturbation growth rates. The evolution of the perturbation energy is similar to that of the kinetic energy spectrum mentioned above, with higher perturbation energy growth rates of ETKF_R than those of the ETKF method. From the above results, we conclude that the analysis error restraint in the ETKF method could reduce the initial perturbation energy (a representation of ensemble spread) and increase the perturbation growth rates. It means that the ETKF_R method could describe a faster error growing mode than the ETKF method.
The vertical distributions of ensemble mean perturbation energy (J kg−1) at (a) 12- and (b) 72-h forecast lead times and perturbation energy growth rates (h−1) averaged from 0000 UTC 7 May 2019 to 1200 UTC 15 May 2019 (twice a day, 18 times in total). Shown are the perturbation kinetic energy (KE; dotted line), and the sum of perturbation kinetic energy and internal energy, referred to as perturbation total energy (TE; solid line). The ETKF method is shown in blue and the ETKF with rescaling method in red.
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
Figure 6 shows the distributions of ensemble mean 500-hPa perturbation total energy at 12- and 72-h forecast lead times and the geopotential height of GRAPES-REPS control forecast at the corresponding times. It indicates that the perturbation energy is large over the dynamic unstable areas, such as trough and ridge areas, while the perturbation energy in flat areas is relatively small. This result implies that the perturbation total energy distribution has a flow-dependent structure for both the ETKF and ETKF_R methods. Moreover, due to the effects of the regional rescaling, the ETKF_R method could lower the perturbation total energy at the initial forecast lead times. However, the ensemble spread converges at the final forecast lead times.
The horizontal distributions of ensemble mean 500-hPa perturbation total energy (J kg−1) at (a),(c) 12- and (b),(d) 72-h forecast lead times starting from 0000 UTC 8 May 2019. (left) ETKF method and (right) ETKF with rescaling method. Black lines are contours of the 500-hPa geopotential height of the control forecast at the corresponding times.
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
The PECA (Wei and Toth 2003) could measure the relationship between ensemble forecast perturbations and forecast errors. A higher PECA value means that ensemble perturbations could better explain the forecast errors. The absolute values of the PECA at different lead times are shown in Figs. 7a1–d2. For each variable, the ETKF_R method has higher averaged PECA values than the ETKF method for short lead times (12–24-h forecast lead times). For example, the PECA values of the ETKF and ETKF_R perturbation methods are 0.177 and 0.198 at 12-h forecast lead time for 1000-hPa meridional wind, respectively. It means that the ETKF_R perturbations can explain approximately 19.8% of the forecast errors, which is an improvement of 2.1% relative to the ETKF method. We apply the Student’s t test (Wilks 2006) to test the significance of the PECA differences between the ETKF_R and ETKF methods. The differences between the two methods are statistically significant with a confidence limit of 90% for all variables until 24-h forecast lead time. However, the differences are not significant (not shown) for longer lead times. Therefore, the ETKF_R initial perturbations have advantages over the ETKF methods in explaining the forecast errors at short lead times.
The absolute values of PECA averaged from 0000 UTC 7 May 2019 to 1200 UTC 15 May 2019 (twice a day, 18 times in total). (a1),(a2) Meridional wind V; (b1),(b2) zonal wind U; (c1),(c2) temperature T; and (d1),(d2) geopotential height H. (left) 12- and (right) 24-h forecast lead time. The ETKF method is shown in blue and the ETKF with rescaling method in red.
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
4. Ensemble forecast verification
The verification metrics for ensemble forecast quality (including ensemble spread and the probabilistic distribution) were selected referred to Du (2007) and Du and Zhou (2017). Spread–skill relationships and rank histograms are used for verifying ensemble spread. The continuous ranked probability score (CRPS), reliability and relative operating characteristics (ROC) curve are applied to verify the probability forecasts for variables at pressure levels (Wang et al. 2018a). The probability forecasts for precipitation are measured by Brier and AROC (area under the ROC) scores.
a. Ensemble spread
The similarity between the ensemble mean root-mean-square error (RMSE) and ensemble spread is a desired feature. Figures 8a1–c2 compares the spatial distributions of RMSE and spread of 500-hPa zonal wind at 12-h forecast lead time between the ETKF and ETKF_R methods. The RMSE is obviously larger than the ensemble spread over all of China areas for GRAPES-REPS, indicating underdispersion over the domain. The maximum RMSE is 4.9, while the maximum spread is only 3.5 in the ETKF method. The evolution of domain-averaged RMSEs and spreads with forecast hours (not shown) also reveals that the GRAPES-REPS is severely underdispersive at all forecast lead times for both the ETKF and ETKF_R methods. Moreover, by applying the regional rescaling procedure, the ensemble spread distribution at initial forecast lead times has changed considerably, but the RMSE is fundamentally unchanged. The ETKF_R method has a lower domain-averaged spread at the initial forecast hour and a higher ensemble spread growth rate compared with the ETKF method. The evolution of the domain-averaged spread could explain the evolution of above perturbation kinetic energy spectrum and perturbation energy. In addition, the horizontal distributions of RMSE and spread are nearly the same between these two initial perturbation methods at later forecast lead times (not shown).
The horizontal distributions of (a1),(a2) the ensemble mean forecast RMSE; (b1),(b2) the ensemble spread; and (c1),(c2) the consistency (defined as the ratio of the RMSE to the spread) at 12-h forecast lead time. The results are averaged from 0000 UTC 7 May 2019 to 1200 UTC 15 May 2019 (twice a day, 18 times in total). The variable is the 500-hPa zonal wind. (top) ETKF method and (bottom) ETKF with rescaling method.
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
To quantitatively compare the RMSE and spread grid point by grid point, the consistency (defined as the ratio of RMSE to ensemble spread) is shown in Figs. 8c1 and c2. The consistency is larger than 1.0 in western China (Qinghai–Tibet Plateau). The rescaling effect makes the consistency of ETKF_R closer to 1 compared with the ETKF method, especially in the large consistency areas. It shows a more perfect match between the ensemble spread and ensemble mean RMSE in magnitude (the consistency value ranges from 0.5 to 1.5, the white area in Figs. 8c1 and c2) after applying the ETKF_R method. Figure 9 shows the domain-averaged consistency for the ETKF and ETKF_R methods. The consistency is reduced from severe underdispersion of the ETKF method to the more perfect value “1.0” of the ETKF_R method at 12-h forecast lead time. This result indicates that the rescaling effect makes the ensemble spread closer to RMSE at the initial forecast lead time with a better ensemble spread quantity. The differences are not statistically significant at 72-h lead time for all variables and pressure levels.
The vertical distributions of domain-averaged consistency (defined as the ratio of the RMSE to the spread) derived from Figs. 8c1 and 8c2 for (a) zonal wind U, (b) meridional wind V, (c) temperature T, and (d) geopotential height H. The results are averaged from 0000 UTC 7 May 2019 to 1200 UTC 15 May 2019 (twice a day, 18 times in total). Shown are the 12-h forecast lead time (dotted line) and 72-h forecast lead time (solid line). The ETKF method is in blue, and ETKF with rescaling method is in red.
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
Besides the size match between the ensemble mean RMSE and spread, the ensemble spread is also supposed to simulate the spatial structure of the ensemble mean RMSE (Du et al. 2014). Figure 10 shows the spatial correlation coefficient between the ensemble mean RMSE and spread of different variables for the ETKF and ETKF_R methods. Similar to the magnitude improvement, the improvement in spatial structure matching is also dramatic at 12-h forecast lead time for all variables after employing the rescaling method in ETKF. For example, the correlation coefficients increase by approximately 83.67% from 0.098 (ETKF method) to 0.18 (ETKF_R method) for the 600-hPa zonal wind. However, the improvement in the spatial correlation is not statistically significant at 72-h forecast lead time.
As in Fig. 9, but for the vertical distributions of the domain-averaged correlation coefficient between the ensemble mean RMSE and spread.
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
The rank histogram is another commonly used verification metric for ensemble spread (Talagrand et al. 1997; Hamill and Colucci 1997; Candille and Talagrand 2005). The distribution of the rank histogram should be flat over n + 1 sorted bins ranging from the smallest to largest values (n is the ensemble member number) for a desirable ensemble forecast system. The outlier score is measured by the sum of two end bins, which indicates the frequency of an observed event falls outside of the ensemble envelope. As the outlier should be 2/(n + 1) for a perfect EPS, the GRAPES-REPS with 15 ensemble members leads to the expected outlier near 12.5%. Figure 11 shows the rank histograms and corresponding outliers at 12-h lead time for the ETKF and ETKF_R methods. The rank histograms of given variables appear to be “U” shaped, and the outlier scores are high in the ETKF method, which implies that the numbers of observations being either smaller than the ensemble minimum or greater than the ensemble maximum are large. After imposing rescaling into the ETKF method, the distribution of “U” becomes flatter, and the outlier is closer to the expected value at the initial forecast lead times. For example, the outlier score decreases from 0.268 158 (ETKF) to 0.227 574 (ETKF_R) at 12-h forecast lead time for the 200-hPa zonal wind. Therefore, the GRAPES-REPS spread (magnitude) changes from very bad to more perfect regarding rank histograms and outlier scores at the initial forecast lead times after applying the regional rescaling factor. The differences between the ETKF and ETKF_R methods are not significant with the increase of forecast hours.
The rank histograms of (left) zonal wind and (right) meridional wind at 12-h forecast lead time and the corresponding outliers for the ETKF (blue) and ETKF_R (red) methods. (a),(d) 200; (b),(e) 500; and (c),(f) 850 hPa. The results are averaged from 0000 UTC 7 May 2019 to 1200 UTC 15 May 2019 (twice a day, 18 times in total).
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
The above results have suggested the effect of the rescaling factor on ensemble forecast spread verifications for variables at pressure levels. We have repeated the above verifications with near-surface elements (such as the 10-m zonal wind and 2-m temperature), as shown in Figs. 12 and 13 . Qualitatively speaking, the results are the same as what we have seen for the variables at pressure levels. By applying the regional rescaling factor, the horizontal distribution of the ensemble spread changes considerably, and the horizontal distribution of consistency decreases from very large values to relatively small values at the initial forecast lead times. The domain-averaged consistency also decreases from 5 to 4.58 at 12-h forecast lead time for the 2-m temperature (Fig. 13a1), and the difference is significant at the 99% level (t test). Apart from the quantity of ensemble spread, its quality is also a key criterion to measure how the ensemble spread behaves. In response to this, the correlation coefficient between the ensemble spread and ensemble mean RMSE is shown in Figs. 13b1 and 13b2. The correlation coefficient of ETKF_R has a noticeable improvement compared with the ETKF method before 36-h forecast lead time, which passes the significance t test. For example, the correlation coefficient of the 2-m temperature at 12-h forecast lead time changes from 0.159 (ETKF) to 0.207 (ETKF_R). The better consistency and correlation coefficient between RMSE and ensemble spread at initial lead times give rise to a better spread–skill relationship.
As in Fig. 8, but for the zonal wind at 10-m height.
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
The evolution of domain-averaged (a1),(a2) consistency (defined as the ratio of the RMSE to the spread) and (b1),(b2) correlation coefficient between the ensemble mean RMSE and spread with forecast lead times for (top) temperature at 2-m height, and (bottom) zonal wind at 10-m height. The results are averaged from 0000 UTC 7 May 2019 to 1200 UTC 15 May 2019 (twice a day, 18 times in total). The ETKF method is shown in blue and the ETKF with rescaling method in red.
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
b. Probabilistic forecast
The above results have demonstrated that the rescaling effect could effectively improve the ensemble spread with a better spread–skill relationship (including quantity and quality between the RMSE and ensemble spread), a faster spread growth rate, and a better outlier score. The effect of rescaling on probabilistic ensemble forecasts also needs to be further investigated. The comparisons of the verification summary statistics of probabilistic forecasts for ETKF and ETKF_R methods are presented in Fig. 14. The metrics used here include CRPS (Hersbach 2000; Zhu and Toth 2008), statistical reliability (Du and Deng 2010), and ROC (Mason 1979). The probability threshold used in the figure is “exceeding 5 m s−1 over climatology.” The rescaling effect has only a minor impact on the assessment of the sharpness and reliability of probabilistic forecasts. The slight differences in verification metrics have not passed the significance t test.
(a1),(a2) The domain-averaged CRPS; (b1),(b2) reliability diagram, the diagonal line represents perfect reliability; and (c1),(c2) the ROC diagram at 12-h forecast lead time for the ETKF (blue) and ETKF_R (red) methods. (left) 500-hPa zonal wind and (right) 500-hPa meridional wind. The probability of exceeding 5 m s−1 over climatology is used for (b1), (b2), (c1), and (c2). The results are averaged from 0000 UTC 7 May 2019 to 1200 UTC 15 May 2019 (twice a day, 18 times in total).
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
To measure probabilistic forecasts of precipitation for the ETKF and ETKF_R methods, we introduce the Brier (Brier 1950) and AROC scores. Brier evaluates the error between observed probability and forecast probability, which is a simplification of CRPS, and it is the smaller the better. The ROC measures the combined effects of the probability of detection (POD) and the false alarm rate (FAR), and the AROC curve is calculated to measure whether the forecast has a skill. A perfect AROC is 1 (100% POD and 0% FAR). Figure 15 shows the Brier and AROC scores of 12-h accumulated precipitation exceeding 0.1, 5, and 15 mm. There is a moderately significant (at the 40% level) improvement by the ETKF_R method over the ETKF method for the light rain at 36-h forecast time. However, there are no significant differences between these two methods at other forecast lengths and precipitation categories. Therefore, precipitation is much less impacted by the rescaling procedure.
The evolution of (top) the Brier score and (bottom) AROC score for the 12-h accumulated precipitation with forecast lead times. (a1),(b1) Light rain; (a2),(b2) moderate rain; and (a3),(b3) heavy rain. The results are averaged from 0000 UTC 7 May 2019 to 1200 UTC 15 May 2019 (twice a day, 18 times in total). The ETKF method is in blue, and the ETKF with rescaling method is in red.
Citation: Weather and Forecasting 36, 3; 10.1175/WAF-D-20-0176.1
5. Summary and discussion
To accurately describe the initial condition uncertainties in the ensemble prediction system, the initial perturbation structures and growth rates should be rational. An ideal initial perturbation method cannot only effectively represent the uncertainty of the atmospheric state at the time of analysis, but also capture fast error growing modes that reflect the dynamic property of error growth in the atmosphere. The ETKF initial perturbation method has been widely used in the GRAPES-REPS of CMA and could capture sort of fast error growing modes because it combines the 12-h first guess errors. However, an initial perturbation distribution similar to the analysis error cannot be produced in the ETKF method of the GRAPES-REPS. It destroys the desirable characteristics of ideal initial perturbations. To solve this problem, following the rescaling factor of the ETR and masked BV methods, we introduce a regional rescaling mask into the ETKF method. The rescaling factor is defined as the ratio of the square root of the long-term averaged analysis error variance in the kinetic energy norm (referred to as MASK) to the square root of the kinetic energy norm of the analysis perturbation. The former can be obtained from the ECMWF analysis and GRAPES regional model analysis fields, and the latter is derived from the GRAPES-REPS ETKF initial perturbation method. The MASK distribution is small over regions with dense observations. By applying a regional rescaling mask, the perturbation magnitude is expected to be restrained by the analysis uncertainties, and the ensemble spread is assumed to be rational.
This study describes the theoretical formulations of the ETKF and ETKF_R methods, which are used to generate the initial perturbations. The study also compares the results of the ETKF and ETKF_R initial perturbation methods and further demonstrates how rescaling can affect the initial perturbations as well as the ensemble forecast skills. Some commonly used ensemble verification metrics are employed in the comparison. The perturbation structures and its growth characteristics (e.g., the spatial distribution of the initial perturbation, kinetic energy spectrum analysis, perturbation energy evolution, and PECA) are used to illustrate the effect of rescaling on the initial perturbation itself. The ensemble spread and probabilistic forecast are used for demonstrating the ensemble forecast skills. The spread–skill relationship (e.g., the quantity and quality relations between the ensemble spread and ensemble mean forecast error, referred to as the consistency and correlation coefficient), the growth rates of ensemble spread and rank histograms are used to verify the ensemble spread; CRPS, reliability diagrams and ROC curves are used to evaluate the quality of probabilistic forecasts for variables at pressure levels; Brier and AROC scores are measures for the probabilistic forecasts of precipitation.
The characteristics of the initial ensemble perturbation reveal that the initial perturbation structures and its growth rates are reasonable for both the ETKF and ETKF_R methods. Specifically, the perturbation kinetic energy spectrum increases with forecast lead times. The internal and kinetic perturbation energy are the prevailing perturbation energy at low and high vertical levels, and the perturbation energy distribution has a flow-dependent structure. Additionally, the initial perturbations could explain the forecast errors to a certain degree. Compared with the ETKF initial perturbation method, the characteristics of the initial ensemble perturbation become better after applying the ETKF_R method. For example, the initial perturbation structures become more reasonable, the perturbations are able to explain the forecast errors better with a higher PECA value at short lead times, and a lower kinetic energy spectrum as well as perturbation energy at the initial forecast times can lead to a higher growth rate of themselves, which could help the ensemble forecast capture a faster error growing mode. Therefore, the ETKF_R method has more reasonable initial perturbation structures and faster error growing modes than the ETKF method, and solves the problem of being unable to effectively sample analysis uncertainties in the ETKF method.
The ensemble forecast verification results suggest that the rescaling has a large effect on the horizontal distribution of the ensemble spread at the initial forecast lead times, but a very small effect on RMSE. The rescaling leads to a lower domain-averaged spread at initial lead times and a higher growth rate of the ensemble spread. The horizontal distribution and domain-averaged consistency all demonstrate that the rescaling effect makes the ensemble spread closer to the RMSE at initial forecast lead times with a better quantity between the RMSE and spread. The improvement of the ETKF_R method in the spatial structure matching between the ensemble spread and RMSE is also dramatic at initial forecast lead times for all variables compared with the ETKF method. This improvement leads to a better quality between the RMSE and spread. The better performances of quality and quantity between ensemble mean RMSE and ensemble spread bring about a noticeably improved spread–skill relationship. Furthermore, the distributions of rank histogram become flatter, and the outlier scores are closer to the expected values at the initial lead times after applying the ETKF_R method. The above ensemble spread verification results are suitable for variables both at pressure levels and near the surface. In summary, the ETKF_R method has a better spread–skill relationship, including the quality and quantity between the ensemble spread and ensemble mean RMSE, a faster ensemble spread growth rate and more logical rank histogram distributions and outlier scores than the ETKF. However, the rescaling has only a minor impact on the assessment of the sharpness of probabilistic forecasts in terms of the CRPS, reliability and ROC for variables at pressure levels, as well as the Brier and AROC for precipitation. The differences are not statistically significant based on a t test.
The above results reveal that the rescaling method improves the initial perturbation characteristics and ensemble spread skills. Notably, in this study, we only apply a climatologically fixed analysis error mask to restrain the ETKF initial perturbation. With DA being widely applied in the GRAPES-REPS in the near future, there is a need to update the mask with new analysis error estimates based on the real-time DA system (Fisher and Courtier 1995; Barkmeijer et al. 1998). Thus, the initial perturbation is more consistent with the DA system. A reliable EFS could provide accurate background covariance for the DA system, and a good DA system will produce accurate estimates of analysis error variance for EFS.
Acknowledgments
This work is sponsored by National Key Research and Development Program of China (Grant 2018YFF0300103), National Science and Technology Major Project of Ministry of Science and Technology of China (Grant 2018YFC1507405), and GRAPES development project of China Meteorological Administration (Grant GRAPES-FZZX-2019).
REFERENCES
Barkmeijer, J., F. Bouttier, and M. Van Gijzen, 1998: Singular vectors and estimates of the analysis-error covariance metric. Quart. J. Roy. Meteor. Soc., 124, 1695–1713, https://doi.org/10.1002/qj.49712454916.
Bishop, C. H., and Z. Toth, 1999: Ensemble transformation and adaptive observations. J. Atmos. Sci., 56, 1748–1765, https://doi.org/10.1175/1520-0469(1999)056<1748:ETAAO>2.0.CO;2.
Bishop, C. H., B. J. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420–436, https://doi.org/10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2.
Bowler, N. E., A. Arribas, K. R. Mylne, K. B. Robertson, and S. E. Beare, 2008: The MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor., 134, 703–722, https://doi.org/10.1002/qj.234.
Bowler, N. E., A. Arribas, S. E. Beare, K. R. Mylne, and G. J. Shutts, 2009: The local ETKF and SKEB: Upgrades to the MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 135, 767–776, https://doi.org/10.1002/qj.394.
Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78, 1–3, https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.
Buizza, R., and T. N. Palmer, 1995: The singular-vector structure of the atmospheric global circulation. J. Atmos. Sci., 52, 1434–1456, https://doi.org/10.1175/1520-0469(1995)052<1434:TSVSOT>2.0.CO;2.
Buizza, R., M. Milleer, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 2887–2908, https://doi.org/10.1002/qj.49712556006.
Buizza, R., P. L. Houtekamer, Z. Toth, G. Pellerin, M. Z. Wei, and Y. J. Zhu, 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction system. Mon. Wea. Rev., 133, 1076–1097, https://doi.org/10.1175/MWR2905.1.
Candille, G., and O. Talagrand, 2005: Evaluation of probabilistic prediction systems for a scalar variable. Quart. J. Roy. Meteor. Soc., 131, 2131–2150, https://doi.org/10.1256/qj.04.71.
Chen, D. H., and Coauthors, 2008: New generation of multi-scale NWP system (GRAPES): General scientific design. Chin. Sci. Bull., 53, 3433–3445, https://doi.org/10.1007/s11434-008-0494-z.
Chen, J., J. Z. Wang, J. Du, Y. Xia, F. J. Chen, and H. Q. Li, 2020: Forecast bias correction through model integration: A dynamic wholesale approach. Quart. J. Roy. Meteor. Soc., 146, 1149–1168, https://doi.org/10.1002/qj.3730.
Denis, B., J. Cote, and R. Laprise, 2002: Spectral decomposition of two-dimensional atmospheric fields on limited-area domains using the discrete cosine transform (DCT). Mon. Wea. Rev., 130, 1812–1829, https://doi.org/10.1175/1520-0493(2002)130<1812:SDOTDA>2.0.CO;2.
Descamps, L., and O. Talagrand, 2007: On some aspects of the definition of initial conditions for ensemble prediction. Mon. Wea. Rev., 135, 3260–3272, https://doi.org/10.1175/MWR3452.1.
Du, J., 2007: Uncertainty and ensemble forecast. NOAA/NWS Science and Technology Infusion Lecture Series, 42 pp.
Du, J., and G. Deng, 2010: The utility of the transition from deterministic to probabilistic weather forecasts—Verification and application of probabilistic forecasts. Meteor. Mon., 36, 10–18.
Du, J., and B. Zhou, 2017: Ensemble fog prediction. Marine Fog: Challenges and Advancements in Observations, Modeling, and Forecasting, D. Koracin and C. E. Dorman, Eds., Springer, 477–509, https://doi.org/10.1007/978-3-319-45229-6_10.
Du, J., R. Yu, C. Cui, and J. Li, 2014: Using a mesoscale ensemble to predict forecast error and perform targeted observation. Acta Oceanol. Sin., 33, 83–91, https://doi.org/10.1007/s13131-014-0426-5.
Duan, W. S., and Z. H. Huo, 2016: An approach to generating mutually independent initial perturbations for ensemble forecasts: Orthogonal conditional nonlinear optimal perturbations. J. Atmos. Sci., 73, 997–1014, https://doi.org/10.1175/JAS-D-15-0138.1.
Dudhia, J., 1989: Numerical study of convective observed during the winter monsoon experiment using a mesoscale two-dimensional model. J. Atmos. Sci., 46, 3077–3107, https://doi.org/10.1175/1520-0469(1989)046<3077:NSOCOD>2.0.CO;2.
Ehrendorfer, M., 1994: The Liouville equation and its potential usefulness for the prediction of forecast skill. Part I: Theory. Mon. Wea. Rev., 122, 703–713, https://doi.org/10.1175/1520-0493(1994)122<0703:TLEAIP>2.0.CO;2.
Feng, J., R. Q. Ding, D. Q. Liu, and J. P. Li, 2014: The application of nonlinear local Lyapunov vectors to ensemble predictions in the Lorenz systems. J. Atmos. Sci., 71, 3554–3567, https://doi.org/10.1175/JAS-D-13-0270.1.
Feng, J., Z. Toth, and M. Pena, 2017: Spatially extended estimates of analysis and short-range forecast error variances. Tellus, 69A, 1325301, https://doi.org/10.1080/16000870.2017.1325301.
Feng, J., Z. Toth, M. Pena, and J. Zhang, 2020: Partition of analysis and forecast error variance into growing and decaying components. Quart. J. Roy. Meteor. Soc., 146, 1302–1321, https://doi.org/10.1002/qj.3738.
Feng, Y. X., J. Z. Min, X. R. Zhuang, and S. Q. Wang, 2019: Ensemble sensitivity analysis-based ensemble transform with 3D rescaling initialization method for storm-scale ensemble forecast. Atmosphere, 10, 24, https://doi.org/10.3390/atmos10010024.
Fisher, M., and P. Courtier, 1995: Estimating the covariance matrices of analysis and forecast error in variational data assimilation. ECMWF Tech. Memo. 220, 28 pp.
Hamill, T., and S. J. Colucci, 1997: Verification of Eta/RSM short-range ensemble forecasts. Mon. Wea. Rev., 125, 1312–1327, https://doi.org/10.1175/1520-0493(1997)125<1312:VOERSR>2.0.CO;2.
Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15, 559–570, https://doi.org/10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2.
Hong, S. Y., and H. L. Pan, 1996: Nonlocal boundary layer vertical diffusion in a medium-range forecast model. Mon. Wea. Rev., 124, 2322–2339, https://doi.org/10.1175/1520-0493(1996)124<2322:NBLVDI>2.0.CO;2.
Hong, S. Y., and J. O. Lim, 2006: The WRF single moment 6-class microphysics scheme (WSM6). J. Korean Meteor. Soc., 42, 129–151.
Houtekamer, P. L., L. Lefaivre, J. Derome, H. Ritchie, and H. L. Mitchell, 1996: A system simulation approach to ensemble prediction. Mon. Wea. Rev., 124, 1225–1242, https://doi.org/10.1175/1520-0493(1996)124<1225:ASSATE>2.0.CO;2.
Kain, J. S., 2004: The Kain–Fritsch convective parameterization: An update. J. Appl. Meteor., 43, 170–181, https://doi.org/10.1175/1520-0450(2004)043<0170:TKCPAU>2.0.CO;2.
Kay, J. K., and H. M. Kim, 2014: Characteristics of initial perturbations in the ensemble prediction system of the Korea Meteorological Administration. Wea. Forecasting, 29, 563–581, https://doi.org/10.1175/WAF-D-13-00097.1.
Kurihara, Y., M. A. Bender, R. E. Tuleya, and R. J. Ross, 1995: Improvements in the GFDL hurricane prediction system. Mon. Wea. Rev., 123, 2791–2801, https://doi.org/10.1175/1520-0493(1995)123<2791:IITGHP>2.0.CO;2.
Langland, R. H., R. N. Maue, and C. H. Bishop, 2008: Uncertainty in atmospheric temperature analyses. Tellus, 60A, 598–603, https://doi.org/10.1111/j.1600-0870.2008.00336.x.
Leith, C., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102, 409–418, https://doi.org/10.1175/1520-0493(1974)102<0409:TSOMCF>2.0.CO;2.
Long, K. J., J. Chen, X. L. Ma, and Y. M. Ji, 2011: The preliminary study on ensemble prediction of GRAPES-meso based on ETKF. J. Chengdu Univ. Info. Tech., 26, 37–46.
Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130–141, https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.
Lorenz, E. N., 1965: A study of the predictability of a 28-variable atmospheric model. Tellus, 17, 321–333, https://doi.org/10.3402/tellusa.v17i3.9076.
Ma, J. H., Y. J. Zhu, D. C. Hou, X. Q. Zhou, and M. Pena, 2014: Ensemble transform with 3D rescaling initialization method. Mon. Wea. Rev., 142, 4053–4073, https://doi.org/10.1175/MWR-D-13-00367.1.
Magnusson, L., E. Kallen, and J. Nycander, 2008: Initial state perturbations in ensemble forecasting. Nonlinear Processes Geophys., 15, 751–759, https://doi.org/10.5194/npg-15-751-2008.
Magnusson, L., J. Nycander, and E. Kallen, 2009: Flow-dependent versus flow-independent initial perturbations for ensemble predictuion. Tellus, 61A, 194–209, https://doi.org/10.1111/j.1600-0870.2008.00385.x.
Mahrt, L., and M. Ek, 1984: The influence of atmosphere stability on potential evaporation. J. Climate Appl. Meteor., 23, 222–234, https://doi.org/10.1175/1520-0450(1984)023<0222:TIOASO>2.0.CO;2.
Mason, I. B., 1979: On reducing probability forecasts to yes/no forecasts. Mon. Wea. Rev., 107, 207–211, https://doi.org/10.1175/1520-0493(1979)107<0207:ORPFTY>2.0.CO;2.
McLay, J., C. H. Bishop, and C. A. Reynolds, 2008: Evaluation of the ensemble transform analysis perturbation scheme at NRL. Mon. Wea. Rev., 136, 1093–1108, https://doi.org/10.1175/2007MWR2010.1.
Mlawer, E. J., S. J. Taubman, P. D. Brown, M. J. Iacono, and S. A. Clough, 1997: Radiative transfer for inhomogeneous atmosphere: RRTM, a validated corrected-k model for the longwave. J. Geophys. Res., 102, 16 663–16 682, https://doi.org/10.1029/97JD00237.
Molteni, F., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122, 73–119, https://doi.org/10.1002/qj.49712252905.
Noilhan, J., and S. Planton, 1989: A simple parameterization of land surface processes for meteorological models. Mon. Wea. Rev., 117, 536–549, https://doi.org/10.1175/1520-0493(1989)117<0536:ASPOLS>2.0.CO;2.
Palmer, T. N., 1993: Extended-range atmospheric prediction and the Lorenz model. Bull. Amer. Meteor. Soc., 74, 49–65, https://doi.org/10.1175/1520-0477(1993)074<0049:ERAPAT>2.0.CO;2.
Palmer, T. N., 1999: Predicting uncertainty in forecasts of weather and climate. ECMWF Tech. Memo. 294, 48 pp.
Palmer, T. N., R. Gelaro, J. Barkmeijer, and R. Buizza, 1998: Singular vectors, metrics, and adaptive observations. J. Atmos. Sci., 55, 633–653, https://doi.org/10.1175/1520-0469(1998)055<0633:SVMAAO>2.0.CO;2.
Pena, M., and Z. Toth, 2014: Estimation of analysis and forecast error variances. Tellus, 66A, 21767, https://doi.org/10.3402/tellusa.v66.21767.
Raynaud, L., and F. Bouttier, 2016: Comparison of initial perturbation methods for ensemble prediction at convective scale. Quart. J. Roy. Meteor. Soc., 142, 854–866, https://doi.org/10.1002/qj.2686.
Saito, K., H. Seko, M. Kunii, and T. Miyoshi, 2012: Effect on lateral boundary perturbations on the breeding method and the local ensemble transform Kalman filter for mesoscale ensemble prediction. Tellus, 64A, 11594, https://doi.org/10.3402/tellusa.v64i0.11594.
Steven Tracton, M., and E. Kalnay, 1993: Operational ensemble prediction at the National Meteorological Center: Practical aspects. Wea. Forecasting, 8, 379–398, https://doi.org/10.1175/1520-0434(1993)008<0379:OEPATN>2.0.CO;2.
Swanson, K. L., and P. Roebber, 2008: The impact of analysis error on medium-range weather forecast. Mon. Wea. Rev., 136, 3425–3431, https://doi.org/10.1175/2008MWR2475.1.
Szunyogh, I., and Z. Toth, 2002: The effect of increased horizontal resolution on the NCEP global ensemble mean forecasts. Mon. Wea. Rev., 130, 1125–1143, https://doi.org/10.1175/1520-0493(2002)130<1125;TEOIHR>2.0.CO;2.
Talagrand, O., R. Vautard, and B. Strauss, 1997: Evaluation of probabilistic prediction systems. Proc. ECMWF Workshop on Predictability, Reading, United Kingdom, ECMWF, 25 pp.
Talagrand, O., G. Candille, and L. Descamps, 2007: On some aspects of validation of probabilistic prediction, Proc. Workshop on Ensemble Prediction, Reading, United Kingdom, ECMWF, 28 pp.
Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 2317–2330, https://doi.org/10.1175/1520-0477(1993)074<2317:EFANTG>2.0.CO;2.
Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125, 3297–3319, https://doi.org/10.1175/1520-0493(1997)125<3297:EFANAT>2.0.CO;2.
Wang, J. Z., J. Chen, J. Du, Y. T. Zhang, Y. Xia, and G. Deng, 2018a: Sensitivity of ensemble forecast verification to model bias. Mon. Wea. Rev., 146, 781–796, https://doi.org/10.1175/MWR-D-17-0223.1.
Wang, J. Z., J. Chen, Z. R. Zhuang, H. B. Zhang, and J. Wang, 2018b: Characteristics of initial perturbation growth rate in the regional ensemble prediction system of GRAPES. Chin. J. Atmos. Sci., 42, 367–382.
Wang, T. W., 2008: Impact of initial perturbation on meso-scale model uncertainty [D]. M.S. thesis, Department of Atmospheric Science, Chinese Academy of Meteorological Sciences, 93 pp.
Wang, X. G., and C. H. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60, 1140–1158, https://doi.org/10.1175/1520-0469(2003)060<1140:ACOBAE>2.0.CO;2.
Wang, X. G., C. H. Bishop, and S. J. Julier, 2004: Which is better, an ensemble of positive negative pairs or a centered spherical simplex ensemble? Mon. Wea. Rev., 132, 1590–1605, https://doi.org/10.1175/1520-0493(2004)132<1590:WIBAEO>2.0.CO;2.
Wang, X. G., D. Parrish, D. Kleist, and J. Whitaker, 2013: GSI 3DVar-based ensemble variational hybrid data assimilation for NCEP global forecast system: Single-resolution experiments. Mon. Wea. Rev., 141, 4098–4117, https://doi.org/10.1175/MWR-D-12-00141.1.
Wei, M. Z., and Z. Toth, 2003: A new measure of ensemble performance: Perturbations versus error correlation analysis (PECA). Mon. Wea. Rev., 131, 1549–1565, https://doi.org/10.1175//1520-0493(2003)131<1549:ANMOEP>2.0.CO;2.
Wei, M. Z., Z. Toth, D. Wobus, and Y. J. Zhu, 2005: Initial perturbations for NCEP ensemble forecast system. Thorpex Symp.: Proc. First THORPEX Internal Science Symp., Montreal, Canada, WMO, WMO-TD-1237, WWRP-THORPEX-6, WMO, 227–230.
Wei, M. Z., Z. Toth, R. Wobus, Y. J. Zhu, C. H. Bishop, and X. G. Wang, 2006: Ensemble transform Kalman filter-based ensemble perturbations in an operational global prediction system at NCEP. Tellus, 58A, 28–44, https://doi.org/10.1111/j.1600-0870.2006.00159.x.
Wei, M. Z., Z. Toth, R. Wobus, and Y. J. Zhu, 2008: Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system. Tellus, 60A, 62–79, https://doi.org/10.1111/j.1600-0870.2007.00273.x.
Wei, M. Z., Z. Toth, and Y. J. Zhu, 2010: Analysis differences and error variance estimates from multi-center analysis data. Aust. Meteor. Oceanogr. J., 59, 25–34, https://doi.org/10.22499/2.5901.005.
Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. International Geophysics Series, Vol. 100, Academic Press, 648 pp.
Wu, Z. Q., J. Zhang, J. Chen, B. Pang, Y. Xia, and F. J. Chen, 2020: The study on the method of conditional typhoon vortex relocation for GRAPES regional ensemble prediction. Acta Meteor. Sin., 78, 163–176.
Yuan, Y., X. L. Li, J. Chen, and Y. Xia, 2016: Stochastic parameterization toward model uncertainty for the GRAPES mesoscale ensemble prediction system. Meteor. Mon., 42, 1161–1175.
Zhang, H. B., J. Chen, X. F. Zhi, Y. L. Li, and Y. Sun, 2014: Study on the application of GRAPES regional ensemble prediction system. Meteor. Mon., 40, 1076–1087.
Zhang, H. B., J. Chen, X. F. Zhi, and Y. N. Wang, 2015: A comparison of ETKF and downscaling in a regional ensemble prediction system. Atmosphere, 6, 341–360, https://doi.org/10.3390/atmos6030341.
Zhu, L. J., J. D. Gong, L. P. Huang, D. H. Chen, Y. Jiang, and L. T. Deng, 2007: Three-dimensional cloud initial field created and applied to GRAPES numerical weather prediction nowcasting. Yingyong Qixiang Xuebao, 28, 38–51.
Zhu, Y., and Z. Toth, 2008: Ensemble based probabilistic forecast verification. 19th Conf. on Probability and Statistics, New Orleans, LA, Amer. Meteor. Soc., 2.2, https://ams.confex.com/ams/88Annual/techprogram/paper_131645.htm.