Performance of a Convective-Scale Ensemble Prediction System on 2017 Warm-Season Afternoon Thunderstorms over Taiwan

I-Han Chen aCentral Weather Bureau, Taipei, Taiwan

Search for other papers by I-Han Chen in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-3610-6793
,
Yi-Jui Su aCentral Weather Bureau, Taipei, Taiwan

Search for other papers by Yi-Jui Su in
Current site
Google Scholar
PubMed
Close
,
Hsiao-Wei Lai aCentral Weather Bureau, Taipei, Taiwan

Search for other papers by Hsiao-Wei Lai in
Current site
Google Scholar
PubMed
Close
,
Jing-Shan Hong aCentral Weather Bureau, Taipei, Taiwan

Search for other papers by Jing-Shan Hong in
Current site
Google Scholar
PubMed
Close
,
Chih-Hsin Li aCentral Weather Bureau, Taipei, Taiwan

Search for other papers by Chih-Hsin Li in
Current site
Google Scholar
PubMed
Close
,
Pao-Liang Chang aCentral Weather Bureau, Taipei, Taiwan

Search for other papers by Pao-Liang Chang in
Current site
Google Scholar
PubMed
Close
, and
Ying-Jhang Wu aCentral Weather Bureau, Taipei, Taiwan

Search for other papers by Ying-Jhang Wu in
Current site
Google Scholar
PubMed
Close
Free access

Abstract

A 16-member convective-scale ensemble prediction system (CEPS) developed at the Central Weather Bureau (CWB) of Taiwan is evaluated for probability forecasts of convective precipitation. To address the issues of limited predictability of convective systems, the CEPS provides short-range forecasts using initial conditions from a rapid-updated ensemble data assimilation system. This study aims to identify the behavior of the CEPS forecasts, especially the impact of different ensemble configurations and forecast lead times. Warm-season afternoon thunderstorms (ATs) from 30 June to 4 July 2017 are selected. Since ATs usually occur between 1300 and 2000 LST, this study compares deterministic and probabilistic quantitative precipitation forecasts (QPFs) launched at 0500, 0800, and 1100 LST. This study demonstrates that initial and boundary perturbations (IBP) are crucial to ensure good spread–skill consistency over the 18-h forecasts. On top of IBP, additional model perturbations have insignificant impacts on upper-air and precipitation forecasts. The deterministic QPFs launched at 1100 LST outperform those launched at 0500 and 0800 LST, likely because the most-recent data assimilation analyses enhance the practical predictability. However, it cannot improve the probabilistic QPFs launched at 1100 LST due to inadequate ensemble spreads resulting from limited error growth time. This study points out the importance of sufficient initial condition uncertainty on short-range probabilistic forecasts to exploit the benefits of rapid-update data assimilation analyses.

Significance Statement

This study aims to understand the behavior of convective-scale short-range probabilistic forecasts in Taiwan and the surrounding area. Taiwan is influenced by diverse weather systems, including typhoons, mei-yu fronts, and local thunderstorms. During the past decade, there has been promising improvement in predicting mesoscale weather systems (e.g., typhoons and mei-yu fronts). However, it is still challenging to provide timely and accurate forecasts for rapid-evolving high-impact convection. This study provides a reference for the designation of convective-scale ensemble prediction systems; in particular, those with a goal to provide short-range probabilistic forecasts. While the findings cannot be extrapolated to all ensemble prediction systems, this study demonstrates that initial and boundary perturbations are the most important factors, while the model perturbation has an insignificant effect. This study suggests that in-depth studies are required to improve the convective-scale initial condition accuracy and uncertainty to provide reliable probabilistic forecasts within short lead times.

© 2023 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Jing-Shan Hong, rfs14@cwb.gov.tw

Abstract

A 16-member convective-scale ensemble prediction system (CEPS) developed at the Central Weather Bureau (CWB) of Taiwan is evaluated for probability forecasts of convective precipitation. To address the issues of limited predictability of convective systems, the CEPS provides short-range forecasts using initial conditions from a rapid-updated ensemble data assimilation system. This study aims to identify the behavior of the CEPS forecasts, especially the impact of different ensemble configurations and forecast lead times. Warm-season afternoon thunderstorms (ATs) from 30 June to 4 July 2017 are selected. Since ATs usually occur between 1300 and 2000 LST, this study compares deterministic and probabilistic quantitative precipitation forecasts (QPFs) launched at 0500, 0800, and 1100 LST. This study demonstrates that initial and boundary perturbations (IBP) are crucial to ensure good spread–skill consistency over the 18-h forecasts. On top of IBP, additional model perturbations have insignificant impacts on upper-air and precipitation forecasts. The deterministic QPFs launched at 1100 LST outperform those launched at 0500 and 0800 LST, likely because the most-recent data assimilation analyses enhance the practical predictability. However, it cannot improve the probabilistic QPFs launched at 1100 LST due to inadequate ensemble spreads resulting from limited error growth time. This study points out the importance of sufficient initial condition uncertainty on short-range probabilistic forecasts to exploit the benefits of rapid-update data assimilation analyses.

Significance Statement

This study aims to understand the behavior of convective-scale short-range probabilistic forecasts in Taiwan and the surrounding area. Taiwan is influenced by diverse weather systems, including typhoons, mei-yu fronts, and local thunderstorms. During the past decade, there has been promising improvement in predicting mesoscale weather systems (e.g., typhoons and mei-yu fronts). However, it is still challenging to provide timely and accurate forecasts for rapid-evolving high-impact convection. This study provides a reference for the designation of convective-scale ensemble prediction systems; in particular, those with a goal to provide short-range probabilistic forecasts. While the findings cannot be extrapolated to all ensemble prediction systems, this study demonstrates that initial and boundary perturbations are the most important factors, while the model perturbation has an insignificant effect. This study suggests that in-depth studies are required to improve the convective-scale initial condition accuracy and uncertainty to provide reliable probabilistic forecasts within short lead times.

© 2023 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Jing-Shan Hong, rfs14@cwb.gov.tw

1. Introduction

According to Lorenz (1969), predictability issues can be classified into 1) practical predictability (Melhauser and Zhang 2012), that is, the best-possible prediction based on current forecast procedures, such as data assimilations and forecast models, and 2) intrinsic predictability (Zhang et al. 2003, 2007): the best-possible prediction using nearly perfect initial conditions, boundary conditions, and forecast procedures. Although intrinsic predictability is limited in large part due to weather systems, it is possible to enhance practical predictability by improving initial conditions, boundary conditions, and model processes (Zhang et al. 2002, 2019; Frogner et al. 2019). Nowadays, cloud-resolving numerical weather prediction (NWP) models with a grid spacing of 1–4 km are commonly used for scientific research and operational purposes (Golding et al. 2016; Schwartz et al. 2018; Kalina et al. 2021). Since 2016, the Central Weather Bureau (CWB) of Taiwan has developed high-resolution hourly updated data assimilation systems to provide deterministic predictions for convective precipitation (Chen et al. 2020; Jiang et al. 2021). Despite recent progress, it is still challenging to predict the timing, location, and intensity of convection. Considering the limited deterministic predictability, a convective-scale ensemble prediction system (CEPS) has been developed at the CWB to provide short-range probability forecasts. It is important to mention that the CWB CEPS aims to provide probabilistic forecasts for individual storms with sufficient lead times, which is very similar to the purpose of the Warn-on-Forecast System (WoFS; Stensrud et al. 2009, 2013) developed at the National Severe Storms Laboratory (NSSL). Aiming to improve forecasts for hazardous weather, the WoFS has a tentative plan to be operational at the National Weather Service (NWS) during the 2025–30 time frame (Guerra et al. 2022).

The development of CEPS is still at the cutting edge and cannot be analogous to large-scale and mesoscale ensemble prediction systems (EPS) due to the distinct behaviors of atmospheric convection. For example, Hohenegger and Schär (2007) pointed out that baroclinic and convective instabilities are the major synoptic-scale and convective-scale error growth mechanisms, respectively. Also, the cloud-resolving simulation has error growth rates 10 times larger than the synoptic-scale simulation. According to Frogner et al. (2019), the operational CEPS (for Norway, Sweden, and Finland) loses predictability rapidly in the first 6-h forecast for small-scale precipitation (smaller than ∼60 km). Despite the inherently limited predictability, studies demonstrated that assimilating radar (Bachmann et al. 2018; Sun et al. 2014) and surface (Chen et al. 2020) observations can improve forecast skill for convective precipitation. To enhance model forecast skills for convection, the CWB CEPS was designed with two focuses: 1) using hourly updated ensemble analyses as initial conditions to improve initial condition accuracy and 2) providing reliable probabilistic forecasts by sampling uncertainties that can facilitate proper error growth at the convective scale. In brief, the CEPS aims to provide accurate and reliable short-range probabilistic forecasts for convection. For example, using the CEPS to provide reliable probabilistic forecasts within a lead time of fewer than 6 h. This objective is envisaged to be supported by using the rapid-update ensemble analyses since they assimilate the most recent observations. However, the key to success lies in resolving the issues of model spinup and limited forecast error growth time.

Since a reliable EPS should have an ensemble spread comparable to the error of the ensemble mean (Whitaker and Loughe 1998), it is crucial to sample perturbations that can represent convective-scale forecast uncertainty at different forecast hours. The CWB CEPS considered initial condition uncertainty, lateral boundary condition uncertainty due to a limited-area domain, and model uncertainty. In the past decade, convection-allowing EPSs have been tested at several operational centers (Hagelin et al. 2017; Bouttier et al. 2012; Peralta et al. 2012). As shown in Harnisch and Keil (2015), the ensemble analyses from a local ensemble transform Kalman filter (LETKF; Hunt et al. 2007) data assimilation system (Schraff et al. 2016) is competitive with the downscaled initial conditions used at the German Weather Service at that time. It is worth mentioning that high-resolution analyses are helpful in mitigating model spinup due to downscaled interpolation and contain more fine-scale perturbations (Sun et al. 2014), both of which are valuable for convective-scale forecasts. In addition to initial perturbations, including lateral boundary perturbations to represent large-scale uncertainty is essential for limited-area EPSs (Saito et al. 2011). Depending on ensemble configuration and weather regimes, Kühnlein et al. (2014) pointed out that initial perturbations dominate forecasts up to 6–9 h and are outweighed by boundary perturbations afterward.

Besides initial and boundary condition uncertainty, forecast errors also originate from imperfect numerical models. Although knowledge of convective-scale error growth mechanisms is incomplete, the small-scale variability of low-level wind, temperature, and water vapor was reported to influence precipitation predictability (Weckwerth 2000). Studies have shown that small-scale perturbations in the planetary boundary layer (PBL) scheme can influence the convective initiation, precipitation amount, and location (Hirt and Craig 2021; Hirt et al. 2019). Also, small-scale perturbations in microphysics schemes facilitate error growth in areas with larger convective instability (Hermoso et al. 2021; Thompson et al. 2021). Existing model error representation approaches include multidynamics (Roberts et al. 2018), multiphysics (García-Ortega et al. 2017), multiparameters (Gebhardt et al. 2011), and stochastic perturbation schemes (Romine et al. 2014; Jankov et al. 2017; Li et al. 2020; Berner et al. 2016). Although a multiphysics ensemble does not represent an equally likely outcome, it was adopted in the CWB CEPS since it is practical and still extensively used. For example, an 8-member multiphysics and multidynamics High-Resolution Ensemble Forecast system (HREFv2; Roberts et al. 2018) was developed at the National Centers for Environmental Prediction (NCEP) and was reported to be very effective (Clark et al. 2019).

In warm seasons (May–October), convection frequently occurs in the afternoon and produces intense rainfall and lightning flashes. Since it can lead to urban flooding and threaten aviation safety with little response time (Harding 2011), there are high demands from society for timely and accurate warnings. For this reason, the CWB CEPS has an initial objective to provide short-range forecast guidance for afternoon thunderstorms (ATs). In this study, we aim to explore the behavior of the CEPS, including the following aspects.

  1. Since the behaviors of different perturbations can depend on ensemble configuration and weather regimes, what is the impact of initial, lateral boundary, and model perturbations on the CEPS over the subtropical Taiwan island? In particular, what ensemble configuration can produce the highest spread–skill consistency? It should be noted that the treatment of lateral boundary conditions is specifically important for limited-area model applications. Especially for limited area ensemble systems, the spread is envisaged to collapse quickly if the boundary perturbations are not well sampled.

  2. The CWB CEPS aims to provide reliable probabilistic forecasts within short forecast lead times since the predictability of small-scale precipitation decreases rapidly in the first 6-h forecast (Frogner et al. 2019). For a rapid-updated (e.g., every 3 h) CEPS, an optimal situation could be that the most-recent analyses can lead to the most reliable probabilistic forecasts. According to Chen et al. (2020), an hourly updated three-dimensional variational (3DVAR) data assimilation system improved deterministic QPF skills for ATs as lead times decreased. The reason is that initial condition accuracy can be improved at each hour in the morning. The current study uses initial conditions from an hourly update LETKF system (Jiang et al. 2021). In this study, we first evaluate the impact of ensemble configuration and lead times on the deterministic QPF skills of each ensemble member. After that, the behavior of resulting probabilistic forecasts is discussed.

In this study, we also explore the capability of the CEPS to predict the hourly evolution of convective echoes over Taiwan. Section 2 contains descriptions of AT cases, experimental design, and verification dataset. Section 3 provides statistical results of upper air and surface forecasts. Verification results of deterministic and probabilistic QPFs are discussed in section 4. Section 5 illustrates the capability of CEPS in predicting hourly convective echoes. Conclusions are provided in section 6.

2. Experimental design

a. Case description

This study includes five consecutive warm-season ATs from 30 June to 4 July 2017. As shown in Fig. 1, Taiwan was dominated by a subtropical high in the early study period. In addition, the tropical cyclone Nanmadol passed the Pacific Ocean from the east of Taiwan during 2–3 July 2017. Each AT case had maximum accumulated rainfall exceeding 130 mm (24 h)−1 with different rainfall locations and spatial coverage (Fig. 2). Although scattered rainfall systems over the surrounding ocean evolved at various times (Figs. 2 and 3a), the five cases have similar precipitation time evolution over land. As shown in Fig. 3b, the precipitation over land initiated at 12–14 LST, reached its peak at 1500–1700 LST, and dissipated at 1800–2000 LST. One objective of this study is to evaluate the probabilistic precipitation forecast skills for these fast-evolving ATs since it is one of the most demanding forecast problems in Taiwan.

Fig. 1.
Fig. 1.

Synoptic conditions valid at 0000 UTC each day from 29 Jun to 4 Jul 2017. The ECMWF Reanalysis v5 (ERA5) is downloaded from Copernicus Climate Change Service (C3S) Climate Data Store. Variables included 850-hPa geopotential height (solid lines), 850-hPa relative humidity (%; shaded), and 850-hPa wind field (m s−1; wind barb).

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

Fig. 2.
Fig. 2.

The spatial distribution of 24-h accumulated precipitation (mm; shaded) from 29 Jun to 4 Jul 2017. The dataset is CWB operational radar-derived rainfall estimation products from QPESUMS system.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

Fig. 3.
Fig. 3.

Hourly evolutions of area-averaged 1-h accumulated precipitation (mm) from 29 Jun to 4 Jul 2017 (colored lines). (a) Calculation over the ocean grid points within the area in Fig. 2. (b) Calculation over the land grid points within the area in Fig. 2.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

b. Model configuration

The designation of the 16-member CEPS was based on CWB operational convective-scale deterministic NWP systems (Chen et al. 2020). In this study, the Advanced Research version of the Weather Research and Forecasting (WRF) Model version 3.8.1 was employed (Skamarock et al. 2008). The computing domain was centered on Taiwan island (Fig. 4) with a 2-km grid spacing (450 × 450 grid points) in the horizontal and 52 eta levels in the vertical. The model top was located at 20 hPa. To understand the impact of different sources of uncertainty, this study designed four ensemble experiments composed of different perturbations, including initial condition perturbations, lateral boundary condition perturbations, model perturbations, and their combinations. Ensemble forecasts were launched at 0500, 0800, and 1100 LST each day to investigate the impact of forecast lead times. That is, each experiment has 15 ensemble forecasts (15 initial time × 16 members) initialized between 30 June and 4 July 2017 and provides 18-h forecasts to cover the lifetime of ATs. Configurations of initial perturbations, boundary perturbations, model perturbations, and their corresponding ensemble experiments are described in sections 2c and 2d.

Fig. 4.
Fig. 4.

Configuration of 10-km (blue dashed box) and 2-km (solid green box) computing domains. In this study, the 10-km domain provides lateral boundary conditions to the CEPS 2-km domain.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

c. Configurations of initial and lateral boundary perturbations

The ensemble forecasts were initialized by the first 16 members of the 32-member ensemble analyses from a 2-km LETKF data assimilation system (Jiang et al. 2021). In this study, the LETKF system assimilated radar and surface observations using hourly update cycles starting from 1200 UTC 29 June 2017. Yang (2005) proposed a blending scheme based on an incremental spatial filtering scheme as part of the initialization process in a limited area model system. Also, (Hsiao et al. 2015) demonstrated that the blending scheme could improve the typhoon track and rainfall forecast by combining the large-scale global analysis and the regional mesoscale analysis. In this study, the LETKF applied the blending scheme (Jiang et al. 2021) two times a day at 0600 and 1800 UTC to mitigate large-scale model errors accumulated during continuous cycles. To provide sufficient ensemble spreads, the LETKF system included multiplicative inflation, additive inflation, and relaxation to prior spread schemes, which was expected to mitigate the initial deficiencies of small-scale perturbations (Harnisch and Keil 2015).

Due to limited computing resources, high-resolution NWP models commonly integrate over a finite area and require additional lateral boundary conditions. This study generated a set of boundary conditions by adding ensemble perturbations downscaled from a 10-km EPS (EPS10) onto ensemble means downscaled from a 10-km deterministic system (DET10; Chen et al. 2020). The perturbed variables included three-dimensional wind components, temperature, water vapor mixing ratio, column dry air mass, and geopotential height. This configuration aims to utilize more accurate ensemble means from DET10 since EPS10 requires enough error growth time to reach sufficient spreads, which usually yields a loss of forecast accuracy. In this study, forecasts from DET10 and EPS10 had a lead time of 0–6 and 12–18 h, respectively. The computing domain of DET10 is shown in Fig. 4. The DET10 and EPS10 were initialized by the 0.25° NCEP Global Forecast System (GFS) and 0.5° NCEP Global Ensemble Forecast System (GEFS), respectively. The EPS10 included initial, lateral boundary, multiphysics, and stochastic perturbations with settings based on the CWB operational 15-km ensemble system (Li et al. 2020). In this study, the 10–2-km downscaling process used the stand-alone WRF ndown package.

To evaluate the role of initial and boundary perturbations, the 1) ICP experiment, 2) BCP experiment, and 3) IBP experiment utilized 1) initial perturbations, 2) lateral boundary perturbations, and 3) both perturbations, respectively. Configurations of all ensemble experiments and their acronyms are summarized in Table 1. Model settings in the three experiments are identical to the CWB operational convection-scale deterministic systems, including the Yonsei University PBL scheme (Hong et al. 2006), MM5 similarity scheme (Paulson 1970), Goddard microphysics scheme (Tao et al. 1989, 2016), RRTMG shortwave and longwave schemes (Iacono et al. 2008), and Noah land surface model (Tewari et al. 2004).

Table 1

Configuration of ensemble experiments. DET10 is downscaled from deterministic 10-km forecasts. EPS10 is downscaled from 10-km regional ensemble forecasts.

Table 1

d. Configurations of model perturbations

In addition to the initial and boundary perturbations, this study used multiple physics schemes to represent model uncertainties. It should be noted that various combinations of physics schemes have been evaluated to figure out an optimal mixed-physics ensemble (not shown). The selected PBL scheme included the YSU scheme, Bougeault–Lacarrere scheme (BouLac; Bougeault and Lacarrere 1989), Shin–Hong scale-aware scheme (Shin and Hong 2015), and Mellor–Yamada–Nakanishi–Niino (MYNN; Nakanishi and Niino 2009) Level 2.5 and Level 3 schemes (MYNN3; Nakanishi and Niino 2006). The selected surface layer schemes included the MM5 similarity scheme and the MYNN scheme. The selected microphysics schemes included the Goddard scheme, Morrison two-moment scheme (Morrison et al. 2009), WRF single-moment 5-class schemes (WSM5, Hong et al. 2004), and WRF single-moment 6-class scheme (WSM6, Hong and Lim 2006). Details of multiphysics configurations are listed in Table 2. Aside from the experiments mentioned above (ICP, BCP, and IBP), an additional IBP_PHY experiment (see Table 1) with multiphysics was configured. Since IBP_PHY included both initial and boundary perturbations, comparing IBP and IBP_PHY reveals the impact of mixed-physics schemes.

Table 2

Configuration of multiphysics in the ensemble. Abbreviations of PBL schemes are Yonsei University (YSU), Bougeault–Lacarrere scheme (BouLac), Shin–Hong scale-aware scheme (Shin-Hong), Mellor–Yamada–Nakanishi–Niino Level 2.5 (MYNN), and Level 3 (MYNN3) schemes. Abbreviations of surface layers are the MM5 similarity scheme (MM5), and MYNN scheme. Abbreviations of microphysics schemes are Goddard scheme, Morrison two-moment scheme (Morrison), WRF single-moment 5-class schemes (WSM5), and WRF single-moment 6-class scheme (WSM6). Details on the physical parameterization packages refer to Skamarock et al. (2008) and references mentioned in section 2.

Table 2

e. Observations

This study utilized two radar-derived products, including column maximum of three-dimensional mosaic reflectivity (MREF; Zhang et al. 2005) and quantitative precipitation estimation (QPE; Fig. 2). The radar observation used a hybrid-scan strategy and quality control procedures described in (Chang et al. 2009). The QPE product was adopted from the Quantitative Precipitation Estimation and Segregation Using Multiple Sensors (QPESUMS) system (Chang et al. 2021). This QPESUMS QPE is derived by single-polarization and dual-polarization relations explicitly designed for the Taiwan area with additional gauge correction. Since MREF and QPE were issued on the latitude–longitude coordinate system with a horizontal resolution of 0.0125°, they were remapped onto the 2-km model grid using cubic spline interpolation to compare with the model forecasts. The verification compared model precipitation to QPE and model reflectivity to MREF over Taiwan island and the surrounding ocean (as indicated by the dashed box in Fig. 5) in a temporal resolution of 1 h.

Fig. 5.
Fig. 5.

The distribution of surface stations (black dots) and the height of 2-km resolution model terrain (m; shaded). Rainfall and radar verification were computed over Taiwan and the surrounding ocean (indicated by the black dashed box).

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

Over 300 conventional and automated surface stations distributed as Fig. 5 were used to evaluate model near-surface forecasts. Verified variables included 10-m wind speed, 2-m temperature, and 2-m water vapor mixing ratio. The verification compared observations to forecasts of the nearest model grid point, excluding grid points identified as ocean points. The model temperature was corrected using a lapse rate of 0.65°C (100 m)−1 if the difference between model terrain and station height is within 150 m (otherwise excluded). Besides, this study utilized ECMWF Reanalysis v5 (ERA5; Hersbach et al. 2020) issued hourly on a 0.25° latitude–longitude coordinate system. The upper-air verification was performed on 15 pressure levels (100, 200, 300, 400, 500, 550, 600, 650, 700, 750, 800, 850, 900, 950, and 1000 hPa). Verified variables included u and v components of wind, relative humidity, and temperature. To match the two model grids, CEPS forecasts were remapped from the 2-km grid to the ERA5 grid by selecting the nearest grid points. The verification included only ocean grid points for model levels below 700 hPa to deal with terrain mismatch between the two resolutions. For model levels higher than 700 hPa, all grid points except those located below the 2-km terrain were considered.

3. Evaluation of upper-air and near-surface forecasts

a. Evaluation of upper-air forecasts

According to Leith (1974), an accurate ensemble mean and an ensemble spread that can provide a good measure of forecast uncertainty are two objective goals for EPSs. A reliable EPS is expected to have a smaller (larger) ensemble spread implying a higher (lower) forecast accuracy. This spread–error relation is commonly quantified by degrees of agreement between root-mean-square error (RMSE) of the ensemble mean and ensemble spread. To assess the domain-wide forecast skill, three-dimensional wind, temperature, and relative humidity forecasts were verified against ECMWF ERA5 (Fig. 6). First, ICP has the lowest RMSEs for temperature, relative humidity, and wind fields within roughly 6-, 6-, and 3-h forecasts, respectively (Fig. 6a). Although BCP has the largest temperature and relative humidity errors at the early forecast hours, perturbing lateral boundaries reduces RMSEs of the ensemble mean after 3–6-h forecasts. It is important to note that the difference between the four experiments is not significant as the confidence intervals are quite wide, likely due to the relatively small sample size used in this study. To sum up, the upper-air verification shows that IBP and IBP_PHY have overall higher forecast accuracy but do not reach statistical significance.

Fig. 6.
Fig. 6.

(a) Root-mean-square error (RMSE) of the ensemble mean and (b) ensemble spread of temperature (K), relative humidity (%), u and υ component of winds (m s−1) averaged over three height levels (100–500, 550–750, and 800–1000 hPa). The model forecasts are verified against ECMWF ERA5. Experiments are shown in different colors, and each has a total of 15 cases. The upper and lower bounds of 95% confidence intervals are marked by dots.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

Second, ICP has the overall smallest ensemble spreads for all variables compared with the other three experiments (Fig. 6b). Its ensemble spread decreases from the initial time as forecast length increases, indicating a significant deficiency in forecast error growth if the perturbations arise only from the current LETKF ensemble analyses. In contrast, BCP has an initial zero and a fast-growing ensemble spread, which grows from the initial time up to 18-h forecasts. The relative impact of BCP compared with ICP can depend on domain size, synoptic weather regimes, and ensemble configurations. For current CEPS and for ATs, the spread of BCP exceeds ICP within 3–6 h with a most significant impact on the upper levels (100–500 hPa). Compared to RMSEs, the difference in ensemble spread among the four experiments is more significant. ICP has spread–skill consistency higher than BCP within about 3–6-h forecasts. After that, the impact of boundary perturbations outperforms initial perturbations due to the rather small computing domain. Since initial perturbations and boundary perturbations can compensate for each other at different forecast hours, IBP has better ensemble spreads considering all forecast hours. Based on IBP, additional multiphysics (IBP_PHY) has a relatively small impact on the domain-wide spread. As a result, IBP and IBP_PHY outperform ICP and BCP since they are more reliable over the entire forecast hours. Their spread–error ratios grow from initially 0.3–0.55 to 0.9–1.1 at the 18-h forecast depending on different variables and height levels. To conclude, the upper-air verification points out that the impact of boundary perturbations exceeds initial perturbations after 3–6-h forecasts in the CWB operational convective-scale domain. Although sampling both initial and boundary perturbations improves the overall spread–skill consistency, the underdispersive forecasts at short lead times emphasize that the initial perturbations should be further improved due to limited error growth time for boundary and model perturbations.

b. Evaluation of near-surface forecasts

In this study, model diagnostic 10-m wind, 2-m temperature, and 2-m water vapor mixing ratio were verified against surface observations (Fig. 7). Since surface heating, local circulation, storm-produced cold pool, and outflow boundaries are associated with the storm evolution, the model near-surface forecasts are an informative index to assess the precipitation forecasts. It is noted that 10-m wind, 2-m wind, and 2-m temperature are diagnostic variables and can be influenced by the use of different PBL and land surface schemes. As shown in Figs. 7g–i, ICP has larger spreads than BCP within the first 6-h wind, 10-h temperature, and 10-h water vapor forecasts. After that, BCP outweighs ICP for all variables. In general, ICP and BCP have comparable RMSEs, except BCP has larger wind errors within 10-h forecasts (Fig. 7a). These results imply that initial conditions dominate short lead-time (∼10 h) near-surface forecasts over Taiwan. It should be noted that the differences in ensemble spread between experiments are significant with a 95% confidence level, while the differences in RMSE are not. Compared with ICP and BCP, IBP has a higher spread–skill consistency due to more consistent ensemble spreads at most forecast hours for all surface variables. Here, the increasing RMSEs in short forecast hours and decreasing spreads in long forecast hours are partly influenced by the diurnal variation of surface variables.

Fig. 7.
Fig. 7.

Root-mean-square error (RMSE), mean error (ME), and ensemble spread of (a),(d),(g) 10-m wind (m s−1); (b),(e),(h) 2-m temperature (K); and (c),(f),(i) 2-m water vapor mixing ratio (g kg−1) forecasts against surface observations. Experiments are shown in different colors, and each has a total of 15 cases. The upper and lower bounds of 95% confidence intervals are marked by dots.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

Although IBP_PHY has a negligible impact on the upper-air spreads (Fig. 6b), it increases the spread and alters the accuracy of 10-m wind speed and 2-m water vapor (Fig. 7). For these two variables, IBP_PHY has RMSE, ME, and spread different from others with 95% confidence level. Compared with IBP, IBP_PHY improves the 2-m water vapor forecast by reducing the dry bias (Figs. 7c,f) but deteriorates the 10-m wind forecast due to overpredicted wind speed (Figs. 7a,d). Since IBP and IBP_PHY used identical initial and lateral boundary conditions, the difference originated purely from multiphysics schemes. To explore the sources of errors, the RMSE differences between IBP and IBP_PHY members were computed (IBP_PHY-IBP) and averaged by their PBL and microphysics schemes.

As shown in Fig. 8a, the green bar denoted as YSU is the RMSE difference averaged from four YSU members in IBP_PHY (m1, m6, m11, and m16). Since they all use the YSU PBL scheme, it points out that multimicrophysics schemes slightly reduce wind speed errors compared with a single microphysics scheme (IBP). The blue bar denoted as Goddard (Fig. 8c) points out that Goddard with multi-PBL schemes performs worse than Goddard with a single-PBL scheme, degrading the 10-m wind ensemble mean forecast accuracy in IBP_PHY (Fig. 7a). In this case, multi-PBL schemes lead to serious wind speed error clustering. The BouLac, MYNN, and MYNN3 members cause larger errors than the YSU and Shin–Hong members (Fig. 8a). For the 2-m water vapor forecast, the reduced RMSE in the YSU members points out that YSU with multiphysics schemes outperforms YSU with a single Goddard scheme (Fig. 8b). Indeed, Morrison, WSM5, and WSM6 members have substantially smaller RMSEs than Goddard members (Fig. 8d). It is not a desirable EPS designation since the probability of possible outcomes is usually computed with a hypothesis that each ensemble member is equally likely (Ziehmann 2000). To generate an equally likely ensemble, the CWB has ongoing tasks to develop parameter perturbations in the YSU scheme because of its statistically higher accuracy over Taiwan.

Fig. 8.
Fig. 8.

RMSE differences (RMSED) between IBP and IBP_PHY (IBP_PHY-IBP). (a) RMSED of 10-m wind forecasts (m s−1) grouped by PBL schemes, (b) RMSED of 2-m water vapor maxing ratio forecasts (g kg−1) grouped by PBL schemes, (c) RMSED of 10-m wind forecasts (m) grouped by microphysics schemes, and (d) RMSED of 2-m water vapor forecasts (g) grouped by microphysics schemes. Each verification has a total of 15 cases.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

4. Evaluation of precipitation forecasts for AT events

According to Chen et al. (2020), the forecast lead time is an indicative factor for the model deterministic QPF skills for ATs. Specifically, QPFs initialized each hour from 0800 to 1300 LST generally have increasing deterministic QPF skills as lead time decreases. The increasing forecast skills come from more accurate initial conditions through an hourly update surface and radar variational data assimilation system. Since the CEPS used initial conditions from the ensemble-based LETKF radar and surface data assimilation system, we first examined the capability of its analysis members and analysis mean in predicting the AT precipitation. This study aims to assess 1) the impact of different ensemble configurations and initial times (0500, 0800, and 1100 LST) on model deterministic QPF skills and 2) whether more skillful ensemble members can produce a more skillful probabilistic forecast for ATs. This study verified 7-h accumulated rainfall between 1300 and 2000 LST to evaluate precipitation associated with AT events.

a. Evaluation of model deterministic QPF skills

When verifying high-resolution forecasts, traditional gridpoint verifications often lead to low forecast skills even though the forecast storms are realistic. Accordingly, this study selected the fractional skill score (FSS; Roberts and Lean 2008) to verify the model QPF skills with a tolerable location error of an 8-km radius. The FSS score ranges from 0 (implying a no-skill forecast) to 1 (implying a perfect forecast). The formulation of FSS is described in section 4b. As shown in Fig. 9a, LETKF ensemble mean forecasts (annotated as LETKF_D) initialized at 0500 and 0800 LST have comparable FSSs, which is reasonable since the effect of surface data assimilation is more distinct during the late morning (Chen et al. 2020). Consistent with the 3DVAR system (Chen et al. 2020), the LETKF_D initialized at 1100 LST has the overall highest deterministic QPF skill (Fig. 9a), especially for precipitation thresholds larger than 25 mm. These highest QPF skills at 1100 LST are credible when considering individual AT cases, except those on 2 July 2017 (Fig. 9d).

Fig. 9.
Fig. 9.

Fractions skill score (FSS) of LETKF ensemble mean forecasts against the QPESUMS QPE. The verification included three initial times at 0500 (black line), 0800 (blue line), and 1100 (red line) LST each day from 30 Jun to 4 Jul 2017. To focus on afternoon thunderstorm events, verification only considered precipitation from 1300 to 2000 LST.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

Since ensemble forecasts construct the probabilistic density function (PDF) from ensemble members, the behaviors of individual members were first evaluated. Figure 10 includes FSSs of CEPS members and FSSs of the LETKF_D at different initial times. For ICP, the only difference compared with LETKF_D is that it used analysis members instead of the analysis mean as initial conditions. According to Figs. 10a1–a3, ICP members generally have lower FSSs than LETKF_D, especially at short lead times. In other words, the mean of ensemble analyses is able to produce more accurate deterministic QPFs than most of the analysis members in the current convective-scale LETKF system. For BCP, the only difference compared with LETKF_D is the additional lateral boundary perturbations. As shown in Fig. 10b1, the FSSs of BCP members are comparable to the LETKF_D, indicating boundary perturbations have a relatively small effect on the deterministic QPFs launched at 1100 LST in terms of 7-h accumulated rainfall from 1300 to 2000 LST. Since earlier initial times provide longer error growth times, FSS differences between BCP members and LETKF_D also increase from 1100 to 0500 LST.

Fig. 10.
Fig. 10.

FSSs (40-mm threshold) of LETKF ensemble mean (annotated as LETKF_D and green dash lines) and FSSs of CEPS ensemble members (gray bars) against the QPESUMS QPE. Columns are different experiments, and rows are initial times. Each verification consists of five cases from 30 Jun to 4 Jul 2017 and considered precipitation accumulated from 1300 to 2000 LST. For ICP, BCP, and IBP_PHY, the blue bars are the FSSs difference against IBP at the same initial time. For 0500 and 0800 LST, the yellow bars are the FSSs difference against forecasts initialized at 1100 LST for each experiment.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

It should be noted that FSSs of different forecast lead times are influenced by initial condition accuracy, boundary condition accuracy, and model errors. Here, the red bars show the FSS differences of each member against itself initialized at 1100 LST to measure the lead time effect. For the 40-mm threshold, the FSSs of LETKF_D decrease from 1100 to 0500 LST. In general, the FSSs of ensemble members decrease from 1100 to 0500 LST for all experiments. To conclude, ensemble members have the highest deterministic QPF skills at 1100 LST and lowest QPF skills at 0500 LST no matter which perturbation is used.

For the same initial time, FSSs of ICP, BCP, and IBP_PHY were compared to FSSs of IBP (blue bars in Fig. 10). These FSS differences measure the effect of different ensemble configurations on the deterministic QPF skills of ensemble members. First, the differences between ICP and IBP increase as the forecast length increases due to the growth of boundary perturbations in IBP. In general, sampling boundary perturbations have a neutral effect on the QPF skills of the ensemble members since there are no systematic differences between ICP and IBP. On the contrary, BCP consists of more skillful members than IBP, especially for forecasts initialized at 1100 LST. Again, this result indicates that the analysis mean outperforms analysis members on deterministic QPF skills for ATs. On the whole, the deterministic QPF skills of ensemble members are 1100 LST > 0800 LST > 0500 LST for all experiments, pointing out the importance of short forecast lead times for AT prediction. Among all, BCP has the overall highest deterministic QPF skills, indicating that the LETKF analysis mean can provide more accurate initial conditions than analysis members.

b. Evaluation of model probabilistic QPF skills

The performance of a probabilistic forecast depends on how well ensemble members represent the distribution of a forecasting variable. As shown above, different ensemble configurations and lead times alter the deterministic FSSs of ensemble members. To evaluate the resulting rainfall PDFs, we compare the probabilistic QPF skills among ensemble configurations and initial times. For ATs, it is unproven if a short lead-time ensemble (e.g., 1100 LST) has higher probabilistic QPF scores than a long lead-time ensemble (e.g., 0500 LST) because of its more skillful members when they are evaluated as deterministic forecasts.

1) Verification matrices

This study utilized the neighborhood maximum ensemble probability (NMEP; Schwartz and Sobash 2017) method to verify probabilistic QPF. In NMEP, the probability of exceedance considered grid points within a radius of 8 km (about 2 model grids around verification points in the CEPS domain). The binary classification of events and nonevents used absolute rainfall thresholds including 0, 5, 10, 20, and 50 mm for 7-h accumulated precipitation. The forecast skill was quantified by the relative operating characteristic (ROC; Mason and Graham 2002) and the fractional skill score (FSS; Roberts and Lean 2008). The ROC measures the forecast skills to discriminate events and nonevents conditioned on the observations by computing the probability of detection (POD) and false alarm rate (FAR). It does not measure forecast reliability and is relatively insensitive to forecast bias. In this study, the total area under the ROC curve is examined (AUC, higher is better). Generally, an AUC of 0.5, 0.7, and 1.0 indicates a no-skill, useful, and perfect forecast, respectively.

Second, the FSS measures the forecast reliability by comparing the forecast probability to the observed probability and is defined as
FSS=11NN(PfcPo)21N(NPfc2+nPo2)=1FBSFBSworse,
where Pfc, Po, and N are the probability of model forecast, probability of observation, and the number of points in the verification domain, respectively. The FBS provides a measure similar to the Brier score (BS; Brier 1950), which quantifies the accuracy of forecast probability by its square difference against observed probability. An FBS of 0 means that the forecast probability matches the observed probability at all grid points. The FBSworse represents the worst possible forecast when there are no overlaps between nonzero forecasts and observed probabilities. As pointed out by Mittermaier and Roberts (2010), forecast bias can significantly influence FSS values. A more biased forecast generally gives lower FSSs and can possibly produce higher FSSs at small scales.

2) Results

As shown in Fig. 11, probabilistic QPFs initialized at 0500, 0800, and 1100 LST have comparable FSS despite FSS differences for ensemble members between initial times (Fig. 10). That is, more skillful ensemble member does not lead to more skillful probabilistic forecasts for the five ATs. In general, the four experiments have comparable FSSs at all lead times. For probabilistic QPFs initialized at 1100 LST, ICP alone produces comparable FSSs with IBP and IBP_PHY, indicating additional boundary perturbation and model perturbation have no significant impact at short lead time. Without initial perturbations, BCP has slightly lower FSSs than other experiments at 1100 UTC due to higher FBSs and the largest FBSworse. Although the difference is not statistically significant as the 95% confidence intervals overlap, it still informs that BCP does not have higher probabilistic QPF skills for the five ATs, although it has more skillful members than other experiments (blue bars in Fig. 10). One possible explanation is that BCP initialized at 1100 UTC has smaller precipitation spreads compared to other experiments. This inference can be supported by its largest FBSworse at 1100 LST, which comes from its biased forecast probability since observed probability holds constants for all experiments (Fig. 11c). This explanation is also conceptually reasonable since BCP lacks initial perturbations and has limited error growth time for boundary perturbations. For probabilistic QPFs initialized at 0500 LST, BCP has comparable FSS, FBS, and FBSworse with other experiments. At the longer lead time, ICP alone has slightly lower FSS, higher FBS, and higher FBSworse, implying a collapse of rainfall spreads due to lacking lateral boundary perturbations. Although the differences are not statistically significant due to the small sample size, the results suggest that combining initial and boundary perturbations are essential to guarantee reliable probabilistic QPFs at different lead times.

Fig. 11.
Fig. 11.

FSS, FBS, FBS_worse, and AUC scores for ensemble forecasts initialized at (a) 0500, (b) 0800, and (c) 1100 LST. The FSS and NMEP neighborhood both used a radius of 8 km. The binary classification thresholds included 0, 5, 10, 20, and 50 mm, and the rainfall is accumulated from 1300 to 2000 LST. Experiments are shown in different colors, and each line consists of five different ATs from 30 Jun to 4 Jul 2017. The upper and lower bounds of the 95% confidence interval are marked by dots.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

Although it is possible to improve the reliability of probabilistic forecasts through statistical calibration methods (Johnson and Bowler 2009; Flowerdew 2014), it is difficult to correct the spatial structure and improve the discrimination skill (e.g., AUC). As shown in Fig. 11, all experiments and lead times have AUC greater than 0.7, indicating useful forecasts in an ensemble framework. Comparing different initial times, BCP has AUC increased from 1100 to 0500 LST, implying that growing lateral boundary perturbation improves the discrimination skill. Its lowest AUC at 1100 LST again indicates the critical role of initial perturbations at short lead times. In general, ICP and BCP contribute to the shorter and longer lead-time AUC, respectively. Therefore, IBP can produce optimal AUC when all initial times are considered. Similar to previous conclusions, multiphysics has a small impact at all lead times.

Although this study has a major shortcoming of limited sample size, the results still identify that forecast lead times have different effects on deterministic and probabilistic QPF skills. For deterministic QPFs, shorter lead times improve FSS due to more accurate initial conditions from the hourly updated LETKF system. In contrast, a short lead time does not provide better probabilistic QPF skills for the five ATs, primarily due to insufficient ensemble spreads. To conclude, the multiphysics schemes have little impact, and boundary perturbations require sufficient time to propagate into the inner domain. As a result, the initial perturbations largely dominate probabilistic QPFs at short lead times. For the CEPS, this issue is particularly important since we aim to take advantage of the most recent initial conditions that can improve the prediction of ATs (Chen et al. 2020 and section 4a). Also, this study suggests that it is important to improve initial condition uncertainty by, for example, sampling lower boundary and model perturbations during the data assimilation cycles.

5. Evaluating the prediction of hourly radar echoes

a. Brier score

In previous studies (Chen et al. 2020; Jiang et al. 2021), the assessment of convective-scale precipitation systems largely focused on model QPF skills, such as 3- or 6-h accumulated rainfall. In this study, we aim to explore the capability of the CWB CEPS to predict the hourly evolution of convective echoes. The model column maximum reflectivity (MCMR) was compared to radar MREF observations at each hour (Fig. 12). The Brier score (BS) for binary outcomes classified by 20 and 40 dBZ were used to evaluate the probabilistic forecast skills for medium and strong echoes, respectively. In general, the reliability of probabilistic reflectivity forecasts increases from 0500 to 1100 LST, especially for the 40-dBZ threshold. This decreasing reliability exists in all experiments, no matter which perturbations are included. To clarify the systematic impact of lead times on reflectivity forecasts, the behavior of convection in different forecasts was explored. The following discussion focuses on IBP_PHY since it has the lowest BSs for the five ATs.

Fig. 12.
Fig. 12.

Brier scores for 20- and 40-dBZ column maximum radar reflectivity against radar MREF observations over Taiwan and surroundings (as Fig. 5) in the function of valid time (LST). Experiments are shown in different colors, and each line consists of five different ATs from 30 Jun to 4 Jul 2017.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

b. Time phase of strong (40 dBZ) convective cells

A correct convection prediction includes the precise location, size, timing, and intensity of convective cells. In this study, our goal is to evaluate if the CEPS can capture the correct time phase of convective cells. The assessment of forecast MCMR was designed as the following steps to explore the natural cell behavior: 1) Two groups of 40-dBZ MCMR were identified: the first with sizes larger than 50 km2 and a second with sizes larger than 250 km2 in hourly model output between 1100 and 2000 LST (e.g., each ensemble member and valid hour). 2) The total cell numbers at each hour were normalized by the total cell number counted in each 16-member ensemble run. That is, the intensity differences were neglected in order to extract the time phase of convective cells and enable a clear comparison between different datasets (e.g., observed and model time phase). For example, although not discussed in the present study, it is possible that the 16-member ensemble forecasts initialized at 1100 LST have double cell numbers compared to those initialized at 0500 LST. After all, the present assessment was designed to understand the convection behavior in observations and forecasts launched at 0500, 0800, and 1100 LST.

As shown in Fig. 13, the observed convective cells greater than 50 km2 generally have two peaks. The first peak occurred during 1400–1600 LST, and the second peak occurred after 1800 LST. Currently, the CEPS ensemble members cover the first peak better and miss the second peak for almost all initial times. This result implies that the CEPS forecasts have convective cells dissipate too early compared with observations. Especially for AT cases on the first three days, the two groups (>50 and >250 km2) have cells occurred 1–2 h earlier in forecasts initialized at 1100 LST (green lines) compared to those at 0500 LST (red lines). It should be mentioned that, regardless of ensemble configurations, all experiments have the early initiated convective cells in the 1100 LST runs (not shown). This systematic time phase error is partly responsible for the large reflectivity BSs at 1100 LST (Fig. 12). Additional sensitivity tests prove that the early-triggering cells are caused by higher temperatures in 1100 LST initial conditions (not shown). As mentioned in Figs. 9 and 10, the deterministic forecasts initialized at 1100 LST generally have the highest FSSs for 7-h accumulated rainfall than 0500 and 0800 LST. That is, the LETKF analyses at 1100 LST produce the most skillful rainfall forecasts due to hourly updated radar and surface data assimilation in the morning. However, forecasts initialized at 1100 LST have significant phase errors in the time evolution of convection, suggesting that the LETKF analyses should be further tuned and assessed more thoroughly in order to predict the convective cells at the correct hours.

Fig. 13.
Fig. 13.

The forecast distribution of convective cells in the function of valid time (LST) for each AT event. Convective cells are counted if the area of 40-dBZ column maximum radar reflectivity is larger than (left) 50 and (right) 250 km2, respectively. The number of convective cells is normalized by the total number of cells for each ensemble forecast. That is, each line has a total area of 1. Experiments are shown in different colors, and each line consists of a single 16-member ensemble forecast.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

c. Case illustration

The AT case on 30 June 2017 was selected to present the current CEPS performance in predicting the hourly AT evolution. As shown in Fig. 14, the initial convective echoes were triggered over mountain areas at 1200 LST (marked as A). Afterward, several convective cells developed rapidly between 1300 and 1400 LST (marked as B and C). During 1500–1700 LST, convective cells moved downslope and reached the mature stage (marked as D). After 1700 LST, the systems move offshore with the second intensification at 2000 LST.

Fig. 14.
Fig. 14.

Column maximum of composite radar reflectivity observations (dBZ; shaded) from 1100 to 2200 LST 30 Jun 2017. The 750-m height terrain is shown in black contour lines.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

This study identified the time stamp of the first 20-dBZ MCMR echoes at all grid points in each ensemble member. Then, the ensemble mean time stamp was derived and compared to the observed time stamp (Fig. 15). It is encouraging that the observed time stamp has a similar spatial structure to the ensemble mean time stamps of all experiments and lead times. It indicates that the CEPS potentially can predict the initiation and movement of convective cells. Consistent with the statistical results in Fig. 13, the early-initiated radar echoes over mountains are more severe in forecasts launched at 1100 LST than at 0500 and 0800 LST. This early initiation occurred in all experiments, suggesting that initial conditions play a critical role.

Fig. 15.
Fig. 15.

Ensemble mean time stamp of first occurred 20-dBZ column maximum radar reflectivity (LST; shaded). The case shown here is AT event on 30 Jun 2017. Columns are time stamps of radar observations and different experiments. Rows are time stamps of different initial times.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

In addition, this study examined the forecast probability of 20-dBZ MCMR echoes valid at 1400 and 1900 LST 30 June 2017 (Figs. 16 and 17). For this case, the CEPS predicts the observed MREF echoes better in the mature stage (not shown) than in the initiation stage (as illustrated in Fig. 16) and dissipation stage (as illustrated in Fig. 17). In general, ensemble forecasts initialized at 0500 LST perform better than those initialized at 1100 LST although it still triggers convection over mountains earlier than observations (Fig. 16). As shown in Fig. 17, the forecasts initialized at 1100 LST also have more severe early-dissipated 20-dBZ MCMR, which mirrors the statistical results in Figs. 13 and 15. Except for BCP, which has significant underdispersive forecasts at short lead times, the probabilistic forecasts are similar among the ensemble experiments. In particular, all experiments initialized at 1100 UTC have the early initiation and dissipation forecast errors, suggesting that future studies should address the time phase error for forecasts launched in the late morning (e.g., 1100 LST). For instance, improving the temperature analysis in the current hourly updated LETKF system is necessary. In particular, it is important to assess the data assimilation analysis more comprehensively. For example, forecasts initialized at 1100 LST have the highest 7-h QPF skill. However, the convection is triggered earlier than observations and other forecast runs.

Fig. 16.
Fig. 16.

Forecast probability (%; shaded) of 20-dBZ column maximum reflectivity valid at 1400 LST 30 Jun 2017. Columns and rows are results of ensemble experiments and results of different initial times, respectively. Black contours show 20-dBZ radar observations valid at 1400 LST 30 Jun 2017.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

Fig. 17.
Fig. 17.

As in Fig. 16, but with valid time at 1900 LST 30 Jun 2017.

Citation: Weather and Forecasting 38, 4; 10.1175/WAF-D-22-0082.1

6. Summary and discussion

Recently, the Central Weather Bureau of Taiwan has developed a convective-scale ensemble prediction system (CEPS) to provide short-range probabilistic forecasts for convection. The CEPS used initial conditions from an hourly updated high-resolution LETKF system to enhance practical predictability. The lateral boundary perturbations were downscaled from a coarser-resolution EPS, and the model perturbations were represented by multiphysics schemes. The CWB CEPS has an initial objective to provide operational forecast guidance for warm-season afternoon thunderstorms (ATs) over Taiwan island. Since the rapid-evolving ATs can produce hazards within a short time, there are strong demands for timely and accurate forecast products. This study aims to explore the behavior of different CEPS configurations using AT events from 30 June to 4 July 2017. Our findings are summarized as follows.

a. Impact of initial, lateral boundary, and model perturbations on the CEPS

The current CWB operational convective-scale model has a 900 km × 900 km domain size (Chen et al. 2020; Jiang et al. 2021). In this domain, initial perturbations and boundary perturbations dominate the upper-air forecasts in the first 3–6 h and afterward, respectively. Initial perturbations alone lead to decreasing ensemble spreads as forecast hours increase, emphasizing the importance of lateral boundary perturbations in limited-area ensemble systems. Based on initial and boundary perturbations, additional multiphysics has a negligible impact. In this study, there are no significant differences in upper-air forecast errors between experiments. For near-surface forecasts over Taiwan island, the impact of the initial perturbation is larger than the boundary perturbation in the first 6–10-h forecasts. In contrast to its negligible effect on upper-air fields, multiphysics alters the forecast accuracy of the ensemble mean and increases the spread of 10-m wind and 2-m water vapor. Compared with single-physics forecasts, multiphysics perturbations improve the 2-m water vapor forecasts by reducing dry biases and degrade the 10-m wind forecasts due to overpredicted wind speed. To sum up, the results suggest that improving initial and boundary perturbations should be prioritized during the development of a limited-area ensemble, in particular for those targeted at short lead times. Compared to initial and boundary perturbations, additional model perturbations have insignificant effects on the CWB CEPS. In this study, different PBL and microphysics schemes lead to error clustering near the surface. To generate equally likely members, suitable model perturbation schemes based on the single physics suite are required. This development could be resource demanding since each operational center has its optimal physic suites. For example, there is ongoing work at the CWB to develop parameter perturbations in the YSU PBL scheme.

b. Impact of ensemble configuration and lead times on the deterministic and probabilistic QPF skills

Although each perturbation has distinct error growth behavior, the difference in probabilistic QPF skills between experiments is small and might not have practical importance. Despite that, verification results from the five AT cases indicate that IBP has overall better probabilistic QPF scores since ICP and BCP can compensate for each other at different forecast hours. Based on IBP, additional multiphysics perturbations have minor effects. For deterministic forecasts (LETKF_D and individual members), a shorter lead time generally provides more skillful QPFs for ATs since the initial conditions are corrected by the most recent observations (Chen et al. 2020). However, a short lead-time forecast with a higher deterministic QPF skill does not necessarily improve the probabilistic QPF skills due to inadequate rainfall spreads. Since there is limited error growth time for boundary and model perturbations, generating sufficient rainfall spreads from initial perturbations is the key to improving probabilistic QPFs at short lead times. Accordingly, this study suggests more resources should be invested to improve initial condition uncertainty. It can be achieved by, for example, improving the model, lateral boundary, and lower boundary perturbations in the data assimilation cycles, which is cross-domain research since it involves data assimilation and model expertise.

c. Current CEPS capability in predicting the hourly evolution of convective echoes

For the five ATs, the time phase of strong (40 dBZ) convective cells generally has two peaks: 1400–1600 LST and after 1800 LST. Current CEPS ensemble members are able to cover the first peak. However, the second peak is missed in almost all forecasts, which implies that the model cannot capture the second intensification of convective cells. In general, all experiments have forecast reliability decreases from 0500 to 1100 LST, no matter which perturbations are included. Ensemble forecasts launched at 1100 LST have convective cells initiated and dissipated earlier than those launched at 0500 and 0800 LST. This problem is caused by higher temperatures in initial conditions (not shown). Although analyses at 1100 LST produce the highest deterministic QPF scores, it leads to systematic time phase error in all experiments. This result emphasizes the need for accurate initial conditions that are evaluated by more comprehensive metrics in addition to rainfall verification (e.g., the impact of data assimilation on the predicted hourly convective cells).

This study provides a first assessment of the CWB CEPS using five continuous AT events. Since the behavior of different perturbations is also influenced by the configuration of ensemble prediction systems and flow regimes, the above findings should not be taken as general conclusions due to the limited cases and their potentially correlated synoptic condition. In the future, it is crucial to investigate the behavior of the CEPS under different synoptic regimes. Since initial and boundary perturbations collectively dominate limited-area short-range forecasts and the model perturbations can only have minor effects. In particular for small computing domains such as the 2-km domain used in this study, this study suggests evaluating model perturbation schemes in data assimilation cycles in addition to the CEPS itself. Besides, this study demonstrates that the multiphysics perturbation leads to significant error clustering and is exceptionally unsuitable to be applied in data assimilation cycles. Therefore, it is conducive to assessing the performance of different stochastic schemes (Romine et al. 2014; Jankov et al. 2017; Li et al. 2020; Berner et al. 2011, 2016) in high-resolution data assimilation systems. To provide accurate and informative probabilistic forecasts that meet operational purposes, Demuth et al. (2020) pointed out the importance of developing convection-related products that meet the forecasters’ needs during the development of convection-allowing ensembles.

Acknowledgments.

This research was supported by the Central Weather Bureau (CWB) of Taiwan and the Ministry of Science and Technology of Taiwan under Grant MOST 110-2625-M-052-001. We thank the Meteorology Information Center at CWB for providing the high-performance computer. The ECMWF Reanalysis v5 hourly dataset was downloaded from the Copernicus Climate Change Service (C3S) Climate Data Store (Hersbach et al. 2018).

Data availability statement.

The outputs of ensemble simulations are too large to archive and transfer. Instead, we archived all the required datasets including the namelist files, initial condition files, and boundary condition files to replicate the ensemble simulations. More information is available from Hsiao-Wei Lai (hlai@cwb.gov.tw) at the Central Weather Bureau of Taiwan.

REFERENCES

  • Bachmann, K., C. Keil, and M. Weissmann, 2018: Impact of radar data assimilation and orography on predictability of deep convection. Quart. J. Roy. Meteor. Soc., 145, 117130, https://doi.org/10.1002/qj.3412.

    • Search Google Scholar
    • Export Citation
  • Berner, J., S.-Y. Ha, J. P. Hacker, A. Fournier, and C. Snyder, 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 19721995, https://doi.org/10.1175/2010MWR3595.1.

    • Search Google Scholar
    • Export Citation
  • Berner, J., and Coauthors, 2016: Stochastic parameterization: Toward a new view of weather and climate models. Bull. Amer. Meteor. Soc., 98, 565588, https://doi.org/10.1175/BAMS-D-15-00268.1.

    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and P. Lacarrere, 1989: Parameterization of orography-induced turbulence in a mesobeta-scale model. Mon. Wea. Rev., 117, 18721890, https://doi.org/10.1175/1520-0493(1989)117<1872:POOITI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Bouttier, F., B. Vié, O. Nuissier, and L. Raynaud, 2012: Impact of stochastic physics in a convection-permitting ensemble. Mon. Wea. Rev., 140, 37063721, https://doi.org/10.1175/MWR-D-12-00031.1.

    • Search Google Scholar
    • Export Citation
  • Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78, 13, https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Chang, P.-L., P.-F. Lin, B. J.-D. Jou, and J. Zhang, 2009: An application of reflectivity climatology in constructing radar hybrid scans over complex terrain. J. Atmos. Oceanic Technol., 26, 13151327, https://doi.org/10.1175/2009JTECHA1162.1.

    • Search Google Scholar
    • Export Citation
  • Chang, P.-L., and Coauthors, 2021: An operational multi-radar multi-sensor QPE system in Taiwan. Bull. Amer. Meteor. Soc., 102, E555E577, https://doi.org/10.1175/BAMS-D-20-0043.1.

    • Search Google Scholar
    • Export Citation
  • Chen, I. H., J. S. Hong, Y. T. Tsai, and C. T. Fong, 2020: Improving afternoon thunderstorm prediction over Taiwan through 3DVAR-based radar and surface data assimilation. Wea. Forecasting, 35, 26032620, https://doi.org/10.1175/WAF-D-20-0037.1.

    • Search Google Scholar
    • Export Citation
  • Clark, A., and Coauthors, 2019: Spring forecasting experiment 2019: Preliminary findings and results. NOAA Hazardous Weather Testbed, NOAA, 77 pp., https://hwt.nssl.noaa.gov/sfe/2019/docs/HWT_SFE_2019_Prelim_Findings_FINAL.pdf.

  • Demuth, J. L., and Coauthors, 2020: Recommendations for developing useful and usable convection-allowing model ensemble information for NWS forecasters. Wea. Forecasting, 35, 13811406, https://doi.org/10.1175/WAF-D-19-0108.1.

    • Search Google Scholar
    • Export Citation
  • Flowerdew, J., 2014: Calibrating ensemble reliability whilst preserving spatial structure. Tellus, 66A, 22662, https://doi.org/10.3402/tellusa.v66.22662.

    • Search Google Scholar
    • Export Citation
  • Frogner, I., A. T. Singleton, M. Ø. Køltzow, and U. Andrae, 2019: Convection‐permitting ensembles: Challenges related to their design and use. Quart. J. Roy. Meteor. Soc., 145, 90106, https://doi.org/10.1002/qj.3525.

    • Search Google Scholar
    • Export Citation
  • García-Ortega, E., J. Lorenzana, A. Merino, S. Fernández-González, L. López, and J. L. Sánchez, 2017: Performance of multiphysics ensembles in convective precipitation events over northeastern Spain. Atmos. Res., 190, 5567, https://doi.org/10.1016/j.atmosres.2017.02.009.

    • Search Google Scholar
    • Export Citation
  • Gebhardt, C., S. E. Theis, M. Paulat, and Z. B. Bouallègue, 2011: Uncertainties in COSMO-DE precipitation forecasts introduced by model perturbations and variation of lateral boundaries. Atmos. Res., 100, 168177, https://doi.org/10.1016/j.atmosres.2010.12.008.

    • Search Google Scholar
    • Export Citation
  • Golding, B., N. Roberts, G. Leoncini, K. Mylne, and R. Swinbank, 2016: MOGREPS-UK convection-permitting ensemble products for surface water flood forecasting: Rationale and first results. J. Hydrometeor., 17, 13831406, https://doi.org/10.1175/JHM-D-15-0083.1.

    • Search Google Scholar
    • Export Citation
  • Guerra, J. E., P. S. Skinner, A. Clark, M. Flora, B. Matilla, K. Knopfmeier, and A. E. Reinhart, 2022: Quantification of NSSL Warn-on-Forecast System accuracy by storm age using object-based verification. Wea. Forecasting, 37, 19731983, https://doi.org/10.1175/WAF-D-22-0043.1.

    • Search Google Scholar
    • Export Citation
  • Hagelin, S., J. Son, R. Swinbank, A. McCabe, N. Roberts, and W. Tennant, 2017: The Met Office convective‐scale ensemble, MOGREPS‐UK. Quart. J. Roy. Meteor. Soc., 143, 28462861, https://doi.org/10.1002/qj.3135.

    • Search Google Scholar
    • Export Citation
  • Harding, K., 2011: Thunderstorm formation and aviation hazards. NOAA/National Weather Service, 1–4.

  • Harnisch, F., and C. Keil, 2015: Initial conditions for convective-scale ensemble forecasting provided by ensemble data assimilation. Mon. Wea. Rev., 143, 15831600, https://doi.org/10.1175/MWR-D-14-00209.1.

    • Search Google Scholar
    • Export Citation
  • Hermoso, A., V. Homar, and R. S. Plant, 2021: Potential of stochastic methods for improving convection-permitting ensemble forecasts of extreme events over the Western Mediterranean. Atmos. Res., 257, 105571, https://doi.org/10.1016/j.atmosres.2021.105571.

    • Search Google Scholar
    • Export Citation
  • Hersbach, H., and Coauthors, 2018: ERA5 hourly data on single levels from 1940 to present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS), accessed 14 April 2021, https://doi.org/10.24381/cds.adbb2d47.

  • Hersbach, H., and Coauthors, 2020: The ERA5 global reanalysis. Quart. J. Roy. Meteor. Soc., 146, 19992049, https://doi.org/10.1002/qj.3803.

    • Search Google Scholar
    • Export Citation
  • Hirt, M., and G. C. Craig, 2021: A cold pool perturbation scheme to improve convective initiation in convection‐permitting models. Quart. J. Roy. Meteor. Soc., 147, 24292447, https://doi.org/10.1002/qj.4032.

    • Search Google Scholar
    • Export Citation
  • Hirt, M., S. Rasp, U. Blahak, and G. C. Craig, 2019: Stochastic parameterization of processes leading to convective initiation in kilometer-scale models. Mon. Wea. Rev., 147, 39173934, https://doi.org/10.1175/MWR-D-19-0060.1.

    • Search Google Scholar
    • Export Citation
  • Hohenegger, C., and C. Schär, 2007: Atmospheric predictability at synoptic versus cloud-resolving scales. Bull. Amer. Meteor. Soc., 88, 17831794, https://doi.org/10.1175/BAMS-88-11-1783.

    • Search Google Scholar
    • Export Citation
  • Hong, S.-Y., and J.-O. J. Lim, 2006: The WRF Single-Moment 6-Class Microphysics Scheme (WSM6). Asia-Pac. J. Atmos. Sci., 42, 129151.

  • Hong, S.-Y., J. Dudhia, and S.-H. Chen, 2004: A revised approach to ice microphysical processes for the bulk parameterization of clouds and precipitation. Mon. Wea. Rev., 132, 103120, https://doi.org/10.1175/1520-0493(2004)132<0103:ARATIM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hong, S.-Y., Y. Noh, and J. Dudhia, 2006: A new vertical diffusion package with an explicit treatment of entrainment processes. Mon. Wea. Rev., 134, 23182341, https://doi.org/10.1175/MWR3199.1.

    • Search Google Scholar
    • Export Citation
  • Hsiao, L.-F., and Coauthors, 2015: Blending of global and regional analyses with a spatial filter: Application to typhoon prediction over the western North Pacific Ocean. Wea. Forecasting, 30, 754770, https://doi.org/10.1175/WAF-D-14-00047.1.

    • Search Google Scholar
    • Export Citation
  • Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112126, https://doi.org/10.1016/j.physd.2006.11.008.

    • Search Google Scholar
    • Export Citation
  • Iacono, M. J., J. S. Delamere, E. J. Mlawer, M. W. Shephard, S. A. Clough, and W. D. Collins, 2008: Radiative forcing by long-lived greenhouse gases: Calculations with the AER radiative transfer models. J. Geophys. Res., 113, D13103, https://doi.org/10.1029/2008JD009944.

    • Search Google Scholar
    • Export Citation
  • Jankov, I., and Coauthors, 2017: A performance comparison between multiphysics and stochastic approaches within a North American RAP ensemble. Mon. Wea. Rev., 145, 11611179, https://doi.org/10.1175/MWR-D-16-0160.1.

    • Search Google Scholar
    • Export Citation
  • Jiang, S.-Y., J.-S. Hong, and B. J.-D. Jou, 2021: Blend global analyses into the cyclic convective-scale data assimilation system to improve short-term QPF. Terr. Atmos. Oceanic Sci., 32, 633647, https://doi.org/10.3319/TAO.2021.09.13.01.

    • Search Google Scholar
    • Export Citation
  • Johnson, C., and N. Bowler, 2009: On the reliability and calibration of ensemble forecasts. Mon. Wea. Rev., 137, 17171720, https://doi.org/10.1175/2009MWR2715.1.

    • Search Google Scholar
    • Export Citation
  • Kalina, E. A., I. Jankov, T. Alcott, J. Olson, J. Beck, J. Berner, D. Dowell, and C. Alexander, 2021: A progress report on the development of the High-Resolution Rapid Refresh ensemble. Wea. Forecasting, 36, 791804, https://doi.org/10.1175/WAF-D-20-0098.1.

    • Search Google Scholar
    • Export Citation
  • Kühnlein, C., C. Keil, G. C. Craig, and C. Gebhardt, 2014: The impact of downscaled initial condition perturbations on convective‐scale ensemble forecasts of precipitation. Quart. J. Roy. Meteor. Soc., 140, 15521562, https://doi.org/10.1002/qj.2238.

    • Search Google Scholar
    • Export Citation
  • Leith, C. E., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102, 409418, https://doi.org/10.1175/1520-0493(1974)102<0409:TSOMCF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Li, C.-H., J. Berner, J.-S. Hong, C.-T. Fong, and Y.-H. Kuo, 2020: The Taiwan WRF ensemble prediction system: Scientific description, model-error representation and performance results. Asia-Pac. J. Atmos. Sci., 56, 115, https://doi.org/10.1007/s13143-019-00127-8.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1969: Atmospheric predictability as revealed by naturally occurring analogues. J. Atmos. Sci., 26, 636646, https://doi.org/10.1175/1520-0469(1969)26<636:APARBN>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Mason, S. J., and N. E. Graham, 2002: Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quart. J. Roy. Meteor. Soc., 128, 21452166, https://doi.org/10.1256/003590002320603584.

    • Search Google Scholar
    • Export Citation
  • Melhauser, C., and F. Zhang, 2012: Practical and intrinsic predictability of severe and convective weather at the mesoscales. J. Atmos. Sci., 69, 33503371, https://doi.org/10.1175/JAS-D-11-0315.1.

    • Search Google Scholar
    • Export Citation
  • Mittermaier, M., and N. Roberts, 2010: Intercomparison of spatial forecast verification methods: Identifying skillful spatial scales using the fractions skill score. Wea. Forecasting, 25, 343354, https://doi.org/10.1175/2009WAF2222260.1.

    • Search Google Scholar
    • Export Citation
  • Morrison, H., G. Thompson, and V. Tatarskii, 2009: Impact of cloud microphysics on the development of trailing stratiform precipitation in a simulated squall line: Comparison of one- and two-moment schemes. Mon. Wea. Rev., 137, 9911007, https://doi.org/10.1175/2008MWR2556.1.

    • Search Google Scholar
    • Export Citation
  • Nakanishi, M., and H. Niino, 2006: An improved Mellor–Yamada level-3 model: Its numerical stability and application to a regional prediction of advection fog. Bound.-Layer Meteor., 119, 397407, https://doi.org/10.1007/s10546-005-9030-8.

    • Search Google Scholar
    • Export Citation
  • Nakanishi, M., and H. Niino, 2009: Development of an improved turbulence closure model for the atmospheric boundary layer. J. Meteor. Soc. Japan, 87, 895912, https://doi.org/10.2151/jmsj.87.895.

    • Search Google Scholar
    • Export Citation
  • Paulson, C. A., 1970: The mathematical representation of wind speed and temperature profiles in the unstable atmospheric surface layer. J. Appl. Meteor. Climatol., 9, 857861, https://doi.org/10.1175/1520-0450(1970)009<0857:TMROWS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Peralta, C., Z. B. Bouallègue, S. E. Theis, C. Gebhardt, and M. Buchhold, 2012: Accounting for initial condition uncertainties in COSMO‐DE‐EPS. J. Geophys. Res., 117, D07108, https://doi.org/10.1029/2011JD016581.

    • Search Google Scholar
    • Export Citation
  • Roberts, B., I. L. Jirak, A. J. Clark, S. J. Weiss, and J. S. Kain, 2018: Post-processing and visualization techniques for convection-allowing ensembles. Bull. Amer. Meteor. Soc., 100, 12451258, https://doi.org/10.1175/BAMS-D-18-0041.1.

    • Search Google Scholar
    • Export Citation
  • Roberts, N. M., and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 7897, https://doi.org/10.1175/2007MWR2123.1.

    • Search Google Scholar
    • Export Citation
  • Romine, G. S., C. S. Schwartz, J. Berner, K. R. Fossell, C. Snyder, J. L. Anderson, and M. L. Weisman, 2014: Representing forecast error in a convection-permitting ensemble system. Mon. Wea. Rev., 142, 45194541, https://doi.org/10.1175/MWR-D-14-00100.1.

    • Search Google Scholar
    • Export Citation
  • Saito, K., H. Seko, M. Kunii, and T. Miyoshi, 2011: Effect of lateral boundary perturbations on the breeding method and the local ensemble transform Kalman filter for mesoscale ensemble prediction. Tellus, 64A, 11594, https://doi.org/10.3402/tellusa.v64i0.11594.

    • Search Google Scholar
    • Export Citation
  • Schraff, C., H. Reich, A. Rhodin, A. Schomburg, K. Stephan, A. Periáñez, and R. Potthast, 2016: Kilometre‐scale ensemble data assimilation for the COSMO model (KENDA). Quart. J. Roy. Meteor. Soc., 142, 14531472, https://doi.org/10.1002/qj.2748.

    • Search Google Scholar
    • Export Citation
  • Schwartz, C. S., and R. A. Sobash, 2017: Generating probabilistic forecasts from convection-allowing ensembles using neighborhood approaches: A review and recommendations. Mon. Wea. Rev., 145, 33973418, https://doi.org/10.1175/MWR-D-16-0400.1.

    • Search Google Scholar
    • Export Citation
  • Schwartz, C. S., G. S. Romine, R. A. Sobash, K. R. Fossell, and M. L. Weisman, 2018: NCAR’s real-time convection-allowing ensemble project. Bull. Amer. Meteor. Soc., 100, 321343, https://doi.org/10.1175/BAMS-D-17-0297.1.

    • Search Google Scholar
    • Export Citation
  • Shin, H. H., and S.-Y. Hong, 2015: Representation of the subgrid-scale turbulent transport in convective boundary layers at gray-zone resolutions. Mon. Wea. Rev., 143, 250271, https://doi.org/10.1175/MWR-D-14-00116.1.

    • Search Google Scholar
    • Export Citation
  • Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-475+STR, 113 pp., https://doi.org/10.5065/D68S4MVH.

  • Stensrud, D. J., and Coauthors, 2009: Convective-scale Warn-on-Forecast System: A vision for 2020. Bull. Amer. Meteor. Soc., 90, 14871500, https://doi.org/10.1175/2009BAMS2795.1.

    • Search Google Scholar
    • Export Citation
  • Stensrud, D. J., and Coauthors, 2013: Progress and challenges with warn-on-forecast. Atmos. Res., 123, 216, https://doi.org/10.1016/j.atmosres.2012.04.004.

    • Search Google Scholar
    • Export Citation
  • Sun, J., and Coauthors, 2014: Use of NWP for nowcasting convective precipitation: Recent progress and challenges. Bull. Amer. Meteor. Soc., 95, 409426, https://doi.org/10.1175/BAMS-D-11-00263.1.

    • Search Google Scholar
    • Export Citation
  • Tao, W.-K., J. Simpson, and M. McCumber, 1989: An ice-water saturation adjustment. Mon. Wea. Rev., 117, 231235, https://doi.org/10.1175/1520-0493(1989)117<0231:AIWSA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Tao, W.-K., D. Wu, S. Lang, J. Chern, C. Peters‐Lidard, A. Fridlind, and T. Matsui, 2016: High‐resolution NU‐WRF simulations of a deep convective‐precipitation system during MC3E: Further improvements and comparisons between Goddard microphysics schemes and observations. J. Geophys. Res. Atmos., 121, 12781305, https://doi.org/10.1002/2015JD023986.

    • Search Google Scholar
    • Export Citation
  • Tewari, M., and Coauthors, 2004: Implementation and verification of the unified Noah land surface model in the WRF model. 20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., 14.2a, https://ams.confex.com/ams/84Annual/techprogram/paper_69061.htm.

  • Thompson, G., J. Berner, M. Frediani, J. A. Otkin, and S. M. Griffin, 2021: A stochastic parameter perturbation method to represent uncertainty in a microphysics scheme. Mon. Wea. Rev., 149, 14811497, https://doi.org/10.1175/MWR-D-20-0077.1.

    • Search Google Scholar
    • Export Citation
  • Weckwerth, T. M., 2000: The effect of small-scale moisture variability on thunderstorm initiation. Mon. Wea. Rev., 128, 40174030, https://doi.org/10.1175/1520-0493(2000)129<4017:TEOSSM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Whitaker, J. S., and A. F. Loughe, 1998: The relationship between ensemble spread and ensemble mean skill. Mon. Wea. Rev., 126, 32923302, https://doi.org/10.1175/1520-0493(1998)126<3292:TRBESA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Yang, X., 2005: Analysis blending using a spatial filter in grid-point model coupling. HIRLAM Newsletter, No. 48, HIRLAM, 49–55.

  • Zhang, F., C. Snyder, and R. Rotunno, 2002: Mesoscale predictability of the “surprise” snowstorm of 24–25 January 2000. Mon. Wea. Rev., 130, 16171632, https://doi.org/10.1175/1520-0493(2002)130<1617:MPOTSS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Zhang, F., C. Snyder, and R. Rotunno, 2003: Effects of moist convection on mesoscale predictability. J. Atmos. Sci., 60, 11731185, https://doi.org/10.1175/1520-0469(2003)060<1173:EOMCOM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Zhang, F., N. Bei, R. Rotunno, C. Snyder, and C. C. Epifanio, 2007: Mesoscale predictability of moist baroclinic waves: Convection-permitting experiments and multistage error growth dynamics. J. Atmos. Sci., 64, 35793594, https://doi.org/10.1175/JAS4028.1.

    • Search Google Scholar
    • Export Citation
  • Zhang, F., Y. Q. Sun, L. Magnusson, R. Buizza, S.-J. Lin, J.-H. Chen, and K. Emanuel, 2019: What is the predictability limit of midlatitude weather? J. Atmos. Sci., 76, 10771091, https://doi.org/10.1175/JAS-D-18-0269.1.

    • Search Google Scholar
    • Export Citation
  • Zhang, J., K. Howard, and J. J. Gourley, 2005: Constructing three-dimensional multiple-radar reflectivity mosaics: Examples of convective storms and stratiform rain echoes. J. Atmos. Oceanic Technol., 22, 3042, https://doi.org/10.1175/JTECH-1689.1.

    • Search Google Scholar
    • Export Citation
  • Ziehmann, C., 2000: Comparison of a single-model EPS with a multi-model ensemble consisting of a few operational models. Tellus, 52A, 280299, https://doi.org/10.3402/tellusa.v52i3.12266.

    • Search Google Scholar
    • Export Citation
Save
  • Bachmann, K., C. Keil, and M. Weissmann, 2018: Impact of radar data assimilation and orography on predictability of deep convection. Quart. J. Roy. Meteor. Soc., 145, 117130, https://doi.org/10.1002/qj.3412.

    • Search Google Scholar
    • Export Citation
  • Berner, J., S.-Y. Ha, J. P. Hacker, A. Fournier, and C. Snyder, 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 19721995, https://doi.org/10.1175/2010MWR3595.1.

    • Search Google Scholar
    • Export Citation
  • Berner, J., and Coauthors, 2016: Stochastic parameterization: Toward a new view of weather and climate models. Bull. Amer. Meteor. Soc., 98, 565588, https://doi.org/10.1175/BAMS-D-15-00268.1.

    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and P. Lacarrere, 1989: Parameterization of orography-induced turbulence in a mesobeta-scale model. Mon. Wea. Rev., 117, 18721890, https://doi.org/10.1175/1520-0493(1989)117<1872:POOITI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Bouttier, F., B. Vié, O. Nuissier, and L. Raynaud, 2012: Impact of stochastic physics in a convection-permitting ensemble. Mon. Wea. Rev., 140, 37063721, https://doi.org/10.1175/MWR-D-12-00031.1.

    • Search Google Scholar
    • Export Citation
  • Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78, 13, https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Chang, P.-L., P.-F. Lin, B. J.-D. Jou, and J. Zhang, 2009: An application of reflectivity climatology in constructing radar hybrid scans over complex terrain. J. Atmos. Oceanic Technol., 26, 13151327, https://doi.org/10.1175/2009JTECHA1162.1.

    • Search Google Scholar
    • Export Citation
  • Chang, P.-L., and Coauthors, 2021: An operational multi-radar multi-sensor QPE system in Taiwan. Bull. Amer. Meteor. Soc., 102, E555E577, https://doi.org/10.1175/BAMS-D-20-0043.1.

    • Search Google Scholar
    • Export Citation
  • Chen, I. H., J. S. Hong, Y. T. Tsai, and C. T. Fong, 2020: Improving afternoon thunderstorm prediction over Taiwan through 3DVAR-based radar and surface data assimilation. Wea. Forecasting, 35, 26032620, https://doi.org/10.1175/WAF-D-20-0037.1.

    • Search Google Scholar
    • Export Citation
  • Clark, A., and Coauthors, 2019: Spring forecasting experiment 2019: Preliminary findings and results. NOAA Hazardous Weather Testbed, NOAA, 77 pp., https://hwt.nssl.noaa.gov/sfe/2019/docs/HWT_SFE_2019_Prelim_Findings_FINAL.pdf.

  • Demuth, J. L., and Coauthors, 2020: Recommendations for developing useful and usable convection-allowing model ensemble information for NWS forecasters. Wea. Forecasting, 35, 13811406, https://doi.org/10.1175/WAF-D-19-0108.1.

    • Search Google Scholar
    • Export Citation
  • Flowerdew, J., 2014: Calibrating ensemble reliability whilst preserving spatial structure. Tellus, 66A, 22662, https://doi.org/10.3402/tellusa.v66.22662.

    • Search Google Scholar
    • Export Citation
  • Frogner, I., A. T. Singleton, M. Ø. Køltzow, and U. Andrae, 2019: Convection‐permitting ensembles: Challenges related to their design and use. Quart. J. Roy. Meteor. Soc., 145, 90106, https://doi.org/10.1002/qj.3525.

    • Search Google Scholar
    • Export Citation
  • García-Ortega, E., J. Lorenzana, A. Merino, S. Fernández-González, L. López, and J. L. Sánchez, 2017: Performance of multiphysics ensembles in convective precipitation events over northeastern Spain. Atmos. Res., 190, 5567, https://doi.org/10.1016/j.atmosres.2017.02.009.

    • Search Google Scholar
    • Export Citation
  • Gebhardt, C., S. E. Theis, M. Paulat, and Z. B. Bouallègue, 2011: Uncertainties in COSMO-DE precipitation forecasts introduced by model perturbations and variation of lateral boundaries. Atmos. Res., 100, 168177, https://doi.org/10.1016/j.atmosres.2010.12.008.

    • Search Google Scholar
    • Export Citation
  • Golding, B., N. Roberts, G. Leoncini, K. Mylne, and R. Swinbank, 2016: MOGREPS-UK convection-permitting ensemble products for surface water flood forecasting: Rationale and first results. J. Hydrometeor., 17, 13831406, https://doi.org/10.1175/JHM-D-15-0083.1.

    • Search Google Scholar
    • Export Citation
  • Guerra, J. E., P. S. Skinner, A. Clark, M. Flora, B. Matilla, K. Knopfmeier, and A. E. Reinhart, 2022: Quantification of NSSL Warn-on-Forecast System accuracy by storm age using object-based verification. Wea. Forecasting, 37, 19731983, https://doi.org/10.1175/WAF-D-22-0043.1.

    • Search Google Scholar
    • Export Citation
  • Hagelin, S., J. Son, R. Swinbank, A. McCabe, N. Roberts, and W. Tennant, 2017: The Met Office convective‐scale ensemble, MOGREPS‐UK. Quart. J. Roy. Meteor. Soc., 143, 28462861, https://doi.org/10.1002/qj.3135.

    • Search Google Scholar
    • Export Citation
  • Harding, K., 2011: Thunderstorm formation and aviation hazards. NOAA/National Weather Service, 1–4.

  • Harnisch, F., and C. Keil, 2015: Initial conditions for convective-scale ensemble forecasting provided by ensemble data assimilation. Mon. Wea. Rev., 143, 15831600, https://doi.org/10.1175/MWR-D-14-00209.1.

    • Search Google Scholar
    • Export Citation
  • Hermoso, A., V. Homar, and R. S. Plant, 2021: Potential of stochastic methods for improving convection-permitting ensemble forecasts of extreme events over the Western Mediterranean. Atmos. Res., 257, 105571, https://doi.org/10.1016/j.atmosres.2021.105571.

    • Search Google Scholar
    • Export Citation
  • Hersbach, H., and Coauthors, 2018: ERA5 hourly data on single levels from 1940 to present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS), accessed 14 April 2021, https://doi.org/10.24381/cds.adbb2d47.

  • Hersbach, H., and Coauthors, 2020: The ERA5 global reanalysis. Quart. J. Roy. Meteor. Soc., 146, 19992049, https://doi.org/10.1002/qj.3803.

    • Search Google Scholar
    • Export Citation
  • Hirt, M., and G. C. Craig, 2021: A cold pool perturbation scheme to improve convective initiation in convection‐permitting models. Quart. J. Roy. Meteor. Soc., 147, 24292447, https://doi.org/10.1002/qj.4032.

    • Search Google Scholar
    • Export Citation