Heavy rainfall events account for most socioeconomic damages caused by natural disasters in South Korea. However, the microphysical understanding of heavy rain is still lacking, leading to uncertainties in quantitative rainfall prediction. This study is aimed at evaluating rainfall forecasts in the Local Data Assimilation and Prediction System (LDAPS), a high-resolution configuration of the Unified Model over the Korean Peninsula. The rainfall of LDAPS forecasts was evaluated with observations based on two types of heavy rain events classified from K-means clustering for the relationship between surface rainfall intensity and cloud-top height. LDAPS forecasts were characterized by more heavy rain cases with high cloud-top heights (cold-type heavy rain) in contrast to observations showing frequent moderate-intensity rain systems with relatively lower cloud-top heights (warm-type heavy rain) over South Korea. The observed cold-type and warm-type events accounted for 32.7% and 67.3% of total rainfall, whereas LDAPS forecasts accounted for 65.3% and 34.7%, respectively. This indicates severe overestimation and underestimation of total rainfall for the cold-type and warm-type forecast events, respectively. The overestimation of cold-type heavy rainfall was mainly due to its frequent occurrence, whereas the underestimation of warm-type heavy rainfall was affected by both its low occurrence and weak intensity. The rainfall forecast skill for the warm-type events was much lower than for the cold-type events, due to the lower rainfall intensity and smaller rain area of the warm-type. Therefore, cloud parameterizations for warm-type heavy rain should be improved to enhance rainfall forecasts over the Korean Peninsula.
During the recent 35 years in South Korea, heavy rainfall events have been responsible for about 90% of the socioeconomic damage by natural disasters (Ahn et al. 2015). Considering the tremendous damage and costs of recovering from damages caused by heavy rainfall, a considerable number of studies have been conducted on the characteristics of heavy rain and its formation mechanisms over South Korea. Three synoptic mechanisms have been identified for heavy rainfall over the Korean Peninsula: the divergence with an upper jet, the increased baroclinic instability by northwesterly cold-air advection in the middle layer, and the water vapor advection by a southwesterly low-level jet (Hwang and Lee 1993; Lee et al. 1998; Park et al. 1986). The heavy rainfall events over the Korean Peninsula are also associated with the changma front during the period from late June to late July (Lee et al. 2017), mesoscale disturbances (Kim and Lee 2006), localized thermodynamic instability (Jung et al. 2015), and the effects of typhoons (Byun and Lee 2012). However, the understanding of heavy rain–forming microphysics is still insufficient, leading to uncertainties in rainfall forecasting, especially in terms of rainfall amount. Consequently, quantitative rainfall prediction remains poor and the current forecast accuracy for heavy rainfall does not meet the expectations of people.
The Korean Meteorological Administration (KMA), which handles rainfall forecasts in South Korea, has been using forecasting systems based on the Unified Model (UM) of the Met Office since May 2010. The KMA UM forecast systems consist of the Global Data Assimilation and Prediction System (GDAPS) over the global domain, Regional Data Assimilation and Prediction System (RDAPS) over East Asia regions, and Local Data Assimilation and Prediction System (LDAPS) over the Korean Peninsula. After June 2016, the horizontal resolution of GDAPS was improved from 25 to 17 km, and the GDAPS data have been used as boundary conditions for the LDAPS with a 1.5-km horizontal resolution. The resolution of GDAPS was further improved to 10 km since January 2018. Because the horizontal resolutions of GDAPS (10–25 km) and RDAPS (12 km) are too coarse to resolve heavy rain systems, convection-permitting models with resolutions of a few kilometers (e.g., LDAPS) have been used to improve performance for heavy rainfall forecasts. For example, Roberts and Lean (2008) and Lean et al. (2008) reported that a 1-km UM better predicts heavy precipitation accompanied by convective rain systems than a 12-km UM over the United Kingdom regions. The high-resolution model also has the advantage of reducing uncertainty attributable to cumulus parameterization as well as a realistic representation of topography. In fact, the prediction of precipitation at coarse resolution (e.g., GDAPS, RDAPS) tends to be determined by both cumulus and microphysics parameterizations, whereas grid-resolvable-scale precipitation in the high-resolution model (e.g., LDAPS) is solely driven by the microphysics scheme. On the other hand, numerical models with a grid spacing of a few kilometers are known for being sensitive to subgrid mixing representation, because they are at the same scale of convective gray zones (Clark et al. 2016). Currently, with high-resolution modeling facilitated by steady increases in computing power, local-scale weather prediction models are commonly run at horizontal resolutions of a few kilometers (Kain et al. 2008). For example, the United Kingdom, South Korea, Australia, New Zealand, India, Singapore, and South Africa have been operating forecast systems with resolutions of a few kilometers based on the Unified Model.
The verification of LDAPS rainfall forecast over the Korean Peninsula has been mainly carried out by South Korean scientists operationally. Routine verification of LDAPS rainfall forecasts is being performed in KMA operations using various evaluation indices (e.g., accuracy, bias score, critical success index, and equitable threat score). Lim et al. (2014) reported that the data assimilation of mobile rawinsonde observations could improve the equitable threat score of rainfall in the LDAPS by about 2%. However, the equitable threat score for heavy rainfall events (for 12-h rainfall greater than 25 mm) was only about 40% that of total rainfall events, representing the difficulty of heavy rainfall forecasting (Lim et al. 2014). Kim et al. (2015) evaluated the performance of the LDAPS ensemble model on rainfall forecast during August 2012, reporting that the forecast accuracy for heavy rain events is lower than that for all rainfall events. As indicated by previous studies, the forecast accuracy of heavy rainfall is generally low for not only GDAPS and RDAPS but even LDAPS. Considering the significant public concerns over heavy rainfall, it is necessary to carefully examine the heavy rainfall forecasts of LDAPS over South Korea.
The forecasting accuracy of heavy rainfall can strongly depend on the characteristics of local rain. Recent satellite studies (Sohn et al. 2013; Song and Sohn 2015) have suggested most rainfall events over the Korean Peninsula and humid East Asian monsoon regions (hereafter referred to as “warm-type heavy rain”) are attributable to heavy rain systems with much lower storm height (<8 km) than those found in continental regimes (e.g., the central United States and mainland China). Zhang et al. (2006) and Oue et al. (2010) using ground-based radars also have been pointing out that “medium-depth convection” (similar to warm-type heavy rain) is the main contributor to the total rainfall of mei-yu and baiu monsoon fronts. Therefore, warm-type heavy rain system, mainly consist of medium-depth clouds, can be regarded as a general structure over the humid monsoon regions (e.g., South Korea, Japan, Taiwan, and adjacent ocean areas). Note that the warm-type heavy rain is different concept from conventional warm rain below freezing levels (4–5 km) usually associated with light rainfall. Song et al. (2017) showed that the production of warm-type heavy rainfall is most affected by collision and coalescence processes of raindrops under a humid environment, from idealized model experiments.
Therefore, numerical weather prediction over the Korean Peninsula is extremely difficult owing to the wide difference between warm-type heavy rain and conventional deep convective cloud-producing heavy rainfall (hereafter referred to as “cold-type heavy rain”), which is mainly found in continental regime areas related with thermodynamic instability by solar heating. Hong (2004) reported that the removal of convective instability by a cloud parameterization scheme is not suitable for producing heavy rainfall over the Korean Peninsula, unlike in the continental United States. This implies that a cloud parameterization scheme for simulating continental-type heavy rain such as in the United States is not suitable for rain clouds found in the Korean Peninsula. Furthermore, Song and Sohn (2018) evaluated the performance of eight Weather Research and Forecasting (WRF) Model microphysics schemes for simulating the warm-type heavy rain in South Korea against the satellite-based radar observations. They noted that all microphysics schemes commonly indicated an overestimation of ice water content for heavy rain cases of rainfall intensities greater than 10 mm h−1. This suggests that the cold-type heavy rain process is more dominant in the WRF model in contrast to the warm-type heavy rain found in observations.
Owing to limitations of temporal and spatial coverage in satellite radar observations, the result of Song and Sohn (2018) remained the only climatological comparison of heavy rain structure between observations and model simulations, as shown in radar reflectivity profiles, over the southern part of the Korean Peninsula. In this study, we aim to further evaluate the performance of the high-resolution UM (i.e., LDAPS) on rainfall forecasts according to heavy rain types over all of South Korea. Here, we are interested in the entire rain system, rather than to the heavy-rain core only (as in Song and Sohn 2018); in fact, our study is primarily concerned with the evaluation of the total rainfall forecast for both heavy and light rain. To classify the heavy rain types, we assume that there are multiple relationships between surface rainfall and cloud-top height for the observation and model. As the two heavy rain types (i.e., cold-type versus warm-type) suggested in Sohn et al. (2013) and Song and Sohn (2015) are a relatively new concept over the Korean Peninsula, rainfall forecasts according to these heavy rain types have not been verified thus far. In general, we investigate whether the high-resolution UM captures the dominant feature of warm-type heavy rain over South Korea. If differences exist between observed and modeled heavy rain types, the fundamental cause for the difference is explored to shed light on heavy rain prediction. The results obtained through these analyses could provide valuable guidance to weather forecasting in interpreting high-resolution model results and improving the development of physical parameterizations for numerical models.
The UM is a finite-difference grid model with semi-implicit and semi-Lagrangian numerical schemes. The model uses a rotated horizontal grid system with Arakawa-C staggering (Arakawa and Lamb 1977) and a terrain-following vertical coordinate with Charney–Philips staggering (Charney and Phillips 1953). The model includes various physical parameterizations: Edward–Slingo general two-stream radiation scheme (Edwards and Slingo 1996), Joint U.K. Land Environment Simulator (JULES) nine-layer soil model using the soil hydrology of van Genuchten (1980), first-order nonlocal boundary layer scheme (Lock et al. 2000), diagnostic cloud scheme (Smith 1990), mixed-phase cloud microphysics scheme (Wilson and Ballard 1999) with modified raindrop fall speed (Abel and Shipway 2007), mass flux cumulus convection scheme with a convective available potential energy closure (Gregory and Rowntree 1990), a Smagorinsky-type subgrid turbulence scheme (Hanley et al. 2015) with the mixing length of 300 m, and gravity wave drag due to orography (Lott and Miller 1997). Detailed updates for the recent version of the UM are available in Walters et al. (2017).
Version 10.1 of the KMA LDAPS model over the Korean Peninsula has 1188 × 1148 horizontal grid points with a horizontal grid spacing of 1.5 km and 70 vertical levels up to 40 km. The LDAPS system provides forecasts in 36 h for initial conditions of 0000, 0600, 1200, and 1800 UTC (four times a day) and boundary conditions from GDAPS model data. Data assimilation for the initial conditions is based on a three-dimensional variational data assimilation (3DVAR) system with latent heat nudging of ground-based radars. The convection scheme is turned off in the LDAPS model because the high-resolution model can depict convection explicitly. Although the LDAPS model integrates with a time step of 1 min, the forecast data are stored every hour. We selected June–August (JJA) of 2016–18 as the research period because the version of LDAPS (i.e., version 10.1), including physics schemes and dynamical process, was largely updated in June 2016.
To evaluate the LDAPS rainfall forecast, this study used hourly accumulated rainfall with a 5-km horizontal resolution provided by the KMA Numerical Modeling Center. Rainfall data were obtained by combining rain data from surface rain gauges at more than 800 sites and ground-based weather radar networks over a 480-km area in South Korea. The quality control processes (e.g., the conversion of radar rain rates to hourly accumulated rainfall) were applied to produce homogeneous rainfall data from different datasets (i.e., rain gauge and radar). However, the rainfall data can be relatively inaccurate for areas without rain gauge data (e.g., sea and North Korea) because it is derived from radar observation alone. Thus, we limited our research domain to the land region over South Korea where both highly accurate rain gauge and radar data are available.
We also use hourly instantaneous cloud-top height (CTH) data with 5-km horizontal resolution acquired from the Meteorological Imager of the Communication, Oceanography and Meteorology Satellite (COMS)—the first geostationary satellite of South Korea. The retrieval algorithm for the cloud-top properties (i.e., top temperature, pressure, and height) is based on Choi et al. (2007). According to the KMA National Meteorological Satellite Center, COMS CTH data have a root-mean-square error (RMSE) less than 1.5 km for all clouds, compared with cloud products from the Moderate Resolution Imaging Spectroradiometer (MODIS) with high spatial resolution and more channels than COMS. In the European geostationary satellite (i.e., Meteosat Second Generation) similar to COMS, the Spinning Enhanced Visible and Infrared Imager (SEVIRI) CTH is known to have mean bias of −0.27 km and standard deviation of 1.61 km for all clouds when compared to the CloudSat radar observation, which can be considered a reference (Chung et al. 2017). Since the emissions from the cloud top are close to those from the blackbody (i.e., emissivity = 1) of optically thick clouds (i.e., rain clouds), the accuracy of the CTH retrieval for the rain clouds was expected to be higher than for others (including optically thin and spatially inhomogeneous clouds that are difficult to distinguish from land surface signals). Choi et al. (2007) noted that the retrieval uncertainty of cloud-top temperature is lower than 3 K for opaque clouds. If we assume a moist adiabatic lapse rate of 6 K km−1, an uncertainty of 0.5 km arises for the CTH retrieval. Another error of CTH retrieval occurs in the conversion process from cloud-top temperature into CTH. The KMA National Meteorological Satellite Center utilizes hourly temperature profiles from the GDAPS forecasts as ancillary data for estimating CTH. Although data smoothing with the use of the GDAPS data with a low horizontal resolution may cause an error in the CTH retrieval, it is considered insignificant in the long-term perspective.
This study also analyzed hourly accumulated rainfall and CTH data in the LDAPS model. Because initial fields are provided in 6-h intervals and LDAPS provides 36-h predictions, LDAPS forecasts were divided into six categories according to the forecast times (0–6, 6–12, 12–18, 18–24, 24–30, and 30–36 h forecasts). In other words, six LDAPS forecasts provide hourly accumulated rainfall and CTHs for the period of six hours. For comparison with 5-km observations, the 1.5-km data of the LDAPS model were upscaled to 5-km resolution (a total of 3860 grids over the South Korean land area). To obtain 5-km data from an original resolution of 1.5 km, we assigned a 90% weighting to three successive 1.5-km data (i.e., 81% weighting in 4.5 km × 4.5 km grids), and the remaining weight to the neighboring grid points of three 1.5-km data among total 7.5 km × 7.5 km grids. After that, the LDAPS 5-km data were linearly interpolated to the observation 5 km × 5 km grids. Note that the occurrence frequency at 5 km may differ from that at 1.5 km without the upscaling process, because rainfall intensity is spatially inhomogeneous, with skewness ~0 (e.g., frequent nonraining cases). In addition, we averaged two subsequent hourly CTHs in order to represent the accumulated rainfall during each hour (i.e., the CTH represents the values of 30 min every hour). Since the CTH value represented an average of values registered during a period of ~30 min, the above approach can lead to errors for quick movements and insolated cloud systems. However, the error is not expected to be large in terms of long-term analysis (including the summer months of three years) over the entire South Korean region.
The CTH in the LDAPS model was empirically defined as the highest altitude at which cloud fraction was larger than the half of an okta (i.e., 1/16 = 6.25%). In the data assimilation system of cloud-affected satellite infrared radiances above overcast clouds in the UM, a cloud fraction of 10% is used for the criteria of CTH (Pavelin et al. 2008), implying that, physically, the CTH condition in the LDAPS would not significantly differ from CTH inferred from satellite infrared radiance. The issue of CTH definitions will be further discussed in the next section.
a. General characteristics
Hourly rainfall accumulations from observations and six LDAPS forecasts (0–6, 6–12, 12–18, 18–24, 24–30, and 30–36 h) were collected for the period of JJA 2016–18 in order to identify the mean spatial distribution of unconditional rainfall (i.e., nonrain + rain cases). Figure 1 represents the spatial distribution of mean rainfall from observations and LDAPS forecasts over South Korea. A simple average of the six LDAPS forecasts was calculated to produce the horizontal mean rainfall distribution shown in Fig. 1b. LDAPS forecasts tend to overestimate rainfall over the southeastern part of the Korean Peninsula as well as around Jeju Island, whereas it underestimates rainfall in the most areas of the rest. Despite the regional biases, the mean unconditional rainfall in the LDAPS forecasts (6.18 mm day−1) was found to be comparable to observations (i.e., 6.57 mm day−1). Additionally, Figure S1 in the online supplemental material shows that there are large differences in average intensity among the six LDAPS forecasts (i.e., 7.67, 5.18, 5.43, 5.78, 6.29, and 6.72 mm day−1 for 0–6, 6–12, 12–18, 18–24, 24–30, and 30–36 h forecasts, respectively). The LDAPS forecasts overestimated rainfall in the first six hours (Fig. S1a) after the initialization process, while it substantially underestimated rainfall in the following six hours (Figs. S1b–f). Those features appear to be spinup problems with initialization processes (e.g., downscaling from the coarse-resolution GDAPS to the high-resolution LDAPS, excessive latent heat nudging of radar data).
Likely, the model overestimated rainfall in the first six hours after the initialization process (e.g., downscaling from the coarse-resolution GDAPS to the high-resolution LDAPS and excessive latent heat nudging of radar data), while it substantially underestimated rainfall in the following six hours. Furthermore, the mean unconditional rainfall obtained for the observation and the LDAPS forecasts was different every year. In particular, the mean rainfall in the LDAPS forecasts for the period of JJA 2018 was underestimated compared to the observations (7.33 versus 5.87 mm day−1), in contrast to those in 2016 (5.21 versus 5.92 mm day−1). The underestimation of LDAPS rainfall in 2018 is relatively large compared with those in 2017 (7.18 versus 6.73 mm day−1). In fact, the subversion of the LDAPS was updated after 2018, while keeping its core version (i.e., version 10.1). The updates included various improvements in data assimilation (related to soil moisture, new satellite data, background error covariance, and latent heat nudging of radar data), as well as in numerical modeling, concerning the semi-Lagrangian advection coefficient and the size distribution of raindrops in the microphysics scheme. Still, the rainfall underestimation for the year 2018 could have been affected by both observational features of year and updates in the LDAPS. Finding the cause of rainfall underestimation was not main purpose of this study; hence, we did not give much meaning to the rainfall bias in the LDAPS forecasts shown in Fig. 1.
The probability density functions (PDFs) of binned hourly rainfall intensities from observation and LDAPS forecasts were also examined (Fig. 2). The PDFs were normalized by the total number of cases (i.e., nonrain + rain cases). Different x- and y-axis scales were applied for light rain (<10 mm h−1) and heavy rain (>10 mm h−1) cases in Fig. 2. Six forecasts (0–6, 6–12, 12–18, 18–24, 24–30, and 30–36 h) of LDAPS are named as 6, 12, 18, 24, 30, and 36 h. The PDFs of LDAPS 12–36 h were in good agreement with those of observations. In addition, LDAPS 12–36 h tended to simulate less frequent light rain cases of 1–10 mm h−1 by 29% and less/more frequent heavy rain cases (>10 mm h−1) from −6% (LDAPS 12 h) to 27% (LDAPS 36 h) compared to observations. The overestimation of heavy rain cases is consistent with that of McBeath et al. (2014) who showed UM simulations with a 1.5-km grid spacing overestimated heavy rain events over the British Isles. However, the LDAPS 6 h exhibited a strong overestimation in frequency for rain cases greater than 5 mm h−1 by 19%, leading to an overestimation of the PDF of rainfall intensities. This unique feature in LDAPS 6 h appears to be a problem related to the initial condition and model spinup.
This study further investigated total volumetric rainfall (i.e., rain intensity × rain area) for each bin (1-km interval) of CTH for all rainfall cases (Fig. 3). The observed volumetric rainfall was distributed over a wide range lower than the CTH of 13 km and it sharply increased at the CTH of 15–16 km (Fig. 3a). These two features appear to be consistent with two types of heavy rain over the Korean Peninsula noted in previous studies (Sohn et al. 2013; Song and Sohn 2015). However, LDAPS forecasts show a single-mode feature of rainfall, which is mostly concentrated on clouds with high CTH between 14 and 15 km, indicating the cold-type heavy rain (Fig. 3a). The volumetric rainfall of LDAPS forecasts with CTH higher than 13 km is about twice as much as that of observations, whereas the LDAPS volumetric rainfall with CTH lower than 13 km is a half of observations, indicating that the LDAPS forecasts produce more rainfall from cold-type events compared to observations, and less from warm-type events. Those features of the LDAPS forecasts seem highly related with the occurrence frequency of heavy rain cases (>10 mm h−1): compared with the observations, the LDAPS data showed much higher frequencies at CTHs above 13 km and lower frequencies at CTHs below 13 km (Fig. 3b).
A critical argument arises as to whether the CTHs from observation and forecasts are comparable. Figure 3b shows the mean CTHs of heavy rain clouds (>10 mm h−1), which can be assumed as vertically uniform convective clouds; the value based on the satellite observations is of 11.6 km, but that based on the LDAPS forecasts is of 13.1 km. The difference between these values (i.e., 1.5 km) could be attributed to the different characteristics between observation and model, or to their different physical definitions on CTH. We also found that using a threshold of 32.7% for the cloud fraction in the LDAPS model, produced the mean CTH of heavy rain (i.e., 11.6 km) found in the observation. However, this threshold (i.e., 32.7%) seems too strict compared with the value (i.e., 10%) used in Pavelin et al. (2008), because infrared radiances in window channels are highly sensitive to the presence of clouds. If the cloud fraction thresholds used were within generally accepted ranges for the cloud top (e.g., 10%), we would not expect a much different conclusion from Fig. 3.
Stein et al. (2015) noted that the CTH and the ice water content of high-level clouds in a 1.5-km UM were lower than those obtained from satellite observations over the West African monsoon region. As West Africa is characterized by a continental rain regime (showing frequent deep convective clouds; Xu and Zipser 2012), the UM may simulate CTH values lower than those of the observations. On the other hand, Franklin et al. (2016) found that the convection-permitting UM simulations over the Darwin region (Australia), comparable to an oceanic monsoon regime (Xu and Zipser 2012), exhibit higher cloud-top heights and ice water content compared to the radar observations. This situation appears similar to that on the Korean Peninsula where most rainfall is attributable to medium-depth convection (i.e., warm-type heavy rain), which are part of the oceanic monsoon system. Recent results of Li et al. (2018) in the Chinese continent indicate that the convection-permitting simulations based on the UM (4.4-km horizontal grid spacing) overestimated rainfall in the late afternoon but underestimated it from midnight to the morning. These diurnal features are closely related to the cold-type and warm-type heavy rainfalls that, in turn, are linked to the thermodynamic instability caused by solar heating and by the radiative cooling effect, respectively (Song and Sohn 2015). The results of Furtado et al. (2018) over Southern China showed that the Wilson and Ballard (1999) microphysics scheme (used in the UM model) frequently produced OLR cases lower than 100 W m−2, which can be considered as deep convection (i.e., cold-type heavy rain), than the observation and results from other microphysics parameterizations. All these results strongly support the overestimation of cold-type heavy rainfall and the underestimation of warm-type heavy rainfall in the UM simulations at convection-permitting scales.
b. Heavy rain types
In this study, the representation of heavy rain types in observations and LDAPS forecasts was first investigated. As in Song and Sohn (2015), the K-means clustering algorithm was used, but with a different methodology. The classification of heavy rain types in Song and Sohn (2015) was based on the difference in vertical structure of heavy rain. This study considered a classification of rain systems accompanied by a heavy rain core, rather than considering only an isolated heavy rain area; in fact, our focus was to evaluate the forecast performance of total rainfall for heavy rain events. Therefore, we developed the classification method using the frequency distributions of CTH normalized conditional on rainfall intensity over South Korea. Here, 6-hourly frequency statistics were accumulated from hourly relationships between rainfall intensity and CTH, and thus the K-means clustering algorithm was performed to the joint distribution of rainfall intensity and CTH, asking two clusters to be returned. Since initial value of the joint distribution is randomly prescribed in the first step, iteration process is performed until convergence has been reached. Here, the observation and LDAPS forecasts were considered simultaneously in the K-means clustering. The number of observations was considered to be virtually increased by 6 times for a fair comparison with six LDAPS forecasts. Note that this study does not consider the K-means clustering separately from the observation and LDAPS forecasts (i.e., two cold-type and two warm-type events), because the four-type concepts would make it difficult to fairly compare observation and forecast results based on the common cold-type and warm-type events. Different features between observation and forecast are dealt with occurrence frequencies of two-type events in this study. The classification of heavy rain types in this study risks to be inaccurate and not very useful for no or very few heavy rain cases. For these reasons, we empirically removed those events exhibiting heavy rain (>10 mm h−1) areas smaller than 675 km2, which corresponded to the top 99% of the cumulative heavy rain area. Although we could have used more strict conditions (i.e., extreme heavy rain cases), this would have reduced the representation of total rainfall, since many rain cases would have been removed. After the removal process, 752 observation events and 699 forecast events (average of six LDAPS forecasts) were excluded from 1104 events (i.e., 6-hourly events during JJA 2016–18). The removed cases correspond to 6.14% (7.27% in observation and 5.00% in forecasts) of total rainfall.
The frequency distributions of CTH with rainfall intensity for the classified cold-type and warm-type heavy rain events (simply “cold-type events” and “warm-type events” here) are presented in Figs. 4a and 4d. Here, nonraining and cloud-free areas were not considered in the K-means clustering. Heavy rain type events in this study represent 6-hourly cases for areas of heavy rain (>10 mm h−1) greater than 675 km2 in South Korean land regions. It refers the whole heavy rain system, containing various precipitating cloud pixels, in terms of large-scale condition. In the classification of heavy rain types, we considered individual events in time in the observation and forecast (with the same weighting for each). However, a common feature (i.e., CTH distributions with rainfall intensity) representing both the observation and forecast should be identified in order to compare the characteristics of the classified heavy rain types. For this purpose, the K-means clustering analysis assuming two clusters was applied to categorize representative types both for the observation and forecast. As a result, 67 and 285 events were classified as cold-type and warm-type events in the observation, respectively. On the other hand, the LDAPS forecasts were characterized by very frequent cold-type events (184 events) but less frequent warm-type events (221 events) compared to the observation. The number of events for the LDAPS is an average of the results from six forecasts (0–6, 6–12, 12–18, 18–24, 24–30, and 30–36 h). The numbers of heavy rain type events are summarized in Table 1.
The cold-type events show that most CTHs are concentrated at altitudes higher than 13 km (Fig. 4a), whereas the CTH of warm-type events is spread over a wide range (Fig. 4d). The mean CTHs for the cold-type and warm-type events are 13.9 and 9.0 km, respectively. Although included in the same class type, the distributions of the CTHs were somewhat different in their mean and shape between the observation and LDAPS forecasts. The CTH distributions for the cold-type events showed relatively consistent patterns between the observation and LDAPS forecasts; however, the widely distributed CTHs, with low observation frequency (Figs. 4b and 4e), were not found in the LDAPS (Figs. 4c and 4f). The mean CTHs for the cold-type observation and forecast events were 13.5 and 14.0 km, respectively. The difference among the CTH distributions for the warm-type events was larger than for the cold-type events (Figs. 4c and 4f). CTHs frequently occurred at around 12 km in the warm-type forecast events, but they were relatively rare in the warm-type observation events. Although the patterns for the warm-type events are different, the mean CTHs for the warm-type observation and forecast events were 8.5 and 9.5 km, respectively. Note that the mean CTHs are generally higher than storm heights (about 9.5 km for cold-type and 6.5 km for warm-type cases) shown in Sohn et al. (2013). This is because storm height, defined as the altitude at which 15-dBZ radar reflectivity is observed, is mostly affected by large-size precipitating particles (e.g., snow, graupel, and rain), whereas CTH is also sensitive to small-size cloud-ice particles.
Figure 5 compares the relative contribution of rainfall for the cold-type and warm-type events between the observations and LDAPS model. Warm-type events account for most of the rainfall events in the observations (Fig. 5b), whereas cold-type events are dominant in LDAPS forecasts (Fig. 5c). The correspondent statistical values are given in Table 1. The observed cold-type and warm-type events occupy 32.7% and 67.3% of all the rainfall events over the land area of South Korea, whereas the LDAPS forecasts correspond to 65.3% and 34.7%, respectively. The spatial distributions (Fig. 5a) indicate that higher rainfall values were observed for cold-type events over the western part, rather than over the eastern part of the Korean Peninsula. A west–east gradient was also identified in the forecast events (Fig. 5c); still, the relative contribution of rainfall was somehow different in the forecast and in the observations.
We further investigated the PDF distributions of rainfall intensity for the two heavy rain events of the observations and LDAPS forecasts (Fig. 6). Since the PDF of rainfall intensity is normalized by the occurrence of each type, the result is not affected by their frequency of occurrence. The total percentage of moderate-to-heavy (1–30 mm h−1) cases for the LDAPS forecast events was about 60% of the observations (64% for cold-type and 56% for warm-type events). These values were affected by the severe underestimation of mean rainfall in forecast events, when compared to the observations (Table 1). Although most of moderate rain cases and mean rainfall values were underestimated compared to the observations, warm-type forecast events tended to overestimate extreme rain cases compared to the observations (Fig. 6b).
Figure 7 compares total volumetric rainfall at each CTH bin for heavy rain types. The results are cold-type and warm-type split versions of Fig. 3 for total rain events. The similarity between Figs. 3 and 7 also implies that the removal of some cases during the classification of heavy rain types (i.e., heavy rain area less than 675 km2) does not have any noticeable influence on the results with total volumetric rainfall. As expected, the characteristics of volumetric rainfall based on CTH are clear for cold-type and warm-type events. For cold-type events, LDAPS volumetric rainfall with CTHs higher than 13 km is 2.11 times than that of the observation, indicating a severe overestimation of total rainfall for LDAPS cold-type events (Fig. 7a). However, LDAPS volumetric rainfall with CTHs lower than 13 km is 48% of the observation for the warm-type events, representing a significant underestimation of total rainfall for LDAPS warm-type events (Fig. 7b). It is also of note that the overestimated volumetric rainfall for cold-type forecast events is mainly related to their frequent occurrence because the mean rainfall intensity for the cold-type events is smaller than that from the observations, whereas the underestimated volumetric rainfall of warm-type forecast events is affected by both frequency and intensity (Table 1). Figures 7c and 7d represent the mean volumetric rainfall with CTHs normalized by the numbers of cold-type and warm-type events. A good agreement between cold-type observation and forecast events is found in mean rainfall for the altitude above 13 km (Fig. 7c), indicating that the severe overestimation of total rainfall for CTHs higher than 13 km (Fig. 7a) is due to the frequent occurrence of LDAPS forecasts. Furthermore, cold-type forecast events do not exhibit significant mean rainfall with CTHs lower than 13 km, unlike the observation (Fig. 7c). This feature implies that the LDAPS forecasts do not sufficiently produce rain clouds with relatively lower CTHs (<13 km) accompanied by heavy rain cores in cold-type dominant event (remember the type classification were done over the whole of South Korea). It contributed the most to the underestimated mean rainfall for the cold-type forecast events in Table 1. Warm-type forecast events also show the underestimated mean rainfall for the altitude below 12 km compared with the observation (Fig. 7d), consistent with the mean rainfall intensity for the warm-type shown in Table 1. Therefore, the underestimation of mean rainfall for clouds with relatively lower CTHs is a common problem in both cold-type and warm-type dominant events (Figs. 7c and 7d).
c. Forecast skill
The above results are believed to be more affected by physical parameterizations of numerical models rather than forecast errors associated with large-scale dynamics, because spatial or temporal matching is not performed in observation and forecast events. We investigated hit H, false alarm F, miss M, and correct negative C cases with 6-hourly rainfall thresholds for a given 5-km grid in order to quantitatively evaluate the forecast performance of LDAPS rainfall according to the heavy rain types. Here, H indicates that both the observation and forecast produce more rainfall than the threshold for the given period (6 h) and location (5-km grid box), whereas F represents that the forecast produces more rainfall while the observation produces less rainfall. The M and C are the opposite cases of F and H, respectively. The accuracy of the forecast can be considered H + C divided by H + C + F + M. BIAS is the ratio of H + F to H + M, indicating a measure of systematic bias between the forecast and observation. The critical success index (CSI) is defined as H divided by H + M + F, implying the hit probability to event frequency (observation or forecast). The BIAS and CSI scores are popularly used in the forecast verification of rainfall. In this analysis, in addition to physical parameterization, the role of data assimilation with initial condition could be important because the timing of rainfall events greatly influences the evaluation results.
Figure 8 represents BIAS and CSI scores with 6-hourly accumulated rainfall threshold for two heavy rain types. The classification of rain types is based on the LDAPS forecasts, regardless of the observation. The LDAPS 6 h recorded the highest bias score, while the LDAPS 12 h showed the lowest bias score. The results are consistent with the contrast in mean unconditional rainfall between the LDAPS 6 h (7.67 mm day−1) and the LDAPS 12 h (5.18 mm day−1). Likely, some problem occurred in the data assimilation system immediately after initialization, resulting in the estimation of excessive rainfall; this was successively compensated, causing a large decrease in rainfall values in the following six hours. The CSI scores were as high as 0.5–0.6 (cold-type events) and about 0.4 (warm-type events) for light rain cases but those decreased with increasing rainfall threshold as well as prediction time (Fig. 8c). This implies that the quantitative forecast of heavy rain cases is much difficult. It is also worthy of note that the forecast accuracy for the occurrence of cold-type events was 86% (46 + 132 + 78 + 53 + 636 divided by 1104 events), and the hit rate was 69% (46 divided by 46 + 11 + 10 events), as shown in Table 2. However, the accuracy was 78% (46 + 38 + 132 + 10 + 636 divided by 1104 events) and the hit rate was 46% (132 divided by 100 + 132 + 53 events) for the warm-type events. The higher forecast accuracy for the occurrence of cold-type events is consistent with results from BIAS and CSI scores for the rainfall (i.e., Fig. 8). However, it should be noted that only 25% of the cold-type events in the LDAPS forecasts were observed as such, while 54% were observed as warm-type events. The overestimation of occurrences for the cold-type forecast events was still problem to be solved, because it can cause an excessive false alarm. The rainfall forecast for warm-type events shows the BIAS scores for LDAPS 12 h, 18 h, 24 h, 30h, and 36 h are within about 20% (i.e., 0.8–1.2), similar to that of the cold-type events (Figs. 8a,b). However, the BIAS score for LDAPS 6 h, represents a severe overestimation of the warm-type compared to the cold-type. In addition, the CSI scores for the warm-type events are much worse than those for the cold-type (Figs. 8c,d). The lower forecast skill of rainfall for warm-type events is thought to be related with their low rainfall intensity, compared to the cold-type events (Table 1). Figure 9 shows that the BIAS and the CSI scores tended to 1 while the area-averaged unconditional rainfall increased. In fact, the area-averaged unconditional rainfall and heavy rain area (by applying rainfall threshold of 15 mm) exhibited a strong positive correlation (R2 = 0.96) in this study (not shown). Thus, the results are consistent with the result of Roberts and Lean (2008) showing the higher rainfall forecast skill for widespread rain systems than for localized rain systems. The results of forecast skill regarding heavy rain types could provide valuable guidance to real-time weather forecasting in interpreting high-resolution model results over the Korean Peninsula.
4. Summary and conclusions
This study aims at evaluating the performance of the operational LDAPS model over the Korean Peninsula according to heavy rain types. First, we compared the mean spatial distribution of LDAPS rainfall to observations for the period of JJA 2016–18. The mean rainfall over the South Korean land region between observations and LDAPS forecasts was found to be comparable, although some areas exhibited deviations. The PDF distributions of rainfall intensity revealed that the LDAPS forecasts tend to underestimate the frequency of occurrence of light rain but overestimate the frequency of occurrence of heavy rain. In particular, the LDAPS forecast within 6 h showed a strong overestimation for most rain cases greater than 5 mm h−1. This study also investigated the distribution of volumetric rainfall with CTH. The volumetric rainfall obtained from the observations showed two modes over a wide range: it was lower at the CTH of 13 and 15–16 km; these results are consistent with the occurrence of two heavy rain types (i.e., warm-type and cold-type) over the Korean Peninsula. On the other hand, the LDAPS forecasts exhibited a single-mode feature of rainfall mostly concentrated on clouds with high CTH (i.e., cold-type heavy rain). Despite the inevitable differences in the CTH definition between the observation and model forecasts, the huge difference in volumetric rainfall with CTH distribution could be attributed to the limitation of the LDAPS model in capturing observational characteristics of the rain system over South Korea.
Two heavy rain types were objectively classified based on K-means clustering with frequency distributions of CTH normalized conditional on each rainfall intensity. The K-means clustering was applied together to both the observation and LDAPS forecasts. Cold-type events were mostly attributed to CTHs higher than 13 km (mean CTH is 13.9 km), whereas the warm-type events corresponded to a broad distribution of CTH (mean CTH is 9 km), although lower than that of the cold-type events. In total, 67 and 285 events were classified as cold-type and warm-type events in the observation, whereas the LDAPS forecasts exhibited very frequent cold-type events (184 events) but less frequent warm-type events (221 events). The observed cold-type and warm-type events occupied 32.7% and 67.3% of total rainfall over the South Korean land area, whereas the LDAPS forecasts corresponded to 65.3% and 34.7%, respectively. Therefore, the LDAPS rain forecast results largely deviated from the dominant feature of warm-type heavy rain over South Korea. Furthermore, the spatial distributions of rainfall for the cold-type events indicated more rainfall over the western part of the Korean Peninsula both in the observations and LDAPS forecasts. The PDF distributions of rainfall intensity for two heavy rain types were also examined. The overestimation of extreme rain cases were noticeable for the warm-type forecast events, in association with higher CTHs for the warm-type forecast events, compared with the warm-type observation events. Volumetric rainfall with CTH for the LDAPS forecasts clearly exhibited overestimation of cold-type events but underestimation of warm-type events compared with the observation. The overestimation of cold-type heavy rainfall was mainly due to the frequent occurrence of cold-type forecast events, whereas the underestimation of warm-type heavy rainfall was affected by both its low occurrence and weak intensity. The underestimation of mean rainfall for clouds with relatively lower CTHs was thought to be a common problem in both cold-type and warm-type dominant events. Finally, verification results of rainfall forecast skill indicated a significantly lower forecast performance of rainfall for warm-type events compare to the cold-type events (especially for heavy rain cases). These results likely derive from the lower rainfall intensity and smaller rain area of the warm-type forecast events compared to the cold-type events. The overestimation of occurrences for the cold-type forecast events was also considered as a problem that can cause an excessive false alarm.
Because the scope of this study is limited to the analysis of the bulk property of clouds (i.e., cloud-top height), an in-depth study on the microphysical structure of warm-type heavy rain is essential in the future. For example, Song et al. (2017) emphasized the roles of the collision–coalescence process of rain drops for the warm-type heavy rain. The development of warm-type heavy rain also could be related to the rapid growth of frozen drops in the upper levels, as found in videosonde observations (Takahashi 2006; Takahashi et al. 2015). However, the absence of observations in the upper levels is a big obstacle for improving cloud physics in numerical models. Therefore, remote measurements such as satellites and ground-based radars should be actively utilized for improving model cloud parameterizations. In addition, in situ measurements such as videosonde and aircraft observations are urgently required. The KMA has been operating the first South Korean weather aircraft since 30 January 2018. Therefore, the future improvement of cloud physics parameterizations using the aircraft observations over the Korean Peninsula is highly expected.
This study reveals the need to improve the existing numerical weather models for heavy rain forecasting over the Korean Peninsula. We suggest that a proper physical parameterization (primarily microphysics, but interactions with other physical processes) for heavy rain types would improve the prediction of heavy rainfall. In particular, the warm-type heavy rain systems should be suitably represented because they generate most of the rainfall in humid monsoon regions such as the Korean Peninsula. Although this study is conducted only over South Korea, we expect the verification of rainfall forecasts with heavy rain types to be applicable over any part of the world, for which ground and satellite observations are available. Moreover, the evaluation methodology of LDAPS in this study can be applied to other high-resolution numerical models for achieving more quantitative precipitation forecasts.
We are grateful to the three anonymous reviewers who gave constructive advices. This work was funded by the KMA Research and Development Program “Enhancement of Convergence Technology of Analysis and Forecast on Severe Weather” under Grant 1365003081. We thank the KMA National Meteorological Satellite Center (http://nmsc.kma.go.kr) and Numerical Modeling Center (Yong Hee Lee and So Ra Park) for providing observation and model data. The datasets can be obtained at https://data.kma.go.kr.
Supplemental information related to this paper is available at the Journals Online website: https://doi.org/10.1175/WAF-D-18-0140.s1.