Concern regarding lightning activity as a precursor of severe weather is increasing. Atmospheric electricity, including lightning phenomena, is one of most serious threats to successful space launch operations. The objective of this study was to evaluate the performance of two different lightning detection networks using a time–range correlation method. Understanding lightning detection network performance enables the weather forecaster to support decisions made regarding space launch operations. The relative detection efficiency (ReDE), observation ratio, ellipse area for 50% probability of location, number of sensors reporting (NSR), time difference, and distance, as parameters that predict system performance, were calculated with the time-range correlation method using cloud-to-ground (CG) flash data from the Korea Aerospace Research Institute Total Lightning Detection System (KARITLDS) and from the Korean Meteorology Administration Lightning Detection Network (KLDN). In this study, 15 thunderstorms were selected from 2008–09 data. A total of 41 192 and 28 976 CG flashes were recorded by KARITLDS and KLDN, respectively. In all, 19 044 CG flashes were correlated as being the same flash. The observation ratios, ReDEKARITLDS, and ReDEKLDN were calculated as 1.42, 0.66, and 0.46, respectively. Eighty percent of CG flashes detected by the KARITLDS (KLDN) had elliptical areas less than 5 km2 (12 km2), where the elliptical areas were defined as having a 50% probability of containing the CG flash. Two regions showing a high observation ratio were due to high KARITLDS detection efficiency and to the blocking of electromagnetic wave propagation by Mount Hanla at 1950 m above sea level.
Natural and triggered lightning poses a significant threat to space launch vehicles, launch complexes, and ground range tracking systems at space centers. As of 1 March 2004, a total of 13 of 162 Ariane 4 and 5launch operations at the Centre Spatial Guyanais had to be postponed because of weather conditions (Isabelle 2004). For the Eastern Range at Cape Canaveral and the National Aeronautics and Space Administration (NASA) Space center, a total of 4.7% of the launches from 1 October 1988 to 1 September 1997 were scrubbed and 35% were delayed because of Lightning Commit Criteria (LCC; M. W. Maier 1999, unpublished manuscript). The cost impact of launch operation scrubbing varies from 150 000 U.S. dollars (USD) to over 1 000 000 USD, depending on the launch vehicle (Roeder et al. 1999). For the Korean space center, Naro Space Center, its location on the southern coastal area of the Korean Peninsula puts it in one of the areas that has among the highest lightning densities in South Korea. (Kuk et al. 2010)
Most space centers have established and now operate Weather Launch Commit Criteria (WLCC), including LCC, to check weather conditions, including atmospheric electrical phenomena, vertical and horizontal distribution of clouds, and ground and upper wind speeds, for safe launch operations. The LCC is an essential part of WLCC because it sets the criteria for minimizing natural and triggered lightning hazards to launch vehicles during their ascent and for vehicle electrification due to atmospheric interactions. Weather forecasters at the space center have been focused on forecasting the onset probability of lightning and on the nowcasting of lightning activity.
Many studies on lightning characteristics have been conducted to establish the relationship between lightning parameters, including density and weather radar echo, rainfall rate, and brightness temperature from geostationary weather satellites. (Hohl and Schiesser 2001; Holler et al. 2000; Machado et al. 2009; Piepgrass et al. 1982; Seity et al. 2000). For lightning structure, Stolzenburg has shown a complex structure for thunderstorms by studying individual storm cells, mesoscale convective system (MCS), and supercell storms (Stolzenburg et al. 1998). Krider has also shown that cloud-to-ground (CG) flashes tend to cause electrical field changes when temperatures ranged between −10° and −20°C (Krider et al. 1996).
Many scientists and engineers have endeavored to evaluate lightning detection network performance in terms of detection efficiency (DE), location accuracy (LA), peak current estimation error, and type classification error. A number of different experimental campaigns, such as rocket-triggered lightning validation, tower lightning evaluation, video validation, numerical modeling, and network intercomparison validation have been carried out for determination of ground truth data (Diendorfer et al. 2002; Idone et al. 1998; Jerauld et al. 2004; Maier and Wilson 1996; Parker and Krider 2003). Two methods in particular that used peak current distributions and multiplicity distribution for derivation of relative detection efficiency were explained by Cummins (Cummins and Bardo 2004).
For reliable and accurate forecasting and nowcasting of lightning to support launch operations, good data quality from weather observation systems such as weather radar, lightning detection networks, electric field mills, and geostationary weather satellites is very important. When a meteorologist forecasts lightning, an understanding of the performance of the weather observation system plays a vital role as well.
The present study evaluated the system performance of the networks operated by the Korea Aerospace Research Institute (KARI) and Korean Meteorology Administration (KMA), in terms of relative detection efficiency for lightning detection. For the purpose of this study, sets of CG flash data observed by the KARI and KMA lightning detection networks and the time correlating method for individual flash matching were used.
2. Data and methodology
a. Lightning detection networks in South Korea
Three lightning detection networks are operated by different organizations in South Korea. The Korean Electricity and Power Corporation (KEPCO) operates the KEPCO Lightning Detection and Information Network (KLDNet) to provide intelligent design and engineering support for lightning protection and maintenance in electric power generation and transmission. The KMA Lightning Detection Network (KLDN) consists of 7 Improved Accuracy from Combined Technology Enhanced Sensitivity Performance (IMPACT ESP) sensors for low frequency (LF) sensing and 14 Lightning Detection and Ranging (LDAR) sensors for very high frequency (VHF) sensing. The purpose of KLDN is to identify thunderstorms early to support severe weather forecasting and convective system monitoring over the Korean Peninsula. The KARI Total Lightning Detection System (KARITLDS) that is dedicated to space launch operations at the Naro Space Center has operated since 2007. The KARITLDS with its electric field mill sensor can intensively monitor lightning activities, including cloud-to-cloud (CC) discharge, around the space center with high accurate and sensitivity. The details for three lightning detection networks are summarized in Table 1. In this study, CG flash datasets from KARITLDS and KLDN were analyzed.
b. Selection of storm days and research area
As shown in Tables 2 and 3, 15 storm days were selected for this study. These cases were the days in which more than 1000 CG flashes had occurred in research area 2 (see Fig. 1). The statistical study of lightning over the Korean Peninsula revealed two areas that showed maximum flash density: the southern region of the Korean Peninsula and the southwestern inland region (Kuk et al. 2010). This maximum density occurred because weather systems tend to develop and pass from the southwest to the northeast or from the northwest to the southeast.
To derive the relative detection efficiency, it is necessary to determine a CG flash dataset regarded as truth data (the real number of flashes occurring at each research area). In this study, the CG flash data from KARITLDS are regarded as truth data. As shown in Fig. 1, KARITLDS has a small network coverage dedicated to launch operations at the launch facility, while the 90% coverage of the KLDN detection efficiency was the entire Korean Peninsula. The 90% coverage of detection efficiency of KARITLDS was smaller than that of KLDN. In the present study, two research areas were set up, as shown in Fig. 1. Research area 1 was located between 33.0° and 36.0°N and between 125.0° and 128.0°E, which allowed evaluation of the overall geographical features of CG flashes, observation ratios, and total and ellipse areas, etc. Research area 2 was located between 34.0° and 35.0°N and between 126.0° and 128.0°E, corresponding to 90% coverage of detection efficiency for both of KARITLDS and KLDN. The results for research area 2 were used for comparison of system performance with the same detection efficiency.
All geographical figures in this study were constructed from 900 grid blocks for research area 1 and 200 grid blocks for research area 2, with 0.1° longitude and 0.1° latitude spacing. This corresponds to a spatial resolution of 9 km in the east–west direction and 11 km in the north–south direction. The location of sensors and the detection efficiency contours for KLDN and KARITLDS are presented in Fig. 1.
c. Location-based flash grouping algorithm
KLDN and KARITLDS employ the same low-frequency sensing technology, which is combination of time of arrival (TOA), and magnetic direction finding (MDF) for CG stroke monitoring. Many previous studies have dealt with the principal of sensing technology and the advantages and disadvantages of each technology (Cummins et al. 1998; Lim and Lee 2005; Holle and Lopez 1993). For CG flash data comparison, we reprocessed stroke data from KLDN and KARITLDS to group into flash data using a location-based flash grouping algorithm, as explained by Cummins (Cummins et al. 1998). This is based on spatial and temporal clustering techniques. The advantage of this algorithm is that it avoids overestimated multiplicity, compared to an angle-based flash grouping algorithm (Cummins et al. 1998). The detailed parameters, such as flash spatial radius, determination of polarity, and maximum multiplicity per flash on the location-based flash grouping algorithm are summarized in Table 4. As explained in the previous study by Cummins (Cummins et al. 1998), positive CG flash data lower than 10 kA are regarded as intracloud (IC) flashes and are excluded from the CG flashed dataset.
d. Relative detection efficiency and time correlating for flash matching
The absolute DE of a CG flash can be defined as the fraction of CG flashes detected as a proportion of the total number of actual CG flashes occurring in nature. To acquire true CG flash data from nature, experimental efforts such as rocket-triggered and video camera experiments are necessary. For this reason, many previous studies have endeavored to understand and evaluate relative system performance by using CG flash data observed by different networks. The relative DE of a CG flash is defined as the ratio between the number of CG flashes detected by one network and the number of CG flashes detected by another network. Thus, we can calculate relative detection efficiency (ReDE) as
where ReDEA is “relative detection efficiency (A out of B)” and N(B) denotes the number of CG flashes detected by the B network. (Drue et al. 2007)
To match flash data from different networks, temporal and spatial separation are employed. If both the time difference and the spatial separation are below threshold levels, then the two signals are regarded as the same flash. The threshold levels applied in this study were less than 1.0 s and less than 25.0 km, for temporal and spatial separation, respectively. (Drue et al. 2007; Schulz et al. 2000) The result of previous studies, absence of available data of physical parameters of lightning data, and inherent ambiguity of observation data due to GPS time stamping resolution and flash grouping algorithms for data conversion of stroke data to flash data were considered carefully when the temporal and spatial separation threshold levels were selected. The method and threshold levels for flash matching are same with those used by Drue et al. (2007) and Schulz et al. (2000) KARITLDN and KLDN used GPS clock system for time tagging with 500 ns. The stroke data from KARITLDS and KLDN were converted to flash data using flash groping algorithm with 1.0-s flash time window and other parameters summarized in Table 4.
3. Results and discussion
a. Observation ratio of CG flashes and relative detection efficiency
1) Research area 1
The number of CG flashes detected by KARITLDS and KLDN, the number of CG flashes that were time-range correlated, the observation ratio, and the ReDE for 15 thunderstorm cases for research area 1 are summarized in Table 2. In all, 109 748 and 86 855 CG flashes were detected by KARITLDS and KLDN, respectively. The observation ratio, as the ratio number of CG flashes from KARITLDS and number of CG flashes from KLDN, was calculated as 1.26. A total of 51 741 CG flashes were correlated with the 1.0-s temporal constraint and 25.0-km spatial constraint.
As shown in Fig. 2, there are two regions (region 1: 33.4°N, 126.1°E; region 2: 34.2°N, 126.3°E) that present a high observation ratio. Region 1 shows a high observation ratio, possibly because the electromagnetic waves from CG flashes occurring around region 1 are unable to reach the KLDN detection antenna on Mount Hanla, at an altitude of 1950 m above sea level (see Figs. 1 and 3). In contrast, KARITLDS can efficiently sense the CG flashes occurring at the region 1 area because the KARITLDS detection antennas were positioned in the southwestern coastal area, as shown in Fig. 1. As compared with the observation ratio for region 1, the high observation ratio for region 2 seems to have originated from the high detection efficiency, since KARITLDS antennas are concentrated in the southwestern area (see Figs. 1 and 4).
As described in section 2c, we employed the concept of probability of detection to derive the ReDE for evaluation of the lightning detection network performance. The ReDEKARITLDS (i.e., the relative detection efficiency of KARITLDS in comparison with KLDN) was calculated as 0.66 for 15 cases; whereas ReDEKLDN was calculated as 0.46.
2) Research area 2
As shown in Table 3, KARITLDS detected 41 192 and KLDN detected 28 976 CG flashes for research area 2, giving an observation ratio of 1.42. In the whole of research area 2, KARITLDS observed CG flashes more often than did KLDN, except in the 127.5°N, 128.0°E and 34.0°–34.5°N regions. The location of the KARITLDS site was thought to lean toward the west, as shown in Fig. 1. The 19 044 CG flashes were correlated with a 1.0-s temporal constraint and 25.0-km spatial constraint. As summarized in Table 3, the ReDEKARITLDS (i.e., the relative detection efficiency of KARITLDS in comparison with KLDN) was derived as 0.66 for 15 cases; whereas ReDEKMATLDS (i.e., the relative detection efficiency of KLDN in comparison with KARITLDS) was calculated as 0.46.
b. Ellipse area of 50% probability of location
The ellipse area of 50% probability of location is one of the parameters presenting data quality of the observed flashes, especially location accuracy. As the ellipse area gets larger—in other words, as the semimajor and semiminor portions of the ellipse increase in size—the location accuracy decreases. As shown in Figs. 5 and 6, 80% of the CG flash data detected by KARITLDS and KLDN for research area 1 has an ellipse area of less than 15.0 and 60.0 km2, respectively. The 50% ellipse areas of CG flashes from KARITLDS and KLDN were 40.0 and 200.0 km2, respectively. For research area 2, the ellipse area for 80% of CG flashes from KARITLDS and KLDN was 5.0 and 15.0 km2, respectively, as shown in Figs. 7 and 8. KLDN showed an ellipse area distribution that was broader than the ellipse area of KARITLDS for both research areas 1 and 2. The location accuracy of KLDN was thought to be lower than that of KARITLDS in the research areas because the baseline of the KLDN sensor was longer than that of KARITLDS.
c. Number of sensors reporting, time differences, and distance
The number of sensors reporting (NSR) for KARITLDS and KLDN—one of the parameters of the performance of the lightning detection network—is indicated in Figs. 9 and 10, respectively. To estimate the result of the flash matching method by time correlated with range, the time difference and distance are presented in Figs. 11 –14. As shown in Fig. 1, KARITLDS and KLDN consist of 5 and 7 LF sensors that observe CG flashes. Thus, the maximum NSR is 5 and 7 for KARITLDS and KLDN, respectively. As shown in Fig. 9, most of CG flashes of KARITLDS are reported by 3 sensors, while most flashes of KLDN are reported by 2 NSRs in research area 1. In addition, KARITLDS and KLDN detected more than 20 000 and less than 9000 CG flashes with 5 NSRs in research area 1. This means that more KARITLDS sensors than KLND sensors were used in the location calculation of CG flashes. Thus, KARITLDS revealed lightning with higher detection efficiency and location accuracy than did KLDN. As shown in Figs. 11 and 12, 80% of the correlated CG flashes had a distance less than 15.0 and 12.0 km, for research areas 1 and 2, respectively. Most of correlated flashes showed time differences of <0.05 s, as shown in Figs. 13 and 14. To demonstrate the symmetry and flatness of the time difference distribution and distance distribution, skewness and kurtosis are summarized in Table 5. The high kurtosis observed indicates that the time distribution has a distinct peak near the mean that is zero in this distribution.
In this study, CG flash data for 15 storm days from KARITLDS and KLDN were compared and analyzed to evaluate system performance. The relative detection efficiency, observation ratio, ellipse area for 50% probability location, and number of sensors reporting were used as measures of system performance. The ReDE with a time-range correlating method was used for system performance evaluation, since the true number of CG flashes in nature was unknown. The ReDEKARITLDS and ReDEKLDN in research area 2 were calculated as 0.66 and 0.46, respectively. In research area 2, KARITLDS was found to observe 1.42 times more CG flashes than did KLDN. The reason for two points showing a high observation ratio is discussed. The KARITLDS seems to have higher location accuracy than KLDN, because 80% of the CG flashes from KARITLDS had less than 5 km2 ellipse area, whereas this value was 15 km2 for KLDN.
The results from this study will be useful for weather forecasters at space centers that need to make decisions, as the study supports launch operations with lightning data from two different networks.
Corresponding author address: Bong-Jae Kuk, Bongraemyeon Yenari 1, Naro Space Center, Goheung 548944, South Korea. Email: firstname.lastname@example.org