One of the main goals of the Sydney 2000 Forecast Demonstration Project was to demonstrate the efficacy and utility of automated severe weather detection radar algorithms. As a contribution to this goal, this paper describes the radar-based severe weather algorithms used in the project, their performance, and related radar issues. Participants in this part of the project included the National Severe Storm Laboratory (NSSL) Warning Decision Support System (WDSS), the Meteorological Service of Canada Canadian Radar Decision Support (CARDS) system, the National Center for Atmospheric Research Thunderstorm Initiation, Tracking, Analysis, and Nowcasting (TITAN) system, and a precipitation-typing algorithm from the Bureau of Meteorology Research Centre polarized C-band polarimetric (C-Pol) radar. Three radars were available: the S-band reflectivity-only operational radar, the C-band Doppler Kurnell radar, and the C-band Doppler polarization C-Pol radar.
The radar algorithms attempt to diagnose the presence of storm cells; provide storm tracks; identify mesocyclone circulations, downbursts and/or microbursts, and hail; and provide storm ranking. The tracking and identification of cells was undertaken using TITAN and WDSS. Three versions of TITAN were employed to track weak and strong cells. Results show TITAN cell detection thresholds influence the ability of the algorithm to clearly identify storm cells and also the ability to correctly track the storms. WDSS algorithms are set up with lower-volume thresholds and provided many more tracks. WDSS and CARDS circulation algorithms were adapted to the Southern Hemisphere. CARDS had lower detection thresholds and, hence, detected more circulations than WDSS. Radial-velocity-based and reflectivity-based downburst algorithms were available from CARDS. Since the reflectivity-based algorithm was based on features aloft, it provided an earlier indication of strong surface winds. Three different hail algorithms from WDSS, CARDS, and C-Pol provided output on the presence, the probability, and the size of hail. Although the algorithms differed considerably they provided similar results. Size distributions were similar to observations. The WDSS provided a ranking algorithm to identify the most severe storm.
Many of the algorithms had been adapted and altered to account for differences in radar technology, configuration, and meteorological regime. The various combinations of different algorithms and different radars provided an unprecedented opportunity to study the impact of radar technology on the performance of the severe weather algorithms. The algorithms were able to operate on both single- and dual-pulse repetition frequency Doppler radars and on C- and S-band radars with minimal changes. The biggest influence on the algorithms was data quality. Beamwidth smoothing limited the effective range of the algorithms and ground clutter and ground clutter filtering affected the quality of the low-level radial velocities and the detection of low-level downbursts. Cycle time of the volume scans significantly affected the tracking results.
The World Weather Research Programme (WWRP) conducted the Sydney 2000 Forecast Demonstration Project (FDP) in conjunction with the Bureau of Meteorology (BoM) of Australia to demonstrate the efficacy of tested state-of-the-art nowcast and severe weather detection algorithm systems. The lead group within the BoM was the Bureau of Meteorology Research Centre (BMRC). The WWRP was formed to promote research and development as well as to foster the technology transfer process to operations for member countries. There are many nowcast systems in development throughout the world and the selection of participants was determined by the operational readiness of the systems, the mix of the various nowcast applications, and most importantly the willingness and resources to participate.
The FDP was held from 5 September 2000 to 16 November 2000. It was scheduled to overlap with the Sydney 2000 (S2K) Olympics (Keenan et al. 2001). The two foci of the project were precipitation nowcasting, on a timescale of 0–6 h, and the demonstration of the use of severe weather detection algorithms (Wilson et al. 2001; Burgess et al. 2001). The radar algorithms attempt to detect, classify, and track the presence of severe thunderstorm cells, mesocyclone circulations, tornadic vortex signatures, downbursts/microbursts, and hail, among others. These severe weather detection algorithms, which are the focus of this paper, are radar based and include the Warning Decision Support System (WDSS; Eilts et al. 1996) from the National Severe Storms Laboratory, the Canadian Radar Decision Support System (CARDS; Lapczak et al. 1999), a fuzzy logic hail algorithm and particle classification scheme using C-band polarimetric radar (C-Pol; Keenan 1999), and a BMRC-modified Thunderstorm Initiation, Tracking, Analysis, and Nowcasting (TITAN) system (Dixon and Wiener 1993; Potts et al. 2000).
The algorithms have evolved over the years and there are design differences between the algorithms and in their configuration. For example, the current mesocyclone algorithm originated from an azimuthal shear algorithm that attempted to identify mesocyclones by tuning the detection algorithm to maximize a skill score (Zrnić et al. 1985). In practice, this led to missed detections and short or nonexistent lead times. While the basic algorithm has been retained, the current practice is to tune the shear detection to maximize the probability of detection (Johnson et al. 1998a; Joe et al. 1994) and couple these detections with a classification approach to reduce the false alarms.
The algorithms were designed to work with different radar systems. For example, the algorithms in the WDSS system were designed to work with S-band radars with a Nyquist velocity of 31 m s−1 or so. The algorithms in CARDS were designed to work with C-band radars using dual-pulse repetition frequency (dual-PRF) data with limited Nyquist velocities of 12–16 m s−1. The algorithms can also be configured differently to reflect local meteorological conditions such as for weaker storms. So the project provided an opportunity to compare the various radar processing systems. Many of the severe weather features identified by the algorithms cannot be reasonably verified against observations (e.g., hail aloft or circulations aloft). Side-by-side intercomparisons provide the only opportunity to validate some of the algorithms, particularly the ones that identify features aloft. Also sparse surface observations make it difficult, if not impossible, to accurately compute false alarms or to evaluate the null case. So the project provided a unique opportunity to take a snapshot of the state of severe weather detection algorithms, to conduct side-by-side comparisons, to cross compare the algorithms, and to determine the transferability of the algorithms using the same radar and with the same data in the same meteorological environment.
In addition, three radars were available to the project. Wollongong was an S-band reflectivity-only operational radar, Kurnell was a new operational C-band Doppler radar, and C-Pol was a research Doppler polarization radar. Each had their own scan strategies and characteristics and provided an opportunity to study the impact of various radar technologies and configurations on the utility of severe weather algorithms.
Each of the software processing systems had their own data and product viewing software. However, in order to present the information from these diverse systems for the BoM forecasters in a cohesive fashion, image products were prepared by the Thunderstorm Interactive Forecast System (TIFS; Bally 2001) using the algorithm outputs for display on a WWRP S2K Web page. TIFS was an interactive Web-based product display and forecast preparation system developed specially for the project.
Two system tests were conducted in the year prior to the main test period. The primary goal was to install the software and hardware systems, adapt them to the radar data feeds and to test the interface between systems. These tests also provided an opportunity to introduce the systems to the forecast office and to collect data to develop the algorithm tunings. Algorithm tuning was not permitted once the project began.
The systems and algorithms are partially described elsewhere (see references). The purpose of this paper is to describe the algorithms with a focus on the details that are not described elsewhere—the impact of the various radar configurations and the resulting operational performance of the systems. A new hail algorithm was developed for the project based on a manual radar technique used at the Sydney weather office. Not all of the algorithms could be cross compared due to system differences. The algorithms that were compared in this paper are listed in Table 1.
We begin by briefly describing the radars, the data, the new hail algorithm, and other algorithm modifications required for the project. We then present examples of the WWRP S2K version of the products, compare and summarize the performance of the algorithms, and discuss the impact of the radar systems on their performance. Only the case of 3 November 2000 provided the opportunity to exercise all of the severe weather algorithms (Sills et al. 2001, 2004, in this issue). Companion papers present additional information on the forecaster reaction to the algorithms and the system on 3 November (Fox et al. 2001, 2004, in this issue), on the independent verification of the algorithms (Brown et al. 2001; Ebert et al. 2004, in this issue), and on the impacts (Anderson-Berry et al. 2004, in this issue).
2. The radars and the radar data
Within the project area, three radars were available—Kurnell (WKL), C-Pol (WCP), and Wollongong (WLB)—as shown in Fig. 1. Each radar was significantly different and performed different scan strategies as described in Tables 2 and 3. For the purposes of this paper, the radars are given three-letter identifiers for convenience. May et al. (2004, in this issue) describe the radars, their characteristics, and their performance. The siting of the radar, the ground clutter, the ground clutter filtering, the dual-PRF unfolding errors, and the calibration can affect the performance of the severe detection algorithms. Considerable effort was made in the two experiment test phases to improve the data quality. In particular, the dual-PRF velocity unfolding technique, the adaptive fast Fourier transform ground clutter technique, and a velocity despiking technique were implemented (May and Joe 2001; Joe and May 2003). Detailed radar calibrations were also performed to match reflectivities from the three radars (May et al. 2004).
The primary project radar was the new operational Kurnell Doppler radar. It is situated near the coast of the Tasman Sea on a tall tower (60 m above sea level). The radar was configured to perform dual-PRF scans with Doppler filtering to remove ground clutter. The radar performed full Doppler scans (reflectivity and radial velocity). The data from this radar were processed by all of the systems.
Due to communication bandwidth limitations, the polarization data from the C-Pol radar (Keenan et al. 1998) were not transmitted to the Sydney Regional Weather Office where the project processing systems were located. Polarization-based image products (described below) were created at the radar site and transmitted to the weather office. Reflectivity and radial velocity data were also transmitted to the weather office.
The third radar was the Wollongong radar, which was the existing operational radar for the Sydney area. It was located 350 m above sea level in hilly terrain south of Sydney. This is an S-band radar with a 1.9° beamwidth. This radar only produced coarse-reflectivity data for echoes greater than 0 dBZ at 64 levels in 1-dBZ steps in contrast with the C-Pol and Kurnell radars, which produce reflectivity data at 0.5-dBZ resolution.
All radars presented different data problems as summarized by May et al. (2004). Sea clutter was prevalent on many days from Kurnell (Fig. 2). Sea clutter was relatively low in intensity, had small radial velocities, and had a very smooth texture. In general, it did not pose a problem for the severe weather algorithms. It did affect the precipitation algorithms (Donaldson et al. 2001). However, the ground clutter at Kurnell was significant and posed a problem for the detection of clear-air echoes and low-level downbursts. The C-Pol radar was not filtered, was located inland, and had less ground clutter but had significant beam blockage in the west due to the proximity of the nearby mountains (Fig. 1). The blockage was not a problem for the project since it was to the west of the experiment domain. C-Pol detected more clear air than Kurnell, in spite of its poorer sensitivity. It was the primary radar for the National Center for Atmospheric Research (NCAR) Auto-nowcaster (ANC) that needed clear-air detection for boundary processing (Wilson et al. 2001).
The data from each of the radars were collected by the BoM three-dimensional radar picture (3D-Rapic) processing system and then reformatted into a Next Generation Radar (NEXRAD)-like data stream. The 3D-Rapic system produced basic image products and provides an interactive 3D viewing capability. The WDSS and ANC systems were designed to accept the radar data on a ray-by-ray basis and required the data to be sent on a user datagram protocol (UDP). The CARDS system was designed as a file-based system and expected a volume scan file of data. So a special ingest program was written to read the UDP network port and create a volume scan file of radar data. The data were sent on an isolated network in order to not interfere with the operational forecast network.
3. Overview description of the systems
a. Canadian Radar Decision System (CARDS)
CARDS (Joe et al. 1994; Lapczak et al. 1999) ran the mesocyclone, downburst, severe weather algorithm, the vertically integrated liquid water (VIL)–based downdraft/gust, and hail and VIL algorithms for the S2K project. The basic mesocyclone algorithm is described by Zrnić et al. (1985) using what is commonly known as the pattern vector approach. The CARDS algorithm was designed to work with C-band data that have small Nyquist velocities or with dual-PRF data where there may be isolated unfolding errors (May and Joe 2001; Joe and May 2003). The inherent assumption of the built-in-dealiasing algorithm was that bin-to-bin velocity differences were less than the Nyquist velocity. This can create breaks in the pattern vectors. However, after the pattern vectors are created by the algorithm, a continuity check was performed where nearby pattern vectors were reconnected and the intervening data were interpolated. This also allowed for data gaps in the data that may arise due to other data quality issues (Joe et al. 1997, 1998). The algorithm identified the location and intensity of contiguous areas of azimuthal shear that are interpreted as circulations. These should not be interpreted as mesocyclones or tornado vortex signatures at face value. The thresholds were purposely set low in an effort to detect circulations as early as possible, which results in high false alarm rates. Multiheight analysis and intensity thresholds were used to classify the severity of the circulation (Burgess et al. 1982). A circulation that has height continuity, time continuity, and is intense is then classified as a mesocyclone. The algorithm was trivially modified to search for clockwise rotations to adapt to the Southern Hemisphere. Shear thresholds were not altered for the S2K project because the existing philosophy was to detect all possible circulations. The maximum detected gate-to-gate shear is an output of the mesocyclone algorithm and is used to indicate the potential of a tornado.
The downburst algorithm was a low-level divergent radial shear detection algorithm. The pattern vector approach was again used and the algorithm looks for contiguous areas of radial shear exceeding a specific threshold on a single elevation scan. No attempt was made to detect features aloft. Experience has shown that when downbursts are detected, an effective strategy for public warning applications was to provide a general downburst warning for all thunderstorms in the vicinity rather than for individual thunderstorms. Due to their short life cycle and various warning dissemination issues (mainly timing and specificity), it was not effective to issue specific downburst warnings. Airport operations in Sydney are concerned whenever any thunderstorm threatens the airport. Dry microbursts are not prevalent in Sydney, so the lack of clear-air echoes from Kurnell was not a significant issue and a simple low-level divergence algorithm sufficed. Due to the ground clutter on the lowest tilt, the detection was done only on the second-lowest-level scan (1.5°) and out to a range of 60 km due to beam height effects.
The VIL downdraft (WDRAFT) or gust potential algorithm was based on an empirical relationship between VIL and the resulting downdraft outflow (Stewart 1991). The original concept was based on the penetrative downdraft theory of Emanuel (1981). An evaluation of the algorithm in Canada showed that it was very effective and provided earlier warning than the low-level microburst detection algorithm by about 10–15 min (Amorim et al. 1997). Studies in Canada found that low midlevel equivalent potential temperature (θe) values were needed to enhance the reliability of the algorithm.
The hail algorithm was based on the work by Treloar (1998). It was an empirically based algorithm where the freezing level, VIL, and the maximum height of the 50-dBZ echo were used to predict hail size (Fig. 3). This was a new algorithm that was developed especially for the S2K project. It was similar to an existing “severe weather” algorithm used in CARDS where the height of the 45-dBZ echo was used to identify severe storms (Donaldson 1961). The hail size was estimated from the maximum height of the 50-dBZ level or the VIL given the freezing level. The algorithm used the larger of the two hail size estimates. The freezing level was determined from the most recent sounding taken at Sydney airport.
b. Warning Decision Support System (WDSS)
WDSS (Eilts et al. 1996) ingests Doppler weather radar data, lightning data, surface data, and other weather data sources. Using image processing, artificial intelligence, and statistical techniques, severe weather phenomena (e.g., tornadoes, hail, and high winds) are detected and forecast. The S2K version ran the Storm Classification Identification and Tracking (SCIT), hail, mesocyclone, and tornado detection (TDA) algorithms. A WDSS table with cell attributes and a cell product indicating storm severity and tracks were available on the WWRP Web site.
The SCIT algorithm (Johnson et al. 1998b) identifies, characterizes, tracks, and forecasts the movement of storm cells. It is based on finding centroids by identification of three-dimensional features. SCIT includes outputs of cell-based VIL, cell base and top, maximum dBZ, maximum dBZ height, and speed and direction of cell movement.
The hail detection algorithm (Witt et al. 1998) estimates the probability of hail, the probability of severe hail, and the maximum expected hail size within a cell. The presence of hail is determined by the extent of the 45-dBZ echo above the freezing level. Severe hail is determined by integrating values of kinetic energy flux at heights above the freezing level (Waldvogel et al. 1979). Great weight is given to values at heights where the environmental temperature is colder than −20°C. Previously derived equations are used to determine the probability of (severe) hail and maximum expected hail size.
The mesocyclone detection algorithm (Stumpf et al. 1998) uses the same pattern vector concept that is used in CARDS. Initial radial-shear thresholds are set low to detect circulations and diagnosis rules are used to classify features as mesocyclones. Through adaptable display thresholds, users determine the strength of circulations they wish to see. Time association logic allows for tracking and trending of vortex attributes. Outputs include circulation, mesocyclone, mesocyclone with base at low levels, and low-top mesocyclone. For S2K, the algorithm was modified to detect Southern Hemispheric circulations.
The TDA algorithm (Mitchell et al. 1998) makes use of the magnitude of the gate-to-gate radial-shear concept by finding and vertically associating 2D shear segments. Time association logic allows for tracking and trending of the detections. Outputs include the tornado vortex signature (TVS; low-elevation angle/height of base) and elevated TVS (ETVS) detections.
c. C-band polarimetric radar (C-Pol)
C-Pol uses a “fuzzy” logic technique developed by BMRC (Keenan 1999), to determine the hydrometeor species, including hail. Membership functions, defined in two-dimensional subspaces of polarimetric power and differential phase measurements, are employed in both radial and Cartesian space. These membership functions are temperature dependent. There are 10 hydrometeor species: drizzle, rain, rain–hail mixture, small hail (<2 cm), large hail (>2 cm), dry graupel, wet graupel, dry high-density snow, dry low-density snow, and wet snow. Rain rates are also estimated by polarimetric techniques and are included in the C-Pol products derived every 10 min. This provided the opportunity to compare the empirical reflectivity techniques to the more physically based polarization techniques (Ebert et al. 2004). Real-time output for the WWRP Web site included an image where hail greater than 2 cm is detected and superimposed on contours of rain rate.
d. Thunderstorm Identification, Tracking, Analysis, and Nowcasting (TITAN)
The Thunderstorm Identification, Tracking, Analysis, and Nowcasting (TITAN) application (Dixon and Wiener 1993) identified and tracked storm cells in radar data and provided a nowcast of their movement. In Australia this application has been integrated with the Bureau of Meteorology's advanced operational radar data workstation, 3D-Rapic, to provide better guidance on storm movement and storm characteristics during convective weather events. The polar volumetric radar data are converted to a Cartesian coordinate system with a horizontal grid of 520 km × 520 km centered on the radar, a vertical grid extending from 0.5 to 19.5 km, and a grid spacing of 1 km in the horizontal and vertical. These data are passed to the TITAN application, which identifies a “storm” as a three-dimensional contiguous region in space for which the radar reflectivity, the volume, and the top exceed defined thresholds. The movement of identified storms is determined by minimization of an objective cost function to achieve a logical match of storms from one time step to the next. The methodology includes procedures for handling merging and splitting of storms, which can occur frequently. The identified storms can be displayed in 3D-Rapic and properties, such as location, size, maximum reflectivity, and movement, were made available to the TIFS for display on the S2K WWRP Web site (Bally 2001).
In S2K, the TITAN algorithm was run on the Wollongong and Kurnell radars with various thresholds. The algorithms are designated TITAN, TITAN2, and TITAN3 (see Table 4). The volume threshold for a storm was either 40 or 50 km3 and the minimum top was 2 km above the surface, giving a minimum area on the order of 5 km × 5 km (see Table 4). The volume threshold was based on the minimum height and the resolution of the Cartesian data such that a storm included a reasonable number of grid points. The height threshold was chosen to minimize the possibility of including echoes resulting from anomalous propagation, which can cause strong echoes over water.
For the Wollongong radar, (volume scans at 10-min intervals), reflectivity thresholds at either 35 or 45 dBZ were used. The lower of these was selected to capture the weaker storms that occur in Sydney in the spring and summer and was of primary interest for airport operations (TITAN2). For the more intense and larger-scale convective systems a “35-dBZ storm” may comprise multiple cores and associated areas of stratiform precipitation. The 45-dBZ threshold was selected to identify storms, or the core region of larger-scale systems, that were approaching severe storm criteria for Sydney and was of primary interest to severe weather meteorologists in the Sydney forecasting office (TITAN). Kurnell radar data (volume scans at 5-min intervals) was also processed using a reflectivity threshold of 35 dBZ (TITAN3).
a. Snapshot of WWRP S2K algorithm outputs
Severe weather was not frequent during the S2K project. Useful algorithm detections were limited to 6 days (Webb et al. 2001). By far the largest number of detections occurred on 3 November 2000 (Sills et al. 2001, 2004). On this day, there were many reports of severe hail and wind and tornadoes over a 10-h period. The most significant storm was a long-lived supercell that began southwest of Sydney. It initially moved toward the southern suburbs, then became a “left mover,” and finally struck the western suburbs with 7-cm hail and three tornadoes. Significant activity occurred between 0400 and 0600 UTC, in the metropolitan area of Sydney, with brief tornadic touchdowns around 0500 UTC. The following paragraphs describe some of the new information that was available through the WWRP Web site for this event.
A traditional presentation of a reflectivity image, from the Kurnell radar, for this storm is shown in Fig. 4 from the CARDS system. The storm (∼20 km WNW of Wollongong) is relatively isolated and traveling from the SSW to the NNE. Diagnosis of severe weather from such data requires manual, intensive, and time consuming investigation of the three-dimensional morphology of each storm in the coverage area to look for radar features associated with the severe weather. The CARDS hail algorithm (Fig. 5) showed 7-cm hail to the southwest of Sydney. This figure shows how the automated WWRP systems provide early detections and simplified visualizations of severe weather phenomena. At the time of this image, reports of large hail had not been received. The previous procedure required the analyst to manually estimate the VIL and the height of the 45-dBZ echo, then to manually look up the hail sizes from a chart (Fig. 3). In this figure, cells are conceptualized as ellipses, predicted locations are presented as semiellipses, and ancillary information (hail, mesocyclones, and microbursts) are displayed in iconic and numeric form. This product was intended for non–radar specialists and was designed to remove all but the essential details. It also acted as an alerting and guidance product to prompt the analyst to investigate further. Color coding was used to indicate the severity of the storm that in this case was represented by storm height. Note that the cell identifications were from the TITAN3 (35 dBZ) algorithm but the hail, mesocyclone, and microburst detections were from the CARDS system.
Track information, available from WDSS at 0445UTC (Fig. 6), indicated the left-mover motion associated with the tornadic supercell (color coded red). In contrast, all other cells show regular west to east motion. The product showed how the WWRP systems were able to discern the most severe of the cells through detection and classification.
Figure 7a shows a WWRP S2K version of the WDSS SCIT table information that was displayed by the TIFS system and Fig. 7b shows an image of the typical elements of a WDSS output screen. There is a wealth of information presented in Fig. 7b that is not available in the simplified WWRP products shown in the previous two figures. The animation, interactive, and highlighting capabilities of the WDSS interface cannot be easily shown in this paper but they play an important role in the design of the algorithm products and in their effective usage. The Sydney forecasters had access to both the WWRP S2K TIFS and WDSS versions of the product. The overall forecast concept was that a simplified TIFS product would be used to catch the attention of the forecaster who would then walk over to the S2K systems for detailed analysis and diagnosis.
Frequently there are numerous storms occurring at any one time, as shown in Figs. 4 or 5. To focus attention on the most severe storms, a TITAN product with higher detection thresholds (Table 4) was produced where only the strongest storms are displayed (Fig. 8). This considerably simplified the amount of information presented to forecasters. While this simple technique was effective in this case where only the Sydney tornadic storm was depicted, thereby greatly simplifying the analysis and diagnosis, other cases may require more information to make warning decisions (see Fig. 17).
At 0440UTC, C-Pol indicated the presence of severe hail (defined as hail greater than 2 cm in diameter) in the tornadic storm, located in the western suburbs of Sydney. There were also cells containing hail farther to the west (Fig. 9). This product indicated the presence of each hydrometeor species in a vertical column. Height information on the species was also available, which will play a role in developing future severe weather warnings products from polarimetric radar. The following species were displayed: rain–hail mix, hail less than 2 cm, and hail greater than 2 cm.
b. Summary analysis
Each software system produced different results due to algorithmic differences, technology differences, configuration differences, warning philosophies, and weather regimes. The algorithmic outputs for hail detection and sizing (CARDS, WDSS, C-Pol), cell identification and tracking (WDSS, TITAN, TITAN2, TITAN3), mesocyclone detection (CARDS, WDSS), microburst/downdraft detection (CARDS), and storm ranking (WDSS) are compared and contrasted in this section. The analysis is based on the entire 24-h dataset for the 3 November 2000 case. Detailed meteorological description of this case is provided by Sills et al. (2001, 2004). The impact of the algorithms is described by Fox et al. (2001, 2004). The verification of the algorithms is described by Brown et al. (2001) and Ebert et al. (2004).
Verification of the hail algorithm is presented by Brown et al. (2001) and Ebert et al. (2004). In this section, the performance is examined by compositing the output of each algorithm over the period 0000–2400 UTC on 3 November 2000. Figure 10 shows a multipanel display of the hail algorithm outputs from C-Pol, CARDS, and WDSS. The algorithm and radar are indicated. Limited validation observations available to this study are presented in the observation (OBS) panel. Note that hail was reported only in the populated suburbs west of Sydney, perhaps reflecting an observation bias due to population density.
On the C-Pol panel (Fig. 10a), hail greater than 2 cm is highlighted. This image was created by compositing all the individual hail images and displaying the largest reported hail size. The C-Pol data indicate swaths of hail embedded within smaller swaths of small hail and hail–rain mixture. The limited observations nicely coincide with the C-Pol hail swath to the west of Sydney.
CARDS processed the data from the three available radars. The C-Pol and Wollongong radars ran on 10-min cycles, whereas Kurnell ran on a 5-min cycle; therefore, twice as many detections are possible. The hail algorithm searched for contiguous areas in the hail size field that exceed a threshold (hail cells). The symbols, in Fig. 10, indicate the centroid location of these hail areas. The squares indicate the size of the maximum hail within the hail cell. The three outputs are quite similar but with fewer detections by the Wollongong radar. The correspondence with the C-Pol polarization results (Fig. 10a) is remarkable. Wollongong showed fewer detections, which is attributable to beamwidth smoothing and to “cone of silence” effects.
The average hail size analysis is presented in Fig. 11. For all the algorithms, the occurrence of hail is underreported compared to C-Pol polarization detections. The average hail sizes that are derived from the CARDS algorithms are strongly dependent on the detection threshold chosen, where lower thresholds result in lower average hail sizes. However, the maximum hail size (previous figure) is independent of the chosen threshold.
The WDSS system reported only hail greater than 2 cm and it showed similar patterns and good correspondence to the detections by CARDS and C-Pol. However, the hail detections are limited in range from the radar. There were no detections at a range greater than about 60 km. WDSS does not report hail in the northeast of the domain, north of Penrith, that is reported by the other algorithms.
Figure 12 shows distributions of the maximum hail sizes reported by CARDS, WDSS, and by the observations. Considering the difference in scan cycle time, scan strategies, and location with respect to the storms, the CARDS detections (Figs. 12b,d,e) were similar on C-Pol and Kurnell. The CARDS detections on the Wollongong radar were similar to the C-Pol results (same scan cycle) except that the peak in the distributions, at around 2-cm hail size, is flatter. In this analysis, the hail from C-Pol was derived from the reflectivity algorithm and the polarization capability was not used. The tails of the distributions at sizes greater than 2.5 cm are almost identical.
The Wollongong radar is approximately 100 km from the storms at the northern edge of the study area where the beamwidth effects are greatest. Those storms contained mostly moderately sized hail (∼2 cm). Also the hail within the area of the cone of silence is mainly moderate. Consequently, there are fewer hail detections of the moderate hail size (see Fig. 12e) from Wollongong compared to those from C-Pol and Kurnell. WDSS reported a much narrower range of hail sizes compared to the CARDS detections or the observations (Fig. 12c).
Figure 12a is a plot of the cumulative probability distribution functions of the maximum hail sizes. The cumulative probability distribution of hail sizes shows a consistency in the reported median size (hail size at the 50% or 0.5 value) except for WDSS. These results are quite remarkable in that the origin and nature of the algorithms are quite diverse.
2) Cell identification and tracking
The various cell tracking algorithms have similar overall performance, generally producing tracks with similar speed and direction (Fig. 13). The differences are attributable to differences in cell identification algorithms, thresholds, and radar scan time. The TITAN-based algorithms were set up to improve the detection of weak cells and to identify cells earlier in their life history.
For the Kurnell radar, there were many more WDSS (45 dBZ; Fig. 13a) tracks than TITAN3 tracks (35 dBZ; Fig. 13b). WDSS identified smaller cells and this resulted in the detection of many more cells, more tracks, and many short tracks. The very short lived storms are difficult to track; consequently, the WDSS results gave the appearance of more spurious and chaotic tracks.
TITAN (45 dBZ; Fig. 13c) and TITAN2 (35 dBZ; Fig. 13d) were run on the Letterbox radar data and showed much fewer tracks than on the Kurnell radar (TITAN3, 35dBZ; Fig. 13b). There are several possible reasons for this result. Wollongong ran on a 10-min cycle compared to the 5-min cycle of Kurnell. A track requires two time steps to be displayed in Fig. 13 and so very short tracks are not plotted. In the case of Wollongong, a cell must exist for at least 20 min compared to 10 min on the Kurnell radar. TITAN3 (WKL) used the smallest values for cell definition (35 dBZ) and volume threshold (40 km3) and should produce more cells and tracks. Wollongong has poorer range resolution and beamwidth, which resulted in smoother data. The application of a ground clutter filter mask on the Wollongong radar created holes in the data (May et al. 2004). These factors contributed to fewer cell detections on Wollongong. Calibration differences were small and were not a significant factor (May et al. 2004).
Errors in storm tracking are more likely when smaller, weaker, and short-lived storms are detected. Experience indicates that this is more of a problem when such cells are fast moving and short lived. This results from difficulties in matching the storm location between consecutive volume scans, and for some applications and in some locations more frequent volume scans may be required. In Sydney, the cell tracking generally performed well with the scan strategies used. The availability of 5-min volume scans from the Kurnell radar allowed an assessment of the impact of improved temporal resolution, and May et al. (2004) showed potential improvements of 5–10 min in the forecast lead time for a given spatial accuracy.
The forecast “storm” movement was based on a weighted mean of the movement of the centroid over past volumes and the forecast storm track was based on linear extrapolation. Forecast locations of the storm were provided up to a maximum of 60 min. In general, the forecast skill was better and the forecast track can be lengthened in time when the storms become more intense. This is illustrated in Fig. 14, which shows the cell position error for 35- and 45-dBZ storms observed on 3 November 2000 for lead times of 10–60 min. Although the 35-dBZ storms include the higher-reflectivity storms, the distribution is dominated by smaller, weaker storms (Potts et al. 2000) and the main contribution to the error comes from the smaller storms. For the most intense storms, a simple linear extrapolation can be quite accurate for over an hour.
Both CARDS and WDSS provided mesocyclone detection algorithms. The algorithms look for contiguous areas of azimuthal shear that are interpreted as circulations. If the circulations are intense and deep enough and have the right vertical structure, they are then classified as mesocyclones (Zrnić et al. 1985). Through operational experience, both systems set their detection thresholds low in order to provide early detections and to detect weak circulations and then classify these circulations into mesocyclones.
Algorithm outputs from the CARDS (WKL and WCP), WDSS (WKL), and the tornado touchdown observations are summarized in Fig. 15. The locations identify the detections and the symbols indicate the intensity of the circulations defined by the maximum azimuthal shear within the feature. It should be noted that the tornado in this case may have originated from a surface boundary and may not have been mesocyclonic in origin (Sills et al. 2001, 2004). All of the algorithms were able to detect intense circulations at the time and at the location where the tornadoes were on the ground (Fig. 15d).
The CARDS system identified more circulations than WDSS. The detections were confined to a south–north elongated area within 50 km of the radar. The tuning for CARDS was primarily developed for the southern Ontario, Canada, weather environment. This environment does not have the propensity for supercells like the Canadian prairie provinces or the U.S. midwest and so the algorithm was configured to detect very weak circulations. WDSS produced fewer detections than CARDS reflecting the difference in the algorithm “tuning” (Fig. 15c).
There was a difference between the C-Pol and the Kurnell detections with CARDS (Figs. 15a,b). In overall pattern, the detections appear to be similar but the circulations are classified differently by the algorithm. While the two radars have different scan strategies, velocity unfolding schemes, and viewing angles, it is not clear why there is such a difference. The C-Pol radar collected single-PRF data and hence the radial velocities fold at 13 m s−1 while the Kurnell radar used a dual-PRF unfolding scheme where the extended radial velocities fold at 40 m s−1. In addition, a median filter was applied to the Kurnell data to remove unfolding errors that could create small areas of false azimuthal shear. However, the mesocyclone algorithm was designed to overcome these factors. There were many weak C-Pol circulations detected compared to Kurnell and this appears to be a range effect.
The CARDS system ran two algorithms to detect strong downdraft winds. One algorithm was a simple divergence algorithm that identifies contiguous area of divergent radial shear to identify downbursts at the surface. The second algorithm is an empirical VIL-based algorithm that identified the potential of precipitation-loaded downdrafts. It is based on work by Stewart (1991) and Emanuel (1981) and the relationship is given by
where VIL is the vertically integrated liquid water and h is the echo-top height. This is called the WDRAFT or gust potential algorithm and W is in m s−1.
Figures 16a–c show the output of the gust potential algorithm for the three project radars. There are many more detections on the Kurnell radar than the other two radars, which was expected. However, there were disproportionately more detections on the C-Pol radar than on the Wollongong radar (Figs. 16b,c). They operate with similar elevation angles and so the discrepancy may be attributed to beamwidth smoothing, the location of the storms with respect to the radar, and slight calibration differences between the two radars. It is not clear why there is a difference in the detections between Kurnell and C-Pol in the northern and northwest parts of the project domain.
Figures 16d,e show the microburst detections. There were very few microburst detections on Kurnell compared to the C-Pol radar and to the gust potential results presented in Figs. 16a–c. Due to strong ground clutter and ground clutter filter effects on the Kurnell radar, the microburst algorithm used the second (1.5°) elevation scan. It was expected that the algorithm would be able to detect near-range divergent radial velocity features; however, very few detections were found. There were radial velocity processing differences but this would not explain the lack of detections on the Kurnell radar.
The WDSS system had an algorithm to rank the storms. This is arguably the most useful of the algorithms since it combines the various algorithms into a single parameter by which to prioritize the storms. The following equation was used to compute the rank weight for each storm:
See Eilts et al. (1998) for a detailed discussion of this equation. In brief, the Rank is dependent on hail size (size), probability of severe hail (POSH), Damaging wind index, and Circulation, with the latter being the strongest determining factor. This provides an aid to the forecaster to maintain situational awareness on the severest of storms. This is particularly important on a busy day with many storms, storms with different severe weather features, or when the forecaster must keep watch over many radars as was the case in Sydney. Using Rank defined by the equation above, the storms can be sorted in ordinal sequence. Figure 17 shows a plot of the ordinal storm rankings for the day. A square indicates that the storm was the top ordinal ranked storm at that time. The figure shows that the storm that tracked to the northwest of Sydney was the top-ranked storm for much of its history and that the tornadic storm near Horsley was ranked number 1 before and during the touchdown phase (cf. with Fig. 15d). The tornadic storm does not necessarily need to be the top-ranked storm, but it was true in this case.
This paper presented a brief summary of the severe weather algorithms used in the S2K project by the CARDS, WDSS, TITAN, and C-Pol systems. These algorithms included the hail, cell identification and tracking, mesocyclone, downburst, and storm rank algorithms. Three radars were available to the algorithm systems with the main project radar being the new operational Kurnell C-band Doppler radar. The other two radars included the S-band Wollongong radar with a 1.9° beam and the polarization C-band Doppler radar. This presented an unprecedented opportunity to compare the performance of the severe weather algorithms among the various processing systems, with various radars, and with various scan strategies, on common weather situations in a real-time operational environment.
There were a few important modifications to the WWRP systems that were required to adjust to the BoM radars and to their scan characteristics. For the CARDS system, the main issue was data ingest. A NEXRAD-like data stream was created for the project and a separate data ingest system had to be written for CARDS. The CARDS algorithms were already configured for a single-or dual-PRF C-band Doppler radars. The WDSS system had to be adjusted for the dual-PRF C-band data with its inherent issues (Joe and May 2003; May and Joe 2001). The mesocyclone algorithms in both systems had to be adapted to the Southern Hemisphere. Sea clutter, anticipated to be a problem for the algorithms, proved not to be a problem. It was of relatively low intensity and had a very smooth velocity texture. It did have an effect on the rainfall algorithms (Donaldson et al. 2001).
The hail detection algorithms from the various reflectivity algorithms were, in general, remarkably consistent with each other and with the polarization radar results. However, the WDSS hail detections were limited to near ranges (<75 km) and were disparate from the polarization or the CARDS results. Due to the sparseness of the observations, it is not possible to conclude whether polarization radars provide more accurate detections though scientifically this is a more pleasing algorithm. In the convective case studied, attenuation was not a factor whereas beamwidth smoothing effects were significant as there were few hail detections at long range from the S-band Wollongong radar. The scan strategy differences did not seem to influence the hail detections. The CARDS hail algorithm was originally based on the Wollongong radar but the algorithm transferred well to the C-band C-Pol and Kurnell radars. The hail size distributions from the CARDS algorithms matched observations remarkably well. WDSS hail size distributions were much narrower than the observations. The hail algorithm comparisons provide a clear example of the benefit of the cross-comparison opportunity afforded by the WWRP S2K FDP.
The cell identification and tracking algorithm results differed mainly in the definition of a cell. Not surprisingly, tracks are better defined with 5-min rather than 10-min radar data. In this meteorological environment and for 10-min radar data, lower cell definition thresholds (35 dBZ for a fixed-volume threshold) identified more cells and this improved the cell tracking. Poor tracks and poor extrapolations result from rapidly moving and short-lived cells (relative to the scan cycle time). WDSS produced the most cells and most tracks compared to the TITAN-based algorithms, which is attributable to the lack of a volume threshold in the SCIT algorithm. WDSS used 45 dBZ as a cell identification threshold. While the track motions appear to have more variance than the TITAN tracks, they still appear to be consistent and correct.
Forecast locations of the storm can be provided to a maximum of 60 min. In general, the forecast skill is better and the forecast track can be applied for a longer period as the storms become more intense. Since the forecast track for a storm is a linear extrapolation of past movement, errors will occur in the initial stages when the cell movement is not well known and when the cell decays. Past studies have shown that most storms move with the mean flow (Byers and Braham 1949; Potts et al. 2000) and ways to improve cell tracking and forecast movement in the initial stages need to be explored. Some account of cell growth and decay based on simple models can also be taken into consideration in cell tracking applications like TITAN and this needs to be explored further in the future. Errors in the forecast movement also arise when there is a change in the storm dynamics that causes it to change direction or to split, when there are interactions with a preexisting convergence boundary that causes it to move along the boundary, or when there is a merger with another storm. Accurately forecasting such changes will ultimately require observing systems and cloud-scale models that are not currently available. Nevertheless, experience in Sydney demonstrated that forecast movement based on a weighted mean of past movement does perform quite well and the forecast track responds quickly to direction changes when the greatest weight is placed on the most recent cell movement.
While all the systems reported mesocyclone detections in the Sydney tornadic storm, mesocyclones are not necessarily related to tornadoes and so the algorithm is very difficult or impossible to verify. In this case, the detections may have been mostly false alarms. WDSS had less detection than CARDS, which was attributable to a greater level of filtering in the classification phase of the algorithm. The CARDS algorithm worked on both single- and dual-PRF radar data and produced similar but not identical results. The single-PRF data show that many of the circulations were weak but it was not clear why this was the case (Fig. 15b). The data record was too short to determine whether this was a bias due to single-PRF data, viewing aspect issues, or just false alarms. The tornado detection algorithm (TDA) in the WDSS system was never triggered. CARDS did not have a separate algorithm for tornado detection; the maximum gate-to-gate shear was reported as part of the mesocyclone algorithm. Circulations were not detected within all cells and this, in itself, acts as a filter in the analysis and ranking of storms. The analyst only needs to interpret the radial velocity data when the algorithms indicate a potential circulation.
The CARDS microburst algorithm on the Kurnell radar did not detect many divergent signatures. While this may have been meteorologically accurate, it was not clear whether this was a result of the use of the second elevation angle (1.5° PPI). Lack of time prevented a better algorithm configuration to overcome the ground clutter and ground clutter filtering issues on the lowest elevation scan. High-quality radar data are needed not just in this but in all algorithms.
The CARDS downburst potential algorithm was less influenced by ground clutter and ground clutter filtering. It was a VIL-based algorithm and depends on having a volume scan of reflectivity data. The differences in the algorithm outputs were probably due to the beamwidth differences between the 1° C-Pol and Kurnell radars versus the 1.9° Letterbox radar, which affect the results at long ranges (>100 km).
Cell ranking is arguably the most important algorithm for the forecaster to maintain situational awareness as it combines the information from all of the algorithms. The purpose of the cell rank algorithm and indeed of all of the severe weather algorithms was to analyze and identify the most significant storms in order to aid the forecaster in determining which storms to manually analyze in detail and in issuing weather warnings. In the case of the Sydney storm, it was the top-ranked storm at the time of the tornado and so cell ranking was successful in providing the necessary alert to the forecaster. This is particularly important if the analyst must maintain surveillance over many radars and over a large forecast domain.
Inherent in the design and interpretation of the algorithm outputs is a conceptual model of a severe storm. This conceptual model is not always applicable. For example, the Midwest supercell model may not be entirely applicable in the Sydney area where sea-breeze boundaries may play an important role. So interpretations of the algorithm outputs may have to be modified or adjusted accordingly by the forecaster.
The purpose of the severe weather algorithms is to provide an alert to the forecaster to initiate an analysis of a specific storm that may be potentially hazardous. The algorithms attempt to produce a first-guess interpretation of data. The algorithms are configured with a high probability of detection and so they are not intended to be unequivocal in identifying severe storm features or to be used without intervention to issue the warning. In particular, the radial velocity fields are inherently ambiguous. The intention is to use them as an aid to the forecaster and to be an integral part of the forecaster's severe weather analysis and diagnosis forecast process. Andra et al. (2002) suggest that the TDA algorithm plays a secondary role in the issuance of warnings. In the particular case of this algorithm, due to radar viewing and resolution issues, the identification of a radar tornadic signature is not certain and the issuance of warnings based on the algorithm outputs is not reliable.
An important aspect of the severe weather systems, not discussed in this paper, is the role of the display station in the decision making process (Fig. 7b). The products, access to the products, display functionality, and workstation performance are intertwined and key elements. If the products were perfect, then interactive display functionality or high-end graphical workstations would not be needed. Within the context of the project, it was neither possible nor practical to provide each forecaster with separate workstations connected to each system. The compromise was to provide simple and consistent summary outputs from each system via the TIFS system, thus providing a common interface to the products and to have the forecaster walk over to the individual systems for further detailed analysis. This led to the formulation of the “decision support” concept where simple summary products act as a gateway to more detailed products that enable and support the forecaster's analysis, diagnosis, and decision making process. Another important conclusion of the project was that easy access to the products and systems enables them to be used effectively. The CARDS system was Web accessible through a Java browser on desktop PCs, which allowed Olympic forecasters supporting the Sydney Olympic Committee on the Olympic Games (SOCOG) to make decisions based on real-time radar data. The interactive functionality was limited but sufficient to be useful in a briefing context. The WDSS had a very sophisticated and mature interface, with considerable functionality, which could display a variety of very detailed storm-based products and images that supported the algorithms but required an advanced and dedicated workstation.
The biggest radar technology influence on the algorithms was the beamwidth. The smoothing effect affected the VIL values resulting in loss of hail detections by the Wollongong radar at long range. The cycle time of the scan strategy was a significant factor in the tracking of cells. While reducing the cell detection thresholds improved the detection of weak cells and thus improved the cell tracking, it was not as significant as reducing the cycle time. The use of a different set of elevation angles does not seem to affect the reflectivity-based algorithms where vertical interpolation is used. However, it was very important for the detection of low-level divergent features. In the convective situation on 3 November 2000, where there was not a lot of intervening precipitation, the effect of C-band attenuation on the overall performance of the algorithms was not discernable. However, this may not be true on an individual scan basis. The impact of wavelength, which reduces the Nyquist velocity, on the algorithms was more difficult to assess but the CARDS algorithm was able to provide similar detections on both single- and dual-PRF data in this particular meteorological environment though the dataset was not sufficient to fully answer this question. Storms with greater azimuthal shear may create issues not germane to this study. Radar siting had an impact on the data quality and on the echo that could be detected. C-Pol was sited inland and could detect considerable clear air “events” whereas Kurnell was sited in a location where the ground clutter and ground clutter filtering had an adverse impact on the quality of the data on the low-level 0.5° scan that was important to the downburst detection algorithm. Calibration differences between radars were a minor influence on the overall performance of the severe weather algorithms.
The severe storm dataset available to this study was not large enough for detailed algorithm verification. Many of the algorithmic features, such as mesocyclones, are not possible to verify directly since they are not readily observable nor observed in independent data. In all cases, the algorithms identified what they are designed to find, such as contiguous areas of reflectivity, azimuthal shear, or radial shear. The reaction of forecasters to the various systems is described elsewhere (Fox et al. 2001, 2004; Anderson-Berry et al. 2004).
Using S2K data, WDSS algorithms are being adapted to Australia and 5-cm radar data. The success of the WWRP S2K FDP will hopefully stimulate future FDPs where severe storm algorithms can again be tested. From the perspective of the participants, intercomparison of the algorithms, as presented in this paper, was the only way to assess the performance of the algorithms since many of the severe weather features found using radar cannot be easily verified. Future FDPs will lead to international cooperation in the development of more robust, highly skilled, and user-friendly algorithms.
Many people contributed to the results of this paper. Phil Purdom, Geoff Freeman, and Sandy Dennis of the Bureau of Meteorology Research Centre constructed and maintained the processing infrastructure during the project. Marie Falla of the Meteorological Service of Canada, and Mike Lehman, Karen Cooper, and V. Lakshmanan of the National Severe Storms Laboratory provided the code modifications and interfaced the CARDS and WDSS systems to the Bureau of Meteorology data streams. Peter May developed the median filtering velocity de-aliasing code that enabled the shear algorithms to function. Bill Conway, DeWayne Mitchell, Arthur Witt, and Janelle Janish of the National Severe Storms Laboratory, and Norman Donaldson, David Hudak, Robert Kuhn, David Sills, David Patrick, and Pierre Vaillancourt of the Meteorological Service of Canada provided the forecast and interpretation support during the project. The contributions of Mike Eilts and J. T. Johnson of the National Severe Storms Laboratory are gratefully acknowledged.
Corresponding author address: Dr. Paul Joe, Meteorological Service of Canada, 4905 Dufferin St., Downsview, ON M3H 5T4, Canada. Email: firstname.lastname@example.org