a. Systems present and products displayed

The systems present in Sydney and their origins are shown in Table 1. The system displays had all been designed to operate in isolation, as prior to this project each system had been used alone. There was little time available to give full consideration to the complementarity of the systems and how best to optimize displays with minimum repetition. This led to discrepancies between color scales and some nonoptimal selections of product displays. The cell tracking displays used a small range of colors in an attempt to indicate storm severity as classified by cell-top height. Also prior and forecast tracks were not clearly distinguished by color.

The details of the products produced and displayed by each system can be found in Keenan et al. (2002) and Fox et al. (2001) and are summarized in Table 2. An example of the display from 3 November 2000 is shown in Fig. 1. Twelve panels were displayed on the two screens of the interactive Web display. The two screens had a broad separation between severe weather nowcasting and precipitation products. All of the panels displayed the same cutout area corresponding to the New South Wales (NSW) Severe Weather Warning Area (SWWA). This encompassed the Sydney metropolitan area and the entire range of Olympic sites.

It was possible to click with the mouse on many of the 12 panels. This would bring up a display related to the image clicked on, which would replace half of the 12-panel display. The information accessible by clicking on an individual panel varied and the products available by this procedure are listed in Table 2 for the systems discussed in this paper. The usual image available from a system is referred to as the front panel and that obtained by clicking on this image is referred to as the rear panel. It was very noticeable during the FDP that the forecasters had little time to investigate the information contained in the rear panels. Therefore, the systems that presented their most important features “up front” were of greater use, especially during severe weather events. This was particularly evident during 3 November, as the forecasters were faced with a multitude of tasks and decisions for a lengthy period. Those systems whose major output took the form of animated loops that could only be viewed at the expense of covering other displays were less used. For this reason, this paper concentrates on the impact of the products that were readily accessible, but reserves some space for recommendations as to how the other products could be made more available.

All of the systems analyzed and generated products for a wider area than shown on the common Web display (Fig. 1). In particular, the systems designed to nowcast widespread rainfall displayed only a small fraction of their analysis domain. This restricted the application of their products, as the forecast motion of large-scale structures could not be observed. It was felt, overall, that the consistency of product display was preferable to each system showing the domain considered most suitable by that system's designers.

This paper concentrates on a single series of convective events during which the focus of attention was on severe weather; therefore, little consideration is made of the systems and products that were designed to forecast precipitation fields [the Nowcasting and Initialisation for Modelling Using Regional Observation Data (Nimrod; Golding 1998); the Spectral Prognosis approach (S-PROG; Seed 2003)]. As it happens, the precipitation products [Nimrod, the Generating Advanced Nowcasts for Deployment in Operational Land-surface Flood Forecasts system (Gandolf), and S-PROG] were those that had little additional information on their front-panel display and little information pertinent to severe weather situations on their rear panels. They were, however, used extensively and appreciated during other situations (Anderson-Berry et al. 2004, in this issue).

It is notable that some systems used different detection methods to derive the same feature. For example, in the case of hail, the C-band polarmetric radar (C-Pol) used a direct observation by dual-polarization radar (Keenan et al. 1998), while the Canadian Radar Decision Support system (CARDS) and the Warning Decision Support System (WDSS) used different empirical algorithms that related vertical radar reflectivity profiles within the storm to the occurrence of severe hail. CARDS used a relationship dependent upon the maximum height of the 50-dBZ reflectivity and the freezing level. The WDSS hail detection algorithm is based upon a statistical relationship to assess the probability of hail from the size of the convective cell and the vertically integrated liquid water (VIL) (Kitzmuller et al. 1995). The CARDS algorithm, actually developed locally in the New South Wales (NSW) forecast office (Treloar 1998), is a function that yields maximum hail size from the height of a number of radar reflectivities within the storm.

b. Products available at the WWRP desk

Due to limitations on the display available on the forecasters' desk, many useful products were not shown. Table 3 lists the products that each system produced and displayed, both on the Web-based display available at the forecaster desk and on their own systems in the WWRP area. The WWRP system operators were able to view more information from their own systems and were able to analyze and interpret that information in addition to the information available to the forecasters. Therefore, if a WWRP representative was able to determine some important characteristic of a storm that was not available to the forecasters, that information could be passed to the WWRP “champion” (discussed below), who could then pass it on to the forecasting team. This ensured that valuable information could reach the forecasters with minimal disruption to operations. As the forecasters became more familiar with the systems, they often approached the representatives of individual systems to discover more specific information.

In most cases the WWRP desk display covered the entire domain for which the system produced nowcasts, rather than the selected area available on the combined Web display. This enabled some features to be seen sooner on the WWRP desk than on the Web display. The Auto-nowcaster (ANC), WDSS, and CARDS had the ability (on the WWRP desk) to zoom in on selected areas of interest and look in detail at storm features. This allowed for close examination of the structure of cells and their potential for severe weather. Part of the CARDS display was available as a separate interactive Web tool. This allowed forecasters to view reflectivity cross sections through cells along any chosen direction (not only radials).

Also in Table 3 is a record of those products that the WWRP representatives considered the most valuable for nowcasting. There is a general consensus regarding some of the most valuable indicators of severe weather, for example, the direct notification of the maximum reflectivity in a cell and the height at which it occurs.

In the final column is a list of products that have been developed since the 2000 project, which would be available to forecasters if systems were installed now. A number of these new products have come about as a result of the experience gained in Sydney and the cross-pollination of ideas resulting from the interaction of the different groups present.

c. Forecaster interaction

One of the primary objectives of the project was to demonstrate the nowcasting technology in an operational forecast setting. The Bureau of Meteorology (BoM), as project host, allowed the products of the nowcast systems to be made available to its operational severe weather forecasters under a strict protocol. This involved the composite display system designed within the bureau (described above) and a roster of designated project representatives who liaised with the operational forecasters. During a severe weather event only the designated representative on duty, or champion, was to interact with the forecasters. All decisions regarding the issuing of alerts and warnings lay with the operational forecast team. One experimental tool specifically designed for the FDP was Thunderbox (now referred to as TIFS) (Bally 2001), which used FDP cell track products as a basis for warning areas.

This allowed semiautomatic dissemination of graphic and text warnings to selected clients (e.g., State Emergency Services). Although the tool was managed by the FDP participants, it was overseen by the forecasters and all warnings issued had to conform to the current forecast policy, which was disseminated by “traditional” means. All warnings issued, whether or not they used FDP products, and, in particular, if they were issued through the Thunderbox system, had to agree with the official warning policy of the BoM office valid at the time. The official text warnings remained a priority and the Thunderbox warnings were often issued in a less timely manner, but were used in conjunction with the official warnings. In theory the Thunderbox system could have expedited the issuance of warnings, but, naturally, the text warnings were given priority during the project.

The WWRP champion was responsible for the interpretation of nowcast products and drawing the attention of the severe weather team to important features within them. This gave the forecasters exposure to the novel products without being crowded and confused by all the individuals involved in the project. The display included a text summary of each system's operational status and a message board to highlight any notable features identified by the systems. This was completed by the champion and could be updated at any time.

Prior to the event of 3 November the setup had worked reasonably well, but with a limited amount of severe weather there had not been a significant test of how the forecasters would use the available information or how the interaction with the WWRP groups would proceed. The regime, in fact, proved effective, with efficient transfer of information to the forecasters.

a. Storm initiation

In the early stages of storm development several small cells formed, but it was difficult to anticipate which deserved attention. Early in the event (about 0130 UTC) the forecasters' attention was drawn to the possible presence of hail, diagnosed only by the FDP systems, in a cell to the northwest of Sydney. This was the first indication of severe weather and allowed for a severe thunderstorm warning (STW) to be made earlier than it would have been by the forecasters using conventional systems alone.

The greatest benefit in the initiation stage was the identification of those cells most likely to develop strongly. This was achieved to some extent by Gandolf, but most successfully by ANC. This system predicted the interaction between the sea-breeze front and the most southerly of the cells visible on the radar at around 0200 UTC (Fig. 3). The system showed the imminent collocation of the cell and the sea-breeze front and the probability that the storm would intensify through this interaction. On the other hand, this image fails to show the subsequent steering of the cell along the front. The Thunderbox product, shown in Fig. 4, used the Thunderstorm Identification, Tracking, Analysis, and Nowcasting (TITAN) nowcast track to generate a threat area. In preparing these warnings the forecasters used the tracks provided as a base and then extended the envelope of the threat area within the Thunderbox system to encompass what they perceived to be the true extent of possible severe weather (Bally 2001). This in general indicates a measure of the uncertainty in both cell speed and direction indicated by these products. The tracking algorithms showed the trajectory of the cell over Bowral toward Wollongong allowing warnings of severe hail to be disseminated to the emergency services and the media for these areas. However, all the systems showed the motion of the storm to be primarily eastward. There was no indication of the northward turning that was observed when the cell encountered the sea-breeze front, as the tracking algorithms do not allow for deviation from a linear path. The information regarding the position of the sea-breeze front from ANC provided an indication that turning may occur, but this was only incorporated into warning products once it began. Therefore, there was no apparent threat to the Campbelltown area at that time, nor for the succeeding hour, but a warning for the Wollongong coastal district was issued. There was also an emphasis on the threat posed by the cells to the northwest of Sydney, which were diagnosed as moving toward the western suburbs. In fact, although cells continued to develop in the Blue Mountain region, none of these propagated into the coastal plain. This demonstrates the deficiency of the cell tracking systems: the forecasters' experience was that these cells would dissipate as they moved to the east, but the automated systems, which had no development/dissipation routines, produced forecasts of cell positions for these dissipating storms. On the other hand, as Fig. 3 shows, systems such as ANC, which had cell development routines, did forecast the dissipation of these cells. The threat area delineation based on the 0200 UTC tracks was quickly adjusted by the issuing of a further warning at 0220 UTC, which showed the storm steering farther northward toward Campbelltown (Fig. 5). This latter warning was issued as the forecasters noted the imminent convergence and recognized the potential for development and steering.

All this led to a forecast of severe convection inland of Wollongong that proved very accurate. The convection was initially forecast to move to the east (as indicated by the standard cell tracking systems such as TITAN), but it was evident from ANC that there would be some steering of the cell along the sea-breeze front, along with development of convection in this area. The information was conveyed to the forecasters and they were able to monitor the situation in this area and observe the rapid development earlier than they might otherwise have done. This provided an opportunity to warn for areas in the path of the storm of the probability of severe weather, and this was certainly the case for Campbelltown where 4–5-cm hail was later reported.

b. Storm development

The systems in place provided a great deal of information regarding the environment in which storms were developing and the progress of this development. Much of that information was more accurate and at a far better resolution than that available from models, which had no indication of the small-scale features critical to maintaining powerful storms.

One system designed to explicitly forecast storm development was Gandolf, which is designed to forecast storm development using a life cycle model of thunderstorm evolution (Pierce et al. 2000). Gandolf uses mesoscale model data to determine the atmospheric environment within which the storm exists. This exhibited problems that stemmed from the coarseness of the mesoscale model data that were crucial to its diagnosis of storm development (Sleigh 2002). Thus, it did not forecast cells to develop as well as it was intended to do.

ANC, with its ability to identify convergence lines and, hence, areas of ascent, produced the most useful information regarding possible storm development. While ANC displayed detected convergence lines, the interpretation of the information these provided was not clear. There was a great deal of uncertainty as to whether cells would develop at the sea-breeze front prior to crossing the boundary and then dissipate, or if they would be steered along the front, resulting in continued and prolonged development of cells and motion toward the metropolitan area.

As shown by Sills et al. (2004) there was a complex interaction between several low-level boundaries, and the development of the mesocyclones and tornadoes could not have been unambiguously predicted with the current understanding of such phenomena. Not all these small-scale features could be clearly resolved in real time and their influence on the storm system was not possible to predict. However, strong indicators of environmental conditions conducive to maintaining the severe weather, such as convergence zones, were available from ANC and this information was used.

The forecasters appeared less concerned with future development than with the current state of the storm and its projected path. This was natural given the constraints upon them in terms of time, the development-related products available, and the requirements of the forecast users. They used other products and their knowledge of storm behavior to judge the likely progression of storms based upon the environment into which the storms were moving.

The use of information to gauge convective development and forecast continued development is hard to assess. The systems specifically designed to diagnose these aspects (Gandolf, ANC) did not have complementary display systems that enabled the forecasters to analyze the information easily. To assess the current state of the cell and its development trend, the forecasters, under the time constraints of a critical situation, used more accessible and familiar tools. The operational radar viewer used by the BoM, a three-dimensional radar picture (3D-Rapic), enabled the forecasters to examine the overall structure of the storms, identifying features such as bounded weak echo regions (BWERs), high-altitude reflectivity cores, and so on. The forecasters gained much from the combination of these with the information relayed to them by the WWRP participants, who had expertise in the interpretation of their own products.

c. Storm motion

The impression of users at the time of the event was that, although the cell tracking algorithms (WDSS and TITAN) did a good job, they were occasionally unstable in that they oscillated to the right of the main, left-turning cell. Wilson et al. (2004, in this issue) reveal that, although the tracks were far less erratic than thought, they did indeed tend to show the storms moving farther to the right than actually occurred. The tracks imply a single coherent storm, whereas over the period of hours a succession of cells developed and dissipated. However, during the critical, most damaging phase of the storm a single supercell tracked more slowly than, and to the left of, the forecast positions. This did not appear to prevent the forecasters from using the information to augment their expertise and produce excellent forecasts of areas under the threat of severe weather.

The important point is that a single glance at system output can be misleading. It is natural that, for a given forecast, the track may be in error, but by repeated viewing one can determine the turning of the storm (from the visible past track) even when the forecast track is in error. The forecast tracks also formed the basis of the Thunderbox (Bally 2001) dissemination system and their reliability was therefore critical to the success of that system.

The forecast speed of cell motion appears to have been very well analyzed by the tracking routines [the Storm Cell Identification and Tracking algorithm (SCIT), TITAN]. An examination of the forecast tracks of the storm centroids as produced by the WDSS and TITAN systems, especially those used and broadcast with Thunderbox, revealed many instances when the projected position of the storm corresponded to observations of storm locations up to 1 h in advance. It must be made clear that the forecasters did not use the tracks in such a deterministic manner. Forecasters have an image of the area that may be affected and issue warnings accordingly. This is shown by the way the warnings issued through Thunderbox used the tracks as a basis around which an envelope, or “threat area,” where severe weather is likely to be experienced, is delineated.

A true measure of the accuracy of the forecast storm tracks (Brown et al. 2001) may not realistically reflect the value of the forecast as an indicator of threat to a given area. An example of this, the threat area delineated by the Thunderbox product at 0303 UTC (Fig. 6), was based upon the TITAN tracks of that time and encompassed Campbelltown. This suggested that the storm would reach that area approximately 60 min hence. The tracks produced 50 min later retained the threat to this town and indicated that the storm would arrive in the following 10 min (Fig. 7). The severity of the storm was conveyed by the storm signatures and attributes provided by the WDSS and CARDS systems. The hail detections of the systems were apparent over Campbelltown shortly after 0400 UTC, and reports of large hail arrived just after this time.

d. Hail detection

Figure 6 shows a Thunderbox product with a 60-min forecast track of a cell from 0303 UTC. The delineated threat area just encompassed Campbelltown, suggesting that severe weather would occur there in the subsequent 60 min. Three products showed the presence of severe hail in this cell: CARDS, WDSS, and C-Pol. Both C-Pol and WDSS indicated the presence of hail greater than 2 cm in diameter, while CARDS estimated the maximum hail diameter at over 5 cm. Other evidence of consistency with reflectivity patterns, such as reflectivity cross sections available from CARDS, was presented and this gave further confidence to the forecasters that these hail detections were accurate. Another warning, shown in Fig. 7, with the time stamp 0353 UTC, confirmed this initial warning prior to the onset of severe weather (in the form of large hail and heavy rainfall). Had this product reached the clients at this time it would have given a clear warning that severe weather would impact on Campbelltown in the following 10 min, with the center of the storm passing over approximately 30 min later. The forecasters' log shows this warning was actually issued at 0400 UTC.

At 0410 UTC C-Pol detected a large area of the convective cell that contained hail in excess of 2-cm diameter over Campbelltown (Fig. 8) and continued this detection for the next 30 min. This was corroborated by the WDSS system, which analyzed hail greater than 2 cm. As confirmation of the three systems' independent hail detections, observations were reported and included 2–3-cm diameter hail in Campbelltown from 0415 to 0430 UTC, followed by hail reaching sizes between 4 and 5 cm northwest of the town at 0450 UTC. Such prompt confirmation of the FDP systems' ability to accurately detect severe weather phenomena gave the forecasters great confidence in all the systems' outputs and allowed rapid assimilation of the information later in the event when time became very limited for decision making.

e. Damaging wind warnings

Figure 9 shows the 0423 UTC TITAN tracks (generated while the storm was still centered over Campbelltown) were used as the basis for a Thunderbox product that was not actually issued until around 0455 UTC, although an official text warning was issued earlier. The delay was due to a detailed discussion among the forecast staff regarding the severity of the warning to be issued, along with a large number of other matters that required urgent attention. These tracks remained the basis for the warning, however, as they were thought to provide a clear indication of the storm's progress and likely area of influence. At that stage the storm was propagating quite slowly such that the delineation of the storm area and the forecast track were still valid some time later. Figure 9b shows the radar reflectivity image from which the forecast tracks were generated at this time.

The BoM has an operational arrangement for the specific forecasting and warning of tornadoes, which is to be used only when there is real confidence that a tornado is likely and, in effect, is used when there is knowledge of the existence of a tornado. The BoM forecasters have a capability to upgrade the wording within warnings to mention destructive, instead of damaging, winds when applicable. A warning of very destructive winds was issued at 0523 UTC along with an additional statement that “this is a particularly dangerous storm.” These warnings are shown in Figs. 10 and 11.

This upgraded warning was partly a result of information coming from the ANC system. FDP experts detected signs of a mesocyclone and possibly a tornado vortex signature (TVS) in ANC fields at 0515 UTC, but the detection was not automatic and confirmed those detections of mesocyclonic vortices found by WDSS and CARDS. The ANC information was not displayed on the FDP products page, but the information was deemed so important that it was conveyed in person to the severe weather forecasting team. The wind warning was, however, not issued specifically on the basis of forecaster belief in the probable development of tornadoes, but due to the high probability that damaging winds would be generated by the storm, either in the form of tornadoes or downbursts. At that time the storm was visible from the BoM office, which is situated on the 16th floor of a building in central Sydney. Forecasters and FDP participants were able to see the wall cloud and dust being blown by a rear-flank downdraft.

At 0525 UTC (at the time the upgraded warning was being issued) both WDSS and CARDS detected signs of mesocyclone development and severe hail. In fact, CARDS reported the largest hail of the day—10 cm in diameter. At this point in the event CARDS and WDSS became the focus of attention. WDSS in particular, with its clear graphics and complementary cell attribute table, allowed a rapid assessment of the severe weather associated with individual storm cells. CARDS had the added advantage of clear thunderstorm tracks and superimposed severe weather detections, but these were in its rear panel.

Spotter reports of rotation and tornadic vortices were logged starting at 0530 UTC, but were related to observations made earlier (Sills et al. 2004). There was a further spotter report of cloud rotation at 0541 UTC. It is difficult to accurately locate these events from the reports, but they do confirm the presence of vortices at the time the systems detected them and after warnings had been issued. Some time before 0600 UTC a corrugated iron roof was peeled off, leading to a flooded building in Greystanes, by what a damage survey team later determined to be a tornado. This tornado was the strongest of the day, peaking at F1 intensity and damaging a number of properties and felling many trees.

At approximately 0515 UTC reports of golf-ball-sized hail at Greystanes were received and it was near this area that the roof of a school building collapsed under the weight of accumulated hail. Nearby, another school-block roof was lost to strong winds at approximately the same time. The damage survey suggested that, although this was close to the tornado tracks, it was not actually within them and the pattern of the damage suggested that it was most likely caused by straight-line winds. Similar damage was caused by both nontornadic and tornadic winds and, although tornadoes were present, it is debatable whether the inclusion of a specific tornado warning would have been of additional benefit to the “very destructive wind” warning.

At 0601 UTC the severe thunderstorm warning was reissued, continuing the very destructive winds advisory for a further hour (Fig. 11). This warning and the previous 0523 UTC warning actually postdated most of the damaging severe winds. However, even a timely hindcast was of great use to the SES, which are required to attend to and aid areas that have suffered damage as swiftly as possible.