Abstract

A forecaster-interactive capability was added to an automated convective storm nowcasting system [Auto-Nowcaster (ANC)] to allow forecasters to enhance the performance of 1-h nowcasts of convective storm initiation and evolution produced every 6 min. This Forecaster-Over-The-Loop (FOTL-ANC) system was tested at the National Weather Service Fort Worth–Dallas, Texas, Weather Forecast Office during daily operations from 2005 to 2010. The forecaster’s role was to enter the locations of surface convergence boundaries into the ANC prior to dissemination of nowcasts to the Center Weather Service Unit. Verification of the FOTL-ANC versus ANC (no human) nowcasts was conducted on the convective scale. Categorical verification scores were computed for 30 subdomains within the forecast domain. Special focus was placed on subdomains that included convergence boundaries for evaluation of forecaster involvement and impact on the FOTL-ANC nowcasts. The probability of detection of convective storms increased by 20%–60% with little to no change observed in the false-alarm ratios. Bias values increased from 0.8–1.0 to 1.0–3.0 with human involvement. The accuracy of storm nowcasts notably improved with forecaster involvement; critical success index (CSI) values increased from 0.15–0.25 (ANC) to 0.2–0.4 (FOTL-ANC). Over short time periods, CSI values as large as 0.6 were also observed. This study demonstrated definitively that forecaster involvement led to positive improvement in the nowcasts in most cases while causing no degradation in other cases; a few exceptions are noted. Results show that forecasters can play an important role in the production of rapidly updated, convective storm nowcasts for end users.

1. Motivation

There is a critical need for rapidly updating, accurate, high-resolution, short-term 0–6-h forecasts (nowcasts1) of convective precipitation that can be used in alerting the public of high-impact weather. Yet two of the greatest challenges with short-term quantitative precipitation forecasts (QPFs) are accurately predicting the specific location and the timing of convective storm initiation, development, and evolution. A convective storm is defined here as a storm having radar reflectivities ≥ 35 dBZ.2 The direct goal and benefit in accurate prediction of convective storm initiation is the extra lead time gained in forecasting and warning of high-impact weather (storms of ≥50 dBZ). A third challenge is disseminating convective storm nowcast products in a timely fashion for these products to be useful to an end user, such as those involved in strategic and tactical planning for en route and terminal air traffic around weather-impacted airspace. In this paper we describe the real-time involvement of National Weather Service (NWS) forecasters with an automated convective storm nowcast system (Auto-Nowcaster, ANC) developed by the National Center for Atmospheric Research (NCAR) (Mueller et al. 2003; Roberts and Rutledge 2003; Saxen et al. 2008) to improve and disseminate 1-h nowcast products of storm initiation, growth, and decay. Two new approaches are taken for producing and summarizing the categorical skill scores of the ANC convective storm nowcasts with and without forecasters involved in the process.

Forecasters currently utilize real-time observations and numerical weather prediction (NWP) models to provide convective weather outlooks each day. However, current NWP 0–6-h model forecasts of precipitation lack the accuracy needed by end users in the prediction of site-specific locations of convective-scale precipitation, particularly for events without strong synoptic forcing during the warm season months (Olson et al. 1995; Wilson and Roberts 2006). Median timing errors of 2.5 h have been observed with high-resolution mesoscale model forecasts (Fowle and Roebber 2003) and spatial offsets of as much as 250 km were observed 44% of the time with 3-h Rapid Update Cycle (RUC) operational forecasts (Wilson and Roberts 2006)—quite problematic for providing accurate, high-resolution nowcasts of weather that might impact approach and departure gates at major airports. These systematic NWP forecast errors are due in part to the lack of dense operational observations necessary to provide a complete representation of the atmospheric environment when models are initialized (Benjamin et al. 2004; Stensrud et al. 2009) and to the use of convective parameterization schemes within NWP models. Increasingly, NWP models that are run in the research community at higher horizontal grid resolutions (<12 km) are showing improvement in representing mesoscale storm precipitation structure and intensities (Done et al. 2004), but as the resolution increases, objective verification scores are degraded due to spatial and timing errors in the forecasts (Mass et al. 2002). It is anticipated that the assimilation of radar, satellite, profiler, and other operational datasets into NWP models (Xue et al. 2003; Sun 2005; Weygandt et al. 2009; Benjamin et al. 2009), explicit specification of convection (Done et al. 2004; Weisman et al. 2008), and use of ensemble runs of NWP models to produce probabilistic forecasts will help improve the accuracy of QPF (Stensrud et al. 1999, Roebber et al. 2004) and can be used to increase air traffic capacity in the future (Steiner et al. 2010).

In the absence of spatially accurate QPFs from NWP models, production of accurate convective storm nowcasts lies within the realm and responsibilities of the operational forecaster (Roberts et al. 2005; Sills 2009) and with heuristically based nowcast systems (Keenan et al. 2003; Mueller et al. 2003; Pinto et al. 2010). In the last decade, great attention has been given by forecast offices around the world (e.g., in Australia, Canada, eastern Asia, and Europe) to increasing nowcast capabilities and tools (Keenan et al. 2003; Bally 2004; Sills 2009; White et al. 2009; Wong et al. 2009; Haiden et al. 2011). In particular, the role of the forecaster in the nowcasting of high-impact weather (Roberts et al. 2009; Sills 2009; Wilson et al. 2010) has become a topic of intense interest, especially as the forecast process becomes more automated (Mass 2003; Stuart et al. 2006; Stuart et al. 2007a,b) and automated products are desired for the future generation of the U.S. aviation airspace.

A unique opportunity arose in the United States to test and demonstrate the role forecasters can play in the production of rapidly updated, short-term nowcast products in an operational setting. The Aviation Weather Branch of the NWS supported the demonstration of a Forecaster-Over-The-Loop (FOTL) forecaster-computer system (FOTL-ANC) at the Fort Worth–Dallas (FWD) Weather Forecast Office (WFO) (see Fig. 1a) for the past 6 yr, with the real-time system up and running in March 2005. Two primary objectives of the Forecaster-Over-The-Loop demonstration are to 1) assess NWS forecaster involvement in providing value-added enhancements to gridded, automated, nowcast products produced by the NCAR ANC system and 2) assess the usefulness of these products as guidance for the forecaster/meteorologist in producing terminal aerodrome forecasts (TAFs), short-term forecasts (nowcasts), area forecast discussions (AFDs), and mesoscale area weather updates (AWUs). The overarching goal of the FOTL-ANC demonstration was to improve the consistency, reliability, and accuracy of 1-h convective storm nowcasts for inclusion in automated aviation weather digital products planned for the National Airspace System’s Next Generation (NextGen) Air Transportation System (JPDO 2007) and specifically to improve convection initiation nowcasts.

There are challenges in evaluating these types of nowcasts. The ANC and FOTL-ANC systems produce both convective storm initiation and convective storm growth and decay nowcast products. However, there are difficulties in observing convection initiation objectively and identifying it as a separate stage of the whole storm evolution process. There is large variability in convective weather in space and time. Convective storms typically form over very small regions (5–10 km in scale) and it can be difficult to identify the impacts of forecast improvement across the whole nowcast domain. Evaluations of enhancements to convective nowcasting systems are hampered by large-scale storm characteristics as well as small-scale errors in time and location. A further challenge is how to separate out and evaluate the specific contributions by an operational forecaster toward the accuracy of the convective storm nowcasts and determine if the forecaster adds value to the products. A final challenge is how to conduct the verification using datasets that includes a mix of newly initiated and existing storms typical on active weather days. This paper addresses the additional approaches that were taken to address these challenges and assess the FOTL-ANC system performance.

A brief description of the Auto-Nowcaster system and the tools and fields available to the forecaster on the NWS Advanced Weather Interactive Processing System (AWIPS) display 2-dimensional (D2D) is presented in section 2. Section 3 documents the forecaster interaction with the ANC system during daily operations. Statistical approaches used to evaluate the FOTL-ANC convective storm nowcasts are documented in section 4 with results presented in section 5. A summary discussion follows in section 6.

2. Convective storm nowcasting system

The NCAR Auto-Nowcaster was originally developed for the Federal Aviation Administration to provide completely automated, rapidly updated, very short-term, deterministic thunderstorm nowcast products for aviation traffic managers. It was first run during real-time operations as a stand-alone system at the NWS Washington D.C.–Baltimore WFO as part of the System for Convection Analysis and Nowcasting (SCAN) project (Smith et al. 1998) from 1997 to 2000. A unique attribute of the ANC system is its ability to combine radar, satellite, surface station, and rawinsonde data with NWP output fields to provide gridded, location-specific, 1-h nowcasts of convective storm initiation, storm extrapolations, and storm growth and decay nowcasts every 6 min on the regional scale.

A fuzzy logic methodology (McNeil and Freiberger 1993) is used by the ANC to combine predictor fields to produce both a convective storm initiation nowcast field and a final convective storm (initiation + growth and decay) nowcast field. A different set of fuzzy logic membership functions and weights are applied to the initiation and the growth and decay components of the nowcast fields. The set of predictors represent attributes of the large-scale and boundary layer environments (stability and large-scale forcing), provide characterization of convergence boundaries (e.g., boundary speed, maximum vertical velocities, boundary-relative shear profiles, and atmospheric instability above the boundaries), designate cloud type and rate of cloud development, and characterize storm attributes (e.g., growth rate, areal coverage, maximum intensity). The list of the ANC predictor fields used to produce the convective storm initiation nowcast field is provided in Table 1. Additional predictors for the storm growth and decay portions of the final convective storm nowcast are given in Table 2 of Mueller et al. (2003). The purpose of the convection initiation field is to alert forecasters quickly to areas where new convection is likely to occur in 60 min. This field does not take into account the presence of existing convection; that information is provided in the final convective storm nowcast field.

The fuzzy logic approach uses membership functions to convert predictor fields into interest fields that range in interest values from 1 (high positive interest) to −1 (high negative interest). The creation of these “fuzzy” interest fields ensures that all information at every grid point is retained in the nowcast process. For example, a membership function applied to average convective inhibition (CIN; Table 1) values translates regions of average CIN equal to 0 J kg−1 (no convective inhibition present) into interest values of 1.0 (likelihood of convective storms is high) at those gridpoint locations. Regions with larger values of CIN (more convective inhibition) would be assigned lower interest values at those grid points and set to −1 interest for values of CIN >500 J kg−1. Each predictor interest field is then multiplied by a weighting factor whose magnitude is based on the importance of that particular predictor in the initiation, growth, and decay of storms, as determined from analyses of hundreds of cases. The weighted interest fields are summed together (at each grid point in the domain) to produce a gridded storm initiation likelihood nowcast field (Fig. 1c) and a final gridded, 60-min convective storm nowcast field that includes the storm initiation, growth, and decay components of the nowcast (Fig. 1d). The process for applying fuzzy logic in the ANC is described in great detail in section 2 of Mueller et al. (2003). Several of the membership functions for the ANC are shown in their Fig. 7 and associated weights are listed in their Table 2.

3. Forecaster involvement

A critical step in accurately predicting convective storms is the detection of low-level convergence boundaries (e.g., synoptic fronts, gust fronts, drylines, sea-breeze fronts) that are known to trigger new convection and enhance existing convection when all other ingredients for storm development are present and favorable. These convergence boundaries frequently can be seen in satellite, radar, and surface mesonet data (Purdom 1976; Purdom 1982, Wilson and Schreiber 1986). Stationary, moving, and colliding boundaries in conditionally unstable environments focus new storm development in very specific locations (Boyd 1965; Purdom and Marcus 1982; Wilson and Mueller 1993). The direction and speed of boundaries relative to the motion of the initiated deep convection is also a factor in the continued growth or decay of those storms (Weisman and Klemp 1982; Wilson and Megenhardt 1997).

The ANC system initially included an automatic boundary detection algorithm to detect and extrapolate the positions of gust fronts, sea-breeze fronts, and other local-scale convergence features within 80–120 km of the radar. This algorithm, called the Convergence Line Detection (COLIDE) algorithm, performed with reasonably good results when run over a single radar domain. When the ANC was modified to run over a larger, regional domain and use mosaics of reflectivity data from 7–10 WSR-88Ds spaced 200–300 km apart, the boundary detection algorithm often failed to detect the full extent of mesoscale convergence boundaries at the longer ranges from the radars. The radar beam heights were often too high to observe the near-surface clear-air echoes (thin lines of 5–25-dBZ reflectivity) associated with convergence boundaries in those regions. Fronts, drylines, and mesoscale outflow boundaries in particular were not always well resolved as complete continuous features by the COLIDE algorithm, as can be seen in Fig. 2, which provides a comparison of COLIDE and forecaster-entered boundaries. As a result, given the limitations of the automated algorithm over data-sparse regions, the spatial accuracies of ANC storm initiation nowcasts were diminished. Over the past 20 years, significant effort has been spent developing robust, operational boundary detection algorithms like COLIDE and the Machine Intelligent Gust Front Algorithm (MIGFA; Delanoy and Troxel 1993) that employ various pattern matching, templates and wavelet schemes, but all fail to provide complete detections and extrapolations of boundaries in sparse or data-void regions. While it might be possible that some improvement could be made to existing, automated boundary detection and tracking algorithms, it has proved to be a difficult problem to obtain the spatial precision needed for the scale of prediction of the ANC, without introducing false positives (identification of features that are false, i.e., not actual boundaries). In the foreseeable future, until more sophisticated approaches are available for handling data-void areas or higher density of observations become available, the forecaster plays an important role in alleviating these deficiencies in automated algorithm performance by using conceptual models for thunderstorm outflow propagation and frontal boundary movement, multiple diverse datasets, and assumptions of spatial and temporal continuity to locate the spatial extension and movement of convergence boundary locations.

Real-time tests conducted following the SCAN project showed that the inclusion of human-entered boundaries into the ANC process added consistency, reliability, and accuracy to the 60-min time and location-specific convective storm nowcasts (Roberts et al. 2005; Nelson et al. 2006). Subsequently, forecaster-interactive capabilities have been built into the ANC system to utilize the strengths of both the computer and the human. The computer provides rapid processing of large operational datasets and model output to produce 60-min nowcasts every 6 min. The forecaster skills needed include training on the ANC system, the local knowledge that each WFO possesses about the nuances of their own weather, good subjective mesoanalysis skills for rapid identification of the prevailing weather regime of the day, and identification of convergence features, recognition of limitations in NWP output, and assessment of the quality of real-time observations. The ANC forecaster–computer mix was first tested during the Sydney, Australia, 2000 Olympics Forecast Demonstration Program (Keenan et al. 2003; Wilson et al. 2004). The FOTL demonstration is the first test of this FOTL-ANC forecaster–computer system in an operational NWS setting.

a. Forecaster role and tools

The ANC software system has been transferred to the NWS Meteorological Development Laboratory (MDL) and installed on a prototype AWIPS system running at MDL and on the FWD WFO workstations. Specific ANC fields can be selected for display under the SCAN menu option. The ANC field most frequently viewed by the forecasters is the gridded, storm initiation likelihood field shown in Fig. 1c; a colored-coded interest field indicating where new storm initiation is nowcast to occur within 60 min. The forecasters also have the option to view the ANC final 60-min nowcast product (Fig. 1d) that includes both the initiation component of the nowcast plus the forecast location and intensity of existing storms. This latter component of the product field is not simple storm extrapolation, but also incorporates information about storm growth and decay during the 60-min forecast period (see Mueller et al. 2003 for more detailed discussion). The lowest likelihood for declaring storm initiation (i.e., occurrence of ≥ 35 dBZ) is set at an interest value of 0.7 (pink shade in Fig. 1c; init1 with white shading in Fig. 1d). Darker shades of pink (0.9 interest) and red (1.2 interest) in Fig. 1c represent higher levels of likelihood that initiation will occur, and are represented by gray-shaded regions (init2 and init3) in Fig. 1d.

The forecaster has four options for providing input into the ANC system via the AWIPS display: 1) entering the location of surface convergence boundaries, 2) selecting the dominant weather regime for the day, 3) increasing or decreasing (“nudging”) the storm initiation interest values over the whole domain, and 4) increasing or decreasing the storm initiation values only in a forecaster-selected region of the domain using a “polygon” tool. A boundary can be entered on AWIPS either as a stationary feature or a feature set in motion. The boundary location is automatically updated by the system every 6 min, unless the forecaster modifies it. The forecaster can speed up or slow down the boundary motion, adjust the boundary orientation, enter additional boundaries, and delete boundaries. The boundary locations can be superimposed onto any of the AWIPS fields.

The ANC allows the forecaster to select the dominant weather regime expected to impact the WFO County Warning Area (CWA) during the day or evening. In collaboration with the FWD forecasters, predictors for seven weather regimes have been established representing the modes of convective triggering that commonly occur in Texas: cold fronts, drylines, pulse storms (air masses), stationary/warm fronts, mesoscale convective systems (MCSs), mixed, and no storms regimes. Each regime has a specific set of membership functions and weights that have been tuned for the area using cases identified by the FWD staff. The forecaster can change the regime selection in the ANC whenever desired. During a transition in weather regimes, the forecaster typically would change the regime from the current setting to another specific regime. Mixed regime was generally selected when there is no dominant synoptic setup. During 2008–09, the mixed regime was always run concurrently with the forecaster-selected regime. Choosing an inappropriate regime for the given synoptic situation could lead to incorrect storm initiation nowcasts as the regime predictor fields may have no relevance to the current synoptic setup. For example, choosing the MCS regime, which places lower weights on boundary predictor fields as elevated convection and storm advection dominate under this weather pattern, will result in missed detections if the weather is actually being triggered by surface boundaries (e.g., a cold front). Although it would be ideal to automate the process of regime identification, currently no such algorithms exist because of the complexity in weather patterns and processes. Nelson et al. (2008) provide additional details on these regimes. The storm initiation likelihood fields for dryline and pulse storm regimes are shown in Figs. 3a,b for two different days illustrating the marked difference due to the different surface-based lifting mechanisms and environmental conditions.

To monitor the atmospheric environment and potential instability, the ANC uses the operational Rapid Update Cycle (RUC) model analysis and 1-h forecast fields as input in producing the nowcasts (see Table 1). Situations often arise in which the RUC CAPE and CIN stability fields will either substantially underestimate or overestimate the amount of stability in the atmosphere; that is, the model is running “cold” or “hot,” respectively. This can significantly impact the magnitude of interest values throughout the domain in the storm initiation likelihood field. On “cold” days, it may be impossible for the ANC system to produce a nowcast for new convection, even if a forecaster has entered a convergence boundary into the system. The forecaster is provided with a tool to nudge the interest values to a more appropriate level, based on the forecaster awareness of the conditions on that day. Figures 3c,d illustrate the impact of applying a nudge to the field. Ultimately, the intent in using these tools is to improve the products distributed to the end user.

b. Operational methodology: Daily routine

All forecasters were trained on the system and the AWIPS-ANC tools by the ANC focal point forecaster at the WFO. The short-term forecaster on duty was tasked with the primary responsibility for interaction with the ANC system during shift operations. Table 2 lists the operational scenario, as documented by the FWD Scientific Operations Officer (SOO), for daily interaction with the AWIPS-ANC system. Immediately following forecaster interaction with the system, the ANC ingests any new boundary locations, automatically extrapolating boundaries to the 60-min forecast time and producing nowcast products every 6 min. The system is optimized to streamline and minimize the amount of time a forecaster needs to interact with the system as it produces frequently updated, gridded nowcast products for use by aviation and other end users (see items 6 and 7 in Table 2). Meteorologists at the Fort Worth CWSU can view the ANC product fields in real time via a dedicated web page that displays the fields shown in Fig. 1 and is updated every 6 min with new images.

Figure 4 illustrates the substantial increase in NWS forecaster interaction with the ANC system following the installation of the ANC tools and fields on AWIPS in 2008. Prior to 2008, forecasters were not comfortable with using the different, unfamiliar ANC display system and, in an independent survey of the FWD forecasters conducted by Roberts and Cheatwood-Harris (2010), forecasters stated that “the AWIPS interface was easier to use and the tools for drawing and altering boundary shape were easiest to use.” Figure 5 shows the frequency of regime selection for 2007–09. Cold fronts were by far the most frequent trigger for convective weather in the Dallas–Fort Worth area annually. The mixed regime was often selected, as forecasters were advised during training to select this option when more than one regime is expected to affect the area. Vigorous to severe convection initiated frequently along drylines, but primarily during the spring season. Both surface-based and elevated convective initiation (convection arising from non-surface-based triggering) often occurred during MCS and warm front events. Roughly 50% of all of the convective initiation that occurs over the southern Great Plains is elevated convection (Wilson and Roberts 2006); the ANC system does not currently handle this challenging problem of providing location-specific nowcasts of new elevated convection and is only able to provide extrapolation-based nowcasts under these weather scenarios.

Table 3 provides detail on forecaster interaction with the ANC system for 44 different days of active weather during the period from 2006 to 2009. Because the FOTL-ANC has been running year round, this subset of cases was selected to illustrate a spectrum of weather events, regimes, frequency of boundaries entered, and the general impact of forecaster interaction with the system.

Data from 4 July 2006 are presented in Fig. 6 to illustrate a typical day of forecaster interaction with the ANC system. On 4 July, the forecaster had three interactions with the system (Table 3) in addition to monitoring the system periodically during his or her shift and setting the system in the cold front regime. At 1921 UTC3 (Fig. 6a), several storms were already present within the forecast domain. The ANC storm initiation nowcast field (Fig. 6b), without any forecaster interaction up to that point, was indicating the likelihood for initiation to occur in the southern-central portion of the domain (pink areas; init1 level) ahead of the existing storms. No boundary associated with the cold front had been entered yet by the forecaster. One hour later at 2009 UTC the forecaster entered a stationary convergence boundary. The impact of this boundary on the system can be seen in Fig. 6c at 2021 UTC where nowcasts for new storm initiation extend farther to the north in advance of the leading edge of the existing convection and with significantly higher likelihood (darker pink shades) for new storms in the vicinity of the convergence boundary. At 2111 UTC the forecaster used the AWIPS tools to enter an updated boundary location and set it in motion. By 2121 UTC (Fig. 6d), the FOTL-ANC system produced nowcasts of even higher likelihood for storms in association with the moving boundary.

The contoured location of the 35-dBZ storms at a nowcast valid time 60 min later are also shown in Figs. 6b–d. Visually, the convective storm initiation nowcast field seems to be doing a good job of pinpointing specific locations where convection will likely be occurring. Only a few missed forecasts exist in those areas of the storm initiation field where interest values were just below the nowcast threshold (light green regions of 0.6 interest values); at subsequent times, these areas gained heightened interest (cf. Figs. 6b,c) when the boundary was added into the forecast. Although the boundary was entered after some storms had already formed, the forecaster interaction appears to have had a positive impact on focusing the interest to regions with active convection.

4. Evaluation of the FOTL-ANC system

There has always been a disparity between visual assessments and the statistical evaluation of forecasts. Forecasts that may look quite good visually as in 4 July 2006 may not always result in high values of categorical verification scores that are typical metrics of performance for deterministic forecasts. This disparity results in part from the high sensitivity of these scores to small errors in location or timing and in part from the aggregations of areas that are convectively active with areas that are relatively inactive. After first trying conventional approaches to evaluate the nowcasts, we embarked on two new approaches for computing the categorical verification scores that are described below.

a. Conventional evaluation and verification data

The FOTL-ANC nowcast domain, with a grid resolution of 2 km, is centered over the FWD CWA and extends ~200 km beyond the CWA in all directions, making it possible to track weather propagating into the verification domain (see Fig. 1a). Conventional verification using contingency table verification statistics (Fig. 7) such as probability of detection (POD), false-alarm ratio (FAR), critical success index (CSI), success ratio, and bias were computed using the equations presented in Wilks (2006). Verification was conducted using a gridpoint to gridpoint comparison of FOTL-ANC deterministic nowcasts with the predictand (i.e., the occurrence of radar reflectivity echoes ≥35 dBZ). No relaxation or smoothing was applied to either the forecast or observed field. Thus, nowcasts of storm initiation and storm extrapolation must line up perfectly with the predictand 60 min later for a successful forecast. This is much more challenging for storm initiation nowcasts, considering that the storm initiation nowcast is specifying a precise timing and location for a storm that does not currently exist, in contrast to extrapolation of existing storms, which are more likely to line up with at least some portion of the actual storm location 60 min later.

Ideally it is desirable to evaluate both the convective storm initiation nowcast and the final convective storm nowcast fields. However, historically it has been very difficult to obtain a comprehensive verification dataset (“truth”) that accurately distinguishes between initiated and existing convective storms due to the complex weather interactions and weather patterns that occur on the convective scale such as 1) new convective development on the leading edge of existing convection, as observed with multicellular storms and squall lines; 2) secondary convective storm initiation arising from the collision of outflows from two nearby storms leading to one large area of >35-dBZ echo with embedded cores; 3) new convective storms embedded within existing stratiform precipitation; 4) storms that form along the flanking line of existing lines of convection; and 5) enhancement of existing weak storms into vigorous convective storms due to extra lift provided by passage of a gust front. To produce a truth dataset that includes only newly initiated storms would involve the use of a human to tag every storm as new or existing and this would still fail to handle new embedded convection. Thus, we are limited to using the radar reflectivity data 60 min after nowcast time, which includes a mix of both initiated and existing storms, to evaluate the performance of the final convective storm nowcast field.

The following approach was taken to compare the different nowcast fields. Figure 8 compares simple persistence and storm extrapolations4 nowcasts to the FOTL-ANC storm initiation and growth–decay (GD) nowcasts using different initiation thresholds (init1, init2, init3). The time series graphs as shown in Fig. 8 were produced every day during the FOTL demonstration. In this stepwise approach, the performance of the FOTL-ANC GD nowcasts (which includes forecaster-entered boundaries but not the initiation component of the nowcast) was compared to the performance of the extrapolation nowcasts and persistence. For the majority of the FOTL days, the GD nowcasts usually showed very similar performance to the extrapolation curve with small variations observed due to the effects of the convergence boundary(s) on increasing or decreasing the spatial dimensions of the extrapolated storms compared to the Thunderstorm Identification Tracking Analysis and Nowcasting (TITAN) extrapolated storms. The differences observed by comparing the performance of the FOTL-ANC final convective storm nowcasts (e.g., init1 in Fig. 8), which include both the initiation and GD components of the nowcast, with the GD and extrapolation (both represented by gray curve in Fig. 8) nowcasts are mostly a direct result of the impact of forecaster-entered boundaries on the storm initiation component of the nowcast. In section 4b, these comparisons are taken one step further by removing boundaries from the initiation component of the FOTL-ANC nowcast to examine the human versus no-human nowcasts.

In Fig. 8 for 4 July 2006, higher values of POD and bias are apparent for the FOTL-ANC system compared to values for extrapolation and persistence between 2000 and 0200 UTC valid times when new storm initiation is occurring (see vertical black lines in Fig. 8), with very little difference observed in the FAR values. Yet it is difficult to see much improvement in the CSI values. Because the GD and extrapolation verification curves are so similar, higher values of POD and bias are likely attributable to increases in performance associated with the convective initiation portion of the nowcast. However, convective initiation nowcasts represent a significantly small fraction of events [i.e., they are “rare” events in comparison to the large number of storm extrapolation forecasts; Roberts et al. (2007)]. Assessment of skill is problematic because of the rarity of such events (Stephenson et al. 2008). As the day progresses, it is difficult to discern any appreciable difference in CSI values between the different techniques because storm extrapolation nowcasts dominate the statistics over the ANC domain where many storms exist simultaneously.

b. Verification over spatial subdomain regions

Conventional evaluation techniques are quite limited in their inability to measure and represent the spatial correlations between the nowcasts and precipitation fields; moreover, it is very difficult to interpret these traditional verification results in a physically meaningful way. Recently, new verification approaches have been developed that account for the spatial attributes and the scale of the phenomena being forecast and evaluated. These approaches are summarized in Casati et al. (2008) and include object-based approaches (Ebert and McBride 2000; Davis et al. 2006a,b), scale decomposition and intensity-scale verification (Casati et al. 2004), and neighborhood-based (fuzzy) verification (Roberts and Lean 2008; Ebert 2008; Marsigli et al. 2008). The approach taken here is to evaluate performance on the scale being nowcast, the convective scale, by dividing the ANC verification region into 1° latitude × 1° longitude subdomain boxes (30 boxes) as shown in Fig. 6. One can view this as a single layer of filtering of the data, as opposed to the multiscale decomposition techniques using wavelet analysis (Casati et al. 2004) or using the fractions skill score (FSS) approach to selectively determine the best spatial scale for evaluation of the forecasts (Roberts 2008). For this study, a gridpoint-to-gridpoint comparison was still conducted within the subdomain boxes but it would be of interest to test a fuzzy verification technique in the future that would relax the requirement for exact matches between the nowcasts and the observations (Ebert 2008).

For a more meaningful evaluation of FOTL performance, we have chosen to focus our evaluation on regions where forecaster involvement would have had the largest impact on the ANC nowcasts (i.e., to those subdomain boxes in which a convergence boundary was resident over a period of time). Time series plots of CSI values for 4 July 2006 are shown in Fig. 9 for a selection of subdomains. The curves shown are for human versus no- human-based final convective storm nowcasts of the init2 level. Experience with the ANC system running at Dallas for 6 years has shown that the init2 level is the initiation threshold level that typically produces more accurate, site-specific nowcasts (Figs. 8a,c). The init1 level has generally been viewed as the very lowest likelihood for any new convection to occur somewhere in that region and thus the forecast areas (as seen in Fig. 6c) tend to extend over larger regions to capture the scattered convection, often resulting in nowcasts with larger false alarms and bias values (see Figs. 8b,d). The black dashed lines in Fig. 9 bound the time period when the boundary is located within and affecting one-fifth of the area within that subdomain box. Specifically, this indicates the area where over one-fifth of the grid points in the subdomain reside within boundary-produced interest fields that envelop a boundary [i.e., within the gridded “lifting zone” defined by the ANC system; see Mueller et al. (2003)].

Within these smaller domains, the change in performance of the final convective storm nowcasts is much more evident, with relatively large increases of 0.1–0.2 observed in the CSI scores after the forecaster entered the convergence boundary into the ANC system. Increases of ~0.2 in the CSI scores (Figs. 9b,d) were not evident in the full domain statistics (Figs. 8c) for the same time periods. Experience has shown that increases of this magnitude in the performance of automated convective storm nowcasting systems have been difficult to demonstrate with traditional approaches even when visual inspection of the forecasts suggests otherwise. Following the period of storm initiation, smaller increases in the CSI values are observed in the FOTL-ANC (human) scores compared to the ANC (no human)5 and storm extrapolation scores. These modest improvements are a direct result of convergence boundary information being included in the GD component of the final convective storm nowcast field (Mueller et al. 2003). The corresponding POD and FAR time series plots (not shown) showed that the POD scores increased and FAR scores changed very little between the FOTL-ANC and ANC nowcasts, resulting in the larger CSI scores for FOTL-ANC nowcasts.

It is informative to sum up the statistics from all of the individual subdomain boxes over the duration of the weather event (10 h on 4 July 2006; see Table 3) to evaluate the overall performance in storm initiation, growth, and decay nowcasting when forecaster-entered boundaries were included in the ANC process. Summary histograms are shown in Fig. 10. Positive (negative) changes represent increases (decreases) in the accuracy of ANC nowcasts when forecaster input is included. Most of the values fall within the zero (no change) category, but when there were changes, they mainly landed on the improvement side (POD, CSI) or were evenly split between positive and negative changes (FAR). As we will see later for another day, a larger negative impact can result if the forecaster does not periodically update the location of a stationary or moving boundary.

Initiation likelihood regions along boundaries tend to be larger than the size of individual storms; thus, bias values >1 are observed and the change in bias values end up being mostly positive, which is not ideal statistically because it represents an overforecasting of the area. However, higher PODs and higher bias values may be acceptable to an aviation end user if their greater need is to be forewarned that they will encounter convective storms at some point along their flight path. The large number of time periods clustered at zero (no change) in Fig. 10 is dominated by the correct “no” category; that is, the correct nowcasts for no storms.

c. Performance diagrams

An instrumental new graphic for visually representing several measures of forecast quality (POD, success ratio, CSI, bias) in the same plot (Roebber 2009; C. Wilson 2008, personal communication) has been employed in this study to summarize the FOTL-ANC performance and spatial variability across the subdomains for each of the days in Table 3. These performance diagrams are based on mathematical relationships (Roebber 2009) among the scores presented in Fig. 7 and they provide an easy way to visually compare the performance of different forecasts using multiple verification measures. Figure 11a shows the performance measures for all of the subdomain boxes (color coded) for the duration of the cold front event on 4 July 2006. Triangular icons represent FOTL-ANC nowcasts, circular icons represent ANC nowcasts, and the dark gray lines connect icons for the same subdomain box. The size of the icon is scaled based on the following relation, (a + b + c) (a + b + c + d)−1, using contingency table values (Fig. 7). Applying this factor, larger icons represent subregions with a larger proportion of forecasted events (hits, misses, and false alarms) compared to null values (d); thus, the size of the icons provides a graphical scaling of the amount of weather occurring within each subdomain box. Figure 11a shows that for most of the subdomains there is some small improvement in the nowcasts due to forecaster input but a substantial clustering of the results over a small range of statistical measures.

However, by excluding subdomains that did not have convergence boundaries present within those regions, we are able to examine the forecaster’s specific contribution to nowcast performance within the remaining subdomains and confined to the specific time periods where their actions had the most impact. Figure 11b shows this subset of subdomains and the associated icons now represent the forecasts only for those time periods when a boundary was present in the subdomain. Thus, the change in location from Fig. 11a to Fig. 11b of the subset of icons represents the change in the statistics associated with nowcasts that are primarily boundary-influenced, storm-initiation nowcasts. Figure 11b dramatically shows large increases in POD values and an average increase in CSI values in the range of 0.1–0.2 between the FOTL-ANC and ANC nowcasts (e.g., box 43), while the success ratio (1 − FAR) remains fairly constant. The shaded area in the upper right-hand corner of Fig. 11 represents the ideal level of statistical performance (C. Wilson 2008, personal communication) and examination of the improved performance observed in Fig. 11b indicates the FOTL-ANC skill is headed in the right direction. We also see the spatial variability in performance represented by each triangle–circle pair as the forecaster-entered convergence boundary moved through the various subdomain regions. All but three of the subdomain pairs show increased performance with the FOTL-ANC nowcasts. One exception is box 53 (purple), which is characterized by a slight decrease in CSI resulting from a decrease in success ratio and an increase in bias, due to too large an area being nowcast for storm initiation within this subdomain.

The verification scores improved when the forecaster entered convergence boundaries primarily because the storm initiation component of the ANC captured regions where storm initiation was occurring and existing storm persisted. As explained in section 4a, it is not possible to objectively specify how much of the improvement results from successfully nowcasting initiation. However, visual inspection of many of the cases indicates that successfully nowcasting the initiation played a significant role in improving the nowcasts; this was particularly obvious during the diurnal onset of convective storm development.

5. Nowcast verification results

In this section, verification results for a spectrum of nine additional cases covering six different weather regimes are presented. They were selected as a cross section of cases occurring over a 4-yr period when forecasters had entered boundaries.

On 27 August 2006, the mixed regime was activated and the forecaster entered one moving boundary—a cold front (Fig. 12a), that moved throughout the domain during the afternoon. The performance diagram (Fig. 12b) is similar to that of 4 July (Fig. 11b). What is more dramatic here is the large CSI improvement observed in the instantaneous values of the subdomain time series plots (Figs. 12c–f) that are not as evident in the average values (over the duration of the event) shown in Fig. 12b. FOTL-ANC CSI values increased to 0.3–0.5 over values of ANC that ranged from near zero to 0.1 in magnitude. Large jumps in the FOTL-ANC CSI curves were observed early, indicating that the forecaster-entered boundary was the primary contributor to accurately nowcasting the start of predominantly new convection in subdomains 11, 21, and 22 (e.g., at 1900 UTC in Fig. 12c, at 2100 UTC in Fig. 12d, and at 1830 UTC in Fig. 12e) and the sole reason for accurate prediction of new convection validating at 2300 UTC in Figs. 12c,d, and at 1930 UTC in Figs. 12e,f, as the ANC system did not issue nowcasts of convective storms and no extrapolation nowcasts were made due to the lack of any prior convection (verified by visual inspection).

Figure 13 shows the performance diagrams for four days in 2007 associated with dryline, pulse, and mixed regimes. On 24 April, the forecaster entered a stationary dryline early in the day and then set it in motion once the dryline was observed moving eastward toward the CWA in the afternoon (Fig. 3a). In contrast to the other regimes, the dryline regime includes three additional predictor fields optimized to focus attention and higher interest along the dryline as can be seen in the storm initiation nowcast field in Fig. 3a. These narrow regions along both sides of the dryline are where convection often initiates. Thus, it becomes more critical that the forecaster locate and enter the position of the dryline as accurately as possible. The performance diagram in Fig. 13a illustrates only modest improvement in FOTL-ANC statistical measures over the ANC measures. This is likely because of timing and placement inaccuracies in the entered location of the dryline. The dryline often sets up in western Texas in regions where there are not a sufficient number of surface observations available to accurately pinpoint its precise location.

Under the pulse regime, convection is locally driven through surface heating. Any outflow boundaries present are usually produced by single storms with outflows sometimes merging with other storm gust fronts to become a larger, mesoscale boundary. In discussions with the WFO, it was agreed that forecasters would not enter boundary locations for the storm-scale gust fronts on pulse-storm days; they would only enter the larger-scale outflow boundaries if present. Under the pulse regime, it is generally difficult for the forecaster to improve over the ANC system unless a boundary is entered into the system. On 12 May, the convective storm initiation nowcast field (Fig. 3b) has the popcornlike appearance similar to the type of convection that actually results. On this day, the forecaster did enter two small-scale outflows (Table 3) and the performance diagram (Fig. 13b) highlights the modest improvement in the POD (similar to Fig. 13a) for the small number of subdomains impacted.

Two days under the mixed regime are also shown in Figs. 13c,d. It is evident that the performance for 8 October (Fig. 13d) was quite good. Detailed time series plots and the storm initiation likelihood for this field are presented in Fig. 14. The time series plots show even larger changes in CSI from near 0 [for ANC and for extrapolation (Extrap)] to 0.4 (FOTL-ANC) in Fig. 14c from 2100 to 2200 UTC. In contrast, the changes in POD and CSI for 1 August (Fig. 13c) are much smaller. A comparison of the weather on these two days showed that most of convective weather occurring on 8 October was associated with a solitary cold frontal boundary moving south through the ANC domain while the convection on 1 August was triggered by two boundaries within the domain. As seen in Table 3, forecaster performance in entering the boundaries on 1 August was mixed; one boundary ran N–S throughout the domain and was a good representation of the large-scale convergence in that area while the second moving boundary was set in motion with too high a speed and created some false-alarm nowcasts and several missed nowcasts. Many more missed nowcasts occurred on 1 August compared with 8 October and account for the poorer performance of the nowcast observed in Fig. 13c.

As mentioned in section 3b, MCSs occur frequently in Texas but convergence boundaries are just one of several trigger mechanisms that will produce the widespread, intense convection characteristic of an MCS regime. The storm initiation nowcast field and performance diagram for an MCS case on 30 July 2008 are shown in Fig. 15. Figure 15b shows limited involvement of the forecaster entering and modifying the boundaries. Only a few of the subdomain boxes were affected by the entered boundaries and a modest increase in performance was observed in those regions.

The performance diagram for one of the few warm front regime cases on 25 March 2009 has fairly reasonable results (Figs. 15c,d). CSI scores (0.2–0.3) were higher overall for both FOTL-ANC and ANC systems compared to the MCS case in Fig. 15b. As can be seen in Fig. 15c, widespread convection occurred not only along the frontal boundary but well north of the frontal location as southeasterly flow overran the front and produced large regions of elevated convection. With such widespread echo over a large portion of the ANC domain, it is understandable that the performance scores will be higher as many of the nowcasts will validate. A better methodology than gridpoint-to-gridpoint processing of predictor fields would be desirable to handle the broad spatial triggering of warm frontal and elevated convection.

On 10 June 2009, the forecaster selected the cold front regime and entered the cold front as a moving boundary into the ANC system. Unquestionably, this was one of the best cases of the 2009 season, as can be seen in Fig. 16b by the marked increases in the CSI, success ratio, and POD values evident, as well as the movement of the overall performance toward the upper-right corner of the plot. Average CSI values within some of the subdomains jumped from ~0.2 to almost 0.6 for the FOTL-ANC nowcasts, POD values jumped up to 0.7–0.8, success ratios increased, and smaller increases in bias values were observed compared with other days. Figure 16 presents a dramatic contrast to another cold front case on 6 October 2009 (Fig. 17), when a cold front entered the ANC domain early in the morning initiating intense convection as early as 0800 UTC. A moving boundary was not entered until 1526 UTC, after a change in shift had occurred and the short-term forecaster had time to enter the boundary. (Note, for the FOTL demonstration, boundary entry was optional for the midnight shift forecaster.) Figure 17 shows the convective storm initiation nowcast fields prior to and after the moving boundary was entered, as well as the performance diagram for this case. Figure 17a shows the ANC system highlighting areas for potential convection in the region where the front was actually located (based on surface observations) at 1418 UTC. The problem with the boundary entered an hour later was that it was set in motion at too fast a speed and produced nowcasts ~80 km to the southeast of where convection actually occurred (Fig. 17b). The performance diagram (Fig. 17c) reflects this with several subdomains showing decreases in success ratio values due to the solid region of false-alarm nowcasts. The delta-CSI histogram plots also showed a similar result with more negative delta-CSI values than positive (Table 3).

For most of the cases listed in Table 3, the forecaster involvement in entering convergence boundaries resulted in improvement of the nowcast performance or caused no degradation in performance (i.e., “no change”). But there were specific cases in which negative changes in performance occurred, as discussed above, and these cases were typically associated with the forecaster not monitoring the system closely enough to notice a problem with either the boundary location or motion. Table 3 shows for 6 October that the combined assessment of performance was classified as negative; this was one of the few completely negative FOTL-ANC cases. When intense weather is under way, as on 6 October 2009, the short-term forecaster will typically need to switch over to more pressing tasks and produce storm area updates and thunderstorm warnings, and consider the potential for severe weather. In these situations, based on discussions with WFO staff, the AWIPS-ANC prototype system was designed to allow the short-term forecaster to hand off boundary-entry responsibility to someone else in the forecast office (e.g., long-range forecaster) during severe weather events.

The performance diagram can also be used to summarize skill over multiple days and years, as well as seasonally and based on weather regime. Figure 18 shows a summary of the performance of nowcasts for all subdomains with forecaster involvement, stratified by weather regime. Under most regimes there is a spectrum of performance; nowcast performance was not noticeably better for one particular weather regime versus another. The pulse regime showed the least skill, due to the difficulty in being able to predict the small-scale, primarily thermally driven, short-lived storms. Summary diagrams such as this provide important feedback on how to improve a particular set of nowcasts or improve the system as a whole.

6. Summary and discussion

Historically, it has been challenging to show increased skill in the automated prediction of convective storms as the convection typically forms over small spatial scales, and offsets in time or location can affect the forecast skill significantly. Also, the large number of storms and the variability of weather across the domain hamper our ability to obtain the desired signals out of summary statistics. A primary motivation for conducting the FOTL demonstration at the FWD WFO for 6 yr was to assess the benefit gained by having forecasters provide input into the FOTL-ANC system and to evaluate their impact on nowcast skill. The forecasters’ primary role was to select the daily weather regimes, enter convergence boundaries, monitor the system, and adjust the boundaries as needed. Forecaster input into a mostly automated system created an additional challenge to separate and evaluate their specific contributions toward the accuracy of the nowcasts.

The verification domain was divided into subdomains to better represent the convective scale being nowcast. On the subdomain scale, distinct differences in performance between the FOTL-ANC, ANC, and storm extrapolation nowcasts became evident during the early stages of convective activity. Marked increases in POD and CSI values were observed in time series plots for FOTL-ANC nowcasts compared with POD and CSI values for ANC and for storm extrapolation nowcasts. Performance diagrams were employed to plot the statistical measures used in this study, for each subdomain, as one point on a graph. These diagrams were extremely useful in summarizing the overall nowcast performance for the duration of a convective weather event both with and without the forecaster over the loop and also in documenting the spatial variability of performance. By excluding those subdomains that did not include a forecaster-entered convergence boundary, we were able to examine the direct impact of the forecaster on the system for those time periods and locations where the forecaster-identified boundary was affecting convective storm initiation and evolution. Using these approaches provided solutions for the challenges cited above and enabled us to determine the specific performance and skill associated with nowcasting convective storms both with and without forecaster input.

Figure 19 provides an overall summary of the nowcast performance of the 44 FOTL-ANC cases listed in Table 3, stratified by year, and for all subdomains with forecaster involvement. Improvement in the CSI scores was observed, with most of the events listed in Table 3 associated with forecaster involvement with the ANC system and with daily cumulative CSI scores of 0.1–0.4 observed. However, for individual subdomains and at specific time periods, much larger CSI skill scores ranging from 0.2 to 0.6 were observed with the FOTL-ANC nowcasts compared to the ANC and storm extrapolation nowcasts. The large increases in CSI scores shown in this study were directly related to the large positive change in POD values (20%–60%) and, implicitly, the reduction in missed nowcasts. By entering a convergence boundary, the forecaster is creating higher interest and focusing on a region surrounding the boundary location where surface triggering may lead to storm initiation and enhanced storm growth. It is no surprise then that the POD values should increase with this broader region of interest, nor that bias values got larger for the FOTL-ANC nowcasts. What is surprising and significant is that the magnitude of the FAR values changed very little in a comparison of the two systems while CSI values increased. This suggests that the correct entry and positioning of convergence boundaries are paramount in accurately nowcasting convective storms and minimizing false alarms. Thus, a true improvement of the nowcast performance was achieved through forecaster involvement in the production of rapidly updated, high-resolution nowcast products.

We believe that a significant portion of the improvement in accuracy was the result of the FOTL-ANC successfully nowcasting storm initiation in the vicinity of the forecaster-entered convergence boundaries. Although we cannot confirm this objectively due to the mixture of newly initiated and existing storms in the verification dataset, forecaster observations and visual inspection of the data do suggest that prediction of the initiation is the main factor. There is a definite need to identify methods of isolating new storms from evolving convection in order to perform a definitive evaluation.

Mass (2003) has stated “the greatest failure of the weather forecasting enterprise in the U.S. is its inability to provide the public with detailed information regarding local weather features and their expected evolution during the next few hours.” The 6-yr FOTL demonstration has successfully shown that forecasters can improve the quality of automated, high-resolution, gridded storm initiation and storm evolution nowcasts and that forecasters can and should play an important role in nowcasting in a future where automated forecast processes will increasingly become the norm and be required for operational efficiency (e.g., NextGen automated, gridded, weather products). And although it may seem like computer algorithms should now be able to automatically identify convergence boundaries and determine the weather regime and corresponding set of fuzzy logic for that regime, data voids, the lack of sufficient observations, and the complexity involved with automatically classifying weather patterns ensure that forecasters will be needed to oversee the production of accurate, short-term nowcasts for an end user for some time to come.

The NWS WFOs are already involved in producing NOWs (short-term forecasts). However, for optimal utility of forecaster–computer systems, defining the familiarity and fit of the forecaster–computer mix within their concept of operations is more important than a technology improvement (Dave Sharp, SOO, Melbourne, Florida, WFO, 2009, personal communication). Installation of a computer software application or forecast guidance tool at a WFO can be achieved in a fairly straightforward manner, but if the forecaster does not see how to incorporate and utilize this technology in daily operations to improve their forecasts, there is no benefit gained from the computer–forecaster system. This would be analogous to installing dual-polarization radar capability on the WSR-88Ds, but the forecasters not having the knowledge of how to incorporate these fields in producing their forecasts. These were perhaps the biggest hurdles initially at the FWD WFO: gaining familiarity and defining responsibilities and operational involvement with the ANC system, and fitting this extra effort into their ever-expanding workload. An interesting outcome from this demonstration was the exploratory work by Ted Ryan (forecaster, FWD WFO, 2009, personal communication) on ingesting gridded radar, surface station observations and other observational fields into the Gridded Forecast Editor (GFE) of the Interactive Forecast Preparation System (IFPS) for generating exploratory short-term severe weather forecasts. This looks very promising as an approach for disseminating gridded nowcast products to the public via the National Digital Forecast Database (NDFD) and addressing the need for the public to have access to detailed short-term nowcasts (Mass 2003).

The verification results presented here are positive, illustrating that the performance of automated nowcast products can be improved with forecaster involvement. However, the nowcast products are far from perfect, even with forecaster involvement, as forecasters were quick to comment on in the 2009 FWD WFO survey on the FOTL-ANC demonstration (Roberts and Cheatwood-Harris 2010). This can be seen in Fig. 19, which shows that the FOTL-ANC system generally underforecast areas of convective activity (bias < 1) and missed forecasting the full extent of the new convection. Forecasters felt that the system performed best under situations of strong low-level forcing along drylines and cold fronts, and performed poorly under convection driven by solar heating and elevated or weakly forced convection in the vicinity of warm fronts. This is consistent with the discussion of the ANC regimes and performance in sections 3b, 4, and 5. The ANC system was not designed to nowcast elevated convection, which occurs ~50% of the time over the southern Great Plains (Wilson and Roberts 2006). Forecasters also felt it would be desirable to have the ANC produce probabilistic nowcasts to provide confidence intervals and measures of uncertainty in the forecasts; efforts in this area are already under way. In the future, additional thought needs to be given on how to best utilize the forecaster’s wealth of experience and skill in improving forecaster–computer techniques for nowcasting high-impact weather and disseminating rapidly updated products to end users.

Acknowledgments

This work was funded by the NWS Aviation Weather Branch via Grant DG133W06CN0182, under the direction of Kevin Johnston, Cynthia Abelman, and Curt Neidhart and through the NextGen program under the guidance of Mark Miller. The success of this demonstration was due to the six years of support provided by the FWD staff in entering boundaries and monitoring the products in real time, and to Dan Megenhardt at NCAR who provided numerous engineering enhancements and excellent real-time support. Steven Fano, Gregory Patrick, William Bunting, Ted Ryan, and Lance Bucklew provided us with valuable and insightful feedback throughout the demonstration. Thomas Amis and the Fort Worth CWSU staff provided helpful commentary on the nowcast fields. Collaboration with Stephan Smith, Mamoudou Ba, Kenneth Sperow, Scott O’Donnell, Xuning Tan, John Crockett, and Lingyan Xin of NWS MDL made it possible to completely integrate this system into the WFO environment and provide the forecasters with tools on AWIPS, including the boundary tools developed by Dave Albo at NCAR. Karen Griggs provided terrific assistance with the figures. The three formal reviewers provided detailed, discerning, thoughtful reviews that really helped make the paper much more complete.

REFERENCES

REFERENCES
Bally
,
J.
,
2004
:
The Thunderstorm Interactive Forecast System: Turning automated thunderstorm tracks into severe weather warnings
.
Wea. Forecasting
,
19
,
64
72
.
Benjamin
,
S.
, and
Coauthors
,
2004
:
An hourly assimilation–forecast cycle: The RUC
.
Mon. Wea. Rev.
,
132
,
495
518
.
Benjamin
,
S.
,
M.
Hu
,
S.
Weygandt
, and
D.
Devenyi
,
2009
:
Integrated assimilation of radar, satellite, and METAR cloud data for initial hydrometeor/divergence fields to improve hourly updated short-range forecasts from the RUC, Rapid Refresh, and HRRR. Int. Symp. on Nowcasting and Very Short Term Forecasting, Whistler, BC, Canada, WMO, 3.1. [Available online at http://www.wmo.int/pages/prog/arep/wwrp/new/documents/WSN09_program_booklet.pdf.]
Boyd
,
J. G.
,
1965
:
Observation of two intersecting radar fine lines
.
Mon. Wea. Rev.
,
93
,
188
.
Casati
,
B.
,
G.
Ross
, and
D. B.
Stephenson
,
2004
:
A new intensity-scale approach for the verification of spatial precipitation forecasts
.
Meteor. Appl.
,
11
,
141
154
.
Casati
,
B.
, and
Coauthors
,
2008
:
Forecast verification: Current status and future directions
.
Meteor. Appl.
,
15
,
3
18
.
Davis
,
C.
,
B.
Brown
, and
R.
Bullock
,
2006a
:
Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas
.
Mon. Wea. Rev.
,
134
,
1772
1784
.
Davis
,
C.
,
B.
Brown
, and
R.
Bullock
,
2006b
:
Object-based verification of precipitation forecasts. Part II: Application to convective rain systems
.
Mon. Wea. Rev.
,
134
,
1785
1795
.
Delanoy
,
R. L.
, and
S. W.
Troxel
,
1993
:
A machine intelligent gust front algorithm for Doppler weather radar systems. Preprints, Fifth Int. Conf. on Aviation Weather Systems, Vienna, VA, Amer. Meteor. Soc., 125–129
.
Dixon
,
M.
, and
G.
Wiener
,
1993
:
TITAN: Thunderstorm Identification, Tracking, Analysis, and Nowcasting—A radar based methodology
.
J. Atmos. Oceanic Technol.
,
10
,
785
797
.
Done
,
J.
,
C.
Davis
, and
M.
Weisman
,
2004
:
The next generation of NWP: Explicit forecasts of convection using the Weather Research and Forecasting (WRF) model
.
Atmos. Sci. Lett.
,
5
,
110
117
.
Dye
,
J. E.
,
W. P.
Winn
,
J. J.
Jones
, and
D. W.
Breed
,
1989
:
The electrification of New Mexico thunderstorms. Part I. Relationships between precipitation development and the onset of electrification
.
J. Geophys. Res.
,
94
(
D6
),
8643
8656
.
Ebert
,
E. E.
,
2008
:
Fuzzy verification of high resolution gridded forecasts: A review and proposed framework
.
Meteor. Appl.
,
15
,
53
66
.
Ebert
,
E. E.
, and
J. L.
McBride
,
2000
:
Verification of precipitation in weather systems: Determination of systematic errors
.
J. Hydrol.
,
239
,
179
202
.
Ebert
,
E. E.
,
L. J.
Wilson
,
B. G.
Brown
,
P.
Nurmi
,
H. E.
Brooks
,
J.
Bally
, and
M.
Jaeneke
,
2004
:
Verification of nowcasts from the WWRP Sydney 2000 Forecast Demonstration Project
.
Wea. Forecasting
,
19
,
73
96
.
Fowle
,
M. A.
, and
P. J.
Roebber
,
2003
:
Short-range (0–48 h) numerical prediction of convective occurrence, mode, and location
.
Wea. Forecasting
,
18
,
782
794
.
Gremillion
,
M. S.
, and
R. E.
Orville
,
1999
:
Thunderstorm characteristics of cloud-to-ground lightning at the Kennedy Space Center, Florida: A study of lightning initiation signatures as indicated by WSR-88D
.
Wea. Forecasting
,
14
,
640
649
.
Haiden
,
T.
,
A.
Kann
,
C.
Wittmann
,
G.
Pistotnik
,
B.
Bica
, and
C.
Gruber
,
2011
:
The Integrated Nowcasting through Comprehensive Analysis (INCA) system and its validation over the eastern Alpine region
.
Wea. Forecasting
,
26
,
166
183
.
JPDO
,
2007
:
Concept of operations for the next generation air transportation system. Version 1.2, Joint Planning and Development Office, Washington, DC, 226 pp
.
Keenan
,
T.
, and
Coauthors
,
2003
:
The Sydney 2000 World Weather Research Programme Forecast Demonstration Project: Overview and current status
.
Bull. Amer. Meteor. Soc.
,
84
,
1041
1054
.
Marsigli
,
C.
,
A.
Montani
, and
T.
Paccagnella
,
2008
:
A spatial verification method applied to the evaluation of high-resolution ensemble forecasts
.
Meteor. Appl.
,
15
,
127
145
.
Mass
,
C. F.
,
2003
:
IFPS and the future of the National Weather Service
.
Wea. Forecasting
,
18
,
75
79
.
Mass
,
C. F.
,
D.
Ovens
,
K.
Westrick
, and
B.
Colle
,
2002
:
Does increasing horizontal resolution produce more skillful forecasts?
Bull. Amer. Meteor. Soc.
,
83
,
407
430
.
McNeil
,
D.
, and
P.
Freiberger
,
1993
:
Fuzzy Logic—The Revolutionary Computer Technology That Is Changing Our World. Simon and Schuster, 319 pp
.
Mueller
,
C.
,
T.
Saxen
,
R.
Roberts
,
J.
Wilson
,
T.
Betancourt
,
S.
Dettling
,
N.
Oien
, and
J.
Yee
,
2003
:
NCAR Auto-Nowcast System
.
Wea. Forecasting
,
18
,
545
561
.
Nelson
,
E.
,
R.
Roberts
,
T.
Saxen
, and
H.
Cai
,
2006
:
The NWS/NCAR “Forecaster Over The Loop” Fort Worth operational demonstration: Human enhancement of a thunderstorm nowcasting system. Fifth Int. Conf. on Mesoscale Meteorology and Typhoon, Boulder, CO, East Asia Weather Research Association. [Available online at http://www.eol.ucar.edu/icmcs/program.html.]
Nelson
,
E.
,
R.
Roberts
,
T.
Saxen
,
H.
Cai
,
J.
Pinto
, and
M.
Steiner
,
2008
:
Evaluating the performance of different synoptic regime settings on the NCAR Auto-Nowcaster. Preprints, 13th Conf. on Aviation, Range and Aerospace Meteorology, New Orleans, LA, Amer. Meteor. Soc., 3.2. [Available online at https://ams.confex.com/ams/88Annual/techprogram/paper_132736.htm.]
Olson
,
D. A.
,
N. W.
Junker
, and
B.
Korty
,
1995
:
Evaluation of 33 years of quantitative precipitation forecasting at the NMC
.
Wea. Forecasting
,
10
,
498
511
.
Pierce
,
C. E.
, and
Coauthors
,
2004
:
The nowcasting of precipitation during Sydney 2000: An appraisal of the QPF algorithms
.
Wea. Forecasting
,
19
,
7
21
.
Pinto
,
J. O.
,
W.
Dupree
,
S.
Weygandt
,
M.
Wolfson
,
S.
Benjamin
, and
M.
Steiner
,
2010
:
Advances in the Consolidated Storm Prediction for Aviation (CoSPA). Preprints, 14th Conf. on Aviation, Range, and Aerospace Meteorology, Atlanta, GA, Amer. Meteor. Soc., J11.2. [Available online at http://ams.confex.com/ams/pdfpapers/163811.pdf.]
Purdom
,
J. F. W.
,
1976
:
Some uses of high resolution GOES imagery in the mesoscale forecasting of convection and its behavior
.
Mon. Wea. Rev.
,
104
,
1474
1483
.
Purdom
,
J. F. W.
,
1982
:
Subjective interpretations of geostationary satellite data for nowcasting. Nowcasting, K. A. Browning, Ed., Academic Press, 149–166
.
Purdom
,
J. F. W.
, and
K.
Marcus
,
1982
:
Thunderstorm trigger mechanisms over the southeast United States. Preprints, 12th Conf. on Severe Local Storms, San Antonio, TX, Amer. Meteor. Soc., 487–488
.
Roberts
,
N.
,
2008
:
Assessing the spatial and temporal variation in the skill of precipitation forecasts from an NWP model
.
Meteor. Appl.
,
15
,
163
169
.
Roberts
,
N.
, and
H. W.
Lean
,
2008
:
Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events
.
Mon. Wea. Rev.
,
136
,
78
97
.
Roberts
,
R. D.
, and
S.
Rutledge
,
2003
:
Nowcasting storm initiation and growth using GOES-8 and WSR-88D data
.
Wea. Forecasting
,
18
,
562
584
.
Roberts
,
R. D.
, and
Coauthors
,
2005
:
The NWS/NCAR Man-in-the-Loop (MITL) demonstration: Forecaster input into a thunderstorm nowcasting system. Preprints, World Weather Research Program Symp. on Nowcasting and Very Short Range Forecasting, Toulouse, France, WMO, 7.26. [Available online at http://www.meteo.fr/cic/wsn05/DVD/index.html.]
Roberts
,
R. D.
,
E.
Nelson
,
J.
Pinto
,
T.
Saxen
, and
C.
Phillips
,
2007
:
Status of nowcasting thunderstorm initiation: Where do we go from here? Preprints, 33rd Conf. on Radar Meteorology, Cairns, QLD, Australia, Amer. Meteor. Soc., 5.3. [Available online at http://ams.confex.com/ams/pdfpapers/123659.pdf.]
Roberts
,
R. D.
,
E.
Nelson
, and
B.
Brown
,
2009
:
A role for the forecaster in improving convection initiation nowcasts: Operational methodology and validation of approach. Int. Symp. on Nowcasting and Very Short Range Forecasting, Whistler, BC, Canada, WMO, 2.10. [Available online at http://www.wmo.int/pages/prog/arep/wwrp/new/documents/WSN09_program_booklet.pdf.]
Roberts
,
W.
, and
L.
Cheatwood-Harris
,
2010
:
The 2009 AutoNowcaster evaluation at the Ft. Worth/Dallas Weather Forecast Office. NWS NextGen Program Office Rep., 44 pp
.
Roebber
,
P. J.
,
2009
:
Visualizing multiple measures of forecast quality
.
Wea. Forecasting
,
24
,
601
608
.
Roebber
,
P. J.
,
D. M.
Schultz
,
B. A.
Colle
, and
D. J.
Stensrud
,
2004
:
Toward improved prediction: High-resolution and ensemble modeling systems in operations
.
Wea. Forecasting
,
19
,
936
949
.
Saxen
,
T. R.
, and
Coauthors
,
2008
:
The operational mesogamma-scale analysis and forecast system of the U.S. Army Test and Evaluation Command. Part IV: The White Sands Missile Range Auto-Nowcast System
.
J. Appl. Meteor. Climatol.
,
47
,
1123
1139
.
Sills
,
D.
,
2009
:
On the MSC Forecasters Forums and the future role of the human forecaster
.
Bull. Amer. Meteor. Soc.
,
90
,
619
627
.
Smith
,
S. B.
,
J. T.
Johnson
,
R. D.
Roberts
,
S. M.
Zubrick
, and
S. J.
Weiss
,
1998
:
The System for Convection Analysis and Nowcasting (SCAN): 1997–1998 field test. Preprints, 19th Conf. on Severe Local Storms, Minneapolis, MN, Amer. Meteor. Soc., 790–793
.
Steiner
,
M.
,
R.
Bateman
,
D.
Megenhardt
,
Y.
Liu
,
M.
Xu
,
M.
Pocernich
, and
J.
Krozel
,
2010
:
Translation of ensemble weather forecasts into probabilistic air traffic capacity impact
.
Air Traffic Control Quart.
,
18
,
229
254
.
Stensrud
,
D. J.
,
H. E.
Brooks
,
J.
Du
,
M. S.
Tracton
, and
E.
Rogers
,
1999
:
Using ensembles for short-range forecasting
.
Mon. Wea. Rev.
,
127
,
433
446
.
Stensrud
,
D. J.
, and
Coauthors
,
2009
:
Convective-scale warn-on-forecast system: A vision for 2020
.
Bull. Amer. Meteor. Soc.
,
90
,
1487
1499
.
Stephenson
,
D. B.
,
B.
Casati
,
C. A. T.
Ferro
, and
C. A.
Wilson
,
2008
:
The extreme dependency score: A non-vanishing measure for forecasts of the rare events
.
Meteor. Appl.
,
15
,
41
50
.
Stuart
,
N. A.
, and
Coauthors
,
2006
:
The future of humans in an increasingly automated forecast process
.
Bull. Amer. Meteor. Soc.
,
87
,
1497
1502
.
Stuart
,
N. A.
,
R.
Grumm
,
J.
Moore
,
A. E.
Pietrycha
,
K.
Reeves
,
E.
Abrams
, and
C. F.
Mass
,
2007a
:
Reaching the goals outlined in the first forum
.
Bull. Amer. Meteor. Soc.
,
88
,
1896
1897
.
Stuart
,
N. A.
,
D.
Schultz
, and
G.
Klein
,
2007b
:
Maintaining the role of humans in the forecast process
.
Bull. Amer. Meteor. Soc.
,
88
,
1893
1896
.
Sun
,
J.
,
2005
:
Convective-scale assimilation of radar data; progress and challenges
.
Quart. J. Roy. Meteor. Soc.
,
131
,
3439
3463
.
Sun
,
J.
, and
N. A.
Crook
,
1997
:
Dynamical and microphysical retrieval from Doppler radar observations using a cloud model and its adjoint. Part I: Model development and simulated data experiments
.
J. Atmos. Sci.
,
54
,
1642
1661
.
Weisman
,
M. L.
, and
J. B.
Klemp
,
1982
:
The dependence of numerically simulated convective storms on vertical wind shear and buoyancy
.
Mon. Wea. Rev.
,
110
,
504
520
.
Weisman
,
M. L.
,
C.
Davis
,
W.
Wang
,
K. W.
Manning
, and
J. B.
Klemp
,
2008
:
Experiences with 0–36-h explicit convective forecasts with the WRF-ARW model
.
Wea. Forecasting
,
23
,
407
437
.
Weygandt
,
S.
,
S.
Benjamin
,
T.
Smirnova
,
K.
Brundage
,
C.
Alexander
,
M.
Hu
,
B.
Jamison
, and
S.
Sahm
,
2009
:
Evaluation of the High Resolution Rapid Refresh (HRRR): An hourly updated convection resolving model utilizing radar reflectivity assimilation from the RUC/RR. Int. Symp. on Nowcasting and Very Short Range Forecasting, Whistler, BC, Canada, WMO, 3.17. [Available online at http://www.wmo.int/pages/prog/arep/wwrp/new/documents/WSN09_program_booklet.pdf.]
White
,
A.
,
P.
Neiman
,
D.
Gottas
,
S.
Gutman
, and
I.
Jankov
,
2009
:
A coastal atmospheric river monitoring and early warning system. Int. Symp. on Nowcasting and Very Short Range Forecasting, Whistler, BC, Canada, WMO, 4.11. [Available online at http://www.wmo.int/pages/prog/arep/wwrp/new/documents/WSN09_program_booklet.pdf.]
Wilks
,
D. S.
,
2006
:
Statistical Methods in the Atmospheric Sciences. 2nd ed. Academic Press, 627 pp
.
Wilson
,
J. W.
, and
W. E.
Schreiber
,
1986
:
Initiation of convective storms by radar-observed boundary layer convergent lines
.
Mon. Wea. Rev.
,
114
,
2516
2536
.
Wilson
,
J. W.
, and
C. K.
Mueller
,
1993
:
Nowcasts of thunderstorm initiation and evolution
.
Wea. Forecasting
,
8
,
113
131
.
Wilson
,
J. W.
, and
D. L.
Megenhardt
,
1997
:
Thunderstorm initiation, organization, and lifetime associated with Florida boundary layer convergence lines
.
Mon. Wea. Rev.
,
125
,
1507
1525
.
Wilson
,
J. W.
, and
R. D.
Roberts
,
2006
:
Summary of convective storm initiation and evolution during IHOP: Observational and modeling perspective
.
Mon. Wea. Rev.
,
134
,
23
47
.
Wilson
,
J. W.
,
E. E.
Ebert
,
T. R.
Saxen
,
R. D.
Roberts
,
C. K.
Mueller
,
M.
Sleigh
,
C. E.
Pierce
, and
A.
Seed
,
2004
:
Sydney 2000 Forecast Demonstration Project: Convective storm nowcasting
.
Wea. Forecasting
,
19
,
131
150
.
Wilson
,
J. W.
,
Y.
Feng
,
M.
Chen
, and
R. D.
Roberts
,
2010
:
Nowcasting challenges during the Beijing Olympics: Successes, failures, and implications for future nowcasting systems
.
Wea. Forecasting
,
25
,
1691
1714
.
Wong
,
W.-K.
,
L. H. Y.
Yeung
,
Y.-C.
Wang
, and
M.
Chen
,
2009
:
Towards the blending of NWP with Nowcast–Operation experience in B08FDP. Int. Symp. on Nowcasting and Very Short Range Forecasting, Whistler, BC, Canada, WMO, 4.5. [Available online at http://www.wmo.int/pages/prog/arep/wwrp/new/documents/WSN09_program_booklet.pdf.]
Xue
,
M.
,
D.
Wang
,
J.
Gao
,
K.
Brewster
, and
K. K.
Droegemeier
,
2003
:
The Advanced Regional Prediction System (ARPS), storm-scale numerical weather prediction and data assimilation
.
Meteor. Atmos. Phys.
,
82
,
139
170
.

Footnotes

*

Current affiliation: Neptune and Company, Inc., Lakewood, Colorado.

+

Current affiliation: Lifetouch, Eden Prairie, Minnesota.

#

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

1

The Nowcasting Working Group of the World Meteorological Organization (WMO) World Weather Research Program has defined nowcasting as follows: forecasting with local detail, by any method, over a period from the present to a few hours ahead; this includes a detailed description of the present weather.

2

In lieu of lightning data, which were not available in the FOTL-ANC system, convective storms are defined as storms reaching radar echo intensities ≥35 dBZ. This designation is based partially on studies by Dye et al. (1989) and Gremillion and Orville (1999), who have shown that the onset of storm electrification generally occurs >5 min after storm echoes of 30 dBZ or greater have reached subfreezing levels.

3

All times are listed in UTC; subtract 5 h for Fort Worth, TX, central daylight time.

4

Storm extrapolation nowcasts of existing storms are obtained using the radar-based TITAN (Dixon and Wiener 1993) algorithm. Storm extrapolation and persistence are the typical benchmark nowcasts against which all other short-term precipitation nowcasts are compared (e.g., Mueller et al. 2003; Ebert et al. 2004, Pierce et al. 2004; Wilson et al. 2004; Saxen et al. 2008).

5

Hereafter, FOTL-ANC (ANC) notations are used to represent human (no human) involvement in producing the nowcasts.