Search Results

You are looking at 1 - 10 of 11 items for :

  • Author or Editor: Elizabeth E. Ebert x
  • Weather and Forecasting x
  • Refine by Access: All Content x
Clear All Modify Search
Elizabeth E. Ebert

Abstract

High-resolution forecasts may be quite useful even when they do not match the observations exactly. Neighborhood verification is a strategy for evaluating the “closeness” of the forecast to the observations within space–time neighborhoods rather than at the grid scale. Various properties of the forecast within a neighborhood can be assessed for similarity to the observations, including the mean value, fractional coverage, occurrence of a forecast event sufficiently near an observed event, and so on. By varying the sizes of the neighborhoods, it is possible to determine the scales for which the forecast has sufficient skill for a particular application. Several neighborhood verification methods have been proposed in the literature in the last decade. This paper examines four such methods in detail for idealized and real high-resolution precipitation forecasts, highlighting what can be learned from each of the methods. When applied to idealized and real precipitation forecasts from the Spatial Verification Methods Intercomparison Project, all four methods showed improved forecast performance for neighborhood sizes larger than grid scale, with the optimal scale for each method varying as a function of rainfall intensity.

Full access
Chermelle Engel and Elizabeth E. Ebert

Abstract

This paper describes an extension of an operational consensus forecasting (OCF) scheme from site forecasts to gridded forecasts. OCF is a multimodel consensus scheme including bias correction and weighting. Bias correction and weighting are done on a scale common to almost all multimodel inputs (1.25°), which are then downscaled using a statistical approach to an approximately 5-km-resolution grid. Local and international numerical weather prediction model inputs are found to have coarse scale biases that respond to simple bias correction, with the weighted average consensus at 1.25° outperforming all models at that scale. Statistical downscaling is found to remove the systematic representativeness error when downscaling from 1.25° to 5 km, though it cannot resolve scale differences associated with transient small-scale weather.

Full access
Andrew Brown, Andrew Dowdy, and Elizabeth E. Ebert

Abstract

Epidemic asthma events represent a significant risk to emergency services as well as the wider community. In southeastern Australia, these events occur in conjunction with relatively high amounts of grass pollen during the late spring and early summer, which may become concentrated in populated areas through atmospheric convergence caused by a number of physical mechanisms including thunderstorm outflow. Thunderstorm forecasts are therefore important for identifying epidemic asthma risk factors. However, the representation of thunderstorm environments using regional numerical weather prediction models, which are a key aspect of the construction of these forecasts, have not yet been systematically evaluated in the context of epidemic asthma events. Here, we evaluate diagnostics of thunderstorm environments from historical simulations of weather conditions in the vicinity of Melbourne, Australia, in relation to the identification of epidemic asthma cases based on hospital data from a set of controls. Skillful identification of epidemic asthma cases is achieved using a thunderstorm diagnostic that describes near-surface water vapor mixing ratio. This diagnostic is then used to gain insights on the variability of meteorological environments related to epidemic asthma in this region, including diurnal variations, long-term trends, and the relationship with large-scale climate drivers. Results suggest that there has been a long-term increase in days with high water vapor mixing ratio during the grass pollen season, with large-scale climate drivers having a limited influence on these conditions.

Significance Statement

We investigate the atmospheric conditions associated with epidemic thunderstorm asthma events in Melbourne, Australia, using historical model simulations of the weather. Conditions appear to be associated with high atmospheric moisture content, which relates to environments favorable for severe thunderstorms, but also potentially pollen rupturing as suggested by previous studies. These conditions are shown to be just as important as the concentration of grass pollen for a set of epidemic thunderstorm asthma events in this region. This means that weather model simulations of thunderstorm conditions can be incorporated into the forecasting process for epidemic asthma in Melbourne, Australia. We also investigate long-term variability in atmospheric conditions associated with severe thunderstorms, including relationships with the large-scale climate and long-term trends.

Open access
John L. McBride and Elizabeth E. Ebert

Abstract

Real-time gridded 24-h quantitative precipitation forecasts from seven operational NWP models are verified over the Australian continent. All forecasts have been mapped to a 1° latitude–longitude grid and have been verified against an operational daily rainfall analysis, mapped to the same grid. The verification focuses on two large subregions: the northern tropical monsoon regime and the southeastern subtropical regime. Statistics are presented of the bias score, probability of detection, and false alarm ratio for a range of rainfall threshold values. The basic measure of skill used in this study, however, is the Hanssen and Kuipers (HK) score and its two components: accuracy for events and accuracy for nonevents.

For both regimes the operational models tend to overestimate rainfall in summer and to underestimate it in winter. In the southeastern region the models have HK scores ranging from 0.5 to 0.7, and easily outperform a forecast of persistence. Thus for the current operational NWP models, the 24-h rain forecasts can be considered quite skillful in the subtropics. On the other hand, model skill is quite low in the northern regime with HK values of only 0.2–0.6. During the summer wet season the low skill is associated with an inability to simulate the behavior of tropical convective rain systems. During the winter dry season, it is associated with a low probability of detection for the occasional rainfall event. Thus it could be said that models have no real skill at rainfall forecasts in this monsoonal wet season regime.

Model skill falls dramatically for occurrence thresholds greater than 10 mm day−1. This implies that the models are much better at predicting the occurrence of rain than they are at predicting the magnitude and location of the peak values.

Full access
Elizabeth E. Ebert and William A. Gallus Jr.

Abstract

The contiguous rain area (CRA) method for spatial forecast verification is a features-based approach that evaluates the properties of forecast rain systems, namely, their location, size, intensity, and finescale pattern. It is one of many recently developed spatial verification approaches that are being evaluated as part of a Spatial Forecast Verification Methods Intercomparison Project. To better understand the strengths and weaknesses of the CRA method, it has been tested here on a set of idealized geometric and perturbed forecasts with known errors, as well as nine precipitation forecasts from three high-resolution numerical weather prediction models.

The CRA method was able to identify the known errors for the geometric forecasts, but only after a modification was introduced to allow nonoverlapping forecast and observed features to be matched. For the perturbed cases in which a radar rain field was spatially translated and amplified to simulate forecast errors, the CRA method also reproduced the known errors except when a high-intensity threshold was used to define the CRA (≥10 mm h−1) and a large translation error was imposed (>200 km). The decomposition of total error into displacement, volume, and pattern components reflected the source of the error almost all of the time when a mean squared error formulation was used, but not necessarily when a correlation-based formulation was used.

When applied to real forecasts, the CRA method gave similar results when either best-fit criteria, minimization of the mean squared error, or maximization of the correlation coefficient, was chosen for matching forecast and observed features. The diagnosed displacement error was somewhat sensitive to the choice of search distance. Of the many diagnostics produced by this method, the errors in the mean and peak rain rate between the forecast and observed features showed the best correspondence with subjective evaluations of the forecasts, while the spatial correlation coefficient (after matching) did not reflect the subjective judgments.

Full access
David Ahijevych, Eric Gilleland, Barbara G. Brown, and Elizabeth E. Ebert

Abstract

Several spatial forecast verification methods have been developed that are suited for high-resolution precipitation forecasts. They can account for the spatial coherence of precipitation and give credit to a forecast that does not necessarily match the observation at any particular grid point. The methods were grouped into four broad categories (neighborhood, scale separation, features based, and field deformation) for the Spatial Forecast Verification Methods Intercomparison Project (ICP). Participants were asked to apply their new methods to a set of artificial geometric and perturbed forecasts with prescribed errors, and a set of real forecasts of convective precipitation on a 4-km grid. This paper describes the intercomparison test cases, summarizes results from the geometric cases, and presents subjective scores and traditional scores from the real cases.

All the new methods could detect bias error, and the features-based and field deformation methods were also able to diagnose displacement errors of precipitation features. The best approach for capturing errors in aspect ratio was field deformation. When comparing model forecasts with real cases, the traditional verification scores did not agree with the subjective assessment of the forecasts.

Full access
Eric Gilleland, David Ahijevych, Barbara G. Brown, Barbara Casati, and Elizabeth E. Ebert

Abstract

Advancements in weather forecast models and their enhanced resolution have led to substantially improved and more realistic-appearing forecasts for some variables. However, traditional verification scores often indicate poor performance because of the increased small-scale variability so that the true quality of the forecasts is not always characterized well. As a result, numerous new methods for verifying these forecasts have been proposed. These new methods can mostly be classified into two overall categories: filtering methods and displacement methods. The filtering methods can be further delineated into neighborhood and scale separation, and the displacement methods can be divided into features based and field deformation. Each method gives considerably more information than the traditional scores, but it is not clear which method(s) should be used for which purpose.

A verification methods intercomparison project has been established in order to glean a better understanding of the proposed methods in terms of their various characteristics and to determine what verification questions each method addresses. The study is ongoing, and preliminary qualitative results for the different approaches applied to different situations are described here. In particular, the various methods and their basic characteristics, similarities, and differences are described. In addition, several questions are addressed regarding the application of the methods and the information that they provide. These questions include (i) how the method(s) inform performance at different scales; (ii) how the methods provide information on location errors; (iii) whether the methods provide information on intensity errors and distributions; (iv) whether the methods provide information on structure errors; (v) whether the approaches have the ability to provide information about hits, misses, and false alarms; (vi) whether the methods do anything that is counterintuitive; (vii) whether the methods have selectable parameters and how sensitive the results are to parameter selection; (viii) whether the results can be easily aggregated across multiple cases; (ix) whether the methods can identify timing errors; and (x) whether confidence intervals and hypothesis tests can be readily computed.

Full access
Jeremy S. Grams, Willam A. Gallus Jr., Steven E. Koch, Linda S. Wharton, Andrew Loughe, and Elizabeth E. Ebert

Abstract

The Ebert–McBride technique (EMT) is an entity-oriented method useful for quantitative precipitation verification. The EMT was modified to optimize its ability to identify contiguous rain areas (CRAs) during the 2002 International H2O Project (IHOP). This technique was then used to identify systematic sources of error as a function of observed convective system morphology in three 12-km model simulations run over the IHOP domain: Eta, the fifth-generation Pennsylvania State University–NCAR Mesoscale Model (MM5), and the Weather Research and Forecasting (WRF). The EMT was fine-tuned to optimize the pattern matching of forecasts to observations for the scales of precipitation systems observed during IHOP. To investigate several error measures provided by the EMT, a detailed morphological analysis of observed systems was performed using radar data for all CRAs identified in the IHOP domain. The modified EMT suggests that the Eta Model produced average rain rates, peak rainfall amounts, and total rain volumes that were lower than observed for almost all types of convective systems, likely because of its production of overly smoothed and low-variability quantitative precipitation forecasts. The MM5 and WRF typically produced average rain rates and peak rainfall amounts that were larger than observed in most linear convective systems. However, the rain volume for these models was too low for almost all types of convective systems, implying a sizeable underestimate in areal coverage. All three models forecast rainfall too far northwest for linear systems. The results for the WRF and MM5 are consistent with previous observations of mesoscale models run with explicit microphysics and no convective parameterization scheme, suggesting systematic problems with the prediction of mesoscale convective system cold pool dynamics.

Full access
Elizabeth E. Ebert, Laurence J. Wilson, Barbara G. Brown, Pertti Nurmi, Harold E. Brooks, John Bally, and Matthias Jaeneke

Abstract

The verification phase of the World Weather Research Programme (WWRP) Sydney 2000 Forecast Demonstration Project (FDP) was intended to measure the skill of the participating nowcast algorithms in predicting the location of convection, rainfall rate and occurrence, wind speed and direction, severe thunderstorm wind gusts, and hail location and size. An additional question of interest was whether forecasters could improve the quality of the nowcasts compared to the FDP products alone.

The nowcasts were verified using a variety of statistical techniques. Observational data came from radar reflectivity and rainfall analyses, a network of rain gauges, and human (spotter) observations. The verification results showed that the cell tracking algorithms predicted the location of the strongest cells with a mean error of about 15–30 km for a 1-h forecast, and were usually more accurate than an extrapolation (Lagrangian persistence) forecast. Mean location errors for the area tracking schemes were on the order of 20 km.

Almost all of the algorithms successfully predicted the frequency of rain throughout the forecast period, although most underestimated the frequency of high rain rates. The skill in predicting rain occurrence decreased very quickly into the forecast period. In particular, the algorithms could not predict the precise location of heavy rain beyond the first 10–20 min. Using radar analyses as verification, the algorithms' spatial forecasts were consistently more skillful than simple persistence. However, when verified against rain gauge observations at point locations, the algorithms had difficulty beating persistence, mainly due to differences in spatial and temporal resolution.

Only one algorithm attempted to forecast gust fronts. The results for a limited sample showed a mean absolute error of 7 km h−1 and mean bias of 3 km h−1 in the speed of the gust fronts during the FDP. The errors in sea-breeze front forecasts were half as large, with essentially no bias. Verification of the hail associated with the 3 November tornadic storm showed that the two algorithms that estimated hail size and occurrence successfully diagnosed the onset and cessation of the hail to within 30 min of the reported sightings. The time evolution of hail size was reasonably well captured by the algorithms, and the predicted mean and maximum hail diameters were consistent with the observations.

The Thunderstorm Interactive Forecast System (TIFS) allowed forecasters to modify the output of the cell tracking nowcasts, primarily using it to remove cells that were insignificant or diagnosed with incorrect motion. This manual filtering resulted in markedly reduced mean cell position errors when compared to the unfiltered forecasts. However, when forecasters attempted to adjust the storm tracks for a small number of well-defined intense cells, the position errors increased slightly, suggesting that in such cases the objective guidance is probably the best estimate of storm motion.

Full access
Elizabeth E. Ebert, Michael Turk, Sheldon J. Kusselson, Jianbin Yang, Matthew Seybold, Peter R. Keehn, and Robert J. Kuligowski

Abstract

Ensemble tropical rainfall potential (eTRaP) has been developed to improve short-range forecasts of heavy rainfall in tropical cyclones. Evolving from the tropical rainfall potential (TRaP), a 24-h rain forecast based on estimated rain rates from microwave sensors aboard polar-orbiting satellites, eTRaP combines all single-pass TRaPs generated within ±3 h of 0000, 0600, 1200, and 1800 UTC to form a simple ensemble. This approach addresses uncertainties in satellite-derived rain rates and spatial rain structures by using estimates from different sensors observing the cyclone at different times. Quantitative precipitation forecasts (QPFs) are produced from the ensemble mean field using a probability matching approach to recalibrate the rain-rate distribution against the ensemble members (e.g., input TRaP forecasts) themselves. ETRaPs also provide probabilistic forecasts of heavy rain, which are potentially of enormous benefit to decision makers. Verification of eTRaP forecasts for 16 Atlantic hurricanes making landfall in the United States between 2004 and 2008 shows that the eTRaP rain amounts are more accurate than single-sensor TRaPs. The probabilistic forecasts have useful skill, but the probabilities should be interpreted within a spatial context. A novel concept of a “radius of uncertainty” compensates for the influence of location error in the probability forecasts. The eTRaPs are produced in near–real time for all named tropical storms and cyclones around the globe. They can be viewed online (http://www.ssd.noaa.gov/PS/TROP/etrap.html) and are available in digital form to users.

Full access