Search Results

You are looking at 1 - 2 of 2 items for

  • Author or Editor: Shaocai Yu x
  • Refine by Access: All Content x
Clear All Modify Search
Daiwen Kang
,
Rohit Mathur
,
Kenneth Schere
,
Shaocai Yu
, and
Brian Eder

Abstract

Traditional categorical metrics used in model evaluations are “clear cut” measures in that the model’s ability to predict an “exceedance” is defined by a fixed threshold concentration and the metrics are defined by observation–forecast sets that are paired both in space and time. These metrics are informative but limited in evaluating the performance of air quality forecast (AQF) systems because AQF generally examines exceedances on a regional scale rather than a single monitor. New categorical metrics—the weighted success index (WSI), area hit (aH), and area false-alarm ratio (aFAR)—are developed. In the calculation of WSI, credits are given to the observation–forecast pairs within the observed exceedance region (missed forecast) or the forecast exceedance region (false alarm), depending on the distance of the points from the central line (perfect observation–forecast match line or 1:1 line on scatterplot). The aH and aFAR are defined by matching observed and forecast exceedances within an area (i.e., model grid cells) surrounding the observation location. The concept of aH and aFAR resembles the manner in which forecasts are usually issued. In practice, a warning is issued for a region of interest, such as a metropolitan area, if an exceedance is forecast to occur anywhere within the region. The application of these new categorical metrics, which are supplemental to the traditional counterparts (critical success index, hit rate, and false-alarm ratio), to the Eta Model–Community Multiscale Air Quality (CMAQ) forecast system has demonstrated further insight into evaluating the forecasting capability of the system (e.g., the new metrics can provide information about how the AQF system captures the spatial variations of pollutant concentrations).

Full access
Brian Eder
,
Daiwen Kang
,
S. Trivikrama Rao
,
Rohit Mathur
,
Shaocai Yu
,
Tanya Otte
,
Ken Schere
,
Richard Wayland
,
Scott Jackson
,
Paula Davidson
,
Jeff McQueen
, and
George Bridgers

The National Air Quality Forecast Capability (NAQFC) currently provides next-day forecasts of ozone concentrations over the contiguous United States. It was developed collaboratively by NOAA and Environmental Protection Agency (EPA) in order to provide state and local agencies, as well as the general public, air quality forecast guidance. As part of the development process, the NAQFC has been evaluated utilizing strict monitor-to-gridcell matching criteria, and discrete-type statistics of forecast concentrations. While such an evaluation is important to the developers, it is equally, if not more important, to evaluate the performance using the same protocol as the model's intended application. Accordingly, the purpose of this article is to demonstrate the efficacy of the NAQFC from the perspective of a local forecaster, thereby promoting its use. Such an approach has required the development of a new evaluation protocol: one that examines the ability of the NAQFC to forecast values of the EPA's Air Quality Index (AQI) rather than ambient air concentrations; focuses on the use of categorical-type statistics related to exceedances and nonexceedances; and, most challenging, examines performance, not based on matched grid cells and monitors, but rather over a “local forecast region,” such as an air shed or metropolitan statistical area (MSA). Results from this approach, which is demonstrated for the Charlotte, North Carolina, MSA and subsequently applied to four additional MSAs during the summer of 2007, reveal that the quality of the NAQFC forecasts is generally comparable to forecasts from local agencies. Such findings will hopefully persuade forecasters, whether they are experienced with numerous tools at their disposal or inexperienced with limited resources, to utilize the NAQFC as forecast guidance.

Full access