Search Results

You are looking at 1 - 10 of 10 items for

  • Author or Editor: Randy Bullock x
  • All content x
Clear All Modify Search
Christopher Davis, Barbara Brown, and Randy Bullock

Abstract

The authors develop and apply an algorithm to define coherent areas of precipitation, emphasizing mesoscale convection, and compare properties of these areas with observations obtained from NCEP stage-IV precipitation analyses (gauge and radar combined). In Part II, fully explicit 12–36-h forecasts of rainfall from the Weather Research and Forecasting model (WRF) are evaluated. These forecasts are integrated on a 4-km mesh without a cumulus parameterization. Rain areas are defined similarly to Part I, but emphasize more intense, smaller areas. Furthermore, a time-matching algorithm is devised to group spatially and temporally coherent areas into rain systems that approximate mesoscale convective systems. In general, the WRF model produces too many rain areas with length scales of 80 km or greater. Rain systems typically last too long, and are forecast to occur 1–2 h later than observed. The intensity distribution among rain systems in the 4-km forecasts is generally too broad, especially in the late afternoon, in sharp contrast to the intensity distribution obtained on a coarser grid with parameterized convection in Part I. The model exhibits the largest positive size and intensity bias associated with systems over the Midwest and Mississippi Valley regions, but little size bias over the High Plains, Ohio Valley, and the southeast United States. For rain systems lasting 6 h or more, the critical success index for matching forecast and observed rain systems agrees closely with that obtained in a related study using manually determined rain systems.

Full access
Gregory Thompson, Randy Bullock, and Thomas F. Lee

Abstract

Overprediction of the spatial extent of aircraft icing is a major problem in forecaster products based on numerical model output. Dependence on relative humidity fields, which are inherently broad and smooth, is the cause of this difficulty. Using multispectral satellite analysis based on NOAA Advanced Very High Resolution Radiometer data, this paper shows how the spatial extent of icing potential based on model output can be reduced where there are no subfreezing cloud tops and, therefore, where icing is unlikely. Fifty-one cases were analyzed using two scenarios: 1) model output only and 2) model output screened by a satellite cloud analysis. Average area efficiency, a statistical validation measure of icing potential using coincident pilot reports of icing, improved substantially when satellite screening was applied.

Full access
Christopher Davis, Barbara Brown, and Randy Bullock

Abstract

A recently developed method of defining rain areas for the purpose of verifying precipitation produced by numerical weather prediction models is described. Precipitation objects are defined in both forecasts and observations based on a convolution (smoothing) and thresholding procedure. In an application of the new verification approach, the forecasts produced by the Weather Research and Forecasting (WRF) model are evaluated on a 22-km grid covering the continental United States during July–August 2001. Observed rainfall is derived from the stage-IV product from NCEP on a 4-km grid (averaged to a 22-km grid). It is found that the WRF produces too many large rain areas, and the spatial and temporal distribution of the rain areas reveals regional underestimates of the diurnal cycle in rain-area occurrence frequency. Objects in the two datasets are then matched according to the separation distance of their centroids. Overall, WRF rain errors exhibit no large biases in location, but do suffer from a positive size bias that maximizes during the later afternoon. This coincides with an excessive narrowing of the rainfall intensity range, consistent with the dominance of parameterized convection. Finally, matching ability has a strong dependence on object size and is interpreted as the influence of relatively predictable synoptic-scale systems on the larger areas.

Full access
Marion Mittermaier, Rachel North, Adrian Semple, and Randy Bullock

Abstract

With the resolution of global numerical weather prediction (NWP) models now typically between 10 and 20 km, forecasts are able to capture the evolution of synoptic features that are important drivers for significant surface weather. The position, timing, and intensity of jet cores, surface highs and lows, and changes in the behavior of these forecast features is explored using the Method for Object-based Diagnostic Evaluation (MODE) at the global scale. Previously this was only possible with a more subjective approach. The spatial aspects of the forecast features (objects) and their intensity can be assessed separately. The evolution of paired forecast–analysis object attributes such as location and orientation differences, as well as area ratios, can be considered. The differences in the paired object attribute distributions from various model configurations were evaluated using the k-sample Anderson–Darling (AD) test. Increases or decreases in hits, false alarms (forecast-not-observed), and misses (observed-not-forecast) features were also assessed. It was found that when focusing purely on the forecast features of interest, differences in seasonal spatial extent biases emerged, intensity biases varied as a function of analysis time, and changes in the attribute distributions could be detected but were largely insignificant, primarily due to sample size. As has been shown for kilometer-scale NWP, results from spatial verification methods are more in line with subjective assessment. This type of objective assessment provides a new dimension to the traditional assessment of global NWP, and provides output that is closer to the way in which forecasts are used.

Full access
Christopher A. Davis, Barbara G. Brown, Randy Bullock, and John Halley-Gotway

Abstract

The authors use a procedure called the method for object-based diagnostic evaluation, commonly referred to as MODE, to compare forecasts made from two models representing separate cores of the Weather Research and Forecasting (WRF) model during the 2005 National Severe Storms Laboratory and Storm Prediction Center Spring Program. Both models, the Advanced Research WRF (ARW) and the Nonhydrostatic Mesoscale Model (NMM), were run without a traditional cumulus parameterization scheme on horizontal grid lengths of 4 km (ARW) and 4.5 km (NMM). MODE was used to evaluate 1-h rainfall accumulation from 24-h forecasts valid at 0000 UTC on 32 days between 24 April and 4 June 2005. The primary variable used for evaluation was a “total interest” derived from a fuzzy-logic algorithm that compared several attributes of forecast and observed rain features such as separation distance and spatial orientation. The maximum value of the total interest obtained by comparing an object in one field with all objects in the comparison field was retained as the quality of matching for that object. The median of the distribution of all such maximum-interest values was selected as a metric of the overall forecast quality.

Results from the 32 cases suggest that, overall, the configuration of the ARW model used during the 2005 Spring Program performed slightly better than the configuration of the NMM model. The primary manifestation of the differing levels of performance was fewer false alarms, forecast rain areas with no observed counterpart, in the ARW. However, it was noted that the performance varied considerably from day to day, with most days featuring indistinguishable performance. Thus, a small number of poor NMM forecasts produced the overall difference between the two models.

Full access
Adam J. Clark, Randy G. Bullock, Tara L. Jensen, Ming Xue, and Fanyou Kong

Abstract

Meaningful verification and evaluation of convection-allowing models requires approaches that do not rely on point-to-point matches of forecast and observed fields. In this study, one such approach—a beta version of the Method for Object-Based Diagnostic Evaluation (MODE) that incorporates the time dimension [known as MODE time-domain (MODE-TD)]—was applied to 30-h precipitation forecasts from four 4-km grid-spacing members of the 2010 Storm-Scale Ensemble Forecast system with different microphysics parameterizations. Including time in MODE-TD provides information on rainfall system evolution like lifetime, timing of initiation and dissipation, and translation.

The simulations depicted the spatial distribution of time-domain precipitation objects across the United States quite well. However, all simulations overpredicted the number of objects, with the Thompson microphysics scheme overpredicting the most and the Morrison method the least. For the smallest smoothing radius and rainfall threshold used to define objects [8 km and 0.10 in. (1 in. = 2.54 cm), respectively], the most common object duration was 3 h in both models and observations. With an increased smoothing radius and rainfall threshold, the most common duration became shorter. The simulations depicted the diurnal cycle of object frequencies well, but overpredicted object frequencies uniformly across all forecast hours. The simulations had spurious maxima in initiating objects at the beginning of the forecast and a corresponding spurious maximum in dissipating objects slightly later. Examining average object velocities, a slow bias was found in the simulations, which was most pronounced in the Thompson member. These findings should aid users and developers of convection-allowing models and motivate future work utilizing time-domain methods for verifying high-resolution forecasts.

Full access
Barbara G. Brown, Gregory Thompson, Roelof T. Bruintjes, Randy Bullock, and Tressa Kane

Abstract

Recent research to improve forecasts of in-flight icing conditions has involved the development of algorithms to apply to the output of numerical weather prediction models. The abilities of several of these algorithms to predict icing conditions, as verified by pilot reports (PIREPs), are compared for two numerical weather prediction models (Eta and the Mesoscale Analysis and Prediction System) for the Winter Icing and Storms Program 1994 (WISP94) time period (25 January–25 March 1994). Algorithms included in the comparison were developed by the National Aviation Weather Advisory Unit [NAWAU, now the Aviation Weather Center (AWC)], the National Center for Atmospheric Research’s Research Applications Program (RAP), and the U.S. Air Force. Operational icing forecasts (AIRMETs) issued by NAWAU for the same time period are evaluated to provide a standard of comparison. The capabilities of the Eta Model’s explicit cloud liquid water estimates for identifying icing regions are also evaluated and compared to the algorithm results.

Because PIREPs are not systematic and are biased toward positive reports, it is difficult to estimate standard verification parameters related to overforecasting (e.g., false alarm ratio). Methods are developed to compensate for these attributes of the PIREPs. The primary verification statistics computed include the probability of detection (POD) of yes and no reports, and the areal and volume extent of the forecast region.

None of the individual algorithms were able to obtain both a higher POD and a smaller area than any other algorithm; increases in POD are associated in all cases with increases in area. The RAP algorithm provides additional information by attempting to identify the physical mechanisms associated with the forecast icing conditions. One component of the RAP algorithm, which is designed to detect and forecast icing in regions of“warm” stratiform clouds, is more efficient at detecting icing than the other components. Cloud liquid water shows promise for development as a predictor of icing conditions, with detection rates of 30% or more in this initial study. AIRMETs were able to detect approximately the same percentage of icing reports as the algorithms, but with somewhat smaller forecast areas and somewhat larger forecast volumes on average. The algorithms are able to provide guidance with characteristics that are similar to the AIRMETs and should be useful in their formulation.

Full access
Abayomi A. Abatan, William J. Gutowski Jr., Caspar M. Ammann, Laurna Kaatz, Barbara G. Brown, Lawrence Buja, Randy Bullock, Tressa Fowler, Eric Gilleland, and John Halley Gotway

Abstract

This study analyzes spatial and temporal characteristics of multiyear droughts and pluvials over the southwestern United States with a focus on the upper Colorado River basin. The study uses two multiscalar moisture indices: standardized precipitation evapotranspiration index (SPEI) and standardized precipitation index (SPI) on a 36-month scale (SPEI36 and SPI36, respectively). The indices are calculated from monthly average precipitation and maximum and minimum temperatures from the Parameter-Elevation Regressions on Independent Slopes Model dataset for the period 1950–2012. The study examines the relationship between individual climate variables as well as large-scale atmospheric circulation features found in reanalysis output during drought and pluvial periods. The results indicate that SPEI36 and SPI36 show similar temporal and spatial patterns, but that the inclusion of temperatures in SPEI36 leads to more extreme magnitudes in SPEI36 than in SPI36. Analysis of large-scale atmospheric fields indicates an interplay between different fields that yields extremes over the study region. Widespread drought (pluvial) events are associated with enhanced positive (negative) 500-hPa geopotential height anomaly linked to subsidence (ascent) and negative (positive) moisture convergence and precipitable water anomalies. Considering the broader context of the conditions responsible for the occurrence of prolonged hydrologic anomalies provides water resource managers and other decision-makers with valuable understanding of these events. This perspective also offers evaluation opportunities for climate models.

Full access
Steven D. Miller, Courtney E. Weeks, Randy G. Bullock, John M. Forsythe, Paul A. Kucera, Barbara G. Brown, Cory A. Wolff, Philip T. Partain, Andrew S. Jones, and David B. Johnson

Abstract

Clouds pose many operational hazards to the aviation community in terms of ceilings and visibility, turbulence, and aircraft icing. Realistic descriptions of the three-dimensional (3D) distribution and temporal evolution of clouds in numerical weather prediction models used for flight planning and routing are therefore of central importance. The introduction of satellite-based cloud radar (CloudSat) and Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) sensors to the National Aeronautics and Space Administration A-Train is timely in light of these needs but requires a new paradigm of model-evaluation tools that are capable of exploiting the vertical-profile information. Early results from the National Center for Atmospheric Research Model Evaluation Toolkit (MET), augmented to work with the emergent satellite-based active sensor observations, are presented here. Existing horizontal-plane statistical evaluation techniques have been adapted to operate on observations in the vertical plane and have been extended to 3D object evaluations, leveraging blended datasets from the active and passive A-Train sensors. Case studies of organized synoptic-scale and mesoscale distributed cloud systems are presented to illustrate the multiscale utility of the MET tools. Definition of objects on the basis of radar-reflectivity thresholds was found to be strongly dependent on the model’s ability to resolve details of the cloud’s internal hydrometeor distribution. Contoured-frequency-by-altitude diagrams provide a useful mechanism for evaluating the simulated and observed 3D distributions for regional domains. The expanded MET provides a new dimension to model evaluation and positions the community to better exploit active-sensor satellite observing systems that are slated for launch in the near future.

Full access
Barbara Brown, Tara Jensen, John Halley Gotway, Randy Bullock, Eric Gilleland, Tressa Fowler, Kathryn Newman, Dan Adriaansen, Lindsay Blank, Tatiana Burek, Michelle Harrold, Tracy Hertneky, Christina Kalb, Paul Kucera, Louisa Nance, John Opatz, Jonathan Vigh, and Jamie Wolff

Capsule summary

MET is a community-based package of state-of-the-art tools to evaluate predictions of weather, climate, and other phenomena, with capabilities to display and analyze verification results via the METplus system.

Full access