Search Results
You are looking at 1 - 10 of 67 items for
- Author or Editor: Adam J. Clark x
- Refine by Access: All Content x
Abstract
Methods for generating ensemble mean precipitation forecasts from convection-allowing model (CAM) ensembles based on a simple average of all members at each grid point can have limited utility because of amplitude reduction and overprediction of light precipitation areas caused by averaging complex spatial fields with strong gradients and high-amplitude features. To combat these issues with the simple ensemble mean, a method known as probability matching is commonly used to replace the ensemble mean amounts with amounts sampled from the distribution of ensemble member forecasts, which results in a field that has a bias approximately equal to the average bias of the ensemble members. Thus, the probability matched mean (PM mean hereafter) is viewed as a better representation of the ensemble members compared to the mean, and previous studies find that it is more skillful than any of the individual members. Herein, using nearly a year’s worth of data from a CAM-based ensemble running in real time at the National Severe Storms Laboratory, evidence is provided that the superior performance of the PM mean is at least partially an artifact of the spatial redistribution of precipitation amounts that occur when the PM mean is computed over a large domain. Specifically, the PM mean enlarges big areas of heavy precipitation and shrinks or even eliminates smaller ones. An alternative approach for the PM mean is developed that restricts the grid points used to those within a specified radius of influence. The new approach has an improved spatial representation of precipitation and is found to perform more skillfully than the PM mean at large scales when using neighborhood-based verification metrics.
Abstract
Methods for generating ensemble mean precipitation forecasts from convection-allowing model (CAM) ensembles based on a simple average of all members at each grid point can have limited utility because of amplitude reduction and overprediction of light precipitation areas caused by averaging complex spatial fields with strong gradients and high-amplitude features. To combat these issues with the simple ensemble mean, a method known as probability matching is commonly used to replace the ensemble mean amounts with amounts sampled from the distribution of ensemble member forecasts, which results in a field that has a bias approximately equal to the average bias of the ensemble members. Thus, the probability matched mean (PM mean hereafter) is viewed as a better representation of the ensemble members compared to the mean, and previous studies find that it is more skillful than any of the individual members. Herein, using nearly a year’s worth of data from a CAM-based ensemble running in real time at the National Severe Storms Laboratory, evidence is provided that the superior performance of the PM mean is at least partially an artifact of the spatial redistribution of precipitation amounts that occur when the PM mean is computed over a large domain. Specifically, the PM mean enlarges big areas of heavy precipitation and shrinks or even eliminates smaller ones. An alternative approach for the PM mean is developed that restricts the grid points used to those within a specified radius of influence. The new approach has an improved spatial representation of precipitation and is found to perform more skillfully than the PM mean at large scales when using neighborhood-based verification metrics.
Abstract
This study compares ensemble precipitation forecasts from 10-member, 3-km grid-spacing, CONUS domain single- and multicore ensembles that were a part of the 2016 Community Leveraged Unified Ensemble (CLUE) that was run for the 2016 NOAA Hazardous Weather Testbed Spring Forecasting Experiment. The main results are that a 10-member ARW ensemble was significantly more skillful than a 10-member NMMB ensemble, and a 10-member MIX ensemble (5 ARW and 5 NMMB members) performed about the same as the 10-member ARW ensemble. Skill was measured by area under the relative operating characteristic curve (AUC) and fractions skill score (FSS). Rank histograms in the ARW ensemble were flatter than the NMMB ensemble indicating that the envelope of ensemble members better encompassed observations (i.e., better reliability) in the ARW. Rank histograms in the MIX ensemble were similar to the ARW ensemble. In the context of NOAA’s plans for a Unified Forecast System featuring a CAM ensemble with a single core, the results are positive and indicate that it should be possible to develop a single-core system that performs as well as or better than the current operational CAM ensemble, which is known as the High-Resolution Ensemble Forecast System (HREF). However, as new modeling applications are developed and incremental changes that move HREF toward a single-core system are made possible, more thorough testing and evaluation should be conducted.
Abstract
This study compares ensemble precipitation forecasts from 10-member, 3-km grid-spacing, CONUS domain single- and multicore ensembles that were a part of the 2016 Community Leveraged Unified Ensemble (CLUE) that was run for the 2016 NOAA Hazardous Weather Testbed Spring Forecasting Experiment. The main results are that a 10-member ARW ensemble was significantly more skillful than a 10-member NMMB ensemble, and a 10-member MIX ensemble (5 ARW and 5 NMMB members) performed about the same as the 10-member ARW ensemble. Skill was measured by area under the relative operating characteristic curve (AUC) and fractions skill score (FSS). Rank histograms in the ARW ensemble were flatter than the NMMB ensemble indicating that the envelope of ensemble members better encompassed observations (i.e., better reliability) in the ARW. Rank histograms in the MIX ensemble were similar to the ARW ensemble. In the context of NOAA’s plans for a Unified Forecast System featuring a CAM ensemble with a single core, the results are positive and indicate that it should be possible to develop a single-core system that performs as well as or better than the current operational CAM ensemble, which is known as the High-Resolution Ensemble Forecast System (HREF). However, as new modeling applications are developed and incremental changes that move HREF toward a single-core system are made possible, more thorough testing and evaluation should be conducted.
Abstract
This study investigates probabilistic forecasts made using different convection-allowing ensemble configurations for a three-day period in June 2010 when numerous heavy-rain-producing mesoscale convective systems (MCSs) occurred in the United States. These MCSs developed both along a baroclinic zone in the Great Plains, and in association with a long-lived mesoscale convective vortex (MCV) in Texas and Arkansas. Four different ensemble configurations were developed using an ensemble-based data assimilation system. Two configurations used continuously cycled data assimilation, and two started the assimilation 24 h prior to the initialization of each forecast. Each configuration was run with both a single set of physical parameterizations and a mixture of physical parameterizations. These four ensemble forecasts were also compared with an ensemble run in real time by the Center for the Analysis and Prediction of Storms (CAPS). All five of these ensemble systems produced skillful probabilistic forecasts of the heavy-rain-producing MCSs, with the ensembles using mixed physics providing forecasts with greater skill and less overall bias compared to the single-physics ensembles. The forecasts using ensemble-based assimilation systems generally outperformed the real-time CAPS ensemble at lead times of 6–18 h, whereas the CAPS ensemble was the most skillful at forecast hours 24–30, though it also exhibited a wet bias. The differences between the ensemble precipitation forecasts were found to be related in part to differences in the analysis of the MCV and its environment, which in turn affected the evolution of errors in the forecasts of the MCSs. These results underscore the importance of representing model error in convection-allowing ensemble analysis and prediction systems.
Abstract
This study investigates probabilistic forecasts made using different convection-allowing ensemble configurations for a three-day period in June 2010 when numerous heavy-rain-producing mesoscale convective systems (MCSs) occurred in the United States. These MCSs developed both along a baroclinic zone in the Great Plains, and in association with a long-lived mesoscale convective vortex (MCV) in Texas and Arkansas. Four different ensemble configurations were developed using an ensemble-based data assimilation system. Two configurations used continuously cycled data assimilation, and two started the assimilation 24 h prior to the initialization of each forecast. Each configuration was run with both a single set of physical parameterizations and a mixture of physical parameterizations. These four ensemble forecasts were also compared with an ensemble run in real time by the Center for the Analysis and Prediction of Storms (CAPS). All five of these ensemble systems produced skillful probabilistic forecasts of the heavy-rain-producing MCSs, with the ensembles using mixed physics providing forecasts with greater skill and less overall bias compared to the single-physics ensembles. The forecasts using ensemble-based assimilation systems generally outperformed the real-time CAPS ensemble at lead times of 6–18 h, whereas the CAPS ensemble was the most skillful at forecast hours 24–30, though it also exhibited a wet bias. The differences between the ensemble precipitation forecasts were found to be related in part to differences in the analysis of the MCV and its environment, which in turn affected the evolution of errors in the forecasts of the MCSs. These results underscore the importance of representing model error in convection-allowing ensemble analysis and prediction systems.
Abstract
Severe weather probabilities are derived from the Warn-on-Forecast System (WoFS) run by NOAA’s National Severe Storms Laboratory (NSSL) during spring 2018 using the random forest (RF) machine learning algorithm. Recent work has shown this method generates skillful and reliable forecasts when applied to convection-allowing model ensembles for the “Day 1” time range (i.e., 12–36-h lead times), but it has been tested in only one other study for lead times relevant to WoFS (e.g., 0–6 h). Thus, in this paper, various sets of WoFS predictors, which include both environment and storm-based fields, are input into a RF algorithm and trained using the occurrence of severe weather reports within 39 km of a point to produce severe weather probabilities at 0–3-h lead times. We analyze the skill and reliability of these forecasts, sensitivity to different sets of predictors, and avenues for further improvements. The RF algorithm produced very skillful and reliable severe weather probabilities and significantly outperformed baseline probabilities calculated by finding the best performing updraft helicity (UH) threshold and smoothing parameter. Experiments where different sets of predictors were used to derive RF probabilities revealed 1) storm attribute fields contributed significantly more skill than environmental fields, 2) 2–5 km AGL UH and maximum updraft speed were the best performing storm attribute fields, 3) the most skillful ensemble summary metric was a smoothed mean, and 4) the most skillful forecasts were obtained when smoothed UH from individual ensemble members were used as predictors.
Abstract
Severe weather probabilities are derived from the Warn-on-Forecast System (WoFS) run by NOAA’s National Severe Storms Laboratory (NSSL) during spring 2018 using the random forest (RF) machine learning algorithm. Recent work has shown this method generates skillful and reliable forecasts when applied to convection-allowing model ensembles for the “Day 1” time range (i.e., 12–36-h lead times), but it has been tested in only one other study for lead times relevant to WoFS (e.g., 0–6 h). Thus, in this paper, various sets of WoFS predictors, which include both environment and storm-based fields, are input into a RF algorithm and trained using the occurrence of severe weather reports within 39 km of a point to produce severe weather probabilities at 0–3-h lead times. We analyze the skill and reliability of these forecasts, sensitivity to different sets of predictors, and avenues for further improvements. The RF algorithm produced very skillful and reliable severe weather probabilities and significantly outperformed baseline probabilities calculated by finding the best performing updraft helicity (UH) threshold and smoothing parameter. Experiments where different sets of predictors were used to derive RF probabilities revealed 1) storm attribute fields contributed significantly more skill than environmental fields, 2) 2–5 km AGL UH and maximum updraft speed were the best performing storm attribute fields, 3) the most skillful ensemble summary metric was a smoothed mean, and 4) the most skillful forecasts were obtained when smoothed UH from individual ensemble members were used as predictors.
Abstract
Using a composite procedure, North American Mesoscale Model (NAM) forecast and observed environments associated with zonally oriented, quasi-stationary surface fronts for 64 cases during July–August 2006–08 were examined for a large region encompassing the central United States. NAM adequately simulated the general synoptic features associated with the frontal environments (e.g., patterns in the low-level wind fields) as well as the positions of the fronts. However, kinematic fields important to frontogenesis such as horizontal deformation and convergence were overpredicted. Surface-based convective available potential energy (CAPE) and precipitable water were also overpredicted, which was likely related to the overprediction of the kinematic fields through convergence of water vapor flux. In addition, a spurious coherence between forecast deformation and precipitation was found using spatial correlation coefficients. Composite precipitation forecasts featured a broad area of rainfall stretched parallel to the composite front, whereas the composite observed precipitation covered a smaller area and had a WNW–ESE orientation relative to the front, consistent with mesoscale convective systems (MCSs) propagating at a slight right angle relative to the thermal gradient. Thus, deficiencies in the NAM precipitation forecasts may at least partially result from the inability to depict MCSs properly. It was observed that errors in the precipitation forecasts appeared to lag those of the kinematic fields, and so it seems likely that deficiencies in the precipitation forecasts are related to the overprediction of the kinematic fields such as deformation. However, no attempts were made to establish whether the overpredicted kinematic fields actually contributed to the errors in the precipitation forecasts or whether the overpredicted kinematic fields were simply an artifact of the precipitation errors. Regardless of the relationship between such errors, recognition of typical warm-season environments associated with these errors should be useful to operational forecasters.
Abstract
Using a composite procedure, North American Mesoscale Model (NAM) forecast and observed environments associated with zonally oriented, quasi-stationary surface fronts for 64 cases during July–August 2006–08 were examined for a large region encompassing the central United States. NAM adequately simulated the general synoptic features associated with the frontal environments (e.g., patterns in the low-level wind fields) as well as the positions of the fronts. However, kinematic fields important to frontogenesis such as horizontal deformation and convergence were overpredicted. Surface-based convective available potential energy (CAPE) and precipitable water were also overpredicted, which was likely related to the overprediction of the kinematic fields through convergence of water vapor flux. In addition, a spurious coherence between forecast deformation and precipitation was found using spatial correlation coefficients. Composite precipitation forecasts featured a broad area of rainfall stretched parallel to the composite front, whereas the composite observed precipitation covered a smaller area and had a WNW–ESE orientation relative to the front, consistent with mesoscale convective systems (MCSs) propagating at a slight right angle relative to the thermal gradient. Thus, deficiencies in the NAM precipitation forecasts may at least partially result from the inability to depict MCSs properly. It was observed that errors in the precipitation forecasts appeared to lag those of the kinematic fields, and so it seems likely that deficiencies in the precipitation forecasts are related to the overprediction of the kinematic fields such as deformation. However, no attempts were made to establish whether the overpredicted kinematic fields actually contributed to the errors in the precipitation forecasts or whether the overpredicted kinematic fields were simply an artifact of the precipitation errors. Regardless of the relationship between such errors, recognition of typical warm-season environments associated with these errors should be useful to operational forecasters.
Abstract
An experiment is described that is designed to examine the contributions of model, initial condition (IC), and lateral boundary condition (LBC) errors to the spread and skill of precipitation forecasts from two regional eight-member 15-km grid-spacing Weather Research and Forecasting (WRF) ensembles covering a 1575 km × 1800 km domain. It is widely recognized that a skillful ensemble [i.e., an ensemble with a probability distribution function (PDF) that generates forecast probabilities with high resolution and reliability] should account for both error sources. Previous work suggests that model errors make a larger contribution than IC and LBC errors to forecast uncertainty in the short range before synoptic-scale error growth becomes nonlinear. However, in a regional model with unperturbed LBCs, the infiltration of the lateral boundaries will negate increasing spread. To obtain a better understanding of the contributions to the forecast errors in precipitation and to examine the window of forecast lead time before unperturbed ICs and LBCs begin to cause degradation in ensemble forecast skill, the “perfect model” assumption is made in an ensemble that uses perturbed ICs and LBCs (PILB ensemble), and the “perfect analysis” assumption is made in another ensemble that uses mixed physics–dynamic cores (MP ensemble), thus isolating the error contributions. For the domain and time period used in this study, unperturbed ICs and LBCs in the MP ensemble begin to negate increasing spread around forecast hour 24, and ensemble forecast skill as measured by relative operating characteristic curves (ROC scores) becomes lower in the MP ensemble than in the PILB ensemble, with statistical significance beginning after forecast hour 69. However, degradation in forecast skill in the MP ensemble relative to the PILB ensemble is not observed in an analysis of deterministic forecasts calculated from each ensemble using the probability matching method. Both ensembles were found to lack statistical consistency (i.e., to be underdispersive), with the PILB ensemble (MP ensemble) exhibiting more (less) statistical consistency with respect to forecast lead time. Spread ratios in the PILB ensemble are greater than those in the MP ensemble at all forecast lead times and thresholds examined; however, ensemble variance in the MP ensemble is greater than that in the PILB ensemble during the first 24 h of the forecast. This discrepancy in spread measures likely results from greater bias in the MP ensemble leading to an increase in ensemble variance and decrease in spread ratio relative to the PILB ensemble.
Abstract
An experiment is described that is designed to examine the contributions of model, initial condition (IC), and lateral boundary condition (LBC) errors to the spread and skill of precipitation forecasts from two regional eight-member 15-km grid-spacing Weather Research and Forecasting (WRF) ensembles covering a 1575 km × 1800 km domain. It is widely recognized that a skillful ensemble [i.e., an ensemble with a probability distribution function (PDF) that generates forecast probabilities with high resolution and reliability] should account for both error sources. Previous work suggests that model errors make a larger contribution than IC and LBC errors to forecast uncertainty in the short range before synoptic-scale error growth becomes nonlinear. However, in a regional model with unperturbed LBCs, the infiltration of the lateral boundaries will negate increasing spread. To obtain a better understanding of the contributions to the forecast errors in precipitation and to examine the window of forecast lead time before unperturbed ICs and LBCs begin to cause degradation in ensemble forecast skill, the “perfect model” assumption is made in an ensemble that uses perturbed ICs and LBCs (PILB ensemble), and the “perfect analysis” assumption is made in another ensemble that uses mixed physics–dynamic cores (MP ensemble), thus isolating the error contributions. For the domain and time period used in this study, unperturbed ICs and LBCs in the MP ensemble begin to negate increasing spread around forecast hour 24, and ensemble forecast skill as measured by relative operating characteristic curves (ROC scores) becomes lower in the MP ensemble than in the PILB ensemble, with statistical significance beginning after forecast hour 69. However, degradation in forecast skill in the MP ensemble relative to the PILB ensemble is not observed in an analysis of deterministic forecasts calculated from each ensemble using the probability matching method. Both ensembles were found to lack statistical consistency (i.e., to be underdispersive), with the PILB ensemble (MP ensemble) exhibiting more (less) statistical consistency with respect to forecast lead time. Spread ratios in the PILB ensemble are greater than those in the MP ensemble at all forecast lead times and thresholds examined; however, ensemble variance in the MP ensemble is greater than that in the PILB ensemble during the first 24 h of the forecast. This discrepancy in spread measures likely results from greater bias in the MP ensemble leading to an increase in ensemble variance and decrease in spread ratio relative to the PILB ensemble.
Abstract
The diurnal cycles of rainfall in 5-km grid-spacing convection-resolving and 22-km grid-spacing non-convection-resolving configurations of the Weather Research and Forecasting (WRF) model are compared to see if significant improvements can be obtained by using fine enough grid spacing to explicitly resolve convection. Diurnally averaged Hovmöller diagrams, spatial correlation coefficients computed in Hovmöller space, equitable threat scores (ETSs), and biases for forecasts conducted from 1 April to 25 July 2005 over a large portion of the central United States are used for the comparisons. A subjective comparison using Hovmöller diagrams of diurnally averaged rainfall show that the diurnal cycle representation in the 5-km configuration is clearly superior to that in the 22-km configuration during forecast hours 24–48. The superiority of the 5-km configuration is validated by much higher spatial correlation coefficients than in the 22-km configuration. During the first 24 forecast hours the 5-km model forecasts appear to be more adversely affected by model “spinup” processes than the 22-km model forecasts, and it is less clear, subjectively, which configuration has the better diurnal cycle representation, although spatial correlation coefficients are slightly higher in the 22-km configuration. ETSs in both configurations have diurnal oscillations with relative maxima occurring in both configurations at forecast hours corresponding to 0000–0300 LST, while biases also have diurnal oscillations with relative maxima (largest errors) in the 22-km (5-km) configuration occurring at forecast hours corresponding to 1200 (1800) LST. At all forecast hours, ETSs from the 22-km configuration are higher than those in the 5-km configuration. This inconsistency with some of the results obtained using the aforementioned spatial correlation coefficients reinforces discussion in past literature that cautions against using “traditional” verification statistics, such as ETS, to compare high- to low-resolution forecasts.
Abstract
The diurnal cycles of rainfall in 5-km grid-spacing convection-resolving and 22-km grid-spacing non-convection-resolving configurations of the Weather Research and Forecasting (WRF) model are compared to see if significant improvements can be obtained by using fine enough grid spacing to explicitly resolve convection. Diurnally averaged Hovmöller diagrams, spatial correlation coefficients computed in Hovmöller space, equitable threat scores (ETSs), and biases for forecasts conducted from 1 April to 25 July 2005 over a large portion of the central United States are used for the comparisons. A subjective comparison using Hovmöller diagrams of diurnally averaged rainfall show that the diurnal cycle representation in the 5-km configuration is clearly superior to that in the 22-km configuration during forecast hours 24–48. The superiority of the 5-km configuration is validated by much higher spatial correlation coefficients than in the 22-km configuration. During the first 24 forecast hours the 5-km model forecasts appear to be more adversely affected by model “spinup” processes than the 22-km model forecasts, and it is less clear, subjectively, which configuration has the better diurnal cycle representation, although spatial correlation coefficients are slightly higher in the 22-km configuration. ETSs in both configurations have diurnal oscillations with relative maxima occurring in both configurations at forecast hours corresponding to 0000–0300 LST, while biases also have diurnal oscillations with relative maxima (largest errors) in the 22-km (5-km) configuration occurring at forecast hours corresponding to 1200 (1800) LST. At all forecast hours, ETSs from the 22-km configuration are higher than those in the 5-km configuration. This inconsistency with some of the results obtained using the aforementioned spatial correlation coefficients reinforces discussion in past literature that cautions against using “traditional” verification statistics, such as ETS, to compare high- to low-resolution forecasts.
Abstract
From 9 to 11 June 2010, a mesoscale convective vortex (MCV) was associated with several periods of heavy rainfall that led to flash flooding. During the overnight hours, mesoscale convective systems (MCSs) developed that moved slowly and produced heavy rainfall over small areas in south-central Texas on 9 June, north Texas on 10 June, and western Arkansas on 11 June. In this study, forecasts of this event from the Center for the Analysis and Prediction of Storms' Storm-Scale Ensemble Forecast system are examined. This ensemble, with 26 members at 4-km horizontal grid spacing, included a few members that very accurately predicted the development, maintenance, and evolution of the heavy-rain-producing MCSs, along with a majority of members that had substantial errors in their precipitation forecasts. The processes favorable for the initiation, organization, and maintenance of these heavy-rain-producing MCSs are diagnosed by comparing ensemble members with accurate and inaccurate forecasts. Even within a synoptic environment known to be conducive to extreme local rainfall, there was considerable spread in the ensemble's rainfall predictions. Because all ensemble members included an anomalously moist environment, the precipitation predictions were insensitive to the atmospheric moisture. However, the development of heavy precipitation overnight was very sensitive to the intensity and evolution of convection the previous day. Convective influences on the strength of the MCV and its associated dome of cold air at low levels determined whether subsequent deep convection was initiated and maintained. In all, this ensemble provides quantitative and qualitative information about the mesoscale processes that are most favorable (or unfavorable) for localized extreme rainfall.
Abstract
From 9 to 11 June 2010, a mesoscale convective vortex (MCV) was associated with several periods of heavy rainfall that led to flash flooding. During the overnight hours, mesoscale convective systems (MCSs) developed that moved slowly and produced heavy rainfall over small areas in south-central Texas on 9 June, north Texas on 10 June, and western Arkansas on 11 June. In this study, forecasts of this event from the Center for the Analysis and Prediction of Storms' Storm-Scale Ensemble Forecast system are examined. This ensemble, with 26 members at 4-km horizontal grid spacing, included a few members that very accurately predicted the development, maintenance, and evolution of the heavy-rain-producing MCSs, along with a majority of members that had substantial errors in their precipitation forecasts. The processes favorable for the initiation, organization, and maintenance of these heavy-rain-producing MCSs are diagnosed by comparing ensemble members with accurate and inaccurate forecasts. Even within a synoptic environment known to be conducive to extreme local rainfall, there was considerable spread in the ensemble's rainfall predictions. Because all ensemble members included an anomalously moist environment, the precipitation predictions were insensitive to the atmospheric moisture. However, the development of heavy precipitation overnight was very sensitive to the intensity and evolution of convection the previous day. Convective influences on the strength of the MCV and its associated dome of cold air at low levels determined whether subsequent deep convection was initiated and maintained. In all, this ensemble provides quantitative and qualitative information about the mesoscale processes that are most favorable (or unfavorable) for localized extreme rainfall.
Abstract
Given increasing computing power, an important question is whether additional computational resources would be better spent reducing the horizontal grid spacing of a convection-allowing model (CAM) or adding members to form CAM ensembles. The present study investigates this question as it applies to CAM-derived next-day probabilistic severe weather forecasts created by using forecast updraft helicity as a severe weather proxy for 63 days of the 2010 and 2011 NOAA Hazardous Weather Testbed Spring Forecasting Experiments. Forecasts derived from three sets of Weather Research and Forecasting Model configurations are tested: a 1-km deterministic model, a 4-km deterministic model, and an 11-member, 4-km ensemble. Forecast quality is evaluated using relative operating characteristic (ROC) curves, attributes diagrams, and performance diagrams, and forecasts from five representative cases are analyzed to investigate their relative quality and value in a variety of situations. While no statistically significant differences exist between the 4- and 1-km deterministic forecasts in terms of area under ROC curves, the 4-km ensemble forecasts offer weakly significant improvements over the 4-km deterministic forecasts over the entire 63-day dataset. Further, the 4-km ensemble forecasts generally provide greater forecast quality relative to either of the deterministic forecasts on an individual day. Collectively, these results suggest that, for purposes of improving next-day CAM-derived probabilistic severe weather forecasts, additional computing resources may be better spent on adding members to form CAM ensembles than on reducing the horizontal grid spacing of a deterministic model below 4 km.
Abstract
Given increasing computing power, an important question is whether additional computational resources would be better spent reducing the horizontal grid spacing of a convection-allowing model (CAM) or adding members to form CAM ensembles. The present study investigates this question as it applies to CAM-derived next-day probabilistic severe weather forecasts created by using forecast updraft helicity as a severe weather proxy for 63 days of the 2010 and 2011 NOAA Hazardous Weather Testbed Spring Forecasting Experiments. Forecasts derived from three sets of Weather Research and Forecasting Model configurations are tested: a 1-km deterministic model, a 4-km deterministic model, and an 11-member, 4-km ensemble. Forecast quality is evaluated using relative operating characteristic (ROC) curves, attributes diagrams, and performance diagrams, and forecasts from five representative cases are analyzed to investigate their relative quality and value in a variety of situations. While no statistically significant differences exist between the 4- and 1-km deterministic forecasts in terms of area under ROC curves, the 4-km ensemble forecasts offer weakly significant improvements over the 4-km deterministic forecasts over the entire 63-day dataset. Further, the 4-km ensemble forecasts generally provide greater forecast quality relative to either of the deterministic forecasts on an individual day. Collectively, these results suggest that, for purposes of improving next-day CAM-derived probabilistic severe weather forecasts, additional computing resources may be better spent on adding members to form CAM ensembles than on reducing the horizontal grid spacing of a deterministic model below 4 km.
Abstract
During the 2007 NOAA Hazardous Weather Testbed Spring Experiment, a 10-member 4-km grid-spacing Storm-Scale Ensemble Forecast (SSEF) system was run in real time to provide experimental severe weather forecasting guidance. Five SSEF system members used perturbed initial and lateral boundary conditions (ICs and LBCs) and mixed physics (ENS4), and five members used only mixed physics (ENS4phys). This ensemble configuration facilitates a comparison of ensemble spread generated by a combination of perturbed ICs/LBCs and mixed physics to that generated by only mixed physics, which is examined herein. In addition, spread growth and spread-error metrics for the two SSEF system configurations are compared to similarly configured 20-km grid-spacing convection-parameterizing ensembles (ENS20 and ENS20phys). Twelve forecast fields are examined for 20 cases.
For most fields, ENS4 mean spread growth rates are higher than ENS20 for ensemble configurations with both sets of perturbations, which is expected as smaller scales of motion are resolved at higher resolution. However, when ensembles with only mixed physics are compared, mass-related fields (i.e., geopotential height and mean sea level pressure) in ENS20phys have slightly higher spread growth rates than ENS4phys, likely resulting from the additional physics uncertainty in ENS20phys from varied cumulus parameterizations that were not used at 4-km grid spacing. For 4- and 20-km configurations, the proportion of spread generated by mixed physics in ENS4 and ENS20 increased with increasing forecast lead time. In addition, low-level fields (e.g., 2-m temperature) had a higher proportion of spread generated by mixed physics than mass-related fields. Spread-error analyses revealed that ensemble variance from the current uncalibrated ensemble systems was not a reliable indicator of forecast uncertainty. Furthermore, ENS4 had better statistical consistency than ENS20 for some mass-related fields, wind-related fields, precipitation, and most unstable convective available potential energy (MUCAPE) with no noticeable differences for low-level temperature and dewpoint fields. The variety of results obtained for the different types of fields examined suggests that future ensemble design should give careful consideration to the specific types of forecasts desired by the user.
Abstract
During the 2007 NOAA Hazardous Weather Testbed Spring Experiment, a 10-member 4-km grid-spacing Storm-Scale Ensemble Forecast (SSEF) system was run in real time to provide experimental severe weather forecasting guidance. Five SSEF system members used perturbed initial and lateral boundary conditions (ICs and LBCs) and mixed physics (ENS4), and five members used only mixed physics (ENS4phys). This ensemble configuration facilitates a comparison of ensemble spread generated by a combination of perturbed ICs/LBCs and mixed physics to that generated by only mixed physics, which is examined herein. In addition, spread growth and spread-error metrics for the two SSEF system configurations are compared to similarly configured 20-km grid-spacing convection-parameterizing ensembles (ENS20 and ENS20phys). Twelve forecast fields are examined for 20 cases.
For most fields, ENS4 mean spread growth rates are higher than ENS20 for ensemble configurations with both sets of perturbations, which is expected as smaller scales of motion are resolved at higher resolution. However, when ensembles with only mixed physics are compared, mass-related fields (i.e., geopotential height and mean sea level pressure) in ENS20phys have slightly higher spread growth rates than ENS4phys, likely resulting from the additional physics uncertainty in ENS20phys from varied cumulus parameterizations that were not used at 4-km grid spacing. For 4- and 20-km configurations, the proportion of spread generated by mixed physics in ENS4 and ENS20 increased with increasing forecast lead time. In addition, low-level fields (e.g., 2-m temperature) had a higher proportion of spread generated by mixed physics than mass-related fields. Spread-error analyses revealed that ensemble variance from the current uncalibrated ensemble systems was not a reliable indicator of forecast uncertainty. Furthermore, ENS4 had better statistical consistency than ENS20 for some mass-related fields, wind-related fields, precipitation, and most unstable convective available potential energy (MUCAPE) with no noticeable differences for low-level temperature and dewpoint fields. The variety of results obtained for the different types of fields examined suggests that future ensemble design should give careful consideration to the specific types of forecasts desired by the user.
