1. Introduction
Advances in evaluating and quantifying the tornado threat have recently been made. These advances come from a better understanding of relationships between near-storm regional-scale environmental conditions and the resulting mode of convection [see Smith et al. (2012) and Thompson et al. (2012) for a review of the literature on this topic], and from careful statistical analysis of relationships between radar-based rotational signals at the storm scale and the probability of specific damage rating categories (Smith et al. 2015; Thompson et al. 2017). Cohen et al. (2018) investigated multivariate models as a way to combine environmental and storm-scale factors influencing enhanced Fujita scale (EF) rating probabilities. While these approaches improve the statistical foundation of tornado threat research, there is room for additional improvement. The purpose of the present study is to introduce a statistical model to estimate a per-tornado damage rating (and associated uncertainties) directly and to demonstrate features of the model by using it to estimate damage ratings with environmental factors on days with many tornadoes. The aim differs from earlier studies in that the sole focus is on method, as opposed to operational application. The goal here is to make researchers aware of modern statistical technology that can be leveraged to help them to improve prediction of severe weather. The objective here is to demonstrate the cumulative logistic model for estimating damage rating probabilities.
From a mathematics perspective, the approach that we take is similar to that outlined by Cohen et al. (2018), who fit a linear regression to wind speeds corresponding to midpoints of EF rating intervals and used a logistic regression model to analyze the probability of a tornado occurrence. Our approach differs in that we fit a cumulative logistic regression model to the EF rating distribution directly. From a statistics standpoint, the approach we take is similar to the approach used by Thompson et al. (2017), who estimated conditional empirical probabilities of EF ratings by binning various indicators from WSR-88D. Our approach differs in that we use a multivariate model and we include estimates of uncertainty on the predicted output. In short, our approach is unique in that we use damage ratings directly as ordered categorical outcomes and we provide estimates of uncertainty on the estimated probabilities.
While our focus in this paper is solely methodological, the application might have some operational relevance. This is because environmental ingredients needed to produce an outbreak of severe convective weather are well known and can be leveraged to make predictions. Considerable skill exists in outlining areas under greatest risk of severe weather on a given day Hitchens and Brooks (2014). Outbreaks have large variation in terms of tornado frequency and intensity with much of this variability resulting from the convective mode (Smith et al. 2012; Thompson et al. 2012; Smith et al. 2015). Given a forecast of a severe weather outbreak will conditions favor many violent tornadoes? Dynamical convective-allowing models provide forecast guidance through products like updraft helicity swaths, and such models can anticipate the convective mode to some degree. But statistical models trained on thousands of tornadoes occurring across dozens of outbreaks can provide a baseline climatology for this risk. The U.S. Storm Prediction Center (SPC) currently uses long-run frequency of two or more tornadoes and long-run frequency of at least one strong (EF2–EF5) tornado as climatology.
Our paper is similar to a recent study in the above sense in that it employs a model to estimate the probability of at least one significant (EF2+) tornado on days with at least one tornado-warned supercell (Togstad et al. 2011). It differs in two key ways, however. First, in demonstrating the approach, we condition our model on the occurrence of a tornado “outbreak” (at least 10 tornadoes) rather than on the occurrence of a tornado warning. Second, we use cumulative frequency distribution by EF rating as the outcome variable rather than relative frequency of at least one EF2 + tornado. The paper is outlined as follows. The mathematics of cumulative logistic regression are given in section 2. The data used to demonstrate the model are described in section 3. Model results are presented in section 4, and a summary is given in section 5.
2. Cumulative logistic regression
The objective of this paper is to introduce cumulative logistic regression model as a way to estimate damage rating probabilities directly and to demonstrate its features by using it to estimate damage rating probabilities from large-scale environmental variables. We begin with a description of the model in the context of estimating damage ratings from environmental variables.
A rating EFi is assigned to tornado Ti using an ordered distribution, which is a categorical distribution that takes a vector of probabilities (p = {p0, p1, p2, p3, p4}), one for each EF rating below EF5. Each probability value pk in the vector is defined by its link to the intercept parameter value αk. To include a predictor variable in the model we define the log-cumulative odds as the sum of αk and a linear model term (βjxij), where xij is the value of a population-level or a group-level variable j (e.g., distance to nearest city/town for a population-level variable and month for a group-level variable) associated with tornado Ti and βj is the coefficient (or coefficient vector for group-level variables) associated with that variable. We determine the αks and βjs using Bayesian inference, so the model includes prior distributions on these parameters. We put a flat normal distribution prior on the αks and a flat Student’s t distribution prior on the βjs.
The model gives the correct ordering of the EF ratings while allowing for changes in the likelihood for each tornado based on associated environmental conditions and other factors. The negative sign ensures that as the log-cumulative odds of every EF rating below the highest decreases, the probability mass shifts upward toward higher EF ratings (McElreath 2015).
3. Data
We illustrate the utility of cumulative logistic regression for estimating EF rating categories by fitting the model to a set of data. The data consist of the outcome variable [highest (maximum) per-tornado EF rating], predictor variables (environmental factors and distance to nearest city/town), and grouping variables (month and cluster number). Data are filtered to include only tornadoes occurring on days with at least 10 tornadoes over the period 1994–2017 within the contiguous United States. Ten is a compromise between having too few cases leading to large uncertainty on the statistical model estimates and too many leading to slow convergence to the posterior distribution. Here we describe the procedure that we used to organize the data and provide summary statistics.
First we extract the date, time, genesis location, and maximum EF rating from the tornado record obtained from the SPC. Each row in the record contains information about an individual tornado. The start year of 1994 marks the beginning of extensive use of WSR-88D. There are 29 372 tornadoes over this period of record. We convert the geographic coordinates of the genesis locations to a Lambert conformal conic projection centered on 107° W longitude.
Next we assign a cluster number to each tornado based on space–time differences between genesis locations. If two tornadoes occur close together in space and time (e.g., 1 km and 1 h), they are assigned the same cluster number (see Fig. 1 for an example of a tornado cluster). Clustering stops when the difference between individual tornadoes and an existing cluster exceeds 50 000 s (~14 h). The differences have units of time because we divide the spatial distance by 15 m s−1. Details of the procedure along with a comparison with a subjective grouping are provided in Schroder and Elsner (2019). Last, we filter the tornadoes to include only those occurring as part of clusters with at least 10 tornadoes within a single convective day (from 1200 to 1200 UTC). This filtering results in 16 501 tornadoes in 742 clusters, with the majority of the clusters occurring during April, May, and June (Fig. 2).
Tornado locations (origin) during one tornado cluster used in this study.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Tornado locations (origin) during one tornado cluster used in this study.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Tornado locations (origin) during one tornado cluster used in this study.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Monthly frequency of tornado clusters (convective days with at least 10 tornadoes), 1994–2017. The origin of the circle is 0, and the outermost ring corresponds to 200 tornadoes.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Monthly frequency of tornado clusters (convective days with at least 10 tornadoes), 1994–2017. The origin of the circle is 0, and the outermost ring corresponds to 200 tornadoes.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Monthly frequency of tornado clusters (convective days with at least 10 tornadoes), 1994–2017. The origin of the circle is 0, and the outermost ring corresponds to 200 tornadoes.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Next we extract environmental variables from the National Centers for Environmental Prediction North American Regional Reanalysis (NARR), obtained from the National Center for Atmospheric Research. Variables are available on a 32.4-km grid and are a blend of modeled and observed data. We use the files that contain environmental data for each day, ranging from 1200 UTC to 1200 UTC in 3-h increments. Variables considered include the CAPE and CIN from 180 to 0 hPa above ground level (AGL) (layer 375, 376), the storm-relative helicity from 0 to 3000 m AGL (layer 323), and the u and υ components of storm motion from 0 to 6000 m AGL (layer 324, 325). In addition, we compute total storm motion as the square root of the sum of the velocity components squared and bulk shear as the square root of the sum of the squared differences between the u and υ winds for the 1000- and 500-hPa levels. We consider these variables because they are well known to be associated with tornado activity (Gensini and Ashley 2011; Cheng et al. 2016). For each tornado cluster, we find the closest 3-h time before the appearance of the first tornado in the cluster and use the environmental variables from that time. We pick a time before the event starts to have a sample of conditions prior to the appearance of any tornado. We join the environmental variables at the cluster level with the data at the tornado level.
For each tornado, we compute the distance between the genesis location and the nearest city/town (Elsner et al. 2013; Widen et al. 2013). The city/town location is based on the geographic center. Population values are based on the 2010 U.S. Census data [obtained from Steiner (2019) and accessed through the “USAboundaries” package (Mullen and Bratt 2018) in the R software package] and range from a few hundred people to more the eight million people. The distance between a tornado and the nearest city/town serves as a proxy for the potential number of damage targets. All else being equal, a tornado occurring within a city or town will have a greater opportunity to impact a damage target, on average, than one that occurs in a rural area. With the distance to nearest city/town as predictor variable, we are able to quantify how the potential number of damage targets shifts the distributions, for example, from EF0 to EF1 and from EF1 to EF2. We know that the chance of getting an EF4 + tornado in the dataset increases with the number of targets, but we do not know by how much relative to an EF3. Table 1 lists the predictor variables along with the associated extremes and average values. Averages are computed over all tornadoes. We remove tornadoes occurring during the 30 May 2003 cluster since the maximum helicity value for this cluster had an erroneously high value as determined by examining the coefficient on this term when the cluster was included in the model.
Variables used in the model to estimate damage ratings. The values are based on 16 483 tornadoes in 741 clusters.
4. Results
We begin with a histogram of maximum EF rating per tornado (highest rating given to the tornado as recorded in the SPC data). As expected, the histogram (Fig. 3) shows that the vast majority of tornadoes that occur as part of a big cluster (10+ tornadoes) are rated EF0 or EF1, with far fewer rated EF4 or EF5. Relative to all tornadoes, however, the distribution of tornadoes in big clusters favors higher ratings. For example, 3.5% of tornadoes occurring in big clusters are rated EF3 as compared with 2.3% of all tornadoes; 0.08% of tornadoes occurring in big clusters are rated EF5 as compared with 0.05% of all tornadoes.
Histogram of tornadoes by maximum EF rating. Only tornadoes occurring in big clusters are considered (see text).
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Histogram of tornadoes by maximum EF rating. Only tornadoes occurring in big clusters are considered (see text).
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Histogram of tornadoes by maximum EF rating. Only tornadoes occurring in big clusters are considered (see text).
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Next, we describe this histogram on the log-cumulative-odds scale by constructing the odds of a cumulative probability and then taking logarithms. The logit function is the logarithm of the odds (log odds), so the cumulative logit is the logarithm of the cumulative odds. Both the logit and the cumulative logit constrain the probabilities to lie in the interval between 0 and 1. Predictor variables are added on the cumulative logit scale [Eq. (2)]. The link function takes care of converting the parameter estimates on these variables to the proper probability scale (McElreath 2015). We compute the cumulative probabilities from the histogram, which are the discrete proportions of tornadoes by each EF rating. We then compute the series of intercept parameters to redescribe the histogram in terms of log-cumulative odds [Eq. (1)]. Each intercept is on the log-cumulative-odds scale and stands in for the cumulative probability associated with each EF rating (Fig. 4). The discrete probability for each EF rating Pr(Ti = k) is the successive difference between the elements of the vector of cumulative probabilities. These probabilities are the likelihoods that are conditioned on the values of the predictor variables and combined with the priors to complete the model [Eq. (2)].
(a) Cumulative proportion and (b) log-cumulative odds of a tornado by maximum EF rating for all tornadoes (gray) and for tornadoes occurring in big clusters (black). Note that the cumulative logit for the EF5 rating is infinity.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
(a) Cumulative proportion and (b) log-cumulative odds of a tornado by maximum EF rating for all tornadoes (gray) and for tornadoes occurring in big clusters (black). Note that the cumulative logit for the EF5 rating is infinity.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
(a) Cumulative proportion and (b) log-cumulative odds of a tornado by maximum EF rating for all tornadoes (gray) and for tornadoes occurring in big clusters (black). Note that the cumulative logit for the EF5 rating is infinity.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Posterior distributions on the model parameters are obtained using the Stan computational engine (Carpenter et al. 2017) accessed through the “brms” package (Bürkner 2017). Mildly informative conservative priors are specified to improve convergence of the sampler and to guard against overfitting. To improve the efficiency of the sampler, predictor variables are scaled by subtracting their respective means and dividing by their respective standard deviations. The environmental variables and year are included as population-level effects (fixed effects). The month of the cluster and the unique cluster identification number are included as group-level effects (random effects).
The model reproduces the distribution of tornadoes by EF rating category as expected (Table 2). It slightly underestimates the proportion of EF0 tornadoes and slightly overestimates the number of EF1 tornadoes and EF3 tornadoes, but overall the proportions from the model match the data very well. Signs on the fixed-effect coefficients (Table 3) are consistent with expectations based on physical reasoning derived from the current understanding of how environmental factors influence tornado activity (Smith et al. 2012; Thompson et al. 2017). The coefficient on cluster year is positive indicating a trend toward higher rated tornadoes as discussed in Elsner et al. (2019). The coefficient on the distance-to-nearest-city/town term is negative as expected. The closer a tornado occurs to a city/town, the greater the chance it will get rated at the next higher EF rating relative to the same tornado occurring in a rural area. The largest effect occurs with bulk shear. The sign on the coefficient indicates that greater shear results in a better chance of a higher EF rating, as we would expect from physical reasoning.
Observed and estimated proportions of tornadoes by EF damage rating. The estimated proportions are from a cumulative logistic regression model.
Estimated coefficients on the population-level effects. Abbreviations in the subscripts refer to the variables listed in Table 1; UI is the uncertainty/confidence interval.
Coefficients on the fixed effects and on the month random effect are plotted in Fig. 5. Magnitude of the departure from zero indicates the importance of the variable to the model for estimating damage ratings as discussed above. The monthly variation in the distribution of tornadoes by EF rating is an important model component, with May and June having a significantly lower proportion of most damaging tornadoes after accounting for the fixed effects. January and November have a larger than average proportion of most damaging tornadoes.
Posterior median (circles) and highest posterior probability intervals [66% (thick lines) and 95% (thin lines)] for (a) the fixed effects and (b) the random effect of month.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Posterior median (circles) and highest posterior probability intervals [66% (thick lines) and 95% (thin lines)] for (a) the fixed effects and (b) the random effect of month.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Posterior median (circles) and highest posterior probability intervals [66% (thick lines) and 95% (thin lines)] for (a) the fixed effects and (b) the random effect of month.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
To get an idea how much a particular variable statistically influences the distribution of EF ratings while holding the other variables constant, we examine marginal effects (Fig. 6). A variable’s marginal effect is computed by holding the other variables at their respective mean values. Consider the marginal effect of bulk shear. For tornadoes occurring in environments of low shear (less than 10 m s−1) the model estimates the probability that a tornado gets rated EF0 at nearly 75%. This probability drops to 40% for tornadoes occurring in environments of high shear (greater than 40 m s−1). There are compensating increases in the chance of EF1 and higher ratings across the range of bulk shear values. Further, we see that CAPE and helicity have less of an effect on the probability distribution of EF ratings compared with bulk shear (posterior means on the respective coefficients are farther from the zero line). We also quantify the trend toward higher EF ratings and the relative changes over time depending on where the tornado occurs (near a city/town or outside a city/town; Fig. 7).
Marginal effects of the environmental variables on the distribution of EF rating for (a) bulk shear, (b) CAPE, (c) distance to nearest city/town, and (d) storm-relative helicity.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Marginal effects of the environmental variables on the distribution of EF rating for (a) bulk shear, (b) CAPE, (c) distance to nearest city/town, and (d) storm-relative helicity.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Marginal effects of the environmental variables on the distribution of EF rating for (a) bulk shear, (b) CAPE, (c) distance to nearest city/town, and (d) storm-relative helicity.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Marginal trends in the distribution of EF rating (EF0, EF1, and EF2). Trends are estimated by setting the distance to 135 km for remote areas and 0 km for inside a city or town.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Marginal trends in the distribution of EF rating (EF0, EF1, and EF2). Trends are estimated by setting the distance to 135 km for remote areas and 0 km for inside a city or town.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Marginal trends in the distribution of EF rating (EF0, EF1, and EF2). Trends are estimated by setting the distance to 135 km for remote areas and 0 km for inside a city or town.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
An important point is that we can use the model to get an estimate of the probability distributions for any particular set of predictor values. Since the model uses Bayesian inference, we get posterior predictive samples of the EF probability distribution for any set of values. As an example, we show the posterior predictive samples across a range of bulk shear values setting the variables to their respective averages except distance-to-nearest-city/town, which we set to zero (Fig. 8). Bulk shear is illustrated because it has the largest influence on the outcome (distribution of EF ratings) as noted above. Individual samples (100 of them) of the cumulative proportion of tornadoes for different EF ratings are shown.
Posterior predictions over a range of bulk shear values: (a) cumulative proportion by EF rating for 100 random samples, and (b) probability of EF-level damage by EF rating. The white line indicates the posterior average, and the band indicates the interquartile range over the samples. Values for other variables are set to their respective averages except for distance-to-nearest-city/town, the value of which is set to zero.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Posterior predictions over a range of bulk shear values: (a) cumulative proportion by EF rating for 100 random samples, and (b) probability of EF-level damage by EF rating. The white line indicates the posterior average, and the band indicates the interquartile range over the samples. Values for other variables are set to their respective averages except for distance-to-nearest-city/town, the value of which is set to zero.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
Posterior predictions over a range of bulk shear values: (a) cumulative proportion by EF rating for 100 random samples, and (b) probability of EF-level damage by EF rating. The white line indicates the posterior average, and the band indicates the interquartile range over the samples. Values for other variables are set to their respective averages except for distance-to-nearest-city/town, the value of which is set to zero.
Citation: Journal of Applied Meteorology and Climatology 58, 12; 10.1175/JAMC-D-19-0178.1
When bulk shear is 10 m s−1 the posterior mean relative percentage of an EF0 tornado is 65% [interquartile range (IQR) = (56%, 79%)], but when bulk shear is 40 m s−1 the posterior mean relative percentage of an EF0 tornado drops to 38% [IQR = (25%, 50%)]. This decrease in percent is compensated by increases in the relative percentage of tornadoes rated higher. For example, when bulk shear is 10 m s−1 the posterior mean relative percentage of an EF3 tornado is 2.0% [IQR = (0.9%, 2.5%)] but when bulk shear is 40 m s−1 the posterior mean relative percentage of an EF3 tornado rises to 6.1% [IQR = (2.9%, 8.2%)]. This quantification of the effect of bulk shear on EF ratings is possible with a cumulative logistic regression model.
We can use the model in a similar way to quantify a well-known (but not well quantified) EF rating bias. We find that, under average environmental condition (see Table 1), when a tornado occurs near the center of a city or town, it will get rated EF0 47% [IQR = (31%, 59%)] of the time. This compares with 56% [IQR = (41%, 70%)] of the time when the same tornado occurs 50 km from the center. This increase in the percentage of EF0 tornadoes going from city to rural areas is compensated by corresponding decreases in percentages of tornadoes getting rated higher. For example, the chance that a tornado gets rated as EF3 or higher is 5% in the city compared with 3.6% at a distance of 50 km from the city/town and only 2.4% at a distance of 100 km from the city/town. This quantification of an EF rating bias is possible with a cumulative logistic model. By including an interaction between year and distance to nearest city/town in the model we determine that this bias is not diminishing over time. This differs from the decreasing population bias on the tornado reports as documented and quantified elsewhere (Elsner et al. 2013; Jagger et al. 2015).
5. Summary
We introduced the cumulative logistic regression model to estimate damage rating probabilities directly and we demonstrated features of the model by using it to estimate probabilities from environmental variables for tornadoes occurring in large clusters (10 or more tornadoes). Model parameters were determined by Bayesian inference using the method of Hamiltonian Monte Carlo with the Stan programming language. Stan code was generated from R through the brms package (Bürkner 2017). The flexibility of this approach makes it straightforward to adjust the model to estimate other outbreak characteristics (e.g., overall number of tornadoes) and to include domain-specific knowledge. Results show that the chance of higher damage ratings can be explained statistically by increasing values of bulk shear, CAPE, and helicity, and by decreasing values of distance to nearest city/town.
Coefficients on the environmental variables are consistent with expectations based on physical reasoning derived from the current understanding of how environmental factors influence tornado activity. There is a trend toward higher rated tornadoes with time as inferred in Elsner et al. (2019). The closer a tornado occurs to a city/town, the greater the chance it will get rated at the next higher EF rating. Taken together these two findings could be related to urban sprawl (Strader et al. 2017). Bulk shear has the strongest relationship to damage rating proportions. Under otherwise average conditions, the model estimates a 65% [IQR = (53%, 78%)] chance that any tornado occurring near a city or town will be rated EF0 when the bulk shear is weak (10 m s−1). This probability drops to 38% [IQR = (25%, 50%)] when the bulk shear is strong (40 m s−1) but with compensating increases in the chance of higher ratings. This quantification is only possible with a cumulative logistic regression.
This study makes the case that cumulative logistic regression is the right tool for quantifying the combined role environmental factors play on the distribution of tornadoes by EF rating. It might be tempting to fit a simpler model to these data as was done in Cohen et al. (2018), who suggested that simulated tornado wind speeds from their model can be scaled within the context of the damage ratings, but it is unclear how this can be done while preserving the relative frequency of ratings, given that the model residuals are assumed to be described by a normal distribution centered about the conditional mean wind speed. Cumulative logistic regression makes no such assumption and estimates probabilities on the damage ratings directly.
Although the results from applying the model for demonstration purposes are consistent with past research on this topic, there are limitations to the inferences that can be made with them. In particular, our exclusive focus on days with at least 10 tornadoes is a type of selection bias meaning that the sample of data used to fit the model does not represent the population of all tornadoes, which limits what we can say in general about the effect of convective environments on the probability of a particular EF rating. Further, no attempt was made to assess model skill in the context of its potential value in actual forecast situations. At a minimum a cross-validation exercise (see Elsner and Schmertmann 1994) would be needed. The demonstrated model could be made operationally useful if additional variables related to convective type and radar signatures are included.
Acknowledgments
The authors received no funding for this research. The code and data to produce all of the figures and results of this paper are available at github.com/jelsner/EF-dist.
REFERENCES
Bürkner, P.-C., 2017: brms: An R package for Bayesian multilevel models using Stan. J. Stat. Software, 80, 1–28, https://doi.org/10.18637/jss.v080.i01.
Carpenter, B., and Coauthors, 2017: Stan: A probabilistic programming language. J. Stat. Software, 76 (1), 1–32, https://doi.org/10.18637/jss.v076.i01.
Cheng, V. Y. S., G. B. Arhonditsis, D. M. L. Sills, W. A. Gough, and H. Auld, 2016: Predicting the climatology of tornado occurrences in North America with a Bayesian hierarchical modeling framework. J. Climate, 29, 1899–1917, https://doi.org/10.1175/JCLI-D-15-0404.1.
Cohen, A. E., J. B. Cohen, R. L. Thompson, and B. T. Smith, 2018: Simulating tornado probability and tornado wind speed based on statistical models. Wea. Forecasting, 33, 1099–1108, https://doi.org/10.1175/WAF-D-17-0170.1.
Elsner, J. B., and C. Schmertmann, 1994: Assessing forecast skill through cross validation. Wea. Forecasting, 9, 619–624, https://doi.org/10.1175/1520-0434(1994)009<0619:AFSTCV>2.0.CO;2.
Elsner, J. B., L. E. Michaels, K. N. Scheitlin, and I. J. Elsner, 2013: The decreasing population bias in tornado reports. Wea. Climate Soc., 5, 221–232, https://doi.org/10.1175/WCAS-D-12-00040.1.
Elsner, J. B., T. Fricker, and Z. Schroder, 2019: Increasingly powerful tornadoes in the United States. Geophys. Res. Lett., 46, 392–398, https://doi.org/10.1029/2018GL080819.
Gensini, V. A., and W. S. Ashley, 2011: Climatology of potentially severe convective environments from North American regional reanalysis. Electron. J. Severe Storms Meteor., 6 (8), http://www.ejssm.org/ojs/index.php/ejssm/article/viewArticle/85.
Hitchens, N. M., and H. E. Brooks, 2014: Evaluation of the Storm Prediction Center’s convective outlooks from day 3 through day 1. Wea. Forecasting, 29, 1134–1142, https://doi.org/10.1175/WAF-D-13-00132.1.
Jagger, T. H., J. B. Elsner, and H. M. Widen, 2015: A statistical model for regional tornado climate studies. PLOS ONE, 10, e0131876, https://doi.org/10.1371/journal.pone.0131876.
McElreath, R., 2015: Statistical Rethinking: A Bayesian course with examples in R and Stan. 1st ed. Chapman and Hall/CRC, 469 pp.
Mullen, L. A., and J. Bratt, 2018: USAboundaries: Historical and contemporary boundaries of the United States of America. J. Open Source Software, 3, 314, https://doi.org/10.21105/joss.00314.
Schroder, Z., and J. B. Elsner, 2019: Quantifying relationships between environmental factors and power dissipation on the most prolific days in the largest ‘outbreaks.’ Int. J. Climatol., https://doi.org/10.1002/joc.6388, in press.
Smith, B. T., R. L. Thompson, J. S. Grams, C. Broyles, and H. E. Brooks, 2012: Convective modes for significant severe thunderstorms in the contiguous United States. Part I: Storm classification and climatology. Wea. Forecasting, 27, 1114–1135, https://doi.org/10.1175/WAF-D-11-00115.1.
Smith, B. T., R. L. Thompson, A. R. Dean, and P. T. Marsh, 2015: Diagnosing the conditional probability of tornado damage rating using environmental and radar attributes. Wea. Forecasting, 30, 914–932, https://doi.org/10.1175/WAF-D-14-00122.1.
Steiner, E., 2019: Spatial History Project. Center for Spatial and Textual Analysis, Stanford University, http://web.stanford.edu/group/spatialhistory/cgi-bin/site/index.php.
Strader, S. M., W. S. Ashley, T. J. Pingel, and A. J. Krmenec, 2017: Projected 21st century changes in tornado exposure, risk, and disaster potential. Climatic Change, 141, 301–313, https://doi.org/10.1007/s10584-017-1905-4.
Thompson, R. L., B. T. Smith, J. S. Grams, A. R. Dean, and C. Broyles, 2012: Convective modes for significant severe thunderstorms in the contiguous United States. Part II: Supercell and QLCS tornado environments. Wea. Forecasting, 27, 1136–1154, https://doi.org/10.1175/WAF-D-11-00116.1.
Thompson, R. L., and Coauthors, 2017: Tornado damage rating probabilities derived from WSR-88D data. Wea. Forecasting, 32, 1509–1528, https://doi.org/10.1175/WAF-D-17-0004.1.
Togstad, W. E., J. M. Davies, S. J. Corfidi, D. R. Bright, and A. R. Dean, 2011: Conditional probability estimation for significant tornadoes based on Rapid Update Cycle (RUC) profiles. Wea. Forecasting, 26, 729–743, https://doi.org/10.1175/2011WAF2222440.1.
Widen, H. M., and Coauthors, 2013: Adjusted tornado probabilities. Electron. J. Severe Storms Meteor., 8 (7), http://ejssm.org/ojs/index.php/ejssm/article/view/131.
