1. Introduction
In 2009, the Hurricane Forecast Improvement Project (HFIP) was established with the goal of improving both track and intensity forecasts (Gall et al. 2013). It is well accepted that track forecasts have greatly improved and DeMaria et al. (2014) showed that hurricane intensity guidance has also improved at all forecast times on average at a statistically significant level. However, the prediction of rapid intensification (RI) and rapid weakening (RW) have shown little improvement and remain one of the highest-priority forecast challenges for forecasters at the National Hurricane Center (NHC) and other forecast agencies (Gall et al. 2013). RI and RW prediction is particularly critical for hurricanes approaching land with major implications on emergency management operations. It is important to understand the characteristics of the distribution of operational intensity forecast errors and not just the average error in order to improve intensity forecasts. Understanding how both RI and RW contribute to overall intensity forecast error distributions is a necessary step in improving RI and RW forecasts that has yet to be fully studied.
Rapid change in hurricane intensity is influenced by the large-scale environment, inner-core dynamics, and oceanic processes and requires detailed information across multiple scales to improve our forecast skill (Kaplan et al. 2010). Hendricks et al. (2010) showed statistical differences between the environments of hurricanes that underwent RI and those that did not. An important finding of their study is that the rate of intensification is only weakly dependent on environmental conditions given a favorable environment, indicating the importance of inner-core dynamics to intensification rate. In the current study, we investigate whether official intensity forecast errors have a similar dependence on key large-scale environmental conditions.
Both RI and RW strongly affect intensity forecast errors (Kaplan et al. 2010; Wood and Ritchie 2015), although less attention has been given to RW prediction. RW events can often be attributed to landfall but a significant number of RW events take place over water. RW events associated with landfall generally have lower forecast errors because substantial weakening is already predicted, but track errors can lead to large intensity errors due to changes in the forecasted time of landfall. Wood and Ritchie (2015) found that over-ocean RW events occur when hurricanes transition to environments with low convective available potential energy, cold sea surface temperature (SST), decreasing midlevel relative humidity (RH), and strong vertical wind shear. Over-ocean RW can also occur in somewhat favorable environments (Liang et al. 2016), in which case the inner-core dynamics may also become important. Little work has been done to examine the inner-core processes associated with over-ocean RW or evaluate the errors associated with RW.
A recent study by Na et al. (2018) showed a strong anticorrelation between operational forecast errors and intensity change. Official forecasts struggle with rapid intensity changes, which leads to underestimates of hurricane intensity during RI and overestimates during RW (Cangialosi and Franklin 2014). A limited increase in official forecast skill of RI prediction, relative to persistence and climatology, in recent years was shown by Kaplan et al. (2015), but it is unclear how skillful RW forecasts have been and whether forecast biases have improved. With the improvement of operational intensity models and statistical RI guidance, it is important to understand whether the distributions have narrowed for RI and RW forecast errors or the average errors have improved over the years. Additionally, it is important to know whether over-ocean or landfall RW guidance has improved or not, and if there have been changes to forecast biases at different lead times.
Van Sang et al. (2008) showed that there may be an intrinsic predictability limit of the mesoscale processes in hurricanes that contribute to RI; however, Emanuel and Zhang (2016) showed that there is still a large gap between our current intensity forecast skill and what is theoretically achievable. It is thought that improved models, better observations, and superior data assimilation techniques will lead to more accurate intensity forecasts (Emanuel and Zhang 2016). The representation of the mesoscale properties of the inner-core in forecast models is suggested to be critical for forecasting rapid intensity changes. The modeling study of Aberson et al. (2015) showed that assimilating Doppler radar observations improved short-term intensity forecasts but did not show significant improvements to forecasting RI. The inner-core dynamics have been shown to be important in understanding Hurricane Patricia’s (2017) both record-breaking RI and over-ocean RW (Doyle et al. 2017; Rogers et al. 2017; Martinez et al. 2019). Nystrom and Zhang (2019) found that assimilating inner-core radial velocities in Hurricane Patricia resulted in better forecasts of RI and a 40% reduction in forecast errors. In addition to intensity guidance from dynamical models, the development and implementation of the Statistical Hurricane Intensity Prediction Scheme (SHIPS) rapid intensification index (RII) has played a significant role in RI prediction and is a key operational forecast tool at NHC (Rozoff and Kossin 2011; Kaplan et al. 2015; Cangialosi et al. 2020). SHIPS-RII uses linear discriminant analysis in addition to a Bayesian and logistic regression model to create probabilistic RI guidance. The creation and improvement of the SHIPS decay model (DSHP) has provided guidance for land interactions, but a similar forecast tool explicitly for over-ocean RW has yet to be developed (Kaplan and DeMaria 1995; DeMaria et al. 2006).
The purpose of this study is to evaluate the characteristics and distributions of intensity forecast errors and demonstrate the relative contributions to the forecast errors from both RI and RW events. The spatial and temporal distributions will be analyzed in conjunction with environmental data to highlight where improvements can be made in forecasting RI and RW. Section 2 will discuss the data and RI/RW definitions used in this study. Section 3 will show the distributions of intensity forecast errors and the contributions from RI and RW. Section 4 will show the distribution of RI and RW errors in association with key environmental variables. Finally, section 5 will summarize and discuss the results of this study.
2. Data and methods
In this study, we analyze the operational intensity forecasts of the maximum sustained (1-min average) surface (10 m) winds at 6-h intervals. This study examines the NHC operational intensity forecast errors in the North Atlantic and east Pacific (east of 140°W) from 1989 to 2018. The intensity and track forecast error statistics from NHC can be found online (http://www.nhc.noaa.gov/verification/verify7.shtml). This 30-yr dataset includes tropical cyclones stronger than 20 kt (1 kt ≈ 0.51 m s−1) and excludes extratropical stages and dissipation forecasts. These exclusions are consistent with the verification rules used by NHC and the Joint Typhoon Warning Center (JTWC) (DeMaria et al. 2014). Forecast intensity errors are defined as the difference between the forecasted intensity and the best track intensity at the verifying forecast time. Note that the absolute value of each intensity error is used to calculate the mean absolute error (MAE).
In this study we consider RI over several forecast periods extending out to 48 h to include when NHC issues watches and warnings. We calculate the intensity change over 12-, 24-, 36-, and 48-h forecasts and the official forecast errors associated with those changes. Although there is some correlation with the previous time period, we treat each forecast as being independent to evaluate the full distribution and improve sample size. Here we use the same definitions as SHIPS RII to categorize RI for the four forecast periods that are found in Kaplan et al. (2010). RI is therefore defined as an increase of at least 20 kt in 12 h, 30 kt in 24 h, 45 kt in 36 h, and 55 kt in 48 h. We employ the same definitions corresponding to negative intensity changes to categorize RW over the same time periods for consistency, although past studies have used varying definitions (e.g., Liang et al. (2016): 20 kt in 24 h, Aberson et al. (2015): 25 kt in 24 h, Wood and Ritchie (2015): 30 kt in 24 h). The 24-h RI definition is consistent with that used by Na et al. (2018) and originally defined by Kaplan and DeMaria (2003). The values within the brackets shown in Table 1 are the number of events in the sample for each forecast length and basin.
Official intensity forecast errors for the North Atlantic and east Pacific for the 12-, 24-, 36-, and 48-h forecasts. The mean absolute error for only RI and RW events, all events, and all events excluding RI and RW are shown in knots. The last line is the percentage of MAE reduction relative to the total errors for each category. Units are denoted in parentheses, and the number of events is denoted in brackets.
Environmental variables are obtained from the SHIPS developmental dataset that extends from 1982 to 2017 in both the Atlantic and east Pacific (DeMaria et al. 2005). The SHIPS data are reduced to 1989–2017 based on the availability of NHC operational intensity forecast errors for the analysis. The SHIPS developmental database is based on the gridded analysis from the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) and was used to derive the SHIPS-RII. Environmental variables from SHIPS are used to evaluate the relationship between the thermodynamic environment and forecast errors. We focus here on 850–200-hPa deep-layer vertical wind shear, Reynolds SST, and 850–700-hPa RH, which have been shown to be important predictors for distinguishing RI and RW (Kaplan and DeMaria 2003; Hendricks et al. 2010; Kaplan et al. 2015). The 850–200-hPa vertical wind shear is averaged within a 500-km radius after the vortex circulation is removed from the background flow. The 850–700-hPa RH is averaged over the 200–800-km radii. Because the deep-layer vertical wind shear and midlevel RH are averaged over annuli, asymmetries in the environment are not well resolved and cannot be assessed in this study.
To understand the total distributions of RW events, we include landfall events in our statistics. Landfall events are included unless otherwise noted, as they contribute to a large distribution of forecast errors to be shown later. We distinguish RW events that are attributed to land interactions versus over-ocean events using the distance to land (DTL) variable in the SHIPS developmental database. RW due to land interactions will be identified based on whether the storm is within 50 km of any coastline. Major landmasses in addition to mountainous and large islands such as Puerto Rico and Jamaica are included in DTL; however, we assume that small relatively flat islands have negligible effects on rapid intensity changes (DeMaria et al. 2006).
3. Forecast error distributions
Figure 1 shows the relationship between official intensity errors and the change in surface wind speeds for the four forecast periods analyzed in this study. The RI and RW definitions for the forecast period are denoted by vertical red and blue lines, respectively, showing that the largest error magnitudes typically occur with RI and RW. The anticorrelation between intensity change and forecast errors is similar to that shown in the 24-h time period in Na et al. (2018), but here we expand the analysis with the addition of multiple forecast periods. Forecast errors and intensity change have a correlation around −0.7 at all forecast times and are statistically significant at the 99.9% confidence level. As expected, when the forecast length increases, the distribution of forecast errors widens to include a larger range of errors. The distribution of errors between the Atlantic and east Pacific are similar at the 12- and 24-h forecast periods, but as the forecast period length grows, the distribution widens faster in the east Pacific compared to the Atlantic. In the 36- and 48-h distributions for the Atlantic, the larger absolute intensity forecast errors are shifted more toward RI events than RW events. East Pacific hurricanes are more prone to strong weakening events compared to the Atlantic, due in part to the climatologically unfavorable SST to the north and west of the basin. The wider distributions of errors in the east Pacific at longer forecast times could also be due to the effects of track errors along more common gradients of SST or shear. While longer forecast lead times are not analyzed in detail in this study, it is noted that the error distributions for intensity forecasts do not continue to widen with increasing lead time beyond 48 h. Figure 2 shows the forecast error distributions normalized by the maximum number of events for forecasts between 24 and 120 h. The normalized forecast error distributions for forecasts longer than 48 h are similar, suggesting that there is an intrinsic limit on the magnitude of intensity errors. The forecast error distributions for forecasts longer than 48 h are overall similar despite the differences in sample size and are centered near zero. We will focus on forecasts extending through 48 h for the remainder of the study because 3–5-day forecasts have a significantly lower sample size.
Distribution of NHC official forecast errors and the corresponding intensity change for the (left) Atlantic and (right) east Pacific binned at 10-kt intervals. The (a),(b) 12-, (c),(d) 24-, (e),(f) 36-, and (g),(h) 48-h forecasts correspond to the wind speed change over that same duration of time. The colored boxes are normalized by the total number of events and are shaded by the % of the total. Bins that make up <0.2% of the total are not shaded. The zero forecast error and zero intensity change lines are black. The correlation coefficient is shown for each curve, which are all statistically significant at the 99.9% confidence level. The RI definition for each forecast length is shown by the red dashed lines, and the RW definition is shown by the blue dashed lines. For reference, the zero intensity change and zero error forecasts are found in the square above and to the right of the origin.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
Distribution of NHC official intensity forecast errors for forecasts lengths from 24 to 120 h. The forecast errors for the (a) Atlantic and (b) east Pacific are normalized by the maximum number of events.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
Figure 3 shows the distribution of official intensity forecast errors. The total distributions of all errors are approximately Gaussian with mean errors near zero in both basins for all forecast periods. The distributions for RI and RW confirm that these events represent the tails of the forecast error distributions in addition to the tails of the intensity change distribution (Kaplan and DeMaria 2003). The RI and RW distributions are also approximately Gaussian, but Atlantic and east Pacific RW events at longer lead times show a broader, almost bimodal distribution. The distributions widen with increasing forecast time period for both the total, RI, and RW events as previously noted. More RI and RW events occur in the east Pacific compared to the Atlantic and are accompanied by a slightly wider distribution of errors. The spatial distribution of 24-h RI and RW errors in Fig. 4 helps to explain the differences in number of RI and RW events. The higher number of east Pacific RI compared to Atlantic RI can be attributed to generally more favorable thermodynamic environments at lower latitudes in the east Pacific (Kaplan et al. 2010). The higher number of east Pacific RW events can be attributed to the sharp gradient in SST and lower instability to the northwest of the main east Pacific development region (Wood and Ritchie 2015), in addition to the lower number of recurvatures and extratropical transitions (Jones et al. 2003).
The distribution of official intensity forecast errors for all events (black), RI events (red), and RW events (blue) for the Atlantic (circles) and east Pacific (squares). Forecast errors are shown for forecast lengths of (a) 12, (b) 24, (c) 36, and (d) 48 h. The 12- and 24-h forecast errors are binned at 5-kt intervals while the 36- and 48-h forecast errors are binned at 10-kt intervals. Note the change in ordinate scale for the 36- and 48-h forecast errors. The number of RI and RW events uses the left ordinate, and the number of all events uses the right ordinate.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
Spatial distribution of all 24-h RW (circle) and 24-h RI (plus) events between 1989 and 2018 in the Atlantic and east Pacific. The events are shown at the initialization location and are colored by the intensity forecast error.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
The RI errors are almost always negative indicating that RI is associated almost exclusively with underforecasted intensity change, consistent with Na et al. (2018). These distributions indicate that when RI occurred, the forecasts predicted slower intensification than what occurred on average. The RI distributions are also dependent on the forecast period, with the most frequent errors becoming more negative for longer forecast lead time. The most frequent RI errors are approximately −15, −20, −30, and −40 kt for the 12-, 24-, 36-, and 48-h periods, respectively.
In contrast to RI, the RW events typically have positive forecast errors indicating a general underprediction of the weakening rate. There is, however, a nonnegligible component of negative forecast errors where the weakening was overpredicted. The RW distributions are wider than the RI distributions, and at longer forecast periods becomes slightly bimodal. The broadening and bimodality in the Atlantic can be attributed in part to differences between over-ocean RW and landfall RW for the 24–48-h forecast periods. For 24-h RW cases in the Atlantic, the center of the error distribution when the DTL < 50 km is at 10 kt; however the center of the distribution when the DTL > 50 km is at 20 kt suggesting that over-ocean weakening is harder to forecast (not shown). The spatial distribution of RW errors in Fig. 4 also shows that reduced errors with 24-h RW occur for hurricanes in the Gulf of Mexico approaching landfall. The rather large forecast errors associated with RW in the central Gulf of Mexico shown in Fig. 4 could be related to the initial storm intensity as it approaches landfall shown by Rappaport et al. (2010). The broad distribution in the 48-h RW forecasts for the east Pacific cannot be entirely explained by land interactions but could be related to track errors that become increasingly important at longer forecasts periods.
Table 1 shows how RI and RW events contribute to overall forecast errors. We remove all RI and RW events from the distributions and recalculate the MAE to show the full impact of these forecasts. The average MAE for RI and RW events at all forecast times is larger in the Atlantic than the east Pacific, but the total MAE of all forecasts is lower in the Atlantic than the east Pacific for all forecast periods excluding the 12-h period. MAEs would be reduced by 10%–22% if we neglect RI and RW events from the error calculations. The MAE in the Atlantic is reduced less than that of the east Pacific. The intensity errors of both basins are reduced to nearly the same value at 24-h lead times when RI and RW are neglected. By removing RI and RW events, there is a reduction in MAE of 15.6%, 15.9%, 10.9%, and 9.7% in the Atlantic and a reduction in MAE of 19.8%, 22%, 16.9%, and 15.3% in the east Pacific for the 12-, 24-, 36-, and 48-h forecast periods, respectively. The analysis suggests that the larger number of occurrence of RI and RW events in the east Pacific is a major contributor to why the basin has larger errors on average and would therefore have more improvement if they are neglected from the MAE calculations. We note that if root-mean squared error (RMSE) is used as the performance metric instead of MAE, the error reduction due to excluding RI/RW would increase due to the enhanced weight of more common large errors. Even larger improvements in overall performance would be possible at the 36- and 48-h lead times with improved RI/RW forecasts using the RMSE metric (not shown).
Although we have shown the large forecast errors for RI and RW overall, we have not considered the errors when rapid intensity changes are actually forecast by NHC. Table 2 shows the number of RI events, the number of forecasted RI events, the number of verifying RI forecasts, the verifying percentage or success ratio, the probability of detection, and the corresponding MAE for all RI forecasts. A forecast must meet or exceed the intensity change threshold magnitudes described in section 2 to be considered. Therefore, a 25-kt intensity change forecast over 24 h is considered a missed RI forecast. In the Atlantic, RI is forecasted considerably less often compared to the east Pacific at all forecast periods, which is related to the climatologically larger number of RI events in the east Pacific. The number of verifying forecasts is also higher in the east Pacific at the 12- and 24-h forecast periods. It is difficult to compare RI between the two basins at the 36- and 48-h thresholds because only 2 and 3 forecasts meet that criteria in the Atlantic, respectively. When NHC made an RI forecast, they correctly forecasted the occurrence of 24-h RI events 53% of the time in the Atlantic and 68% of the time in the east Pacific with MAEs of 12.5 and 14.4 kt. The errors when RI is forecast are significantly lower than the average errors with RI/RW shown in Table 1, but still larger than the mean error for the forecast period. The probability of detection for RI cases is low for all forecast periods in both basins with 5%, 3.3%, 0.7%, and 0.7% in the Atlantic and a probability of detection of 8.8%, 10.8%, 5.5%, and 5.3% in the east Pacific for the 12-, 24-, 36-, and 48-h forecast periods, respectively. The probability of detection is 7.5% larger for the 24-h RI in the east Pacific compared to the Atlantic, which suggests that forecasters are more likely to forecast RI in the east Pacific because climatologically there are more frequent RI events there. A higher probability of detection for RI in the east Pacific could also be due to better performance by SHIPS-RII because of the more favorable environments there on average (Kaplan et al. 2010). Although we did not find any trends using the 30 kt in 24-h RI threshold, Cangialosi et al. (2020) showed that using the 20 kt in 24-h RI threshold did suggest improvements in the probability of detection of Atlantic RI events in the last decade.
Error statistics for when NHC forecasted intensity changes that meets the RI criteria thresholds at each forecast period. Verifying RI is the percentage of forecasted RI events that verified to the number of total RI forecasts by NHC. The probability of detection for RI events is shown as the number of verifying RI forecasts divided by the total number of RI events. The MAE is shown for all RI events that were forecasted within the sample.
Next we analyze Table 3, which shows the same elements as Table 2 but for RW events in both basins. Table 3 shows that RW events occur less often compared to RI events at all forecast periods except for 12-h forecasts in the Atlantic, which is also evident in Fig. 3. Despite the lower number of RW events, the number of forecasted RW events is much larger than RI events because RW is common for landfalling hurricanes. RW forecasts in the Atlantic verify 75% of the time for 12-h forecasts and 89% of the time for 24-h forecasts, which is 11% and 13% larger than the same RW forecast periods in the east Pacific. A smaller number of RW forecasts verify in the east Pacific compared to the Atlantic and there is also a lower probability of detection at all forecast periods. The differences in probability of detection of RW events between the basins can largely be explained by the larger number of over-ocean weakening in the east Pacific and the larger number of landfall RW events in the Atlantic. The probability of detection and verifying forecast percentage would suggest that RW in the Atlantic is better forecasted; however, the MAE for RW events in the Atlantic are actually higher for 12–36-h forecast periods despite the fact that landfalling hurricanes are more common and typically are better forecasted. We speculate that the larger MAE in the Atlantic RW events could be due to uncertainty in the track forecasts and the timing of landfall.
While RI and RW events substantially increase the average MAE, there is considerable year-to-year variability in the number of these events. To address this variability we consider the correlation between number of RI and RW events and the yearly mean forecast errors. Figure 5 shows the Pearson correlation between number of 24-h RI and RW events and the yearly MAE. The correlation explains ~20% of the variance in the Atlantic and ~30% of the variance in the east Pacific, which is statistically significant at the 95% confidence level. The positive correlation indicates that the more tropical storms and hurricanes undergo RI or RW, the larger the MAE for a given year. While attribution of the trends and year-to-year variability is beyond the scope of this study, we emphasize here that it is important to consider these trends and variability when looking at the progressive improvement of forecast errors. As we increase the forecast period length, the correlation is reduced in the Atlantic due to decreasing sample size, while there is little change in the correlation with forecast period length in the east Pacific (not shown).
The 24-h MAE and number of RI and RW events for the (a) Atlantic and (b) east Pacific. The yearly averaged 24-h MAE (black) uses the left ordinate, and the number of yearly RI and RW events (red) uses the right ordinate. Linear trend lines are shown in dashed lines and the r value and p value are shown testing the correlation.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
We next analyze the RI and RW forecasts to determine whether or not forecasts associated with RI and RW have improved over the years. To analyze whether the errors have improved we calculate the yearly forecast bias, which is the mean of the error distribution, for RI and RW forecasts. Figure 6 shows the bias of RI and RW events over time with statistically significant slopes at the 90% confidence level shown with stars. At the 12-h forecast period there is statistically significant improvement in RI forecasts in both basins, although 12-h RW forecasts have shown little improvement. For 24-h RI events, the east Pacific has shown statistically significant improvement but RI in the Atlantic has not improved. The improvement of 24-h RI forecasts in the east Pacific can be partially attributed to improved guidance from SHIPS-RII that has been shown provide better RI probabilities in the east Pacific (Kaplan et al. 2010). The 24-h RW cases in the Atlantic have shown statistically significant improvement, but the east Pacific 24-h RW events have not shown improvement. There is clear improvement in 36- and 48-h RI and RW forecast bias in both basins although not all the improvement is significant. The biases for 36-h RI, 48-h RI, and 48-h RW events have all improved in the east Pacific. The large year-to-year variability in RI and RW events in the Atlantic is a contributing reason to the lack of statistically significant improvement in the 36–48-h RI and RW bias and should be considered when referencing the yearly biases. The improvements in some of the biases suggest that forecasts of rapid intensity changes are improving, although it is not because of an increase in forecasts of RI and RW. There are no significant trends in the number of yearly RI and RW forecasts in either basin (not shown).
The yearly intensity forecast bias for only RI and RW events in the Atlantic and the east Pacific. Bias associated with RI and RW are shown for the (a) 12-, (b) 24-, (c) 36-, and (d) 48-h forecast periods using the definitions stated in section 2. Lines with stars indicate that the slope of the bias line is statistically significant from zero at the 90% confidence level using a two-sided t test. The gray lines indicate the zero bias line.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
RW in east Pacific hurricanes is more common than in the Atlantic, but it is unclear why 24-h RW forecasts in the east Pacific have shown no improvement while significant improvements are found in the Atlantic. One potential reason for the difference in 24-h RW trends between the basins is the contribution of RW errors by landfall events. To quantify how landfall events affect the trends in forecast errors associated with RW, we isolate the events where RW occurred within 50 km of any landmass. Figure 7 shows the 24-h MAE trends for over-ocean and land interaction RW events separately for the Atlantic and east Pacific. In the Atlantic, the errors associated with over-ocean RW are larger and have not improved as much compared to the errors associated with RW due to land interactions. Both types of Atlantic RW have improved slightly through the decades that both contribute to the significant improvement in 24-h RI bias shown in Fig. 6. In the east Pacific, the trend for RW events where land interactions are involved is negative at a statistically significant level. RW due to land interaction cases are not as common in the east Pacific but the MAE associated with those events have been reduced in recent years. The over-ocean RW events in the east Pacific are the main contributor to the lack of improvement in the biases shown in Fig. 6. The MAE associated with over-ocean RW events have not improved with time and the trend line is slightly positive. We speculate that a contributing factor may be the difficulty in forecasting the timing of over-ocean RW events as hurricanes cross SST gradients in the east Pacific versus the timing of landfall events in the Atlantic. Improvements in RW forecasts near land can partially be explained by improving track forecasts, although it is unclear why improved track forecasts do not directly result in lower intensity errors for over-ocean RW in the east Pacific. Cangialosi et al. (2020) noted that track and intensity errors have only a correlation of 0.2 from 2010 to 2019 in the Atlantic. One potential reason for the lack of improved over-ocean RW forecasts is that even small cross-track errors can result in large SST differences under east Pacific hurricanes along the climatological SST gradients.
Yearly 24-h MAE for the (a) Atlantic and (b) east Pacific associated with only RW cases with land interactions (brown) and over-ocean RW (blue) events. Data are shown from 1989 to 2017 where there is overlap between the verification database and SHIPS database. The r value and p value are shown for each trend line that is shown in the dashed lines. The p value is determined from a two-sided t test testing the null hypothesis that the slopes of the lines are zero.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
Figure 8 shows the relationship of official intensity forecast errors for the different forecast periods with maximum wind speeds at the forecast initialization time. In general, the distributions are similar between the east Pacific and Atlantic with the largest concentration of forecasts for tropical cyclones between 30 and 70 kt. For the 12-h forecasts, the distributions are centered near zero. As the forecast period length grows the distributions widen with larger magnitudes of intensity forecast errors. The distributions grow asymmetrically for increasing forecast period length with more overestimates (positive bias) in intensity for stronger hurricanes and more underestimates (negative bias) for weaker hurricanes. Bhatia and Nolan (2013) also found a dependence of forecast errors on initial hurricane intensity with larger intensity biases for stronger storms. There is a larger tendency for overestimates in the Atlantic compared to the east Pacific in the 24–48-h forecast periods, which likely is due to more land interactions. The overestimates at larger hurricane intensities may also be due to the difficulty of predicting secondary eyewall formation and eyewall replacement cycles. Kossin and DeMaria (2016) found similar overestimates in SHIPS due to eyewall replacement cycles and created a simple model to reduce the errors. More frequent large underestimates in the east Pacific can be attributed to the larger number of RI events in the basin.
Distribution of forecast errors with maximum wind speeds for the (left) Atlantic and (right) east Pacific. Shown are the distributions for (a),(b) 12-, (c),(d) 24-, (e),(f) 36-, and (g),(h) 48-h forecast periods. Both the intensity forecast errors and maximum wind speeds are binned at 10-kt intervals. The colored boxes are normalized by the total number of events. Bins that make up <0.2% of the total are not shaded. The black line is the zero forecast error line where values above the line are the number of zero error forecasts.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
4. Environmental contribution to forecast errors
In this section we analyze the environmental variables that may contribute to the difficulty in forecasting RI and RW. For conciseness, we will only show select environmental variables from SHIPS for the 24-h forecast period. The environmental variables shown correspond to the average atmospheric state between the time when each forecast was made and the verifying time. While the change in environmental variables over the forecast time is also important for intensity change, it adds another layer of complexity that is not considered here but will remain a topic of future work.
Figure 9 shows the total error distribution in addition to the distribution for exclusively RI and RW events of forecast errors in the Atlantic with respect to 850–200-hPa vertical wind shear, SST, and 850–700-hPa RH. Other variables such as 200-hPa divergence, 850-hPa vorticity, and maximum potential intensity were also considered but showed similar relationships and are not shown for brevity. The errors for all events with respect to vertical wind shear (Fig. 9b) indicate that the magnitude of forecast errors is generally reduced for environments with strong vertical wind shear. For environments with low vertical wind shear, the forecast error distribution widens considerably with both large positive and large negative forecast errors. If we only consider the vertical wind shear of RI and RW events (Fig. 9a), we can see that RW (positive errors > 10 kt) events generally occur in environments with slightly larger shear values and that RI (negative errors < −10 kt) events occur in environments with slightly lower shear consistent with past observations (e.g., Kaplan and DeMaria 2003; Hendricks et al. 2010). The widest distribution of forecast errors for both RI and RW occur when shear is low to moderate (5–20 kt), which is climatologically favorable for hurricane intensification.
The relationship between 24-h official forecast errors and SHIPS environmental data in the Atlantic from 1989 to 2017. (right) The total distribution for all events and (left) the distribution for only RI and RW events. (a),(b) The 850–200-hPa vertical wind shear, (c),(d) SST, and (e),(f) 850–700-hPa RH are binned at 5 kt, 1°C, and 5%, respectively. Intensity forecast errors are binned at 5 kt. The colored boxes are normalized by the total number of events. Bins that make up <0.2% of the total are not shaded. The black line is the zero forecast error line where values above the line are the number of zero error forecasts.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
The errors with respect to SST show a similar pattern as the vertical wind shear (Figs. 9c,d) with more unfavorable conditions (colder SST) resulting in a narrower distribution of intensity forecast errors. As SST increases, the distribution of forecast errors widens and there is an increase in the magnitude of both positive and negative errors. The relationship between SST and forecast errors for RI and RW events shows a similar distribution as the total but with larger error magnitudes. The total distribution of all intensity forecast errors with 850–700-hPa RH (Figs. 9e,f) shows a circular distribution centered at zero errors and 65%–70% RH. The error distribution is wider for larger RH values compared to lower RH values. The distribution of forecast errors associated with RI are shifted slightly toward larger RH values compared to RW. Errors associated with RW occur in environments similar to RI with the largest error magnitudes for higher RH. The largest intensity forecast errors occur at low shear values with warm SST suggesting that additional factors and processes need to be considered to forecast RI and RW, such as inner-core dynamics (Van Sang et al. 2008; Aberson et al. 2015; Nystrom and Zhang 2019).
Figure 10 shows the relationship of environmental conditions with 24-h official intensity forecast errors in the east Pacific. Here we show the error distributions associated with 850–200-hPa vertical wind shear, SST, and 850–700-hPa RH. The overall qualitative distribution of environmental conditions in the east Pacific is similar to the Atlantic, but the forecast error distributions are not as symmetric about the zero forecast error line for SST and RH. Figures 10c and 10e show there are very few large forecast errors for RW over cold SST or drier environments suggesting that when the environment is unfavorable, RW is easier to forecast. As the environment becomes more favorable with reduced vertical wind shear, warmer SST, and higher RH, the width of the distribution of forecast errors increases, which is similar to the Atlantic. Also similar to the Atlantic, the widest forecast error distributions associated with RI and RW typically occur in environments with low shear, warm SST, and moderate RH. This again emphasizes that the inner-core processes are critical for predicting rapid changes in intensity.
As in Fig. 9, but for the east Pacific.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
To better illustrate the role of the environment on total forecast errors, Fig. 11 shows the normalized distribution of all 24-h forecast errors in different environments in the east Pacific from SHIPS. In Fig. 11a the distributions correspond to vertical wind shear greater than 20 kt, between 10 and 20 kt, and less than 10 kt indicating an unfavorable, moderate, and favorable environment for hurricane intensification respectively. The peak of each distribution is normalized so the key differences in the figures are in the widths and skewness of the distributions. The analysis provides further evidence that intensification is underestimated more frequently when there is favorable environmental shear. When vertical wind shear exceeds 20 kt, there are fewer underestimates of hurricane intensity because RI does not frequently occur and slower intensification rates are therefore forecast well. On the positive side of the forecast error distribution, the largest overestimates of hurricane intensity occur with moderate values of vertical wind shear. This suggests that larger errors associated with RW are in environments with moderate wind shear consistent with other observations (Bhatia and Nolan 2013). The distribution of forecast errors for RH (Fig. 11b) and SST (Fig. 11c) show similar relationships as vertical wind shear indicating that larger negative forecast errors are associated with more favorable environments. Also similar to vertical wind shear, marginal values of SST and RH seem to have the largest positive forecast errors.
The distribution of 24-h forecast errors from 1989 to 2017 in the east Pacific conditioned on the SHIPS (a) 850–200-hPa vertical wind shear, (b) 850–700-hPa RH, and (c) SST. For (a)–(c) the SHIPS environmental variable is conditioned into three groups with the forecast error distribution then normalized by the maximum value. (d) The distribution of forecast errors for RH > 75% (orange) and RH < 75% (green) marked with circles and the SST > 28°C (orange) and SST < 28°C (green) marked with squares when the 850–200-hPa vertical wind shear is larger than 15 kt. Forecasts errors are limited to those that occurred greater than 50 km from any major landmass.
Citation: Weather and Forecasting 35, 6; 10.1175/WAF-D-19-0253.1
For any individual forecast one must consider the vertical wind shear, RH, and SST but sometimes one environmental variable can limit possible ranges of intensity change. Thus far all the environmental variables have been considered independently when analyzing the forecast error distributions; however, all the environmental variables covary and play a role in hurricane intensity change. Figure 11d shows the distributions of favorable and unfavorable values of RH and SST given that the vertical wind shear is greater than 15 kt. When vertical wind shear is moderate, the distribution of forecast errors still show that warmer SST and higher RH are associated with larger numbers of negative forecast errors, but the differences between favorable and unfavorable SST/RH have been diminished. When SST is warm and RH is high, there is also a shift toward a positive bias in the distribution compared to a bias centered on zero when all the factors are unfavorable for intensification. The MAE is 12.6 kt in the case where all the variables are favorable (meaning vertical wind shear is less than 15 kt, RH > 75%, and SST > 28°C), but when the environment is unfavorable by these same thresholds, the MAE is reduced considerably to 7.8 kt. Further research is needed to understand the various combinations of environmental parameters on forecast skill.
5. Summary and conclusions
In this study, we have evaluated the characteristics of intensity forecast error distributions and demonstrated the relative contributions of both rapid intensification (RI) and rapid weakening (RW) events. It has been shown that rapid intensity changes are associated with the tails of the distribution of intensity forecast errors, which has been assumed but never analyzed in detail to the authors’ knowledge. Forecast errors associated with both rapid intensification and weakening are nearly always underestimated in magnitude consistent with the analysis by Na et al. (2018). Consistent with DeMaria et al. (2014), there has been a slight improvement in the 24-h intensity forecast error distributions over the years and intensity forecast distributions overall are centered at zero.
RI and RW is associated with large forecast errors on average. Rapid weakening is forecasted more often than RI with a much higher success ratio and probability of detection. Over-ocean RW events cause the reduced probability of detection in the east Pacific compared to the Atlantic despite the larger number of RW events. Rapid intensification occurs more often in the east Pacific compared to the Atlantic, which may lead forecasters to predict intensity changes meeting the RI thresholds more often in the east Pacific. The probability of detection for 24-h RI events is 7.5% larger in the east Pacific, although RI is predicted only ~3% of the time in the Atlantic. Although NHC rarely forecasts RI, when NHC does predict rapid intensification at 24 h, the forecasts verify ~50% of the time in the Atlantic. Using a probability of detection approach to evaluating RI and RW forecasts has some limitations because a 25-kt intensity change forecast in 24 h would be considered a missed RI forecast despite the fact that the error would be low. However, using only mean absolute error as a metric is also limited by the fact that accurately forecasting RI thresholds produces minimal error improvement.
This study is novel in its examination of intensity forecast error distributions in association with environmental conditions over multiple decades. We have shown that the largest error distributions associated with RI occur in climatologically favorable environments for intensification. Although RW has received less attention in the literature, RW events cause a larger distribution of intensity forecast errors than RI. In the Atlantic, the wide distribution is attributed to the difference in over-ocean events and landfalling hurricanes. Over-ocean RW causes larger forecast errors on average because substantial weakening is usually predicted for landfalling hurricanes unless there are substantial track errors associated with the landfall timing. Rapid weakening events that occur in more unfavorable environments, such as with cold SST or stronger vertical wind shear, have lower forecast errors on average suggesting a lower degree of intensity forecast uncertainty. RW events can occur in moderate to favorable thermodynamic and dynamic environments for intensification, suggesting that improved understanding of inner core processes is required for both RW and RI events.
As the forecast period length increases from 12 to 48 h, the width of the intensity error distributions also increases as both positive and negative errors grow with time. The intensity forecast error distributions show similar widths beyond 48 h. The larger forecast errors for RI and RW events explains roughly 20% of the variance in the yearly mean absolute errors in the Atlantic and 30% of the variance in the east Pacific. A positive correlation between the number of RI and RW events and the yearly mean forecast error has been assumed in previous discussions of forecast errors (Cangialosi and Franklin 2014) and here we explicitly show this to be true. In the east Pacific, the intensity bias associated with RI has decreased at a statistically significant level for the 12-, 24-, 36-, and 48-h forecast periods, while significant improvement is found only in 12-h RI forecasts in the Atlantic. Intensity biases associated with RW have decreased at a statistically significant level for the 24-h period in the Atlantic and in the 36- and 48-h period in the east Pacific. The lack of improvement in east Pacific 24-h RW forecasts can be attributed to large forecast errors for over-ocean RW events.
Understanding the environment is important in determining the potential for a tropical disturbance to intensify, but we show here that the largest forecast errors occur due to RI when there is a favorable environment. The largest RW errors also occur in moderate to favorable environments. We attribute the effects of convective and mesoscale processes to the increased spread of short-term intensity forecast errors when the large-scale environment is favorable for intensification. This suggests that when hurricanes are in favorable environments for intensification the intensity forecasts have a higher probability of large errors and thus a larger degree of uncertainty. The analysis presented in this study suggests that improved understanding of the inner-core dynamics of hurricanes in favorable environments is paramount and an important area for future work to improve intensity forecasts and reduce the width of the intensity error distribution.
When RI occurred, the distribution of errors suggests that forecasts were in general too slow to intensify and nearly always underestimated the intensification. We have evaluated these errors in prediction of RI without considering forecast-to-forecast continuity or changes in numerical model guidance over time. Although we treat each forecast as independent in this study, real-time forecasts are correlated from one forecast cycle to the next. The relative contribution to intensity forecast errors from individual factors such as forecast continuity or numerical model guidance remain a topic for future work. The uncertainty of hurricane rapid intensity change forecasts will be dependent on a combination of the uncertainty in track forecasts (particularly near land or gradients in thermodynamic variables), the thermodynamic environment, and the consistency of model guidance to give forecasters the confidence to forecast RI and RW thresholds. New tools for RI and RW prediction are needed at NHC in order to improve intensity forecasts (Cangialosi et al. 2020).
Acknowledgments
This work has been funded by the Office of Naval Research Awards N000141613033, N000141712230, and N000142012069, and National Science Foundation Award AGS-1701225. The authors thank Naufal Razin, Jhordanne Jones, and three anonymous reviewers for their insightful comments and suggestions.
REFERENCES
Aberson, S. D., A. Aksoy, K. J. Sellwood, T. Vukicevic, and X. Zhang, 2015: Assimilation of high-resolution tropical cyclone observations with an ensemble Kalman filter using HEDAS: Evaluation of 2008–11 HWRF forecasts. Mon. Wea. Rev., 143, 511–523, https://doi.org/10.1175/MWR-D-14-00138.1.
Bhatia, K. T., and D. S. Nolan, 2013: Relating the skill of tropical cyclone intensity forecasts to the synoptic environment. Wea. Forecasting, 28, 961–980, https://doi.org/10.1175/WAF-D-12-00110.1.
Cangialosi, J. P., and J. L. Franklin, 2014: 2013 National Hurricane Center Forecast verification report. NOAA/National Hurricane Center, 84 pp., https://www.nhc.noaa.gov/verification/pdfs/Verification_2013.pdf.
Cangialosi, J. P., E. Blake, M. DeMaria, A. Penny, A. Latto, E. N. Rappaport, and V. Tallapragada, 2020: Recent progress in tropical cyclone intensity forecasting at the National Hurricane Center. Wea. Forecasting, 35, 1913–1922, https://doi.org/10.1175/WAF-D-20-0059.1.
DeMaria, M., M. Mainelli, L. K. Shay, J. A. Knaff, and J. Kaplan, 2005: Further improvements to the Statistical Hurricane Intensity Prediction Scheme (SHIPS). Wea. Forecasting, 20, 531–543, https://doi.org/10.1175/WAF862.1.
DeMaria, M., J. A. Knaff, and J. Kaplan, 2006: On the decay of tropical cyclone winds crossing narrow landmasses. J. Appl. Meteor. Climatol., 45, 491–499, https://doi.org/10.1175/JAM2351.1.
DeMaria, M., C. R. Sampson, J. A. Knaff, and K. D. Musgrave, 2014: Is tropical cyclone intensity guidance improving? Bull. Amer. Meteor. Soc., 95, 387–398, https://doi.org/10.1175/BAMS-D-12-00240.1.
Doyle, J. D., and Coauthors, 2017: A view of tropical cyclones from above. Bull. Amer. Meteor. Soc., 98, 2113–2134, https://doi.org/10.1175/BAMS-D-16-0055.1.
Emanuel, K., and F. Zhang, 2016: On the predictability and error sources of tropical cyclone intensity forecasts. J. Atmos. Sci., 73, 3739–3747, https://doi.org/10.1175/JAS-D-16-0100.1.
Gall, R., J. Franklin, F. Marks, E. N. Rappaport, and F. Toepfer, 2013: The hurricane forecast improvement project. Bull. Amer. Meteor. Soc., 94, 329–343, https://doi.org/10.1175/BAMS-D-12-00071.1.
Hendricks, E. A., M. S. Peng, B. Fu, and T. Li, 2010: Quantifying environmental control of tropical cyclone intensity change. Mon. Wea. Rev., 138, 3243–3271, https://doi.org/10.1175/2010MWR3185.1.
Jones, S. C., and Coauthors, 2003: The extratropical transition of tropical cyclones: Forecast challenges, current understanding, and future directions. Wea. Forecasting, 18, 1052–1092, https://doi.org/10.1175/1520-0434(2003)018<1052:TETOTC>2.0.CO;2.
Kaplan, J., and M. DeMaria, 1995: A simple empirical model for predicting the decay of tropical cyclone winds after landfall. J. Appl. Meteor., 34, 2499–2512, https://doi.org/10.1175/1520-0450(1995)034<2499:ASEMFP>2.0.CO;2.
Kaplan, J., and M. DeMaria, 2003: Large-scale characteristics of rapidly intensifying tropical cyclones in the North Atlantic basin. Wea. Forecasting, 18, 1093–1108, https://doi.org/10.1175/1520-0434(2003)018<1093:LCORIT>2.0.CO;2.
Kaplan, J., M. DeMaria, and J. A. Knaff, 2010: A revised tropical cyclone rapid intensification index for the Atlantic and eastern North Pacific basins. Wea. Forecasting, 25, 220–241, https://doi.org/10.1175/2009WAF2222280.1.
Kaplan, J., and Coauthors, 2015: Evaluating environmental impacts on tropical cyclone rapid intensification predictability utilizing statistical models. Wea. Forecasting, 30, 1374–1396, https://doi.org/10.1175/WAF-D-15-0032.1.
Kossin, J. P., and M. DeMaria, 2016: Reducing operational hurricane intensity forecast errors during eyewall replacement cycles. Wea. Forecasting, 31, 601–608, https://doi.org/10.1175/WAF-D-15-0123.1.
Liang, J., L. Wu, G. Gu, and Q. Liu, 2016: Rapid weakening of Typhoon Chan-Hom (2015) in a monsoon gyre. J. Geophys. Res. Atmos., 121, 9508–9520, https://doi.org/10.1002/2016JD025214.
Martinez, J., M. M. Bell, R. F. Rogers, and J. D. Doyle, 2019: Axisymmetric potential vorticity evolution of Hurricane Patricia (2015). J. Atmos. Sci., 76, 2043–2063, https://doi.org/10.1175/JAS-D-18-0373.1.
Na, W., J. L. McBride, X.-H. Zhang, and Y.-H. Duan, 2018: Understanding biases in tropical cyclone intensity forecast error. Wea. Forecasting, 33, 129–138, https://doi.org/10.1175/WAF-D-17-0106.1.
Nystrom, R. G., and F. Zhang, 2019: Practical uncertainties in the limited predictability of the record-breaking intensification of Hurricane Patricia (2015). Mon. Wea. Rev., 147, 3535–3556, https://doi.org/10.1175/MWR-D-18-0450.1.
Rappaport, E. N., J. L. Franklin, A. B. Schumacher, M. DeMaria, L. K. Shay, and E. J. Gibney, 2010: Tropical cyclone intensity change before U.S. Gulf Coast landfall. Wea. Forecasting, 25, 1380–1396, https://doi.org/10.1175/2010WAF2222369.1.
Rogers, R. F., and Coauthors, 2017: Rewriting the tropical record books: The extraordinary intensification of Hurricane Patricia (2015). Bull. Amer. Meteor. Soc., 98, 2091–2112, https://doi.org/10.1175/BAMS-D-16-0039.1.
Rozoff, C. M., and J. P. Kossin, 2011: New probabilistic forecast models for the prediction of tropical cyclone rapid intensification. Wea. Forecasting, 26, 677–689, https://doi.org/10.1175/WAF-D-10-05059.1.
Van Sang, N., R. K. Smith, and M. T. Montgomery, 2008: Tropical cyclone intensification and predictability in three dimensions. Quart. J. Roy. Meteor. Soc., 134, 563–582, https://doi.org/10.1002/qj.235.
Wood, K. M., and E. A. Ritchie, 2015: A definition for rapid weakening of North Atlantic and eastern North Pacific tropical cyclones. Geophys. Res. Lett., 42, 10 091–10 097, https://doi.org/10.1002/2015GL066697.
