a. Storm, environmental, and comparison data

Two datasets, similar to the ones in Lee et al. (2015, 2016), are used in this study to develop and test the model. The first dataset is used for RI labeling and storm feature calculations and is taken from the National Hurricane Center (NHC; Landsea and Franklin 2013) and the Joint Typhoon Warning Center best track data (JTWC; Chu et al. 2002). The second dataset contains the environmental features, which are calculated using the monthly European Centre for Medium-Range Weather Forecasts interim reanalysis (ERA-Interim; Dee et al. 2011). Although the monthly averaged environmental features contain less detailed information than the higher temporal resolution data that are used in operational predictions, Lee et al. (2015) showed that in a statistical model for intensity prediction there was only a modest reduction in error when using daily environmental data instead of monthly averages, and our experiments support this statement, showing no better performance achieved using daily data. Therefore, we present results here using monthly environmental data and discuss the impact of using higher-frequency data in the Conclusions. The dataset includes storms from five basins: Atlantic, eastern North Pacific (including central North Pacific), northern Indian Ocean, western North Pacific, and Southern Hemisphere. Each storm was recorded in a time-sequential format with descriptions of the storm’s status and its surrounding environment conditions every 6 h. Because we use monthly average environmental fields (centered at 15th of the month) interpolated to daily, changes in the storm environment from one time step to the next are primary due only to the change in storm location. We use storms whose lifetime maximum intensity reached at least tropical storm strength (34 kt). Storm intensity and environmental features are calculated from their genesis (the first record) to landfall. We do not include information after landfall. Extratropical stages are also included. More details of the dataset are discussed in Lee et al. (2015).

The model is trained using data from the period 1981–2009 (training dataset). Hyperparameters (e.g., model configuration and learning rate) are tuned on data from the period 2010–13 (validation dataset). Data from the period 2014–17 (test dataset) are used for model evaluation and are completely independent of the training and validation datasets. There are 51 629, 4007, and 6857 samples in the training, validation, and testing datasets, respectively. The information regarding the number of RI cases is summarized in Table 1. Model predictions during the period 2014–17 for the Atlantic and eastern North Pacific are compared with the operational SHIPS Rapid Intensification Index (SHIPS RII).

Table 1.

The number of RI case in the training, validation, and test datasets for thresholds of 25, 30, and 35 kt.

View Table

b. Feature engineering

A total of 35 features (8 storm features, 22 environmental features, and 5 basins) are included in the model. Storm features represent the storm status at the time steps prior to the initialization time, including storm maximum wind, minimum sea level pressure, wind speed changes, basin, and so on (Table 2). Environment features describe the large-scale conditions near the storm and include ocean temperature, wind shear, and so on (Tables 3 and 4 ). In these two tables, we can see that some variables are averaged over a small disk and others over a large annulus. The former ones contain storm structure information while variables averaged over a larger annulus describe environment features in which the storms are embedded. Now we briefly discuss some of the features.

Table 2.

Storm features.

View Table

Table 3.

Environment features. Variables presented here are either derived at storm center or averaged over a disk or over an annulus. Disk-averaged variables are averaged over the area within 500 km of the storm center while annulus-averaged ones are averaged over a ring-shape area with inner radius of 200 km and outer radius 800 km from storm center.

View Table

Table 4.

Environment features (continued).

View Table

One of the most important environment controls on TC behavior is maximum potential intensity (MPI; Camargo et al. 2009, 2007). Here, we used the MPI definition developed in Emanuel (1988) and modified in Emanuel (1995) and Bister and Emanuel (2002). MPI calculated using sea surface temperature is a function of outflow temperature which is defined as the 200 hPa temperature away from the storm center. The temperature at 200 hPa ($T_{{\text{air}}_{200\text{mb}}}$) is also included as a separate feature (Kaplan and DeMaria 2003). MPI also depends on sea surface temperature. Storm-centered and area-averaged values of ocean temperature averaged over the top 100 m are included as predictors (Price 2009). Vertical wind shear variables (meridional, zonal, and annulus averaged) are included since strong vertical wind shear has a negative impact on the TC intensification process (DeMaria 1996; Tang and Emanuel 2012b). Annulus averages are computed over the annulus region extending 200–800 km from the storm center to capture surrounding conditions. Moisture is another useful predictor for the RI process. Kaplan and DeMaria (2003) found that storms were more likely to undergo RI when they were embedded in moister environments. For our model, storm-center and annulus-averaged high-level relative humidity (RH_500–300mb) and low-level relative humidity (RH_850–700mb) are considered. Instead of convective instability used in DeMaria and Kaplan (1994), we considered storm-centered and annulus averages of conditional instability defined as the vertical gradient of saturated equivalent potential temperature (dθ_e/dz). Storms that undergo RI often show indications of intensity changes at previous time steps, and previous changes in intensity (dS/dt₆, dS/dt₁₂, dS/dt₁₈, and dS/dt₂₄) are also included. Finally, the model is global, and every storm has a location feature vector with five Boolean elements to indicate the basin.

To interpret our model more comprehensively in section 5, we introduce two passive features: the norm of vertical wind shear (Shear) and the difference between potential max intensity and current storm intensity (P − S). Experiments demonstrate that the inclusion of these two features does not add skill to our model because these two variables are dependent on other features that we have already included in the model. Therefore, they are not used for model training but only for model interpretation.

c. Data preprocessing

To address the data insufficiency problem, we first split each storm into overlapping 48-h time chunks. This allows multiple training examples to be yielded from each storm. This method is called “rolling window with overlapping,” and is illustrated in Fig. 1. A storm’s lifetime consists of four components in this figure: time, environment features, storm features, and intensity. Time defines the time steps in a storm’s lifetime, and intensity is used to construct RI/no-RI labels. The rolling windows are overlapping 48-h windows. The data preprocessing starts with a black window covering the period from time step 0 to time step 7. The features at the eight time steps become the first sample of the storm denoted as s₁, and its label y₁ indicating whether RI occurs in the next 24 h is defined as y₁ = 1[(Speed_max11 − Speed_max7) ≥ threshold]; the function 1[⋅] is one when its argument is true and zero otherwise. The RI/no-RI label y₁ is the quantity to be predicted. Thus, the first sample for a given storm will be the features of the first 48 h (8 time steps) of the storm. This sample is then used to predict whether RI occurs in the next 24 h (hour 72 of the storm). In the training dataset, the RI threshold used is 25 kt due to the limited number of RI cases for higher thresholds. When testing the model’s performance for different thresholds, thresholds of 25, 30, or 35 kt can be used. Pairs of consecutive windows are then shifted by one time step of 6 h. Thus, the window following the black window is the red one. The red window contains the features from time step 1 to time step 8, which become the second sample s₂ with its associated label y₂ = 1[(Speed_max12 − Speed_max8) ≥ threshold]. Similarly, the green window covers the third sample s₃ with label y₃ = 1[(Speed_max13 − Speed_max9) ≥ threshold]. Following this procedure, the windows roll to the end of the storm’s lifetime, and those samples without 4 future time steps are not included in our dataset since we do not have access to their RI labels. In the end, each storm record is divided into multiple overlapping samples. Every sample s_i consists of the features during eight time steps: $s_i=\left\{{\mathbf{x}}_{j_i},\dots ,{\mathbf{x}}_t,\dots ,{\mathbf{x}}_{j_i+7}\right\}$, where we call the vector x_t of length d a “description” that denotes the features at time step t, and d is the number of features included in the single time step’s description. The term j is a list whose ith element j_i indicates the starting time point of sample s_i in a storm. In what follows, d = 36, and it will sometimes be convenient to consider x_t as a row vector of length 36, which is the number of features (30 storm/environment features and the location feature vector of length 5; one of the environment features, Landmask, occupies two Boolean elements to indicate two types of storm location: land or ocean).

View Full Size

Rolling window data preprocessing.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide

d. Data balance

To cope with the data imbalance issue (RI is relatively rare), we applied the synthetic minority-oversampling technique (SMOTE; Chawla et al. 2002) on the preprocessed dataset. For the convenience of visualization, Fig. 2 illustrates the underlying idea of SMOTE in a two-dimensional space instead of in the original high-dimensional feature space.

View Full Size

For the convenience of visualization, high-dimensional feature space is represented in a two-dimensional space. (a) Two sample distributions, where blue points represent non-RI cases and yellow points represent RI cases. (b) A RI case is chosen denoted as a black point, and it is connected to its two nearest neighbors denoted as green points by dashed lines. (c) Two RI cases denoted as red points are oversampled on those dashed lines.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide

Figure 2a shows two sample distributions, with yellow points representing RI cases and blue points representing non-RI cases. The rarity of RI cases is reflected by the smaller number of yellow points compared to the number of blue points. To apply SMOTE, we randomly choose an RI case denoted as the black point in Fig. 2b. Second, the chosen RI case is connected with its nearest RI neighbors denoted as green points, which are determined by Euclidean distance in the feature space. The number of nearest neighbors used is a hyper-parameter of this technique, which can be adjusted to achieve the best performance. In our task, experiments (validation data) demonstrate that five is the best choice. For simplicity, only two green points are drawn in Fig. 2b. Last, a point is linearly interpolated randomly on each connected line, and they are regarded as oversampled (artificial) RI cases (red points in Fig. 2c). Following the above procedures, the original unbalanced dataset consisting of 4231 RI cases and 47 398 no-RI cases was balanced to form a dataset with 47 398 cases for both RI and no-RI.

a. Model structure

The model used for RI prediction has two parts: long short-term memory (LSTM) and classifier as shown in Fig. 3. LSTM is used as a feature extractor. A sample s_i with its eight descriptions $\left\{{\mathbf{x}}_{j_i},\dots ,{\mathbf{x}}_t,\dots ,{\mathbf{x}}_{j_i+7}\right\}$ is fed to the LSTM one by one in time sequence. After inputting a description x_t, the LSTM’s current state h_t is generated. The state vector h_t has length k where k is the size of the hidden state and takes information not only from current time step’s description x_t but also from the previous time step’s state h_t−1. As descriptions are input time sequentially, the state of the LSTM updates, which allows LSTM to capture the temporal characteristics of the input samples. After feeding in all the descriptions in sample s_i, LSTM outputs the final state ${\mathbf{h}}_{j_i+7}$. This state is a condensed representation of s_i and should include the most important information for predicting RI. In other words, ${\mathbf{h}}_{j_i+7}$ is the feature vector that LSTM extracts from s_i, summarized as ${\mathbf{h}}_{j_i+7}=\text{LSTM}\left(s_i\right)$.

View Full Size

The model consists of two parts: LSTM and classifier. A sample s_i with its eight descriptions $\left\{{\mathbf{x}}_{j_i},\dots ,{\mathbf{x}}_t,\dots ,{\mathbf{x}}_{j_i+7}\right\}$ is fed to the LSTM. As descriptions are input to LSTM one by one, LSTM’s current state updates from ${\mathbf{h}}_{j_i}$ until ${\mathbf{h}}_{j_i+7}$. Then ${\mathbf{h}}_{j_i+7}$ as an extracted feature vector is fed to the classifier and RI probability is finally obtained.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide

Then the extracted feature vector is input to the model’s second part, which is the classifier that will output the probability of RI. Here, the classifier is a one-layer fully connected neural network (Jain et al. 1996), which is comprised of a neuron. It computes a linear combination of the extracted feature from the LSTM. The weights in the linear combination are learned as in a linear regression model; then the neuron applies a nonlinear activation function—sigmoid activation function. The function forces the output to be in the range from 0 to 1, which is the probability P(RI_i) of RI (25-kt threshold) occurring in the following 24 h based on the sample s_i: P(RI_i) = Classifer[LSTM(s_i)]. RI probabilities for thresholds of 30 and 35 kt are computed in an offline calibration procedure by logistic regression which will be discussed in section 3e.

b. Long short-term memory

The ability of LSTM to encode temporal relationships comes from its unique gating mechanism, which makes it suitable for temporal extrapolation and temporal feature extraction. In short, LSTM extracts the feature ${\mathbf{h}}_{j_i+7}$ from the input sample s_i, and thus LSTM can be viewed as a complicated function with ${\mathbf{h}}_{j_i+7}=\text{LSTM}\left(s_i\right)$. In this section, we describe the basic structure and mathematics of the LSTM and refer readers to Goodfellow et al. (2016) for further details. Readers who are not interested in LSTM details can skip this section.

To understand its feature extraction details, LSTM’s structure is shown in Fig. 4. In addition to the state variable h_t, LSTM has a memory cell variable ${\mathbf{c}}_t\in {\mathbb{R}}^{1\times k}$ to store information from the storm descriptions. LSTM has three gates that control the stream of state information: a forget gate $F\in {\mathbb{R}}^{1\times k}$, an input gate $I\in {\mathbb{R}}^{1\times k}$, and an output gate $O\in {\mathbb{R}}^{1\times k}$. Those gates at time step t are updated by the following equations:

The sigmoid is an activation function mapping from $\mathbb{R}$ to (0, 1); the $\mathbf{W}\in {\mathbb{R}}^{d\times k}$ and $\mathbf{U}\in {\mathbb{R}}^{k\times k}$ are weight matrices, and the $\mathbf{b}\in {\mathbb{R}}^{1\times k}$ are bias terms; “⋅” is the usual matrix multiplication. In the LSTM unit, the forget gate and the input gate use the current storm description x_t and the previous state h_t−1 to control the information kept in the memory cell c_t as described by the equation:

where ⊙ is the Schur product (element-wise multiplication), and $\widetilde{{\mathbf{c}}_t}$ is a memory cell candidate. The forget gate F controls how much information from the last memory cell to forget, and the input gate I decides how much information from the candidate memory cell to keep. The memory candidate is computed from

where again the ${\mathsf{W}}_{xc}\in {\mathbb{R}}^{d\times k}$ and ${\mathsf{U}}_{hc}\in {\mathbb{R}}^{k\times k}$ are weights, ${\mathbf{b}}_c\in {\mathbb{R}}^{1\times k}$ is a bias term, and tanh is the activation function. The output gate O_t is the last gate and controls the state update by determining how much information from the memory cell goes into the new state: h_t = O_t ⊙ tanh(c_t).

View Full Size

Long short-term memory structure. The terms I, F, and O indicate the input gate, forget gate, and output gate, respectively. The terms x_t−1, x_t, and x_t+1 represent consecutive three time steps’ descriptions. The terms h and c are a time step’s state and memory cell. The tanh is the hyperbolic-tangent activation function, and σ is the sigmoid activation function. The × and + marks mean element-wise multiplication and addition, respectively.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide

c. Model training

Before the model can be used for RI prediction, its parameters must be estimated using the training dataset, which are the data from the period 1981–2009. The parameters consist of two sets: the parameters in LSTM and the parameters of the classifier. As mentioned before, we estimate the model parameters for the case where RI is defined as intensification greater than 25 kt in 24 h since there are substantially fewer RI cases at higher thresholds. In the model training, the first step is to initialize the parameters of the LSTM and the classifier with random values. Second, the model takes in training data samples s_i and outputs predicted RI probability P(RI_i) for all i. Third, the distance between the predicted RI probabilities and the RI labels y_i is measured by a loss function, which is logarithm loss in our case. Optimization of the loss function yields the model parameters. The training steps can be expressed by the following formula:

where θ refers to the trainable parameters from both the LSTM and the classifier. N is the number of samples, and L is the loss function used to measure the difference between predictions and observations. Because our task is a standard binary classification, the loss function used is logarithm loss by convention, which is often called cross-entropy loss (De Boer et al. 2005) in machine learning and the negative of the log or ignorance score in weather applications (Roulston and Smith 2002). Explicitly, for a forecast probability p and occurrence y, the cross-entropy loss is

When RI occurs (y = 1), the loss function becomes −logp ∈ [0, ∞) given probability p ∈ [0, 1], which is made small to zero by making p as close as possible to one. When RI does not occur (y = 0), the loss function becomes −log(1 − p) ∈ [0, ∞) given probability p ∈ [0, 1], which is made small by making p as close as possible to zero. As Eq. (4) shows, the parameters θ are adjusted to reduce the average (over all training cases) of the loss function L. There are many optimizers available to solve this optimization problem such as Adam, RMSprop, and Adagrad. Experiments found that RMSprop by Tieleman and Hinton (2012) works well here making training process more stable.

d. Model ensemble

To improve predictive performance further, we trained two models and combined their predictions. The following formula shows the prediction mixture formula:

where M₊ is a positive model trained on a SMOTE-balanced dataset, and M₋ is a negative model trained on the originally unbalanced dataset. Both positive model and negative model have the same structure as shown in Fig. 3 and are trained on 25-kt threshold RI dataset. The best ratio for combining the models for each threshold is obtained from the training dataset of the corresponding RI threshold and summarized in Tables 5–7. Using the ratio associated with each threshold, the ensemble model for each threshold is created. The values of the ratio are quite small for the 25-kt threshold (Table 5), which is an indication of the modest improvement due to using the SMOTE-balanced dataset. The ratios are almost zero for the higher thresholds (Tables 6 and 7), indicating that SMOTE contributes little to ensemble models’ predictions for higher thresholds. Recalling that the two components of the ensemble models are fit using the 25-kt threshold, and that the ratio for each threshold is estimated from the corresponding threshold’s training data, this differing behavior of the ratio depending on threshold indicates that the SMOTE-balanced low-threshold dataset does not capture well the distribution of the higher threshold datasets.

Table 5.

Global and basin values of Brier skill score (BSS) and RI cases for RI threshold of 25 kt.

View Table

Table 6.

As in Table 5, but for a RI threshold of 30 kt. BSSc is the Brier skill score for the calibrated (logistic regression) predictions.

View Table

Table 7.

As in Table 5, but for a RI threshold of 35 kt.

View Table

e. Offline calibration for thresholds of 30 and 35 kt

We explored three approaches for generating predictions of RI for the thresholds of 30 and 35 kt. The first one is to directly use the ensemble model associated with each threshold (30 and 35 kt). However, it is not satisfying since the probability of intensifying 35 kt or more should be less than that of intensifying 25 kt or more, and that is not the case for the two components of the ensemble models, which are trained on 25-kt threshold RI dataset. The second approach is to directly train a new ensemble model on 30/35-kt threshold RI dataset, where two components of each ensemble model are also trained on associated threshold’s dataset. The third approach is to view the probability of intensification greater than 25 kt as an extracted feature, which then is used to predict the probability of exceedance of the higher thresholds via probit transformation and logistic regression. Since the LSTM performs effectively on 25-kt threshold RI dataset, we see it as having extracted all predictive information from input storm features and environment features and transformed it into the predicted probability. That is, the predicted probability includes all the predictive information from inputs. This is the basis for using the prediction probability to fit logistic regressions. Although the achieved prediction performance by the third approach is comparable with that of the second approach, the third approach is preferred for two reasons. The first reason is that the interpretation of LSTM model is complicated. Using the third approach means that only a single pair of LSTM models needs to be interpreted. The second reason is that LSTM models are more computationally expensive to train than logistic regression models.

In the third approach, we fit a logistic regression for each basin whose input is the probit-transformed 25-kt exceedance probabilities and whose outputs are the probability of exceeding 30- and 35-kt thresholds on unbalanced training datasets. Probit transformation is defined as $\sqrt{2}{\mathrm{erf}}^{-1}\left[2P\left(\text{RI}\right)-1\right]$. The primary reason for the probit transformation is to convert the probability input to have larger range so that logistic regression output has the full range 0–1 as it should. Although the number of cases is small, the number of parameters to be fit in logistic regressions is substantially smaller—two for each basin and threshold.

a. Skill scores

The ensemble models are evaluated on independent (testing) storm data during the period 2014–17. The testing data are not used in any part of the development of the ensemble models. To have a better sense of our models’ performance, the quality of the probabilistic predictions is measured by the Brier skill score (BSS; Brier 1950). Positive values of BSS indicate that the RI predictions are better than reference predictions, which means that mean squared error between RI predictions and observations is smaller than the error between reference predictions and observations, while negative values would indicate the opposite. The reference prediction is the basin-dependent RI base rate, computed from storm data during the period 1981–2009. BSS values were computed globally and by basins for each RI threshold. BSS values are positive globally, in all basins, and for all three RI thresholds (Tables 5–7). BSS values decrease in all basins as the RI threshold increases, likely in part due to the two components of ensemble models being trained on the 25-kt threshold RI dataset, though the Brier score has been noted to depend on event frequency (Stephenson et al. 2008). Comparing BSS values across basins at the same threshold, our model performs best in eastern North Pacific and northern Indian Ocean basins, and worst in the Southern Hemisphere. The performance in Atlantic is the most stable one as the RI threshold increases because most RI cases in Atlantic are above high thresholds. Over half of the 25-kt RI cases there are 35-kt RI cases.

A natural question is how the performance of the ensemble models compares with that of previously proposed methods and real-time predictions. First, we compare the BSS of the ensemble models in Atlantic and eastern North Pacific with that of previously published methods, noting that this is an imperfect comparison since different independent (test) periods are used. Then we compare the BSS of the ensemble models with that of real-time predictions on the same period.

Kaplan et al. (2010) derived two RI prediction models and computed their BSS on independent data during 2006 and 2007. For Atlantic basin, the best model’s BSS values were 0.068, 0.165, and 0.145 for thresholds of 25, 30, and 35 kt, respectively. These BSS values in independent data were substantially lower than the ones in the training data, which can be an indication that training data and test data have dissimilar distributions or that there is overfitting. For the eastern North Pacific, the best model’s BSS values in independent data (2006–07) were 0.167, 0.14, and 0.036. For both basins, the BSS values of the ensemble models here are similar or higher, but are computed on the different independent period (2014–17). Rozoff et al. (2015) developed four models for RI prediction, with some including passive microwave predictors, and assessed their performance on independent data from 2004 to 2013. For the Atlantic basin, the best model’s BSS values were 0.205, 0.175, and 0.097. For the eastern North Pacific, the best model’s BSS values on independent data (2006–07) were 0.227, 0.168, and 0.13. Again, for both basins, the BSS values of the ensemble models here are similar or higher, but are computed on the different independent period (2014–17). The best models in Rozoff et al. (2015) were those that used passive microwave predictors, which agrees with Kieper and Jiang (2012) and suggests additional room for improvement in the model here. Overall, the above comparison demonstrates that the ensemble models have a level of prediction skill that is comparable with that of previously proposed methods.

To exclude the effect of different testing periods on model evaluation, we compare the ensemble model predictions with SHIPS real-time predictions for the same period 2014–17. One advantage we have over SHIPS is that we are using the best track and reanalysis data while SHIPS is using the working best track and the analysis/forecast fields that are available in real time. On the other hand, SHIPS uses more predictors and high-frequency data instead of monthly averages. More predictors and high-frequency data provide the SHIPS model with more comprehensive information regarding the storm and its surrounding environment, which gives it better a chance to make correct predictions. Overall, it is not immediately clear whether the model here is at an advantage or disadvantage. The details of the comparison between our model predictions with SHIPS real-time predictions are shown in Fig. 5 for the Atlantic and eastern North Pacific. For the 25-kt threshold, our ensemble model’s performance is better than the SHIPS real-time predictions in both the Atlantic and eastern North Pacific. To evaluate the significance of this relationship, we run a sign test on the 25-kt threshold Atlantic and eastern North Pacific test datasets (DelSole and Tippett 2014). The sign test uses the Brier score as the evaluation criterion, and the null hypothesis is that our ensemble model has the same performance as SHIPS for the 25-kt threshold. The resultant p values are 1.463 × 10⁻²¹ and 1.242 × 10⁻⁷⁷ for the Atlantic and eastern North Pacific, respectively, which is a strong indication of statistical significance. As the RI threshold increases, there is some obvious degradation of our models’ performance. For the RI threshold of 35 kt, the performance of the ensemble model is slightly worse than the SHIPS real-time predictions. A similar sign test is performed on the 35-kt threshold Atlantic and eastern North Pacific test datasets. The resultant p values indicate the relationship is not as statistically significant as the relationship on the 25-kt threshold Atlantic and eastern North Pacific test datasets. Figure 5 also shows that the two calibrations for thresholds of 30 and 35 kt have overall good performance. In eastern North Pacific, the BSS increases from 0.264 (0.126) to 0.298 (0.255) for thresholds of 30 (35) kt. The BSS for the 35-kt threshold more than doubles. The calibrated models outperform SHIPS for the 30- and 35-kt thresholds in the eastern North Pacific significantly. Similar sign tests are also performed to evaluate the relationship for the 30 and 35-kt thresholds in the eastern North Pacific. The resultant p values are 2.222 × 10⁻¹⁰² and 7.085 × 10⁻¹¹⁷ for the 30 and 35-kt thresholds, again strong indications of statistical significance. In the Atlantic, BSS goes from 0.172 (0.125) to 0.163 (0.149) with calibration for the 30- and 35-kt thresholds, respectively. The BSS in the Atlantic for 30 kt suffered from slight degradation. One possible reason for the large increase of BSS in eastern North Pacific but the slight decrease of BSS in Atlantic is that the environment climatology in the eastern North Pacific during 2014–17 is similar to that during 1981–2009, while the environment climatology in Atlantic during 2014–17 is different.

View Full Size

BSS comparison between our proposed method and SHIPS real-time prediction model in Atlantic and eastern North Pacific. BSS values are computed from storm data of the same time period (2014–17). The blue column and yellow column are the BSS values of the SHIPS model in the Atlantic and eastern North Pacific, respectively. The orange column and purple column represent our ensemble models’ performance in the Atlantic and eastern North Pacific without calibration (logistic regression), respectively. The gray column and green column show our ensemble models’ BSS values after calibration in the Atlantic and eastern North Pacific, respectively.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide

Besides BSS, we also evaluate our models via the reliability diagram and performance diagram. In Fig. 6, most of the predicted probabilities are in the range from 0 to 0.4. This is reasonable because RI is such a rare event. As the threshold goes up, the number of mean predicted values at the range from 0 to 0.1 increases, which means ensemble models are correctly calibrated to lower prediction probabilities for 30- and 35-kt thresholds. In the upper panel of Fig. 6, the blue line (for the 25-kt model) is right on top of the dashed line, while the orange line and green line (for the 30- and 35-kt models) are always under the dashed line, suggesting that the ensemble models’ performance degrades as threshold increases. This is reasonable as two of the models’ components are trained on 25-kt threshold. Conversely, the red and purple lines (the calibrated 30- and 35-kt models) are above the dashed line indicating underconfidence. Overall, the reliability of the forecast probabilities is fairly good, and the logistic regression procedure has a positive effect on improving ensemble models’ performance for high thresholds.

View Full Size

(top) Reliability diagram for five models in global scale: ensemble models for 25-, 30-, and 35-kt thresholds and calibrated ensemble models for 30- and 35-kt thresholds denoted as blue, orange, green, red, and purple. Only squares associated with bins to which over 50 predictions belong are plotted. The dashed line represents the perfect reliability as standard. (bottom) The prediction count diagram. The five models are denoted using the same colors as the top panel.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide

Figure 7 summarizes the probability of detection (POD), success ratio (SR), and critical success index (CSI), and bias for our models. For uncalibrated ensemble models, the ensemble model for the 25-kt threshold (the red dot) basically has the best performance. It gives better scores for SR, CSI, and bias than other uncalibrated ensemble models. As the threshold increases, uncalibrated ensemble model’s performance degrades, which is indicated by yellow dot and green dot being located farther to the upper left than the red dot. Comparing models for the same threshold before and after calibration, the calibrated model achieves better performance. For example, the calibrated model for the 30-kt threshold (the orange dot) gives better CSI, bias, and SR than the corresponding uncalibrated model (the green dot). This illustrates that the calibration can helps to improve model’s discriminant performance.

View Full Size

The performance diagram summarizes critical success index (CSI), success ratio (SR), probability of detection (POD), and bias. Dashed lines represent bias scores, while colored contour fields are CSI. The performance of five models in global scale: ensemble models for 25-, 30-, and 35-kt thresholds and calibrated ensemble models for 30- and 35-kt thresholds, which are represented by red, green, yellow, orange, and pink dots, is demonstrated on this diagram. Each dot is chosen by maximizing the difference between POD and false alarm ratio (FAR) for each model.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide

b. Case studies

To give an indication of the character of the ensemble model predictions, we show four case studies from the test data (Fig. 8). They are three hurricanes (Maria, Harvey, Patricia) and one typhoon (Meranti). For the Atlantic and eastern North Pacific cases we also show the SHIPS real-time predictions. Overall, our model predicts higher probabilities when RI is observed than when RI is not observed. When comparing performance among the four cases, we notice that our model has better performance on Maria, Patricia, and Meranti than on Harvey. One possible reason is that Maria, Patricia, and Meranti all have relatively steady intensification process which provides useful time sequence information to predict intensity changes. Since intensity change (derivative of intensity with respect to time) plays an important role in RI prediction, it is plausible that RI of storms with steady intensification processes can be better predicted. Like our model, the SHIPS model produces higher probabilities when RI is observed. However, our model predicts lower RI probabilities than SHIPS when RI is not observed, especially for Maria and Harvey. Most of the green patches, representing the predictions of our model, are under the purple lines which indicate the SHIPS predictions. The Patricia figure demonstrates there is a delay between the SHIPS RI predictions and RI occurrences, while our ensemble model predicts RI promptly. These cases are consistent with our model being more skillful (higher BSS) than the SHIPS model in these basins.

View Full Size

Probabilistic RI predictions from the ensemble model (green) for Hurricanes Maria, Harvey, Patricia and Typhoon Meranti. Black lines show the observed storm intensity and gray patches label when RI greater than 30 kt occurs. Purple lines are SHIPS real-time RI probabilistic predictions for Maria, Harvey, and Patricia for the 30-kt threshold. There is no SHIPS real-time prediction for Typhoon Meranti. The x axis represents the lifetime of the hurricanes and typhoon. The inset figures show storm tracks.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide

Composite feature images

To interpret the basic behaviors of the model, we examine composite feature images associated with RI and non-RI cases for the 25-kt threshold. The positive composite feature image (Image_p) and the negative composite feature image (Image_n) are averages of the samples when the model predicts the probability of RI to be greater than the cutoff and when the model predicts the probability of RI to be less than the cutoff, respectively. The cutoff is chosen from the receiver operating characteristic (ROC) curve in the training dataset by maximizing Peirce score, which is the difference between true positive rate and false positive rate. Composite images are defined by the following two equations:

where N₁ and N₂ are the numbers of samples predicted as RI cases and non-RI cases, respectively. Image_p describes how storms behave on average in the 48 h before RI is predicted. Image_n describes how storms behave on average in the 48 h before RI is not predicted. These composite images can be computed globally and by basin. The difference image Image_d = Image_p − Image_n provides information as to how, on average, the ensemble model discriminates RI cases from non-RI cases. Features are standardized to have zero mean and unit variance. Figure 9 shows the difference image for each basin and for the globe.

View Full Size

Image difference (Image_d = Image_p − Image_n) for each basin and for the globe. The eight columns of each image correspond to eight 6-h time steps. Each of the 31 rows shows the time evolution of one feature during the 48 h prior to prediction. The rows are ordered by each feature’s mean magnitude over the 48 h period and vary by basin.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide

The Atlantic difference image shows that storms in Atlantic that are predicted to undergo RI move into environments with higher ocean temperatures and are intensifying faster than those that are not predicted to undergo RI, as indicated by ocean temperature stripes and intensification rate stripes that become redder over the 48-h period prior to prediction. This conclusion is consistent with Rozoff et al. (2015). The predictor P − S is positive, which agrees with the work by Kaplan and DeMaria (2003) showing the intensity of storms undergoing RI are farther away from their potential maximum than those not going through RI. The P − S stripe becomes a lighter red near the end of the 48-h period, which is physically reasonable as well. Storms often undergo small intensification before RI. The Shear stripe is constantly blue at the bottom of the image and demonstrates that the storms tend to be in low vertical wind shear situations before RI, in agreement with Tang and Emanuel (2012b). Rows corresponding to moisture variables are red and near the top of the image, reflecting the known preference of moister environments for RI (Kaplan and DeMaria 2003). The conclusions from the Atlantic are nearly the same as for the eastern North Pacific, with ocean temperatures and upper-level outflow temperatures being exceptions. There are no pronounced differences in these temperatures between RI cases and non-RI cases in eastern North Pacific, which may reflect the differing environments of the basins. Another notable characteristic in the eastern North Pacific image is that divergence stripes are redder than those in Atlantic image. It is not surprising for the Div_200mb stripe to be red because higher divergence is favorable for RI (Rozoff et al. 2015). The redness of the Div_200mb stripe indicates that divergence is an excellent feature to discriminate RI cases from non-RI cases for storms in eastern North Pacific. Interestingly, Rozoff et al. (2015) did not use this predictor for their model in eastern North Pacific.

In the northern Indian Ocean, storms are more likely to undergo RI in environments that are relatively more humid and have larger 200-hPa divergence. The red strips in Fig. 9 for these two variables become less red with increasing time step, meaning that the values of divergence and humidity tend to decrease (but are still anomalously positive) prior to RI. The reddish strip for $T_{{\text{ocean}}_{0-100\text{m}}}$ and the almost white Shear stripe do match our understanding that RI is more likely to occur over warm ocean and under low shear, but the closeness to white color suggests that they are not good discriminators in the basin. The changes in the storm intensity and storm intensity itself are always good indicators of RI. Similar results are seen in the Southern Hemisphere and northwestern Pacific Ocean with weaker anomalies, except for variables associated with storm intensity.

Since each basin has its own climatological distribution of environments, the predictor pattern favorable for RI is also different. Colors of some variables are mildly inconsistent across basins, such as dθ_e/dz, the conditional instability variable. To compare the difference images, the pattern correlations between each basin’s difference images are computed via cosine similarity (Fig. 10). The lowest similarity is between the difference images for the Atlantic and northern Indian Ocean. To compare the image differences for these two basins in more detail we plot them with the same feature ordering (Fig. 11). There are a few notable points. First, the stripes for the shear variables are bluer in the Atlantic difference image than in the northern Indian Ocean difference image. When computing the average of Shear over eight time steps, the value of Atlantic image is less than that of northern Indian Ocean image by 0.55. Shear is a less informative feature for predicting RI in the northern Indian Ocean than in the Atlantic. Second, ocean temperature stripes in Atlantic difference image are redder than in the northern Indian Ocean difference image. For example, the average of $T_{{\text{ocean}}_{0-100\text{m}}}$ over eight time steps for Atlantic is 0.52 greater than that for northern Indian Ocean, indicating that in the Atlantic relatively higher ocean temperature is more favorable for RI. This may be because the gradient of ocean temperature is weaker in the Indian Ocean due to both the limited latitude expansion and the deep ocean circulations. Third, humidity variable stripes are redder in northern Indian Ocean image. The northern Indian Ocean averages of RH_850–700mb, $\overline{{\text{RH}}_{850-700{\text{mb}}_a}}$, RH_500–300mb, and $\overline{{\text{RH}}_{500-300{\text{mb}}_a}}$ are greater than Atlantic ones by 0.06, 0.04, 0.16, and 0.26, respectively. This shows that a moister environment is informative for predicting RI in the northern Indian Ocean. Interestingly, the difference between potential intensity and storm intensity, which is a good discriminator for RI in the Atlantic, is much less so on average in the northern Indian Ocean.

View Full Size

Pattern correlation between the global and basin difference images via cosine similarity.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Image difference (Image_d = Image_p − Image_n) for the Atlantic and northern Indian basins. Features (rows) are in the same order to highlight differences.

Citation: Weather and Forecasting 35, 4; 10.1175/WAF-D-19-0199.1

• Download Figure
• Download figure as PowerPoint slide