Abstract

Ensemble sensitivity is often a diagonal approximation to the multivariate regression, leading to a simple univariate regression. Comparatively, the multivariate ensemble sensitivity retains the full covariance matrix when computing the multivariate regression. The performances of both univariate and multivariate ensemble sensitivities in multiscale flows have not been thoroughly examined, and the demonstration of the latter in realistic applications has been sparse. A high-resolution ensemble forecast of Typhoon Haiyan (2013) is used to examine the performances of the two ensemble sensitivities. Compared to the multivariate sensitivity, the univariate sensitivity overestimates the forecast metric, especially at higher levels. To increase the predicted Haiyan’s intensity, multivariate ensemble sensitivity gives initial perturbations characterized by a warming area around the center of the storm, an increased moisture area around the eyewall, a stronger primary circulation around the radius of maximum wind, and stronger inflow at low levels and stronger outflow at high levels. Perturbed initial condition experiments verify that the predicted response from the multivariate sensitivity is more accurate than that from the univariate sensitivity. Therefore, the ability of multivariate sensitivity to provide more accurate predicted responses than the univariate sensitivity has been demonstrated in a realistic multiscale flow application.

1. Introduction

Forecast sensitivity analysis provides an objective and quantitative way to evaluate the impact of changes in initial conditions on subsequent forecasts. The changes to initial conditions are often linear, based on a scalar metric of the forecast variables (Ancell and Hakim 2007; Torn and Hakim 2008). The forecast sensitivity analysis provides a basis for understanding the dynamics of the forecast error, and an accurate evaluation of the sensitivity can point out key variables associated with critical regions from which additional observations may be collected to improve the forecast, as measured by the forecast metric (e.g., Torn and Hakim 2008).

Traditionally, forecast sensitivity analysis is accomplished by perturbing the state variables or regions of interests, advancing the model forward, and evaluating the forecast responses. However, tremendous efforts and computational costs would be spent on rerunning the model, since choices of the perturbed state variables or regions are often subjective and sometimes difficult to obtain. Therefore, objective sensitivity analysis approaches, including the adjoint sensitivity (e.g., Errico and Vukicevic 1992) and ensemble sensitivity (e.g., Hakim and Torn 2008), have been proposed and widely used.

Adjoint sensitivity utilizes a tangent linear model and its adjoint to analyze the sensitivity measured by a forecast metric with respect to all variables through a backward integration (e.g., Errico and Vukicevic 1992; Langland et al. 1995; Errico 1997; Doyle et al. 2014). For instance, Xiao et al. (2008) developed an adjoint model for the adiabatic simplified Weather Research and Forecasting (WRF) Model and applied it for sensitivity analysis of a 24-h windstorm event in Antarctica. The adjoint sensitivity has been applied to tropical cyclones, and previous results showed that small perturbations to the initial state provided by the adjoint sensitivity could improve the prediction of intensification of storms (e.g., Chu et al. 2011; Doyle et al. 2012). However, adjoint sensitivity has difficulties in assumed linearity, and coding the adjoint of a tangent linear model that is especially challenging for irreversible processes.

Ensemble sensitivity, as a linear regression method, uses sample statistics from ensemble forecasts to estimate relationships between a forecast metric and changes in initial conditions (Hakim and Torn 2008). Typically ensemble sensitivity calculates the response to a forecast metric resulted from each state variable on a grid independently, and neglects the covariances across state variables. This is referred as univariate ensemble sensitivity. Ancell and Hakim (2007) compared univariate ensemble sensitivity with adjoint sensitivity for a wintertime flow pattern, and found that univariate ensemble sensitivity provided accurate estimates of the impact of initial condition changes on a forecast metric. Since then, univariate ensemble sensitivity analysis has been widely used to understand the dynamics of forecast errors and improve the forecasts for synoptic-scale extratropical cyclones (e.g., Garcies and Homar 2009; Chang et al. 2013), mesoscale events including the African easterly waves (e.g., Torn 2010) and tropical cyclones (e.g., Brown and Hakim 2015; Rios-Berrios et al. 2016a,b), and convective-scale events (e.g., Hanley et al. 2013; Bednarczyk and Ancell 2015).

However, univariate ensemble sensitivity can overestimate the response of a forecast metric to initial perturbations, which possibly results from sample error (Torn and Hakim 2008). To remedy this overprediction, Hacker and Lei (2015) proposed the multivariate ensemble sensitivity that accounts for collective contributions to the forecast metric from all the initial perturbation variables simultaneously. Using the two-scale model of Lorenz (2005), multivariate ensemble sensitivity analysis had superior skill in predicting the response when compared to univariate ensemble sensitivity analysis with diagonal approximation of the covariance, particularly when fast scales and model errors were presented, and observations were sparse (Hacker and Lei 2015). Limpert and Houston (2018) applied multivariate regression on each grid point in an idealized supercell simulation. But their multivariate sensitivity analysis showed difficulties to identify physically meaningful variables, possibly because the contributions of the covariances from variables at adjacent grid points are neglected. Therefore, the multivariate ensemble sensitivity that accounts for multicollinearity among perturbation variables can be an effective method to estimate forecast responses to initial perturbations.

Application of ensemble sensitivity analysis to high-resolution forecast problems has been sparse. Previous research explored the multivariate ensemble sensitivity analysis is based on simple models and idealized simulations, so demonstration of multivariate ensemble sensitivity in realistic simulations, especially for multiscale flow like tropical cyclones, is necessary. Thus this work examines multivariate ensemble sensitivity analysis in a high-resolution ensemble simulation for the Super Typhoon Haiyan (2013). By comparing the univariate and multivariate ensemble sensitivity analyses, the potential deleterious effects of the diagonal approximation in univariate ensemble sensitivity are analyzed in this real case. Perturbed initial condition experiments are used to evaluate the univariate and multivariate ensemble sensitivities.

The remainder of this paper is organized as follows. Section 2 describes the methodologies of the univariate and multivariate ensemble sensitivities. Section 3 gives a brief introduction of Typhoon Haiyan. Experimental design of the ensemble sensitivity test is explained in section 4. Results of the univariate and multivariate ensemble sensitivities are discussed in section 5, and the perturbed initial condition experiments are provided in section 6. Section 7 gives the summary and conclusions.

2. Ensemble sensitivities analysis

Univariate ensemble sensitivity, a common approach of ensemble sensitivity analysis, is typically performed as a linear regression between a single perturbation variable x and forecast response variable J with the sensitivity measured by the slope (e.g., Ancell and Hakim 2007; Torn and Hakim 2008). Thus univariate ensemble sensitivity counts the impact of each model state variable from a single grid point on the forecast metric, neglecting contributions from collocated state variables and adjacent grids. The linear regression model can be written as

 
J=bx+ε,
(1)

where b and ε are the slope (i.e., regression coefficient) and residual, respectively. Given an ensemble forecast that provides samples of response functions and model state variables, it is natural to estimate the slope within an ensemble context. Thus, univariate ensemble sensitivity becomes a statistical linearization of Eq. (1), in which ensemble means Je and xe are used.

Let Je, a K × 1 vector where K is ensemble size, denote the forecast response vector collecting elements Je,k from the kth ensemble member with ensemble mean removed. Let x denotes a state vector that has dimension of P × 1, where P is the dimension of the state vector; and xpe, a 1 × K vector, presents the ensemble perturbations of the pth state variable. The linear sensitivity can be calculated with the ensemble statistics. By solving the ordinary least squares (OLS) equation, the change in x needed to produce a given change in J is solved, which leads to the ensemble sensitivity:

 
bp=Jxp=cov(Je,xpe)var(xpe),p=1,2,,P.
(2)

The sensitivity multiplied by an expected analysis increment from a new or hypothetical observation in a data assimilation system can provide an estimate of the forecast response by assimilating that observation. Assume a hypothetical observation that is the same type as the pth state variable and leads to an analysis increment equal to the ensemble spread σp, the corresponding change of the forecast response function can be written as

 
δJpu=σp×Jxp,p=1,2,,P,
(3)

in which the superscript u denotes univariate ensemble sensitivity.

As shown by Eqs. (1) and (2), univariate ensemble sensitivity approximates the covariance matrix of x with corresponding diagonal matrix, and it is also affected by sampling error due to a finite ensemble. Thus univariate ensemble sensitivity can provide an overestimated response to an assimilated observation. Hacker and Lei (2015) proposed a multivariate ensemble sensitivity that computes the sensitivity with a multivariate regression that retains the full covariance matrix. The multivariate linear regression problem can be written as

 
J=b1x1+b2x2++bPxP+ε.
(4)

Solving the multivariate regression coefficients yields

 
β=Jx=X(XTX)1Je,
(5)

in which x is state vector, X is a P × K matrix composed by ensemble perturbations x1e,x2e,,xPe in each row, and β, a P × 1 vector, denotes the multivariate ensemble sensitivity.

Given this multivariate ensemble sensitivity, the change of forecast response function resulted from assimilating a hypothetical observation that is the same type as the pth state variable can be obtained by multiplying the associated analysis increment δx to the multivariate ensemble sensitivity ∂J/∂x:

 
δJp=(Jx)Tδx.
(6)

The analysis increment δx, a P × 1 vector, contains ensemble spread σp for the pth state variable, and perturbations of σp regressed onto the other state variables using the ensemble statistics. Elements δx that consists δx can be expressed as

 
δ(xi)={σp,i=p,σp×cov(xie,xpe)var(xie),i=1,,p1,p+1,,P.
(7)

Hacker and Lei (2015) demonstrated that δJp from Eq. (6) is equivalent to δJpu from Eq. (3).

However, finite ensemble sizes result in sampling error in the analysis increment and the ensemble sensitivity estimates. In ensemble data assimilation, sampling error is typically addressed with a localization function. Localized analysis increment has the form of ρ(δx), where ρ is localization function with the same dimension as state vector x, and denotes the Hadamard, or Schur, product. The most commonly used localization function is the Gaspari–Cohn (GC; Gaspari and Cohn 1999) localization function that is a fifth-order piecewise polynomial function of spatial distance. Adaptive localization approaches that are not explicitly distance dependent and require no assumptions about spatial relationships, have been proposed to count for sampling errors between collocated observations and different physical quantities (e.g., Bishop and Hodyss 2009; Lei and Anderson 2014b; Lei et al. 2016). For simplicity, the GC localization function is used in this study.

The estimated change of forecast response function using the multivariate ensemble sensitivity with localized analysis increment can be written as

 
δJpm=(Jx)Tρ(δx),
(8)

in which the superscript m denotes multivariate ensemble sensitivity. Localization shortens δx and influences the predicted forecast response δJpm, so that δJpm is not equivalent to δJpu.

For both univariate and multivariate ensemble sensitivities, an adaptive method is proposed by Hacker and Lei (2015) to mitigate the impact of sampling error on the ensemble sensitivity. Following the Bayesian hierarchical filter (Anderson 2007; Lei and Anderson 2014a), a “localization” factor can be estimated by comparing the regression coefficients among the groups to the mean of the regression coefficients across the groups, which provides one measure of the relative sampling error based on the signal-to-noise concept. When the diversity of regression coefficients among the groups is large relative to the mean, it is noisy and leads to a small localization value, and vice versa. Since significant computational cost is required to operate a group of high-resolution ensemble simulations for a real case, the adaptive localization method for ensemble sensitivity is not applied here.

3. Case overview

Typhoon Haiyan (2013) is the most intense typhoons of the year 2013, and since then it has kept the record of maximum wind speed for tropical cyclones in the western North Pacific Ocean (Velden et al. 2017; Shimada et al. 2018). According to the National Hurricane Center’s (NHC’s) Tropical Cyclone Vitals (TCVitals) database, Haiyan was named at 0600 UTC 4 November with its center positioned at (6.0°N, 150.2°E). In the following three days, Haiyan continued to rapidly intensify while moving northwestward. At 1800 UTC 7 November, its reached peak intensity with maximum wind speed of 87 m s−1 and minimum sea level pressure (SLP) of 895 hPa. A few hours after that, Haiyan passed through the Philippines and caused about 36 000 casualties (NDRRMC 2014) and tremendous economic loss.

Previous studies reported good track prediction but substantial underestimation of intensity forecast for Haiyan (e.g., Islam et al. 2015). Similar results are obtained from the ensemble forecast launched at 0000 UTC 4 November (Figs. 1 and 2). The ensemble forecast captures the movement of Haiyan, although biased to the north. The ensemble mean of the minimum SLP is larger than the observed value. Large ensemble spread of the minimum SLP indicates significant uncertainty in intensity forecast. Several ensemble members intensify to approximately 930 hPa, but still less intense than the observed value. Although the track and intensity ensemble forecasts are biased compared to the observations, the ensemble perturbations in which the ensemble mean is removed are used for ensemble sensitivity analysis [Eqs. (2) and (5)].

Fig. 1.

The track of Typhoon Haiyan. The ensemble mean (blue) and ensemble members (gray) from the ensemble forecasts, compared with the observation (red).

Fig. 1.

The track of Typhoon Haiyan. The ensemble mean (blue) and ensemble members (gray) from the ensemble forecasts, compared with the observation (red).

Fig. 2.

As in Fig. 1, but for the intensity measured by the minimum sea level pressure. The black dashed line indicates 0000 UTC 8 Nov.

Fig. 2.

As in Fig. 1, but for the intensity measured by the minimum sea level pressure. The black dashed line indicates 0000 UTC 8 Nov.

Previous studies found that the intensity forecast of Typhoon Haiyan related with the atmospheric environment, ocean support and other technical details such as horizontal grid resolution and physical parameterization schemes (Lin et al. 2014; Islam et al. 2015; Wada et al. 2018). To understand the factors in atmospheric initial conditions that can improve the intensity forecast, ensemble sensitivity analysis is performed here for Typhoon Haiyan. The impact of changes in initial conditions on subsequent intensity forecast is evaluated, and the comparison between univariate and multivariate ensemble sensitivity analysis is discussed in this real multiscale case.

4. Numerical model and experimental design

The Advanced Research Weather Research and Forecasting (WRF-ARW) Model version 3.4 is used here for ensemble simulation. Domain setup is shown in Fig. 3, in which the outermost domain (D01) is a fixed domain with horizontal grid spacing of 27 km and the other two domains (D02 and D03) are vortex following domains with horizontal grid spacing of 9 and 3 km, respectively. The vortex following domains can provide high-resolution simulation for the vortex structure. The three nested domains have grid numbers of 320 × 270, 198 × 198, and 360 × 360, respectively. There are 56 model levels in the vertical with model top at 10 hPa.

Fig. 3.

Domain settings at the beginning of ensemble forecast. The outermost rectangle is the model domain 1 (D01) with 27-km horizontal grid spacing, while the inner domains (D02 and D03) are the vortex following domains with horizontal grid spacing of 9 and 3 km, respectively.

Fig. 3.

Domain settings at the beginning of ensemble forecast. The outermost rectangle is the model domain 1 (D01) with 27-km horizontal grid spacing, while the inner domains (D02 and D03) are the vortex following domains with horizontal grid spacing of 9 and 3 km, respectively.

Regarding model physics, the Rapid Radiative Transfer Model (RRTM) longwave scheme (Mlawer et al. 1997) and the RRTM shortwave scheme (Iacono et al. 2008) are used. The Unified Noah land surface model (Ek et al. 2003), the Yonsei University (YSU) planetary boundary layer (PBL) scheme (Hong et al. 2006), and the WRF single-moment (WSM) 6-class microphysics scheme (Hong et al. 2004) are adopted. The cumulus parameterization uses the modified Tiedtke cumulus scheme (Zhang et al. 2011), which is only applied in the 27-km D01.

The Global Forecast System (GFS) analysis data of 0.25° resolution from the National Centers for Environmental Prediction (NCEP) is interpolated and perturbed to provide the ensemble lateral boundary conditions (LBCs) every 6 h. The fixed-covariance perturbation technique of Torn et al. (2006) is used to produce random perturbations that sample the NCEP background error covariance by the use of the WRFDA-3DVAR (Barker et al. 2012) for the ensemble LBCs. Ensemble initial conditions (ICs) at the starting date 0000 UTC 1 November are generated in a similar manner to the ensemble LBCs. The ensemble ICs later than the starting date are generated by a cycling ensemble Kalman filter (EnKF) system [for details refer to Lei et al. (2018), manuscript submitted to J. Adv. Model. Earth Syst.]. Initialized from the ensemble analyses at 0000 UTC 4 November, an 80-member ensemble forecast is integrated to 0600 UTC 9 November. The ensemble forecast that covers the peak intensity of Haiyan is then used for ensemble sensitivity analysis.

To conduct ensemble sensitivity analysis, the forecast response function is set to a forecast error of minimum sea level pressure (SLP) [i.e., (SLPfc − SLPobs)], at 0000 UTC 8 November (i.e., the model’s integrated timing of 96 h, at which Haiyan just passed its peak intensity shown by the black dashed line in Fig. 2). Similar results are obtained when the forecast error of maximal wind speed is used as the forecast response function (not shown). State variables, including the potential temperature (T), water vapor mixing ratio (Q), tangential wind (TW, anticlockwise flow is set to be positive), and radial wind (RW, outflow is set to be positive) are selected for perturbation variables. Note that to better understand the physics provided by the ensemble sensitivity analysis, the RW and TW interpolated to the mass grid are chosen here, rather than the zonal and meridional winds. The perturbation dry air mass in column (MU) is also used in δx when computing the multivariate ensemble sensitivity. Localization scales used in the multivariate ensemble sensitivity are 2000 km in the horizontal and 1.5 ln(hPa) in the vertical, which are the same as those used in the cycling EnKF. The perturbation variables are from the 3-km resolution D03 and valid at 24, 48, and 72 h earlier than the verification time of the forecast response function.

To verify the estimated forecast responses from both univariate and multivariate ensemble sensitivities, perturbed initial condition experiments are conducted. Due to computational cost, the “best” member that has predicted SLP closest to the ensemble mean at 0000 UTC 8 November is perturbed. At each lead time, 100 randomly chosen hypothetical observations that have the same type as state variables with analysis increment equal to the ensemble spread are assimilated in the best member. Then forecasts can be integrated from the perturbed states, and the actual forecast responses can be obtained.

5. Results

a. Univariate and multivariate ensemble sensitivities

The ensemble sensitivity is examined from the storm-centered high-resolution D03. Figures 47 show both univariate and multivariate sensitivities of potential temperature (T), water vapor mixing ratio (Q), tangential wind (TW), and radial wind (RW) at different vertical levels with 24-h lead time. Negative (positive) sensitivities indicate that a positive (negative) analysis increment with magnitude of σp at the pth grid point yields a decreased forecast error of minimum SLP (i.e., an intensified storm) 24 h later.

Fig. 4.

(a)–(c) The estimated univariate ensemble sensitivity δJu and (d)–(f) the multivariate ensemble sensitivity δJm of potential temperature (T) with lead time 24 h. (top) Model level whose 500–600 km annular mean pressure closest to 850 hPa, and (middle),(bottom) model levels closest to 500 and 200 hPa, respectively.

Fig. 4.

(a)–(c) The estimated univariate ensemble sensitivity δJu and (d)–(f) the multivariate ensemble sensitivity δJm of potential temperature (T) with lead time 24 h. (top) Model level whose 500–600 km annular mean pressure closest to 850 hPa, and (middle),(bottom) model levels closest to 500 and 200 hPa, respectively.

Fig. 5.

As in Fig. 4, but for water vapor mixing ratio (Q) at (a),(c) 850 hPa, and (b),(d) 500 hPa.

Fig. 5.

As in Fig. 4, but for water vapor mixing ratio (Q) at (a),(c) 850 hPa, and (b),(d) 500 hPa.

Fig. 6.

As in Fig. 4, but for tangential wind (TW).

Fig. 6.

As in Fig. 4, but for tangential wind (TW).

Fig. 7.

As in Fig. 4, but for radial wind (RW).

Fig. 7.

As in Fig. 4, but for radial wind (RW).

The univariate sensitivity of the forecast metric to T at approximately 850, 500, and 200 presents negative values around the center of the storm (Figs. 4a–c). The negative sensitivity of T broadens from a few hundred kilometers in radius to about a thousand kilometers with increasing height, which indicates that warming these areas in the analysis leads to decreased minimum SLP 24 h later. The sensitivity patterns are consistent with a warm core whose strengthening is associated with an intensified storm (Kidder et al. 2000; Wang and Jiang 2019). The multivariate ensemble sensitivity of T has similar sensitivity patterns to the univariate ensemble sensitivity, but with smaller magnitudes (Figs. 4d–f). Please note that the range of color bar for the multivariate ensemble sensitivity at 500 and 200 hPa is smaller than that for the univariate ensemble sensitivity. For a grid near the inner core at 850 hPa, the forecast metrics from univariate and multivariate ensemble sensitivities are about −12.6 and −10.8 hPa, respectively. The forecast metrics become −12.1 and −2.9 hPa at 500 hPa, and −13.3 and −0.4 hPa at 200 hPa. Thus compared to the multivariate ensemble sensitivity, the univariate ensemble sensitivity tends to predict larger forecast responses, especially at upper levels, which is an overestimation of the sensitivity as discussed in section 6. The comparison is consistent with Hacker and Lei (2015) in which the univariate method tends to overestimate sensitivity when fast-moving waves are involved.

Sensitivity to Q is also found mainly near the center of the storm (Fig. 5). The sensitivity patterns show negative sensitivity from the inner core to several hundred kilometers with height increasing from 850 to 500 hPa. Thus moistening these areas in the analysis leads to an intensified storm 24 h later. Regarding the sensitivity magnitude, the univariate ensemble sensitivity overestimates the quantity compared to the multivariate ensemble sensitivity. The minimum forecast metrics at 850 hPa from univariate and multivariate ensemble sensitivities are −10.6 and −8.8 hPa, respectively. The overestimation becomes more substantial at 500 hPa, since the minimum forecast metrics from univariate and multivariate ensemble sensitivities are −10.3 and −2.5 hPa, respectively. At 500 hPa, there are positive sensitivities in the eye, which indicate that drying in the eye at high level is helpful for intensifying the storm.

The sensitivity patterns shown by Figs. 4 and 5 also display positive sensitivity in banded structures outside the storm center. At 850 hPa, the areas of positive sensitivity of T appear northwest of the storm center (Figs. 4a,d), while the areas of positive sensitivity of Q appear north of the storm center (Figs. 5a,c). At 500 hPa, the banded areas of positive sensitivity of T appear at a few hundred kilometers outside of the storm center (Figs. 4b,e), while the areas of positive sensitivity of Q appear mainly southwest of storm center (Figs. 5b,d). These sensitivity patterns implicate that cooling and drying at a few hundred kilometers surrounding the storm center has a positive effect on the storm intensification. Multivariate ensemble sensitivity again produces smaller magnitudes of the banded structures outside the storm center than the univariate ensemble sensitivity.

Both the univariate and multivariate sensitivity patterns of TW in Fig. 6 emphasize the core of the storm, which suggest that to achieve storm intensification 24 h later, the TW should be increased around the radius of maximum wind from 850 to 200 hPa. The sensitivity patterns of RW indicate that stronger inflow in east of storm center at 850 hPa (Figs. 7a,d), stronger inflow in west of storm center at 500 hPa (Figs. 7b,e), and stronger outflow around storm center at 200 hPa (Figs. 7c,f) can lead to an intensified storm 24 h later. This general pattern is partially due to the large spatial correlations between points in the core of the storm center that are influenced by the strong primary circulation. Thus a method of sensitivity that can account for correlations between spatial points is needed. Compared to the multivariate ensemble sensitivity, the univariate ensemble sensitivity overestimates the forecast metric, especially at higher levels.

In general, the univariate and multivariate ensemble sensitivity analyses produce similar sensitivity patterns, while the former overestimates the forecast metric compared to the latter. This indicates that the diagonal approximation of the covariance that is used by the univariate ensemble sensitivity captures the prominent information, but is influenced by the sampling error. As demonstrated by Hacker and Lei (2015), without localization ρ, the multivariate ensemble sensitivity is identical to the univariate ensemble sensitivity. Due to sampling errors resulted from finite ensemble sizes, localization is needed for ensemble data assimilation and ensemble sensitivity analysis. Consequently, the multivariate ensemble sensitivity that counts for sampling errors better predicts the forecast metric than the univariate ensemble sensitivity. Results from Figs. 47 also show that the differences between the univariate and multivariate ensemble sensitivity analyses become larger with increased height. This is possibly due to the broader correlation length scales among state variables at higher levels (Lei et al. 2015). Since a uniform localization length scale is used at different levels, larger differences between the diagonal approximation of the covariance and full covariance are expected at higher levels. Thus the overestimation of the univariate ensemble sensitivity compared to the multivariate ensemble sensitivity is more prominent with increased height.

b. Impact of lead times

The univariate and multivariate ensemble sensitivities at 48- and 72-h lead time are examined in this section. Please note that the forecast response function is fixed at 0000 UTC 8 November, while the perturbation variables are valid at 48- and 72-h earlier than the verification time of the forecast response function. Figure 8 shows the ensemble sensitivity of T, Q, TW, and RW at 850 hPa with 48-h lead time. With 48-h lead time, the univariate ensemble sensitivity still overestimates the forecast metric compared to the multivariate ensemble sensitivity. In general, for both the univariate and multivariate ensemble sensitivities, the sensitivity patterns with 48-h lead time are similar to those with 24-h lead time, but with smaller magnitudes.

Fig. 8.

The estimated ensemble sensitivity of (a),(e) T, (b),(f) Q, (c),(g) TW, and (d),(h) RW at 850 hPa with lead time 48 h. (left) Univariate ensemble sensitivity, and (right) multivariate ensemble sensitivity.

Fig. 8.

The estimated ensemble sensitivity of (a),(e) T, (b),(f) Q, (c),(g) TW, and (d),(h) RW at 850 hPa with lead time 48 h. (left) Univariate ensemble sensitivity, and (right) multivariate ensemble sensitivity.

However, when lead time increases to 72 h, the structures of sensitivity patterns for T and Q shown at 24- and 48-h lead times disappear (Figs. 9a,b,e,f). For instance, the warm core structure shown in Figs. 4a and 4d and the moistening inner core shown in Figs. 5a and 5c become invisible. But similar sensitivity patterns of TW and RW with 72-h lead time to those with 24- and 48-h lead times are observed, although the magnitudes are significantly reduced (Figs. 9c,d,g,h). The univariate ensemble sensitivity also provides larger sensitivity values for TW and RW than the multivariate ensemble sensitivity.

Fig. 9.

As in Fig. 8, but for 72-h lead time.

Fig. 9.

As in Fig. 8, but for 72-h lead time.

Therefore, for both univariate and multivariate ensemble sensitivities, the forecast response becomes smaller with increasing lead times. At different lead times, the univariate ensemble sensitivity still overestimates the forecast metric compared to the multivariate ensemble sensitivity. The sensitivity for T and Q decreases faster in time than the sensitivity for TW and RW, which may relate with the temporal length scale of different state variables.

6. Perturbed initial condition experiment

The univariate and multivariate ensemble sensitivity patterns are identified in the previous section. Although the sensitivity patterns are similar between the two approaches, the sensitivity magnitudes are quite different, especially at higher levels. As discussed in section 2, both sensitivity methods are subject to assumptions made in the derivation of ensemble sensitivity. Thus the actual forecast responses to initial perturbations from perturbed initial condition experiments are performed, which provide evaluation for the two approaches. The predicted responses of univariate and multivariate ensemble sensitivities utilize Eqs. (3) and (8), respectively. The actual responses are calculated from the perturbed initial condition experiments, in which the initial conditions are perturbed by ρ(δx) in Eq. (8). Elements δx that consist δx follow Eq. (7), and ρ is localization function and denotes the Schur product.

Figure 10 shows the predicted response versus the actual response from 100 trials on a scatter diagram for different lead times. Greater distances from the 1:1 (diagonal) line indicate less accurate response predictions, which are quantitatively presented by the root-mean-square error (RMSE) in the legend. Least squares best-fit lines are also reported by the red lines. At each forecast lead time, the abscissa values are the same for both univariate and multivariate ensemble sensitivities, because the actual responses to initial perturbations are independent of the sensitivity method.

Fig. 10.

Actual perturbation responses against predicted responses from (a)–(c) univariate ensemble sensitivity and (d)–(f) multivariate ensemble sensitivity. Results with (left) 24-h lead time, and (middle),(right) 48 and 72-h lead times, respectively. Least squares best-fit lines (red) and RMSE are also reported.

Fig. 10.

Actual perturbation responses against predicted responses from (a)–(c) univariate ensemble sensitivity and (d)–(f) multivariate ensemble sensitivity. Results with (left) 24-h lead time, and (middle),(right) 48 and 72-h lead times, respectively. Least squares best-fit lines (red) and RMSE are also reported.

At 24-h lead time, the scatter diagram and least squares best-fit line of the univariate ensemble sensitivity show overestimation of the actual response (Fig. 10a). Compared to the univariate ensemble sensitivity, the predicted response from the multivariate ensemble sensitivity better matches with the actual response, and the corresponding least squares best-fit line is much closer to the diagonal line (Fig. 10d), which indicates better consistency with the nonlinear model. The RMSE from the univariate and multivariate ensemble sensitivities are 2.96 and 1.30, respectively, so the RMSE from the multivariate method is less than half of that from the univariate method. When the lead time increases to 48 h, multivariate ensemble sensitivity also provides better predicted response than the univariate ensemble sensitivity when verified to the actual response (Figs. 10b,e). The RMSE from the multivariate ensemble sensitivity slightly increases to 1.43 while the RMSE from the univariate ensemble sensitivity increases to 3.56, so the RMSE from the multivariate method is also less than half of that from the univariate method. Therefore, the ability to provide more accurate predicted response by the multivariate ensemble sensitivity compared to the univariate ensemble sensitivity has been demonstrated.

However, when the lead time increases to 72 h, both the univariate and multivariate methods show a significantly biased predicted response compared to the actual response (Figs. 10c,f). As previously discussed, the estimated sensitivities from both univariate and multivariate methods are significantly reduced at 72-h lead time, which indicates that correlations between the forecast responses and initial perturbations at 72 h earlier become very limited due to nonlinearity. Thus at 72-h lead time, the predicted response from both methods cannot present the actual response from the nonlinear model.

To illustrate how the variability in the initial conditions can lead to different intensity changes, perturbed initial condition experiments with 24-h lead time that assimilate hypothetical observations associated with the largest 10 sensitivities at 850 hPa for T, Q, TW, and RW, respectively, are performed (named ALL). Similar to Rios-Berrios et al. (2016a,b), two additional experiments are conducted, in which only dynamic state variables (named DRY) or only water vapor mixing ratio (named MOIST) are perturbed by the hypothetical observations. Figure 11 displays the predicted response versus the actual response for univariate and multivariate ensemble sensitivities. Results from the ALL experiments show that the actual responses of TW are slightly larger than those of T, and both have larger magnitudes than those of Q, while the actual responses of RW have the smallest magnitudes. These response patterns are well predicted by the univariate and multivariate ensemble sensitivities, but both ensemble sensitivities, especially the univariate ensemble sensitivity, show biased predicted responses. Comparing the ALL, DRY, and MOIST experiments reveals that perturbing only dynamic variables has a similar effect to perturbing all state variables and both give much larger changes of minimum SLP than perturbing only water vapor mixing ratio. Thus the initial dry dynamical differences play a more important role than the initial moist differences for the intensity changes of Typhoon Haiyan.

Fig. 11.

Actual perturbation responses against predicted responses for (a) univariate ensemble sensitivity and (b) multivariate ensemble sensitivity. Responses resulted from hypothetical observations of T, Q, TW, and RW are denoted by circle, triangle, cross, and plus, respectively. The ALL, DRY, and MOIST experiments are shown by black, red, and blue, respectively.

Fig. 11.

Actual perturbation responses against predicted responses for (a) univariate ensemble sensitivity and (b) multivariate ensemble sensitivity. Responses resulted from hypothetical observations of T, Q, TW, and RW are denoted by circle, triangle, cross, and plus, respectively. The ALL, DRY, and MOIST experiments are shown by black, red, and blue, respectively.

7. Summary and conclusions

The ensemble sensitivity has been applied to a realistic high-resolution ensemble simulation of the Super Typhoon Haiyan (2013). Typhoon Haiyan is one of the most intense typhoons, and it presents great challenges to forecast its observed intensity. Thus the high-resolution ensemble forecasts of Haiyan provide a testbed for examining the performances of both the univariate and multivariate ensemble sensitivities. Ensemble sensitivities are derived from ordinary least squares. The univariate ensemble sensitivity uses diagonal approximation to the multivariate regression on predicting a response to an initial perturbation. The multivariate ensemble sensitivity retains the full covariance matrix when computing the multivariate regression. The intensity forecast responses from both the univariate and multivariate ensemble sensitivities to initial perturbations at different lead times are analyzed.

Results show that both the univariate and multivariate ensemble sensitivities provide similar sensitivity patterns that lead to increased storm intensity 24 h later. Both sensitivity methods suggest that initial perturbations that are characterized by a warming area around the center of the storm from 850 to 200 hPa, an increased moisture area around the eyewall from 850 to 500 hPa, a stronger primary circulation around the radius of maximum wind from 850 to 200 hPa, and stronger inflow at low levels and stronger outflow at high levels, can lead to storm intensification 24 h later. These results are consistent with univariate ensemble sensitivity analysis in Brown and Hakim (2015) that intensify tropical cyclone in initial conditions can bring about stronger predictions. Despite similar sensitivity patterns obtained from the two sensitivity methods, the magnitudes of the predicted forecast responses from the two approaches are different. Compared to the multivariate ensemble sensitivity, the univariate ensemble sensitivity overestimates the forecast metric, especially at higher levels.

When the lead time of initial perturbations increases to 48 h, both univariate and multivariate ensemble sensitivities produce similar sensitivity patterns to those with 24-h lead time, but with smaller magnitudes. The univariate ensemble sensitivity still overestimates the forecast metric compared to the multivariate ensemble sensitivity. When the lead time of initial perturbations increases to 72 h, there are slight sensitivity patterns remain in the wind field, while the sensitivity patterns of temperature and humidity disappear. Thus ensemble sensitivity for the mass field decreases faster in time than that for the wind field, which may relate with different temporal length scale of state variables.

Perturbed initial condition experiments are conducted by assimilating hypothetical observations that are the same type as state variables and give observation increment equivalent to the corresponding ensemble spread. At 24- and 48-h lead times, multivariate ensemble sensitivity provides better predicted response than the univariate ensemble sensitivity, when verified to the actual response. The univariate ensemble sensitivity tends to overpredict the forecast response than the multivariate ensemble sensitivity, which is consistent with previous studies (e.g., Hacker and Lei 2015). Therefore, the ability of multivariate ensemble sensitivity to provide more accurate predicted responses than the univariate ensemble sensitivity has been demonstrated. At 72-h lead time, both sensitivity methods show significantly biased predicted response compared to the actual response, thus the two methods have a limited temporal length scale to predict the actual response from a nonlinear model. Perturbed initial condition experiments with variability in the initial conditions reveal that the initial dry dynamical differences play a more important role than the initial moist differences for the intensity changes of Typhoon Haiyan.

Promising results are obtained from the multivariate ensemble sensitivity for the Super Typhoon Haiyan (2013). The next step is to include more dynamic and thermodynamic factors in the initial perturbations, like the sea surface temperature, salinity, and so on, aiming to improve the understanding of the processes of storm intensification and achieve better intensity forecasts of storms. Moreover, the application of the multivariate ensemble sensitivity for other mesoscale and smaller-scale phenomena needs further investigation.

Acknowledgments

This work is jointly sponsored by the National Key R&D Program of China through Grant 2017YFC1501603, and the National Natural Science Foundation of China through Grants 41675052 and 41775057. Insightful comments from three anonymous reviewers significantly improved this report.

REFERENCES

REFERENCES
Ancell
,
B.
, and
G. J.
Hakim
,
2007
:
Comparing adjoint- and ensemble-sensitivity analysis with applications to observation targeting
.
Mon. Wea. Rev.
,
135
,
4117
4134
, https://doi.org/10.1175/2007MWR1904.1.
Anderson
,
J. L.
,
2007
:
Exploring the need for localization in ensemble data assimilation using a hierarchical ensemble filter
.
Physica D
,
230
,
99
111
, https://doi.org/10.1016/j.physd.2006.02.011.
Barker
,
D.
, and Coauthors
,
2012
:
The Weather Research and Forecasting Model’s Community Variational/Ensemble Data Assimilation System: WRFDA
.
Bull. Amer. Meteor. Soc.
,
93
,
831
843
, https://doi.org/10.1175/BAMS-D-11-00167.1.
Bednarczyk
,
C. N.
, and
B. C.
Ancell
,
2015
:
Ensemble sensitivity analysis applied to a southern plains convective event
.
Mon. Wea. Rev.
,
143
,
230
249
, https://doi.org/10.1175/MWR-D-13-00321.1.
Bishop
,
C. H.
, and
D.
Hodyss
,
2009
:
Ensemble covariances adaptively localized with ECO-RAP. Part 2: A strategy for the atmosphere
.
Tellus
,
61A
,
97
111
, https://doi.org/10.1111/j.1600-0870.2008.00372.x.
Brown
,
B. R.
, and
G. J.
Hakim
,
2015
:
Sensitivity of intensifying Atlantic hurricanes to vortex structure
.
Quart. J. Roy. Meteor. Soc.
,
141
,
2538
2551
, https://doi.org/10.1002/qj.2540.
Chang
,
E. K.
,
M.
Zheng
, and
K.
Raeder
,
2013
:
Medium-range ensemble sensitivity analysis of two extreme Pacific extratropical cyclones
.
Mon. Wea. Rev.
,
141
,
211
231
, https://doi.org/10.1175/MWR-D-11-00304.1.
Chu
,
K.
,
Q.
Xiao
,
Z.
Tan
, and
J.
Gu
,
2011
:
A forecast sensitivity study on the intensity change of Typhoon Sinlaku (2008)
.
J. Geophys. Res.
,
116
,
D22109
, https://doi.org/10.1029/2011JD016127.
Doyle
,
J. D.
,
C. A.
Reynolds
,
C.
Amerault
, and
J.
Moskaitis
,
2012
:
Adjoint sensitivity and predictability of tropical cyclogenesis
.
J. Atmos. Sci.
,
69
,
3535
3557
, https://doi.org/10.1175/JAS-D-12-0110.1.
Doyle
,
J. D.
,
C.
Amerault
,
C. A.
Reynolds
, and
P. A.
Reinecke
,
2014
:
Initial condition sensitivity and predictability of a severe extratropical cyclone using a moist adjoint
.
Mon. Wea. Rev.
,
142
,
320
342
, https://doi.org/10.1175/MWR-D-13-00201.1.
Ek
,
M. B.
,
K. E.
Mitchell
,
Y.
Lin
,
E.
Rogers
,
P.
Grunmann
,
V.
Koren
,
G.
Gayno
, and
J. D.
Tarpley
,
2003
:
Implementation of Noah land surface model advances in the National Centers for Environmental Prediction operational mesoscale Eta model
.
J. Geophys. Res.
,
108
,
8851
, https://doi.org/10.1029/2002JD003296.
Errico
,
R. M.
,
1997
:
What is an adjoint model?
Bull. Amer. Meteor. Soc.
,
78
,
2577
2592
, https://doi.org/10.1175/1520-0477(1997)078<2577:WIAAM>2.0.CO;2.
Errico
,
R. M.
, and
T.
Vukicevic
,
1992
:
Sensitivity analysis using an adjoint of the PSU–NCAR mesoscale model
.
Mon. Wea. Rev.
,
120
,
1644
1660
, https://doi.org/10.1175/1520-0493(1992)120<1644:SAUAAO>2.0.CO;2.
Garcies
,
L.
, and
V.
Homar
,
2009
:
Ensemble sensitivities of the real atmosphere: Application to Mediterranean intense cyclones
.
Tellus
,
61A
,
394
406
, https://doi.org/10.1111/j.1600-0870.2009.00392.x.
Gaspari
,
G.
, and
S. E.
Cohn
,
1999
:
Construction of correlation functions in two and three dimensions
.
Quart. J. Roy. Meteor. Soc.
,
125
,
723
757
, https://doi.org/10.1002/qj.49712555417.
Hacker
,
J. P.
, and
L.
Lei
,
2015
:
Multivariate ensemble sensitivity with localization
.
Mon. Wea. Rev.
,
143
,
2013
2027
, https://doi.org/10.1175/MWR-D-14-00309.1.
Hakim
,
G. J.
, and
R. D.
Torn
,
2008
: Ensemble synoptic analysis. Synoptic-Dynamic Meteorology and Weather Analysis and Forecasting: A Tribute to Fred Sanders, Meteor. Monogr., No. 55, 147–162, https://doi.org/10.1175/0065-9401-33.55.147.
Hanley
,
K. E.
,
D. J.
Kirshbaum
,
N. M.
Roberts
, and
G.
Leoncini
,
2013
:
Sensitivities of a squall line over central Europe in a convective-scale ensemble
.
Mon. Wea. Rev.
,
141
,
112
133
, https://doi.org/10.1175/MWR-D-12-00013.1.
Hong
,
S.
,
J.
Dudhia
, and
S.
Chen
,
2004
:
A revised approach to ice microphysical processes for the bulk parameterization of clouds and precipitation
.
Mon. Wea. Rev.
,
132
,
103
120
, https://doi.org/10.1175/1520-0493(2004)132<0103:ARATIM>2.0.CO;2.
Hong
,
S.
,
Y.
Noh
, and
J.
Dudhia
,
2006
:
A new vertical diffusion package with an explicit treatment of entrainment processes
.
Mon. Wea. Rev.
,
134
,
2318
2341
, https://doi.org/10.1175/MWR3199.1.
Iacono
,
M. J.
,
J. S.
Delamere
,
E. J.
Mlawer
,
M. W.
Shephard
,
S. A.
Clough
, and
W. D.
Collins
,
2008
:
Radiative forcing by long-lived greenhouse gases: Calculations with the AER radiative transfer models
.
J. Geophys. Res.
,
113
,
D13103
, https://doi.org/10.1029/2008JD009944.
Islam
,
T.
,
P. K.
Srivastava
,
M. A.
Rico-Ramirez
,
Q.
Dai
,
M.
Gupta
, and
S. K.
Singh
,
2015
:
Tracking a tropical cyclone through WRF–ARW simulation and sensitivity of model physics
.
Nat. Hazards
,
76
,
1473
1495
, https://doi.org/10.1007/s11069-014-1494-8.
Kidder
,
S. Q.
,
M. D.
Goldberg
,
R. M.
Zehr
,
M.
DeMaria
,
J. F. W.
Purdom
,
C. S.
Velden
,
N. C.
Grody
, and
S. J.
Kusselson
,
2000
:
Satellite analysis of tropical cyclones using the Advanced Microwave Sounding Unit (AMSU)
.
Bull. Amer. Meteor. Soc.
,
81
,
1241
1260
, https://doi.org/10.1175/1520-0477(2000)081<1241:SAOTCU>2.3.CO;2.
Langland
,
R. H.
,
R. L.
Elsberry
, and
R. M.
Errico
,
1995
:
Evaluation of physical processes in an idealized extratropical cyclone using adjoint sensitivity
.
Quart. J. Roy. Meteor. Soc.
,
121
,
1349
1386
, https://doi.org/10.1002/qj.49712152608.
Lei
,
L.
, and
J. L.
Anderson
,
2014a
:
Comparisons of empirical localization techniques for ensemble Kalman filters in a simple atmospheric general circulation model
.
Mon. Wea. Rev.
,
142
,
739
754
, https://doi.org/10.1175/MWR-D-13-00152.1.
Lei
,
L.
, and
J. L.
Anderson
,
2014b
:
Empirical localization of observations for serial ensemble Kalman filter data assimilation in an atmospheric general circulation model
.
Mon. Wea. Rev.
,
142
,
1835
1851
, https://doi.org/10.1175/MWR-D-13-00288.1.
Lei
,
L.
,
J. L.
Anderson
, and
G. S.
Romine
,
2015
:
Empirical localization functions for ensemble Kalman filter data assimilation in regions with and without precipitation
.
Mon. Wea. Rev.
,
143
,
3664
3679
, https://doi.org/10.1175/MWR-D-14-00415.1.
Lei
,
L.
,
J. L.
Anderson
, and
J. S.
Whitaker
,
2016
:
Localizing the impact of satellite radiance observations using a global group ensemble filter
.
J. Adv. Model. Earth Syst.
,
8
,
719
734
, https://doi.org/10.1002/2016MS000627.
Limpert
,
G. L.
, and
A. L.
Houston
,
2018
:
Ensemble sensitivity analysis for targeted observations of supercell thunderstorms
.
Mon. Wea. Rev.
,
146
,
1705
1721
, https://doi.org/10.1175/MWR-D-17-0029.1.
Lin
,
I. I.
,
I.-F.
Pun
, and
C.-C.
Lien
,
2014
:
“Category-6” supertyphoon Haiyan in global warming hiatus: Contribution from subsurface ocean warming
.
Geophys. Res. Lett.
,
41
,
8547
8553
, https://doi.org/10.1002/2014GL061281.
Lorenz
,
E. N.
,
2005
:
Designing chaotic models
.
J. Atmos. Sci.
,
62
,
1574
1587
, https://doi.org/10.1175/JAS3430.1.
Mlawer
,
E. J.
,
S. J.
Taubman
,
P. D.
Brown
,
M. J.
Iacono
, and
S. A.
Clough
,
1997
:
Radiative transfer for inhomogeneous atmosphere: RRTM, a validated correlated-k model for the long-wave
.
J. Geophys. Res.
,
102
,
16 663
16 682
, https://doi.org/10.1029/97JD00237.
NDRRMC
,
2014
: Effects of Typhoon “Yolanda” (Haiyan). NDRRMC SitRep. 107, National Disaster Risk Reduction and Management Council, 62 pp., http://www.ndrrmc.gov.ph/attachments/article/1329/Effects_of_Typhoon_YOLANDA_%28HAIYAN%29_SitRep_No_107_14MAR2014.pdf.
Rios-Berrios
,
R.
,
R. D.
Torn
, and
C. A.
Davis
,
2016a
:
An ensemble approach to investigate tropical cyclone intensification in sheared environments. Part I: Katia (2011)
.
J. Atmos. Sci.
,
73
,
71
93
, https://doi.org/10.1175/JAS-D-15-0052.1.
Rios-Berrios
,
R.
,
R. D.
Torn
, and
C. A.
Davis
,
2016b
:
An ensemble approach to investigate tropical cyclone intensification in sheared environments. Part II: Ophelia (2011)
.
J. Atmos. Sci.
,
73
,
1555
1575
, https://doi.org/10.1175/JAS-D-15-0245.1.
Shimada
,
U.
,
H.
Kubota
,
H.
Yamada
,
E. O.
Cayanan
, and
F. D.
Hilario
,
2018
:
Intensity and inner-core structure of Typhoon Haiyan (2013) near landfall: Doppler radar analysis
.
Mon. Wea. Rev.
,
146
,
583
597
, https://doi.org/10.1175/MWR-D-17-0120.1.
Torn
,
R. D.
,
2010
:
Ensemble-based sensitivity analysis applied to African easterly waves
.
Wea. Forecasting
,
25
,
61
78
, https://doi.org/10.1175/2009WAF2222255.1.
Torn
,
R. D.
, and
G. J.
Hakim
,
2008
:
Ensemble-based sensitivity analysis
.
Mon. Wea. Rev.
,
136
,
663
677
, https://doi.org/10.1175/2007MWR2132.1.
Torn
,
R. D.
,
G. J.
Hakim
, and
C.
Synder
,
2006
:
Boundary conditions for limited-area ensemble Kalman filters
.
Mon. Wea. Rev.
,
134
,
2490
2502
, https://doi.org/10.1175/MWR3187.1.
Velden
,
C.
,
T.
Olander
,
D.
Herndon
, and
J. P.
Kossin
,
2017
:
Reprocessing the most intense historical tropical cyclones in the satellite era using the Advanced Dvorak Technique
.
Mon. Wea. Rev.
,
145
,
971
983
, https://doi.org/10.1175/MWR-D-16-0312.1.
Wada
,
A.
,
S.
Kanada
, and
H.
Yamada
,
2018
:
Effect of air-sea environmental conditions and interfacial processes on extremely intense Typhoon Haiyan (2013)
.
J. Geophys. Res. Atmos.
,
123
,
10 379
10 405
, https://doi.org/10.1029/2017JD028139.
Wang
,
X.
, and
H.
Jiang
,
2019
:
A 13-year global climatology of tropical cyclone warm-core structures from AIRS data
.
Mon. Wea. Rev.
,
147
,
773
790
, https://doi.org/10.1175/MWR-D-18-0276.1.
Xiao
,
Q.
,
Y.
Kuo
,
Z.
Ma
,
W.
Huang
,
X.
Huang
,
X.
Zhang
,
D. M.
Barker
,
J.
Michalakes
, and
J.
Dudhia
,
2008
:
Application of an adiabatic WRF adjoint to the investigation of the May 2004 McMurdo, Antarctica, severe wind event
.
Mon. Wea. Rev.
,
136
,
3696
3713
, https://doi.org/10.1175/2008MWR2235.1.
Zhang
,
C.
,
Y.
Wang
, and
K.
Hamilton
,
2011
:
Improved representation of boundary layer clouds over the southeast Pacific in ARW–WRF using a modified Tiedtke cumulus parameterization scheme
.
Mon. Wea. Rev.
,
139
,
3489
3513
, https://doi.org/10.1175/MWR-D-10-05091.1.

Footnotes

© 2019 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).