The excellent forecasts made by ECMWF predicting the devastating landfall of Hurricane Sandy attracted a great deal of publicity and praise in the immediate aftermath of the event. The almost unprecedented and sudden “left hook” of the storm toward the coast of New Jersey was attributed to interactions with the large-scale atmospheric flow. This led to speculation that satellite observations may play an important role in the successful forecasting of this event. To investigate the role of satellite data a number of experiments have been performed at ECMWF where different satellite observations are deliberately withheld and forecasts of the hurricane rerun. Without observations from geostationary satellites the correct landfall of the storm is still reasonably well predicted albeit with a slight timing shift compared to the control forecast. On the other hand, without polar-orbiting satellites (which represent 90% of the volume of currently ingested observations) the ECMWF system would have given no useful guidance 4–5 days ahead that the storm would make landfall on the New Jersey coast. Instead the hurricane is predicted to stay well offshore in the Atlantic and hit the Maine coast 24 h later. If background errors estimated from the ECMWF Ensemble of Data Assimilations (EDA) are allowed to evolve and adapt to the depleted observing system, then some of the performance loss suffered by withholding polar satellite data can be recovered. The use of the appropriate EDA errors results in a more enhanced use of geostationary satellite observations, which partly compensates for the loss of polar satellite data.
Hurricane Sandy devastated areas of the Caribbean and the numerous locations along the eastern seaboard of the United States and Canada in late October 2012. It has been designated the largest Atlantic storm on record (reaching a diameter of over 1500 km) and at its most intense had a central core pressure of 940 hPa. The storm is thought to have been responsible for the loss of over 250 lives and caused over $60 billion (U.S. dollars) of damage.
It is generally accepted that this storm was well forecasted by all of the major NWP centers allowing preparations to be made that undoubtedly saved lives. However, the fact that the European Centre for Medium-Range Weather Forecasts (ECMWF) gave an early indication that the storm would take a sharp westward turn and make landfall in the mid-Atlantic states, attracted a lot of attention in the media (particularly in the United States). Some warning signals could be seen in ECMWF forecasts 7 or 8 days in advance and by 5 days out (at 0000 UTC 25 October) there was a strong convergence between the high-resolution forecast (HRES) and the associated ensemble system (ENS). Indeed at this stage many NWP centers [including the National Centers for Environmental Prediction (NCEP)] were forecasting a westward turn of the storm.
This turn (or left hook as it was dubbed in the U.S. media) has been widely attributed to the interaction of the storm with large-scale weather patterns lying to the north. Thus, one might expect that successful medium-range predictions of the storm’s path from the Caribbean to the midlatitudes would require an accurate description of the larger-scale meteorological environment. Information from the constellation of operational weather satellites gives a unique view of the large-scale atmospheric conditions—particularly over oceans where very few conventional measurements are available (e.g., from balloons or aircraft). Geostationary spacecraft located 36 000 km above Earth provide near-continuous measurements in the visible and infrared spectrum of low and midlatitudes. Polar-orbiting spacecraft flying at a much lower altitude (below 1000 km) provide global measurements, but at the expense of a reduced time sampling—revisiting the same location typically only twice per day. However, multiple polar satellites can collectively provide information 4–6 times per day for a given region.
Observing system experiments (OSEs) are classically used to assess the value of observations in a given NWP system. They usually consist in denying (adding) a given set of observations from (to) a baseline observing system scenario, and provide a measure of the impact of these observations on the weather forecast skill. OSEs can be run to assess the value of observations in specific regions of the globe (Kelly et al. 2007), to document the respective contribution of different observation types on the average quality of the forecasts (Bouttier and Kelly 2001; Bauer and Radnoti 2009), or to look at the impact of a specific dataset on the forecast of particular weather systems during field campaign experiments (Harnisch and Weissmann 2010; Harnisch et al. 2011). In this paper, we have adopted a standard OSE framework to test the sensitivity of the ECMWF forecasts of Hurricane Sandy to the denial of geostationary satellite data and the denial of polar-orbiting satellite data.
2. Data from polar-orbiting spacecraft
Polar-orbiting spacecraft circle Earth at altitudes typically below 1000 km and carry numerous active and passive sensors to observe the atmosphere and surface (listed in Fig. 1). With these spacecraft the main emphasis is making measurements at all latitudes with a variety of onboard sensors, but this is achieved at the expense of temporal resolution. A satellite may make measurements from the same location just once or twice per day. However, increasingly the same sensors [e.g., the Advanced Microwave Sounding Unit (AMSU)] are carried on multiple polar platforms so that similar measurements (albeit not from the same satellite) are obtained from a given location many times per day. At very high latitudes the time sampling is significantly improved—with the same satellite typically returning to the pole every 100 min. The heterogeneous array of highly sophisticated sensors carried by polar-orbiting spacecraft provides complementary observations. Microwave and infrared sounders and imagers provide information on temperature and humidity by measuring emitted radiation along nadir or near-nadir paths—resulting in typical spatial resolutions between 15 and 50 km. However, the vertical resolution that can be obtained is rather limited (between 1 and 2 km for infrared and 3 and 6 km for microwave). In contrast GPS radio occultation sensors provide similar information over broader horizontal scales (around 200 km), but with very fine vertical resolution (a few hundred meters). Active scatterometer instruments provide ocean wind speed information, which is useful in its own right, but also assists the interpretation of surface emitted radiation measured by passive sensors. For infrared and microwave sounding (and imager) data, radiance measurements are assimilated directly. For the GPS data, bending angles are assimilated and for the scatterometer, ambiguous (multiangle) winds are used.
3. Data from geostationary spacecraft
Geostationary spacecraft orbit Earth at an altitude of 36 000 km and tend to carry just one passive infrared and visible sensor. With these spacecraft the main emphasis is making measurements at low to midlatitudes with very fine spatial scale and very high (near continuous) temporal resolution. However, because of view geometry at high latitudes the data become more difficult to use [atmospheric motion vectors (AMVs) are not routinely generated >60° latitude, see Fig. 2] and the geostationary sensors are restricted to visible and infrared sounders with very poor spectral resolution. This limits their ability to make any measurements at all below clouds, and even in clear sky, the temperature and humidity information has much poorer vertical resolution than that provided by sensors carried on polar platforms. However, the very high temporal sampling allows the motion of atmospheric features such as clouds and water vapor to be tracked very accurately to provide wind information. This tracking is done explicitly by the data provider (particularly in cloudy conditions) to produce AMVs (see Velden et al. 2005) or implicitly by the data assimilation scheme (mainly in clear conditions) using time sequences of geostationary radiance measurements.
4. Details of the satellite denial experiments
The experiments have been conducted using the current operational version of the ECMWF forecasting system (CY38R1) with 91 levels in the vertical (up to 0.01 hPa) and a horizontal resolution of T1279 (typical grid spacing of 16 km). The initial conditions for the forecast model come from a 12-h window incremental four-dimensional variation data assimilation (4D-Var) system with T1279 outer loop resolution (used to compare observations with model forecasts) and T159 and T255 inner loop minimizations. It should be noted that this is not identical to the ECMWF daily operational forecast that is based on an early-delivery suite. The operational system is designed to deliver forecast products to users as early as possible, with the loss of late-arriving observations being offset by an additional short cutoff 6-h update 4D-Var. For retrospective research studies (where late-arriving observations and scheduling are not an issue), the computational expense of the additional short cutoff update cycle is traditionally avoided.
A control system has been run that uses all operationally available conventional observations (from the surface, balloons, ships, and aircraft) and satellite observations (polar and geostationary). It has been verified that the control forecasts in this study (using initial conditions from a 12-h window incremental 4D-Var) are almost identical to those produced operationally at the time of Hurricane Sandy (by the early delivery suite configuration of 4D-Var).
In the denial experiments (henceforth NOPOLAR and NOGEO) the respective observations are deliberately withheld from 0000 UTC 20 October 2012 onward. The NOPOLAR experiment removes all polar-orbiting satellite observations and the NOGEO experiment removes all geostationary data. The systems are then cycled without these data for 5 days until 0000 UTC 25 October after which forecasts are launched each day until the landfall of the Hurricane (on 30 October). These particular dates are chosen as the 5-day operational forecasts from 25 October (both high resolution and ensemble) were the first to accurately and consistently predict the exact timing and location of the landfall on the New Jersey coast.
5. Impact of satellite data denials on the analysis
Figure 3 shows how the NOPOLAR and NOGEO surface pressure analyses have deviated from the control after 5 days of cycling without observations. As expected, the analysis without polar-orbiting data exhibits more extensive differences than when geostationary data are denied. Changes in the immediate vicinity of the storm are rather modest—with just a 1–2-hPa weakening of the original tropical cyclone near Cuba. In the North Pacific the absence of polar satellite data causes a weakening of an extratropical depression—again by just 1–2 hPa, but over a much wider area.
A useful measure of how much the NOGEO and NOPOLAR systems are degraded as a result of the denial of satellite data is found in the fit to other assimilated observations. The fit to radiosonde wind data (both u and υ components) computed over the extratropical Northern Hemisphere and normalized differences are shown in Fig. 4 for the two denial experiments. The analysis fit (dotted lines) is determined by the very low observation errors (typically between 1 and 2 m s−1) assigned to radiosonde observations (i.e., the analysis is forced to fit these data locally). However, it can be seen from the wind data fit to the short-range forecast that the wind field of the NOPOLAR system is degraded with respect to the control by up to 5% whereas the NOGEO system is almost unaffected. Such a degradation is significant compared to the wind observation error and demonstrates the very strong constraining effect the polar-orbiting data have upon the analysis of the large-scale flow and wind field (despite the fact that most of the information provided is temperature rather than wind observations). In comparison the more direct wind information provided by the geostationary data have a much weaker impact on the analyzed wind field. This finding is in agreement with previous studies of the relative impact of various components of the satellite observing system (e.g., Radnoti et al. 2010). Balance is a very strong constraint imposed upon data assimilation systems designed for NWP and the translation of three-dimensional thermal information from the overwhelming volume of polar satellite data dominates the determination of the wind field.
6. Impact of satellite data denials on the forecasts
Forecast have been launched from the NOPOLAR and NOGEO analyses and compared to those of the control. Starting at 0000 UTC 25 October, initially all three systems give a rather consistent picture of the storm exiting the Caribbean and entering the Atlantic, such that after 72 h only very small differences can be seen between the forecasts (shown in Fig. 5 and 6). Even after 96 h, while some small departures in the hurricane’s forecasted position exist, the NOPOLAR and NOGEO systems are still close to the control and arguably giving useful forecasts. At 120 h the NOGEO forecast remains very close to the control and it correctly predicts the landfall of the storm (albeit with a very slight timing error) on the New Jersey coast at 0000 UTC 30 October. However, at this forecast range the NOPOLAR system deviates dramatically from the control and fails to capture the sudden westward turn of the storm to impact the coast. It keeps the hurricane position well offshore in the Atlantic and 36 h later (not shown) it goes on to hit the Maine seaboard some 800 km to the north. In Fig. 5 significant differences can be seen not only in the position of the storm after 120 h, but also in the large-scale flow in the immediate vicinity. In particular the trough–ridge wave structure to the west and north has a visibly stronger amplitude in the NOPOLAR system compared to the control as indicated by the highlighted red and blue 1025-hPa isobars. Tracing the origin of this wave amplification back in time shows that it is associated with the weakening of the North Pacific depression 5 days earlier in the initial conditions of the NOPOLAR system (Fig. 3).
Shorter-range forecasts of the storm have been examined but results are not shown. In brief the failure of the NOPOLAR 5-day forecast is repeated again in the 4-day forecast from the next day (0000 UTC 26 October). Only 3 days out from the event does the NOPOLAR system predict the correct landfall of the storm. The control and the NOGEO forecasts are consistent and accurate at all ranges from 5 days and shorter until the event happens on 30 October.
Longer-range forecasts have been examined, but are also not shown here. As discussed in the introduction, the control system from 7 days onward gives a reasonably consistent indication that the storm will make landfall somewhere on the eastern seaboard, with decreasing errors in timing and location between successive forecasts. The NOGEO forecasts are rather similar to the control while the NOPOLAR system only starts to suggest a landfall trajectory 6 days before the event, but with large errors in both timing and location.
7. Individual instrument denial experiments
Additional experiments have been performed in the context of Hurricane Sandy. Given the strong sensitivity to removing all polar-orbiting satellite observations, individual data types have been withheld to investigate if any single element was key to the successful prediction (or more precisely identify if losing any single element would reproduce the failure of the NOPOLAR system). The impact of individually removing microwave and infrared radiance observations (NORAD), GPS radio occultation data (NOGPS), and finally scatterometer winds (NOSCAT) upon the analysis quality and forecasts was tested. After 5 days of cycling (as before from 20 to 25 October) the NORAD system demonstrates the most significantly degraded analysis. Indeed it can be seen in Fig. 7 that the NORAD denial accounts for a large proportion of the changes seen in the degraded initial conditions of the NOPOLAR system (Fig. 3). The NOSCAT and NOGPS denials result in smaller changes to the analysis, but importantly these are located in the vicinity of the depression in the North Pacific (particularly in the case of the NOSCAT experiment). However, the 5-day forecast of all three experiments launched from 25 October correctly predicts the timing and location of the storm landfall (i.e., no single data denial reproduces the forecast failure of the NOPOLAR system).
8. Denial experiments with modified background errors
ECMWF runs an Ensemble of Data Assimilations (EDA; Isaksen et al. 2010; Bonavita et al. 2012) in parallel to the main high-resolution assimilation and forecasting system. The EDA consists of 10 independent, lower-resolution 4D-Var analyses where the observations, the model and the sea surface temperature are randomly perturbed according to their expected errors. By sampling the main sources of errors in the assimilation system the EDA aims to estimate uncertainty in the analysis and in the short-range forecast that is used as a background estimate of the atmospheric state in the high resolution ECMWF 4D-Var. The background errors estimated from the EDA present spatial variations that not only reflect changes in different meteorological conditions, but also changes related to the distribution and density of available observations. For practical and computational reasons observing system experiments (OSEs) are usually performed using background errors from the operational system (and this was the case for all the experiments described so far in this study). This is obviously suboptimal, but arguably a reasonable approximation when the changes to the observing system are small. However, this is clearly not the case for the NOPOLAR experiment where roughly 90% of the total volume of assimilated observations over ocean is removed. In this case the operational background errors are unlikely to describe those of the significantly depleted system. Thus, the NOPOLAR experiment has been rerun using background errors from a consistent EDA, which is cycled without polar satellite data (henceforth NOPOLAR-EDA). Like the experiments described in the previous section, the 4D-Var data assimilation experiments were run without the polar satellite data from 20 October, but after each 12-h assimilation window the background errors are recomputed from the spread of the 10 member EDA (where each individual member has the polar data removed). After 5 days without the polar satellites, background errors have evolved to larger values than the operational background errors (on average 50% larger), with the main differences found in oceanic areas, as shown in the top panel of Fig. 8. The largest increase is in the immediate vicinity of the storm, but there are also substantial changes in error over the North Pacific.
In the context of these denial experiments, larger background errors have two main consequences: first, more weight is given to those observations that are retained in the assimilation system (in this case geostationary satellite data and conventional data); and second, quality control checks based upon observation departures from the background are also effectively relaxed. The threshold for rejection is formulated as a multiple of the combined observation and background error variances—such that an increase of the latter (in response to the degraded observing system) renders observations less likely to be rejected. This effect is illustrated in the middle panel of Fig. 8 where the difference between geostationary AMV data coverage in the NOPOLAR and NOPOLAR-EDA is shown. More AMV data are used in the North Pacific by the NOPOLAR-EDA system, although in the vicinity of the storm there is no extra data usage. The combination of more weight being given to observations and extra data being used results in analysis differences between the NOPOLAR and NOPOLAR-EDA systems shown in the bottom panel of Fig. 8. Changes are generally rather small and certainly less than those seen between the NOPOLAR and control system (note that the contour interval of Fig. 8 is half that of Fig. 3). However, the changes are located in the dynamically active area in the North Pacific, which was previously identified as important for the correct forecast of the cyclone track.
When the modified NOPOLAR-EDA analyses are used as initial conditions there is a significant impact upon the forecasts of the storm. Figure 9 shows track predictions from the NOPOLAR and NOPOLAR-EDA systems initialized on 25 and 26 October (5 and 4 days before landfall). It can be seen that, for the 5-day forecast, almost all of the accuracy lost because of the denial of polar satellite is recovered by the NOPOLAR-EDA system—the forecast being almost as good as the control. Figure 10 illustrates the downstream propagation of small differences in the NOPOLAR-EDA analysis of the North Pacific depression at 0000 UTC 25 October dramatically affecting the forecast of Sandy’s 5 days later. Unfortunately such a remarkable improvement is not repeated in the 4-day forecast (from 26 October), but the NOPOLAR-EDA does provide a marginal improvement over the poor NOPOLAR forecast. At longer forecast ranges (6 days and beyond), the use of modified background errors in the NOPOLAR-EDA system does not produce better predictions of the storm.
It is reassuring to note that ensemble forecasts initialized from the operational EDA and the NOPOLAR EDA analysis provide a picture that is very consistent with the respective high-resolution (HRES) deterministic runs. This is shown in Fig. 11, where we present the hurricane tracks computed from forecasts of the operational EDA (left column) and the NOPOLAR EDA (right column) started at (from left to right) 0000 UTC 23 October, 0000 UTC 24 October, 0000 UTC 25 October, and 0000 UTC 26 October. Throughout the period, the tracks from the operational EDA show a coherent picture of the cyclone making landfall on the eastern seaboard of the United States, with increasing accuracy in terms of location and timing closer to the landfall date. On the other hand, the tracks from the NOPOLAR EDA show it is only from the 25 October run that the degraded system starts to give a clear signal of the westward turning of the storm, again in accordance with the NOPOLAR-EDA high-resolution forecasts.
The effect of retuning background errors for the individual instrument denial experiments has been tested. It was found that the resulting increases of EDA spread were very small (typically just a few percent averaged over the globe) for individual instrument denials and using these background errors produced analyses and forecasts that were effectively unchanged.
9. Summary and conclusions from the study
The fact that the withdrawal of polar-orbiting satellite data introduces large time and location errors in the ECMWF forecasts of Hurricane Sandy’s track illustrates the importance of these observations for accurate medium-range weather forecasting. Polar satellites provide unique information on the large-scale atmospheric conditions over areas that would otherwise be sparsely observed—information, which in this case, proved crucial and undoubtedly helped to mitigate the consequences of the hurricane. These results also corroborate previous OSE impact studies on the important role of polar-orbiting satellites play in current global NWP (McNally 2012).
In the case of the 5-day forecast of the storm landfall, it is found that a considerable fraction of the accuracy lost because of the denial of polar satellite data was recovered using consistent background errors from the ECMWF EDA (which better describe errors in the degraded NOPOLAR observing system). These larger errors increased the use of and the weight given to other observations—particularly geostationary winds over the North Pacific. Although the impact of EDA background errors was less for other forecasts (e.g., days 6 and 4) the Sandy case clearly demonstrates the value of a sophisticated data assimilation algorithm with flow-dependent and data-dependent uncertainty estimation. It is encouraging that such systems are now being developed and implemented in operational NWP centers.
It is interesting that for Hurricane Sandy none of the other data denials (including the NOGEO system) resulted in a failure in the prediction of the storm. However, when GEO data were more extensively used and given more weight in the NOPOLAR_EDA experiment, they clearly provided crucial information that improved the forecasts and compensated (at least partially) for the loss of the polar satellite data. It is also important to note that the geostationary satellites provided vital real-time monitoring of the storm’s actual progress during the event.
Finally, it should be emphasized that the conclusions drawn from this study are made only within the context of the ECMWF assimilation and forecasting system and only for this one meteorological case. In particular, one cannot draw conclusions from this specific case about the mean impact or overall value of the various observing systems for NWP forecast skill in general.