1. Introduction
Over the past decade the number of numerical weather prediction (NWP) models capable of producing high-quality tropical cyclone (TC) track forecasts has grown. Today, for the western North Pacific, forecasters at the Joint Typhoon Warning Center (JTWC) routinely use TC track forecasts from eight operational NWP models. Three of these models are run operationally at the Fleet Numerical Meteorology and Oceanography Center (FNMOC): the Navy Operational Global Atmospheric Prediction System (NOGAPS; Hogan and Rosmond 1991; Goerss and Jeffries 1994), the Geophysical Fluid Dynamics Laboratory (GFDL) Hurricane Prediction System (GFDN; Kurihara et al. 1993, 1995, 1998; Rennick 1999), and the Coupled Ocean–Atmosphere Mesoscale Prediction System (COAMPS; Hodur 19971). Two models are run operationally at the Japan Meteorological Agency (Kuma 1996): the global spectral model (GSM) and typhoon model (TYM). The remaining three models are the U. K. Meteorological Office global model (UKMO; Cullen 1993; Heming et al. 1995), the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS; Lord 1993), and the fifth-generation Pennsylvania State University–National Center for Atmospheric Research Mesoscale Model (MM5) run operationally by the Air Force Weather Agency (AFWA; Grell et al. 1995). A time line indicating when TC track forecasts from these eight NWP models became available to the forecasters at JTWC is displayed in Fig. 1.
The benefits of consensus forecasting have long been recognized by the meteorological community (Sanders 1973; Thompson 1977). Leslie and Fraedrich (1990) and Mundell and Rupp (1995) applied this approach to TC track prediction and illustrated the forecast improvement that resulted from using linear combinations of forecasts from various TC track prediction models. Goerss (2000) first illustrated the superior TC track forecasting performance of simple ensemble average or consensus forecasts formed using operational NWP models. For the western North Pacific in 1997 he demonstrated that the TC track forecast errors for both a global-model consensus (GSM, NOGAPS, and UKMO) and a regional-model consensus (GFDN and TYM) were significantly less than the errors for the best of the individual models. Versions of these global- and regional-model consensuses were installed on the Automated Tropical Cyclone Forecasting System (ATCF; Sampson and Schrader 2000) in 1998 and run operationally (although somewhat intermittently) during the 1998–2000 seasons. Elsberry and Carr (2000) expanded upon this work and made a five-model consensus (using the models from the Goerss global-model consensus and regional-model consensus) an integral part of the expert system developed for use by the TC forecasters at JTWC, the Systematic Approach Forecasting Aid (SAFA; Peak et al. 2000; Carr et al. 2001). Currently, consensus forecast aids are routinely used by the forecasters at both JTWC and the National Hurricane Center.
In the next section we describe how the TC forecast tracks from the individual NWP models are prepared for use and how a consensus forecast is determined. In section 3, we examine the forecast performance for the western North Pacific of individual NWP models and various consensus forecasts since 1992 and conclude with a summary of our results and a discussion of their implications for the future.
2. Methods
Forecast tracks discussed in this paper are processed as they would be in an operational setting. Since forecast track output for the NWP models become available to the forecaster 6 or 12 h after NWP model run time, they arrive too late to be used directly. Instead, the NWP model tracks are interpolated to intermediate times, and then interpolated positions are relocated to reflect the forecaster-analyzed (best track) position. The version of the interpolator used in this study includes a cubic spline (M. DeMaria 2000, personal communication) and a 10-pass, 3-point filter. All interpolated tracks are computed from real-time tracks, not postseason analyzed tracks (best tracks). Quality control for the interpolator includes a linear interpolator to fill in missing 12- and 36-h forecasts, a forecast position check (the 6-h/12-h old NWP model 6-h/12-h interpolated forecast position must be within 333 km of the current forecaster analyzed position), and a forecast track speed check (60-kt maximum) for all forecast periods beyond 12 h. NWP model interpolated tracks that fail the 12-h forecast position check are eliminated from the interpolator, while those failing the 60-kt speed check are truncated before the 60-kt speed is encountered.
A consensus for a given forecast period is a simple average of the interpolated members that pass the interpolator quality control tests described above. An attempt is made to compute a consensus forecast at the 12-, 24-, 36-, 48-, 72-, 96-, and 120-h forecast periods. This consensus is computed if two or more members exist for a given forecast period. If less than two members exist, the consensus is not computed.
3. Results and conclusions
First, we examine the forecast performance of the interpolated versions of the three NWP models that have been available to the JTWC forecasters since 1992: NOGAPS, UKMO, and TYM. The 12-h forecast errors for the three models from 1992 to 2002 are displayed in Fig. 2a. We see that the forecast errors for the three models declined over the decade from about 120–165 km to just under 100 km. The 24-h forecast errors for the three models declined from about 220–275 km to under 170 km (Fig. 2b). Similar results are seen in Fig. 2c for the 48-h forecast errors, which have improved from about 430–520 km to about 300 km between 1992 and 2002. Finally, we see in Fig. 2d that the 72-h forecast errors declined from roughly 600 km in 1992 to just over 400 km in 2002. Over the decade NWP model forecasts have improved so that 24-h forecasts today are only a little worse than 12-h forecasts in the early 1990s, and 72-h forecasts today are better than 48-h forecasts from the early 1990s.
Forecast performance since 1992 of consensus forecasts created from the interpolated versions of the eight NWP models shown in Fig. 1 was examined. While the total number of models available to these consensus forecasts varies, we only require that forecasts from at least two of the models be available to create a consensus forecast. Henceforth, we will identify the consensus forecasts by the number of models in the specified pool of models from which the consensus forecast is created. For example, as illustrated in Fig. 1, the pool for the three-model consensus consists of NOGAPS, UKMO, and TYM, while the pool for the five-model consensus consists of those three models along with GFDN and GSM. Thus, for the five-model consensus, consensus forecasts can be created when forecasts from two, three, four, or five models are available. For each forecast length, the consensus forecast is merely the arithmetic mean of individual forecasts of available members. The 12-h forecast errors for various consensus forecasts from 1992–2002 are displayed in Fig. 3a. The steady decline in the forecast error of the three-model consensus over the decade from about 120 km to about 80 km is indicative of improvements made to individual NWP models that make up the three-model consensus (NOGAPS, UKMO, and TYM). Making more models available to consensus has resulted in small but consistent gains in skill. For 2002, forecast error for the three-model consensus was about 80 km, while that for the eight-model consensus was just less than 75 km. We see similar results for all forecast lengths. The 24-h forecast error of the three-model consensus declined from about 185 km to about 135 km over the decade (Fig. 3b), while for the eight-model consensus in 2002 it was just less than 120 km. In Fig. 3c, the 48-h forecast error of the three-model consensus declined from about 350 km in 1992 to about 250 km in 2002, while that for the eight-model consensus in 2002 was about 210 km. Finally, the 72-h forecast error of the two-model consensus (NOGAPS and UKMO) declined from about 550 km to about 360 km over the decade (Fig. 3d), while for the eight-model consensus in 2002 it was about 310 km. As we saw for individual NWP models (Fig. 2), over the decade consensus forecasts have improved so that 24-h forecasts today are only a little worse than 12-h forecasts from the early 1990s, and 72-h forecasts today are better than 48-h forecasts from the early 1990s.
We have seen that addition of models to the consensus results in small but consistent gains in skill. For example, in 2002, the errors for the two-, five-, seven-, and eight-model-consensus 72-h forecasts were 357, 342, 329, and 309 km, respectively. However, this is not the only benefit of making more models available to the consensus. By increasing the number of models in the specified pool available to the consensus, we make it more likely that forecasts from at least two models will be available and that a consensus forecast can be formed. In Fig. 4, the availability percentages for the various consensus models in 2002 are displayed. We define the forecast availability percentage to be the percent of the time that consensus forecasts were available to the JTWC forecaster when he/she was required to make a TC forecast. In 2002, the availability percentages for the two-, five-, seven-, and eight-model-consensus 72-h forecasts were 84%, 89%, 92%, and 97%, respectively. By increasing the number of models in the specified pool from two to eight, we have significantly increased the percent of the time that forecasts from at least two of the models are available so that a consensus forecast can be created. For all forecast lengths, the availability of the five-model consensus ranged from 89% to 94%, while that for the eight-model consensus ranged from 97% to 98%. To an operational forecaster, this increase in availability may be just as valuable as the increase in forecast skill.
The forecast difficulty level (FDL) for a particular year is defined as the forecast error of a climatology and persistence (CLIPER) model run on best-track initial data (Neumann 1981). The FDLs for the western North Pacific for 1992–2002 were 169, 405, and 611 km at 24, 48, and 72 h, respectively. These values compare quite closely with the values reported by McBride and Holland (1987) of 183, 417, and 632 km. The percent improvement (positive values) or degradation (negative values) with respect to best-track CLIPER of the various consensus forecasts is displayed in Fig. 5. For the three-model consensus the 12-h forecast skill increased from nearly −90% in 1992 to about −15% in 2002 (Fig. 5a). Once again, we see that making more models available to the consensus has resulted in small but consistent gains in skill. For 2002, 12-h forecast skill increased from about −15% for the three-model consensus to about −5% for the eight-model consensus. The 24-h forecast skill for the three-model consensus increased from about −20% in 1992 to nearly 20% in 2002 (Fig. 5b), while the forecast skill for the eight-model consensus in 2002 was greater than 25%. In Fig. 5c, we see that the 48-h forecast skill for the three-model consensus increased from less than 10% to greater than 30% over the decade, while that for the eight-model consensus in 2002 increased further, to greater than 40%. Finally, the 72-h forecast skill for the two-model consensus increased from less than 5% in 1992 to about 40% in 2002 (Fig. 5d), while that for the eight-model consensus increased further, to nearly 50%.
In conclusion, we have seen that the TC track forecasting skill of NWP models for the western North Pacific has improved dramatically over the past decade (1992–2002). This improvement has contributed to a similar improvement in consensus forecasts created from these NWP models. We have also seen that the addition of models to the consensus results in improvements to both consensus track forecast skill and consensus forecast availability. Finally, we have seen the forecasters successfully integrate consensus forecasting into their operational procedures at JTWC. While a consensus demonstrates superior performance with respect to the individual NWP models in the long run, forecasts from individual models can certainly outperform a consensus in the short run. The forecasters use consensus as a baseline or starting point, then modify the forecast as they see fit. In the future, as the TC track forecasting skill of NWP models continues to improve and more high-quality NWP models become available, we can look forward to further improvement in consensus track forecast skill and forecast availability.
Acknowledgments
The authors would like to acknowledge Ann Schrader for her work with the ATCF read/write utilities, Mark DeMaria for development of the spline interpolator, Mike Fiorino for his invaluable vortex trackers, and the entire staff at JTWC for advice and the care and feeding required to make a consensus work in an operational setting. This work was supported by the Oceanographer of the Navy through the program office at the Space and Naval Warfare Systems Command (PMW-150), Program Element 0603207N.
REFERENCES
Carr, L. E., Elsberry R. L. , and Peak J. E. , 2001: Beta test of the systematic approach expert system prototype as a tropical cyclone track forecasting aid. Wea. Forecasting, 16 , 355–368.
Cullen, M. J. P., 1993: The Unified Forecast/Climate Model. Meteor. Mag, 122 , 81–122.
Elsberry, R. L., and Carr L. E. , 2000: Consensus of dynamical tropical cyclone track forecasts—Errors versus spread. Mon. Wea. Rev, 128 , 4131–4138.
Goerss, J. S., 2000: Tropical cyclone track forecasts using an ensemble of dynamical models. Mon. Wea. Rev, 128 , 1187–1193.
Goerss, J. S., and Jeffries R. A. , 1994: Assimilation of synthetic tropical cyclone observations into the Navy Operational Global Atmospheric Prediction System. Wea. Forecasting, 9 , 557–576.
Grell, G. A., Dudhia J. , and Stauffer D. R. , 1995: A description of the fifth-generation Penn State/NCAR Mesoscale Model (MM5). NCAR Tech. Note NCAR/TN-398+STR, 122 pp.
Heming, J. T., Chan J. C. L. , and Radford A. M. , 1995: A new scheme for the initialization of tropical cyclones in the UK Meteorological Office global model. Meteor. Appl, 2 , 171–184.
Hodur, R. M., 1997: The Naval Research Laboratory's Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS). Mon. Wea. Rev, 125 , 1414–1430.
Hogan, T. F., and Rosmond T. E. , 1991: The description of the Navy Operational Global Atmospheric Prediction System's spectral forecast model. Mon. Wea. Rev, 119 , 1786–1815.
Kuma, K., 1996: NWP activities at Japan Meteorological Agency. Preprints, 11th Conf. on Numerical Weather Prediction, Norfolk, VA, Amer. Meteor. Soc., J15–J16.
Kurihara, Y., Bender M. A. , and Ross R. J. , 1993: An initialization scheme of hurricane models by vortex specification. Mon. Wea. Rev, 121 , 2030–2045.
Kurihara, Y., Bender M. A. , Tuleya R. E. , and Ross R. J. , 1995: Improvements in the GFDL hurricane prediction system. Mon. Wea. Rev, 123 , 2791–2801.
Kurihara, Y., Tuleya R. E. , and Bender M. A. , 1998: The GFDL hurricane prediction system and its performance in the 1995 hurricane season. Mon. Wea. Rev, 126 , 1306–1322.
Leslie, L. M., and Fraedrich K. , 1990: Reduction of tropical cyclone position errors using an optimal combination of independent forecasts. Wea. Forecasting, 5 , 158–161.
Lord, S. J., 1993: Recent developments in tropical cyclone track forecasting with the NMC global analysis and forecast system. Preprints, 20th Conf. on Hurricanes and Tropical Meteorology, San Antonio, TX, Amer. Meteor. Soc., 290–291.
McBride, J., and Holland G. , 1987: Tropical-cyclone forecasting: A worldwide summary of techniques and verification statistics. Bull. Amer. Meteor. Soc, 68 , 1230–1238.
Mundell, D. B., and Rupp J. A. , 1995: Hybrid forecast aids at the Joint Typhoon Warning Center: Applications and results. Preprints, 21st Conf. on Hurricanes and Tropical Meteorology, Miami, FL, Amer. Meteor. Soc., 216–218.
Neumann, C., 1981: Trend in forecasting the tracks of Atlantic tropical cyclones. Bull. Amer. Meteor. Soc, 62 , 1473–1485.
Peak, J. E., Carr L. E. , and Elsberry R. L. , 2000: Developing a systematic approach to tropical cyclone track forecasting expert system—1: Information management, visualization, and proactivity considerations. Preprints, 24th Conf. on Hurricanes and Tropical Meteorology, Ft. Lauderdale, FL, Amer. Meteor. Soc., 502–503.
Rennick, M. A., 1999: Performance of the Navy's tropical cyclone prediction model in the western North Pacific basin during 1996. Wea. Forecasting, 14 , 3–14.
Sampson, C. R., and Schrader A. J. , 2000: The Automated Tropical Cyclone Forecasting System (Version 3.2). Bull. Amer. Meteor. Soc, 81 , 1131–1240.
Sanders, F., 1973: Skill in forecasting daily temperature and precipitation: Some experimental results. Bull. Amer. Meteor. Soc, 54 , 1171–1179.
Thompson, P. D., 1977: How to improve accuracy by combining independent forecasts. Mon. Wea. Rev, 105 , 228–229.
Time line indicating when TC track forecasts from the operational NWP models became available to the forecasters at JTWC
Citation: Weather and Forecasting 19, 3; 10.1175/1520-0434(2004)019<0633:AHOWNP>2.0.CO;2
Mean position errors (km) for NOGAPS (triangles), UKMO (squares), and TYM (dashed line) (a) 12- (b) 24- (c) 48-, and (d) 72-h forecasts for the western North Pacific (1992–2002)
Citation: Weather and Forecasting 19, 3; 10.1175/1520-0434(2004)019<0633:AHOWNP>2.0.CO;2
As in Fig. 2, but for the three-model (solid line), five-model (dashed line), seven-model (triangles), and eight-model (squares) consensuses. The number of cases appears below each year in parentheses
Citation: Weather and Forecasting 19, 3; 10.1175/1520-0434(2004)019<0633:AHOWNP>2.0.CO;2
Forecast availability percentage for the eight-model (solid black), seven-model (descending hatch), five-model (solid gray), and three-model (ascending hatch) consensuses for the western North Pacific in 2002
Citation: Weather and Forecasting 19, 3; 10.1175/1520-0434(2004)019<0633:AHOWNP>2.0.CO;2
As in Fig. 3, but for percent improvement (indicated by a positive number) or degradation (indicated by a negative number) with respect to best-track CLIPER. The number of cases appears below each year in parentheses
Citation: Weather and Forecasting 19, 3; 10.1175/1520-0434(2004)019<0633:AHOWNP>2.0.CO;2
COAMPS is a trademark of the Naval Research Laboratory.
