• Charles, M., 2008: Verification of extratropical cyclones within NCEP forecast models using an automated cyclone tracking algorithm. M.S. thesis, School of Marine and Atmospheric Sciences, Stony Brook University, 135 pp. [Available from SoMAS, Stony Brook University, Stony Brook, NY 11794-5000].

  • Charles, M., , and Colle B. , 2009: Verification of extratropical cyclones within the NCEP operational models. Part II: The Short-Range Ensemble Forecast system. Wea. Forecasting, 24 , 11911214.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Du, J., , McQueen J. , , DiMego G. , , Toth Z. , , Jovic D. , , Zhou B. , , and Chuang H. , 2006: New dimension of NCEP Short-Range Ensemble Forecasting (SREF) System: Inclusion of WRF Members. Preprints, WMO Expert Team Meeting on Ensemble Prediction System, Exeter, United Kingdom, World Meteorological Organization. [Available online at http://wwwt.emc.ncep.noaa.gov/mmb/SREF/reference.html].

    • Search Google Scholar
    • Export Citation
  • Froude, L. S. R., , Bengtsson L. , , and Hodges K. I. , 2007: The predictability of extratropical storm tracks and the sensitivity of their prediction to the observing system. Mon. Wea. Rev., 135 , 315333.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hoskins, B. J., , and Hodges K. I. , 2002: New perspectives on the Northern Hemisphere winter storm tracks. J. Atmos. Sci., 59 , 10411061.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marchok, T. P., 2002: How the NCEP tropical cyclone tracker works. Preprints, 25th Conf. on Hurricanes and Tropical Meteorology, San Diego, CA, Amer. Meteor. Soc., P1.13. [Available online at http://ams.confex.com/ams/pdfpapers/37628.pdf].

    • Search Google Scholar
    • Export Citation
  • McMurdie, L., , and Mass C. , 2004: Major numerical forecast failures over the northeast Pacific. Wea. Forecasting, 19 , 338356.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McMurdie, L., , and Casola J. H. , 2009: Weather regimes and forecast errors in the Pacific Northwest. Wea. Forecasting, 24 , 829842.

  • Mesinger, F., and Coauthors, 2006: North American Regional Reanalysis. Bull. Amer. Meteor. Soc., 87 , 343360.

  • Mullen, S. L., , and Smith B. B. , 1990: An analysis of sea-level cyclone errors in NMC’s Nested Grid Model (NGM) during the 1987–88 winter season. Wea. Forecasting, 5 , 433447.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Olson, J. B., , and Colle B. A. , 2007: A modified approach to initialize an idealized extratropical cyclone within a mesoscale model. Mon. Wea. Rev., 135 , 16141624.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Olson, J. B., , and Colle B. A. , 2009: Three-dimensional idealized simulations of barrier jets along the southeastern coast of Alaska. Mon. Wea. Rev., 137 , 391413.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shapiro, M., , and Thorpe A. , 2004: THORPEX—A global atmospheric research programme: International science plan. World Meteorological Organization, 66 pp. [Available online at http://www.wmo.int/pages/prog/arep/wwrp/new/documents/executivesummary.pdf].

    • Search Google Scholar
    • Export Citation
  • Silberberg, S. R., , and Bosart L. F. , 1982: An analysis of systematic cyclone errors in the NMC LFM-II model during the 1978–79 cool season. Mon. Wea. Rev., 110 , 254271.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, B. B., , and Mullen S. L. , 1993: An evaluation of sea level cyclone forecasts produced by NMC’s Nested Grid Model and Global Spectral Model. Wea. Forecasting, 8 , 3756.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Uccellini, L. W., , Kocin P. J. , , Schneider R. S. , , Stokols P. M. , , and Dorr R. A. , 1995: Forecasting the 12–14 March 1993 Superstorm. Bull. Amer. Meteor. Soc., 76 , 183199.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Uccellini, L. W., , Kocin P. J. , , Sienkiewicz J. M. , , Kistler R. , , and Baker M. , 2009: Fred Sanders’ roles in the transformation of synoptic meteorology, the study of rapid cyclogenesis, the prediction of marine cyclones, and the forecast of New York City’s ‘Big Snow’ of December 1947. Synoptic–Dynamic Meteorology and Weather Analysis and Forecasting: A Tribute to Fred Sanders, Meteor. Monogr., No. 55, Amer. Meteor. Soc., 269–294.

    • Search Google Scholar
    • Export Citation
  • Wedam, G. B., , McMurdie L. A. , , and Mass C. F. , 2009: Comparisons of model forecast skill over the east and west coasts of the United States. Wea. Forecasting, 24 , 843854.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, F., , Snyder C. , , and Rotunno R. , 2002: Mesoscale predictability of the “surprise” snowstorm of 24–25 January 2000. Mon. Wea. Rev., 130 , 16171632.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zishka, K., , and Smith P. , 1980: The climatology of cyclones and anticyclones over North America and surrounding ocean environs for January and July, 1950–77. Mon. Wea. Rev., 108 , 387401.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zwiers, F. W., 1990: The effect of serial correlation on statistical inferences made with resampling procedures. J. Climate, 3 , 14521461.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Geographical regions (1–6) used for the cyclone verification. The surface stations used to verify the SLP in the model analyses are also shown by the three-letter (or five number) identifier.

  • View in gallery

    Average number of cyclones (shaded every four and contoured every one) for each cool season during 2002–07 in the GFS per 2.5° × 2.5°.

  • View in gallery

    Differences in cyclone central pressure (mb) between the (a) NARR and GFS (NARR-GFS) and (b) NAM and GFS (NAM-GFS) on a 2.5° grid for the 2002–07 cool seasons. The X locations denote areas where the SLP is greater in the GFS.

  • View in gallery

    SLP analysis (a) MAE and (b) ME for the stations from west to east in Fig. 1 for the GFS (solid black), NAM (dashed), and NARR (gray). The numbers of cyclones verified between 2002 and 2007 are shown in the parentheses. The dashed horizontal lines represent the average error during the period and the 90% confidence intervals are shown using the vertical bar on the right.

  • View in gallery

    MAEs of SLP (mb) in the (a) NARR, (c) NAM, and (e) GFS analyses using observations within 500 km of a cyclone for each of the 2002–07 cool seasons. The regions (1, 3–4, and 5–6) are denoted by the inset legend. (b),(d),(f) As in (a),(c),(e), but for the mean error (mb). Confidence intervals at the 90% significance level are given by the vertical bars.

  • View in gallery

    Spatial distribution of errors (18–36 h) of the cyclone central pressure (filled in mb) for the positive bias errors in the (a) NAM and (b) GFS and the negative bias errors in the (c) NAM and (d) GFS. The X locations mark points where the GFS pressure is significantly different than the NAM at the 90% level.

  • View in gallery

    Cyclone MAEs (mb) vs forecast hour for the 2002–07 cool seasons for the (a) NAM and (b) GFS for regions 1–6 specified in Fig. 1. Confidence intervals at the 90% significance level are given by the vertical bars.

  • View in gallery

    As in Fig. 7, but for the ME of the cyclone central pressure (mb).

  • View in gallery

    Mean absolute errors for cyclone central pressure (mb) averaged for hours 42–60 during each 2002–07 cool season for the (a) NAM and (b) GFS. Confidence intervals at the 90% significance level are given by the vertical bars.

  • View in gallery

    As in Fig. 9, but for the mean cyclone errors.

  • View in gallery

    As in Fig. 7, but for the cyclone displacement in kilometers.

  • View in gallery

    As in Fig. 9, but for the cyclone displacement.

  • View in gallery

    Histograms showing the frequency of model cyclone positions relative to the observed position (center point) for 45° bins centered on N, NE, E, etc. in the NAM for hours 42–60 h for regions 1, 4, and 6. The dashed range circles are every 10%, from 0% to 30%.The solid range circles represent the 90% confidence intervals below and above the range of the bins. The numbers for each radial represent the average displacement error (km) within that directional bin. The black vectors indicate the mean displacement vector, with each dashed range ring every 50 km.

  • View in gallery

    As in Fig. 13, but for the GFS.

  • View in gallery

    Time series of cyclone central pressure errors at hour 48 for region 1 during the five cool seasons (separated by gray dashed vertical lines) for the (a) NAM and (b) GFS. Horizontal black dashed lines denote two standard deviations above and below the mean central pressure error for region 1. The boldface black line is the 5-day running mean of the cyclone central pressure errors. Interesting periods for text discussion are highlighted with the numbered boxes.

  • View in gallery

    Extratropical cyclone displacement errors (km) vs forecast hour for the LFM-II: Silberberg and Bosart 1982, 1978–79 cool season, and conterminous United States (CONUS) and oceans; the NGM and AVN, Smith and Mullen 1993, 1987–88 and 1989–90 cool seasons, and Atlantic; and the NAM and GFS, 2002–07 cool seasons, and Atlantic.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 79 80 16
PDF Downloads 27 27 6

Verification of Extratropical Cyclones within the NCEP Operational Models. Part I: Analysis Errors and Short-Term NAM and GFS Forecasts

View More View Less
  • 1 School of Marine and Atmospheric Sciences, Stony Brook University, Stony Brook, New York
© Get Permissions
Full access

Abstract

This paper verifies extratropical cyclones around North America and the adjacent oceans within the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) and North American Mesoscale (NAM) models during the 2002–07 cool seasons (October–March). The analyzed cyclones in the Global Forecast System (GFS) model, North American Mesoscale (NAM) model, and the North American Regional Reanalysis (NARR) were also compared against sea level pressure (SLP) observations around extratropical cyclones. The GFS analysis of SLP was clearly superior to the NAM and NARR analyses. The analyzed cyclone pressures in the NAM improved in 2006–07 when its data assimilation was switched to the Gridpoint Statistical Interpolation (GSI).

The NCEP GFS has more skillful cyclone intensity and position forecasts than the NAM over the continental United States and adjacent oceans, especially over the eastern Pacific, where the NAM has a large positive (underdeepening) bias in cyclone central pressure. For the short-term (0–60 h) forecasts, the GFS and NAM cyclone errors over the eastern Pacific are larger than the other regions to the east. There are relatively large biases in cyclone position for both models, which vary spatially around North America. The eastern Pacific has four to eight cyclone events per year on average, with errors >10 mb at hour 48 in the GFS; this number has not decreased in recent years. There has been little improvement in the 0–2-day cyclone forecasts during the past 5 yr over the eastern United States, while there has been a relatively large improvement in the cyclone pressure predictions over the eastern Pacific in the NAM.

* Current affiliation: NOAA/NWS/NCEP/Climate Prediction Center, Camp Springs, Maryland.

Corresponding author address: Dr. Brian A. Colle, School of Marine and Atmospheric Sciences, Stony Brook University, Stony Brook, NY 11794-5000. Email: brian.colle@stonybrook.edu

Abstract

This paper verifies extratropical cyclones around North America and the adjacent oceans within the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) and North American Mesoscale (NAM) models during the 2002–07 cool seasons (October–March). The analyzed cyclones in the Global Forecast System (GFS) model, North American Mesoscale (NAM) model, and the North American Regional Reanalysis (NARR) were also compared against sea level pressure (SLP) observations around extratropical cyclones. The GFS analysis of SLP was clearly superior to the NAM and NARR analyses. The analyzed cyclone pressures in the NAM improved in 2006–07 when its data assimilation was switched to the Gridpoint Statistical Interpolation (GSI).

The NCEP GFS has more skillful cyclone intensity and position forecasts than the NAM over the continental United States and adjacent oceans, especially over the eastern Pacific, where the NAM has a large positive (underdeepening) bias in cyclone central pressure. For the short-term (0–60 h) forecasts, the GFS and NAM cyclone errors over the eastern Pacific are larger than the other regions to the east. There are relatively large biases in cyclone position for both models, which vary spatially around North America. The eastern Pacific has four to eight cyclone events per year on average, with errors >10 mb at hour 48 in the GFS; this number has not decreased in recent years. There has been little improvement in the 0–2-day cyclone forecasts during the past 5 yr over the eastern United States, while there has been a relatively large improvement in the cyclone pressure predictions over the eastern Pacific in the NAM.

* Current affiliation: NOAA/NWS/NCEP/Climate Prediction Center, Camp Springs, Maryland.

Corresponding author address: Dr. Brian A. Colle, School of Marine and Atmospheric Sciences, Stony Brook University, Stony Brook, NY 11794-5000. Email: brian.colle@stonybrook.edu

1. Introduction

a. Background

Extratropical cyclones are responsible for a wide range of hazardous weather, including flooding, severe convection, strong winds, snow, and ice. The skill of numerical weather prediction models in forecasting these major storm events has varied. Some storms, such as the Superstorm of 1993 along the U.S. east coast, have been relatively well forecast several days in advance (Uccellini et al. 1995). In contrast, operational models have poorly predicted other cyclones 1–2 days in advance, such as the 25 January 2000 cyclone event along the North Carolina coast (Zhang et al. 2002). Along the U.S. west coast, there have been several poorly predicted cyclone events related to poorly initialized cyclones over the relatively data-sparse Pacific Ocean (McMurdie and Mass 2004).

Improving our understanding of extratropical cyclone predictability requires an objective model verification of many historic cyclone events. It has been several years since a large number of extratropical cyclones have been verified around North America within the National Centers for Environmental Prediction (NCEP) operational models. For example, Silberberg and Bosart (1982) investigated forecast errors within the Limited-Area Fine Mesh model (LFM) in 1972, which was operational at 190.5-km horizontal grid spacing. They found that the LFM underdeepened Atlantic and Pacific cyclones by 6–10 mb at forecast hour 48, and overdeepened cyclones to the east of the Rockies through the eastern United States by up to 4 mb. Mullen and Smith (1990) found that the Nested Grid Model (NGM) during the 1987–88 cool season for forecast hours 24 and 48 also tended to underdeepen oceanic cyclones (4–8 mb in the Pacific, and 2–6 mb in the Atlantic) and cyclones in the lee of the Rockies (up to 2 mb). Smith and Mullen (1993) found that the NGM, in the 1987–88 and 1989–90 cool seasons at forecast hours 24 and 48, slightly overdeepened weak and continental cyclones (2–4 mb), and underdeepened deep and oceanic cyclones (2–6 mb in the Pacific and 2–4 mb in the Atlantic). Meanwhile, the medium-range Aviation Model (AVN) during the same cool seasons underdeepened all cyclones (2–4 mb, except up to 6 mb over the Pacific). The mean displacement error for the AVN at hour 48 was at least 30 km less than the NGM, and at least 60 km less for stronger cyclones (<980 mb). The NGM had a westward track bias over the Atlantic of ∼100 (∼150) km at 24 (48) h, while the track bias for the AVN was not statistically different than zero.

More recently, Uccellini et al. (2009) compared the skill of oceanic cyclone predictions over the Atlantic and Pacific by the forecasters at the National Oceanic and Atmospheric Administration (NOAA)/NCEP/Ocean Prediction Center (OPC) between the 1992–93 and 2002–05 cool seasons. Since the early 1990s, the day 4 cyclone central pressure and displacement errors by OPC forecasters have improved by ∼15%. The cyclone strength for 2002–05 was underforecast by ∼3 (∼4.3) mb over the Pacific (Atlantic). Cyclone position errors are generally ∼7% smaller in the Atlantic than the Pacific. For more intense cyclones, the Pacific position errors have decreased 15%–20% since the early 1990s.

Froude et al. (2007) globally tracked extratropical cyclones using 850-mb vorticity fields from European Centre for Medium-Range Weather Forecasts (ECMWF) 0–7-day forecasts during two warm and cool seasons. It was found that the model skill in forecasting cyclone intensity and position was greater in the Northern Hemisphere (NH) than the Southern Hemisphere (SH), with about 1 day of skill added in the NH. When considering only intense storms (850 mb vorticity > 8.0 × 10−5 s−1), the NH model skill was reduced by about 1 day for both seasons, but there was little change in the SH.

b. Motivation

A major goal of the World Meteorological Organization’s (WMO) The Observing System Research and Predictability Experiment (THORPEX) is to better understand the atmospheric predictability of high-impact weather events from the Pacific to the Atlantic (Shapiro and Thorpe 2004). Recently, Wedam et al. (2009) showed that operational model errors tend to be larger over the U.S. west coast than the east coast on average, but these regional differences have not been quantified for specific weather phenomena, such as extratropical cyclones.

To quantify the future benefits of THORPEX on numerical weather prediction, more event-based verification is needed for operational models. However, since the mid-1990s, there has been little objective cyclone verification of the NCEP short-range operational models around North America and its adjacent oceans. McMurdie and Mass (2004) only focused on a select number of cyclone events over the eastern Pacific to illustrate the large errors that can develop in this relatively data-sparse region. Uccellini et al. (2009) focused on the skill of forecasters in predicting cyclones, and not on the NCEP model performance. Thus, a major goal of this paper is to present a multiyear climatology of cyclone errors across the United States and surrounding oceans for the short-term (2 day) forecast period for the Global Forecast System (GFS) and North American Mesoscale (NAM) models. Overall, this study will address the following motivational questions:

  1. How do the various NCEP analyses (GFS and NAM) and North American Regional Reanalysis (NARR) compare to surface pressure observations around cyclones?
  2. What are the cyclone position and strength errors in NCEP operational models for the 0–2-day forecasts, and how have they changed in the last 5 yr?
  3. How do the forecast errors vary from the eastern Pacific eastward to the western Atlantic?
Part II of this paper (Charles and Colle 2009, hereafter Part II) presents the cyclone verification for the Short-Range Ensemble Forecast (SREF) system of NCEP, while subsequent papers will verify the medium-range (3–5 day) GFS cyclones, as well as investigate some of the large-scale flow patterns and cyclone tracks associated with the errors and biases.

Section 2 of the current paper will describe the data used to generate the cyclone climatology and the methods used to analyze the data. Section 3 will cover the cyclone errors in several data analysis systems. Section 4 will show the GFS and NAM central pressure and displacement errors, including an analysis of deep cyclones and intraseasonal cyclone errors. Section 5 will summarize the results.

2. Data and methods

Observed and forecast cyclones were identified in the GFS and NAM (formerly the Eta Model) for five cool seasons (October–March) from 2002 to 2007. The GFS is a global spectral model that is run at NCEP 4 times daily at a resolution of T254. Its global Gaussian grid is roughly equivalent to a horizontal resolution of ∼0.5°. The GFS has undergone several changes since 2002 (Table 1). The NAM is a regional mesoscale model run 4 times daily. Since 2001, the NAM has been run at 12-km grid spacing with 60 vertical layers. The NAM has also undergone many upgrades over the past several years (Table 2), including a change to the Weather Research and Forecasting- Nonhydrostatic Mesoscale Model (WRF-NMM) core in 2006 (Du et al. 2006).

The cyclones within the GFS and NAM were evaluated over central North America and ∼1000 km offshore on either side of the continent (Fig. 1). For the first three cool seasons (2002–05), the GFS grids were available 4 times daily every 6 h at ∼95 km grid spacing, while the NAM was available every 6 h on a ∼40 km grid. The GFS and NAM runs at 0000 and 1200 UTC were available to hour 60, while the 0600 and 1800 UTC runs were available to hour 48. For the 2005–07 cool seasons, both the NAM and GFS were available to hour 60 four times daily, with the NAM on a 12-km grid and the GFS on a 1° grid. The analyzed cyclones within the North American Regional Reanalysis (NARR) were also evaluated for the 2002–07 cool seasons over the same region (Mesinger et al. 2006). The NARR grids were available at ∼32 km resolution every 3 h. Since the automated cyclone tracker program utilized model data on a latitude–longitude grid, all grids were interpolated to 0.8° latitude–longitude grids. This resolution was applied to all models since it is closest to the GFS data, which has the coarsest resolution of all models. To fairly compare the models, their analyses and forecasts were evaluated for the same cyclones at each particular forecast hour.

The automated cyclone identification routine used in this study is similar to that in Marchok (2002), which was applied for tropical cyclones. First, a potential cyclone was identified by locating the lowest sea level pressure (SLP) point on a latitude–longitude grid, and then the cyclone position was interpolated to a location between latitude–longitude points using four Barnes analysis passes. To determine whether this point was associated with a regional area of lower pressure and not just an artifact of a highly variable SLP field, an SLP gradient of 1.5 mb per 1000 km was required anywhere within 300 km of this lowest pressure point. If this gradient was satisfied, then it was determined whether a 2-mb closed contour surrounded the grid point. If either of these tests failed, the grid point was masked out and the point with the next lowest SLP on the grid was tested. If a 2-mb closed contour was found, then the grid point was flagged as a cyclone, and the cyclone’s latitude, longitude, and central pressure were recorded. To prevent locating any additional points with this cyclone in future iterations, all grid points outward from the cyclone were masked out to the location where the SLP gradient reversed. All the steps above were repeated until either all grid points were masked out, or all remaining SLP values were greater than one-half standard deviation above the domain-wide mean SLP. The cyclone tracking capabilities of this automated program are not discussed in this paper, but will be utilized in a future paper.

To test the accuracy of this automated cyclone identification algorithm, all GFS forecast hour 48 (F048) cyclones identified by the automated program for October 2004 were checked manually using GFS SLP images created at F048. A “false alarm” was counted if a cyclone was marked by the routine but it did not have at least one closed 2-mb contour. Those cyclones that had at least one closed 2-mb contour, but were not marked by the routine were considered “missed.” For October 2004, there were 647 cyclones in the GFS between 20° and 70°N. Of these 647 cyclones, 11 were missed by the automated routine and there were 78 false alarms. The probability of detection (number matched divided by the total number that occurred) was found to be ∼98%, and the false alarm rate (number of false alarms divided by the total number that were matched) was found to be ∼11%. Over 90% of the false alarms occurred over the high terrain of the western United States, where SLP reduction can create irregularly shaped pressure gradients over steep topography. Overall, the manual evaluation suggests that the automated cyclone routine is reliable.

Once all forecast and analyzed cyclones were located, each forecast cyclone was paired with the appropriate observed cyclone. For each model run, the distance between each forecast and observed cyclones was calculated. These distances were ordered from smallest to largest, and the forecast and observed cyclones were paired by choosing the two with the smallest separation. This pairing process continued until either 1) all forecast cyclones were matched, in which case any remaining observed cyclones were considered missed by the model; 2) all observed cyclones were matched, in which case any remaining forecast cyclones were considered false alarms; or 3) the pairing distance exceeded 800 km, in which case any remaining observed (forecast) cyclones were considered missed (false alarms). This pairing process was also manually checked for the observed cyclones in October 2004 at hour 48 in the GFS, for which there were 524 GFS forecast and analyzed cyclones matched. There were only 10 pairs that were incorrectly matched (∼98% reliable).

Figure 2 shows the number of GFS-analyzed cyclones identified by the automated approach for the 2002–07 cool seasons on a 2.5° × 2.5° grid. The three largest maxima in the cyclone density occurred in the Gulf of Alaska, east of the southern tip of Greenland, and in the lee of the Rockies, which agrees with previous studies (Zishka and Smith 1980, Hoskins and Hodges 2002). The Gulf of Alaska and Greenland have especially high cyclone densities because these locations are within major climatological cyclone tracks and are frequent sites of cyclolysis (Hoskins and Hodges 2002). Overall, the cyclone climatology produced by our automated approach is realistic and suggests that it is useful to validate models.

The cyclones in this study were grouped into six distinct geographical regions (Fig. 1). The regions were chosen to relate the model errors to particular geographical areas, and to ensure there were a large number of cyclone densities per forecast hour for each region (>∼150).

To test for statistical significance, a bootstrapping approach was used to resample the data and obtain proper confidence intervals around the means (Zwiers 1990). For each region and parameter (e.g., cyclone central pressure errors over 5 yr), a new sample of the same size was obtained by randomly selecting from the original sample and allowing for repeated selections. The mean was calculated and this process was repeated 1000 times. The 90% confidence intervals around the mean were determined by finding the 5th and 95th percentiles of the means of all 1000 resamples. If the confidence intervals of two particular regions did not overlap, then they were considered significantly different at the 90% level.

3. Analyzed cyclone central pressure errors

To illustrate the systematic differences in the analyzed cyclones between the GFS, NAM, and NARR covering 2002–07, Figs. 3a and 3b show the differences in cyclone central pressures between the GFS analyses and the corresponding NARR and NAM analyses (NARR-GFS and NAM-GFS), respectively, on a 2.5° × 2.5° grid. The analyzed cyclone intensity is similar for the NAM, NARR, and GFS across the eastern half of the United States. Meanwhile, the NARR cyclones are 1–3 mb weaker (higher pressure) than the GFS on average over the oceans, eastern Rockies, and much of Canada, and these differences with the GFS are statistically significant at the 90% level (not shown). The NARR-GFS yields a spatial pattern that is similar to the NAM-GFS across all regions, but the NAM-GFS difference is ∼1 mb smaller in many regions. In contrast, the NARR- and NAM-analyzed cyclones are 1–2 mb stronger than the GFS along portions of the southwest United States. The similarity between the NARR and NAM is not surprising, since both used the Eta Data Assimilation System (EDAS) and similar physics packages originally developed for the Eta Model.

To determine the accuracy of the SLP analyses, 44 surface National Weather Service stations were randomly selected across the United States, Canada, the eastern Pacific, and the western Atlantic (Fig. 1). For each station, a bilinear interpolation was applied to obtain the analyzed SLP at the observation point when there was a cyclone within 500 km of the station. Figure 4a shows the mean absolute error (MAE) of the analyzed SLPs for all stations from west to east, as well as the 90% confidence intervals (error bar on right side of figure) for the domain-averaged error (horizontal lines). The NARR and NAM MAEs (1.5–2.5 mb) are larger than those of the GFS (∼1 mb) over the eastern Pacific, with the error in all models decreasing toward the coast. The NARR and NAM have large MAEs (1.5–3 mb) over the Rockies, and the GFS has about 0.5 mb smaller MAEs in this region. Over the eastern half of the United States, the MAEs are about 1 and 1.5 mb for the NAM and NARR, and just over 0.5 mb in the GFS. The eastern U.S. MAEs are lower than anywhere else in the domain, presumably because of the wealth of data upstream over the United States. Over the western Atlantic, the MAEs are about 2 mb in the NARR and NAM, and 1–2 mb in the GFS. Across the full domain, the NARR and NAM have average MAEs of ∼1.6 and ∼1.4 mb, respectively, while the GFS analysis MAE is about 1 mb. Overall, the MAEs in the GFS analysis are significantly (hereafter to mean at least statistically significant at the 90% level) less than the NARR and NAM analyses. In fact, averaged across the domain, the GFS has the best analysis of the station SLP near a cyclone 89% of the time (not shown), while NAM and NARR are best only 8% and 3% of the time, respectively.

Figure 4b shows the cyclone central pressure mean error bias (ME) for each analysis. Except for Revelstroke, British Columbia, Canada (YRV), and Las Vegas, Nevada (LAS) and Birmingham, Alabama (BHM), the NARR has a 0.5–2.0-mb positive ME at every station, with the largest positive bias over the eastern Pacific, western Atlantic, and the Rockies. The NAM has a similar distribution of pressure biases, except for a smaller positive bias over the United States. The GFS has a positive bias along the West Coast and the western Rockies, a slight negative bias over the eastern Rockies, and a near-zero bias at other regions.

To determine whether the model analyses have improved over the last several years, Fig. 5 shows the error in analyzed SLP (averaged over all surface stations) for each cool season and region across North America for regions 1, 3–4, and 5–6 (Fig. 1). The NARR mean absolute errors vary only slightly between years and the changes are not significant (Fig. 5a), while the pressure bias around the cyclones in the NARR remains positive for all regions and years (Fig. 5b). Since NARR used the same model and data sources during the period of study, any interannual variations in NARR performance depend on the synoptic-scale flow regime prevalent each year.

The NAM analysis has a large decrease in SLP MAEs (by 0.3–0.5 mb) between the 2005–06 and the 2006–07 cool seasons in all regions (Fig. 5c), which is significant in regions 3–6. This improvement coincides with the change to the WRF core and Gridpoint Statistical Interpolation (GSI; see Table 2). The positive bias also decreases slightly during this period as well as from 2004–05 to 2006–07 (Fig. 5d), during which there were improvements made to the precipitation assimilation, land surface model, and cloud and radiation schemes (Table 2).

Meanwhile, the MAEs for the GFS SLP analyses have changed less over the last 5 yr (Fig. 5e). The pressure bias in the GFS for the eastern Pacific is positive (∼0.3–0.5 mb) for all years (Fig. 5f). It appears that the increase in GFS resolution and improvements to the land–sea mask and downward longwave radiation schemes in August 2006 (Table 2) had little positive impact on its cyclone analysis performance.

The above results indicate that the GFS analysis has significantly better cyclone pressures than NAM and NARR from 2002 to 2007; thus, the GFS analysis was used to verify the central pressure forecasts below, although some of the verification results below were checked using the NAM analysis as truth. Since it is difficult to determine whether the GFS or NAM had the best analyzed cyclone positions using scattered observations, the mean positions of the GFS and NAM analysis cyclone were used as the observed cyclone positions.

4. Short-range GFS and NAM cyclone forecast verification

a. Central pressure errors

To illustrate the spatial distribution of cyclone mean errors (bias), the errors for each cyclone were interpolated onto a 2.5° latitude–longitude grid and averaged over the five cool seasons for the 18–36-h period (Fig. 6). The hatched boxes represent grid points in which the NAM error was significantly (90% level) different than the GFS error. The eastern Pacific errors in the NAM are positive (cyclones underdeepened) by 2–4 mb (Fig. 6a), while the positive errors in the GFS are 0–2 mb over the Pacific Ocean (Fig. 6b), which is significant at many grid points. The NAM errors are relatively small (0–3 mb) from the southern and central plains eastward to the mid-Atlantic, with a negative (overdeepening) bias (−1 to −2 mb) over much of eastern Canada and the Great Lakes (Fig. 6c).

Meanwhile, the MEs in the GFS are small (<1 mb) for much of the plains and Midwest, while a negative bias (−1 to −3 mb) exists along the western Canadian coast (Fig. 6d). This is likely the result of too little cyclolysis in the simulated cyclones that interact with a coastal barrier that is less steep in the model than reality. A negative GFS bias also exists over much of the northeast United States, while GFS cyclones are strongly underdeepened by 2–4 mb around Hudson Bay. These results are similar for the 42–60-h period (not shown).

To show the evolution of errors in the forecast, Figs. 7a and 7b show the MAEs for the cyclone central pressure versus the forecast hour for each region in the NAM and GFS, respectively. The MAEs over the eastern Pacific (region 1) are 1–2 mb larger than the other regions in the NAM and after hour 48 in the GFS, with the eastern Pacific errors increasing rapidly to ∼5.2 (∼4.2) mb in the NAM (GFS) by forecast hour 54 (F54). The western Atlantic (region 6) errors in the NAM increase to ∼4.2 mb by F60, which is the second largest of all regions at this time. Meanwhile, the GFS has a ∼3 mb error over the western Atlantic at F60, which is not significantly different than the other regions except for region 4, which has the smallest error of all regions. Given these results, there is 18–24 h of additional skill in the GFS cyclone SLP predictions over the eastern Pacific and western Atlantic as compared to the NAM. Also, the results suggest that lee cyclogenesis to the east of the Rockies (region 4) is more predictable than the oceanic storm tracks by F60.

The bias in the central pressure is shown in Fig. 8. The NAM has a large (2–3 mb) positive bias over the eastern Pacific (Fig. 8a), which reaches ∼3 mb by F48. This underdeepening in the NAM forecast is likely the result of cyclones being initialized too weak on average (Fig. 4a). A similar bias was noted years ago in the LFM (Silberberg and Bosart 1982) and the NGM (Mullen and Smith 1990). The GFS has a relatively small (∼0.6 mb) positive bias for the eastern Pacific before F24 (Fig. 8b), which becomes negative after F36 (though not significantly different from zero until after F48). The GFS has a ∼1 mb positive bias over central and eastern Canada (region 3), which is significantly different from zero. In contrast, the NAM has a 1–1.5-mb negative bias after F30 in region 3, which is also significantly different than zero. There are significant negative biases in the GFS and the NAM of about −0.6 and −0.8 mb, respectively, over the eastern United States (region 5). The GFS has significant negative errors (−1 mb) over the West Coast and Rockies (region 2).

Figure 9 shows the annual variations of the cyclone central pressure MAEs at F42–F60 for the NAM and GFS for each region. For the NAM, the most significant change in MAE was between the 2003–04 and 2004–05 cool seasons, when Pacific MAEs decreased from ∼6.5 to ∼4.25 mb (Fig. 9a). Since there were no significant model improvements during this period, the reduction in MAEs is probably due to differences in performance relative to the large-scale flow regimes. The GFS shows a similar pattern over the eastern Pacific (region 1) between the 2002–03 and 2004–05 cool seasons, but errors increase again (by ∼1 mb) by the 2006–07 cool season (Fig. 9b). The central United States (region 4) has a slight decrease in error for the GFS during the five cool seasons.

There are also interesting trends in the biases of the cyclone central pressures (Fig. 10). For the NAM, only the eastern Pacific (region 1) had a significant decrease in MEs since the first cool season (Fig. 10a). After 2004–05, improvements in the NAM model (Table 2) probably contributed to improvements in the forecasts. Interestingly, when the NAM switched to using GSI and WRF, the eastern Pacific MEs decreased little, while in the other regions the NAM MEs decreased by 0.5–1.5 from 2005–06 to 2006–07 and the cyclones were too deep by ∼1 mb on average. For the GFS (Fig. 10b), the increase in spatial resolution between 2002–03 and 2005–06 did not change the cyclone MEs much in all regions. Only the GFS region 1 errors became slightly more negative.

Since the results presented above use the GFS analysis as truth, it is important to determine whether using the NAM analysis as truth will change the results. From F12 to F60 (not shown), the NAM errors are still 0.5–1.0 mb larger than the GFS for all regions (Charles 2008), with Pacific (region 1) MAEs in the NAM peaking at just over 5 mb. The mean error does decrease in a few regions by ∼1 mb using the NAM as truth, but this is likely the result of the positive bias in the NAM analyses as shown above in section 3.

b. Displacement errors for 2002–07

The average cyclone displacement errors were quantified for each region (Fig. 11). The largest displacement errors in the NAM are in the western United States (region 2), peaking at ∼300 km by F60 (Fig. 11a), and by F42 the eastern Pacific (region 1) errors are comparable. The large error over the western United States is not surprising given the difficulty in reducing SLP over complex terrain. In the GFS (Fig. 11b), the largest cyclone displacement errors are also in the western United States (region 2), which peak at about 270 km at F48 and are significantly larger than the other regions. The GFS displacement errors in region 1 peak at about 260 km at F54, and are significantly larger than regions 3–6 by F48. The other regions have similar errors, peaking at 270–300 km (240–260 km) in the NAM (GFS). For all regions, the NAM has significantly larger displacement errors than the GFS (except for Canada).

Cyclone displacement errors have also varied substantially from 2002 to 2007 during the cool season (Fig. 12). For the NAM, the most significant trend is in eastern Canada (region 3), where displacement errors have increased consistently by an average of ∼15 km each cool season. Pacific (region 1) errors were at their lowest in the 2002–03 cool season, and their highest by the 2006–07 cool season (Fig. 12a). The other regions had no discernable trends. Apparently, upgrades to the NAM helped improve the cyclone central pressures in recent years, but not the position errors. The improvement in strength is likely the result of the improved analysis of the cyclone central pressure (Fig. 5), but the position error forecasts may still result from amplifying the initial condition errors and deficiencies in the model physics.

For the GFS (Fig. 12b), eastern Canada (region 3) had a significant increase in displacement errors between the 2002–03 and 2003–04 cool seasons, but after the 2003–04 cool season the displacement errors have consistently decreased by about 5–10 km yr−1. The eastern Pacific and western United States had significant decreases in displacement errors from the 2003–04 cool season to the 2005–06 cool season, but then displacement errors increased again by the 2006–07 cool season. The central and eastern United States (regions 4 and 5), as well as the western Atlantic (region 6), have had variable displacement trends over the years, and regions 5 and 6 show a significant decrease in displacement errors since the 2004–05 cool season.

The cyclone position biases in the operational models were also quantified for each of the 2002–07 seasons. Figures 13 and 14 show histograms of the NAM and GFS cyclone position errors relative to the observed for the 42–60-h forecasts of 2002–03, 2004–05, and 2006–07 for the oceans and the central United States (regions 1, 4, and 6). The GFS and NAM cyclones in region 1 are generally displaced to the east and south of the observed. For the NAM, the displacement is significant at the 90% level to the south for all years,1 and to the east in 2002–03 and 2006–07. It is hypothesized that this east–south displacement bias is from too little southerly flow deflection approaching North America, since it has been shown that the Rocky Mountain plateau can influence the flow and cyclone evolution ∼1000 km upstream of the coast (Olson and Colle 2007, 2009). There was little position bias in the GFS for region 1 during 2006–07, which suggests that it may now be better simulating these upstream terrain effects than the NAM. In region 4, cyclones in the NAM and GFS are significantly displaced to the northwest through northeast for all years. This same bias was prevalent in NCEP operational models 10–15 yr ago (Smith and Mullen 1993), which suggests that lee cyclogenesis is still occurring too far north in these models on average even though the model terrain has become more realistic. The NAM in region 6 has a significant westward or southwestward bias during 2002–03, which became more southwest in 2004–05 and 2006–07. The GFS has less of a bias, though there is still a tendency to favor a large fraction of west and southwest displacement errors. Smith and Mullen (1993) showed very similar results for both the GFS and NAM.

For the other regions (not shown), the western U.S. (region 2) cyclones are significantly displaced to the northwest in both models for all years (Charles 2008). In Canada (region 3), most cyclones are displaced from the southwest, west, to the north sectors in the NAM (significant to the southwest), and south and southwest in the GFS (both significant). Eastern U.S. (region 5) cyclone displacements are very similar to the western Atlantic (region 6) displacements.

c. Intraseasonal cyclone errors

There are also intraseasonal variations in cyclone errors as illustrated in a time series of cyclone central pressure errors at F48 during the five cool seasons for the eastern Pacific (Fig. 15). Both the NAM and GFS cyclone SLP errors vary greatly over the five cool seasons. The NAM underdeveloped cyclones during most of first two cool seasons (2002–03 and 2003–04; see Fig. 15a, box 1), while this is not the case for the GFS (Fig. 15b). Another period of underdeveloped cyclones in the NAM is from the end of January 2005 to the end of March 2005 (Fig. 15a, box 2). This period matches a period of overdeepened GFS cyclones during most of January 2005 (Fig. 15b, box 1), and then a short period of underpredicted GFS cyclones (Fig. 15b, box 2). Finally, during most of November and December 2006, the GFS overdeepened many cyclone events (Fig. 15b, box 3). Overall, the central pressure biases in the models vary over relatively slow periods (∼15–30 days), which suggest some dependence of the model errors for particular large-scale flow regimes (McMurdie and Casola 2009).

The number of large-error events (more than two standard deviations above average or −10 to +15 mb) per year in the eastern Pacific in the NAM is consistent with McMurdie and Mass (2004), who also found three to six relatively large SLP error occurrences at hour 48 h in the Eta Model per cool season. Most of the largest cyclone errors (>10 mb) in the NAM over the past several years in this region were associated with underdeepened events (positive error). The GFS has around six to eight large- (two standard deviation) error events per cool season (except for two in the 2002–03 cool season), although the two standard deviation MAE threshold is ∼5 mb smaller than for the NAM on the positive side. In the central United States, the frequency of large-error events in the NAM (−8 to + 8 mb) is more variable than in the eastern Pacific, ranging from only 2 in the 2003–04 cool season, to 14 in the 2005–06 cool season. In the GFS, large-error (−5 to +5 mb) frequency varies between 3 (2006–07) and 15 (2003–04). Over the western Atlantic, there were much fewer error events >10 mb than in the Pacific or central United States, with about one to four events per cool season in the NAM, and two to six events per cool season in the GFS. This is consistent with Wedam et al. (2009), who showed that there are more large SLP error events over the West Coast than over the East Coast.

d. Verification of relatively deep cyclones

Cyclone verification was also completed for relatively deep cyclone events, in which the central pressure (forecast or observed) was more than 1.5 standard deviations below the mean cyclone central pressure for each region. This corresponds to the following SLP thresholds for regions 1–6: 978.7, 992.4, 983.5, 996.1, 986.7, and 991.1 mb. For the NAM, deep cyclones have larger absolute errors in central pressure (by 1–2 mb) then for all cyclones, especially for the oceans (regions 1 and 6) (not shown). For the GFS, deep cyclones have larger absolute central pressure errors in all regions (∼0.5–1 mb higher). Deep cyclones forecast by the NAM tend to have a larger (smaller) positive (negative) bias than the verification of all cyclones, in all regions (not shown). For the GFS, deep cyclones in the central and eastern United States (regions 4 and 5) and the western Atlantic (region 6) have a larger positive bias after hour 48 than for all cyclones, and the western United States (region 2) has a larger negative bias. For the most part, cyclone displacements are smaller for deep cyclones than for all cyclones in both the NAM and GFS (not shown). The direction of displacement of deep cyclones is very similar to that for all cyclones (not shown).

5. Summary and conclusions

The goal of this paper is to quantify the extratropical cyclone forecast errors in NCEP GFS and NAM operational models during the cool seasons from 2002 to 2007. To evaluate the accuracy of the initial conditions for cyclone SLP, the GFS, NAM, and NARR analyses were verified against surface station observations within 500 km of a cyclone. The GFS analysis has the lowest MAEs in SLP, while the NARR has the greatest errors in SLP. All three analyses have a positive bias in SLP, though the GFS bias is significantly smaller than the NAM and NARR. These results suggest that users should be cautious when using the NARR for cyclone case studies, especially over the oceanic regions. Given these results, the GFS was used as the observed analysis to verify cyclone central pressure. Since there was no easy way to determine the most accurate model analysis for the observed cyclone position, an average of the NAM and GFS cyclone positions was used for this study.

For all regions the cyclone central pressure MAEs are 0.5–1.5 mb larger in the NAM than the GFS. For both models, cyclones over the Pacific (region 1) consistently have higher central pressure absolute errors than any other region. It is hypothesized that these errors are largest due to the deficiency in observations available over the Pacific as compared to the rest of the domain. Except for the western United States and Canada (regions 2 and 3), the NAM has a more positive bias in all regions than in the GFS. For example, in the Pacific (region 1) the positive bias peaks at ∼3 mb at F48 in the NAM, while the Pacific bias is not significantly different than zero after F18 in the GFS, until it becomes significantly negative by F48. It is hypothesized that the reason cyclone errors (especially for the Pacific) are larger in the NAM than in the GFS is partially due to the NAM’s inferior data assimilation (before the switch to GSI in 2006). Now that the NAM uses the GSI scheme, as in the GFS, a few more years of cyclone data are required to document any improvements that the NAM may exhibit.

Over the 60-h forecast, cyclone displacement in the NAM is also larger in every region than the GFS. The western United States (region 2) has the largest displacement in both models because of the complex terrain, making SLP reduction difficult. It was found that forecast cyclones in the Pacific were consistently displaced to the east and south of the observed cyclones. This suggests that the simulated cyclones make landfall too rapidly and are not deflected to the north by the terrain as much as was observed. In eastern Canada (region 3), cyclones are usually forecast to the west and south of the observed cyclones. In the central United States (region 4), cyclones were mostly displaced to the north in both models. This same northward bias was prevalent in earlier studies (Smith and Mullen 1993), which suggests that the lee cyclogenesis is shifted too far north. In the Atlantic (region 6), a large fraction of the cyclones are displaced to the west of the observed cyclone toward the coast, particularly in the NAM.

There are some large interannual variations in the cyclone errors in the NAM and GFS. The NAM central pressure errors decreased over the central Pacific from 2002 to 2007, while there was little improvement in the other regions and cool seasons. The positive central pressure bias over the eastern Pacific in the NAM decreased from nearly 5 mb in the 2002–03 season to ∼1.2 by the 2006–07 season. All regions experienced a reduction in the NAM central pressure MEs from the 2005–06 to 2006–07 cool seasons, which suggests that a combination of the switch to WRF and GSI resulted in deeper forecast cyclones in the NAM. Although the central pressures improved in the NAM, the NAM cyclone displacement errors increased over the same 5-yr period in many regions, while the GFS errors had a slight decreasing trend.

An important question is whether the model cyclone errors in this paper are smaller than the models 10–15 yr ago. Cyclone pressure biases in the NGM and AVN were 4–6 mb over the oceans in the earlier 1990s (Smith and Mullen 1993), but now the errors are 3 mb or less. Figure 16 shows how the cyclone displacement errors in the NAM and GFS from 2002 to 2007 compare with previous studies of extratropical cyclones in the NCEP operational models. The LFM-II displacement errors over the continental United States and surrounding oceans for the 1978–79 cool season ranged from ∼300 km to ∼440 km from hour 24 to 48 (Silberberg and Bosart 1982). By the late 1980s, cyclone position errors over the western Atlantic were ∼30% smaller than those of the late 1970s. The displacement errors in the NAM and GFS for the 2002–07 cool seasons are 30%–40% smaller than those of the late 1980s, which suggests that cyclone position forecasts have continued to improve since these earlier studies, albeit at a modest rate.

Overall, this paper shows that for a majority of the United States and over the oceans, the GFS gives a more accurate and less biased forecast of cyclone position and intensity. It was also shown that the GFS analysis is more accurate over the majority of the domain. This is consistent with Smith and Mullen (1993), who showed that the AVN had smaller cyclone forecast errors than the Eta Model (NAM), and with Wedam et al. (2009), who showed that the GFS provides more accurate sea level pressure forecasts than the NAM. The next paper of this series will verify the Short-Range Ensemble Forecast (SREF) system of NCEP to see if the ensemble has probabilistic skill for cyclones prediction. A subsequent paper will verify the extended GFS to day 5 and provide some composites of the large-scale flow and tracks attached to the particular cyclone errors.

Acknowledgments

This work represents a portion of the first author’s M.S. thesis. The research was supported by UCAR–COMET (Grants S02-38662 and S07-66814). We thank Timothy Marchok for his help and assistance in using the NCEP cyclone tracking program.

REFERENCES

  • Charles, M., 2008: Verification of extratropical cyclones within NCEP forecast models using an automated cyclone tracking algorithm. M.S. thesis, School of Marine and Atmospheric Sciences, Stony Brook University, 135 pp. [Available from SoMAS, Stony Brook University, Stony Brook, NY 11794-5000].

  • Charles, M., , and Colle B. , 2009: Verification of extratropical cyclones within the NCEP operational models. Part II: The Short-Range Ensemble Forecast system. Wea. Forecasting, 24 , 11911214.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Du, J., , McQueen J. , , DiMego G. , , Toth Z. , , Jovic D. , , Zhou B. , , and Chuang H. , 2006: New dimension of NCEP Short-Range Ensemble Forecasting (SREF) System: Inclusion of WRF Members. Preprints, WMO Expert Team Meeting on Ensemble Prediction System, Exeter, United Kingdom, World Meteorological Organization. [Available online at http://wwwt.emc.ncep.noaa.gov/mmb/SREF/reference.html].

    • Search Google Scholar
    • Export Citation
  • Froude, L. S. R., , Bengtsson L. , , and Hodges K. I. , 2007: The predictability of extratropical storm tracks and the sensitivity of their prediction to the observing system. Mon. Wea. Rev., 135 , 315333.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hoskins, B. J., , and Hodges K. I. , 2002: New perspectives on the Northern Hemisphere winter storm tracks. J. Atmos. Sci., 59 , 10411061.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marchok, T. P., 2002: How the NCEP tropical cyclone tracker works. Preprints, 25th Conf. on Hurricanes and Tropical Meteorology, San Diego, CA, Amer. Meteor. Soc., P1.13. [Available online at http://ams.confex.com/ams/pdfpapers/37628.pdf].

    • Search Google Scholar
    • Export Citation
  • McMurdie, L., , and Mass C. , 2004: Major numerical forecast failures over the northeast Pacific. Wea. Forecasting, 19 , 338356.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McMurdie, L., , and Casola J. H. , 2009: Weather regimes and forecast errors in the Pacific Northwest. Wea. Forecasting, 24 , 829842.

  • Mesinger, F., and Coauthors, 2006: North American Regional Reanalysis. Bull. Amer. Meteor. Soc., 87 , 343360.

  • Mullen, S. L., , and Smith B. B. , 1990: An analysis of sea-level cyclone errors in NMC’s Nested Grid Model (NGM) during the 1987–88 winter season. Wea. Forecasting, 5 , 433447.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Olson, J. B., , and Colle B. A. , 2007: A modified approach to initialize an idealized extratropical cyclone within a mesoscale model. Mon. Wea. Rev., 135 , 16141624.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Olson, J. B., , and Colle B. A. , 2009: Three-dimensional idealized simulations of barrier jets along the southeastern coast of Alaska. Mon. Wea. Rev., 137 , 391413.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shapiro, M., , and Thorpe A. , 2004: THORPEX—A global atmospheric research programme: International science plan. World Meteorological Organization, 66 pp. [Available online at http://www.wmo.int/pages/prog/arep/wwrp/new/documents/executivesummary.pdf].

    • Search Google Scholar
    • Export Citation
  • Silberberg, S. R., , and Bosart L. F. , 1982: An analysis of systematic cyclone errors in the NMC LFM-II model during the 1978–79 cool season. Mon. Wea. Rev., 110 , 254271.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, B. B., , and Mullen S. L. , 1993: An evaluation of sea level cyclone forecasts produced by NMC’s Nested Grid Model and Global Spectral Model. Wea. Forecasting, 8 , 3756.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Uccellini, L. W., , Kocin P. J. , , Schneider R. S. , , Stokols P. M. , , and Dorr R. A. , 1995: Forecasting the 12–14 March 1993 Superstorm. Bull. Amer. Meteor. Soc., 76 , 183199.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Uccellini, L. W., , Kocin P. J. , , Sienkiewicz J. M. , , Kistler R. , , and Baker M. , 2009: Fred Sanders’ roles in the transformation of synoptic meteorology, the study of rapid cyclogenesis, the prediction of marine cyclones, and the forecast of New York City’s ‘Big Snow’ of December 1947. Synoptic–Dynamic Meteorology and Weather Analysis and Forecasting: A Tribute to Fred Sanders, Meteor. Monogr., No. 55, Amer. Meteor. Soc., 269–294.

    • Search Google Scholar
    • Export Citation
  • Wedam, G. B., , McMurdie L. A. , , and Mass C. F. , 2009: Comparisons of model forecast skill over the east and west coasts of the United States. Wea. Forecasting, 24 , 843854.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, F., , Snyder C. , , and Rotunno R. , 2002: Mesoscale predictability of the “surprise” snowstorm of 24–25 January 2000. Mon. Wea. Rev., 130 , 16171632.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zishka, K., , and Smith P. , 1980: The climatology of cyclones and anticyclones over North America and surrounding ocean environs for January and July, 1950–77. Mon. Wea. Rev., 108 , 387401.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zwiers, F. W., 1990: The effect of serial correlation on statistical inferences made with resampling procedures. J. Climate, 3 , 14521461.

    • Crossref
    • Search Google Scholar
    • Export Citation

Fig. 1.
Fig. 1.

Geographical regions (1–6) used for the cyclone verification. The surface stations used to verify the SLP in the model analyses are also shown by the three-letter (or five number) identifier.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 2.
Fig. 2.

Average number of cyclones (shaded every four and contoured every one) for each cool season during 2002–07 in the GFS per 2.5° × 2.5°.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 3.
Fig. 3.

Differences in cyclone central pressure (mb) between the (a) NARR and GFS (NARR-GFS) and (b) NAM and GFS (NAM-GFS) on a 2.5° grid for the 2002–07 cool seasons. The X locations denote areas where the SLP is greater in the GFS.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 4.
Fig. 4.

SLP analysis (a) MAE and (b) ME for the stations from west to east in Fig. 1 for the GFS (solid black), NAM (dashed), and NARR (gray). The numbers of cyclones verified between 2002 and 2007 are shown in the parentheses. The dashed horizontal lines represent the average error during the period and the 90% confidence intervals are shown using the vertical bar on the right.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 5.
Fig. 5.

MAEs of SLP (mb) in the (a) NARR, (c) NAM, and (e) GFS analyses using observations within 500 km of a cyclone for each of the 2002–07 cool seasons. The regions (1, 3–4, and 5–6) are denoted by the inset legend. (b),(d),(f) As in (a),(c),(e), but for the mean error (mb). Confidence intervals at the 90% significance level are given by the vertical bars.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 6.
Fig. 6.

Spatial distribution of errors (18–36 h) of the cyclone central pressure (filled in mb) for the positive bias errors in the (a) NAM and (b) GFS and the negative bias errors in the (c) NAM and (d) GFS. The X locations mark points where the GFS pressure is significantly different than the NAM at the 90% level.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 7.
Fig. 7.

Cyclone MAEs (mb) vs forecast hour for the 2002–07 cool seasons for the (a) NAM and (b) GFS for regions 1–6 specified in Fig. 1. Confidence intervals at the 90% significance level are given by the vertical bars.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 8.
Fig. 8.

As in Fig. 7, but for the ME of the cyclone central pressure (mb).

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 9.
Fig. 9.

Mean absolute errors for cyclone central pressure (mb) averaged for hours 42–60 during each 2002–07 cool season for the (a) NAM and (b) GFS. Confidence intervals at the 90% significance level are given by the vertical bars.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 10.
Fig. 10.

As in Fig. 9, but for the mean cyclone errors.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 11.
Fig. 11.

As in Fig. 7, but for the cyclone displacement in kilometers.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 12.
Fig. 12.

As in Fig. 9, but for the cyclone displacement.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 13.
Fig. 13.

Histograms showing the frequency of model cyclone positions relative to the observed position (center point) for 45° bins centered on N, NE, E, etc. in the NAM for hours 42–60 h for regions 1, 4, and 6. The dashed range circles are every 10%, from 0% to 30%.The solid range circles represent the 90% confidence intervals below and above the range of the bins. The numbers for each radial represent the average displacement error (km) within that directional bin. The black vectors indicate the mean displacement vector, with each dashed range ring every 50 km.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 14.
Fig. 14.

As in Fig. 13, but for the GFS.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 15.
Fig. 15.

Time series of cyclone central pressure errors at hour 48 for region 1 during the five cool seasons (separated by gray dashed vertical lines) for the (a) NAM and (b) GFS. Horizontal black dashed lines denote two standard deviations above and below the mean central pressure error for region 1. The boldface black line is the 5-day running mean of the cyclone central pressure errors. Interesting periods for text discussion are highlighted with the numbered boxes.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Fig. 16.
Fig. 16.

Extratropical cyclone displacement errors (km) vs forecast hour for the LFM-II: Silberberg and Bosart 1982, 1978–79 cool season, and conterminous United States (CONUS) and oceans; the NGM and AVN, Smith and Mullen 1993, 1987–88 and 1989–90 cool seasons, and Atlantic; and the NAM and GFS, 2002–07 cool seasons, and Atlantic.

Citation: Weather and Forecasting 24, 5; 10.1175/2009WAF2222169.1

Table 1.

Major updates to the GFS model during 2002–07.

Table 1.
Table 2.

As in Table 1, but for the NAM model.

Table 2.

1

For a given direction of displacement to have statistical significance, the number of cyclones falling within that directional bin is significantly different at the 90% level than the average number of cyclones in each bin.

Save