• Anwender, D., Harr P. A. , and Jones S. C. , 2008: Predictability associated with the downstream impacts of the extratropical transition of tropical cyclones: Case studies. Mon. Wea. Rev., 136, 32263247, doi:10.1175/2008MWR2249.1.

    • Search Google Scholar
    • Export Citation
  • Barnes, E. A., and Screen J. A. , 2015: The impact of Arctic warming on the midlatitude jet-stream: Can it? Has it? Will it? Wiley Interdiscip. Rev.: Climate Change, 6, 277286, doi:10.1002/wcc.337.

    • Search Google Scholar
    • Export Citation
  • Blackburn, M., Methven J. , and Roberts N. , 2008: Large-scale context for the UK floods in summer 2007. Weather, 63 (9), 280288, doi:10.1002/wea.322.

    • Search Google Scholar
    • Export Citation
  • Chang, E. K., 2005: The impact of wave packets propagating across Asia on Pacific cyclone development. Mon. Wea. Rev., 133, 19982015, doi:10.1175/MWR2953.1.

    • Search Google Scholar
    • Export Citation
  • Cressman, G. P., 1948: On the forecasting of long waves in the upper westerlies. J. Meteor., 5, 4457, doi:10.1175/1520-0469(1948)005<0044:OTFOLW>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Davies, H. C., and Didone M. , 2013: Diagnosis and dynamics of forecast error growth. Mon. Wea. Rev., 141, 24832501, doi:10.1175/MWR-D-12-00242.1.

    • Search Google Scholar
    • Export Citation
  • Davis, C., Brown B. , and Bullock R. , 2006: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134, 17721784, doi:10.1175/MWR3145.1.

    • Search Google Scholar
    • Export Citation
  • Davis, C., Brown B. , Bullock R. , and Halley-Gotway J. , 2009: The Method for Object-based Diagnostic Evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC Spring Program. Wea. Forecasting, 24, 12521267, doi:10.1175/2009WAF2222241.1.

    • Search Google Scholar
    • Export Citation
  • Dee, D. P., and Coauthors, 2011: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553597, doi:10.1002/qj.828.

    • Search Google Scholar
    • Export Citation
  • ECMWF, 2008: Changes in ECMWF model: CY33r1. [Available online at http://www.ecmwf.int/en/forecasts/documentation-and-support/changes-ecmwf-model.]

  • Fehlmann, R., and Davies H. C. , 1997: Misforecasts of synoptic systems: Diagnosis via PV retrodiction. Mon. Wea. Rev., 125, 22472264, doi:10.1175/1520-0493(1997)125<2247:MOSSDV>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Gilleland, E., Ahijevych D. , Brown B. G. , Casati B. , and Ebert E. E. , 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24, 14161430, doi:10.1175/2009WAF2222269.1.

    • Search Google Scholar
    • Export Citation
  • Gray, S., Dunning C. , Methven J. , Masato G. , and Chagnon J. , 2014: Systematic model forecast error in Rossby wave structure. Geophys. Res. Lett., 41, 29792987, doi:10.1002/2014GL059282.

    • Search Google Scholar
    • Export Citation
  • Grazzini, F., and Vitart F. , 2015: Atmospheric predictability and Rossby wave packets. Quart. J. Roy. Meteor. Soc., 141, 27932802, doi:10.1002/qj.2564.

    • Search Google Scholar
    • Export Citation
  • Hoskins, B., and Woollings T. , 2015: Persistent extratropical regimes and climate extremes. Curr. Climate Change Rep., 1 (3), 115124, doi:10.1007/s40641-015-0020-8.

    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., and Stephenson D. B. , 2012: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. John Wiley and Sons, 292 pp.

    • Search Google Scholar
    • Export Citation
  • Langland, R. H., Shapiro M. A. , and Gelaro R. , 2002: Initial condition sensitivity and error growth in forecasts of the 25 January 2000 East Coast snowstorm. Mon. Wea. Rev., 130, 957974, doi:10.1175/1520-0493(2002)130<0957:ICSAEG>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Lee, S., and Held I. M. , 1993: Baroclinic wave packets in models and observations. J. Atmos. Sci., 50, 14131428, doi:10.1175/1520-0469(1993)050<1413:BWPIMA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Martínez-Alvarado, O., Madonna E. , Gray S. , and Joos H. , 2015: A route to systematic error in forecasts of Rossby waves. Quart. J. Roy. Meteor. Soc., 142, 196210, doi:10.1002/qj.2645.

    • Search Google Scholar
    • Export Citation
  • Martius, O., Schwierz C. , and Davies H. , 2008: Far-upstream precursors of heavy precipitation events on the Alpine south-side. Quart. J. Roy. Meteor. Soc., 134, 417428, doi:10.1002/qj.229.

    • Search Google Scholar
    • Export Citation
  • Martius, O., Schwierz C. , and Davies H. , 2010: Tropopause-level waveguides. J. Atmos. Sci., 67, 866879, doi:10.1175/2009JAS2995.1.

  • Rodwell, M. J., and Coauthors, 2013: Characteristics of occasional poor medium-range weather forecasts for Europe. Bull. Amer. Meteor. Soc., 94, 13931405, doi:10.1175/BAMS-D-12-00099.1.

    • Search Google Scholar
    • Export Citation
  • Rossby, C.-G., 1945: On the propagation of frequencies and energy in certain types of oceanic and atmospheric waves. J. Meteor., 2, 187204, doi:10.1175/1520-0469(1945)002<0187:OTPOFA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Schlemmer, L., Martius O. , Sprenger M. , Schwierz C. , and Twitchett A. , 2010: Disentangling the forcing mechanisms of a heavy precipitation event along the Alpine south side using potential vorticity. Mon. Wea. Rev., 138, 23362353, doi:10.1175/2009MWR3202.1.

    • Search Google Scholar
    • Export Citation
  • Schwierz, C., Dirren S. , and Davies H. C. , 2004: Forced waves on a zonally aligned jet stream. J. Atmos. Sci., 61, 7387, doi:10.1175/1520-0469(2004)061<0073:FWOAZA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Waliser, D., and Moncrieff M. , 2008: The Year of Tropical Convection (YOTC) Science Plan: A joint WCRP–WWRP/THORPEX international initiative. WMO/TD-No. 1452, 26 pp. [Available online at https://www.wmo.int/pages/prog/arep/wwrp/new/documents/YOTC_Science_Plan.pdf.]

  • Wirth, V., and Eichhorn J. , 2014: Long-lived Rossby wave trains as precursors to strong winter cyclones over Europe. Quart. J. Roy. Meteor. Soc., 140, 729737, doi:10.1002/qj.2191.

    • Search Google Scholar
    • Export Citation
  • View in gallery

    For 1200 UTC 29 Oct 2008 (a) PV patterns on the 330-K isentropic surface (shaded; PVU) for the ERA-Interim reanalysis and wind vectors at 300 hPa (only wind vectors greater than 25 m s−1 are shown). (b) As in (a), but for the 120-h forecast. (c) PV gradient on the 330-K isentropic surface (shaded; PVU 1000−1 km−1) for the ERA-Interim reanalysis. (d) As in (c), but for the 120-h forecast. (e) As in (c), but with a filter of 15 PVU 1000−1 km−1 applied. (f) As in (d), but with a filter of 15 PVU 1000−1 km−1 applied.

  • View in gallery

    Total area of the objects normalized by the area of the 2-PVU line ±3° (y axis) as a function of the matching thresholds (PVU 1000−1 km−1; x axis; markers are the tested thresholds and the stars indicate the selected thresholds for the test application). Test runs of 120-h forecast period (1 Jan–31 Dec 2008) for the verification domain over the Northern Hemisphere for (a) DJF, (b) MAM, (c) JJA, and (d) SON.

  • View in gallery

    For 1200 UTC 29 Oct 2008, (a) the defined objects in the reanalysis field. (b) As in (a), but for the 120-h forecast. (c) The matching objects in the reanalysis after the matching and clustering process. (d) As in (c), but for the 120-h forecast. (e) The objects in the reanalysis field that will be considered in the verification. (f) As in (e), but for the 120-h forecast.

  • View in gallery

    A simplified representation of the error measures: (a) area errors, where the dark gray object is larger than the light gray object; (b) shift vectors, phase error, where the dark gray object is located more to the west than the light gray object, and (c) shift vectors, meridional error, where the dark gray object is located more to the south than the light gray object.

  • View in gallery

    An idealized representation of the resulting clustered objects for two matching thresholds (threshold 1 > threshold 2). Threshold 1 results in three objects that can be shifted independently for the calculation of the shift vectors and threshold 2 results in one big object.

  • View in gallery

    Total area of the objects as a function of the matching thresholds (km2; solid line) and the mean number of clusters in a time step as a function of the matching thresholds (dashed line). The markers (circles and asterisks) represent the tested thresholds. Based on test runs during the 120-h forecast period (1 Jan–31 Dec 2008) for the verification domain over the Northern Hemisphere.

  • View in gallery

    Total percentage area of the missing (continuous line) and false objects (dashed line) for all the forecast lead times. The percentages are calculated based on the area of the identified objects that overlap with the 2-PVU line. For the period 1 Jan–31 Dec 2008, the verification domain is over Europe (red boxes in Fig. 3).

  • View in gallery

    Probability density functions of ERA-Interim in comparison with the (a) 24-, (b) 72-, (c) 120-, (d) 168-, and (e) 240-h forecast lead times. Only PV gradients > 14 PVU 1000−1 km−1 are shown. For small values of the PV gradients, the probability density function is larger in the forecast than in the reanalysis (not shown). For the period 1 Jan–31 Dec 2008, the verification domain is over the Northern Hemisphere.

  • View in gallery

    Percentages of the shifted objects for all the forecast lead times. The numbers along the horizontal axis show the percentage of forecast objects that have to be shifted to the west or to the east. Numbers along the vertical axis show the percentage of forecast objects that were shifted south or north. The percentages in the four quadrants indicate the fractions of objects that need to be shifted west or east and north or south. The range of the shift errors increases with forecast lead time (not shown).

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 67 60 3
PDF Downloads 51 41 1

An Object-Based Forecast Verification Tool for Synoptic-Scale Rossby Waveguides

View More View Less
  • 1 Oeschger Centre for Climate Change Research, Institute of Geography, University of Bern, Bern, Switzerland
© Get Permissions
Full access

Abstract

An accurate representation of synoptic-scale Rossby waves in numerical weather forecast models is very important as these waves are closely linked to weather formation at the surface. Enhanced potential vorticity (PV) gradients at the tropopause levels act as waveguides for synoptic-scale Rossby waves, so spatial errors in the waveguides imply errors in the amplification and propagation of Rossby waves. This paper focuses on evaluating the forecast representation of these waveguides and presents an object-based forecast verification tool. In both forecast and the verification data, Rossby waveguide objects are defined based on enhanced PV gradient fields on isentropic surfaces. The tool automatically pairs the complex objects, compares their properties, and assesses the number of objects without a matching partner in either the forecast or the reanalysis. In the last step, error measures are calculated for the area and the location of the objects. As proof-of-concept application of the method for the year 2008, five lead times of the Integrated Forecast System (IFS) from the ECMWF are compared with the ECMWF reanalysis dataset. The majority of the waveguide objects are found to be in the correct position, and there are no systematic positional errors; however, the forecast objects and hence the areas of enhanced PV gradients are smaller.

Corresponding author address: Paraskevi Giannakaki, Institute of Geography, Hallerstrasse 12, 3012 Bern, Switzerland. E-mail: paraskevi.giannakaki@gmail.com

This article is included in the Predictability and Dynamics of Weather Systems in the Atlantic-European Sector (PANDOWAE) Special Collection.

Abstract

An accurate representation of synoptic-scale Rossby waves in numerical weather forecast models is very important as these waves are closely linked to weather formation at the surface. Enhanced potential vorticity (PV) gradients at the tropopause levels act as waveguides for synoptic-scale Rossby waves, so spatial errors in the waveguides imply errors in the amplification and propagation of Rossby waves. This paper focuses on evaluating the forecast representation of these waveguides and presents an object-based forecast verification tool. In both forecast and the verification data, Rossby waveguide objects are defined based on enhanced PV gradient fields on isentropic surfaces. The tool automatically pairs the complex objects, compares their properties, and assesses the number of objects without a matching partner in either the forecast or the reanalysis. In the last step, error measures are calculated for the area and the location of the objects. As proof-of-concept application of the method for the year 2008, five lead times of the Integrated Forecast System (IFS) from the ECMWF are compared with the ECMWF reanalysis dataset. The majority of the waveguide objects are found to be in the correct position, and there are no systematic positional errors; however, the forecast objects and hence the areas of enhanced PV gradients are smaller.

Corresponding author address: Paraskevi Giannakaki, Institute of Geography, Hallerstrasse 12, 3012 Bern, Switzerland. E-mail: paraskevi.giannakaki@gmail.com

This article is included in the Predictability and Dynamics of Weather Systems in the Atlantic-European Sector (PANDOWAE) Special Collection.

1. Introduction

Synoptic-scale Rossby waves are manifest in the tropopause-level flow as a succession of troughs and ridges (Rossby 1945). They play a vital role in the formation of midlatitude weather. In addition, high-impact weather events can be preceded by synoptic-scale Rossby waves (Chang 2005; Blackburn et al. 2008; Martius et al. 2008; Wirth and Eichhorn 2014).

Since they are present in the atmosphere for several days, they indicate the potential for improved predictability on the medium-range time scale (Cressman 1948; Lee and Held 1993; Grazzini and Vitart 2015), provided that their representation in numerical weather prediction (NWP) models is accurate. However, Gray et al. (2014) found that both the meridional extent of upper-level Rossby waves and the tropopause sharpness in ridges decrease with forecast lead time. Errors in Rossby waves can be associated with errors in the forecasting of warm conveyor belts (Martínez-Alvarado et al. 2015) and convection over North America (Rodwell et al. 2013). Forecast errors closely follow the time evolution of the waves (Davies and Didone 2013) and propagate with the group velocity of the waves (Langland et al. 2002; Anwender et al. 2008). These errors can have significant impacts on forecasting extreme weather events. For example, Fehlmann and Davies (1997) and Schlemmer et al. (2010) show that an underestimation of the meridional extent of breaking synoptic-scale Rossby waves over western Europe results in significant errors in the forecasting of heavy precipitation in southern Switzerland.

The focus of this study is on the model representation of one particular aspect of synoptic-scale Rossby waves, namely their waveguides. In the extratropics and subtropics, narrow and zonally extended areas of enhanced potential vorticity (PV) gradients, which are coaligned with the jet streams, act as waveguides for synoptic-scale Rossby waves (e.g., Schwierz et al. 2004; Martius et al. 2010). A misrepresentation of the Rossby waveguides will result in a misforecast of Rossby wave propagation and amplification.

The topic of waveguides is not only relevant for weather prediction. There is an active debate in the climate community with regard to whether a decrease in Arctic Sea ice affects the waveguide ability of the jet stream and hence the propagation and amplitude of Rossby waves (see, e.g., Hoskins and Woollings 2015; Barnes and Screen 2015).

The aim of this paper is to introduce a tool for the quantification and characterization of errors in the representation of synoptic-scale Rossby waveguides, that is, areas of elongated strong PV gradients that are collocated with the jet streams and the dynamical tropopause in forecast data.

Numerous object-based verification techniques have been developed in recent years [see Gilleland et al. (2009) and Jolliffe and Stephenson (2012) for details]. Here, we modify the Method for Object-based Diagnostic Evaluation (MODE; Davis et al. 2006). The objective of the presented method is to verify to what extent waveguides in the forecast have the same location and strength as those in the reanalysis.

2. Methodology

In this paper, the application of the method is demonstrated using forecast data from the Integrated Forecast System (IFS) model (ECMWF 2008) (T799/L91) and the ERA-Interim reanalysis dataset from ECMWF (Dee et al. 2011) (T255/L60) for the year 2008. Both datasets are interpolated at 1° horizontal resolution. The method is illustrated step by step with one example synoptic situation above Europe (1200 UTC 29 October 2008).

a. First step: Identification of the objects

The object-based forecast verification tool for synoptic-scale Rossby waveguides is based on the MODE method by Davis et al. (2006). As a first step, waveguide objects are identified. The objects of interest are narrow and long bands of strong PV gradients () on isentropic surfaces, which represent the waveguides and are collocated with the dynamical tropopause. The isentropic levels that are best suited to capture changes in the extratropical waveguides throughout the course of the year are for this study 340 K for JJA, 330 K for SON, and 325 K for DJF and MAM. In cases where different levels were used, the results were different, and the dynamical tropopause was not captured.

Figures 1a and 1b depict the PV distribution on the 330-K isentropic surface in the reanalysis and in the 120-h forecast over Europe at 1200 UTC 29 October 2008. Figures 1c and 1d show . To identify the waveguides, a threshold that isolates the strong gradients is needed.

Fig. 1.
Fig. 1.

For 1200 UTC 29 Oct 2008 (a) PV patterns on the 330-K isentropic surface (shaded; PVU) for the ERA-Interim reanalysis and wind vectors at 300 hPa (only wind vectors greater than 25 m s−1 are shown). (b) As in (a), but for the 120-h forecast. (c) PV gradient on the 330-K isentropic surface (shaded; PVU 1000−1 km−1) for the ERA-Interim reanalysis. (d) As in (c), but for the 120-h forecast. (e) As in (c), but with a filter of 15 PVU 1000−1 km−1 applied. (f) As in (d), but with a filter of 15 PVU 1000−1 km−1 applied.

Citation: Weather and Forecasting 31, 3; 10.1175/WAF-D-15-0147.1

The use of very large (small) thresholds results in very small (large) objects that do not have the characteristics of the waveguides. Namely, the waveguide objects should contain areas of strong PV gradients collocated with the dynamical tropopause on an isentropic surface. We use the 2-PVU contour (1 PVU = 1 × 10−6 m2 s−1 K kg−1) that encircles the pole as a proxy for the dynamical tropopause on an middle-world isentropic surface. To choose the optimal grad PV threshold, we calculate the area of the objects for different thresholds normalized by the area of the 2 PVU line ±3° (Fig. 2). Thresholds that result in normalized area values close to one and a small instantaneous range of change (slope) are considered representative for the waveguides.

Fig. 2.
Fig. 2.

Total area of the objects normalized by the area of the 2-PVU line ±3° (y axis) as a function of the matching thresholds (PVU 1000−1 km−1; x axis; markers are the tested thresholds and the stars indicate the selected thresholds for the test application). Test runs of 120-h forecast period (1 Jan–31 Dec 2008) for the verification domain over the Northern Hemisphere for (a) DJF, (b) MAM, (c) JJA, and (d) SON.

Citation: Weather and Forecasting 31, 3; 10.1175/WAF-D-15-0147.1

The threshold tests are performed separately for each season, as the verification of the waveguides is performed on different isentropic levels for each season. Figure 2 suggests that the best thresholds for the summer season are between 13 and 20 PVU 1000−1 km−1 and between 13 and 17 PVU 1000−1 km−1 for the remaining seasons. In this paper we used the following thresholds: 13 PVU 1000−1 km−1 for DJF, 17 PVU 1000−1 km−1 for MAM, 20 PVU 1000−1 km−1 for JJA, and 15 PVU 1000−1 km−1 for SON.

Spatially connected areas of strong PV gradients (for this season >15 PVU 1000−1 km−1) are the waveguide objects (Figs. 1e,f). Very small objects are filtered out using an area threshold of 100 000 km2; these objects are not dynamically relevant and make the matching procedure described in the next paragraph very time consuming if they are not removed. In the example, five objects are identified in the reanalysis field and seven in the forecast field, as indicated by different colors and numbers (Figs. 3a,b).

Fig. 3.
Fig. 3.

For 1200 UTC 29 Oct 2008, (a) the defined objects in the reanalysis field. (b) As in (a), but for the 120-h forecast. (c) The matching objects in the reanalysis after the matching and clustering process. (d) As in (c), but for the 120-h forecast. (e) The objects in the reanalysis field that will be considered in the verification. (f) As in (e), but for the 120-h forecast.

Citation: Weather and Forecasting 31, 3; 10.1175/WAF-D-15-0147.1

b. Second step: Matching of the objects

The next step is the matching of the objects in the forecast and the reanalysis fields. If a forecast object and a reanalysis object overlap spatially, then they are matched. The objects have complex structures, and there are cases in which reanalysis objects and forecast objects do not overlap. MODE is an ideal method for these cases, as a fuzzy logic algorithm is used to match the objects. If the objects do not overlap, a total interest value is calculated for the two objects by taking into account the area ratio and the centroid distance based on the following equation:
e1
[simplified Eq. (1) from Davis et al. (2009)], where is the total interest value () for the jth object pair, is the centroid distance (normalized by scaling , where 1 stands for the smallest distance) for the jth object pair, and is the area of the smaller of the two objects divided by the area of the larger. More weight is given to the centroid distance to ensure that the matched objects are close enough. The user needs to set a threshold value for the total interest value to define a match. If the chosen is very low, then objects that are spatially distant are matched and compared. A very high value of prevents the matching of related but not overlapping objects. In this paper we use a threshold value of 0.75 (see also section 3).

Another advantage of the MODE method is that multiple matches and merges are possible between objects. All objects with an interest value of 0.75 and higher are matched, and if several objects in the reanalysis match an object in the forecast (or vice versa), they are merged into one object. This procedure works stepwise and is bidirectional. In our example, a double match exists between the number 2 forecast object (Fig. 3b) and the number 3 and 4 reanalysis objects (Fig. 3a). Hence, the two reanalysis objects (3 and 4) are merged into one object. In addition, a multiple match exists between forecast objects 2–4 and reanalysis object 3, which results in the final merging of objects 3 and 4 in the reanalysis and objects 2–4 in the forecast (Figs. 3c,d).

There are objects that are not part of the waveguide; that is, they are not connected to the circumpolar band of strong PV gradients. Examples are objects 2 and 5 in Fig. 3a. To eliminate these objects, we require that all waveguide objects must overlap with the 2-PVU contour that encircles the pole.

If an unmatched object is present in the reanalysis, we refer to it as a missing object. If an unmatched object is present in the forecast field, we refer to it as a false object. The results of the verification have limited informative value if the area of unmatched objects is large.

c. Third step: Forecast evaluation

The spatial domain for the forecast verification is flexible; we choose a domain over the North Atlantic and Europe (20°–70°N, 70°W–30°E), as indicated by the red boxes in Fig. 3. The verification domain can be as large as the whole Northern Hemisphere. Figure 4 schematically depicts the error measures that are used. The matched objects are compared with respect to their area and their location. As an area measure, we define the area difference (AD) between the two matched objects normalized by the area of the reanalysis object:
e2
where is the area of the forecast object and is the area of the reanalysis object.
Fig. 4.
Fig. 4.

A simplified representation of the error measures: (a) area errors, where the dark gray object is larger than the light gray object; (b) shift vectors, phase error, where the dark gray object is located more to the west than the light gray object, and (c) shift vectors, meridional error, where the dark gray object is located more to the south than the light gray object.

Citation: Weather and Forecasting 31, 3; 10.1175/WAF-D-15-0147.1

Values of AD greater (less) than zero denote an overestimation (underestimation) of the waveguide area in the forecast. In the example case, the area error is equal to −0.2, which means that the forecast slightly underestimates the area of the waveguide objects.

The second verification measure describes location errors (Figs. 4b,c). Every forecast object is moved in 1° steps in both the latitudinal and longitudinal directions to find the position with the highest spatial overlap with the reanalysis object. To decrease the computation time, we recommend setting a maximum shift in both the latitudinal and longitudinal directions; here, we use ±6°. This results in 144 shift positions for which the area overlap is calculated. A positive latitudinal shift means that the forecast object is located too far north compared to the reanalysis object. A positive longitudinal shift means that the forecast object is located too far east. The shift vectors with the greatest overlapping area between forecast and reanalysis correspond to the location errors.

In the example case, the shift of the forecast object by 1° to the south and 1° to the west resulted in the greatest overlapping area.

3. Test application to one year’s worth of forecast data

This section expands on the application of the Rossby waveguide verification tool to one year’s worth of forecast data, focusing on the North Atlantic and Europe as the verification domain (red boxes in Fig. 3).

a. Sensitivity test for matching thresholds

For the year of the illustrated example (2008, 120-h forecast lead time), tests were performed to determine how the matching thresholds are related to the number of clusters and their area. The verification domain was the whole Northern Hemisphere. The ideal threshold should result in a large number of clusters. The number of clusters is relevant for the calculation of the shift vectors. In Fig. 5, threshold 1 resulted in three different objects that can be shifted independently and the smaller threshold 2 resulted in only one object. So in cases where threshold 1 is used, we have three objects moving independently and three shift vectors. These three shift vectors in the case of threshold 1 are considered more accurate than the one shift vector in the case of threshold 2.

Fig. 5.
Fig. 5.

An idealized representation of the resulting clustered objects for two matching thresholds (threshold 1 > threshold 2). Threshold 1 results in three objects that can be shifted independently for the calculation of the shift vectors and threshold 2 results in one big object.

Citation: Weather and Forecasting 31, 3; 10.1175/WAF-D-15-0147.1

Figure 6 shows the number of clusters and their area as a function of the tested matching thresholds. Matching thresholds less than 0.55 result in few and very large clusters. A rapid increase in the number of objects is noted for thresholds between 0.6 and 0.75, while the total area of the matched objects decreases slowly. When a total interest value greater than 0.75 is required for the matching, both the number of clusters and their area decrease. In this paper a total interest value of 0.75 is used as a minimum requirement for the matching of two objects.

Fig. 6.
Fig. 6.

Total area of the objects as a function of the matching thresholds (km2; solid line) and the mean number of clusters in a time step as a function of the matching thresholds (dashed line). The markers (circles and asterisks) represent the tested thresholds. Based on test runs during the 120-h forecast period (1 Jan–31 Dec 2008) for the verification domain over the Northern Hemisphere.

Citation: Weather and Forecasting 31, 3; 10.1175/WAF-D-15-0147.1

The year 2008 is chosen because it is included in the Year of Tropical Convection (YOTC) period. YOTC data explicitly contain diabatic forcing terms (Waliser and Moncrieff 2008), and follow-up studies that focus in more detail on the physical processes responsible for the forecast errors are possible. A 1-yr forecast error climatology was produced by comparing output from the IFS model at five different forecast lead times (24, 72, 120, 168, and 240 h) with the ERA-Interim reanalysis dataset.

b. Missing and false forecast areas for the year 2008

Figure 7 shows the area of missing and false objects in percent of the area of all objects in the reanalysis that overlap with the 2-PVU contour for five forecast lead times. For all the forecast lead times, the model errors are more frequently associated with missed objects rather than with false objects. This can be explained by an underestimation of the intensity of the field in the forecast (Fig. 8). The percentage of missing and false objects increases with the forecast lead time.

Fig. 7.
Fig. 7.

Total percentage area of the missing (continuous line) and false objects (dashed line) for all the forecast lead times. The percentages are calculated based on the area of the identified objects that overlap with the 2-PVU line. For the period 1 Jan–31 Dec 2008, the verification domain is over Europe (red boxes in Fig. 3).

Citation: Weather and Forecasting 31, 3; 10.1175/WAF-D-15-0147.1

Fig. 8.
Fig. 8.

Probability density functions of ERA-Interim in comparison with the (a) 24-, (b) 72-, (c) 120-, (d) 168-, and (e) 240-h forecast lead times. Only PV gradients > 14 PVU 1000−1 km−1 are shown. For small values of the PV gradients, the probability density function is larger in the forecast than in the reanalysis (not shown). For the period 1 Jan–31 Dec 2008, the verification domain is over the Northern Hemisphere.

Citation: Weather and Forecasting 31, 3; 10.1175/WAF-D-15-0147.1

c. Results of verification

The area fraction of the missing and false objects compared to the matched objects is very low and therefore verification of the matched objects is considered to be informative. As described in the previous section, the verification measures are the area errors and the shift vectors. The IFS model forecasts smaller waveguide objects than the reanalysis (Table 1); moreover, are weaker in the forecast than in the reanalysis. A reduction in the area of the forecast objects with forecast lead time is also evident.

Table 1.

Area difference between the matched objects (forecasts − reanalysis) normalized by the area of the reanalysis objects.

Table 1.

Figure 9 shows how the forecast objects were shifted in order to ensure the greatest amount of overlap with the reanalysis objects. The number of forecast objects that were shifted south is greater than the number of forecast objects that were shifted north for all forecast lead times. For example, approximately 19% of the objects were shifted at least 1° south and approximately 9% of the objects were shifted north in the 120-h forecast. No differences of note are found in the number of objects that were shifted either to the west or to the east. The majority of the objects were located in the correct position; for example, in the 120-h forecast approximately 70% of the objects were correctly located. We also found that shift and area errors do not vary much by season (not shown).

Fig. 9.
Fig. 9.

Percentages of the shifted objects for all the forecast lead times. The numbers along the horizontal axis show the percentage of forecast objects that have to be shifted to the west or to the east. Numbers along the vertical axis show the percentage of forecast objects that were shifted south or north. The percentages in the four quadrants indicate the fractions of objects that need to be shifted west or east and north or south. The range of the shift errors increases with forecast lead time (not shown).

Citation: Weather and Forecasting 31, 3; 10.1175/WAF-D-15-0147.1

4. Discussion

The presented object-based Rossby waveguide verification tool is applicable to any coarse resolution numerical weather prediction or climate model dataset (horizontal resolution > 0.5°). Most likely, it is not suitable for high-resolution fields, as the objects are expected to be very complex. To compare waveguide errors among different forecast lead times. it is essential to take into consideration that the number of the matched objects decreases with lead time (see also Fig. 7).

To apply the tool, three thresholds need to be defined subjectively. The first identifies the objects, the second removes small structures that are not part of the waveguide, and the third defines the matching objects.

The ERA-Interim reanalysis dataset was used as verification data. Although reanalysis is considered the best estimate, noise may be introduced through observation or calculation errors. The PV gradient requires the calculation of three derivatives starting from the wind field. Therefore, there is a possibility that occasionally a forecast could be closer to reality than the reanalysis. In general, the strength of the gradients was weaker in the forecast than in the reanalysis (for all the lead times) and systematically decreased with lead time. Moreover, an underestimation of the areas of the enhanced PV gradients for the year 2008 was found in the IFS model in comparison with the reanalysis dataset. Weaker gradients in the forecast imply spatial and temporal errors in the representation of wave breaking events and of wave amplification. Additionally, according to Gray et al. (2014), an underestimation of the PV gradient from the forecast can be accompanied by an underestimation of the humidity gradient, which leads to an underestimation of the radiative cooling.

Overall, the IFS model captures the positions of the waveguides, and most of the errors are associated with a northward shift in the forecast objects. This may be due to a systematic shift of the waveguides in the reanalysis or an amplitude error in the waveguides. Test runs propose that, in cases where ECMWF analysis data were used instead of the ERA-Interim reanalysis, there would be no differences in the shift vectors but the area errors would be smaller. The presented tool is able to highlight forecast errors in the representation of Rossby waves and could be combined with PV inversion tools to study the downstream behavior of the errors and the linked dynamical processes. A next step in the application of the tool is to compare it with other novel verification tools.

5. Summary

An object-based forecast verification tool for synoptic-scale Rossby waveguides based on the MODE method is presented. The aim of the verification tool is to analyze whether the waveguides have the same location and magnitude in the forecast as compared to the verification data. In this paper, the application is demonstrated using the IFS forecasts from ECMWF and the ERA-Interim reanalysis dataset.

The main steps in the Rossby waveguide verification tool are 1) the definition and automated identification of the waveguide objects, 2) the automated pairing of objects and the merging of objects, and 3) the calculation of object-based verification measures.

In the second part of this paper, the object-based forecast verification methodology has been applied to the IFS model for the year 2008 using five different forecast lead times. Results show that the model generally underestimates the area and the strength of the waveguides. This is in agreement with previous studies, such as that of Gray et al. (2014), who found a systematic underestimation of the PV gradient. Further on, the waveguide objects are generally in the correct position; however, more objects are located too far north compared with the reanalysis than too far south.

Studying long-term forecast errors is very helpful for identifying systematic errors that can be used to improve forecasting predictability. However, studying physical processes that lead to a misrepresentation of the synoptic-scale Rossby waveguides is also very important. The results of the waveguide verification tool provide the basis for the subsequent analysis of these processes by pinpointing time periods and areas in which the forecast was particularly poor.

Acknowledgments

This research was funded by the SNSF (200021-137543). We thank Dr. Suzanne Gray for valuable input and discussions. We thank the reviewers for their constructive comments.

REFERENCES

  • Anwender, D., Harr P. A. , and Jones S. C. , 2008: Predictability associated with the downstream impacts of the extratropical transition of tropical cyclones: Case studies. Mon. Wea. Rev., 136, 32263247, doi:10.1175/2008MWR2249.1.

    • Search Google Scholar
    • Export Citation
  • Barnes, E. A., and Screen J. A. , 2015: The impact of Arctic warming on the midlatitude jet-stream: Can it? Has it? Will it? Wiley Interdiscip. Rev.: Climate Change, 6, 277286, doi:10.1002/wcc.337.

    • Search Google Scholar
    • Export Citation
  • Blackburn, M., Methven J. , and Roberts N. , 2008: Large-scale context for the UK floods in summer 2007. Weather, 63 (9), 280288, doi:10.1002/wea.322.

    • Search Google Scholar
    • Export Citation
  • Chang, E. K., 2005: The impact of wave packets propagating across Asia on Pacific cyclone development. Mon. Wea. Rev., 133, 19982015, doi:10.1175/MWR2953.1.

    • Search Google Scholar
    • Export Citation
  • Cressman, G. P., 1948: On the forecasting of long waves in the upper westerlies. J. Meteor., 5, 4457, doi:10.1175/1520-0469(1948)005<0044:OTFOLW>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Davies, H. C., and Didone M. , 2013: Diagnosis and dynamics of forecast error growth. Mon. Wea. Rev., 141, 24832501, doi:10.1175/MWR-D-12-00242.1.

    • Search Google Scholar
    • Export Citation
  • Davis, C., Brown B. , and Bullock R. , 2006: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134, 17721784, doi:10.1175/MWR3145.1.

    • Search Google Scholar
    • Export Citation
  • Davis, C., Brown B. , Bullock R. , and Halley-Gotway J. , 2009: The Method for Object-based Diagnostic Evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC Spring Program. Wea. Forecasting, 24, 12521267, doi:10.1175/2009WAF2222241.1.

    • Search Google Scholar
    • Export Citation
  • Dee, D. P., and Coauthors, 2011: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553597, doi:10.1002/qj.828.

    • Search Google Scholar
    • Export Citation
  • ECMWF, 2008: Changes in ECMWF model: CY33r1. [Available online at http://www.ecmwf.int/en/forecasts/documentation-and-support/changes-ecmwf-model.]

  • Fehlmann, R., and Davies H. C. , 1997: Misforecasts of synoptic systems: Diagnosis via PV retrodiction. Mon. Wea. Rev., 125, 22472264, doi:10.1175/1520-0493(1997)125<2247:MOSSDV>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Gilleland, E., Ahijevych D. , Brown B. G. , Casati B. , and Ebert E. E. , 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24, 14161430, doi:10.1175/2009WAF2222269.1.

    • Search Google Scholar
    • Export Citation
  • Gray, S., Dunning C. , Methven J. , Masato G. , and Chagnon J. , 2014: Systematic model forecast error in Rossby wave structure. Geophys. Res. Lett., 41, 29792987, doi:10.1002/2014GL059282.

    • Search Google Scholar
    • Export Citation
  • Grazzini, F., and Vitart F. , 2015: Atmospheric predictability and Rossby wave packets. Quart. J. Roy. Meteor. Soc., 141, 27932802, doi:10.1002/qj.2564.

    • Search Google Scholar
    • Export Citation
  • Hoskins, B., and Woollings T. , 2015: Persistent extratropical regimes and climate extremes. Curr. Climate Change Rep., 1 (3), 115124, doi:10.1007/s40641-015-0020-8.

    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., and Stephenson D. B. , 2012: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. John Wiley and Sons, 292 pp.

    • Search Google Scholar
    • Export Citation
  • Langland, R. H., Shapiro M. A. , and Gelaro R. , 2002: Initial condition sensitivity and error growth in forecasts of the 25 January 2000 East Coast snowstorm. Mon. Wea. Rev., 130, 957974, doi:10.1175/1520-0493(2002)130<0957:ICSAEG>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Lee, S., and Held I. M. , 1993: Baroclinic wave packets in models and observations. J. Atmos. Sci., 50, 14131428, doi:10.1175/1520-0469(1993)050<1413:BWPIMA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Martínez-Alvarado, O., Madonna E. , Gray S. , and Joos H. , 2015: A route to systematic error in forecasts of Rossby waves. Quart. J. Roy. Meteor. Soc., 142, 196210, doi:10.1002/qj.2645.

    • Search Google Scholar
    • Export Citation
  • Martius, O., Schwierz C. , and Davies H. , 2008: Far-upstream precursors of heavy precipitation events on the Alpine south-side. Quart. J. Roy. Meteor. Soc., 134, 417428, doi:10.1002/qj.229.

    • Search Google Scholar
    • Export Citation
  • Martius, O., Schwierz C. , and Davies H. , 2010: Tropopause-level waveguides. J. Atmos. Sci., 67, 866879, doi:10.1175/2009JAS2995.1.

  • Rodwell, M. J., and Coauthors, 2013: Characteristics of occasional poor medium-range weather forecasts for Europe. Bull. Amer. Meteor. Soc., 94, 13931405, doi:10.1175/BAMS-D-12-00099.1.

    • Search Google Scholar
    • Export Citation
  • Rossby, C.-G., 1945: On the propagation of frequencies and energy in certain types of oceanic and atmospheric waves. J. Meteor., 2, 187204, doi:10.1175/1520-0469(1945)002<0187:OTPOFA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Schlemmer, L., Martius O. , Sprenger M. , Schwierz C. , and Twitchett A. , 2010: Disentangling the forcing mechanisms of a heavy precipitation event along the Alpine south side using potential vorticity. Mon. Wea. Rev., 138, 23362353, doi:10.1175/2009MWR3202.1.

    • Search Google Scholar
    • Export Citation
  • Schwierz, C., Dirren S. , and Davies H. C. , 2004: Forced waves on a zonally aligned jet stream. J. Atmos. Sci., 61, 7387, doi:10.1175/1520-0469(2004)061<0073:FWOAZA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Waliser, D., and Moncrieff M. , 2008: The Year of Tropical Convection (YOTC) Science Plan: A joint WCRP–WWRP/THORPEX international initiative. WMO/TD-No. 1452, 26 pp. [Available online at https://www.wmo.int/pages/prog/arep/wwrp/new/documents/YOTC_Science_Plan.pdf.]

  • Wirth, V., and Eichhorn J. , 2014: Long-lived Rossby wave trains as precursors to strong winter cyclones over Europe. Quart. J. Roy. Meteor. Soc., 140, 729737, doi:10.1002/qj.2191.

    • Search Google Scholar
    • Export Citation
Save