1. Introduction
Even when considering the sizeable progress recently achieved with global ensemble prediction systems (EPSs; e.g., Buizza et al. 2005), deterministic global models still constitute a very important tool for medium-range numerical weather prediction (NWP) at most national centers. Over the last decade, the horizontal and vertical resolutions of deterministic models have greatly increased, along with significant improvements in their representation of physical processes. For example, the current Global Spectral Model (GSM) used at the Japan Meteorological Agency (JMA) now has a spectral resolution of TL959 (about 20-km grid size) and includes recent improvements to convection and radiation. At the European Centre for Medium-Range Weather Forecasts (ECMWF), the spectral resolution of the Integrated Forecast System (IFS) is T799 (about 25 km), with improved physics for convection, condensation, radiation, and surface drag (see Bougeault 2008). At the U.S. National Centers for Environment Prediction (NCEP), the horizontal spectral resolution is now T382 (estimated to be approximately equivalent to a grid size of 35 km), with many new improvements to the surface processes.
All of these efforts have contributed to a reduction in the errors for medium-range predictions of mass and wind fields in the Northern and Southern Hemispheres (e.g., Jung et al. 2006; Drusch and Viterbo 2007; Morcrette et al. 2007). But their impact on medium-range quantitative precipitation forecasts (QPFs) is less well documented, if at all. Obviously, improving medium-range QPFs is of primary importance, not only in meteorology, with all its impact on the general population, transport, as well as on industrial and touristic activities, but also for other applications such as hydrology (river flow management, flood prediction) and agriculture (soil wetness).
Several years ago, within the context of short-range NWP, it was realized that increasing horizontal resolution alone is not sufficient to improve QPFs, unless such an increase is combined with physical schemes that are more appropriate for the new model grid size or, in this particular case, for meso-β-scale models [i.e., 10–40-km grid size; Bélair et al. (1994), (2000)]. This “combined” strategy has been adopted in Canada, where a 33-km version of the Global Environmental Multiscale (GEM; Côté et al. 1998a,b) model has been implemented at the Meteorological Service of Canada (MSC) for medium-range NWP, together with a physical package that is very similar to that used in the 15-km regional short-range NWP system (Mailhot et al. 2006). As discussed in Bélair et al. (2005), this physical configuration is particularly well suited for the prediction of clouds and precipitation in meso-β-scale models.
Our main objective in this study is thus to examine the impact on medium-range QPFs resulting from the operational implementation of this 33-km global model with improved “mesoscale” physics.
Details of the new 33-km global forecast system are given in the next section. The experimental setup is described in section 3. An objective evaluation of QPFs resulting from the new global system is presented in section 4, along with similar evaluations for the lower-resolution global system previously operational at MSC and for the higher-resolution regional short-range forecast system. Analysis and discussion follow in section 5, and the final section offers an outlook on future developments at MSC related to medium-range QPFs.
2. Canada’s 33-km global forecast system
The new GEM version for global medium-range forecasting was implemented at MSC on 31 October 2006. In this implementation, both the horizontal and vertical resolutions of the global forecast system were significantly increased. The horizontal grid size was reduced from 100 to 33 km, and the number of levels in the vertical was increased from 28 to 58. As shown in Table 1, the differences between MSC’s new and old global systems are broader and more general than just increasing the resolution, and involve important modifications to GEM’s numerics–dynamics and physics. In Table 1, GLBOLD refers to the global system previously operational at MSC, GLBNEW refers to the new global system, and REG refers to MSC’s short-range regional (i.e., continental) system (Mailhot et al. 2006).
For GLBNEW’s numerical–dynamical configuration, the time step was decreased from 45 to 15 min and the sponge layer at the top of the model (to minimize the negative impact of spurious waves reflecting from the model top) was increased from one to four levels. It should also be noted that the latitudinal and longitudinal grid lengths of the computational grid are not the same with the new global system (i.e., 0.45° grid size in the longitudinal direction of the grid and 0.3° in the latitudinal direction), which translates into equal spatial scales for both directions at about 49° north and south (grid size ∼33 km). This is in contrast with GLBOLD, in which the latitudinal and longitudinal grid sizes are the same, which leads to isotropic spatial resolution at the computational grid’s equator (grid size ∼100 km).
The modifications that were done to the physical package are as compelling, as they relate to many important physical processes. One major objective with this new global model was to represent condensation and precipitation in a way similar to what is done in MSC’s REG system (Bélair et al. 2000; Mailhot et al. 2006). To achieve this, the Kuo scheme (see Kuo 1974; Mailhot et al. 1989) for deep convection was replaced by the Kain and Fritsch (1990, 1993) scheme. And a new scheme for shallow convection [Kuo transient, based on the work of Girard and colleagues; see Bélair et al. (2005)] was included, together with a modified version of the grid-scale stable condensation scheme (Sundqvist 1978; Pudykiewicz et al. 1992). Changes in this last scheme are related to the improved representation of evaporation below the cloud base. In the turbulent vertical diffusion scheme, the Bougeault and Lacarrère (1989) mixing length is now used (see also Bélair et al. 1999). The Interactions between Surface, Biosphere, and Atmosphere (ISBA; see Noilhan and Planton 1989) land surface scheme is now replacing the previous force–restore scheme (Benoit et al. 1989), with initial conditions provided by a sequential data assimilation based on optimal interpolation (see Bélair et al. 2003a,b for details).
As can be seen in Table 1, GLBNEW’s configuration is indeed similar to that of REG, but is not identical. Because REG’s main concern is with short forecast ranges (up to 48 h), its numerical–dynamical configuration is quite different from that of GLBNEW. Even though its computational grid is still global, REG’s horizontal resolution is variable with a uniform-resolution domain over North America (see Fig. 1 in Bélair et al. 2000). Also, REG’s horizontal grid size is smaller compared with GLBNEW (approximately 15 km over North America) and the time step is shorter (7.5 min).
There are also a few differences in the vertical diffusion schemes for turbulence. The regional system includes a newly implemented statistical treatment of boundary layer clouds called MoisTKE (see Mailhot and Bélair 2002; Bélair et al. 2005), compared with the global model in which the effect of boundary layer clouds is represented by a simple modification of the Richardson number (based on Geleyn 1987). Also, REG’s turbulent mixing length is given by an asymptotic value (see Mailhot et al. 1998), and not the Bougeault and Lacarrère (1989) length. It should also be mentioned that a four-dimensional variational data assimilation (4DVAR) approach is used for upper-air data assimilation in the global forecast system (Gauthier et al. 2007; Laroche et al. 2007), whereas REG still operates with a three-dimensional variational data assimilation system (3DVAR; Laroche et al. 1999).
3. Parallel run and experimental setup
Prior to its operational implementation, the new 33-km global forecast system was thoroughly evaluated, both subjectively and objectively. As a last and crucial part of this evaluation, GLBNEW was run in parallel with MSC’s previous global system (i.e., GLBOLD). This parallel run lasted 2 months, starting on 31 August 2006 and ending with GLBNEW’s operational implementation on 31 October 2006. It produced a total of 122 medium-range (10 day) forecasts for each of the two systems (61 days with two integrations per day, at 0000 and 1200 UTC).
Objective evaluation of this parallel run against upper- air analyses and radiosondes observations revealed better performance at the short and medium ranges for GLBNEW. As an example, Fig. 1 shows biases and root-mean-square errors (RMSEs) obtained from comparing GLBOLD’s and GLBNEW’s 96-h forecasts against observations from North American radiosondes. At this medium range, a reduction in the RMSEs is evident for geopotential height and temperature (black lines in the top panels of Fig. 1), whereas the RMSEs are similar for winds and slightly increased for humidity (represented by the dewpoint depression, i.e., the difference between the temperature and the dewpoint temperature). Although beyond the scope of this article, it is worth mentioning that further investigation of these last results indicates that comparison of the two global systems’ RMSEs may not be fair for variables such as winds and humidity, due to their much larger variances in the higher-resolution GLBNEW system.
In general, biases are also improved with GLBNEW. As shown in Fig. 1b, GLBOLD’s warm bias below 600 hPa is reduced with the new system, as well as the cold bias above 500 hPa. This leads to a bias for geopotential height that is more constant with height (Fig. 1a), even though it is slightly larger then that of GLBOLD for most of the troposphere. It is worth mentioning that wind speeds are increased with GLBNEW, with smaller biases for most of the troposphere (Fig. 1c). For humidity, the dewpoint depression biases are more positive in the upper troposphere (i.e., 300–700 hPa), indicating that GLBNEW is generally drier for this layer (Fig. 1d).
It is these 122 parallel-run forecasts that are used in this study to evaluate the QPF improvement resulting from implementing GLBNEW. Two different observational surface networks are commonly used at MSC to objectively evaluate QPFs over North America: the conventional surface synoptic observations (SYNOPs) and the U.S. Cooperative Observer Program (COOP) available in standard hydrometeorological exchange format (SHEF; NOAA 1996). The spatial density of the COOP network is much greater than that of the SYNOPs network. But for several reasons, including the facts that the COOP network does not cover Canada, that observations from this network suffer from observer bias (see Daly et al. 2007), and that only one observation per day is available for most COOP stations (daily accumulations valid at 1200 UTC), only evaluation results using SYNOPs are presented in this study.
It should be mentioned, however, that objective evaluations of QPFs were performed using both networks, and were found to lead to equivalent conclusions. In fact, the statistical performance measures presented in this study are relatively stable due to the large number of cases. But confidence intervals (which could have been computed using resampling techniques such as the bootstrap method) are not plotted in the figures below.
As can be seen in Fig. 2, the spatial coverage of SYNOPs is relatively dense over the southern portion of Canada, where most of the country’s population resides. This is particularly the case over southern Alberta, Saskatchewan, and Manitoba, and in the southern Ontario–St. Lawrence River corridor. Interestingly, a greater number of U.S. SYNOP stations are manual (red), in contrast with Canada where more stations are automatic (blue).
Other relevant information concerning the objective evaluation of QPFs is as follows: (i) the evaluation period is in the fall season, that is, preceding the first snowfall for most of the stations shown in Fig. 2, thus avoiding biases that could be induced by severe underestimation of measured snowfall from automatic precipitation gauges (Goodison et al. 1998; Fortin et al. 2008); (ii) objective evaluation scores of QPFs for the 15-km short-range forecast system (REG) are shown and compared with those of the two global systems (GLBNEW and GLBOLD), but only for forecast ranges less than or equal to 48 h; (iii) the observation–prediction pairs used for the objective evaluation of QPFs are directly obtained by horizontally interpolating model outputs to surface stations; as discussed in Tustison et al. (2001), this process does not account for scale-dependent errors associated with the interpolation, referred to as representativeness errors; and (iv) otherwise, the comparison between GLBNEW and GLBOLD is clean, in the sense that the initial conditions for both systems are produced using 4DVAR, with identical cutoff times for the observations (i.e., 3-h delay after the initial forecast times at 0000 and 1200 UTC); this is not so for the comparison with REG, for which the assimilation is less advanced (i.e., 3DVAR) and the cutoff time is shorter (i.e., 1-h 40-min delay).
4. Objective evaluation of QPFs
a. Contingency tables
All the performance measures presented in this study are based on contingency tables, which are often used to analyze the statistical relationship between two categorical variables. In this study the variables are the observed and predicted daily precipitation accumulations (referred to in the text below as daily accumulations or precipitation intensities). The performance measures are not obtained from averaging scores that have been computed separately at each station or at each date, but are rather computed from summary contingency tables. These tables have the form as shown in Table 2, where the numbers 1, 2, … , K − 1, and K on the left and top of Table 2 refer to specific categories of daily accumulations (e.g., between 5 and 10 mm day−1): n(Fi,Oj) is the number of events with predicted (or forecast) accumulations in the ith category and observed accumulations in the jth category, N(Fi) is the marginal total of predicted events in the ith category, N(Oj) is the marginal total of observed events in the jth category, and N is the grand total of all the events.
Many of the performance measures discussed in this section are based on simpler 2 × 2 contingency tables (see Table 3) in which H is for the hits, that is, the number of events both predicted and observed; FA is for the false alarms, that is, the number of events predicted but not observed; M is for the misses, that is, the number of events observed but not predicted; and CN is for the correct negatives, that is, the number of events not predicted and not observed.
b. Marginal totals analysis and biases
A first approach to evaluating the performance of each forecast system (GLBOLD, GLBNEW, and REG) is to compare the distributions of marginal totals for predictions and observations, that is, N(Fi) versus N(Oi). In Fig. 3, the total number of events is shown for each category of daily accumulations (0–0.2 mm day−1, 0.2–5 mm day−1, …). Because the REG system produces forecasts up to 48 h, only results from the first 2 days of integration are used (statistics from 0–24-h accumulations and from 24–48-h accumulations are combined) to properly compare distributions and biases from the global configurations (GLBOLD and GLBNEW) with those from the regional system (REG). This does not influence the interpretation of Fig. 3’s histograms for the two global systems since these are relatively insensitive to model lead time (not shown). It is noteworthy that a logarithmic scale is used for the ordinate axis, due to the exponential nature of precipitation intensity distributions (Tremblay 2005).
Results displayed in Fig. 3 indicate that the old global system (GLBOLD, in yellow) underestimates the number of events in the no-precipitation category (0–0.2 mm day−1) and overestimates the number of events for small daily accumulations (0.2–5 mm day−1). There is also an important underestimation with this configuration for events with substantial accumulations (greater than 30 mm day−1). The new global system (GLBNEW, in red) better represents the marginal totals distribution, even though it slightly overestimates the number of events for accumulations between 0.2 and 30 mm day−1, and the opposite for accumulations greater than 40 mm day−1. These results are similar to those of the regional forecast system (REG, in blue), except for the number of events, which is larger with the regional system for all the categories (i.e., accumulations greater than 0.2 mm day−1) and which compares better for larger daily accumulations (i.e., greater than 40 mm day−1).
Information on the marginal totals can be displayed differently by multiplying the number of events with the mean precipitation for each of the daily precipitation categories. In this approach, the mean precipitation of each specific category is obtained by assuming an exponential form of the marginal totals distribution (see Fig. 3). The resulting plot displays the total precipitation accumulations (in m) for each category, referred to below as the total precipitation per category.
These totals, shown in Fig. 4, emphasize the differences pointed out from the frequency distributions shown in Fig. 3. For instance, total precipitation per category with GLBOLD is greatly overestimated for weak precipitation intensities (0.2–5 mm day−1) and severely underestimated for larger intensities (greater than 20 mm day−1). The other two systems, GLBNEW and REG, have totals of precipitation per category that compare more favorably with the observations, but they still overestimate this quantity for daily accumulations between 0.2 and 30 mm day−1. And as expected, the total precipitation per category from the REG system is larger than that from GLBNEW for all the categories.
An interesting quantity that can be derived from this figure is the total precipitation (including all categories) for observations and for the three forecast systems. The summation results, given by the horizontal lines in the right portion of Fig. 4, indicate that GLBOLD performs best for this quantity, in spite of its poor representation of the frequency and precipitation distributions. It appears that GLBOLD’s overestimation for weak precipitation intensities is canceled by its underestimation for greater intensities. The new global forecast system, GLBNEW, has total precipitation that is slightly larger than GLBOLD, but is much better than the REG system, which overestimates the total precipitation by about 20%. (This difference between GLBNEW and REG is linked to the parameters of the Kuo transient shallow convective scheme that are not the same for the two model configurations; these parameters are mainly related to the treatment of precipitation under the cloud base and will be harmonized in upcoming implementations at MSC.) This objective evaluation of the total precipitation is important for hydrological applications. For instance, the overestimation of the total water with the REG system has been found to have a negative impact on soil moisture assimilation and on river flow modeling systems currently under development at Environment Canada.
Figure 5 confirms that GLBOLD markedly overestimates the number of events with daily accumulations larger than small quantities (i.e., with thresholds smaller than about 5 mm day−1) and severely underestimates the number of events with large daily accumulations. With this system, the bias is about 0.2 for daily accumulations greater than 40 mm day−1. Overestimation of the number of predicted events with small daily accumulations is diminished with GLBNEW, with a crossover (B = 1) at daily accumulations of about 20 mm day−1. The bias is approximately 0.6 for daily accumulations greater than 50 mm day−1. REG, on the other hand, mainly overestimates the number of precipitation events for thresholds smaller than 50 mm day−1. This large (and positive) bias for REG has been a permanent characteristics of this system for many years (see Bélair et al. 2000; Mailhot et al. 2006).
c. Equitable threat scores
Proposed by Schaefer (1990), who called it the Gilbert skill score, the ETS is considered to be an improvement over the threat score [also called the critical success index; see Donaldson et al. (1975)], which was shown to depend on bias (Mason 1989; Baldwin and Kain 2006). Yet, the ETS is also sensitive to bias and somehow benefits from bias values above one (e.g., Hamill 1999). This deficiency led to further variants of this score proposed by Mesinger and Brill (2004) and Baldwin and Kain (2006). But as argued in this last study, these new scores as well as other scores such as the true skill statistic (Doswell et al. 1990) and the odds ratio skill score (Stephenson 2000) are all susceptible to hedging in one form or another. The ETS has been chosen in the present study because of its wide use in the operational community, including at MSC where it has been utilized for more than a decade.
Figure 6 displays the ETSs for several categories of precipitation intensity for the three forecast systems. It is clear from this figure that MSC’s new global system (GLBNEW) generally improves the ETS compared to GLBOLD. In particular, the ETS for the 0–0.2 mm day−1 category is noticeably increased, showing the new model’s improved accuracy in predicting the occurrence (or nonoccurrence) of precipitation. Remarkably, the performance difference between the two systems remains noticeable for this category even for day 5 predictions (96–120 h). For the other categories, GLBNEW’s greatest improvements over GLBOLD are for the largest precipitation intensities. This improvement, though, is reduced with lead time so that the day 5 ETSs from both GLBNEW and GLBOLD are similar for most categories of precipitation intensity.
In Fig. 7, the ETS is presented as a function of the precipitation intensity thresholds instead of the accumulation categories. The same type of separation between the two global systems is evidenced in Fig. 7, with larger ETS values from GLBNEW compared with GLBOLD for all thresholds of precipitation intensity. As observed for the categorical score shown in Fig. 6, the differences between GLBNEW’s and GLBOLD’s ETS values tend to decrease in the medium range. As an example, for the 30 mm day−1 threshold, the ETS differences between the two global systems go from 0.10 for day 1 to 0.08 for day 3 and to 0.04 for day 5. Both Figs. 6 and 7 indicate, as expected, that the day 1 ETS for GLBNEW is comparable with the ETS from the REG system, for all categories.
The ETS drop with lead time is evidenced in Fig. 8 for three different precipitation intensities: 0.2, 10, and 30 mm day−1. For each of these thresholds, GLBNEW’s ETS values are much larger than those of GLBOLD, and similar to those of REG. The ETS comparison for the two global systems obviously depends on precipitation intensity. For prediction of precipitation occurrence versus nonoccurrence, that is, for the 0.2 mm day−1 threshold, the gain in predictability at the medium range is of the order of 60 h. That is, GLBNEW’s ETS at 84 h is as good as GLBOLD’s ETS at 24 h. The gain in predictability is on the order of 24 h for the 10 mm day−1 threshold, and between 36 and 48 h for the 30 mm day−1 threshold.
The convergence of GLBNEW and GLBOLD ETSs at day 5 coincides with a general lack of accuracy for precipitation predicted by the two systems. This behavior is more evident for the 10 and 30 mm day−1 thresholds, as GLBOLD’s ETS seems to stabilize toward an asymptotic value that appears to be on the order of 0.1. Unfortunately, the objective evaluation of QPFs conducted prior to GLBNEW’s operational implementation was not done for predictions longer than 5 days; this would have allowed for a more precise documentation of the asymptotic behavior of the ETS.
Implicitly, the ETS should tend toward zero as lead time increases, that is, as the model loses accuracy and the number of hits H tends toward the number of hits associated with random chance [i.e., Hr in (11)]. But because precipitation may be organized in response to stationary forcing from the surface, such as precipitation forced by coastal orography, atmospheric models may preserve some residual or climatological accuracy to predict QPFs, which would lead to asymptotic values for ETS that would be greater than zero. This nonzero asymptotic value, which may be different for GLBOLD and GLBNEW, due to their different representations of the physical processes and different resolutions, could be used as a reference for new accuracy measures that would indicate the atmospheric models’ performance relative to a “residual accuracy” state.
d. Relative economic value
The relative economic values of the QPFs from GLBOLD, GLBNEW, and REG are shown in Fig. 9 for day 1, day 3, and day 5 predictions, and for intensity thresholds of 0.2, 10, and 30 mm day−1. It should first be noted that the position (cost–loss ratio) of the maximum relative economic value does not depend on the forecast system, but is rather equal to the climatological occurrence frequency of the event (Richardson 2000). Peaks of the economic value thus occur at similar cost–loss ratios for a particular precipitation intensity, insensitive of lead time, and at smaller cost–loss ratios for more intense precipitation. For instance, the maximum relative economic values are found at C/L ∼ 0.4 for the 0.2 mm day−1 intensity, at C/L ∼ 0.08 for the 10 mm day−1 intensity, and at C/L ∼ 0.02 for the 30 mm day−1 intensity.
The range of cost–loss ratios with positive economic values (hereafter referred to as the “beneficial” range) is centered near the C/L ∼ 0.4 ratio for the 0.2 mm day−1 precipitation intensity (i.e., occurrence versus nonoccurrence; see the top panels in Fig. 9). For this low-intensity threshold, the maximum economic value is greater with GLBNEW, with a wider beneficial range of cost–loss ratios and a slight shift toward larger ratios. Although slightly diminished, this advantage of GLBNEW over GLBOLD is also found for day 3 precipitation. The two systems mostly have the same economic values for day 5 precipitation, both with narrow beneficial ranges.
For more intense precipitation (10 and 30 mm day−1, middle and bottom panels in Fig. 9), the beneficial range is shifted toward smaller ratios for the three forecast systems, due to the lower frequency of these events. This transition indicates that predictions of large precipitation accumulations are profitable to users for which the loss associated with an unpredicted event greatly exceeds the cost of preventive actions. Such users are less sensitive to false alarms, which explains the positive relative economic values over this range. For these precipitation intensities, the economic value of GLBNEW’s day 1 and day 3 QPFs is greater than those for GLBOLD. This improvement is more important for the 30 mm day−1 intensity. The beneficial range is similar for the two global forecast systems.
Interestingly, GLBNEW’s economic values compare well with those of the higher-resolution regional system REG. Although the maximum economic value of GLBNEW is either the same or slightly below that of REG, its beneficial range is marginally wider.
According to this measure, GLBNEW’s QPFs have more value to users than those from the previous low-resolution global system GLBOLD, and similar value to those from the high-resolution short-range continental system REG. The greatest impact of the new system is found for the 0.2 mm day−1 precipitation threshold. For this precipitation intensity, the integrated economic value of GLBNEW’s day 3 (72 h) QPFs is as large as the value for GLBOLD’s day 1 (24 h) QPFs, that is, a gain of 48 h. This difference between the two global systems shrinks with lead time. Day 4 (96 h) QPFs from GLBNEW have as much value as GLBOLD’s day 2 (48 h) QPFs, a gain of more than 40 h. For day 5 QPFs, GLBNEW’s edge is on the order of 20 h.
For the larger intensity thresholds (10 and 30 mm day−1), the integrated economic values of GLBNEW’s QPFs also surpass that of GLBOLD, but to a lesser degree. For the 10 mm day−1 threshold, GLBNEW’s gain compared with GLBOLD is on the order of 18–24 h, except for day 5 predictions, which have similar values for the two systems. For the 30 mm day−1 intensity, GLBNEW’s QPFs still have more value, but this value remains low even for short-range forecasts.
5. Analysis and discussion
To provide a broad and general assessment of the quality and improvements of QPFs resulting from Canada’s implementation of a new 33-km global deterministic forecast system, several types of performance measures were examined in the previous section. These metrics essentially focus on three aspects of the evaluation of QPFs: bias, accuracy, and user value.
Evidence presented in the preceding section is consistent for these three aspects: it essentially indicates that the new global forecast system (GLBNEW) agrees considerably better with observations, relative to the performance of the previous global system (GLBOLD). The biases, for instance, are considerably less with GLBNEW, compared with GLBOLD, which severely overpredicts (underpredicts) the frequencies and total amounts associated with weak (strong) precipitation intensities. The accuracy and value scores, on the other hand, reveal gains of at least 12 h and even up to 72 h for medium-range QPFs (i.e., day 3–5 predictions).
Several factors are responsible for this substantial improvement in medium-range QPFs. But because this study’s experimental setup is intimately linked with GLBNEW’s operational implementation at MSC, the impact of each individual component of this implementation (e.g., vertical and horizontal resolutions, physical processes) could not be precisely quantified. Our objective in this study is rather to describe the overall impact of the new system, including all changes to the model numerics–dynamics, physics, and assimilation. Quantifying the impact that each of these changes has on QPFs is beyond the scope of the present article and would, in fact, require the publication of several articles.
Certainly, modifications to GEM’s dynamical and numerical configuration (see Table 1) have a positive impact on medium-range weather predictions. A significant portion of the positive results for upper-air evaluation (shown in Fig. 1) can be directly attributed to the increase in the horizontal and vertical resolutions, although changes to the representation of physical processes (e.g., condensation–precipitation, turbulence, and surface) also contribute in an important manner.
Increasing horizontal resolution also has an impact on QPFs’ objective evaluation. It is not surprising, for instance, that biases and precipitation distributions shown in Figs. 3 –5 are notably improved with the high-resolution GLBNEW. As is often observed, the number of events with small (large) accumulations is decreased (increased) with the higher-resolution system because of its ability to represent smaller-scale circulations and more intense precipitation events. For QPFs’ accuracy measures, on the other hand, increasing horizontal resolution may have a negative impact due to the larger spatial and temporal variances of the QPFs outputs, which normally translate into a deterioration of accuracy scores (e.g., ETS, RMSE). The important improvement displayed in this study for ETS and relative economic value is thus more likely related to a better treatment of the physical processes in GLBNEW (for clouds, precipitation, turbulence, and surface processes) than to horizontal resolution.
Obviously, QPFs are sensitive to GEM’s physical schemes for clouds, condensation, and precipitation. Bélair et al. (1994, 2000) have shown that deep convection schemes such as Kain–Fritsch or Fritsch and Chappell (1980), based on convective inhibition (CIN) and the release of convective available potential energy (CAPE), allow for more realistic interactions between implicit (subgrid scale) and explicit (grid scale) cloud processes. This partition between implicit and explicit processes is of critical importance for QPFs, as discussed in Molinari and Dudek (1992), Zhang et al. (1994), and Bélair and Mailhot (2001). When using the Kain–Fritsch convective scheme, the grid-scale condensation scheme (i.e., Sundqvist) is responsible for a larger portion of the total precipitation. This is in comparison with GLBOLD’s Kuo scheme, based on an equilibrium between the convective precipitation and the large-scale production of low-level convergence, which is overly active and dominates for all types of precipitation, producing weak precipitation for very large areas (Bélair et al. 1994, 2000). This explains the major adjustment of QPFs’ biases with the new system, for which the number of weak (strong) precipitation events is diminished (increased) (Fig. 5).
In addition to the Kain–Fritsch scheme for deep convection, the Kuo transient scheme for shallow cumulus activity also influences the balance between implicit and explicit cloud processes, by more realistically parameterizing the effects of low-level cloud activity that otherwise would have been captured by the grid-scale condensation scheme. As shown in Bélair et al. (2005), the Kuo transient scheme has a stabilizing effect (in the lower troposphere) that differs from the increased vertical transport typically associated with boundary layer cloud schemes. This vertical adjustment by the Kuo transient was found in Bélair et al. (2005) to reduce the amount of nonconvective cloud in a large-scale system over the Pacific Ocean, in better agreement with satellite observations. Of greater importance for the present study, it was also found to reduce the precipitation in the rear portion of this system, which could be another reason explaining the decrease of QPFs’ biases with GLBNEW.
Other components of GLBNEW’s implementation also have beneficial effects on QPFs. For instance, the implementation of the ISBA land surface scheme, together with a sequential assimilation for soil moisture and surface temperature, was found to have a noticeable positive impact on medium-range QPFs—even larger than results presented in Bélair et al. (2003a) for short-range prediction. Surface fluxes of heat and humidity strongly influence the diurnal evolution and vertical structure of the planetary boundary layer. Since precipitation and condensation processes are quite sensitive to low-level heat and humidity contents, there is a direct link between an improved representation of the daytime well-mixed layer (in large part driven by surface fluxes from the new land surface scheme) and QPFs.
The improvement of the QPFs resulting from GLBNEW’s implementation (compared with GLBOLD’s QPFs) decreases with lead time as the two global systems tend to perform similarly at longer range. (For 5 day QPFs, the two systems exhibit similar scores.) The asymptotic behavior observed for some of the performance measures could be used as an indicator of QPFs’ lack of skill and value for longer-range predictions. This is certainly obvious with GLBOLD’s forecasts, for which QPFs’ scores tend to plateau around days 4 or 5. This stabilization of the scores depends on the precipitation intensity; it occurs earlier for larger intensities.
For GLBNEW, however, the performance measures tend to decrease almost linearly with time without reaching this asymptotic state, even for the 30 mm day−1 precipitation threshold. Presumably, GLBNEW’s performance measures for QPFs would get to such a plateau if the evaluation was performed over longer integrations. The fact that this does not happen in the first 5 days of integration adds to the difficulty of attributing skill and value in an absolute manner to GLBNEW’s QPFs. Indeed, asymptotic values of ETS or of the area under economic value curves could provide additional information regarding the absolute performance of the prediction system, that is, not relative to another prediction system. In this respect, all we can say from the results presented in this study is that the asymptotic values are reached later in the integrations for the GLBNEW system, indicating that the skill and value in GLBNEW’s QPFs for day 5 predictions are still greater than that of climatology. Moreover, the area under the relative economic value curves for the 0.2 and 10 mm day−1 thresholds remains important for day 3 QPFs, and is still positive for day 5 QPFs (although small).
It is worth mentioning that the objective evaluation methods used in this study could be refined to consider the scale-dependent representativeness errors discussed in Tustison et al. (2001). Even though it is unlikely that this uncertainty would considerably influence the conclusions drawn in the present work (due to the sizeable differences in performance between GLBOLD and GLBNEW), it may be necessary in future implementations to examine more sophisticated verification approaches, such as the composite method proposed in Tustison et al. (2001). It may also be helpful to examine techniques such as the intensity-scale approach proposed by Casati et al. (2004) and the object-oriented approach described in Ebert and McBride (2000), among others.
6. Outlook
Improving QPFs at the medium-range should remain a priority for many years to come. In Canada, efforts to achieve this objective include the continued development of probabilistic products from the global EPS system, the improvement of the global deterministic model’s numerics and physics, the improvement of initial conditions by increasing the amount of data assimilated in the 4DVAR and in the ensemble Kalman filter used for the global EPS (Houtekamer and Mitchell 2005), and the use of more sophisticated approaches to evaluate the quality of medium-range QPFs.
Several modifications to MSC’s global deterministic system are expected for operational transfer in the near future. An upgrade of the global system with the model lid at 0.1 hPa (as opposed to 10 hPa in GLBNEW) is in its final phase of acceptance at MSC Operations. In this model, the number of computational levels has been increased from 58 to 80, a nonorographic gravity wave drag based on Hines (1997a,b) has been implemented, and a new radiation scheme from Li and Barker (2005) using a correlated-k distribution for gaseous transmission is included. Other improvements to the model physics, related to the representation of deep convection [e.g., inclusion of the Bechtold et al. (2001) scheme], of slantwise convection (see Ma 2000; Morcrette and Browning 2006), and of land surface processes (inclusion of the Canadian Land Surface Scheme, CLASS; Verseghy 1991; Verseghy et al. 1993) are currently being explored.
Improvements to medium-range QPFs are also expected from better initial conditions, both in the atmosphere and at the surface. For upper-air data assimilation, positive impact should come from substantial increases in the volume of space-based observations that are expected for the 4DVAR data assimilation system. The assimilation of cloud and precipitation data may also be an important source of improvement (see Errico et al. 2007). For the surface, a new Canadian Land Data Assimilation System (Balsamo et al. 2006, 2007), including space-based information on soil moisture, terrestrial snow, and vegetation, is expected to be available for operation in 3–5 yr. Progress toward these achievements is anticipated in the next few years.
Acknowledgments
The operational implementation of Canada’s new 33-km global system for medium-range weather forecast required several years of research and development, and included contributions from many scientists, research assistants, operational meteorologists, computer experts, and managers. Although the list of these people is too long to be detailed here, we thank all of them. More specifically, we would like to highlight the contributions of Jean-Marc Bélanger and François Lemay, who had the task of running the final preimplementation tests prior to submission to MSC’s decisional committee. The precious help from Rochdi Lahlou in the production of the QPF scores is acknowledged. Thanks are also due to senior managers both in the Science and Technology Branch (Gilbert Brunet) and at MSC (Pierre Dubreuil, André Méthot, and Louis Lefaivre) for their support throughout this project.
REFERENCES
Baldwin, M. E., and Kain J. S. , 2006: Sensitivity of several performance measures to displacement error, bias, and event frequency. Wea. Forecasting, 21 , 636–648.
Balsamo, G., Mahfouf J-F. , Bélair S. , and Deblonde G. , 2006: A global root-zone soil moisture analysis using simulated L-band brightness temperature in preparation for the Hydros satellite mission. J. Hydrometeor., 7 , 1126–1146.
Balsamo, G., Mahfouf J-F. , Bélair S. , and Deblonde G. , 2007: A land data assimilation system for soil moisture and temperature: An information content study. J. Hydrometeor., 8 , 1225–1242.
Bechtold, P., Bazile E. , Guichard F. , Mascart P. , and Richard E. , 2001: A mass-flux convection scheme for regional and global models. Quart. J. Roy. Meteor. Soc., 127 , 869–886.
Bélair, S., and Mailhot J. , 2001: Impact of horizontal resolution on the numerical simulation of a midlatitude squall line: Implicit and explicit condensation. Mon. Wea. Rev., 129 , 2362–2376.
Bélair, S., Zhang D-L. , and Mailhot J. , 1994: Numerical prediction of the 10–11 June 1985 squall line with the Canadian regional finite-element model. Wea. Forecasting, 9 , 157–172.
Bélair, S., Mailhot J. , Strapp J. W. , and MacPherson J. I. , 1999: An examination of local versus nonlocal aspects of a TKE-based boundary layer scheme in clear convective conditions. J. Appl. Meteor., 38 , 1499–1518.
Bélair, S., Méthot A. , Mailhot J. , Bilodeau B. , Patoine A. , Pellerin G. , and Côté J. , 2000: Operational implementation of the Fritsch–Chappell convective scheme in the 24-km Canadian regional model. Wea. Forecasting, 15 , 257–274.
Bélair, S., Crevier L-P. , Mailhot J. , Bilodeau B. , and Delage Y. , 2003a: Operational implementation of the ISBA land surface scheme in the Canadian regional weather forecast model. Part I: Warm season results. J. Hydrometeor., 4 , 352–370.
Bélair, S., Brown R. , Mailhot J. , Bilodeau B. , and Crevier L-P. , 2003b: Operational implementation of the ISBA land surface scheme in the Canadian regional weather forecast model. Part II: Cold season results. J. Hydrometeor., 4 , 371–386.
Bélair, S., Mailhot J. , Girard C. , and Vaillancourt P. , 2005: Boundary layer and shallow cumulus clouds in a medium-range forecast of large-scale weather system. Mon. Wea. Rev., 133 , 1938–1960.
Benoit, R., Côté J. , and Mailhot J. , 1989: Inclusion of a TKE boundary layer parameterization in the Canadian regional finite-element model. Mon. Wea. Rev., 117 , 1726–1750.
Bougeault, P., 2008: Our changing forecast model. ECMWF Newsletter, No. 114, ECMWF, Reading, United Kingdom, 1.
Bougeault, P., and Lacarrère P. , 1989: Parameterization of orography-induced turbulence in a mesobeta-scale model. Mon. Wea. Rev., 117 , 1872–1890.
Buizza, R., Houtekamer P. L. , Toth Z. , Pellerin G. , Wei M. , and Zhu Y. , 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133 , 1076–1097.
Casati, B., Ross G. , and Stephenson D. B. , 2004: A new intensity-scale approach for the verification of spatial precipitation forecasts. Meteor. Appl., 11 , 141–154.
Côté, J., Desmarais J-G. , Gravel S. , Méthot A. , Patoine A. , Roch M. , and Staniforth A. , 1998a: The operational CMC–MRB Global Environmental Multiscale (GEM) model. Part II: Results. Mon. Wea. Rev., 126 , 1397–1418.
Côté, J., Gravel S. , Méthot A. , Patoine A. , Roch M. , and Staniforth A. , 1998b: The operational CMC–MRB Global Environmental Multiscale (GEM) model. Part I: Design considerations and formulation. Mon. Wea. Rev., 126 , 1373–1395.
Daly, C., Gibson W. P. , Taylor G. H. , Doggett M. K. , and Smith J. I. , 2007: Observer bias in daily precipitation measurements at United States cooperative network stations. Bull. Amer. Meteor. Soc., 88 , 899–912.
Donaldson, R. J., Dyer R. M. , and Kraus R. M. , 1975: An objective evaluator of techniques for predicting severe weather events. Preprints, Ninth Conf. on Severe Local Storms, Norman, OK, Amer. Meteor. Soc., 321–326.
Doswell, C. A., Davies-Jones R. , and Keller D. L. , 1990: On summary measures of skill in rare event forecasting based on contingency tables. Wea. Forecasting, 5 , 576–585.
Drusch, M., and Viterbo P. , 2007: Assimilation of screen-level variables in ECMWF’s Integrated Forecast System: A study on the impact on the forecast quality and analyzed soil moisture. Mon. Wea. Rev., 135 , 300–314.
Ebert, E. E., and McBride J. L. , 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239 , 179–202.
Ebert, E. E., Damrath U. , Wergen W. , and Baldwin M. E. , 2003: The WGNE assessment of short-term quantitative precipitation forecasts. Bull. Amer. Meteor. Soc., 84 , 481–492.
Errico, R. M., Bauer P. , and Mahfouf J-M. , 2007: Issues regarding the assimilation of cloud and precipitation data. J. Atmos. Sci., 64 , 3785–3798.
Fortin, V., Therrien C. , and Anctil F. , 2008: Correcting wind-induced bias in solid precipitation measurements in case of limited and uncertain data. Hydrol. Processes, 22 , 3393–3402. doi:10.1002/hyp.6959.
Fritsch, J. M., and Chappell C. F. , 1980: Numerical prediction of convectively driven mesoscale pressure systems. Part I: Convective parameterization. J. Atmos. Sci., 37 , 1722–1733.
Gallus W. A. Jr., , and Segal M. , 2001: Impact of improved initialization of mesoscale features on convective system rainfall in 10-km Eta simulations. Wea. Forecasting, 16 , 680–696.
Gauthier, P., Tanguay M. , Laroche S. , Pellerin S. , and Morneau J. , 2007: Extension of 3DVAR to 4DVAR: Implementation of 4DVAR at the Meteorological Service of Canada. Mon. Wea. Rev., 135 , 2339–2354.
Geleyn, J-F., 1987: Use of a modified Richardson number for parameterizing the effect of shallow convection. Short- and Medium-range Numerical Weather Prediction: Collection of Papers Presented at the WMO/IUGG NWP Symposium, Tokyo, Japan, WMO, 141–149.
Goodison, B. E., Louie P. Y. T. , and Yang D. , 1998: WMO solid precipitation measurement intercomparison final report. WMO/TD 872, World Meteorological Organization, Geneva, Switzerland, 212 pp.
Hamill, T. M., 1999: Hypothesis tests for evaluating numerical precipitation forecasts. Wea. Forecasting, 14 , 155–167.
Hines, C. O., 1997a: Doppler-spread parameterization of gravity-wave momentum deposition in the middle atmosphere. Part I: Basic formulation. J. Atmos. Sol. Terr. Phys., 59 , 371–386.
Hines, C. O., 1997b: Doppler-spread parameterization of gravity-wave momentum deposition in the middle atmosphere. Part II: Broad and quasi monochromatic spectra, and implementation. J. Atmos. Sol. Terr. Phys., 59 , 387–400.
Houtekamer, P. L., and Mitchell H. L. , 2005: Ensemble Kalman filtering. Quart. J. Roy. Meteor. Soc., 131 , 3269–3289.
Jung, T., Gulev S. K. , Rudeva I. , and Soloviov V. , 2006: Sensitivity of extratropical cyclone characteristics to horizontal resolution in the ECMWF model. Quart. J. Roy. Meteor. Soc., 132 , 1839–1857.
Kain, J., and Fritsch J. M. , 1990: A one-dimensional entraining/detraining plume model and its application in convective parameterization. J. Atmos. Sci., 47 , 2784–2802.
Kain, J., and Fritsch J. M. , 1993: Convective parameterization for mesoscale models: The Kain–Fritsch scheme. The Representation of Cumulus Convection in Numerical Models, Meteror. Monogr., No. 46, Amer. Meteor. Soc., 165–170.
Katz, R. W., and Murphy A. H. , Eds. 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.
Kuo, H. L., 1974: Further studies of the parameterization of the influence of cumulus convection on large-scale flow. J. Atmos. Sci., 31 , 1232–1240.
Laroche, S., Gauthier P. , St-James J. , and Morneau J. , 1999: Implementation of a 3D variational data assimilation system at the Canadian Meteorological Centre. Part II: The regional analysis. Atmos.–Ocean, 37 , 281–307.
Laroche, S., Gauthier P. , Tanguay M. , Pellerin S. , and Morneau J. , 2007: Impact of the different components of 4DVAR on the Global Forecast System of the Meteorological Service of Canada. Mon. Wea. Rev., 135 , 2355–2364.
Li, J., and Barker H. W. , 2005: A radiation algorithm with correlated-k distribution. Part I: Local thermal equilibrium. J. Atmos. Sci., 62 , 286–309.
Ma, L., 2000: On the parameterization of slantwise convection in general circulation models. Ph.D. thesis, McGill University, Montreal, QC, Canada, 175 pp.
Mailhot, J., and Bélair S. , 2002: An examination of a unified cloudiness–turbulence scheme with various types of cloudy boundary layers. Preprints, 15th Symp. on Boundary Layer and Turbulence, Wageningen, Netherlands, Amer. Meteor. Soc., 215–218.
Mailhot, J., Chouinard C. , Benoit R. , Roch M. , Verner G. , Côté J. , and Pudykiewicz J. , 1989: Numerical forecasting of winter coastal storms during CASP: Evaluation of the regional finite-element model. Atmos.–Ocean, 27 , 24–58.
Mailhot, J., and Coauthors, 1998: Scientific description of RPN physics library: Version 3.6. Recherche en Prévision Numérique, 188 pp. [Available from RPN, 2121 Trans-Canada, Dorval, PQ H9P 1J3, Canada].
Mailhot, J., and Coauthors, 2006: The 15-km version of the Canadian regional forecast system. Atmos.–Ocean, 44 , 133–149.
Mason, I., 1989: Dependence of the critical success index on sample climate and threshold probability. Aust. Meteor. Mag., 37 , 75–81.
Mesinger, F., and Brill K. , 2004: Bias normalized precipitation scores. Preprints, 17th Conf. on Probability and Statistics in the Atmospheric Sciences/20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., J12.6. [Available online at http://ams.confex.com/ams/pdfpapers/69561.pdf].
Molinari, J., and Dudek M. , 1992: Parameterization of convective precipitation in mesoscale numerical models: A critical review. Mon. Wea. Rev., 120 , 326–344.
Morcrette, C. J., and Browning K. A. , 2006: Formation and release of symmetric instability following delta-M adjustment. Quart. J. Roy. Meteor. Soc., 132 , 1073–1089.
Morcrette, J-J., Barker H. W. , Iacono M. J. , Mozdzynski G. , Pincus R. , Salmond D. , and Serrar S. , 2007: A new radiation package: McRad. ECMWF Newsletter, No. 112, ECMWF, Reading, United Kingdom, 22–32.
Murphy, A. H., and Ehrendorfer M. , 1987: On the relationship between the accuracy and value of forecasts in the cost–loss ratio situation. Wea. Forecasting, 2 , 243–251.
NOAA, 1996: Standard hydrometeorological exchange format. Weather Service Hydrology Handbook 1, National Oceanic and Atmospheric Administration, 61 pp. [Available from Office of Hydrology, National Weather Service, Silver Spring, MD 20910].
Noilhan, J., and Planton S. , 1989: A simple parameterization of land surface processes for meteorological models. Mon. Wea. Rev., 117 , 536–549.
Pudykiewicz, J., Benoit R. , and Mailhot J. , 1992: Inclusion and verification of a predictive cloud water scheme in a regional weather prediction model. Mon. Wea. Rev., 120 , 612–626.
Richardson, D. S., 2000: Skill and relative economic value of the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc., 126 , 649–667.
Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5 , 570–575.
Stephenson, D. B., 2000: Use of the “odds ratio” for diagnosing forecast skill. Wea. Forecasting, 15 , 221–232.
Sundqvist, H., 1978: A parameterization scheme for non-convective condensation including prediction of cloud water content. Quart. J. Roy. Meteor. Soc., 104 , 677–690.
Tremblay, A., 2005: The stratiform and convective components of surface precipitation. J. Atmos. Sci., 62 , 1513–1528.
Tustison, B., Harris D. , and Foufoula-Georgiou E. , 2001: Scale issues in verification of precipitation forecasts. J. Geophys. Res., 106 , (D11). 11775–11784.
Verseghy, D. L., 1991: CLASS—A Canadian Land Surface Scheme for GCMs. I. Soil model. Int. J. Climatol., 11 , 111–133.
Verseghy, D. L., McFarlane N. A. , and Lazare M. , 1993: CLASS—A Canadian Land Surface Scheme for GCMs. II. Vegetation model and coupled runs. Int. J. Climatol., 13 , 347–370.
Wilks, D. S., and Hamill T. M. , 1995: Potential economic value of ensemble forecasts. Mon. Wea. Rev., 123 , 3565–3575.
Zhang, D-L., Kain J. S. , Fritsch M. , and Gao K. , 1994: Comments on “Parameterization of convective precipitation in mesoscale numerical models: A critical review.”. Mon. Wea. Rev., 122 , 2222–2231.
Zhu, Y., Toth Z. , Wobus R. , Richardson D. , and Mylne K. , 2002: The economic value of ensemble-based weather forecasts. Bull. Amer. Meteor. Soc., 83 , 73–83.
Objective evaluation of 96-h forecasts against radiosondes for the GLBOLD (dotted lines) and GLBNEW (dash lines) global forecast systems for (a) geopotential height (dam), (b) temperature (K), (c) wind speed (m s−1), and (d) dewpoint depression (K). RMSE (black lines) and bias (gray lines) are shown. Verification is done over North America, for a set of 122 integrations initialized from 4DVAR analyses between 31 Aug and 31 Oct 2006.
Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1
SYNOP surface observation network over North America. Red exes show the locations of stations where manual measurements are taken. Blue exes show the locations of automatic stations.
Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1
Histograms showing the number of observed and predicted events with daily precipitation accumulations between 0 and 0.2 mm day−1, 0.2 and 5 mm day−1, etc. The histogram for the observations is shown in black, whereas histograms for the forecast systems are shown in yellow (GLBOLD), red (GLBNEW), and blue (REG).
Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1
Total precipitation (m) for each daily accumulations category (i.e., between 0 and 0.2 mm day−1, 0.2 and 5 mm day−1, etc). Totals for the observations are given by the thick full line, whereas totals for the GLBOLD, GLBNEW, and REG forecast systems are indicated by the dotted, dashed, and dashed–dotted lines. The total precipitation (including all categories) is shown on the right.
Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1
Threshold biases for the GLBOLD (dotted line), GLBNEW (dashed line), and REG (dashed–dotted line) forecast systems vs precipitation intensity.
Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1
ETS as a function of categories of precipitation intensity (mm day−1) for the GLBOLD (yellow), GLBNEW (red), and REG (blue) configurations. (top) The 0–24-, (middle) 48–72-, and (bottom) 96–120-h accumulations.
Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1
ETS as a function of thresholds for precipitation intensity (mm day−1) for the GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines) forecast systems. (top) The 0–24-, (middle) 48–72-, and (bottom) 96–120-h predictions.
Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1
ETS as a function of lead time for the GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines) forecast systems. (top) The 0.2, (middle) 10, and (bottom) 30 mm day−1 threshold results. Values at a certain lead time represents the ETS for the preceding 24-h period (e.g., values at 24 h are for the day 1 ETS).
Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1
Relative economic values of the predicted daily precipitation as a function of the cost–loss ratio (C/L) for three lead times (days 1, 3, and 5) and three thresholds of precipitation intensity (0.2, 10, and 30 mm day−1) for GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines).
Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1
Area between zero and the positive portion of the relative economic value curves given in Fig. 9, as a function of lead time. Results for GLBNEW (dashed lines), GLBOLD (dotted lines), and REG (dashed–dotted lines) are given for thresholds of precipitation intensity of 0.2, 10, and 30 mm day−1.
Citation: Weather and Forecasting 24, 3; 10.1175/2008WAF2222175.1
Configurations of GLBNEW, GLBOLD, and REG forecast systems.
Structure of summary contingency tables.
Structure of 2 × 2 contingency tables.
