1. Introduction

The three-dimensional variational data assimilation (3DVAR) system was introduced at the Meteorological Service of Canada (MSC) in 1997 in preparation for the direct assimilation of satellite radiances (Gauthier et al. 1999a). After several improvements (Chouinard et al. 2001), this system has recently been extended to the four-dimensional variational data assimilation (4DVAR) by including the model forecast integration as part of the observation operator. All the benefits of this approach as well as the main characteristics of this new data assimilation system are presented in Gauthier et al. (2007). Intercomparison of the operational 3DVAR and 4DVAR over 2 months of data assimilation cycles in the winter of 2003–04 and the summer of 2004 has revealed a consistent improvement in the extratropics with 4DVAR for both periods. Based on these positive results, 4DVAR system was implemented in the MSC global (medium range) forecast system on 15 March 2005.

The implementation of 4DVAR relaxes the stationarity assumption implicit in 3DVAR. Several inherent sources of error related to time inconsistency between the background field and observations over the assimilation window are thus eliminated. First, the observations are compared to the first guess (background field) at the appropriate time (FGAT). Although the FGAT can be implemented in a three-dimensional scheme, this procedure was not part of our operational 3DVAR. Second, the use of the tangent-linear model (TLM) and its adjoint permit the proper propagation of information over the assimilation window, which can be interpreted as implicit flow-dependent propagation of the background error covariances. The evolution of the error structure in the assimilation window can be easily seen when only one observation is assimilated in the 4DVAR context (Thépaut et al. 1993, 1996; Rabier et al. 2000). Fisher and Andersson (2001) also showed that the propagation of the gradient of the background departure to the beginning of the assimilation window is a key element of 4DVAR. There are other benefits to implement a 4DVAR scheme. Thépaut et al. (1996) showed the close relationship between the evolving analysis increment and the fastest-growing perturbations in the assimilation window, which are the most important structures to capture in order to control short-range forecast errors. 4DVAR is also able to extract tendency information from the time series of observations, which is difficult to achieve in any static data assimilation scheme (Järvinen et al. 1999).

The implementation of a set of simplified physical parameterizations in the TLM and its adjoint is also an important part of 4DVAR. Buizza (1994) showed that a linear vertical diffusion to represent the planetary boundary layer should minimally be included in the TLM to eliminate spurious fast-growing structures that may appear near the surface. In addition to the vertical diffusion, simplified parameterizations of subgrid-scale orographic drag, stratiform, and deep convection precipitation have been developed for 4DVAR (Gauthier et al. 2007) and singular vector studies (Zadra et al. 2004). Mahfouf and Rabier (2000) showed that the implementation of such a comprehensive set of simplified parameterizations in the last inner loop of 4DVAR is overall beneficial, especially in the Tropics.

In this paper, the performance over a 1-month period of several data assimilation configurations that range in complexity from our former 3DVAR to the implemented 4DVAR are compared. These experiments aim at better understanding and quantifying the contribution of the new components that were necessary to extend our 3DVAR to 4DVAR. This article also presents the impact of 4DVAR on the tropical cyclone tracks as well as the role of the full-resolution model in their propagation. It was indeed found in the trials that the forecast of the extratropical transition of these storms from both 3DVAR and 4DVAR is often premature. Finally, the impact of using a shorter cutoff time for the availability of observations for launching operational forecasts at 0000 and 1200 UTC is discussed.

2. Variational data assimilation formulation

The complete description of the 3DVAR and 4DVAR schemes developed at the MSC for the global forecast system can be found in Gauthier et al. (1999a) and Gauthier et al. (2007). The main equations and key features of the variational formulation, which is based on the incremental approach (Courtier et al. 1994), are summarized in this section. The analysis increment is calculated at a lower horizontal resolution (1.5°) than the forecast model, which is 0.9° for the current global forecast system (Côté et al. 1998). The analysis increment is obtained by minimizing the following cost function:

View Expanded

where y′^(k)_i = y_i − 𝗛_i[x^(k)_i] is the innovation vector; k is the outer loop index; y_i is the observation vector in the time interval i; 𝗕 and 𝗥 are the background and observation error covariance matrices respectively; 𝗛_i is the nonlinear observation operator that maps the full-resolution trajectory x^(k)_i = L(t_o, t_i)x^(k)₀ into the observation space; L represents the full-resolution model integration; and the caret indicates the low-resolution state. The high-resolution analysis update is obtained from

View Expanded

where Π_L is the interpolation operator from high to low resolution, Π^−I_L is its pseudoinverse, δx̂^(k)_i = 𝗟(t_o, t_i)δx̂^(k)₀ represents the low-resolution analysis increment propagated in time with the TLM 𝗟 linearized around the nonlinear trajectory x̂^(k)_i at low-resolution during the iterative minimization. This process is referred to as the inner loop.

The minimization is performed with the quasi-Newton algorithm developed by Gilbert and Lemaréchal (1989). The initial state of the low-resolution trajectory is also updated after each outer loop as follows:

View Expanded

where x̂⁽⁰⁾₀ = x̂^b₀ is the background field at low resolution.

In the former operational 3DVAR, the background field is a 6-h forecast from the previous analysis, as shown in Fig. 1. A single outer loop is performed (k = 0) with only one time interval (n = 0) covering the whole assimilation window. The analysis increment is not evolved in time and is estimated at the center of the assimilation window, corresponding to the synoptic time (t₀ = 0 h in Fig. 1). The minimization is stopped when the gradient of the cost function is reduced by two orders of magnitude, which is usually achieved within 90 inner loops. A 6-h assimilation window, centered at the synoptic time is used for the satellite radiances. For the other data types, the assimilation window is restricted to 3 h, because the time inconsistency between the background and observation beyond 90 min from the synoptic time may introduce large errors in the innovation vector. This is especially true for wind data. After making the quality control of observations, satellite and aircraft data are spatially thinned to avoid observation error correlation. The data are also temporally screened to retain only the closest observation to the synoptic time. This process is referred to as 3D thinning.

In the 4DVAR scheme, the background field now corresponds to a trajectory covering the whole assimilation window (−3 h < t₀ < +3 h in Fig. 1) and is obtained from a 9-h forecast initiated with the previous analysis (valid at t₀ = −6 h in Fig. 1). The TLM of the global forecast model at low resolution and its adjoint (Tanguay and Polavarapu 1999) are employed in the inner loop to propagate the analysis increment and the gradient of the cost function over the assimilation window. The analysis is obtained after two outer loops (k = 0, 1). In the first outer loop, 40 inner loops are performed with only the vertical diffusion as simplified linearized physics in 𝗟 (Laroche et al. 2002). After updating the full- and low-resolution trajectories, 30 inner loops are executed with a set of simplified physical parameterizations that includes vertical diffusion, subgrid-scale orographic effects, large-scale precipitation, and deep moist convection (Zadra et al. 2004; Mahfouf 2005). The assimilation window is now 6 h for most observations. It is divided into seven time intervals of 45 min, and two intervals of 22.5 min at both ends of the assimilation window (n = 8). The thinning procedure of observations used in the former 3DVAR has been extended to retain the closest observation to the middle of the assimilation time intervals (i.e., 4D thinning), as proposed by Rabier et al. (2000). This has considerably increased the number of assimilated data from aircrafts, geostationary satellites, and wind profilers. The number of satellite radiances has also increased, especially over the high latitudes where several satellite orbits overlap. Overall, the number of observations has increased by 60% with the introduction of the 4D thinning procedure. The so-called upper-air analysis, in its 4DVAR implementation, is actually the result of an additional 3-h integration of the full-resolution nonlinear model, as shown in Fig. 1.

Finally, the surface analysis is the same in both 3DVAR and 4DVAR schemes, as are the background error statistics (Gauthier et al. 1999b) and the data quality control procedures (Gauthier et al. 2003).

In summary, the new aspects of 4DVAR recently implemented are the following:

• calculation of the innovation vector at the appropriate time;

• use of the TLM to propagate the analysis increment over the assimilation window and the adjoint model to propagate the observation departure back to the start of the assimilation window;

• two outer loops with updates of the innovation vector from the full-resolution trajectory and updates of the low-resolution reference state vector for the linear models;

• a comprehensive set of simplified physical parameterizations; and

• an increased number of observations provided by the 4D thinning procedure.

3. Impact of the new 4DVAR components

The contribution of each 4DVAR component to forecast improvements is assessed in this section. This is done by performing data assimilation cycles over a 1-month period (August 2004) with various data assimilation configurations ranging in complexity from our former 3DVAR to the new 4DVAR. Table 1 summarizes the six different configurations considered in this study: the 3DVAR and 4DVAR as described in the previous section, a variant of 3DVAR in which the FGAT is implemented, and three variants of 4DVAR in which one of the following components is withdrawn: the 4D thinning (3D thin), the second outer/inner loop (1 loop), and the set of physical parameterization in the second inner loop except the vertical diffusion (simpler).

Figure 2 shows the RMS forecast errors against analyses for the 500-hPa geopotential height over the Northern and Southern Hemispheres (extratropics) are displayed for the six data assimilation configurations. Overall, the forecast errors with 4DVAR are smaller than with 3DVAR in both hemispheres, and the forecast performance for the other data assimilation configurations lie between those for 3DVAR and 4DVAR. Although the rankings are similar in both verification areas, the differences are more visible in the Southern Hemisphere. This is explained by the fact that the winter season prevails in this area in August, thus, the weather is more dynamically active. Also, satellite data, which are distributed over the whole assimilation window, are largely dominant in this part of the world. We thus considered the scores for the Southern Hemisphere where consistent reduction of errors, when including the new features of 4DVAR in the data assimilation system, can be better seen. Figure 3 displays the rankings based on the average RMS forecast errors (days 1–5) over the Southern Hemisphere, for each configuration (Table 1). The relative improvements are expressed in percentages, 100% representing the improvement of the new 4DVAR with respect to the former 3DVAR. The impact of each element of 4DVAR is estimated from the difference of average RMS forecast errors between the various configurations.

The most significant improvement (50%) comes from the use of the TLM and its adjoint model (ADM; TLM/ADM in Fig. 3). A total of 36% of the improvement is explained by the introduction of the 4D thinning procedure, while the implementation of the FGAT contributes 14%. The use of a more complete simplified physics improves the results by 7%. The update of model trajectories between the first and second inner loops leads to very little improvement (3%). This may be explained by the small difference in horizontal resolution between the simplified model used in the inner loops (1.5°) and the current high-resolution model (0.9°) in the global forecast system. Note that the benefit of using the TLM and its adjoint as estimated here also includes the slight contribution coming from the updates of the full-resolution trajectory.

Finally, it is important to note that the contributions of each feature are not necessarily independent in the sense that the improvement obtained by combining two components may be greater or smaller than the sum of their individual impact.

4. Impact on tropical cyclone tracks

Results from the 2-month assimilation periods in the winter of 2003–04 and the summer of 2004 clearly demonstrate the consistent improvement in the extratropics with 4DVAR, as shown in Gauthier et al. (2007). However, we found that the medium-range forecast of tropical cyclones is generally not improved, in particular, those over the western North Pacific during the summer and fall seasons. The best tracks determined by the Joint Typhoon Warning Center of the individual storms that developed in August 2004 are shown in Fig. 4. The number of typhoons during that period was above normal, which makes the comparison of the various configurations of 3DVAR and 4DVAR even more relevant. Figure 5 shows the 500-hPa geopotential height scores over the western North Pacific for 3DVAR, 4DVAR (simpler), and 4DVAR (the results for the other configurations listed in Table 1 are omitted for clarity). The rankings of the different data assimilation configurations is basically the same as the one obtained for the Southern Hemisphere for the first 3 forecast days. The forecast error difference at day 2 between 4DVAR and the configuration without the set of simplified physical parameterizations in the second minimization (i.e., 4DVAR simpler) is remarkable. This suggests that the simplified physical parameterizations help improve the short-term forecast of tropical storms. However, the performances of the 4DVAR configurations become worse than the one for 3DVAR beyond day 3 (although the differences between mean scores may become less significant since they are only computed over a 1-month period). To better understand the reasons for this, we examined the performance of Super Typhoon Chaba for which its track from 18 to 31 August 2004 is displayed by the dashed line in Fig. 4. Special attention was given to forecasts from the various 3DVAR and 4DVAR analyses at 1200 UTC 24 August 2004. Figure 6 shows the 5-day forecast tracks from 3DVAR and 4DVAR, and the best estimate from the Joint Typhoon Warning Center. It is noteworthy that the accuracy of the analyzed and forecast typhoon positions is limited to the horizontal resolution of the forecast model, which is around 100 km. Although the central positions of Chaba in the 3DVAR and 4DVAR analyses are too far north and northeast with respect to the best estimate, the direction and propagation speed of the super typhoon is well forecast during the first 2 days by both configurations. Then, the forecast trajectories turn to the northeast as the predicted typhoon interacts with the midlatitude flow (this is also the case for all the configurations, not shown here for clarity). In fact, the midlatitude transition of Super Typhoon Chaba actually occurred, but 5 days later over Japan.

The premature extratropical transition of typhoons with the MSC global forecast system has been known for years (this is also true for hurricanes in the Atlantic Ocean). We found that this problem mainly stems from the relatively coarse horizontal resolution (0.9°) and long time step (45 min) of the global forecast model, which is unable to properly simulate tropical storms. However, it seems that 4DVAR further amplifies the storm speed, already systematically too fast with 3DVAR, which explains the degradation shown in Fig. 5 after day 3 (this is also true for all the other 4DVAR variants). This indicates that weather elements in the full-resolution model should be well represented to get the full benefit from 4DVAR. To support this argument, we replaced the operational global model by its upgraded version that was in preimplementation evaluation at the time of the writing of this article. The main model improvements are the increased spatial resolution (0.33° latitude, 0.45° longitude, and 58 levels) and a complete revision of the physical parameterizations (Bélair et al. 2005). Figure 7 shows the typhoon forecast tracks for Chaba obtained with the upgraded model from the 3DVAR (thin solid line) and 4DVAR (thick solid line) analyses. We can see that the medium-range forecast tracks are substantially improved. We also performed a full data assimilation cycle in which the upgraded model provides the background field. The track for Chaba obtained with this cycle (dotted line) is similar to the others as shown in Fig. 7, although better at the beginning but not as good after day 3. The spread in the tracks seen in Fig. 7 is simply attributed to the sensitivity to initial conditions. The new version of the global model is now able to properly represent parts of the tropical storms that are important for their propagation. Although the forecasts of central pressure are improved with the new model (not shown), it is still underestimated, especially at the beginning of the model integration. More details about this upgraded model and the handling of tropical storms will be reported on soon in another study.

5. Impact of data availability

Based on the good performance of 4DVAR from the trials presented in Gauthier et al. (2007) as well as in this companion paper, a parallel suite with 4DVAR was initiated in December 2004 for the final evaluation before implementation in the MSC global forecast system. The first 4DVAR results from this parallel suite were not quite as good as those from the preimplementation experiments. After investigation, it was found that the performance of 4DVAR is more sensitive to the data availability than 3DVAR.

Operationally, the analysis for the medium-range forecast at 0000 and 1200 UTC is performed after a 3-h delay. This delay is referred to as the cutoff time for availability of observations (formally T + 300 for 3DVAR and T + 310 for 4DVAR, T being either 0000 or 1200 UTC). For the preimplementation experiments, the forecasts used in the evaluation were initiated from analyses with a full dataset using a 9-h cutoff time, corresponding to that of the final data assimilation cycling. The mean percentage of satellite data available 3 h after the synoptic time is between 50% and 60% of the full dataset (see Fig. 7 in Gauthier et al. 2007). Most of the missing data with the 3-h cutoff time are also in the second half period of the assimilation window. Another difference between the 4DVAR setup used in the trials and the one for the medium-range forecast at 0000 and 1200 UTC is the number of observations that is not the same in the first and second inner loops, as explained in Gauthier et al. (2007). The additional observations introduced in the second inner loop may impact the minimization and hence deteriorate the analysis. However, we verified that this is not the case and the number of iterations in the second inner loop (i.e., 30) is sufficient to accurately reach the minimum of the cost function.

Figure 8 shows the 500-hPa geopotential height scores for the Southern Hemisphere in January 2005 from 3DVAR and 4DVAR with 3- and 9-h cutoff times. The use of a shorter cutoff time is less detrimental for 3DVAR than 4DVAR. The reduction of the forecast performance with a reduced cutoff time is nearly 23% with 4DVAR whereas it is only about 5% with 3DVAR. These results suggest that 4DVAR is better than 3DVAR at extracting information from observations, especially satellite data, near the end of the assimilation window. On the other hand, the stationary assumption may lessen the benefit of assimilating extra satellite observations in 3DVAR. Moreover, with the implementation of the 4D thinning procedure, there are more observations in this part of the assimilation window in 4DVAR than in 3DVAR, which also contribute to the forecast improvement when a longer cutoff time is used in 4DVAR. Nevertheless, the performance of 4DVAR remains superior to 3DVAR with a 3-h cutoff time.

6. Conclusions

The impact on forecast skill of the additional or improved features that have been implemented into the MSC global forecast system to extend its data assimilation component from 3DVAR to 4DVAR has been assessed in this paper. Over the Southern Hemisphere, the implementation of the TLM of the forecast model and its adjoint to propagate the information in the assimilation window explains half of the 500-hPa geopotential height improvement. The 4D thinning contributes one-third of the forecast improvement. The FGAT procedure, updating the full-resolution trajectory after the first minimization and the full set of physical parameterization in the last inner loop are responsible for the remaining improvement of 4DVAR over 3DVAR.

For tropical storms, the short-term forecast is better with 4DVAR, especially when the set of simplified parameterizations (which includes moist processes) is used in the last inner loop. However, 4DVAR does not improve the medium-range forecast of these storms. The tendency of the model to interact prematurely with the midlatitude flow is a systematic problem already noticed with 3DVAR and, which has worsen with 4DVAR beyond day 3. The solution to this problem is to upgrade the current global model with the higher-resolution version with improved physical parameterizations. Preliminary comparisons between 3DVAR and 4DVAR cycles with this upgraded model are currently under way and the results for tropical cyclones will be reported in a future study.

A shorter delayed cutoff time for the availability of observations, as the one used in the operational context for the 0000 and 1200 UTC forecasts, is more detrimental to 4DVAR, which remains superior to 3DVAR with a longer cutoff time. This indicates that the observations at the end of the assimilation period play an important role in 4DVAR.