Système d’analyse fournissant des renseignements atmosphériques à la neige (SAFRAN) is a mesoscale atmospheric analysis system for surface variables. It produces an analysis at the hourly time step using ground data observations. One of SAFRAN’s main features is that it is based on climatically homogeneous zones and is able to take vertical variations into account. Originally intended for mountainous areas, it was later extended to cover France. This paper focuses on the validation of the extended version. The principle of the analysis is described and its quality was tested for five parameters (air temperature, humidity, wind speed, rainfall, and incoming radiation), using Météo-France’s observation network and data of some well-instrumented stations. Moreover, SAFRAN’s rainfall was compared with another analysis, known as analyse utilisant le relief pour l’hydrométéorologie (Aurelhy). Last, two different versions of SAFRAN were compared for mountain conditions. Temperature and relative humidity were well reproduced, presenting no bias. Wind speed was also well reproduced; however, its bias was −0.3 m s–1. The interpolation from the 6-h time step of the analysis to the 1-h time step was one of the sources of error. The precipitation analysis was robust and not biased; its root-mean-square error was 2.4 mm day−1. This error was mainly due to the spatial heterogeneity of the precipitation within the geographical zones of analysis (1000 km2). The analysis of incoming solar radiation presented some biases, especially in coastal areas. The results of the comparison with some well-instrumented sites were encouraging. SAFRAN is being run operationally at Météo-France on a real-time basis for various applications.
The knowledge of the atmospheric conditions during the last decades, days, or hours is useful for numerous activities such as climatology, hydrology, agronomy, or insurance.
Large-scale global analyses of the atmospheric conditions are computed by the main meteorological centers such as the European Centre for Medium-Range Weather Forecasts (ECMWF) or the National Centers for Environmental Prediction. Although the general quality of such analyses is very good and resolution is improving, some key aspects are not well treated, especially some screen-level parameters such as the precipitation.
Thus, to study the spatial and temporal evolution of the soil moisture, the Global Soil Wetness Project (GSWP) (http://www.iges.org/gswp/) has rebuilt a new analysis of the atmospheric conditions using other sources of information for precipitation. Nonetheless, the general quality of the precipitation is debatable (Decharme and Douville 2006).
Several projects aim at building a good-quality near-real-time analysis of the screen-level parameters by combining several kinds of observations: the Development of a European Land Data Assimilation System to Predict Floods and Droughts (ELDAS) project over Europe (http://www.knmi.nl/samenw/eldas/), the North American Land Data Assimilation Systems (NLDAS) project over the United States (Mitchell et al. 2004), and even the Global Land Data Assimilation Systems (GLDAS) project over the globe (Rodell et al. 2004). Those projects associate ground data observations with satellite data for incoming solar radiation or for estimations of daily heating rates.
In France, a mesoscale atmospheric analysis system devoted to hilly regions was built in the 1990s to provide the atmospheric forcing to a snow model for the forecast of avalanche hazards (Durand et al. 1993, 1999). This system was called système d’analyse fournissant des renseignements atmosphériques à la neige (SAFRAN), which means analysis system that provides data for the snow model.
Basic specifications of SAFRAN were to provide relevant parameters for the snow model “CROCUS” (Brun et al. 1989, 1992) every hour, for predefined vertical levels, for several mountain ranges, and for different slopes and aspects. The combination of SAFRAN, CROCUS, and “MEPRA” (an expert system for avalanche hazard forecasting) led to an operational application used to forecast avalanche hazards in France (Durand et al. 1999).
More recently, a special effort was made to develop a tool to model the main components of the continental water cycle at the regional scale in France. The soil–vegetation–atmosphere transfer (SVAT) model Interactions between Soil, Biosphere, and Atmosphere (ISBA; Noilhan and Planton 1989; Noilhan and Mahfouf 1996) was used in conjunction with the distributed hydrogeological model “MODCOU” (Ledoux et al. 1989) for the simulation of the water table and the streamflow of rivers. On this occasion, the SAFRAN analysis system was extended to cover France in order to feed the SVAT model with meteorological data.
This paper will focus on the validation of the SAFRAN analysis extended to France. The principle of the SAFRAN analysis and its adaptation to the context of this study will be described briefly. The validation of several analyzed parameters against the Météo-France observation network and a small number of well-instrumented sites will be presented for the hydrological years 2001/02 and 2004/05, which start on 1 August. Last, a short overview of the present applications of SAFRAN/France and perspectives on improvements will be given.
2. General description of the SAFRAN analysis
The main objective of SAFRAN is to produce an accurate estimation of the variables and downward fluxes needed by the SVAT model. SAFRAN is described in detail in Durand et al. (1993, 1999). In this section, only the main characteristics of the system are presented.
SAFRAN uses an optimal interpolation method to analyze most of the parameters (Gandin 1963). One of the main features of SAFRAN is that the analyses are performed over climatically homogeneous zones, which are areas of irregular shape covering a surface usually smaller than 1000 km2 and where the horizontal climatic gradients (specially for precipitation) are weak. SAFRAN estimates one value of each parameter for each zone at several altitude levels. Within the zone, analyzed parameters depend only on elevation and aspect. Zones are not isolated: observations from the neighboring zones are used if necessary.
First, SAFRAN does a quality control of the observations. This is an iterative procedure based on the comparison between observed and analyzed quantities at the observation location.
The analyses of temperature, humidity, wind speed, and cloudiness are performed every 6 h using all available observations (see subsection below). For this part, the first guess comes from the large-scale operational weather prediction model Arpege (Courtier et al. 1991) or from the ECMWF operational archives, as in this study.
The analysis is performed in two stages. In the first stage the vertical profiles of temperature, wind speed, humidity, and cloudiness are analyzed. These profiles are calculated with a vertical step of 300 m. In the second stage SAFRAN determines the surface parameters, using a cruder method. The first guess is, in this case, deduced and updated from the result of the previous analyses.
The precipitation analysis is performed daily at 0600 UTC, to include in the analysis the numerous rain gauges that measure precipitation on a daily basis (in particular in the climatological and snow networks). The first guess is in this case deduced from climatological fields (a constant altitudinal gradient or more elaborate fields, e.g., depending on weather type, if available).
Next, the analyzed values are interpolated to an hourly time step:
All altitude profiles (temperature, humidity, and cloudiness) and surface wind are linearly interpolated.
It is also at this stage that solar (direct and diffuse) and longwave radiation are calculated using a radiative transfer scheme (Ritter and Geleyn 1992), which uses the vertical profiles previously calculated.
The analysis of temperature at 2 m is slightly corrected at 1200 UTC using the daily observed maximum.
Then the temperature at 2 m is adjusted hourly using the solar radiation and a relaxation to an equilibrium temperature (Martin and Mainguy 1988).
To keep the moisture content of the air constant, some corrections to the surface humidity are also applied.
The determination of the hourly precipitation is solved in four steps: 1) determination of the highest hourly 0°C isotherm; 2) determination of the hourly snow–rain transition altitude, based on the previous step, observations, and vertical temperature gradient; 3) determination of the daily rain–snow ratio based on the observed or estimated rain–snow ratio at each observation site. This is done hourly using the results of the previous step; and 4) last, the hourly precipitation and phase are estimated, using the relative humidity and with the constraint of the daily rain–snow ratio estimated above.
a. Adaptation of the SAFRAN analysis to the whole of France
Two main versions of SAFRAN are used at the Centre National de Recherches Météorologiques (CNRM). The original version (Durand et al. 1993, 1999) will be referred to as SAFRAN/Avalanche (SAFRAN/A). The second one is the version validated in the present paper and will be referred as SAFRAN/France (SAFRAN/F). In the case of statements valid for both versions, the term SAFRAN will be employed.
In SAFRAN/F, the climatic zones as used to perform the analysis are the same ones used by Météo-France to produce fully automatic or expert meteorological forecasting. These zones were determined using the expertise of the local forecasters in each French administrative department. They were built to be as homogeneous as possible from a meteorological point of view. There are 612 zones in France; all of them have their lowest level below 1500 m and 93% of them below 300 m. On the other hand, 31% of the zones have a maximum height above 1000 m and 7% above 2100 m. Ninety-four percent of the zones have an elevation range higher than 300 m. Actually, a third of the zones correspond to mountainous regions, while the remaining are to be found in flat areas. For the hydrological application using ISBA and MODCOU, the SAFRAN/F analysis is interpolated on an 8 km × 8 km grid. The orography was derived from the “GTOPO30” global digital elevation model, available from the U.S. Geological Survey. Figure 1 shows the SAFRAN/F zones and the 8 km × 8 km orography.
Some options of SAFRAN were not used in SAFRAN/F. First, each aspect had the same weight. Second, a very simple and constant guess field was used in the precipitation analysis, instead of the guess being based on a climatological study and depending on the weather type used by SAFRAN/A.
Last, an improvement of the SAFRAN/A hourly interpolation analysis, which also used hourly observations, was proposed by Durand et al. (2000) in parallel to the present study and has not yet been transferred to SAFRAN/F. This is mainly due to 1) the less important role of aspects for the application over France, in which the results of the analysis were expected to be used over a larger and mostly flat area (France) and not only in mountains, and 2) the absence of a comprehensive climatological study of precipitation by weather types for the whole of France. The relative performance of both versions of SAFRAN at a site in the Alps will be discussed in a subsequent section of this paper.
To validate SAFRAN/F, several comparisons were carried out. First of all, SAFRAN/F was directly compared with the observations over France. For precipitation, SAFRAN/F’s yearly mean was also compared with the results of analyse utilisant le relief pour l’hydrométéorologie (Aurelhy; Bénichou and LeBreton 1987), which is another analysis (see description and results in section 5). Afterward, another validation was done through comparison with observations obtained by a small set of completely independent and well-instrumented stations. Moreover, SAFRAN/F was compared with SAFRAN/A for a site in the mountains.
Two periods of 1 yr were used for the validation. The first one starts on 1 August 2001 and the second starts on 1 August 2004. From now on, these two periods will be called 2001/02 and 2004/05. Both periods were chosen to start in August because of the hydrological applications of SAFRAN/F. These two specific years were selected because their precipitation cycles are contrasted, even though their yearly mean is similar (see Table 1).
a. Data used for the validation
The observations used by SAFRAN/F were extracted from the climatological database of Météo-France. Data from manually operated and automatic weather stations were used. For the hydrological year 2004/05, 3675 precipitation stations were available, most of them coming from the climatological network. For temperature, wind speed, and relative humidity, which are analyzed every 6 h, 1062, 465, and 819 stations were available, respectively. About the same number of stations were available for hydrological year 2001/02.
Data from the snow network (an ancillary network of voluntary observers, generally situated in ski resorts) were not used because they were not available in the climatological database at the time of this study.
b. Validation over France
In this section, the analysis is compared with observations over France. The validated parameters are temperature, wind speed, relative humidity, incoming solar radiation, and precipitation. Longwave radiation was not validated because too few observations were available.
Even though data were obtained from the same database that was used by the analysis, most of them could be considered as independent because the validation was done using hourly data and the analysis only used data 4 times a day (at 0000, 0600, 1200, and 1800 UTC) for temperature, wind speed, relative humidity, and cloudiness. Therefore, for these variables, almost all observations were not used by the analysis (20 out of 24). Furthermore, for incoming solar radiation all data were independent as they were not analyzed from the observations. On the contrary, for precipitation all data were dependent as the analysis was done once a day, which coincides with the frequency of the observations.
The discrimination between dependent and independent data is useful to understand the source of various bias. In this section, observations at the times of analysis will be called “dependent data” and the other ones “independent data.”
As stated in section 2 the analysis calculates one value of the considered parameter for several altitude layers of each zone. Afterward, the analysis is interpolated to a given grid depending on the altitude and zone of each cell.
To compare the observations with the analysis, the analysis must be interpolated to the altitude of the observation. To do this, the values of the zones, which were available at different vertical levels, were used. Then using a linear interpolation, from the two nearest levels, a value of the analysis at the altitude of the observation was obtained.
The results of the validation are presented in Table 1 for both years. In this section more attention is given to the most recent year (2004/05), and the results of 2001/02 are only presented in cases where there are notable differences with 2004/05.
Data from 1062 stations was used by SAFRAN/F to analyze the temperature for 2004/05. There were 612 climatically homogeneous zones, giving an average of 1.7 stations per zone. This number is low, but the calculation includes coastal and small zones that do not have any stations and some geographic areas that lack observations, as can be seen in Fig. 2. There are 460 zones with measurements; 188 of them have one station, 150 have more than 2 stations, and last, 48 zones have more than 4 stations.
The analyzed yearly mean temperature field is shown in Fig. 3. As expected, the yearly mean was correlated with latitude and altitude with maximal temperature on the Mediterranean coast. Inside each zone of analysis, the value of each cell only depended on its altitude. As a consequence, over zones with no relief, the resulting analysis was constant. In these areas, the pattern of the analyzed temperature reproduced the structure of the zones.
The monthly and yearly statistics of the analysis can be found on the first column of Table 1. The analysis was not biased and the root-mean-square error (RMSE) was 1.5°C. However, there was a seasonal cycle of the bias, which was positive during summer, negative during spring and autumn, and very small during winter.
The same monthly statistics were also calculated using only the dependent data. The results show that SAFRAN/F was not biased at the 4 h of analysis (i.e., 0000, 0600, 1200, and 1800 UTC). Therefore, the bias found in Table 1 was introduced by the hourly interpolation. The algorithm used to do the interpolation did not reproduce well the observed cycle during the whole year, introducing biases at some seasons.
The RMSE for the year calculated using only dependent data is 1.0°C and using only independent data becomes 1.5°C.
Figure 2 also shows the geographical distribution of the bias and the RMSE. The size of the spots is proportional to the bias and the RMSE. The RMSE map shows that the error is higher in the Mediterranean climate area of France, the maximal errors being over the Southern Alps. This pattern is not linked to orography (as the RMSE is lower in the Northern Alps, which are more subject to the westerly atmospheric circulation) and seems to be related to long diurnal amplitudes hardly seen by SAFRAN/F.
For temperature, the field produced by the ECMWF Reanalysis (ERA; Uppala et al. 2005) is shown in Figure 4. A visual comparison of SAFRAN/F’s and ERA’s fields shows the gain in resolution of SAFRAN/F. ERA’s relief is poor, as it does not show the effect of the Massif Central and of the Rhone valley. In terms of statistics, the comparison of ERA with the same observations used to evaluate SAFRAN/F gives a bias of 0.3°C and an RMSE of 2.2°C, which are higher than SAFRAN/F’s (0° and 1.5°C, respectively).
2) Wind speed
For wind speed, relatively few observations were available. Data from 465 stations were used. Therefore, there were 345 zones (out of 612) with observations (56%); 258 of them have only 1 station and only 25 zones have more than 2 stations.
The resulting analysis is in Fig. 5, which presents the annual mean wind speed over France. The stronger winds were on the northwestern half of the country, being stronger closer to the coast. The Mediterranean coast was also windy, and the patterns found coincide with the known local winds, Tramontane and mistral. As fewer observations than for temperature were available for the analysis, the structure of the field is less smooth, following the pattern of the zones. The lack of observations is more important over coastal areas, where stations are often located on nonrepresentative sites like cliffs.
As for temperature, the RMSE (Fig. 6) was higher over the Mediterranean part of France. But it also was remarkably strong over the Pyrenees, on the border with Spain, and over the Atlantic coast. The difference between the Alps and the Pyrenees might be due to a problem of representativeness of the stations situated in that region. Anabatic and katabatic winds could also play a role in the errors in mountainous regions. On the other hand, the bias is systematically negative over the south, especially the southeast, of France and its pattern is less clear in the northern half of the country. As for other analyzed variables, the RMSE is generally higher on the coast because of the lack of data and their anisotropic spatial distribution.
Table 1 shows that the wind speed bias and RMSE were stable throughout the year. The bias was systematically negative. In general, SAFRAN/F’s wind speed was 10% lower than the observed and the RMSE was high.
As for temperature, the bias in wind speed was introduced by the hourly interpolation. At the analysis times, the bias was only −0.05 m s−1, while it reached −0.24 m s−1 for the independent data. For wind speed, a linear interpolation was used. The presence of a bias means that the linear interpolation is too simple to take into account some phenomena, like the breeze, which affect the observed diurnal cycle.
3) Relative humidity
Data for relative humidity (RH) were available at 819 stations. There were 428 zones with observations; half of them (230) had only 1 station and 94 had more than 2 observation stations. Again, there was an important number of zones without any measurement and the majority of zones with observations had only one or two of them.
Figure 7 shows the resulting mean RH over France. There was a gradient from the northwest to the southeast, RH being stronger over the northwest. The Mediterranean southeast had a drier signature. In this case, the geographic distribution of bias and RMSE were not very significant.
Table 1 shows that globally the bias was zero and that the RMSE was 7% for both years. Monthly results show a yearly cycle. This cycle is present in both dependent and independent data. The fact that temperature is used to calculate RH might affect the results incorporating the error of this variable with the RH.
4) Incoming solar radiation
Incoming solar radiation (ISR) is not analyzed from the observations, as the other variables are, but calculated by SAFRAN’s radiation scheme.
For this parameter, the comparison was done using 239 stations.
Figure 8 shows the ISR mean for 2004/05. As expected ISR was stronger at high altitudes and lower latitudes, but the latitudinal gradient was not as smooth as expected. In this case the structure of the zones is easy to see, as there are neighboring zones with strong differences in the plains.
Table 1 shows that RMSE was important. For 2004/05 it was 93 W m−2, which corresponds to 60% of its mean value. The bias was not very high, but its sign differed between the two studied years. In both years the bias followed a similar pattern, it was positive during the winter (low ISR) and mostly negative during the rest of the year (high ISR).
The bias of ISR was more negative near the coast (Fig. 9), meaning that the transfer scheme did not take into account local phenomena in this area. Inland, the bias was more mixed, tending to be positive in the southwestern plains and the Vosges (northeast) and more negative in other areas. In 2001/02 the pattern was very similar, but the coastal negative bias was enhanced.
These patterns might also be due to the cloudiness analysis; if some kinds of clouds and their altitude are not well analyzed, the error is transferred to the radiation scheme. It can be the case in coastal areas where local small-scale clouds occur. This point should be examined in detail in the future by using cloudiness from observations, models, and satellites.
Precipitation is the most important parameter for a great number of applications of SAFRAN/F, but at the same time, it is a difficult parameter because of its discontinuity in space and time.
SAFRAN/F performs the precipitation analysis daily, instead of every 6 h, because this is the frequency in which data of the climatological network are collected and stored. As a consequence, in this study, all statistics were calculated on a daily time step. An important consequence is that all data used for the comparison had already been used in the analysis.
The network of rain gauges was the denser measuring network used in the present study. There were 3675 measurement stations for 2004/05, which covered 595 zones (97%). There were 512 zones with more than 2 stations, 298 had more than 5, and last, 136 of them had more than 8 observations.
The mean daily precipitation for 2004/05 is shown in Fig. 10. On this figure the same field calculated using the Aurelhy (Bénichou and LeBreton 1987) analysis is also plotted. In this method, the local topography is used to explain rainfall variables by multivariate linear correlation. The regression residuals are interpolated by kriging The experience showed that Aurelhy results are stable for a long time step (a month or a year). It is not able to reproduce a daily, or even hourly, precipitation as SAFRAN/F does.
For the sake of comparison, the resolution of the Aurelhy analysis was reduced from 1 to 8 km (SAFRAN/F’s resolution) by averaging. Both analyses are shown in Fig. 10. SAFRAN/F’s zones were plotted over both maps, even though Aurelhy does not use these zones to make the analysis. Comparing both annual means, it was seen that SAFRAN/F and Aurelhy did reproduce the same precipitation structures. Nevertheless, Aurelhy’s precipitation field was smoother than SAFRAN’s, as discontinuities occur at SAFRAN’s zone borders.
The SAFRAN/F’s analysis was not biased but the RMSE remained high (2.4 mm day−1; 109%). Neither RMSE or bias followed a seasonal cycle; RMSE was always proportional to precipitation and bias remained zero. A large part of the RMSE was probably linked to the intrazone precipitation variability, which could not be captured by SAFRAN, as the zones are supposed to be climatically homogeneous.
Figure 11 shows the geographical distribution of the bias and the RMSE. The bias was usually small, with some noticeable exceptions near the Cotentin Peninsula in the Normandie Region and in mountainous areas (Vosges, Massif Central, Pyrenees, and Alps). In these regions, stations with positive and negative biases were usually found very close to each other. This was a consequence of the high horizontal precipitation gradient (see also Fig. 10). SAFRAN/F was unable to account for the intrazone variability in these areas, as one of its basic assumptions was the homogeneity for each zone. A solution to improve the analysis in this case would be to modify the zone boundaries or to add some new zones, to make the zones more homogeneous, or to better tune SAFRAN’s statistical assumptions.
Apart from the mountainous areas already cited, the RMSE was high in the Rhone valley and the Mediterranean border of the Massif Central (Cévennes). Here, the high spatial variability of the convective precipitation explained these figures.
Over most of France, the bias and RMSE were small.
Figure 12 shows the observed and analyzed frequencies of occurrence of different classes of precipitation. The SAFRAN/F analysis was usually good. However, it underestimated the occurrence of no precipitation (45% instead of 52% in the observations) and overestimated the days of precipitation in the range between 0 and 5 mm day−1. These two errors compensated for each other and were linked to the analysis method.
The analysis also underestimates the occurrence of strong precipitation (P > 20 mm day−1).
Some standard statistical criteria, usually used to assess the quality of predictions, were also calculated for the precipitation analysis. These criteria were the critical success index (CSI), the probability of detection (POD), the false-alarm ratio (FAR), the true skill score (TSS), and the equitable threat score (ETS). Figure 13 shows the results for 2004/05. Eleven classes had been selected: no precipitation and precipitation rate higher than 1, 2, 4, 6, 8, 10, 20, 30, 40, and 50 mm day−1. These criteria are defined in the appendix.
All criteria behaved in a very similar way; they showed that the ability of SAFRAN/F to reproduce precipitation events decreased when the intensity of these events augmented. For 2004/05 the CSI decreased from 70% to 35%. On the other hand, the FAR remained very low and increased at a low pace, starting at 20% for P > 0 and increasing to almost 30% for P > 50 mm day−1. The increase of false alarms might be related to the fact that strong local events might increase the mean value of precipitation in the zone and, as a consequence, create some false alarms on neighboring stations within the zone. For 2001/02 the results were very similar.
c. SAFRAN/F compared with well-instrumented stations
SAFRAN/F was also validated by comparing the analysis with a set of well-instrumented stations that were not used in the analysis.
Five stations were used. Three of them belonged to the CarboEurope Integrated Project (http://www. carboeurope.org/) and were situated in the northeast, southwest, and south of France. The fourth one was the Smosrex station (De Rosnay et al. 2006) and was situated in the south, near Toulouse. The last one was a mountain station situated on the Col de Porte (CdP), in the Alps. Details about theses stations may be found in Table 2.
Analyzed and observed temperature, wind speed, relative humidity, incoming solar radiation, IR radiation, and precipitation were compared. The comparison was done using an hourly time step. In the previous section, precipitation was validated at the daily time step; in this section, a daily time step is also used to allow the comparison.
For 2004/05 there were only data available for the Col de Porte, so more emphasis will be given to 2001/02 in this section.
Tables 3 and 4 show the statistics of the time series over each station for 2001/02 and 2004/05. As before, the variables used to validate SAFRAN/F are the bias and the RMSE, but also the correlation.
Table 3 shows that the bias could be important in some stations (−0.9°C). The RMSE was consistent with the 1.5°C found before. Both observed and analyzed data were very well correlated.
2) Wind speed
The RMSEs of wind speed were also coherent with those found in the previous section. Biases could be very strong, ranging from −46% (−0.6 m s−1) at Col de Porte to 30% (−0.6 m s−1) at the Smosrex site. Correlations were very different at each station but none of them reached 0.9.
3) Relative humidity
Bias was always smaller than 6% for relative humidity, and the RMSE was higher than the mean over the whole of France. Correlations were very different from one station to the other, two of them being higher than 0.9.
4) Incoming solar radiation
Observations and the analysis were very well correlated for ISR. This high correlation might be due to the important diurnal cycle. To test the importance of this cycle on correlation, it was calculated again but using daily means. The results were very similar. In the same order as in Table 3, these correlations were 0.92, 0.90, 0.92, 0.88, and 0.87, respectively. The last one, Col de Porte, was the only one where correlation was better. Therefore, both series were well correlated.
In terms of RMSE and bias, Col de Porte had a different behavior in comparison with that of the other stations. The bias was positive, and the relative RMSE was more important.
The statistics were also calculated using only the values at noon. In relative terms at noon the RMSE was smaller and, for four of these stations, the bias was negative. Therefore SAFRAN/F underestimated the daily maximum. At noon the correlation was worse than the one found for the daily mean and for the full series.
5) Longwave radiation
Infrared radiation could not have been validated before because there were no measurements all over France. But on two of these well-instrumented stations there were IR sensors. The results on two stations are not significant, but they give an idea of the errors that SAFRAN/F might produce. On both stations the bias was negative, being more important at Col de Porte. The RMSE was moderate, relative to the ISR, and the correlation was not as good. Nevertheless, when calculating the correlations using daily means, as was done for ISR, the results ameliorated considerably, becoming 0.85 for Smosrex and 0.91 for Col de Porte. This might be linked to the analysis of cloudiness, which is an important parameter to calculate the IR with the transfer scheme. On average SAFRAN/F analyzed well the daily mean of this variable, but at the hourly time step the analysis was degraded.
Precipitation at a daily time step was also coherent with the global RMSE calculated over France, but the bias was not zero, as seen before, but positive on all these stations. This might be due to the fact that these were automatic gauges, which might underestimate precipitation in comparison with the nonautomatic gauges used by SAFRAN/F (Canellas 2005). Correlation was very good on a daily time step but it dropped at the hourly step. The method used to interpolate from the daily analysis to hourly data did not well reproduce observed data.
d. SAFRAN/France compared with another analysis: SAFRAN/Avalanche
Tables 3 and 4 show two values at Col de Porte. The first one corresponds to the SAFRAN/F (CdP) analysis, but the second one to SAFRAN/A (CdP/A). As expected, in general, SAFRAN/A and SAFRAN/F obtained similar results. Precipitation was an exception; SAFRAN/F had a bias of about 10% (16% for 2004/05) and SAFRAN/A had a bias of 1%. This difference is probably due to differences in the precipitation guess fields (constant altitudinal gradient for SAFRAN/F vs variable gradient based on a detailed climatological study for SAFRAN/A) and differences in the input data (some observations from the French snow network are not yet included in the central database). In terms of RMSE, SAFRAN/F was slightly better one year and slightly worse the other, so, in terms of RMSE, both analyses were equivalent.
SAFRAN/F was implemented on a real-time basis at the end of 2003. Every morning, the analysis is performed ranging from 0600 UTC of the previous day to 0600 UTC of the present day, building a continuous series of hourly fields.
For the precipitation analysis, SAFRAN/F uses 1000 observations, mainly from the automatic network and some man-operated stations. To take into account information from the climatological network (3000 daily observations of precipitation, available with a delay of one month) and late observations, another SAFRAN analysis is also run monthly. This second run (called climatological run) allows a significant improvement of the precipitation analysis. This run is very important for the hydrological monitoring applications, which need a good estimation of the precipitation amounts over long periods.
SAFRAN/F is mainly used in the framework of the SAFRAN–ISBA–MODCOU (SIM) suite. The SIM suite was used in several studies over some of the major basins in France: the Rhone basin (Habets et al. 1999; Etchevers 2000), the Garonne basin (Morel 2003), and the Seine basin (Rousset et al. 2004). The development of SAFRAN/F, in conjunction with the extension of the physiographic and hydrological database for ISBA and MODCOU, allowed the extension of SIM to the whole of France (Habets et al. 2007, manuscript submitted to J. Geophys. Res.).
The main application of SIM is the soil wetness real-time monitoring. The soil wetness index (SWI) is calculated and compared with a climatology of the past 10 yr. Maps are published on a regular basis (www. meteo.fr) as part of the French climatological synthesis bulletin in Fig. 14. These data are also used in the hydrological bulletin published by the French Ministère de l’écologie et du dévelopement durable. Snow cover simulated by SIM (relative to the 10-yr mean) is published in spring.
SAFRAN/F is also used to force ISBA-Routes (a specific version of ISBA that simulates the water and heat transfers in roads; Bouilloud and Martin 2006), which is used to monitor the road surface temperature and provide initial conditions for road surface temperature forecasts. In addition, SAFRAN/F analyzed fields are used in other punctual applications.
On a research mode, SIM is validated using observed discharge, piezometric head, soil temperature, and snow heights (Habets et al. 2007, manuscript submitted to J. Geophys. Res.). It is also used to provide an initial state for hydrological forecasts (e.g., Habets et al. 2004) and in climate impact studies (Etchevers et al. 2002). The ongoing extension of the SAFRAN/F analysis back to 1981 will give strength to the future studies of the recent hydrological changes in France.
5. Conclusions and perspectives
The detailed validation presented here and feedback from the operational implementation showed that SAFRAN/F was robust and provided accurate meteorological values to force the soil–vegetation–atmosphere transfer model ISBA and the hydrogeological model MODCOU as well as other applications.
Temperature, wind, and relative humidity were very well reproduced, considering the number of observations available: 1000, 500, and 800, respectively. Results were less accurate between the four analysis times (0000, 0600, 1200, and 1800 UTC) because of the simple time interpolation. They were also less accurate in some regions with a higher spatial or temporal variability, mainly the Mediterranean area of France.
The precipitation analysis was robust, as the comparison with the Aurelhy annual precipitation fields (calculated using a very different method) showed. However, the RMS error was relatively large because of the intrazone variability that could not be taken into account.
As radiation terms were calculated using a radiation scheme fed by analyzed altitude fields of temperature, humidity, and cloudiness, the ISR presented some biases, especially in the coastal areas. The seasonal cycle was also not well reproduced. The lack of measurement of downward longwave radiation did not allow a proper validation of this variable. Last, the comparison of SAFRAN/F results and observations at a few well-instrumented sites were very encouraging.
Feedback from users and the results of the present validation will help the improvement of the surface analysis in the future. Another related element is the continuous increase of the resolution of the numerical weather prediction models: the grid mesh of the future French model AROME (Ducrocq et al. 2005) will be on the order of 2–3 km, the quality of remote sensing products, and the number of automatic weather stations, most of them having at least an hourly data transmission.
The hypothesis of the climatologically homogeneous zone is less valid in plain areas than in the mountains. In mountainous areas, the climatic zones are well defined, because mountain barriers create a discontinuity in the meteorological and climatological fields. In the plains, natural barriers inducing discontinuities in the climate are less obvious. To account for these points, the solution will probably be to use analysis techniques that diminish the influence of the borders of the zones.
Apart from changing the analysis technique itself, the implementation of an hourly analysis will allow a better use of observation (as it is done in SAFRAN/A). Another way of improvement will be to spatialize the ISR measurements derived from the Meteosat data, instead of using a radiative scheme.
The use of radar data will help to improve the precipitation fields—both the temporal and spatial distribution. Another possible advantage of the use of radar data will be to improve the real-time precipitation analysis (based only on 1000 observations, instead of 3000 for the climatological version of SAFRAN) to provide accurate precipitation fields more rapidly.
Data from the fully instrumented sites came from three French eddy flux towers within the UE programs Carboeurope and Carboeurope-IP (Hesse, Bray, and Puchabon). We thank the teams that made these data available.
Definitions of Statistical Criteria
Five criteria were used to assess the quality of the analysis. For each threshold on the rainfall rate, a contingency table is built that shows the frequency of observed and analyzed events (Table A1). The critical success index (or threat score) is the number of correct event forecasts divided by the number of cases forecast and/or observed: CSI = D/(B + C + D). The probability of detection measures the success of the forecast in correctly predicting the occurrence of events: POD = D/(D + B). The false-alarm ratio measures the fraction of event forecasts that were actually nonevents: FAR = C/(D + C). The true skill score is the accuracy for events plus accuracy for nonevents minus 1: TSS = (AD − BC)/(A + C)(B + D). The equitable threat score is a modification to the CSI that takes into account the number of correct forecasts of events that would be expected purely as a result of chance: ETS = (D − chanceD)/(B + C + D − chanceD), with chanceD = (D + B)(D + C)/(A + B + C + D).
* Current affiliation: UMR-Sisyphe, Paris, France
Corresponding author address: P. Quintana-Seguí, Météo-France, CNRM/GMME/MC2, 42 ave. G. Coriolis, 31057 Toulouse CEDEX 1, France. Email: firstname.lastname@example.org