In situ soil moisture from 117 stations across the world and under different biome and climate conditions are used to evaluate two soil moisture products from the European Centre for Medium-Range Weather Forecasts (ECMWF)—namely, the operational analysis and the interim reanalysis [ECMWF Re-Analysis Interim (ERA-Interim)]. ECMWF’s operational Integrated Forecasting System (IFS) is based on a continuous effort to improve the analysis and modeling systems, resulting in frequent updates (a few times a year). The ERA-Interim reanalysis is produced by a fixed IFS version (for the main component of the atmospheric model and data assimilation). It has the advantage of being consistent over the whole period from 1979 onward and by design, reanalysis products are more suitable than their operational counterparts for use in climate studies. Although the two analyses show good skills in capturing surface soil moisture variability, they tend to overestimate soil moisture, particularly for dry land. Over the 2008–10 period, averaged statistical scores (correlation, bias, and root-mean-square difference) are 0.70, −0.081 m3 m−3, and 0.113 m3 m−3 for the operational product and 0.63, −0.079 m3 m−3, and 0.121 m3 m−3 for ERA-Interim. Compared to the scheme used in ERA-Interim, the current model used in the IFS has an improved match to soil moisture that is attributed to recent changes in the IFS. Indeed, major upgrades recently implemented in the operational land surface analysis and modeling system improve the surface and the root-zone soil moisture analyses.
Soil moisture is a crucial variable for numerical weather and climate prediction as it controls the partitioning of energy into latent and sensible heat fluxes at the soil–atmosphere interface. In addition it is a key variable in hydrological processes (i.e., runoff, evaporation from bare soil, and transpiration from the vegetation cover) and has an impact on plant growth and carbon fluxes (Dirmeyer et al. 1999; Entekhabi et al. 1999). Soil moisture is also important for monitoring land surface conditions that can enhance extreme events such as droughts, heat waves, and floods. As a consequence, many studies have been conducted to obtain estimates of soil moisture. These have shown that land surface modeling (e.g., Dirmeyer et al. 1999; Georgakakos and Carpenter 2006) and remote sensing techniques (Wagner et al. 1999, 2007; Kerr et al. 2001; Kerr 2007; Njoku et al. 2003) have great potential for providing reliable estimates of soil moisture.
In recent years, major upgrades have been implemented in the land surface modeling and analysis systems of the Integrated Forecasting System (IFS) used operationally at the European Centre for Medium-Range Weather Forecasts (ECMWF). The upgrades include an improved soil hydrology (Balsamo et al. 2009), a new snow scheme (Dutra et al. 2010), and a multiyear satellite-based vegetation climatology (Boussetta et al. 2011). Also a new soil moisture analysis scheme based on a pointwise extended Kalman filter (EKF) for the global land surface has been developed and implemented (Drusch et al. 2009; de Rosnay et al. 2011, 2012). As with the previous optimal interpolation (OI; Mahfouf 1991; Mahfouf et al. 2000) scheme, it uses proxy observations to analyze soil moisture (temperature and relative humidity at 2 m from the global network of conventional observations). Additionally, compared to the OI method, assimilation techniques based on the Kalman filter are more general and allow use of new types of observations such as satellite data. Recent technological advances in remote sensing enable the retrieval of soil moisture from satellite observations. Indeed, quantitative information about the soil water content of a shallow near-surface layer can be obtained from spaceborne microwave instruments (Schmugge 1983), particularly in the low-frequency microwave region from 1 to 10 GHz (and to a lesser extent up to 23.8 GHz; Calvet et al. 2011). Although satellite sensors provide a means of quantitatively describing the top few centimeters of soil, the variable of interest for applications in short- and medium-range meteorological modeling, as well as hydrological studies over vegetated areas, is the root-zone soil moisture content—this controls plant transpiration. Both near-surface and root-zone soil moisture are related through diffusion processes. When using assimilation algorithms (such as the EKF), model data and observations related to surface soil moisture (SSM) are optimally combined. The resulting information is propagated by the physics of the model downward through the root-zone, thereby allowing the retrieval of root-zone soil moisture (Entekhabi et al. 1994; Houser et al. 1998; Walker et al. 2001a,b; Sabater et al. 2007, 2008; Albergel et al. 2010a). It is clear that the EKF soil moisture analysis offers a range of developments options for the exploitation of satellite observations.
This study provides an evaluation of the ECMWF’s operational analysis of soil moisture based on in situ soil moisture observations from more than 100 stations across the world (Australia, Africa, America, and Europe) for the period from 2007 to early 2011. Along with the operational product, the new ECMWF Re-Analysis Interim product (ERA-Interim) (from 1979 onward; Dee et al. 2011) is evaluated. While the operational analysis is obtained from frequently updated versions of the IFS (including changes in spatial and vertical resolutions, data assimilation, parameterizations, and sources of data), ERA-Interim guarantees a higher level of consistency (e.g., in skill) because of its frozen configuration. After a description of the soil moisture datasets used in this study, ECMWF’s soil moisture products are initially evaluated against observations from the Soil Moisture Observing System–Meteorological Automatic Network Integrated Application (SMOSMANIA) network (Calvet et al. 2007; Albergel et al. 2008) and the Surface Monitoring of Soil Reservoir Experiment (SMOSREX) experimental site (de Rosnay et al. 2006) in southwestern France. In other studies in situ data from those sites have already been used to assess the quality of several SSM datasets including operational products from meteorological services as well as remotely sensed products such as the Advanced Scatterometer (ASCAT) SSM (Albergel et al. 2009, 2010b). ECMWF operational and reanalysis products are evaluated for a 4-yr period (2007–10) to show the evolution of the quality of the operational product. Note that the aim is to evaluate the ERA-Interim soil moisture product rather than trend detection as multidecadal period would be necessary to investigate the trend in soil moisture. The comparison between ECMWF’s analyses and soil moisture observations from the SMOSMANIA network and the SMOSREX experiment provide the framework of this study. The evaluation of ECMWF’s product is then extended to include in situ observations from networks across the world. The remainder of this paper is organized as follows: section 2 describes the soil moisture products used in this study (analyses and in situ) and results of the comparisons are analyzed in section 3 and discussed in section 4.
2. Material and methods
In situ soil moisture observations are highly relevant to evaluate soil moisture products derived from either modeling or remote sensing. In this study in situ data from more than 12 networks across four continents were gathered. Some of them were freely available on the Internet such as data from the Natural Resources Conservation Service–Soil Climate Analysis Network (NRCS–SCAN) in the United States (Schaefer and Paetzold 2000; http://www.wcc.nrcs.usda.gov/scan/) or the OzNet hydrological monitoring network in Australia (Young et al. 2008; http://www.oznet.org.au/). Use was also made of information from the International Soil Moisture Network (ISMN; Dorigo et al. 2011; http://www.ipf.tuwien.ac.at/), a new data hosting center where globally available ground-based soil moisture measurements are collected, harmonized, and made available to users. Other datasets were obtained by request from organizations such as Météo-France and the European Short-Range Numerical Weather Prediction (SRNWP) Programme. Data at 133 stations were collected and a first visual quality check was performed. When suspicious data were observed they were discarded, leading to the retention of 117 stations to evaluate ECMWF’s products in the first model layer of soil (0–7 cm) and 69 for the second model layer (7–28 cm). Indeed, most of the stations only measure SSM. When possible, soil moisture over the first meter of soil was also evaluated.
Soil moisture analyses from either the deterministic operational suite or ERA-Interim are available at four depths (0–7, 7–28, 28–100, and 100–289 cm; Balsamo et al. 2009). The ECMWF soil moisture analysis is evaluated from January 2007 to April 2011. In situ data are collected within this period, but their availability does not necessarily cover the whole period. The soil moisture datasets used in this study are presented in Table 1.
a. ECMWF’s IFS
Data produced at ECMWF include a large variety of surface parameters, describing atmosphere as well as ocean wave and land surface conditions. The analysis is produced using a 12-h assimilation scheme. Available observations are combined with prior information provided by a forecast model to estimate the evolving state of the global atmosphere and its underlying surface. It involves the computation of a variational analysis for the basic upper-layer atmospheric fields followed by separate analyses of near-surface parameters, soil moisture and temperature, and snow and ocean waves. Analyses are then used to initialize a short-range model forecast, which provides the prior state estimates needed for the next analysis cycle. While producing the forecast, the model estimates a wide variety of physical parameters including precipitation, turbulent fluxes, radiation fields, and soil moisture. Even if not directly observed, these are constrained by the observations used to initialize the forecast and their accuracy relies on the quality of the model physics as well as that of the analysis. A full description of ECMWF’s model physic and data assimilation is available at http://www.ecmwf.int/research/ifsdocs/.
1) Upper-air analysis
The core atmospheric assimilation system at ECMWF relies on the four-dimensional variational (4DVAR) data assimilation scheme (Rabier et al. 2000; Mahfouf and Rabier 2000) with an observation time window of 12 h (Bouttier 2001). Data provided by satellite sensors (both from microwave and infrared radiometers) as well as conventional observations (e.g., radiosonde network) are ingested within the 4DVAR. Use is also made of surface observations such as surface pressure, humidity, and wind.
2) Surface analysis
The model forecast for the land surface analysis is provided by the Tiled ECMWF Scheme of Surface Exchanges over Land (TESSEL) land surface scheme (van den Hurk et al. 2000), which was then upgraded with an improved soil hydrology (H-TESSEL) (van den Hurk and Viterbo 2003; Balsamo et al. 2009). H-TESSEL development was a response to weaknesses in the TESSEL hydrology: a Hortonian runoff scheme hardly producing surface runoff and the choice of a single global soil texture was not able to characterize different soil moisture regimes. So, for H-TESSEL the formulation of the soil hydrological conductivity and diffusivity was revised to be spatially variable according to a global soil texture map [Food and Agriculture Organization (FAO)/United Nations Educational, Scientific and Cultural Organization (UNESCO) Digital Soil Map of the World (DSMW); FAO 2003]. In addition, surface runoff is based on variable infiltration capacity. The soil heat budget follows a Fourier diffusion law, modified to take into account soil water freezing/melting according to Viterbo et al. (1999). The energy equation is solved with a net ground heat flux as the top boundary condition and a zero flux at the bottom. The water balance at the surface (i.e., the change in water storage of the soil moisture, interception reservoir, and accumulated snowpack) is computed as the difference between the precipitation and (i) the evaporation of soil, vegetation, and interception water and (ii) surface and subsurface runoff. First precipitation is collected in the interception reservoir until it saturated. Then, excess precipitation is partitioned between surface runoff and infiltration into the soil column. At the end of each data assimilation cycle an adjustment to the model forecast (e.g., soil moisture) is produced; it usually referred to analysis increment and represents the net response of the variational data assimilation to all observations used. H-TESSEL was implemented by Balsamo et al. (2009) and verified in a variety of ways including field site, data assimilation, and modeling experiments. However, they considered only a few selected soil moisture stations. Also in Balsamo et al. (2011), evaporation processes were revised and a monthly leaf area index (LAI) climatology (Bousseta et al. 2011) together with an improved bare ground evaporation parameterization (Balsamo et al. 2011) became operational in November 2010. Bare ground evaporation over dry lands has been enhanced by adopting a lower stress threshold than for the vegetation, allowing a higher evaporation. This is in agreement with the experimental findings of Mahfouf and Noilhan (1991) and results in more realistic soil moisture for dry land (Balsamo et al. 2011). Three analysis schemes for the surface (and near surface) variables are currently used in operations. They are based on spatial OI (for snow depth and screen-level analysis), column OI (for soil and snow temperature analysis), and a simplified EKF (for the soil moisture analysis). The analysis of surface parameters is decoupled from the main atmospheric analysis. Firstly, an optimal interpolation scheme produces estimates of screen-level temperature and relative humidity by combining synoptic observations over land with background estimates (short-range forecasts) from the most recent analysis (Douville et al. 2000). Analyzed fields of screen-level temperature and relative humidity are then used to update soil moisture (and soil temperature) estimates for the various layers of the model, either by a simple empirical approach (OI, for ERA-Interim and in operations before November 2010) or the EKF (in operations after November 2010). A description of the EKF can be found in Drusch et al. (2009) and a full description of the EKF implementation and evaluation is given in de Rosnay et al. (2011, 2012).
3) Soil moisture products
In this section a description is given of the major differences between the deterministic operational suite and ERA-Interim, with respect to soil moisture analyses.
The version of the IFS used in operations at ECMWF within the period from January 2007 to December 2010 spans cycles 31r2 to 36r4 (more information at http://www.ecmwf.int/research/ifsdocs/). The land surface scheme used in operations is TESSEL and its revised version, H-TESSEL, was implemented in operations on 9 November 2007. The revised bare ground evaporation and the monthly LAI were implemented in November 2010. Before the implementation of cycle 36r4 in November 2010, the assimilation technique used was OI (Mahfouf 1991; Mahfouf et al. 2000). On 9 November 2010, an advanced surface data assimilation scheme was implemented in operations to optimally combine model data with conventional observations and satellite measurements. It is based on an EKF, as described in Drusch et al. (2009) and de Rosnay et al. (2012). In its current configuration, the EKF soil moisture analysis uses only meteorological observations of screen-level parameters such as air temperature and relative humidity close to the surface, as with the previous OI method. However, owing to the flexibility of Kalman-based techniques, the EKF can handle different sources of observations (Mahfouf et al. 2009) and allows satellites data such as ASCAT to be assimilated (de Rosnay et al. 2012; Albergel et al. 2010b). The operational IFS soil moisture analysis is produced daily at 0000, 0600, 1200, and 1800 UTC, at a spatial resolution of about 25 km (T799) until 26 January 2010 and then at about 16 km (T1279). Analyses at 0000 UTC are considered in this study. For the 0000 UTC analysis, the 12-h 4DVAR analysis is run using observations in the time window 2100–0900 UTC. A summary of the improvements implemented in operation between 2007 and 2010, with respect to soil moisture, is given in Table 2.
ERA-Interim is the latest global atmospheric reanalysis produced by ECMWF (Dee et al. 2011); it uses IFS cycle 31r1. It covers the period from 1 January 1979 onward, and continues to be extended forward in near–real time (with a delay of approximately 1 month). Berrisford et al. (2009) provide a detailed description of the ERA-Interim product archive. ERA-Interim uses TESSEL for the whole period so it does not take into account the hydrological improvements discussed above. The assimilation technique used for soil moisture is OI with a resolution of about 80 km (T255).
b. In situ soil moisture observations
The different in situ soil moisture datasets used in this study are described in Table 1 and depicted in Fig. 1. The nearest-neighbor approach was retained to match soil moisture analysis with in situ data.
1) SMOSMANIA, SMOSMANIA-E, and SMOSREX
The SMOSMANIA network is a long-term data acquisition effort to obtain profiles of soil moisture observations in southwestern France (Calvet et al. 2007; Albergel et al. 2008). With this project, soil moisture profiles have been obtained from 12 automated weather stations from the Réseau d’Acquisition de Données d’Observations Météorologiques Etendu (RADOME) network run by Météo-France. Since January 2007 observations have been available from four depths (5, 10, 20, and 30 cm) following a Mediterranean–Atlantic transect. However, since January 2009, nine additional RADOME stations were equipped with soil moisture measurements in south and southeastern France (referred to as SMOSMANIA-E). For all the stations used, most are at low altitude and on reasonably flat terrains, four of them are above 450 m MSL, and one above 1200 m MSL. The vegetation cover at those sites consists of natural fallow. The soil moisture measurements are derived from capacitance probes: ThetaProbe ML2X of Delta-T Devices, which are easily interfaced with the RADOME stations. A ThetaProbe provides a signal in units of volts and its variation is virtually proportional to changes in the soil moisture content over a large dynamic range. To convert the voltage signal into a volumetric soil moisture content, site-specific calibration curves were developed using in situ gravimetric soil samples for each station and each depth (i.e., 84 calibration curves). In situ SSM (5 cm) are compared to the first layer of ECMWF analyses (0–7 cm) and an average of in situ data (10 and 20 cm) is compared to the second layer of ECMWF analyses (7–28 cm). The period used for the comparison is specified in Table 1. Located along the SMOSMANIA transect, the SMOSREX experimental site (de Rosnay et al. 2006) is also used in this study as it provides profiles of soil moisture since 2001. SM measurements are performed with a vertically installed ThetaProbe (0–6 cm) and at every 10 cm until almost 1-m depth (10, 20, 30, 40, 50, 60, 70, 80, and 90 cm).
In situ data at 38 stations of the OzNet network (Young et al. 2008; http://www.oznet.org.au) are used in this study. They are all located within the Murrumbidgee experimental catchment in southern New South Wales, Australia. Climate variations in the Murrumbidgee experimental catchment are primarily associated with elevation, varying from semiarid in the west (altitude from ~50 m MSL) to temperate in the east (altitude up to ~2000 m MSL). The highest station is 937 m MSL. Land use in the catchment is predominantly agricultural with some forested areas in the steeper parts of the catchment (Young et al. 2008). Each soil moisture site of the Murrumbidgee monitoring network measures the soil moisture at 0–5 cm with soil dielectric sensor (Stevens Hydra Probe) or 0–8, 0–30, 30–60, and 60–90 cm with water content reflectometers (Campbell Scientific). Hydra probes are soil dielectric sensor (operating at 50 MHz). At each measurement point, a volumetric soil moisture value is inferred from the real component of the measured relative dielectric constant and the conductivity from the imaginary component. Reflectometers measure the travel time of an output pulse to estimate changes in the bulk soil dielectric constant. Each measurement is converted to volumetric water content with a calibration equation parameterized with soil type and soil temperature. As the sensor response to soil moisture may vary with soil characteristics (e.g., salinity, density, soil type, and temperature), the sensor calibration was undertaken using both laboratory and field measurements. Reflectometer measurements were compared with both field gravimetric samples and time-domain reflectometry (TDR) measurements (these measurements are based on the relationship between the dielectric properties of soils and their moisture content).
The SCAN network (http://www.wcc.nrcs.usda.gov/scan/) is a comprehensive, nationwide soil moisture and climate information system designed to provide data to support natural resource assessments and conservation activities. Administered by the United States Department of Agriculture NRCS through the National Water and Climate Center, in cooperation with the NRCS National Soil Survey Center, the system focuses on agricultural areas of the United States. The observing network monitors soil temperature and soil moisture at several depths, soil water level, air temperature, relative humidity, solar radiation, wind, precipitation, and barometric pressure, amongst others. SCAN data are used for various studies from global climate modeling to agricultural studies. In total, 28 stations providing continuous measurements of soil moisture between 2007 and April 2011 were randomly selected within the whole United States. The vegetation cover at those sites consists of either natural fallow or short grass. Data are collected by a dielectric constant measuring device and typically measurements are made at 5, 10, 20, 50, and 100 cm.
West Africa has been extensively instrumented in the framework of the African Monsoon Multidisciplinary Analysis (AMMA), which is a project dedicated to improving our understanding and our modeling capabilities of the effect of land surface processes on monsoon intensity, variability, and predictability (Redelsperger et al. 2006). Three mesoscale sites were implemented in Mali (de Rosnay et al. 2009), Niger (Pellarin et al. 2009a), and Benin (Pellarin et al. 2009b), providing information along the north–south gradient between the Sahelian and Soudanian regions. Land use ranges from natural rangeland through crops to wooded savanna. Amongst others, soil moisture data are collected from stations within the three mesoscale sites. The same installation protocol is used for all the soil moisture stations where TDR sensors are used (Campbell CS616). When they were not suitable (e.g., because of soil texture), Delta-T ThetaProbes were used. In this study, data collected at 5 and 20 cm are used from 10 stations in Mali, Niger, and Benin.
The goal of this program is to support the development of soil–vegetation–atmosphere transfer models within the European SRNWP community by providing good quality operational data from a limited set of well-instrumented and high-quality observation sites, including soil moisture. SRNWP gathers soil moisture data from several European networks such as the SMOSREX experimental site, which has already been mentioned. In addition to SMOSREX data observations, the Lindenberg station is used. Lindenberg is a small village situated in a rural landscape in the east of Germany about 65 km to the southeast of the center of Berlin. The central part of the field site is a flat meadow covered by short grass. Soil moisture is measured at the upper level by four sensors (TDR) at 8 cm. Soil moisture determination using the gravimetric method is performed regularly during frost-free periods for comparison with the continuous sensor measurements.
6) ISMN soil moisture: REMEDHUS, UMSUOL, SWEXPOLAND, UDC-SMOS
21 stations from the Red de Medición de la Humedad del Suelo (REMEDHUS) network in Spain are available through the ISMN website. This network is located in the central sector of the Duero basin; the climate is semiarid continental Mediterranean and the land use is predominantly agricultural with some patchy forest. Each station has been equipped with capacitance probes (Stevens Hydra Probes) installed horizontally at a depth of 5 cm. Analysis of soil sample were carried out to verify the capacitances probes and to assess soil properties at each station (Martínez-Fernández and Ceballos 2005).
The San Pietro Capofiume station belongs to the UMidità del Suolo (UMSUOL) network located in northern Italy. It was installed by the Service of Hydrology, Meteorology and Climate of the Regional Agency for Environmental Protection in Emilia–Romagna (ARPA-SIMC; http://www.arpa.emr.it/sim/). The surrounding area is characterized by a Mediterranean semihumid climate and is covered by grass. Data are collected at 10 cm with TDR (TDR100, Campbell Scientific, Inc.). The Trzebieszow station from the Soil Water and Energy Exchange Poland (SWEXPOLAND) network in western Poland was also used; data are collected by means of a TDR instrument [EasyTest, a moisture/capillary pressure/temperature/salinity datalogger (D-LOG/mpts)] at 10-cm depth between January 2007 and September 2009 in an agricultural area.
Ten stations near the city of Munich in Germany from the Upper Danube Catchment Soil Moisture Observing System network (UDC-SMOS; Loew et al. 2009) are included in this study. Data are collected with TDR (IMKO-TDR) at 5 cm. This soil moisture network is run in cooperation with the Bavarian State Research Center for Agriculture and is carried out as part of the SMOSHYD project (Integrative analysis of SMOS soil moisture products at the Upper Danube, project number 50EE0731) funded by the German Aerospace Center (DLR). The area is characterized by agricultural land use (intensively used grassland).
The Arctic Research Center of the Finnish Meteorological Institute (ARC-FMI) monitors soil moisture at Sodankyla in a boreal forest. It contains multiple soil moisture measurements at 2 and 10 cm with ThetaProbes. Data at 10 cm are used in this study.
c. Statistical comparison between analysis and in situ observations
All the soil moisture products are in units of m3 m−3, but they might correspond to soil surface layers with different thicknesses (e.g., 0–7 cm for ECMWF products, 5 cm for the SMOSMANIA stations, and 8 cm for the OzNet stations). As both ECMWF products (operational analysis and ERA-Interim) use a multilayer soil moisture representation, in addition to SSM, the analyzed soil moisture content in the second layer of soil (7–28 cm) can be compared to the in situ observations at a deeper layer (an average at 10 and 20 cm for SMOSMANIA and 30 cm for OzNet) when available (69 stations). Albergel et al. (2008) used measurements at 0–6, 10, 20, 30, 40, 50, 70, 80, and 90 cm from the SMOSREX site (southwest France) to make an estimate of the entire root-zone soil moisture; Balsamo et al. (2009) used similar estimates from the Boreal Ecosystem Research and Monitoring Sites (BERMS) Old Aspen site in Canada to evaluate the performance of H-TESSEL. Considering our pool of stations, such estimate can be obtained for
OzNet where observations are available over three different depths: 0–30, 30–60, and 60–90 cm;
NRCS–SCAN with observations at about 5, 10, 20, 50, and 100 cm; and
SMOSREX with observations at 5, 10, and every 10 cm down to 1 m.
Data over the SMOSMANIA network could also be considered. In the case of SMOSMANIA, soil moisture observations are only available from 5 to 30 cm. Soil layers below 30 cm are not observed, and integrated root-zone soil moisture (over the first meter of soil) cannot be computed. However, according to Albergel et al. (2008), the analysis of the SMOSREX data (located within the SMOSMANIA network) indicates that individual soil moisture observations, at depths ranging from 20 to 50 cm, are significantly correlated to the integrated root-zone soil moisture. For the stations in the SMOSMANIA network, it is assumed that, based on correlation criteria soil moisture, observations at 30 cm are a good proxy of the root-zone soil moisture. Stations of the SMOSNANIA network are used to having a significant number of stations available. The resulting dataset of the entire in situ root zone is compared to its modeled equivalent for both ECMWF soil moisture products (weighted average of the three first layers of soil). In situ observations as well as the analysis are based on the 0000 UTC data. As in situ observations of soil moisture are frequently associated with soil temperature measurements, the soil moisture observations were flagged for temperatures lower than 4°C. For each of the 117 stations available in this study and for both operational (OPER) and ERA-Interim (ERA-I) analyses, correlation [R; Eq. (1)], bias (in situ minus analysis), and root-mean-square difference [RMSD, Eq. (2)] are computed for each year and for the whole 2008–10 period (extended to April 2011 for the stations of NRCS–SCAN network) between observations and analyses:
The rationale for using root-mean-square difference instead of root-mean-square error is to emphasize that, as well as for the model analysis, in situ data contain errors (instrumental and representativeness), so they are not considered as “true” soil moisture. Additionally, the normalized standard deviation (SDV) and the centered RMSD analysis and in situ patterns, normalized by the in situ standard deviation (E), are computed. SDV is the ratio between analyzed and in situ standard deviations; it gives the relative amplitude while E quantifies errors in the pattern variations. It does not include any information on biases since the means of the fields are subtracted before computing second order errors. SDV and E are expressed by Eqs. (3) and (4), respectively:
Taylor diagrams are used to represent these three statistics using two-dimensional plots. The normalized standard deviation is displayed as a radial distance and the correlation with in situ data as an angle in the polar plot. In situ data are represented by a point located on the x axis at R = 1 and SDV = 1. The distance to this point represents the centered normalized RMS difference (E) between the analysis and in situ patterns.
To avoid seasonal effects, monthly anomaly time series are also calculated. The difference from the mean is calculated for a sliding window of 5 weeks (if there are at least five measurements in this period), and the difference is scaled to the standard deviation. For each SSM estimate at day (i), a period F is defined, with F = [i − 17, i + 17] (corresponding to a 5-week window). If at least five measurements are available in this period, the average SSM value and the standard deviation are calculated. The anomaly (Ano), dimensionless, is then given by the following:
The same equation is used to compute in situ anomaly time series that can be compared with that from the operational and ERA-Interim soil moisture product. Anomaly time series reflect the time-integrated impact of antecedent meteorological forcing. The latter is mainly reflected in the first layer of soil, so anomaly time series are calculated only for this layer.
The p value (Schervish 1996), a measure of the correlation significance, is calculated for both anomaly and volumetric time series. It indicates the significance of the test; if it is small (e.g., below 0.05), it means that the correlation is not a coincidence. Only cases with p values below 0.05 are considered.
a. Using in situ data in southwestern France
The statistical scores for OPER and ERA-I are presented in Table 3 for the stations from SMOSMANIA and SMOSREX. Figure 2 illustrates the three soil moisture products used in this study for three stations from the SMOSMANIA network (Sabres, Lahas, and St Felix) over the 2007–10 period. The implementation of the H-TESSEL land surface scheme within OPER in November 2007 (black line) resulted in a shift in the soil moisture range (e.g., a shift down for Sabres and up for Lahas station). It is clear from Fig. 2 that, after the implementation of H-TESSEL, OPER has a larger variability than ERA-I (red line), which uses TESSEL for the whole period. Note that OPER and ERA-I are only similar for the period from January to October 2007. After November 2007, OPER has a larger dynamical range and is in better agreement with the observations. Statistical scores are computed for 2007, 2008, 2009, 2010, and for the period 2008–10. The shift induced by the implementation of H-TESSEL in November 2007 is an artifact that decreases the stability of the scores. That is why data from 2007 are not used when considering the whole period.
A comparison between in situ data and ECMWF products shows good temporal correlations for the 2008–10 period with R ranging from 0.73 to 0.87 with an average of 0.80 for OPER and 0.62 to 0.83 with an average of 0.77 for ERA-I. The SSM temporal dynamic is well captured by both OPER and ERA-I analyses. Biases range from −0.208 to 0.041 m3 m−3 with an average value of −0.050 m3 m−3 for OPER and from −0.175 to 0.087 m3 m−3 with an average of −0.035 m3 m−3 for ERA-I. No systematic biases are observed for this group of stations, though most stations have negatives values (10 of 13). RMSD are ranging from 0.044 to 0.211 m3 m−3 with an average value of 0.088 m3 m−3 for OPER and from 0.038 to 0.179 m3 m−3 with an average value of 0.097 m3 m−3 for ERA-I.
Statistics are also computed for the second layer of soil, too. Soil moisture analyses between 7 and 28 cm are compared to averaged in situ data at 10 and 20 cm. Correlations values range from 0.66 to 0.90 with an average of 0.82 for OPER and from 0.60 to 0.84 with an average of 0.78 for ERA-I. Biases range from −0.215 to 0.022 m3 m−3 with an average value of −0.056 m3 m−3 for OPER and from −0.177 to 0.049 m3 m−3 with an average of −0.049 m3 m−3 for ERA-I. RMSD range from 0.039 to 0.219 m3 m−3 with an average value of 0.081 m3 m−3 for OPER while they range from 0.035 to 0.184 m3 m−3 with an average value of 0.091 m3 m−3 for ERA-I. For the various periods considered (2007, 2008, 2009, 2010, and 2008–10), OPER has higher correlations along with a smaller bias and RMSD than ERA-I. All p values are below 0.05, indicating that all stations have significant level of correlations. Statistics for the root zone are presented in the next section.
Figure 3 shows two Taylor diagrams illustrating the statistics from the comparison between OPER and ERA-I analyses with in situ data for the 12 stations from the SMOSMANIA network and SMOSREX experimental site for 2007, 2008, 2009, and 2010 (the diagram on the left is for the first layer of soil and the one on the right is for the second layer). These results underline the good range of correlations with most values being between 0.70 and 0.90. Also, they show that the variability of ERA-I product (triangles) is smaller than that of the OPER analysis (circles) compared to in situ data. According to Fig. 3, for most stations the dynamical range of ERA-I is also smaller than for OPER and the in situ data. The triangular symbols, representing the ERA-I analysis, are systematically below the SDV value of 1 (blue dashed line on Fig. 3). As SDV is the ratio between analyzed and in situ standard deviations [see Eq. (3)], it indicates that the variability of the in situ data is higher than for ERA-I. The Taylor diagrams shown in Fig. 3 are consistent with the statistical scores presented in Table 3. They are complementary as they permit a better appreciation of the dynamics of the two ECMWF analyses, giving an additional indication on the relative amplitude and pattern variation. Taylor diagrams provide a global view of the dynamics of ECMWF soil moisture analyses (correlation and SDV). These results show the added value of the various upgrades affecting the operational analysis (including the change in spatial resolution in January 2010). The main contribution to the improvements in terms of soil moisture dynamic comes from the revised H-TESSEL land surface scheme. The soil physiographic parameters (wilting point and field capacity) associated with each soil texture in the new analysis produce a larger water holding capacity leading to a better representation of the observed dynamical range of in situ soil moisture.
b. Extension to other countries and root-zone soil moisture estimates
In addition to the stations from the SMOSMANIA network and the SMOSREX experimental site, data from other networks across the world are used. Consequently, a total of 117 stations can be used in the study. The results are presented in Table 4. On average, correlations are 0.70 (ranging from 0.52 to 0.84) for OPER and slightly lower at 0.63 for ERA-I (ranging from 0.47 to 0.81). Also ERA-I has a smaller bias (−0.079 against −0.081 m3 m−3) but higher RMSD (0.121 against 0.113 m3 m−3) than OPER. All p values are below 0.05, indicating that all correlations are significant.
The strong negatives biases shown in Table 4 (in situ minus analyses) indicate that OPER and ERA-I tend to overestimate soil moisture. Figure 4 displays a time series of soil moisture products used in this study for stations from three networks: Concejo del Monte in Spain (REMEDHUS network), Ginniderra in Australia (OzNet network), and Uapb_Earl in the United State (NRCS–SCAN network), from top to bottom. The seasonal water cycle for Ginniderra in Australia (Southern Hemisphere), which has maximum values in July–September (local winter), is different to that of Concejo del Monte in Spain (Northern Hemisphere), with minimum values in July–September (local summer). As with the SMOSMANIA and SMOSREX results, OPER has a higher variability than the observations and ERA-I has a smaller one. Figure 5 has two Taylor diagrams illustrating the statistics from the comparison between ECMWF products (OPER and ERA-I) and in situ data for 117 stations for 2008–10 (for NRCS–SCAN data, the period is extended to April 2011 and for SMOSMANIA-E only 2009 is considered).
Stations from SMOSMANIA, SMOSMANIA-E, SMOSREX, OzNet, NRCS–SCAN, and AMMA (Benin) are also used to analyze OPER and ERA-I soil moisture in the second layer of soil (69 stations). Results are presented in Table 5 and illustrated by Fig. 6. As with SSM, OPER (Fig. 6, left) and ERA-I (Fig. 6, right) have high correlations. Compared to the first layer of soil, the variability of the analysis in the second layer is lower and much closer to the in situ data (blue dashed line, SDV value of 1) with most stations having SDV values between 0.5 and 1.5. No systematic tendency is observed. Correlations are better with OPER (0.77 in average) than with ERA-I (0.70 in average). However, ERA-I has a smaller bias and RMSD (−0.058 and 0.092 m3 m−3) than OPER (−0.094 and 0.116 m3 m−3).
Results of the comparison using an estimate of the root-zone soil moisture within the first meter of soil are presented in Table 6 using 48 stations. As for the first two layers of soil evaluated in this study, better averaged correlations are found for the stations from the SMOSMANIA network than for the OzNet and NRCS–SCAN. These new results are consistent with the data shown in Tables 4 and 5. Additionally, except for SMOSREX stations, better correlations are found for ERA-I than for OPER. In situ root-zone soil moisture (integrated over 0–100 cm), presents a smaller variability than SSM. This lack of variability is more in line with ERA-I. Comparison between in situ data and both ECMWF products shows good temporal correlations for 2008–10 with correlations ranging from 0.47 to 0.80 with an average of 0.70 for the OPER and 0.45 to 0.85 with an average of 0.67 for ERA-I. On average, the bias and RMSD (−0.033 and 0.079 m3 m−3, respectively) are better than for the two first layers of soil.
c. Comparison of the anomaly time series
Results presented above give an overview of the comparison of products on the annual scale. To address the ability of the products to capture the short-term SSM variations, anomaly time series were derived (section 2c) and correlations were computed for the anomaly time series over the 2008–10 period. Soil moisture for the same time within that period are used for the three products (OPER, ERA-I, and in situ) to take account of a potentially large temporal variability in the data. Only configurations associated with significant correlation values (p value < 0.05) are considered. Correlations of the anomaly time series are reported in Table 7; they range from 0.29 to 0.61 with an average of 0.51 for OPER and from 0.27 to 0.62 with an average of 0.49 for ERA-I.
In general, both OPER and ERA-I analyses captured well the temporal dynamics of the observed soil moisture. Better scores are found for the SMOSMANIA network and the SMOSREX experimental site than for the other networks. Information from observations of air temperature and air humidity close to the surface is used to analyze soil moisture. Therefore, this analysis is more effective in data-rich areas, such as southwestern France, which have higher correlations and smaller RMSDs than in other areas.
a. Soil moisture range
Results presented in the previous section show that OPER and ERA-I tend to overestimate soil moisture. This is particularly clear in dry areas such as in Australia where all the stations used for the comparison have negatives biases (from −0.272 to 0.033 m3 m−3 and −0.178 to −0.014 m3 m−3 for OPER and ERA-I, respectively). The improved bare ground evaporation over dry land, which was implemented in operations in 2010 (Balsamo et al. 2011), reduces biases. Its impact is illustrated by Fig. 7, which depicts observed soil moisture time series at two stations from the NSCR-SCAN between January 2010 and 15 May 2011. Enterprise station (in Utah) and Pine Nut station (Nevada) are located in areas with less than 400 mm of rain according to the Parameter-elevation Regressions on Independent Slopes Model (PRISM; http://www.prism.oregonstate.edu) annual climatology computed for 1971–2000. Before 9 November 2010, the operational SSM minimum values were limited by the dominant wilting point vegetation types’ parameter values; however, ground data indicate much drier conditions, as is clearly seen in Fig. 7 from May to September 2010. In spring 2011, the new bare ground evaporation allows the model to go below this wilting point value so the operational analysis is in better agreement with the observations. Longer-term evaluation over multiple sites will be necessary to consolidate this result, but the first 6 months of the operational analysis with the improved model are very encouraging. Albergel et al. (2010b) have already highlighted that the biases observed for OPER might be caused by shortcomings in the soil characteristics and pedotransfer functions that are employed, as well as by the difficulty of representing the spatial heterogeneity of these properties. Further improvements might be obtained by a better representation of soil texture. The soil texture map currently used at ECMWF is from the FAO dataset (FAO 2003) and the implementation of a new map such as the new comprehensive Harmonized World Soil Database (HWSD) (FAO/IIASA/ISRIC/ISSCAS/JRC 2009) could lead to better results.
b. Soil moisture variability
In this study observations at a specific site are compared with model output at either 16- or 25- and even 80-km scale. Several authors have demonstrated that local measurements could be used to validate model output as well as remotely sensed SSM at a different scale (e.g., Albergel et al. 2009, 2010b; Rüdiger et al. 2009; Brocca et al. 2010). However, spatial variability of SSM is very high and can vary from centimeters to meters. Precipitation, evapotranspiration, soil texture, topography, vegetation, and land use could either enhance or reduce the spatial variability of soil moisture depending on how it is distributed and combined with other factors (Famiglietti et al. 2008). Differences in soil properties could imply important variations in the mean and variance on soil moisture, even over small distances. While comparisons between ECMWF products and in situ data provide good correlations, they still have high RMSDs as discussed above. These findings are in agreement with the suggestion of Saleem and Salvucci (2002) and Koster et al. (2009, 2011) that the true information content of modeled soil moisture does not necessarily rely on their absolute magnitudes but instead on their time variation. The latter represents the time-integrated impacts of antecedent meteorological forcing on the hydrological state of the soil system within the model. When compared with field site experiments, H-TESSEL modifications show a shift in the range of the soil moisture and give a better agreement with observations. The soil physiographic parameters (wilting point and field capacity) associated to each soil texture produce a larger water holding capacity. Also it permits a better representation of the soil moisture annual cycle. Looking at the short-term variability, it is interesting to note that ERA-I and OPER have similar levels of correlation on average. For the AMMA stations in western Africa, correlations are driven by the annual cycle and the representation of the short-term variability by ECMWF’s two products is poor. The same conclusions apply for the Sodankyla site in north Finland. Short-term variability allows the assessment of soil moisture temporal pattern response to large-scale environmental conditions such as atmospheric forcing (e.g., precipitation) but also to the small-scale environmental conditions such as soil physiographic parameters. Compared to TESSEL, the larger water holding capacity and the improved soil properties parameters in H-TESSEL produce a more realistic decrease in soil moisture after a precipitation event. This explains the slightly better scores obtained with OPER than with ERA-I (correlations of 0.51 and 0.49, respectively). The large overestimation of soil moisture over dry areas (e.g., over the AMMA network) explains the poor level of correlation obtained for the monthly anomalies over such areas. Albergel et al. (2012) have analyzed the short-term variability of another soil moisture product from ECMWF. They found that even if most peaks and troughs are well represented, some precipitations events are missed or added by the model. ECMWF’s analysis does not assimilate observations of ground-based precipitations. Over land, the information used by the model to generate rain is strongly constrained by in situ measurements of temperature and humidity. Hence the quality of precipitation estimates tends to be better over well-observed land locations. It affects the quality of the short-term variability (precipitation is the main driver of soil moisture temporal pattern). However, the use of precipitation data in the analysis continues to be studied at ECMWF; Lopez (2011) has demonstrated a positive impact on model performance of the direct 4DVAR assimilation of 6-hourly radar and rain gauge rainfall accumulations. Similarly, as only observations of temperature and relative humidity close to the surface are used, the analyzed soil moisture is of better quality over well-observed land areas (e.g., SMOSMANIA network). Correlations for stations from the AMMA network and from Sodankyla are among the weakest from both the volumetric and monthly anomaly time series.
The high correlations associated with the ECMWF’s products are supportive of the development of a root-zone soil moisture index, which could be of interest for potential users. In the framework of the Satellite Application Facility on Support to Operational Hydrology and Water Management (H-SAF) EUMETSAT project, an advanced surface data assimilation system is being developed at ECMWF to retrieve a root-zone soil moisture profile index from satellite data. It is based on the new EKF soil moisture analysis and uses the ASCAT SSM. This results in the first global product containing consistent surface and root-zone soil moisture available in near real time for the numerical weather prediction, climate, and hydrological communities (Albergel et al. 2012).
In this study, soil moisture observations from various countries, under different biome and climate conditions, were used to evaluate two ECMWF soil moisture products, the one used in operations and the interim reanalysis, ERA-Interim. Datasets from 117 stations from the 133 available were used to evaluate ECMWF analyses in the first soil layer. Among them, 69 stations also provide data that were used to verify the performance of soil moisture products in the second layer of soil and 48 over the first meter of soil. The operational product relies on an analysis and model system that is revised on a regular basis, while ERA-Interim is produced by a fixed analysis and model system. It is shown that a major difference between the two products with respect to soil moisture is the use of an improved soil hydrology (H-TESSEL) in operations from November 2007, as well as new bare ground evaporation and an extended Kalman filter soil moisture analysis implemented from November 2010. In addition, the spatial resolution of the operational product increased from 25 to 16 km in January 2010, leading to a general improvement of the atmospheric forecast. In general, both operational and ERA-Interim analyses captured well the temporal dynamics of the observed soil moisture, with average correlations of 0.70 for the operational product and 0.63 for ERA-Interim (in the first layer of soil, for the 2008–10 period against 117 stations). However, ECMWF’s soil moisture products have a large RMSD (average of 0.113 and 0.121 m3 m−3 for the operational product and ERA-Interim, respectively) and tend to overestimate soil moisture with negative biases of −0.081 and −0.079 m3 m−3, respectively, for the operational product and ERA-Interim.
Strong negatives biases (in situ minus analysis) and high RMSD are found especially over dry areas (OzNet in Australia, REMEDHUS in Spain, and AMMA in West Africa). The improvements introduced in November 2010 discussed above have overcome this weakness. The first 6 months of operational analysis with the new bare ground evaporation are shown to decrease the bias and RMSD. The added value of the EKF analysis at ECMWF has already been demonstrated in previous studies. The flexibility of the EKF soil moisture analysis compared to the former OI analysis opens a wide range of development possibilities. In addition to the use of satellite-based soil moisture information from both active and passive microwave sensors [e.g., ASCAT, SMOS, and the upcoming Soil Moisture Active/Passive mission (SMAP)], an extension of the EKF to analyze other variables such as snow mass and vegetation parameters is under development at ECMWF. In recent years, the operational soil moisture analysis is shown to have performed better than the ERA-Interim version for most of the stations. However, the ERA-Interim product has a consistent and good performance over the period 2007–10 with an average correlation of 0.70 for the 117 sites. This result adds to the robustness of ERA-Interim for climate studies as shown in Simmons et al. (2010). It also highlights the potential of future reanalyses stemming from the European Union (EU)-funded ERA-Clim project, which will include recent model and data assimilation advances. Finally the root-zone soil moisture developed at ECMWF in the framework of the H-SAF project combines satellite-derived soil moisture information through the EKF analysis (ASCAT SSM). It will provide, for the first time, a global product of consistent surface and root-zone soil moisture index available in near real time. Realistic initial states for the soil moisture variables are required from many applications, from forecasts of weather and seasonal climate variations to models of plant growth and carbon fluxes.
Authors would like to acknowledge the re-analysis group at ECMWF for making the data available, J.-C. Calvet from Météo-France for giving access to the SMOSMANIA and SMOSMANIA-E soil moisture data, as well as T. Pellarin from LTHE for giving access to AMMA soil moisture data and the SRNWP Data Exchange Programme for the Lindeberg data. J. Walker and C. Rüdiger are thanked for the OZNET soil moisture data. The initial setup and maintenance of the OzNet monitoring network was funded by two Australian Research Council grants (DP0343778, DP0557543). R. Riddaway from ECMWF is thanked for his valuable comments on the English style. A. Bowen from ECMWF is thanked for her help in improving the figures. Authors thank the EUMETSAT Satellite Application Facility on Support to Operational Hydrology and Water Management (HSAF) for their funding support.