## 1. Introduction

The soil heat flux quantifies transport of heat through planes perpendicular to the soil surface. At the surface it quantifies the energy transport (positive downward) through the interface between the air–vegetation continuum and the soil. The thus-defined surface soil heat flux *G*_{0} is in balance with the net radiation *R _{N}* and the atmospheric transport (positive upward) of sensible heat

*H*and latent heat LE (Stull 1988).

During daytime and for vegetated surfaces, *G*_{0} is relatively small—about 10% of *R _{N}* (De Bruin and Holtslag 1982; Stull 1988). For sparsely vegetated and nonvegetated surfaces in semiarid regions,

*G*

_{0}becomes a more-important component of the surface energy budget (SEB; Passerat de Silans et al. 1997; Heusinkveld et al. 2004). Also, during nighttime

*G*

_{0}is important, because

*H*and LE are much smaller in nocturnal conditions than during daytime. As a consequence,

*G*

_{0}plays a significant role in the development of stable boundary layers (Van de Wiel et al. 2002) and in fog (Duynkerke 1991) and dew formation (Garratt and Segal 1988).

Local measurements of the soil heat flux at certain depths are obtained using heat flux plates (Portman 1958). To determine *G*_{0}, heat flux plates should be placed very close to the surface. This approach is, however, only applicable when a sensor is used whose thermal properties are comparable to the thermal properties of the soil (Mayocchi and Bristow 1995). Using such a configuration, Heusinkveld et al. (2004) were able to measure the soil heat flux in the Negev Desert.

In general, values of the thermal properties of the soil differ from thermal properties of heat flux plates that are used at measurement sites. In those situations, *G*_{0} is inferred from a combination of heat flux plates that are placed at depths that are deeper than a few centimeters and one or more soil temperature sensors. The estimation of *G*_{0} involves an extrapolation toward the surface, for which three methods are available. The first method is the calorimetric method (Fuchs and Tanner 1968; Massman 1992; Heusinkveld et al. 2004), in which *G*_{0} is calculated by integrating the one-dimensional heat conduction equation between the surface and the installation depth of the heat flux plate. The second method is the harmonic analysis method (Van Wijk and de Vries 1963; Massman 1992; Heusinkveld et al. 2004), in which *G*_{0} follows from extrapolated harmonic functions that are fitted to the vertical profile of measured soil heat fluxes and soil temperatures. The third method is the lambda method (De Bruin and Holtslag 1982), in which *G*_{0} is calculated as the product of the observed surface gradient of soil temperature and an apparent thermal soil conductivity at the surface, which is derived from harmonic analysis of the observed soil heat fluxes and the observed surface temperature gradient.

A major disadvantage of the calorimetric method is that this method requires detailed knowledge of the thermal properties of the soil—most notable, the volumetric heat capacity of the soil, but also the soil heat conductivity of the soil, because it is an important parameter for relating the heat flux over the plate to an undistorted soil heat flux (Philip 1961). In contrast, the harmonic analysis method does provide estimates of the thermal properties of the soil, provided that at least two measurements of the soil temperature are taken in conjunction with at least one measurement of the soil heat flux. A major disadvantage of this method is that it requires the slowest Fourier mode to be identifiable and measureable. Furthermore, it cannot calculate temporal variations in the soil characteristic that are more rapid than the identified slowest cycle. A major disadvantage of the lambda method is that it determines the gradient at the surface by means of the difference between a measurement at the surface and a measurement at a few centimeters below the surface. In some situations, this approach might lead to underestimations or overestimations of the gradient at the surface.

In this paper, we pursue the use of an explicit model for the diffusion of heat from the vegetation canopy to the soil surface and within the soil column in conjunction with a variational data assimilation technique. Using variational data assimilation techniques has a long history in weather forecasting (Daley 1991). Their application has recently been extended to 1D column models of the atmosphere (Lopez et al. 2006; Margulis and Entekhabi 2001) and land surface models (Boni et al. 2001). The main advantage of this method is that it provides optimal estimates of the soil characteristics in a user-defined data assimilation window. Other advantages over the traditional methods to estimate the surface soil heat flux are that 1) the system provides a more flexible framework in which it is easy to incorporate different types of observations that contain information on the thermodynamic state of the soil in a consistent way, 2) the system is able to incorporate instrumental corrections that depend on the thermal parameters of the soil, 3) an objective quality assessment of the system and different sensors that are used within the system is possible, and 4) the system can be extended to incorporate known inhomogeneities of the soil. The novel method is evaluated for measurements taken during 2003 at the Cabauw measurement facility in the Netherlands (Van Ulden and Wieringa 1996).

In section 2, the variational data assimilation framework is described. Section 3 addresses the measurements that are used in the variational data assimilation framework to estimate the surface soil heat flux. In section 4, the numerical setup for the validation study is given. In section 5, results of the validation study are presented and discussed. Section 6 contains a summary and conclusions.

## 2. Variational data assimilation method

### a. Forward model

*T*. The soil temperature is found by integrating the one-dimensional diffusion (Fourier) equation from a starting time

*t*=

*t*

_{0}at which an initial profile for

*T*is specified:

*c*is the volumetric heat capacity of the soil,

_{p}*λ*is the soil heat conductivity,

*z*is the depth below the surface, and

*f*

_{1}(

*z*) gives the initial profile of

*T*. Applying Eq. (1a) requires specification of boundary conditions at the top and bottom of the soil column. The upper boundary condition at the top of the soil column is formed by the surface soil heat flux. The surface soil heat flux follows from the transport of heat between the vegetation canopy and the surface of the soil, the longwave radiative flux difference between the vegetation canopy and the soil surface, and the incoming shortwave radiation that is absorbed at the soil surface. It is parameterized according to the formulation given by Duynkerke (1992), which translates to a Robin boundary condition for Eq. (1a) that reads as

*T*is the vegetation canopy temperature,

_{υ}*τ*is the transmissivity of the vegetation, and

*K*

_{in}is the incoming shortwave radiation flux (here, the angle brackets indicate the outermost level of nesting of terms, rather than averaging). At the lower boundary, a Dirichlet boundary condition is implemented:

*z*denotes the depth at which the bottom of the soil column is located and

_{b}*f*

_{2}(

*t*) specifies the soil temperature at the bottom of the column.

### b. Optimization of parameters

*λ*,

*c*, Λ, and

_{p}*τ*need to be specified. These are found using a variational data assimilation framework. Within this framework, we seek statistically optimal values for these parameters as a function of misfits between uncertain measurements and simulated equivalents of the measurements, and uncertain prior estimates of the parameters. Assuming that both measurements and prior value estimates can be considered as realizations drawn from a Gaussian distribution, deriving statistically optimal values for the parameters is equal to finding parameter values that minimize a cost function that reads

**p**is the vector containing the parameter values:

**p**= (

*λ*,

*c*, Λ,

_{p}*τ*)

^{T}(T indicates transpose);

**p**

_{pr}is the vector that contains the prior estimates of the parameter values:

**p**

_{pr}= (

*λ*

_{pr},

*c*

_{p}_{,pr}, Λ

_{pr},

*τ*

_{pr})

^{T}, where

*λ*

_{pr}is the prior value of

*λ*,

*c*

_{p}_{,pr}is the prior value of

*c*, Λ

_{p}_{pr}is the prior value of Λ, and

*τ*

_{pr}is the prior value of

*τ*; 𝗖

*is the error-covariance matrix of the prior values;*

_{p}**Z**is the vector containing the measurements;

**M**is the vector that gives the modeled equivalents of the measurements that result from propagating the forward model; and 𝗖

*is the error-covariance matrix of the measurements.*

_{υ}*L*is the Lagrange function,

*t*is the end time of the integration (end time of data assimilation window), and

_{f}*μ**(

*z*,

*t*) is the Lagrange multiplier function.

### c. Numerical implementation

Obtaining analytical solutions for both the forward model and the adjoint model is not straightforward. The partial differential equation describing the forward model is homogeneous, but it includes inhomogeneous initial and boundary conditions. The adjoint model has homogeneous terminal and boundary conditions, but the partial differential equation describing its evolution includes nonlinear source terms. Because, in addition, Eqs. (6a)–(6d) depend nonlinearly on the model parameters, a numerical approach has been adopted to find optimal parameters.

First, the forward model is solved using a Crank–Nicolson finite-difference scheme. This leads to a vector propagation equation that calculates the temperature for a set of discrete depths and time steps starting from a discretized initial condition. Second, a discrete formulation of the Lagrange function is derived, whereby, instead of the continuous forward model, the discretized forward model is added as a constraint. Third, the discrete adjoint model is derived by differentiating the discrete formulation of the Lagrange function to the different components of the discrete temperature vector. Last, the discrete formulation for the Lagrange function is differentiated with respect to the model variables, yielding discrete equivalents of Eqs. (6a)–(6d). Our approach can be identified as the adjoint of finite-difference approach, as outlined by Sirkes and Tziperman (1997). The resulting equations are given in the appendix. Equations (A1a) and (A1b) describe the finite-difference form of the forward model, Eqs. (A3a) and (A3b) describe the resulting discrete adjoint model, and Eq. (A4) summarizes the derived finite-difference equivalents of Eqs. (6a)–(6d).

Because Eqs. (6a)–(6d) and their numerical equivalents represented by Eq. (A4) are nonlinear in the model parameters, finding the minimal values for the model parameters requires an iterative procedure. We follow Margulis and Entekhabi (2004) and employ a steepest descent technique to minimize *L* with respect to the control variables. This method is summarized in algorithm 1, in which *q* refers to the iteration step counter, *γ _{q}* is the step size matrix, ε is the scalar convergence factor, and the subscript refers to the iteration step under scope. Algorithm 1 consists of the following eight steps:

set

*q*= 1,set

**p**=_{q}**p**_{pr},calculate

**T**(_{q}*t*),_{i}*i*= 1, 2, 3, … ,*N*using Eqs. (A1a) and (A1b),calculate

*μ*(_{q}*t*),_{i}*i*= 1, 2, 3, … ,*N*using Eqs. (A3a) and (A3b),calculate ‖∂

*L*/∂**p**‖using Eq. (A4), where the double bars denote the norm,_{q}- calculateusing Eq. (A4),
set

*q*=*q*+ 1, andrepeat steps 3–7 until ‖∂

*L*/∂**p**‖/‖∂_{q}*L*/∂**p**‖_{1}< ε.

## 3. Observations

### a. General characteristics

The tower at the Cabauw measurement facility site is located at 51°58′N, 4°56′E, in the central Netherlands. It consists of flat grass meadows and ditches, surrounded by a flat area with villages, orchards, and lines of trees (Beljaars and Bosveld 1997).

Jager et al. (1976) determined the soil in the Cabauw area using laboratory analysis of soil samples and inspection of a soil column in a 120-cm-deep pit (see Beljaars and Bosveld 1997). The soil in Cabauw consists, from the surface downward, of a 2-cm turf layer, an 18-cm upper clay layer that is relatively rich in organic matter, a 42-cm-thick (from 18 to 60 cm) lower clay layer that is relatively poor in organic matter, and a 15-cm-thick mixed clay/peat layer, and the layer underneath the soil consists of peat. Based on these characteristics, the upper 18 cm of the upper layer are identified as B11 (fairly heavy clay) in the soil-type classification for the Netherlands as proposed by Wösten et al. (1994).

### b. Soil heat flux

*G*is the soil heat flux,

*G*is the heat flux through the plate,

_{p}*T*is the thickness of the plate,

_{p}*d*is the diameter of the plate, and

_{p}*λ*is the heat conductivity of the plate material. The heat flux through the plate is calculated as the product of the conductivity of the plate and the temperature gradient over the body of the plate. The latter is determined by the ratio of the temperature difference over the plate (which is measured using a thermopile) and the thickness of the plate.

_{p}At the Cabauw site, six WS31S soil heat flux plates manufactured by TNO-Delft are installed. The body of these plates consists of ceramic–plastic material, has a diameter of 110 mm, and is 5 mm thick. These plates are buried at depths of approximately 5 and 10 cm below the surface, located at the three vertices of an equilateral triangle. First, for each depth and at each vertex, 10-min averages of the soil heat flux are obtained. Second, a linear average of the fluxes at the three vertices is performed so that average heat fluxes over the soil terrain at the Cabauw site at depths of approximately 5 and 10 cm are obtained.

### c. Soil temperature

Ten-minute averaged profiles of soil temperatures are measured at the center of the equilateral triangle used to determine the average soil heat flux profile. The thermometers are made at Koninklijk Nederlands Meteorologisch Instituut (KNMI) and consist of a 35-cm-long nickel needle with a temperature-dependent electrical resistance of approximately 500 Ω. The thermometers were placed at depths of approximately 0.004, 0.02, 0.04, 0.08, 0.12, 0.20, 0.3, and 0.5 m. Depths were obtained using a metal plate at the grass layer at the edge of the pit, whereby the lower side of the plate served as the zero reference level.

### d. Incoming shortwave radiation, vegetation canopy temperature, and soil moisture content

At the Cabauw site, 10-min averages of the incoming shortwave radiation fluxes are saved. Incoming shortwave radiation is measured at the radiation field, which is located about 20 m away from the soil site. Incoming shortwave radiation is measured at a height of 1.5 m above the surface using a ventilated and heated Kipp & Zonen CM11 pyranometer.

The vegetation canopy temperature is estimated from 10-min averages of the infrared radiation temperature, as measured with a Heimann K-15–85 radiation thermometer. The thermometer is mounted at 2 m, and it looks downward on the soil site with an M6 lens with a field of view of 40°. The field of view is thus a circle with a diameter of about 1.5 m, which is centered around the location at which the soil temperature profile is taken.

Soil water content is measured at depths of 0.03 and 0.08 m below the surface of the soil terrain. It is measured using time domain reflectometry sensors manufactured by Campbell Scientific, Inc. In Cabauw, sensor type CS615 is used, which has a rod length of 0.3 m, and the width between the rods is 0.0032 m. For calibration, the standard calibration function provided by the manufacturer is used. Note that calibration for the (clayey) soil found at Cabauw is uncertain. Averages over 10 min are saved.

## 4. Setup of study

### a. General numerical setup and assimilation window

We have run the (adjoint) model for 2003. As the soil type changes at a depth of 0.18 m below the surface, the bottom of the modeled soil column is chosen to be at 0.12 m (*z _{b}* = 0.12 m); Δ

*z*is set at 0.005 m. To optimize the involved parameters, a data assimilation time window of 24 h is used. Each 24-h period starts at 0000 UTC. The model is initialized by linear spatial interpolation of the first 10-min-averaged temperature profile after 0000 UTC. The Δ

*t*is set at 600 s, implying that

*N*= 144. At the lower boundary, the model is forced by the measured 10-min averages of the soil temperature at

*z*. At the upper boundary of the soil column, the model is forced using the 10-min averages of the temperature measured by the Heimann thermometer and 10-min averages of the incoming shortwave radiation flux.

_{b}### b. Parameter prior values and uncertainty

Because a time-assimilation window of 24 h is adopted, 24-h averages for the control variables are required. As prior values for the control variables for each 24-h period, their values on the previous 24-h period are used. Prior uncertainties in these estimates are assumed to be uncorrelated. The off-diagonal elements of the a priori error-covariance matrix are thus set to zero. For the diagonal elements *C _{p}* of the error-covariance matrix 𝗖

*, we assume that the error in each parameter is 50% of the prior value:*

_{p}*C*(1, 1) = (

_{p}*λ*

_{pr}/2)

^{2},

*C*(2, 2) = (

_{p}*c*

_{p}_{,pr}/2)

^{2},

*C*(3, 3) = (Λ

_{p}_{pr}/2)

^{2}, and

*C*(4, 4) = (

_{p}*τ*

_{pr}/2)

^{2}.

*θ*is the soil moisture content. Here, we estimate the soil moisture content as the average of the daily averaged soil moisture content at 0.03 m, and the daily averaged soil moisture content measured at 0.08 m. For the volumetric heat capacity, the prior value at the beginning of 2003 is calculated as

^{−2}K

^{−1}at the beginning of the year, and

*τ*is initialized at a prior value of 0.05 (Duynkerke 1992).

### c. Applied measurements and uncertainty

Ten-minute averages of the soil temperature and soil heat flux measurements are available. Because the first 10-min average after 0000 UTC is used for initialization, at each depth only 143 of the 10-min averages of the soil temperature profile and the soil heat flux profile are independent measurements that can be used to optimize the parameters. Though the measurements can be assumed to be independent, modeling errors might be biased, leading to time correlations among misfits between measurements and modeled equivalents (Bosveld and Bouten 2001). To mitigate this, we only use measurements taken at *t* = 0300, 0600, 0900, 1200, 1500, 1800, 2100, and 0000 UTC. Thus, the observations obtained at each depth and at each time step are assumed to be independent and the off-diagonal elements of the matrix 𝗖* _{υ}* are zero. The diagonal elements refer to the variance associated with the error in the soil temperature measurement and soil heat flux measurements that results from both instrumental errors and representation errors.

*C*(G05) is the element of 𝗖

_{υ}*that refers to the daily averaged variance of the plate heat flux measurements taken at approximately 0.5 m;*

_{υ}*C*(G10) is the element of 𝗖

_{υ}*that refers to the daily averaged variance of the plate flux measurements taken at approximately 0.1 m;*

_{υ}*i*denotes the index of the 10-min averages; GS05, GW05, and GE05 denote the plate heat flux measurements taken at 0.5 m at, respectively, the southern, western, and eastern vertex of the equilateral triangle; GS10, GW10, and GE10 denote the plate heat flux measurements taken at 0.1 m at, respectively, the southern, western, and eastern vertex ofthe equilateral triangle; G05 denotes the average plate heat measurement at 0.5 m; and G10 denotes the average plate heat flux measurement at 0.1 m.

*C*(

_{υ}*T*) are the elements of 𝗖

*that refer to the variance of the error in the soil temperature measurements at the different depths and dn is the yearday (DOY). Also, only temperatures taken at 0300, 0600, 0900, 1200, 1500, 1800, 2100, and 0000 UTC are used.*

_{υ}### d. Minimization algorithm parameters

*η*is the scalar step size, which is set to a constant value of 1 × 10

^{−4}. For the convergence factor ε, we adopt a value of 1 × 10

^{−3}.

## 5. Results and discussion

Our intent is to evaluate the performance of the proposed variational data assimilation approach in providing estimates of *G*_{0} as a term of the SEB. First, the approach is analyzed and evaluated by examining the characteristics of the cost function that it provides. Second, important side products such as the optimal values for the model parameters are presented and discussed. Third, estimates of the surface soil heat flux obtained using the approach proposed in section 2 are compared with estimates of the surface soil heat flux obtained by the lambda approach. Last, the effect of using the proposed approach to estimate the surface soil heat flux on the SEB is explored.

### a. Cost function characteristics

The approach as presented in section 2 has been applied for the entire year of 2003. It appears that for 2003 there are 223 days for which all of the data described in section 3 are available. For these days in Fig. 2, the posterior value of the cost function (denoted *J*_{post}), that is, the value of the cost function after optimizing, is plotted against the prior value of the cost function (denoted *J*_{prior}), that is, the value of the cost function when it is evaluated using the prior values for the parameters. Also shown are values of 33.1 and 65.2, which define the 5% and 95% quantile of the chi-square distribution with 48 degrees of freedom.

From Fig. 2 it appears that, on average, *J*_{post} is about 70% of *J*_{prior}. For values of *J*_{prior} that are lower than 35, the difference between the prior cost function and the posterior cost function is relatively small. In these cases, the observations contain only little extra information on the model parameters. For higher values of *J*_{prior}, the difference between *J*_{prior} and *J*_{post} becomes much larger, indicating that in these cases the observations put a much stronger constraint on the model parameters. Note that on some days the reduction of *J*_{post} with respect to *J*_{prior} exceeds 50%. As Margulis and Entekhabi (2004) argued, values of *J*_{post} on each day can be used to evaluate the validity of the prescribed error characteristics. To understand this, it is important to acknowledge that, for independent observational errors, *J*_{post} is the sum of 48 (8 × 6 + 4 − 4) independent Gaussian distributions. As a result, it should theoretically be distributed according to a chi-square distribution with 48 degrees of freedom (Menke 1989; Margulis and Entekhabi 2004). Such a distribution has a mean of 48 and a standard deviation of 9.8. If the posterior cost function is much smaller than 48, this suggests that either the prescribed observation error characteristics overestimate the uncertainty in the observations, or errors among different observations show correlations that are not accounted for. If the posterior cost function is much larger than 48, the prescribed uncertainty might be too small. Thus, Margulis and Entekhabi (2004) assessed the statistical confidence in the posterior parameter estimates and the statistical assumptions in the variational data assimilation setup by evaluating whether the value of the posterior cost function was within one or two standard deviations from the mean value. We follow a slightly different approach and validate the posterior cost function by evaluating whether it is within the 90% confidence interval of its theoretical probability distribution. As such, posterior cost functions are only valid for days in which the posterior cost function lies between 33.1 and 65.2. The 101 days for which this condition is met will be further analyzed, and the remaining days are not used.

For the selected days we show in Fig. 3, the contribution of different components to the entire posterior cost function [Eq. (2)] as given in Fig. 2. Components include a component penalizing deviations from the prior values, denoted by *J*_{ap} (Fig. 3a), a component penalizing deviations from the observed soil temperatures, denoted by *J _{T}* (Fig. 3b), and a component penalizing deviations from the observed soil heat fluxes, denoted by

*J*(Fig. 3c). It appears that

_{G}*J*dominates the posterior cost function throughout 2003. During the entire year,

_{G}*J*scatters between a value of 20 and 60, with an average value of about 40. In contrast,

_{G}*J*is smaller, ranging from a value of about 13 around DOY 2003 50 and 350 to a value of about 4 around DOY 2003 200. This is surprising, because the data assimilation cycle uses on each day 32 soil temperature measurements and 16 soil heat flux measurements. This suggests that the prescribed uncertainty in the soil heat flux as derived using Eq. (10) underestimates the real uncertainty in the soil heat flux measurements, while the prescribed uncertainty in the soil temperature measurements as derived using Eq. (11) overestimates the real uncertainty in the soil temperature measurements. Also,

_{T}*J*has a clear seasonal cycle, with values of about 13 in winter conditions, around DOY 2003 50 and 350, and values of about 3 in summer conditions, around DOY 2003 200. Of interest is that the observed seasonal cycle in the values of the posterior cost function is reflected in the prescribed seasonal cycle in the standard deviation as inferred from Scharringa (1976). In winter conditions, Eq. (11) predicts relatively low values of the standard deviation leading to high values of the cost function or given discrepancies between observed soil temperatures and modeled equivalents. In summer conditions, Eq. (11) predicts relatively high values of the standard deviation leading to low values of the cost function or given discrepancies between observed soil temperatures and modeled equivalents. This suggests that the prescribed seasonal cycle as inferred from Scharringa (1976) overestimates the real seasonal cycle in the standard deviation.

_{T}The component *J*_{ap} is very small. Most points are very close to zero, although, especially at the end of 2003, values of 0.8 are obtained. It is apparent that the prior estimates only play a small role in determining the final estimates of the parameter values. Their prime role is to serve as a starting point for the iteration in algorithm 1.

### b. Control parameter values

Figure 4 shows posterior values of the soil heat conductivity, denoted by *λ*_{post} (Fig. 4a), the volumetric heat capacity, denoted by *c _{p}*

_{,post}(Fig. 4b), the skin conductivity, denoted by Λ

_{post}(Fig. 4c), and the transmissivity of the vegetation, denoted by

*τ*

_{post}(Fig. 4d). Also, in Figs. 4a and 4b, values for

*λ*and

*c*are given that are calculated using, respectively, Eqs. (8) or (9) and the average of the daily averaged soil moisture contents at 0.03 and 0.08 m. The temporal evolution of the soil moisture at 0.08 m is not given, because its calibration is uncertain. Its temporal evolution can be derived from the volumetric heat capacity as calculated using Eq. (9), because this quantity is linearly related to the soil moisture content.

_{p}It appears that from DOY 2003 50 to DOY 2003 230, *λ*_{post} scatters around 0.9 W m^{−1} K^{−1}. This value is comparable to the value derived from Eq. (8), although the latter shows somewhat less interdiurnal variation. At about DOY 2003 230, *λ*_{post} decreases toward a value of about 0.6 W m^{−1} K^{−1}, around which value it scatters from DOY 2003 230 onward. In contrast, the value for *λ* derived from Eq. (8) reaches a minimum value of about 0.8 W m^{−1} K^{−1} around DOY 2003 230, and it increases again to a value of about 1.0 W m^{−1} K^{−1} at DOY 2003 350.

The volumetric heat capacity follows a pattern similar to that of the soil heat conductivity. From DOY 2003 50 to DOY 2003 230, *c _{p}*

_{,post}is relatively close to the value for

*c*derived from Eq. (9). Both decrease from a value of about 3.3 MJ m

_{p}^{−3}K

^{−3}at DOY 2003 50 to a value of about 1.7 MJ m

^{−3}K

^{−3}at DOY 2003 230. From DOY 2003 210 onward,

*c*

_{p}_{,post}differs considerably from the value for

*c*derived using Eq. (9). From DOY 2003 210 to DOY 2003 230,

_{p}*c*

_{p}_{,post}decreases further to a value of about 1.5 MJ m

^{−3}K

^{−3}, after which it increases sharply toward a value of about 2.7 MJ m

^{−3}K

^{−3}from DOY 2003 260 to DOY 2003 350. In contrast, the value for

*c*derived from Eq. (9) reaches a minimum value of about 1.7 MJ m

_{p}^{−3}K

^{−3}around DOY 2003 230, after which it increases toward a value of about 2.5 MJ m

^{−3}K

^{−3}at DOY 2003 350.

Similar to *λ*_{post} and *c _{p}*

_{,post}, the temporal variation of Λ

_{post}shows a sharp change at DOY 2003 230. Before DOY 2003 230, Λ

_{post}scatters around a value of about 4.0 W m

^{−2}K

^{−1}, whereas from DOY 2003 230 onward, Λ

_{post}scatters around a value of about 3.0 W m

^{−2}K

^{−1}. For

*τ*

_{post}, there is no clear regime change at DOY 2003 230. Throughout 2003 it decreases steadily from a value of about 0.03 at DOY 2003 50 to a value of about 0.005 at the end of 2003.

Parameter *τ* is only a function of the vegetation cover and the characteristics of the incoming shortwave radiation. Parameters *λ* and *c _{p}* are mainly soil characteristics, and Λ is influenced by both the characteristics of the (top) soil and the temperature distribution within the vegetation cover. Changes in the soil characteristics are thus the most probable explanation for the regime change that is observed for

*λ*

_{post},

*c*

_{p}_{,post}, and Λ

_{post}at DOY 2003 230. A possible explanation for the regime change is the well-known property of clay soil to crack in very dry conditions, such as those experienced during the summer of 2003 (Wallender et al. 2006). In very dry conditions, cracking leads to the appearance of fissures in the clay soil that are filled with air that has both a very low conductivity and a very low volumetric heat capacity, hence leading to the very low posterior values of the soil heat conductivity, the volumetric heat capacity, and the skin conductivity from DOY 2003 230 to DOY 2003 250. From DOY 2003 250 onward, the soil starts to wet again because of precipitation [note the increasing value of soil heat conductivity as derived from Eq. (8)]. In these conditions, the fissures are filled with water that has a very large volumetric heat capacity but a conductivity that is lower than the surrounding soil, hence leading to the relatively large values of the volumetric heat capacity and the relatively low values of the soil heat conductivity and the skin conductivity from DOY 2003 250 onward. Note, however, hat the current method assumes a homogeneous soil in which the vertical heat transfer can be described using the diffusion equation as given in Eq. (1a). It is obvious that this assumption is violated when the clay soil cracks, and another much more complex formulation should be applied to calculate the heat transfer within the soil column (Wallender et al. 2006).

### c. Surface soil heat flux

In Fig. 5, 10-min averages of *G*_{0} estimated by the current approach are compared with 10-min averages of *G*_{0} estimated using the lambda approach as described by De Bruin and Holtslag (1982). Here, we have adapted the original approach described by De Bruin and Holtslag (1982) to account for the Philip (1961) correction [Eq. (7)] on the soil heat flux estimates. It is unfortunate that, unlike the new method, the lambda method does not yield values for *λ* that are representative for the soil column between the surface and 0.12. To account for the Philip correction, a constant “representative” value for *λ* of 0.9 W m^{−1} K^{−1} is adopted. Also shown in Fig. 5 is the 1:1 line. It appears that in conditions in which *G*_{0} derived using the new approach is between −40 and −20 W m^{−2} (directed upward), the new approach gives lower values of *G*_{0} than does the adapted lambda approach: *G*_{0} as estimated by the new approach is −40 W m^{−2} and the adapted lambda approach gives values of about −30 W m^{−2}. When *G*_{0} is between −20 and 30 W m^{−2} (directed downward), the new approach gives estimates of the surface soil flux that are similar to the estimates: in these situations the points in Fig. 5 scatter around the 1:1 line. When *G*_{0} is larger than 30 W m^{−2} (directed downward), the new approach gives lower estimates of *G*_{0} than does the adapted lambda method: when *G*_{0} as estimated by the new approach is 50 W m^{−2}, the estimate obtained by the adapted lambda approach is about 70 W m^{−2}.

It appears that the underprediction of *G*_{0} by the new approach with respect to the lambda approach can be mitigated when only daytime values of measured ground heat flux and soil temperature profile obtained are assimilated into the model. In a similar way, the underprediction of the ground heat fluxes by the new approach with respect to the lambda approach can be mitigated when only nocturnal values of measured ground heat flux and soil temperature profile are assimilated into the model. Furthermore, using only daytime measurements leads to somewhat larger values of the skin conductivity than using nocturnal profiles only. This suggests that especially Λ shows a diurnal cycle that favors large surface soil heat fluxes during daytime and decreases surface soil heat fluxes in nocturnal conditions.

The larger nighttime (negative) values of the surface soil heat flux as calculated using the novel approach result in a lower imbalance of the surface energy budget, as can be seen in Fig. 6 in which we plot, for both the adapted lambda approach (Fig. 6a) and the new approach (Fig. 6b), *G*_{0} against the measurements of (*R _{N}* − LE −

*H*). It appears that for the lambda approach the estimated

*G*

_{0}is not equal to (

*R*− LE −

_{N}*H*). When (

*R*− LE −

_{N}*H*) is negative, the difference amounts to a maximum value of 25 W m

^{−2}, that is, about 50% of the value of (

*R*− LE −

_{N}*H*) at this point. For positive values of (

*R*− LE −

_{N}*H*), the difference between the estimated surface soil heat flux and the (

*R*− LE −

_{N}*H*) is about 100 W m

^{−2}. The imbalance of the surface energy budget gets somewhat smaller when the novel approach to estimate the surface soil heat flux is used. When (

*R*− LE −

_{N}*H*) is positive, however, there is still a large imbalance between the measured (

*R*− LE −

_{N}*H*) and the determined surface soil heat flux.

## 6. Summary and conclusions

Our aim was to present and evaluate the performance of a new method for estimating the surface soil heat flux at the meteorological measurement facility at Cabauw in the Netherlands. The proposed method calculates the soil surface heat flux using an optimized land surface model that describes the transport of heat between the vegetation and the soil surface, the transport of heat through the interface between the soil and air/vegetation continuum, and the transport of heat within the soil column. The model is optimized using a variational data assimilation framework in which, for each day, optimal values for the parameters are calculated as a function of misfits between 48 uncertain measurements and simulated equivalents, and 4 uncertain prior parameter value estimates. Boundary conditions for the model are the (radiative) temperature of the vegetation, the temperature measured at 12 cm below the surface, and the incoming shortwave radiation at the surface.

For the ground heat fluxes observational uncertainty is derived from the field measurements themselves, whereas for the soil temperatures the observational uncertainty is assigned according to values for the lateral variations in the soil temperature found in literature (Scharringa 1976). After insertion of the resulting values into the data assimilation setup, it appears that the variational data assimilation setup yields, on 101 days, values of the cost function that are between 33 and 67, the 90% confidence interval of a chi-square distribution with 48 degrees of freedom, which is the distribution that the cost function theoretically should follow assuming independent observational errors.

As important evaluation products, the proposed approach yields values for the following model parameters: the soil conductivity, the volumetric heat capacity of the soil, the skin conductivity, and the transmissivity of the vegetation. Up to DOY 2003 230, values for the soil conductivity and the volumetric heat capacity follow theoretical values as derived from literature (Peters-Lidard et al. 1998). From DOY 2003 230, however, deviations from the values derived from Peters-Lidard et al. (1998) start to occur, possibly due to the cracking of the clay soil in Cabauw.

In general, the proposed method gives estimates of the soil heat flux that are in good agreement with the currently operational lambda approach, provided that the lambda approach is adapted to include a term to account for a correction term that translates the heat flux over a heat flux plate to a soil heat flux. However, unlike the proposed approach, which allows the incorporation of instrumental corrections that depend on the thermal parameters of the soil, the correction cannot be extracted by the method itself but has to be quantified separately. It is unfortunate that the proposed method gives very large estimates of the surface soil heat flux in conditions in which the soil heat flux is larger than about 50 W m^{−2} and gives too-small estimates when the soil heat flux is smaller than about −20 W m^{−2}. It appears that these discrepancies are caused by the lack of intradiurnal variations in the skin conductivity in the proposed method. Therefore, it is recommended that the proposed approach be further refined so that the skin conductivity is allowed to vary within the day.

The conclusion is that the proposed approach is a good candidate to replace current operational algorithms to infer the surface soil heat flux from measured soil temperature and soil heat flux profiles. Its main advantages are that (1) the prescribed error statistics can be evaluated objectively by studying the distribution and characteristics of the cost function, (2) the model can incorporate different measurements that have different error characteristics, and (3) the model can include in a consistent way measurements that need corrections that depend on the modeled soil parameters themselves, as is the case for the soil heat flux plates. However, the current approach should be extended so that inhomogeneities of the soil as well as intradiurnal variations in the parameters can be accommodated.

## Acknowledgments

This study is performed within the framework of the Climate Scenario 2 (CS2) project and the Mitigation Experiment 2 (ME2) project, which are both part of the BSIK Climate Changes for Spatial Planning (CcSP) research program. The authors thank two anonymous reviewers for their useful comments.

## REFERENCES

Beljaars, A. C. M., and F. C. Bosveld, 1997: Cabauw data for the validation of land surface parameterization schemes.

,*J. Climate***10****,**1172–1193.Boni, G., D. Enthekhabi, and F. Castelli, 2001: Land data assimilation with satellite measurements for the estimation of surface energy balance components and surface control on evaporation.

,*Water Resour. Res.***37****,**1713–1722.Bosveld, F. C., and W. Bouten, 2001: Evaluation of transpiration models with observations over a Douglas-fir forest.

,*Agric. For. Meteor.***188****,**247–264.Daley, R., 1991:

*Atmospheric Data Analysis*. Cambridge University Press, 457 pp.De Bruin, H. A. R., and A. A. M. Holtslag, 1982: A simple parameterization of the surface fluxes of sensible and latent heat during daytime compared with the Penman–Monteith concept.

,*J. Appl. Meteor.***21****,**1610–1621.Duynkerke, P. G., 1991: Radiation fog: A comparison of model simulations with detailed observations.

,*Mon. Wea. Rev.***119****,**324–341.Duynkerke, P. G., 1992: The roughness length for heat and other vegetation parameters for a surface of short grass.

,*J. Appl. Meteor.***31****,**579–586.Fuchs, M., and C. B. Tanner, 1968: Calibration and field test of soil heat flux plates.

,*Soil Sci. Soc. Amer. Proc.***32****,**326–328.Garratt, J. R., and M. Segal, 1988: On the contribution of atmospheric moisture to dew formation.

,*Bound.-Layer Meteor.***45****,**209–236.Heusinkveld, B. G., A. F. G. Jacobs, A. A. M. Holtslag, and S. M. Berkowicz, 2004: Surface energy balance closure in an arid region: Role of soil heat flux.

,*Agric. For. Meteor.***122****,**21–37.Jager, C. J., T. C. Nakken, and C. L. Palland, 1976:

*Bodemkundig Onderzoek van twee graslandpercelen Nabij Cabauw (Soil Research of Two Grassland Plots near Cabauw)*. NV Heidemaatschappij Beheer, 9 pp.Johansen, O., 1975: Thermal conductivity of soils, Ph.D. thesis, University of Trondheim, 236 pp.

Lopez, P., A. Benedetti, P. Bauer, and M. Janiskova, 2006: Experimental 2D-VAR assimilation of ARM cloud and precipitation observations.

,*Quart. J. Roy. Meteor. Soc.***132****,**1325–1347.Margulis, S. A., and D. Entekhabi, 2001: A coupled land surface–boundary layer model and its adjoint.

,*J. Hydrometeor.***2****,**274–296.Margulis, S. A., and D. Entekhabi, 2004: Boundary-layer entrainment estimation through assimilation of radiosonde and micrometeorological data into a mixed-layer model.

,*Bound.-Layer Meteor.***110****,**405–433.Massman, W. J., 1992: Correcting errors associated with soil heat flux measurements and estimating soil thermal properties from soil temperature and heat flux plate data.

,*Agric. For. Meteor.***59****,**249–266.Mayocchi, C. L., and K. L. Bristow, 1995: Soil surface heat flux: Some general questions and comments on measurements.

,*Agric. For. Meteor.***75****,**43–50.Menke, W., 1989:

*Geophysical Data Analysis: Discrete Inverse Theory*. International Geophysics Series, Vol. 45, Academic Press, 289 pp.Passerat de Silans, A., B. A. Monteny, and J. P. Lhomme, 1997: The correction of soil heat flux measurements to derive an accurate surface energy balance by the Bowen ratio method.

,*J. Hydrol.***188–189****,**453–465.Peters-Lidard, C. D., E. Blackburn, X. Liang, and E. F. Wood, 1998: The effect of soil thermal conductivity parameterization on surface energy fluxes and temperatures.

,*J. Atmos. Sci.***55****,**1209–1224.Philip, J. R., 1961: Theory of heat flux meters.

,*J. Geophys. Res.***66****,**571–579.Portman, D. J., 1958: Conductivity and length relationships in heat-flow transducer performance.

,*Trans. Amer. Geophys. Union***39****,**1089–1094.Press, W. H., B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, 1986:

*Numerical Recipes: The Art of Scientific Computing*. Cambridge University Press, 818 pp.Scharringa, M., 1976: On the representativeness of soil temperature measurements.

,*J. Agric. Meteor.***16****,**263–276.Sirkes, Z., and E. Tziperman, 1997: Finite difference of adjoint or adjoint of finite difference.

,*Mon. Wea. Rev.***125****,**3373–3378.Stull, R. B., 1988:

*An Introduction to Boundary Layer Meteorology*. Kluwer Academic, 666 pp.Van de Wiel, B. J. H., R. J. Ronda, A. F. Moene, H. A. R. de Bruin, and A. A. M. Holtslag, 2002: Intermittent turbulence and oscillations in the stable boundary layer. Part I: A bulk model.

,*J. Atmos. Sci.***59****,**2567–2581.Van Ulden, A. P., and J. Wieringa, 1996: Atmospheric boundary layer research at Cabauw.

,*Bound.-Layer Meteor.***78****,**39–69.Van Wijk, W. R., and D. A. de Vries, 1963: Periodic temperature variations in a homogeneous soil.

*Physics in Plant Environment,*W. R. van Wijk, Ed., North Holland, 103–143.Wallender, W. W., K. K. Tanji, J. R. Gilley, R. W. Hill, J. M. Lord, C. V. Moore, R. R. Robinson, and E. C. Stegman, 2006: Water flow and salt transport in cracking clay soils of the Imperial Valley, California.

,*Irrig. Drain. Syst.***20****,**361–387.Wösten, J. H. M., G. J. Veerman, and J. Stolte, 1994: Waterretentie- en doorlatendheidskarakteristieken van boven- en ondergronden in Nederland; de Staringreeks (Water retention and permeability characteristics of surface and subsoils in the Netherlands; the Staringreeks). Tech. Doc. 19, Winand Staring Centre for Integrated Land, Soil, and Water Research, Wageningen, Netherlands, 66 pp.

## APPENDIX

### The Adjoint of Finite-Difference Approach

*z*is the vertical grid spacing and

*M*=

*z*/Δ

_{b}*z*. For time integration of the values at discrete points in the vertical direction, the semi-implicit Crank–Nicolson technique is used, resulting in the following equation for the model given by Eqs. (1a)–(1d) (Press et al. 1986):

*t*=

_{i}*t*

_{0}+

*i*Δ

*t*, 𝗔 and 𝗕 are

*M*×

*M*matrices controlling the propagation of the soil temperature, Δ

*t*is the temporal grid spacing,

*N*is the number of time steps,

**T**(

*t*) is an

_{i}*M*× 1 vector with elements

**T**(

*t*){

_{i}*k*} =

*T*(

*z*,

_{k}*t*), and

_{i}**V**(

*t*) is an

_{i}*M*× 1 vector that contains information on boundary conditions. Matrices 𝗔 and 𝗕 are tridiagonal, implying that their elements are zero valued everywhere, except for the values on the main diagonal, the first diagonal below this, and the first diagonal above this:

*σ*=

*λ*Δ

*t*/[

*c*(Δ

_{p}*z*)

^{2}]. Elements of the vector

**V**(

*t*) are given by

_{i}*L*is the discrete formulation of the Lagrange function and

_{d}**(**

*μ**t*) is an

_{i}*M*× 1 Lagrange multiplier vector.

Comparison of posterior values of the cost function vs prior values of the cost function for days on which all of the available data are available (dots). Also shown are the 1:1 line (dashed line), and the 0.05 (dotted line) and 0.95 quantile (solid line) of a chi-square distribution with 48 degrees of freedom.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

Comparison of posterior values of the cost function vs prior values of the cost function for days on which all of the available data are available (dots). Also shown are the 1:1 line (dashed line), and the 0.05 (dotted line) and 0.95 quantile (solid line) of a chi-square distribution with 48 degrees of freedom.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

Comparison of posterior values of the cost function vs prior values of the cost function for days on which all of the available data are available (dots). Also shown are the 1:1 line (dashed line), and the 0.05 (dotted line) and 0.95 quantile (solid line) of a chi-square distribution with 48 degrees of freedom.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

Daily values of (a) the component of the posterior cost function that penalizes deviations from the prior values, (b) the component of the cost function that penalizes deviations from the observed soil temperatures, and (c) the component of the cost function that penalizes deviations from the observed soil heat fluxes for the selected days.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

Daily values of (a) the component of the posterior cost function that penalizes deviations from the prior values, (b) the component of the cost function that penalizes deviations from the observed soil temperatures, and (c) the component of the cost function that penalizes deviations from the observed soil heat fluxes for the selected days.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

Daily values of (a) the component of the posterior cost function that penalizes deviations from the prior values, (b) the component of the cost function that penalizes deviations from the observed soil temperatures, and (c) the component of the cost function that penalizes deviations from the observed soil heat fluxes for the selected days.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

(a) Posterior values of the soil heat conductivity (open circles), and values of the soil heat conductivity calculated according to Eq. (8) (solid squares); (b) posterior values of the volumetric heat capacity of the soil (open circles), and values of the volumetric heat capacity of the soil calculated according to Eq. (9) (solid squares); (c) posterior values of the skin conductivity; (d) posterior values of the transmissivity of the vegetation for the selected days.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

(a) Posterior values of the soil heat conductivity (open circles), and values of the soil heat conductivity calculated according to Eq. (8) (solid squares); (b) posterior values of the volumetric heat capacity of the soil (open circles), and values of the volumetric heat capacity of the soil calculated according to Eq. (9) (solid squares); (c) posterior values of the skin conductivity; (d) posterior values of the transmissivity of the vegetation for the selected days.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

(a) Posterior values of the soil heat conductivity (open circles), and values of the soil heat conductivity calculated according to Eq. (8) (solid squares); (b) posterior values of the volumetric heat capacity of the soil (open circles), and values of the volumetric heat capacity of the soil calculated according to Eq. (9) (solid squares); (c) posterior values of the skin conductivity; (d) posterior values of the transmissivity of the vegetation for the selected days.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

Comparison of the surface soil heat as calculated using the new approach with estimates of the surface soil heat flux using the adapted lambda approach. Also shown is the 1:1 line.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

Comparison of the surface soil heat as calculated using the new approach with estimates of the surface soil heat flux using the adapted lambda approach. Also shown is the 1:1 line.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

Comparison of the surface soil heat as calculated using the new approach with estimates of the surface soil heat flux using the adapted lambda approach. Also shown is the 1:1 line.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

The surface soil heat flux vs the net radiation minus the atmospheric transport (positive) upward for (a) the new approach and (b) the adapted lambda approach. Also shown is the 1:1 line.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

The surface soil heat flux vs the net radiation minus the atmospheric transport (positive) upward for (a) the new approach and (b) the adapted lambda approach. Also shown is the 1:1 line.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1

The surface soil heat flux vs the net radiation minus the atmospheric transport (positive) upward for (a) the new approach and (b) the adapted lambda approach. Also shown is the 1:1 line.

Citation: Journal of Applied Meteorology and Climatology 48, 3; 10.1175/2008JAMC1930.1