a. Skjern catchment

The Skjern River catchment is located in the western part of the Danish peninsula and covers 2500 km². The catchment serves as the main research site of the Danish hydrological observatory (HOBE) project and has been studied intensively for almost a decade (Jensen and Illangasekare 2011). With a maritime climate, mean annual precipitation amounts to 990 mm and mean reference evapotranspiration accounts for 575 mm of the annual water balance. Soils are predominately sandy and originate from glacial outwash plains with intertwined sections of clay and till. The topography slopes gently from the coast on the western side to 125 m MSL at the eastern side of the catchment. Figure 1 depicts the land-use map at the Skjern catchment, where agriculture is the dominant class followed by forest and heath. The map does not indicate the extensive monitoring network, which consists of state-of-the-art instrumentations for continuous measurements of eddy covariance towers (Ringgaard et al. 2011), a wireless soil moisture network (Bircher et al. 2012), and all other relevant hydrological states and fluxes throughout the catchment.

View Full Size

The land-use map of the Skjern River catchment at 500 m resolution, including one discharge station.

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

b. Hydrological model

The hydrological model of the Skjern River catchment is set up in the MIKE Système Hydrologique Européen (MIKE SHE) modeling system (Abbott et al. 1986) and comprises fully coupled modules of 3D saturated flow (finite difference), 1D unsaturated flow (Richards’ equation), river routing, and overland flow. The traditional MIKE SHE setup is extended by an additional coupling of a land surface model (SW-ET) that solves the diurnal energy balance (Shuttleworth and Wallace 1985). The land surface model provides a more physical description of the processes at the land surface–atmosphere interface than the more conceptual model based on the potential evapotranspiration approach (Kristensen and Jensen 1975). This extension requires hourly climate forcing and time steps in order to reflect the complex diurnal variability correctly (Overgaard 2005). In addition, a vegetation parameterization scheme is derived from the Moderate Resolution Imaging Spectroradiometer (MODIS), and details are presented in the following section. The Skjern model is set up for a period from 2002 to 2010 at 500 m resolution, and the spatial sensitivity analysis focuses exclusively on the last 3 years. At 500 m resolution, relevant small-scale processes such as hillslope–riparian–stream interactions are not reflected accordingly. However, reliable parameters and forcing datasets as well as remote sensing observations are not available at finer scales, which limit the model’s predictability below 500 m resolution.

The model is calibrated using a multiconstraint framework that is briefly introduced in Table 1. The calibration is based on five independent observational datasets: discharge, hydraulic head, actual evapotranspiration, soil moisture, and satellite-derived LST. In total, 11 model parameters are included in the optimization. They are related to the land surface, saturated zone, and unsaturated zone modules. The parameter values are estimated using a global optimization scheme based on covariance matrix adaptation estimation strategies (CMA-ES) as introduced by Hansen and Ostermeier (2001) and later applied by Bayer and Finkel (2007) to calibrate a groundwater model against concentration. The Parameter Estimation (PEST; Doherty 2016) has a built-in CMA-ES module, which is used for this calibration task. The calibration reduces errors in 11 objective functions, and the final values are stated in Table 1. In general, the root-mean-square error (RMSE) and bias are selected as metrics for the five different observational variables. However, a few exceptions are made for soil moisture and LST where the slope of the linear regression line or the correlation coefficient are considered as well. Current calibration frameworks of catchment models target the fitting of time series, and hence objective functions of discharge, hydraulic head, and soil moisture can effectively be reduced. Although remotely sensed spatial patterns of LST are included in the calibration, the spatial performance shows limited improvements, because the calibration design is more efficient at generating temporal, as opposed to spatial, coherence. Further details of the calibration are not covered by this study because the focus lies on the successive spatial sensitivity analysis. Hence, the purpose of the calibration is to grant a baseline scenario that is consistent with observations as shown by Table 1.

Table 1.

Overview of the calibration results. The calibration scheme incorporates five observational datasets and 11 objective functions. The objective functions are defined as RMSE, slope of the linear regression line, bias, and correlation coefficient. The values are based on averages; for example, the RMSE value for discharge represents the average of all eight stations. Fields marked with an em dash are not considered as objective functions in the calibration.

View Table

c. Remote sensing data

The MODIS sensor flies on board the polar-orbiting platforms Terra and Aqua and has four overpasses per day at midlatitudes. For the vegetation parameterization of the distributed Skjern model, we incorporated different MODIS products obtained from the NASA Land Processes Distributed Active Archive Center (LP DAAC). Table 2 presents a brief description of the remote sensing products used in this study. During some periods, low sun zenith angles and snow cover cause low coverage of the MODIS leaf area index (LAI) product (MCD15A2) data marked with best-quality flags. Therefore, we build a site-specific empirical relationship between the LAI and the normalized difference vegetation index (NDVI; Rouse et al. 1973). The NDVI is an index sensitive to biomass and chlorophyll content of the vegetation and has been used extensively to monitor different properties of the vegetation by establishing empirical relationships between the index and the variable of interest. For example, Boegh et al. (2004) derived LAI data for Denmark from NDVI using remote sensing data with an exponential function and in situ LAI data for Denmark. The Nadir Bidirectional Reflectance Distribution Function (BRDF)-Adjusted Reflectance (NBAR) from the MCD43B4 product is used to calculate the NDVI by

View Expanded

where B_x indicates the MODIS band number. In the following, TIMESAT, version 3.2 (Jönsson and Eklundh 2004, 2002), is utilized to smooth the dataset and to ensure consistency in the data. TIMESAT fits a function to the data based on least squares criteria. The Savitzky–Golay filter is selected as it preserves the maximum and minimum values of the original dataset (Savitzky and Golay 1964).

Table 2.

Description of the applied MODIS products.

View Table

The empirical relationship to obtain the fitting equation is based on the input LAI data used in the Danish National Water Resources model (Stisen et al. 2012). In the Skjern catchment, the main land use is agriculture, and thus the fitting equation is based on mean LAI and NDVI values at collocated agricultural cells. For this study we applied a similar approach as in Boegh et al. (2004) by adjusting the coefficients to match data:

View Expanded

Spatially distributed root depth (RD) is derived using a similar approach to Andersen et al. (2002), where RD is calculated by

View Expanded

where is RD in pixel i, is the LAI in that cell, and and indicate the maximum values at i. This equation is modified and LAI is substituted by NDVI,

View Expanded

and used for forested land-cover types, poorly vegetated areas, and urban areas. It should be noted that RD is an effective parameter with limited physical relevancy. For instance, Eq. (4) equips forest cells with a temporal-varying RD that is contrary to our physical understanding, but it compensates for mixed land-use cells and scaling issues. RD of croplands and grasslands, denoted as , is estimated by implementing some modifications to Eq. (4). The varies depending on the soil type, and therefore an empirical relation between the plant available water (PAW) and is established for the different soil types. PAW reflects the difference in soil water content between the field capacity and wilting point. This equation is then included as a substitute of . To allow RD to reach zero, the second term in Eq. (4) is normalized by including in the equation:

View Expanded

Following Stisen et al. (2011), the vegetation height (VH) is calculated by assuming a simple linear regression,

View Expanded

and forest cells are set to a year-round constant VH of 7 m. The shortwave broadband albedo is estimated every 8 days at 1-km spatial resolution from the individual bands of 8-day surface reflectance product (MOD09A1) containing atmospherically corrected narrowband MODIS reflectance, using the algorithm by Liang et al. (2003).

d. Model scenarios

The highly parameterized Skjern model is based on a distinct degree of spatial detail, and Table 3 lists all parameters and forcing data that have a spatial dimension. In total, 43 rain gauges within and around the catchment are utilized to generate daily interpolated precipitation fields. The remaining climate variables are available from up to 16 climate stations with temporal resolutions of 1 and 3 h. The soil hydraulic parameters for the van Genuchten model, which describe the soil water retention curve of unsaturated soils, are given for 16 soil types in the catchment, and the spatial distribution is derived from a national soil map at 500 m resolution. The geological model of the top 3 m consists of three units (sand, clay, and till) that have individual values for hydraulic conductivity, specific storage, and specific yield. Deeper layers of the geological model are not addressed in this study because of their limited coupling to surface processes. LAI, root depth, albedo, and vegetation height are all derived from remote sensing data as described previously. The root distribution is derived from national bulk density maps of the top soil layer. Values for the remaining vegetation parameters (interception, stomata resistance, leaf width, and light extinction) are given for the three major classes (agriculture, forest, and grassland/heath) of the land-use map (Fig. 1) following literature values found in Larsen et al. (2016a).

Table 3.

Overview of the 22 scenarios and the model forcings and parameters that have a spatial dimension. Further, the scenarios are grouped with respect to the model module they affect.

View Table

The baseline model considers the full integration of all spatially distributed parameters and forcing data mentioned above. Table 3 describes the 22 scenarios that are applied in this study, each one including a deterioration of spatial detail of the baseline model by perturbing one potential driver at a time. The premise is that if a perturbation in spatial input alters the simulated spatial patterns of LST and ET significantly, the model is deemed sensitive to that specific driver. Vice versa, little change in the simulated patterns of a scenario indicates limited sensitivity. The MODIS-derived vegetation parameters have the highest degree of spatial heterogeneity; therefore, the spatial variability is reduced in two steps: 1) by a moving average 10-km smoothing filter and 2) by spatially averaging to a constant value for the entire catchment. Scenario 9, which addresses the sensitivity of groundwater coupling, represents a special case. The spatial variability of potential drivers remains unchanged, but the conceptual model setup is simplified by disabling the saturated zone and setting a constant groundwater head boundary below the root depth to decouple the land surface from potential groundwater influences.

Interactions and feedbacks between potential drivers are disregarded because the applied sensitivity analysis is based on one-at-a-time perturbations. The underlying assumption that there exist no interactions between potential drivers and the fact that the changes in model response are always related to the same baseline may limit the conclusion in terms of identifying global sensitivities. However, we are not aware of any formal approach of how to assess the global sensitivity of spatial patterns of input parameters; instead, it is typically applied for specific parameter values (Razavi and Gupta 2015; Sobol’ and Kucherenko 2009). In this way, this study treats the term sensitivity as practical rather than mathematically sound, with the aim to guide model builders in setting up spatially distributed models.

The baseline model and the scenarios perform reasonably well with Nash–Sutcliffe efficiency values ranging from 0.81 to 0.87 for station 200082 (Fig. 1). This underlines that all scenarios are capable of simulating the temporal discharge dynamics adequately while maintaining a correct water balance as well. Furthermore, this stresses the limited sensitivity of discharge to the spatial patterns of catchment inherent processes (Koch et al. 2016a; Pokhrel and Gupta 2011). Scenario 9 is excluded from the discharge analysis because reasonable discharge cannot be simulated, as most discharge is generated through drainage flow and base flow at the Skjern catchment.

a. Variogram

Variogram analysis is a frequently applied geostatistical tool that analyses the spatial autocorrelation structure of spatial data as a function of distance (lag; Deutsch and Journel 1998). Several studies have incorporated variograms to assess spatial patterns of soil moisture at catchment scale (Rosenbaum et al. 2012; Western et al. 1998). Furthermore, Koch et al. (2016a) and Korres et al. (2015) incorporate a variogram analysis of observed and simulated soil moisture patterns to assess the capability of integrated catchment models to represent the observed spatial patterns. Besides these previous efforts, the variogram analysis is typically not applied on spatial patterns of other land surface variables, such as ET or LST.

The omnidirectional empirical semivariance γ of variable Z at lag h is calculated as

View Expanded

where n(h) is the total number of data pairs at a given lag h, represents the value of Z at location i, and is the value separated by lag h from i. In the Skjern catchment, γ is calculated at eight irregularly spaced lags from 1 up to 25 km, which is about half of the maximum possible lag distance in the catchment. To quantify agreement between baseline [γ(h)_ref] and scenario [γ(h)_scen] semivariances, the root-mean-square error (RMSE_Vario) is computed at the eight given lags:

View Expanded

In summary, the variogram metric focuses on the spatial autocorrelation as a function of distance and states a global measure of spatial performance that is not constrained by local agreement. Furthermore, the metric is bias insensitive because it only considers the relative deviation between values.

b. Empirical orthogonal functions

The EOF analysis is often applied to gain insight into the spatiotemporal variability of hydrological states and fluxes, such as soil moisture (Graf et al. 2014; Perry and Niemann 2007) or LST (Mascaro et al. 2015). The main feature of the EOF analysis is its capability to decompose a large spatiotemporal dataset of a single hydrological variable into two insightful components: 1) a set of orthogonal EOF maps that are invariant in time and 2) a set of loadings that quantify the importance of each EOF over time. The first EOF captures the highest portion of variance and the following EOFs subsequently add to the amount of explained variance. A more detailed mathematical description of the EOF methodology is given by Perry and Niemann (2008). While most studies have applied the EOF analysis on either observational or modeled data, more recent studies successfully applied it as a tool to spatially validate distributed hydrological models at catchment scale (Fang et al. 2015; Koch et al. 2015). Koch et al. (2015) brought forward the novel concept of performing a joint EOF analysis on an integral data matrix that contains both reference and modeled data. This joint analysis guarantees that the obtained EOF maps honor the spatiotemporal variability of both datasets and allows that the deviation between the loadings at a certain time step can be utilized as an indicator of spatial similarity. Following this approach, the associated loadings of highly similar maps at a certain time step will show a minimal deviation. Vice versa, a large loading deviation can be attested to pattern dissimilarity. The loading deviation must be weighted according to the variance contribution of the corresponding EOF to ensure a reliable pattern skill score. Thus, the EOF-based similarity score S_EOF between a reference and a scenario map at time x can be formulated as

View Expanded

where is the variance contribution of the ith EOF, is the scenario loading, and is the reference loading at time x. Prior to the EOF analysis, the mean is removed from each map; thus, the methodology is based on the spatial anomalies, which makes it bias insensitive.

c. Fractions skill score

The FSS was initially developed by Roberts and Lean (2008) to obtain a scale-dependent quantitative measure of spatial performance for various competing precipitation forecasts. This methodology is based on fractions that are calculated at each cell and relate to occurrences of values exceeding a certain threshold at a given window size (scale). In most FSS applications, the thresholds represent a few selected percentiles of interest, which has the purpose to eliminate any impact of a potential bias. As an example, Roberts (2008) analyzed precipitation forecasts at different extreme percentiles (75th, 90th, 95th, and 99th). To our knowledge, FSS has exclusively been applied on precipitation forecasts (e.g., Gilleland et al. 2009; Mittermaier et al. 2013; Wolff et al. 2014), but the methodology can easily be adapted to any continuous spatial data, such as ET or LST. In summary, FSS can be described via the following steps: 1) for each threshold of interest, truncate the reference (ref) and the scenario (scen) spatial patterns into binary maps; 2) for each cell, compute fractions within a given scale n based on the number of cells that exceed the threshold and lie within a window of size n × n; and 3) calculate the mean-square error (MSE) between the reference and scenario fractions and normalize it with a worst-case MSE that reflects the condition with zero agreement between ref and scen (MSE_wc). The MSE is based on all cells N_xy that lie within the modeling domain with dimension of N_x and N_y. For a certain threshold, FSS at scale n is given by

View Expanded

where

View Expanded

and

View Expanded

FSS ranges from zero to one, where one indicates a perfect agreement between reference and scenario patterns and zero reflects the worst possible performance. The methodology is bias insensitive because of the percentile truncation, which is a favorable feature for our application. Figure 2 illustrates the essential FSS steps for a summer ET pattern at the Skjern catchment. The 80th percentile is chosen to address the highest 20% of cells, and fractions at two different scales (5 and 10 km) underline the flexibility of this methodology. Adjusting the scale allows the user to tolerate placement errors, and thus uncertainties of the pattern, in dependence of the threshold. For instance, it may be desirable to accept higher uncertainties in the top 1% than in the top 25% of ET cells.

View Full Size

(top left) The methodology of the FSS is illustrated in a map of simulated ET (mm day⁻¹) for a single day in June 2008. First, a percentile of interest is chosen and the continuous map is classified into a binary map by truncation at the given percentile. (top right) For this example, the 80th percentile is selected to display the top 20% of ET cells. The binary map is smoothed using (bottom left) 5 and (bottom right) 10 km, and the fractions are given for each cell.

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

d. Connectivity

In hydrogeology, connectivity is a common measure to quantify aquifer heterogeneity (dell’Arciprete et al. 2012; Koch et al. 2014; Renard et al. 2011), where the degree of connectivity has direct physical implications on groundwater flow and solute transport. Only a few studies have implemented the concept of connectivity to describe spatial patterns of land surface variables. In this context, Western et al. (2001) and Grayson et al. (2002) analyzed connectivity of soil moisture patterns at the small Tarrawarra research catchment in Australia and linked it to discharge dynamics. These studies underline the physical relevance of connectivity as a characteristic of spatial patterns of hydrological states and fluxes at the land surface. Recently, Koch et al. (2016b) implemented a connectivity analysis in a spatial validation study of continental-scale land surface models using remotely sensed LST. Connectivity provided a reliable measure of heterogeneity and the general structure of the spatial LST patterns, which helped to identify specific shortcomings of the spatial performance. Renard and Allard (2013) suggested the following steps to conduct a connectivity analysis of a continuous variable: 1) decompose the domain into a series of binary maps using a stepwise increasing threshold that truncates the data along a range of selected thresholds that are typically defined by the variable’s percentiles, 2) perform a cluster analysis on each of the binary maps to identify connected clusters, and 3) describe the transition from many disconnected clusters to a single connected cluster as the threshold increases. The latter can be quantified with principles of percolation theory; Hovadik and Larue (2007) proposed the probability of connection as a suitable metric to quantify how percolated clusters are. This metric, denoted as Γ(t), is computed for each threshold t and describes the proportion of pairs of cells that are connected among all possible pairs of connected cells:

View Expanded

where is the total number of cells in the binary map below or above threshold t, which has distinct clusters in total, and n_i is the number of cells in the ith cluster in . The percolation is described through an increasing threshold that moves along all percentiles of the variable’s range. Focusing on the percentiles makes this methodology bias insensitive, and it allows investigating the clusters of high and low values. In the following, the connectivity analysis is separated and applied individually on cells that exceed a given threshold and those that fall below. The two approaches are referred to as high and low phase, respectively.

To compare the connectivity of a reference spatial pattern with a scenario , the root-mean-square error between the reference and scenario connectivity Γ(t) (RMSE_CON) can be computed for both phases:

View Expanded

The RMSE provides a global skill assessment of the connectivity that is not constrained by local agreement.

e. Multiple-point compatibility

In the field of geostatistics, multiple-point simulation algorithms are frequently applied frameworks to simulate heterogeneous geological structures with the aim to address geological uncertainty (He et al. 2013; Mariethoz et al. 2010). In comparison to conventional two-point simulations, by means of variograms or transition probabilities (He et al. 2014) that are solely based on data, multiple-point simulations are informed by training images that incorporate the geologist’s contextual insights as prior information (Mariethoz and Caers 2014; Mariethoz and Kelly 2011). Obtaining an accurate training image is crucial for the quality of a simulation, and Pérez et al. (2014) suggested a methodology to identify the most suitable training image by quantifying the spatial similarity of multiple competing training images with respect to reference data. The algorithm can easily be applied on spatial patterns of continuous land surface variables. For this study, the scenarios are considered as training images and the baseline model is set as reference. This metric is referred to as MPC in the coming sections. The basic idea behind MPC is to quantify spatial similarity as the proportion of spatial patterns that are found in both the reference and the simulated map. As indicated by Fig. 3, a spatial pattern is defined by a spiral search that starts at every cell that contains data and collects data from all cells it passes through. The number of data that are detected by the spiral search defines the pattern order. The user has to set a search radius (dxy) that defines the maximum extent of the spiral search and a sampling percentage that specifies how many data within dxy are recorded by the spiral search. The sampling option is included in the MPC algorithm to allow investigating large spatial patterns rather than focusing on small-scale spatial variability. After a spatial pattern in the reference is inventoried, the occurrence of this pattern is searched using a random search path across the scenario map. The pattern is checked off if the positions and values in the reference pattern are matched by a scenario pattern. Since it is difficult to find a perfect match, an error tolerance is introduced. To sidestep the bias sensitivity of MPC, we decided to transform the data to their spatial anomalies by removing the spatial mean at each time step. The metric ranges from zero to one and indicates the fraction of patterns contained in the reference that exist at any location in the simulation. Hence, MPC does not reflect any local agreement between simulation and reference. Instead, it focuses on the global assessment of the spatial structure. Last, the parameterization enables flexibility; for example, the search radius can easily be adjusted to investigate large and small patterns separately.

View Full Size

(left) The spiral search that characterizes a spatial pattern in the MPC analysis is illustrated for a map of simulated LST in January 2008. (center) First, a window size (dy and dx) is selected that defines the size of the pattern, then (right) a spiral search registers all cells that contain data and the number of cells defines the order of the spatial pattern. The window is sampled with a certain percentage to allow investigating large patterns rather than focusing on small-scale variability.

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

a. Variogram

The omnidirectional semivariance is computed at eight irregularly spaced lag distances between 1 and 25 km. Figure 4 illustrates the resulting LST and ET variograms for the baseline model and the scenarios for a summer day in July 2009. The degree of spatial detail differs between the scenarios, which results in different simulated spatial patterns that are equipped with individual variograms. Some scenarios have distinctly similar variograms in comparison to the baseline model, which suggests a limited spatial sensitivity. Others show a clear reduction in semivariance, indicating a decreased spatial variability and hence a high spatial sensitivity, such as the precipitation scenario for both variables and the groundwater coupling scenario for ET. However, some scenarios have a slightly increased semivariance with respect to the baseline, which seems counterintuitive at first, because the spatial detail is reduced in the scenarios. In general, homogenizing parameter or forcing fields poses complex transformations to the spatial patterns that are intractable and may result in a shift from energy limitation to water limitation or vice versa, which can lead to an increase of spatial variability.

View Full Size

Results from the variogram analysis for ET and LST patterns simulated by the baseline model (red) and all 22 scenarios (gray) on a single day (2 Jul 2009). Two scenarios are highlighted: scenario 6 (dashed) and scenario 9 (dotted). The omnidirectional semivariance is calculated at 10 lags spanning from 1 to 25 km.

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

b. EOF

The EOF analysis is computed based on daily spatial anomalies, and the integral spatiotemporal data matrix includes data from both scenario and baseline. The matrix dimensions are thus 2410 × 2192, where each row represents a position in space (2410 cells) and each column constitutes a daily time step from the 3-yr period (1096 days), which is doubled in order to accommodate data from both baseline and scenario. The resulting EOF maps and their associated loadings are presented in Fig. 5 for the assessment of simulated ET patterns of scenario 9 (disabled groundwater coupling). The first three EOFs are shown, which, as linear combination with their loadings, explain 64% of the variance. In general, multiplying the loadings with the EOF maps indicates how cells are related to the mean at a given time step. The predominant spatial pattern is captured by the first EOF, and it depicts the temporal evolution of the growing season with positive values for the agricultural cells that have a strong seasonal signal and negative values for the forest cells that show a dampened seasonal signal. The loadings indicate that scenario 9 comprises this pattern well, but honors it to a lesser degree because the loadings are generally lower in comparison to the baseline. Wintertime loadings are close to zero because little spatial variability is present in the ET patterns. The second EOF (EOF2) adds 21% to the explained variance and the loadings change sign, which indicates that this pattern is inverted during the season. The physical meaning behind EOF2 is less clear; however, the loadings of the baseline model and scenario 9 are almost identical, indicating that EOF2 captures the similarities between baseline and scenario ET patterns. Because of the spiky loadings time series, it can be speculated that EOF2 is related to the coupling of ET and precipitation. Furthermore, ET patterns from the baseline model and scenario 9 can be expected to be most similar during times when ET is controlled by precipitation, which diminishes the local groundwater influence. Last, the loadings of the third EOF (EOF3) are very divergent between the baseline model and scenario 9, and hence EOF3 captures major dissimilarities. Negative values in the EOF3 map reflect the critical zone where the water table is shallow and influences ET. The loadings of scenario 9 are positive and peak in the summertime, which underlines that ET is clearly underestimated in those areas with respect to the baseline model. Following Eq. (9), the weighted loading deviation can be utilized as a metric to quantify spatial similarity.

View Full Size

Results from the EOF analysis comparing the spatial patterns of ET between the baseline model and scenario 9 (no groundwater coupling) for a 3-yr period (2008–10). (top) EOF maps and the amount of explained variance by each EOF. (bottom) The associated loadings for the baseline model and for scenario 9.

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

c. FSS

The unique feature of FSS is its flexibility with respect to threshold and scale, and this is, to our knowledge, the first FSS application on hydrological land surface variables. Figure 6 depicts results for the evaluation of ET patterns for a summer day in June 2010 for three threshold percentiles: 1st, 5th, and 20th. At all thresholds FSS increases with scale, and FSS tends to one at large scales, which is an elemental characteristic because the number of cells in the binary maps for the baseline model and the scenarios are equal at each threshold. However, the rate of increase differs among the thresholds, because a natural higher degree of uncertainty is associated with the low percentiles in comparison to the higher percentiles. Overall, FSS clearly differentiates the scenarios with respect to their pattern similarity to the baseline model. To reduce computational time, the entire combination of scales and thresholds does not need to be calculated, and instead we suggest defining critical scales at which FSS is calculated. Uncertainty associated with the lowest 1% of cells may be tolerated better than higher percentiles. Therefore, the three percentiles are assessed at individual critical scales: 1st at 25 km, 5th at 15 km, and 20th at 5 km. These critical scales are subjective, but they were found suitable for our application. This is conducted analogously for the three top percentiles—80th, 95th, and 99th—that focus on the highest 20%, 5%, and 1% of cells, respectively. The averages of the three top and bottom percentiles are calculated as an overall pattern similarity score and are referred to as “FSS high” and “FSS low” in the following analysis. One drawback of FSS is that focusing on percentiles may force spatial variability onto maps; for example, wintertime ET is small and spatial variability is marginal in the Skjern catchment, but classifying the map using percentiles artificially creates spatial variability. However, the percentile approach makes this methodology bias insensitive, which is favorable in our opinion.

View Full Size

The variability of FSS obtained from the comparison of ET patterns on 1 Jul 2010 between the baseline model and 22 scenarios. Each gray line represents the spatial performance of a single scenario, and the red lines mark the critical scales for each percentile that is used to calculate the merged FSS. The critical scales are 25 km for 1%, 15 km for 5%, and 5 km for 20%. Three scenarios are highlighted: scenario 6 (dashed), scenario 9 (dotted), and scenario 15 (dash–dotted).

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

d. Connectivity

The connectivity analysis provides insights into the spatial structure of a pattern and quantifies the degree of percolation as a function of threshold. The methodology is exemplified in Fig. 7 for LST patterns on a spring day in March 2009 for scenario 1 (no spatial variability in air temperature) and the baseline model. The cluster analysis at threshold percentiles is an elementary feature of the connectivity analysis, and Fig. 7 depicts the resulting cluster maps at two percentiles: the 20th and 80th percentiles that focus on the lowest 20% and highest 20% of cells, respectively. In general, LST is strongly coupled to air temperature (Vancutsem et al. 2010), and the air temperature gradient in the Skjern catchment is characterized by warmer temperatures near the coastline in the west and a decrease in temperature toward east. Outside the growing season, where vegetation activity is minor, LST patterns exhibit a similar gradient. This results in a clear separation of the highest and lowest 20% of cells that form well-connected clusters in the baseline model, as indicated in Fig. 7. In scenario 1, this gradient is eliminated, which leads to a more scattered cluster allocation for the two given percentiles. Finally, the probability of connection at all thresholds can serve as a metric to quantify percolation as a function of threshold percentile. Abrupt increases in connectivity are distinct characteristics of connectivity curves and are unique attributes of spatial patterns (Renard and Allard 2013). The connectivity of both phases (low phase: cluster analysis for cells < threshold; high phase: cluster analysis for cells > threshold) is generally higher for the baseline model, which indicates that the spatial patterns are generally smoother and more homogeneous in comparison to scenario 1. As indicated by Eq. (14), the RMSE between the connectivity curves is incorporated as a metric to quantify the spatial performance for the low (CON low) and high phase (CON high) separately. Analogous to FSS, the connectivity analysis is based on threshold percentiles that may force spatial variability to patterns that are very homogeneous.

View Full Size

(top) Connectivity analysis illustrated for a comparison of LST maps simulated on 28 Mar 2008 by (left) the baseline model and (right) scenario 1 (no spatial variability in air temperature). The continuous maps are first truncated at specific thresholds, and then a cluster analysis is performed on the basis of the binary maps. (middle) The results from the cluster analysis at the 20th and 80th percentiles. The first analyzes the coldest 20% of cells and the latter focuses on the warmest 20% of cells. A unique color is assigned to each connected cluster. (bottom) The computed connectivity metric is shown for the high and low phase at each percentile.

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

e. MPC

The MPC algorithm is very flexible because of its parameterization, but at the same time, care must be taken when defining the essential parameters, namely, pattern order, search radius, sampling percentage, and error threshold. Figure 8 reflects on this issue and plots the number of valid patterns as a function of the MPC parameters. An invalid pattern occurs if the spiral search does not reach the required pattern order within the given search radius, which is typically the case for cells close to the catchment boundary. These boundary effects are caused by the geometry of the catchment domain. The spiral search starts at each of the 2410 cells that comprise the Skjern catchment. With 100% sampling, ~1800 cells have valid patterns with full coverage when applying a 2-km search radius, which implies that the 5 × 5 search window contains 25 cells with data. The fully covered 5-km search radius that defines an 11 × 11 spiral search and has a pattern order of 121 is valid for ~1200 cells. The number of valid patterns increases for lower pattern orders, because empty cells are tolerated in the search window. Sampling reduces the maximum pattern order; for instance, 33% sampling of the 5-km search radius lowers the maximum pattern order from originally 121 (11 × 11 search window) to 40. This allows MPC to assess large patterns without focusing on small-scale variability. For this study, we define two MPC parameterizations (indicated in gray in Fig. 8). The first investigates large patterns at a 5-km search radius with 10% sampling, and it considers pattern orders between 8 and 12. The second setup focuses on small patterns with a search radius of 2 km. Here the sampling percentage can be higher (20%), because small-scale variability is more desirable and pattern orders of 4 and 5 are taken into consideration. In the following the two described MPC parameterization schemes are referred to as “MPC large” and “MPC small.” Furthermore, the error threshold, which is used for the comparison of inventoried patterns, has to be defined. The error threshold should be determined in dependence of the variable of interest. A straightforward and objective option is to draw the value from the variable’s standard deviation. We define MPC large to tolerate an error of one standard deviation when comparing patterns of LST and ET, and MPC small is set up with an error threshold of half a standard deviation. This differentiation enables MPC large to address larger gradients that have a natural higher variability, whereas small patterns have lower gradients, and thus, accuracy has to be higher. The standard deviations for LST and ET are derived from the baseline model and are 0.62°C and 0.27 mm, respectively.

View Full Size

The number of cells that contain data within a window of size dxy defines the order of the spatial pattern for the MPC analysis. Sampling is included in order to investigate large patterns rather than focusing on small-scale variability. The number of valid events represents the cells that ensure a certain pattern order given a window size (dxy) and a sampling percentage. The gray lines mark the range of order and the sampling that is chosen for this study to reflect small patterns (dxy = 2; sampling = 20%; orders 4 and 5) and large patterns (dxy = 5; sampling = 10%; orders 8–12).

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

f. Seasonal sensitivity

The set of eight spatial performance metrics described above (variogram, EOF, FSS high, FSS low, CON low, CON high, MPC small, and MPC large) is applied on 3 years of daily LST and ET patterns of the 22 scenarios, and spatial similarity is quantified with respect to the baseline model. As an example, Fig. 9 depicts the daily spatial performance based on the connectivity analysis of the high phase of LST patterns for two scenarios. Both scenarios 10 and 14 perturb the LAI input, where the first is based on a 10-km smoothing and the second homogenizes the LAI fields to spatially uniform values. The connectivity analysis identifies small changes in the spatial patterns during wintertime, but two peaks, one at the start and one at the end of the growing season, are detectable in all 3 years. These peaks underline the distinct seasonality in LST sensitivity to LAI. Furthermore, the connectivity of the warm LST clusters shows similar scores for the late growing season in both scenarios. In this period, a homogeneous LAI input informs the model equally well as a 10-km smoothed LAI input. However, during the early growing season, the latter performs clearly better than scenario 14 with constant LAI. In summary, the spatial patterns of LST are most complex during the late cropping season, where information at the 10-km scale is clearly insufficient. During the middle of the growing season, where vegetation is active throughout the catchment, the connectivity of the scenarios is in good agreement with the baseline model, which supports the interpretation that spatial patterns are simple and detailed information on LAI is less crucial.

View Full Size

The results obtained from the connectivity analysis of the high phase of LST is given for scenario 10 (10-km smoothed spatial variability of LAI) and scenario 14 (no spatial variability of LAI). Low values indicate spatial similarity. The metric is computed on the daily maps and, for better visualization, a 7-day smoothing filter is applied on the daily data.

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

To summarize the most important findings, Figs. 10 and 11 show the spatial performance of all eight metrics for the four most sensitive scenarios for ET and LST in a single year. The most sensitive scenarios are identified by simply selecting the ones that exhibit the lowest pattern performance on average for the 3-yr period. The results for the ET patterns in 2008 (Fig. 10) stress that the sensitivity of most scenarios underlies a distinct seasonality. Furthermore, the seasonality of sensitivities is partly rated similar by the spatial performance metrics. Scenarios 6 and 9 are related to groundwater coupling and precipitation, respectively, and their sensitivity is highest in early summer. This conclusion is clearly indicated by EOF and variogram, whereas the other metrics provide a more nuanced interpretation. This underlines the complexity of comparing spatial patterns, because different metrics are expected to provide different interpretations of the spatial similarity. Perturbations in LAI and vegetation height are considered in scenarios 14 and 17, respectively, and their sensitivity is highest toward the end of the growing season. Scenario 17 has a distinct pattern sensitivity in wintertime, as indicated by FSS and connectivity, because the year-round constant vegetation height of the forest is averaged out with all nonforest areas. This underlines that a detailed land-use map and hence the correct allocation of forest cells is essential to represent wintertime processes correctly. EOF, variogram, and MPC that are not based on percentile thresholds attest little sensitivity to the scenarios in wintertime, because spatial variability of ET is low during the energy-limited months.

View Full Size

Eight spatial performance metrics that assess the spatial similarity between the baseline model and four scenarios are shown for simulated ET patterns. The temporal variability of spatial similarity is given as averages of the smoothed metric scores for a 3-yr period (2008–10): scenario 6 (no spatial variability of precipitation), scenario 9 (no groundwater coupling), scenario 14 (no spatial variability of LAI), and scenario 17 (no spatial variability of vegetation height).

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Eight spatial performance metrics that assess the spatial similarity between the baseline model and four scenarios are shown for simulated LST patterns. The temporal variability of spatial similarity is given as averages of the smoothed metric scores for a 3-yr period (2008–10): scenario 1 (no spatial variability of air temperature), scenario 9 (no groundwater coupling), scenario 14 (no spatial variability of LAI), and scenario 17 (no spatial variability of vegetation height).

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

Figure 11 depicts results for LST sensitivity, and in comparison to ET, scenario 9 is the sole scenario that shows a clear seasonality that follows the growing season. Additionally, the spatial sensitivity of air temperature is given, and it is rated most significant during winter by FSS and connectivity, which stresses the importance of the east–west air temperature gradient outside the growing season. The other metrics, which are not dependent on percentiles, evaluate air temperature as sensitive as well, but attest less seasonality. Again, scenario 17 is most sensitive during wintertime, which underlines the importance of vegetation height as a driver of spatial variability of LST. The sensitivity of LAI is addressed by scenario 14 and, similar to ET, two peaks are identified by most of the metrics, with the first at the start and the second at the end of the growing season.

g. Spatial information content

Comparing spatial patterns is not a trivial task because many dimensions of spatial information have to be considered simultaneously. Typically, a spatial metric focuses on a limited number of dimensions, such as the overall global structure or the local agreement, as indicated by Table 4. Metrics will naturally give ambiguous results that should not challenge their reliability and legitimacy. Instead, diverging interpretations between metrics should be embraced, because this stresses that different dimensions of spatial information are covered. On the other hand, metrics with similar scores can be considered redundant. To analyze the uniqueness and redundancy of the set of applied performance metrics, Table 5 states the coefficient of determination R² between the metrics based on individual daily pattern similarity scores for ET and LST of all scenarios. The spatial correlation coefficient R, the overall bias, and the spatial RMSE are added in order to include some traditional and easy-to-compute metrics, which are based on cell-to-cell comparisons. Overall, the differences between the R² scores for the LST and ET patterns are marginal and hence interpretations are analogous. The bias explains negligible amounts of variance of the other metrics, which is logical given the bias insensitivity of the other metrics. To better understand drivers of spatial variability, the bias can be a relevant metric; for example, scenario 9, which decouples groundwater and land surface, results in a negative bias of up to 2 mm day⁻¹ (not shown) due to the reduced water availability. However, we focus on bias-insensitive metrics and suggest applying these in combination with the bias when validating or calibrating a model spatially. The separation is crucial, because a metric that is sensitive to both the spatial pattern and the bias may be ambivalent and misleading. From a modeling perspective, the spatial pattern and the bias are typically controlled by different parameters, and model diagnostics are most straightforward if metrics treat them separately.

Table 4.

Overview of the spatial performance metrics.

View Table

Table 5.

The coefficient of determination between the applied spatial performance metrics. The analysis is based on the daily spatial similarity scores for a 3-yr period (2008–10) for all scenarios. Values above 0.5 are given in boldface to visualize significant dependencies. High R² indicates redundant information content between the assessed metrics.

View Table

The pattern scores from the spatial correlation coefficient are in reasonable agreement with FSS and the connectivity analysis because all are controlled by the tails of the distribution. Variogram, EOF, MPC, and RMSE are somewhat dependent on the spatial variability of the patterns, which results in high R² scores. Overall, part of the information of more complex spatial performance metrics can be covered by simpler cell-to-cell metrics. Based on the results of Table 5, the EOF analysis has the highest average R², which allows the conclusion that it is the most suitable standalone metric for the LST and ET patterns of the Skjern catchment. Connectivity analysis and FSS show least redundancy to the remaining metrics, which underlines that assessing spatial patterns at threshold percentiles clearly adds information to a pattern comparison. MPC is related to the variogram analysis because both assess a pattern via a sequence of cells: MPC takes multiple locations into consideration, whereas variogram analysis focuses on two locations. The differentiation of MPC into MPC small and MPC large provided limited additional information content to the pattern assessment. The connectivity of the low phase and the high phase are attributed with R² values of 0.55 and 0.66 for ET and LST, respectively, and hence they are not fully independent. However, the differentiation of FSS into FSS high and FSS low proves to be very reasonable, because they exhibit low R² scores.

h. Relative sensitivity

To summarize the results of all 11 spatial performance metrics, Fig. 12 depicts the relative sensitivity of each scenario with respect to spatial patterns of ET and LST. The relative sensitivity is computed based on the average spatial similarity score over the entire 3 years. For each metric, the scenario with the lowest score is marked as most sensitive and the scenario with the best score is rated as the least sensitive scenario. In the following the relative sensitivity is linearly scaled between zero and one, where one defines the highest spatial sensitivity. The general agreement between the spatial performance metrics is striking. The only peculiar metric is the bias, which is not surprising, given that the other metrics are bias insensitive. Differences between spatial drivers of ET and LST are clear. Vegetation, groundwater, and precipitation are the most dominant drivers of ET patterns, whereas LST is most sensitive to climate forcing and vegetation height (scenario 17). It is unexpected to see low spatial sensitivity attested to albedo (scenario 16) and root depth (scenario 15) for the simulation of the two given land surface variables, because these are frequently used for calibrating integrated catchment model against discharge. Consequently, albedo and root depth are discharge sensitive because they are affecting the water balance, but their pattern sensitivity appears limited. The vegetation parameters derived from the land-use map (scenarios 19–22) are more sensitive to ET patterns than LST patterns. In particular, interception is a sensitive parameter to the simulated ET patterns. This is logical from a modeling perspective, because it is the first storage term to be evaporated, before transpiration or soil evaporation is initiated. Interestingly, scenarios 7 and 8, which are related to the soil type map and the hydrogeological model of the near surface, respectively, are mostly rated insensitive to the spatial patterns of LST and ET. These model parameters are typically included in calibration frameworks of catchment models against discharge, but our spatial sensitivity analysis stresses that soil physical parameters and hydraulic conductivity have a limited sensitivity to energy balance–related variables at the land surface. The soils in the Skjern catchment are predominantly of sandy origin and variability in soil texture is less pronounced, which may cause the reduced relative sensitivity of scenarios 7 and 8.

View Full Size

The relative sensitivity is given for each metric for the respective scenarios. The sensitivity is linearly normalized relative to the most sensitive scenario (1.0) and the least sensitive scenario (0.0). The analysis is based on mean values for each scenario based on 3 years (2008–09).

Citation: Journal of Hydrometeorology 18, 4; 10.1175/JHM-D-16-0148.1

• Download Figure
• Download figure as PowerPoint slide

Whether state-of-the-art or not, a model will always depict a simplified version of the natural systems. Therefore, the identified spatial sensitivities are not directly transferrable to the natural system. Furthermore, it is questionable if the conclusions are applicable to other catchments that are equipped with different characteristics.