1. Introduction

Snow cover and snow depth are important parameters for numerical weather prediction (NWP) and hydrological models, especially in springtime during the melting period. Essential characteristics include snow water equivalent (SWE), snow depth, and snow covered area. For hydrology, the maximum SWE prior to the onset of spring snowmelt is typically the most important snow characteristic for operational runoff and river discharge forecasts. In NWP models, the snow cover affects the surface albedo and the heat flux between the ground and the atmosphere, and therefore has a strong effect on the lower part of the atmosphere. The variable inside NWP models is typically SWE. However, because the SWE can only be analyzed indirectly from temporally and spatially sparse snow depth and snow cover observations, the accuracy of analysis is limited. Thus there is a pressing need for improved snow analysis in numerical models.

In the foreseeable future, there will be no improvements to in situ snow observations, but the satellite technology is advancing rapidly and the progress in snow analysis will be based on the increasing use of satellite products. Snow cover analyses produced from satellite information have much better spatial coverage, and both snow and no-snow observations are readily made. However, methods are often based on visible and near-infrared channels and suffer from cloud cover and limited sunlight during the polar winter. On the other hand, methods based on passive microwave instruments are not limited by clouds but suffer from coarser spatial resolution and they work only for dry snow conditions (Ulaby et al. 1986).

The purpose of the current study was to assess the current situation: What satellite products are available and how consistent are their results with each other and with NWP analyses of snow cover? This study concentrated on satellite analyses with methods based on visible and near-infrared channels, as they are more widely used in NWP than methods-based microwave instruments [e.g., Advanced Microwave Scanning Radiometer for Earth Observing System (EOS) (AMSR-E)]. With methods based on visible and near-infrared channels, we are in practice restricted to information about the existence of snow in binary form. However, for many applications, SWE is a more important parameter than the snow cover, and therefore developments in this area will be welcome (e.g., Comiso et al. 2003; Pulliainen 2006). Also, our emphasis is more on operational NWP use than on climatological studies [for more climatological and SWE-oriented comparison, see, e.g., Frei et al. (2005)].

Satellite and NWP snow analyses were compared with each other from January to June 2006, first in the whole Northern Hemisphere and then concentrating on northern Europe. The satellite analyses were from the Interactive Multisensor Snow and Ice Mapping System (IMS) from the National Oceanic and Atmospheric Administration (NOAA), Moderate Resolution Imaging Spectroradiometer (MODIS) snow product from the National Aeronautics and Space Administration (NASA), and the snow cover analysis of the Land Surface Analysis (LSA) Satellite Applications Facility (SAF) from the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT). The NWP analyses were from the High-Resolution Limited-Area Model (HIRLAM) and European Centre for Medium-Range Weather Forecasts (ECMWF). HIRLAM uses only in situ observations for its analysis, while ECMWF uses both in situ and satellite observations (the analysis from NOAA). ECMWF and NASA analyses are global, NOAA covers the Northern Hemisphere, HIRLAM covers northern Europe, and LSA SAF uses the geostationary satellite instrument and is thus constricted by the field of view of the instrument.

Numerous studies of snow analyses or comparisons between analyses and ground truth have been made. Producers of snow analyses have published validation studies; for example, Hall and Riggs (2007) reported that the absolute accuracy [the proportion correct (PC), as defined below, against in situ observations] of the MODIS analysis is around 93%, but varied on the land cover type. Most problems arose from cloud detection over snow and from very thin (less than 1 cm) snow. Other studies have compared MODIS snow analyses of in situ observations in, for example, China (Wang et al. 2008) and Europe (Parajka and Blöschl 2006; Sorman et al. 2007) with comparable results. Brubaker et al. (2005) compared MODIS and IMS snow analyses with in situ observations in the United States and reported that IMS underestimates snow in the transition seasons but outperforms MODIS analysis in cloudy areas with new snow and during winter. Drusch et al. (2004) shows how the inclusion of IMS snow improves the ECMWF snow analysis. However, to our knowledge, the present study is novel because comparisons between more than two snow analyses have not been widely performed.

2. Data

The snow analyses used in this study are called both products and analyses. The difference between an analysis and a product is a matter of definition. The term product emphasizes that the analysis is made for the general use, while, for example, NWP analyses are more restricted both in distribution and in intended use. For consistency in this study, after the introduction of analyses and products, only the term analysis will be used.

3. Methods

In this study we concentrate only on binary snow cover, that is, if the land is snow covered or not. As detailed above, data are disseminated in a variety of projections, but first all data were reprojected to Lambert cylindrical equal-area projection. In this area-preserving projection, comparisons of analyses extending over large geographical areas are meaningful. For the comparisons of ECMWF, IMS, and MODIS analyses in the Northern Hemisphere, analyses were reprojected with a resolution of 25 km. This resolution is close to the original resolution of ECMWF data. For comparisons of HIRLAM, ECWMF, IMS, MODIS, and LSA SAF analyses in northern Europe, analyses were reprojected with a resolution of 5 km, which is close to the original resolution of analyses other than ECMWF. All reprojections were done using the freely available Geospatial Data Abstraction Library (GDAL).

Resamplings were done using the simple nearest-neighbor method.

NWP analyses are for SWE, not snow cover. To transform this to snow cover, we simply classified grid boxes with SWE greater than zero as snow covered. This may result in a little overestimation of snow in comparisons. MODIS analyses have the fractional snow area for each grid point and we classified the fractional snow area greater than 20% as snow. This may result in a little underestimation of snow in comparisons. In addition, only MODIS grid points more than 50% cloud free were processed.

Snow behaves differently in mountainous areas and near the sea surface level, and therefore, using the freely available global topography with 30 arc s resolution from the U.S. Geological Survey’s (USGS) Center for Earth Resources Observation and Science (EROS) (GTOPO30), we divided the land areas into two classes: lowlands and highlands with a threshold of 600 m. This threshold is somewhat arbitrary and was selected so that the same threshold could be used in the Northern Hemisphere and northern Europe. In the Northern Hemisphere, lowlands will cover 65% of land and highlands 35% (Fig. 2). The area for which the comparison in northern Europe is done is shown in Fig. 1. Here lowlands will cover 95% of the chosen area and highlands 5% of the area (Fig. 3).

The treatment of clouds in MODIS and LSA SAF analyses needs to be considered. As the purpose of this study was to compare day-to-day analyses, no effort was made to lessen the impact of clouds, by, for example, aggregating cloudy analyses from different days. A simple solution adopted was to compare analyses only in cloud-free areas and ignore the cloudy area.

To further simplify the comparison, only 1200 UTC analyses of ECMWF and HIRLAM and only MODIS Terra snow analyses were used. The amount of snow varies from one analysis time to the next or between Terra and Aqua analyses, but we found the difference to be smaller than the difference between different analyses.

The results of comparison between two analyses can be shown in a 2 × 2 contingency table (Table 1), where a is the number of cases where both analyses reported snow, d is the number of cases where neither analyses reported snow, b is the number of cases where only the first analysis reported snow, and c is the number of cases where only the second analysis reported snow. There is extensive literature of different measures calculated from this table (e.g., Jolliffe and Stephenson 2003). The basic measure is proportion correct, defined as

View Expanded

PC is the fraction of items classified correctly when one of the analyses represents the truth. The best value for PC is 1, and the worst value is 0. This measure alone is not sufficient, in particular when one of the categories dominates.

Another widely used measure is the Heidke skill score. HSS is PC adjusted to account for the agreement between two analyses due to chance. HSS is defined as

View Expanded

and has the same value regardless of whether analysis 1 or analysis 2 is the truth, because the value does not change if we switch b and c. HSS is important for our study, as the amount of the snow varies considerably during our study period and PC can give quite misleading results. HSS has been independently discovered and named many times. For example, in the remote sensing community, it is called kappa (Congalton and Green 1998).

One of the challenges of this study is that there is no independent ground truth. As analyses use the same input data or even directly use other analyses, none of them are independent and none can be said to represent the truth. IMS uses MODIS and SEVIRI data as input and is the basis for ECMWF snow. Although HIRLAM does not use satellite data, it uses the same in situ SYNOP observations as ECMWF. Had it been possible to rerun NWP analyses, independent in situ observations could be done by cross validation (e.g., Wilks 2006) of ground measurements, but rerunning analyses would require resources not at our disposal. This lack of ground truth means we cannot assess the accuracy of, for example, ECMWF compared with IMS, but can only assess the difference of analyses (e.g., the impact of in situ measurements on ECMWF).

We cannot rank the analyses with, for example, HSS, because the ground truth does not exist. Instead of ranking, we can compare the consistency between analyses by calculating HSS for each possible analysis pair. There are however many pairings, and results can be hard to visualize. One way forward is to note that HSS tells how similar analyses are and can be considered as a measure of distance between analyses. This suggests we can use multidimensional scaling (MDS) (e.g., Ripley 1996) to construct a simpler two-dimensional mapping of results. MDS tries to retain the distances between variables, that is, distances between variables in a map should be proportional to the actual distances. These measures of distance, or dissimilarity, do not have to satisfy all requirements for a proper mathematical distance; it is enough for them to be nonnegative symmetric numbers. Often MDS is done using the Euclidean distance or the correlation matrix between variables, but here HSS, or 1 − HSS in practice, is used. As a perfect match has the maximum value HSS = 1, 1 − HSS is nonnegative. In addition, 1 − HSS is symmetric because the score remains the same when the reference and the test datasets are swapped. There are different methods of how to implement MDS, and for this study Sammon mapping (Ripley 1996) was chosen, as it converged most reliably with the best solution in the current dataset.

Finally, days should be aggregated unless one mapping for each day is desired. Using the objective k-means algorithm (e.g., Ripley 1996), the test period is partitioned into two clusters. Tests with more than two clusters did not alter the conclusions below, but unnecessarily complicated the analysis. The clustering was done using all components of the HSS matrix as features, so in the Northern Hemisphere there were three features and in northern Europe ten features. In the clustering, no temporal information is used, so clusters need not be continuous in time. After clustering, Sammon mappings are calculated for the mean values of each cluster.

In comparisons below, the following plan is followed: first, images of different analyses for a particular day are presented. As snow cover changed considerably during our study period, simple static images are not enough and time series are needed. Time series of the snow amount in analyses and values of HSS with IMS as a baseline are shown. Finally, HSS matrices and derived Sammon mappings are presented.

4. Comparison of snow analyses in the Northern Hemisphere

First we compare the IMS analysis with ECMWF and MODIS analyses in the Northern Hemisphere. An example (Fig. 4) shows a comparison of analyses for day 93, 3 April. ECMWF and IMS agreed for the most part (Fig. 4b), though ECMWF had more snow. MODIS appeared quite similar to IMS in cloud-free areas (Fig. 4c), but overall visual comparison was difficult if not impossible, as the area was very much obscured by clouds, even though this day was chosen because it was relatively cloud free.

The change of the percentage of snow-covered areas and cloud-free areas in different analyses and two different height classes, lowlands and highlands, is shown in Fig. 5. The comparison between ECMWF and IMS is straightforward because snow cover is available both for cloudy and cloud-free conditions in ECMWF and there are no clouds in IMS. As expected, the snow started to melt as the spring progressed. In the lowlands, melting started in late February (Fig. 5a). In the highlands, melting started a little earlier, in late January (Fig. 5b), probably because large parts of its area, like the Tibetan Plateau, are rather far south. ECMWF consistently detected more snow than IMS, especially in the highlands. The proportion of snow-covered area detected in IMS is almost the same in the highlands and lowlands, but ECMWF detected more snow in the highlands than in the lowlands. Most of this extra snow is detected in the Tibetan Plateau (Fig. 4b). The Tibetan Plateau is a difficult area to analyze (Drusch et al. 2004). It is characterized by intermittent and patchy snow cover, which is problematic for methods based on visible and infrared measurements from satellite. Also, few reliable real-time observations are available.

Because of the clouds, MODIS analyses detected much less surface than ECMWF and IMS analyses (Figs. 5c,d). In addition, MODIS analyses detected even less snow. For example, in the lowlands on 1 January, there was approximately as much snow-free area as snow-covered area in the IMS analysis (Fig. 5a), but there was 15 times more snow-free area than snow-covered area in cloud-free areas of the MODIS analysis (Fig. 5c). And in MODIS analyses of the lowlands, the snow area actually increased until the end of May. Thus the clouds in MODIS analyses are not randomly distributed, but are more likely to be detected over snow areas. One possible reason for this is that even when using the best available instruments and methods, there is always some uncertainty in discriminating between low clouds and snow and the cloud mask is designed to be cloud biased. But in addition to this, there probably exists a physical reason for this difference in snow cover in cloud-free and cloudy conditions. This would be an interesting topic for a future study. In cloud-free areas, the difference between MODIS and IMS is not as evident as the difference between ECMWF and IMS.

The absolute differences between analyses are quite small and PC is over 0.95 for all comparisons, except for ECMWF and IMS in the highlands where it is slightly lower, around 0.85 (not shown). These results might be a little optimistic, and HSS gives a more realistic view. The HSS for comparison of ECMWF and MODIS against IMS is shown in Fig. 6. Both ECMWF and MODIS agree better with IMS in the lowlands (Figs. 6a,c) than in the highlands (Figs. 6b,d). MODIS agrees with IMS better than ECMWF both in the lowlands and highlands, even when there is more scatter in values of MODIS in the highlands (Fig. 6d). Results are still rather stable, with some downward trend in the lowlands from the beginning of April (Fig. 6a).

For Sammon mapping, the HSS values are combined into the HSS matrix. It is almost complete; only HSS between ECMWF and MODIS is missing and has to be computed. After that the values are divided into two clusters using k-means, and the resulting classification of days is shown in Fig. 6. In the lowlands the first cluster includes days from the middle of the study period while the second cluster has days at the start and end of the period. In the highlands, the first cluster has more days at the end of the study period and the second cluster at the start of the period.

With only three members, the HSS matrix is quite simple, and it is helpful to look at in more detail how Sammon mapping compares to the matrix before moving to the more complex HSS matrix in the next section. In the first cluster of lowlands (Fig. 7a), IMS and MODIS are very similar (HSS = 0.97), IMS and ECMWF are still quite similar (HSS = 0.93), and even between ECMWF and MODIS the similarity is high (HSS = 0.90). In Sammon mapping this means MODIS and IMS are nearer to each other than to ECMWF. The distance in between IMS and MODIS in Sammon mapping should be 0.03 (i.e., 1 − 0.97), between IMS and ECMWF 0.07, and between MODIS and ECMWF 0.10. Had all analyses been identical (HSS = 1), all points would have collocated at the center. For a more complex matrix, it cannot be guaranteed that distances are exactly preserved in lower-dimensional Sammon mapping. Compared to the first cluster of lowlands, the similarity of IMS and ECMWF does not change much (from HSS = 0.93 to HSS = 0.90) in the second cluster of lowlands (Fig. 7b), but the similarity of MODIS with others drops, especially the similarity of MODIS and ECMWF (from HSS = 0.90 to HSS = 0.78). In Sammon mapping, MODIS moves some distance away from IMS analysis and much farther away from ECMWF. Actually, in all clusters, the IMS is more similar to ECMWF and MODIS than ECMWF and MODIS are to each other. Thus in Sammon mapping, IMS is more or less in the middle of the other two analyses. Both in the lowlands (Fig. 7) and highlands (Fig. 8), there is not much temporal change in the difference between IMS and ECMWF. The greatest differences and changes in differences come from MODIS and ECMWF, which could not be seen in Fig. 6. In the lowlands, all analyses were rather similar in the middle of the period as HSS is greater than 0.90 for all comparison (Fig. 7a), but MODIS deviates from the other two at the start and end of the study period as HSS drops from 0.97 (0.90) to 0.89 (0.78) with the comparison with IMS (ECMWF) (Fig. 7b). In Highlands MODIS came much closer to IMS and ECMWF at the end of the study period (Fig. 8b). Both in the lowlands and highlands, the physical interpretation of clusters is that days are partitioned to cases of lesser and greater agreement between analyses. With only two clusters, this is the rather obvious solution we could find by hand, but the use of a clustering algorithm like k-means is motivated by the fact that the partitioning can be done objectively and consistently.

5. Comparison of snow analyses in northern Europe

An example (Fig. 9) shows a comparison of analyses against IMS for day 99, 9 April. Day 99 was subjectively chosen as it was relatively cloud free in MODIS with about half of the snow cover still remaining. ECMWF and IMS agreed for the most part, even though ECMWF had more snow (Fig. 9b), especially spotty areas of snow in central Europe. MODIS looked quite similar to IMS in cloud-free areas but, again, overall visual comparison is difficult as the area is very much obscured by clouds (Fig. 9c). LSA SAF was obscured by clouds to a lesser degree, but in Scandinavia its analysis had large areas of no snow where other analyses detected snow (Fig. 9d). Similarly, HIRLAM had large areas of snow where there was no snow in IMS as well as areas of no snow where there was snow in IMS (Fig. 9e). HIRLAM suffers from the unfortunate practice that SYNOP stations often do not report no-snow conditions, so for large areas no observations of no-snow are available. In our example, this can be seen in Baltic countries, where information of snow observations in the north is spread too far south.

In contrast to the Northern Hemisphere (Fig. 5), there is more snow in northern Europe at the start of the study period and almost no snow at the end of the period (Fig. 10: unfortunately, HIRLAM analyses in January 2006 were unavailable, and only days from February onward are shown). Nearly the entire region was covered by snow before the end of March, so there was little difference between most analyses, except HIRLAM in the highlands (Fig. 10h) and LSA SAF (Figs. 10e,f). As the snow melts away the differences can be more clearly seen. The snow started to melt in April as the spring progressed, a little later than in the whole hemisphere data, and most of the snow was gone by the middle of May in the lowlands. As in the Northern Hemisphere, MODIS and LSA SAF analyses with clouds detected much less surface than snow analyses of IMS, ECMWF, and HIRLAM with no clouds, and most of the time the ratio of snow-free and snow-covered areas was higher in MODIS and LSA SAF analyses than in other analyses. LSA SAF sees much more cloud-free area than MODIS, as it aggregates information from all SEVIRI images for a day. Both in MODIS (Fig. 10c) and LSA SAF (Fig. 10e), the less cloudy period can be seen in the beginning of May in the lowlands. ECMWF detected more snow than IMS analyses both in the lowlands (Fig. 10a) and highlands (Fig. 10b). The small amount of cloud-free area that MODIS sees, especially in the highlands (Fig. 10d), makes the behavior of MODIS rather erratic, and sometimes MODIS and IMS disagree greatly but the sign of the difference can change from day to day. For the entire period and in both areas, LSA SAF saw much less snow than IMS. In the highlands (Fig. 10h), HIRLAM detected less snow than IMS analyses; in the lowlands (Fig. 10g) this was not as apparent.

Agreement between analyses measured with PC is good (not shown). For MODIS and ECMWF, PC values were around 0.8, near an optimum 1.0, as it also was in the Northern Hemisphere. Disagreement was larger for HIRLAM and for LSA SAF, but in the lowlands PC tended to optimum as the snow melted away. Again, PC was too optimistic, because the dominance of snow in early spring and snow-free areas in later spring will influence the results, and HSS gives a more realistic view (Fig. 11). HSS fluctuated greatly during the study period and values were lower than in the Northern Hemisphere. All comparisons showed results of no skill in March in the highlands, when almost the whole area was snow covered. Additionally, at the end of the study period in the lowlands, when PC reached optimum values, HSS had much lower values.

According to HSS, MODIS and ECMWF agreed for the most part with IMS. For MODIS in the lowlands, results were good except at the very end of the study period (Fig. 11c). In the highlands the values were low at the start of the study period, but improved as the melting started (Fig. 11d). In both cases the results fluctuated a lot, probably because only small portions of the earth’s surface could be seen because of clouds. ECMWF had slightly lower values but not as much noise in the results (Figs. 11a,b). LSA SAF had consistently mediocre results both in the lowlands and highlands (Figs. 11e,f). HIRLAM had the biggest differences in the highlands (Fig. 11h).

Looking at the HSS values of comparisons with IMS as the baseline (Fig. 11), we can see how the highlands are easier to subjectively divide into two clusters than the lowlands. In the highlands HSS had much lower values before the snow melted than after the melting started. This kind of behavior is not as easily seen in the lowlands. The k-means follows this reasoning and two clusters it found in the highlands are reasonably the same as what we would choose by hand if we used only the information in Fig. 11. In the lowlands the second cluster is much smaller and contains mainly days at the end of the study period when most of the snow had melted away. As in the previous section, these HSS clusters represent the situations of larger and lesser agreement between analyses. In the lowlands (Fig. 12) agreement between analyses is best when the most snow is present, while in the highlands (Fig. 13) agreement is highest when most snow had melted away.

With IMS as the baseline, it can be seen how LSA SAF and HIRLAM deviated most from IMS, while the other two analyses were close to IMS. The broader view can be seen with Sammon mapping. In the lowlands, the larger cluster (Fig. 12a), when most snow was present, has MODIS, IMS, and ECMWF reasonably close to each other (HSS > 0.60), while HIRLAM and LSA SAF deviated more from the other three and from each other (HSS < 0.60). The second cluster (Fig. 12b) with less snow represents days where the disagreement between analyses was greatest. Most notably, MODIS that agreed well with IMS and ECMWF in the first cluster disagreed with others, and only ECMWF and IMS agreed well with each other (HSS = 0.65). In the highlands, at the start of the study period (Fig. 13a), there was not much consensus between analyses (HSS < 0.20), and at the end of the period (Fig. 13b) the agreement was better. The best agreements (HSS > 0.60) are between IMS and MODIS, between IMS and ECMWF, and surprisingly, between MODIS and LSA SAF.

6. Summary

In this paper, satellite-based snow cover analyses were compared with NWP snow analyses. The major requirement of any validation is the independent validation data or ground truth. However, we did not have any independent data for validation. The IMS snow analysis from NOAA uses data from the MODIS and SEVIRI instruments; that is the basis for snow analyses from NASA and LSA SAF. The snow analysis of ECMWF uses IMS as an input. Although the snow analysis of HIRLAM does not use any satellite data, it uses the same SYNOP observations as ECMWF. So we could only compare the consistency between analyses. To this end, we calculated the Heidke skill score between all analyses.

The snow analyses were generally consistent with each other. IMS and ECMWF analyses were quite similar, though ECMWF had more snow. Terra MODIS snow and IMS snow were also quite consistent most of the time, but MODIS was greatly hampered by clouds. This seriously decreases the usefulness of MODIS data in NWP, especially because the cloud problems seem to be concentrated in snow-covered areas. LSA SAF was often the least consistent with other analyses. However, LSA SAF was at the preoperational stage, and improved soon after [preliminary new results have been presented in Siljamo and Hyvärinen (2008)].

Based only on in situ measurements and thus hampered by the lack of observations of no snow, HIRLAM had problems in defining the snow edge and behaved rather erratically during the snow melting season. The surface analysis schemes in ECMWF and HIRLAM are quite similar and the biggest difference is the use of IMS in ECMWF. The use of IMS data in ECMWF analysis helps to analyze more accurately the snow edge. In the HIRLAM analysis, the snow edge is often analyzed too far into the no-snow areas because of the practice of not reporting no snow in the SYNOP reports. This suggests that the HIRLAM, or any other NWP snow analysis based only on in situ measurements, would greatly benefit from the inclusion of satellite-based snow/no-snow information. This information can be, for example, IMS, or at a later stage, the improved LSA SAF snow cover analysis.