Satellite-derived high-resolution precipitation products (HRPP) have been developed to address the needs of the user community and are now available with 0.25° × 0.25° (or less) subdaily resolutions. This paper evaluates a number of commonly available satellite-derived HRPPs covering northwest Europe over a 6-yr period. Precipitation products include the Tropical Rainfall Measuring Mission (TRMM) Multisatellite Precipitation Analysis (TMPA), the Climate Prediction Center (CPC) morphing (CMORPH) technique, the CPC merged microwave technique, the Naval Research Laboratory (NRL) blended technique, and the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) technique. In addition, the Geosynchronous Operational Environmental Satellite (GOES) precipitation index (GPI) and the European Centre for Medium-Range Weather Forecasting (ECMWF) operational forecast model products are included for comparison. Surface reference data from the European radar network is used as ground truth, supported by the Global Precipitation Climatology Centre (GPCC) precipitation gauge analysis and gauge data over the United Kingdom. Measures of correlation, bias ratio, probability of detection, and false alarm ratio are used to evaluate the products. Results show that satellite products generally exhibit a seasonal cycle in correlation, bias ratio, probability of detection, and false alarm ratio, with poorer statistics during the winter. The ECMWF model also shows a seasonal cycle in the correlation, although the results are poorer during the summer, while the bias ratio, probability of detection, and false alarm ratio are consistent through all seasons. Importantly, all the satellite HRPPs underestimate precipitation over northwest Europe in all seasons.
Precipitation is a vital component of the global water and energy cycle and helps to regulate the climate system. The measurement of precipitation on a global scale is therefore crucial for a comprehensive understanding of the climate and of the hydrological cycle, as well as the proper management of water resources, agriculture, and disaster management (Kidd et al. 2009).
Conventional measurements through the use of rain (snow) gauges (“gauge” will denote precipitation, or rain gauge) provide a direct physical measurement of surface precipitation. However, such measurements are prone to errors arising from the actual measurement itself, such as wind effects and evaporation (Strangeways 2004), and the representativeness of the gauge measurements to the rainfall over the surrounding region is a major problem. Over the oceans very few gauges exist, while over land gauge networks can be sparse and unrepresentative, particularly, for example, in regions of varying topography. While weather radars can provide a spatial measure of precipitation, these too are prone to inaccuracies through imprecise backscatter: rain-intensity relationships, range effects, clutter, etc. (e.g., Jameson and Kostinski 2002). In addition, conventional high-resolution surface precipitation datasets are generally not comprehensively available in near–real time.
The measurement of precipitation on a global basis, particularly for near-real-time applications, must therefore rely upon satellite systems capable of providing global observations at regular intervals (see Kidd and Levizzani 2011; Kidd and Huffman 2011). Satellite systems that provide such observations are divided broadly into the geostationary (GEO) satellites and low earth-orbiting (LEO) satellites. The GEO satellites provide frequent, large-area observations using visible (Vis) and thermal infrared (IR) sensors capable of resolutions up to 1 and 4 km, respectively. IR data from these satellites are now routinely combined into a single global (60°N–60°S) IR product by the Climate Prediction Center (Janowiak et al. 2001).
The GEO satellites are complemented by the LEO satellites, typically in a near-polar orbit, and carry a variety of sensors including Vis/IR and passive microwave (PMW). The main sensors on LEO satellites for precipitation studies are the PMW instruments, which include the Special Sensor Microwave Imager (SSM/I) and the Special Sensor Microwave Imager/Sounder (SSMIS) on the Defense Meteorological Satellite Program (DMSP) satellites, the Microwave Humidity Sounder and the Advanced Microwave Sounding Unit B (AMSU-B) on the National Oceanographic and Atmospheric Administration (NOAA) missions, and the Tropical Rainfall Measuring Mission (TRMM) microwave imager (TMI) and the Advanced Scanning Microwave Radiometer (AMSR) on the Aqua satellite.
Vis/IR techniques based upon the presence and/or absence of clouds and the cloud characteristics typically exhibit low skill for individual events, although some, such as the Geostationary Operational Environmental Satellite (GOES) precipitation index (GPI; Arkin and Meisner 1987) have proven remarkably resilient. Other techniques exploit the availability of multichannel Vis/IR observations, such as the GOES Multispectral Rainfall Algorithm (GMSRA; Ba and Gruber 2001) and the Clouds–Aerosols–Precipitation Satellite Analysis Tool (CAPSAT; Lensky and Rosenfeld 2008), while artificial neural networks (ANN) have also been usefully exploited (e.g., Murao et al. 1993). Techniques to exploit the more direct nature of PMW observations have been developed based upon enhanced emissions from precipitation (e.g., Wilheit et al. 1991) or the scattering of upwelling radiation (e.g., Spencer et al. 1989). Multispectral techniques have been developed based upon both empirical calibrations and physical modeling (see Kidd et al. 1998; Bauer 2001). The dichotomy of Vis/IR (frequent, indirect) and passive microwave (infrequent, direct) retrievals have been emphasized in a number of algorithm intercomparisons; passive microwave techniques generally provide better instantaneous estimates of precipitation, while Vis/IR techniques generally provide better longer-term estimates (see Ebert et al. 2007).
The combination of Vis/IR and passive microwave observations offers the opportunity to combine good sampling (Vis/IR) with better retrievals (PMW) to provide not only better estimates, but improved temporal and spatial resolution estimates; early studies include those of Barrett et al. (1987) and Adler et al. (1993). Current combination schemes fall broadly into two main categories. The first relies upon the PMW to calibrate infrared observations and includes techniques such as the NRL-blended technique (NRLBLD; Turk and Miller 2005) and the passive microwave–infrared (PMIR) technique (Kidd et al. 2003), together with ANN techniques such as the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) technique (Sorooshian et al. 2000). The second category is the advection or morphing schemes (see Behrangi et al. 2010). These techniques, which include the Climate Prediction Center (CPC) morphing (CMORPH) technique (Joyce et al. 2004) and the Global Satellite Mapping of Precipitation (GSMaP; Kubota et al. 2007), are based on the fact that PMW estimates provide the best measure of precipitation, while the IR observations provide information about the movement of the precipitation system. Both of these schemes are capable of producing precipitation products at a nominal resolution of 3 hourly, 0.25° × 0.25°, although finer-resolution data products are available up to the resolution and sampling of the component datasets.
The verification of precipitation products is an integral part of the development and refinement of the retrieval techniques. Past intercomparison studies include the Global Precipitation Climatology Project (GPCP) Algorithm Intercomparison Program (see Ebert et al. 1996; Arkin and Xie 1994; Ebert and Manton 1998) and the National Aeronautics and Space Administration (NASA) Precipitation Intercomparison Projects (see Dodge and Goodman 1994; Barrett et al. 1994; Smith et al. 1998; Adler et al. 2001). In 2001, the International Precipitation Working Group (IPWG) was established to help coordinate the development and refinement of techniques (see Kidd et al. 2010; Huffman and Klepp 2011). An offshoot of the IPWG has been the Program to Evaluate High-Resolution Precipitation Products (PEHRPP; Turk et al. 2008), aimed at the new generation of high-resolution precipitation products and their intercomparison.
Recent intercomparisons of high-resolution precipitation products have included Sapiano and Arkin (2009), who compared the CMORPH, TRMM Multisatellite Precipitation Analysis (TMPA), NRLBLD, and PERSIANN products over the U.S. Southern Great Plains and over the Pacific Ocean. Over land, correlations between the products and gauges were generally better during the warm season, with CMORPH producing the highest correlations. All products had a small positive bias during winter, although in summer CMORPH, NRLBLD, and PERSIANN had ~100% positive bias, attributed to the overestimation of convective events. The TMPA product, being calibrated against the gauges, had nearly zero bias. Over the ocean, products generally underestimated the precipitation, particularly over the eastern Pacific region (east of 150°W). Although the CMORPH product produced high correlations—in terms of probability of detection (POD), false alarm ratio (FAR), and the Heidke skill score (HSS)—its success was less clear; for these statistics the TMPA was better than CMORPH over land, particularly in summer. Over the oceans west of 150°W the TMPA was best, with CMORPH marginally better east of 150°W.
Sohn et al. (2010) studied the validation of HRPPs over the Korean Peninsula (33°–39°N, 125°–130°E) using a dense network of 520 gauges. Four HRPP techniques were selected—namely TMPA, CMORPH, PERSIANN, and NRLBLD—together with the 2A12 product from the TMI. Results indicated that for accumulated rainfall, the TMPA was best, not least because it incorporates gauge information into the final product. The TMI product that is used as the input into the selected HRPP techniques underestimated the precipitation; Sohn et al. (2010) noted that this might be indicative of the poorer representation of midlatitude profiles in its database. Statistically, the CMORPH product produced the highest correlation (0.58) although with a negative bias; the TMPA correlation was good (0.47) with a slight positive bias. The CMORPH also produced the best POD score, although TMPA produced the best FAR score.
Both these studies show significant regional variations in performance of the different precipitation products that can be attributed to a number of sources such as sensitivity to light precipitation and environmental conditions (see Berg et al. 2002, 2006).
The use of numerical weather prediction (NWP) models for generation of global HRPPs has been limited to date primarily because of the computational requirements. However, NWP models occupy a unique position since they incorporate a large volume of diverse, mostly satellite-based observations that may also include cloud- and precipitation-affected regions. The global atmospheric models provide precipitation estimates based on a physically consistent environment that, despite many modeling uncertainties, can outperform products constructed from individual observation types (Ebert et al. 2007; Lu et al. 2010).
This paper addresses the performance of satellite precipitation products and the European Centre for Medium-Range Weather Forecasting (ECMWF) NWP model estimates over northwest Europe at the higher midlatitudes. The region covered by this study extends from 30° to 60°N, 20°W to 20°E encompassing a range of climatological zones from maritime to continental and from semiarid to temperate. Importantly, this region covers the extreme limits of current “operational” precipitation retrievals, where cold season precipitation estimation is still an open issue (Turk et al. 2008). The estimation of precipitation over these higher-latitude regions is somewhat challenging because of paucity in surface measurements, particularly in the Southern Hemisphere, the occurrence of low-intensity precipitation, and the increasing dominance of mixed or frozen precipitation. Importantly, the higher-latitude regions are particularly sensitive to changes in climates (Serreze et al. 2000; Dai et al. 1997), which requires comprehensive measurement of precipitation.
Although there are now a number of HRPPs available, not all can be included in this analysis. Key selection criteria included high spatial–temporal resolution and availability over the period of the study (March 2005–February 2011), coverage of the region of interest, and availability in near–real time. The last criterion was included to address the user community, who often need the best available product as soon as possible; thus the real-time version of the TMPA (3B42RT) was used in this analysis rather than the later gauge-adjusted product. Other HRPPs such as the GSMaP (Kubota et al. 2007), HydroEstimator (Scofield and Kuligowski 2003), and PMIR (Kidd et al. 2003) are excluded from the study since they did not meet all the necessary criteria. Consequently the CMORPH, NRLBLD, PERSIANN, and 3B42RT products were chosen, together with comparative data from the GPI, the ECMWF operational forecast model, surface radar, monthly Global Precipitation Climatology Centre (GPCC) gauge analysis, and U.K. national hourly gauge data.
a. Satellite precipitation products
1) CMORPH technique
The CMORPH technique was developed by Joyce et al. (2004) to exploit the fact that the retrievals of precipitation from PMW observations are better than those derived from IR techniques, although the IR data are capable of providing information on the movement (and development) of the precipitation systems. PMW observations from multiple satellite sensors—including the AMSU-B, SSM/I, TMI, and AMSR—are used to derive precipitation estimates. The SSM/I and TMI are used as the baseline estimates with the AMSU-B estimates scaled to fit using a frequency-distribution matching procedure. The microwave-only component of the CMORPH techniques is termed the CPC merged microwave product (CPCMMW). The IR-derived cloud motion vectors (see Purdom and Dills 1994) derived from the global IR composite (Janowiak et al. 2001) are used to propagate the PMW-derived rainfall field forward and backward in time, permitting linear interpolation between the rainfall field from the PMW following overpasses (i.e., in a Lagrangian framework) to estimate the rainfall field on half-hourly intervals.
2) NRL-blended technique
The NRLBLD technique is based upon temporally and spatially matched pixels from all available GEO Vis/IR and PMW observations and TRMM Precipitation Radar (PR) 2A25 data (Turk et al. 2010; Turk and Miller 2005). The baseline product is a global (60°N–60°S) 3-h accumulated precipitation product, updated every three hours. Collocated data from all operational GEO Vis/IR and LEO PMW imagers are used to dynamically build 2° × 2° latitude (lat)–longitude (lon) lookup tables of IR brightness temperature (Tb) to PMW-retrieved rain rates (RR). These lookup tables are then used to adjust VIS–IR data into instantaneous rain rates at each 3-h synoptic time (00, 03, … 21 UTC); each GEO-based rain rate value is weighted according to its time proximity to the nearest PMW overpass. Additional corrections are applied for upslope and downslope orographic effects and the growth–decay of the clouds (Vicente et al. 1998).
3) PERSIANN technique
PERSIANN exploits the ability of adaptive ANNs to extract and combine information from various sources—such as IR and PMW satellite observations, surface gauges, and radar—along with ancillary information such as topography (Hsu et al. 1997; Sorooshian et al. 2000). Input variables for the PERSIANN technique include the IR Tb, mean 3 × 3 IR Tb, 3 × 3 IR Tb standard deviation, mean 5 × 5 IR Tb, 5 × 5 IR Tb standard deviation, and surface type (land–sea–coast). Calibration of the technique was performed over Japan and Florida because of the availability of high-quality surface reference datasets, range of climate regimes, and ability to compare the results with existing studies. The PERSIANN technique showed that the ANN is capable of deriving good results even with sparse model updates, which is ideal for combining the sparse PMW estimates with the more frequent and regular IR observations.
4) TMPA 3B42RT
The TMPA comprises three components—the merged microwave product (3B40), the microwave-calibrated IR product (3B41), and the combined microwave–infrared product (3B42)—and is described by Huffman et al. (2007). The methodology builds upon earlier, coarser-resolution precipitation products (see Huffman et al. 2001) to provide a routinely generated and distributed product at 0.25° × 0.25° 3-h resolution. Precipitation estimates from a number of different satellite systems are combined, together with surface gauge data where available/applicable. The 3B42RT product is generated in near–real time and is usually available within 9 h of the satellite observations. Critically, only the real time (RT) version of the 3B42 product is available (with caveats) over 60°N–60°S—the calibrated product being restricted to 50°N–50°S.
The critical datasets in the TMPA include the TMI, SSM/I, AMSR, and AMSU-B/Microwave Humidity Sounder (MHS) and the GEO IR. Precipitation estimates from the PMW imagers are derived processed using the Goddard profiling (GPROF) algorithm (Kummerow et al. 1996; Olson et al. 1999), while those from PMW sounding data uses the ice water path technique described by Zhao and Weng (2002) and Weng et al. (2003). The Janowiak et al. (2001) IR data product is averaged to 0.25° × 0.25° before conversion to precipitation estimates using a probability matching against PMW estimates. The combination scheme assumes that if PMW estimates are available these are used; if not, PMW-calibrated IR estimates are used.
For the real-time product, the main calibrator is the TRMM 2A12 precipitation dataset. It should be noted that, unlike the other data products used in this analysis, the 3B42RT product represents an instantaneous snapshot of the precipitation at the synoptic hour, rather than an accumulation over a 3-h period. Therefore a discrepancy exists between the 3B42RT product and the other products; consequently in this study, separate time-matched radar data are used for the evaluation of the 3B42RT product.
In addition to the “standard” HRPP estimates, the GPI product was generated from the CPC global IR product so that comparison could be made with a “low-skill” precipitation product. IR Tbs were analyzed at the full 4-km resolution every 30 min and clouds with Tbs less than 235 K assigned a rain rate of 3 mm h−1. These were then averaged over the 0.25° × 0.25° resolution and accumulated to produce 3-hourly totals commensurate with the other HRPPs. The GPI is designed for use in regions where convective systems dominate, therefore use poleward of 40°N–S is not ideal; it is included here purely for reference and benchmarking against the other techniques.
b. Model data
Precipitation forecasts have been extracted from the operational ECMWF archive for the period January 2005–February 2011. The forecasts have been stored in 3-h intervals over the first two days of the forecast, although only the first day’s forecast is utilized in this study. In the 2005–11 period, the operational model, data assimilation, and employed observing system have been updated on average three times per year impacting the model’s accuracy, representation of the hydrological cycle, and, thus, precipitation forecasts. The spatial resolution and number of vertical levels were upgraded in February 2006 from 40 to 25 km and 60 to 91 model levels when the model ceiling was also raised from 0.1 to 0.01 hPa. Horizontal resolution was increased again in January 2010 from 25 to 16 km. One of the most important changes to the model physics that affected mostly tropical convection over land surfaces was implemented in November 2007. Following this change, verification of the model precipitation climate showed reduced biases over the central–eastern tropical Pacific and South America (see Jung et al. 2010).
NWP models tend to exhibit a spinup of precipitation between the initial state and the short- to medium-range forecast that are caused by the adjustments applied to the hydrological cycle during the analysis by moisture-sensitive observations and by the model converging back to its climatological state during the forecast (Bauer et al. 2010; Geer et al. 2010). On a global average, the ECMWF model has a wet bias; that is, mean rainfall increases along the forecast until it reaches a steadier state after 3–4 days. During the first day, global mean rainfall overshoots this state. This is because the model tends to enhance precipitation in areas where moisture has been added in the analysis and subsequently relaxes back to its preferred mean state. The mean difference between analysis and medium-range rainfall is about 30% and the overshooting amounts to 10% on the first day.
c. Surface reference datasets
1) Radar data
Surface radar data are used to provide the primary surface rainfall reference dataset and is collected from British Atmospheric Data Centre (BADC). The Met Office generates the Nimrod radar product (see Harrison et al. 2000) composited from operational C-band radars across Europe, primarily the United Kingdom, France, Germany, Belgium, and the Netherlands. Data are mapped to a polar-stereographic projection at a resolution of 5 km, quantified every mm h−1. Quality control is carried out by the contributing countries, often using real-time gauges to adjust the radar estimates. The radar composites are available every 15 min—the time of the image relating to the end time of the radar scans during the preceding 15 min.
2) Gauge data
Two gauge datasets were obtained: the GPCC monthly 0.5° × 0.5° resolution gauge analysis was obtained for comparison of monthly and annual products. This analysis is based upon all available hourly, daily, and monthly gauge data, interpolated where necessary to a 0.5° × 0.5° lat–lon grid. The interpolation is computed for station deviations from climatology before the final field is composed from the anomaly analysis and the climatology (Beck et al. 2005). For the European region, each 0.5° × 0.5° grid is the product of multiple gauge measurements. Over the U.K. region 3-hourly gauge measurements were generated from available hourly gauge data; this is included primarily as a check for the radar data. This gauge data were obtained from the Met Office Integrated Data Archive System (MIDAS) through the BADC. The quality-controlled gauge data were then mapped to the 0.25° × 0.25° grid used in this analysis and accumulated to provide 3-hourly totals.
3. Data processing
One aspect of these different precipitation products is that no standard data format exists (see Table 1); therefore, the first step in the processing was to organize the data into a common format. The satellite precipitation products were converted to 3-hourly 0.25° × 0.25° accumulations with a common start position at 180°W and 60°N. The ECMWF model data were extracted from the GRIB format, collated with the structure of the Gaussian grid, and remapped to the 0.25° × 0.25° grid. Because of changes in the resolution of the model over the period of this study, the earlier coarser-resolution product was interpolated to fill the lat–lon grid, while for later products the mean model product in each lat–lon grid box was calculated. These products were then subsetted for the region of interest, 30°–60°N, 40°W–40°E.
The radar data, with an original polar-stereographic projection at a 5-km resolution, was remapped to the 0.25° × 0.25° lat–lon grid through the use of a lookup table for the subsetted region of interest and accumulated over the 3-h period; a minimum of 10 out of the 12 possible 15-min radar products was required to compute each 3-hourly average to ensure temporal representativeness. The hourly gauge data, available as a lat–lon location and value, were mapped directly to the region of interest for each 3-h period. Although the gauge density is generally sufficient to fill all land-based grid boxes over the United Kingdom, an inverse-weighted interpolation scheme was used to assign values to unfilled adjacent grid boxes. The GPCC gauge data product (0.5° × 0.5° resolution) was included to verify the monthly and annual precipitation products.
Figure 1 shows the data availability for each of the datasets: each day produces up to eight samples per product (one every 3 h), although sometimes a product might be “available” but filled with missing data. Analysis of the datasets is done on a pairwise basis where both the individual HRPP and surface datasets are present; precise comparison between the different products is therefore not possible. The product–surface datasets were compared using four main statistics: the correlation, bias ratio, POD, and FAR, all of which have been used in previous intercomparison studies (e.g., Ebert et al. 2007). Spatial maps of the correlation and bias ratio statistics have been generated at each grid box so that variations in the performance of the product can be seen over the study region. The results are split into the four seasons: winter [December–February (DJF)], spring [March–May (MAM)], summer [June–August (JJA)], and autumn [September–November (SON)]. Each grid box has a maximum of 17 528 samples available, or about 4400 samples per season.
In this paper the radar is used as the main verification source primarily because of its superior spatial and temporal resolution, although it is not without errors. The full extent of the radar has been included in this study since the statistics are based upon each 0.25° grid rather than over the entire radar region; the results at the extremes of radar coverage should be treated with caution. The density of gauge data over Europe and the United Kingdom is generally very good when considering gauge products at coarser resolutions; the GPCC gauge product provides an overview of satellite–surface comparisons. However, at finer resolutions (e.g., 0.25°, 3-hourly) the number of available gauges is dramatically reduced and not necessarily contiguous across the region under study. A high-quality surface gauge dataset has been generated from hourly gauge data for a smaller, topographically homogeneous region of southern England (2°W–0°E and 51°–52.5°N), providing 48 0.25° × 0.25° resolution grid boxes.
a. Latitudinal variations
To emphasize the need to investigate regional variations in rainfall retrievals a latitudinal cross section of mean annual rainfall over land has been generated for 2009; this is shown in Fig. 2. The cross section represents the average daily precipitation for longitudes from 20°W to 20°E over the latitudes from 60°N to 35°S (since no land data are available south of 35°S). Over the tropics (30°N to 30°S) the agreement between the different satellite techniques is extremely strong. Agreement between the satellite-derived products and the ECMWF model is generally very strong, although the model suggests less precipitation on the northern side of the tropical rainfall belt. This offset is related to a systematic position bias of the intertopical convergence zone (ITCZ) over tropical western Africa that reduces with forecast range (Agusti-Panareda and Beljaars 2008). There is a large discrepancy between the GPCC gauge analysis and the satellite–model precipitation values, with the gauge data suggesting a third less rainfall between 10°N and 10°S. There are two possible explanations for this: first, the gauge density across central Africa is relatively sparse and therefore might not fully capture the true precipitation over this region. Second, the similarity among the satellite precipitation products is due to them being tuned to the TRMM 2A12 product.
For the region north of 30°N the ECMWF model and the GPCC gauge product agree reasonably well, while the satellite products underestimate the precipitation to varying degrees (with respect to ECMWF and GPCC). It is reasonable to assume that, given the gauge density across Europe, the gauge-derived precipitation values are correct, with a typical mean value of 2 mm day−1 between ~35° and 60°N. The CMORPH technique produces the least precipitation (between 0.5 and 1 mm day−1), while the NRLBLD and 3B42RT techniques produce the most (around 1.5 mm day−1). This hints at a problem common to all rainfall retrievals that is related to the lack of sensitivity to light precipitation (Klepp et al. 2003) and frozen precipitation over land surfaces. Most algorithms have been tuned with datasets representing warm and moderate conditions; consequently, midlatitude estimates are not retrieved well.
b. Annual precipitation
Maps of annual precipitation for each product have been generated for 2009, as shown in Figs. 3a–g; a minimum of 2000 (out of 2920) 3-h samples were required for the satellite estimates. The GPCC 0.5° × 0.5° gauge-derived precipitation product is used here as the surface reference dataset (Fig. 3h), although only available over the land area. It should be noted that despite the distribution of the gauges used in the GPCC analysis varying greatly across the globe, the gauge density over the European region is generally very good, with the CPCC analysis deemed “truth.”
The 3B42RT product (Fig. 3a) reveals a generally smooth distribution of the rainfall across Europe, although rainfall totals are lower than those indicated by the gauge data, particularly over the windward coasts of the western United Kingdom and northwest Spain and Portugal. The higher totals along the Pyrenees (Spain–France border) and over the Alps are subtly different; in the 3B42RT the maximum over the Pyrenees is over the central–eastern part of the range, while the GPCC product suggests a maximum in the west. Similarly, the structure of the precipitation over the Alps differs. Overall the 3B42RT can be seen to underestimate the precipitation with some evidence of problems along the coastlines.
The CMORPH technique (Fig. 3b) uses the CPCMMW (Fig. 3c) as the principal input dataset; consequently, these two products are somewhat similar. Both products produce less precipitation than the GPCC surface dataset and, more importantly, with differences in the regional distribution of precipitation. In particular, the precipitation along the coastal margins is subdued, and very noticeable in the CPCMMW. CPCMMW, and to a lesser extent CMORPH, shows an artificial maximum line of precipitation along the Greenwich meridian at 0° longitude as a result of a mapping issue. As with the 3B42RT, both of these products do not capture the higher precipitation totals along the windward coasts. The low values of precipitation over the Alps is also a processing artefact: the CPCMMW product masks the Alpine region as “no retrieval” because of the cold/snow surface, which is then interpreted in the CMORPH scheme as “no rain,” thus leading to an underestimation of precipitation in this region.
The NRLBLD product, shown in Fig. 3d, is similar to that of the 3B42RT although with generally lighter precipitation overall. The NRLBLD also reveals significant coastline artefacts with most of the coastal regions around the United Kingdom and northern Europe showing little or no precipitation. This lack of precipitation in these products is an artefact of the retrieval process; pixels over the coastline contain a fraction of sea and land surface that is either masked as no retrieval or is poorly estimated by the technique. The reliance of the PERSIANN technique (Fig. 3e) upon the IR data results in little or no coastal effect, although a line across the precipitation field at 10°W can be seen, presumably related to the IR calibration procedure. The overall PERSIANN precipitation field is rather bland with few features of maximum precipitation matching those of the GPCC gauge analysis.
The GPI product (Fig. 3f) reveals some artefacts probably stemming from the original IR dataset; there is a general area of maximum precipitation either side of the Greenwich meridian suggesting that when data from the Meteosat satellite is missing, these areas are in filled using data from the GOES-East satellite and the Meteosat satellite positioned over the Indian Ocean, albeit at extreme zenith angle. This would suggest that the zenith angle correction applied to the merged IR product needs further refinement. The generally higher totals over the eastern part of the region relates to an increase in the occurrence of convective-related cold cloud.
The ECMWF model product (Fig. 3g) is the best matched against the GPCC analysis; the product captures the orographic effects over the Pyrenees and Alps and along the windward coasts and along the north coast of Africa. Subtle differences are evident, such as along the length of Italy where the ECMWF product suggests higher precipitation than in the GPCC product.
c. Monthly precipitation
The mean monthly precipitation (shown as mm day−1) for the region over southern England (2°W–0°E, 51°–52.5°N) is shown in Fig. 4. The ECMWF model and the surface radar and gauge agree very well throughout the 6 years of the study. The satellite techniques, particularly the 3B42RT, CMORPH, CPCMMW, and NRLBLD all underestimate the precipitation, particularly during the wintertime, where most record mean daily rainfall values of 0.5–1.0 mm day−1 or less. In the other seasons, these techniques do reasonably well in tracking the month-to-month variations in precipitation. The PERSIANN technique is somewhat more variable, producing a couple of very noticeable spikes in precipitation: one during 2005 and another at the end of 2009; these are not precipitation related and are likely to be caused by the self calibration of the PERSIANN technique. The low-skill GPI technique produces high precipitation during the winters of 2006/07, 2007/08, and 2008/09, suggesting that the occurrence of cold cloud during the wintertime is significantly greater than during the summer, and therefore reducing the usefulness of this technique outside the tropics.
d. Temporal correlations
To analyze the performance of the precipitation products against surface radar data, spatial maps of correlation were calculated, each 0.25° × 0.25° grid box having a maximum of 17 528 samples over the 6 years of the study. The results for each product by season are shown in Fig. 5, covering the region 40°–60°N, 12°W–18°E and. The correlations essentially relate to how well changes in one product are reflected in the other, without regard for the magnitude.
Most of individual precipitation products have relatively low, seasonally variable correlations. The 3B42RT shows relatively poor performance during the wintertime (DJF) with correlations up to only 0.4 in the southwest of the region. This situation improves somewhat in the summer with correlations up to 0.6 across most of Europe. The correlations for CMORPH are higher with wintertime correlations up to 0.7 in the southwest of the region, rising to 0.8 during the summer months. One notable feature is the area of low correlations over the Alps during springtime, possibly linked to snow cover being misclassified as precipitation. Results for the CPCMMW product mirror those of CMORPH, although the correlations are typically 0.1 lower.
The NRLBLD produces generally poorer correlations than the 3B42RT, with the patterns of correlation being somewhat similar, but 0.1 lower. Correlations between PERSIANN and surface radar show the greatest range; during winter and autumn correlations are very low with large areas with values less than 0.1, although in summer correlations up to 0.6 can be observed over France. Perhaps surprisingly, the low-skill GPI technique produces higher correlations than PERSIANN during winter and autumn, and higher correlations than NRLBLD and 3B42RT during winter. This result can be attributed almost entirely to the overestimation of rain extent by the GPI during the winter months, which leads to a higher correlation coefficient; thus the seemly better performance is an artefact of the statistical analysis, rather than the greater skill of the technique.
The ECMWF model shows a different seasonal pattern; unlike the satellite-derived precipitation products, the ECMWF produces highest correlations during wintertime—typically 0.7–0.8. The correlations are lowest during summer, with correlations as low as 0.2–0.3 in the eastern part of the region. This suggests that the ECMWF model is good at forecasting large-scale precipitation, but less good at convective precipitation, which dominates the summertime precipitation regimes, especially over continental Europe.
e. Seasonal bias
Maps of bias ratio were generated to analyze the ability of the techniques to quantify precipitation; these are shown in Fig. 6. It should be noted that the surface radar network suffers from range effects causing the radars to underestimate rainfall at long range and overestimate it close to the radar locations. These effects are evident to a greater or lesser extent in Fig. 6, as indicated by the blue fringes in most of the bias-ratio maps.
All the satellite techniques, except for the GPI, generally underestimate the precipitation. This is particularly true during the cold seasons. The 3B42RT generally produces half of the radar-derived rainfall during winter, although this improves during summer, with an underestimation of ~10% over continental Europe. However, over windward coasts, even during the summer, there is still significant underestimation. Both CMORPH and CPCMMW seriously underestimate precipitation over land during winter, producing less than 25% of the precipitation identified by the radar. This may relate to the fact that most satellite products lack sensitivity to drizzle and frozen precipitation. During the summer these techniques improve somewhat but still underestimate rainfall away from continental Europe, with the NRLBLD showing very similar results. PERSIANN produces somewhat different results with more variation in the bias ratio across the region. During winter PERSIANN generally underestimates precipitation in the northwest of the region, but overestimates it in the east for all seasons except spring and in the south during winter. The GPI technique significantly overestimates precipitation during winter, echoing the results in Fig. 4, while underestimating precipitation during summer. Another important feature is the consistent discrepancy of ratios along coastlines as compared to land or ocean. This indicates systematic rainfall retrieval errors due to the treatment of land versus ocean surfaces in the algorithm’s training datasets.
The ECMWF model is the most consistent of the different precipitation products analyzed. Over the four seasons the areas of overestimation and underestimation are about equal, ranging from ~80% to 120% of the radar precipitation. The model is actually very good at identifying the range effects in the radar data, showing where the radar is overestimating (i.e., model is underestimating) and underestimating (i.e., model overestimating) precipitation.
f. Precipitation detection
Two further quantitative measures are used to assess the performance of the techniques against surface radar: POD and FAR. Figure 7 shows plots of FAR against POD, together with the critical success ratio (CSI; see Roebber 2009), for the 3-hourly 0.25° precipitation products for each season over the region in southern England; gauge data are included for reference. The 48 boxes that make up this region show the spread of results for each product; the variations in performance within each product being a result of the product itself and the accuracy and representativeness of the radar validation product. It should be noted that the rain/no-rain boundary is as defined in each dataset: that is, a value of zero in the precipitation product (and surface radar dataset) is interpreted as no rain and a nonzero (excluding “no data”) is taken as “rain.”
Results for winter are essentially split into four clusters: ECMWF with high POD (0.8–0.9) but relatively high FAR (~0.5), gauges with good POD (0.6–0.8) and low FAR (0.2–0.3), GPI with low POD (~0.3) and high FAR (0.3–0.5), and other products with very low POD (<0.1) and variable FAR (0.0–0.6). These reflect patterns in the spatial plots of correlation and bias ratio that show that during winter the ECMWF and GPI tend to overestimate precipitation (to a lesser or greater degree) while other techniques underestimate the precipitation. The curved lines of the CSI show that most of the satellite products have a CSI <0.1, although the GPI’s CSI lies between 0.2 and 0.3. These CSI results are somewhat lower than the ECMWF (0.4–0.5) and the gauges (0.5–0.6). Results during spring show a greater variation of FAR and POD in these groupings with the ECMWF and gauge still providing relatively high POD and moderate–low FAR, respectively. Among the satellite products the GPI POD has fallen to 0.1–0.2 while other techniques have generally improved their POD, particularly CMORPH with POD of up to 0.25, although with a large range in FAR (0.05–0.5). The CSI of the GPI during spring has reduced to around 0.15, while that of the CMORPH improved to 0.2.
Summer provides the best results overall for most satellite products with improved POD and lower FAR, with the ECMWF and gauge retaining their POD–FAR positions. CMORPH has POD up to 0.45, although the FAR still has a large range from 0.05 to 0.45. Other techniques are relatively well clustered around a POD of 0.2 with generally lower FAR, although PERSIANN has slightly higher FAR up to 0.35. CMORPH shows the best CSI performance amongst the satellite products, rising to between 0.3 and 0.4, although still below that of the ECMWF product. In autumn the POD of the satellite techniques falls with the highest, CMORPH, having POD of 0.3, but generally clustered around a POD of 0.1. PERSIANN has a slightly higher POD, although with a high FAR. The CSIs of the satellite products have fallen back, with the best, CMORPH, now averaging between 0.2 and 0.3.
This study has highlighted the challenges of mid- to high-latitude retrievals resulting from precipitation type and intensities, as well as surface background problems, particularly during winter. In terms of correlation, the satellite products show a seasonal cycle with relatively good results during the summer but relatively poor during winter. Some of this can be attributed to the inability of the techniques to retrieve precipitation during winter, either through a greater occurrence of low-intensity precipitation, or because of cold surface backgrounds affecting the PMW retrievals. Conversely, the ECMWF model shows the opposite seasonal cycle: good correlations occur during the cooler months with poorer correlations in summer. This behavior is indicative of problems representing convective processes in the numerical models. Among the satellite techniques, CMORPH provided the best correlation scores overall, mirroring the results of Sapiano and Arkin (2009).
A noticeable feature of all the mainstream HRPPs is that they underestimate precipitation (except at the extremes of radar range, which is a radar artefact). CMORPH and CPCMMW in particular show significant underestimation during winter. These findings echo those of Sohn et al. (2010), who found satellite algorithms underestimated precipitation over the Korean Peninsula. The GPI has the greatest annual range, greatly overestimating precipitation during winter while underestimating during summer. Since the GPI is a measure of cold (<235 K) cloud occurrence, it is perhaps not surprising that, at the midlatitudes where large-scale precipitation systems dominate, cold clouds occur more often during the winter than in the summer. Meanwhile, the ECMWF model shows relatively consistent performance over all seasons; in fact, the regions of over- and underestimation are related more to the radar network than features within the model product.
Analysis of POD and FAR show that even with good radar and gauge datasets a “perfect” score is rarely possible; the POD/FAR of the gauge–radar comparison shows the “best possible” situation. There is usually a trade-off between POD and FAR: improving the POD is likely to increase the FAR and thus not necessarily improve the overall product unless there is an increase in the overall skill. While during winter the GPI has the best POD among the satellite techniques, it also has consistently the highest FAR. This suggests that the GPI overestimates occurrence of precipitation and is therefore not a good discriminator of the rain/no-rain boundary. The “best” product during the summer is CMORPH with a reasonably good POD score, although with generally higher FAR compared to other techniques. In terms of CSI, the ECMWF product provides the most consistent product across all seasons with CSIs of between 0.4 and 0.5. This exceeds the best performing satellite technique, CMORPH, even in the most favorable season.
The results reinforce the findings of earlier studies. Sapiano and Arkin (2009) suggested that the overestimation of satellite techniques during summer was due to the preponderance of convective systems. Analysis of the spatial maps of the ratio suggests that there is some overestimation in the satellite HRPPs over Germany; this region experiences more convective regimes during the summer. Underestimation by the satellite products was also found by Sohn et al. (2010) who noted that the GPROF algorithm, against which the HRPP techniques used here are calibrated, was instrumental in the underestimation. The underestimation found by Sohn et al. (2010) is also evident in the northwest European region. It is therefore evident that while the 2A12 algorithm is well tuned to tropical precipitation, it lacks the representativeness of midlatitude precipitation profiles in the retrieval database to generate realistic precipitation in extratropical regions. This and other studies have shown that the success of merged products is very much dependent upon the component products used in their techniques.
This region is important for a number of reasons: at present, it is at the latitudinal extremes of current satellite precipitation products, and where there are known issues in the retrieval of precipitation. Furthermore, the expansion of satellite algorithms poleward of these latitudes is critical, yet challenging. This can be attributed to three main factors: first, the prevalence of low-intensity precipitation events; second, the increasing dominance of frozen precipitation; and third, contamination from cold/frozen surface backgrounds. These issues need to be addressed as a matter of urgency and relate directly to algorithm development. The retrieval of frozen precipitation is under development with some encouraging results being published exploiting the LEO high-frequency PMW datasets (Surussavadee and Staelin 2010). In addition, PMW sounding instruments are relatively insensitive to surface emissions and therefore are essentially immune to cold/frozen background issues. Critically, for global precipitation measurements these regions need to be incorporated into the precipitation products.
This study also highlights the need for caution when assessing the statistical performance of precipitation products. Techniques that overestimate precipitation occurrence tend to produce high POD, while those that underestimate precipitation occurrence will produce low FAR: these alone should not be used as a measure of success. More problematic is the interpretation of correlation statistics. The correlation between the satellite and surface datasets is very much dependent upon the ratio of no-rain–rain area; if there is a high occurrence (or extent) of precipitation, the correlations will tend to be higher than for low occurrences of precipitation. Therefore, techniques that overestimate precipitation are likely to have a higher correlation than those that underestimate precipitation.
It should be noted that the products evaluated here are not truly independent since they share common observations through the use of the GEO and LEO datasets and, perhaps more importantly, CMORPH, CPCMMW, NRLBLD, and PERSIANN are tuned to the TRMM datasets and explicitly use the 2A12 precipitation product as the reference dataset. Thus retrieval errors in the 2A12 product tend to be propagated through to the merged products. The results shown here highlight the necessity to address current deficiencies in retrieval techniques during the cold season. Although this study has limited oceanic extent, it is clear that the underestimation is not just limited to land areas, but also extends over the adjacent ocean regions. In addition, many of these techniques exhibit artefacts relating to mapping, calibration regions, or coastal boundaries; these artefacts need addressing to ensure a complete and accurate data product is available.
This study has also shown that the global NWP model forecasts, despite being weakly constrained by cloud and precipitation observations, are capable of producing consistent and accurate precipitation estimates, particularly in midlatitude winter conditions. This underlines the importance of capturing the entire atmospheric state in a physically consistent way as a condition for also producing accurate rainfall estimates as well as exploiting sufficient observations and density of measurements necessary to capture the precipitation at appropriate scales.
The authors thank the algorithm developers for the provision of the precipitation products, the NASA Goddard Space Flight Center Distributed Active Archive Center for provision of the global IR data sets for generation of the GPI, the ECMWF for the provision of precipitation forecasts, the U.K. BADC and Met Office for the provision of the surface radar data and hourly gauge data, and the GPCC for provision of their gauge analysis. The authors would also like to acknowledge the contribution made by the members of the International Precipitation Working Group, particularly those active in the Project for the Evaluation High-Resolution Precipitation Products.