An extensive evaluation of nine global-scale high-resolution satellite-based rainfall (SBR) products is performed using a minimum of 6 years (within the period of 2000–13) of reference rainfall data derived from rain gauge networks in nine mountainous regions across the globe. The SBR products are compared to a recently released global reanalysis dataset from the European Centre for Medium-Range Weather Forecasts (ECMWF). The study areas include the eastern Italian Alps, the Swiss Alps, the western Black Sea of Turkey, the French Cévennes, the Peruvian Andes, the Colombian Andes, the Himalayas over Nepal, the Blue Nile in East Africa, Taiwan, and the U.S. Rocky Mountains. Evaluation is performed at annual, monthly, and daily time scales and 0.25° spatial resolution. The SBR datasets are based on the following retrieval algorithms: Tropical Rainfall Measuring Mission Multisatellite Precipitation Analysis (TMPA), the NOAA/Climate Prediction Center morphing technique (CMORPH), Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks (PERSIANN), and Global Satellite Mapping of Precipitation (GSMaP). SBR products are categorized into those that include gauge adjustment versus unadjusted. Results show that performance of SBR is highly dependent on the rainfall variability. Many SBR products usually underestimate wet season and overestimate dry season precipitation. The performance of gauge adjustment to the SBR products varies by region and depends greatly on the representativeness of the rain gauge network.
Rainfall is a fundamental component of the global water cycle. To understand and manage water systems under a changing climate and meet an increasing demand for water, a quantitative understanding of the hydrological cycle is important on a regional to global scale. Hence, quality and availability of rainfall estimates affects accuracy and reliability of hydrological studies.
Rainfall measurements can be conducted at the ground surface (such as by rain gauge and radar networks) or with satellite sensors. Rain gauges provide direct physical measurement of surface rainfall; however, they are susceptible to uncertainty sources (such as the size of the collector, evaporative loss, wind, siting of gauges, etc.; Michaelides et al. 2009; Strangeways 2011; Groisman and Legates 1994). Also, spatial representation of rainfall variability from rain gauge measurements is limited because of the sparseness of gauge networks over remote parts of the world, such as mountainous regions. On the other hand, weather radar networks provide rainfall measurement in high spatial and temporal resolutions, but they are also susceptible to certain limitations and errors, especially over mountainous regions where accuracy tends to degrade because of beam overshooting, range effects, beam blockage, and brightband effects (Young et al. 2000; Krajewski and Smith 2002). In addition, establishing and maintaining ground-based gauge or radar networks is often cost prohibitive, especially over remote parts of the world or in regions with limiting financial resources. Thus, observations are generally available over lowlands, leaving an observational gap in global precipitation data over complex terrain where rainfall is typically characterized by high spatiotemporal variability. This scarcity of ground-based observations affects the reliability of hydrological modeling and water resources assessment studies over complex terrain (Barros et al. 2004; Immerzeel et al. 2014; Salerno et al. 2015).
Difficulties in representation of spatial rainfall variability from ground-based observations highlights the need to use satellite-based rainfall (SBR) datasets because of their ability to represent the space–time variability of rainfall with quasi-global coverage. However, SBR products are susceptible to significant uncertainty that necessitates the use of error correction procedures based upon more accurate rainfall measurements from data-rich ground-validation case studies. Satellite measurements are based on one or more remotely sensed characteristics of clouds, such as reflectivity [visible (VIS)], cloud-top temperature [infrared (IR) imagery], or from the scattering/emission effects of raindrops or ice particles [passive microwave (PMW) radiation; Sapiano and Arkin 2009; Kidd and Levizzani 2011]. Since VIS and IR sensors are on board geostationary satellites, they provide data at fine temporal scales (Sapiano and Arkin 2009). However, the link between cloud-top temperature and surface precipitation is indirect (Joyce et al. 2004; Anagnostou et al. 2010). PMW sensors are on polar-orbiting satellites; therefore, their temporal resolution is coarse, resulting in large sampling errors for short-term rainfall events. Rainfall estimates from PMW are more accurate since observations are related to hydrometeor content present within the atmospheric column. Since all these approaches are indirect measurements, they have strengths and weaknesses. To take advantage of these strengths, the most recent algorithms combine IR and PMW observations (Ebert et al. 2007). As a result, global-scale, high-resolution SBR products are now available. Among these products are the Tropical Rainfall Measuring Mission (TRMM) Multisatellite Precipitation Analysis (TMPA; no longer operational) near-real-time (3B42RT; hereafter referred to as TMPART) and gauge-adjusted, version 7 (3B42v7; hereafter referred to as TMPAV7), products produced at the National Aeronautics and Space Administration (NASA) Goddard Space Flight Center (GSFC; Huffman et al. 2007, 2010; Huffman 2015); the National Oceanic and Atmospheric Administration (NOAA) Climate Prediction Center (CPC) morphing technique (CMORPH; Joyce et al. 2004), which has both satellite-only and gauge-adjusted products (Xie et al. 2011); the Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks (PERSIANN), which has both satellite-only and gauge-adjusted products (Sorooshian et al. 2000); and the satellite-only Global Satellite Mapping of Precipitation Microwave-IR Combined Product (GSMaP-MVK) and gauge-adjusted (GSMaP-Gauge) datasets produced at the Earth Observation Research Center (EORC) of the Japan Aerospace Exploration Agency (JAXA; Kubota et al. 2007; Ushio et al. 2013). Even though SBR estimates contain considerable errors, the ongoing improvements and new satellite missions such as the Global Precipitation Measurement (GPM) continuation of the TRMM make them potentially useful for hydrologic modeling in large basins (Yilmaz et al. 2005; Su et al. 2008). In the post-GPM era (since 2014), the two TMPA products have transitioned to a new integrated product, Integrated Multisatellite Retrievals for GPM (IMERG; Huffman et al. 2015), which contributes higher-resolution near-real-time and gauge-adjusted global precipitation estimates.
Studies have shown that the performance of SBR products largely depends on the hydroclimatic characteristics of the region (Yilmaz et al. 2005). Therefore, evaluation of these products over multiple regions will provide comprehensive error characteristics to the algorithm developers. SBR products have been evaluated for different continental regimes in the past two decades (Petty and Krajewski 1996; Anagnostou 2004; Tian et al. 2007). Regional studies have been conducted over the continental United States (CONUS; McCollum et al. 2002; Gottschalck et al. 2005; Hossain and Huffman 2008; Anagnostou et al. 2010), South America (Su et al. 2008; Dinku et al. 2010; Scheel et al. 2011), Europe (Stampoulis and Anagnostou 2012; Mei et al. 2014; Derin and Yilmaz 2014), Africa (Dinku et al. 2007, 2008; Hirpa et al. 2010; Thiemig et al. 2012; Habib et al. 2012; Milewski et al. 2015), and Asia (Chen et al. 2013; Yong et al. 2013). Derin and Yilmaz (2014) found that SBR products have difficulties in representing the rainfall precipitation gradient normal to the orography, and in general, they observed an underestimation along the windward region and an overestimation on the leeward side of the mountains. Stampoulis and Anagnostou (2012) showed that SBR products underestimate heavy precipitation occurring over higher elevations of the Alps, especially during the fall season. Moreover, they observed a seasonal dependence of the SBR products’ performances. Thiemig et al. (2012), who evaluated six SBR products over four African river basins, found a superior performance over the tropical wet and dry zones relative to semiarid or mountainous regions. Although there are several studies on SBR error analysis, only Ebert et al. (2007) evaluated SBR datasets over multiple regions of the earth. They evaluated seven SBR products, three global numerical weather prediction model outputs, and one regional model over the CONUS, Australia, and northwestern Europe. They showed that SBR performances are highly dependent on the rainfall variability and that they are more accurate when the precipitation regime is characterized by deep convection. Comprehensive error analysis of SBR products across different mountainous regions is needed to evaluate the relative performance characteristics of the available satellite precipitation datasets in order to develop error correction procedures globally.
In this study, we address this goal by evaluating the major global SBR products over multiple regions of the earth characterized by complex terrain. The evaluated SBR products include TMPAV7, TMPART, TMPA 3B42 climatological calibration algorithm (CCA; hereafter referred to as TMPACCA), CMORPH (both satellite-only and gauge-adjusted versions), PERSIANN (both satellite-only and gauge-adjusted versions), GSMaP-MVK, GSMaP-Gauge, and the recently released ECMWF interim reanalysis (ERA-Interim) product (Weedon et al. 2011; Dee et al. 2011). The products are evaluated over nine mountainous regions: the Italian and Swiss Alps, the French Cévennes, the western Black Sea region of Turkey, the Colombian Andes, the Peruvian Andes, the U.S. Rocky Mountains, the upper Blue Nile, western Taiwan, and the Himalayas over Nepal. Evaluation of the products over these regions is conducted by taking averages of rain gauges over corresponding 0.25° SBR grid cells. The performance of SBR products is assessed using statistical measures and visual comparison methods. The paper is organized as follows: the details of the study area and datasets are provided in section 2. Evaluation methodology is presented in section 3. The discussion of results is presented in section 4, and the conclusions and recommendations are summarized in section 5.
2. Study areas and data
a. Study domains and rain gauge datasets
As mentioned above, the study areas involve nine data-rich mountainous ground-validation sites (Fig. 1). Every region has a different rain gauge network density and time periods of data record (Fig. 2). To make this study consistent across different domains, ground-validation datasets are limited to daily time resolution in which in situ data records span a minimum of 6 years within the period of 2000–13 (Table 1). To represent complex terrain observations, only gauge locations over moderate to high elevations are included. It should be noted that since no universal quality control has been applied to the data, the quality of the rain gauge data most likely varies a lot and therefore affects the general results.
The Taiwan study region focuses on the Tsengwen basin, which is located in the southwest of the island. The highest elevation in the watershed is 2609 m. The Taiwan study region could conceptually be separated into two climate regions on opposite sides of latitude 23.5°N, which crosses the northern part of the Tsengwen watershed. It is characterized by a tropical climate with an average temperature of 18°C in winter. The annual precipitation can exceed 3300 mm and is unevenly distributed throughout the year. Most of the precipitation occurs during the East Asian rainy season (plum rain) in April as well as from typhoons from May to September. The remaining months represent the dry period exhibiting low precipitation.
The Swiss Alps are located in the midlatitude region (46°N) in western Europe. Westerly winds are dominant and explain the larger rain amount on the western foothills of the Alps. Shadow effects result in drier inner alpine valleys, while southern circulation from the Mediterranean generates intense precipitation on the southern side of the Alps (southeast of Switzerland and north of Italy) in autumn. Annual precipitation amounts vary roughly from 500 to 3000 mm. Given the typical altitude range of the freezing level, snowfall represents a significant proportion of precipitation over a large part of the alpine region (above 1500 m MSL).
The Blue Nile study area is the upper Blue Nile basin, which is located in the mountain range of the Great Horn of Africa. Rain gauge density in this study area is higher than the typical coverage in East Africa. There are 70 daily rain gauge stations over three subbasins representing a total catchment area of 46 202 km2. Because of the widely varying topography of the region, altitude strongly influences climate from cool highlands to hot deserts. Most precipitation occurs in the wet season (June–September), which is driven by the intertropical convergence zone that drives the summer monsoon.
The study area for the Italian Alps is the upper Adige River basin closed at Bronzolo, with an area of around 7000 km2. This region is characterized by steep topographic gradients with elevation ranging from 200 to approximately 3900 m MSL, with a mean elevation of about 1800 m MSL. Precipitation in the region is primarily attributed to mesoscale convective systems during summer to early fall and frontal systems during fall and early winter (Frei and Schär 1998; Norbiato et al. 2009). Precipitation generally occurs as snowfall, even at low altitudes, in the period of November–April. The precipitation monthly distribution shows a peak in early summer and a second one during fall. Spatial distribution of the station mean annual precipitation, computed over the study period, ranges from 692 to 912 mm. Low precipitation amounts are mainly due to the sheltering effect of the mountainous ranges to both the southerly and northerly winds. It should be noted that Italian Alps and Swiss Alps are combined into one region referred to as the Alps.
Validation data for the French Cévennes region are available from Météo‐France, Électricité de France, and the Service de Prévision des Crues du Grand‐Delta in the framework of the Cévennes-Vivarais Mediterranean Hydrometeorological Observatory. By using 600 rain gauges over the 2007–12 period, kriging-based area interpolation maps of the region were created (Delrieu et al. 2014). Rain gauge elevations range between 140 and 1567 m where precipitation increase can be observed from Fig. 3b. The region has a largely Mediterranean climate. Annual rainfall totals range from about 500 mm near the sea to 2000 mm over the mountains. In the Mediterranean part of the region, temperatures are mild in winter (e.g., average maximum temperature is 11°C in Nîmes in January) and hot in summer (31°C in July). The rainfall is low in summer (<30 mm on average in Nîmes in July) and higher during the fall season (~120 mm on average in October). In the northern part of the region, the climate is mitigated by oceanic conditions, and rainfall is more regular. On the northwestern mountainous part, a Mediterranean climate is observed, with more precipitation and colder winters.
The Turkey study area is located in the western Black Sea region. This region is located south of the mountains and is characterized by a dry/subhumid continental climate (Derin and Yilmaz 2014). The study area in general is impacted by polar air masses with continental origin of cold Siberian high and maritime origin of Iceland low in the winter and by subtropical air masses (Azores high and part of the Pakistan low) in the summer. When the Siberian high crosses the Black Sea and approaches the northern coasts of Turkey, cold and dry air turn into a maritime continental air mass because of the acquired moisture content (Yucel and Onen 2014). Extension of the mountains and effects of the Black Sea determine the climate types of the region. The mountains, which are parallel to the shoreline, restrict transfer of precipitation to inland zones (south of the mountains), hence decreasing the temperature and precipitation (mean annual value as low as 400 mm south of the mountains). For this reason, the inland zones observe a more continental climate. The inland region has an average of 70 rainy days per year, with the majority of precipitation occurring in the fall and winter seasons (Yucel and Onen 2014).
The Himalayan study area is located in Nepal over the eastern part of the central Himalayas. It is based on five rain gauges reporting data for the period 2000–13. The gauge elevations ranged between 2660 and 5600 m. Mean annual precipitation of all rain gauges is 698 mm, while the maximum recorded daily rainfall in the study period is 88 mm. The region is located in the subtropical zone and is influenced by the monsoon system (Salerno et al. 2015). Moreover, local circulation is dominated by a system of mountain and valley breezes. Orographic effects on precipitation have been studied over this region (Singh and Kumar 1997; Ichiyanagi et al. 2007; Salerno et al. 2015), and it was found that precipitation increases with altitude below 2000 m MSL and decreases for elevations above 2000 m MSL, which can also be seen from Fig. 3i.
The U.S. Rocky Mountains region is located over western North America. Validation data for the U.S. Rocky Mountains are based on the CPC unified gauge-based analysis of daily precipitation over the CONUS. This dataset is created by combining all information sources available at CPC and taking advantage of the optimal interpolation objective analysis technique at 0.25° daily resolution (Chen et al. 2008). The mountainous regions within CONUS were extracted by removing grid cells with elevations less than 1500 m and without rain gauges. Hence, 319 grid cells were extracted for analysis over the U.S. Rocky Mountains reporting data for the period of 2000–13. Gridcell elevations ranged between 1500 and 3550 m. Mean annual precipitation of all grid cells is 522 mm, while the maximum reported area-average daily rainfall is 83 mm. The region is characterized by a highland (alpine) climate.
The Peruvian Andes study area consists of 147 rain gauges with measurements for the period of 2000–13 and elevations ranging between 2000 and 5020 m. Rain gauges are maintained by the national agency Servicio Nacional de Meteorología e Hidrología del Perú (SENAMHI). The orographic effect on precipitation can be seen from Fig. 3c. Mean annual precipitation of all rain gauges in the region is 713 mm, while the maximum reported area-average daily rainfall in the study period is 540 mm. Precipitation patterns are controlled by the interaction of synoptic-scale atmospheric currents and the complex Andean topography (Manz et al. 2016). The Pacific coastline is arid and experiences precipitation less than 100 mm yr−1 as a result of the cold von Humboldt current (Manz et al. 2016), which increases toward the Andes mountainous range because of strong topographic gradients that result in pronounced orographic effects (Espinoza et al. 2015; Espinoza Villar et al. 2009; Bookhagen and Strecker 2008).
The Colombian Andes study area consists of 98 rain gauges with measurements for the period of 2000–12 and elevations ranging between 286 and 3666 m. Rain gauges are maintained by the national agency Instituto de Hidrología, Meteorología y Estudios Ambientales (IDEAM). The mean annual precipitation based on all rain gauges in the region is 1130 mm, while the maximum reported area-average daily rainfall in the study period is 439 mm. The intra-Andean valleys create great variability in this region where annual rainfall ranges from less than 1000 to about 6000 mm over the Colombian Pacific coast. Precipitation in this area is represented by a complex interaction of low-level jet stream with local topography and the intertropical convergence zone (ITCZ). Central Colombia experiences a bimodal annual cycle of precipitation with high rain seasons in the April–May and September–November months resulting from the double passage of the ITCZ over the region (Álvarez-Villa et al. 2011).
b. Satellite-based precipitation products
We examined nine SBR products and one global reanalysis product. Table 2 summarizes the satellite products and the ECMWF global reanalysis dataset. The examined SBR products are based on different techniques applied to PMW and IR measurements. CMORPH derives precipitation estimates from PMW-only satellite estimates, which are propagated by motion vectors derived from IR data (Joyce et al. 2004). PERSIANN is IR based and uses a coincident PMW-calibrated neural network technique to relate IR observations to rainfall estimates (Sorooshian et al. 2000). GSMaP is very similar to CMORPH, where precipitation estimates are derived from PMW estimates and propagated by using IR estimates. The main difference between CMORPH and GSMaP is that GSMaP uses IR estimates at times when PMW estimates are not present (Kubota et al. 2007). TMPA derives precipitation by combining IR and PMW. First, PMW estimates are calibrated and combined. Next, IR estimates are generated using a PMW calibration procedure, and finally, these two datasets are combined in 3-hourly time windows.
All products conduct gauge-based corrections to the satellite-only rainfall measurements. TMPAV7 conducts gauge adjustments at monthly time scales by using Global Precipitation Climatology Project (GPCP) monthly rain gauge analysis developed by the Global Precipitation Clımatology Centre (GPCC; Rudolf 1993) and Climate Anomaly Monitoring system (CAMS) monthly rain gauge analysis developed by the CPC (Xie and Arkin 1996). CMORPH conducts gauge adjustment at daily time scales by using approximately 30 000 stations that are interpolated by using the optimum interpolation technique with orographic correction on 0.125° latitude/longitude, then averaged on 0.5° latitude/longitude over globe. GSMaP conducts gauge adjustment at daily time scales by using the CPC global gauge data analysis by Xie et al. (2007) and Chen et al. (2008).
In addition to comparing different products, in order to understand the performance of gauge adjustment to the satellite-only products, a comparison between satellite-only products and gauge-adjusted products is performed. Moreover, TMPART has undergone another adjustment, which is based on a climatological gauge-based calibration algorithm. The main difference between TMPACCA and TMPAV7 is that the former does not require month-to-month gauge observations. Yong et al. (2013) evaluated the performance of TMPART and TMPACCA over two basins in China (one located at lower latitudes and one at higher latitudes) and concluded that the performance of TMPACCA over low latitudes is superior when compared to the performance over higher latitudes primarily in winter. The ERA-Interim precipitation product corrected by a global gauge dataset is also utilized in this study (Dee et al. 2011).
3. Evaluation methodology
The primary objective of this study is to evaluate the performance of various SBR products over complex terrain using the numerous rain gauge networks gathered over the nine different regions. All datasets used in this study were scaled at the same spatial (0.25°) grid resolution and at daily temporal resolution. Gauges were interpolated at the 0.25° grid cells by averaging gauge measurements within common satellite grid cells. Satellite grid cells with no overlapping gauges were excluded in this analysis. Evaluation was conducted by creating pairs of the average rain gauge rainfall values and the SBR product over each study region. It is noted that the statistical analysis disregarded cold seasons for the Alps, Turkey, and the U.S. Rocky Mountain regions (November–April) and for the Himalayas (September–May) to avoid effects due to snow precipitation. The evaluations are conducted at daily, monthly, and annual time scales.
The agreements between different products were investigated using quantitative, categorical, and graphical measures. The quantitative statistics include mean relative error (MRE), relative central root-mean-square error (CRMSE), and correlation coefficient:
where S and G represent SBR products and rain gauge estimates, respectively, and M represents total number of days. The MRE is an error metric measuring the systematic error component with values greater or smaller than zero indicating over- or underestimation, respectively. CRMSE is a metric measuring the random component of error, as bias has been removed. Correlation coefficient is an indicator of the temporal similarity between rain gauge and SBR.
The contingency-table-based categorical statistics measure the daily rain-detection capability and include normalized missed rainfall volume (NMRV) and normalized false alarm satellite rainfall volume (NFASRV) error metrics:
The NMRV metric quantifies the missed rainfall volume by SBR products normalized by the total reference rainfall volume throughout the study period. The NFASRV metric measures falsely detected rainfall volume by SBR products normalized by the total reference rainfall volume during the same period of time. A good performance of the SBR would imply low systematic error (i.e., MRE) accompanied by low magnitude of the random error component (i.e., CRMSE), high covariation (i.e., correlation), and accurate rainfall area detection (i.e., low NMRV and NFASRV).
a. Annual comparisons
Figure 4 shows mean annual average precipitation over all regions, where green represents the ECMWF reanalysis product, blue represents the gauge-adjusted SBR products (TMPAV7, TMPACCA, CMCT, PNCT, and GSCT), red represents the satellite-only SBR products (TMPART, CM, PN, and GS), and the red thick line represents the mean annual average rain gauge precipitation values. Overall, ECMWF is shown to closely follow rain gauge in terms of annual averages, while satellite-only SBR products tend to underestimate the reference magnitudes, and the performance of these products improves with gauge adjustment. The performance of TMPACCA relative to TMPART varies considerably, exhibiting improvement only in a few cases (Taiwan and the Colombian Andes), while for most cases TMPACCA seems to elevate TMPART estimates to overestimation (Fig. 4). Over Blue Nile (Fig. 4g), all products underestimate annual precipitation. In this region the performance of PERSIANN is the lowest while CMORPH outperforms all products, which is consistent with findings by other studies (Dinku et al. 2007, 2008; Hirpa et al. 2010; Romilly and Gebremichael 2011; Thiemig et al. 2012). This is because IR-based rainfall algorithms have limitations over mountainous regions in East Africa, while PMW is more physically based and free of the cold surface of snow effects. Rain gauges over Turkey are located inland, a region primarily characterized by rain shadow effects. Most of the gauge-corrected SBR products over this region overestimate the rain gauge values (Fig. 4e). This is consistent with the study of Derin and Yilmaz (2014), which showed that inland region precipitation is overestimated by SBR products. It is noted that gauge adjustments over Turkey and the Colombian Andes do not improve the performance of satellite products. Over the French Cévennes, Taiwan, the Peruvian Andes, and the Alps (Figs. 4a–d), orographic enhancement is observed. PMW-based SBR products (CMORPH and GSMaP) over these regions exhibit underestimation. As has been stated in past studies (e.g., Dinku et al. 2007), warm orographic rain does not produce ice aloft, and since PMW retrievals are based on scattering by ice aloft, this causes an underestimation of surface rainfall.
b. Seasonal comparisons
Figure 5 compares the performances of SBR products in capturing the temporal dynamics of mean monthly precipitation. The thick black line represents the area-average rain gauge precipitation, the green solid line represents the corresponding reanalysis product, the blue lines represent the gauge-corrected SBR products, and the red lines represent the unadjusted SBR products. The SBR products are differentiated using different symbols, which are consistently used across this analysis. The majority of the SBR products and the reanalysis dataset sufficiently capture the seasonal trends in all regions; the ECMWF reanalysis in particular closely follows rain gauge values at the monthly time scale, with the only exception being the Alps. The gauge-adjusted SBR products closely follow the observations, with the exception of Turkey and the Colombian Andes (Figs. 5e,f), which tend to overestimate the gauge observations. The Blue Nile region, which displays only the wet period months, demonstrates a close agreement for all products and reference datasets in terms of monthly variability, but a consistent underestimation of the SBR and reanalysis exists relative to the gauge dataset. The best monthly estimates over the region are from the gauge-corrected CMORPH product, which is followed by TMPAV7. The worst performance is by PERSIANN, which is confirmed by the studies of Thiemig et al. (2012), Dinku et al. (2007, 2008), and Hirpa et al. (2010). Over the Peruvian Andes region, SBR products underestimate relative to the gauge precipitation during the wet season (December–February) and overestimate during the dry season (Fig. 5c). Over Taiwan, there is a distinct improvement in the performance of gauge-corrected SBR products relative to their unadjusted counterparts. Over French Cévennes, the climatologically adjusted TMPACCA decreases the performance of the satellite-only TMPART product; it is especially unable to represent the higher rainfall amounts during cold season (Fig. 5b). Over Taiwan SBR products fail to capture the high rainfall values in the summer and autumn months, associated with low-level orography-enhanced warm rain and tropical systems (typhoons), which are generally underestimated by passive microwave retrievals.
c. Regional comparisons
Over the Alps there is a general underestimation by the SBR products at annual time scales (Fig. 4a). Generally, gauge-corrected SBR products’ performance improves relative to satellite-only SBR products in terms of systematic error. CMORPH has the highest underestimation over this region, which is related to the snow effect, as shown by the seasonal comparisons in Fig. 5a. Even though SBR products are able to capture the seasonal variability, they generally underestimate precipitation in winter months, with the exception of TMPAV7 and PNCT, which include IR rainfall to bridge across microwave precipitation estimates. Overall, TMPAV7 and PNCT exhibit the smallest annual and seasonal systematic errors for this study area.
The performance of SBR products over the French Cévennes is similar to that presented over the Alps. There is an overall underestimation from all SBR products at the annual time scale (Fig. 4b), where TMPAV7 and PNCT mean annual precipitation values exhibit the smallest bias and closely follow seasonal variations. In general, seasonal variability is captured well (Fig. 5b), but almost all SBR products significantly underestimate mean rain gauge values over winter seasons, with the exception of TMPAV7 and PNCT products.
Over the Peruvian Andes at annual time scales, SBR products closely follow the mean annual rain gauge value (Fig. 4c). TMPACCA significantly overestimates rain gauge values over the winter season, which also affects the mean annual overestimation of this product. TMPAV7 and PNCT over this region closely follow the seasonal variation of the rain gauge rainfall. The ECMWF reanalysis product is not able to capture the light precipitation values over the summer (reporting almost zero values), while the rest of the SBR products overestimate these light precipitation values (Fig. 5c). Overall, TMPAV7 exhibits the smallest annual and seasonal systematic errors for this study area.
The performance evaluation over Taiwan exhibits significant underestimation of all SBR products. Gauge adjustment slightly reduces the SBR products’ bias. Seasonal variability is captured; however, there is a significant underestimation in the summer and fall months associated with warm rain and tropical systems. Overall, TMPAV7 and GSCT products exhibit the smallest annual and seasonal systematic errors for this area.
Over Turkey there is a clear pattern of underestimation by satellite-based SBR products and overestimation of the gauge-corrected SBR products, which highlights the issue of using limited gauge data to adjust precipitation estimates in complex terrain areas. Over this region most of the SBR products are not able to capture seasonal variability; on the other hand, the ECMWF reanalysis product did capture the seasonal variation well. Over this region the CMCT product exhibits the smallest annual and seasonal systemic errors.
Over the Colombian Andes, the performance of SBR products is similar to the Turkey region. At annual time scales the SBR products show underestimation, while the gauge-corrected SBR products turn into overestimation. Over this region SBR products are able to capture the seasonal variability, but there is a clear distinction between unadjusted and gauge-corrected SBR products, with the exception of CMORPH, which did not improve with the gauge correction. Over this region TMPAV7 exhibits the smallest seasonal systematic errors while PNCT and GSCT exhibit the smallest annual systematic errors.
Over Blue Nile all products underestimate the rainfall (Fig. 4g). Moreover, gauge correction slightly improves the performance of SBR products at annual time scales. Over this region all products’ performances are similar to each other.
Over the U.S. Rocky Mountains, SBR products underestimate rain gauge values at annual time scales, except TMPART and TMPACCA (Fig. 4h). Over this region rain gauges give no seasonal variation, and a few products (TMPAV7, PNCT, and ECMWF) are consistent with this seasonal pattern. Although some satellite products (CMCT, TMPAV7, PNCT, and PN) capture the seasonal magnitudes for the warm months (summer/fall seasons) well, they give strong underestimation in the winter season months because of the snow and mixed phase precipitation effect on passive microwave retrievals. TMPART and TMPACCA are an exception, exhibiting strong overestimation (Fig. 5h). As a result, these performance characteristics depict a seasonal variability in CMCT, TMPAV7, PNCT, and PN satellite products that does not agree with the gauge measurements. Overall, TMPAV7 and PNCT exhibit the smallest annual and seasonal systematic errors for this study area.
Over Nepal SBR products’ performance is low. TMPA products in general overestimate over this region, especially after gauge correction. Seasonal variation is captured by all products, but there is a significant overestimation by TMPAV7, TMPACCA, and ECMWF over the summer months. In general, all other products underestimate in summer over this region. There is also apparent overestimation in winter months by all products. Overall, PNCT and TMPART exhibit the smallest annual and seasonal systematic errors for this area.
d. Daily comparisons
Figure 6 provides the frequency of occurrence of daily precipitation for different rainfall intensities of the SBR products versus rain gauges. Rainfall intensities are based on reference rainfall. Since ECMWF is available at fixed daily time periods, it was not possible to convert these values to the local time zones at which gauge data are available. Hence, this product is not examined in daily comparisons. Overall, SBR products are able to capture the distribution of rainfall in most of the regions. The PERSIANN products (both gauge adjusted and unadjusted) capture the distribution of rainfall well over almost all regions. Over the Alps, the French Cévennes, Turkey, and the Peruvian Andes, the lowest range of rainfall magnitude (0 < R < 0.5 mm h−1) is the clear mode of the pdf, and the uncertainty is especially large for this range of rainfall values for SBR products. Over the Blue Nile there is a general trend of underestimating observed values in the midrange while shifting to overestimating for high-range values. This trend was also captured for GSMaP, PERSIANN, and CMORPH products over Ethiopia by the study of Thiemig et al. (2012). Over the Peruvian Andes, SBR products in general underestimate low-range values and overestimate medium-range values. Moreover, the TMPACCA does not show any improvement over the unadjusted TMPART distribution of rainfall rates. Over Taiwan uncertainty is large at both low and high rainfall values, with relatively better estimates for the midrange of the distribution.
e. Products accuracy and uncertainty
Figure 7 presents the products’ accuracy in capturing the rainfall occurrence by analyzing NMRV (solid colors) and NFASRV (faded colors) values. SBR products’ performance in capturing rainfall occurrence is best over the Blue Nile region, which is followed by the Alps, the French Cévennes, Taiwan, and the Peruvian Andes regions. Over almost all regions, and especially over the French Cévennes, the performance of TMPACCA is decreased compared to TMPART. Yong et al. (2013) also found that the probability of detection and false alarm ratio values increased for TMPACCA. In their study, they suggested that this low performance is due to reintroduction of satellite data sources taken by the TRMM Microwave Imager (TMI)–TRMM Combined Instrument (TCI) and TCI-3B43, which as a result gradually increases the frequency of input datasets. Therefore, this situation increases the number of rainy events detected by SBR products, which directly contributes to the observed increases in NMRV and NFASRV. In general, the Peruvian Andes, Turkey, and Colombian Andes NFASRV values are higher than NMRV, causing overestimation over these regions, especially in drier seasons. Over the Alps and Blue Nile, these values are similar; meanwhile, over the French Cévennes, Taiwan, the U.S. Rocky Mountains, and Nepal, NMRV values are higher than NFASRV values where significant underestimations are observed in the winter season.
To capture the systematic error component of SBR products, MRE is presented in Fig. 8. Furthermore, the error statistics presented in Figs. 8–10 are conducted by considering all precipitation values (unconditional) and the 99th quantile of rain gauge measurements (>Q99) represented with solid colors and faded colors, respectively. The 99th quantile represents the rainfall value exceeded by 1% of the rain gauge rainfall data, which is used to evaluate the SBR estimates of extreme rainfall values. A comparison is performed to understand improvements of gauge-adjusted SBR products relative to the corresponding unadjusted products’ unconditional values and extreme precipitation values. Over the Blue Nile all products are underestimated, with the exception of the gauge-adjusted CMORPH and gauge-adjusted GSMaP at Q99. As noted above, the performance of PERSIANN is the lowest, followed by GSMaP, which is also confirmed by the studies of Thiemig et al. (2012), Dinku et al. (2007, 2008), and Hirpa et al. (2010). CMORPH (both gauge adjusted and unadjusted) performance is better than the other products, followed by TMPAV7 and TMPART. Over the Colombian Andes, the 99th quantiles of SBR products are all underestimated relative to reference. The CMORPH performance for the 99th quantile estimation is the worst of all products. Over Taiwan, all SBR products significantly underestimate rain gauge values, which is also consistent with the findings of Chen et al. (2013). Over the Alps, all products underestimate rain gauge measurements, with the TMPA products being the exception. Also in agreement with this study are the findings of Mei et al. (2014), who found that CMORPH and PERSIANN products underestimate the gauge rainfall values in all quantile ranges, especially during winter periods. Moreover, Stampoulis and Anagnostou (2012) found that TMPA 3B42, version 6, exhibited a general overestimation, while CMORPH seemed to mostly underestimate over eastern parts of the Alps, which corresponds to the findings of this study. To understand relative improvements of gauge adjustment, Tables 3 and 4 report relative improvements of the gauge-adjusted satellite product to the unadjusted product for unconditional and >99th quantile values. Relative improvement is evaluated by dividing the difference of adjusted and unadjusted SBR products to the unadjusted SBR products’ MRE and CRMSE values. Positive (negative) values represent performance improvement (worsening) from gauge adjustment. When the relative improvements of unconditional rainfall are considered in detail, it is clear that TMPAV7 is able to improve the performance of TMPRT [relative improvement values are close to 1 (0.43–1.83, with Nepal being an exception); Table 3]; TMPACCA shows no improvement except over the Colombian Andes (0.99); and CMORPH, PERSIANN, and GSMaP show fair to good relative improvements, with values ranging from −0.57 to 4.46. When the 99th quantile of relative improvements of MRE performances are considered (Table 4), it is apparent that there are improvements, with Taiwan, the U.S. Rocky Mountains, and Nepal/Himalayas being exceptions.
Figure 9 captures the correlation coefficient performances of SBR products. Results are based on daily precipitation values where unconditional and conditional (99th quantiles) satellite–gauge pairs are considered. It is clear from Fig. 9 that the performance of most products diminishes at extreme values. In general when unconditional precipitation amounts are checked, correlation coefficients of the SBR products are sufficient for every region except Taiwan, the U.S. Rocky Mountains, and Nepal/Himalayas. Over the Blue Nile region, the lowest correlation values are reported for the gauge-corrected GSMaP and both PERSIANN products, while the highest correlation performance is exhibited for the CMORPH products. Over Turkey, the highest correlation is shown for the gauge-corrected GSMaP, both unconditional and for values exceeding the 99th quantile. Over the Peruvian Andes, the correlation of all SBR products is high; however, when values exceeding the 99th quantile range are checked, all SBR product performances diminish. The TMPACCA correlation is low over all regions, and it is the lowest over the French Cévennes.
Finally, Fig. 10 captures the random component of error (CRMSE) for unconditional rainfall amounts and values exceeding the 99th quantile of rain gauge rainfall amounts. Over the Blue Nile the unadjusted PERSIANN and GSMaP products exhibit high CRMSE. However, when compared to other regions, the values of CRMSE over this region are the lowest, which indicates that the random component of error is low in this region. Over the Turkey study region, CRMSE for gauge-corrected CMORPH and GSMaP are the lowest, while the worst performance is exhibited by the PERSIANN products. Similarly, over the Colombian Andes the worst performance is exhibited by both PERSIANN products. Over the Colombian Andes region, it is noted that the gauge-adjusted (TMPAV7) TMPA performance is worse than the unadjusted (TMPART) products. As TMPAV7 is gauge corrected, there may be an expectation for it to perform better; however, it is not clear why the adjusted product shows higher CRMSE, which is also confirmed by the study of Dinku et al. (2010). TMPACCA seems to elevate TMPART estimates in order to reduce the random error and bias; however, this causes a boost of overestimation, especially over regions where overestimation is already observed with TMPART. Yong et al. (2013) concluded in their study that TMPACCA simply reduces the error and bias over the globe rather than focusing on error structures of SBR data for local areas. Over the Peruvian Andes, other than PERSIANN, all the products’ performances are improved relative to the unadjusted products.
An evaluation of nine satellite-based rainfall products and one reanalysis product over nine different regions around the globe, characterized by complex terrain, is conducted using daily rain gauge rainfall data over the 2000–13 data period. The satellite-based rainfall data are available at 0.25° regular latitude–longitude grid and rain gauge measurements were averaged within that 0.25° grid. The satellite-based rainfall products and reanalysis used in this study were TMPART, TMPAV7 (gauge adjusted), TMPACCA (climatological gauge adjusted), CMORPH, gauge-adjusted CMORPH, PERSIANN, gauge-adjusted PERSIANN, GSMaP, gauge-adjusted GSMaP, and ECMWF. The study areas of this work were the Alps, the French Cévennes, the western Black Sea region of Turkey, the Colombian Andes, the Peruvian Andes, the U.S. Rocky Mountains, the Blue Nile, western Taiwan, and the Himalayas over Nepal. Various evaluation statistics were used to assess the performance of the satellite-based rainfall products.
All study regions are over complex terrain characterized by different hydroclimatic characteristics. Over the Alps, the French Cévennes, the Peruvian Andes, and Turkey, significant orographic effects are observed. Taiwan, the Blue Nile, and the Colombian Andes are characterized by tropical conditions; the Colombian Andes is more complex than others, with a bimodal annual cycle of precipitation resulting from double passage of ITCZ. The U.S. Rocky Mountains and Nepal/Himalayas can be grouped as highland climate; Nepal/Himalayas is mostly influenced by monsoon systems.
The study showed that the performance of satellite-based rainfall products depends highly on rainfall structure. Many SBR products were shown to underestimate wet season and overestimate dry season precipitation. The study compared the performance of gauge adjustment for every satellite product and showed that gauge adjustment to the SBR products is highly dependent on the representativeness of the chosen rain gauges to the local satellite measurement conditions. Hence, over data-sparse regions and complex terrain, gauge adjustment could result in no improvement compared to satellite-only rainfall products. Taking into account all presented results in this study, we conclude the following:
ECMWF performance at mean annual time scales closely follows rain gauge values, while satellite-only SBR products tend to underestimate and performance of products improves with gauge adjustment, with exceptions. Over most of the regions, TMPACCA seems to elevate TMPART estimates to overestimation. When gauge-corrected SBR products are compared with each other, superiority of TMPAV7 can be observed at mean annual time scales.
Performance of SBR products highly depends on precipitation structure and variability imposed by orography. Specifically, PMW-based SBR products (i.e., CMORPH and GSMaP) significantly underestimated the warm orographic rain systems, which, because of the low ice concentration aloft, are not captured well by PMW sensors. On the other hand, the IR-based PERSIANN product did not capture well the deep convective systems in the upper Blue Nile basin.
In general, most of the SBR products captured the seasonal trends in all regions; ECMWF especially closely followed the rain gauge values at the monthly time scale, with the Alps being an exception. Gauge-adjusted SBR products closely follow observations except over Turkey and the Colombian Andes, where they tend to overestimate the gauge observations. Over Turkey, climatology changes suddenly as with the Colombian Andes; hence, rain gauges that are used in bias adjustment algorithm heavily affect the correction. The Alps, the Colombian Andes, the Peruvian Andes, the Blue Nile, and Taiwan exhibit clear distinctions between wet and dry seasons. Over these regions SBR products provide fair to good trends in capturing this distinction between wet and dry seasons. On the other hand, when there is no clear distinction between dry and wet seasons, which is the case for the French Cévennes, Turkey, the U.S. Rocky Mountains, and Nepal/Himalayas, different SBR products show inconsistent temporal distributions compared to gauge values.
SBR products are able to capture the distribution of rainfall rates for most of the regions. However, there is a general trend of overestimating the occurrence of low rain rates and underestimating the occurrence of high rain rates.
SBR performance for occurrence of rainfall is best observed over the Blue Nile followed by the Alps, the French Cévennes, Taiwan, and the Peruvian Andes. Over regions characterized by orographic enhancement, an IR-based product (PERSIANN) shows high NFASRV. Over regions characterized by highland climate, most of the products have high NMRV.
To understand performances of SBR products at unconditional and extreme cases (>Q99) we evaluated the systematic error component. All products’ performances diminished at extreme cases. Rain gauge–adjusted products’ performances at unconditional range were shown to not always improve throughout all regions, but at extreme cases rain gauge adjustment seems to be improving satellite products. TMPACCA seems to elevate TMPART estimates in order to reduce the random error and bias; however, this was shown to cause a boost of overestimation, especially over regions where overestimation is already observed with TMPART.
In summary, we have shown that the performance of integrated satellite rainfall products is highly dependent on rainfall variability and seasonal changes. All the regions studied are complex terrain; hence, results may not be readily applicable to other regions. Future extensions of this study can include evaluation of the high-resolution rainfall data from the newly available Global Precipitation Measurement products. Moreover, evaluation of the instantaneous PMW-based precipitation datasets that constitute the basis of the main satellite rainfall algorithms using high-temporal-resolution ground-validation datasets can give further insight to the error characteristics of the SBR products. Finally, investigating possible links between the errors and physiographic features within each region can provide more detailed error characteristics, which can be useful information to algorithm developers and data users.
This work was supported in part by the EU-funded Earth2Observe project (ENVE.2013.6.3-3) and in part by NASA Precipitation Measurement Mission (Award NNX07AE31G).
This article is included in the Seventh International Precipitation Working Group (IPWG) Workshop special collection.