Introduction
Reliable observation of precipitation is an important task to the hydrologic and climate research communities. Because ground observation from gauge and radar suffers from spatial and temporal gaps, rainfall measurements derived from meteorological satellites become an attractive option because of their high spatial and temporal sampling frequencies.
In our earlier reported work, we developed the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) algorithm, using satellite longwave infrared (IR) images (Hsu et al. 1997, 1999; Sorooshian et al. 2000). The PERSIANN algorithm fits the pixel brightness temperature and its neighbor temperature textures, in terms of means and standard deviations, to the calculated pixel rain rates based on an Artificial Neural Network (ANN) model. The PERSIANN algorithm is operated at a spatial resolution of 0.25° latitude × 0.25° longitude, with model parameters routinely adjusted by other low-orbital satellite microwave rainfall estimates (Sorooshian et al. 2000).
Improvement of the PERSIANN system for estimating rainfall rates at higher spatial and temporal resolution is ongoing. In this paper, the development of the PERSIANN Cloud Classification System (CCS) is introduced. Instead of direct pixel-to-pixel fitting of infrared cloud images to the rain rate, the PERSIANN CCS takes into account image segmentation and objective classification methods to process cloud images into a set of disjointed cloud-patch regions; informative features are extracted from cloud patches and classified into a number of patch groups based on the similarity of selected features, such as the patch size and texture. Pixel rain rates under the cloud-patch coverage are assigned using the histogram-matching method and exponential regression fitting of pixel brightness temperatures Tb and rainfall rates R. Each classified cloud-patch group is specified a unique temperature–rainfall (Tb–R) curve, and the parameters of the Tb–R curve are calibrated based on rain estimates of the Next-Generation Weather Radar (NEXRAD) network.
The scope of this paper is as follows. Section 2 describes algorithms in rainfall estimation using infrared imagery. The structure of PERSIANN CCS is presented in section 3, followed by the network training and some insights provided by PERSIANN CCS calibration in section 4. Section 5 presents validation results, and section 6 discusses conclusions and suggestions for further improvements to PERSIANN CCS.
Background
Satellite IR-based rainfall estimation algorithms offer unique advantages of extensive global coverage at relatively high temporal sampling rates (Ebert and Manton 1998). The algorithms developed to date may be classified into three groups, depending on the level of information extracted from the infrared cloud images. There are
cloud-pixel-based algorithms,
cloud-local-texture-based algorithms, and
cloud-patch-based algorithms.
An example of the cloud-pixel-based algorithm is the Geostationary Operational Environmental Satellite (GOES) precipitation index (GPI; Arkin and Meisner 1987), which assigns a constant conditional rain rate of 3 mm h−1 to pixels with a Tb lower than 235 K and zero rain rates otherwise. The adjusted GPI (AGPI; Adler and Negri 1993) adjusts rain rate by time- and space-matched GPI and Special Sensor Microwave Imager (SSM/I)–based rain estimates accumulated for a month. Xu et al. (1999a) proposed a universal AGPI that is similar to AGPI but allows both the rain rate and temperature threshold to vary. Another pixel-based fitting is the autoestimator (AE) algorithm (Vicente et al. 1998). The AE algorithm utilizes a power-law function to fit the Tb–R relationship. Pixel rainfall values are further adjusted using several correction factors, such as relative humidity and precipitable water.
Cloud-local-texture-based approaches retrieve pixel rain rates by extending the mapping from one single pixel to a range of the neighborhood pixel coverage. Wu et al. (1985) used 24 brightness temperature texture features to retrieve rainfall within a neighborhood size of 20 km × 20 km. Hsu et al. (1997) developed the PERSIANN system, which calculates rain rate at 0.25° latitude × 0.25° longitude resolution, based on the brightness temperature variations in the neighboring coverage of 1.25° × 1.25°.
Cloud-patch-based approaches estimate rainfall based on the cloud coverage under a specified temperature threshold, such as 253 K. The Griffith–Woodley Technique (Griffith et al. 1978; Woodley et al. 1980) tracked a cloud through its lifetime. The total rain volume of the cloud patch is determined based on the ratios of satellite cloud area to its maximum coverage. The Convective–Stratiform Technique (Adler and Negri 1988) is another example of the cloud-patch-based approach. It screens convective cells based on the local minimum of infrared temperature and assigns different rainfall amounts to convective and stratiform components separately. Pixel rain rates are proportionally distributed starting from the coldest pixel to higher-temperature pixels. Xu et al. (1999b) proposed another approach, which determines different temperature thresholds by separating the rain/no-rain pixels in a cloud patch using SSM/I microwave rainfall estimates.
With respect to the use of information, both pixel- and local-texture-based approaches only utilize limited attributes of the cloud patches. Rain rates retrieved from these methods tend to be nonunique and may be insufficient to identify the relationships between cloud types and surface rain rates. Cloud-patch-based approaches, on the other hand, attempt to include more information from the cloud images and are likely to provide a more reliable rainfall retrieval system than do the pixel-based approaches. Therefore, successful characterization of cloud patches can be one step toward better estimation of rainfall. However, developing a desirable cloud-patch-based algorithm depends on many factors, including 1) how effectively the cloud image may be separated into distinct cloud systems or cells; 2) how our knowledge of the cloud systems may be converted into a set of measurable numerical feature vectors, in terms of patch temperature, size, and texture; 3) how these feature vectors may be effectively clustered into a set of separable cloud-patch “classes” or “groups;” and 4) how the cloud-patch groups are associated with the rainfall distributions under the cloud coverage. Following the above requirements, the PERSIANN CCS is developed. A description of the algorithm is listed in the following section.
Method
The PERSIANN CCS establishes different Tb–R relationships, calibrated by collocated cloud images and gauge-corrected radar rainfall data, for every classified cloud group by implementing a sequence of four steps (Fig. 1). The steps are described as follows.
Segmentation of satellite infrared cloud images
A cloud image is shown in Fig. 2a, and cloud-patch segmentation of this image, using a fixed threshold (253 K), is shown in Fig. 2b. Note that even though the cloud image contains several convective cells, the single threshold cannot separate them effectively.
Our proposed approach, called the incremental temperature threshold (ITT), achieves segmentation by gradually increasing threshold temperatures (Figs. 2c– h). The algorithm (for a complete description of ITT, please see appendix A) locates the local minimum temperature Tmin and places the first set of seeds (illustrated as cross markers in Fig. 2). Next, the threshold temperature is raised and extended to the neighboring areas from the seeded points until the border of other seeded regions or cloud-free regions is reached.
Extraction of cloud-patch features
Fully developed convective clouds have distinguishable overshooting tops, tight temperature gradients, and higher local pixel temperature variations. Stratiform clouds, on the other hand, show more gradual temperature gradients and lower temperature variations. The three categories used to discriminate between cloud types are coldness, geometry, and texture. A further breakdown of measurable features is provided in Table 1, and a detailed explanation of these features is listed in appendix B.
As one might expect, there is a potential interrelationship between the selected cloud-patch features and the amounts of patch rainfall. Figure 3 illustrates the scatterplots of the four selected features with respect to hourly patch rainfall (millimeters per patch) and patch rain probability, based on 1-month (June 1999) GOES images and gauge-corrected radar-rainfall data over the region of 30°–40°N and 120°–100°W. Note that the rain probability is the ratio of the rain patches (≥10 millimeters per patch) to the total number of patches within a small interval of displayed feature. The left-hand panels show a negative correlation trend with respect to the minimum temperature (Fig. 3a) and positive correlation for the remaining three features (Figs. 3b–d). The wide-scattering samples in the figures are partly due to the interaction among the patch features. In the case of Fig. 3a, for example, the amount of patch rainfall is not only determined by Tmin, but is also influenced by other factors, such as the patch size and texture. In the case of the rain probability plots depicted in the right-hand panels of Fig. 3, we witness a monotonically decreasing trend in the case of Tmin (i.e., a lower patch minimum temperature is related to a higher probability of rain in the patch) and increasing trends for the other three features.
Classification of cloud patches
A clustering algorithm, the self-organizing feature map (SOFM), is used to classify the cloud-patch features into a number of groups (Kohonen 1982). This approach has already been described in our earlier work (Hsu et al. 1997), and therefore only a brief description is provided here. SOFM projects patterns of high-dimensional space to lower-dimensional space. The projection enables the classification of input patterns of many variables into a number of clusters arranged in a two-dimensional coordinate (Fig. 1c). The two main steps involved are
- calculating the distance between patch features and the SOFM cluster center usingwhere xi is the cloud-patch input feature i, and wij is the connection parameter of SOFM from input feature i to the specified cluster center (node) j, and
- finding the best matching SOFM cluster center c, from the minimum distance between the input feature vector and the SOFM connection weights wij:dcdjjn.
Estimation of patch and pixel rainfall
The relationship of pixel temperature Tb and surface rainfall R of a cloud patch is likely to vary in different stages of its life cycle. Therefore, we assign different Tb–R relationships to various cloud patches based on the classification resulting from the previous clustering stage.
PERSIANN CCS calibration
System architecture and calibration
To implement the PERSIANN CCS, we first need to specify the number of cloud-patch groups. There is a trade-off between the number of cloud-patch groups and the accuracy and efficiency of the calibrated model. To ensure good accuracy, a large number of classified cloud groups are required, but the computational requirements for model training and testing increase dramatically. Hsu et al. (2002) show that, given an objective function (root-mean-square error, e.g.), an optimal group size can be found by progressively testing the size of groups. In this study, a group size (20 × 20) was determined in the SOFM classification layer.
The nine input features listed in Table 1, except Tmin and the gradient of the cloud-top brightness temperature TOPG, are calculated at three separate temperature levels (220, 235, and 253 K) for every cloud patch. Figure 4 is a demonstration of cloud feature extraction for two adjacent patches (shown as m1 and m2). Cloud m1 depicts the situation of a patch in its mature stage with extensive vertical growth and an overshooting top, and patch m2 is still in the towering stage with its top temperature not yet at the 220-K level. Features extracted at the 220-K level are related to the cold core of the taller cloud patch, while features extracted at the warmer 235-K level represent information of larger extent. Patch features extracted from the threshold 253-K level give information of the entire cloud patch ranging from Tmin to 253 K. However, not every cloud patch contains temperatures ranging up to the colder levels, such as 235 or 220 K, where default values are assigned. For example, in the case of cloud-patch area, a zero is assigned for Tmin at the 220-K level for patch m2, and it has a nonzero value for temperature levels of 235 and 253 K.
After completing the cloud-patch feature extraction, the system is calibrated using GOES infrared images and radar-rainfall maps for June 1999 over the region of 25°–45°N and 100°–130°W (both datasets are mapped to 0.04° latitude × 0.04° longitude scale). The hourly rainfall used for calibration is a rain gauge–corrected multisensor precipitation dataset, developed by the National Oceanic and Atmospheric Administration National Centers for Environmental Prediction (NCEP) (NCEP 2003).
Calibration of the algorithm is a two-stage process. The first stage is an unsupervised clustering of input features. In this stage, all of the segmented cloud patches are classified into 400 groups based on the similarity of cloud features using the SOFM clustering technique. In the next stage, the rainfall data are coregistered and are assigned into the classified patch groups. The concurrent infrared images and rainfall data in each classified patch group are further sorted, based on a probability matching method. Last, the parameters of the nonlinear fitting function [Eq. (3)] are calculated, based on the sorted pixel infrared and rainfall data. The PERSIANN CCS eventually includes 23 cloud features in the input layer and 400 cloud groups arranged in a 20 × 20 matrix in the classification layer, with each group being assigned its unique Tb–R function.
Insights provided by the calibration results
The SOFM clustering technique allows exploration of the classified cloud-patch features through its two-dimensional feature map. Figure 5a shows the 20 × 20 groups arranged in two-dimensional coordinates. Each classified group is described by a set of input patch features. The status of the cloud patch can be explained by listing the input features on each classified group (see Figs. 5b–d). Figure 5b, for instance, shows the contour map of cloud-patch minimum temperatures (Tmin) on 20 × 20 groups. Cloud-patch groups in the upper-right corner are associated with warmer clouds where cloud-patch Tmin are higher than 235 K. Following the arrow, the cloud patch's Tmin are gradually reduced, and the patches in the lower-right corner are associated with cold clouds, where their Tmin are lower than 205 K.
The contour plot of TOPG on 20 × 20 groups in Fig. 5c shows the cloud patch's temperature gradient near the coldest pixel. This index helps us to locate those cloud-patch groups having strong upwelling movement near the coldest convective core area. Heavy convective clouds usually present tight temperature gradients near the cold core area. They are marked with an “X” in Fig. 5c. An interesting point that should be highlighted is an observation that was made when Figs. 5c and 5b were compared with respect to the location of the X marks. Note that those high-TOPG patch groups cover a wide range of temperatures, suggesting that convective clouds are not exclusive to the cold cloud regions.
Next we examine the contour plot of the local texture variation [the mean value of the local standard deviation of the cloud temperature (MSTD5)] in 20 × 20 groups, as shown in Fig. 5d. This feature indicates the average of local standard deviation of pixel temperature in a neighborhood of 5 × 5 pixels for all pixels under the cloud-patch coverage. Again, the higher MSTD5 groups are marked with an X. Note that patches with a higher MSTD5 correspond very closely to a higher TOPG in Fig. 5c. The averaged rain rate in the classified 20 × 20 groups shown in Fig. 5e is assigned based on 1-month NCEP gauge-corrected radar data. Cloud patches with higher average rain rates are similarly marked with an X. Note that patches with higher rain rates correspond well to the groups with higher TOPG and MSTD5 (Figs. 5c–e).
Last, we examine the two other feature maps. Figure 5f shows the contour map of cloud-patch size (AREA) on 20 × 20 groups, and the total patch rainfall amounts are depicted in Fig. 5g. Patch groups with AREA larger than 3000 pixels are circled. When one compares Figs. 5f and 5g, it is evident that those circled large AREA regions correspond to higher total patch rainfall. Interesting to note is that, when we make a comparison with Fig. 5e, the encircled patches with larger AREA groups do not necessarily correspond to a high rainfall rate, which indicates that a large portion of the cloud patch generates little or no rain in those large AREA groups.
What remains to be done is the transformation of the patch information to the pixel rainfall rate for the areas under the patch coverage. Figure 6a is intended to provide a visualization of the calibrated Tb–R curves for each of the 20 × 20 patch groups. Each Tb–R curve represents the temperature and rainfall-rate relationship at the pixel level for a classified cloud group. For closer examination, seven categories of cloud groups (C0–C6) are displayed. Note that the C0 category is associated with warmer and no-rain cloud patches, whereas the other categories (C1–C6) show significant variations of slope and maximum rain rate for temperatures ranging from Tmin > 235 K (in C1 and C2), 215 K > Tmin > 235 K (in C3 and C4), and Tmin < 215 K in (C5 and C6). The Tb–R curves of C0–C6 are unique as shown in Fig. 6b. Table 2 summarizes the input features with respect to the seven highlighted categories, and they are explained in appendix C.
Case study and validation
Several evaluation criteria were selected to validate PERSIANN CCS. The quantitative accuracy of estimates is evaluated using bias: correlation coefficient (CORR), and root-mean-square error (rmse). The performance of rain/no-rain detection is evaluated by the probability of detection (POD), false-alarm ratio (FAR), skill score (SKILL), and critical success index (CSI). Definitions of these criteria are listed in the report of the third Algorithm Intercomparison Project of the Global Precipitation Climatology Project (Ebert 1996). PERSIANN CCS rainfall estimates are validated on an hourly and daily basis for different spatial (0.04°, 0.12°, 0.25°, and 0.5°) scales. Two rainfall observation datasets were used in this validation: high-temporal-and-spatial-resolution NCEP radar data in section 5a and high-quality daily rain gauge data for both the cold and warm seasons in section 5b. PERSIANN CCS was also compared with the PERSIANN product (Sorooshian et al. 2000). The PERSIANN estimates for 1999 were trained using the instantaneous rain rate derived from the Tropical Rainfall Measuring Mission Microwave Imager (TMI). The PERSIANN products for 2002 were calibrated using the rain rates merged from the SSM/I and TMI.
Two-day storm event near Las Vegas, Nevada
Hourly rainfall evaluation
PERSIANN CCS estimates were compared with those of PERSIANN for a flash-flood storm that occurred on 8 July 1999 in Las Vegas, Nevada, and its surrounding area (approximately 300 km × 300 km). This intense storm, monitored form 1400 through 1900 UTC (0600– 1100 LST), reportedly caused the worst flooding of the century in the Las Vegas vicinity, resulting in the loss of two human lives and $20 million in property damage. At the instantaneous time scale (at 0.25° × 0.25° spatial resolution), PERSIANN CCS estimates show a substantial improvement in accuracy and the detection of rain/ no-rain pixels and compared with PERSIANN for this storm (Table 3). A time series of rain-rate and evaluation indices from satellite and radar data is shown in Fig. 7. This result illustrates the advantage of PERSIANN CCS in capturing the high variability in short-term rainfall.
Daily rainfall evaluation
Daily comparison of PERSIANN CCS and radar rainfall
Hourly rainfall on 8–9 July 1999 was simulated over a relatively large area (25°–45°N, 100°–130°W) and then was accumulated to daily rainfall for model evaluation. PERSIANN CCS was evaluated at 0.04°, 0.12°, 0.24°, 0.5°, and 1° resolution. Table 4 shows that the PERSIANN CCS's performance improved as the grid resolution changed from high (0.04° × 0.04°) to low (1° × 1°). The estimates show a good fit at the resolution of 0.12° and lower, with CORR (including 0.0 rainfall grids) of approximately 0.6–0.8, rmse of 3–7 mm day−1, POD of 0.8–0.9, and FAR of 0.08–0.17. The bias ratio was approximately 1.11–1.02 for the estimates at a resolution of 0.04°–1.0°, indicating a minor bias.
Figure 8 shows the scatterplots of PERSIANN CCS estimates against radar observations. The PERSIANN CCS tends to underestimate high rainfall at a small spatial scale (0.04°). When the resolution is reduced to 0.12° and 0.24°, the underestimation of high rainfall is improved.
Daily comparison of PERSIANN CCS with PERSIANN
A comparison of PERSIANN CCS with GPI (the widely used) and PERSIANN results, after remapping at a daily 0.25° × 0.25° scale, is shown in Table 5. PERSIANN CCS outperformed PERSIANN on both test days, especially for CORR, which improves from 0.54 to 0.75 on 8 July and from 0.605 to 0.72 on 9 July. Figure 9 shows that PERSIANN estimates were less varied in spatial distribution than those of PERSIANN CCS. Part of the reason for this result is that radar rainfall and PERSIANN CCS are computed at the 0.04° grid scale but are remapped to a PERSIANN elemental resolution of 0.25° for comparison. Both PERSIANN CCS and PERSIANN show statistically significant improvement over GPI, particularly in the correlation and bias ratio.
Patch rainfall validation
PERSIANN CCS is a patch-based approach, which starts with patch segmentation and classification, followed by the probability matching of the Tb–R data distribution and subsequent derivation of an optimized nonlinear Tb–R data relationship for each classified cloud patch. During the evolution of a convective cloud, the Tb–R distribution varies greatly from the initial stage to the dissipated stage of the cloud patch. Figure 10a illustrates the evolution of a convective cloud patch from 1430 through 2345 UTC 9 July 1999. Figure 10b shows a good fit between the Tb–R curves generated from the PERSIANN CCS model and the data dots averaged from infrared images and radar rainfall at each time step. Figure 10c shows the time series of the coldest temperature (Tmin) and cloud-area coverage of the cloud patch. Patch-averaged pixel rain rate and patch rainfall derived from PERSIANN CCS are compared with radar (Fig. 10d). As shown in Fig. 10c, this convective cloud has one major growing stage that peaked at the sixth time step, resulting in maximum rain intensity. As shown in Figs. 10c,d, the time step of maximum patch rain intensity matches well with the time step of minimum cloud-patch temperature. Likewise, the time step of maximum patch rainfall volume associates well with the time step of maximum patch area. The case demonstrated in Fig. 10 is unique, with the cloud patches clearly separable. For cases in which multiple convective storms are mixed together (not shown here), the time evolution of the patch-averaged pixel rain intensity and patch rainfall to the selected features may be more complicated.
Figure 10a shows the evolution of the cloud patch from the beginning (small, warm), through the pre-mature (midsized, cold), mature (maximum sized, coldest), dissipating (midsized, cold), and disappearing life stages. Note that the corresponding Tb–R functions in Fig. 6a also change, in sequence, from cluster 2, through clusters 4, 6, 3, 1, and 0 for each life stage, respectively. This result clearly shows the temporal correspondence between cloud features (coldness, geometry, and texture) and precipitation characteristics (rainfall volume and intensity) at every life stage of the cloud patch.
As shown in Fig. 10d, the correlation coefficients of the observed and estimated patch intensity and volume time series are 0.96 and 0.93, respectively. The bias ratios are around 1.0, showing that the biases of the estimates are low. The fitting curves, as shown in Figs. 10a,b, imply that the Tb–R distribution varies with respect to the cloud patch in different stages and cannot fit well if using a single fixed Tb–R function.
Figure 11 shows the curve fitting of the Tb–R distribution using PERSIANN CCS and several other approaches. The pixel-to-pixel Tb–R relationship is plotted in Fig. 11a and shows a large variation between the cloud-top temperature and surface rainfall at the small scale (hourly measurements at 0.04° grids). Note that only 10% of the data were randomly selected from the whole training domain samples for this scatterplot, which could represent the distribution of the total dataset. It is evident that the Tb–R relationship cannot be fitted well by a single function. Figure 11b shows that the testing data are fitted by multiple (400) Tb–R curves from the PERSIANN CCS model. Figure 11c illustrates the fitted curves using threshold, linear, or nonlinear regression curves. The PERSIANN CCS model dispatches the Tb–R curves based on the specific cloud-patch properties, whereas threshold/linear/nonlinear approaches produce rainfall estimates with only one unique fitting function. For the improvement of the single-function approaches, applications are made to spatial/temporal accumulation of rainfall to lower resolutions, such as 1° × 1° daily/monthly, or are made to rely on temporal adjustment of the Tb–R function from other sources of rainfall observations, such as low-orbiting satellite microwave rainfall estimates or surface radar/gauge measurements.
Seasonal validation with U.S. daily gauge analyses
The PERSIANN CCS estimates were validated using gauge data over the southwestern United States for both the cold and warm seasons. The source of the data is the Climate Prediction Center Real-Time Daily Gauge Analysis (Higgins et al. 2000), which is composed of 5000 National Weather Service stations and several hundred stations per day from the Climate Anomaly Data Base. This dataset has undergone quality control (duplicate station check, buddy check, and standard deviation check against climatological mean) and has been objectively analyzed at a 0.25° grid scale. The validation of PERSIANN CCS was conducted over the continental United States within 25°–45°N, 100°–125°W during a cold month (February 2002) and a warm season (June– August 2002). All datasets were gridded into the same resolution (daily 0.25°). To ensure quality control, only days with more than 500 effective grids are considered; otherwise, no statistical comparison is made.
Time series of statistical comparisons for both the cold and warm seasons
Time series of statistical comparisons of daily U.S. rain gauge analyses with PERSIANN and PERSIANN CCS estimates are illustrated in Fig. 12. PERSIANN CCS compares more favorably than PERSIANN with the gauge analyses in all statistics during both the cold (Fig. 12a) and the warm (Fig. 12b) seasons. Both algorithms exhibit similar variability in the time series of the statistics in the two seasons; however, the flatter curves of statistics indicate that PERSIANN CCS performs in a more stable manner, particularly for CORR, SKILL, POD, and FAR.
As shown in Table 6, both algorithms perform better in the warm season than in the cold season, possibly because of the absence of snow cover–screening procedures in the two algorithms or the contamination of cold cirrus clouds during the cold season.
Histogram of rain rates for the cold and warm seasons
The distributions of rain rates among the rain gauge, PERSIANN, and PERSIANN CCS during the two seasons are depicted in Figs. 13a and 13b, respectively. During the cold season (Fig. 13a), both algorithms overdetect areas at low rainfall intensities. PERSIANN CCS slightly underdetects areas starting at 22 mm day−1 and upward; PERSIANN starts to converge with gauge analyses at 32 mm day−1 and substantially underestimates rain rates at 45 mm day−1 and beyond. As shown in Fig. 13b, during the summer season both algorithms detect more areas with rainfall when compared with the gauge analyses. PERSIANN CCS incrementally converges with gauge analyses at high rain rates over 30 mm day−1; PERSIANN overdetects areas with high rainfall intensity. Note that both algorithms share a very similar distribution in the summer season and exhibit higher variation in February, possibly due to many fewer counting samples in 1-month data samples. The distributions of both satellite rainfall estimates in the summer season more closely correlate with gauge data than in the cold season. When compared with PERSIANN, PERSIANN CCS demonstrates better agreement with the rain gauge data in the plot of rainfall rates, especially in high–rainfall intensity regions (above 22 mm day−1).
The t test for “correlated correlations” between PERSIANN and CCS
As shown in Tables 3, 5, and 6, PERSIANN CCS has higher correlations than PERSIANN when compared with gauge/radar observations. The question is whether the difference between the correlations is statistically significant or is just sampling variation. A t test for correlated correlations is needed because the two correlations being compared share a variable (rainfall observation, in this case). The long-time standard t test for correlated correlations is Hotelling's t test (Meng et al. 1992).
Given significant level α = 0.05 and N ≥ 120, R12 is significantly different from R13 if |t| > tα, where tα (=1.65) is a critical value from the t table. This test was applied in the comparison cases in Tables 3, 5, and 6, respectively. The results (Table 7) reveal that the correlations of PERSIANN CCS are significantly improved from the correlations of PERSIANN when compared with gauge/radar rainfall.
Summary, conclusions, and future work
To date, numerous applications of artificial neural networks to solve large-scale problems have been reported in the literature and have provided impressive performances over conventional techniques (Govindaraju et al. 2000). In the case of precipitation, an ANN model, the PERSIANN system, was developed (Hsu et al. 1997, 1999), using the modified counter propagation network, which consists of an SOFM layer and a linear output layer to fit the pixel cloud-top temperature to rain rates. The PERSIANN system estimates surface rainfall at a 0.25° latitude × 0.25° longitude grid every 30 min and can accommodate a resolution of 1° × 1° on a daily scale in various applications (Sorooshian et al. 2000).
One of the limitations of the PERSIANN architecture is that it estimates rainfall based on local pixel information. To overcome this limitation and to extend PERSIANN's ability to utilize additional satellite imagery information from the local pixel scale to cloud-patch scale, PERSIANN CCS is developed. The structural and functional improvements of PERSIANN CCS over PERSIANN are
a cloud segmentation procedure that is designed to preprocess IR imagery into a large number of separated cloud patches through an incremental temperature threshold algorithm, as shown in Fig. 1 and appendix A;
a feature extraction scheme that retrieves both local pixel temperature textures and regional cloud-patch features instead of only local pixel features as in PERSIANN;
a cloud classification algorithm (SOFM) that clusters cloud patches into 400 (20 × 20 matrix) groups, based on the cloud-patch features extracted from 230-, 235-, and 253-K temperature levels, respectively; and
a cloud-rainfall fitting scheme that calibrates different Tb–R functions for each classified cloud cluster instead of a single mapping function in PERSIANN.
The performance of the algorithm was evaluated using daily gauge and hourly radar rainfall at various spatial scales (0.04°, 0.12°, 0.24°, 0.25°, and 0.5°). The results show that PERSIANN CCS, because of its comprehensive cloud-patch features and its ability to address the variability of rainfall distributions in different cloud clusters, outperforms PERSIANN. In addition, PERSIANN CCS offers insights in explaining the classified patch features with respect to their pixel rainfall distributions. PERSIANN CCS can be used not only as a finescale rainfall estimation scheme, but also as an explanatory tool to analyze the cloud–rainfall system.
Our evaluation of PERSIANN CCS so far has focused on rainfall estimation over land, where ground-based radar/gauge measurements are available to quantify the rainfall distribution. We are currently extending this work to evaluate PERSIANN CCS's performance over the ocean.
Acknowledgments
Partial support for this research is from NSF STC for “Sustainability of Semi-Arid Hydrology and Riparian Areas” (SAHRA) (Grant EAR-9876800), NASA-EOS (Grant NA56GPO185), and TRMM (Grant NAG5-7716). We are grateful to Dr. R. Adler and others of NASA GSFC for their valuable suggestions. The authors also thank the anonymous reviewers and editor for their constructive comments.
REFERENCES
Adler, R. F. and A. J. Negri. 1988. A satellite infrared technique to estimate tropical convective and stratiform rainfall. J. Appl. Meteor 27:30–51.
Adler, R. F. and A. J. Negri. 1993. Estimation of monthly rainfall over Japan and surrounding waters from a combination of low-orbit microwave and geosynchronous IR data. J. Appl. Meteor 32:335–356.
Arkin, P. A. and B. N. Meisner. 1987. The relationship between large-scale convective rainfall and cold cloud over the Western Hemisphere during 1982–84. Mon. Wea. Rev 115:51–74.
Atlas, D., D. Rosenfeld, and D. B. Wolff. 1990. Climatologically tuned reflectivity–rain rate relations and links to area–time integrals. J. Appl. Meteor 29:1120–1135.
Ebert, E. E. 1996. Results of the 3rd Algorithm Intercomparison Project (AIP-3) of the Global Precipitation Climatology Project (GPCP). Bureau of Meteorology Centre Res. Rep. 55, 204 pp.
Ebert, E. E. and M. J. Manton. 1998. Performance of satellite rainfall estimation algorithms during TOGA COARE. J. Atmos. Sci 55:1537–1557.
Govindaraju, R. S. and A. R. Rao. 2000. Artificial Neural Networks in Hydrology. Kluwer Academic, 329 pp.
Griffith, C. G., W. L. Woodley, P. G. Grube, D. W. Martin, J. Stout, and D. N. Sikdar. 1978. Rain estimation from geosynchronous satellite imagery—Visible and infrared studies. Mon. Wea. Rev 106:1153–1171.
Haralick, R. M., K. Shanmuban, and I. Dinstein. 1973. Texture features for image classification. IEEE Trans. Syst. Man Cybernetics 6:610–621.
Higgins, R. W., W. Shi, E. Yarosh, and R. Joyce. cited 2000. Improved United States Precipitation Quality Control System and Analysis. NCEP/CPC Atlas No. 7 [Available online at http://www.cpc.ncep.noaa.gov/research_papers/ncep_cpc_atlas/7/index.html.].
Hsu, K., X. Gao, S. Sorooshian, and H. V. Gupta. 1997. Precipitation estimation from remotely sensed information using artificial neural networks. J. Appl. Meteor 36:1176–1190.
Hsu, K., H. V. Gupta, X. Gao, and S. Sorooshian. 1999. Estimation of physical variables from multichannel remotely sensed imagery using a neural network: Application to rainfall estimation. Water Resour. Res 35:1605–1618.
Hsu, K., H. V. Gupta, X. Gao, S. Sorooshian, and B. Iman. 2002. Self-organizing linear output (SONO): An artificial neural network suitable for hydrologic modeling and analysis. Water Resour. Res 38:1302–1318.
Kohonen, T. 1982. Self-organized formation of topologically correct feature maps. Biol. Cybernetics 43:59–69.
Meng, X. L., R. Rosenthal, and D. B. Rubin. 1992. Comparing correlated correction coefficients. Psychol. Bull 111:172–175.
NCEP, cited 2003. National stage IV QPE product. [Available online at http://wwwt.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage4/;.].
Negri, A. J., T. L. Bell, and L. Xu. 2002. Sampling of the diurnal cycle of precipitation using TRMM. J. Atmos. Oceanic Technol 19:1333–1344.
Sorooshian, S., K. Hsu, X. Gao, H. V. Gupta, B. Imam, and D. Braithwaite. 2000. Evolution of PERSIANN system satellite-based estimates of tropical rainfall. Bull. Amer. Meteor. Soc 81:2035–2046.
Vicente, G., R. A. Scofield, and W. P. Mensel. 1998. The operational GOES infrared rainfall estimation technique. Bull. Amer. Meteor. Soc 79:1881–1898.
Woodley, W. L., C. G. Griffith, J. S. Griffin, and S. C. Stroomatt. 1980. The inference of GATE convective rainfall from SMS-1 imagery. J. Appl. Meteor 19:338–408.
Wu, R., J. A. Weinman, and R. T. Chin. 1985. Determination of rainfall rates from GOES satellite images by a pattern recognition technique. J. Atmos. Oceanic Technol 2:314–330.
Xu, L., X. Gao, S. Sorooshian, P. A. Arkin, and B. Imam. 1999a. A microwave infrared threshold technique to improve the GOES precipitation index. J. Appl. Meteor 38:569–579.
Xu, L., S. Sorooshian, X. Gao, and H. Gupta. 1999b. A cloud-patch technique for identification and removal of no-rain clouds from satellite infrared imagery. J. Appl. Meteor 38:1170–1181.
APPENDIX A
The Processing Steps of the ITT Algorithm
The incremental temperature threshold algorithm segments cloud IR images starting from the coldest pixels to warmer thresholds, controlled by the following steps.
- Step 1: Locate pixels with the coldest temperature Tmin and local minimum temperatures (initializing seeds). Compute the incremental thresholds using the following:Here Tmin is the pixel temperature minimum, Tu is the temperature that discriminates between a cloudy and clear sky, ΔT is temperature interval equal to 3 K, and {}int takes the integer values.
- Step 2: Grow seeded regions from THTm to THTm+1. Assuming that at step m there are n1 seeded regions covered by temperatures lower than THTm (the seeded region Ai is numerically labeled as i = 1, 2, … , n1):AmAm1Am2AmiAmn1The coldest top temperature for each cloud region iswhere g(x, y) is the temperature of the pixel (x, y). At each step, the algorithm grows first by labeling new seed(s), then by expanding the existing regions Am until reaching the threshold THTm+1. The external boundary EB of the existing regions iswhere NB(x, y) is the set of eight immediate neighbors of the pixel (x, y). If NB(x, y) connects only to one existing region (
Ami Ami Amj jiδix,yigx,yiRepeat the process until the current THTm+1 is reached. - Step 3: Find new seeds. A new seed is assigned if a pixel is colder than THTm+1 but isolated from any existing region Am:New seeds are numerically labeled as n + 1, n + 2, … , n2. After completing the m + 1 iterations, the cloud segments areAm+1Am+11Am+12Am+1iAm+1n1Am+1n2
Step 4: Repeat steps 2 and 3 until THTm ≥ Tu.
APPENDIX B
Cloud-Patch Feature Description
Given a cloud patch with a size of Ak, total pixel count N, and pixel brightness temperature of g(x, y), those patch features listed in Table 1 are described as follows.
- Minimum temperature of a cloud patch (Tmin) is
- Mean temperature of a cloud patch (Tmean) is
- Cloud-patch area (Ak) isAkN
- Shape index (SI) is defined as the ratio of the geometric eccentricity of a cloud patch (I) to that of a round patch with the same size (I0):II0where(xi, yi) is the coordinate of pixel i, (Gx, Gy) is the geometric center of the cloud patch, withand I0 is the geometric inertia of a round patch with the same area. Given a certain size of cloud patch, the rounder-shaped patches give a smaller shape index. A perfectly circular cloud patch has the smallest SI value of 1.
- Standard deviation of cloud-patch top temperature (STD) is
- Mean value of the local standard deviation of cloud-patch top temperature (MSTD5) iswhere
STD5i - Standard deviation of local standard deviation of cloud-top temperature (
STD5std - Temperature gradient of cloud cold core (TOPG) measures the cloud-patch average pixel temperature gradient around the cold core pixel (Tmin) and serves a similar purpose as a temperature slope parameter addressed in Negri et al. (2002) to determine convective cores in order to eliminate nonprecipitating cirrus. The threshold of 15 K above the coldest pixel Tmin is empirically chosen to calculate TOPG from the Tmin to the surrounding pixels having a temperature of Tmin + 15 K:where N is the number of pixels along the border of Tmin + 15 K and μ(i) is the distance from the border pixel i to the coldest pixel.
- Gray-image texture [maximum angular second moment (MASM)] information is characterized by a set of gray-level co-occurrence matrices Hd,ϕ, with the element h(i, j)d,ϕ as the estimate of the probability that gray level i and gray level j are located at a distance d and angle ϕ, with a fixed direction (Haralick et al. 1973). The co-occurrence matrices Hd,ϕ delineate the texture information of an image on various scales and directions. The angular second moment (ASM) of the co-occurence matrix located at various distances and directions isThe ASM features computed at various directions are usually redundant. In our case, only the maximum ASM was selected:
APPENDIX C
Explanation of the Seven Highlighted Cloud Categories
The calibrated 20 × 20 Tb–R curves (Fig. 6a) form seven highlighted categories (Fig. 6b) noted as C0–C6, and they are explained as follows.
C0: Cloud minimum temperature (Tmin) is higher than 245 K, and there is a small cloud-patch size (<100 pixels), a low temperature gradient near Tmin, and mostly no rain. The slope of Tb–R is approximately 0. This category is related to warm, no-rain clouds.
C1: The range of the coldest patch pixel temperatures is 235–245 K, and the texture indices are low (TOPG ∼ 1; MSTD5 ∼ 1; STD < 4). Cloud-patch size is relatively small, and the slope of the Tb–R curve is flat. The cloud-patch average pixel rain rate is less than 0.5 mm h−1, and the cloud-patch rainfall is less than 100 mm h−1.
C2: The coldest pixel temperature of a cloud patch is similar to that of the C1 category, but the texture indices are much higher (TOPG ∼ 3; MSTD5 ∼ 3.5; STD ∼ 4–8). The slope of the Tb–R curve is high, with an average pixel rain rate of around 0.5– 1.0 mm h−1 and a total rainfall per patch of about 1000 mm h−1, which is about 10 times that of the C1 category.
C3: The coldest patch pixel temperature is around 215–235 K, and the texture indices are low (TOPG ∼ 1; MSTD5 ∼ 2.5; STD ∼ 6–8). The average pixel rain rate is around 0.1–0.5 mm h−1, and the accumulated rainfall per patch is around 500–2000 mm h−1. As shown in Fig. 6b, the slope of the Tb– R curve is flat.
C4: The coldest patch pixel temperature is around 215–235 K, and the texture indices are relatively high (TOPG ∼ 3.5; MSTD5 ∼ 4; STD ∼ 8–10). The average pixel rain rate is around 0.5–1.0 mm h−1, and the accumulated rainfall per patch is around 1000–2500 mm h−1. The maximum pixel rain rate can be as high as 60 mm h−1.
C5: This category is associated with cold cloud patches with Tmin < 215 K and low texture indices (TOPG ∼ 1.5; MSTD5 ∼ 3). The average pixel rain rate is around 0.5–1.5 mm h−1, and the accumulated patch rainfall is around 3000–6000 mm h−1. The Tb–R curve is long and mildly steep.
C6: The coldest patch pixel temperature is lower than 215 K, and the texture indices are relatively high (TOPG ∼ 4.5; MSTD5 ∼ 5). The average pixel rain rate is around 1.5–4.5 mm h−1, and the accumulated patch rainfall is around 2000–3500 mm h−1. This type of cloud patch has a Tb–R curve with a steep slope. Maximum pixel rainfall could reach 100 mm h−1.
The PERSIANN CCS model structure: (a) image preprocessing through a cloud segmentation procedure, (b) input feature extraction of the cloud patch, (c) cloud-patch feature classification (20 × 20 clusters), and (d) nonlinear temperature and rainfall (Tb–R) fitting for the 20 × 20 classified cloud clusters, with one fitting curve per cluster. Note that W and V are the connection weights for the SOFM layer and multiparameter nonlinear fitting layer, respectively. The final output of this network is the calibrated Tb–R for every classified cloud cluster
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
(a) The IR cloud image, (b) segmentation of the cloud image, using the fixed threshold of 253 K, and (c)–(h) the cloud-patch segmentation process using the ITT algorithm in this study
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
Cloud-patch features of Tmin, AREA, TOPG, and STD vs patch rainfall and patch rain probability. Note that not all of the selected features are shown here. (left) Values of cloud-patch features vs cloud-patch rainfall, and (right) interval-averaged values of cloud-patch features vs patch rain probability
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
Cloud-patch features calculated at three separate temperature levels (220, 235, and 253 K)
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
(a) The SOFM classified 20 × 20 nodes (groups) of cloud patches and (b)–(g) their associated feature maps and rainfall by training using the Jun 1999 GOES IR and radar rainfall data
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
(a) The calibrated cloud–rainfall relationships (400 Tb–R curves) from the PERSIANN CCS algorithm using IR and radar data, with each curve referring to a cloud cluster in the SOFM classification map (Fig. 5a.). (b) Several typical Tb–R curves from clusters in (a). Note that the curve of cluster 0 does not show in (b) because it is short and flat
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
Time series of statistics of hourly rain rates derived from PERSIANN CCS and PERSIANN over the Las Vegas, NV, vicinity flash-flooding storm from 1400 through 1900 UTC 8 Jul 1999. (The results were calculated at 0.25° × 0.25° grid resolution)
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
Scatterplots of radar measurements vs PERSIANN CCS–estimated daily rainfall at four spatial scales for a region located at 25°–45°N, 100°–120°W on (left) 8 Jul and (right) 9 Jul 1999
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
Comparison of daily rainfall from radar, PERSIANN, and PERSIANN CCS estimates at 0.25° × 0.25° on 9 Jul 1999
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
Cloud-patch rainfall event from 1430 to 2345 UTC 9 Jul 1999 over a region centered at 37°N, 104°W: (a) cloud evolution from beginning to end (only partial stages are shown), (b) the corresponding rainfall estimation curves along with the cloud development (dots are the Tb–rain rate data pairs after being averaged at every 0.5-K interval; curves are the nonlinear functions regressed from the data pairs), (c) time series of cloud-patch minimum pixel temperature and patch size, and (d) time series of patch-averaged pixel rain rate and patch total rainfall estimated from PERSIANN CCS and radar
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
Multiple-curve-fitting model (PERSIANN CCS) vs single-curve-fitting models: (a) the scatterplots of cloud-top pixel temperature and radar rain rates from a set of training data, (b) the Tb–R curves fitted by the PERSIANN CCS model, and (c) the Tb–R curves from several fixed function models (constant threshold, linear, or nonlinear fitting functions)
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
Time series of selected statistics evaluated at daily 0.25° scale over 25°–45°N, 100°–125°W during (a) Feb 2002 and (b) Jun–Aug 2002.
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
Plots of rain rates (mm day−1) during (a) Feb 2002 and (b) Jun–Aug 2002 for (left) rain rates from 0 to 20 mm day−1 and (right) rain rates from 20 to 60 mm day−1, both in 2 mm day−1 increments. Numbers on the x axis represent rain-rate boundaries, and numbers on the y axis are the number of rain-rate occurrences. The validation region is the same as in Fig. 12
Citation: Journal of Applied Meteorology 43, 12; 10.1175/JAM2173.1
Input features extracted from cloud patches
Input features and precipitation characteristics of selected clusters on SOFM
Statistics for PERSIANN CCS and PERSIANN estimates of hourly rain rates under a range of spatial resolutions when compared with radar observation—a case study. Note: zero-rain-rate grids were included to compute the statistical information. Sample size for comparison is 1890 at 0.25° grid resolution over the entire storm
Comparison of daily rainfall estimates from PERSIANN CCS and radar observation on 8–9 Jul 1999
Intercomparison of GPI, PERISANN, and CCS estimation vs gauge–radar data. Note that the sample size for this comparison is 2925
Averaged time series of statistical comparisons of daily rain gauge data with PERSIANN and PERSIANN CCS estimates for both cold and warm seasons
The significance test of correlation between PERSIANN and PERSIANN CCS in comparison with gauge/radar observation. Given significance level α = 0.05, tα (=1.65) < t
