• Barnes, S. L., 1964: A technique for maximizing details in numerical weather map analysis. J. Appl. Meteor., 3 , 396409.

  • Basu, B. K., 2003: Verification of model predicted precipitation over India during the summer monsoon of 1997. Mausam, 54 , 359376.

  • Basu, B. K., 2005: Some characteristics of model-predicted precipitation during the summer monsoon over India. J. Appl. Meteor., 44 , 324339.

    • Search Google Scholar
    • Export Citation
  • Cherubini, T., , Ghelli A. , , and Lalaurette F. , 2002: Verification of precipitation forecasts over the Alpine region using a high-density observing network. Wea. Forecasting, 17 , 238248.

    • Search Google Scholar
    • Export Citation
  • Cressman, G. P., 1959: An operational objective analysis system. Mon. Wea. Rev., 87 , 367374.

  • Daley, R., 1991: Atmospheric Data Analysis. Cambridge University Press, 457 pp.

  • Ghelli, A., , and Lalaurette F. , 2000: Verifying precipitation forecast using upscaled observations. ECMWF Newsletter, No. 87, ECMWF, Reading, United Kingdom, 9–17.

    • Search Google Scholar
    • Export Citation
  • Goswami, B. N., , and Mohan R. S. , 2000: A common spatial mode for intra-seasonal and inter-annual variation and predictability of the Indian summer monsoon. Curr. Sci., 79 , 11061111.

    • Search Google Scholar
    • Export Citation
  • IMD, 2002: Forecasting circular. Office of the DDGM Rep. W-969/4496-4570, India Meteorological Department, New Delhi, India, 3 pp.

  • Jolliffe, I. T., , and Stephenson D. B. , 2003: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. John Wiley and Sons, 240 pp.

  • Kalnay, E., , Sela J. G. , , Campana K. A. , , Basu B. K. , , Schwarzkopf M. D. , , Long P. E. , , Caplan P. M. , , and Alpert J. C. , 1988: Documentation of research version of the NMC medium range forecasting model. National Meteorological Center, 132 pp.

  • Karl, T. R., , Wang W. C. , , Schlesinger M. E. , , Knight R. W. , , and Portman D. , 1990: A method of relating general circulation model simulated climate to observed local climate. Part I: Seasonal statistics. J. Climate, 3 , 10531079.

    • Search Google Scholar
    • Export Citation
  • Katz, W. R., , and Murphy A. H. , 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.

  • Kuo, H. L., 1974: Further studies of the parameterization of the influence of cumulus convection on large-scale flow. J. Atmos. Sci., 31 , 12321240.

    • Search Google Scholar
    • Export Citation
  • Manabe, S., , Smagorinsky J. , , and Strickler R. F. , 1965: Simulated climatology of a general circulation model with a hydrologic cycle. Mon. Wea. Rev., 93 , 769798.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8 , 281293.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., , and Winkler R. L. , 1987: A general framework for forecast verification. Mon. Wea. Rev., 115 , 13301338.

  • Rao, Y. P., 1976: South West Monsoon. Meteor. Monogr., No. 1, India Meteorological Department, 367 pp.

  • Şen, Z., 1997: Objective analysis by cumulative semivariogram technique and its application in Turkey. J. Appl. Meteor., 36 , 17121724.

    • Search Google Scholar
    • Export Citation
  • Skelly, W. C., , and Handerson-Sellers A. , 1996: Grid box or grid point: What type of data do GCMs deliver to climate impact research? Int. J. Climatol., 16 , 10791086.

    • Search Google Scholar
    • Export Citation
  • Thiebaux, H. J., , and Pedder M. A. , 1987: Spatial Objective Analysis. Academic Press, 299 pp.

  • Tiedtke, M., 1983: The sensitivity of the time-mean large-scale flow to cumulus convection in the ECMWF model. Proc. Workshop on Convection in Large-Scale Numerical Models, Reading, United Kingdom, ECMWF, 297–316.

  • Wigly, T. M. L., , and Santer B. D. , 1990: Statistical comparison of spatial fields in model validation, perturbation and predictability experiments. J. Geophys. Res., 95 , D1. 851865.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 1995: Statistical Method in Atmospheric Science. Academic Press, 467 pp.

  • WMO, 1994: Guide to hydrological practices. WMO Rep. 168, 735 pp.

  • View in gallery

    Spatial distribution of rain gauge stations over India and NCMRWF grid points over the Andhra Pradesh, Rajasthan, and West Bengal regions. Open dots represents observation stations and solid dots represents grid points.

  • View in gallery

    (a) Observed seasonal anomaly pattern for (a) 1997 and (b) 1999. The contours are drawn as departures (%) at an interval of 5%.

  • View in gallery

    Seasonal anomaly for (a) day-1 forecasts during 1997 and (b) day-3 forecasts during 1999. The contours are drawn as departures (%) at an interval of (a) 10% and (b) 20%.

  • View in gallery

    Observed and model forecast precipitation amounts accumulated over the monsoon seasons and averaged over grid points of the three regions following AA: (top left) Andhra Pradesh, (top right) Rajasthan, and (bottom) West Bengal.

  • View in gallery

    Pattern correlation between CMAP data and (a) observed rainfall, (b) day-1 forecasts, and (c) day-3 forecasts at an interval of 0.1.

  • View in gallery

    Anomaly correlation of precipitation over India.

  • View in gallery

    Monthly RMSE distributions of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

  • View in gallery

    Monthly correlation distributions of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

  • View in gallery

    Bias scores of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

  • View in gallery

    PODs of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

  • View in gallery

    FAR scores of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

  • View in gallery

    TSS scores of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

  • View in gallery

    The year-to-year variation of weekly average precipitation as observed and forecasted for the West Bengal region, starting from June.

  • View in gallery

    The active and break phase representations in terms of daily pentad rainfall variation in different regions over West Bengal in 1999 starting from 1 Jun for the (top) sub-Himalayan, (middle) transition, and (bottom) Gangatic regions of West Bengal.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 0 0 0
PDF Downloads 0 0 0

Precipitation Forecast Verification of the Indian Summer Monsoon with Intercomparison of Three Diverse Regions

View More View Less
  • 1 School of Environmental Studies, Jadavpur University, Kolkata, India
  • 2 NCMRWF, NOIDA, Uttar Pradesh, India
© Get Permissions
Full access

Abstract

The Indian summer monsoon precipitation forecast, as well as its verification, are always of great interest because of their socioeconomic impact on the Indian subcontinent. The present work highlights the verification of quantitative precipitation forecasts of the Global Spectral Model, running at the National Center for Medium Range Weather Forecasting (NCMRWF), Noida, India. Studies like pattern correlation and anomaly correlation over all of India confirm that the model is applicable over the subcontinent. Some comparative studies are done for three diverse regions like West Bengal, Andhra Pradesh, and Rajasthan. Verification studies include both measure-oriented methods like root-mean-square error (RMSE) and Pearson’s correlation coefficient and distribution-oriented methods like bias score, false alarm ratio, probability of detection, and true skill score. The distribution-oriented verification (yes–no) is done for the daily threshold precipitation of 0.254, 2.54, 6.4, 12.8, 19.2, 32.0, 44.8, and 64.0 mm. Two years of data from 1997 and 1999 for Andhra Pradesh and Rajasthan and 5 yr of data from 1997 to 2001 are used for West Bengal. The distribution of model output during a severe rainfall situation over West Bengal is also examined to understand the usefulness of the model forecasts during those events. It can be concluded that the model is most efficient in predicting precipitation in the 2.54–12.8-mm range but the efficiency decreases rapidly for higher thresholds. Performance of the model during active and break phases of the monsoon is examined and it is found to be reasonably good. On the whole, it can be concluded that the performance of the NCMRWF model is reasonably good for day-1 forecasts and the weekly rainfall forecast is quite good for all forecast lead times.

Corresponding author address: Dr. U. K. De, School of Environmental Studies, Jadavpur University, Kolkata 700032, India. Email: deutpal2003@yahoo.com

Abstract

The Indian summer monsoon precipitation forecast, as well as its verification, are always of great interest because of their socioeconomic impact on the Indian subcontinent. The present work highlights the verification of quantitative precipitation forecasts of the Global Spectral Model, running at the National Center for Medium Range Weather Forecasting (NCMRWF), Noida, India. Studies like pattern correlation and anomaly correlation over all of India confirm that the model is applicable over the subcontinent. Some comparative studies are done for three diverse regions like West Bengal, Andhra Pradesh, and Rajasthan. Verification studies include both measure-oriented methods like root-mean-square error (RMSE) and Pearson’s correlation coefficient and distribution-oriented methods like bias score, false alarm ratio, probability of detection, and true skill score. The distribution-oriented verification (yes–no) is done for the daily threshold precipitation of 0.254, 2.54, 6.4, 12.8, 19.2, 32.0, 44.8, and 64.0 mm. Two years of data from 1997 and 1999 for Andhra Pradesh and Rajasthan and 5 yr of data from 1997 to 2001 are used for West Bengal. The distribution of model output during a severe rainfall situation over West Bengal is also examined to understand the usefulness of the model forecasts during those events. It can be concluded that the model is most efficient in predicting precipitation in the 2.54–12.8-mm range but the efficiency decreases rapidly for higher thresholds. Performance of the model during active and break phases of the monsoon is examined and it is found to be reasonably good. On the whole, it can be concluded that the performance of the NCMRWF model is reasonably good for day-1 forecasts and the weekly rainfall forecast is quite good for all forecast lead times.

Corresponding author address: Dr. U. K. De, School of Environmental Studies, Jadavpur University, Kolkata 700032, India. Email: deutpal2003@yahoo.com

1. Introduction

With the inclusion of different physical processes and the development of better parameterization processes, an operational NWP model, in the age of the supercomputing era, is now capable of predicting different seasonal phenomena successfully, especially for uniform terrain. But the predictability of an NWP model for the Indian summer monsoon (Goswami and Mohan 2000) is quite low and the situation is similar in other monsoon regions. The Indian summer monsoon season is one of the most spectacular manifestations of atmospheric circulation features, having some dominant regional characteristics as well as global connections. The southwest monsoon is characterized by the establishment of a pressure trough over all of northern India, giving rise to much-needed and widespread rainfall for cultivation. The prediction of the Indian summer monsoon is highly challenging and has considerable importance over the Indian subcontinent. So, the verification of an operational NWP model has become equally important for a season like the summer monsoon in India. Forecast verification is the process of determining the quality of a forecast through the assessment of the degree of similarity between that forecast and the observed conditions (Murphy and Winkler 1987; Murphy 1993). The verification process incorporates different methodologies to study the quality of the forecast and may be either quantitative or probabilistic. An extensive review of this aspect of the process is already available in the literature (Katz and Murphy 1997; Jolliffe and Stephenson 2003).

The present article includes verification of a quantitative precipitation forecast from the Indian National Center for Medium Range Weather Forecasting (NCMRWF) model. Some of the studies, such as (i) the monsoon rainfall anomaly from seasonal normal; (ii) the pattern correlation (Wilks 1995) between the Climate Prediction Center Merged Analysis of Precipitation (CMAP) data, and the observational data and the NCMRWF model forecast data, respectively; and (iii) the anomaly correlation (Wilks 1995) between the observed data and the NCMRWF forecast, were done over all of India, mainly to test the applicability of the model in the subcontinent.

The use of grid analyses of precipitation for forecast verification assumes some level of certainty as a product, as no standard objective analysis method has been developed for an intermittent field like precipitation. In fact, there are basically two different approaches for deriving the gridpoint values. The methods are the weight-based gridpoint average (GPA) and areal average (AA) approaches. One group of scientists treats precipitation as an areal averaged quantity (Karl et al. 1990; Skelly and Handerson-Sellers 1996) while another group (Wigley and Santer 1990; Ghelli and Lalaurette 2000; Cherubini et al. 2002) prefers to use the gridpoint approach. In the present paper, the simulated precipitation is presented as following the AA values. It is found that the two approaches do not yield very different results, so long as the precipitation is not high. The comparative study between the two approaches is presented for the West Bengal region only, which is, incidentally, a high rainfall region.

There are 36 meteorological districts in India and each meteorological district has a homogeneous character. Further verification study was carried over three geographically diverse and meteorologically contrasting regions of the country: West Bengal, Andhra Pradesh, and Rajasthan.

Some concise statistics of measure-oriented and distribution-oriented verification schemes are presented for the three concerned regions. However, additional detailed study has been performed over the entire West Bengal region, where the period of study is over 5 yr, from 1997 to 2001. The Gangetic Plain of West Bengal region falls at one end of the monsoon pressure trough, where a moist convective process is a dominant feature during the monsoon time period. The agriculturally fertile region of the country with seasonal mean rainfall of nearly 120 cm needs proper agrometeorological services for the benefit of the economy. An intensive verification of T-80 model output over this region is supposed to help to that end. There are two meteorological districts within the West Bengal region. The Gangetic Plain of West Bengal forms one meteorological district. The northern part of West Bengal and the state of Sikkim constitute the other meteorological district with very high seasonal normal rainfall of about 200 cm. As West Bengal falls in a high rainfall zone within the eastern part of the country, this region may also be considered as a representative of the entire eastern zone.

In section 2, the model overview, nature of data used, and the period of study are discussed. In section 3, the paper describes the procedures for the conversion of observation data to grid point or grid box to compare quantitative precipitation forecasts as gridpoint output or grid-area output. In section 4, the verification procedures being followed are briefly discussed. The results are presented in section 5. The paper ends with the conclusions in section 6.

2. Model overview and data used

At the NCMRWF, a global spectral model (T80) is integrated to produce 5-day forecasts from 0000 UTC initial conditions operationally. In the present study the grid spacing over India is nearly 150 km. The global model at NCMRWF uses a spectral technique for solving the dynamical set of equations. For the model (Basu 2003), global atmospheric observations collected either via in situ or remote sensing are obtained from a Global Telecommunication System computer at the Indian Meteorological Department (IMD), New Delhi, and subjected to rigorous quality control before being used for the preparation of initial conditions. The data assimilation procedure used at NCMRWF is very similar to that of NCEP (Kalnay et al. 1988). Altogether, there are 135 Gaussian grid points in the T80 model over all of India. The characteristics of the model are given in Table 1. There are 22, 23, and 12 grid points for the model in the Andhra Pradesh, Rajasthan, and West Bengal regions, respectively. All conclusions are based on the model outputs of those grid points. As already stated, extensive study has been performed for the West Bengal region for 5 yr and all conclusions are drawn from validations preformed at 12 grid points located within the region. A semidense network of rain gauge stations over India and the grid points over the three regions of study is presented in Fig. 1.

The data for all of India were collected from the IMD archive. Necessary data were created at T80 model grid points following an objective analysis procedure. The rainfall stations include nearly 500 surface observatories of IMD and more than 400 part-time observatories maintained by different government agencies. The rain gauge network is, however, scanty over West Rajasthan and comparatively richer in Gangetic West Bengal as the rain gauge stations of the Department of Agriculture, government of West Bengal, are included in the present study. All rainfall observations are for the accumulative rainfall at 0300 UTC. Precipitation is collected by standard manually operated rain gauges and measured up to the first decimal place of a millimeter. The verification was done for the summer monsoon season, which briefly covers 1 June to 30 September over India.

A simple quality control for consistency was done to identify errors in the data. In this approach the data were tested against various independent estimates of the field and accepted/rejected depending upon the degree of consistency with neighboring stations. After that, control experiments and analyses were carried out on the basis of data from more than 100 observation stations of each region for each year.

3. Objective analysis

The model predicts the meteorological parameters at specified grid points or it may be taken as an average over a grid box, which are different from the observational stations. To validate the model output at a particular grid point, one needs to upscale the data at the grid point from the data of irregularly spaced observation stations following the usual technique of objective analysis.

a. GPA approach

A wide range of objective analysis techniques are used in meteorology and oceanography. Different experiments indicate that the cumulative semivariogram (CSV) method is not subject to some of the well-known limitations (e.g., Daley 1991; Thiebaux and Pedder 1987) of empirical weighting functions. The CSV method suggested by Şen (1997), is used here to find the radius of influence as well as the best-suited mean weighing function for this region under study. It is observed that the radius of influence (R) varies from week to week. But on average, it is around 300 km for the present period of study. But that large of a radius of influence for a field like precipitation looks unusual. It is cross-checked by spatial correlation study. The scatter diagram of the correlation coefficient with distance shows that, beyond 125-km distance, the correlation becomes erratic (not shown). In fact, the maximum correlation of magnitude 0.4 is reached at 125 km when making the necessary sensitivity study. So the 125-km distance may be finalized as the radius of influence for the period of study. From the comparative analysis of experimental CSV functions of weekly averaged data, with different theoretical weighing functions like that of Cressman (1959), Barnes (1964), and Cressman powers 2, 3, 4, . . . , etc., it was concluded that the suggested function with Cressman power 2 was close to the original CSV values for the maximum number of weeks. So one can also conclude that rainfall at the region of study changes with distance with the following relation:
i1520-0434-22-3-428-e1a
i1520-0434-22-3-428-e1b
where ri,m = the distance between the ith station and the mth grid point.
In this analytical technique, the main goal is the estimation of parameter at any point, as a weighted average of the measured values at irregular sites (Şen 1997):
i1520-0434-22-3-428-e2
where i = 1, 2, . . . , n are the measurement sites, m is the prediction site, Zi is the observed meteorological value at the ith site, and Zm is the estimated value at point m.

b. AA approach

The irregularly spaced observations of precipitation may be averaged over an area around the grid point. It is assumed that in a model the gridpoint value is the representative of the average condition over the whole area associated with that grid point. In this work the area associated with each grid point (henceforth, grid box) is obtained by drawing the bisector of lines joining the nearest grid points of the model. The weights are calculated by the Thiessen polygon method (WMO 1994). The area (S) associated with each grid box is divided into a large number of smaller areas (dS). Each of the smaller areas is assigned to its nearest observation station. So the ratio of total area acquired by each station to the total area of grid box is the weight of that station under that grid box. This method of averaging depends simply on the geometry of the distribution of rain gauge stations:
i1520-0434-22-3-428-e3
where Wi is the weighted area of a Thiessen polygon, Ap is the area of one Thiessen polygon, and A is the total area of influence.
Now, the average precipitation over a grid box is estimated using the following equation:
i1520-0434-22-3-428-e4
where P is the average precipitation over a grid box, Pi is precipitation for the ith polygon, and n is the number of Thiessen polygons.

The only limitation of this method is that it does not consider the inhomogeneity in the topography or circulation. However, in coastal regions, due to smooth homogeneities, there is no discernible bias in rainfall over the sea and the adjoining land areas, and the average-based method shows better correlation with the observations.

4. Verification measures

a. Monsoon characteristics, choice of three regions, and two particular years

The southwest summer monsoon over India is a unique system in and of itself. It always manifests itself over the same time of the year. Onset of the monsoon takes place over the Kerala coast from late May to early June. The moist wind divides itself into two branches: the Arabian Sea branch moves directly through the western part of the country and the Bay of Bengal branch through the northeastern part of the country. The monsoon prevails over the entire country in July and August, and the withdrawal starts around the middle of September (Rao 1976).

In the present study, emphasis has been placed on West Bengal, rather than the Andhra Pradesh and Rajasthan regions. There are three meteorological districts within Andhra Pradesh. In each of the three districts, the monsoon normal rain is significantly less compared with either of the meteorological districts in the West Bengal region. The three meteorological districts depict three different facets of the monsoon. According to IMD records, the interior northern part of the state, called the Telengana meteorological district, has a seasonal normal rainfall of about 76 cm; the coastal meteorological district has a seasonal normal rainfall of about 57 cm; while the rain-shadow district of Rayasalema has a lower seasonal normal rainfall of only 38 cm. This district is contiguous with the state of Tamil Nadu, where the contribution from southwest monsoon rain is not so conspicuous.

The western part of the state of Rajasthan is occupied by the Thar Desert, but the eastern part of the state is comparatively greener. Two meteorological districts fall within the state. The eastern district has a seasonal normal rainfall of nearly 63 cm, but the western district has only 26 cm of seasonal normal rain. The western Rajasthan district has, in fact, the lowest seasonal normal rainfall in India. The seasonal normal rainfall total successively decreases from the West Bengal region to Andhra Pradesh and then to the Rajasthan region. As the monsoonal rainfall anomaly pattern has contrasting behavior in the two years of 1997 and 1999 over the three regions, these two particular years have been chosen for the present study (Figs. 2a and 2b).

According to IMD, there was a slightly delayed onset to the 1997 monsoon season, with heavy rains beginning approximately 1 week after the normal date. The monsoon covered most of the entire country by 19 July. Seasonal rainfall was near normal over major regions of India. It was below normal over northern Andhra Pradesh and eastern Rajsthan, and there was excessive rainfall over western Rajasthan and West Bengal (Fig. 2a). In northern India, excessive rainfall is sometimes associated with strong midlatitude westerly disturbances moving across the region. In 1997, these rains brought the total to near normal for the season as a whole, despite a weaker-than-normal and highly variable monsoon circulation during most of the period.

During the 1999 monsoon season, precipitation was below average for the Indian subcontinent as a whole, with much of the rainfall deficit being observed during July and August. The precipitation deficits were observed over western and southern India, and excess rain was observed dominantly for the West Bengal region (Fig. 2b).

b. Measures-oriented verification schemes

The World Meteorological Organization (WMO) has recommended routine computation of verification indices for the basic variables of the model. It is possible to develop simple measures that convey some information about the forecast performance. All of the measures-oriented results are spatially analyzed for day-1 and day-3 prediction during June–September of 1997 and 1999 for the above-mentioned three regions.

Pattern correlation for seasonal rainfall is done to compare observations with the NCEP merged CMAP forecast data over all of India, and to elucidate the monsoon variability over the subcontinent.

Another standard WMO-prescribed proof scheme is anomaly correlation, which helps to examine the model forecast in the limit of climatic normal. In general, precipitation has strong variability; in spite of that, this study over the entirety of India can enrich our understanding about the model’s performance on a seasonal time scale.

The RMSE is a good measure of the accuracy of prediction on either side and it is better than the mean absolute error. However, the results cannot convey the relation between observed and predicted data.

Pearson’s correlation provides the degree of similarity in a relationship that exists between the observed and forecasted variables. The value is a measure of the phase relationship as well as the reliability of the model forecast.

c. Distributions-oriented verification schemes

In addition to the measures-oriented forecast value, a number of categorical (yes–no) statistics are applied for the distributions-oriented verification scheme. In this study four different measures are used to evaluate the performance of NCMRWF model forecasts following the contingency table for different threshold values. The precipitation thresholds for the study are chosen to be 0.254, 2.54, 6.4, 12.8, 19.2, 32.0, 44.8, and 64.0 mm (equivalent to 0.01, 0.1, 0.25, 0.5, 0.75, 1.25, 1.75, and 2.25 in.) (Basu 2003). The eight precipitation thresholds give rise to classes I–IX, with precipitation in excess of 64.0 mm considered as class IX. Standard verification schemes like bias score (BS), probability of detection (POD), false alarm ratio (FAR), and true skill score (TSS) are used to consider the performance of the model (Wilks 1995).

Here, the BS is equal to the number of rain forecasts divided by the total number of rain events observed. POD is equal to the number of hits divided by the total number of rain observations. This is a simple measure of the proportion of rain events successfully forecasted by the model. FAR is equal to the number of false alarms divided by the total number of times rain is forecasted. This represents the model tendency to forecast rain events at other observations. TSS is the ratio of the number of correct predictions above that obtainable by chance to the number of occurrences above that likely by chance. When the number of correct predictions equals that obtainable by chance, TSS is zero. But, when the prediction is perfect, TSS is 1.

5. Verification results

a. Measures-oriented results

The overall seasonal rainfall predicted by the T80 model for 1997 captures well the excess seasonal rainfall over the West Bengal and Rajasthan regions, but it also overpredicts slightly for southern India (Fig. 3a represents only the day-1 forecast). The anomaly is presented in terms of percentage of departure. In the year 1999, the model again captures well the excess and below normal seasonal rain in West Bengal and Rajasthan, respectively, but the rain is overpredicted in the eastern part of southern India. This is true for both day-1 and day-3 forecasts (Fig. 3b represents only the day-3 forecast). In general, the model is not efficient at predicting small rainfall amounts correctly. Apart from that, the Western Ghat mountains influence the model rainfall in southern India. This might be the reason for the failure of the model in that region.

The seasonal rainfall forecasted by the model is plotted for various forecast lead times (Fig. 4). The observed precipitation is taken as the initial value for comparison with the forecasts. The average rainfall forecasted over Andhra Pradesh was nearly the same for 1997 and 1999. But the tendency to predict excessive amounts of rainfall with forecast lead time may be noted. In the case of Rajasthan, the excessive rainfall in 1997 cannot be captured by the model. However, the normal seasonal rainfall in 1999 can be captured by the model. In the West Bengal region, the model has an unusual tendency of forecasting with lead time. A comparatively dry bias in the day-1 forecast stands out as the dominant feature of this region.

The pattern correlation is created by considering the combined data of 2 yr, and taking the monthly mean value at each grid point. The pattern correlation (Fig. 5a) of the observed data with the NCEP merged CMAP data produces strong persistence over a large part of India and the value rises to 0.9 over a large area of central India. However, the value falls to 0.4 over the Ragasthan region. In the case of the CMAP data and the T80 1-day forecast data (Fig. 5b), the value decreases to 0.7 over central India, though it increases over the Rajasthan region and remains almost the same as in the previous case for other regions. In the case of CMAP data and the T80 3-day forecast data (Fig. 5c), the value decreases over central India and it again decreases to 0.4 over the Rajasthan region. The magnitude is lowest over southern India.

The anomaly correlation was done over all of India by considering the 1997 and 1999 data separately, taking the monthly mean at each grid point, and taking the 30-yr climatic normal at each grid point as the base value. The value never went below 0.35 when considering the 1-day to 4-day forecasts (Fig. 6). This shows that the model has some skill at predicting precipitation over India and the model forecasts have a higher trend correlation with the observed precipitation in comparison with the European Centre for Medium-Range Weather Forecasts model forecasts (Basu 2005).

The average RMSE of the daily rainfall is 13.0 mm over Andhra Pradesh and it remains relatively steady over time. However, the RMSE increases with longer forecast lead times. The RMSE was maximum during the two active months of July 1997 and August 1999. July 1999 was abnormally dry and the RMSE was also low. While Rajasthan experiences low rainfall, the RMSE is also low (below 10 mm for most of the period). There is also no significant difference between different lead times. In July 1997, when Rajasthan experienced above normal rain, the RMSE also increased appreciably. In Fig. 7, the monthly RMSE values are presented for different years.

In the case of the West Bengal region, the RMSEs showed greater values as the rainfall increased. They varied from 15 to 33 mm for the day-1 forecast during the 5-yr period of study and the value was higher for the day-3 forecast. However, when the precipitation was heavy, the RMSE values were less for the day-3 forecasts, as the model could not capture the heavy rain and the gap decreased in the day-3 forecasts. The RMSE was significantly large during July 1997 and September 1999. On both occasions, the rainfall was quite high. Higher RMSEs were also observed in July 1998, June 2000, and September 2000. On all of these occasions, the rainfall increased. It should be noted that the presentation is done for 2 yr only. Also, there is little difference between the GPA and AA approaches.

The Pearson correlation coefficient over a month for daily rainfall evaluated at all grid points varied from 0.6 to 0.25 during the period of study (1997 and 1999 for Andhra Pradesh and Rajasthan, 1997–2001 for West Bengal). It was above 0.3 on most of occasions. The day-1 correlation was always much better than the day-3 correlation. The model correlations were better over the Andhra Pradesh and West Bengal regions compared with the Rajasthan region (Fig. 8). Sometimes the model cannot capture heavy rainfall in both time and space. This trend is particularly noticeable for West Bengal, which brought down the correlation value. In the case of Rajasthan, the observation network is comparatively poorer, which might be one of the reasons for the low correlation value. Again, the two objective analysis approaches produce almost the same results.

b. Distributions-oriented results

The results presented in this section are for day-1 and day-2 forecasts, considering data of 2 yr for Andhra Pradesh and Rajasthan, and of 5 yr for West Bengal. Altogether, there are nine classes used to verify the distributions-oriented rainfall results.

These results mainly point to the capability of the model to capture different rainfall classes. The bias aspect of the model is expressed in Fig. 9. The score indicates underprediction or overprediction depending on value of BS, if it is less or greater than 1, respectively. In general, BS shows underprediction at higher classes (VIII–IX) and also for class I. Overprediction mainly occurs for the middle classes (III–VI). In the case of Andhra Pradesh, the model is strongly biased for class V. In Rajasthan, the first two classes are more bias free (values close to 1). In the case of West Bengal, the model bias is nondispersive and systematic for classes I–III. Higher dispersion is noted for the rest of the threshold values. In this case, the bias is higher for day-1 prediction compared with day-2 prediction. The model bias is nearly zero for class IX. This is true for both of the approaches.

The equivalent diagrams for POD as a function of rainfall are shown in Fig. 10. In general, the model is inefficient at detecting higher classes, particularly above class VII. In the cases of Andhra Pradesh and West Bengal, the model looks more reliable for classes III and IV, as 50% of the cases are detected. However, for Andhra Pradesh, there exist significant differences for day-1 and day-2 forecasts up to class IV. For Rajasthan, the model is more reliable for detecting classes I and II. In general, most of the NWP models detect rainfall of lower threshold values with greater efficiency. Detection power also depends on the distribution of rainfall among the different classes. In the present study, a trace or no-rain situation (class I) is much more likely for Rajasthan compared with other two regions. It is also noted that the detection power gradually falls beyond class IV.

The FARs are quite low for lower and higher threshold values (Fig. 11). In the case of Andhra Pradesh and West Bengal, the different values of FAR follow an almost Gaussian distribution. However, the peak occurs at class IV for Andhra Pradesh and at class III for West Bengal. In the case of Rajasthan, the value decreases linearly, if plotted against different classes. The FAR becomes zero for higher rainfall classes as the model has less capability to forecast these classes of rainfall. Thus, the model FAR response is more consistent compared with the other scores.

TSS measures the discrimination power of the model for different classes. In general, one can conclude that the model is quite efficient at detecting the lower value of rainfall and starting from class V, the efficiency decreases systematically with higher values of rainfall (Fig. 12). In the case of West Bengal, the day-2 prediction is slightly better than the day-1 prediction for the lower classes for both the objective analysis techniques. The TSSs have significant differences for the day-1 and day-2 forecasts in the case of Andhra Pradesh. If the overall performance is taken into account, the day-1 prediction is best when considering all three regions.

c. Intensive study over West Bengal

The experimental region is broadly classified into two districts by the IMD according to their weather characteristics. These are the sub-Himalayan foothills of West Bengal and Gangetic West Bengal. The foothills region has high topography and most of the time receives high rainfall totals. Meanwhile, the coastal Gangetic plain experiences different weather systems formed at the Bay of Bengal: low pressure, depressions, deep depressions, or cyclonic storms. But a mixed behavior is observed in between the two regions. In fact, five grid points each fall within the sub-Himalayan foothills region and the coastal Gangetic plain region, and only two grid points fall within the transitional region. The seasonal accumulated rainfall has significant year to year variability. The interannual variation with respect to the 5-yr mean rainfall is negative for the year 2001, that is, less rainfall was received; otherwise, a slight excess of rainfall was received in the other years. The mean difference between the forecasts and observations is dipole in nature, with excess rainfall in the southwest region and less in the northeast region. Thus, an overprediction in the model is noticed at the minimum rainfall region whereas the model predicts less for the maximum rainfall region. On the other hand, the middle region has less error in its prediction. This difference in rainfall is quite persistent for the 5 yr of study.

The intraseasonal variability is another important feature of the Indian summer monsoon that needs to be reproduced by any atmospheric model to be successful. There is an increase in weekly rainfall after the first or second week of June with the advent of monsoon onset. The weekly rainfall gradually increases through the second week of July and then decreases depending upon the break phase condition. Another weekly peak in rainfall is noticed during August depending upon favorable condition. During September, deep depressions and cyclonic storms occasionally form over the Bay of Bengal, severely affecting the region with sudden rises in weekly rainfall.

During June an active monsoon epoch is usually associated with enhanced activity in the westerly wind regime of the monsoon, while in September such activity is more often due to the interaction of a monsoon trough with the extratropical westerlies, moving across the north of the country, or enhanced activity of the easterly wind from the Bay of Bengal in association with a low pressure system moving westward along the seasonal monsoon trough.

The weekly accumulated precipitation during the monsoons of 1997–2001, as forecasted by the NCMRWF model, is shown in Fig. 13, along with the observed (GPA) variation. It is noted that the forecasts for different lead times are quite consistent with the observations. The correlation coefficients for different lead times are as follows: day 1, 0.72; day 2, 0.73; day 3, 0.71; and day 4, 0.62.

The Indian summer monsoon usually has a low-frequency oscillation. But, in addition to this low-frequency oscillation, shorter-period fluctuations of different dry and wet spells are also noted in the monsoon rainfall. These are directly related to the break and active periods, respectively. During break monsoon conditions, rainfall vanishes over the Gangetic West Bengal, but it increases over the foothills of the Himalayas. In one season there may be a number of active and break spells of the monsoon. It is also found that cloud bands with zonal orientation move northward from the near-equatorial latitudes in the monsoon region. The fluctuation in cloud cover is strongly related to the active and break spells of the monsoon.

The active and break monsoon attributes are discussed in the next section, with respect to three regions: the northern, middle, and southern regions. To analyze the above attributes, five-point averaging was applied over daily rainfall time series along with regional weather reports (Fig. 14). The upper, middle, and lower panels in Fig. 14 represent the northern, middle, and southern regions, respectively. Fast Fourier transform filter smoothening is applied by removing Fourier components with frequencies higher than a cutoff frequency. The methodology is available online (at Originlab’s Web page: http://www.originlab.com/index.aspx?s=8&lm=115&pid=109).

The active and break phases can be detected well by the model, provided the rain intensity in the active phase is less. However, in some cases the model fails to detect close rain peaks. In early June, the model overpredicts the rainfall in the southern region. In fact, the model-predicted rain has greater spatial homogeneity than the observations. So, any observed rain patch is predicted over a larger area, though it might be of less intensity, which in the long run leads to a greater magnitude of regional rainfall.

In addition to the above verification, some direct comparisons of the different monsoon features were also done. This frequency-based direct comparison is intended to help a forecaster gather knowledge about the response of the model during different events. For a detailed study of the frequency distribution, the precipitation amounts are divided into six main categories in Table 2 and the limits of these categories are similar to those introduced by the IMD (2002). The averaging made at each grid point/box reduces the magnitude of the heaviest precipitation and increases the trace–no-rain situation. The average daily precipitation has nearly equal frequencies (30.0%) for the trace and moderate categories (Table 2). Occurrences of the very heavy rain category decrease from 0.8% to 0.14% as we consider different years. A distinction between GPA and AA may be noticed. In comparison with the observations, the model forecast was less frequent except for the light and moderate categories. The maximum deviation occurred in the year 2001 in terms of the percentage of frequency. The maximum positive deviation in the frequency was noticed for light rain but the maximum negative deviation occurred for the trace–no-rain situation. The model generally sensed the later category of rain as the former category. On the other hand, the model had no response for very heavy precipitation (>130.0 mm). The average forecast by the global model is presented in Table 3 for various forecast lead times. The average 24-h precipitation amount per grid point varies from year to year and it is lowest for 2001, whereas it is highest for 1998. It may be noted from Fig. 4 that day-1 forecasts for the seasonal rainfall are always less than the observations, and then they rise for the other lead time forecasts.

Occasionally, a heavy and incessant rainfall situation occurred during the favorable active monsoon phase. It was observed that sometimes nearly 80 mm day−1 of rainfall occurred at a station for two to four consecutive days. This would lead to a rise in the water level of all major rivers as well as their tributaries over the region. This situation gives rise to severe flooding.

The statistical distribution of forecasted rainfall up to 48 h is given in Table 3 when 80 mm or more rain is observed. It is noted that the model maximum forecast of rain was less than 25 mm in almost 50% of cases and the capabilities for forecasting rainfall above 55 mm are barely 7.57% and 4.87% for GPA and AA, respectively, for day-1 predictions. The highest rainfall estimate delivered by the model varied from 23.9 to 112.3 mm for the 5 yr of study. Sometimes the day-2 prediction was more than the day-1 prediction. The rain amounts derived by GPA and AA differed considerably in many cases. It should also be pointed out that the model response to a severe rainfall situation was best in 1999 and worst in 2001.

6. Conclusions

It is well known that the T80 model used by NCMRWF is an older version of the present NCEP model. In spite of that, the overall performance of the T80 model for the prediction of the Indian summer monsoon is reasonably satisfactory.

From anomaly study, it turns out that the anomaly between forecasts and observations is consistent over a large part of India, though the difference is large near higher or lower observed rainfall regions. One can infer from the pattern and anomaly correlation studies over all of India that NCMRWF forecasts of seasonal rainfall have good consistency. This consistency, however, decreases systematically with increasing forecast lead time.

The average RMSE of daily rainfall is expectedly high in higher rainfall regions, and low in lower rainfall regions. The RMSE rises with lead time. The overall Pearson correlation coefficient of daily rainfall varies from 0.6 to 0.25, though it is mostly above 0.3. The range of correlation values is consistent for each of the three diverse regions being considered.

From distributions-oriented studies like BS, POD, FAR, and TSS, one can conclude that the model has good capability to forecast rainfall from classes I–VI but the efficiency decreases rapidly for higher thresholds. However, the capability to predict class I rainfall is quite poor for the West Bengal region.

From intensive studies over West Bengal for 5 yr, some additional conclusions may be made. First, the weekly accumulative rainfall is reproduced remarkably well by the model for all forecast lead times. The model has a general tendency toward overpredicting the frequency of occurrence of precipitation events in the light and moderate categories, and to underpredict the frequency in the higher rainfall categories. It can also be concluded that the performance of the model during the active and break phases of the monsoon is reasonably good.

It should be mentioned in closing that, in general, the model fails to capture heavy rainfall; otherwise, the performance of the model is quite good. So long as the rainfall is not heavy, the objective analysis methodologies AA and GPA produce similar results.

Acknowledgments

The authors thank the Department of Science and Technology, government of India, for the sanction of the research project, “Intercomparison between NCMRWF output and observed data in the Gangetic Plains of West Bengal.” The present work is a part of that project. The authors are also thankful to the Agriculture Department, government of West Bengal; IMD; and NCMRWF for providing the respective data.

REFERENCES

  • Barnes, S. L., 1964: A technique for maximizing details in numerical weather map analysis. J. Appl. Meteor., 3 , 396409.

  • Basu, B. K., 2003: Verification of model predicted precipitation over India during the summer monsoon of 1997. Mausam, 54 , 359376.

  • Basu, B. K., 2005: Some characteristics of model-predicted precipitation during the summer monsoon over India. J. Appl. Meteor., 44 , 324339.

    • Search Google Scholar
    • Export Citation
  • Cherubini, T., , Ghelli A. , , and Lalaurette F. , 2002: Verification of precipitation forecasts over the Alpine region using a high-density observing network. Wea. Forecasting, 17 , 238248.

    • Search Google Scholar
    • Export Citation
  • Cressman, G. P., 1959: An operational objective analysis system. Mon. Wea. Rev., 87 , 367374.

  • Daley, R., 1991: Atmospheric Data Analysis. Cambridge University Press, 457 pp.

  • Ghelli, A., , and Lalaurette F. , 2000: Verifying precipitation forecast using upscaled observations. ECMWF Newsletter, No. 87, ECMWF, Reading, United Kingdom, 9–17.

    • Search Google Scholar
    • Export Citation
  • Goswami, B. N., , and Mohan R. S. , 2000: A common spatial mode for intra-seasonal and inter-annual variation and predictability of the Indian summer monsoon. Curr. Sci., 79 , 11061111.

    • Search Google Scholar
    • Export Citation
  • IMD, 2002: Forecasting circular. Office of the DDGM Rep. W-969/4496-4570, India Meteorological Department, New Delhi, India, 3 pp.

  • Jolliffe, I. T., , and Stephenson D. B. , 2003: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. John Wiley and Sons, 240 pp.

  • Kalnay, E., , Sela J. G. , , Campana K. A. , , Basu B. K. , , Schwarzkopf M. D. , , Long P. E. , , Caplan P. M. , , and Alpert J. C. , 1988: Documentation of research version of the NMC medium range forecasting model. National Meteorological Center, 132 pp.

  • Karl, T. R., , Wang W. C. , , Schlesinger M. E. , , Knight R. W. , , and Portman D. , 1990: A method of relating general circulation model simulated climate to observed local climate. Part I: Seasonal statistics. J. Climate, 3 , 10531079.

    • Search Google Scholar
    • Export Citation
  • Katz, W. R., , and Murphy A. H. , 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.

  • Kuo, H. L., 1974: Further studies of the parameterization of the influence of cumulus convection on large-scale flow. J. Atmos. Sci., 31 , 12321240.

    • Search Google Scholar
    • Export Citation
  • Manabe, S., , Smagorinsky J. , , and Strickler R. F. , 1965: Simulated climatology of a general circulation model with a hydrologic cycle. Mon. Wea. Rev., 93 , 769798.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8 , 281293.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., , and Winkler R. L. , 1987: A general framework for forecast verification. Mon. Wea. Rev., 115 , 13301338.

  • Rao, Y. P., 1976: South West Monsoon. Meteor. Monogr., No. 1, India Meteorological Department, 367 pp.

  • Şen, Z., 1997: Objective analysis by cumulative semivariogram technique and its application in Turkey. J. Appl. Meteor., 36 , 17121724.

    • Search Google Scholar
    • Export Citation
  • Skelly, W. C., , and Handerson-Sellers A. , 1996: Grid box or grid point: What type of data do GCMs deliver to climate impact research? Int. J. Climatol., 16 , 10791086.

    • Search Google Scholar
    • Export Citation
  • Thiebaux, H. J., , and Pedder M. A. , 1987: Spatial Objective Analysis. Academic Press, 299 pp.

  • Tiedtke, M., 1983: The sensitivity of the time-mean large-scale flow to cumulus convection in the ECMWF model. Proc. Workshop on Convection in Large-Scale Numerical Models, Reading, United Kingdom, ECMWF, 297–316.

  • Wigly, T. M. L., , and Santer B. D. , 1990: Statistical comparison of spatial fields in model validation, perturbation and predictability experiments. J. Geophys. Res., 95 , D1. 851865.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 1995: Statistical Method in Atmospheric Science. Academic Press, 467 pp.

  • WMO, 1994: Guide to hydrological practices. WMO Rep. 168, 735 pp.

Fig. 1.
Fig. 1.

Spatial distribution of rain gauge stations over India and NCMRWF grid points over the Andhra Pradesh, Rajasthan, and West Bengal regions. Open dots represents observation stations and solid dots represents grid points.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 2.
Fig. 2.

(a) Observed seasonal anomaly pattern for (a) 1997 and (b) 1999. The contours are drawn as departures (%) at an interval of 5%.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 3.
Fig. 3.

Seasonal anomaly for (a) day-1 forecasts during 1997 and (b) day-3 forecasts during 1999. The contours are drawn as departures (%) at an interval of (a) 10% and (b) 20%.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 4.
Fig. 4.

Observed and model forecast precipitation amounts accumulated over the monsoon seasons and averaged over grid points of the three regions following AA: (top left) Andhra Pradesh, (top right) Rajasthan, and (bottom) West Bengal.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 5.
Fig. 5.

Pattern correlation between CMAP data and (a) observed rainfall, (b) day-1 forecasts, and (c) day-3 forecasts at an interval of 0.1.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 6.
Fig. 6.

Anomaly correlation of precipitation over India.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 7.
Fig. 7.

Monthly RMSE distributions of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 8.
Fig. 8.

Monthly correlation distributions of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 9.
Fig. 9.

Bias scores of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 10.
Fig. 10.

PODs of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 11.
Fig. 11.

FAR scores of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 12.
Fig. 12.

TSS scores of model-predicted rainfall at grid points over (top left) Andhra Pradesh, (top right) Rajasthan, (bottom left) West Bengal GPA, and (bottom right) West Bengal AA.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 13.
Fig. 13.

The year-to-year variation of weekly average precipitation as observed and forecasted for the West Bengal region, starting from June.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Fig. 14.
Fig. 14.

The active and break phase representations in terms of daily pentad rainfall variation in different regions over West Bengal in 1999 starting from 1 Jun for the (top) sub-Himalayan, (middle) transition, and (bottom) Gangatic regions of West Bengal.

Citation: Weather and Forecasting 22, 3; 10.1175/WAF1010.1

Table 1.

Characteristics of the NCMRWF model.

Table 1.
Table 2.

Percentage of frequency distribution of the number of rainy days in different precipitation categories.

Table 2.
Table 3.

Frequency-based characteristics of model prediction during extreme rainfall (≥80 mm day−1) situations over West Bengal.

Table 3.
Save