• Badescu, V., 1993: Use of Willmott’s index of agreement to the validation of meteorological models. Meteor. Mag.,122, 282–286.

  • Barnston, A. G., and H. M. Van den Dool, 1993: A degeneracy in cross-validated skill in regression-based forecasts. J. Climate,6, 963–977.

    • Crossref
    • Export Citation
  • Browne, M. W., 1970: A critical evaluation of some reduced-rank regression procedures. Research Bulletin 70-21, Educational Testing Service, Princeton, NJ.

    • Crossref
    • Export Citation
  • ——, 1975a: Predictive validity of a linear regression equation. Br. J. Math. Statist. Psychol.,28, 79–87.

    • Crossref
    • Export Citation
  • ——, 1975b: A comparison of single sample and cross-validation methods for estimating the mean squared error of prediction in multiple linear regression. Br. J. Math. Statist. Psychol.,28, 112–120.

    • Crossref
    • Export Citation
  • ——, and R. Cudeck, 1989: Single sample cross-validation indices for covariance structures. Mult. Behav. Res.,24, 445–455.

    • Crossref
    • Export Citation
  • ——, and ——, 1992: Alternative ways of assessing model fit. Sociol. Meth. Res.,21, 230–258.

    • Crossref
    • Export Citation
  • Camstra, A., and A. Boomsma, 1992: Cross-validation in regression and covariance structure analysis. Soc. Meth. Res.,21, 89–115.

    • Crossref
    • Export Citation
  • Copas, J. B., 1983: Regression, prediction, and shrinkage. J. Roy. Statist. Soc.,45B, 311–354.

    • Crossref
    • Export Citation
  • Cotton, W. R., G. Thompson, and P. W. Mielke, 1994: Real-time mesoscale prediction on workstations. Bull. Amer. Meteor. Soc.,75, 349–362.

    • Crossref
    • Export Citation
  • Efron, B., 1983: Estimating the error rate of a prediction rule: Improvement on cross-validation. J. Amer. Statist. Assoc.,78, 316–331.

    • Crossref
    • Export Citation
  • Elsner, J. B., and C. P. Schmertmann, 1993: Improving extended- range seasonal predictions of intense Atlantic hurricane activity. Wea. Forecasting,8, 345–351.

  • ——, and ——, 1994: Assessing forecast skill through cross-validation. Wea. Forecasting,9, 619–624.

    • Crossref
    • Export Citation
  • Geisser, S., 1975: The predictive sample reuse method with applications. J. Amer. Statist. Assoc.,70, 320–328.

    • Crossref
    • Export Citation
  • Glick, N., 1978: Additive estimators for probabilities of correct classification. Pattern Recog.,10, 211–222.

  • Gray, W. M., C. W. Landsea, P. W. Mielke, and K. J. Berry, 1992: Predicting Atlantic seasonal hurricane activity 6–11 months in advance. Wea. Forecasting,7, 440–455.

  • Hess, J. C., and J. B. Elsner, 1994: Extended-range hindcasts of tropical-origin Atlantic hurricane activity. Geophys. Res. Lett.,21, 365–368.

    • Crossref
    • Export Citation
  • Hora, S. C., and J. B. Wilcox, 1982: Estimation of error rates in several-population discriminant analysis. J. Marketing Res.,19, 57–61.

    • Crossref
    • Export Citation
  • Horst, P., 1966: Psychological Measurement and Prediction. Wadsworth, 455 pp.

  • Huberty, C. J., J. M. Wisenbaker, and J. C. Smith, 1987: Assessing predictive accuracy in discriminant analysis. Mult. Behav. Res.,22, 307–329.

    • Crossref
    • Export Citation
  • Kelly, F. P., T. H. Vonder Haar, and P. W. Mielke, 1989: Imagery randomized block analysis (IRBA) applied to the verification of cloud edge detectors. J. Atmos. Oceanic Technol.,6, 671–679.

    • Crossref
    • Export Citation
  • Lachenbruch, P. A., 1967: An almost unbiased method of obtaining confidence intervals for the probability of misclassification in discriminant analysis. Biometrics,23, 639–645.

    • Crossref
    • Export Citation
  • ——, and M. R. Mickey, 1968: Estimation of error rates in discriminant analysis. Technometrics,10, 1–11.

    • Crossref
    • Export Citation
  • Lee, T. J., R. A. Pielke, and P. W. Mielke, 1995: Modeling the clear- sky surface energy budget during FIFE 1987. J. Geophys. Res.,100, 25585–25593.

    • Crossref
    • Export Citation
  • Livezey, R. E., A. G. Barnston, and B. K. Neumeister, 1990: Mixed analog/persistence prediction of seasonal mean temperatures for the USA. Int. J. Climatol.,10, 329–340.

  • MacCallum, R. C., M. Roznowski, C. M. Mar, and J. V. Reith, 1994:Alternative strategies for cross-validation of covariance structure models. Mult. Behav. Res.,29, 1–32.

    • Crossref
    • Export Citation
  • Maltz, M. D., 1994: Deviating from the mean: The declining significance of significance. J. Res. Crime Delinq.,31, 434–463.

    • Crossref
    • Export Citation
  • McCabe, G. J., and D. R. Legates, 1992: General-circulation model simulations of winter and summer sea-level pressures over North America. Int. J. Climatol.,12, 815–827.

    • Crossref
    • Export Citation
  • Michaelsen, J., 1987: Cross-validation in statistical climate forecast models. J. Climate Appl. Meteor.,26, 1589–1600.

    • Crossref
    • Export Citation
  • Mielke, P. W., K. J. Berry, C. W. Landsea, and W. M. Gray, 1996: Artificial skill and validation in meteorological forecasting. Wea. Forecasting,11, 153–169.

    • Crossref
    • Export Citation
  • Mosier, C. I., 1951: Symposium: The need and means of cross-validation, I. Problems and designs of cross-validation. Educ. Psych. Meas.,11, 5–11.

    • Crossref
    • Export Citation
  • Mosteller, F., and J. W. Tukey, 1977: Data Analysis and Regression. Addison-Wesley, 586 pp.

  • Murphy, A. H., and R. L. Winkler, 1984: Probability forecasting in meteorology. J. Amer. Statist. Assoc.,79, 489–500.

    • Crossref
    • Export Citation
  • Nicholls, N., 1985: Predictability of interannual variations of Australian seasonal tropical cyclone activity. Mon. Wea. Rev.,113, 1144–1149.

    • Crossref
    • Export Citation
  • Picard, R. R., and R. D. Cook, 1984: Cross-validation of regression models. J. Amer. Statist. Assoc.,79, 575–583.

    • Crossref
    • Export Citation
  • ——, and K. N. Berk, 1990: Data splitting. Amer. Statist.,44, 140–147.

    • Crossref
    • Export Citation
  • Snee, R. D., 1977: Validation of regression models: Methods and examples. Technometrics,19, 415–428.

    • Crossref
    • Export Citation
  • Stone, M., 1974: Cross-validatory choice and assessment of statistical predictions. J. Roy. Statist. Soc.,36B, 111–147.

    • Crossref
    • Export Citation
  • ——, 1978: Cross-validation: A review. Math. Operationsforsch. Statist., Ser. Statistics,9, 127–139.

    • Crossref
    • Export Citation
  • Subrahmanyam, M., 1972: A property of simple least squares estimates. Sankhya,34B, 355–356.

  • Toussaint, G. T., 1974: Bibliography on estimation of missclassification. IEEE Trans. Inf. Theory,20, 472–479.

    • Crossref
    • Export Citation
  • Tucker, D. F., P. W. Mielke, and E. R. Reiter, 1989: The verification of numerical models with multivariate randomized block permutation procedures. Meteor. Atmos. Phys.,40, 181–188.

    • Crossref
    • Export Citation
  • Watterson, I. G., 1996: Nondimensional measures of climate model performance. Int. J. Climatol.,16, 379–391.

    • Crossref
    • Export Citation
  • Willmott, C. J., 1982: Some comments on the evaluation of model performance. Bull. Amer. Meteor. Soc.,63, 1309–1313.

    • Crossref
    • Export Citation
  • ——, S. G. Ackleson, R. E. Davis, J. J. Feddema, K. M. Klink, D. R. Legates, J. O’Donnell, and C. M. Rowe, 1985: Statistics for the evaluation and comparison of models. J. Geophys. Res.,90, 8995–9005.

    • Crossref
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 120 20 3
PDF Downloads 18 9 0

A Single-Sample Estimate of Shrinkage in Meteorological Forecasting

View More View Less
  • 1 Department of Statistics, Colorado State University, Fort Collins, Colorado
  • | 2 Department of Sociology, Colorado State University, Fort Collins, Colorado
  • | 3 NOAA Climate and Global Change Fellowship, NOAA/AOML/Hurricane Research Division, Miami, Florida
  • | 4 Department of Atmospheric Science, Colorado State University, Fort Collins, Colorado
Restricted access

Abstract

An estimator of shrinkage based on information contained in a single sample is presented and the results of a simulation study are reported. The effects of sample size, amount, and severity of nonrepresentative data in the population, inclusion of noninformative predictors, and least (sum of) absolute deviations and least (sum of) squared deviations regression models are examined on the estimator. A single-sample estimator of shrinkage based on drop-one cross-validation is shown to be highly accurate under a wide variety of research conditions.

* Current affiliation: NOAA/AOML/Hurricane Research Division, Miami, Florida.

Corresponding author address: Dr. Paul W. Mielke Jr., Department of Statistics, Colorado State University, Fort Collins, CO 80523-1877.

Email: mielke@lamar.colostate.edu

Abstract

An estimator of shrinkage based on information contained in a single sample is presented and the results of a simulation study are reported. The effects of sample size, amount, and severity of nonrepresentative data in the population, inclusion of noninformative predictors, and least (sum of) absolute deviations and least (sum of) squared deviations regression models are examined on the estimator. A single-sample estimator of shrinkage based on drop-one cross-validation is shown to be highly accurate under a wide variety of research conditions.

* Current affiliation: NOAA/AOML/Hurricane Research Division, Miami, Florida.

Corresponding author address: Dr. Paul W. Mielke Jr., Department of Statistics, Colorado State University, Fort Collins, CO 80523-1877.

Email: mielke@lamar.colostate.edu

Save