Editorial Type:
Article
Article Type:
Research Article

Neural Network Training for Prediction of Climatological Time Series, Regularized by Minimization of the Generalized Cross-Validation Function

Yuval Department of Earth and Ocean Sciences, University of British Columbia, Vancouver, British Columbia, Canada

Search for other papers by Yuval in
Current site
Google Scholar
PubMed Close
Print Publication:
01 May 2000
DOI:
https://doi.org/10.1175/1520-0493(2000)128<1456:NNTFPO>2.0.CO;2
Page(s):
1456–1473
Restricted access

Abstract

Neural network (NN) training is the optimization process by which the relation between the NN input and output is established. A new formulation for the NN training is presented where an NN model is reconstructed such that it produces predicted output data optimally fitting the observed ones. The optimal level of fit is determined by minimization of the generalized cross-validation function, which is integrated in the training. The training process is fully automated, does not require the user to set aside data for validation, and enables objective testing and evaluation of the predictions. Results are demonstrated and discussed using synthetic data produced by Lorenz’s low-order circulation model and on real data from the equatorial Pacific.

Corresponding author address: Yuval, Dept. of Earth and Ocean Sciences, University of British Columbia, 1461-6270 University Boulevard, Vancouver, BC V6T 1Z4, Canada.

Email: yuval@ocgy.ubc.ca

Abstract

Neural network (NN) training is the optimization process by which the relation between the NN input and output is established. A new formulation for the NN training is presented where an NN model is reconstructed such that it produces predicted output data optimally fitting the observed ones. The optimal level of fit is determined by minimization of the generalized cross-validation function, which is integrated in the training. The training process is fully automated, does not require the user to set aside data for validation, and enables objective testing and evaluation of the predictions. Results are demonstrated and discussed using synthetic data produced by Lorenz’s low-order circulation model and on real data from the equatorial Pacific.

Corresponding author address: Yuval, Dept. of Earth and Ocean Sciences, University of British Columbia, 1461-6270 University Boulevard, Vancouver, BC V6T 1Z4, Canada.

Email: yuval@ocgy.ubc.ca

Save
Cover Monthly Weather Review
Monthly Weather Review

  • Abu-Mostafa, Y., 1990: Learning from hints in neural networks. J. Complex.,6, 192–198.

  • ——, 1993: Hints and the VC dimension. Neural Comput.,5, 278–288.

  • Allen, D., 1974: The relationship between variable selection and data augmentation and a method for prediction. Technometrics,16, 125–127.

  • Barnston, A. G., and Coauthors, 1994: Long-lead seasonal forecasting—Where do we stand? Bull. Amer. Meteor. Soc.,75, 2097–2114.

  • Bishop, C. M., 1995: Neural Networks for Pattern Recognition. Clarendon Press, 482 pp.

  • Chauvin, Y., 1990: Generalization performance of overtrained backpropagation. Neural Networks. EURASIP Workshop Proceedings, L. B. Almedia and C. J. Wellekens, Eds., Springer-Verlag, 46–55.

  • Cichocki, A., and R. Unbehauen, 1993: Neural Networks for Optimization and Signal Processing. John Wiley and Sons, 526 pp.

  • Cybenko, G., 1989: Approximation by superpositions of a sigmoidal function. Math. Control, Signal, Syst.,2, 303–314.

  • Demuth, H., and M. Beale, 1998: Neural Network Toolbox version 3.1. The Math Works, Inc. [Available from The Math Works, Inc., 24 Prime Park Way, Natick, MA 01760-1500.].

  • Finnoff, W., F. Hergert, and A. G. Zimmermann, 1993: Improving models selection by nonconvergent methods. Neural Networks,6, 771–783.

  • Geman, S., E. Bienenstock, and R. Doursat, 1992: Neural networks and the bias/variance dilemma. Neural Comput.,4, 1–58.

  • Golub, G. H., M. Heath, and G. Wahba, 1979: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 21, 215–223.

  • Goswami, P., and Srividya, 1996: A novel neural network design for long range prediction of rainfall pattern. Current Sci.,70, 447–457.

  • Grieger, B., and M. Latif, 1994: Reconstruction of the El-Niño attractor with neural networks. Climate Dyn.,10, 267–276.

  • Haber, E., and D. W. Oldenburg, 2000: A GCV based method for nonlinear ill-posed problems. Comput. Geosci., in press.

  • Hastenrath, S., L. Greischar, and J. van Heerden, 1995: Prediction of the summer rainfall over South Africa. J. Climate,8, 1511–1518.

  • Hewitson, B. C., and R. G. Crane, 1996: Climate downscaling: Techniques and application. Climate Res.,7, 85–95.

  • Hornik, K., M. Stinchcombe, and H. White, 1989: Multilayer feedforward networks are universal approximators. Neural Networks,2, 359–366.

  • Lorenz, E. N., 1984: Irregularity: A fundamental property of the atmosphere. Tellus,36A, 98–110.

  • ——, 1990: Can chaos and intransitivity lead to interannual variability? Tellus,42A, 378–389.

  • MacKay, D. J. C., 1995: Probable networks and the plausible predictions—A review of practical Bayesian methods for supervised neural networks. Network Comput. Neural Syst.,6, 469–505.

  • Michaelsen, J., 1987: Cross-validation in statistical forecast models. J. Climate. Appl. Meteor.,26, 1589–1600.

  • Navone, H. D., and H. A. Ceccatto, 1994: Predicting Indian monsoon rainfall—A neural network approach. Climate Dyn.,10, 305–312.

  • Reynolds, R. W., and T. M. Smith, 1994: Improved global sea surface temperature analysis using optimum interpolation. J. Climate,7, 929–948.

  • Rumelhart, D. E., G. E. Hinton, and R. J. Williams, 1986: Learning internal representations by error propagation. Parallel Distributed Processing, D. E. Rumelhart, J. L. McClelland, and P. R. Groups, Eds., Vol. 1, The MIT Press, 318–362.

  • Smith, T. M., R. W. Reynolds, R. E. Livezey, and D. C. Stokes, 1996:Reconstruction of historical sea surface temperatures using orthogonal functions. J. Climate,9, 1403–1420.

  • Tang, B., W. W. Hsieh, A. H. Monahan, and F. T. Tangang, 2000: Skill comparisons between neural networks and canonical correlation analysis in predicting the equatorial Pacific sea surface temperatures. J. Climate,13, 287–293.

  • Tangang, F. T., W. W. Hsieh, and B. Tang, 1997: Forecasting the equatorial Pacific sea surface temperature by neural network models. Climate Dyn.,13, 135–147.

  • ——, ——, and ——, 1998a: Forecasting the regional sea surface temperature of the tropical Pacific by neural network models, with wind stress and sea level pressure as predictors. J. Geophys. Res.,103, 7511–7522.

  • ——, B. Tang, A. H. Monahan, and W. W. Hsieh, 1998b: Forecasting ENSO events: A neural network–extended EOF approach. J. Climate,11, 29–41.

  • Tichonov, A. N., and V. Y. Arsenin, 1977: Solutions of Ill-Posed Problems. John Wiley and Sons, 258 pp.

  • Wahba, G., 1977: Practical approximate solutions to the linear operator equations when the data are noisy. SIAM J. Numer. Anal.,14, 651–667.

  • ——, 1990: Spline Models for Observational Data. SIAM Philadelphia, 169 pp.

  • ——, 1995: Generalization and regularization in nonlinear learning systems. The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed., The MIT Press, 426–430.

  • Weigend, A. S., 1996: Time series analysis and prediction. Mathematical Perspectives on Neural Networks, P. Smolensky, M. C. Mozer, and D. E. Rumelhart, Eds., Lawrence Earlbaum, 395–449.

  • Woodruff, S. D., R. J. Slutz, R. L. Jenne, and P. M. Steurer, 1987: A comprehensive ocean-atmosphere data set. Bull. Amer. Meteor. Soc.,68, 1239–1250.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 204 66 12
PDF Downloads 100 32 5