Design and Analysis of Numerical Experiments

Kenneth P. Bowman Department of Atmospheric Sciences, University of Illinois at Urbana-Champaign, Urbana, Illinois

Search for other papers by Kenneth P. Bowman in
Current site
Google Scholar
PubMed
Close
,
Jerome Sacks Department of Statistics and The Beckman Institute, University of Illinois at Urbana-Champaign, Urbana, Illinois

Search for other papers by Jerome Sacks in
Current site
Google Scholar
PubMed
Close
, and
Yue-Fang Chang Department of Statistics and The Beckman Institute, University of Illinois at Urbana-Champaign, Urbana, Illinois

Search for other papers by Yue-Fang Chang in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

Calculations with numerical models are often referred to as numerical experiments, by analogy to classical laboratory experiments. Usually, many numerical experiments are carried out to determine the response of a numerical model to variations of internal or external parameters over some range of interest. If individual experiments are inexpensive to carry out, and if the number of independent parameters is small, it may be possible to search the entire parameter space of the model. This is difficult, however, if the dimension of the parameter space is even moderately large or the codes are expensive to run.

In this paper methods are presented for the design and analysis of numerical experiments that are especially useful and efficient in multidimensional parameter spaces. The analysis method, which is similar to kriging in the spatial analysis literature, fits a statistical model to the output of the numerical model. As an example, the method is applied to a fully nonlinear, global, equivalent-barotropic dynamical model. The statistical model also provides estimates of the uncertainty of predicted numerical model output, which can provide guidance on where in the parameter space to conduct further experiments, if necessary. The method can provide major improvements in the efficiency with which numerical sensitivity experiments are conducted.

Abstract

Calculations with numerical models are often referred to as numerical experiments, by analogy to classical laboratory experiments. Usually, many numerical experiments are carried out to determine the response of a numerical model to variations of internal or external parameters over some range of interest. If individual experiments are inexpensive to carry out, and if the number of independent parameters is small, it may be possible to search the entire parameter space of the model. This is difficult, however, if the dimension of the parameter space is even moderately large or the codes are expensive to run.

In this paper methods are presented for the design and analysis of numerical experiments that are especially useful and efficient in multidimensional parameter spaces. The analysis method, which is similar to kriging in the spatial analysis literature, fits a statistical model to the output of the numerical model. As an example, the method is applied to a fully nonlinear, global, equivalent-barotropic dynamical model. The statistical model also provides estimates of the uncertainty of predicted numerical model output, which can provide guidance on where in the parameter space to conduct further experiments, if necessary. The method can provide major improvements in the efficiency with which numerical sensitivity experiments are conducted.

Save