1. Introduction and scope

The development of conceptual rainfall–runoff models for use in hydrologic predictions has been driven by several needs, one of which is operational flood forecasting (Sorooshian 1997). These conceptual models vary in complexity, but nearly all have parameters for which values must be estimated. In spite of the advances in “physically” based modeling, there is general agreement among scientists and practicing hydrologists that a certain level of calibration is required to obtain successful streamflow predictions. Although some parameters can be derived directly from knowledge of physical watershed characteristics, others must be adjusted or “tuned” to get acceptable simulations of observed streamflows.

The traditional and most widespread approach to model calibration involves “manual” (also called “expert”) adjustment of the parameter values to improve the model response, based on visual inspection of the observed and simulated hydrographs. The hydrologist will typically attempt to reproduce the hydrograph peaks (amount and timing), flood volumes, recession slopes, and base flow. However, for models such as the Sacramento Soil Moisture Accounting Model (SAC-SMA;Burnash et al. 1973; Burnash 1995) and “SNOW-17” (Anderson 1973, 1978), used by the National Weather Service (NWS) for flood forecasting, this approach requires considerable training and experience. Further, it is typically laborious and time consuming, particularly when numerous parameters with interacting effects must be adjusted.

The NWS is currently undergoing a process of modernization, including installation of the Advanced Hydrologic Prediction System (AHPS). Under AHPS, the river forecast centers (RFCs) have been issued deadlines for the calibration of the SAC-SMA and SNOW-17 models to the roughly 4000 forecast points in this system (J. Ingram 1996, personal communication). The North Central RFC (NCRFC), one of the NWS’s largest river forecast centers, is in the process of calibrating these models for approximately 900 basins. To help in meeting these deadlines, the NCRFC (among others) has been investigating the availability of reliable methods to reduce the calibration time.

During the past three decades, considerable research has been performed on the development of automated methods to aid the model calibration process. The classical single-objective automatic approach, based on optimization theory, requires the definition of a mathematical measure (an objective function such as least squares or maximum likelihood) of the differences between the observed and simulated hydrograph. An optimization algorithm is then used to adjust the parameters toward values that minimize (or maximize, if appropriate) this function. Although the method is fast and objective, it has not received widespread acceptance among operational hydrologists. For example, NWS personnel have explored the use of automatic calibration methods (with various single-objective functions) but found that visual inspection of the hydrograph reveals areas of concern, such as poor matching of recessions and unacceptable flow biases. Use of different objective functions has not helped to resolve this problem. Therefore, poor confidence in the capabilities of automatic methods has inhibited their usage at RFCs for speeding model calibration.

The work reported in this paper is an outcome of discussions between University of Arizona (UA) researchers and NWS personnel at calibration workshops and training sessions organized by the NWS Office of Hydrology (Arkansas–Red Basin RFC in September 1998, NWS Hydrologic Research Laboratory in April 1999, and Northwest RFC in July 1999). The discussions motivated interest in the development of “hybrid” methodologies that would combine the strengths of the manual (expert) and automatic calibration strategies. The goal is to satisfy the operational need for a calibration scheme that will rapidly provide parameter estimates that are acceptable to NWS operational hydrologists (i.e., “first-cut” calibrations that provide acceptable simulations at the forecast points) and that can be refined manually later if necessary. As a result of this collaborative effort, a multistep automatic calibration scheme (MACS) procedure is under development that seeks to emulate the operational progression of steps followed by NWS hydrologists during manual calibration.

This paper presents the results of a study in which the MACS approach was applied to six operational forecast points in the NCRFC forecast area. The procedure compares favorably with the manual calibrations conducted by NCRFC hydrologists. Based on this study, the NCRFC has opted to perform further testing of the MACS procedure at a large number of forecast points that constitute the Grand River (Michigan) forecast group. Although the MACS technique reported here has been developed with specific reference to the SAC-SMA and SNOW-17 models of the NWS, these models are generally representative of the class of conceptual watershed models, and the methods presented here are, therefore, generally appropriate for calibration of various hydrologic models of this type.

A background discussion on automatic calibration is presented in the next section. The study area and forecast points used for this study are specified in section 3. Details of the SAC-SMA and SNOW-17 models are described in section 4, along with a brief overview of the UA Shuffled Complex Evolution (SCE-UA) scheme. The MACS methodology is presented in detail in section 5. Results are presented in section 6, and conclusions are summarized in section 7.

2. Automatic calibration

Automatic calibration methods have evolved significantly since early endeavors reported by Dawdy and O’Donnell (1965), Nash and Sutcliffe (1970), Ibbitt (1970), Ibbitt and O’Donnell (1971), Monroe (1971), and Johnston and Pilgrim (1976). During the last two decades, the evolution of these methods has been motivated by 1) the need to simplify and speed up the calibration process, 2) the need to assign some objectivity and confidence to the calibration process (and hence, model predictions), and 3) the lack of numerous expert calibrators available for each watershed model (Sorooshian and Gupta 1995).

Various issues that have arisen in the context of research into automatic calibration methods have included conceptually unrealistic parameter values, poor model performance on validation period (vs calibration period), and the inability of the algorithms to find a “single” best parameter set [see discussions by Gupta and Sorooshian (1994), Duan et al. (1993), and Gupta et al. (1998)]. More recent research has resulted in the development of global search procedures (L. E. Brazil and W. F. Krajewski 1987, personal communication; Brazil 1988; Duan et al. 1992, 1993; Sorooshian et al. 1993) and multicriteria optimization schemes (Gupta et al. 1998; Yapo et al. 1998), providing calibrators with better tools for obtaining a “best” parameter set for their watershed models.

The implementation of an automatic calibration procedure includes the selection of:

1. an objective function,
2. an algorithm to search the parameter space,
3. a period of historical data against which to calibrate the model, and
4. termination criteria used to determine when to stop the search.

The selected objective function implicitly gives rise to a response surface, and an iterative optimization algorithm or search procedure is used to search this surface for an “optimum” point, within the user-defined constraints (usually parameter ranges). Optimization methods have been categorized as “local search strategies,” such as the pattern search method (Hooke and Jeeves 1961), the Rosenbrock method (Rosenbrock 1960), and the simplex method (Nelder and Mead 1965), or as “global search strategies,” including the random search method (Karnopp 1963), the adaptive random search (Masri et al. 1978; Pronzato et al. 1984), and, more recently, the SCE-UA method (Duan et al. 1992). The SCE-UA method, a global search procedure based on the nonlinear simplex method, uses concepts from the random search procedures, along with the strength of the downhill simplex method.

As part of the automatic calibration process, a historical period of data against which to calibrate the model must also be selected. In selecting a calibration period, the goal is to choose a dataset representative of the various hydrologic phenomena experienced by the watershed. Research by Yapo et al. (1996) indicates that approximately 8 years of data, specifically including some of the wettest years, are adequate to ensure a quality calibration. This is consistent with the experience of NWS hydrologists.

Termination criteria are needed to determine when to stop the iterative search. Methods that have been used include parameter convergence, function convergence, or a maximum number of iterations. When an algorithm is unable to appreciably change parameter values and improve the objective function value, parameter convergence is achieved. Function convergence occurs when the algorithm is unable to improve the objective function beyond a predefined increment in one or more iterations. A calibrator also may set a maximum number of iterations to stop the search procedure, ensuring that the algorithm does not enter an endless loop.

3. Study area

This study involved collaboration between UA and NCRFC, located in Chanhassen, Minnesota. NCRFC’s area of forecasting responsibility encompasses all of Minnesota, Wisconsin, and Michigan, portions of North Dakota, South Dakota, Iowa, Illinois, and Indiana, along with portions of southern Canada. Major drainages include the upper Mississippi River, St. Lawrence River, and Hudson Bay.

The six headwater basins selected by the NCRFC for this study are all located within the upper Mississippi-River drainage. Four are located within the Des Moines River Forecast Group: the South River, at Ackworth, Iowa; Cedar Creek, near Bussey, Iowa; the Middle River, near Indianola, Iowa; and the North River, near Norwalk Iowa. The remaining two belong to the Minnesota River Forecast Group: the Redwood River, near Russell, Minnesota, and the Le Sueur River, near Rapidan, Minnesota (Fig. 1). Although the time series data available for these watersheds may not be of “research quality,” they represent the type and quality of data with which NWS hydrologists must typically work. Each of the time series was checked and quality controlled by NCRFC personnel and has been used by the NCRFC hydrologists for manual calibration of the models to these watersheds. They are, therefore, representative of typical operational conditions and provide a rigorous and realistic test of the automatic methods under development.

The study basins, along with relevant data, are listed in Table 1. The available data include basin precipitation, temperature, and streamflow records from 1948 to 1993. Average midmonth evapotranspiration (ET) values are used to estimate the ET-demand curve in the SAC-SMA. An 11-yr period, 1971–81, was used for automated calibration of the models. The entire period from 1948 to 1993 was used for evaluation of the results.

4. Models

Since 1985, the majority of RFCs have been using a highly integrated, centralized software system known as the NWS River Forecast System (NWSRFS; Page 1996). The system is a highly interdependent batch-operating system containing various models and algorithms to aid the RFCs in their hydrologic forecasting responsibilities. The operational part of the system contains several river forecasting models that the RFC may use, including the Antecedent Precipitation Index (API) and the SAC-SMA, along with a snowmelt model, SNOW-17.

5. Multistep automatic calibration scheme

a. Introduction

As mentioned in the previous section, the goal of this study was to develop a viable automatic calibration strategy that uses only the resources currently available (within the OPT3 procedure) to NWS hydrologists and that gives results comparable to those that can be obtained by manual means. To set the stage for this goal, an initial series of independent single-step calibration runs was performed using each of the six objective functions available within NWSRFS. The results of a DRMS calibration are shown in Fig. 3. The RFC manual calibration run for the 1976 water year (WY) is shown in Fig. 3a, and the single-step optimization run is displayed in Fig. 3b. The tendency of the DRMS to try to match the peaks while providing strongly biased simulations of other parts of the hydrograph is clearly apparent. The DRMS calibration never adequately returns to observed base flow. Forecasters rely on reasonable simulations of hydrograph recessions to have confidence that the soil-moisture states are being simulated correctly in preparation for a subsequent flood event. Other criteria gave varying results, but all had similarly unacceptable biases in various portions of the hydrograph.

Based on this initial analysis, two objective functions were chosen as suitable for use in the development of a multistep automatic calibration approach. The LOG function [Eq. (1)] was selected for calibration of the parameters that influence the hydrograph recessions (lower-zone parameters), and the DRMS function [Eq. (2)] was selected for calibration of the parameters affecting high-flow events. These functions are defined as follows:

where Q_sim,t = simulated flows, and Q_obs,t = observed flows at time step t.

Automatic stepwise schemes to calibrate hydrologic models have been investigated by other researchers. Harlin (1991) developed the process-oriented calibration (POC) automatic calibration scheme to apply to the Hydrologiska Byråns Vattenbalansavdelning (HBV) model. The POC method involves two steps, calibrating different sets of parameters with different objective functions over relevant subperiods of the data. Brazil (1988) developed an interactive multistage semiautomated (manual) method to calibrate the SAC-SMA. However, discussions with NWS personnel indicate that this method has never been integrated into NWS operations.

6. Results

a. Calibration period

The 11-yr period of WY 1971–81 was selected for model calibration because it consisted of several high-flood events along with periods where drier conditions prevailed. The parameter estimates provided by both the MACS approach and the NCRFC manual calibrations were compared and evaluated by running simulations over the entire historical record of data, 1948–93.

The evaluations are based both on subjective visual inspection of the modeled versus simulated hydrographs and on several statistics used routinely by the NWS RFCs to evaluate model performance. These statistics include monthly and flow-interval percent biases, overall percent biases, and overall DRMS values. The DRMS is calculated using Eq. (2), and the percent bias is defined as follows:

DRMS and %Bias statistics were computed for all model runs during this study, using a statistical package within NWSRFS (“STAT-QME”). Visual inspections of hydrographs included comparisons of peak simulations, recession slopes, precipitation typing, and timing of spring snowmelt. These comparisons were performed for both the 11-yr calibration period (WY 1971–81) and the entire 46-yr evaluation period (WY 1948–93).

1) Step 1: Estimation of base flow parameters

Figure 4 illustrates the results for the South River basin for WY 1973. Daily MAP and MAT time series are at the top of Fig. 4a, with hydrographs of the RFC manual calibration versus observed flows shown in Fig. 4b and MACS Step 1 calibration versus observed flows displayed in Fig. 4d. Calibration residuals (the difference between observed and simulated flows) for each of the simulations are displayed in Figs. 4c and 4e. The hydrographs depicted in Figs. 4b and 4d are presented using a simple transformation of the flows that expands the lower end of the flow scale (for a better view of the recessions) while still maintaining a reasonable visual perspective of the higher flows. The transformed flow Q is calculated as follows:

where λ = 0.3 in this transformation. (Note: a value of λ = 0 equates to a log transformation, and λ = 1 equates to no transformation. The transformation λ = 0.3 was chosen after evaluating the visual effect provided by a full range of λ values.)

Both the MACS step-1 and RFC calibrations slightly underestimate peak flow values; this underestimation is especially evident for the flow event near day 90, where both calibration methods have missed this event. Other high-flow events for WY 1973 are somewhat better estimated by MACS than by the RFC calibration. The MACS step-1 simulation also does a better job of simulating recession flows than does the RFC manual calibration. Figure 5 shows the results for a different water year (WY 1978). Both simulations capture most of the high-flow events (near days 200 and 355), but the MACS method models recession of the flood peak at day 200 much better than does the RFC simulation. Other flow events during this water year (days 40 and 220) are generally estimated more accurately (particularly the overall recessions) with MACS than with RFC calibrations.

2) Step 2: Calibration of upper-zone SAC-SMA and SNOW-17 parameters

Step 2 of MACS is designed to estimate higher-flow events better and to match peak flows while reducing overall DRMS and %Bias on flows. Figure 6 shows the evolution of the DRMS and %Bias statistics for the different steps in the MACS procedure, as well as a comparison with the RFC manual calibration results for the South River basin. The second step in MACS can be seen to reduce dramatically the overall %Bias (from 20% to 4.9%) while slightly improving the overall DRMS (this is typical of the results for all six basins).

3) Step 3: Refinement of base flow parameters

MACS step 2 provides a significant improvement in model performance when viewed in terms of the overall DRMS and %Bias statistics, but the monthly percent biases for most of the months (see Fig. 7) are still considerably larger than those provided by the RFC calibration. This difference is especially evident during the winter months, where MACS step-2 %Biases run as large as 40%. MACS step 3 performs a fine-tuning adjustment of the base flow parameters, while keeping the upper-zone and snow parameters fixed at the values estimated through steps 1 and 2. Figures 6 and 7 show that the monthly %Biases for the South River are significantly improved through this step, with only slight deterioration in the overall %Bias statistic. The monthly biases are now less than approximately 5% and much more in line with RFC simulations. However, this step provided significant improvements for only some of the study basins.

Figures 8 and 9 display the WY 1973 hydrographs resulting from MACS step 3 and the RFC manual calibrations for the Cedar Creek (Iowa) and Le Sueur (Minnesota) River basins, respectively. Note that the hydrologic behavior of the Minnesota River basins is different from that of the Iowa River basins. On the Cedar Creek basin, the MACS calibration (Fig. 8d) provides results that are nearly identical to the RFC manual calibration (Fig. 8b). Both simulations slightly underestimate most of the flow events while fitting the recessions fairly well. Both methods miss the small event near day 250, indicating possible problems with the precipitation data. On the Le Sueur River basin, the MACS calibration (Fig. 9d) does a slightly better job than the RFC calibration does (Fig. 9b) in matching the observed flows for most of this water year (Fig. 9). Both calibrations have a slight problem near day 110, but MACS does a better job of estimating the first big runoff event on day 160. Both simulations underestimate the next big flow event, near day 225. The MACS and RFC procedures were both able to give reasonable quality calibrations for the basins belonging to the Iowa River forecast group, but they both had difficulty providing good-quality calibrations for the two Minnesota River basins (Le Sueur and Redwood). We speculate that this result may be due to a combination of problems with the data (precipitation inputs) and/or difficulties in modeling the Minnesota terrain (topographically very flat, low flows, deep snowpack, etc.).

As an additional test, flow duration curves were constructed for the six basins. The observed flows and results for the RFC and MACS calibrations are presented (Fig. 10) for the 11-yr calibration period for the step-3 calibration of the South River. The MACS calibration tends to match the observed flows throughout the range of flows. However, although this traditional flow duration curve plot shows the general trend of the simulations, it is difficult to visualize how well the simulation performs at the higher range of flow values. Figure 11 illustrates modified flow duration curves, with the x axis expanded at the higher flow values by use of a logarithmic scale. Five of the basins are plotted with parameters from step 3 of MACS, and the Le Sueur River basin is illustrated with parameters estimated after step 4 (discussed below). On four of the six basins (South River, Cedar Creek, North River, and the Redwood River), the MACS simulation tends to match higher flow values better. On the Middle River basin, both RFC and MACS tend to oversimulate flow values, and, on the Le Sueur River basin, both RFC and MACS undersimulate observed flow values at the high end.

4) Step 4: Adjustment of ET values

As a final, but optional, step, a final check of monthly %Biases may reveal trends that call for an adjustment of previously estimated ET parameters. The current version of OPT3 does not allow for automatic optimization of the monthly ET demand curve. A manual fine-tuning or adjustment of the ET curve (ETadj), using monthly percent biases as a guide, may produce more accurate streamflows during all seasons. After the first three calibration steps with MACS, the Le Sueur and Middle River basins still had monthly biases that were unreasonably high and showed seasonal trends. Using monthly percent biases and ET values from nearby basins as a guide, monthly ET parameters for both basins were adjusted manually to bring the %Biases into a more reasonable range (ideally <10%). Figure 12 illustrates final monthly %Biases for the MACS method for the Middle River and Le Sueur River basins after ET adjustment. Monthly %Biases are plotted for steps 1, 2, 3, and 4. Because there was no change in base flow parameters when running step 3 on the Le Sueur River, steps 2 and 3 are plotted together. The plot for the Middle River basin displays all four steps of the MACS method. The adjustment of the ET parameters significantly reduced the monthly biases for both basins, as evidenced by the decrease in %Biases from step 3 to step 4 (ETadj) for most of the months. This step was only necessary on the Middle River and Le Sueur River basins and was not performed on other basins.

Final parameter values obtained for the six study basins are listed in Table 4. The MACS and RFC calibrations converged to slightly different values on the majority of basins. Parameter bounds used in the calibration were considered representative for the study area and are the same bounds that the NCRFC “typically” uses in their calibrations. It is also noted that, although four of the headwater basins are within the Des Moines River Forecast Group (South River, Cedar Creek, Middle River, and North River), it is obvious from the RFC and MACS calibrations that the headwaters respond differently and that parameter values obtained in these four basins do vary from one another. This result lends credence to the NWS requisite that a headwater in each river basin should be calibrated before proceeding downstream in the river system.

Last, it is observed that parameters on some of the calibrations, most notably UZTWM, LZTWM, and ZPERC, tended to be at or near the bounds of the parameter ranges specified by NCRFC. Additional studies were performed by widening the limits on the above-mentioned parameters; however, preliminary results from these studies were inconsistent and inconclusive. Further research into the “acceptable” optimization limits for specific regions is required but is beyond the scope of this paper.

7. Conclusions

Advances in technology during the past few decades have made improved calibration tools available to the NWS hydrologists responsible for producing timely and accurate river forecasts. There is increasing interest among RFC hydrologists in the use of automatic procedures for reducing the number of person hours required for producing model calibrations. A simple and straightforward multistep automatic calibration scheme for calibration of river forecasting models has been developed and tested. The study reported here illustrates how the MACS procedure can be incorporated into the operational setting to provide quality first-cut calibrations in a timely manner. A number of statistical measures, as well as visual hydrograph comparisons, were used in this study. Based on these measures, the MACS procedure yielded calibrations that are comparable to, or, in several cases, slightly better than, those obtained by NCRFC hydrologists using the traditional manual approach. Under the NWS modernization program, RFCs are under pressure to produce quality calibrations for all river forecast points in a relatively short period of time. A time-saving automated or semiautomated procedure such as MACS can help the RFCs to achieve this goal. The number of person hours spent using MACS was estimated to be around 3–4 hours per basin, with computer time per calibration estimated to be from 6 to 8 hours per run (IBM-RS6000, Model 591, single processor). An extra advantage of an automated procedure is that the hydrologists are free to perform other forecast duties while the computer runs are being performed. In contrast, the NCRFC hydrologists spend at least 15–20 hours for manual calibration of a forecast point.

Although the MACS procedure developed here provides acceptable first-cut calibrations for operational basins, there are limitations with the stepwise optimization approach. The multiobjective approach under development at UA (Gupta et al. 1998; Boyle et al. 2000) allows hydrologists to examine the trade-off associated with using different objective functions and is expected to provide better support for decision making in the calibration of river forecast points. However, it will be some time before the multiobjective procedure is fully understood and will be ready for operational use. In the short term, methods such as MACS can allow the NWS River Forecast Centers to exploit more fully the time-saving tools available to them within NWSRFS in the production of timely and accurate river-flow forecasts.