1. Introduction
The U.S. Army Corps of Engineers (USACE) Wave Information Study (WIS) program (Tracy and Cialone 2004) is establishing a multidecade wave climatology for the Pacific basin to support a variety of coastal planning and engineering activities. To facilitate selection of an appropriate wave hindcast technology, the performance of three modern numerical spectral wave models is evaluated in the Pacific basin over calendar year 2000. The technologies evaluated include the third-generation wave model WAM cycle 4.5 (Günther 2002), the third-generation wave model WAVEWATCH III version 2.22 (Tolman 1997, 1999, 2002), and the second-generation wave model WAVAD version 4c (Resio and Perrie 1989).
A significant challenge in evaluating large temporal- or spatial-scale wave hindcasts is the need to statistically reduce millions of spectral estimates to a meaningful measure of prediction skill yet retain a sufficient level of detail to identify model strengths and deficiencies. Wave model validation is typically accomplished using the limited amount of information contained in mean or “bulk” wave parameters, obtained from integral properties of the spectrum (Cardone et al. 1996; Hsu et al. 2002; O’Reilly et al. 1996; Tolman et al. 2002). As these quantities represent averages over all existing wave systems, they provide only a general measure of model performance and can mask higher-order deficiencies.
Here we show that improved measures of hindcast skill can be obtained through use of wave spectral components. A wave component is defined as a region of enhanced energy in the directional wave spectrum S( f, θ) attributed to a windsea or swell system propagating from a specific wind-generation event on the ocean surface. Wave components are identified and extracted using spectral partitioning methods. Originally proposed by Gerling (1992), wave partitioning allows the identification and grouping of component wave systems from spatially and temporally distributed observations of directional wave spectra. A limitation to the Gerling approach is that only those portions of the spectrum that rise above a constant threshold are used to determine the features of particular wave components. Hasselmann et al. (1994) improved on this method by dividing the spectrum into subset domains based on an inverted catchment area approach, thus using the entire spectral region of each peak to compute wave component parameters. Voorrips et al. (1997) used the Hasselmann et al. method to implement a buoy data assimilation scheme for numerical modeling. Hanson (1996) and Hanson and Phillips (2001, hereafter HP01) made improvements to the Hasselmann et al. method by adding wave height, period, and direction clustering routines to track the evolution of individual wave components. Furthermore, they performed wave dispersion calculations to estimate the source time and location of the resulting wave systems. Two primary limitations of the technique have been the lack of an efficient approach to partitioning large arrays of wave spectra and the requirement for directional wave data. The majority of available ground truth stations are nondirectional.
Using wave component data in wave model verification would provide a significant advantage over bulk spectral parameters. Although Beal (1989) and Hasselmann et al. (1994) compared wave model output to buoy and satellite observations at the wave system level, the results were only qualitative in nature and lacked specific metrics for evaluating model performance. Bidlot et al. (2005) showed how important it is to look into the wave spectral domain to gain some insight into model deficiencies. Here we demonstrate a new wave component approach to quantify numerical wave model performance and provide key diagnostic information on model deficiencies. This technique is facilitated through improvements to the spectral partitioning methods of HP01 to allow efficient processing of large wave spectra arrays and through inclusion of nondirectional spectral data in the analysis. The results of temporal correlation and quantile–quantile analyses on wave component data are synthesized into a convenient set of performance scores. We demonstrate this capability on full Pacific basin hindcasts obtained from three numerical modeling technologies forced by identical wind fields over calendar year 2000. Model validations were conducted at seven deep-water buoy sites from the National Data Buoy Center (NDBC) and the Coastal Data Information Program (CDIP). The results facilitated selection of a hindcast technology for the WIS multidecade study.
2. Performance evaluation method
The method presented here, termed the Wave Model Evaluation and Diagnostics System (WaveMEDS), uses wave component attributes of evolving wave spectra to quantify model skill across a variety of metrics, folds these metrics into overall measures of performance, and diagnoses model deficiencies. The specific analysis steps are outlined in Fig. 1. A detailed description of each analysis step follows.
a. Wave partitioning
The basic approach to our wave spectral partitioning method is described by Hanson (1996) and HP01. Details on significant improvements that have been made since HP01 follow. Starting with a directional wave spectrum S( f, θ), HP01 isolated spectral regions Si( f, θ) associated with individual energy peaks with a time-consuming recursive algorithm that assigned each spectrum value to a path of steepest ascent associated with a local peak. This step has been improved with efficient height (H) maxima and watershed delineation transforms originally designed to identify drainage areas from smoothed topographic imagery (Soille 1999; Vincent and Soille 1991). Basic steps to this approach are as follows (see also Tracy et al. 2006):
Invert S( f, θ) so spectral peaks become valleys.
Apply the 8-point connected H-maxima morphological transform to remove finescale noise in the spectrum. The H-maxima transform suppresses all maxima in the spectra whose height is less than a specified threshold. An H-maxima threshold of 2.8 × 10−4 was used in this study.
Round the resulting spectral values to integers between 1 and 100.
Sort integer heights in ascending order.
Starting with the lowest value, incrementally “flood” the morphologically represented spectrum, assigning partition values as flooding continues.
The end result is a label matrix 𝗧( f, θ) that assigns a partition value to each grid point in S( f, θ). This change in basic methodology reduced the record processing time by an order of magnitude, with essentially no change in output results.
The resulting partitions are sorted into wind sea or swell. To be classified as wind sea, the waves represented by a spectral peak must be forced by the component of the wind in the wave direction. A directional wave-age criterion [Eqs. (4) and (5) in HP01] is used to identify and combine the windsea partitions. All remaining peaks are labeled as swell. Adjacent swell peaks that are contiguous in frequency are combined if certain conditions are met. A swell angle test requires mean directions of adjacent peaks be separated by less than a user-supplied threshold angle. A threshold value of 30° yielded optimum results with this study and provided a comfortable margin outside the stated ±10° buoy-direction accuracy. A second test compares the polar ( f, θ) distance between peaks in relation to their spectral spread. Peaks are combined if the spread of either peak is large compared to the distance between the two peaks [Eqs. (6) through (9) in HP01]. As a final step in producing consistent results with minimal noise, any windsea or swell component that falls below a significant wave height threshold of 0.2 m is removed from analysis.
Partition labels stored in 𝗧( f, θ) are modified based on the windsea and swell identification process. Label values contained in 𝗧( f, θ) can be one of the following:
0 represents wind sea;
1 n represents swell systems;
−1 represents below height threshold.
An example of a partitioned buoy spectrum containing a windsea and two swell components appears in Fig. 2.
b. Component attributes
c. Error analysis
In this step, we quantify differences between hindcast and observation wave components. Each hourly hindcast spectrum is time paired to the corresponding buoy spectrum. Time lags of up to 10 min between hindcast and buoy times are allowed. For each wave component, the buoy partition template 𝗧( f, θ) is used to identify the corresponding spectral domain in the hindcast spectra. A matching set of hindcast attributes (Hs, Tp, and
To facilitate a diagnostic interpretation of results, the resulting data pairs are divided into three wave maturity classes: wind sea, young swell, or mature swell. The HP01 directional wave-age criterion is used to classify spectral peaks forced by the local wind as wind sea. Remaining wave components that have a peak frequency of 0.09 Hz or greater are labeled as young swell, and those with a peak frequency less than 0.09 Hz are labeled as mature swell. This frequency division was found to roughly separate regionally generated young swell with swell that has traveled significant distances in the Pacific. An alternative approach to wave field classification is addressed in the discussion.
For each monthly subset, the hindcast wave component attributes are evaluated against the observed quantities using temporal correlation (TC) analyses and quantile–quantile (QQ) distributions in 99 percentile bins. The TC analysis, performed on time-paired observation and hindcast data, provides an indication of how well the hindcast quantities match the observed quantities in absolute time. In contrast, the QQ analysis divides both the observation and hindcast datasets into quantiles and is used to indicate if the distribution of magnitudes is correct, regardless of occurrence time. The TC comparisons were performed on the height, period, and direction attributes and the QQ distributions were performed on the height and period attributes only. The error metrics used in these analyses are defined below.
This analysis results in a set of monthly error metrics (b, ERMS, SI, ba, and cor) that quantify hindcast skill in reproducing physical attributes (Hs, Tp, and
d. Performance calculation
e. Wave system analysis
3. Pacific hindcast study
As the purpose of this investigation was to identify the best-performing technology for the WIS Pacific hindcast, each modeling group was allowed to select a bathymetry grid and develop model setup parameters to optimize individual model performance in their own computing environment. However, all model runs used a common set of high-quality wind fields that spanned the entire Pacific Ocean basin for the full calendar year 2000. An overview of the participating modeling groups and resulting hindcast runs appear in Table 1. Specific details on the wind fields, model technologies, and ground truth data appear in the following sections.
a. Wind fields
High-quality, consistent, neutral stability wind fields (NRAQ+) at 3-h intervals on a 0.5° spatial hindcast grid are being developed for WIS with the goal of accurately representing the full range of meteorological events that occur in the Pacific. Wind fields are generated by the marine meteorology group at Oceanweather, Inc. (OWI) using baseline National Centers for Environmental Prediction–National Center for Atmospheric Research (NCEP–NCAR) global reanalysis (NRA) 6-hourly, 10-m surface winds on a Gaussian geographic grid (Kalnay et al. 1996). NRA fields are adjusted using Quick Scatterometer [QuikSCAT (Q/S)] winds by linear regressions through QQ plots in 45° wind direction sectors. NRA data from the full year (2000) are included in the QQ analysis, as seasonally stratified regressions are not statistically independent. Since tropical cyclone winds are poorly resolved in the NRA wind fields, OWI’s mesoscale planetary boundary layer (PBL) cyclone model used available tropical cyclone information over the full domain to recreate tropical cyclone winds for blending into the NRAQ+ winds as previously detailed in Swail et al. (2000) for Atlantic hindcast wind fields. Beyond the Q/S adjustments, no additional observations are used in the analysis of the NRA–QuikSCAT winds. Compared to the NRA winds, the NRAQ+ winds are superior in capturing synoptic and mesoscale events.
b. Wave models
Each numerical wave model (Table 1) was set up to define directional spectra in terms of 25 logarithmically spaced frequency bins from 0.03 to 0.4 Hz and 24 regularly spaced direction bins of 15° width. The two highest frequency bins were dropped from the analysis in order to match the 0.03–0.34-Hz frequency range of the ground truth buoy data. Each model was run at 0.5° spatial resolution covering 64°S–64°N latitude and 110°E–60°W longitude. The models were initiated on 1 January 2000 with the NRAQ+ winds and run for a full calendar year. The first two weeks of spinup in January were excluded from the analysis. Specific details on each model run appear below.
1) WAM model
The third-generation WAM cycle 4.5 (Komen et al. 1994; Günther 2002) was run on a Cray X1 platform with no parallelization (Table 1). Water depths were obtained from General Bathymetric Chart of the Oceans (GEBCO), a digital bathymetry database with nominal horizontal resolution of 3 min. Depth data were then subsampled to a 0.5° fixed longitude–latitude grid and hand edited to include spatially unresolved islands. Full directional wave spectra were exported hourly at the locations of the Pacific wave buoys (1D and 2D) used in this study (Table 2).
2) WAVEWATCH III
The third-generation numerical wave model WAVEWATCH III version 2.22 (Tolman 2002) was run with the standard operational default settings that include the Tolman and Chalikov (1996) source functions. The operational National Oceanic and Atmospheric Administration (NOAA) bathymetry grid was used with island obstructions based on the 2-minute gridded elevations/bathymetry for the world (ETOPO2; Tolman 2003). WAVEWATCH III was run on an Origin 3800 (O3K) platform in Message Passing Interface (MPI) parallel mode using 16 processors (Table 1). Hourly wave parameter and directional wave spectra were saved at each of the ground truth stations used by this study.
3) WAVAD
The second-generation (2G) spectral wave model WAVAD version 4c (Resio and Perrie 1989) maintains equilibrium between the input winds and the nonlinear wave energy flux with an assumed f−4 spectral shape. Wave growth is based on a combined Phillips and Miles mechanism. Weak nonlinear wave–wave interactions are represented as a momentum flux to both lower and higher frequencies away from the spectral peak. Energy transferred to higher frequencies is assumed to be lost by breaking. WAVAD hindcasts were produced on a 3.4-GHz personal computer using the identical bathymetry and obstruction grids used for WAVEWATCH III. Spectral output was archived at hourly intervals for the selected buoy locations, and wave parameter fields over the entire grid were saved at 6-h intervals.
c. Ground truth observations
The WaveMEDS approach requires observations of wind (for partitioning) and directional (2D) or nondirectional (1D) wave spectra as input. These ground truth data were obtained from the NDBC and CDIP buoy networks. Locations of the seven deep-water wave stations used in this study are shown on Fig. 4. These stations cover offshore conditions for much of the U.S. west coast and are sufficiently well dispersed to include a wide range of wave-generation and swell-propagation environments. Specific details on these stations appear in Table 2. They include a variety of measurement platforms and instrument payloads, including three stations equipped with directional wave sensors (46042, 071, and 51028). Station 071 is a commercial Datawell MK II Directional Waverider buoy. Details on NDBC payload configurations are available at the NDBC Web site.
CDIP directional wave data are computed from half-hour records over the frequency band 0.025–0.58 Hz. Reported quantities include E( f ),
Resulting spectra were linearly interpolated to a 23-frequency (0.04–0.34 Hz), 15° bin resolution used in the partitioning analysis. Nondirectional spectra were interpolated in frequency only. These interpolations were necessary to make one-to-one comparisons between buoy and hindcast spectral features.
4. Model performance
At any given instance, the Pacific Ocean contains numerous temporally and spatially evolving wave systems originating from a wide variety of distributed wind-generation events. The challenge of numerical wave modeling is to capture the essence of this dynamic wave field. A wave vector history shown in Fig. 5 from a 2-week period of November 2000 at station 51028 provides a convenient display of wave system attributes. Included are the buoy-observed wave systems (Fig. 5a) and the results from our three model hindcasts (Figs. 5b–d). The complexity of the wave field is evident with 3–5 distinct windsea and swell wave systems present at any given instant in time and each lasting for several days duration (Fig. 5a). The result is 12 or more wave systems passing through this area during the 2-week period shown. Numerical models vary in their skill of capturing the evolution of the various wave systems present (Figs. 5b–d). Although these plots provide an indication of hindcast fidelity, a more precise measure of model skill was required to select a technology for the WIS Pacific hindcast.
The WaveMEDS approach quantifies differences in the three Pacific basin wave hindcasts, with seven deep-water NDBC and CDIP buoys depicted in Fig. 4 and described in Table 2 used as ground truth stations in the analysis. As will be demonstrated, results show that WAVEWATCH III provides a superior hindcast for the input winds, boundary conditions, and model settings employed in this study. In the following sections, a top-down reporting of results compares overall model performance for the three hindcasts and explores spatial and temporal variability in prediction skill for specific wave field attributes. A wave system analysis is performed to identify hindcast deficiencies and guide future model improvements.
a. Performance summary
Annual model performance scores for significant wave height, peak wave period, and mean wave direction appear in Tables 3, 4, and 5, respectively. In each table, results of temporal correlations and quantile–quantile distributions are provided for windsea, young swell, and mature swell components. As discussed above, performance scores can range from 0.0 to 1.0, with 1.0 being a perfect match of hindcast data to observations. Combined scores (in the bottom rows) represent the weighted average (by sample size) of the three wave component classes and provide an overall measure of model skill in predicting each physical attribute (height, period, and direction).
The three hindcasts exhibit varied performance with combined wave height scores of 0.78–0.88, combined wave period scores of 0.88–0.96, and combined wave direction scores of 0.83–0.91. QQ scores are higher than the corresponding TC scores, suggesting that the hindcasts are better skilled at capturing event distributions than correctly matching event times. Furthermore, wave period and wave direction scores for each model are higher than wave height scores. Although the three models are rather close in overall performance, WAVEWATCH III consistently has the highest combined scores in each category.
Performance scores for each wave component reveal that hindcast skill varies with wave maturity. Mature swell height (Table 3) has significantly lower scores than windsea or young swell height in each model hindcast. As will be demonstrated, mature swell height is the most significant factor degrading Pacific wave model performance.
b. Error diagnostics
Using performance scores as a guide, TC and QQ errors are explored to identify patterns in hindcast deficiencies. Here we will focus exclusively on exploring wave height error trends, since this attribute produced the lowest performance scores in all three hindcasts. Furthermore, we will limit our discussion to windsea and mature swell errors, as young swell errors tend to reflect a blend of the other two.
There is an important issue to be raised regarding the comparison of error metrics (RMS error, bias, and scatter index) obtained in this study to those obtained by other case studies (i.e., Guillaume 1990; Cardone et al. 1996). These previous studies typically report combined errors for all stations and over the entire hindcast. Since we are reporting errors by station and month, the means rather than extremes from our figures need to be compared to previous studies. Furthermore, we compute errors for partitions of the spectrum, whereas most other studies compute total wave height errors. As spectral energy estimates are expected to have a larger random error than mean wave height, errors computed from partitioned parameters are expected to be larger than those computed from total wave height.
1) Windsea temporal correlations
Wave height TC errors for wind sea appear in Fig. 6. Monthly error metrics (ERMS, b, and SI) are depicted at all seven ground truth stations and for each hindcast. Error plots are organized by hindcast (columns) and error metric (rows). Windsea height error trends are similar for both of the 3G hindcasts, with WAVEWATCH exhibiting the lowest errors of all three (center column). With the exception of coastal California stations 46042 and 00071, windsea RMS errors (top row) tend to be lowest in the summer months, with WAVEWATCH ERMS < 0.5 m for all but 3 occurrences. Windsea height biases (middle row) are significantly larger in WAM and WAVAD, with WAM windsea bias generally negative, except for positive biases at the two coastal California stations during spring, summer, and fall. WAVEWATCH windsea height biases for all stations cluster around zero, except for overestimation of summer wave heights at 46042. Wind seas from WAVAD exhibit the greatest amount of variability, with biases up to 0.6 m at 46042 and 00071. On average, WAVEWATCH exhibits the lowest scatter index values.
2) Mature swell temporal correlations
The relatively poor mature swell wave height performance (Table 3) is clearly evident in the TC error trends of Fig. 7. There is a definitive seasonal trend in mature swell height RMS error at all stations and in all three hindcasts. This trend shows RMS errors increasing during the Northern Hemisphere winter months (November through March), coinciding with the period of increased North Pacific cyclogenesis. Furthermore, these errors are most significant at stations 46001, 46005, and 51001, which are directly in the path of winter swells emanating from the North Pacific. It is noteworthy that, in summer months, when southern swell dominates, lower wave height errors prevail. Mature swell height bias from WAM and WAVEWATCH III exhibits a similar seasonal trend, with a positive bias in winter months and near zero bias during the remainder of the year. This trend is most distinctive in the WAVEWATCH III hindcast. In contrast, the WAVAD mature swell height biases tend to be slightly negative (−0.5–0.0 m at most stations) with no discernable seasonal trend. At all but one station, mature swell height scatter index does not exhibit any specific trends across time or location, other than being slightly more variable in summer months when wave heights are lower. In general, WAVEWATCH III mature swell heights exhibit lower scatter index values with the least amount of variability.
3) Wave height quantile–quantile distributions
Monthly wave height QQ distributions provide additional details on systematic errors not associated with event timing offsets. Example wave height QQ distributions from all three hindcasts during February, May, August, and November 2000 at station 51028 appear in Fig. 8. These plots compare the observed and hindcast windsea, young swell, and mature swell wave height distributions computed in 99 percentile bins. The solid black line represents a perfect agreement between observation and hindcast height distributions. The two 3G models have the best overall agreements in spring and summer. WAM wind seas show the best agreement at elevated wave heights, with lower heights typically exhibiting a negative bias. This is supported by the Hanson and Jensen (2004) finding that WAM wind seas exhibit a slow response to changing wind conditions and that elevated or consistently steady winds are required to match observed spectral levels. This issue has been also addressed by Bidlot et al. (2007). Mature swell from the 3G models exhibits a nearly constant positive bias at observed wave heights >1 m. In contrast, WAVAD mature swells have a constant negative bias in May, an increasing positive bias in November, and mixed results the remaining two months.
Results of the various hindcast performance analyses facilitated the selection of WAVEWATCH III for the ongoing WIS multidecade hindcast study. Additional results and discussion will focus on this modeling technology.
c. Swell bias
The WaveMEDS hindcast analysis led us to the conclusion that winter mature swell wave heights are the most significant attribute degrading Pacific hindcast accuracy in all three modeling technologies. A wave system approach is now taken to further diagnose the source of these errors in the WAVEWATCH III hindcast. As a representative example, we focus on the mature swell hindcast errors during November 2000 at the Christmas Island NDBC station 51028. To identify events most likely to contribute to swell height bias, integrated wave power [Eq. (25)] was used to identify the five most energetic wave systems during this period. The resulting wave systems (A through E) are depicted in the wave vector displays of Fig. 9. Events A and B are the result of high-latitude storms deep in the Southern Ocean, producing mature swells that can travel 5000–6000 km to reach station 51028. Events C and D are a result of the low pressure winter cyclones that regularly pass across the North Pacific during winter months and contribute to a dynamic multicomponent wave field in the Gulf of Alaska (HP01). Mature swells reaching station 51028 from this generation region have traveled up to 6500 km. Event E is a young swell resulting from the northeast trade wind belt centered at about 15°N latitude.
Examination of wave system events from several stations indicates that mature swell height bias has a strong geographic dependence on swell origin. Comparisons of hindcast and measured wave system Hs, Tp, and
5. Discussion
The purpose of this investigation was to develop an approach for the validation and diagnostic evaluation of numerical ocean-surface wave model performance and use this technique to select a Pacific hindcast modeling technology. In this discussion we will review limitations and alternative methods to this approach and address issues that may be impacting our Pacific hindcast performance.
a. Model performance
Although overall performance results from the three Pacific hindcasts are reasonably similar, we will focus our analysis on the WAVEWATCH III hindcast. Within this study, mature swell height bias appears to be the most significant hindcast limitation. A separate analysis was performed to compare our WAVEWATCH III hindcast total significant wave height fields to Ocean Topography Experiment (TOPEX)/Poseidon altimeter data (Scott 2005). Mean wave height bias was computed for year 2000 and appears in Fig. 12. Note that the maximum average height biases (>0.5 m) cover a broad area of the central and eastern North Pacific Ocean and include the areas of intensive wave generation resulting from low pressure cyclogenesis during these months.
When investigating wave height bias one must start with an examination of the driving winds. Certainly both the WAVEWATCH III and WAM results, along with the buoy and altimeter comparisons, suggest that wind speed bias may be a significant issue contributing to mature swell height bias. It is interesting to note, however, that the WAVAD mature swell height bias is generally negative at most stations with no discernable seasonal trend (Fig. 7) and hence does not readily support the hypothesis that the North Pacific wintertime winds are elevated above realistic values. To evaluate the impact of a more carefully constructed wind field, the marine meteorology experts at Oceanweather, Inc. conducted a full kinematic analysis (Cox et al. 1996; Cox and Cardone 2000) on the nine most intense Northern Hemisphere storms occurring in March 2000. The kinematic analysis was supported by QuikSCAT winds and included the assimilation of available buoy data. Results were blended into the March 2000 baseline NRAQ+ wind fields, resulting in a new NRAQ+K wind field for this month only. NRAQ+K winds were used to generate a new WAVEWATCH III hindcast. Although the NRAQ+K hindcast results exhibit lower wave heights during peak events, improvement is only a small percentage of the total bias. Hence, it appears that input wind magnitudes are certainly part of the swell bias problem but not the only contributing factor. Furthermore, it is also noteworthy that a positive height bias in the northern Pacific Ocean during winter has been a persistent feature of the operational WAVEWATCH III model at NCEP, based on comparisons with Jason-1, Geosat Follow-On (GFO), and Envisat altimeter data (H. L. Tolman, 2009 unpublished manuscript) as well as with buoy data (Bidlot et al. 2007). Considering the different sources and resolutions of the wind fields involved, it is not likely that this wave model bias is solely a consequence of the winds.
Another potential source of error is the parameterization of atmospheric drag in the wind input source term. The standard WAVEWATCH III formulation that was available at the time of this study extrapolated the drag coefficient (CD) to continually increase as a function of wind speed. However, recent observations suggest that CD caps in the neighborhood of approximately 2.5 × 10−3 as whitecapping fully develops at wind speeds above approximately 30 m s−1 (Powell et al. 2003). In extreme winds, the extrapolated CD in WAVEWATCH III could potentially lead to increased wave development and an associated wave height bias. To test if elevated drag coefficients are contributing to the WAVEWATCH III swell height bias, an additional year 2000 hindcast was made with a wind input source term modified to provide a CD cap of 2.5 × 10−3. A map of the resulting annual mean wave height bias based on TOPEX/Poseidon altimeter data (Scott 2005) appears in Fig. 13. This map can be directly compared with the annual mean bias map of Fig. 12, which depicts WAVEWATCH III height bias with no CD cap imposed on the wind input source term. Indeed, implementation of the CD cap results in a 30%–40% reduction in wave height bias in the central North Pacific generation region and a 20%–30% reduction in total wave height bias along the U.S. coast. In an update to his earlier work, Powell (2007) notes that CD has a very complicated behavior with both wind speed and storm orientation. Additional work in this area is clearly warranted.
The remaining source terms (wave–wave interaction and dissipation) are also likely candidates for contributing to the observed swell height bias in WAM and WAVEWATCH III. The wave–wave interaction term uses the discrete interaction approximation (DIA) of Hasselmann and Hasselmann (1985). Although this computationally efficient algorithm is at the core of most 3G wave modeling, evidence suggests that away from the spectral peak and in multimodal wave fields the DIA greatly undersamples the complex set of nonlinear interactions taking place, leading to erroneous estimates of the spectral source function, which will potentially result in spectral shapes deviating from observations (Resio and Perrie 2006). Furthermore, the parameterized dissipation term has largely been used as a device to tune model performance rather than correctly capturing the physical mechanisms of wave decay. Fortunately, there are efforts aimed at adding a more physical representation of dissipation (Ardhuin et al. 2008). It is likely that significant gains in hindcast skill will not be made until significant improvements are made in these source term formulations.
b. Technique limitations and alternatives
The primary limitation to the wave system validation and diagnostics approach is the scarcity of available ground truth data. The number of operational directional wave stations is too small to allow a fully comprehensive verification of model output. Furthermore, floating buoys do not resolve the full directional distribution, which may have an impact on the results. The primary benefit of these tools in their present form is to provide an augmentation of traditional bulk validation techniques.
Additional sources of information can be incorporated to make this approach more operationally viable. For example, extension of the partitioning approach to the 1D spectral domain has added several stations of available ground truth information. An additional source of ground truth information can be obtained from remote sensing of the sea surface. In particular, satellite synthetic aperture radar (SAR) measurements (Beal 1989) provide estimates of ocean surface directional wave spectra along the SAR track. These could be used to validate the spatial representation of evolving hindcast wave systems. Furthermore, SAR data only capture a limited range of the spectral space. Therefore, it is essential in validation and calibration of SAR data that only resolved wave components are compared with in situ data. This can be done most efficiently with partitioning of the SAR and in situ data as in the present study. Finally, significant wave height estimates from satellite altimeters can be used to verify the spatial distribution of bulk hindcast wave heights. As shown here, the use of altimetry data in the Pacific hindcast study has provided valuable insight into the spatial distribution of hindcast height bias.
As a result of this study, a few recommendations for improving the WaveMEDS technique can be made. To identify matching domains in buoy and hindcast spectra in this study we employed a partition template overlay method. A strength of this approach is that the resulting error statistics account for all of the energy present (above the low-energy threshold) in the wave field. However portions of the energy from more than one hindcast peak may fall into any given spectral domain computed from the buoy data. An alternative approach would be to compute a separate partition template for each hindcast spectrum and use a clustering approach to match components. This could provide added benefits, such as a separate accounting of extraneous energy peaks present in either the observations or the hindcast.
The approach to classifying wave components can also be improved. At present wind seas are identified using a directional wave-age criterion, and young and mature swell groups are formed based on the peak frequency of each swell partition. Although this rudimentary separation of swell types works fairly well in the Pacific basin, it would not necessarily be appropriate in other oceanographic regions. A more universal approach would be to separate wave components based on the amount of wind forcing they are subject to. This would allow the separation of wave components into pure windsea, mixed sea, and pure swell categories. This is currently a topic of active testing by two of the authors (Hanson and Tolman).
6. Conclusions
Three numerical spectral wave models were tested to identify the best technology for conducting a multidecade WIS Pacific hindcast. Each technology was evaluated with identical forcing over the year 2000 with seven deep-water NDBC and CDIP buoys employed as ground truth. The Wave Model Evaluation and Diagnostics System (WaveMEDS) provided an efficient mechanism for reducing millions of spectral values from the three hindcasts into a convenient database of monthly hindcast errors organized as a function of physical attribute (height, period, and direction), wave maturity (wind sea, young swell, and mature swell), and station location. Application of a unique set of performance calculations further reduced this information into a concise set of nine overall performance scores providing a robust assessment of model prediction skill and guiding additional diagnostic evaluations.
The three models exhibited varied performance in the depiction of wind sea and young swell physical attributes. A noteworthy problem area is in the prediction of mature swell in winter months, with elevated height errors in all three models. Overall the WAVEWATCH III hindcast exhibited consistently higher performance scores than those from WAM and WAVAD.
Diagnostic evaluation of the 3G hindcast mature swell height errors suggests that this problem emanates from winter swell produced in the North Pacific. This finding is also confirmed by inspection of WAVEWATCH III North Pacific hindcast bias derived from satellite altimetry. Further analysis of the data suggests that underresolved storms in the wind fields contribute only a small percentage to the total error. An examination of source term behavior shows that a cap on the atmospheric drag coefficient has a fairly positive impact on reducing wave height bias in the primary wave-generation areas; however, recent work indicates that a constant cap is not the full story. It is further suspected that the wave–wave interaction and dissipation source terms are likely contributors to swell height error and that significant model advancements are not likely until these source terms are improved.
As a result of this analysis, WAVEWATCH III was selected for use in a new 1981–2004 Pacific basin hindcast that is now available on the USACE WIS Web page (http://www.frf.usace.army.mil/cgi-bin/wis/pac/pac_main.html). Furthermore, it should be noted that improved versions are now available for both WAM and WAVEWATCH III.
Acknowledgments
The authors are grateful to Elizabeth Orelup, Erin Harris, Andrew Cox, and Dr. Vince Cardone of Oceanweather, Inc. for producing the outstanding WIS wind fields; to Dr. Bob Jensen for his skillful preparation of the WAM hindcasts; to Dr. Donald Resio for several insightful suggestions to this investigation; to Dr. Charles Long for patiently working his editorial magic on an early version of the manuscript; to Dr. Heidi Wadman for her skillful help in preparing the figures; and to the anonymous reviewers for their insightful suggestions.
REFERENCES
Ardhuin, F., Hamon M. , Collard F. , Chaperon B. , and Queffeulou P. , 2008: Spectral wave evolution and spectral dissipation based on observations: A global validation of new source functions. Proc. Fourth Chinese–German Joint Symposium on Coastal and Ocean Engineering, Darmstadt, Germany, TU Darmstadt.
Beal, R. C., 1989: Directional Ocean Wave Spectra. The Johns Hopkins University Press, 218 pp.
Bidlot, J-R., Janssen P. A. E. M. , and Abdalla S. , 2005: On the importance of spectral wave observations in the continued development of global wave models. Proc. Fifth Int. Symp. on Ocean Wave Measurement and Analysis: WAVES2005, Madrid, Spain, CEDEX, 16 pp.
Bidlot, J-R., and Coauthors, 2007: Inter-comparison of operational wave forecasting systems. Proc. 10th Int. Workshop on Wave Hindcasting and Forecasting and Coastal Hazard Symp., North Shore, Oahu, HI, European Centre for Medium-Range Weather Forecasts, 22 pp.
Bowers, J. A., Morton I. D. , and Mould G. I. , 2000: Directional statistics of the wind and waves. Appl. Ocean Res., 22 , 13–30.
Cardone, V. J., Jensen R. E. , Resio D. T. , Swail V. R. , and Cox A. T. , 1996: Evaluation of contemporary ocean wave models in rare extreme events: The “Halloween Storm” of October 1991 and the “Storm of the Century” of March 1993. J. Atmos. Oceanic Technol., 13 , 198–230.
Cox, A. T., and Cardone V. J. , 2000: Operational system for the prediction of tropical cyclone generated winds and waves. Proc. Sixth Int. Workshop on Wave Hindcasting and Forecasting, Monterey, CA, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, Fleet Numerical Meteorological Center, and Meteorological Service of Canada, 158–166.
Cox, A. T., Greenwood A. R. , Cardone V. J. , and Swail V. R. , 1996: An interactive objective kinematic analysis system. Proc. Fourth Workshop on Wave Hindcasting and Forecasting, Banff, AB, Federal Panel on Energy Research and Development and Atmospheric Environment Service of Environment Canada, 171–183.
Gerling, T. W., 1992: Partitioning sequences and arrays of directional ocean wave spectra into component wave systems. J. Atmos. Oceanic Technol., 9 , 444–458.
Guillaume, A., 1990: Statistical tests for the comparison of surface gravity wave spectra with application to model validation. J. Atmos. Oceanic Technol., 7 , 551–567.
Günther, H., 2002: WAM cycle 4.5. Institute for Coastal Research, GKSS Research Centre, Geesthacht, Germany, 43 pp.
Hanson, J. L., 1996: Wind sea growth and swell evolution in the Gulf of Alaska. Ph.D. dissertation, The Johns Hopkins University, 151 pp.
Hanson, J. L., and Phillips O. M. , 2001: Automated analysis of ocean surface directional wave spectra. J. Atmos. Oceanic Technol., 18 , 277–293.
Hanson, J. L., and Jensen R. E. , 2004: Wave system diagnostics for numerical wave models. Proc. Eighth Int. Workshop on Wave Hindcasting and Forecasting, Oahu, HI, Joint WMO/IOC Technical Commission for Oceanography and Marine Meteorology Tech. Rep. 29, WMO/TD-1319, 19 pp. [Available online at http://www.waveworkshop.org/8thWaves/Papers/E3.pdf].
Hasselmann, S., and Hasselmann K. , 1985: Computations and parameterizations of the nonlinear energy transfer in a gravity wave spectrum. Part I: A new method for efficient computations of the exact nonlinear transfer integral. J. Phys. Oceanogr., 15 , 1369–1377.
Hasselmann, S., and Bruning C. , 1994: Extraction of wave spectra from SAR image spectra. Dynamics and Modelling of Ocean Waves, G. J. Komen et al., Eds., Cambridge University Press, 391–401.
Hsu, L. Y., Rogers W. E. , and Dykes J. D. , 2002: WAM performance in the Gulf of Mexico with COAMPS wind. Proc. Seventh Int. Workshop on Wave Hindcasting and Forecasting, Banff, AB, Meteorological Service of Canada, 151–159.
Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc., 77 , 437–471.
Komen, G. J., Cavaleri L. , Donelan M. , Hasselmann K. , Hasselmann S. , and Janssen P. A. E. M. , 1994: Dynamics and Modelling of Ocean Waves. Cambridge University Press, 560 pp.
Oltman-Shay, J. M., and Guza R. T. , 1984: A data adaptive ocean wave directional-spectrum estimator for pitch and roll type measurements. J. Phys. Oceanogr., 14 , 1800–1810.
O’Reilly, W. C., Herbers T. H. C. , Seymour R. J. , and Guza R. T. , 1996: A comparison of directional buoy and fixed platform measurements of Pacific swell. J. Atmos. Oceanic Technol., 13 , 231–238.
Powell, M. D., 2007: New findings on hurricane intensity, wind field extent, and surface drag coefficient behavior. Proc. 10th Int. Workshop on Wave Hindcasting and Forecasting and Coastal Hazard Symp., North Shore, Oahu, HI, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, Environment Canada, and WMO/IOC Joint Technical Commission for Oceanography and Marine Meteorology (JCOMM), M5. [Available online at http://www.waveworkshop.org/10thWaves/ProgramFrameset.htm].
Powell, M. D., Vickery P. J. , and Reinhold T. A. , 2003: Reduced drag-coefficient for high wind speeds in tropical cyclones. Nature, 422 , 279–283.
Resio, D., and Perrie W. , 1989: Implications of an f −4 equilibrium range for wind-generated waves. J. Phys. Oceanogr., 19 , 193.
Resio, D., and Perrie W. , 2006: A two-scale approximation for efficient representation of nonlinear energy transfers in a wind wave spectrum. Proc. 9th Int. Workshop on Wave Hindcasting and Forecasting, Victoria, BC, Environment Canada, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, and the WMO/IOC Joint Technical Commission for Oceanography and Marine Meteorology (JCOMM), 37 pp. [Available online at http://www.waveworkshop.org/9thWaves/Papers/Resio.pdf].
Scott, D., 2005: Pacific Ocean wave information study validation of wave model results against satellite altimeter data. Draft Rep., W.F. Baird and Associates, 187 pp.
Soille, P., 1999: Geodesic transformations. Morphological Image Analysis: Principles and Applications,. Springer-Verlag, 155–182.
Steele, K. E., Teng C. C. , and Wang D. W. C. , 1992: Wave direction measurements using pitch-roll buoys. Ocean Eng., 19 , 349–375.
Swail, V. R., Ceccacci E. A. , and Cox A. T. , 2000: The AES40 North Atlantic wave reanalysis: Validation and climate assessment. Proc. Sixth Int. Workshop on Wave Hindcasting and Forecasting, Monterey, CA, U.S. Army Research and Development Center’s Coastal and Hydraulics Laboratory, Fleet Numerical Meteorological Center, and Meteorological Service of Canada, 15 pp. [Available online at http://www.waveworkshop.org/6thWaves/A1.pdf].
Tolman, H. L., 1997: A new global wave forecast system at NCEP. Ocean Wave Measurements and Analysis, Vol. 2, B. L. Edge and J. M. Helmsley, Eds., ASCE, 777–786.
Tolman, H. L., 1999: User manual and system documentation of WAVEWATCH-III version 1.18. Tech. Note, 110 pp.
Tolman, H. L., 2002: User manual and system documentation of WAVEWATCH-III version 2.22. Tech. Note, 139 pp.
Tolman, H. L., 2003: Treatment of unresolved islands and sea ice in wind wave models. Ocean Modell., 5 , 219–231.
Tolman, H. L., and Chalikov D. V. , 1996: Source terms in a third-generation wind wave model. J. Phys. Oceanogr., 26 , 2497–2518.
Tolman, H. L., Balasubramaniyan B. , Burroughs L. D. , Chalikov D. V. , Chao Y. Y. , Chen H. S. , and Gerald V. M. , 2002: Development and implementation of wind-generated ocean surface wave models at NCEP. Wea. Forecasting, 17 , 311–333.
Tracy, B. A., 2002: Directional characteristics of the 1990–1999 Wave Information Studies Gulf of Mexico hindcast. Proc. Seventh Int. Workshop on Wave Hindcasting and Forecasting, Meteorological Service of Canada, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, and Fleet Numerical Meteorology and Oceanography Center, 10 pp. [Available online at http://www.waveworkshop.org/7thWaves/Papers/Tracy.pdf].
Tracy, B. A., and Cialone A. , 2004: Comparison of Gulf of Mexico Wave Information Studies (WIS) 2-G hindcast with 3-G hindcasting. Proc. Eighth Int. Workshop on Wave Hindcasting and Forecasting, Oahu, HI, Meteorological Service of Canada and U.S. Army Engineer Research and Development Center, 15 pp. [Available online at http://www.waveworkshop.org/8thWaves/Papers/C2.pdf].
Tracy, F. T., Tracy B. A. , and Hanson J. L. , 2006: Sorting out waves with a fast sort algorithm. ERDC MSRC Resource (Fall 2006), U.S. Army Engineer Research and Development Center, Vicksburg, MS, 6 pp.
U.S. Army Corps of Engineers, 2003: Coastal engineering manual. Publication Number EM 1110-2-1100, 215 pp.
Vincent, L., and Soille P. , 1991: Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell., 13 , 583–598.
Voorrips, A. C., Makin V. K. , and Hasselmann S. , 1997: Assimilation of wave spectra from pitch-and-roll buoys in a North Sea wave model. J. Geophys. Res., 102 , 5829–5849.
WaveMEDS.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Example partition results for NDBC buoy station 46042 directional wave spectrum on 10 Feb 2000 (2050 UTC). White lines denote the boundaries of each wave component as represented in the partition template 𝗧( f, θ). Note that this record contains windsea (0), primary swell (1), and secondary swell (2) wave components.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Example nondirectional partition results for NDBC buoy station 46042 wave spectrum on 10 Feb 2000 (2050 UTC). The windsea, primary swell, and secondary swell wave component domains are indicated.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Pacific hindcast ground truth stations.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Station 51028 wave systems for 1–15 Nov 2000. Wave component height, period, and direction are represented by vector length, origin, and azimuth, respectively. (a) NDBC buoy measurements. (b) WAVEWATCH III hindcast. (c) WAM hindcast. (d) WAVAD hindcast.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Monthly windsea height errors.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Monthly mature swell height errors.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Wave height QQ results from station 51028.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Vector history of most energetic wave system events during November 2000 at station 51028: (a) WAVEWATCH III hindcast and (b) NDBC station 51028 observations.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Station 51028 event A wave system analysis results for WAVEWATCH III; (a) wave height comparison, (b) wave period comparison, and (c) wave direction comparison.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Station 51028 event D wave system analysis results for WAVEWATCH III; (a) wave height comparison, (b) wave period comparison, (c) wave direction comparison.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
WAVEWATCH III mean significant wave height bias for year 2000 as determined from TOPEX/Poseidon satellite altimetry.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
WAVEWATCH III year 2000 mean significant wave height bias from TOPEX/Poseidon satellite altimetry with a capped drag coefficient (CD ≤ 2.5 × 10−3) in the WAVEWATCH wind input source term.
Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1
Wave model hindcast runs.
Observation stations. [Data Acquisition and Control Telemetry (DACT); General Service Buoy Payload (GSBP)]
Significant wave height performance summary (WAVEWATCH III: WW III).
Peak wave period performance summary.
Mean wave direction performance summary.