• Ardhuin, F., , Hamon M. , , Collard F. , , Chaperon B. , , and Queffeulou P. , 2008: Spectral wave evolution and spectral dissipation based on observations: A global validation of new source functions. Proc. Fourth Chinese–German Joint Symposium on Coastal and Ocean Engineering, Darmstadt, Germany, TU Darmstadt.

    • Search Google Scholar
    • Export Citation
  • Beal, R. C., 1989: Directional Ocean Wave Spectra. The Johns Hopkins University Press, 218 pp.

  • Bidlot, J-R., , Janssen P. A. E. M. , , and Abdalla S. , 2005: On the importance of spectral wave observations in the continued development of global wave models. Proc. Fifth Int. Symp. on Ocean Wave Measurement and Analysis: WAVES2005, Madrid, Spain, CEDEX, 16 pp.

    • Search Google Scholar
    • Export Citation
  • Bidlot, J-R., and Coauthors, 2007: Inter-comparison of operational wave forecasting systems. Proc. 10th Int. Workshop on Wave Hindcasting and Forecasting and Coastal Hazard Symp., North Shore, Oahu, HI, European Centre for Medium-Range Weather Forecasts, 22 pp.

    • Search Google Scholar
    • Export Citation
  • Bowers, J. A., , Morton I. D. , , and Mould G. I. , 2000: Directional statistics of the wind and waves. Appl. Ocean Res., 22 , 1330.

  • Cardone, V. J., , Jensen R. E. , , Resio D. T. , , Swail V. R. , , and Cox A. T. , 1996: Evaluation of contemporary ocean wave models in rare extreme events: The “Halloween Storm” of October 1991 and the “Storm of the Century” of March 1993. J. Atmos. Oceanic Technol., 13 , 198230.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cox, A. T., , and Cardone V. J. , 2000: Operational system for the prediction of tropical cyclone generated winds and waves. Proc. Sixth Int. Workshop on Wave Hindcasting and Forecasting, Monterey, CA, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, Fleet Numerical Meteorological Center, and Meteorological Service of Canada, 158–166.

    • Search Google Scholar
    • Export Citation
  • Cox, A. T., , Greenwood A. R. , , Cardone V. J. , , and Swail V. R. , 1996: An interactive objective kinematic analysis system. Proc. Fourth Workshop on Wave Hindcasting and Forecasting, Banff, AB, Federal Panel on Energy Research and Development and Atmospheric Environment Service of Environment Canada, 171–183.

    • Search Google Scholar
    • Export Citation
  • Gerling, T. W., 1992: Partitioning sequences and arrays of directional ocean wave spectra into component wave systems. J. Atmos. Oceanic Technol., 9 , 444458.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guillaume, A., 1990: Statistical tests for the comparison of surface gravity wave spectra with application to model validation. J. Atmos. Oceanic Technol., 7 , 551567.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Günther, H., 2002: WAM cycle 4.5. Institute for Coastal Research, GKSS Research Centre, Geesthacht, Germany, 43 pp.

  • Hanson, J. L., 1996: Wind sea growth and swell evolution in the Gulf of Alaska. Ph.D. dissertation, The Johns Hopkins University, 151 pp.

    • Search Google Scholar
    • Export Citation
  • Hanson, J. L., , and Phillips O. M. , 2001: Automated analysis of ocean surface directional wave spectra. J. Atmos. Oceanic Technol., 18 , 277293.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hanson, J. L., , and Jensen R. E. , 2004: Wave system diagnostics for numerical wave models. Proc. Eighth Int. Workshop on Wave Hindcasting and Forecasting, Oahu, HI, Joint WMO/IOC Technical Commission for Oceanography and Marine Meteorology Tech. Rep. 29, WMO/TD-1319, 19 pp. [Available online at http://www.waveworkshop.org/8thWaves/Papers/E3.pdf].

    • Search Google Scholar
    • Export Citation
  • Hasselmann, S., , and Hasselmann K. , 1985: Computations and parameterizations of the nonlinear energy transfer in a gravity wave spectrum. Part I: A new method for efficient computations of the exact nonlinear transfer integral. J. Phys. Oceanogr., 15 , 13691377.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hasselmann, S., , and Bruning C. , 1994: Extraction of wave spectra from SAR image spectra. Dynamics and Modelling of Ocean Waves, G. J. Komen et al., Eds., Cambridge University Press, 391–401.

    • Search Google Scholar
    • Export Citation
  • Hsu, L. Y., , Rogers W. E. , , and Dykes J. D. , 2002: WAM performance in the Gulf of Mexico with COAMPS wind. Proc. Seventh Int. Workshop on Wave Hindcasting and Forecasting, Banff, AB, Meteorological Service of Canada, 151–159.

    • Search Google Scholar
    • Export Citation
  • Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc., 77 , 437471.

  • Komen, G. J., , Cavaleri L. , , Donelan M. , , Hasselmann K. , , Hasselmann S. , , and Janssen P. A. E. M. , 1994: Dynamics and Modelling of Ocean Waves. Cambridge University Press, 560 pp.

    • Search Google Scholar
    • Export Citation
  • Oltman-Shay, J. M., , and Guza R. T. , 1984: A data adaptive ocean wave directional-spectrum estimator for pitch and roll type measurements. J. Phys. Oceanogr., 14 , 18001810.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • O’Reilly, W. C., , Herbers T. H. C. , , Seymour R. J. , , and Guza R. T. , 1996: A comparison of directional buoy and fixed platform measurements of Pacific swell. J. Atmos. Oceanic Technol., 13 , 231238.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Powell, M. D., 2007: New findings on hurricane intensity, wind field extent, and surface drag coefficient behavior. Proc. 10th Int. Workshop on Wave Hindcasting and Forecasting and Coastal Hazard Symp., North Shore, Oahu, HI, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, Environment Canada, and WMO/IOC Joint Technical Commission for Oceanography and Marine Meteorology (JCOMM), M5. [Available online at http://www.waveworkshop.org/10thWaves/ProgramFrameset.htm].

    • Search Google Scholar
    • Export Citation
  • Powell, M. D., , Vickery P. J. , , and Reinhold T. A. , 2003: Reduced drag-coefficient for high wind speeds in tropical cyclones. Nature, 422 , 279283.

  • Resio, D., , and Perrie W. , 1989: Implications of an f−4 equilibrium range for wind-generated waves. J. Phys. Oceanogr., 19 , 193.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Resio, D., , and Perrie W. , 2006: A two-scale approximation for efficient representation of nonlinear energy transfers in a wind wave spectrum. Proc. 9th Int. Workshop on Wave Hindcasting and Forecasting, Victoria, BC, Environment Canada, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, and the WMO/IOC Joint Technical Commission for Oceanography and Marine Meteorology (JCOMM), 37 pp. [Available online at http://www.waveworkshop.org/9thWaves/Papers/Resio.pdf].

    • Search Google Scholar
    • Export Citation
  • Scott, D., 2005: Pacific Ocean wave information study validation of wave model results against satellite altimeter data. Draft Rep., W.F. Baird and Associates, 187 pp.

    • Search Google Scholar
    • Export Citation
  • Soille, P., 1999: Geodesic transformations. Morphological Image Analysis: Principles and Applications,. Springer-Verlag, 155–182.

  • Steele, K. E., , Teng C. C. , , and Wang D. W. C. , 1992: Wave direction measurements using pitch-roll buoys. Ocean Eng., 19 , 349375.

  • Swail, V. R., , Ceccacci E. A. , , and Cox A. T. , 2000: The AES40 North Atlantic wave reanalysis: Validation and climate assessment. Proc. Sixth Int. Workshop on Wave Hindcasting and Forecasting, Monterey, CA, U.S. Army Research and Development Center’s Coastal and Hydraulics Laboratory, Fleet Numerical Meteorological Center, and Meteorological Service of Canada, 15 pp. [Available online at http://www.waveworkshop.org/6thWaves/A1.pdf].

    • Search Google Scholar
    • Export Citation
  • Tolman, H. L., 1997: A new global wave forecast system at NCEP. Ocean Wave Measurements and Analysis, Vol. 2, B. L. Edge and J. M. Helmsley, Eds., ASCE, 777–786.

    • Search Google Scholar
    • Export Citation
  • Tolman, H. L., 1999: User manual and system documentation of WAVEWATCH-III version 1.18. Tech. Note, 110 pp.

  • Tolman, H. L., 2002: User manual and system documentation of WAVEWATCH-III version 2.22. Tech. Note, 139 pp.

  • Tolman, H. L., 2003: Treatment of unresolved islands and sea ice in wind wave models. Ocean Modell., 5 , 219231.

  • Tolman, H. L., , and Chalikov D. V. , 1996: Source terms in a third-generation wind wave model. J. Phys. Oceanogr., 26 , 24972518.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tolman, H. L., , Balasubramaniyan B. , , Burroughs L. D. , , Chalikov D. V. , , Chao Y. Y. , , Chen H. S. , , and Gerald V. M. , 2002: Development and implementation of wind-generated ocean surface wave models at NCEP. Wea. Forecasting, 17 , 311333.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tracy, B. A., 2002: Directional characteristics of the 1990–1999 Wave Information Studies Gulf of Mexico hindcast. Proc. Seventh Int. Workshop on Wave Hindcasting and Forecasting, Meteorological Service of Canada, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, and Fleet Numerical Meteorology and Oceanography Center, 10 pp. [Available online at http://www.waveworkshop.org/7thWaves/Papers/Tracy.pdf].

    • Search Google Scholar
    • Export Citation
  • Tracy, B. A., , and Cialone A. , 2004: Comparison of Gulf of Mexico Wave Information Studies (WIS) 2-G hindcast with 3-G hindcasting. Proc. Eighth Int. Workshop on Wave Hindcasting and Forecasting, Oahu, HI, Meteorological Service of Canada and U.S. Army Engineer Research and Development Center, 15 pp. [Available online at http://www.waveworkshop.org/8thWaves/Papers/C2.pdf].

    • Search Google Scholar
    • Export Citation
  • Tracy, F. T., , Tracy B. A. , , and Hanson J. L. , 2006: Sorting out waves with a fast sort algorithm. ERDC MSRC Resource (Fall 2006), U.S. Army Engineer Research and Development Center, Vicksburg, MS, 6 pp.

    • Search Google Scholar
    • Export Citation
  • U.S. Army Corps of Engineers, 2003: Coastal engineering manual. Publication Number EM 1110-2-1100, 215 pp.

  • Vincent, L., , and Soille P. , 1991: Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell., 13 , 583598.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Voorrips, A. C., , Makin V. K. , , and Hasselmann S. , 1997: Assimilation of wave spectra from pitch-and-roll buoys in a North Sea wave model. J. Geophys. Res., 102 , 58295849.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    WaveMEDS.

  • View in gallery

    Example partition results for NDBC buoy station 46042 directional wave spectrum on 10 Feb 2000 (2050 UTC). White lines denote the boundaries of each wave component as represented in the partition template 𝗧( f, θ). Note that this record contains windsea (0), primary swell (1), and secondary swell (2) wave components.

  • View in gallery

    Example nondirectional partition results for NDBC buoy station 46042 wave spectrum on 10 Feb 2000 (2050 UTC). The windsea, primary swell, and secondary swell wave component domains are indicated.

  • View in gallery

    Pacific hindcast ground truth stations.

  • View in gallery

    Station 51028 wave systems for 1–15 Nov 2000. Wave component height, period, and direction are represented by vector length, origin, and azimuth, respectively. (a) NDBC buoy measurements. (b) WAVEWATCH III hindcast. (c) WAM hindcast. (d) WAVAD hindcast.

  • View in gallery

    Monthly windsea height errors.

  • View in gallery

    Monthly mature swell height errors.

  • View in gallery

    Wave height QQ results from station 51028.

  • View in gallery

    Vector history of most energetic wave system events during November 2000 at station 51028: (a) WAVEWATCH III hindcast and (b) NDBC station 51028 observations.

  • View in gallery

    Station 51028 event A wave system analysis results for WAVEWATCH III; (a) wave height comparison, (b) wave period comparison, and (c) wave direction comparison.

  • View in gallery

    Station 51028 event D wave system analysis results for WAVEWATCH III; (a) wave height comparison, (b) wave period comparison, (c) wave direction comparison.

  • View in gallery

    WAVEWATCH III mean significant wave height bias for year 2000 as determined from TOPEX/Poseidon satellite altimetry.

  • View in gallery

    WAVEWATCH III year 2000 mean significant wave height bias from TOPEX/Poseidon satellite altimetry with a capped drag coefficient (CD ≤ 2.5 × 10−3) in the WAVEWATCH wind input source term.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 182 182 16
PDF Downloads 189 189 13

Pacific Hindcast Performance of Three Numerical Wave Models

View More View Less
  • 1 U.S. Army Corps of Engineers, Field Research Facility, Duck, North Carolina
  • | 2 U.S. Army Corps of Engineers, Waterways Experiment Station, Vicksburg, Mississippi
  • | 3 NOAA/NCEP Environmental Modeling Center, Camp Springs, Maryland
  • | 4 W.F. Baird & Associates Coastal Engineers, Ltd., Ottawa, Ontario, Canada
© Get Permissions
Full access

Abstract

Although mean or integral properties of wave spectra are typically used to evaluate numerical wave model performance, one must look into the spectral details to identify sources of model deficiencies. This creates a significant problem, as basin-scale wave models can generate millions of independent spectral values. To facilitate selection of a wave modeling technology for producing a multidecade Pacific hindcast, a new approach was developed to reduce the spectral content contained in detailed wave hindcasts to a convenient set of performance indicators. The method employs efficient image processing tools to extract windsea and swell wave components from monthly series of nondirectional and directional wave spectra. Using buoy observations as ground truth, both temporal correlation (TC) and quantile–quantile (QQ) statistical analyses are used to quantify hindcast skill in reproducing measured wave component height, period, and direction attributes. An integrated performance analysis synthesizes the TC and QQ results into a robust assessment of prediction skill and yields distinctive diagnostics on model inputs and source term behavior. The method is applied to a set of Pacific basin hindcasts computed using the WAM, WAVEWATCH III, and WAVAD numerical wave models. The results provide a unique assessment of model performance and have guided the selection of WAVEWATCH III for use in Pacific hindcast production runs for the U.S. Army Corps of Engineers Wave Information Studies Program.

Corresponding author address: Jeffrey L. Hanson, USACE Field Research Facility, 1261 Duck Rd., Kitty Hawk, NC 27949. Email: jeffrey.l.hanson@usace.army.mil

Abstract

Although mean or integral properties of wave spectra are typically used to evaluate numerical wave model performance, one must look into the spectral details to identify sources of model deficiencies. This creates a significant problem, as basin-scale wave models can generate millions of independent spectral values. To facilitate selection of a wave modeling technology for producing a multidecade Pacific hindcast, a new approach was developed to reduce the spectral content contained in detailed wave hindcasts to a convenient set of performance indicators. The method employs efficient image processing tools to extract windsea and swell wave components from monthly series of nondirectional and directional wave spectra. Using buoy observations as ground truth, both temporal correlation (TC) and quantile–quantile (QQ) statistical analyses are used to quantify hindcast skill in reproducing measured wave component height, period, and direction attributes. An integrated performance analysis synthesizes the TC and QQ results into a robust assessment of prediction skill and yields distinctive diagnostics on model inputs and source term behavior. The method is applied to a set of Pacific basin hindcasts computed using the WAM, WAVEWATCH III, and WAVAD numerical wave models. The results provide a unique assessment of model performance and have guided the selection of WAVEWATCH III for use in Pacific hindcast production runs for the U.S. Army Corps of Engineers Wave Information Studies Program.

Corresponding author address: Jeffrey L. Hanson, USACE Field Research Facility, 1261 Duck Rd., Kitty Hawk, NC 27949. Email: jeffrey.l.hanson@usace.army.mil

1. Introduction

The U.S. Army Corps of Engineers (USACE) Wave Information Study (WIS) program (Tracy and Cialone 2004) is establishing a multidecade wave climatology for the Pacific basin to support a variety of coastal planning and engineering activities. To facilitate selection of an appropriate wave hindcast technology, the performance of three modern numerical spectral wave models is evaluated in the Pacific basin over calendar year 2000. The technologies evaluated include the third-generation wave model WAM cycle 4.5 (Günther 2002), the third-generation wave model WAVEWATCH III version 2.22 (Tolman 1997, 1999, 2002), and the second-generation wave model WAVAD version 4c (Resio and Perrie 1989).

A significant challenge in evaluating large temporal- or spatial-scale wave hindcasts is the need to statistically reduce millions of spectral estimates to a meaningful measure of prediction skill yet retain a sufficient level of detail to identify model strengths and deficiencies. Wave model validation is typically accomplished using the limited amount of information contained in mean or “bulk” wave parameters, obtained from integral properties of the spectrum (Cardone et al. 1996; Hsu et al. 2002; O’Reilly et al. 1996; Tolman et al. 2002). As these quantities represent averages over all existing wave systems, they provide only a general measure of model performance and can mask higher-order deficiencies.

Here we show that improved measures of hindcast skill can be obtained through use of wave spectral components. A wave component is defined as a region of enhanced energy in the directional wave spectrum S( f, θ) attributed to a windsea or swell system propagating from a specific wind-generation event on the ocean surface. Wave components are identified and extracted using spectral partitioning methods. Originally proposed by Gerling (1992), wave partitioning allows the identification and grouping of component wave systems from spatially and temporally distributed observations of directional wave spectra. A limitation to the Gerling approach is that only those portions of the spectrum that rise above a constant threshold are used to determine the features of particular wave components. Hasselmann et al. (1994) improved on this method by dividing the spectrum into subset domains based on an inverted catchment area approach, thus using the entire spectral region of each peak to compute wave component parameters. Voorrips et al. (1997) used the Hasselmann et al. method to implement a buoy data assimilation scheme for numerical modeling. Hanson (1996) and Hanson and Phillips (2001, hereafter HP01) made improvements to the Hasselmann et al. method by adding wave height, period, and direction clustering routines to track the evolution of individual wave components. Furthermore, they performed wave dispersion calculations to estimate the source time and location of the resulting wave systems. Two primary limitations of the technique have been the lack of an efficient approach to partitioning large arrays of wave spectra and the requirement for directional wave data. The majority of available ground truth stations are nondirectional.

Using wave component data in wave model verification would provide a significant advantage over bulk spectral parameters. Although Beal (1989) and Hasselmann et al. (1994) compared wave model output to buoy and satellite observations at the wave system level, the results were only qualitative in nature and lacked specific metrics for evaluating model performance. Bidlot et al. (2005) showed how important it is to look into the wave spectral domain to gain some insight into model deficiencies. Here we demonstrate a new wave component approach to quantify numerical wave model performance and provide key diagnostic information on model deficiencies. This technique is facilitated through improvements to the spectral partitioning methods of HP01 to allow efficient processing of large wave spectra arrays and through inclusion of nondirectional spectral data in the analysis. The results of temporal correlation and quantile–quantile analyses on wave component data are synthesized into a convenient set of performance scores. We demonstrate this capability on full Pacific basin hindcasts obtained from three numerical modeling technologies forced by identical wind fields over calendar year 2000. Model validations were conducted at seven deep-water buoy sites from the National Data Buoy Center (NDBC) and the Coastal Data Information Program (CDIP). The results facilitated selection of a hindcast technology for the WIS multidecade study.

2. Performance evaluation method

The method presented here, termed the Wave Model Evaluation and Diagnostics System (WaveMEDS), uses wave component attributes of evolving wave spectra to quantify model skill across a variety of metrics, folds these metrics into overall measures of performance, and diagnoses model deficiencies. The specific analysis steps are outlined in Fig. 1. A detailed description of each analysis step follows.

a. Wave partitioning

The basic approach to our wave spectral partitioning method is described by Hanson (1996) and HP01. Details on significant improvements that have been made since HP01 follow. Starting with a directional wave spectrum S( f, θ), HP01 isolated spectral regions Si( f, θ) associated with individual energy peaks with a time-consuming recursive algorithm that assigned each spectrum value to a path of steepest ascent associated with a local peak. This step has been improved with efficient height (H) maxima and watershed delineation transforms originally designed to identify drainage areas from smoothed topographic imagery (Soille 1999; Vincent and Soille 1991). Basic steps to this approach are as follows (see also Tracy et al. 2006):

  1. Invert S( f, θ) so spectral peaks become valleys.
  2. Apply the 8-point connected H-maxima morphological transform to remove finescale noise in the spectrum. The H-maxima transform suppresses all maxima in the spectra whose height is less than a specified threshold. An H-maxima threshold of 2.8 × 10−4 was used in this study.
  3. Round the resulting spectral values to integers between 1 and 100.
  4. Sort integer heights in ascending order.
  5. Starting with the lowest value, incrementally “flood” the morphologically represented spectrum, assigning partition values as flooding continues.

The end result is a label matrix 𝗧( f, θ) that assigns a partition value to each grid point in S( f, θ). This change in basic methodology reduced the record processing time by an order of magnitude, with essentially no change in output results.

The resulting partitions are sorted into wind sea or swell. To be classified as wind sea, the waves represented by a spectral peak must be forced by the component of the wind in the wave direction. A directional wave-age criterion [Eqs. (4) and (5) in HP01] is used to identify and combine the windsea partitions. All remaining peaks are labeled as swell. Adjacent swell peaks that are contiguous in frequency are combined if certain conditions are met. A swell angle test requires mean directions of adjacent peaks be separated by less than a user-supplied threshold angle. A threshold value of 30° yielded optimum results with this study and provided a comfortable margin outside the stated ±10° buoy-direction accuracy. A second test compares the polar ( f, θ) distance between peaks in relation to their spectral spread. Peaks are combined if the spread of either peak is large compared to the distance between the two peaks [Eqs. (6) through (9) in HP01]. As a final step in producing consistent results with minimal noise, any windsea or swell component that falls below a significant wave height threshold of 0.2 m is removed from analysis.

Partition labels stored in 𝗧( f, θ) are modified based on the windsea and swell identification process. Label values contained in 𝗧( f, θ) can be one of the following:

  • 0 represents wind sea;
  • 1 n represents swell systems;
  • −1 represents below height threshold.

An example of a partitioned buoy spectrum containing a windsea and two swell components appears in Fig. 2.

To assess hindcast performance at nondirectional buoy stations an extension to the partitioning method was developed. A pseudodirectional wave spectrum ( f, θ) is created from the nondirectional wave spectrum E( f ) by
i1520-0426-26-8-1614-e1
where D( f, θ) is an arbitrary spread function. The form used in this study is given by
i1520-0426-26-8-1614-e2
with mean wave direction θ set to any arbitrary constant direction (we used 180°) and the result normalized to integrate to unity. The pseudodirectional spectrum is then partitioned with the existing directional method. The partition domains in the spectrum are collapsed back into nondirectional form by integration over θ, resulting in a set of nondirectional spectral partitions. The results of this method applied to a nondirectional version of the wave spectrum depicted in Fig. 2 appear in Fig. 3.

b. Component attributes

A variety of wave component statistics are computed to aid further processing. Integration domains are implied to be over all frequency bins and from 0 to 2π in direction. For computing statistics of a spectral component, all spectral values not falling within the partition domain are set to zero. Statistical descriptors include total energy in the spectral domain
i1520-0426-26-8-1614-e3
significant wave height, approximated by Hmo
i1520-0426-26-8-1614-e4
and peak wave period
i1520-0426-26-8-1614-e5
with peak wave frequency fp computed from a 3-point parabolic fit to the 1D spectral peak. Also computed is the vector mean wave direction,
i1520-0426-26-8-1614-e6
where
i1520-0426-26-8-1614-e7
Both full spectrum and spectral component statistics are computed using the above relationships. Obviously, directional statistics are relevant for directional data only.

c. Error analysis

In this step, we quantify differences between hindcast and observation wave components. Each hourly hindcast spectrum is time paired to the corresponding buoy spectrum. Time lags of up to 10 min between hindcast and buoy times are allowed. For each wave component, the buoy partition template 𝗧( f, θ) is used to identify the corresponding spectral domain in the hindcast spectra. A matching set of hindcast attributes (Hs, Tp, and θ) are computed from these hindcast domains using Eqs. (3)(7), resulting in a set of paired (buoy–hindcast) wave component attributes. Differences between the hindcast and observed components are attributed to model errors, which makes the bold assumption that buoy data are truth.

To facilitate a diagnostic interpretation of results, the resulting data pairs are divided into three wave maturity classes: wind sea, young swell, or mature swell. The HP01 directional wave-age criterion is used to classify spectral peaks forced by the local wind as wind sea. Remaining wave components that have a peak frequency of 0.09 Hz or greater are labeled as young swell, and those with a peak frequency less than 0.09 Hz are labeled as mature swell. This frequency division was found to roughly separate regionally generated young swell with swell that has traveled significant distances in the Pacific. An alternative approach to wave field classification is addressed in the discussion.

For each monthly subset, the hindcast wave component attributes are evaluated against the observed quantities using temporal correlation (TC) analyses and quantile–quantile (QQ) distributions in 99 percentile bins. The TC analysis, performed on time-paired observation and hindcast data, provides an indication of how well the hindcast quantities match the observed quantities in absolute time. In contrast, the QQ analysis divides both the observation and hindcast datasets into quantiles and is used to indicate if the distribution of magnitudes is correct, regardless of occurrence time. The TC comparisons were performed on the height, period, and direction attributes and the QQ distributions were performed on the height and period attributes only. The error metrics used in these analyses are defined below.

A variety of established metrics were used to quantify the monthly TC and QQ comparisons. For n values of buoy measurements m and hindcasts h these metrics include the bias (hindcast–buoy)
i1520-0426-26-8-1614-e8
root-mean-square (RMS) error
i1520-0426-26-8-1614-e9
scatter index
i1520-0426-26-8-1614-e10
where the standard deviation of difference is given by
i1520-0426-26-8-1614-e11
(Guillaume 1990; Cardone et al. 1996); and, for directional data, angular bias (Bowers et al. 2000)
i1520-0426-26-8-1614-e12
where S and C are computed from the directional differences Δθ = |θhθm| by
i1520-0426-26-8-1614-e13
and the circular correlation (Tracy 2002)
i1520-0426-26-8-1614-e14

This analysis results in a set of monthly error metrics (b, ERMS, SI, ba, and cor) that quantify hindcast skill in reproducing physical attributes (Hs, Tp, and θ) of windsea, young swell, and mature swell wave systems at each station. The year 2000 Pacific hindcast study produced a database of approximately 3500 independent measures of skill for each model run.

d. Performance calculation

A performance scoring method was developed to reduce the large error metric database into a small set of performance indicators for overall skill assessment. The first step in this process is to generate raw performance scores by normalizing wave component metrics to mean quantities. These estimators include RMS error performance
i1520-0426-26-8-1614-e15
where root-mean-square of the measurements is given by
i1520-0426-26-8-1614-e16
bias performance
i1520-0426-26-8-1614-e17
scatter index performance
i1520-0426-26-8-1614-e18
and for directional data, angular bias performance
i1520-0426-26-8-1614-e19
and circular correlation performance (already normalized)
i1520-0426-26-8-1614-e20
The nondimensional performance scores range from 0 (uncorrelated) to 1 (perfect correlation) and are averaged across metrics, months, and stations with contributions weighted by sample size. Hence, for a particular wave component attribute, performance for a given month at a given station is
i1520-0426-26-8-1614-e21
with the weighted overall performance across all months and stations for each attribute
i1520-0426-26-8-1614-e22
where n denotes the total number of observations in each subset (i subscript) and for all subsets combined (c subscript).

e. Wave system analysis

Additional hindcast diagnostics can be performed through examination of evolving wave systems, formed through application of a clustering algorithm to link like wave components through time (HP01). As Pacific Ocean buoy stations can be subjected to dozens of wave systems over the course of a month, the total wave power I is used to identify and select the most energetic systems for analysis. Average flux of wave energy E per unit wave crest is governed by wave group velocity Cg such that
i1520-0426-26-8-1614-e23
where P is often referred to as the wave power per unit crest length and Cg is approximated by
i1520-0426-26-8-1614-e24
Integrating P over the duration of a particular wave system yields the integrated wave power or total intensity of a wave event per unit crest length (U.S. Army Corps of Engineers 2003):
i1520-0426-26-8-1614-e25
Adopting certain units of ρ (kg m−3), g (m s−2), Hs (m), Cg (m s−1), and t (s) in the above yields I in Joules per meter. This can be interpreted as the total work done by a single wave system over a crest length of 1 m.

3. Pacific hindcast study

As the purpose of this investigation was to identify the best-performing technology for the WIS Pacific hindcast, each modeling group was allowed to select a bathymetry grid and develop model setup parameters to optimize individual model performance in their own computing environment. However, all model runs used a common set of high-quality wind fields that spanned the entire Pacific Ocean basin for the full calendar year 2000. An overview of the participating modeling groups and resulting hindcast runs appear in Table 1. Specific details on the wind fields, model technologies, and ground truth data appear in the following sections.

a. Wind fields

High-quality, consistent, neutral stability wind fields (NRAQ+) at 3-h intervals on a 0.5° spatial hindcast grid are being developed for WIS with the goal of accurately representing the full range of meteorological events that occur in the Pacific. Wind fields are generated by the marine meteorology group at Oceanweather, Inc. (OWI) using baseline National Centers for Environmental Prediction–National Center for Atmospheric Research (NCEP–NCAR) global reanalysis (NRA) 6-hourly, 10-m surface winds on a Gaussian geographic grid (Kalnay et al. 1996). NRA fields are adjusted using Quick Scatterometer [QuikSCAT (Q/S)] winds by linear regressions through QQ plots in 45° wind direction sectors. NRA data from the full year (2000) are included in the QQ analysis, as seasonally stratified regressions are not statistically independent. Since tropical cyclone winds are poorly resolved in the NRA wind fields, OWI’s mesoscale planetary boundary layer (PBL) cyclone model used available tropical cyclone information over the full domain to recreate tropical cyclone winds for blending into the NRAQ+ winds as previously detailed in Swail et al. (2000) for Atlantic hindcast wind fields. Beyond the Q/S adjustments, no additional observations are used in the analysis of the NRA–QuikSCAT winds. Compared to the NRA winds, the NRAQ+ winds are superior in capturing synoptic and mesoscale events.

b. Wave models

Each numerical wave model (Table 1) was set up to define directional spectra in terms of 25 logarithmically spaced frequency bins from 0.03 to 0.4 Hz and 24 regularly spaced direction bins of 15° width. The two highest frequency bins were dropped from the analysis in order to match the 0.03–0.34-Hz frequency range of the ground truth buoy data. Each model was run at 0.5° spatial resolution covering 64°S–64°N latitude and 110°E–60°W longitude. The models were initiated on 1 January 2000 with the NRAQ+ winds and run for a full calendar year. The first two weeks of spinup in January were excluded from the analysis. Specific details on each model run appear below.

1) WAM model

The third-generation WAM cycle 4.5 (Komen et al. 1994; Günther 2002) was run on a Cray X1 platform with no parallelization (Table 1). Water depths were obtained from General Bathymetric Chart of the Oceans (GEBCO), a digital bathymetry database with nominal horizontal resolution of 3 min. Depth data were then subsampled to a 0.5° fixed longitude–latitude grid and hand edited to include spatially unresolved islands. Full directional wave spectra were exported hourly at the locations of the Pacific wave buoys (1D and 2D) used in this study (Table 2).

2) WAVEWATCH III

The third-generation numerical wave model WAVEWATCH III version 2.22 (Tolman 2002) was run with the standard operational default settings that include the Tolman and Chalikov (1996) source functions. The operational National Oceanic and Atmospheric Administration (NOAA) bathymetry grid was used with island obstructions based on the 2-minute gridded elevations/bathymetry for the world (ETOPO2; Tolman 2003). WAVEWATCH III was run on an Origin 3800 (O3K) platform in Message Passing Interface (MPI) parallel mode using 16 processors (Table 1). Hourly wave parameter and directional wave spectra were saved at each of the ground truth stations used by this study.

3) WAVAD

The second-generation (2G) spectral wave model WAVAD version 4c (Resio and Perrie 1989) maintains equilibrium between the input winds and the nonlinear wave energy flux with an assumed f4 spectral shape. Wave growth is based on a combined Phillips and Miles mechanism. Weak nonlinear wave–wave interactions are represented as a momentum flux to both lower and higher frequencies away from the spectral peak. Energy transferred to higher frequencies is assumed to be lost by breaking. WAVAD hindcasts were produced on a 3.4-GHz personal computer using the identical bathymetry and obstruction grids used for WAVEWATCH III. Spectral output was archived at hourly intervals for the selected buoy locations, and wave parameter fields over the entire grid were saved at 6-h intervals.

c. Ground truth observations

The WaveMEDS approach requires observations of wind (for partitioning) and directional (2D) or nondirectional (1D) wave spectra as input. These ground truth data were obtained from the NDBC and CDIP buoy networks. Locations of the seven deep-water wave stations used in this study are shown on Fig. 4. These stations cover offshore conditions for much of the U.S. west coast and are sufficiently well dispersed to include a wide range of wave-generation and swell-propagation environments. Specific details on these stations appear in Table 2. They include a variety of measurement platforms and instrument payloads, including three stations equipped with directional wave sensors (46042, 071, and 51028). Station 071 is a commercial Datawell MK II Directional Waverider buoy. Details on NDBC payload configurations are available at the NDBC Web site.

The NDBC stations provide hourly 8-min average wind speed and direction at a sensor height of 5 m above sea level. Measurement accuracy is ±1.0 m s−1 for wind speed and ±10° for wind direction. There is no wind sensor at CDIP station 071; however, winds from NDBC station 46063, located 23 km southeast of 071, were used to estimate local wind-generated forcing around CDIP station 071. Wave spectra from the NDBC stations are computed hourly from 20-min records over a frequency range of 0.03–0.4 Hz. Reported quantities include the nondirectional (1D) energy-frequency spectrum E( f ) and, for directional buoys, the vector mean direction θ( f ) and the directional distribution parameters r1( f ), r2( f ), α1( f ), and α2( f ). These quantities reported by NDBC incorporate various corrections for hull-mooring response (Steele et al. 1992) and can be described in terms of the Longuet–Higgins Fourier coefficients a1, a2, b1, and b2 as
i1520-0426-26-8-1614-e26
where a° = E( f ). According to NDBC (http://www.ndbc.noaa.gov/), wave measurement accuracies are ±0.2 m in wave height, ±1.0 s in wave period, and ±10° in wave direction.

CDIP directional wave data are computed from half-hour records over the frequency band 0.025–0.58 Hz. Reported quantities include E( f ), θ( f ) and the Fourier coefficients a1, a2, b1, and b2. Stated accuracies of the Datawell directional wave buoy used by CDIP are 3% of buoy heave and 0.4°–2° in direction. The maximum-likelihood estimator of Oltman-Shay and Guza (1984) is used to compute the directional wave spectrum S( f, θ) from both the NDBC and CDIP Fourier coefficients.

Resulting spectra were linearly interpolated to a 23-frequency (0.04–0.34 Hz), 15° bin resolution used in the partitioning analysis. Nondirectional spectra were interpolated in frequency only. These interpolations were necessary to make one-to-one comparisons between buoy and hindcast spectral features.

4. Model performance

At any given instance, the Pacific Ocean contains numerous temporally and spatially evolving wave systems originating from a wide variety of distributed wind-generation events. The challenge of numerical wave modeling is to capture the essence of this dynamic wave field. A wave vector history shown in Fig. 5 from a 2-week period of November 2000 at station 51028 provides a convenient display of wave system attributes. Included are the buoy-observed wave systems (Fig. 5a) and the results from our three model hindcasts (Figs. 5b–d). The complexity of the wave field is evident with 3–5 distinct windsea and swell wave systems present at any given instant in time and each lasting for several days duration (Fig. 5a). The result is 12 or more wave systems passing through this area during the 2-week period shown. Numerical models vary in their skill of capturing the evolution of the various wave systems present (Figs. 5b–d). Although these plots provide an indication of hindcast fidelity, a more precise measure of model skill was required to select a technology for the WIS Pacific hindcast.

The WaveMEDS approach quantifies differences in the three Pacific basin wave hindcasts, with seven deep-water NDBC and CDIP buoys depicted in Fig. 4 and described in Table 2 used as ground truth stations in the analysis. As will be demonstrated, results show that WAVEWATCH III provides a superior hindcast for the input winds, boundary conditions, and model settings employed in this study. In the following sections, a top-down reporting of results compares overall model performance for the three hindcasts and explores spatial and temporal variability in prediction skill for specific wave field attributes. A wave system analysis is performed to identify hindcast deficiencies and guide future model improvements.

a. Performance summary

Annual model performance scores for significant wave height, peak wave period, and mean wave direction appear in Tables 3, 4, and 5, respectively. In each table, results of temporal correlations and quantile–quantile distributions are provided for windsea, young swell, and mature swell components. As discussed above, performance scores can range from 0.0 to 1.0, with 1.0 being a perfect match of hindcast data to observations. Combined scores (in the bottom rows) represent the weighted average (by sample size) of the three wave component classes and provide an overall measure of model skill in predicting each physical attribute (height, period, and direction).

The three hindcasts exhibit varied performance with combined wave height scores of 0.78–0.88, combined wave period scores of 0.88–0.96, and combined wave direction scores of 0.83–0.91. QQ scores are higher than the corresponding TC scores, suggesting that the hindcasts are better skilled at capturing event distributions than correctly matching event times. Furthermore, wave period and wave direction scores for each model are higher than wave height scores. Although the three models are rather close in overall performance, WAVEWATCH III consistently has the highest combined scores in each category.

Performance scores for each wave component reveal that hindcast skill varies with wave maturity. Mature swell height (Table 3) has significantly lower scores than windsea or young swell height in each model hindcast. As will be demonstrated, mature swell height is the most significant factor degrading Pacific wave model performance.

b. Error diagnostics

Using performance scores as a guide, TC and QQ errors are explored to identify patterns in hindcast deficiencies. Here we will focus exclusively on exploring wave height error trends, since this attribute produced the lowest performance scores in all three hindcasts. Furthermore, we will limit our discussion to windsea and mature swell errors, as young swell errors tend to reflect a blend of the other two.

There is an important issue to be raised regarding the comparison of error metrics (RMS error, bias, and scatter index) obtained in this study to those obtained by other case studies (i.e., Guillaume 1990; Cardone et al. 1996). These previous studies typically report combined errors for all stations and over the entire hindcast. Since we are reporting errors by station and month, the means rather than extremes from our figures need to be compared to previous studies. Furthermore, we compute errors for partitions of the spectrum, whereas most other studies compute total wave height errors. As spectral energy estimates are expected to have a larger random error than mean wave height, errors computed from partitioned parameters are expected to be larger than those computed from total wave height.

1) Windsea temporal correlations

Wave height TC errors for wind sea appear in Fig. 6. Monthly error metrics (ERMS, b, and SI) are depicted at all seven ground truth stations and for each hindcast. Error plots are organized by hindcast (columns) and error metric (rows). Windsea height error trends are similar for both of the 3G hindcasts, with WAVEWATCH exhibiting the lowest errors of all three (center column). With the exception of coastal California stations 46042 and 00071, windsea RMS errors (top row) tend to be lowest in the summer months, with WAVEWATCH ERMS < 0.5 m for all but 3 occurrences. Windsea height biases (middle row) are significantly larger in WAM and WAVAD, with WAM windsea bias generally negative, except for positive biases at the two coastal California stations during spring, summer, and fall. WAVEWATCH windsea height biases for all stations cluster around zero, except for overestimation of summer wave heights at 46042. Wind seas from WAVAD exhibit the greatest amount of variability, with biases up to 0.6 m at 46042 and 00071. On average, WAVEWATCH exhibits the lowest scatter index values.

2) Mature swell temporal correlations

The relatively poor mature swell wave height performance (Table 3) is clearly evident in the TC error trends of Fig. 7. There is a definitive seasonal trend in mature swell height RMS error at all stations and in all three hindcasts. This trend shows RMS errors increasing during the Northern Hemisphere winter months (November through March), coinciding with the period of increased North Pacific cyclogenesis. Furthermore, these errors are most significant at stations 46001, 46005, and 51001, which are directly in the path of winter swells emanating from the North Pacific. It is noteworthy that, in summer months, when southern swell dominates, lower wave height errors prevail. Mature swell height bias from WAM and WAVEWATCH III exhibits a similar seasonal trend, with a positive bias in winter months and near zero bias during the remainder of the year. This trend is most distinctive in the WAVEWATCH III hindcast. In contrast, the WAVAD mature swell height biases tend to be slightly negative (−0.5–0.0 m at most stations) with no discernable seasonal trend. At all but one station, mature swell height scatter index does not exhibit any specific trends across time or location, other than being slightly more variable in summer months when wave heights are lower. In general, WAVEWATCH III mature swell heights exhibit lower scatter index values with the least amount of variability.

3) Wave height quantile–quantile distributions

Monthly wave height QQ distributions provide additional details on systematic errors not associated with event timing offsets. Example wave height QQ distributions from all three hindcasts during February, May, August, and November 2000 at station 51028 appear in Fig. 8. These plots compare the observed and hindcast windsea, young swell, and mature swell wave height distributions computed in 99 percentile bins. The solid black line represents a perfect agreement between observation and hindcast height distributions. The two 3G models have the best overall agreements in spring and summer. WAM wind seas show the best agreement at elevated wave heights, with lower heights typically exhibiting a negative bias. This is supported by the Hanson and Jensen (2004) finding that WAM wind seas exhibit a slow response to changing wind conditions and that elevated or consistently steady winds are required to match observed spectral levels. This issue has been also addressed by Bidlot et al. (2007). Mature swell from the 3G models exhibits a nearly constant positive bias at observed wave heights >1 m. In contrast, WAVAD mature swells have a constant negative bias in May, an increasing positive bias in November, and mixed results the remaining two months.

Results of the various hindcast performance analyses facilitated the selection of WAVEWATCH III for the ongoing WIS multidecade hindcast study. Additional results and discussion will focus on this modeling technology.

c. Swell bias

The WaveMEDS hindcast analysis led us to the conclusion that winter mature swell wave heights are the most significant attribute degrading Pacific hindcast accuracy in all three modeling technologies. A wave system approach is now taken to further diagnose the source of these errors in the WAVEWATCH III hindcast. As a representative example, we focus on the mature swell hindcast errors during November 2000 at the Christmas Island NDBC station 51028. To identify events most likely to contribute to swell height bias, integrated wave power [Eq. (25)] was used to identify the five most energetic wave systems during this period. The resulting wave systems (A through E) are depicted in the wave vector displays of Fig. 9. Events A and B are the result of high-latitude storms deep in the Southern Ocean, producing mature swells that can travel 5000–6000 km to reach station 51028. Events C and D are a result of the low pressure winter cyclones that regularly pass across the North Pacific during winter months and contribute to a dynamic multicomponent wave field in the Gulf of Alaska (HP01). Mature swells reaching station 51028 from this generation region have traveled up to 6500 km. Event E is a young swell resulting from the northeast trade wind belt centered at about 15°N latitude.

Examination of wave system events from several stations indicates that mature swell height bias has a strong geographic dependence on swell origin. Comparisons of hindcast and measured wave system Hs, Tp, and θ statistics from representative wave events A and D (from Fig. 9) appear in Figs. 10 and 11. Heights, periods, and directions of the Southern Hemisphere wave system A (Fig. 10) are captured by WAVEWATCH III with minimal errors. A slight temporal offset between the hindcast and observed wave periods suggest hindcast generation time was somewhat earlier than the actual generation time for this swell event. Wave direction variability is within the 15° angular resolution of the spectral data. The duration of this event is a few days longer in the hindcast record; however, this is expected since very low energy components get lost in the noise of buoy data from high-energy environments. Most of the southern mature swell events investigated are very similar to these. These results suggest that Southern Ocean mature swell is reasonably well represented in the WAVEWATCH III hindcast. In contrast, a significant wave height bias of nearly 1.5 m exists at the peak of northern swell event D (Fig. 11). Corresponding wave periods show a remarkable agreement between hindcast and observation. Wave direction variability is mostly within the 15° resolution of the data. Although this is an extreme event selected to make a point, this trend of positive height bias with reasonable period and direction agreement is typical of many WAVEWATCH III hindcast wave systems emanating from the North Pacific in winter months.

5. Discussion

The purpose of this investigation was to develop an approach for the validation and diagnostic evaluation of numerical ocean-surface wave model performance and use this technique to select a Pacific hindcast modeling technology. In this discussion we will review limitations and alternative methods to this approach and address issues that may be impacting our Pacific hindcast performance.

a. Model performance

Although overall performance results from the three Pacific hindcasts are reasonably similar, we will focus our analysis on the WAVEWATCH III hindcast. Within this study, mature swell height bias appears to be the most significant hindcast limitation. A separate analysis was performed to compare our WAVEWATCH III hindcast total significant wave height fields to Ocean Topography Experiment (TOPEX)/Poseidon altimeter data (Scott 2005). Mean wave height bias was computed for year 2000 and appears in Fig. 12. Note that the maximum average height biases (>0.5 m) cover a broad area of the central and eastern North Pacific Ocean and include the areas of intensive wave generation resulting from low pressure cyclogenesis during these months.

When investigating wave height bias one must start with an examination of the driving winds. Certainly both the WAVEWATCH III and WAM results, along with the buoy and altimeter comparisons, suggest that wind speed bias may be a significant issue contributing to mature swell height bias. It is interesting to note, however, that the WAVAD mature swell height bias is generally negative at most stations with no discernable seasonal trend (Fig. 7) and hence does not readily support the hypothesis that the North Pacific wintertime winds are elevated above realistic values. To evaluate the impact of a more carefully constructed wind field, the marine meteorology experts at Oceanweather, Inc. conducted a full kinematic analysis (Cox et al. 1996; Cox and Cardone 2000) on the nine most intense Northern Hemisphere storms occurring in March 2000. The kinematic analysis was supported by QuikSCAT winds and included the assimilation of available buoy data. Results were blended into the March 2000 baseline NRAQ+ wind fields, resulting in a new NRAQ+K wind field for this month only. NRAQ+K winds were used to generate a new WAVEWATCH III hindcast. Although the NRAQ+K hindcast results exhibit lower wave heights during peak events, improvement is only a small percentage of the total bias. Hence, it appears that input wind magnitudes are certainly part of the swell bias problem but not the only contributing factor. Furthermore, it is also noteworthy that a positive height bias in the northern Pacific Ocean during winter has been a persistent feature of the operational WAVEWATCH III model at NCEP, based on comparisons with Jason-1, Geosat Follow-On (GFO), and Envisat altimeter data (H. L. Tolman, 2009 unpublished manuscript) as well as with buoy data (Bidlot et al. 2007). Considering the different sources and resolutions of the wind fields involved, it is not likely that this wave model bias is solely a consequence of the winds.

Another potential source of error is the parameterization of atmospheric drag in the wind input source term. The standard WAVEWATCH III formulation that was available at the time of this study extrapolated the drag coefficient (CD) to continually increase as a function of wind speed. However, recent observations suggest that CD caps in the neighborhood of approximately 2.5 × 10−3 as whitecapping fully develops at wind speeds above approximately 30 m s−1 (Powell et al. 2003). In extreme winds, the extrapolated CD in WAVEWATCH III could potentially lead to increased wave development and an associated wave height bias. To test if elevated drag coefficients are contributing to the WAVEWATCH III swell height bias, an additional year 2000 hindcast was made with a wind input source term modified to provide a CD cap of 2.5 × 10−3. A map of the resulting annual mean wave height bias based on TOPEX/Poseidon altimeter data (Scott 2005) appears in Fig. 13. This map can be directly compared with the annual mean bias map of Fig. 12, which depicts WAVEWATCH III height bias with no CD cap imposed on the wind input source term. Indeed, implementation of the CD cap results in a 30%–40% reduction in wave height bias in the central North Pacific generation region and a 20%–30% reduction in total wave height bias along the U.S. coast. In an update to his earlier work, Powell (2007) notes that CD has a very complicated behavior with both wind speed and storm orientation. Additional work in this area is clearly warranted.

The remaining source terms (wave–wave interaction and dissipation) are also likely candidates for contributing to the observed swell height bias in WAM and WAVEWATCH III. The wave–wave interaction term uses the discrete interaction approximation (DIA) of Hasselmann and Hasselmann (1985). Although this computationally efficient algorithm is at the core of most 3G wave modeling, evidence suggests that away from the spectral peak and in multimodal wave fields the DIA greatly undersamples the complex set of nonlinear interactions taking place, leading to erroneous estimates of the spectral source function, which will potentially result in spectral shapes deviating from observations (Resio and Perrie 2006). Furthermore, the parameterized dissipation term has largely been used as a device to tune model performance rather than correctly capturing the physical mechanisms of wave decay. Fortunately, there are efforts aimed at adding a more physical representation of dissipation (Ardhuin et al. 2008). It is likely that significant gains in hindcast skill will not be made until significant improvements are made in these source term formulations.

b. Technique limitations and alternatives

The primary limitation to the wave system validation and diagnostics approach is the scarcity of available ground truth data. The number of operational directional wave stations is too small to allow a fully comprehensive verification of model output. Furthermore, floating buoys do not resolve the full directional distribution, which may have an impact on the results. The primary benefit of these tools in their present form is to provide an augmentation of traditional bulk validation techniques.

Additional sources of information can be incorporated to make this approach more operationally viable. For example, extension of the partitioning approach to the 1D spectral domain has added several stations of available ground truth information. An additional source of ground truth information can be obtained from remote sensing of the sea surface. In particular, satellite synthetic aperture radar (SAR) measurements (Beal 1989) provide estimates of ocean surface directional wave spectra along the SAR track. These could be used to validate the spatial representation of evolving hindcast wave systems. Furthermore, SAR data only capture a limited range of the spectral space. Therefore, it is essential in validation and calibration of SAR data that only resolved wave components are compared with in situ data. This can be done most efficiently with partitioning of the SAR and in situ data as in the present study. Finally, significant wave height estimates from satellite altimeters can be used to verify the spatial distribution of bulk hindcast wave heights. As shown here, the use of altimetry data in the Pacific hindcast study has provided valuable insight into the spatial distribution of hindcast height bias.

As a result of this study, a few recommendations for improving the WaveMEDS technique can be made. To identify matching domains in buoy and hindcast spectra in this study we employed a partition template overlay method. A strength of this approach is that the resulting error statistics account for all of the energy present (above the low-energy threshold) in the wave field. However portions of the energy from more than one hindcast peak may fall into any given spectral domain computed from the buoy data. An alternative approach would be to compute a separate partition template for each hindcast spectrum and use a clustering approach to match components. This could provide added benefits, such as a separate accounting of extraneous energy peaks present in either the observations or the hindcast.

The approach to classifying wave components can also be improved. At present wind seas are identified using a directional wave-age criterion, and young and mature swell groups are formed based on the peak frequency of each swell partition. Although this rudimentary separation of swell types works fairly well in the Pacific basin, it would not necessarily be appropriate in other oceanographic regions. A more universal approach would be to separate wave components based on the amount of wind forcing they are subject to. This would allow the separation of wave components into pure windsea, mixed sea, and pure swell categories. This is currently a topic of active testing by two of the authors (Hanson and Tolman).

6. Conclusions

Three numerical spectral wave models were tested to identify the best technology for conducting a multidecade WIS Pacific hindcast. Each technology was evaluated with identical forcing over the year 2000 with seven deep-water NDBC and CDIP buoys employed as ground truth. The Wave Model Evaluation and Diagnostics System (WaveMEDS) provided an efficient mechanism for reducing millions of spectral values from the three hindcasts into a convenient database of monthly hindcast errors organized as a function of physical attribute (height, period, and direction), wave maturity (wind sea, young swell, and mature swell), and station location. Application of a unique set of performance calculations further reduced this information into a concise set of nine overall performance scores providing a robust assessment of model prediction skill and guiding additional diagnostic evaluations.

The three models exhibited varied performance in the depiction of wind sea and young swell physical attributes. A noteworthy problem area is in the prediction of mature swell in winter months, with elevated height errors in all three models. Overall the WAVEWATCH III hindcast exhibited consistently higher performance scores than those from WAM and WAVAD.

Diagnostic evaluation of the 3G hindcast mature swell height errors suggests that this problem emanates from winter swell produced in the North Pacific. This finding is also confirmed by inspection of WAVEWATCH III North Pacific hindcast bias derived from satellite altimetry. Further analysis of the data suggests that underresolved storms in the wind fields contribute only a small percentage to the total error. An examination of source term behavior shows that a cap on the atmospheric drag coefficient has a fairly positive impact on reducing wave height bias in the primary wave-generation areas; however, recent work indicates that a constant cap is not the full story. It is further suspected that the wave–wave interaction and dissipation source terms are likely contributors to swell height error and that significant model advancements are not likely until these source terms are improved.

As a result of this analysis, WAVEWATCH III was selected for use in a new 1981–2004 Pacific basin hindcast that is now available on the USACE WIS Web page (http://www.frf.usace.army.mil/cgi-bin/wis/pac/pac_main.html). Furthermore, it should be noted that improved versions are now available for both WAM and WAVEWATCH III.

Acknowledgments

The authors are grateful to Elizabeth Orelup, Erin Harris, Andrew Cox, and Dr. Vince Cardone of Oceanweather, Inc. for producing the outstanding WIS wind fields; to Dr. Bob Jensen for his skillful preparation of the WAM hindcasts; to Dr. Donald Resio for several insightful suggestions to this investigation; to Dr. Charles Long for patiently working his editorial magic on an early version of the manuscript; to Dr. Heidi Wadman for her skillful help in preparing the figures; and to the anonymous reviewers for their insightful suggestions.

REFERENCES

  • Ardhuin, F., , Hamon M. , , Collard F. , , Chaperon B. , , and Queffeulou P. , 2008: Spectral wave evolution and spectral dissipation based on observations: A global validation of new source functions. Proc. Fourth Chinese–German Joint Symposium on Coastal and Ocean Engineering, Darmstadt, Germany, TU Darmstadt.

    • Search Google Scholar
    • Export Citation
  • Beal, R. C., 1989: Directional Ocean Wave Spectra. The Johns Hopkins University Press, 218 pp.

  • Bidlot, J-R., , Janssen P. A. E. M. , , and Abdalla S. , 2005: On the importance of spectral wave observations in the continued development of global wave models. Proc. Fifth Int. Symp. on Ocean Wave Measurement and Analysis: WAVES2005, Madrid, Spain, CEDEX, 16 pp.

    • Search Google Scholar
    • Export Citation
  • Bidlot, J-R., and Coauthors, 2007: Inter-comparison of operational wave forecasting systems. Proc. 10th Int. Workshop on Wave Hindcasting and Forecasting and Coastal Hazard Symp., North Shore, Oahu, HI, European Centre for Medium-Range Weather Forecasts, 22 pp.

    • Search Google Scholar
    • Export Citation
  • Bowers, J. A., , Morton I. D. , , and Mould G. I. , 2000: Directional statistics of the wind and waves. Appl. Ocean Res., 22 , 1330.

  • Cardone, V. J., , Jensen R. E. , , Resio D. T. , , Swail V. R. , , and Cox A. T. , 1996: Evaluation of contemporary ocean wave models in rare extreme events: The “Halloween Storm” of October 1991 and the “Storm of the Century” of March 1993. J. Atmos. Oceanic Technol., 13 , 198230.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cox, A. T., , and Cardone V. J. , 2000: Operational system for the prediction of tropical cyclone generated winds and waves. Proc. Sixth Int. Workshop on Wave Hindcasting and Forecasting, Monterey, CA, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, Fleet Numerical Meteorological Center, and Meteorological Service of Canada, 158–166.

    • Search Google Scholar
    • Export Citation
  • Cox, A. T., , Greenwood A. R. , , Cardone V. J. , , and Swail V. R. , 1996: An interactive objective kinematic analysis system. Proc. Fourth Workshop on Wave Hindcasting and Forecasting, Banff, AB, Federal Panel on Energy Research and Development and Atmospheric Environment Service of Environment Canada, 171–183.

    • Search Google Scholar
    • Export Citation
  • Gerling, T. W., 1992: Partitioning sequences and arrays of directional ocean wave spectra into component wave systems. J. Atmos. Oceanic Technol., 9 , 444458.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guillaume, A., 1990: Statistical tests for the comparison of surface gravity wave spectra with application to model validation. J. Atmos. Oceanic Technol., 7 , 551567.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Günther, H., 2002: WAM cycle 4.5. Institute for Coastal Research, GKSS Research Centre, Geesthacht, Germany, 43 pp.

  • Hanson, J. L., 1996: Wind sea growth and swell evolution in the Gulf of Alaska. Ph.D. dissertation, The Johns Hopkins University, 151 pp.

    • Search Google Scholar
    • Export Citation
  • Hanson, J. L., , and Phillips O. M. , 2001: Automated analysis of ocean surface directional wave spectra. J. Atmos. Oceanic Technol., 18 , 277293.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hanson, J. L., , and Jensen R. E. , 2004: Wave system diagnostics for numerical wave models. Proc. Eighth Int. Workshop on Wave Hindcasting and Forecasting, Oahu, HI, Joint WMO/IOC Technical Commission for Oceanography and Marine Meteorology Tech. Rep. 29, WMO/TD-1319, 19 pp. [Available online at http://www.waveworkshop.org/8thWaves/Papers/E3.pdf].

    • Search Google Scholar
    • Export Citation
  • Hasselmann, S., , and Hasselmann K. , 1985: Computations and parameterizations of the nonlinear energy transfer in a gravity wave spectrum. Part I: A new method for efficient computations of the exact nonlinear transfer integral. J. Phys. Oceanogr., 15 , 13691377.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hasselmann, S., , and Bruning C. , 1994: Extraction of wave spectra from SAR image spectra. Dynamics and Modelling of Ocean Waves, G. J. Komen et al., Eds., Cambridge University Press, 391–401.

    • Search Google Scholar
    • Export Citation
  • Hsu, L. Y., , Rogers W. E. , , and Dykes J. D. , 2002: WAM performance in the Gulf of Mexico with COAMPS wind. Proc. Seventh Int. Workshop on Wave Hindcasting and Forecasting, Banff, AB, Meteorological Service of Canada, 151–159.

    • Search Google Scholar
    • Export Citation
  • Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc., 77 , 437471.

  • Komen, G. J., , Cavaleri L. , , Donelan M. , , Hasselmann K. , , Hasselmann S. , , and Janssen P. A. E. M. , 1994: Dynamics and Modelling of Ocean Waves. Cambridge University Press, 560 pp.

    • Search Google Scholar
    • Export Citation
  • Oltman-Shay, J. M., , and Guza R. T. , 1984: A data adaptive ocean wave directional-spectrum estimator for pitch and roll type measurements. J. Phys. Oceanogr., 14 , 18001810.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • O’Reilly, W. C., , Herbers T. H. C. , , Seymour R. J. , , and Guza R. T. , 1996: A comparison of directional buoy and fixed platform measurements of Pacific swell. J. Atmos. Oceanic Technol., 13 , 231238.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Powell, M. D., 2007: New findings on hurricane intensity, wind field extent, and surface drag coefficient behavior. Proc. 10th Int. Workshop on Wave Hindcasting and Forecasting and Coastal Hazard Symp., North Shore, Oahu, HI, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, Environment Canada, and WMO/IOC Joint Technical Commission for Oceanography and Marine Meteorology (JCOMM), M5. [Available online at http://www.waveworkshop.org/10thWaves/ProgramFrameset.htm].

    • Search Google Scholar
    • Export Citation
  • Powell, M. D., , Vickery P. J. , , and Reinhold T. A. , 2003: Reduced drag-coefficient for high wind speeds in tropical cyclones. Nature, 422 , 279283.

  • Resio, D., , and Perrie W. , 1989: Implications of an f−4 equilibrium range for wind-generated waves. J. Phys. Oceanogr., 19 , 193.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Resio, D., , and Perrie W. , 2006: A two-scale approximation for efficient representation of nonlinear energy transfers in a wind wave spectrum. Proc. 9th Int. Workshop on Wave Hindcasting and Forecasting, Victoria, BC, Environment Canada, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, and the WMO/IOC Joint Technical Commission for Oceanography and Marine Meteorology (JCOMM), 37 pp. [Available online at http://www.waveworkshop.org/9thWaves/Papers/Resio.pdf].

    • Search Google Scholar
    • Export Citation
  • Scott, D., 2005: Pacific Ocean wave information study validation of wave model results against satellite altimeter data. Draft Rep., W.F. Baird and Associates, 187 pp.

    • Search Google Scholar
    • Export Citation
  • Soille, P., 1999: Geodesic transformations. Morphological Image Analysis: Principles and Applications,. Springer-Verlag, 155–182.

  • Steele, K. E., , Teng C. C. , , and Wang D. W. C. , 1992: Wave direction measurements using pitch-roll buoys. Ocean Eng., 19 , 349375.

  • Swail, V. R., , Ceccacci E. A. , , and Cox A. T. , 2000: The AES40 North Atlantic wave reanalysis: Validation and climate assessment. Proc. Sixth Int. Workshop on Wave Hindcasting and Forecasting, Monterey, CA, U.S. Army Research and Development Center’s Coastal and Hydraulics Laboratory, Fleet Numerical Meteorological Center, and Meteorological Service of Canada, 15 pp. [Available online at http://www.waveworkshop.org/6thWaves/A1.pdf].

    • Search Google Scholar
    • Export Citation
  • Tolman, H. L., 1997: A new global wave forecast system at NCEP. Ocean Wave Measurements and Analysis, Vol. 2, B. L. Edge and J. M. Helmsley, Eds., ASCE, 777–786.

    • Search Google Scholar
    • Export Citation
  • Tolman, H. L., 1999: User manual and system documentation of WAVEWATCH-III version 1.18. Tech. Note, 110 pp.

  • Tolman, H. L., 2002: User manual and system documentation of WAVEWATCH-III version 2.22. Tech. Note, 139 pp.

  • Tolman, H. L., 2003: Treatment of unresolved islands and sea ice in wind wave models. Ocean Modell., 5 , 219231.

  • Tolman, H. L., , and Chalikov D. V. , 1996: Source terms in a third-generation wind wave model. J. Phys. Oceanogr., 26 , 24972518.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tolman, H. L., , Balasubramaniyan B. , , Burroughs L. D. , , Chalikov D. V. , , Chao Y. Y. , , Chen H. S. , , and Gerald V. M. , 2002: Development and implementation of wind-generated ocean surface wave models at NCEP. Wea. Forecasting, 17 , 311333.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tracy, B. A., 2002: Directional characteristics of the 1990–1999 Wave Information Studies Gulf of Mexico hindcast. Proc. Seventh Int. Workshop on Wave Hindcasting and Forecasting, Meteorological Service of Canada, U.S. Army Engineer Research and Development Center’s Coastal and Hydraulics Laboratory, and Fleet Numerical Meteorology and Oceanography Center, 10 pp. [Available online at http://www.waveworkshop.org/7thWaves/Papers/Tracy.pdf].

    • Search Google Scholar
    • Export Citation
  • Tracy, B. A., , and Cialone A. , 2004: Comparison of Gulf of Mexico Wave Information Studies (WIS) 2-G hindcast with 3-G hindcasting. Proc. Eighth Int. Workshop on Wave Hindcasting and Forecasting, Oahu, HI, Meteorological Service of Canada and U.S. Army Engineer Research and Development Center, 15 pp. [Available online at http://www.waveworkshop.org/8thWaves/Papers/C2.pdf].

    • Search Google Scholar
    • Export Citation
  • Tracy, F. T., , Tracy B. A. , , and Hanson J. L. , 2006: Sorting out waves with a fast sort algorithm. ERDC MSRC Resource (Fall 2006), U.S. Army Engineer Research and Development Center, Vicksburg, MS, 6 pp.

    • Search Google Scholar
    • Export Citation
  • U.S. Army Corps of Engineers, 2003: Coastal engineering manual. Publication Number EM 1110-2-1100, 215 pp.

  • Vincent, L., , and Soille P. , 1991: Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell., 13 , 583598.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Voorrips, A. C., , Makin V. K. , , and Hasselmann S. , 1997: Assimilation of wave spectra from pitch-and-roll buoys in a North Sea wave model. J. Geophys. Res., 102 , 58295849.

    • Crossref
    • Search Google Scholar
    • Export Citation

Fig. 1.
Fig. 1.

WaveMEDS.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 2.
Fig. 2.

Example partition results for NDBC buoy station 46042 directional wave spectrum on 10 Feb 2000 (2050 UTC). White lines denote the boundaries of each wave component as represented in the partition template 𝗧( f, θ). Note that this record contains windsea (0), primary swell (1), and secondary swell (2) wave components.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 3.
Fig. 3.

Example nondirectional partition results for NDBC buoy station 46042 wave spectrum on 10 Feb 2000 (2050 UTC). The windsea, primary swell, and secondary swell wave component domains are indicated.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 4.
Fig. 4.

Pacific hindcast ground truth stations.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 5.
Fig. 5.

Station 51028 wave systems for 1–15 Nov 2000. Wave component height, period, and direction are represented by vector length, origin, and azimuth, respectively. (a) NDBC buoy measurements. (b) WAVEWATCH III hindcast. (c) WAM hindcast. (d) WAVAD hindcast.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 6.
Fig. 6.

Monthly windsea height errors.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 7.
Fig. 7.

Monthly mature swell height errors.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 8.
Fig. 8.

Wave height QQ results from station 51028.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 9.
Fig. 9.

Vector history of most energetic wave system events during November 2000 at station 51028: (a) WAVEWATCH III hindcast and (b) NDBC station 51028 observations.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 10.
Fig. 10.

Station 51028 event A wave system analysis results for WAVEWATCH III; (a) wave height comparison, (b) wave period comparison, and (c) wave direction comparison.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 11.
Fig. 11.

Station 51028 event D wave system analysis results for WAVEWATCH III; (a) wave height comparison, (b) wave period comparison, (c) wave direction comparison.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 12.
Fig. 12.

WAVEWATCH III mean significant wave height bias for year 2000 as determined from TOPEX/Poseidon satellite altimetry.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Fig. 13.
Fig. 13.

WAVEWATCH III year 2000 mean significant wave height bias from TOPEX/Poseidon satellite altimetry with a capped drag coefficient (CD ≤ 2.5 × 10−3) in the WAVEWATCH wind input source term.

Citation: Journal of Atmospheric and Oceanic Technology 26, 8; 10.1175/2009JTECHO650.1

Table 1.

Wave model hindcast runs.

Table 1.
Table 2.

Observation stations. [Data Acquisition and Control Telemetry (DACT); General Service Buoy Payload (GSBP)]

Table 2.
Table 3.

Significant wave height performance summary (WAVEWATCH III: WW III).

Table 3.
Table 4.

Peak wave period performance summary.

Table 4.
Table 5.

Mean wave direction performance summary.

Table 5.
Save