a. Computed quantities

We group all output quantities into four categories:

1. Station metadata are anything that is specific to the sensor (and is not directly related to waves or the sea state). This includes both metadata describing the raw data source (to ensure reproducibility; more in section 2d) and the conditions in which it was recorded (latitude/longitude and water depth).

2. Wave-specific parameters are all quantities that describe a single wave, such as wave height or maximum slope. A typical study using FOWD aims to determine how a wave-specific parameter depends on one or several sea state parameters.

3. Aggregated sea state parameters describe the circumstances in which each wave occurred; that is, they relate to the past sea state of each wave. They are computed from the immediate 10- and 30-min history prior to (but not including) the current wave (see also section 2b for more on this running-window approach). Quantities are computed using only the raw sea surface elevation as input (either directly or by computing a spectrum first).

4. Directional sea state parameters: Some sensors (like the CDIP buoys) might include additional directional information that is not computable from the raw surface elevation time series. When such directional information (in form of a directional spectrum) is given, FOWD computes some directional parameters from it and includes them in the output. Note that this does not use the same running-window approach as the aggregated sea state parameters. Instead, each wave is mapped to the nearest (in time) available directional measurement. I.e., directional information usually includes some information relating to the future of the wave. But since directional information is robust to the influence of individual extreme events, we do not consider this a problem.

A complete overview of all computed quantities is shown in Table A1 in the appendix. Here, we outline some important quantities (as suggested in literature) and how they are estimated from the observed time series.

5) Crest–trough correlation

Tayfun (1990) suggests another key parameter to describe wave height distributions, the correlation coefficient r between squared crest height $A_0^2$ and squared trough depth $A_1^2$, which we refer to as “crest–trough correlation.” This parameter r is closely related to spectral bandwidth (as, for narrowband seas, crests and troughs are approximately of the same size, becoming increasingly chaotic/uncorrelated as more harmonics are added). By extension, it is also a measure for the tendency of the sea state to form wave groups (Fig. 1).

View Full Size

The crest–trough correlation r is higher in “groupy,” low-bandwidth sea states. Shown are surface elevations generated from Ochi–Hubble spectra (Ochi and Hubble 1976) with increasing spectral bandwidth (from top to bottom) and the corresponding value of r.

Citation: Journal of Atmospheric and Oceanic Technology 38, 7; 10.1175/JTECH-D-20-0185.1

• Download Figure
• Download figure as PowerPoint slide

The estimation of crest–trough correlation from the spectral density $\mathcal{S}$ is further elaborated in Tayfun and Fedele (2007). Following these lines, we compute r via

$r={\displaystyle \frac{1}{m_0}}\sqrt{{\rho }^2+{\lambda }^2},$(15)

with

$\rho ={{\displaystyle \int }}_0^{\infty }\mathcal{S}\left(\omega \right)\hspace{0.17em}\cos \left(\omega {\displaystyle \frac{\overline{T}}{2}}\right)\hspace{0.17em}d\omega \text{and}$(16)

$\lambda ={{\displaystyle \int }}_0^{\infty }\mathcal{S}\left(\omega \right)\hspace{0.17em}\sin \left(\omega {\displaystyle \frac{\overline{T}}{2}}\right)\hspace{0.17em}d\omega ,$(17)

where $\overline{T}=m_0/m_1$ is the spectral mean period and ω = 2πf is the angular frequency.

6) Spectral partitioning

To characterize processes that act mostly on short or long waves, spectral energy content is often more indicative than quantities based on the whole spectrum (such as mean period). Therefore, FOWD includes the relative energy content $Ɛ$ over several spectral bands, computed as a definite integral over the spectral density $\mathcal{S}$:

${Ɛ}_i={\displaystyle \frac{{{\displaystyle \int }}_{f_i}\mathcal{S}\left(f\right)\hspace{0.17em}df}{{{\displaystyle \int }}_0^{\infty }\mathcal{S}\left(f\right)\hspace{0.17em}df}}={\displaystyle \frac{1}{m_0}}{\displaystyle {\int }_{f_i}\mathcal{S}\left(f\right)\hspace{0.17em}df}.$(18)

We use five distinct spectral bands (with limits f_i), each characteristic for a different physical regime (Table 1). [This is a crude way to perform spectral partitioning as compared with more-sophisticated approaches that take directionality into account (Portilla-Yandún et al. 2016; Portilla-Yandún 2018). However, this simple integral is straightforward to compute and interpret, and can be estimated using only a surface displacement time series].

Table 1.

Frequency bands used by FOWD and their approximate corresponding physical regime [as, e.g., given in Holthuijsen (2010)]. Here, and elsewhere ID is identifier.

View Table

Similarly to the relative energy content, we also compute the total energy density contained in each frequency band (in joules per meter squared):

$P_i=\rho g{\displaystyle {\int }_{f_i}\mathcal{S}\left(f\right)\hspace{0.17em}df},$(19)

with approximate density of seawater ρ = 1024 kg m⁻³ and gravitational acceleration g = 9.81 m s⁻².

b. Running-window processing

Usually, studies that investigate extreme wave observations divide all data into blocks of equal length in time, e.g., 30-min chunks, that are then analyzed separately (e.g., Casas-Prat and Holthuijsen 2010; Cattrell et al. 2018). However, the transient nature of the ocean has long been identified as a potential source for systematic error (Adcock and Taylor 2014; Gemmrich and Garrett 2011; Gemmrich et al. 2016), as it is not clear that the wave height distribution is constant within each chunk.

A related consideration is that the estimated quantities must be agnostic of the future—that is, look-aheads must be impossible. This property is critical for machine-learning applications, where future state leaking into the training data may completely invalidate the generalization abilities of a machine-learning algorithm.

We have therefore decided to use a running-window approach in FOWD. Here, we iterate through the raw data one zero-upcrossing at a time, computing the characteristic sea state parameters based on the immediate history of every wave. This implies that there is no time gap between the end of the aggregation period and the current wave, at the expense of additional computation time (since the sea state has to be recomputed for every wave).

Picking a window length is always a trade-off between bias (longer windows are more prone to nonstationarity) and variance (shorter windows leave us with less data with which to work). Therefore, all parameters are computed three times:

• The parameters are calculated twice using fixed 30- and 10-min windows. This makes it possible to investigate the stationarity of the current sea state by comparing the values obtained from each window length.

• The parameters are calculated one more time using a variable, data-dependent window as suggested in Boccotti (2000) and used in Fedele et al. (2019). We define the optimal window size n to be the one that minimizes

  $\text{std}\left({\displaystyle \frac{{\sigma }_{n,i+1}}{{\sigma }_{n,i}}}-1\right),$(24)

  where σ_n,i is the standard deviation of the sea surface elevation in the ith chunk with length n, applied to the past 12 h of time series.

  • To make this process more robust, we recompute (24) 10 times for each candidate window with a different time offset. FOWD tries a total of 11 different windows lengths between 10 and 60 min and selects the one that minimizes the sum of (24) across all trials. This process tends to generate time windows longer than 40 min in most conditions but is also capable of reducing the window size if needed (Fig. 2).

  • Because the standard deviation of the sea surface elevation σ is directly related to significant wave height, we expect this to yield near-optimal window sizes for significant wave height and other slowly drifting quantities (such as mean period and energy content), but suboptimal results for faster drifting parameters (such as steepness, peak period, and kurtosis).

View Full Size

Most dynamic windows are longer than 30 min. Shown is a histogram of the determined optimal window size across all Hawaiian CDIP stations.

Citation: Journal of Atmospheric and Oceanic Technology 38, 7; 10.1175/JTECH-D-20-0185.1

• Download Figure
• Download figure as PowerPoint slide

c. Quality control

FOWD uses a combination of QC flags, most of which are inspired by the process suggested in Christou and Ewans (2014). A measurement is discarded if any of the following conditions are met when applied to the past 30-min surface elevation:

1. There are any waves with zero-crossing period >25 s.

2. The rate of change of the surface elevation η exceeds the limit rate of change by a factor of 2 or more at any point; that is,

   $\left|{\displaystyle \frac{\partial \eta }{\partial t}}\right|>2U_{\lim }.$(25)

   The limit rate of change U_lim is defined as

   $U_{\lim }=2\pi {\displaystyle \frac{\text{std}\left(\eta \right)}{\langle T_{d,0}\rangle }}\sqrt{2\hspace{0.17em}\ln N},$(26)

   with standard deviation std, mean observed zero-crossing periods ⟨T_d,0⟩, and number of waves in the record N. This criterion removes records containing waves that are much steeper than the average rate of change std(η)/⟨T_d,0⟩—that is, records with single, very steep waves—but leaves sea states with many steep waves intact.

3. There are 10 consecutive data points of the same value.

4. There is any absolute crest or trough elevation that is greater than 8 times the normalized median absolute deviation (MADN) of the surface elevation; that is,

   $\left|h\right|>8\kappa \hspace{0.17em}\text{median}\left[\left|\eta -\text{median}\left(\eta \right)\right|\right],$(27)

   with κ = 1.483, which ensures that MADN converges to standard deviation for Gaussian distributed η with growing sample size (see, e.g., Huber and Ronchetti 2009). This criterion permits crest heights and trough depths of up to about 2 times the significant wave height, which should be more than enough for any real signal. [In a linear sea, a crest exceeding 2H_S would have a probability of exp(−32) ≈ 10⁻¹⁴].

5. Surface elevations are not equally spaced in time (but they may contain “NaN” values).

6. The ratio of missing (NaN) data to valid data exceeds 5%.

7. There are less than 100 individual zero crossings.

All waves that fail QC and are larger than 2 times the significant wave height are written to a log file to allow for manual inspection. In addition, all waves that are larger than 2.5 times the significant wave height are written to the log file, regardless of whether they pass QC. This enables us to evaluate the QC process and tweak thresholds or exclude faulty subdatasets as needed. A brief evaluation of this QC process when applied to the CDIP data is given in section 4b.

d. Additional metadata and reproducibility

All FOWD output files are self-documenting in the sense that they include all relevant metadata as netCDF4 attributes, both for each variable and the dataset as a whole. Apart from the static metadata documenting the coordinates and parameters (which is the same for every FOWD output file), we also include some metadata related to the processing environment and raw data source to ensure reproducibility. Specifically, each wave record includes the time stamp, file name, and a unique file identifier (UUID) of the raw source file from which it came (see Table A1). The output files also include the exact version of the FOWD processing implementation used to create the file in form of a “git” tag, along with a UUID. That way, we enable users to reproduce any result by allowing them to use the exact same processing version and input file.

a. Memory efficiency

Because of FOWD’s running-window approach (see section 2b), FOWD output datasets are about 10 times as big as the input surface elevation time series (since every wave results in about 80 output features). This demands that the processing implementation does not store entire output files in memory.

We achieve this by keeping only the immediate 30-min history of the current processing time in memory. Each new record is flushed to disk using Python’s “pickle” format. After the processing has finished, these pickle files are read back by the main process in chunks, reformatted to the netCDF4 output format, and flushed to disk again. This ensures that the main process uses only a negligible amount of memory while each worker process only keeps the input data in memory. In other words, if the input data fit in memory, processing will succeed.

b. Testing strategy

In software engineering, automated tests are an invaluable tool to ensure proper functionality of a product. Unfortunately, writing automated tests for processing workflows of physical data is often impossible or infeasible because of the lack of ground-truth answers with which to compare. On the other hand, faulty results are often easy to detect for humans when they fall outside of reasonable physical limits or show the wrong scaling behavior. We have therefore opted for semiautomated sanity checks instead of fully automated unit tests for the core processing.

Each sanity check test case generates a random surface elevation time series from a different ground-truth wave spectrum and runs it through the FOWD processing. Here, only the spectral shape is prescribed externally, surface elevations are drawn as harmonics with random phases from the spectrum. The resulting output parameters can then be inspected manually.

Indeed, all of these expectations are met for this particular test case (Fig. 3). Other sanity checks feature idealized spectra, for example, containing just a single harmonic, that allow us to validate parameters that are more difficult to interpret like crest–trough correlation, or idealized directional spectra. Because of these sanity checks, we are confident that the FOWD core processing produces meaningful results.

View Full Size

Sanity check test cases allow us to verify manually that computed parameters are reasonable. Shown are test (left) inputs and (right) outputs for (top) high-frequency and (bottom) low-frequency seas. Estimated sea state parameters are defined in Table A1. Spectral parameters are input parameters of the Ochi–Hubble spectrum used to generate each test case (as shown in upper-left panels).

Citation: Journal of Atmospheric and Oceanic Technology 38, 7; 10.1175/JTECH-D-20-0185.1

• Download Figure
• Download figure as PowerPoint slide

a. Input data and processing

In total, the CDIP catalog spans about 750 years of continuous surface elevation measurements (almost all at sampling rates of 1.28 Hz) and is available in netCDF4 format through a THREDDS server. This amounts to about 270 GByte of raw data.

While CDIP data files also include horizontal displacements and a number of derived quantities (like significant wave height, peak period, and others), we use only the raw vertical surface displacement, station metadata, and directional quantities for processing. This ensures that FOWD is applicable to any instrument that delivers a surface displacement time series (including radar or laser sensors).

We applied only minimal preprocessing to the data, which consists of removing all data that have an error flag set and subtracting the 30-min running mean from the raw vertical surface elevation. After that, we processed all data in about 72 h on 10 cluster nodes in parallel (using the FOWD reference implementation described in section 3). The resulting output dataset has a total (compressed) size of 1.1 TB. We create one output file per CDIP station, with individual file sizes ranging between 1.7 MByte and 38 GByte.

In total, FOWD contains about 4.2 billion individual waves and sea states. An interactive map indicating all data locations and some key statistics is available in the online supplemental material.

b. Quality control and filtering

As outlined in section 2c, FOWD automatically logs waves failing QC that are higher than 2 significant wave heights, and all waves higher than 2.5 significant wave heights (whether they pass QC or not). This allows us to assemble some higher-order statistics to get an idea of how prevalent quality issues are in the CDIP data and to verify that FOWD’s QC system works as intended.

In total, just under 80 000 waves fail QC (Table 2). About 80% of these QC failures occur in only 5 CDIP locations (of 161). This suggests that relatively few deployments with general quality problems cause a majority of QC failures.

Table 2.

The number of times each QC flag was triggered for the whole CDIP catalog. See section 2c for a definition of flags a–g. Note that multiple flags can be active for the same wave.

View Table

To investigate this further and isolate faulty deployments, the FOWD implementation includes a postprocessing command that produces plots of all records in the QC logs. These plots show the raw surface elevation of the failing wave and its immediate 30-min history.

After inspecting each of these plots, we decided to blacklist 38 deployments and 4 entire CDIP stations that showed obvious quality problems like frequent spikes, extreme oscillations, unphysical values, or jumps (Table 3). On top of excluding these blacklisted CDIP deployments, we also removed all records in conditions in which buoys are known to be unreliable [similar to McAllister and van den Bremer (2020)]:

1. records with 30-min significant wave height smaller than 1 m,

2. records with spectral mean frequency higher than 1/3.2 of the Nyquist frequency (for 1.28-Hz data, this is equivalent to filtering all records with a mean wave period below 5 s), and

3. records where the relative energy content of frequency band 1 exceeds 10% (extensive low-frequency drift).

Table 3.

Blacklisted CDIP deployments that failed visual inspection.

View Table

Since FOWD is also intended for use by non–wave experts, it is essential to provide access to a precleaned dataset. Therefore, the filtered FOWD–CDIP dataset is available for download along with the unfiltered one (see the data availability statement).

c. Impact of running-window processing

After processing the CDIP data, we can now investigate how large of a difference FOWD’s running-window processing (as described in section 2b) makes in practice, relative to the usual fixed-window approach.

To this end, we divide the FOWD catalog for one particular CDIP station (with ID 188p1, containing about 30 million waves) into 30-min chunks. The last measurement in each of these chunks (concerning the past 30-min sea state) then represents what would have been obtained for all waves if FOWD did not use running windows.

We can then quantify the influence of the running-window approach by computing the root-mean-square (RMS) difference between this last measurement of every chunk and all other data points in it. To make it easier to compare the different parameters, we divide each by a characteristic scale to obtain a normalized RMS (Table 4).

Table 4.

Characteristic scale used to normalize root-mean-square residual for each parameter (Fig. 4).

View Table

The resulting distribution of the normalized RMS in each chunk shows that, while deviations are typically below 10% of the characteristic scale, they can reach up to 50% in extreme cases (Fig. 4). As expected, some parameters (such as kurtosis and maximum wave height) are much more prone to drift than others (such as significant wave height and spectral energy). However, this result is sensitive to which characteristic scale we choose, so comparisons between parameters remain qualitative.

View Full Size

In extreme cases, using running windows (instead of fixed chunks) leads to RMS differences of up to 50% of the characteristic scale of a parameter (Table 4). Shown is the distribution of normalized RMS difference between processing based on running windows and fixed chunks for some parameters.

Citation: Journal of Atmospheric and Oceanic Technology 38, 7; 10.1175/JTECH-D-20-0185.1

• Download Figure
• Download figure as PowerPoint slide

A particularly important quantity in this context is the significant wave height. If the significant wave height is underestimated with an error of only 5%, a wave with true abnormality index AI = 2 is estimated as a wave with AI = 2.1, which is less than one-half as likely to occur (assuming Rayleigh-distributed waves).

We conclude that the running-window approach can lead to significantly different results, apart from the more important effect of preventing look-aheads (as discussed in section 2b). In other words, explicitly accounting for a drifting sea state provides an opportunity to reduce bias by a nontrivial amount—although we did not measure how much this approach influences final results or conclusions.

d. Shortcomings of buoy data

Although any dataset that provides surface elevation measurements can be processed into a FOWD dataset, buoy measurements remain a dominant data source due to their relatively large availability (at least in comparison with radar and laser measurements). Therefore, this section discusses some of the known problems with buoy data, and how they carry over to FOWD and its possible applications.

First and foremost, buoys tend to linearize surface elevations to some degree [see McAllister and van den Bremer (2020, 2019) for a discussion]. This is especially problematic in rough seas with high steepness, because buoys can be dragged through a steep crest or move laterally around it and underestimate the true wave height. Combined with the inherent sampling variability of a point measurement (the two-dimensional wave has to hit the buoy at the crest to be registered at full height; see Benetazzo et al. 2015), wave estimates based on buoy data tend to be too conservative (see also Casas-Prat and Holthuijsen 2010).

This is inconvenient for studies with the goal to estimate absolute rogue wave risk, since one needs to take additional steps to correct for these biases, include other data sources, or accept that the results represent a lower bound for rogue wave risk. However, this is not a problem when estimating the relative importance of sea state risk factors, as buoys should be similarly inaccurate across a wide range of different sea states (after the most problematic conditions are filtered; see section 4b—perhaps with the exception of very steep seas). We therefore see no problem with using buoy data for the type of study presented in section 5.

Another issue to keep in mind is selection bias. Buoys tend to be placed in locations that are easy to reach and of special interest for humans. This implies that coastal areas are overrepresented, and therefore results derived from the whole dataset will be less representative for open-ocean conditions.

No reasonable amount of one-dimensional time series data can tell us about truly exceptional events. In offshore engineering contexts, an important quantity is the “10 000 year wave,” which is the largest expected wave in a 10 000 yr period. Events of this rarity cannot be estimated with this dataset without additional work (such as further theoretical assumptions, or data augmentation via simulations).

a. Confounding and roguish sea states

To get a feeling for the data, we investigate correlations between some of the sea state parameters and have a look at the probability density functions of sea states in which we find rogues with AI > 2 and AI > 2.4.

The correlation matrix of the sea state parameters (Fig. 5) provides yet another important sanity check for FOWD, since many parameters are correlated by definition (such as BFI, which is computed based on steepness and spectral bandwidth). Furthermore, it serves as an important reminder that there are many nonobvious correlations, such as the one between spectral bandwidth and mean period. Any conclusion we draw about the influence of a parameter on rogue wave activity thus has to take possible confounders into account.

View Full Size

Linear (Pearson) correlation matrix of selected parameters. Almost all parameters are strongly correlated with at least one other parameter, but exceptions exist (e.g., skewness, kurtosis/maximum wave height, and wind sea directional spread).

Citation: Journal of Atmospheric and Oceanic Technology 38, 7; 10.1175/JTECH-D-20-0185.1

• Download Figure
• Download figure as PowerPoint slide

The probability density functions of roguish seas (Fig. 6) indicate several potential controlling parameters for rogue wave occurrence, where the distribution of seas containing a rogue wave differs substantially from that of all waves (with, e.g., skewness, spectral bandwidth, and maximum wave height being promising candidates). This analysis, while intuitively approachable, yields little quantitative insight into the relative importance of each parameter, and it neglects the influence of sample size effects. The following section addresses this through a simple analytical Bayesian parameter estimation.

View Full Size

Most parameters show a clear difference between the probability distributions of all sea states and those containing an extreme wave, but some just show a weak dependence (e.g., directional spread, significant wave height, and steepness). Shown are the probability density functions (PDFs) of various sea state parameters, estimated via histograms. Each parameter includes PDFs for the sea states of all waves, waves with AI > 2, and waves with AI > 2.4.

Citation: Journal of Atmospheric and Oceanic Technology 38, 7; 10.1175/JTECH-D-20-0185.1

• Download Figure
• Download figure as PowerPoint slide

b. Estimation of rogue wave probabilities with uncertainties

This posterior is simple to evaluate analytically. In particular, we can use widely available library functions to compute the minimum credible interval (highest posterior credible interval) for p. This gives us the possibility to quantify our uncertainty in p based on the number of available samples, expressed as, for example, the 95% credible interval.

To finally investigate the influence of the sea state on the rogue wave probability p, we split each sea state parameter into 15 equally sized bins. We assume that, within each bin, p is independently and identically distributed (iid) with a distribution according to (33), and we evaluate the mean and credible interval of p independently for each bin. We also exclude bins that contain less than 10 rogue wave events (i.e., where n⁺ < 10) to eliminate overly uncertain estimates. As a result, we can study how p behaves as a function of each sea state parameter and quantify our uncertainty based on how much data we have in each regime.

We stress that this uncertainty is based on the assumption that p is iid. Beta distributed within each bin, which is clearly not the case if we acknowledge that p depends on more than one parameter. Therefore, these uncertainties can only serve as an indicator whether or not there are enough data to make a statement about this marginalized version of the true, multivariate distribution of p. In other words, they indicate how confident we can be in the best estimate of p for this dataset if we can only measure one parameter at a time.

The results of this process show a clear, highly significant dependence of the rogue wave probability on some sea state parameters, and the lack of such a dependence on others (Fig. 7). In particular, we find the following:

1. Surface elevation kurtosis, relative maximum wave height, and skewness are the strongest predictors for rogue wave risk. For relative maximum wave height, P(AI > 2) ranges between 2.9 × 10⁻⁵ and 1.0 × 10⁻³. So if an up-to-date, in situ surface elevation time series is available, these parameters are able to quantify rogue wave risk with a factor of about 35 in variation.

2. Crest–trough correlation and spectral bandwidth (peakedness) are the strongest spectral predictors, with P(AI > 2) varying between 2.4 × 10⁻⁵ and 1.4 × 10⁻⁴ for crest–trough correlation—that is, almost one order of magnitude in variation from the spectrum alone.

3. The Tayfun wave height distribution (Tayfun 1990; Tayfun and Fedele 2007) seems to be an excellent baseline for rogue wave activity.

4. There is, at this level of detail, only a minor dependency of rogue wave occurrence on directional spread, Benjamin–Feir index, significant wave height, and steepness.

View Full Size

Some sea state parameters are much more informative for rogue wave activity than others. Shown is the dependence of the rogue wave probability on several sea state parameters for AI > 2 and AI > 2.4. Symbols represent rogue wave probability posterior mean; shading represents the 95% minimum credible interval. Dashed lines indicate the values predicted by the Tayfun wave height distribution (Tayfun and Fedele 2007).

Citation: Journal of Atmospheric and Oceanic Technology 38, 7; 10.1175/JTECH-D-20-0185.1

• Download Figure
• Download figure as PowerPoint slide

So, in this first analysis, it seems that bandwidth effects are the dominant modifier of rogue wave risk, whereas nonlinear effects (at least those governed by steepness and BFI) seem to play a minor corrective role in comparison with that. However, it is important to keep in mind that we are only looking at one set of stations and only one sea state parameter at a time.