Separation of Short Time Series of Currents into “Fluctuations,” “Tides,” and “Mean” Flow
1. Introduction
Standard current meter mooring data have been useful in the past for determining low-frequency “mean” circulation and the magnitude of various tidal constituents over the mooring record length. Typically, low-pass filtering has been employed to extract a mean, allowing for evolution of mean circulation to be determined despite the possible presence of high-energy tidal variability. Variability remaining in a mean thus determined depends upon the cutoff frequency chosen for the low-pass filter. Examination of tides typically relies on fitting tidal constituents to extended velocity records: the accuracy of the “tide” thus determined increases with the number of constituents determined, hence with the length of record.
The evolution of higher-frequency sampling enabled by the power (and to a lesser extent the bandwidth) provided by cabled ocean observatories presents the opportunity to examine all scales involved in relatively short time-scale circulation “events,” and hence to determine causal relationships among them. The power of such data has been demonstrated in the analysis of measurements from two coastal sites off the East Coast of the United States (Gargett and Wells 2007; Gargett and Grosch 2014; Gargett et al. 2014). The opportunity comes, however, with related needs for 1) a method of separating high-period variability from low-period variability in short (1–7 days) data records that does not incur the data loss and smoothing of abrupt transitions (i.e., the onset of storms) associated with standard filtering and 2) a determination of the local tide that is more accurate than that determined by fitting tidal harmonics over short records.
A successful method is presented for use of short data records, first separating “fluctuations” from evolving longer-period flows and then subsequently separating “tides” from “mean” velocities. The quotation marks around each category at this point acknowledge that the definitions of each are application dependent. For our studies of turbulence associated with wind/waves and/or heat fluxes, “fluctuations” are defined as having (apparent) periods less than ~1 h. In certain conditions, this may also include motions associated with internal waves (Gargett et al. 2014). For the study of the influence of tides on turbulence, the “tidal” portion must resolve the minimum semidiurnal tidal frequency (~12.5-h period), including potentially large spring–neap modulation within a week. The more slowly varying “mean” portion should reasonably represent variability due to time-dependent wind-driven and/or geostrophic flows. Dependent on latitude, inertial oscillations may inhabit any of these three categories. A method to separate three distinct frequency bands should not be expected to help differentiate overlapping frequency processes within any one of the three separable bands. At the midlatitudes of the data used here, the inertial period is roughly equal to that of the diurnal tide. Because the initial impetus for this work was to determine the effect of a dominant semidiurnal tide on Langmuir circulations, further separation of inertial oscillations from tidal signals was not carried out. Should such separation be required, rotary methods exist (spectral and empirical orthogonal function analysis), subject to the standard limitations imposed on spectral accuracy by record length (Gonella 1972; Denbo and Allen 1984). In what follows, the use of the terms fluctuation, tide, and mean without quotes implies the above-mentioned definitions.
2. Data and context
This work was motivated by data from two extended ocean observatory deployments of vertical acoustic Doppler current profilers (VADCPs). For the first of these deployments [at Long-Term Ecosystem Observatory at 15 m (LEO-15), an inner shelf cabled observatory off New Jersey; Gargett et al. 2004], data were grouped into time-continuous “sessions” of approximately 4-day duration because the VADCP accumulated unacceptable errors (for unknown reasons) beyond ~4 days, hence requiring resetting for continued quality acquisition. Data from a second deployment off the Georgia coast at U.S. Navy tower R2 in 27-m-depth water (Savidge et al. 2008) were structured into sessions with lengths of ~7 days. While the instrument deployed in Georgia did not require resetting every few days to maintain data quality, dividing the 1-Hz data into subsets of several days’ length was a precaution against major data loss from potential failure of a microwave link between tower and shore stations. From a physical perspective, 4–7 days approximates the frequency of the dominant meteorological forcing on the U.S. East Coast.
During strong downwelling wind events at both locations, dominant fluctuations were found to be the turbulent structures termed Langmuir supercells (LS) by Gargett et al. (2004). While previous work (Gargett and Wells 2007; Tejada-Martínez and Grosch 2007) has extracted the high-frequency signals characteristic of LS by removing a linear least squares fit over a time period of ~2 h from each bin, it is useful to have a method that extracts the fluctuations continuously over an entire session length. In addition, it is desirable to have time-local estimates of tidal velocities, since the magnitude and direction of tidal flows may be expected to influence LS (Kukulka et al. 2011) and other high-frequency phenomena.
The challenge of the separation task is illustrated in Fig. 1a, which shows a component of horizontal velocity U, estimated for a range bin located 11.2 m above bottom (mab), over a typical session length. The data shown have been subsampled from original 1-Hz data after prefiltering to remove surface wave velocities; the resulting subsampled data have a sample period SP = 10 s. In addition to quasi-periodic variations with a dominant semidiurnal period, note the large and abrupt change in the evolving mean velocity near midsession, an approximately −0.25 m s−1 shift in the background flow upon which the oscillating semidiurnal tide is superimposed. This shift was coincident with a southwestward wind event, accompanied by high waves and LS structures in the water column.
(a) Horizontal velocity component U (instrument coordinates) as a function of the sample number of data filtered to remove surface waves (sample period SP = 10 s). Data shown are from bin 10 (11.2 mab) over a typical ~7-day session length at R2, illustrating a large change in low-frequency (mean) flow around sample number 3 × 104. (b) Expansion of the section marked in (a), showing results of successive decimations. Cyan: original data. Blue curve/black curve/red dots: U subsampled by 8/64/512 (SP = 80/640/5120 s). The red curve (5120-s data successively interpolated to the initial SP = 10 s) provides the first estimate of low frequency content L; the difference between it and the original data is the first estimate of high frequency content F.
Citation: Journal of Atmospheric and Oceanic Technology 33, 5; 10.1175/JTECH-D-15-0232.1
3. Three-component separation of time-limited records
The new separation method developed and presented here uses two processing stages to separate a velocity time series U(ti), i = 1:N such as that in Fig. 1a into the desired three components. Stage 1 separates high-frequency fluctuations F from low-frequency variation L = T + M associated with combined tidal (T) and more slowly varying mean (M) flows. A second stage separates L into T and M.
For stage 1, we first explored empirical mode decomposition (EMD; Huang et al. 1998). However, EMD was found to exhibit large variation in the number of modes required to model the low-frequency portion of different sessions; hence, because we sought a method that would not require adjustment for each individual data record, this method was dismissed and a new method, termed multiple decimate and interpolate (MDI), was developed.
MDI makes use of the MATLAB “decimate” and “interp” commands. The built-in function decimate prefilters (using an eighth-order Chebyshev type I low-pass filter) and then subsamples data at 1/D, where D is a user-defined decimation factor. Although not documented in MATLAB, we find that to preserve phase in a series of length N, D must be chosen so that M = (N − 1)/D is an integer. The behavior of filters involved in both decimate and interp is optimized (in the sense of the usual compromise between cutoff steepness and ripple) if D < 10, with higher values of D obtainable by sequential decimation. With these requirements in mind, the original data series is successively decimated by 8 three times, resulting in a sampling period SPd = SP(8 × 8 × 8 = 512) = 5120 s = ~85 min that is sufficient to resolve the semidiurnal tidal period. Note that if successive decimations use different values of D, then the requirement of integer M must be met at all individual stages. Integer M is provided by end-padding the time series. Instead of zero padding, pad values are calculated to better represent values near the end of the time series, as a linear interpolation from the last available value U(N) to a mean calculated over approximately two semidiurnal periods at the end of the session. Padding values are removed at the end of processing. At each decimation stage, the low-pass prefilter applied by the decimate function allows the filtered result to be subsampled every eighth point without aliasing. The final result is a low-frequency decimated time series 
(a) Original time series U of Fig. 1a. (b) Final low frequency L and (c) final high frequency F signals resulting from stage 1 (MDI) processing. (d) Correction (corr) to the first low-frequency estimate [note scale difference from (b); for description, see text].
Citation: Journal of Atmospheric and Oceanic Technology 33, 5; 10.1175/JTECH-D-15-0232.1
It should be noted that users without the MATLAB commands employed here may duplicate the results of MDI using digital filters and interpolation routines of their own design. However achieved, the essential output of the first stage of processing must be an acceptable version of 
Subsequent separation of L will be a function of the processes understood to be operative in a particular region/time. Motivated by our interest in the effects of the semidiurnal tide on LS, the second stage of processing was designed to separate L into tide and mean. Existing methods were found to have various flaws for this application. Separation by standard filtering has disadvantages, including smoothing often-abrupt changes in background mean flows (cf. Fig. 1) and further shortening already short records, while the coincidence of the time scale of event-scale changes in the mean with that of the semidiurnal tide made separation by wavelet analysis (Flinchem and Jay 2000) impossible. As will be documented further below, standard harmonic analysis of short records produces significant errors in both tidal amplitude and phase. Consequently, we developed a new method, termed peak identification and interpolation (PII), as follows.
The decimated (SPd = 5120 s) low-frequency time series Ld computed as part of stage 1 (Fig. 2a) resolves tidal variation but is sufficiently smooth that the MATLAB subroutine “peakfinder” returns single points at maxima (black dots) and minima (open squares). The average values of successive extrema (black dots in Fig. 3b) would be zero for a perfectly symmetric tidal oscillation but are nonzero if, as here, there is also a mean flow. A cubic spline fit to these average values produces a smoothly varying mean (Md, heavy curve in Fig. 3b): M is produced by successive reinterpolation of 
(a) Low frequency decimated time series Ld (SPd = 5120 s) generated during stage 1, with identified maxima (black dots) and minima (white squares). (b) The (decimated) mean Md is a spline fit (heavy curve) to averages (white dots) of successive maxima and minima (black dots).
Citation: Journal of Atmospheric and Oceanic Technology 33, 5; 10.1175/JTECH-D-15-0232.1
The PII process at no time fits the low-frequency signal to specific tidal periods, nor is it claimed that identification of minima and maxima within short records constitutes a rigorous representation of tidal variability for the study site. Instead the primary goals are to 1) identify the timing and magnitude of maximum and minimum tidal velocities, in order to 2) estimate the more slowly varying event-driven mean as the average of successive maxima and minima. Secondary extrema caused by the remaining variation on periods between that of the semidiurnal tide and those (<~1 h) associated with the definition of fluctuation can affect this estimation. Although secondary extrema were extremely infrequent with the first stage processing described here, they were not totally absent. Thus, deformation of the mean estimate that would result from the presence of secondary extrema is avoided by rejecting successive extrema with an amplitude difference smaller than a chosen limit, here taken as 0.04 m s−1 (~20% of observed tidal amplitude). This difference, set in the “peakfinder” routine, is the only free variable in the PII process, and it should be chosen by examination in locations with different tidal amplitudes. Alternately, secondary extrema can be eliminated with a temporal requirement (requiring a reprogrammed alternate version of peakfinder) that acceptable successive peaks are separated in time longer than some chosen fraction of the local dominant tidal period. Results from use of temporal limits are essentially identical to those, shown in Fig. 3, that use amplitude limits.
The PII process described above requires at least three tidal extrema within the record and hence will be applicable to any record longer than about one and a half periods of the dominant tide, here semidiurnal.
PII provides a smooth mean component that, removed from the observed tide plus mean time series L, produces a tide component with maxima and minima at the times observed. The more traditional method of accomplishing tide/mean separation (such as T_Tide; Pawlowicz et al. 2002) instead uses harmonic analysis of L to produce a predicted tide Tp and then calculates a mean as Mp = L − Tp. If Tp is a faithful representation of the tide, then it follows that there will be no residual variability at tidal frequencies in Mp. However, with short temporal records, phase and amplitude errors of each estimated tidal constituent are large relative to their magnitudes: extracting such tidal fits from the measured records propagates those errors to the residual that defines the mean. For the period shown in Fig. 1, a mean calculated from L as M11 = L − T11, where T11 is a tide estimated from an 11-day record, clearly retains significant variability at the semidiurnal tidal period (Fig. 4a). While the deployments at both LEO-15 and R2 occasionally extended over a number of consecutive several-day segments, allowing tides to be fit over more extended time-continuous records, means computed using tidal fits to even these relatively longer records (e.g., see M37, the mean using 37-day period fits shown in Fig. 4b) still retain detectable amounts of tidal variability. In all cases, errors result from failures of the harmonic method, when used with short record lengths, to match both amplitude and phase. As might be expected, the error in the estimated tide depends on how many constituents are included. Using T_Tide’s estimate of signal-to-noise ratio as a threshold to determine which constituents to use, two different thresholds show significantly different results (Fig. 4). Determination of an appropriate threshold is subjective, depending on record length, time of year, and location. It has been our experience that successful removal of tidal variation by extracting a harmonically fitted tide is rare.
Results for mean component M as calculated from the low frequency signal L (thin line) by MDI–PII (red) and T_Tide (green: using constituents fit with signal to noise ratio > 6, blue: using constituents fit with signal-to-noise ratio > 2), displayed over the time period shown in Figs. 1–3, during which the water column was predominantly unstratified. T_Tide computations (a) M11 using an 11-day record spanning the period shown and (b) M37 using a record length extended to 37 days.
Citation: Journal of Atmospheric and Oceanic Technology 33, 5; 10.1175/JTECH-D-15-0232.1
Figure 5 illustrates the result of a similar comparison for data taken during a period of summer stratification. A diurnal signal, evident in the PII result but hidden by remnant semidiurnal contamination in the T_Tide fits, may be either a diurnal tide or wind-forced near-diurnal inertial motion. If desired, this diurnal signal could also be removed from a mean by iteration of the PII process.
Results for mean component M as calculated from the low frequency signal L (thin line) by MDI–PII (red) and M10 as calculated by T_Tide (green: using constituents fit with signal to noise ratio > 6, blue: using constituents fit with signal-to-noise ratio > 2) using a 10-day record spanning the period shown, during a period of water column stratification. Note the diurnal signal evident in the PII result but hidden by remnant semidiurnal contamination in the T_Tide fits.
Citation: Journal of Atmospheric and Oceanic Technology 33, 5; 10.1175/JTECH-D-15-0232.1
To summarize, the advantages of PII are that 1) tides can be removed from a mean without the error inherent in tidal fits to short datasets, 2) and without the data loss at the beginning and ends of the time series that typical 40-h low-pass filtering would entail, and 3) the exact timing of the local tidal extrema can be identified—something that is not possible with the implicit assumption of stationarity when applying harmonic analysis. Finally, 4) the separated slowly varying mean is not smoothed in time as it would be with low-pass filtering, permitting the evolution of fluctuations to be examined relative to the sometimes rapid onset or cessation of events represented in nontidal forcing.
Success of the tide/mean separation carried out by our method is evidenced by comparing a hodograph of total velocities 
Hodographs of (a) total low frequency (tide plus mean) velocities 
Citation: Journal of Atmospheric and Oceanic Technology 33, 5; 10.1175/JTECH-D-15-0232.1
Carrying out the PII separation on all available bins yields the tide fields 
Horizontal velocity estimates for session 17: (a),(c) 
Citation: Journal of Atmospheric and Oceanic Technology 33, 5; 10.1175/JTECH-D-15-0232.1
Although there remains some distortion of a “pure” tide by variability at time scales between semidiurnal and that of the fluctuations removed, the overall vertical structure of the extracted tidal components seen in Fig. 7 is, as expected based on much longer records, largely barotropic although with occasional phase lag with height above bottom. The variation from spring to neap tides over the course of the session is clear, as is the instantaneous phase of the tide, necessary for any investigation of effects associated with tidal modulation/interference.
4. Conclusions
This paper addresses the problem of separating short records of coastal ocean velocities into fluctuations [here defined as short-period (<~1 h) motions associated with turbulence and/or internal waves], tides, and mean flows, the latter often large in magnitude and varying substantially, often relatively abruptly, in time during the record period. Standard methods having been examined and found wanting, a two-stage process (MDI–PII) was devised that first separates fluctuations from tides plus mean (MDI) and then subsequently separates tides from mean (PII). We have demonstrated the success of this process using data characterized by both tides and large changes in mean velocities. The PII procedure for separation of tide from mean requires at least three tidal extrema; hence, the method will be applicable to records longer than approximately one and a half periods of the dominant tide.
Acknowledgments
Data analyzed here were obtained as part of the Coastal Benthic Exchange Dynamics Experiment funded by the National Science Foundation (0926852).
REFERENCES
Denbo, D. W., and Allen J. S. , 1984: Rotary empirical orthogonal function analysis of currents near the Oregon coast. J. Phys. Oceanogr., 14, 35–46, doi:10.1175/1520-0485(1984)014<0035:REOFAO>2.0.CO;2.
Flinchem, E. P., and Jay D. A. , 2000: An introduction to wavelet transform tidal analysis methods. Estuarine Coastal Shelf Sci., 51, 177–200, doi:10.1006/ecss.2000.0586.
Foreman, M. G. G., 1977: Manual for tidal heights analysis and prediction. Institute of Ocean Sciences, Pacific Marine Science Rep. 77-10, 97 pp.
Gargett, A. E., and Wells J. R. , 2007: Langmuir turbulence in shallow water. Part I. Observations. J. Fluid Mech., 576, 27–61, doi:10.1017/S0022112006004575.
Gargett, A. E., and Grosch C. E. , 2014: Turbulence process domination under the combined forcings of wind stress, the Langmuir vortex force, and surface cooling. J. Phys. Oceanogr., 44, 44–67, doi:10.1175/JPO-D-13-021.1.
Gargett, A. E., Wells J. R. , Tejada-Martínez A. E. , and Grosch C. E. , 2004: Langmuir supercells: A mechanism for sediment resuspension and transport in shallow seas. Science, 306, 1925–1928, doi:10.1126/science.1100849.
Gargett, A. E., Savidge D. K. , and Wells J. R. , 2014: Anatomy of a Langmuir supercell event. J. Mar. Res., 72, 127–163, doi:10.1357/002224014814901976.
Gonella, J., 1972: Rotary-component method for analyzing meteorological and oceanographic time series. Deep-Sea Res. Oceanogr. Abstr., 19, 833–846, doi:10.1016/0011-7471(72)90002-2.
Huang, N. E., and Coauthors, 1998: The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc. Roy. Soc. London, 454A, 903–995, doi:10.1098/rspa.1998.0193.
Kukulka, T., Plueddemann A. J. , Trowbridge J. H. , and Sullivan P. P. , 2011: The influence of crosswind tidal currents on Langmuir circulation in a shallow ocean. J. Geophys. Res., 116, C08005, doi:10.1029/2011JC006971.
Pawlowicz, R., Beardsley B. , and Lentz S. , 2002: Classical tidal harmonic analysis including error estimates in MATLAB using T_TIDE. Comput. Geosci., 28, 929–937, doi:10.1016/S0098-3004(02)00013-4.
Savidge, D. K., Amft J. A. , Gargett A. E. , Archer M. , Conley D. , Voulgaris G. , Wyatt L. , and Gurgel K.-W. , 2011: Assessment of WERA long-range HF-radar performance from the user’s perspective. 2011 IEEE/OES/CWTM Tenth Working Conference on Current, Waves and Turbulence Measurements (CWTM), IEEE, 31–38, doi:10.1109/CWTM.2011.5759520.
Savidge, W. B., Gargett A. E. , Jahnke R. A. , Nelson J. R. , Savidge D. K. , Short R. T. , and Voulgaris G. , 2008: Forcing and dynamics of seafloor-water column exchange on a broad continental shelf. Oceanography, 21 (4), 179–184, doi:10.5670/oceanog.2008.16.
Tejada-Martínez, A. E., and Grosch C. E. , 2007: Langmuir turbulence in shallow water. Part 2. Large-eddy simulation. J. Fluid Mech., 576, 63–108, doi:10.1017/S0022112006004587.
