• Beaton, A. E., , and Tukey J. W. , 1974: The fitting of power series, meaning polynomials, illustrated on band-spectroscopic data. Technometrics, 16, 147185, doi:10.1080/00401706.1974.10489171.

    • Search Google Scholar
    • Export Citation
  • Brock, F. V., 1986: A nonlinear filter to remove impulse noise from meteorological data. J. Atmos. Oceanic Technol., 3, 5158, doi:10.1175/1520-0426(1986)003<0051:ANFTRI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Daubechies, I., 1992: Ten Lectures on Wavelets. CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 61, Society for Industrial and Applied Mathematics, 350 pp.

  • Donoho, D. L., , and Johnstone I. M. , 1994: Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81, 425455, doi:10.1093/biomet/81.3.425.

    • Search Google Scholar
    • Export Citation
  • Felber, R., , Münger A. , , Neftel A. , , and Ammann C. , 2015: Eddy covariance methane flux measurements over a grazed pasture: Effect of cows as moving point sources. Biogeosciences, 12, 39253940, doi:10.5194/bg-12-3925-2015.

    • Search Google Scholar
    • Export Citation
  • Fochesatto, G. J., , and Sloan J. , 2008: Signal processing of multicomponent Raman spectra of particulate matter. Int. J. High Speed Electron. Syst., 18, 277294, doi:10.1142/S0129156408005345.

    • Search Google Scholar
    • Export Citation
  • Foken, T., , and Wichura B. , 1996: Tools for quality assessment of surface-based flux measurements. Agric. For. Meteor., 78, 83105, doi:10.1016/0168-1923(95)02248-1.

    • Search Google Scholar
    • Export Citation
  • Foken, T., and Coauthors, 2010: Energy balance closure for the LITFASS-2003 experiment. Theor. Appl. Climatol., 101, 149160, doi:10.1007/s00704-009-0216-8.

    • Search Google Scholar
    • Export Citation
  • Fratini, G., , and Mauder M. , 2014: Towards a consistent eddy-covariance processing: An intercomparison of EddyPro and TK3. Atmos. Meas. Tech., 7, 22732281, doi:10.5194/amt-7-2273-2014.

    • Search Google Scholar
    • Export Citation
  • Gallagher, N. C., , and Wise G. L. , 1981: A theoretical analysis of the properties of median filters. IEEE Trans. Acoust. Speech Signal Process., 29, 11361141, doi:10.1109/TASSP.1981.1163708.

    • Search Google Scholar
    • Export Citation
  • Gao, W., , Shaw R. H. , , and Paw U K. T. , 1989: Observation of organized structure in turbulent flow within and above a forest canopy. Bound.-Layer Meteor., 47, 349377, doi:10.1007/BF00122339.

    • Search Google Scholar
    • Export Citation
  • Goring, D. G., , and Nikora V. I. , 2002: Despiking acoustic Doppler velocimeter data. J. Hydraul. Eng., 128, 117126, doi:10.1061/(ASCE)0733-9429(2002)128:1(117).

    • Search Google Scholar
    • Export Citation
  • Hill, W., , and Rogalia D. , 1992: Spike-correction of weak signals from charge-coupled devices and its applications to Raman spectroscopy. Anal. Chem., 64, 25752579, doi:10.1021/ac00045a019.

    • Search Google Scholar
    • Export Citation
  • Højstrup, J., 1993: A statistical data screening procedure. Meas. Sci. Technol., 4, 153157, doi:10.1088/0957-0233/4/2/003.

  • Lee, X., , Neumann H. H. , , Hartog G. , , Mickle R. E. , , Fuentes J. D. , , Black T. A. , , Yang P. C. , , and Blanken P. D. , 1997: Observation of gravity waves in a boreal forest. Bound.-Layer Meteor., 84, 383398, doi:10.1023/A:1000454030493.

    • Search Google Scholar
    • Export Citation
  • Lee, X., , Massman W. J. , , and Law B. E. , Eds., 2004: Handbook of Micrometeorology: A Guide for Surface Flux Measurement and Analysis. Atmospheric and Oceanographic Sciences Library, Vol. 29, Springer, 250 pp.

  • LI-COR, 2012: EddyPro 3.0 help and user’s guide: Despiking and raw data statistical screening. Accessed 8 June 2015. [Available online at http://envsupport.licor.com/envhelp/EddyPro3/Content/Topics/Despiking_Raw_Stat_Screening.htm.]

  • Mauder, M., , and Foken T. , 2006: Impact of post-field data processing on eddy covariance flux estimates and energy balance closure. Meteor. Z., 15, 597609, doi:10.1127/0941-2948/2006/0167.

    • Search Google Scholar
    • Export Citation
  • Mauder, M., , and Foken T. , 2011: Documentation and instruction manual of the eddy-covariance software package TK3. Universität Bayreuth Dept. of Micrometeorology Work Results 46, 60 pp.

  • Mauder, M., , Cuntz M. , , Drüe C. , , Graf A. , , Rebmann C. , , Schmid H. P. , , Schmidt M. , , and Steinbrecher R. , 2013: A strategy for quality and uncertainty assessment of long-term eddy-covariance measurements. Agric. For. Meteor., 169, 122135, doi:10.1016/j.agrformet.2012.09.006.

    • Search Google Scholar
    • Export Citation
  • Metzger, S., , Junkermann W. , , Mauder M. , , Beyrich F. , , Butterbach-Bahl K. , , Schmid H. P. , , and Foken T. , 2012: Eddy-covariance flux measurements with a weight-shift microlight aircraft. Atmos. Meas. Tech., 5, 16991717, doi:10.5194/amt-5-1699-2012.

    • Search Google Scholar
    • Export Citation
  • Nikora, V. I., , and Goring D. G. , 2000: Flow turbulence over fixed and weakly mobile gravel beds. J. Hydraul. Eng., 126, 679690, doi:10.1061/(ASCE)0733-9429(2000)126:9(679).

    • Search Google Scholar
    • Export Citation
  • Papale, D., and Coauthors, 2006: Towards a standardized processing of Net Ecosystem Exchange measured with eddy covariance technique: Algorithms and uncertainty estimation. Biogeosciences, 3, 571583, doi:10.5194/bg-3-571-2006.

    • Search Google Scholar
    • Export Citation
  • Pasian, F., , and Crise A. , 1984: Restoration of signals degraded by impulse noise by means of a low-distortion nonlinear filter. Signal Process., 6, 6776, doi:10.1016/0165-1684(84)90052-5.

    • Search Google Scholar
    • Export Citation
  • Phillips, G. R., , and Harris J. M. , 1990: Polynomial filters for data sets with outlying or missing observations: Application to charge-coupled-device-detected Raman spectra contaminated by cosmic rays. Anal. Chem., 62, 23512357, doi:10.1021/ac00220a017.

    • Search Google Scholar
    • Export Citation
  • Starkenburg, D., , Fochesatto G. J. , , Prakash A. , , Cristóbal J. , , Gens R. , , and Kane D. L. , 2013: The role of coherent flow structures in the sensible heat fluxes of an Alaskan boreal forest. J. Geophys. Res. Atmos., 118, 81408155, doi:10.1002/jgrd.50625.

    • Search Google Scholar
    • Export Citation
  • Vickers, D., , and Mahrt L. , 1997: Quality control and flux sampling problems for tower and aircraft data. J. Atmos. Oceanic Technol., 14, 512526, doi:10.1175/1520-0426(1997)014<0512:QCAFSP>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 107 107 36
PDF Downloads 99 99 24

Assessment of Despiking Methods for Turbulence Data in Micrometeorology

View More View Less
  • 1 Geophysical Institute, and Department of Atmospheric Sciences, College of Natural Science and Mathematics, University of Alaska Fairbanks, Fairbanks, Alaska
  • 2 Fundamental Instrument Unit, National Ecological Observatory Network, and Institute of Arctic and Alpine Research, University of Colorado Boulder, Boulder, Colorado
  • 3 Geophysical Institute, and Department of Atmospheric Sciences, College of Natural Science and Mathematics, University of Alaska Fairbanks, Fairbanks, Alaska
  • 4 Hydrology and Remote Sensing Laboratory, ARS, USDA, Beltsville, Maryland
  • 5 Geophysical Institute, University of Alaska Fairbanks, Fairbanks, Alaska
© Get Permissions
Restricted access

Abstract

The computation of turbulent fluxes of heat, momentum, and greenhouse gases requires measurements taken at high sampling frequencies. An important step in this process involves the detection and removal of sudden, short-lived variations that do not represent physical processes and that contaminate the data (i.e., spikes). The objective of this study is to assess the performance of several noteworthy despiking methodologies in order to provide a benchmark assessment and to provide a recommendation that is most applicable to high-frequency micrometeorological data in terms of efficiency and simplicity. The performance of a statistical time window–based algorithm widely used in micrometeorology is compared to three other methodologies (phase space, wavelet based, and median filter). These algorithms are first applied to a synthetic signal (a clean reference version and then one with spikes) in order to assess general performance. Afterward, testing is done on a time series of actual CO2 concentrations that contains extreme systematic spikes every hour owing to instrument interference, as well as several smaller random spike points. The study finds that the median filter and wavelet threshold methods are most reliable, and that their performance by far exceeds statistical time window–based methodologies that use the median or arithmetic mean operator (−34% and −71% reduced root-mean-square deviation, respectively). Overall, the median filter is recommended, as it is most easily automatable for a variety of micrometeorological data types, including data with missing points and low-frequency coherent turbulence.

Corresponding author address: Gilberto J. Fochesatto, Geophysical Institute, University of Alaska Fairbanks, 903 Koyukuk Drive, Fairbanks, AK 99775-7320. E-mail: gjfochesatto@alaska.edu

Abstract

The computation of turbulent fluxes of heat, momentum, and greenhouse gases requires measurements taken at high sampling frequencies. An important step in this process involves the detection and removal of sudden, short-lived variations that do not represent physical processes and that contaminate the data (i.e., spikes). The objective of this study is to assess the performance of several noteworthy despiking methodologies in order to provide a benchmark assessment and to provide a recommendation that is most applicable to high-frequency micrometeorological data in terms of efficiency and simplicity. The performance of a statistical time window–based algorithm widely used in micrometeorology is compared to three other methodologies (phase space, wavelet based, and median filter). These algorithms are first applied to a synthetic signal (a clean reference version and then one with spikes) in order to assess general performance. Afterward, testing is done on a time series of actual CO2 concentrations that contains extreme systematic spikes every hour owing to instrument interference, as well as several smaller random spike points. The study finds that the median filter and wavelet threshold methods are most reliable, and that their performance by far exceeds statistical time window–based methodologies that use the median or arithmetic mean operator (−34% and −71% reduced root-mean-square deviation, respectively). Overall, the median filter is recommended, as it is most easily automatable for a variety of micrometeorological data types, including data with missing points and low-frequency coherent turbulence.

Corresponding author address: Gilberto J. Fochesatto, Geophysical Institute, University of Alaska Fairbanks, 903 Koyukuk Drive, Fairbanks, AK 99775-7320. E-mail: gjfochesatto@alaska.edu
Save