A Horizon-Tracking Method for Shipboard Video Stabilization and Rectification

Michael Schwendeman Applied Physics Laboratory, University of Washington, Seattle, Washington

Search for other papers by Michael Schwendeman in
Current site
Google Scholar
PubMed
Close
and
Jim Thomson Applied Physics Laboratory, University of Washington, Seattle, Washington

Search for other papers by Jim Thomson in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

An algorithm is presented for the stabilization and rectification of digital video from floating platforms. The method relies on a horizon-tracking technique that was tested under a variety of lighting and sea-state conditions for 48 h of video data over 12 days during a research cruise in the North Pacific Ocean. In this dataset, the horizon was correctly labeled in 92% of the frames in which it was present. The idealized camera model assumes pure pitch-and-roll motion, a flat sea surface, and an unobstructed horizon line. Pitch and roll are defined along the camera look direction rather than in traditional ship coordinates, such that the method can be used for any heading relative to the ship. The uncertainty in pitch and roll is estimated from the uncertainties of the horizon-finding method. These errors are found to be of the order 0.6° in roll and 0.3° in pitch. Errors in rectification are shown to be dominated by the uncertainty in camera height, which may change with the heave motion of a floating platform. The propagation of these errors is demonstrated for the breaking-wave distribution Λ(c). A toolbox for implementation of this method in MATLAB is shared via the MATLAB File Exchange.

Supplemental information related to this paper is available at the Journals Online website: http://dx.doi.org/10.1175/JTECH-D-14-00047.s1.

Corresponding author address: Michael Schwendeman, Applied Physics Laboratory, University of Washington, 1013 NE 40th Street, Box 355640, Seattle, WA 98105-6698. E-mail: mss28@u.washington.edu

Abstract

An algorithm is presented for the stabilization and rectification of digital video from floating platforms. The method relies on a horizon-tracking technique that was tested under a variety of lighting and sea-state conditions for 48 h of video data over 12 days during a research cruise in the North Pacific Ocean. In this dataset, the horizon was correctly labeled in 92% of the frames in which it was present. The idealized camera model assumes pure pitch-and-roll motion, a flat sea surface, and an unobstructed horizon line. Pitch and roll are defined along the camera look direction rather than in traditional ship coordinates, such that the method can be used for any heading relative to the ship. The uncertainty in pitch and roll is estimated from the uncertainties of the horizon-finding method. These errors are found to be of the order 0.6° in roll and 0.3° in pitch. Errors in rectification are shown to be dominated by the uncertainty in camera height, which may change with the heave motion of a floating platform. The propagation of these errors is demonstrated for the breaking-wave distribution Λ(c). A toolbox for implementation of this method in MATLAB is shared via the MATLAB File Exchange.

Supplemental information related to this paper is available at the Journals Online website: http://dx.doi.org/10.1175/JTECH-D-14-00047.s1.

Corresponding author address: Michael Schwendeman, Applied Physics Laboratory, University of Washington, 1013 NE 40th Street, Box 355640, Seattle, WA 98105-6698. E-mail: mss28@u.washington.edu
Save
  • Bao, G.-Q., Xiong S.-S. , and Zhou Z.-Z. , 2005: Vision-based horizon extraction for micro air vehicle flight control. IEEE Trans. Instrum. Meas.,54, 10671072, doi:10.1109/TIM.2005.847234.

    • Search Google Scholar
    • Export Citation
  • Benetazzo, A., 2006: Measurements of short water waves using stereo matched image sequences. Coastal Eng., 53, 10131032, doi:10.1016/j.coastaleng.2006.06.012.

    • Search Google Scholar
    • Export Citation
  • Bohren, C. F., and Fraser A. B. , 1986: At what altitude does the horizon cease to be visible? Amer. J. Phys., 54, 222227, doi:10.1119/1.14659.

    • Search Google Scholar
    • Export Citation
  • Callaghan, A. H., and White M. , 2009: Automated processing of sea surface images for the determination of whitecap coverage. J. Atmos. Oceanic Technol., 26, 383394, doi:10.1175/2008JTECHO634.1.

    • Search Google Scholar
    • Export Citation
  • Canny, J., 1986: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell., PAMI-8, 679698, doi:10.1109/TPAMI.1986.4767851.

    • Search Google Scholar
    • Export Citation
  • Cao, H., and Zhang J. , 2007: Video stabilizing and tracking by horizontal line for maritime cruise ship. Proceedings of the IEEE International Conference on Control and Automation, 2007, IEEE, 1202–1206.

  • Chickadel, C. C., Holman R. A. , and Freilich M. H. , 2003: An optical technique for the measurement of longshore currents. J. Geophys. Res., 108, 3364, doi:10.1029/2003JC001774.

    • Search Google Scholar
    • Export Citation
  • Dickey, T., Lewis M. , and Chang G. , 2006: Optical oceanography: Recent advances and future directions using global remote sensing and in situ observations. Rev. Geophys., 44, RG1001, doi:10.1029/2003RG000148.

    • Search Google Scholar
    • Export Citation
  • Duda, R. O., and Hart P. E. , 1972: Use of the Hough transformation to detect lines and curves in pictures. Commun. ACM, 15, 1115, doi:10.1145/361237.361242.

    • Search Google Scholar
    • Export Citation
  • Fefilatyev, S., Goldgof D. , Shreve M. , and Lembke C. , 2012: Detection and tracking of ships in open sea with rapidly moving buoy-mounted camera system. Ocean Eng., 54, 112, doi:10.1016/j.oceaneng.2012.06.028.

    • Search Google Scholar
    • Export Citation
  • French, A. P., 1982: How far away is the horizon? Amer. J. Phys., 50, 795799, doi:10.1119/1.13099.

  • Gemmrich, J. R., Banner M. L. , and Garrett C. , 2008: Spectrally resolved energy dissipation rate and momentum flux of breaking waves. J. Phys. Oceanogr., 38, 12961312, doi:10.1175/2007JPO3762.1.

    • Search Google Scholar
    • Export Citation
  • Heikkila, J., and Silven O. , 1997: A four-step camera calibration procedure with implicit image correction. Proceedings: 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, D. Plummer and I. Torwick, Eds., IEEE, 1106–1112.

  • Holland, K. T., Holman R. A. , Lippmann T. C. , Stanley J. , and Plant N. , 1997: Practical use of video imagery in nearshore oceanographic field studies. IEEE J. Oceanic Eng., 22, 8192, doi:10.1109/48.557542.

    • Search Google Scholar
    • Export Citation
  • Holman, R., and Haller M. C. , 2013: Remote sensing of the nearshore. Annu. Rev. Mar. Sci., 5, 95113, doi:10.1146/annurev-marine-121211-172408.

    • Search Google Scholar
    • Export Citation
  • Illingworth, J., and Kittler J. , 1988: A survey of the Hough transform. Comput. Vision Graphics Image Process., 44, 87116, doi:10.1016/S0734-189X(88)80033-1.

    • Search Google Scholar
    • Export Citation
  • IOCCG, 2000: Remote sensing of ocean colour in coastal, and other optically-complex, waters. S. Sathyendranath, Ed., IOCCG Rep. 3, 140 pp.

  • Kleiss, J. M., and Melville W. K. , 2010: Observations of wave breaking kinematics in fetch-limited seas. J. Phys. Oceanogr., 40, 25752604, doi:10.1175/2010JPO4383.1.

    • Search Google Scholar
    • Export Citation
  • Kleiss, J. M., and Melville W. K. , 2011: The analysis of sea surface imagery for whitecap kinematics. J. Atmos. Oceanic Technol., 28, 219243, doi:10.1175/2010JTECHO744.1.

    • Search Google Scholar
    • Export Citation
  • Moore, R. J. D., Thurrowgood S. , Bland D. , Soccol D. , and Srinivasan M. , 2011a: A fast and adaptive method for estimating UAV attitude from the visual horizon. 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, N. M. Amato et al., Eds., IEEE, 4935–4940.

  • Moore, R. J. D., Thurrowgood S. , Soccol D. , Bland D. , and Srinivasan M. V. , 2011b: A method for the visual estimation and control of 3-DOF attitude for UAVs. Australasian Conference on Robotics and Automation 2011 (ACRA 11), Australian Robotics and Automation Association, 267–275.

  • Morris, D. D., Colonna B. R. , and Snyder F. D. , 2007: Image-based motion stabilization for maritime surveillance. Image Processing: Algorithms and Systems V, J. T. Astola, K. O. Egiazarian, and E. R. Dougherty, Eds., International Society for Optical Engineering (SPIE Proceedings, Vol. 6497), 64970F, doi:10.1117/12.706099.

  • Phillips, O. M., 1985: Spectral and statistical properties of the equilibrium range in wind-generated gravity waves. J. Fluid Mech., 156, 505531, doi:10.1017/S0022112085002221.

    • Search Google Scholar
    • Export Citation
  • Romero, L., Melville W. K. , and Kleiss J. M. , 2012: Spectral energy dissipation due to surface-wave breaking. J. Phys. Oceanogr., 42, 14211444, doi:10.1175/JPO-D-11-072.1.

    • Search Google Scholar
    • Export Citation
  • Schwendeman, M., Thomson J. , and Gemmrich J. R. , 2014: Wave breaking dissipation in a fetch-limited sea. J. Phys. Oceanogr., 44, 104127, doi:10.1175/JPO-D-12-0237.1.

    • Search Google Scholar
    • Export Citation
  • Stockdon, H. F., and Holman R. A. , 2000: Estimation of wave phase speed and nearshore bathymetry from video imagery. J. Geophys. Res., 105, 22 01522 033, doi:10.1029/1999JC000124.

    • Search Google Scholar
    • Export Citation
  • Sutherland, P., and Melville W. K. , 2013: Field measurements and scaling of ocean surface wave-breaking statistics. Geophys. Res. Lett., 40, 30743079, doi:10.1002/grl.50584.

    • Search Google Scholar
    • Export Citation
  • Szeliski, R., 2010: Computer Vision: Algorithms and Applications. Texts in Computer Science, Springer, 812 pp.

  • Thomson, J., and Jessup A. T. , 2009: A Fourier-based method for the distribution of breaking crests from video observations. J. Atmos. Oceanic Technol., 26, 16631671, doi:10.1175/2009JTECHO622.1.

    • Search Google Scholar
    • Export Citation
  • Thomson, J., Gemmrich J. R. , and Jessup A. T. , 2009: Energy dissipation and the spectral distribution of whitecaps. Geophys. Res. Lett.,36, L11601, doi:10.1029/2009GL038201.

  • Thurrowgood, S., Soccol D. , Moore R. , Bland D. , and Srinivasan M. V. , 2009: A vision based system for attitude estimation of UAVS. Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2009), IEEE, 5725–5730.

  • Thurrowgood, S., Moore R. J. D. , Soccol D. , Knight M. , and Srinivasan M. V. , 2014: A biologically inspired, vision-based guidance system for automatic landing of a fixed-wing aircraft. J. Field Rob., 31, 699727, doi:10.1002/rob.21527.

    • Search Google Scholar
    • Export Citation
  • Weissling, B., Ackley S. , Wagner P. , and Xie H. , 2009: Eiscam—Digital image acquisition and processing for sea ice parameters from ships. Cold Reg. Sci. Technol., 57, 4960, doi:10.1016/j.coldregions.2009.01.001.

    • Search Google Scholar
    • Export Citation
  • Zappa, C. J., Banner M. L. , Schultz H. , Gemmrich J. R. , Morison R. P. , LeBel D. A. , and Dickey T. , 2012: An overview of sea state conditions and air-sea fluxes during RaDyO. J. Geophys. Res.,117, C00H19, doi:10.1029/2011JC007336.

  • Zhang, H., Yin P. , Zhang X. , and Shen X. , 2011: A robust adaptive horizon recognizing algorithm based on projection. Trans. Inst. Meas. Control, 33, 734751, doi:10.1177/0142331209342201.

    • Search Google Scholar
    • Export Citation
  • Zhang, Z., 2000: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell., 22, 13301334, doi:10.1109/34.888718.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1960 612 39
PDF Downloads 1440 218 15