Wavelet-Based Optical Flow for Two-Component Wind Field Estimation from Single Aerosol Lidar Data

a. Wind measurement strategies

Air motion is represented by a three-component vector and may be defined at all points in the atmosphere. The wind is generally regarded as the vector consisting of two horizontal components. Active remote wind measurement techniques may be subdivided into Doppler and non-Doppler approaches.

Ground-based radars and lidars typically collect data in a spherical coordinate system. Doppler radars and lidars directly measure only the radial (line of sight) component of air motion. For a Doppler radar or lidar to measure the wind, specific scanning strategies and assumptions about the air motion over space and time must be made. Wind profiling describes the use of a remote sensor to provide a vertical profile of horizontal wind vectors at a single location above the surface of the earth. Alternatively, two Doppler radars or lidars, separated by some horizontal distance, may be used to probe an area from different angles and to obtain a two-component wind field. This approach is known as “dual Doppler” (Stawiarski et al. 2013).

Non-Doppler approaches estimate wind fields from the spatial and temporal movement of features observed by the instrument. Eloranta et al. (1975) provided some of the first remote wind measurements by lidar in the lower atmosphere. Since that time, hardware and software have advanced greatly and a small number of validation experiments have been conducted, for example, Mayor et al. (2012). Meanwhile, other fields—in particular, experimental fluid dynamics—have developed similar approaches to retrieve motions. This concept is also known to the computer vision community, where it is associated with the wide family of motion estimation techniques.

b. Fluid motion estimation: The vision approach

The idea of using the apparent motion of tracers to infer the invisible underlying fluid flow is not new. It “could probably be traced far back in history to the first time a person possessing the concept of velocity watched small debris moving on the surface of a flowing stream” (Adrian 2005, p. 159). Many visualization methods have been developed, such as using droplets, dye, smoke, or shadows for the purpose of revealing fluid flow structures and dynamics (Van Dyke 1982). This led in particular to the well-known PIV techniques, which have been used in experimental fluid dynamics for almost 30 years (Adrian 2005). Our 2D two-component wind measurement approach fits in the motion estimation context: the tracers are the aerosol features, visualized by the lidar system, and the motion estimation technique is usually the cross correlation. This configuration is very comparable to PIV, with the important differences that the distribution of aerosols in the atmosphere (the “seeding” of the flow) cannot be controlled, and that the images are not of individual particles, but instead of a field that approximately represents particle concentration (Held et al. 2012). In these aspects, this problem is closer to AMV computation. An important difference is the temporal and spatial resolutions covered by these two approaches: typically, on the order of 15 s and 10 m for the considered lidar data versus 15 min and several kilometers for geostationary satellite imagery, respectively (García-Pereda and Borde 2014).

c. Motion estimation framework

Motion estimation aims to recover the apparent displacements within a sequence of images. The time and space variations of an observable image quantity are used to infer the underlying motion field occurring in the image plane between two consecutive frames of the sequence. In this work, input images are the scans provided by the lidar, and the movements of the variations of aerosol backscatter intensity are used to estimate the wind field.

In the following, the scan domain is noted as . The observable backscatter intensity is noted as at pixel and at discrete time , . The apparent displacement between two consecutive scans , is a 2D vector field :

View Expanded

This displacement is measured in pixel units and occurs over the time s. If the scan has a resolution of m per pixel, then an estimation of the instantaneous wind velocity (m s⁻¹) is therefore given by

View Expanded

As such, the motion is assumed to be stationary during the time step .

Velocity components , are the in-plane components; that is, they belong to the image plane. Because of the very low value of the elevation angle of the lidar scan plane (typically <6°), these components coincide with the horizontal wind components (usually denoted u, υ in atmospheric sciences). The out-of-plane component (normal to the scan plane), which remains unestimated, thus corresponds to the vertical component w.

The question of the accuracy of motion estimation techniques is often raised. The answer is complex, since it involves the data characteristics (spatial, temporal resolutions), the information given by the visualization method (the image content), and the underlying motion field itself. In the current context, the later contributions are difficult to quantify, as they depend largely on the conditions (e.g., the presence of particulate matter, the scales, and variability of the wind field). However, assuming ideal conditions and a perfect model, errors related to the resolution of data may be quantified. If displacements are measured as integers on the image grid, then the systematic error is ±0.5 pixel, which then gives m s^–1 for each motion component. In practice, various interpolation techniques allow for subpixel estimation, reducing this error. The error can be also lowered by using a smaller and/or a larger . However, for a given motion field, a smaller results in larger apparent displacements, which can be more challenging for estimation algorithms. Otherwise, larger leads to less accurate perception of the instantaneous velocity, since the assumption of stationarity of the motion field is less valid over longer periods.

Any motion estimation technique features two main aspects. The first one, known as the data model, describes the link between observations I (the aerosol backscatter intensity) and the underlying unknown displacement . This model should take into account the nature of observed data and its relevant dynamics. Then, as an inverse problem, motion estimation is usually ill posed. The second aspect is therefore the regularization, which is required in order to close the estimation problem. The regularization may also provide information where the data model fails locally. The various estimation techniques feature different data models, regularizations, or implementation strategies.

d. The cross-correlation algorithm, concept, and limitations

The cross-correlation technique performs independent local motion estimations on subregions (blocks) of the scan domain. It consists of correlating a block of the first scan with a translated block of the second scan ; the translation vector , which induces a correlation peak, is considered to be the displacement at the center of the block (Schols and Eloranta 1992). The estimation problem, presented in its basic form, is written as

View Expanded

where is the set of block centers (and therefore the set of locations of estimated vectors); is the block centered on ; and and are the mean and standard deviations, respectively, of backscatter intensity over block . Note that in practice, this cross-correlation function (CCF) is computed using the FFT for computational efficiency.

In this case, the data model is the CCF [(2)] itself; the regularization is implicitly given by the size of block , which should be large enough to contain reliable information, yet as small as possible to resolve small-scale motions. Typically, neighboring blocks overlap by 50%, so that the estimated motion field is sparse (fewer motion vectors than pixels). Each vector is the result of a single independent problem, which makes the CCF algorithm pleasingly parallel (Mauzey et al. 2012). This cross-correlation approach and its numerous variants have become widely used in PIV (Adrian and Westerweel 2010); in geosciences it is often applied to satellite imagery to retrieve, for instance, glacier velocities (Scambos et al. 1992). It is also the standard method to derive AMVs (Schmetz et al. 1993; García-Pereda and Borde 2014), and it has given good results with aerosol backscatter lidar data, as shown in Schols and Eloranta (1992), Mayor and Eloranta (2001), and Mayor et al. (2012).

However, this method as presented in (2) is not exempt from drawbacks. First, the displacement within an entire block is explained by a single vector , which implies that this displacement is assumed to be uniform (constant) over the block. The larger the block, the less likely this assumption is to be true. Yet, as overly small blocks may result in uncertainties due to lack of information, “large” blocks are usually preferred. This leads to the second point: as displacements occurring within large blocks are likely not uniform, the estimated corresponds to a power-weighted average of the apparent displacements within the corresponding block (Hamada 2014), which results in an oversmoothed motion field. To address these issues, this study proposes to evaluate a recently developed motion estimation algorithm dedicated to fluid flows.

a. Optical flow from observations to motion

The proposed approach has two major differences with respect to the cross-correlation algorithm presented above. First, this wavelet-based optical flow uses a global formulation: all vectors of the displacement field are estimated simultaneously by solving a single problem, whereas the cross-correlation approach in (2) has as many independent problems as vectors . Second, this method provides a dense estimate—that is to say, one displacement vector at every point of the scan domain , whereas the CCF solution is usually sparse. The estimate is obtained by minimizing a functional, similar to an energy, defined over the whole scan domain:

View Expanded

where is the data model that depends on observations I and unknown displacement , while the regularization depends on only. The parameter balances the two terms and is fixed by the user.

The data model used in Typhoon is known as the displaced frame difference (DFD):

View Expanded

It is analogous to finding the displacement field that “warps” an image into the next one. This model assumes the consistency of backscatter intensity along the trajectory of an aerosol feature during the time interval ; in other words, an aerosol feature will present the same intensity, the same “signature,” in both scans , . Therefore, any phenomena inducing a significant change in intensity, such as turbulent diffusion or out-of-plane motion, can possibly lead to false apparent motions.¹ Such phenomena are not uncommon, but it can be reasonably assumed that the time scales at which they act are significantly larger than the interscan time step , so that the DFD [(4)] remains valid. It is also important to note that from formulation (3), the data model is not strictly enforced. Instead, the solution achieves a balance between trying to follow the model on one hand and the regularization on the other—hence, the role of the parameter α, which allows the user to give more weight to one term over the other.

Regularization schemes usually encourage the estimate to follow some smoothness assumption. This work uses the most simple first-order regularization, originally introduced in Horn and Schunck (1981), which penalizes strong velocity gradients. For each displacement component , :

View Expanded

Note that the square root is later cancelled by the square in (3). If the regularization is given much more weight than the data model [ in (3)], then the solution that minimizes (3) moves toward a uniform motion field (with for ). The regularizer also takes precedence over the data model locally where the latter is inefficient, for instance, within uniform regions of the input images. Other regularizers are available in Typhoon, penalizing, for instance, the vorticity or divergence of the flow, or the gradient of vorticity, divergence; some of these schemes have proven to be very efficient with PIV and water vapor satellite images (Corpetti et al. 2002). However, as the regularization becomes more complex, the associated computational costs increase, which may reduce the ability to achieve real-time estimation. Moreover, in the context of aerosol backscatter lidar images, little to no improvement brought by the use of these advanced schemes was found. This could be linked to the specificities of this lidar data, which will be detailed further in section 4.

The DFD model [(4)] and the Horn and Schunck regularizer [(5)] inserted into (3) complete the motion estimation problem:

View Expanded

A particularity of this problem is that the DFD model [(4)] is not linear in , so that the whole functional is not quadratic. This complicates the minimization process, as the existence of a global minimum is not guaranteed. This is another role for the regularization term: it convexifies the functional as . But, as large α values are unmanageable, to ensure a successful minimization it is important for the solution to lie “close” to the first guess.² This calls for the use of an incremental strategy, often known as “multiresolution”: the displacement field is estimated following a coarse-to-fine process, starting with coarse structures of large amplitudes and progressively refining toward smaller scales. This last point motivates the use of the wavelet framework.

b. Introduction to the wavelet framework

In signal processing, the spectral space is often used to analyze or exhibit some properties of a given signal. The FFT leads to a representation in terms of sine and cosine functions of specific frequencies. Any spatial information is lost in the process: the Fourier coefficients, which form an equivalent representation of the input signal, yield no information as to where their associated frequency is or is not present. This is due to the fact that the sine and cosine functions, which form the basis of the spectral space, are very well localized in frequency but have an infinite support in space. Conversely, looking at the signal in the physical space does not give any information on the frequency content. The wavelet formalism offers a trade-off: the wavelet functions are localized both in space and frequency; thus, they enable access to information on the frequency content and the spatial location simultaneously—at the cost of lower precision. A wavelet representation of a given signal consists of a coarse approximation of the signal, along with several sets of details containing spatially localized information at various ranges of frequencies. Note that instead of frequency, the wavelet formalism prefers the equivalent but reciprocal notion of scale.

This multiscale (or, multiresolution) representation offered by the wavelet transform is the main motivation to adopt wavelet bases for displacement components , . It leads to a “natural” coarse-to-fine strategy suitable to motion estimation (Dérian et al. 2013). Approximation and coarse detail coefficients are estimated first and then finescale details are successively added until the smallest scale is reached. Besides the multiscale framework, wavelet bases also allow the representation of arbitrary regular functions (a 3D fluid motion field should at least be continuous). While the continuity might not be a relevant assumption for the 2D field, a sufficient regularity is required in order to compute the regularizing terms presented in section 3a, which involve spatial derivatives. Finally, these regularization schemes find a relatively simple yet very accurate implementation in that context (Kadri Harouna et al. 2013). Similarly to the Fourier transform, the wavelet transform is a linear, separable³ operator, with fast algorithms [fast wavelet transform (FWT)] for computational efficiency. Wavelets are also used in many fields, from signal denoising to video compression; Mallat (2008) discusses an extensive presentation of the theory and applications.

Conceptually, the use of wavelet bases does not lead to significant changes to the estimation problem [(6)]. Each motion component is expressed as the inverse transform (reconstruction) of its corresponding wavelet coefficients :

View Expanded

where denotes the inverse wavelet transform. The set of wavelet coefficients thus is the unknown to the estimation problem.

c. Recent improvements

The original algorithm detailed in Dérian et al. (2013) would accept square images only. If input images were rectangular, then they had to be padded to turn them square, which increases the computational burden. The current version has been modified to accept rectangular images.

The main improvement is the result of redesigning the code to run in real time. To keep up with real time, the estimate of wind field from scans , must be complete by the time the next scan is made available, with the interscan time step typically on the order of 10–20 s. Since the whole motion field is estimated simultaneously, the number of variables is quite large: a dense estimate from 512 × 512 pixel images represents about half a million unknowns. Wavelet transforms lie at the core of the estimation process. Each evaluation of the functional (6) requires two inverse FWTs (to reconstruct the displacement from its coefficients) and two forward FWTs (to compute the gradient). To achieve the necessary reduction in computation time, the low-level functions of the algorithm—in particular, the wavelet transforms—were rewritten in Compute Unified Device Architecture (CUDA) language, which enables it to execute on NVIDIA’s graphics processing units (GPUs). GPUs designed for scientific computing rely on several thousands of small computing units, thus providing massive parallelization capabilities. The CUDA version of Typhoon running on an NVIDIA GeForce GTX Titan is 10–100 times faster than the original version (Mauzey et al. 2014) and is sufficient to meet the real-time requirements.

a. Data preprocessing

Before motion estimation takes place, the raw signal delivered by the REAL must be preprocessed. Lidar data are sampled on a polar grid, with the lidar at the origin. Each scan is composed of shots, with a shot being a 1D array of backscatter samples, uniformly spaced along the range r every 1.5 m, collected at a given angular position θ from a single laser pulse.

The raw backscatter intensity corresponds to the actual backscatter signal and an additive noise ,

View Expanded

The noise ε combines contributions from the atmosphere and the instrument, and can be modeled by a random variable that follows a normal distribution of mean and standard deviation . Values of and change slightly from one shot to another, hence their dependency in θ; they can be estimated for each shot from background data. As explained in Mayor et al. (2012), first the noise mean is subtracted:

View Expanded

with the now-centered random noise. The raw signal-to-noise ratio (SNR) is computed at that point:

View Expanded

Shots are then multiplied by the square of the range to compensate for the one-over-range-squared decay of the backscatter β:

View Expanded

Note that the noise amplitude now increases as the square of the range. For optimal results, it is then essential to discard irrelevant noisy data, which is discussed further.

After conversion to decibels, shots are filtered in the range dimension. The low-pass median filter of length 7 points (10.5 m) removes high-intensity spikes typically caused by hard targets, such as birds and insects, while the high-pass median filter of length 333 points (500 m) removes the very large structures to reveal local fluctuations. Figure 1 presents an example of preprocessed backscatter data (Fig. 1a), along with the corresponding raw SNR [(7)] (Fig. 1b).

View Full Size

Example of preprocessing applied to a horizontal scan collected at 2314:10 UTC 3 Oct 2013. (a) Preprocessed backscatter data. (b) Raw SNR [(7)], revealing a decay. (c) Image SNR [(8)] computed using a 384-m window. (d) Valid data domain computed from image SNR. Motion is estimated in the white area only, excluding far-range noisy regions. The far-range boundary [(9)] of this area is also shown in (a) as a white line. An example of estimated vector flow field has been added to the valid area in (d), decimated for the sake of visualization.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

b. Detecting coherent features

Two different aspects complicate the motion estimation process. First, because of the nature of backscatter data, the raw SNR [(7)] decays as one-over-range-squared. Typically, for the REAL operating in Chico, the SNR resulting from a single laser pulse drops below 5 at r = 3 km. Such high levels of noise in the far range are challenging for optical flow. Second, for the purpose of motion estimation, a good SNR in the near range does not necessarily imply useful information. For instance, coherent features can be absent from a region of the scan, yielding much uncertainties as to the underlying wind field in that region.

To maximize the quality of the results, the scan areas presenting no coherent aerosol features are discarded. Because of the regularization schemes provided by optical flow (section 3a), wind vectors estimated over noisy areas could be relevant. However, judging so proves to be difficult, as often even a basic visual confirmation is impossible in noisy regions. Hence, it is safer to simply discard the noisy image data before motion estimation.

To detect the presence of coherent aerosol features, the image SNR is used. It is defined as the ratio of the local standard deviation of coherent signal to the local standard deviation of noise :

View Expanded

This ratio is estimated from the autocovariance function of preprocessed data . For every point , the autocovariance is computed along the range from data in . Then, the local variance of coherent signal is given by the average of coefficients at lag 1 and lag −1:

View Expanded

while the local variance of noise is obtained from the 0-lag coefficient and :

View Expanded

An example of image SNR is shown in Fig. 1c. A 256-point window was used to compute the autocovariance, corresponding to m.

From the image SNR, a valid data domain is computed for each scan. It is assumed that the best data are in the near range; therefore, the valid domain is simply defined by a far-range boundary. For each shot (azimuth θ), this far-range boundary is given by the smallest range above which the image SNR remains below a threshold τ fixed by the user:

View Expanded

Finally, a low-pass median filter of width 25 points and a Gaussian filter of parameter points are applied to the set of , to exclude small isolated features and to smooth the boundary. An example of mask representing the valid data domain is shown in Fig. 1d, using .

c. Correction of image distortions

A lidar scan does not correspond to an instantaneous view of the aerosol distribution. The shots that compose the scan are acquired sequentially. In the event of high wind speeds, this leads to apparent distortions of the aerosol features in the lidar images, which in turn causes the estimated motion to be biased. This issue was first noted by Sasano et al. (1982), who proposed an iterative correction method. Assuming that the aerosol features are transported without deformation by a uniform wind vector, scans can be warped to reconstruct an approximated instantaneous view of the aerosols, thus improving the accuracy of motion estimation. In this study, implementation proceeds as follows for a given scan pair:

1. Estimate the displacement field from the pair of scans with the Typhoon algorithm.

2. Convert to velocity field using (1).

3. Correct both scans for distortions using wind field , following Sasano et al. (1982). The time of the beam at the center of the scan is used as the reference time.

4. Repeat steps 1–3 until mean wind speed changes by either less than 1% or less than . Typically, it requires two to three iterations.

d. Cartesian gridding

After preprocessing, masking, and correcting for distortions, the backscatter data are interpolated on a Cartesian grid of spacing m. It is possible to perform the motion estimation directly on the original polar grid; however, as mentioned above, the correction step destroys the regularity of the mesh. Fast interpolation on large sets of scattered data can be challenging, considering real-time requirements. In this work, a CUDA implementation of nearest-neighbor interpolation was used.

a. Temporal validation

In this experiment, the REAL scans between −15° and 45° azimuth, with a 4° elevation, every 17 s. This places the scan at 100 m AGL at the range of the DL. The DL operates in vertical profiling mode (VAD scan), providing a profile of two-component horizontal wind vectors about every 15 s.

A typical example of aerosol motion estimation is presented in Fig. 2. It features a close-up of two motion fields estimated from three successive position plan indicator (PPI) scans. The flow is relatively uniform, and can be visually identified due to a large aerosol feature that moves toward the southeast. The DL wind vectors at 100 m AGL are displayed for comparison and show a good agreement with the Typhoon estimates.

View Full Size

Illustration of the experimental design for the temporal validation of motion estimation vectors. (a) Short sequence of three consecutive PPI scans collected on 14 Oct 2013 by the REAL. The displayed area is a close-up centered on the Doppler lidar (white marker) used for validation. The copper shading indicates the intensity (dB) of aerosol backscatter. A large aerosol feature is being advected southeast and passes over the DL. (b),(c) Velocity fields estimated by Typhoon (black arrows) from each pair of scans; they were decimated by a factor of 6 along both dimensions for the sake of visualization. Measurements from the DL (red arrows) at 100 m AGL show good agreement with estimates, with a wind speed of ≈5.4 m s^–1.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

In this paper, an effort is made to establish the potential of the Typhoon algorithm when applied to aerosol backscatter lidar data. However, quality of the data depends upon the performance of the instrument and the state of the atmosphere. Therefore, we selected the days presenting the best potential for this validation among data collected in Chico from mid-September to mid-November 2013, with the expectation that future advances in hardware will lead to increases in data quality and availability. First, because of the local typical conditions in Chico, aerosol backscatter imagery is much better for this application during the daytime than during nighttime. Therefore, this study was restricted to daytime only. Second, the percentage of valid backscatter data (section 4b), during daytime, in a 50-m radius around the DL was computed. These values are plotted against the mean wind speed measured by the DL the same day in Fig. 3. With a sufficient spatial distribution of aerosol features, dense two-component wind fields can be delivered up to several kilometers in range. Figure 4 shows an example of such wind field on a day with high speed and uniform direction, with vectors available out to 4-km range. The low-SNR area in the far range was dynamically excluded. Figure 5 presents a view of a ≈200-m vortex, illustrating the ability of Typhoon to extract coherent structures at intermediate scales.

View Full Size

Distribution of days in terms of valid image SNR (section 4b) in a 50-m radius around DL location (horizontal axis) vs mean wind speed measured by the DL at 100 m AGL (vertical axis), during daytime. Days for which time series are presented (Figs. 6–8) are represented with a black diamond. A total of 57 days are considered, of which 9 have less than 60% valid and are not visible here. The 15 days having more than 85% valid were investigated for the statistics shown in Fig. 9.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Wind field obtained by Typhoon at 1845:07 UTC 3 Oct 2013, superimposed on the first scan of the pair used for estimation. Wind velocity was m s^–1. The motion field was decimated along both dimensions by a factor of 24. The turquoise circle represents the conic section sampled by the DL during the VAD scan.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Wind field obtained by Typhoon at 2332:04 UTC 23 Oct 2013, superimposed on the first scan of the pair used for estimation. The top panel shows a close-up of a vortex with radius m. The vector field was decimated along both dimensions by a factor of 6 and 12 for the top and bottom panels, respectively. The turquoise circle represents the conic section sampled by the DL during the VAD scan.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

Three specific cases are described below: light, moderate, and strong wind conditions. These days are represented by solid diamonds in Fig. 3. For each case, time series of instantaneous and 10-min averaged wind measurements are presented. The 10-min averages are the reference measures for instrument validation in the wind power industry (Brower 2012). Then, statistics on 10-min averages for the 15 days having more than 85% valid data are presented.

The VAD scan strategy used by the DL assumes that the wind is uniform throughout the swept area (Mann et al. 2009, 2010; Sathe et al. 2011; Sathe and Mann 2012); in this case this region is a disc of about 100-m radius, represented by a turquoise circle in Figs. 4 and 5. To compare results of the study to the DL measures, instantaneous Typhoon estimates are averaged in space over a similar sized area centered on the DL location.

Occasionally, the estimation may fail and result in obvious outliers. Those outliers can be detected and removed under the assumption of temporal coherence of the wind field. The normalized median test, commonly used in PIV (Adrian and Westerweel 2010), was implemented. Similar concepts are used with radar wind profilers (Weber et al. 1993). Within each 10-min window, the median wind vector is computed, as well as the residuals for each vector of the window. Vectors for which the residual is twice larger than the median of residuals are discarded.

1) Light wind case

Figure 6 shows the wind speed and direction measured by the DL at 100 m AGL and estimated by Typhoon for a 12-h period starting at 1500 UTC 23 October. It is a light wind episode with speeds remaining below 3 m s^–1 and variable direction. Estimates are missing over a period approximatively covering 1500–1700 UTC. This is due to the coherent feature detection presented in section 4b: no significant features were present in the region of interest at that time; therefore, no motion estimates are available. Then, between 1700 and 1800 UTC, Typhoon speed and direction estimates are in systematic error. Visual inspection of the aerosol imagery reveals the mixed layer growing with the entrainment zone passing through the altitude of the intercomparison. It appears that the plumes and wind shear in the entrainment zone result in false apparent motions that bias the motion estimations. Later, two reversals of wind direction occurred at 2230 and 2330 UTC that correspond to the passage of a vortex of diameter ≈200 m over the region of interest (see also Fig. 5 for a spatial visualization). This microscale circulation resembles those that have resulted from large-eddy simulation of convective boundary layers (Schmidt and Schumann 1989; Kanak 2005; Sullivan and Patton 2011). Correlation coefficients for the 10-min averaged wind components are 0.951 and 0.600 for u and υ, respectively. Excluding the 1700–1800 UTC period with false apparent motions, values increase to 0.966 and 0.866, respectively.

View Full Size

Times series of (top) wind speed and (bottom) direction as measured by the DL (blue) and estimated by Typhoon (orange), for a 12-h period starting at 1500 UTC 23 Oct 2013 (light wind case). Light plus (+) markers are instantaneous values; darker lines are the 10-min rolling averages. The rapid change in direction is the signature of the vortex presented in Fig. 5.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

2) Moderate wind case

Figure 7 shows wind speed and direction measured by the DL at 100 m AGL and estimated by Typhoon for a 12-h period starting at 1500 UTC 17 September. This wind episode features speeds ranging from 0 to 10 m s^–1 and direction mostly stationary except for a 2-h fluctuating episode (corresponding to the lowest wind speeds). Wind speed is underestimated at two occasions, both corresponding to rapid and large changes in direction around 2230 and 2300 UTC. Otherwise, both series of data are in very good agreement. This is confirmed by the 10-min averaged wind components: correlation coefficients are 0.979 and 0.991 for u and υ, respectively.

View Full Size

Times series of (top) wind speed and (bottom) direction as measured by the DL (blue) and estimated by Typhoon (orange), for a 12-h period starting at 1500 UTC 17 Sep 2013 (moderate wind case). Light plus (+) markers are instantaneous values; darker lines are the 10-min rolling averages.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

3) Strong wind case

Figure 8 shows wind speed and direction measured by the DL at 100 m AGL and estimated by Typhoon for a 12-h period starting at 1500 UTC 9 October. It is a strong wind episode with speeds up to 16 m s^–1 and very consistent flow from the northwest direction. Both time series are again in very good agreement. Correlation coefficients for the 10-min averaged wind components are 0.984 and 0.929 for u and υ, respectively.

View Full Size

Times series of (top) wind speed and (bottom) direction as measured by the DL (blue) and estimated by Typhoon (orange), for a 12-h period starting 1500 UTC 9 Oct 2013 (strong wind case). Light plus (+) markers are instantaneous values; darker lines are the 10-min rolling averages.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

4) Overall considerations

Scatterplots of 10-min averaged wind components measured during the daytime⁴ for the 15 “best” days (Fig. 3) are presented in Fig. 9. They show overall excellent agreement of Typhoon estimates with DL measurements at 100 m AGL: correlation coefficients are 0.995 and 0.997 for u and υ, respectively. Detailed statistics on u and υ are available in Tables 2 and 3. In terms of wind speed, a linear regression gives a slope of 1.000 with an offset of −0.10 m s^–1, and the coefficient is 0.991. Regarding the wind direction, the offset is 1.1° and the coefficient is 0.944.⁵ This ≈1° offset observed for the direction corresponds to the precision at which the DL was oriented during its deployment. The root-mean-square error (RMSE) between Typhoon’s estimates and the DL observations is 0.29 m s^–1 on both on u and υ components. This is slightly higher than the expected systematic error of m s^–1, which assumes perfect data and model (section 2c). The few remaining outliers mostly correspond to false apparent motions, typically occurring at the beginning and end of the day as the boundary layer depth evolves.

View Full Size

(a),(b) Scatterplots of 10-min averaged wind components u and υ measured by the DL at 100 m AGL (horizontal axis) vs estimated by Typhoon (vertical axis), combining the 15 days having >85% valid image SNR during daytime (Fig. 3)—892 points total. (c),(d) Distribution of differences for the same dataset.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

Table 2.

RMS error, linear regression variables (slope, offset), correlation coefficient , number of points, and recovery percentage w.r.t. DL reference for the 10-min averaged wind component u (west–east) for the temporal validation results (section 5a).

View Table

Table 3.

RMS error, linear regression variables (slope, offset), correlation coefficient , number of points, and recovery percentage w.r.t. DL reference for the 10-min averaged wind component υ (south–north) for the temporal validation results (section 5a).

View Table

From the time series shown in Figs. 6–8, it appears the variability of the wind speed obtained by Typhoon is less than that measured by the Doppler. Figure 10 is a scatterplot of turbulent kinetic energy (TKE) as measured by the Doppler and Typhoon over 10-min intervals. A linear regression suggests that the TKE from Typhoon is about 50% smaller than the Doppler’s. This could be linked to the fact that Typhoon measures apparent displacements, which are later converted to velocities (section 2c). Small-scale velocity structures, either in time or space, are less accurately perceived. Using a faster scan rate is likely to improve the results. Nevertheless, Typhoon performs better than the cross-correlation technique: the optimized algorithm presented in Hamada et al. (2015, manuscript submitted to J. Atmos. Oceanic Technol.) recovers 39% of the TKE on the same dataset.

View Full Size

Scatterplot of the TKE measured over 10-min intervals, by the DL at 100 m AGL (horizontal axis) vs estimated by the proposed method (vertical axis)—892 points total. The gray shading indicates the mean wind speed measured over the interval. A linear regression (dashed line) gives a slope of 0.49 and an offset of −0.03.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

b. Spatial validation

During this phase of the experiment, the DL was collocated with the REAL. The REAL swept between 15° and 75° azimuth at 2° elevation every 17 s. The DL held its beam fixed at 45° azimuth and 2° elevation, measuring the radial velocity component as a function of range and time; see Fig. 11. DL measurements were integrated over 1 s, with a range gate of 48 m. The temporal resolution of DL measurements is therefore much finer than that of the REAL flow fields, and conversely for the spatial resolution (see Table 1).

Instead of holding the DL beam fixed, a PPI sweeping strategy identical to the REAL’s could have been used, thus allowing the comparison of radial components over the whole scan domain. However, two arguments support the choice of a fixed beam:

• With a moving beam setup, the integration time for DL measurements was reduced to less than 0.1 s. This would cause the SNR to decrease very rapidly. Typically in Chico the maximum range with useful data would be on the order of 1500 m, significantly below that of the REAL’s maximum range.

• The radial velocity fields collected by the DL would suffer from the same distortions as the backscatter data (section 4c), so correcting these distortions would be challenging.

View Full Size

Illustration of the experimental design for the spatial validation of motion estimation vectors. (a) Subsets of two consecutive PPI scans collected on 8 Jan 2014 by the REAL. The displayed area is a close-up centered on the DL line of sight at 45° azimuth (dashed white line). The copper shading indicates the intensity (dB) of aerosol backscatter. A large aerosol feature is being advected north. (b) The velocity field (black arrows) estimated by Typhoon from these two scans; the vector field was decimated by a factor of 15 along both dimensions for the sake of visualization. The color shading in the background indicates the corresponding radial velocity. (c) Comparison of the radial velocities measured by the Doppler (black line) and extracted from the two-component field estimated by Typhoon (red line), along the DL line of sight.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

The data used for the spatial validation were recorded in December 2013 and January 2014. In Chico, the days are shorter and the air is cleaner during this season than in the autumn when time series data were collected. Both the DL and the REAL are affected. Data are of lower quality than shown for the temporal validation. The availability of 10-min averages falls below 50% after 3 km for both instruments and at 5 km it is below 5%. Therefore, the analysis is restricted to the first 3 km. Furthermore, it should be noted that the prevailing wind direction during this time over Chico is northwesterly. At 45° azimuth, the line-of-sight component corresponds mostly to the cross-stream, turbulent wind perturbations. In these data, its magnitude remains mostly below 3 m s^–1. Figure 12 shows a comparison of radial velocity measured by the DL and extracted from the two-component fields obtained by Typhoon for an 8-h period starting at 1700 UTC 8 January 2014.

View Full Size

Comparison of radial wind component at 45° azimuth and 2° elevation (top) measured by the DL and (bottom) estimated by Typhoon, as a function of time (horizontal axis) and range (vertical axis), for an 8-h period starting at 1700 UTC 8 Jan 2014. Gray shading indicates missing or discarded data.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

To compute statistics, radial velocities were averaged. First, the spatial resolutions are matched by averaging Typhoon velocities in space according to DL range gates; then, 10-min time averages are computed at every range. A scatterplot of these 10-min averages is presented in Fig. 13, along with linear regression slopes, coefficient, and distribution of differences. These values were obtained from 8-h periods (1700–0100 UTC) for 8 days in December 2013 and January 2014. The coefficient (Fig. 13d) decreases with the range and this is expected, as both instruments are affected by the gradual reduction in SNR. The remains above 0.95 over the first 1.5 km and then slowly decreases to about 0.8 at 3 km. The overall is 0.928. While the relation between Typhoon and DL velocities remains linear, the slope (Fig. 13c) increases with the range, from about 0.95 at 0.5 km to 1.3 at 3 km. Velocities obtained by the cross-correlation method show a similar trend (Hamada et al. 2015, manuscript submitted to J. Atmos. Oceanic Technol.). This leads to a theory that these discrepancies are due to a mismatch in the actual elevation angles of the beams during this phase of the experiment, especially considering the unbiased results of the temporal validation. At a lower elevation angle and therefore lower altitude, the DL would measure lower velocities.

View Full Size

(a) Scatterplot of 10-min averaged radial wind component measured by the DL (horizontal axis) vs estimated by Typhoon (vertical axis). Color indicates the range from blue (0.5 km) to red (3 km). (b) Histogram of differences. (c) Slope of linear regression as a function of range. (d) The coefficient as a function of range. Dashed red lines indicate overall slope and values.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

c. Spectral analysis

In this section, temporal and spatial power spectra of the velocity components produced by Typhoon are presented, with the objective of characterizing the filtering effect of the algorithm—in particular, in the spatial domain. The velocity data analyzed were collected during the daytime and within the turbulent lower atmospheric boundary layer.⁶ Therefore, an inertial subrange in the power spectra of the actual velocity field is expected.

The spectra are computed in natural coordinates to account for the anisotropy of atmospheric boundary layer turbulence. The west–east and south–north wind velocity components are projected, according to the mean wind direction, as streamwise () and cross-stream () components, such that carries the mean speed and has a null mean. The mean wind vector is defined accordingly to the investigated dimension, either in time or space. The spectra are finally averaged together according to the mean wind speed, using bins of 0–4, 4–8, 8–12, and 12–16 m s^–1, in order to exhibit their evolution with increasing wind speed and turbulent kinetic energy. The resulting power spectral densities (S) are multiplied by frequency (f) or wavenumber squared (), so that an inertial subrange would appear as a slope and white noise would appear as a slope.

1) Temporal power spectra

During the experiment, the REAL collected PPI scans every 17 s and one RHI scan every 15 min. The RHI scan resulted in a 30-s interruption of the PPI scan sequence. The scan strategy of the DL provided vertical profiles of horizontal winds every 15 ± 1 s. Since the FFT requires data points at a uniform time interval, the Typhoon and DL wind measurements were interpolated to a 5-s time series. From the 5-s time series data, we computed power spectra over consecutive 10-min intervals. The 10-min mean wind vector was used for the projection in natural coordinates and the binning of spectra, as defined above. The resulting spectra have a Nyquist frequency of 0.029 Hz (34-s period) for the Typhoon velocities and 0.033 Hz (30-s period) for the DL. The lowest frequency is 1.67 × 10^–3 Hz (10-min period).

The spectra are presented in Fig. 14. Those from the Doppler lidar are consistently higher than the spectra from Typhoon; this is consistent with our observation that the TKE measured from Doppler velocities is larger than those from Typhoon (Fig. 10). The temporal spectra appear to become flatter as the mean wind speed increases. We hypothesize that this may be caused by the challenges that both Typhoon and the DL face under windy conditions. For the DL, the increased variability of the actual wind velocity field in the VAD sample area results in more error in the horizontal wind vector estimate. The increased error appears as noise at these time scales and flattens the spectrum. For Typhoon, windy conditions result in larger horizontal displacements between scans and faster deformation of aerosol coherent structures.

View Full Size

Temporal spectra for (left) streamwise component and (right) cross-stream component obtained by Typhoon (solid lines) and the DL (dashed lines). The shadings from light to dark gray correspond to wind speed ranges of (0;4), (4;8), and (8;12) m s⁻¹. The dotted line represents the −2/3 slope of the inertial subrange predicted by theory.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

2) Spatial power spectra

An independent observation of the two-component 2D velocity field does not exist for comparison with those produced by Typhoon. A dual-Doppler lidar setup could have provided it, but it would have doubled the cost and complexity of the project. Therefore, to investigate the integrity of the vector flow fields in space, spatial power spectra are considered.

A 1-km-diameter circular area is considered, centered on the DL at 1.53-km range. All of the vectors within this area (from a single flow field in time) are used to compute the spatial mean wind vector, which then defines a natural coordinate system. Vectors of the flow field are interpolated on a 128 × 128 point grid (1024 m × 1024 m) that is centered on the DL and aligned with the natural coordinate system, and then projected as streamwise () and cross-stream () components. This operation was performed for each flow field independently and resulted in 30 092 flow fields over 15 days. At 4° elevation, the 1024 m × 1024 m area covers a range of altitudes from about 50 to 150 m AGL. A possible impact of this is that the turbulence statistics within this sloped domain are slightly inhomogeneous. Nevertheless, for each component , , the 2D power spectral densities computed by FFT for each flow field are averaged together according to the mean spatial wind speed. Finally, slices of the resulting 2D power spectra were extracted along the streamwise and cross-stream directions. This results in four 1D spectra for each wind speed bin: along the streamwise and cross-stream directions, for each of the streamwise and cross-stream components. The Nyquist wavenumber is m^–1 (16-m wavelength); the lowest wavenumber is 9.77 × 10^–4 m^–1 (1204 m).

The spectra in the top row of Fig. 15 show the TKE increasing as expected as a function of wind speed. Each spectrum has a maximum amplitude at low wavenumbers. We hypothesize that the peak corresponds to one over the Eulerian length scale and is within the energy-containing range (Kaimal and Finnigan 1994). However, the spectra are steeper than . We attribute this to two factors. The first is the likely absence of aerosol features at all scales and all locations in the scan area at all times. The second is the regularization used in Typhoon, which favors a smooth motion field, especially as the estimation reaches the smallest scales.

View Full Size

(top) Slices of 2D power spectral density and (bottom) corresponding transfer functions for the streamwise component u in the (a) streamwise and (b) cross-stream directions, and the cross-wise component υ in the (c) streamwise and (d) cross-stream directions. The shadings from light gray to black correspond to wind speed ranges of (0;4), (4;8), (8;12) and (12;16) m s⁻¹. The dotted line represents the −2/3 slope of the inertial subrange predicted by theory.

Citation: Journal of Atmospheric and Oceanic Technology 32, 10; 10.1175/JTECH-D-15-0010.1

• Download Figure
• Download figure as PowerPoint slide

A transfer function describes the ratio of two spectra and, in the present work, represents the attenuation of the actual wind field caused by the motion estimation as a function of wavenumber. A highly idealized spectrum is constructed to serve as the reference. This is done by first locating the maximum of each mean spatial spectrum shown in Fig. 15. We assume that the observed power at wavenumbers smaller than the peak in the spectra is accurately captured by the algorithm and serve as a proper approximation of the power at those large scales. For scales smaller than the peak, we extrapolate by a power-law dependence through the higher wavenumbers that mimics the inertial subrange (a spectrum). The transfer functions are then given by the ratio of the observed mean spectra over the idealized spectrum and are presented in the bottom row of Fig. 15. The higher the wind speed, the more energy is missing at small scales. The ratio typically drops below 50% at scales of ≈100 m ( m^–1) for the highest wind speeds and ≈75 m for the lowest wind speeds.