1. Introduction

Traditionally, optical flow (OF) techniques are used in meteorology to estimate the motion of precipitation features from successive radar images and produce short-term forecasts of the future location of these precipitation features (e.g., Pulkkinen et al. 2019, among many others). The other main key application of optical flow estimation is computer vision (e.g., Brox et al. 2004). Two-dimensional motion fields derived from OF are then generally used to nowcast precipitation. A potential application of OF that has not received attention in the literature is the “unfolding” (also referred to as “dealiasing”) of Doppler velocities. The objective of this study is to explore the potential of OF techniques for Doppler velocity unfolding. The principle of “folded” Doppler velocities is thoroughly described in all radar textbooks (e.g., Doviak and Zrnić 2006) and therefore will not be described in detail here. Briefly, radars measure Doppler velocity in an ambiguous velocity interval [−V_N, V_N], where V_N is called the Nyquist velocity, which is a simple function of the radar wavelength λ and the pulse repetition frequency (PRF): V_N = λ × PRF/4. Any true Doppler velocity outside the Nyquist velocity interval will be “folded” back inside this interval. As an example, if the true Doppler velocity is +26 m s⁻¹, but the Nyquist velocity is 25 m s⁻¹, the measured Doppler velocity will be folded within the Nyquist interval by −2 V_N, yielding a measured value of −24 m s⁻¹.

When designing radar scanning strategies, there is a trade-off between achieving the highest possible Nyquist velocity (which requires the highest possible PRF) and the largest possible unambiguous maximum range (which requires the lowest possible PRF). This is referred to as the Doppler dilemma. This trade-off becomes increasingly challenging with decreasing wavelength. In Australia, the operational weather radar network includes a mixture of S-band (∼10-cm wavelength) and C-band (∼5-cm wavelength) radars. For a typical PRF of 1000 Hz, V_N = 25 m s⁻¹ at S band but only 13 m s⁻¹ at C band, for a maximum unambiguous range of about 150 km. Reducing this maximum range further to increase the Nyquist velocity to the typical requirement of 35 m s⁻¹ (the damaging wind threshold used in Australia) is not acceptable for operational radar surveillance purposes.

To increase V_N while still achieving long unambiguous range, the most popular technique is to transmit signals at two or more PRFs for adjacent alternating radar radials as the radar antenna rotates (e.g., Doviak et al. 1976; Joe and May 2003; Tabary et al. 2006). The change in PRF can be done on a pulse pair to pulse pair basis (technique called staggered PRF) or from successive blocks of pulses corresponding to a fraction of a degree in azimuth (technique called dual PRF). Usually, the dual-PRF technique is employed operationally (it is in Australia) for reasons detailed in Joe and May (2003). PRF ratios of 4:3 or 3:2 are popular choices, yielding a tripling or doubling of the Nyquist of the ray with the higher PRF of the two, respectively (Joe and May 2003). Although such techniques produce good Doppler velocities most of the time, they tend to generate problematic artifacts in regions of large wind shear or turbulence and regions of low signal-to-noise ratio (radar noise is amplified by dual-PRF techniques), as discussed in, e.g., Altube et al. (2017) and Joe and May (2003). Increasing the number of pulses for each PRF to mitigate dual-PRF artifacts due to noisy data can only be achieved by scanning slower, which is generally not possible operationally, or by using the two PRFs over a larger azimuthal area, which increases the risk of wrong unfolding in high wind shear situations.

A summary of the full scanning strategy currently used in Australia is presented in Table 1. Operationally, the requirement in Australia is to achieve an unambiguous low-level range of at least 250 km for the S-band radars (PRF = 600 Hz) and 150 km for the C-band radars (PRF = 1000 Hz). Corresponding Nyquist velocities using only one PRF are very low, 15 and 13 m s⁻¹ for the S-band and C-band radars, respectively. Dual PRF with a 3:2 ratio is only used in the three lowest elevation angles for the S-band radars, resulting in Nyquist velocities ranging from 36.1 to 46.7 m s⁻¹ depending on settings and elevation angle, but a single PRF of 1200 Hz is used for higher elevations, resulting in a V_N of 31.1 m s⁻¹. For C-band radars of the Australian radar network, dual PRF is used at most elevations but with different settings at different elevations, resulting in Nyquist velocities of at least 26.6 m s⁻¹ (and up to 42.5 m s⁻¹). These numbers set a clear expectation that an operational Doppler unfolding technique ingesting these operational Doppler radar observations should be capable of handling minimum Nyquist velocities of about 26 m s⁻¹. Ideally, such technique should also be able to identify and hopefully correct for dual-PRF artifacts [using techniques such as Altube et al. (2017) and Joe and May (2003)] to ultimately produce high-quality, unfolded Doppler radar velocities required for real-time severe weather detection algorithms (e.g., Smith et al. 2004), Doppler radar data assimilation in numerical weather prediction models (e.g., Sun and Crook 2001), and operational applications such as wind and hail nowcasting (e.g., Brook et al. 2021).

Table 1.

The Australian operational scanning strategy for S-band and C-band radars, including elevation angle, pulse repetition frequency, maximum range, and Nyquist velocity.

View Table

While our objective is not to provide an exhaustive description of families of Doppler unfolding techniques [the reader is referred to the comprehensive review and references provided in the introduction of Feldmann et al. (2020)], it is important to describe the general features that are employed in existing techniques to provide some background to our own development. Doppler unfolding techniques all use folded radial velocities as inputs. As part of the unfolding process, most if not all techniques use some combination of continuity tests between the radar gate to unfold and its neighbors in the radial and azimuthal directions (e.g., He et al. 2012; Feldmann et al. 2020; Louf et al. 2020). Another ingredient commonly used in several techniques is the vertical continuity between successive elevations (e.g., James and Houze 2001; He et al. 2012; Louf et al. 2020). Although this carries the risk of propagating unfolding errors to other elevations, such approach is usually quite successful at unfolding Doppler velocities aloft, where the density of observations is generally lower. Some techniques also use temporal continuity as an additional ingredient, such as the 4DD technique (James and Houze 2001), and the R2D2 technique (Feldmann et al. 2020). The challenge here is the initialization of the temporal continuity process and the associated risk of propagating unfolding errors forward in time. Some techniques, referred to as “regional” unfolding techniques, aim at reducing the risk of incorrect unfolding decisions based on single radar gates and their immediate neighbors by identifying contiguous Doppler areas to build more “regional” unfolding decisions (e.g., Helmus and Collis 2016; Feldmann et al. 2020).

Importantly, several Doppler unfolding techniques also use supplemental information about the environmental velocity field to constrain the unfolding process. A popular choice is to use the velocity–azimuth display (VAD; Browning and Wexler 1968) analysis as a proxy for horizontal wind (e.g., He et al. 2012; Feldmann et al. 2020). It is important to note that the VAD horizontal winds are themselves derived from an analysis of folded Doppler velocities (e.g., Tabary et al. 2001), assuming that the horizontal wind components are linear, i.e., the first-order spatial derivatives of the wind components are constant. Therefore, such approximation of horizontal wind motions can themselves be contaminated by Doppler folding and cannot be used to constrain areas where the horizontal wind is strongly nonlinear. This is the reason why other techniques elected to use high-resolution numerical model winds instead of VAD winds [e.g., the R2D2 technique from Feldmann et al. (2020)], but the assumption here is that the model does capture reasonably well the local velocity field details. Finally, the Feldmann et al. (2020) technique employs a recursive approach where complicated regions with high wind shear are first masked out, allowing for the simpler areas to be unfolded first. The high wind shear areas are then brought back and unfolded using radial and azimuthal continuity.

The fundamental challenge of these existing unfolding techniques is that they all use folded Doppler velocities to guess where they are folded. In contrast, OF techniques use radar reflectivity, not Doppler velocity, to track the advection of radar reflectivity features. The horizontal displacement of features retrieved with OF is used as a proxy for horizontal winds in single-Doppler wind retrieval techniques using a conservation equation for radar reflectivity to compensate for the lack of dual-Doppler information (e.g., Laroche and Zawadzki 1994; Shapiro et al. 2010). In this study, we explore the potential of using horizontal winds derived from OF to unfold Doppler velocities. The OF technique we use (based on Brox et al. 2004) and the simulation of synthetic Doppler velocities using OF winds are introduced in section 2. In section 3, we describe our OF-based unfolding technique named Two-Step Optical Flow Unfolding (TOFU). In section 4, the performance of the technique is quantified using high Nyquist velocity data, and qualitatively benchmarked against the research-grade UNRAVEL technique being currently implemented operationally in Australia (Louf et al. 2020). Conclusions of this work are presented in section 5.

2. Optical flow to produce synthetic Doppler velocities

Optical flow techniques aimed at tracking radar reflectivity features typically ingest two successive two-dimensional (2D) horizontal cross sections of gridded radar reflectivity at a given height to produce a 2D field of horizontal displacement of the detected features. From the horizontal displacement and the time difference between successive images, a displacement speed vector can be produced for each pixel of the 2D horizontal cross section. In this work, following the assumption made in single-Doppler 3D wind retrieval techniques, this 2D displacement speed vector is assumed to be a proxy for the horizontal wind vector V_H(U, V) where U and V are the eastward and northward components of the horizontal wind. The accuracy of this assumption will determine if such pseudowind information is good enough for Doppler unfolding given a Nyquist velocity V_N. In the remainder of this paper, the OF-derived proxies for 2D horizontal wind will be referred to as the “OF winds” or “OF-derived winds” to avoid confusion with the true 2D horizontal wind.

In Australia, a slightly modified version of the C++ version of the OF technique developed for computer vision by Brox et al. (2004) is used for precipitation nowcasting. For this precipitation application, a horizontal cross section of ground reflectivity is first produced at a horizontal resolution of 500 m in a horizontal domain 300 × 300 km² centered on each operational radar, using the full radar volumetric sampling and a vertical profile of reflectivity correction. The main advantage of the Brox et al. (2004) OF technique is its variational formulation, allowing for multiple constraints to be minimized at once. In short, as described in Brox et al. (2004), the technique is based on the combination of three assumptions or constraints: a nonlinear version of the reflectivity conservation equation, which is the basis of all OF techniques, a gradient constancy assumption (the horizontal gradients of matched reflectivity features are minimized), and a discontinuity-preserving spatiotemporal smoothness constraint (minimizing the first-order derivatives of the two retrieved flow components to mitigate retrieval outliers). An important aspect of the technique is the use of a multiscale approach, aimed at mitigating the risk of the minimization process to be trapped in a local minimum of the cost function, not the absolute minimum, which is a well-documented problem in variational techniques if not well managed using smoothness constraints [e.g., Laroche and Zawadzki (1994) and Protat and Zawadzki (1999) for 3D wind retrieval applications].

To use the Brox et al. (2004) OF technique for the sake of Doppler unfolding, the technique needs to produce gridded estimates of horizontal winds over the full vertical extent of the radar domain from the radar in polar geometry (volumetric scans every 5 min, including 14 elevations ranging from 0.5° to 90°, 1° azimuthal resolution, and 250-m range resolution), not just near ground level. To do so, the operational technique has simply been applied to each 2D height level of gridded radar reflectivity (hereinafter referred to as Z) data at 500-m vertical resolution. While experimenting with the technique, some specific lessons were learned about applying OF techniques in the upper levels of the troposphere. In the lower troposphere, reflectivity fields generally contain much more structures that can be tracked, even in stratiform precipitation, yielding qualitatively sound OF-derived winds, as estimated from qualitative comparisons with horizontal wind magnitude and direction inferred visually from measured Doppler velocities. Higher up, reflectivity fields are generally much smoother, except when convective cells are present, which tends to produce underestimated OF-derived horizontal wind estimates. The typical radar scanning strategy also generates much larger gaps in the upper levels, which are generally filled by an interpolation scheme (regardless of which interpolation scheme is used), resulting in smoother reflectivity structures, further hindering the tracking of reflectivity structures. Our mitigation strategy to partly overcome that problem is to only retain OF wind retrievals when radar reflectivity is greater than 20 dBZ. This 20-dBZ threshold has been determined empirically by trial and error and qualitative comparisons between OF outputs and measured Doppler velocities. To assign an OF wind estimate to all these sub-20-dBZ points, we have computed the mean OF wind in each horizontal layer from all OF winds associated with Z > 20 dBZ in that layer and assigned this mean (constant) OF wind estimate to all sub-20-dBZ points in that layer. When there are no such points with Z > 20 dBZ in a layer, we cannot compute a mean OF wind, so there are no OF wind estimates available in this layer for Doppler unfolding. This specific issue of not having OF wind estimates in areas where Doppler needs to be unfolded will be discussed further in the next section, and solutions to this problem will be proposed.

The major unknown about potential use of OF winds for Doppler unfolding is whether OF winds are a good enough proxy for true horizontal winds. Here, “good enough” means that the OF-derived mean absolute wind speed errors are almost always lower than the Nyquist velocity of 26.6 m s⁻¹, which is the lowest value used for the operational radars in Australia, as explained earlier. To gain quantitative insights into this, a full year (2021) of horizontal wind retrievals derived using a modified version of the Protat and Zawadzki (1999) multi-Doppler 3D wind retrieval technique applied to three operational radars with overlapping coverage in the Sydney region (the Sydney, Newcastle, and Wollongong radars; http://www.bom.gov.au/australia/radar/) has been produced and directly compared with the OF-derived winds. The original Protat and Zawadzki (1999) technique has been extended to include single-Doppler retrievals for grids where there is only one Doppler velocity measurement available (using OF winds as an additional constraint). Wind retrievals are produced in a Cartesian domain with a 1.5 × 1.5 × 0.5 km³ grid resolution. Only wind retrieval grids where two or more Doppler velocities are available were used for this evaluation, ensuring that the best possible reference is used for the evaluation of the OF winds. Severe weather events have been identified for the year 2021 in the Sydney region, and the most intense period of each has been selected and included in the comparisons (ranging from a few hours to a full day). This has resulted in a substantial dataset for verification.

Figure 1 shows cumulative distributions of mean absolute errors on OF wind speed and direction, and the joint distribution of OF wind speed and direction errors. The objective of Fig. 1c is to investigate if the highest errors in OF wind speed and direction happen at the same time. From the shape of this joint distribution, it appears clearly that increasingly larger errors in OF wind direction are associated with increasingly smaller errors in OF wind speed statistically, and vice versa. Figures 1a and 1b show that mean absolute OF wind speed and wind direction errors range from 2 to 5 m s⁻¹ and 15°–25° below 5-km height, respectively. Errors are found to substantially increase above 5-km height, reaching mean error values up to 13–14 m s⁻¹ and 30°–40° at 10–15-km height. Although the OF wind speed errors aloft are substantial, it is important to note again that the specific requirement for Doppler unfolding is that these errors be less than the Nyquist velocity used for the Australian operational radar network (about 26 m s⁻¹). The cumulative distribution of OF wind speed errors (Fig. 1, left) indicates that more than 99.9% of all points included in this analysis are characterized by mean absolute errors smaller than the 26 m s⁻¹ Nyquist velocity below 5-km height, which corresponds to a potential fractional error on Doppler unfolding of less than 0.1%. This is a very low fractional error, especially considering that only severe weather cases are included in the statistics. For the sake of comparisons, state-of-the-art techniques like UNRAVEL reported fractional errors of 0.1%–0.2% in similar severe weather conditions (Louf et al. 2020), albeit for smaller (more challenging) Nyquist velocities.

View Full Size

Cumulative distribution of mean absolute difference between OF and multi-Doppler retrievals of horizontal (a) wind speed and (b) direction as a function of height, and (c) joint distribution of mean absolute OF wind speed and wind direction errors. The colors in (a) and (b) show selected percentiles of the cumulative error distribution. Colors in (c) show the number of samples in each bin using a logarithmic scale instead of a linear scale, defined such that the bin with the highest number of samples has 0 dB and each 50% decrease in occurrence has a 3-dB decrease on the color scale. This figure includes 44 severe weather events in the Sydney region of Australia in 2021. The dashed profile is the number of points at each height, scaled to the maximum number of points reported in the panel (N = 38 080 114). The solid profile is the mean profile of absolute error.

Citation: Journal of Atmospheric and Oceanic Technology 40, 10; 10.1175/JTECH-D-23-0057.1

• Download Figure
• Download figure as PowerPoint slide

The potential fractional error of using OF winds for Doppler unfolding increases to about 1%–3% above 7-km height (only 97%–99% of all points are characterized by OF wind speed errors smaller than 26.6 m s⁻¹ for heights between 7 and 15 km), which makes these OF winds aloft clearly less suitable for Doppler unfolding purposes. It is important to note that a separate analysis using only points with reflectivity lower than 20 dBZ resulted in very similar error characteristics (not shown), indicating that the in-filling procedure used to produce OF winds at lower reflectivities performs equally well.

Results from Fig. 1 can also be used to investigate the potential of applying OF-based Doppler unfolding techniques to lower Nyquist velocities, or in other words to find out how high does the Nyquist velocity need to be to achieve a specified maximum fractional error. For sake of illustration, if we set a fractional error requirement for OF-based techniques of 1% or better (i.e., 99% percentile or higher in Fig. 1), Fig. 1 indicates that the minimum Nyquist velocity that can be used for heights lower than 6-km height is 19 m s⁻¹ but increases up to about 30 m s⁻¹ above 8-km height. These results also show that at the typical value of a single-PRF C-band Nyquist velocity of 13 m s⁻¹, expected fractional errors from OF-based techniques would be unacceptable for Doppler unfolding (2%–5% below 5-km height, and up to 50% aloft). This result also demonstrates that even at a higher Nyquist velocity of 26 m s⁻¹, which is our target for the Australian radar network, OF-based Doppler unfolding techniques should work well in the lower troposphere, but additional solutions are needed to mitigate potentially higher fractional errors aloft. An efficient solution to this problem is proposed and evaluated in the following sections within the TOFU framework.

3. Description of the TOFU technique

As explained in section 1, the Australian radar network includes a mixture of S-band and C-band radars, and a mixture of single-PRF and dual-PRF settings is used at different elevations to achieve a Nyquist velocity of at least 30 m s⁻¹ for the S-band radars and 26.6 m s⁻¹ for the C-band radars. As a result, the requirements for an operational Doppler unfolding technique ingesting these Doppler radar observations are the ability to handle minimum Nyquist velocities of about 26 m s⁻¹ and be minimally affected by noisy data and dual-PRF artifacts.

As discussed in the previous section, synthetic Doppler velocities in polar coordinates can be reconstructed from OF horizontal wind estimates to serve as references for Doppler unfolding. When OF winds are not available due to the unavailability of points with Z > 20 dBZ at any given height level, some measured Doppler velocities cannot be matched to a reference VD_OF. Since the objective of TOFU is to provide a solution for all measured Doppler velocities, a solution needs to be devised for these instances. The proposed solution is a two-step process:

• Step 1: Unfold all measured Doppler velocities VD_M where VD_OF is available as a first step, subtracting 2V_N to the measured Doppler velocities VD_M when VD_M − VD_OF > V_N and adding 2V_N to the measured Doppler velocities VD_M when VD_M − VD_OF < −V_N.

• Step 2: Use a vertical continuity test of Doppler velocities to unfold the points without a VD_OF reference with unfolded Doppler measurements below or above. This second step is also designed to mitigate the potential for higher error rates of OF winds in the upper troposphere (see discussion from Fig. 1).

Three important types of radar data issues need to be carefully mitigated in the design of Doppler unfolding techniques: residual ground clutter pixels, noisy Doppler measurements where the signal-to-noise ratio is low, and dual-PRF artifacts inevitably produced by the dual-PRF techniques in low signal-to-noise ratio areas and regions of high wind shear and/or turbulence (e.g., Joe and May 2003). Although great progress has been made in signal processing and Doppler quality control, it is not unusual to still have ground clutter and noisy radar gates creeping through the quality control chain, especially close to the radar. Ground clutter data points are typically characterized by zero Doppler velocity and large reflectivity, always at the same location, resulting in retrievals of very small OF wind speeds. Using such spurious ground clutter wind speeds as references in a vertical continuity scheme starting from the lowest elevation could produce unfolding errors in the low elevations propagating upward to the higher elevation scans. In the same way, unfolding errors due to very noisy measured Doppler velocities (typically, isolated points or points at the edge of reflectivity objects) or dual-PRF artifacts at one elevation would potentially propagate to other elevations with vertical continuity tests.

To prevent incorrect unfolding of ground clutter points and vertical propagation of unfolding errors due to noise, dual-PRF and ground clutter artifacts, the following additional rules have been implemented in TOFU, to great effect, as will be demonstrated in the next section:

• No Doppler unfolding is done for absolute VD_M lower than 4 m s⁻¹ if the absolute value of VD_OF is lower than 1.5V_N. In other words, we consider that such low Doppler values cannot be the result of folding unless OF winds are very strong. For V_N = 26.6 m s⁻¹, Doppler folding producing absolute velocities lower than 4 m s⁻¹ would correspond to extreme absolute Doppler velocity values in most cases (>49.2 m s⁻¹, or 177 km h⁻¹), which is quite rare. These thresholds of 4 m s⁻¹ and 1.5V_N have been determined by trial and error. Using such combination of thresholds (4 m s⁻¹ and 1.5V_N) was found to mitigate all ground clutter points that were incorrectly unfolded without this test in our high wind cases while retaining the possibility for real high Doppler velocities to be unfolded.

• The third elevation scan (1.3° or 1.4° elevation in the Australian network scanning strategy) is the reference scan from which the vertical continuity tests proceeds downward and upward, elevation by elevation. The reason is that the third elevation was found to be void of residual ground clutter points for all radars investigated. This can be adjusted to a higher elevation for radars that still exhibit ground clutter points.

• To prevent propagation of noisy data on the edges of reflectivity structures that could be incorrectly unfolded on the reference elevation scan, the vertical continuity test compares the Doppler velocities from the elevation scan located below or above the reference elevation scan with an average unfolded Doppler from the reference scan over a box of 5° azimuth × 5 km range instead of a single radar pixel. Again, the size of the averaging box has been determined by trial and error. The selected size was found to filter out the noisy Doppler data efficiently while preserving sufficient variability of the averaged Doppler field.

• Once the downward and upward vertical continuity tests initiated at the third elevation scan are completed, the elevation scan just above the third elevation scan is used to unfold any residual noisy data using the same vertical continuity process as described just above, but with the 5° azimuth × 5 km range box average performed on the Doppler velocities from the fourth elevation scan to correct Doppler velocities from the third elevation scan.

In terms of computing performance, the TOFU technique is compatible with operational use, because it takes about 2 s to process a busy volumetric scan using an average Linux computer (1 CPU and 8 GBytes of RAM).

4. Assessment of the TOFU technique

As explained earlier, our challenge is to develop a Doppler unfolding technique that can unfold residual folded Doppler velocities with a Nyquist velocity of 26 m s⁻¹, because that is the lowest value of Nyquist velocity used operationally for the C-band and S-band radars in Australia. We also want to build a technique that is as immune as possible to noisy data (we have older radars in the Australian radar network with low Doppler data quality) and dual-PRF artifacts (some older radars do exhibit a high level of dual-PRF artifacts due to noisy data). All radar data considered in this section are folded using a Nyquist velocity of 26 m s⁻¹ at all elevations. The easiest cases where all techniques are expected to perform well are stratiform precipitation cases. It is also particularly well suited to TOFU, because in our experience, the main flow is easily picked by tracking reflectivity structures embedded in stratiform precipitation. While developing the technique, we were able to confirm by visual inspection of several cases that the TOFU and UNRAVEL techniques were both performing perfectly, with no remaining folded velocities. Such easy cases are not discussed further. In this section, we instead analyze in some detail challenging case studies to investigate the robustness of the TOFU technique against specific well-known archetypes of Doppler unfolding challenges. The TOFU technique is then evaluated using high Nyquist velocity data collected recently at elevation 1.4° by our operational S-band radars, and statistically benchmarked against the more advanced UNRAVEL technique that uses the true (higher) Nyquist velocities.

a. Analysis of representative challenging case studies

As discussed previously, a major challenge for all Doppler unfolding techniques is to prevent erroneous unfolding of Doppler data due to residual ground clutter (generating spurious near zero Doppler velocities), noisy Doppler data in low signal-to-noise ratio areas, and dual-PRF artifacts. Other major challenging situations for all unfolding techniques are cases with isolated pockets of data with folded radial velocities (isolated convective cells for instance). Techniques such as UNRAVEL (Louf et al. 2020) based on unfolding from reference radials (zero radial velocities when the azimuth angle is perpendicular to the wind direction) and radial and azimuthal continuity tests can occasionally struggle with such cases, as there are generally no reference radials to start the process from, and the radial and azimuthal continuity tests will tend to let the whole pockets of folded velocities unchanged because neighboring radial velocities are consistent with each other. Such cases are also problematic for techniques like 4DD (James and Houze 2001) or R2D2 (Feldmann et al. 2020) relying on temporal continuity, as there is no easy way to find a reference volumetric scan with unfolded velocities. Doppler unfolding techniques make use of a background velocity field (VAD or numerical model wind field) to solve that problem. In contrast, TOFU seems to be very well suited to this task, as there is no strong linear assumption about the wind field like with VAD, and it does not rely on the spatial accuracy of the wind forecast either.

Most challenging cases for OF-based techniques like TOFU are expected to be those that include small-scale wind gradients associated with high azimuthal, radial, or vertical wind shear, because optical flow techniques may not be able to detect reflectivity advection associated with such small-scale dynamical features. With a Nyquist velocity of 26 m s⁻¹ though, Doppler velocity gradients would have to be very high to produce differences between measured and OF-derived Doppler velocities large enough to produce unfolding errors (26 m s⁻¹ over 500 m in range is a 5.2 10⁻² s⁻¹ shear). In general, Doppler unfolding techniques making use of radial and azimuthal continuity should perform better if areas surrounding high shear areas are well unfolded. Another situation where TOFU is expected to struggle is when the main assumption that OF displacements are a good proxy for horizontal wind breaks down. As clearly shown in Fig. 1, this is most likely to happen in the mid- to upper troposphere, where OF winds are difficult to retrieve due to the lack of reflectivity structures to track.

To investigate the performance of TOFU in these challenging situations, several cases have been selected and analyzed using S-band and C-band radar data in different parts of Australia. A short list of representative cases is used in this paper for illustrative purposes, which includes residual ground clutter in some volumetric scans, very noisy data in either isolated locations or at the edge of reflectivity structures, dual-PRF artifacts, isolated convective cells associated with extreme winds, and high Doppler gradients associated with tornadoes and supercells. The first selected case is from 26 July 2021 in the Perth region, where damaging and destructive wind thresholds have been exceeded several times, causing fatalities and widespread damage to critical infrastructure in the region. The operational C-band South Doodlakine radar covers that region. The data quality of this radar is relatively poor due to an operational long range monitoring requirement, which results in a degradation of Doppler data quality. Dual PRF with a 3:2 ratio was used during that period (1000/666 Hz), extending the Nyquist velocity to 26.6 m s⁻¹. The Doppler radar data have been unfolded in real time using the operational Doppler unfolding technique [an implementation of the 4DD technique of James and Houze (2001)], and offline using the UNRAVEL and TOFU techniques. Overall, the operational unfolding technique performed poorly on that day, due to a combination of very high Doppler velocities associated with isolated pockets of convection, and intermittently poor data quality with several instances of dual-PRF artifacts and residual noisy data not fully masked by the data quality control.

Figure 2 shows the radar scan at time 1100 UTC for elevation 1.8°. The reflectivity field (not shown) is characterized by an intense convective line southwest of the radar, with extensive widespread precipitation around. Very strong westerly winds can be inferred visually from the Doppler velocity data, generating Doppler folding for about half of the Doppler velocities measured at this elevation (Fig. 2a). Doppler velocities exceeding 40 m s⁻¹ at that time once unfolded (Figs. 2c,d), with some strong gradients associated with the convective line. Figure 2b shows the synthetic Doppler velocity reconstructed from the gridded OF wind estimates (VD_OF). As can be seen from this figure, the OF technique has accurately captured both wind speed and direction, yielding accurate unfolding of Doppler velocities when the full TOFU technique described in section 3 is used. In contrast, UNRAVEL was found to struggle with that case (Fig. 2c).

View Full Size

Case study 1: destructive wind case near Perth, 1100 UTC 26 Jul 2021, showing radar scan at elevation 1.8° of (a) folded Doppler velocity, (b) Doppler velocity reconstructed from gridded OF wind estimates, and Doppler velocity unfolded using (c) UNRAVEL and (d) TOFU.

Citation: Journal of Atmospheric and Oceanic Technology 40, 10; 10.1175/JTECH-D-23-0057.1

• Download Figure
• Download figure as PowerPoint slide

This particular elevation scan for that case exhibits all the main challenges discussed previously. A detailed analysis of all elevation scans reveals that incorrectly unfolded Doppler velocities in highlighted area A (Fig. 2c) at this elevation are due to unfolding errors associated with isolated noisy radar pixels at elevation 0.5° (not shown), propagating upward with the vertical continuity test in UNRAVEL that starts at the lowest elevation. Unfolding errors in area B are in an isolated pocket of precipitation located away from any reference, which is always challenging for methods based on radial and azimuthal continuity. This unfolding error also propagates to higher elevations (not shown). Area C (see insert in Figs. 2c,d) illustrates the challenge associated with unfolding isolated and noisy radar pixels. Again, as discussed previously, using OF winds as a reference is well suited to this task. Area D illustrates the challenges associated with the unfolding of areas that include dual-PRF artifacts. As can be seen in Fig. 2d, there are dual-PRF artifacts for that case on the northern edge of the reflectivity structure in area D. Although this clearly generates wrong Doppler velocities derived from the TOFU technique, it is important to note that these dual-PRF artifacts do not propagate to neighboring pixels. This is due to the fact that the TOFU technique does not use radial or azimuthal continuity as part of its unfolding process. In contrast, the UNRAVEL technique produces an extended area of wrongly unfolded velocities (Fig. 2c), due to the radial and azimuthal continuity constraints. Again, this pocket of spurious Doppler velocities propagates upward due to the vertical continuity test used in UNRAVEL.

Figure 3a shows folded Doppler velocities at a higher elevation scan (3.1°) for the same case and same volumetric scan to illustrate the importance of using a vertical continuity test in TOFU. The evaluation of OF winds presented in Fig. 1 clearly demonstrated that OF wind errors strongly increase for heights above 5–6 km, due to the lack of reflectivity structures to track, resulting in very light OF winds. Figure 3b presents an illustration of this challenge. For ranges beyond approximately 100 km (highlighted ellipse in Fig. 3), retrieved OF winds are unrealistically small, yielding unfolding errors (Fig. 3c) when a vertical continuity test is not used in TOFU. In contrast, the UNRAVEL technique performs well in this specific area (Fig. 3d), thanks to the radial and azimuthal continuity tests, the only issue being the vertical propagation of unfolding errors seen in Fig. 3. north from the radar (area B). When introducing the vertical continuity test in TOFU (Fig. 3e), the correct unfolding from lower elevations propagates upward and corrects the unfolding errors.

View Full Size

Case study 1: destructive wind case near Perth, 1100 UTC 26 Jul 2021, showing radar scan at elevation 3.1° of (a) folded Doppler velocity, (b) Doppler velocity reconstructed from gridded OF wind estimates, and Doppler velocity unfolded using (c) TOFU without the vertical continuity test, (d) UNRAVEL, and (e) TOFU with the vertical continuity test.

Citation: Journal of Atmospheric and Oceanic Technology 40, 10; 10.1175/JTECH-D-23-0057.1

• Download Figure
• Download figure as PowerPoint slide

Although the case presented in Figs. 2 and 3 includes strong Doppler velocity gradients associated with the convective line, we investigated several other high impact severe weather cases conducive to even stronger Doppler gradients, such as thunderstorms producing large hail and tornadoes. Overall, the TOFU technique was found to perform well. An illustration is presented in Fig. 4, which is the “Kurnell tornado” case that occurred on 15 December 2015 and was captured by the operational Terrey Hills S-band radar in Sydney. This tornado produced extensive property damage over the Kurnell peninsula, the Sydney airport, and knocked down the Sydney back-up operational radar.

View Full Size

Case study 2: the “Kurnell” tornado, 2319 UTC 15 Dec 2015, showing radar scan at elevation 0.9°of Doppler velocity unfolded using (a) UNRAVEL, and (b) TOFU; (c) A close-up on the strong azimuthal shear area associated with the tornado.

Citation: Journal of Atmospheric and Oceanic Technology 40, 10; 10.1175/JTECH-D-23-0057.1

• Download Figure
• Download figure as PowerPoint slide

As can be seen in Fig. 4c (inset), this tornado is associated with Doppler velocity differences of about 45 m s⁻¹ over a few kilometers. Despite this large Doppler velocity gradient, both branches of the circulations have been well unfolded by TOFU (Fig. 4b) and UNRAVEL (Fig. 4a). Such good performance was found for the whole life cycle of this circulation and at all elevations.

When investigating the several cases as part of this evaluation work, we have identified a specific and relatively frequent meteorological situation in the Sydney region as a challenge for TOFU. This challenging situation is associated with high low-level Doppler velocity shear in the 0–500-m vertical layer, which is captured at short range by the radar (Fig. 5 for a hailstorm case from the Terrey Hills radar). This signature was found in six major severe weather events over the sixty events analyzed over the period 18 September 2021–31 March 2022. For two of the 6 cases, this low-level shear feature caused unfolding issues such as that illustrated in Fig. 5. In both unsuccessful cases, the unfolding errors happen in areas of low reflectivity (below 20 dBZ), where the reference OF winds are the mean OF wind components derived from data points with Z > 20 dBZ. In those instances, the mean OF wind is very different from the wind associated with this high low-level Doppler velocity shear (wind direction is almost perpendicular, as schematically highlighted by the arrows in Fig. 5), resulting in erroneous unfolding of Doppler velocities that were not folded in the first place. There is no obvious solution to this problem within the TOFU framework. This is an inherent limitation of assigning to the sub-20-dBZ regions (representative of the larger-scale environmental conditions) the mean OF wind derived from regions with reflectivities greater than 20 dBZ (representative of the convective-scale dynamics). An alternative option would be to not fill these regions with mean OF winds, and therefore assume that the associated Doppler velocities do not need to be unfolded (it would work well for the case presented in Fig. 5), but this would potentially generate a large number of unfolding errors for high-impact wind events.

View Full Size

Case study 3: Sydney hailstorm, 0718 UTC 20 Dec 2018, showing radar scan at elevation 0.5° of (a) radar reflectivity (dBZ), (b) Doppler reconstructed from gridded OF winds, and Doppler velocity unfolded using (c) UNRAVEL, and (d) TOFU. The ellipse highlights an area with incorrectly unfolded Doppler velocities by TOFU. The two arrows on (c) qualitatively indicate large changes in wind direction that can be inferred from Doppler velocity.

Citation: Journal of Atmospheric and Oceanic Technology 40, 10; 10.1175/JTECH-D-23-0057.1

• Download Figure
• Download figure as PowerPoint slide

c. Statistical comparisons between UNRAVEL and TOFU

In this section, the same 6-month Sydney radar dataset as that discussed in section 4b is used to statistically benchmark TOFU against the advanced UNRAVEL technique (Figs. 6 and 7). It is important to note that UNRAVEL uses Doppler velocities derived from the true V_N (ranging from 31.4 to 47.2 m s⁻¹ depending on elevation) while TOFU uses V_N = 26 m s⁻¹ for all elevations (i.e., radar data have been folded using this Nyquist value and unfolded using TOFU). As a result, UNRAVEL is expected to outperform TOFU. On another hand, the current implementation of UNRAVEL does not use the third elevation with highest Nyquist velocity to do the vertical continuity test downward and upward but uses the lowest elevation and proceeds with a continuity test upward. Therefore, it could be prone to the propagation of spurious ground clutter Doppler velocities upward, which has been carefully mitigated in TOFU. As a result, this comparison cannot be fully considered as a validation, but merely as a very good benchmark for TOFU.

View Full Size

TOFU fractional error (using UNRAVEL Doppler unfolding results as a reference) as a function of elevation (blue profile), for data points where Doppler unfolding from UNRAVEL indicated that Doppler velocities did not need to be unfolded. The total number of points (“all points”) and the mean frequency of occurrence of differences (“failed”) are also indicated. The number of points included in the analysis at each elevation linearly decreases with increasing elevation, from 561 995 220 at elevation 0.5° to 55 155 675 at elevation 32°.

Citation: Journal of Atmospheric and Oceanic Technology 40, 10; 10.1175/JTECH-D-23-0057.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

As in Fig. 6, but for Doppler velocities that needed to be unfolded according to UNRAVEL. The minimum and maximum number of points included in the analysis are 95 578 at elevation 32° and 493 629 at elevation 1.4°, respectively.

Citation: Journal of Atmospheric and Oceanic Technology 40, 10; 10.1175/JTECH-D-23-0057.1

• Download Figure
• Download figure as PowerPoint slide

Figures 6 and 7 show the failure rate as a function of elevation for Doppler velocities that are lower in absolute value than V_N (blue), which means that they did not need to be unfolded by TOFU according to UNRAVEL, and for Doppler velocities that are higher in absolute value than V_N (red), which means that TOFU should unfold them according to UNRAVEL. When points did not need to be unfolded (Fig. 6), the failure rate of TOFU is very low for all elevations, peaking at about 0.017% for elevation 0.8°. In contrast, failure rates are about 10 times as high for points that needed to be unfolded (Fig. 7), with a peak value of 0.57% for elevation 4.7°, which is still reasonably low, but high enough to warrant further investigations presented below.

By investigating the dates and times when the higher failure rates happen in Fig. 7, it appears that differences in performance between Figs. 6 and 7 at elevations 0.5° and 0.8° are predominantly associated with radar data quality issues (dual-PRF artifacts, spurious ground clutter points, second trip echoes, and more rarely noisy Doppler estimations close to the minimum detectable signal). This source of differences between UNRAVEL and TOFU is not a concern at all, as it does not point to flaws in either of the Doppler unfolding techniques. The hope is that these spurious points will progressively disappear as our operational radar data quality control continuously improves. In contrast, two very specific conditions conducive to TOFU unfolding errors have been identified using the 6-month dataset:

• Issue 1: Two cases are characterized by an elevated cloud layer at 5–7-km height with low reflectivity and high Doppler velocities (∼30 m s⁻¹). At those heights, retrieved OF winds are very low, owing to a lack of structures to track. Since there are no data points at lower elevations, the vertical continuity test could not be used to correct these folding errors. These unfolding errors are not a major concern for severe weather monitoring and alerting since they are not associated with severe weather. However, they are a major concern for radar data assimilation. These two cases are the primary contributors to the peak failure rate observed between elevations 3.5° and 6° in Fig. 7. For these two cases, UNRAVEL correctly unfolded the Doppler velocities associated with the elevated cloud layer.

• Issue 2: Two cases are characterized by strong low-level vertical shear of horizontal winds. Optical flow has not picked this shallow but strong change in wind direction for the two cases, resulting in wrongly unfolded data at short range. An illustration of such issue was presented earlier (Fig. 5) for a hailstorm case from 2018. For one of the cases of the 6-month comparison period, the unfolding errors were associated with convection, while for the second case, they were associated with reflectivities lower than 10 dBZ, so this part of the domain had been filled with the mean optical flow that is representative of a remote area that was including convection. UNRAVEL correctly unfolded these high low-level wind shear areas.

A thorough individual inspection of the other periods of discrepancy between UNRAVEL and TOFU indicates that the primary reason for discrepancies is the presence of spurious radar data points not removed by the operational quality control in small pockets of data, resulting in either of the two techniques wrongly unfolding these spurious radar data. In general, UNRAVEL was found to be more sensitive to such spurious radar data, due to the azimuthal and radial continuity tests spreading the influence of spurious data to a larger area (as in area D of Fig. 2).

The TOFU failure rate has also been binned as a function of radar range and radar reflectivity to investigate whether discrepancies between TOFU and UNRAVEL were associated with systematic features (not shown). As expected from the discussion just above, peak failure rates were found at ranges shorter than 20 km (associated with the two low-level wind shear cases), and for reflectivities lower than 10 dBZ (associated with the elevated cloud layers) and greater than 45 dBZ (associated with ground clutter points and spurious data points).

5. Conclusions

The main objective of this work was to assess the potential of horizontal wind estimates derived from optical flow (OF) to unfold Doppler velocities. The evaluation of OF-derived winds against multi-Doppler winds indicated that an OF-based Doppler unfolding technique would require Nyquist velocities of at least 25 m s⁻¹ to produce acceptable unfolding failure rates, and that a vertical continuity test was needed as part of such technique to mitigate issues with OF wind underestimations at higher elevation angles. This set the challenge for the development of our TOFU Doppler unfolding technique: it should be capable of unfolding Doppler velocities collected with a Nyquist velocity of about 25 m s⁻¹. This number is consistent with the current settings used for the Australian operational radar network, where the lowest Nyquist velocity currently used is about 30 m s⁻¹ for the S-band radars (using single PRF), and 26.6 m s⁻¹ for the C-band radars (using dual PRF).

The principle of the two-step TOFU technique is simple. The first step is to project gridded OF winds back into the natural polar geometry (azimuth, elevation, range) of radar observations, and use that reference to unfold Doppler velocities. A vertical continuity test is then used as a second step to leverage from the lower errors of OF winds below 5-km height to mitigate underestimated OF winds aloft. The vertical continuity test proceeds downward and upward from the third elevation to mitigate potential Doppler contamination from spurious ground clutter data points creeping through the data quality control procedures at the lowest elevations. A thorough analysis of several case studies clearly demonstrated the value of this two-step strategy.

An evaluation of TOFU was conducted using high Nyquist Doppler velocity measurements collected over a 6-month period in the Sydney region by an operational S-band radar. This 6-month period included a wide range of severe weather conditions, including high wind gusts, east coast lows, and hailstorms. The failure rate of TOFU was found to be very low (about 0.015%) for both situations where the Doppler velocities needed to be unfolded or not. TOFU was also compared statistically with the advanced UNRAVEL technique using the same period of observations. The two techniques were found to produce different results less than 0.017% of the time when Doppler velocities did not need to be unfolded, and less than 0.57% of the time when Doppler velocities needed to be unfolded, with several elevations exhibiting differences less than 0.1% of the time. A thorough analysis of dates and times when these differences happen clearly showed that the “background” differences were associated with spurious data points. However, the peaks with failure rates greater than 0.1% when Doppler velocities needed to be unfolded were found to be associated with two specific conditions: two cases with few hours of elevated cloud layers associated with high winds, and two cases with strong low-level wind shear not well retrieved by the OF technique. Most of the real unfolding errors not associated with spurious radar data were concentrated in only a few hours of only 4 days of 6 months of data, which indicates the robust performance of TOFU in a wide range of severe weather situations.

Overall, this validation work indicated that although the performance of TOFU is statistically very impressive for such a simple technique, TOFU is not fully self-sufficient, as the technique did not always perform well in areas of high low-level wind shear or elevated cloud layers associated with high wind conditions. Such areas could more efficiently be unfolded by introducing radial and azimuthal continuity tests from the existing Doppler unfolding techniques, but that would defy the purpose of having a technique that does not use folded velocities to unfold themselves. A promising avenue here would be to use TOFU as the first step of more advanced techniques such as UNRAVEL, just like some techniques do with VAD winds or a numerical weather prediction forecast. This should solve issues related to unresolved convective-scale circulations with VAD or inaccurate forecasts. This would provide the best of both worlds, with a first unfolding step that does not use Doppler as a constraint to produce an improved reference, then all the smarts of advanced Doppler unfolding techniques.

Data availability statement.

All radar data used in this paper are available online (https://www.openradar.io/operational-network).