Editorial Type:
Article
Article Type:
Research Article

A Method for Efficient Implementation of a Radial-Based Noise Power Estimator for Weather Radars

Martin Hurtado aResearch Institute in Electronics, Control and Signal Processing, UNLP-CONICET, La Plata, Argentina

Search for other papers by Martin Hurtado in
Current site
Google Scholar
PubMed Close
Online Publication:
19 Aug 2021
Print Publication:
01 Aug 2021
DOI:
https://doi.org/10.1175/JTECH-D-19-0140.1
Page(s):
1395–1398
Full access

Abstract

In a previous work, a weather radar algorithm with low computational cost was developed to estimate the background noise power from the data collected at each radial. The algorithm consists of a sequence of steps designed to identify signal-free range volumes that are subsequently used to estimate the noise power. In this paper, we derive compact, closed-form expressions to replace the numerical formulations used in the first two steps of the algorithm proposed in the original paper. The goal is to facilitate efficient implementation of the algorithm.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Martin Hurtado, martin.hurtado@ing.unlp.edu.ar

Abstract

In a previous work, a weather radar algorithm with low computational cost was developed to estimate the background noise power from the data collected at each radial. The algorithm consists of a sequence of steps designed to identify signal-free range volumes that are subsequently used to estimate the noise power. In this paper, we derive compact, closed-form expressions to replace the numerical formulations used in the first two steps of the algorithm proposed in the original paper. The goal is to facilitate efficient implementation of the algorithm.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Martin Hurtado, martin.hurtado@ing.unlp.edu.ar
Keywords: Weather radar signal processing

1. Introduction

In a paper by Ivić et al. (2013) a method for estimating the noise power from the range power profile at each radial is proposed. Because this approach is not based on spectral domain processing, it can be applied even when the number of pulses per radial is small. The resulting algorithm is efficient and robust, which makes it attractive for real-time applications.

Under the assumption that every range gate is contaminated by independent and identically distributed, additive Gaussian noise, the algorithm consists of a sequence of steps intended to detect and reject range volumes that contain echoes either from weather phenomena or clutter. The remaining signal-free volumes are used to compute the power of the background noise. All the steps operate by exploiting the statistical properties of the range power profile. The first two steps are complementary and fundamental for discarding volumes containing strong signals. While the first detects and removes range volumes with sharp power discontinuities, the second detects flat sections in the power profile. Both steps are implemented via statistical testing. In Ivić et al. (2013) the integrals involved in executing these steps were left to be evaluated numerically. However, the accuracy of the numerical integration can be sensitive to the selected method, especially when solving improper integrals. This provides an impetus for deriving the closed-form solutions to these integrals.

Without changing the nature of the algorithm or its performance, herein we propose analytical solutions to the integrals used in its two first steps. First, in section 2, we develop the expression of the probability of falsely detecting noise as point clutter. Then, in section 3, we provide closed-form solutions for the moments of the power variance. These analytical results are intended to replace their numerically evaluated counterparts proposed in Ivić et al. (2013) and avoid the inconvenience of implementing a numerical integration. This paper follows the notation defined in Ivić et al. (2013). Consequently, only a few new variables, not present in the original paper, are declared.

2. High gradient echo detection

The first step of the algorithm detects volumes contaminated with point clutter or pulsed interference. Then the kth volume is discarded if its estimated power is sufficiently larger than that of the near neighbors [i.e. if P^⁡(k)>PCT P^⁡(k−2) or P^⁡(k)>PCT P^⁡(k+2)], where the point clutter threshold (PCT) multiplier is set for a desired probability of false alarm PFA and is dependent on the number of pulses M.

The challenge herein is to determine the value of the threshold multiplier PCT. Following the analysis of the smallest-of constant false alarm rate (SO-CFAR) detector (Weiss 1982), the reference power is defined as Q=min[P^⁡(k−2),P^⁡(k+2)]. Under the assumption of white Gaussian noise of power N, the power estimator P^ in a signal-free volume follows an Erlang distribution (which is the special case of the gamma distribution) with shape parameter M and scale parameter N/M (Ivić et al. 2009). In appendix A, the probability density function of the random variable Q is derived as
fQ⁡(q)=2MN1⁡(M−1)!e−2qM/N∑m=0M−11m!⁡(qMN)M+m−1,
for q ≥ 0. Because both P^⁡(k) and Q are random variables, the probability of false alarm is obtained by integrating the probability of the event P^⁡(k)>PCT Q, conditional on a fixed value of the threshold, over the probability density function of the reference power. This results in PFA of (appendix A)
PFA=∫0∞Pr [P^⁡(k)>PCT q|q] fQ⁡(q)dq=2⁡(M−1)!∑m,n=0M−1⁡(M+m+n−1)!m!n!PCTm⁡(PCT+2)M+m+n.
We highlight that (2) is the analytical solution to the integral (A5) posted in Ivić et al. (2013). It is validated via Monte Carlo simulations using 108 trials, for different values of M. At each trial, the data of the volume under test and those for reference are synthesized under the signal-free assumption, using the white Gaussian noise model of normalized power N = 1. The analytical PFA overlaps the probability calculated from simulated data as shown in Fig. 1. The MATLAB code required to reproduce the presented result is available in the online supplemental material.
Fig. 1.
Fig. 1.

Probability PFA of falsely detecting noise as point clutter vs the point clutter threshold (PCT) multiplier. The analytical probability curves (solid lines) overlap the probability values obtained from Monte Carlo simulations (circles).

Citation: Journal of Atmospheric and Oceanic Technology 38, 8; 10.1175/JTECH-D-19-0140.1

In addition, Fig. 1 shows that the probability PFA is a smooth function of the PCT multiplier. Thus, the value of the PCT multiplier for a desired probability PFA is easily computed by piecewise linear interpolation of a set of data points generated via Eq. (2). On the other hand, Ivić et al. (2013) conduct this computation by solving the optimization for the numerically evaluated objective function [created using Eq. (A5) and denoted (A8)], via the iterative Newton method (refer to appendix C for a brief description of the algorithm). In this regard, the expression (2), derived herein, may also be used instead of (A5) to create an objective function and find the PCT value via Newton method as in Ivić et al. (2013) [likewise, (A5) may be used in place of (2) for PCT computation via interpolation as suggested here]. Table 1 summarizes the values of the PCT multiplier for M = 4, 8, 16, 32, 64 and PFA = 10−3, 10−4, 10−5, 10−6.

Table 1.

Point clutter threshold (PCT) multiplier for detecting high gradient echoes vs the probability PFA and the number of pulses M.

Table 1.

3. Flat power profile detection

The second step detects flat sections of the power profile, which are associated with potential signal-free regions. A running window of length K is applied to estimate the variance of log10P^⁡(k), named Var. Then the kth volume is assumed to contain signal if the estimated variance is larger than a certain threshold (i.e., if Var > THR), where the threshold THR is set for a desired upper-tail probability.

Similar to the former section, the challenge is to find the value of THR because of the complexity of the random variable Var. Nevertheless, Ivić et al. (2013) showed that the histogram obtained from simulations of Var matches the distribution of a gamma random variable with shape α and scale θ. By the method of moments, the parameters of the gamma distribution are related to the variable Var as follows
α=E2⁡(Var)E⁡(Var2)−E2⁡(Var),
θ=E⁡(Var2)−E2⁡(Var)E⁡(Var).
The first two moments of the variable Var were derived in Ivić et al. (2013):1
E⁡(Var)=(K−1)[E⁡(Y2)−E2⁡(Y)],
E⁡(Var2)=E⁡(Y4)⁡(K−2+1/K)+E2⁡(Y2)⁡(K2−3K+5+3/K)+E⁡(Y2)E2⁡(Y)⁡(−2K2+12K−22+12/K)+E⁡(Y3)E⁡(Y)4⁡(−K+2−1/K)+E4⁡(Y)⁡(K−1)⁡(K−2)⁡(K−3)/K,
where Y=log10P^⁡(k) is the power in logarithmic scale, as defined by Ivić et al. (2013). Equivalently, this variable can be computed as Y=ln P^⁡(k)/ln 10. Note that the random variable Y is a monotonously increasing function of P^~Erlang⁡(M,N/M). Thus, using the latter definition of Y and a change of variables, its probability density function is
fY⁡(y)=ln10⁡(M−1)!⁡(Meyln10N)M exp⁡(−Meyln10N),
which is equivalent to Eq. (B4) in Ivić et al. (2013). Then it is shown in appendix B that the nth moment of Y is
E⁡(Yn)=1lnn10∑k=0n⁡(nk)lnn−k⁡(NM)Γ⁡(k)⁡(M)Γ⁡(M),
where Γ(⋅) and Γ(k)(⋅) are the gamma function and its kth derivative, respectively. The latter is solved recursively [Sun and Qin 2017, Eq. (11)]
Γ⁡(k)⁡(M)=∑i=0k−1⁡(k−1i)Γ⁡(i)⁡(M)ψ⁡(k−1−i)⁡(M),
for k ≥ 1, where ψ(n)(⋅) is the polygamma function of order n [Abramowitz and Stegun 1972, Eq. (6.4.1)]. For k = 0, Γ(0)(⋅) = Γ(⋅). Note that Eq. (8) is the closed-form solution of the integral (B5) computed numerically in Ivić et al. (2013). Therefore, the moments generated by (8) are substituted into (5) and (6), and then these last equations into (3) and (4) to yield the following result:
α=[ψ⁡(1)⁡(M)⁡(K−1)]2ψ⁡(3)⁡(M)⁡(K−2+1/K)+[ψ⁡(1)⁡(M)]22⁡(K−1),
θ=ψ⁡(3)⁡(M)⁡(K−2+1/K)+[ψ⁡(1)⁡(M)]22⁡(K−1)ψ⁡(1)⁡(M)⁡(K−1)ln210.
Note these two parameters depend only on the number of pulses M and the window length K, but not on the noise power N. This property was discussed in Ivić et al. (2013), however not formally demonstrated.

To validate the Eqs. (10) and (11), histograms of the variable Var were computed using Monte Carlo simulations, for different values of M and K. Each histogram was generated from 104 trials of signal-free volumes contaminated with white noise of normalized power N = 1. Figure 2 shows that the distribution Γ(α,θ) accurately predicts the simulated data.

Fig. 2.
Fig. 2.

Probability density function of the power variance Var for different values of M and K. The analytical pdfs (dashed lines) overlap the histograms (solid lines) of simulated data generated with power N = 1.

Citation: Journal of Atmospheric and Oceanic Technology 38, 8; 10.1175/JTECH-D-19-0140.1

Once the distribution of the variable Var has been identified, the threshold THR is found from the upper-tail probability:
Pr[Var>THR]=∫THR∞fVar⁡(υ;α,θ)dυX¯=1−FVar⁡(THR;α,θ),
where fVar and FVar are the probability density and cumulative distribution functions of the gamma random variable with parameters α and θ. The integral equation above has no analytical solution for the threshold THR. It can be solved numerically via Newton’s iterative method; see Eq. (B7) in Ivić et al. (2013) and appendix C herein. Because this problem occurs frequently, its solution has been commonly implemented in mathematical software such as MATLAB. Table 2 summarizes the values of the threshold THR for M, K = 4, 8, 16, 32, 64, and the desired tail probability of value Pr = 10−2.
Table 2.

Threshold THR for detecting large values of power variance vs the number of pulses M and the window length K for a desired upper-tail probability of Pr = 10−2.

Table 2.

4. Discussion

The first two steps of the algorithm proposed by Ivić et al. (2013) rely on lookup tables of the detection threshold, which are later retrieved during real-time processing. These tables are precomputed based on the numerical approximation of the integrals (A5) and (B5) presented in their manuscript. Herein, these two integrals are solved analytically. Therefore, the closed-form solutions provide an alternative and more straightforward procedure to build the lookup tables.

Acknowledgments

The radar data used to test the algorithm were provided by “Secretaría de Infraestructura y Política Hídrica, Ministerio de Obras Públicas” of the Argentinean National Government and INVAP S.E. framed within the SINARAME Project. The author thanks Sebastián M. Torres for providing comments that improved the manuscript. M. Hurtado is funded by the following grants: FONCYT PICT-2017-0857, UNLP I+D 11-I-209, and CIN-CONICET PDTS-269.

APPENDIX A

Probability of False Alarm for Step 1

To simplify the notation, the estimated powers of the neighbor volumes are renamed as U=P^⁡(k−2) and V=P^⁡(k+2). Under the assumption of signal-free volumes, U and V are both random variables distributed as Erlang (M, N/M). Then the cumulative distribution function of Q is FQ(q) = FU(q) + FV(q) − FU(q)FV(q). By solving the derivative, its probability density function is fQ(q) = 2fU(q)[1 − FU(q)]. Consider that the cumulative distribution and probability density functions of the Erlang random variable are [Papoulis and Pillai 2002, Eqs. (4-37) and (4-38)]
FU⁡(q)=1−∑m=0M−11m!⁡(qMN)me−qM/N,
fU⁡(q)=MN1⁡(M−1)!⁡(qMN)M−1e−qM/N.
Then substituting (A1) and (A2) into the definition of fQ(q) results in expression (1).
The probability of false alarm is calculated under the assumption that P^⁡(k) is also an Erlang (M, N/M) random variable, which yields
PFA=∫0∞[1−FP⁡(PCT q)]fQ⁡(q)dq=∑m=0M−11m!∫0∞⁡(PCT qMN)me−PCT qM/NfQ⁡(q)dq.
After substituting s = qM/N, the integral becomes
PFA=∑m,n=0M−12PCTm⁡(M−1)!m!n!∫0∞e−s⁡(PCT+2)sM+m+n−1ds=∑m,n=0M−12PCTmΓ⁡(M+m+n)⁡(M−1)!m!n!⁡(PCT+2)M+m+n,
where the second line follows from substituting t = s(PCT + 2) and noting that the integral becomes the gamma function.

APPENDIX B

Moments of Y for Step 2

The definition of the nth moment of the random variable Y is given by
E⁡(Yn)=∫−∞∞ynfY⁡(y) dy.
Using expression (7) of the probability density function of Y and substituting u = exp(yln10)M/N
E⁡(Yn)=1Γ⁡(M)∫0∞[ln⁡(Nu/M)ln⁡(10)]nuM−1e−udu=1lnn⁡(10)Γ⁡(M)∫0∞[ln⁡(N/M)+ln u]nuM−1e−udu=lnn⁡(N/M)lnn⁡(10)Γ⁡(M)∫0∞[1+ln uln⁡(N/M)]nuM−1e−udu.
Recalling that the binomial expansion identity is
⁡(1+x)n=∑k=0n⁡(nk)xk.
Then the moment expression becomes
E⁡(Yn)=1lnn⁡(10)Γ⁡(M)∑k=0n⁡(nk)lnn−k⁡(N/M)×∫0∞⁡(ln u)kuM−1e−udu,
where the integral above is the kth derivative of the gamma function Γ(k)(M) [Sun and Qin 2017, Eq. (3)], which is solved recursively in Eq. (9).

APPENDIX C

Newton’s Method

The Newton’s method, also referred to as the Newton–Raphson method, is an iterative procedure for solving a nonlinear equation f(x) = 0 (Ypma 1995). Given that f is differentiable, it can be locally approximated by a linear model. The root of the approximation is easily computed producing the updated equation:
xi+1=xi−f⁡(xi)f′⁡(xi),
where the first derivative can be approximated by finite differences:
f′⁡(xi)=f⁡(xi)−f⁡(xi−1)xi−xi−1.
Provided that the initial iterate x0 is reasonably close to the true root, the sequence xi will converge to the unique solution.
As an example of usage, the threshold THR can be determined by injecting Eq. (12) into the nonlinear function
f⁡(x)=∫x∞fVar⁡(υ;α,θ)dυ−Pr,
where Pr = 10−2 is the desired tail probability.

REFERENCES

  • Abramowitz, M., and I. A. Stegun, Eds., 1972: Handbook of Mathematical Functions with Formulas, Graphs and Mathematical Tables. 9th ed. Dover Publications, 1046 pp.

    • Search Google Scholar
    • Export Citation

  • Ivić, I. R., D. S. Zrnić, and T.-Y. Yu, 2009: The use of coherency to improve signal detection in dual-polarization weather radars. J. Atmos. Oceanic Technol., 26, 2474–2487, https://doi.org/10.1175/2009JTECHA1154.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ivić, I. R., C. Curtis, and S. M. Torres, 2013: Radial-based noise power estimation for weather radars. J. Atmos. Oceanic Technol., 30, 2737–2753, https://doi.org/10.1175/JTECH-D-13-00008.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Papoulis, A., and S. U. Pillai, 2002: Probability, Random Variables and Stochastic Processes. 4th ed. McGraw-Hill, 852 pp.

  • Sun, Z., and H. Qin, 2017: Some results on the derivatives of the gamma and incomplete gamma function for non-positive integers. IAENG Int. J. Appl. Math., 47, 265–270, http://www.iaeng.org/IJAM/issues_v47/issue_3/IJAM_47_3_04.pdf.

    • Search Google Scholar
    • Export Citation

  • Weiss, M., 1982: Analysis of some modified cell-averaging CFAR processors in multiple-target situations. IEEE Trans. Aerosp. Electron. Syst., 18, 102–114, https://doi.org/10.1109/TAES.1982.309210.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ypma, T. J., 1995: Historical development of the Newton–Raphson method. SIAM Rev., 37, 531–551, https://doi.org/10.1137/1037125.

1

Equations (5) and (6) correspond to (B2) and (B3) in Ivić et al. (2013); however, a typo in (B3) has been corrected in (6).

Supplementary Materials

Save
Cover Journal of Atmospheric and Oceanic Technology
Journal of Atmospheric and Oceanic Technology

  • Abramowitz, M., and I. A. Stegun, Eds., 1972: Handbook of Mathematical Functions with Formulas, Graphs and Mathematical Tables. 9th ed. Dover Publications, 1046 pp.

    • Search Google Scholar
    • Export Citation

  • Ivić, I. R., D. S. Zrnić, and T.-Y. Yu, 2009: The use of coherency to improve signal detection in dual-polarization weather radars. J. Atmos. Oceanic Technol., 26, 2474–2487, https://doi.org/10.1175/2009JTECHA1154.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ivić, I. R., C. Curtis, and S. M. Torres, 2013: Radial-based noise power estimation for weather radars. J. Atmos. Oceanic Technol., 30, 2737–2753, https://doi.org/10.1175/JTECH-D-13-00008.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Papoulis, A., and S. U. Pillai, 2002: Probability, Random Variables and Stochastic Processes. 4th ed. McGraw-Hill, 852 pp.

  • Sun, Z., and H. Qin, 2017: Some results on the derivatives of the gamma and incomplete gamma function for non-positive integers. IAENG Int. J. Appl. Math., 47, 265–270, http://www.iaeng.org/IJAM/issues_v47/issue_3/IJAM_47_3_04.pdf.

    • Search Google Scholar
    • Export Citation

  • Weiss, M., 1982: Analysis of some modified cell-averaging CFAR processors in multiple-target situations. IEEE Trans. Aerosp. Electron. Syst., 18, 102–114, https://doi.org/10.1109/TAES.1982.309210.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ypma, T. J., 1995: Historical development of the Newton–Raphson method. SIAM Rev., 37, 531–551, https://doi.org/10.1137/1037125.

  • <sc>Fig</sc>. 1.
    View in gallery
    Fig. 1.

    Probability PFA of falsely detecting noise as point clutter vs the point clutter threshold (PCT) multiplier. The analytical probability curves (solid lines) overlap the probability values obtained from Monte Carlo simulations (circles).

  • <sc>Fig</sc>. 2.
    View in gallery
    Fig. 2.

    Probability density function of the power variance Var for different values of M and K. The analytical pdfs (dashed lines) overlap the histograms (solid lines) of simulated data generated with power N = 1.

Fig. 1.

Probability PFA of falsely detecting noise as point clutter vs the point clutter threshold (PCT) multiplier. The analytical probability curves (solid lines) overlap the probability values obtained from Monte Carlo simulations (circles).
Fig. 2.

Probability density function of the power variance Var for different values of M and K. The analytical pdfs (dashed lines) overlap the histograms (solid lines) of simulated data generated with power N = 1.
All Time Past Year Past 30 Days
Abstract Views 119 0 0
Full Text Views 250 94 5
PDF Downloads 197 64 3