• Achtemeier, G. L., 1991: The use of insects as tracers for “clear-air” boundary-layer studies by Doppler radar. J. Atmos. Oceanic Technol., 8, 746765, https://doi.org/10.1175/1520-0426(1991)008<0746:TUOIAT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Aralimarad, P., A. M. Reynolds, K. S. Lim, D. R. Reynolds, and J. W. Chapman, 2011: Flight altitude selection increases orientation performance in high-flying nocturnal insect migrants. Anim. Behav., 82, 12211225, https://doi.org/10.1016/j.anbehav.2011.09.013.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Browning, K. A., J. C. Nicol, J. H. Marsham, P. Rogberg, and E. G. Norton, 2011: Layers of insect echoes near a thunderstorm and implications for the interpretation of radar data in terms of airflow. Quart. J. Roy. Meteor. Soc., 137, 723735, https://doi.org/10.1002/qj.800.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chapman, J. W., D. R. Reynolds, H. Mouritsen, J. K. Hill, J. R. Riley, D. Sivell, A. D. Smith, and I. P. Woiwod, 2008: Wind selection and drift compensation optimize migratory pathways in a high-flying moth. Curr. Biol., 18, 514518, https://doi.org/10.1016/j.cub.2008.02.080.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chapman, J. W., R. L. Nesbit, L. E. Burgin, D. R. Reynolds, A. D. Smith, D. R. Middleton, and J. K. Hill, 2010: Flight orientation behaviors promote optimal migration trajectories in high-flying insects. Science, 327, 682685, https://doi.org/10.1126/science.1182990.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Desroziers, G., L. Berre, B. Chapnik, and P. Poli, 2005: Diagnosis of observation, background and analysis-error statistics in observation space. Quart. J. Roy. Meteor. Soc., 131, 33853396, https://doi.org/10.1256/qj.05.108.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, V. A., 1983: Collective orientation by nocturnally migrating Australian plague locusts, Chortoicetes terminifera (Walker) (Orthoptera: Acrididae): A radar study. Bull. Entomol. Res., 73, 679692, https://doi.org/10.1017/S0007485300009287.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, V. A., 1985: Radar observations of moths migrating in a nocturnal low-level jet. Ecol. Entomol., 10, 259265, https://doi.org/10.1111/j.1365-2311.1985.tb00722.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, V. A., and R. Farrow, 1983: The nocturnal migration of the Australian plague locust, Chortoicetes terminifera (Walker) (Orthoptera: Acrididae): Quantitative radar observations of a series of northward flights. Bull. Entomol. Res., 73, 567585, https://doi.org/10.1017/S0007485300009172.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, V. A., and R. Farrow, 1988: The influence of atmospheric structure and motions on insect migration. Ann. Rev. Entomol., 33, 183210, https://doi.org/10.1146/annurev.en.33.010188.001151.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, V. A., K. Helm, J. Readshaw, and D. Reid, 1981: Insect migration across Bass Strait during spring: A radar study. Bull. Entomol. Res., 71, 449466, https://doi.org/10.1017/S0007485300008476.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fabry, F., 2010: Radial velocity measurement simulations: Common errors, approximations, or omissions and their impact on estimation accuracy. Proc. Sixth European Conf. on Radar in Meteorology and Hydrology, Sibiu, Romania, ERAD, 138–144, http://erad2010.com/pdf/oral/thursday/nwp1/02_ERAD2010_0154.pdf.

  • Gregg, P., G. Fitt, M. Coombs, and G. Henderson, 1993: Migrating moths (Lepidoptera) collected in tower-mounted light-traps in northern New South Wales, Australia: Species composition and seasonal abundance. Bull. Entomol. Res., 83, 563578, https://doi.org/10.1017/S0007485300039997.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hannesen, R., S. Kauczok, and A. Weipert, 2014: Quality of clear-air radar radial velocity data: Do insects matter? Proc. Eighth European Conf. on Radar in Meteorology and Hydrology, Garmisch-Partenkirchen, Germany, ERAD, 13.2, http://www.pa.op.dlr.de/erad2014/programme/ExtendedAbstracts/055_Hannesen.pdf.

  • Hollingsworth, A., and P. Lönnberg, 1986: The statistical structure of short-range forecast errors as determined from radiosonde data. Part I: The wind field. Tellus, 38A, 111136, https://doi.org/10.3402/tellusa.v38i2.11707.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lindskog, M., K. Salonen, H. Järvinen, and D. B. Michelson, 2004: Doppler radar wind data assimilation with HIRLAM 3DVAR. Mon. Wea. Rev., 132, 10811092, https://doi.org/10.1175/1520-0493(2004)132<1081:DRWDAW>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lorenc, A. C., and O. Hammon, 1988: Objective quality control of observations using Bayesian methods: Theory, and a practical implementation. Quart. J. Roy. Meteor. Soc., 114, 205239, https://doi.org/10.1002/qj.49711447911.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Michelson, D., R. Lewandowski, M. Szewczykowski, and H. Beekhuis, 2011: EUMETNET OPERA weather radar information model for implementation with the HDF5 file format. Version 2.1, EUMETNET Working Doc. WD_2008_03, 36 pp.

  • Montmerle, T., and C. Faccani, 2009: Mesoscale assimilation of radial velocities from Doppler radars in a preoperational framework. Mon. Wea. Rev., 137, 19391953, https://doi.org/10.1175/2008MWR2725.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ośrodka, K., J. Szturc, and A. Jurczyk, 2014: Chain of data quality algorithms for 3-D single-polarization radar reflectivity (RADVOL-QC system). Meteor. Appl., 21, 256270, https://doi.org/10.1002/met.1323.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Puri, K., and Coauthors, 2013:Implementation of the initial ACCESS numerical weather prediction system. Aust. Meteor. Oceanogr. J., 63, 265284.

  • Rennie, S. J., 2014: Common orientation and layering of migrating insects in south-eastern Australia observed with a Doppler weather radar. Meteor. Appl., 21, 218229, https://doi.org/10.1002/met.1378.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rennie, S. J., A. J. Illingworth, and S. L. Dance, 2010a: On differentiating ground clutter and insect echoes from Doppler weather radars using archived data. Atmos. Meas. Tech. Discuss., 3, 18431860, https://doi.org/10.5194/amtd-3-1843-2010.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rennie, S. J., A. J. Illingworth, S. L. Dance, and S. P. Ballard, 2010b: The accuracy of Doppler radar wind retrievals using insects as targets. Meteor. Appl., 17, 419432.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rennie, S. J., M. Curtis, J. Peter, A. W. Seed, P. J. Steinle, and G. Wen, 2015: Bayesian echo classification for Australian single-polarization weather radar with application to assimilation of radial velocity observations. J. Atmos. Oceanic Technol., 32, 13411355, https://doi.org/10.1175/JTECH-D-14-00206.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reynolds, A. M., D. R. Reynolds, A. D. Smith, and J. W. Chapman, 2010: A single wind-mediated mechanism explains high-altitude “non-goal oriented” headings and layering of nocturnally migrating insects. Proc. Roy. Soc. London, 277B, 765772, https://doi.org/10.1098/rspb.2009.1221.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rihan, F. A., C. G. Collier, S. P. Ballard, and S. Swarbrick, 2008: Assimilation of Doppler radial winds into a 3D-Var system: Errors and impact of radial velocities on the variational analysis and model forecasts. Quart. J. Roy. Meteor. Soc., 134, 17011716, https://doi.org/10.1002/qj.326.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Riley, J. R., and D. R. Reynolds, 1986: Orientation at night by high-flying insects. Insect Flight: Dispersal and Migration, W. Danthanarayana, Ed., Proceedings in Life Sciences, Springer, 71–87, https://doi.org/10.1007/978-3-642-71155-8_6.

    • Crossref
    • Export Citation
  • Salonen, K., H. Järvinen, R. Eresmaa, and S. Niemelä, 2007: Bias estimation of Doppler-radar radial-wind observations. Quart. J. Roy. Meteor. Soc., 133, 15011507, https://doi.org/10.1002/qj.114.

    • Search Google Scholar
    • Export Citation
  • Salonen, K., G. Haase, R. Eresmaa, H. Hohti, and H. Järvinen, 2011: Towards the operational use of Doppler radar radial winds in HIRLAM. Atmos. Res., 100, 190200, https://doi.org/10.1016/j.atmosres.2010.06.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Simonin, D., S. P. Ballard, and Z. Li, 2014: Doppler radar radial wind assimilation using an hourly cycling 3D-Var with a 1.5 km resolution version of the Met Office Unified Model for Nowcasting. Quart. J. Roy. Meteor. Soc., 140, 22982314, https://doi.org/10.1002/qj.2298.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Waller, J. A., D. Simonin, S. L. Dance, N. K. Nichols, and S. P. Ballard, 2016: Diagnosing observation error correlations for Doppler radar radial winds in the Met Office UKV model using observation-minus-background and observation-minus-analysis statistics. Mon. Wea. Rev., 144, 35333551, https://doi.org/10.1175/MWR-D-15-0340.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilson, J., T. M. Weckwerth, J. Vivekanandan, R. M. Wakimoto, and R. W. Russell, 1994: Boundary layer clear-air radar echoes: Origin of echoes and accuracy of derived winds. J. Atmos. Oceanic Technol., 11, 11841206, https://doi.org/10.1175/1520-0426(1994)011<1184:BLCARE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xiao, Q., and Coauthors, 2008: Doppler radar data assimilation in KMA’s operational forecasting. Bull. Amer. Meteor. Soc., 89, 3943, https://doi.org/10.1175/BAMS-89-1-39.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xu, Q., K. Nai, and L. Wei, 2007: An innovation method for estimating radar radial-velocity observation error and background wind error covariances. Quart. J. Roy. Meteor. Soc., 133, 407415, https://doi.org/10.1002/qj.21.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Locations of the nine Doppler radars that provide observations within the ACCESS-Sydney FDP domain. The outer rectangle shows the full domain, and the inner rectangle is the fixed 1.5-km-resolution region. Circles around each radar show the 100-km limit for observations. Triangular icons indicate radars and point to the radar name.

  • View in gallery

    Gridded scan statistics based on first study's dataset. (a) Gridded observation count for Wollongong 0.5° elevation scan and shallow precipitation class. This shows holes from Doppler filtering. (b) Gridded mean OB for the Wollongong scan. The large areas with large absolute values of OB indicate that this radar is strongly affected by sea clutter. (c) Gridded observation count for Brisbane 2.4° elevation for stratiform precipitation. (d) Gridded mean OB for the Brisbane scan. Ground clutter is visible near the radar and at the −90-km range to the southwest of the radar (north is the top of the figure).

  • View in gallery

    The mean speed bias (model minus observation) with standard deviation error bars for each class, radar, and elevation, for the first study. The abbreviations are defined in the text.

  • View in gallery

    The relationship between the number of observations per scan (y axis) and the percentage accepted by OPS QC (x axis), averaged per decile. The circular marker size represents the proportion of scans with that percentage of acceptance. This example is for Wollongong. Other radar results are similar. The abbreviations are defined in the text. Based on the first study’s dataset.

  • View in gallery

    The difference in speed bias for diurnal and nocturnal insect echoes. A positive value implies the nocturnal insects are faster. Differences not significant at the 99% confidence interval using a t test are marked with a cross. Derived using the first study’s dataset.

  • View in gallery

    Number of raw observations from each radar for precipitation and clear air, based on the FDP dataset. For each radar, data points are for elevations in increasing order: 0.9°, 2.4°, 4.2°, 5.6°.

  • View in gallery

    The OB mean and standard deviation for precipitation and clear air based on the FDP dataset. For each radar, data points are for elevations in increasing order: 0.9°, 2.4°, 4.2°, 5.6°.

  • View in gallery

    Speed bias mean and standard error (error bars), and speed bias standard deviation for precipitation and clear air based on the FDP dataset. For each radar, data points are for elevations in increasing order: 0.9°, 2.4°, 4.2°, 5.6°.

  • View in gallery

    As in Fig. 4, but for direction bias.

  • View in gallery

    Mean and standard deviations of superobservation innovations for precipitation and clear air based on the FDP dataset. Circles and diamonds represent values from the four elevations in increasing order: 0.9°, 2.4°, 4.2°, 5.6°. Squares represent the values for all elevations combined.

  • View in gallery

    Fit of a combined truncated Gaussian and uniform distribution to a pdf of normalized innovations from the FDP dataset. The pdf fit parameters are shown on the graph. The weight w indicates the probability of gross error, i.e., the proportion of the pdf represented by the uniform distribution.

  • View in gallery

    Innovation error derived from variance of superobservation innovations as a function of range. Radars with a beamwidth of −1.8° (thicker lines). Based on the FDP dataset. Note that for the radars with wider beamwidths, there are relatively few data points from which to derive statistics for clear air at long range.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 46 46 15
PDF Downloads 41 41 6

Assessment of Doppler Radar Radial Wind Observation Quality from Different Echo Sources for Assimilation during the Sydney 2014 Forecast Demonstration Project

View More View Less
  • 1 Australian Bureau of Meteorology, Melbourne, Victoria, Australia
© Get Permissions
Full access

Abstract

A new quality control system, primarily using a naïve Bayesian classifier, has been developed to enable the assimilation of radial velocity observations from Doppler radar. The ultimate assessment of this system is the assimilation of observations in a pseudo-operational numerical weather prediction system during the Sydney 2014 Forecast Demonstration Project. A statistical analysis of the observations assimilated during this period provides an assessment of the data quality. This will influence how observations will be assimilated in the future, and what quality control and errors are applicable. This study compares observation-minus-background statistics for radial velocities from precipitation and insect echoes. The results show that with the applied level of quality control, these echo types have comparable biases. With the latest quality control, the clear air observations of wind are apparently of similar quality to those from precipitation and are therefore suitable for use in high-resolution NWP assimilation systems.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Susan Rennie, susan.rennie@bom.gov.au

Abstract

A new quality control system, primarily using a naïve Bayesian classifier, has been developed to enable the assimilation of radial velocity observations from Doppler radar. The ultimate assessment of this system is the assimilation of observations in a pseudo-operational numerical weather prediction system during the Sydney 2014 Forecast Demonstration Project. A statistical analysis of the observations assimilated during this period provides an assessment of the data quality. This will influence how observations will be assimilated in the future, and what quality control and errors are applicable. This study compares observation-minus-background statistics for radial velocities from precipitation and insect echoes. The results show that with the applied level of quality control, these echo types have comparable biases. With the latest quality control, the clear air observations of wind are apparently of similar quality to those from precipitation and are therefore suitable for use in high-resolution NWP assimilation systems.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Susan Rennie, susan.rennie@bom.gov.au

1. Introduction

The Australian Bureau of Meteorology (BoM) is developing Doppler radar wind assimilation for high-resolution numerical weather prediction (NWP). A major challenge for assimilating radar observations is automated quality control (QC), and the evaluation of observation quality is an important stage in developing operational assimilation. A Bayesian radar echo classification system for single-polarization radar (Rennie et al. 2015) is the primary method to select observations for data assimilation or other applications. The classifier has been gauged as quite successful (>90%) at identifying precipitation but some types of clutter are identified correctly around only 50% of the time (Rennie et al. 2015). The first major test of Doppler radar wind assimilation was during the Sydney 2014 Forecast Demonstration Project (FDP), which was a 10-week real-time test of BoM’s developmental 1.5-km horizontal-resolution (NWP) system. For the FDP, further improvements to the Bayesian classification QC were sought. This paper presents an analysis of raw and assimilated observation quality before and after these improvements, for both clear air and precipitation echo sources.

Many weather services assimilate precipitation Doppler radial winds (e.g., Lindskog et al. 2004; Salonen et al. 2011; Rihan et al. 2008; Xiao et al. 2008; Montmerle and Faccani 2009), and the Indian National Centre for Medium Range Weather Forecasting (NCMRWF) also routinely assimilates clear air radial winds (J. George 2018, personal communication). However, assimilation using clear air echoes has challenges and is largely unexploited. Depending on the characteristics of the clear air echo and the available technology, these echoes can be difficult to isolate. In addition, the bias and error associated with these echoes must be understood.

This study focuses on both the minimization of unwanted echoes and the quantitative evaluation of error associated with the observations. The total observation error includes contributions from instrument error and biases, the observation operator error (see Fabry 2010), and the representativeness error. In the absence of gross errors (those that do not fit the assumed error distribution; see Lorenc and Hammon 1988), most of the observation error for radial velocity assimilation may be attributed to the representativeness error (Simonin et al. 2014). Radar observation errors that do not fit the assumed error include observations that do not represent the wind velocity, either because the echo source is moving independently of the wind [such as aerofauna (airborne animals)] or not moving (ground clutter). As the echo classification is not perfect, some unwanted echoes may be present. For example, anomalous propagation (anaprop) sea clutter is likely to contaminate the echoes classified as precipitation (Rennie et al. 2015).

The observation error (or accuracy) can be estimated statistically by comparing the observation with its equivalent model estimate, derived using a short forecast (the background). Within the constraints of individual radar coverage during a limited experimental period, it is assumed the model has a relatively uniform and constant error, and relatively low bias. In such conditions, the bulk of variability in the observation-minus-background values can be attributed to observation and representativeness errors. The background error, the observation operator error, and the representativeness error are all expected to be independent of the echo target or class of the observation (as determined by the classifier). Therefore, the differences in statistics among the various echo classes should result from class-dependent error sources. Large enough sample sizes should give a fair estimation of the relative error associated with each class.

The use of insects for wind estimation is complicated, as the accuracy of the estimate and the means of assessing this depend upon several factors. These include the type of aerofauna involved (related to the study’s geographical location), the source of “true” wind velocity measurements used for comparison, and the error margin for the intended application. Therefore, the use of clear air echo in NWP cannot be decided globally by any individual study, but it should be assessed according to a specific location and NWP system. Relevant to the use of Australian insect–derived winds for NWP is the study of insect behavior at an inland Australian radar (Rennie 2014), which showed that the nocturnal insect bloom provides a large quantity of clear air observations and did not disqualify the use of these observations.

Radar studies have shown that clear air echoes provide a view of atmospheric circulation when there is no precipitation (e.g., Achtemeier 1991; Browning et al. 2011; Drake 1985). It has also been demonstrated that in some circumstances, the insects’ velocity does not represent the wind accurately (Achtemeier 1991; Hannesen et al. 2014), although good general agreement with independent observations suggests that wind information can be derived (Rennie et al. 2010a; Wilson et al. 1994). Wilson et al. (1994) attempted to discriminate when insects are intentionally migrating long distances and thus more likely to direct their flight. The en masse directional migration of insects is often signified by the presence of common orientation (Aralimarad et al. 2011; Chapman et al. 2010, 2008; Drake 1983; Rennie 2014; Reynolds et al. 2010; Riley and Reynolds 1986). This orientation is at an angle to the true wind direction (Aralimarad et al. 2011; Rennie 2014; Reynolds et al. 2010), thus insect flight can contribute both a speed and a direction component to the velocity observation. Differences in speed and direction from comparison with independent instrument measurements (Hannesen et al. 2014) have been observed in the range of ±0–4 m s−1 and ±0°–20°. Comparisons against model data in Australia and the United Kingdom showed similar values (Rennie 2014; Rennie et al. 2010b; this study) with directional differences up to several tens of degrees. So, the short answer to the question, Can insects be used to estimate wind velocity using Doppler radar?, is “up to a point.”

This paper presents two analyses using the same methodology. The first is an analysis of observation statistics for the different echo types as determined by the Bayesian classifier as per Rennie et al. (2015): essentially an independent assessment of the classifier presented in that paper. This assessment guided how the radar quality control should be developed and which echo types should be assimilated. The second is an analysis of observation statistics from the FDP, after improvements to the quality control were made. Both analyses involved selecting precipitation- and clear air–derived observations, which were submitted to the Observation Processing System (OPS) in preparation for assimilation. The OPS cleans and processes the raw observations and is a second defense against gross errors. The OPS output can be used to assess observations quality using model-equivalent observations and other quality control information. The quality of the processed and assimilated observations is also assessed with a view to evaluating observation errors during assimilation.

2. Methods

a. The NWP system

These experiments used BoM’s NWP system, the Australian Community Climate and Earth-System Simulator (ACCESS) (Puri et al. 2013). This was run with a global domain (ACCESS-G), inside which was nested the developmental 1.5-km horizontal-resolution ACCESS-City domain, via a stretched grid. ACCESS-City was run with the Met Office Unified Model, version 8.2; observation processing with OPS 29.1; and assimilation with VAR 29.1. Sydney is the test bed for developing the 1.5-km City system with data assimilation. The Sydney domain (Fig. 1) contains a large number of radars within an extensive observation network. The NWP system cycled hourly, with three-dimensional variational data assimilation (3D-Var) using radar radial velocities, aircraft, surface, sonde, ASCAT (scatterometer winds), atmospheric motion vectors (AMV), and ATOVS radiances. Forecasts ran out to 12 or 36 h, with the longer runs every 3 h.

Fig. 1.
Fig. 1.

Locations of the nine Doppler radars that provide observations within the ACCESS-Sydney FDP domain. The outer rectangle shows the full domain, and the inner rectangle is the fixed 1.5-km-resolution region. Circles around each radar show the 100-km limit for observations. Triangular icons indicate radars and point to the radar name.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

For the first experiment, data were collected by reprocessing observations using model background files from an archived run of ACCESS-City. For the second experiment, data were collected in real time during the FDP. For the FDP, clear air winds were assimilated, whereas in the previous run only precipitation winds had been used.

b. Radar data

The radars shown in Fig. 1 and Table 1 are single-polarization S-band or C-band Doppler radars, with 1° or ~1.8° beamwidth, and 250- or 500-m range resolution. Each radar scans over 14 elevations every 6 or 10 min. The radar data undergo on-site quality control including zero-velocity ground clutter filtering and dual-PRF dealiasing before being transmitted offsite. Offsite processing includes further velocity dealiasing and echo classification, including clutter not removed by on-site filtering. The data (Australian Bureau of Meteorology 2014, unpublished data) are stored in Hierarchical Data Format (HDF5) using the OPERA (Operational Programme for the Exchange of Weather Radar Information) Data Information Model (Michelson et al. 2011).

Table 1.

Characteristics of the radars used for radar observations. ID denotes identification number.

Table 1.

The naïve Bayes classifier used to identify echo types is described in detail in Rennie et al. (2015). The classifier was trained to recognize 11 classes, including three types of precipitation (deep or shallow convective, and stratiform), clear air (insects, birds/bats, smoke, and chaff), and normal propagation and anaprop ground clutter and sea clutter. Permanent echoes, which are visible in normal propagation conditions, include ground clutter and sidelobe (SL) sea clutter. The latter refers here to echoes from signals that propagated off axis from the beam center. The classification performs worst for the least frequent echo types (Rennie et al. 2015).

For the FDP, the QC was modified to remove some echo types before applying the Bayesian classifier to the remaining echoes. “Permanent” ground and sea echo were filtered with the aid of probability of detection (POD) maps, created using a 10-dBZ threshold and 6 months’ data. The filter was applied for POD > 0.12. A sea clutter filter was restricted to overocean areas (using a land/sea mask) and a maximum range configured per radar, based on the maximum range of historically observed sea clutter. If the local (7 × 7 kernel) standard deviation of the reflectivity moment was low (below 20.0 dBZ) and the echo-top height was below 2500 m, then a bin was marked as sea clutter. Emission spikes and speckle were removed with algorithms developed for Advanced Weather Radar Network for the Baltic Sea Region (BALTRAD) (Ośrodka et al. 2014). The final effort to remove ground clutter echo, particularly from the higher elevations at Brisbane and Sydney, Australia (an issue highlighted in the first study), was to remove echo < 10 dBZ with a low velocity, with the velocity threshold decreasing with reflectivity.

Observations were selected for assimilation and monitoring based on class, elevation, range, and reflectivity, though the criteria differed for the two studies. Three times were selected for each assimilation cycle, being 0 and ±18–20 min from the analysis time, which can improve the spatial coverage. The observations were passed to the observation processing system to be prepared for assimilation.

c. Observation processing

The OPS creates model-equivalent observations (modelobs) using a 1-h forecast from the previous cycle and performs QC. The OPS output contains the observations, modelobs, and QC information, which is used for this analysis.

The OPS includes an extensive QC routine for radar velocity observations. The software and settings are very similar to Simonin et al. (2014) and more detailed information can be found therein. Raw observation filtering includes maximum (>100–120 km) and minimum (<3 km) range limits, a dealiasing check, an observation-minus-background (OB) threshold of 10 m s−1, removal of isolated pixels, and a 3 × 3 Laplace filter to reject areas with high variability. Observations that passed the initial QC are used to form superobservations, whereby the OB values (innovations) are spatially averaged to reduce their density. Superobservations will also have reduced uncorrelated error. During the construction of the superobservations (within a 3° × 3 km cell), observations exceedingly different from the cell mean are rejected. Superobservations are also rejected if the square difference of the individual observation from the cluster mean substantially exceeds the variance of the model observations in the cell. A circular cell is used (with diameter the minimum of the range and azimuthal distances) so observations outside this circle are unused. Consequently, a substantial proportion of the raw observations may be discarded by the OPS. Superobservations are thinned within a scan to a minimum distance of 6 km, and are further thinned by removing redundant, closely located observations from different scans.

d. Data collection

The OPS outputs diverse information from each cycle, including radial velocity observed and background values, class as per the classifier, OPS QC flags, and superobservation values and cell standard deviations (S. J. Rennie 2014, unpublished data). From each assimilation cycle, various diagnostics were calculated and stored, although not all are presented in this study.

The summary statistics contained a range of time series and aggregated statistics, built from each scan elevation from each radar. The time series data comprised the following:

  • the number of observations,
  • the number of QC flags of each type (which provides the number of observations accepted or rejected prior to superobservation creation),
  • and the whole-scan speed and direction bias (following the method of Salonen et al. 2007).
The speed and direction biases are calculated as model minus observation. The bias calculation errors were estimated for each scan. Their estimate involves calculating a velocity–azimuth display (VAD) for the observed and modelob scans. The uncertainty in the VAD calculations are determined following the method detailed in Rennie et al. (2010b) and summed to get the bias uncertainty. As the bias estimate depends on making a precise VAD estimate (usually requiring observations distributed around the radar), there will be times when a good bias estimate cannot be made. It was also found that the uncertainties in speed and direction estimate are not strongly correlated: a small uncertainty in one does not prevent a large uncertainty in the other. Practical uncertainty thresholds were implemented to ignore unreliable estimates. For the first study, if the errors were >5 m s−1 or >45°, then the values were disregarded because of their uncertainty. For the second study, the thresholds were >6 m s−1 or >90°.

The aggregated statistics include the following:

  • a histogram of superobservation standard deviation of innovations,
  • a histogram of OB values,
  • the mean and variance of OB values in total, which yield the OB bias (the statistics for the histogram) and
  • a spatial grid of the mean OB values (scan bias and scan count) with a resolution of 3° × 5 km.
Statistics were analyzed by class, and for raw observations, observations that passed the OPS QC, and superobservations.

Additionally, the overall probability of gross error (PGE) was estimated, purely as a means of evaluating the superobservations. A histogram of OB values indicates that the errors might reasonably be expected to be Gaussian for radial velocity. The divergence from Gaussian shape provides information about the PGE. Gross error OB values might be expected to have a uniform distribution (Lorenc and Hammon 1988) between a range of feasible values: in this case |OB| < 10 m s−1. The PGE for the superobservations (the quantity actually assimilated) can be estimated by a histogram of normalized superobservation innovations. The innovations are normalized by dividing by , where 〈εo〉 and 〈εb〉 are the expected error values for the observation and background, respectively—that is, the representativeness error and the estimated background error for each observation. The representativeness error is calculated as a function of range in the form W × (A × R + B), where W = 1, A = 0.005 555 6, B = 1.644, and R is the range in kilometers. This gives a value on the order of 2 m s−1 that increases with range. These values were derived by the Met Office (Simonin et al. 2014) and are used by default. A statistical estimation of the representativeness error (e.g., Hollingsworth and Lönnberg 1986; Xu et al. 2007) requires many data and has yet to be conducted for Australian radars. The expected background error was not determined within the OPS and must be estimated. The innovation d is the difference between the observation and background; therefore, it is also the difference between the estimated observation error εo and the background error εb. Rearranging and taking expectation gives
eq1
The normalization of each innovation uses its representativeness error and the estimated mean background error. These values are used to create a histogram of normalized innovations. A pdf f that is a linear combination of truncated Gaussian (GT) and uniform (U) distributions is fit to the histogram:
eq2
where x is the normalized innovation value; the uniform distribution is nonzero between −u and u; t1 and t2 are the lower and upper limits of the truncated Gaussian distribution, respectively; and µ and σ are its mean and standard deviation, respectively. The PGE is then found by the weight w of the uniform distribution. This distribution is fit to the normalized histograms. The fit was found to be insensitive to u, t1, and t2.

The PGE can be used by the assimilation to better handle observations that have large departures from the background. However, the objective here is to assess the quality of the observations and further use of the PGE will be left for later work.

3. Assessment of the classification system

In this section the classifier described in Rennie et al. (2015) is assessed by collecting observation statistics over a period of 40 days.

a. Dataset

This classification assessment used data from 1700 UTC 19 October 2013 to 2300 UTC 28 November 2013. The observations came from seven radars in Australia (Brisbane, Newcastle, Namoi, Sydney, Kurnell, Wollongong, and Canberra; see Fig. 1). Five elevations were selected: 0.5°, 0.9°, 1.8°, 2.4° and 4.2°. Lower elevations have a smaller vertical component to the radial velocity and stay more within the convective boundary layer. However, they are more likely to contain clutter. A range limit of 120 km was applied to the observations.

To create the dataset, the OPS was rerun using archived backgrounds from an hourly cycling assimilation run of ACCESS-City, and observations from both precipitation and clear air echoes. Note that precipitation velocity observations had been assimilated by the reference NWP suite that provided the backgrounds.

The statistics were analyzed for each of five classes: convective precipitation (con), shallow convective precipitation (shc), stratiform precipitation (str), insects (ins), and smoke (smk). A similar analysis was performed for precipitation (prp) and clear air (ca) superclasses (combinations of individual classes). Finally, the superclass analysis was repeated using only observations that were considered by the OPS to be acceptable for inclusion in superobservations. An analysis of the accepted observations demonstrates the quality of observations that would be sent to the assimilation system. Finding quality issues that are not handled by the OPS QC indicated the highest priority areas for improving the radar QC.

Note that “smoke” in this study refers to echoes classified as smoke, which may include echoes from precipitation or other clear air targets when no smoke is present.

b. Radar clutter

Each radar has clutter specific to its geographical situation, and is most visible in the gridded scan count and scan OB bias. Clutter issues differ between higher and lower elevations (Fig. 2). The scan mean OB in regions of low observation counts may also show localized biases because of a small sample size. Zero-velocity ground clutter filtering is applied at all radars, which produces “holes” in the scan count at lower observations [e.g., values near 0 (white) in Fig. 2a]. In some cases, haloes around the holes indicate that ground clutter is not completely filtered. Ground clutter affects the lower elevations of Canberra, Namoi, and Wollongong. At higher elevations, the ground clutter may be less stringently filtered by the on-site filtering, which particularly affects Sydney and Brisbane (Fig. 2c) and Newcastle. Frequent clutter contamination also manifests via a large positive speed bias (Fig. 3) for that elevation. These issues account for many features in the statistics, particularly those that differ between the radars.

Fig. 2.
Fig. 2.

Gridded scan statistics based on first study's dataset. (a) Gridded observation count for Wollongong 0.5° elevation scan and shallow precipitation class. This shows holes from Doppler filtering. (b) Gridded mean OB for the Wollongong scan. The large areas with large absolute values of OB indicate that this radar is strongly affected by sea clutter. (c) Gridded observation count for Brisbane 2.4° elevation for stratiform precipitation. (d) Gridded mean OB for the Brisbane scan. Ground clutter is visible near the radar and at the −90-km range to the southwest of the radar (north is the top of the figure).

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

Fig. 3.
Fig. 3.

The mean speed bias (model minus observation) with standard deviation error bars for each class, radar, and elevation, for the first study. The abbreviations are defined in the text.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

SL sea clutter is common to the coastal radars, and it affects the lowest elevations, more so for radars with larger beamwidth. Wollongong has severe sea clutter (Fig. 2), particularly for shallow convective and stratiform precipitation classes; these classes are also affected for Kurnell and Newcastle. At Sydney it affects the shallow convective and smoke classes. Canberra’s SL sea clutter is a major problem for the precipitation classes, because the classifier fails to detect it as a result of its unusual range (which, unlike Wollongong’s in Fig. 2a, begins beyond 50 km). This is because Canberra was not used to train the classifier, as the radar was upgraded soon after training finished. Kurnell also has anaprop sea clutter issues.

c. Individual class analysis

The analysis by class assesses the efficacy of the classifier and estimates the biases associated with each class. It also further elucidates clutter classification issues such as those noted in section 3b.

The total number of observations is not shown here, as the results in the second study are representative of the differences between radars, but the results are summarized. The insect class provides the most observations at all radars except Newcastle (convective precipitation). Insect observations are numerous because they are often present day and night. For all except Sydney and Brisbane, the insect echoes, though frequent, are of very limited range. Echoes classified as smoke were generally the fewest. Precipitation observations are split between the three precipitation classes in varying proportions. From Rennie et al. (2015) on average at least 50% of precipitation is classified as the correct subclass, with most of the remainder split between the other precipitation classes. Differences occur in the types of contaminants (Rennie et al. 2015), but the frequency of contamination is difficult to estimate without a climatological study.

Both the OB and speed bias statistics reveal issues with ground clutter, which causes observed velocities nearer to zero. As seen in Fig. 3 and noted in section 3b, higher speed biases manifest for some classes and elevations associated with greater clutter. Generally, the speed bias is between 0 and 2 m s−1, and the standard deviation is similar or slightly larger. For several radars, insects have the largest speed bias, which occurs as a result of ground clutter combining with fewer genuine observations. Insect and shallow convection classes are also more likely to include misclassified clutter. The Bayesian classifier uses echo-top height as a feature field, and these classes share characteristic low echo-top height with ground clutter and SL sea clutter. For example, often the shallow convective precipitation class has a higher speed bias than other precipitation classes.

The direction bias is not shown here. Overall, the direction bias is small with values typically between −2° and 2°. However, four radars had a 1° error in nominal azimuth, from a software error that was not discovered until much later. This was detectable in the results but had a negligible effect on other statistics.

Figure 4 shows a typical example of the number of observations per scan and the proportion of those accepted by the OPS QC. The mean number of observations per scan is shown as a function of the deciles of accepted observations. The size of the marker indicates the proportion of scans with that percentage of observations accepted. For example, in the 40%–50% decile (marker at 45%), the average number of observations per scan for shallow convective rain is 1000, and the proportion of scans with 40%–50% acceptance rate is ~10%, as indicated by the marker size key.

Fig. 4.
Fig. 4.

The relationship between the number of observations per scan (y axis) and the percentage accepted by OPS QC (x axis), averaged per decile. The circular marker size represents the proportion of scans with that percentage of acceptance. This example is for Wollongong. Other radar results are similar. The abbreviations are defined in the text. Based on the first study’s dataset.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

Typically, the insect class has the most observations per scan for any acceptance rate. For most radars the insect class modal acceptance rate (large marker size) for scans falls within 40%–70% acceptance and corresponds to 7000–10 000 observations. Smoke has a typical acceptance rate of 20%–40% when the number of observations is on the order of a few thousand.

Stratiform precipitation has an acceptance rate that often reaches 90% and a weaker correlation with the number of observations (not shown: larger spread of the number of observations per scan in each decile). Convective precipitation has a modal acceptance rate of 40%–70%, which is similar to insects. The mean number of observations per scan varies much more, though on average a lower number of observations will yield a higher acceptance rate than for insects. Shallow convective precipitation has an acceptance rate up to 80%, which is on average a little higher than convective precipitation, and usually fewer observations: <10 000 observations at 80% acceptance. Hence, it is concluded that the QC generally rejects proportionally fewer precipitation observations than clear air observations. Most observations are rejected by tests for isolated pixels and spatial variability, and compared to clear air echoes, precipitation more often has contiguous areas with consistent radial velocities, particularly stratiform precipitation.

d. Superclasses

A more realistic assessment of observations submitted to the OPS is found by creating precipitation and clear air superclasses, which would not normally be segregated further within the OPS. Convective, shallow convective, and stratiform precipitation form the precipitation class, while the clear air class comprises insects and smoke. While precipitation echoes are assimilated without concern, an assessment of the quality of clear air echoes is an essential precursor to any further work on their eventual assimilation.

With combined classes, the total number of precipitation observations is similar to or higher than the number of clear air observations (not shown; section 4 results are indicative). The speed bias for the superclasses is similar to the speed bias for individual classes: 0–2 m s−1 except for radars with severe clutter bias. The mean speed bias for clear air is fairly consistently similar to or higher than that of precipitation. The standard deviations of the speed bias are also similar for individual and superclasses: 1.5–3 m s−1. Ground clutter and SL sea clutter are the predominant sources of contamination affecting the high and low elevations, respectively. For some radars, clutter is present at some locations more than half the time. The major contaminants for the superclasses are summarized in Table 2. Even with combined classes, the scan grid biases indicated that there are not always enough data for a climatological analysis. Nevertheless, it is clear that any analysis will show clutter issues to dominate some scans, just as it does in the examples in Fig. 2.

Table 2.

The problems that most affect the superclasses with respect to each radar, as indicated by the scan grid count.

Table 2.

e. Accepted observations

The OPS applies extensive QC to remove isolated pixels, observations that vary from their neighbors, and observations that are more than 10 m s−1 from the modelob equivalent value. Therefore, a substantially reduced number of observations contributes to the statistics in this section.

After QC, permanent clutter still appears in the scan count images (not shown) for precipitation and clear air classes. For some radars the number of observations at a location with clutter is several times greater than the number of precipitation observations at locations not prone to clutter contamination. For clear air, the situation is similar; however, the frequency of insect observations (Fig. 6) means that the degree of clutter contamination at such locations is proportionally smaller. Also, only the smoke class contributes sea clutter to clear air observations. After the OPS QC, the scan bias using accepted observations is noticeably smaller than for the superclasses, with cluttered regions having a bias often 1–2 m s−1 smaller.

The mean OB (not shown) for both precipitation and clear air is usually between −0.5 and 0 m s−1, and it is often systematically related to persistent clutter bias. The OB standard deviation is uniformly around 3.3 m s−1 for precipitation and just under 3 m s−1 for clear air, in part because all |OB| > 10 m s−1 were rejected by the OPS. Clear air and precipitation biases are similar. The mean speed biases for accepted observations are slightly smaller than for the full superclasses, with the most improvement seen for Newcastle, Canberra, Namoi, and Kurnell, indicating that the QC successfully removes significant amounts of some clutter types.

The PGE was calculated for superobservations as per section 2d. For all radars, the range of weights is between 0.15 and 0.24, with a mean of 0.19. This suggests the PGE is on the order of 0.2.

f. Observations using insects

The biases for the insect class are compared with the precipitation superclass to assess the insect class for wind estimation. A consistent bias from insect flight is not seen (which would manifest from the insect speed bias being more negative in Fig. 3). Even though some common migratory insects are able to fly at several meters per second, the speed bias shows the model is faster by typically 1–2 m s−1.

Nocturnal insects are expected to be larger, stronger flyers, including moths and locusts (in inland agricultural regions of Australia) (Drake and Farrow 1983, 1988; Drake et al. 1981; Gregg et al. 1993). The speed bias of nocturnal versus diurnal insects is compared by separating the speed bias for daytime and nighttime. With sunrise and sunset times for the periods 1840–1910 and 0810–0850 UTC, respectively (local standard time is +10 h), this is achieved by selecting night times between 0800 and 1900 UTC.

The mean speed bias for each set of insect observations is compared for all radars (Fig. 5). With the exception of Namoi, nocturnal insects are significantly faster (at the 99% confidence interval) than diurnal insects by around 1 m s−1. Factors besides insect flight speed could impact these results. If the amount of ground clutter is constant, then periods with fewer insects (typically the daytime) may be proportionately more slow biased. A correlation between bias and observation numbers is readily apparent if these are plotted (not shown). Alternatively, there is often a nocturnal jet that yields high wind speeds. Some of the speed discrepancy could therefore be due to the model’s representation of the jet. So, it is possible that the difference in diurnal and nocturnal insect speeds is not purely due to the insect flight speed.

Fig. 5.
Fig. 5.

The difference in speed bias for diurnal and nocturnal insect echoes. A positive value implies the nocturnal insects are faster. Differences not significant at the 99% confidence interval using a t test are marked with a cross. Derived using the first study’s dataset.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

g. Summary

Overall, the abovementioned results show that the radar observations are biased by clutter, which becomes the focus of further QC development. The 0.5° elevation scan is typically the most cluttered, particularly with sea clutter. The difference between precipitation and clear air is comparatively small, although it seems that insect flight bias is smaller than clutter bias, so it is difficult to assess. There does not appear to be much benefit in using observations classified as smoke, as they are few and clutter affected. These results uphold the skill suggested by the analysis in Rennie et al. (2015).

4. FDP analysis

a. Experiment and dataset

The FDP ran from 29 September 2014 to 5 December 2014. Over the 10 weeks, precipitation and clear air (insect) radar observations were assimilated hourly from nine Doppler radars, with only occasional missing data periods. The outages were due to memory overload, a result of the extremely large numbers of insect echo observations from the nocturnal insect bloom during spring. One way of mitigating this issue was to limit the range of ingested observations to 100 km, rather than the 120 km originally planned. Although accepted observations were limited to 100 km, the FDP was intended to monitor observation quality to 120 km. The 100-km range is marked in Fig. 1.

The radar observations included only the 0.9°, 2.4°, 4.2°, and 5.6° elevation scans. The lowest elevation (0.5°) was excluded as being the most cluttered. During the lead up to the FDP, only these four elevations were routinely assimilated and monitored with the latest ground clutter filters. These observations had enhanced clutter removal as described in section 2b. Additionally, a 0-dB filter was applied to assimilated observations, which both reduced clutter issues and the number of observations assimilated. A 1° error in the azimuth for all radars was repaired on 7 October, a week into the FDP. This was considered negligible. Additionally, Namoi had a 1° error in azimuth throughout the FDP (the same issue noted in section 3c), which will yield a 1° bias in direction.

b. Number of observations

The total number of raw observations during the FDP period for different radars and elevations mostly ranged between 5 and 60 million (Fig. 6). Clear air usually provided more observations in the lowest elevation, whereas the number of precipitation observations was more constant with elevation. The OPS QC and superobservation procedures substantially constrain the number of observations that are assimilated. With widespread raw observations, the spatial averaging and thinning results in one-thousandth the number of superobservations per assimilation time; that is, tens of thousands of raw observations yield tens of superobservations. A radar can provide 1000–2000 superobservations at a time with abundant precipitation or clear air echo; however, 85% of the time the number was less than 500 and 60% of the time it was less than 30.

Fig. 6.
Fig. 6.

Number of raw observations from each radar for precipitation and clear air, based on the FDP dataset. For each radar, data points are for elevations in increasing order: 0.9°, 2.4°, 4.2°, 5.6°.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

c. Biases

The average OB bias for raw radial velocity observations was calculated for each radar and elevation (Fig. 7). The average is typically negative, around −0.5 to 0 m s−1 for precipitation and −0.5 to 0.5 m s−1 for clear air. Negative bias can be accounted for by clutter contamination. Surface echoes are expected to constitute the majority of contaminated observations, and the observed velocity will always be closer to zero, especially for ground clutter. Sea clutter often has a nonzero velocity that is slower than the air velocity. Insect echo classification excludes sea clutter (or any echo not over land). The standard deviation was mostly 4–5 m s−1, though higher for Wollongong, Newcastle, and Canberra. These radars are a similar type and have a greater beamwidth, so they irradiate a larger volume. This means that the observed velocity in regions of shear may not be dealiased correctly and that it suffers more from representativeness errors.

Fig. 7.
Fig. 7.

The OB mean and standard deviation for precipitation and clear air based on the FDP dataset. For each radar, data points are for elevations in increasing order: 0.9°, 2.4°, 4.2°, 5.6°.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

The speed bias mean and standard deviation are presented in Fig. 8. The average speed biases are typically between −0.5 and 2 m s−1 with standard errors around 0.05 m s−1; standard deviations are generally 2–2.5 m s−1. The speed bias, like the OB, indicates the presence of clutter. Noticeably larger is the speed bias for Brisbane’s higher elevations, which still have ground clutter issues, particularly for clear air echo. Melbourne’s higher two elevations have so few valid estimates that the means are not meaningful, particularly because the data are likely to be correlated if they are from few weather events. Clear air has a speed bias nearer to zero, probably because there are often proportionately fewer clutter-affected observations and because an insect airspeed bias would counteract the clutter bias.

Fig. 8.
Fig. 8.

Speed bias mean and standard error (error bars), and speed bias standard deviation for precipitation and clear air based on the FDP dataset. For each radar, data points are for elevations in increasing order: 0.9°, 2.4°, 4.2°, 5.6°.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

Positive direction biases indicate the modeled wind is more clockwise than the observed wind. The direction bias statistics are calculated as for circular quantities. The direction bias (Fig. 9) mean values are typically between −3° and 6°, with standard errors on average 0.6°. Melbourne is ignored for its few samples all in one quadrant of the radar scan. Namoi and Yarrawonga have a direction bias consistently around 5° (larger than the expected 1° bias Namoi should have as a result of the azimuth value error). This is interesting, as these are the only two inland radars and as such have clear air observations in all directions, which might account for the similarity between elevations. Each radar’s biases for precipitation and clear air are quite similar, though clear air has a larger variation, often being positive at low elevations and negative at high elevations. The direction bias standard deviation for precipitation varies from 11° to 29°, usually lower at higher elevations. For clear air it is more consistent at 21°–28°, except for Brisbane at 17°.

Fig. 9.
Fig. 9.

As in Fig. 4, but for direction bias.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

An exploration of why Namoi and Yarrawonga have a direction bias near 5° was made by visual comparison of the scans for the last 10 days of October with the associated bias values (not shown). The bias was not found to be correlated with the echo type or time of day but varied over time as the weather changed. This result did not suggest any value in continuing this analysis across the larger dataset. The largest variations often occurred when the wind was changing, where apparently the model mistimed the change. If, for example, a change from northerly to southwesterly is expected as a cold front passes, then the bias will go largely positive if the model is late. The most likely cause of the large direction bias for these two radars is that the model wind happened to be oriented more clockwise than the observations more often during the study period.

d. PGE and superobservation statistics

The statistics of assimilated superobservation innovations are examined to determine the accuracy of quality-controlled observations. This is important, as these are the observations that influence the forecast. The superobservation innovations (the assimilated value) are the average innovations (raw OB values) within the superobservation cell. The mean and standard deviation of the superobservation innovations are shown in Fig. 10. On average the innovations are within −0.4 and 0.2 m s−1, except for Canberra and Melbourne. The standard deviations are 3.1–4.2 m s−1 for precipitation and 2.7–3.6 m s−1 for clear air. The larger standard deviation for precipitation is likely a result of small-scale wind features around showers, where clear air is typically more spatially homogeneous because the source of such wind variation is absent. Overall, the average for each radar is nearer to 0 m s−1 for clear air, except for Canberra. This suggests that the assimilated clear air observations are not substantially biased compared to precipitation observations.

Fig. 10.
Fig. 10.

Mean and standard deviations of superobservation innovations for precipitation and clear air based on the FDP dataset. Circles and diamonds represent values from the four elevations in increasing order: 0.9°, 2.4°, 4.2°, 5.6°. Squares represent the values for all elevations combined.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

The PGE was estimated over the larger dataset of the FDP. As described in section 2d, the histogram of normalized innovations is expected to be represented by a combination of Gaussian and uniform distributions (Lorenc and Hammon 1988). A pdf of this form is fit to the histograms for precipitation and insects for each radar (Fig. 11). The weight of the uniform distribution in the pdf indicates the PGE. PGE values range from 0.1 to 0.18, and are up to 0.08 smaller than in the previous experiment in a per-radar comparison (section 3d). The mean PGEs for precipitation and clear air are 0.15 and 0.13, respectively, with clear air values uniformly lower. All values are shown in Table 3.

Fig. 11.
Fig. 11.

Fit of a combined truncated Gaussian and uniform distribution to a pdf of normalized innovations from the FDP dataset. The pdf fit parameters are shown on the graph. The weight w indicates the probability of gross error, i.e., the proportion of the pdf represented by the uniform distribution.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

Table 3.

Probability of gross error for each radar’s precipitation and clear air observations.

Table 3.

With the superobservation innovations, the “innovation error” with range may be estimated following the method in Simonin et al. (2014). The results (Fig. 12) show values similar to or larger than those found by Simonin et al. (2014), with clear air having smaller errors near the radar. While methods to estimate the observation error covariance have evolved (e.g., Waller et al. 2016), this method does indicate the larger observation error is associated with radars with broader beamwidths.

Fig. 12.
Fig. 12.

Innovation error derived from variance of superobservation innovations as a function of range. Radars with a beamwidth of −1.8° (thicker lines). Based on the FDP dataset. Note that for the radars with wider beamwidths, there are relatively few data points from which to derive statistics for clear air at long range.

Citation: Journal of Atmospheric and Oceanic Technology 35, 8; 10.1175/JTECH-D-17-0183.1

e. Comparison of precipitation and clear air

Clear air observations do not show persistent biases in speed and direction that differ substantially from those of precipitation. The OB means are also comparable. Whether precipitation or clear air has the larger bias is radar dependent; neither has a bias more consistently near zero. The most striking difference in the bias statistics is that the standard deviation of the direction bias is larger on average for clear air (Fig. 10). Also notable is that Brisbane has clutter issues in the higher elevations for clear air echo but not for precipitation (Fig. 8). Coastal radars give similar results to Yarrawonga and Namoi, despite a lack of insects over the ocean. The superobservation innovation standard deviations are smaller for clear air, but this is not unexpected, since the wind field is smoother without the presence of showers.

Exploration of several instances when the clear air direction bias was large revealed that at those times all the observations were very close to the ground (very near the radar), a characteristic that would not occur for precipitation observations. This may mean that there was near-surface shear, so the lower-level wind direction differed in the model, or that clutter near the surface skewed the observed direction. An analysis of biases ignoring observations within 20 km of the radar decreased the large divergences in direction bias, but it did not affect the direction bias at Namoi or Yarrawonga.

It must be noted that the comparison of precipitation and clear air echoes depends upon how well these are identified. It is apparent from the biases that permanent clutter removal is not perfect and that any spatial aggregation of the number of observations (notwithstanding the clear air clutter bias for Brisbane) can confirm this. However, the effect is smaller than that using the straight Bayesian classification QC of the first study. The Bayesian classification that discriminates precipitation and insects (Rennie et al. 2015) does not do this perfectly. The quantitative assessment of skill suggested that 2% of precipitation echo may be classed as insects and that 9% of insect echo may be classed as precipitation. A similar quantitative assessment against an independently classed dataset has not been done for this version of the classification scheme, because of the lack of resources and an inability to apply the new scheme retroactively on the old dataset, but Bayesian classification skill may be assumed similar for these classes. Thus, the similarity in the results for precipitation and insects can be partially explained by the cross contamination between the classes.

5. Conclusions

Two sequential studies of the quality of Doppler radar wind observations are presented. The first was intended to assess a naïve Bayesian classification system, with an intention of determining further quality control requirements for data assimilation. The second study assessed observation quality with these extra quality control measures. The studies involved similar analysis techniques.

The principal findings of the first study, which looked at five echo classes over 40 days, were as follows:

  • The classifier functioned as a useful tool to select radar observations.
  • Further quality control was needed, particularly to reduce ground clutter.
  • Insect echoes appeared to be useful for wind estimation.
  • OPS QC reduced the gross errors present, but it was not sufficient.
The second study used data derived from the 2014 Sydney Forecast Demonstration Project, which ran for 10 weeks. The observations at that time had the best quality control possible with additional clutter filtering. The improvements in the observation biases over the first study’s results were significant, with a reduction in speed bias up to 2 or 3 m s−1 for radars where the clutter was removed more effectively, and PGEs reduced by up to 0.08. However, similar to the first study, it was apparent that surface clutter is a major issue for radar observations. QC is unlikely to be substantially improved without the introduction of dual polarization.

The quality and bias of precipitation and clear air observations were comparable during the FDP. The speed biases sometimes differed substantially between the classes, particularly clear air from Brisbane, which has strong clutter contamination. The speed bias standard deviations were similar across all radars. The direction biases were similar for each class though there were differences between radars. Namoi and Yarrawonga had a higher direction bias most likely because of the weather during the trial period than any inherent bias. Clear air also had a larger direction bias standard deviation as a result of periods with limited, shallow clear air echoes, where the model did not captured the surface wind direction perfectly (or clutter has biased the direction).

The average assimilated (superobservation) innovations were similar for all radars (Canberra is possibly an exception) and were similar to the raw OB values, though the standard deviation of these innovations was smaller than for OB values. The standard deviation of clear air innovations was also smaller than for precipitation.

Overall, differences and similarities in the statistics are anticipated, considering the nature of the precipitation and clear air echoes; that is, clear air often yields a noisy but broadly uniform wind field, whereas precipitation is locally smooth but more variable on the spatial scale of showers. The radial velocities used during the FDP appear to be of fair quality with few outstanding bias issues. With the latest QC, the clear air observations are apparently of sufficient (i.e., comparable to precipitation) quality to be used for assimilation.

These findings will guide further development of radar wind assimilation in Australia, particularly the requirements for estimating updated observation errors and the dependencies for this. The next stage is to run controlled assimilation experiments to evaluate the impact of radar observations. With updated background covariances, the analysis increments should better represent observation scales, compared to those of the FDP. This will also provide the opportunity to collect observation-minus-analysis data, which can be used to develop the estimate of the observation error covariances (Desroziers et al. 2005; Waller et al. 2016).

Acknowledgments

This research/project was undertaken with the assistance of resources and services from the National Computational Infrastructure (NCI), which is supported by the Australian government. This project was funded by the Australian Government New Policy Proposal Strategic Radar Enhancement Project.

REFERENCES

  • Achtemeier, G. L., 1991: The use of insects as tracers for “clear-air” boundary-layer studies by Doppler radar. J. Atmos. Oceanic Technol., 8, 746765, https://doi.org/10.1175/1520-0426(1991)008<0746:TUOIAT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Aralimarad, P., A. M. Reynolds, K. S. Lim, D. R. Reynolds, and J. W. Chapman, 2011: Flight altitude selection increases orientation performance in high-flying nocturnal insect migrants. Anim. Behav., 82, 12211225, https://doi.org/10.1016/j.anbehav.2011.09.013.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Browning, K. A., J. C. Nicol, J. H. Marsham, P. Rogberg, and E. G. Norton, 2011: Layers of insect echoes near a thunderstorm and implications for the interpretation of radar data in terms of airflow. Quart. J. Roy. Meteor. Soc., 137, 723735, https://doi.org/10.1002/qj.800.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chapman, J. W., D. R. Reynolds, H. Mouritsen, J. K. Hill, J. R. Riley, D. Sivell, A. D. Smith, and I. P. Woiwod, 2008: Wind selection and drift compensation optimize migratory pathways in a high-flying moth. Curr. Biol., 18, 514518, https://doi.org/10.1016/j.cub.2008.02.080.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chapman, J. W., R. L. Nesbit, L. E. Burgin, D. R. Reynolds, A. D. Smith, D. R. Middleton, and J. K. Hill, 2010: Flight orientation behaviors promote optimal migration trajectories in high-flying insects. Science, 327, 682685, https://doi.org/10.1126/science.1182990.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Desroziers, G., L. Berre, B. Chapnik, and P. Poli, 2005: Diagnosis of observation, background and analysis-error statistics in observation space. Quart. J. Roy. Meteor. Soc., 131, 33853396, https://doi.org/10.1256/qj.05.108.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, V. A., 1983: Collective orientation by nocturnally migrating Australian plague locusts, Chortoicetes terminifera (Walker) (Orthoptera: Acrididae): A radar study. Bull. Entomol. Res., 73, 679692, https://doi.org/10.1017/S0007485300009287.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, V. A., 1985: Radar observations of moths migrating in a nocturnal low-level jet. Ecol. Entomol., 10, 259265, https://doi.org/10.1111/j.1365-2311.1985.tb00722.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, V. A., and R. Farrow, 1983: The nocturnal migration of the Australian plague locust, Chortoicetes terminifera (Walker) (Orthoptera: Acrididae): Quantitative radar observations of a series of northward flights. Bull. Entomol. Res., 73, 567585, https://doi.org/10.1017/S0007485300009172.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, V. A., and R. Farrow, 1988: The influence of atmospheric structure and motions on insect migration. Ann. Rev. Entomol., 33, 183210, https://doi.org/10.1146/annurev.en.33.010188.001151.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, V. A., K. Helm, J. Readshaw, and D. Reid, 1981: Insect migration across Bass Strait during spring: A radar study. Bull. Entomol. Res., 71, 449466, https://doi.org/10.1017/S0007485300008476.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fabry, F., 2010: Radial velocity measurement simulations: Common errors, approximations, or omissions and their impact on estimation accuracy. Proc. Sixth European Conf. on Radar in Meteorology and Hydrology, Sibiu, Romania, ERAD, 138–144, http://erad2010.com/pdf/oral/thursday/nwp1/02_ERAD2010_0154.pdf.

  • Gregg, P., G. Fitt, M. Coombs, and G. Henderson, 1993: Migrating moths (Lepidoptera) collected in tower-mounted light-traps in northern New South Wales, Australia: Species composition and seasonal abundance. Bull. Entomol. Res., 83, 563578, https://doi.org/10.1017/S0007485300039997.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hannesen, R., S. Kauczok, and A. Weipert, 2014: Quality of clear-air radar radial velocity data: Do insects matter? Proc. Eighth European Conf. on Radar in Meteorology and Hydrology, Garmisch-Partenkirchen, Germany, ERAD, 13.2, http://www.pa.op.dlr.de/erad2014/programme/ExtendedAbstracts/055_Hannesen.pdf.

  • Hollingsworth, A., and P. Lönnberg, 1986: The statistical structure of short-range forecast errors as determined from radiosonde data. Part I: The wind field. Tellus, 38A, 111136, https://doi.org/10.3402/tellusa.v38i2.11707.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lindskog, M., K. Salonen, H. Järvinen, and D. B. Michelson, 2004: Doppler radar wind data assimilation with HIRLAM 3DVAR. Mon. Wea. Rev., 132, 10811092, https://doi.org/10.1175/1520-0493(2004)132<1081:DRWDAW>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lorenc, A. C., and O. Hammon, 1988: Objective quality control of observations using Bayesian methods: Theory, and a practical implementation. Quart. J. Roy. Meteor. Soc., 114, 205239, https://doi.org/10.1002/qj.49711447911.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Michelson, D., R. Lewandowski, M. Szewczykowski, and H. Beekhuis, 2011: EUMETNET OPERA weather radar information model for implementation with the HDF5 file format. Version 2.1, EUMETNET Working Doc. WD_2008_03, 36 pp.

  • Montmerle, T., and C. Faccani, 2009: Mesoscale assimilation of radial velocities from Doppler radars in a preoperational framework. Mon. Wea. Rev., 137, 19391953, https://doi.org/10.1175/2008MWR2725.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ośrodka, K., J. Szturc, and A. Jurczyk, 2014: Chain of data quality algorithms for 3-D single-polarization radar reflectivity (RADVOL-QC system). Meteor. Appl., 21, 256270, https://doi.org/10.1002/met.1323.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Puri, K., and Coauthors, 2013:Implementation of the initial ACCESS numerical weather prediction system. Aust. Meteor. Oceanogr. J., 63, 265284.

  • Rennie, S. J., 2014: Common orientation and layering of migrating insects in south-eastern Australia observed with a Doppler weather radar. Meteor. Appl., 21, 218229, https://doi.org/10.1002/met.1378.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rennie, S. J., A. J. Illingworth, and S. L. Dance, 2010a: On differentiating ground clutter and insect echoes from Doppler weather radars using archived data. Atmos. Meas. Tech. Discuss., 3, 18431860, https://doi.org/10.5194/amtd-3-1843-2010.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rennie, S. J., A. J. Illingworth, S. L. Dance, and S. P. Ballard, 2010b: The accuracy of Doppler radar wind retrievals using insects as targets. Meteor. Appl., 17, 419432.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rennie, S. J., M. Curtis, J. Peter, A. W. Seed, P. J. Steinle, and G. Wen, 2015: Bayesian echo classification for Australian single-polarization weather radar with application to assimilation of radial velocity observations. J. Atmos. Oceanic Technol., 32, 13411355, https://doi.org/10.1175/JTECH-D-14-00206.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reynolds, A. M., D. R. Reynolds, A. D. Smith, and J. W. Chapman, 2010: A single wind-mediated mechanism explains high-altitude “non-goal oriented” headings and layering of nocturnally migrating insects. Proc. Roy. Soc. London, 277B, 765772, https://doi.org/10.1098/rspb.2009.1221.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rihan, F. A., C. G. Collier, S. P. Ballard, and S. Swarbrick, 2008: Assimilation of Doppler radial winds into a 3D-Var system: Errors and impact of radial velocities on the variational analysis and model forecasts. Quart. J. Roy. Meteor. Soc., 134, 17011716, https://doi.org/10.1002/qj.326.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Riley, J. R., and D. R. Reynolds, 1986: Orientation at night by high-flying insects. Insect Flight: Dispersal and Migration, W. Danthanarayana, Ed., Proceedings in Life Sciences, Springer, 71–87, https://doi.org/10.1007/978-3-642-71155-8_6.

    • Crossref
    • Export Citation
  • Salonen, K., H. Järvinen, R. Eresmaa, and S. Niemelä, 2007: Bias estimation of Doppler-radar radial-wind observations. Quart. J. Roy. Meteor. Soc., 133, 15011507, https://doi.org/10.1002/qj.114.

    • Search Google Scholar
    • Export Citation
  • Salonen, K., G. Haase, R. Eresmaa, H. Hohti, and H. Järvinen, 2011: Towards the operational use of Doppler radar radial winds in HIRLAM. Atmos. Res., 100, 190200, https://doi.org/10.1016/j.atmosres.2010.06.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Simonin, D., S. P. Ballard, and Z. Li, 2014: Doppler radar radial wind assimilation using an hourly cycling 3D-Var with a 1.5 km resolution version of the Met Office Unified Model for Nowcasting. Quart. J. Roy. Meteor. Soc., 140, 22982314, https://doi.org/10.1002/qj.2298.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Waller, J. A., D. Simonin, S. L. Dance, N. K. Nichols, and S. P. Ballard, 2016: Diagnosing observation error correlations for Doppler radar radial winds in the Met Office UKV model using observation-minus-background and observation-minus-analysis statistics. Mon. Wea. Rev., 144, 35333551, https://doi.org/10.1175/MWR-D-15-0340.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilson, J., T. M. Weckwerth, J. Vivekanandan, R. M. Wakimoto, and R. W. Russell, 1994: Boundary layer clear-air radar echoes: Origin of echoes and accuracy of derived winds. J. Atmos. Oceanic Technol., 11, 11841206, https://doi.org/10.1175/1520-0426(1994)011<1184:BLCARE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xiao, Q., and Coauthors, 2008: Doppler radar data assimilation in KMA’s operational forecasting. Bull. Amer. Meteor. Soc., 89, 3943, https://doi.org/10.1175/BAMS-89-1-39.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xu, Q., K. Nai, and L. Wei, 2007: An innovation method for estimating radar radial-velocity observation error and background wind error covariances. Quart. J. Roy. Meteor. Soc., 133, 407415, https://doi.org/10.1002/qj.21.

    • Crossref
    • Search Google Scholar
    • Export Citation
Save