Weather radars have been widely used to detect and quantify precipitation and nowcast severe weather for more than 50 years. Operational weather radars generate huge three-dimensional datasets that can accumulate to terabytes per day. So it is essential to review what can be done with existing vast amounts of data, and how we should manage the present datasets for the future climatologists. All weather radars provide the reflectivity factor, and this is the main parameter to be archived. Saving reflectivity as volumetric data in the original spherical coordinates allows for studies of the three-dimensional structure of precipitation, which can be applied to understand a number of processes, for example, analyzing hail or thunderstorm modes. Doppler velocity and polarimetric moments also have numerous applications for climate studies, for example, quality improvement of reflectivity and rain rate retrievals, and for interrogating microphysical and dynamical processes. However, observational data alone are not useful if they are not accompanied by sufficient metadata. Since the lifetime of a radar ranges between 10 and 20 years, instruments are typically replaced or upgraded during climatologically relevant time periods. As a result, present metadata often do not apply to past data. This paper outlines the work of the Radar Task Team set by the Atmospheric Observation Panel for Climate (AOPC) and summarizes results from a recent survey on the existence and availability of long time series. We also provide recommendations for archiving current and future data and examples of climatological studies in which radar data have already been used.
Old measurements are precious: once lost, they cannot be replaced. But without carefully saved information of how the data were measured, we also create a risk of false conclusions.
Over the last century, weather radars have been widely used to detect and quantify precipitation and severe weather. Issuing warnings of severe hail, tornadoes, blizzards, and flooding has greatly benefited from high-resolution data from the radar networks. These networks are now covering a majority of the densely populated areas of the world (Fig. 1). Now that some of the networks have collected data for up to 50 years, scientists have started to use the rich information for climate studies. Unfortunately, a lot of the older radar data have been lost, and not all of them are archived even today. The objective of this paper is, on one hand, to provide an overview about weather radar information that is already available to the climate science community, and on the other hand, to make the radar community aware of the needs of the climate community. Possibilities to improve historical datasets are limited, but the data we archive today are part of the data to be used by future generations.
Weather radars have been used to detect precipitation since the 1940s and 1950s. The first displays were analog, and the image existed only as long as the fluorescent tube was glowing (unless it was saved as a photograph). During the following decades, information technology has been developed, and several weather services have started archiving data and providing the data to the research community; however, most of the archiving procedures were and still are not standardized. For example, some save the data in the original format defined by the radar software manufacturer while others only save the final products such as precipitation intensity maps. Furthermore, upgrading the radar operating software usually introduces changes in the data format. Very few radar operators have executed a reanalysis to create a homogeneous time series. While radar data have primarily been used by radar experts in the past, more and more other research fields are now using radar data. For instance, weather radar data from several data providers are now assimilated into weather forecasting and nowcasting models. As a result, this raises the importance of common data formats and well-documented metadata, for example, among the World Meteorological Organization (WMO).
The Global Climate Observing System (GCOS), a program co-sponsored by the WMO, the Intergovernmental Oceanographic Commission of the United Nations Educational, Scientific and Cultural Organization (IOC-UNESCO), the United Nations Environment Programme (UN Environment), and the International Science Council (ISC), regularly assesses the status of global climate observations and produces guidance for its improvement. GCOS’s vision is for climate observations to be accurate and sustained, and for all users to have free and open access to the climate data they need to address climate-related questions. The Atmospheric Observation Panel for Climate (AOPC) is one of the three scientific panels of GCOS, together with the Ocean Observations Panel for Climate and the Terrestrial Observation Panel for Climate. It was established by the GCOS Steering Committee in recognition of the need for specific scientific and technical input concerning atmospheric observations for climate.
With the increased usage of radar data around the world for diverse applications and increasing demand for archiving historic, current, and future radar data, the AOPC formed a Task Team on Radar Observations for Climate Applications (TTROCA, referred to as Radar Task Team hereafter) to define procedures to use radar data for climate applications. As a first step, the Radar Task Team conducted a survey to determine what operational radar data exist and how weather services have archived data in the past. According to a survey, some radar data have been archived since the 1970s, for instance, in Hong Kong, while the majority of weather services (>35) started archiving around the year 2000 (Fig. 2). Therefore, it is becoming increasingly relevant to review what can be done with existing data and how to manage the present datasets to ensure future climatologists can utilize the full capabilities of the material, which is the goal of the Radar Task Team.
This paper summarizes the work of the Radar Task Team and is organized as follows: we first analyze what data exist, how these data have been used, and then conclude with recommendations for processing historical data and archiving new data. Discussion of these topics concentrates on operational radar networks, most often operated by national meteorological services, and it is based on work conducted by the Radar Task Team of the AOPC.
DECADES OF DATA ARE ALREADY AVAILABLE, BUT THEY ARE OFTEN PATCHY AND HETEROGENEOUS.
The first impulse of climate experts invited to the Radar Task Team—which was soon proven to be wrong—was that radar data volumes are too large to be archived, and that the radar operators are probably not willing to share their archives. The Radar Task Team executed a survey in 2017 to get an estimate of existing radar archives. The invitation to the survey was sent to contact points in different countries suggested by the relevant WMO task teams. To ensure a good response rate, the questions were kept short and straightforward with multiple choice. After the survey the team contacted those countries with the longest archives and asked additional questions.
Out of the 91 weather services which operate weather radars based on the WMO’s radar database (WMO 2019), experts from 45 national weather services replied to this survey. While all these 45 nations currently maintain a radar archive, with 40 of them having more than 10 years of data archived already, only 20 countries have archives that date back to 1995 and earlier (Fig. 2).
According to the interviews, data from the 1970s to 1990s are often inhomogeneous and contain large gaps. Continuous time series of a reasonable quality typically start around 1998, so 20 years of data can already be used from these 20 countries. However, even during this period, radar locations have changed, the coverage has increased, and processing methods have improved, so the time series are by no means homogeneous.
Results from the survey also show that 90% of the archives have saved radar reflectivity (in the unit of dBZ) in native polar coordinates (“Level 2 data,” see sidebar).
In addition to inquiring size and quality of archives, two questions were asked about access to data: “Is the data available for researchers outside of your institute?” and “How would you describe the effort of retrieving one year of old data from your archives?” To the first question, 15 national weather services selected “Yes, and they use it regularly,” and 19 selected “Probably yes, but it may be complicated.” Only 7 replied “No,” while 4 selected “I do not know.” The weather services who replied positively are listed in Table 1. Six of these mentioned they have a graphical interface for selecting the data, while according to most of the other replies the process required programmatic interaction with data as opposed to a graphical interface.
A list of hydrometeorological or meteorological services and other authorities that operate weather radars, with links to their official websites, is provided in the WMO Radar Database (WMO 2019).
EXAMPLES OF EXISTING LARGE ARCHIVES AND PORTALS.
In the United States, the operational implementation of weather radars in the National Weather Service (formerly the U.S. Weather Bureau) began in 1959 with 31 S-band radars installed over a period of 3 years. Fourteen more radars were commissioned in the late 1960s. These radars provided reflectivity data only and were recorded by 35-mm camera and/or Polaroid. The National Climatic Data Center archived thousands of meters of film which exist on microfilm today (Whiton et al. 1998). In the late 1980s and early 1990s, the U.S. Department of Commerce, U.S. Department of Defense, and U.S. Department of Transportation initiated the ground-based Doppler weather radar systems commonly known as the Next-Generation Weather Radar (NEXRAD). Single polarization S-band Doppler weather radars [Weather Surveillance Radar-1988 Doppler (WSR-88D)] were commissioned in the early 1990s until 1997, replacing the older S-band radars, and installed at weather forecast offices and major airports. New to the WSR-88D implementation were the processors and software which encompassed the processing of the microwave signal into product and then to dissemination (Klazura and Imy 1993). Initially, the Level II data (3D reflectivity, mean radial velocity, and spectrum width) were stored on 8-mm tapes and shipped to the National Climatic Data Center for archiving. In the early 2000s a project to transform the dissemination method of these data was started to move data from the 8-mm tapes to a robotic tape storage system, and to broadcast data from the radar site to central storage. Data would soon be available to the public via an ordering system and direct data download (Kelleher et al. 2007). Data volumes have continued to increase due to the increased data resolution in 2008 and the dual-polarization network upgrades in 2012 (Istok et al. 2009). Recently in 2017, the NEXRAD archive was copied to various cloud computing platforms (Amazon Web Services, Google, IBM, Microsoft, and the Open Commons Consortium) under the Big Data Partnership (Ansari et al. 2018).
The Australian weather radar archive consists of 74 sites, with 54 of these sites currently in operation. Radar hardware across the current network is diverse, including both C and S band radars in several configurations, and this diversity increases when historical radar data are considered. Routine procedures to archive volumetric data at operations centers were established in 1997 using physical media (e.g., floppy disks and CDs, later mass storage devices). As a result, several partially overlapping archives exist that use different implementations of the Australian-specific RAPIC data model. This dataset is made available but data requests are subject to a charge for cost recovery of handling procedures. Compounding limitations due to data request costs, complex archives and a nonstandard data model have limited the applications of radar data, in particular for non-expert users.
An initiative to improve the availability and accessibility of Australian weather radar data was coordinated by Monash University, the Australian Bureau of Meteorology, and the National Computational Infrastructure (NCI) in early 2018. They developed a single archive of volumetric Australian weather radar data in a standard format and provide unrestricted access for research use at no cost. Careful quality control for corruption and handling of missing meta data were required before conversion to the Opera Data Information Model (ODIM) in the HDF5 data format, introduced by the EUMETNET radar program OPERA (Huuskonen et al. 2014). This dataset has already supported new research in meteorology and ecology, and education through the 2018 Australian radar training school. In the near future additional radar datasets will be released that provide improved data quality (e.g., satellite corrected reflectivity, dealiased Doppler velocity) and derived datasets (e.g., hail size, rainfall, azimuthal shear).
In Europe, the OPERA data center Odyssey archives European composites of rain rate and precipitation accumulation as well as maximum reflectivity since 2012. These are Cartesian composites of 120–160 radars. They are available for research and education. In addition to OPERA, several national meteorological services have included radar data in their open data portal. Examples of such portals are given in Table 2. Typically, data centers have their own “standard” of data model and data format. In some cases, the data model and format is not consistent across all the years and all the radars in the same archive. An example of a rather large database is the Météo-France radar rainfall reanalysis product Comephore (Tabary et al. 2012), which is, however, not open access but available for a fee. The Royal Netherlands Meteorological Institute (KNMI) in the Netherlands has a portal with volume data and product data from their two radars, all in KNMI-specific HDF5 format and NetCDF4 with conversion tools to image formats. They have plans to move to ODIM HDF5 format, but this will be a long and slow process. The German Weather Service (DWD) provides reprocessed national composites from a routine reanalysis including up to 17 radars with a temporal resolution of 1 h and 5 min at a spatial resolution of 1 km × 1 km, currently covering the time period from 2001 to 2017 in binary and ASCII format (Winterrath et al. 2018). This climatological radar dataset will be updated and made available on a yearly basis. At https://opendata.dwd.de/weather/radar/sites/, operational reflectivity data in the original polar coordinates are available—with open access—every 5 min for the past 2 days, including volume data as well as a set of products in different formats (DWD-specific formats, BUFR sweeps, and HDF5). Volumetric data are also archived at DWD, but are not freely available for historical periods.
NOT ALL DATA ARE AVAILABLE AND FREE TO USE.
Access to radar data archives is, unfortunately, often limited by national data policies defined by the national government and is beyond the control of individual weather services or the WMO. Various data access models can be found around the world ranging from open-access to real-time and archived data through a web portal (e.g., United States), open access of archived products and limited open access of real-time data (e.g., Germany), free access of radar data for scientific purposes processed by request (e.g., Odyssey), and purchasing of archived radar data for both commercial and scientific purposes. Given that taxpayers fund operational weather radars in most countries, there is an underlying commitment to increase the accessibility of this data back to the taxpayer through either open access or cost-recovery services. The Radar Task Team recognizes that open access of weather radar data for research increases the value of the dataset and promotes faster development of diverse applications (e.g., in ecology) and expertise of radar users.
In the European Union (EU), the INSPIRE (Infrastructure for Spatial Information in Europe) directive requires EU member states to share 34 different types of spatial data, from land use classes to weather data, through a network of “services.” Full implementation is required by 2021. INSPIRE itself does not require the data to be free, just that it is made available, but some countries have combined the efforts for availability and open data. When asked in 2015, 11 of the 18 meteorological services had not yet made a decision whether they will make the weather radar data also openly available (Saltikoff 2015).
Harmonization, documentation, and making radar data available is usually triggered by exchange of real-time data. International exchange of Level 2 data (such as in CfRadial2 format) has been encouraged by the WMO Interprogramme Expert Team on Operational Weather Radars (IPET-OWR). While in Europe the bilateral data exchange between national weather services has been operational for more than 20 years (Huuskonen et al. 2014), Asia has only recently started similar exchange projects. For example, Japan, Malaysia, and Thailand are exchanging composites under the umbrella of the WMO Typhoon Committee since 2017 (Kakihara 2018).
RADAR DATA HAVE ALREADY BEEN USED IN CLIMATE STUDIES.
Time series of radar observations that are longer than 10 years have already been used in a number of climatological studies. Compared to rain gauges, the added value of radar for climate monitoring is particularly high for convective precipitation with high spatial and temporal variability, which is typically not captured by conventional rain gauge networks. In radar climatology, the large spatial coverage and resolution of radar observations might allow us to partly make up for short time series by “trading space for time,” that is, by retrieving larger samples of relevant events under the—admittedly strong—assumption that these can be considered as independent.
Out of the numerous studies available, the next section provides a few examples on how radar data have been used for various climatology studies around the world. This list is not comprehensive; instead, the reader is referred to the GCOS report 223 (GCOS 2019) for a summary of studies in Europe. Thorndahl et al. (2017) also list a number of datasets created for climate studies in different geographical areas. They have identified the need to validate the spatial distribution of rainfall derived from regional climate models to understand subdaily extreme precipitation at high spatial resolution.
One of the longest radar-based precipitation climatologies currently available was published by Marra and Morin (2015) and contains 23 years of radar observations from two radars in Israel. The authors applied these radar data in order to derive intensity–duration–frequency curves in different climatic regimes. For the reanalysis process, the authors checked antenna pointing accuracy; corrected for effects of ground clutter, attenuation due to a wet radome and along the radar beam, partial beam blockage, and vertical profile of reflectivity; and limited the maximum values to avoid overestimation due to hail (note that most of these corrections require the data to be available in spherical coordinates).
Devasthale and Norin (2014) used 11 years of data from 12 radars in Sweden to calculate monthly probability distribution functions for precipitation using radar pixels at 5–80-km range from the radar. Even with less than perfect data availability and values discarded in quality control, they calculated margin of error to be only 1.6% of the monthly precipitation.
In our experience, the added value of radar for climate monitoring is particularly high for convective precipitation with high spatial and temporal variations, which cannot correctly be captured by rain gauge networks. However, radar data are often adjusted using rain gauge data in order to derive precipitation climatologies. It is found that radar still underestimates extreme rainfall due to a number of factors, including path-integrated and wet radome attenuation during heavy rainfall. Although methods exist to correct for this, the archived level 3 or 4 radar data do not allow for such corrections. Moreover, data archives from dual-pol radars, allowing for better corrections, are still relatively short. Finally, the spatial density of rain gauges providing hourly or subhourly observations is often sparse, limiting the accuracy of rainfall retrievals at time scales that are necessary to capture the temporal variability of convective precipitation.
However, the use of radar for climatological studies is not limited to precipitation. Several parameters related to severe convection can be derived from radar measurements. The resolution and coverage of radar-based hail climatologies (e.g., Cintineo et al. 2012) exceed that of methods based on hail reports and capture modes of variability not found in report-based climatologies. National radar-derived climatologies of hail occurrence have been published for numerous countries, including the United States (Cintineo et al. 2012), Switzerland (Nisi et al. 2016), Romania (Burcea et al. 2016), and Belgium (Lukach et al. 2017). Meanwhile, other researchers such as Chen et al. (2012) and Grams et al. (2012) have focused on small-scale wind phenomena (tornadoes, downbursts) or other characteristics of convective storms (Goudenhoofdt and Delobbe 2013; Kaltenboeck and Steinheimer 2015).
In the following section, we take a closer look at four exemplary studies around the world: United States, Germany, Australia, and the European Alps. The U.S. study is an example of a continent-wide climatology while the German study focuses on extreme values in a smaller area. The Australian study shows the benefits of using Doppler data in the analysis of mesoscale phenomena, while the studies in the Alps use three-dimensional data to better understand the processes that control precipitation intensity, distribution, and type, in order to better distinguish between natural and climate change–related variations. These four examples demonstrate the four advantages of radar data: they have a large spatial coverage, they have spatial and temporal resolutions suitable to describe a variety of mesoscale weather phenomena, they can provide multiple parameters, and they can describe the three-dimensional structure of weather phenomena.
In the United States, 15-yr continental dataset used for precipitation climatology.
Central dissemination of the NEXRAD Level 2 data provided many opportunities for users to grab and process data. Since 2002 the U.S. National Centers for Environmental Prediction (NCEP) has processed NEXRAD data into conterminous United States (CONUS)-wide maps of precipitation data. These data are called NCEP Stage IV. An overview of the NCEP Stage IV product can be seen in Nelson et al. (2016). The nearly 15-yr record of data allows for the analysis of a radar-based precipitation climatology. Figure 3 shows the long-term daily average precipitation based on the NCEP Stage IV product (which includes all available NEXRAD sites). This CONUS-wide climatology is a compilation of information from regional forecast centers that includes all available radar data as well as blending of the best available rain gauge information. This product is currently the only operational product that provides high-resolution radar-based precipitation estimates over the CONUS, and thus it is used in many studies for comparison of precipitation products (i.e., satellite quantitative precipitation estimation). The authors conclude that the Stage IV product could be useful especially for studies of high rain rates in the eastern and central parts of the United States. A 20-yr reflectivity dataset is also available for the study of convective type precipitation. Fabry et al. (2017) assessed the possibilities of the climatological use of radar reflectivity using this dataset.
Resolving small-scale structures in the heavy precipitation climatology of Germany.
A 17-yr precipitation climatology derived from radar observations for Germany is openly accessible at Winterrath et al. (2018). This climatology has already proven valuable for gaining new insights about the distribution and characteristics of daily and hourly rainfall in Germany (Lengfeld et al. 2018). Nationwide radar observations are available in Germany since 2001. The network operated by the DWD consists of 17 C-band radar systems providing precipitation estimates with 1-km range and 1° azimuthal resolution at a 5-min temporal resolution from a terrain-following precipitation scan. Precipitation estimates are adjusted on an hourly basis with observations from the rain gauge network operated by DWD. Differences and factors between two-thirds of the rain gauges and their corresponding radar pixels are calculated and interpolated over Germany. At the remaining stations the procedure that performs best is assigned the weight 1, the other procedure 0. The weights are interpolated to the whole domain and are used to compute the adjusted rainfall rates.
The operational rainfall dataset contains many inhomogeneities because it is routinely computed using always the latest software version. To ensure comparability of precipitation measurements from different locations and time steps, reprocessing of the time series is mandatory. Therefore, saving radar data as reflectivity fields instead of estimated rain rates is crucial for applying the reanalysis. Still, inconsistencies remain even after reprocessing due to changes in the radar network, for example, system upgrades, relocation of radars, or changes in resolution and range. The radar systems within the German network were upgraded from single-polarization to dual-polarization within the last decade and the maximum range of the radars also changed. A metadata archive helps to minimize computational issues and captures possible gaps or remaining inconsistencies in the radar climatology.
From the 17-yr dataset (2001–17), extreme values of precipitation for 11 different durations between 1 and 72 h that are expected at certain return periods have been determined on a 1 km × 1 km grid (Winterrath et al. 2017). Partial time series of the (2 × the number of years) most significant rainfall events were extracted from the climatological dataset for each of the 11 durations separately. The plotting position, that is, return period, for each element of the partial time series was then calculated based on the sample size and the number of years considered in the radar climatology following the guidelines provided by the German Association for Water, Wastewater and Waste (DWA 2012). A linear regression was performed for the resulting pairs of precipitation rates and return periods to compute rainfall totals for given return periods.
Daily and hourly precipitation totals with return periods of 20 years are depicted in Fig. 4. Daily precipitation amounts (Fig. 4a) are smallest in the lowlands of northern and central Germany and largest in the highlands and mountainous regions, indicating a clear orographic dependency. This spatial pattern for daily rainfall totals with a return period of 20 years is in accordance with estimates from a station-based climatology derived from the DWD rain gauge network (Deutscher Wetterdienst 2018). However, hourly precipitation with a return period of 20 years, as computed from the radar dataset (Fig. 4b), shows a structure that appears to be less dependent on the orography. In general, rainfall totals seem to be slightly smaller for central Germany, but the variability of hourly precipitation occurs on much smaller scales of 10 km or less. This pattern can only be derived from the radar systems with their large spatial coverage and resolution, because the German rain gauge network is too sparse to resolve such small-scale variations in precipitation. Altogether, precipitation maps derived from the climatological radar dataset indicate that heavy short-term rainfall events can occur everywhere in Germany regardless of the orographic structure. Due to the relatively short duration of the 17-yr dataset, it is still unclear whether all patterns are already characteristic for their location or are also dominated by single events. However, this issue will be further investigated in the future as the climatology will be reprocessed, updated, and thus extended at least on an annual basis. In case of a software update (e.g., improved quality control, integration of dual-polarization moments), the complete dataset will be reprocessed and a new version of the radar climatology will be released.
Mesocyclone climatology using 8 years of Doppler radar observations in Sydney.
Radar-based climatologies have proven invaluable for understanding the characteristics, drivers, and occurrence of severe thunderstorms in Peter et al. (2015) and Soderholm et al. (2017). While previous studies have focused on applying reflectivity-based retrievals, the increasing availability of Doppler data archives provides the opportunity to explore retrievals of thunderstorm dynamics. One such retrieval is the change of Doppler speed in azimuthal direction, known as azimuth shear, which is associated with mesocyclone occurrence and strength in supercell thunderstorms. This retrieval can also be applied to produce long-term climatologies as a proxy for supercell occurrence. Simple gate-to-gate difference methods for calculating azimuthal shear are very sensitive to data quality issues, including noise and artifacts, requiring careful quality control. Alternatively, more adaptive approaches, such as the linear least squares derivative (LLSD) technique (Miller et al. 2013), can reduce the effects of outliers while preserving the azimuthal shear signature. However, careful data processing is still required, including robust dealiasing techniques that require minimal supervision. Poor handling of aliased Doppler data can introduce additional artifacts that result to false vortex detection.
Eight years of Doppler radar observations covering the Sydney region of Australia were dealiased using the region-based approach and processed using the LLSD retrieval to detect the storm-scale rotation associated with supercell mesocyclones (Fig. 5). This analysis indicates that while mesocyclones were detected across the domain, the strongest systems occurred offshore, placing the greatest risk on the densely populated coastal cities.
Trends in precipitation from 10-yr time series in Alps.
Combining observations from three operational radars from the Swiss radar network over 10 years (1999–2009) leads to a series of studies on characterizing and quantifying precipitation in the European Alps (Rudolph et al. 2011, 2012; Rudolph and Friedrich 2013). These studies aimed at a better understanding of the processes that modulate precipitation intensity, distribution, and type in order to better distinguish between natural and variations related to climate. The analyses were based on three-dimensional radar data measured every 5 min with a spatial resolution of 1 km.
In Rudolph et al. (2011), the radar data were used to explain how synoptic-scale weather patterns affect mesoscale precipitation distribution over the European Alps. The analysis showed that the highest daily precipitation totals are related to southerly midtropospheric flow, while during convective weather, intensive but short-duration rain showers are most frequent. Based on this link between precipitation characteristics and synoptic patterns, Rudolph et al. (2012) then projected the precipitation climatology for the twenty-first-century for seven Swiss river basins. They used synoptic weather patterns projected by the Community Climate System Model, version 3.0 (CCSM3), which predicted there will be more anticyclonic synoptic patterns, fewer cyclonic patterns, and constant frequency of advective patterns over Switzerland. When coupled with the observed radar-based precipitation estimates for each synoptic pattern, this will result in an approximately 10%–15% decrease in decadal precipitation over the course of the twenty-first century for seven Swiss river basins. The study also shows that heavy precipitation events will not only be more frequent, but as the total precipitation amounts are decreasing, the heavy events will account for a greater proportion of total precipitation in Swiss river basins by the end of the twenty-first century.
The last study conducted by Rudolph and Friedrich (2013) analyzed the seasonal patterns of vertical reflectivity structure. Storms can be divided into high-cloud-top summer type and shallow winter type. In addition to the distinct vertical structure, summer- and winter-type storms are different in duration, intensity, and the interval between storms. Although summer- and winter-type storms result in a similar amount of total precipitation, summer-type storms have greater intensity but shorter duration. When the cold seasons get warmer, the average precipitation intensity may shift toward intense convective precipitation becoming more frequent, while less intense stratiform precipitation might become less frequent.
In addition to providing a survey on existing data archives and future directions, the AOPC also asked the Radar Task Team to provide recommendations for archiving radar and metadata from the perspective of climate research. In principle, the general recommendation for radar data archiving is identical to that for collecting observations for data assimilation—the data quality should be as high as possible, with very detailed documentation. However, the need to process long time series leads to some additional recommendations. The recommendations, which we have itemized below, result from brainstorming sessions of the Task Team consisting of both radar and climate experts and are also published in the GCOS report 223 (GCOS 2019).
Archive at least radar reflectivity factor as full three-dimensional volumetric data.
Radar systems that are used around the world range from single-polarization radars measuring only radar reflectivity to state-of-the-art dual-polarization Doppler weather radars. As such, the main variable that all instruments measure, and that should be always archived, is radar reflectivity. This allows for flexibility when postprocessing a variety of products such as precipitation rates. Precipitation is an essential climate variable, and as more advanced methods for calculating precipitation rates are still being developed, it is recommended to save the original reflectivity values and perform reanalyses (also known as postprocessing) to retrieve precipitation time series that are as homogeneous as possible. Modern weather radars also measure Doppler velocity and polarimetric moments such as differential reflectivity (ZDR), correlation coefficient (ρHV), and differential phase (ϕDP). This additional information could be very useful in climate studies including cloud microphysical processes, improving the accuracy of quantitative precipitation estimates, characterization of tornadoes, and classification of meteorological targets such as hail or nonmeteorological targets such as migrating birds. Since the majority of radars provide three-dimensional information, it is advisable to save the full volumetric data. These level 2 data are needed for physically based corrections in postprocessing, such as corrections for intervening attenuation.
Pay attention to metadata and documentation and think whether your data and metadata can be read and understood in 2050.
Radar measurements alone are not useful if they are not accompanied with a sufficient set of metadata. It is mandatory to know when, where, and how each radar pixel was obtained. This requires the storage of all elevation and azimuth angles as well as time information. A single radar pixel represents a sizable volume of atmosphere, which can be several kilometers wide and tall. These dimensions depend on properties of the antenna and measurement settings. Furthermore, it is also important to know how reliable the calibration of the instrument was.
An especially challenging area to be documented is the processing of the raw data: how the reflectivity values (in dBZ, also known as Level 2 data) are derived from the received power (watts, Level 1 data). This conversion is not just a simple formula: it always includes a calibration factor, and usually many types of filtering to remove undesired noise from the signals. Microwaves transmitted by the radar are scattered from hills, masts, birds, and other obstacles (often known as clutter), and hydrometeors, including raindrops and snowflakes. To distinguish meteorological from nonmeteorological echoes, several methods have been developed over the years. Usually, an upgrade to a new radar generation is best seen in the effectiveness of this filtering process. Improved filtering over multidecadal time series will see fewer and fewer clutter echoes, first from statistical or Doppler filters to remove echoes from nonmoving targets, and then later dual-polarization brings methods for eliminating echoes from birds and other moving targets.
Organize the documentation of change history.
Good documentation of hardware and software changes is particularly important since climatologically significant time series often consist of data spanning sequential instruments at the same site, considering the lifetime of a radar being 10–20 years. Even if the instrument has not been replaced, typically several software or hardware upgrades are implemented over the lifetime of a radar, so the present metadata is most likely invalid for past data. Attention should be paid in planning how the user of time series is made aware of these changes.
Consider adopting a standard open data format and access portal.
The international radar community has been developing international, manufacturer-independent data formats, and major software manufacturers already provide conversion software. Using such common data formats (ODIM_H5, CfRadial) for archives promotes international collaboration and compatibility with many of the open-source software tools (Heistermann et al. 2015). Using these formats also increases the probability that archives can be read in 2050. To promote accessibility of data, widely used data access protocols (DAPs) such as THREDDS (Thematic Real-time Environmental Distributed Data Services) or OpeNDAP (Open-Source Project for a Network DAP) are encouraged. Such services have significant interoperability and can provide both programmatic and web-based access. Currently, the U.S. and Australian operational radar archives are publicly available with THREDDS.
This study was initiated from requirements of AOPC, and we want to thank its chairman Dr. Kenneth Holmlund for his guidance. Development of the azimuthal shear climatology for Sydney, Australia, was supported by Jordan Brook of the University of Queensland. Our thanks go out also to the participants of the radar survey and the international radar community for their inputs and recommendation. The data from the German radar climatology originates from the project “Generation of a decadal radar-based high-resolution precipitation climatology for Germany for the retrieval of recent changes in precipitation extremes.” The manuscript changed significantly after comments of the three anonymous reviewers, and the authors are grateful for their constructive reviews.