Multisensor Agile Adaptive Sampling (MAAS): A Methodology to Collect Radar Observations of Convective Cell Life Cycle
1. Introduction
Convective clouds play important roles in Earth’s climate system; they drive large-scale circulations and are a primary mechanism for the transport of heat, moisture, aerosols, and momentum throughout the troposphere (Hartmann et al. 1992). Uncertainties around the representation of convective clouds in global climate models affect our ability to constrain the Earth climate sensitivity (e.g., Sanderson et al. 2008; Sherwood et al. 2014; Zhao 2014). Furthermore, suggestions that the frequency of occurrence and intensity of convective precipitation will increase in a warming climate highlight the critical importance of understanding the life cycle and drivers of deep convection for resiliency planning (Diffenbaugh et al. 2013; Seeley and Romps 2015).
In simple terms, a deep convective cell can be described as buoyancy-driven updraft(s) that grow hydrometeors, which then precipitate and drive a downdraft(s) that ultimately lead to cloud dissipation (Byers and Braham 1949). However, complicated microphysical and dynamical processes strongly control the details of the life cycle of convective clouds, including thermal evolution, cold pool circulations, hydrometeor size distribution and phase, precipitation efficiency, lightning flash rates, and subsequent convective initiation. Because convective cells evolve rapidly, their life cycle is challenging to represent in models and to observe with instrumentation (e.g., Fridlind et al. 2019; Ladino et al. 2017).
Polarimetric weather radars have been instrumental in advancing our understanding of convective cloud and precipitation processes (e.g., Dawson et al. 2015; Ryzhkov et al. 2013; Schrom and Kumjian 2016; Snyder et al. 2015). That said, most weather radars operate with predetermined “sit and spin” horizontally oriented scan strategies to maximize spatial continuity, which does not allow for rapid updates and leaves researchers with only “snapshots” of evolving cloud systems (Kollias et al. 2022). This approach also leaves gaps in our understanding of the vertical structure of convection, including that of its updraft cores and precipitation shafts. While it may be possible to composite vertical images from the horizontal scans, as exemplified in Fig. 1, such compositing of observations collected over several minutes relies on the assumptions that each of the horizontal slices used in the reconstruction were collected at the same time and/or that the system observed was nonevolving, which is likely not valid for convection. Also, compositing approaches often result in vertical discontinuities and lack vertical resolution, which is critical to accurately capture features such as vertical gradients that can be better interpreted with microphysical models to quantify hydrometeor size growth rates (e.g., Morrison et al. 2020).
Consecutive composite range–height indicator (RHI) constructed from KHGX volume scans for the 7 Aug 2022 convective case. Each panel contains the start and end times in UTC of the KHGX volume scan used and the azimuth of the composite RHI in coordinates relative to the location of the CSAPR2.
Citation: Journal of Atmospheric and Oceanic Technology 40, 11; 10.1175/JTECH-D-23-0043.1
While vertical cross sections through convective clouds have been previously collected using weather radars (e.g., during the CACTI field experiment; Varble et al. 2021), only a few radars have been programmed to automatically adapt their scan pattern to track clouds over their life cycles and thus maximize the number of scans containing cloud observations as opposed to clear air. An example of this is the Collaborative Adaptive Sensing of the Atmosphere (CASA) network (McLaughlin et al. 2009), a network of small, low-cost, short-range radars controlled by a software architecture that uses observations from the radar network itself to automatically track cloud systems based on user preferences for information, data quality, system resources, and the evolving weather phenomena. Another example is the large mechanically scanning Chilbolton radar, which is automatically steered toward storms based on their size, rainfall rate, and distance from the radar using observations from the U.K. radar network (Stein et al. 2015). A similar concept was also applied by Torres et al. (2016) to steer a phased array radar toward regions with significant weather returns; their adaptive digital signal processing algorithm for phased array radar timely scans (ADAPTS) works in real time by classifying individual beam positions as active or inactive based on reflectivity thresholds and devotes more radar time to active beam positions. Another feature of ADAPTS includes using different scan strategies depending on whether the active beams contain clear air, precipitation, or severe precipitation echoes. More recently, Kollias et al. (2020b) introduced a broader framework, the Multisensor Agile Adaptive Sampling (MAAS), which additionally leverages observations external to the “tracking” radar (e.g., satellite and cameras) to enable the sampling of features that are upwind or elevated beyond the sector of the tracking radar and/or that are altogether not detectable by radar (e.g., lightning). MAAS also enables the tracking of fast-evolving processes and/or short-lived systems by eliminating the need for the tracking radar to perform its own surveillance to gain situational awareness. In 2020, MAAS was used to guide a mechanically scanning Ka-band radar and an X-band phased-array radar to track shallow cumuli and a waterspout over parts of their life cycles (Kollias et al. 2020b).
This article describes the operational implementation of a new iteration of MAAS on two distinct mechanically scanning C-band precipitation radars that were operating in Houston, Texas, during the Department of Energy Tracking Aerosol Convection Interactions Experiment (TRACER) and the National Science Foundation Experiment of Sea Breeze Convection, Aerosols, Precipitation, and Environment (ESCAPE; Kollias et al. 2023, manuscript submitted to Bull. Amer. Meteor. Soc.) field campaigns for the specific purpose of sampling isolated convection over their life cycle. This new iteration of MAAS includes the addition of a finessed automatic cell tracking and nowcasting module, as well as two edge computing client modules operating at the locations of, and tailored to the specifications of, the second-generation C-band scanning ARM precipitation radar (CSAPR2) and the Colorado State University C-band Hydrological Instrument for Volumetric Observation (CHIVO) radar.
Between 1 June and 30 September 2022, MAAS enabled the sampling of about 1300 unique isolated cells over parts of their life cycle from a Lagrangian perspective, with most being sampled for at least 15 min at scan repetition times less than 2 min. To the best of our knowledge, this dataset of vertical cross sections through isolated deep convection, collected primarily through automatic means, constitutes the largest dataset of its kind.
2. TRACER/ESCAPE instrumentation
To advance our understanding of convective cloud processes and aerosol impacts on these processes, TRACER focused on the collection of observations of the evolution of convective cloud properties in the context of the environment in which convection initiates, propagates, and decays. TRACER supported the deployment of the first ARM mobile facility (AMF1; Miller et al. 2016) at the La Porte airport (29°40′78″N, 95°03′36″W; Fig. 2; red marker) as well as of the CSAPR2 radar at Croix Memorial Park in Pearland (29°31′14″N, 95°23′38″W; Fig. 2; green marker). The CSAPR2 was positioned 26.4 km away from the AMF1 site such that it would be well positioned to sample convective cells over the AMF1 site. The CSAPR2 was guided by MAAS for 680 h between 1 June and 30 September 2022. These hours include all the intensive observation periods (IOP) as well as additional periods having targets of opportunity identified on the fly by the ESCAPE science team (e.g., unforecasted isolated convection, widespread precipitation, lightning strikes).
Map of the MAAS domain containing the location of the KHGX (blue marker), the CSAPR2 (green marker), and the CHIVO at the AMF1 site (red marker). Overlaid is the spatial distribution of the cells sampled by the CSAPR2 under MAAS guidance during the campaigns.
Citation: Journal of Atmospheric and Oceanic Technology 40, 11; 10.1175/JTECH-D-23-0043.1
The TRACER assets were enhanced by the deployment of the CHIVO radar under the scope of the ESCAPE field campaign. ESCAPE aimed to collect and analyze observations of the fundamental process-level coupling between vertical motions (kinematics), microphysics, and precipitation across a full range meteorological regimes, throughout the life cycle of convective clouds. The CHIVO radar was deployed at the AMF1 main site at the La Porte airport (29°40′78″N, 95°03′36″W; Fig. 2; red marker) and was guided by MAAS for the entire period from 1 August through 30 September 2022.
Operationally, the Houston region is monitored from space by the Geostationary Operational Environmental Satellites (GOES-16) Geostationary Lightning Mapper (GLM; Schmit et al. 2017) and on the ground by the KHGX Next Generation Weather Radar (NEXRAD) radar (Crum et al. 1998) located at 29°28′19″N, 95°04′45″W (Fig. 2; blue marker). During the TRACER and ESCAPE campaigns, MAAS leveraged the routine observations collected by the NEXRAD and GLM to acquire situational awareness of convective activity in the Houston area and subsequently to steer the CSAPR2 and the CHIVO radars for the purpose of collecting rapid sequences of vertical cross-section observations through selected cells as they progressed through their life cycle. In this section, we elaborate on the features of each of these observing systems as they pertain to their ability to measure the properties of convective cells under MAAS guidance.
a. KHGX NEXRAD
The KHGX S-band Doppler dual-polarization radar is part of the NEXRAD network of radars used by the National Weather Service (NWS), the Federal Aviation Administration (FAA), and the U.S. Air Force. KHGX collects volume coverage patterns (VCPs) scan composed of a sequence of 360° (in azimuth) surveillance PPI scans conducted at progressively increasing elevation angles (Brown et al. 2005; Chrisman 2013; Kingfield and French 2022). Specifications of each VCP scan (radar antenna rotation rate, elevation angle, and pulse repetition frequency) depend on the intended use, but are, for the most part, predefined. Regardless, it is well understood that the use of a limited number of elevation angles results in a coarse vertical resolution, especially at long ranges, and in an upper-level data gap, especially near the radar (i.e., a cone of silence; Kollias et al. 2022).
The KHGX radar has a maximum unambiguous range between 300 and 450 km depending on the radar pulse repetition frequency and the processing mode and a range gate spacing of 250 m. MAAS leverages the KHGX’s radar reflectivity and differential phase shift (ϕDP) measurements. More details about how MAAS uses NEXRAD data are given in section 3b.
b. GLM
The GLM aboard GOES-16 continuously maps lightning with 70%–80% efficiency within its spatial observational domain. MAAS looks for lightning strikes occurring within the CHIVO radar observation domain (as depicted by the red pie in Fig. 2). More details about how the GLM data are used by the MAAS framework are given in section 3d.
c. CSAPR2 radar
The CSAPR2 is a state-of-the-art, dual-polarization Doppler radar (Kollias et al. 2020a). The CSAPR2 simultaneously transmits in both horizontal and vertical polarizations, thus providing measurements of radar reflectivity, mean Doppler velocity, and spectrum width in both polarizations as well as of differential phase shift (ϕDP), differential reflectivity (ZDR), and correlation coefficient (ρHV). The CSAPR2 configuration used in Houston has a maximum unambiguous range of 110 km and a range gate spacing of 100 m. The CSAPR2’s sampling sector during the TRACER and ESCAPE campaigns is indicated by the green dashed circle depicted in Fig. 2. Historically, and when not under MAAS control during the campaigns, the CSAPR2 collects observations using a sequence of 360° (in azimuth) surveillance PPI scans conducted at progressively increasing elevation angles followed by a sequence of hemispherical sky RHIs (HSRHI; Kollias et al. 2014; Varble et al. 2021).
The state of the CSAPR2 radar’s scan sequences can be monitored with essentially no delay via a direct Network File System (NFS) mount of the radar’s native file system. This access policy was a critical factor in enabling the MAAS edge computing software, running on a separate server, to maintain a tight feedback loop that maximized system responsiveness to real-time conditions. On the other hand, the CSAPR2 native control software (created by the company Gamic) required the parameters (antenna rotation rate, elevation angles, azimuth spans) for all scan strategies to be predetermined and stored in lookup tables before operation (see section 3a for more details).
d. CHIVO radar
The CHIVO is a C-band dual-polarization Doppler radar capable of collecting measurements of radar reflectivity, mean Doppler velocity, spectrum width as well as differential phase shift (ϕDP) and differential reflectivity (ZDR). In Houston, the CHIVO was configured for a maximum unambiguous range of 125 km and a range gate spacing of 150 m. It should be noted that during the TRACER and ESCAPE campaigns, the CHIVO was not allowed to transmit in the general direction of the CSAPR2 as mandated by local airport operations. The CHIVO’s sampling sector is indicated by the red dashed pie depicted in Fig. 2.
Unlike the CSAPR2, the CHIVO’s native software by Vaisala does not require scan strategies to be predetermined. In other words, it can be given scan commands on the fly thus offering great flexibility. That said, the access policy of the CHIVO’s home institution required us to communicate commands and data through an intermediate server, which by its nature, introduced timing latencies that prevented precise knowledge of radar state. This did not allow us to synchronize commands for new scans with the completion of prior scans. As a result, we found it most efficient to have the CHIVO mirror the scans of CSAPR2 rather than operating autonomously.
3. MAAS framework implementation during the ESCAPE and TRACER campaigns
a. Overview
MAAS was used during the TRACER and ESCAPE campaigns to guide the CSAPR2 and CHIVO in such a way as to increase the likelihood of observing the vertical structure (with little to no gaps), spatial variability (at subkilometer scale), and temporal evolution (at ∼2-min resolution) of convective cells. This was accomplished using surveillance data (from the NEXRAD or GLM) to identify cells and then aiming the C-band radars at the nowcasted location of features within a cell selected either using predetermined rules or by an online user (see section 3b for more details). Edge computing was subsequently employed to rapidly analyze the first set of targeted CSAPR2 scans to identify target features for subsequent CSAPR2 and CHIVO scans (see section 3c for more details). Only when information from the CSAPR2 was unavailable, or when the selected cell was within the CHIVO radar’s blanking sector, CHIVO performed independent scans targeting the most recent and closest lightning strike (if any; see section 3d for more details). Owing to the transient nature of convection, the scans collected by the C-band radars may not have perfectly hit the features that they were targeting (e.g., the maximum reflectivity of the selected cell). That said, MAAS improves our ability to collect sector scans within the same evolving cells and as such reduces the number of cloud-free scans. An example scan sequence and resulting observations for a case where a cell was automatically selected by MAAS is presented in section 3e for reference.
To be more precise, for each selected cell CSAPR2 collected at least one scan bundle composed of three sector PPIs and four to six sector RHIs.
-
The three sector PPIs respectively targeted an elevation 20% below the top of the cell, the center of the cell (both determined through the analysis of NEXRAD observations), and the 3° elevation level.
-
The first sector RHI targeted the centroid of the cell (as determined through the analysis of NEXRAD observations).
-
The second sector RHI targeted the cell’s point of maximum vertically integrated liquid (as determined through the analysis of NEXRAD observations).
-
The third sector RHI targeted the cell’s region of maximum radar reflectivity (as determined through analysis of the CSAPR2 center PPI).
-
The target of the fourth sector RHI varied over the course of the campaigns reflecting lessons learned and evolving interests of the science team. Until around 8 September 2022, it targeted the cell’s region of maximum ZDR and after that, the region of largest along-range mean Doppler velocity gradient in the selected cell.
-
Around 2 August 2022 the fifth and sixth sector RHIs were introduced to assist with the evaluation of the new NASA Atmosphere Observing System (AOS) mission, which proposes to deploy a constellation of satellites that will consecutively sample the same atmospheric feature with the objective of quantifying its rate of change. To emulate this concept, the fifth and sixth sector RHIs respectively targeted the same geospatial location (i.e., latitude, longitude, and elevation sector) as the third and fourth sector RHIs.
For all sector scans, scan span, and antenna rotation speed were set to balance a need to minimize acquisition time and sample a sizable portion of the selected cell. For the sector PPIs, azimuthal span and antenna rotation speed were selected from a lookup table (Table 1) depending on the elevation angle of the third PPI of the bundle (a proxy for cell distance since it targeted near the top of the cell). In contrast, sector RHIs were set to span from 0.5° to the tabulated elevation just above the estimated maximum echo-top height at an antenna rotation speed selected to keep collection time below 60 s (Table 2). An extensive visual analysis of summertime convective cell observations from the NEXRAD and GOES Advanced Baseline Imager (ABI) was used to construct lookup tables for these scan parameters.
CSAPR2 plan position indicator scan parameters lookup table.
CSAPR2 range–height indicator scan parameters lookup table.
In contrast, for each selected cell, CHIVO collected at least one scan bundle composed of three to four RHIs, with at least one mimicking the CSAPR2 by aiming toward the cell’s region of maximum radar reflectivity (as determined through analysis of the CSAPR2 center PPI). The other RHIs were aimed at different targets:
-
The cell’s region of maximum VIL (as determined through the analysis of NEXRAD observations)
-
The cell’s region of maximum ZDR (as determined through analysis of the CSAPR2 center PPI)
-
Azimuths 2 km to the right and left of the cell’s region of maximum radar reflectivity (as determined through analysis of the CSAPR2 center PPI) in the plane perpendicular to the radar beam
-
Respectively, 2 km to the right, toward, 2 km to the left and then again toward the most recent and closest lightning strike, if any (as determined through analysis of the GLM), as measured in the plane perpendicular to the radar beam
The CHIVO sector scan span and antenna rotation speed were also set to balance a need to minimize acquisition time and sample a sizable portion of the selected cell. But since CHIVO can receive scan parameter information on the fly it does not need to rely on a lookup table for these scan parameters. The sector RHIs were set to span from 0.5° to just above the estimated echo-top height at an antenna speed equal to the elevation span divided by nine. Note that antenna speed was limited to values between 3° and 10° s−1.
Under MAAS, radar control was achieved using geographically distributed modules in a client–server architecture (Fig. 3); specialized software clients were located at both edge computing sites (local to the tracking radars in Pearland and LaPorte), at Brookhaven National Laboratory to coordinate the radars and assimilate incoming information, and at Stony Brook University to communicate data about the selected cell and its target regions and to handle real-time data display feeds. With this architecture, components of MAAS were seamlessly coupled via common communication channels, permitting flexible and reconfigurable, essentially worldwide geographical topologies. The communication backbone was provided by two secure SSH tunnel channels, one dedicated to low-latency control and status messages (green lines) and a second for data transfer (red lines). The use of two pathways ensured that data transfer would not compromise the ability to rapidly adapt the radar’s scan strategy, especially under challenging internet connections.
Diagram of the MAAS geographically distributed real-time client–server architecture. Illustrated are the clients and their common communication channels including those of low latency for control and status messages (green lines) and those for data transfer (red lines).
Citation: Journal of Atmospheric and Oceanic Technology 40, 11; 10.1175/JTECH-D-23-0043.1
b. Details of the cell selection and target nowcasting algorithms
When input from an online user is not provided, MAAS utilizes an automated algorithm hosted on a remote server at Stony Brook University. A roadmap to this algorithm is presented in Fig. 4. MAAS first downloads the latest NEXRAD level II VCP scan. Nonmeteorological echoes are masked from the volume scan data using the NEXRAD particle identification (RadxPID) algorithm, part of the Lidar Radar Open Software Environment (LROSE; http://lrose.net). In short, RadxPID computes specific differential phase (KDP) from the radar measurements and uses this variable and others to classify the measurements into 18 categories ranging from cloud to ground clutter. Echoes labeled as “biological,” “second trip,” “ground clutter,” and “receiver saturation” are masked. Each masked radar PPI scan is then gridded onto a horizontal cartesian grid using a KD-Tree interpolation algorithm at a 500-m grid spacing over the 250 km × 250 km Houston domain shown in Fig. 2 (Bentley 1975). The gridded masked radar reflectivity field is used both to construct a 1.5 km AGL constant-altitude plan position indicator (CAPPI) and to create a map of vertically integrated liquid (VIL) for the domain.
Workflow diagram for the automated NEXRAD-based cell selection and target identification algorithm.
Citation: Journal of Atmospheric and Oceanic Technology 40, 11; 10.1175/JTECH-D-23-0043.1
Vertically integrated liquid (color map), horizontal velocity motion field (arrows), and cell IDs (black contours and numbers) automatically estimated by MAAS using KHGX data for the 7 Aug 2022 convective case. Each panel contains the start time in UTC of the KHGX volume scan used.
Citation: Journal of Atmospheric and Oceanic Technology 40, 11; 10.1175/JTECH-D-23-0043.1
MAAS waits for the acquisition of a second volume scan before combining two consecutive CAPPIs to predict the horizontal velocity motion field of each radar echo using the Lucas–Kanade algorithm (Lucas and Kanade 1981) implemented in the Python Framework for short-term ensemble prediction systems (PySTEPS; Ayzel et al. 2019). An example of this motion field is depicted by the arrows in Fig. 5. The velocity motion field is then combined with the cell ID map to coarsely estimate (time horizon of 1 h, resolution of 5 min and 500 m) the trajectory of each cell starting from their location in the second volume scan (for later use in the cell selection process) and to determine the life stage of all identified cells across the two scans.
The life stage of cells larger than five pixels is estimated using a slightly modified version of the MCIT algorithm (Hu et al. 2019). In a nutshell, MAAS advects the VIL and cell ID fields from the previous scan using the motion field and recursively compares the advected cells [cell(n, t)] with the cells in the current scan [cell(n, t + 1)]. Cells can be assigned four life stages:
-
Continuation: The integrated common VIL of cell(n, t) and cell(n, t + 1) is at least 50% of the total VIL of the smaller of the two compared cells. The VIL peaks of both cell(n, t) and cell(n, t + 1) are inside the common area of the two compared cells or the integrated common VIL is greater than 75% of the total VIL of the smaller of the two compared cells. As was the case for cell (2,1852:52 UTC) and cell (3, 1852:52 UTC) in Fig. 5b.
-
Split: If cell(n, t) has more than one continuation cell, only the one with maximum integrated common VIL gets the identity of cell(n, t) and all the rest of the candidate continuation cells are labeled splits. In Fig. 5c cell(4, 1859:10 UTC) was identified as the continuation of cell cell(4, 1852:52 UTC) and cell(20, 1859:10 UTC) was labeled a split.
-
Merge: Similar to splits cell(n, t + 1) has more than one source cell and is therefore labeled as a merger. In Fig. 5c cell(19, 1852:52 UTC) is labeled as having merged with cell(4, 1859:10 UTC).
-
Birth: When cell(n, t + 1) does not overlap with any cell from the previous time step.
Modifications to the original MCIT algorithm were made to promote continuation over split when encountering ambiguous situations in cell tracking. Visual inspection of the data shows that these modifications to the MCIT algorithm provide a more persistent representation of evolving cells.
After establishing the life stage of each cell, MAAS characterizes all cells identified in the latest volume scan. It relies on the VIL map to estimate each cell’s maximum VIL, area (i.e., maximum horizontal coverage), centroid location defined as the center of the smallest latitude and longitude box encompassing the entire cell, mean distance from the CSAPR2, and area of the cell relative to its cluster. It also uses the masked gridded radar reflectivity field, applies an additional filter to remove echoes with Z < −10 dBZ, and then estimates each cell’s maximum radar reflectivity, maximum echo-top height (CTH), minimum echo-base height (CBH), deepest column, and the fraction of grids with a valid echo between CBH and CTH.
A set of predetermined rules is used to select the one cell to be sampled by the tracking C-band radars among all the cells characterized in the domain (see Table 3 for a list of rules). Every time a new NEXRAD KHGX volume scan becomes available (every ∼7 min), MAAS reevaluates whether the selected cell should continue being sampled or if a new selected cell should start being sampled. Since following cells over their life cycle was a key goal of the TRACER and ESCAPE campaigns, a selected cell continues being tracked until at least one of its characteristics reaches the level set in a less stringent set of discontinuation rules listed in Table 3. If more than one cell satisfies the conditions listed in Table 3, MAAS selects the cell that is going to pass over the AMF1 site during its predicted trajectory over the course of 1 h. If there are no cells passing over that location, MAAS selects the cell closest to the CSAPR2.
Criteria for initial cell selection and discontinuing the sampling of a selected cell.
An extensive analysis of a historical NEXRAD KHGX dataset (June–September of years 2018–21) was used to develop the two sets of rules used to sample isolated convective cells in Houston. It is worth pointing out that trial and error was used when creating the two sets of rules to balance simplicity and stability. During development, we observed that an overly simplistic set of rules would lead the algorithm to choose a new selected cell at every iteration thus failing to sample cells over their life cycle. Alternatively, an overly complex set of rules would lead the algorithm to fail to select a cell. For the next iteration of MAAS, we envision incorporating a self-learning algorithm that could use users’ preferences for certain cells to produce a continuously improving set of rules. For the moment, real-time user input was managed by a separate module (see below).
Following cell selection and every time a new NEXRAD volume scan becomes available, the trajectory of the selected cell and hence the latitude and longitude of its centroid, and of its maximum vertically integrated liquid are updated using the latest velocity motion field and the latest location of the selected cell. These nowcasted locations were used to guide the CSAPR2 PPIs and its first two RHIs when operating in automatic cell selection, which operated for 80.34% of the sampling period, including during our example afternoon convective case of 7 August 2022.
Real-time user input to MAAS enables online users to intervene in the cell selection process to enable the sampling of other storm types such as those forming in clusters or producing large anvils. From time to time (for a total of 19.66% of the sampling period during the campaign), a user would visually inspect the latest VCP scans from the NEXRAD web display (https://radar.weather.gov), as well as other real-time data sources, such as GOES-16 visible and outgoing longwave radiation imagery. At any point, the user could override the automatically identified target locations by providing the latitude, longitude, and height of a feature of interest. Ideally, the user would select a latitude, longitude, and height corresponding to where the location of interest was anticipated to advect in a time window to match the time when the tracking C-band radars would perform their scanning sequence. In practice, this was accomplished during TRACER and ESCAPE with a mouse click by the user on a map at the desired location. Special software clients running local to the user extracted and communicated the geolocation information based on the type of map being used at the time. In future iterations, online user input should occur on the cell ID maps such that the automatic cell characterization module could assist the user in predicting the advection of their selected cell. When in online user mode, the selected latitude, longitude, and height is used to guide the CSAPR2’s three PPIs and its first two RHIs.
c. Details of the automated CSAPR2-based target identification algorithm
Information about the cell selected based on NEXRAD observations (e.g., centroid location, echo-top height, and maximum vertically integrated liquid) is passed to the CSAPR2’s edge computer. Upon completion of the CSAPR2’s third PPI, the edge computing system immediately begins analyzing the collected PPI data, while the CSAPR2 begins its first RHI scan targeted toward the NEXRAD nowcasted location of the cell’s centroid.
During this time, edge computing analysis determines the desired azimuths of the RHIs to follow. Specifically, any PPI data outside the geometric boundary of the selected cell are first masked and excluded from consideration. The rest of the PPI data are analyzed to identify the location of the selected cell maximum radar reflectivity, maximum ZDR, and maximum along-range velocity gradient. These exact locations each became potential targets for CSAPR2 and CHIVO RHIs at some point in the campaigns. Note that no effort is made to nowcast the location of these potential targets since the RHI sequence immediately follows the PPI sequence used to identify those locations.
d. Details of the lightning identification algorithm
When the CSAPR2 was not under MAAS guidance or when the selected cell was located within CHIVO’s blanking sector the GLM was used to guide CHIVO. Location of lighting within a 120-km radius around CHIVO was acquired from the GLM; these data are updated by the GLM in a 20-s cycle. The locations of lightning strikes within the most recent GLM update cycle are the first candidates for targeting, with the one closest in range to CHIVO being the one chosen. If no candidates are found, a similar search is repeated for the two prior GLM cycles. If no viable strikes are found, the CHIVO is given no new instructions, and allowed to continue whatever pattern it is currently performing. If no lightning is located within 10 min, and the selected cell remains in the CHIVO blanking sector, CHIVO is instructed to enter an idle state.
e. Example of automatic cell selection mode and associated scan sequence
Taking the event of 7 August 2022 as an example, MAAS identified several cells in the NEXRAD volume scan that ended at 1845:24 UTC (not shown). Upon its second detection by the NEXRAD during the 1846:48 UTC volume scan, cell ID 4 (pictured in Fig. 5a) was selected for tracking given that it best satisfied the rules listed in Table 3. Figure 6 shows the scan sequence that followed this NEXRAD volume scan which ended at 1851:21 UTC (depicted by pink bar).
Scan sequence for the 7 Aug 2022 convective case. (top to bottom) The timing of the KHGX volume scans (bars), the start time and elevation/azimuth of the CSAPR2 scans (triangle markers), and the start time and the azimuth of the CHIVO scans (circle markers). Colors depict distinct scan bundles.
Citation: Journal of Atmospheric and Oceanic Technology 40, 11; 10.1175/JTECH-D-23-0043.1
Within less than 1 min, information about the geometric properties and maximum VIL of the selected cell was used to initiate the collection of a CSAPR2 scan bundle (i.e., three PPIs and six RHIs). The blue triangle markers in Fig. 6 shows the time sequence and angular direction of the individual scans forming this bundle. The sequence began with the CSAPR2 performing a PPI scan at 16.0°, then at 8.0° aiming respectively at the elevation nearest (in the CSAPR2 lookup tables, Table 1) to 80% of the top and center of the selected cell and then at 3° elevation (Figs. 7a–c depict those scans). Immediately after collecting these PPIs, the CSAPR2 collected two consecutive RHIs, one passing through the nowcasted location of the selected cell’s geometric center at 349.56° and the other through the nowcasted location of its maximum VIL at 350.36° (Figs. 7d,e depict those scans). Within seconds of collection, information from the center PPI at 8.0° guided the collection of the following four RHIs, two toward the selected cell maximum reflectivity region at 351.2°, and two toward its maximum ZDR region at 351.47° (shown Figs. 7f–i). Simultaneously, the information from the center PPI at 8.0° guided CHIVO to collect a bundle of three RHIs. The green circle markers in Fig. 6 show the time sequence and angular direction of the individual scans forming this bundle that was meant to target the selected cell maximum reflectivity, maximum VIL, and maximum ZDR (Figs. 7j–l depict those scans). Unfortunately, due to a human error, CHIVO’s aim was about 1° off target between 1 and 9 August 2022. That said, this error was small enough such that CHIVO RHIs were still within the limits of the selected cell.
For the 7 Aug 2022 convective cell (a)–(i) data from the CSAPR2 bundle marked by the dashed blue box in Fig. 5 and (j)–(l) data from the CHIVO bundle marked by the dashed green box in Fig. 5. The plot in (b) also shows the location of the CSAPR2 RHIs (solid black lines) and the CHIVO RHIs (dashed lines) relative to the CSAPR2 center PPI.
Citation: Journal of Atmospheric and Oceanic Technology 40, 11; 10.1175/JTECH-D-23-0043.1
Upon completing the collection of its first scan bundle, the CSAPR2 began collecting a second scan bundle (pink triangle markers) using the nowcasted location of the selected cell based on the same two previous NEXRAD volume observations. In contrast, the CHIVO continued to collect scan bundles until it received information from a new CSAPR2 center PPI at 1855:42 UTC. As shown by the gray bar in Fig. 6, observations from a new NEXRAD volume scan became available at 1857:41 UTC as the CSAPR2 was completing the collection of its third bundle of scans (black triangle markers). This new information was processed to update the selected cell’s characteristics, determine if it should continue being tracked (based on the discontinuation rules listed in Table 3), and update its cell trajectory for use in the collection of the following scan bundle(s).
Comparing RHIs collected by the CSAPR2 and the CHIVO to RHIs reconstructed from the NEXRAD VCPs allows us to begin appreciating the value of MAAS for the study of isolated convection. For our example cell, Fig. 1b shows an RHI reconstructed from the NEXRAD VCP collected between 1852:52 and 1857:41 UTC presented in the CSAPR2 frame of reference. During the same time, the CSAPR2 collected 3 bundles of 3 PPIs and 6 RHIs and the CHIVO collected 14 bundles of 3 RHIs. Figure 8 shows PPI2 (the “center PPI”; Figs. 8a–c) and RHI3 (the “maximum reflectivity” RHI; Figs. 8g–i) from each of these 3 CSAPR2 bundles. It is clear from the CSAPR2 observations that the selected cell evolved significantly during the 5-min period that it took the NEXRAD to complete its VCP scan. From the RHIs, we estimate that the cell’s maximum echo-top height grew from 11.6 to 13.1 km and its maximum reflectivity intensified from 60 to 66 dBZ.
Temporal evolution of the 7 Aug 2022 convective cell between 1852:26 and 1903:41 UTC as viewed by (a)–(f) the CSAPR2 “center” PPI and (g)–(l) the CSAPR2 “maximum reflectivity” RHI. The black line in each “center” PPI panel shows the location of the corresponding “maximum reflectivity” RHI.
Citation: Journal of Atmospheric and Oceanic Technology 40, 11; 10.1175/JTECH-D-23-0043.1
While the NEXRAD provides observations of the entire volume around the cell, a look at one of the CSAPR2 and CHIVO scan bundles shows that they captured at least part of the subkilometer spatial variability of this cell. Figures 7a–c respectively show a horizontal cross section through the base, middle, and near the top of the cell. From those, it is clear that this cell was widest at its base and narrowest at its top and that the cell’s tallest point was in the vicinity of its area of maximum radar reflectivity (near −5 km W, 32 km N of the CSAPR2). Note the black lines overlaid on the center PPI image, which show the location where the tracking radars collected vertical cross sections of the cell only seconds later (solid black for the CSAPR2 and dashed black for the CHIVO). CSAPR2 RHI3–6 (Figs. 7f–i) passed through an intense portion of the storm and revealed an arc shape 60 dBZ reflectivity feature as well as a clear picture of a newly forming child cell at 39 km distance from the CSAPR2. The RHI guided toward the geometric center of the storm (i.e., RHI1; Fig. 7d) instead shows a less intense portion of the storm. Similarly, the RHIs collected by CHIVO (Figs. 6j–l) reveal information about the northeast sector of the storm, which experienced a significant amount of directional shear as evidenced by the tilted features observed. Broadening our comparison to the period between 1852:52 and 1904:35 UTC allows us to look at both a period of cell growth and a period of cell decay. The CSAPR2 RHIs (Figs. 8g–l) reveal vertical structural changes in the storm, such as the gradual deepening of a 40 dBZ column engulfing an intense core (>60 dBZ) core followed by their collapse and the formation of an anvil cloud. Such a progression and amount of detail is not as obvious in the corresponding NEXRAD scans (Figs. 1b,c). Further analysis should help quantify the scientific value of collecting scan bundles containing both sector RHIs and sector PPIs. Such an analysis is however beyond the scope of this study which focuses on detailing the methodology and quantifying the amount of convective cell data collected using MAAS during the TRACER and ESCAPE field campaigns.
4. Convective cell statistics during the campaigns
MASS operated for 680 h during the TRACER and ESCAPE campaigns (1 June–30 September 2022). During that time MAAS identified a total of 245 267 unique cells in the NEXRAD observations within a 250 km × 250 km domain centered around the CSAPR2 location in Houston (the domain pictured in Fig. 2). Most of the cells were identified and tracked in August and September and over 50% lasted less than 10 min (Fig. 9, top panel; black bars). Note that we verified the validity of the NEXRAD-based tracks through visual inspection and feel confident that the several layers of data processing employed in MAAS yield reasonable results.
(a) Distribution of cell duration observed by the KHGX (black bars) and by the CSAPR2 (red bars) during the campaigns and (b) number of scan bundles collected by the CHIVO (green bars) and the CSAPR2 (red bars) during the entire campaign and the subset of CSAPR2 scan bundled acquired using online user input to the MAAS framework.
Citation: Journal of Atmospheric and Oceanic Technology 40, 11; 10.1175/JTECH-D-23-0043.1
Of those unique cells, 1337 were selected for tracking by the CSAPR2. Fifty-three percent of the convective cells tracked by the CSAPR2 were tracked for more than 15 min and 25% of the cells were tracked for more than 30 min (Fig. 9, top panel; red bars). Tracking was performed automatically 80.19% of the time leading to the collection of 17 708 CSAPR2 scan bundles most of which were collected in the late afternoon (Fig. 9, bottom panel; difference between the red and blue bar). The other 19.81% of the time, MAAS relied on input from an online user to select cells for tracking leading to the collection of 4332 CSAPR2 scan bundles most of them collected in the afternoon (Fig. 9, bottom panel; blue bars). As discussed in section 3, every CSAPR2 scan bundle includes three PPIs and four to six RHIs.
The CHIVO came online on 1 August 2022 and operated through 30 September 2022. CHIVO mostly operated based on guidance from CSAPR2 thus sampling the same cells from a different viewing angle. Exceptions to that occurred when the selected cell was within the CHIVO blanking sector (then CHIVO operated under “lightning mode”) and when CSAPR2 was not in operation under MAAS yet a user was online to provide input. All considering, CHIVO collected a total of 75 730 scan bundles most of them in the daytime (Fig. 9, bottom panel; green bars). As discussed in section 3, the CHIVO scan bundle includes three to four RHIs.
5. Summary and conclusions
The Multisensor Agile Adaptive Sampling (MAAS) smart sensing framework is an expandable communications and computation cyberinfrastructure for optimizing atmospheric experimentation. MAAS was adapted to support the TRACER and ESCAPE field campaigns, which collectively aimed to advance observation-based understanding of convective clouds throughout their life cycles in contrasting meteorological regimes. This new iteration of MAAS includes the addition of a finessed cell tracking and advection module operating remotely as well as of two edge computing modules tailored to the second-generation C-band scanning ARM precipitation radar (CSAPR2) and the Colorado State University C-band Hydrological Instrument for Volumetric Observation (CHIVO) radar, which use different command syntax and data structures, and had differing access policies, as determined by their respective owner institutions.
For 680 h between 1 June and 30 September 2022, MAAS was used to steer the C-band radars. Within the 250 km × 250 km domain, MAAS automatically processed operational observations from NEXRAD and GOES GLM within less than a minute of their acquisition to isolate, identify, characterize, and track all convective cells forming in the Houston domain. A set of predefined rules was used to select and nowcast the location of cells that would become the target of the CSAPR2 and CHIVO radar. Alternatively, to satisfy an interest to sample other storm types, such as those forming in clusters or producing large anvils, input from an online user was used to manually identify targets for the CSAPR2 and CHIVO 19.66% of the periods. For each selected cell, the CSAPR2 collected at least one bundle of three sector PPIs and two sector RHIs based on this guidance. Edge computing was then used to analyze the CSAPR2 center PPI in near–real time to identify two additional RHI targets for the CSAPR2. These targets were shared with the CHIVO over the MAAS communication framework so that the CHIVO could perform even higher-temporal-resolution RHI observations of the same convective cell from a complementary vantage point.
The rapid sequence of targeted sector PPI and RHI scan bundles collected by the CSAPR2 and CHIVO enabled the sampling of about 1300 unique isolated cells over parts of their life cycle with most being sampled for at least 15 min at scan repetition times less than 2 min. The combinations of PPIs and RHIs collected was intended to provide unprecedented views through convective systems: 1) high-resolution gap-free views of their vertical structure, 2) views of their spatial variability (at subkilometer scale), and 3) views of their evolution (at sub-2-min time scale) and all that with little to no human intervention. Although preliminary analysis reveals potential benefits, additional research will be necessary to quantify the broader benefits of this particular strategy. That said, this study clearly demonstrated that near-real-time data transfer and communications among different instruments connected in a network enables the collection of large amounts of radar scans (over 315 000 sector RHIs in 680 h) containing cloud/precipitation information (as opposed to clear air) even when traditional mechanically scanning radars are used. We believe that such a cyberinfrastructure is key for integrating future observing technologies like drones and phased-array radars and for optimizing atmospheric experimentation and we are working to make MAAS available to the broader community.
Acknowledgments.
Contributions from P. Kollias, B. Treserras, and M. Oue were performed under the sponsorship of the National Science Foundation (Award AGS 2019932). Contributions from E. Luke were performed under the sponsorship of DOE Atmospheric Radiation Measurement program. M. Oue was also supported by the U.S. Department of Energy, Atmospheric System Research (Contract DE-SC0021160). K. Lamer, E. Luke, and P. Kollias were also supported by the U.S. Department of Energy, Atmospheric System Research (Contract DE-SC0012704). B. Dolan’s contributions were supported by INCUS, a NASA Earth Venture Mission, funded by NASA’s Science Mission Directorate and managed through the Earth System Science Pathfinder Program Office under Contract 80LARC22DA011. K. Lamer: Article conceptualization, writing (original draft); P. Kollias: Methodology, supervision, writing (review and editing), project administration, and funding acquisition; E. Luke: Methodology, investigation, software, writing (review and editing); B. Treserras: Methodology, investigation, software, data curation, visualization; M. Oue: Methodology, investigation, visualization, writing (review and editing); B. Dolan: Methodology.
Data availability statement.
Data collected by the CSAPR2 can be downloaded from the ARM Data Discovery Center (https://adc.arm.gov/discovery/#/results/instrument_class_code::csapr/site_code::hou), while data from the CHIVO during the ESCAPE campaign can be downloaded from the National Center for Atmospheric Research (NCAR) Earth Observations Laboratory (EOL) Field Catalog (https://data.eol.ucar.edu/dataset/619.017).
REFERENCES
Ayzel, G., M. Heistermann, and T. Winterrath, 2019: Optical flow models as an open benchmark for radar-based precipitation nowcasting (rainymotion v0.1). Geosci. Model Dev., 12, 1387–1402, https://doi.org/10.5194/gmd-12-1387-2019.
Bentley, J. L., 1975: Multidimensional binary search trees used for associative searching. Commun. ACM, 18, 509–517, https://doi.org/10.1145/361002.361007.
Brown, R. A., V. T. Wood, R. M. Steadham, R. R. Lee, B. A. Flickinger, and D. Sirmans, 2005: New WSR-88D Volume Coverage Pattern 12: Results of field tests. Wea. Forecasting, 20, 385–393, https://doi.org/10.1175/WAF848.1.
Byers, H. R., and R. R. Braham, 1949: The Thunderstorm: Report of the Thunderstorm Project. U.S. Government Printing Office, 287 pp.
Chrisman, J. N., 2013: Dynamic scanning. NEXRAD Now, No. 22, NEXRAD ROC, Norman, OK, 1–3, https://www.roc.noaa.gov/WSR88D/PublicDocs/NNOW/NNow22c.pdf.
Crum, T. D., R. E. Saffle, and J. W. Wilson, 1998: An update on the NEXRAD program and future WSR-88D support to operations. Wea. Forecasting, 13, 253–262, https://doi.org/10.1175/1520-0434(1998)013<0253:AUOTNP>2.0.CO;2.
Dawson, D. T., E. R. Mansell, and M. R. Kumjian, 2015: Does wind shear cause hydrometeor size sorting? J. Atmos. Sci., 72, 340–348, https://doi.org/10.1175/JAS-D-14-0084.1.
Diffenbaugh, N. S., M. Scherer, and R. J. Trapp, 2013: Robust increases in severe thunderstorm environments in response to greenhouse forcing. Proc. Natl. Acad. Sci. USA, 110, 16 361–16 366, https://doi.org/10.1073/pnas.1307758110.
Fridlind, A. M., and Coauthors, 2019: Use of polarimetric radar measurements to constrain simulated convective cell evolution: A pilot study with Lagrangian tracking. Atmos. Meas. Tech., 12, 2979–3000, https://doi.org/10.5194/amt-12-2979-2019.
Hartmann, D. L., M. E. Ockert-Bell, and M. L. Michelsen, 1992: The effect of cloud type on Earth’s energy balance: Global analysis. J. Climate, 5, 1281–1304, https://doi.org/10.1175/1520-0442(1992)005<1281:TEOCTO>2.0.CO;2.
Hu, J., and Coauthors, 2019: Tracking and characterization of convective cells through their maturation into stratiform storm elements using polarimetric radar and lightning detection. Atmos. Res., 226, 192–207, https://doi.org/10.1016/j.atmosres.2019.04.015.
Kingfield, D. M., and M. M. French, 2022: The influence of WSR-88D intra-volume scanning strategies on thunderstorm observations and warnings in the dual-polarization radar era: 2011–20. Wea. Forecasting, 37, 283–301, https://doi.org/10.1175/WAF-D-21-0127.1.
Kollias, P., N. Bharadwaj, K. Widener, I. Jo, and K. Johnson, 2014: Scanning ARM cloud radars. Part I: Operational sampling strategies. J. Atmos. Oceanic Technol., 31, 569–582, https://doi.org/10.1175/JTECH-D-13-00044.1.
Kollias, P., and Coauthors, 2020a: The ARM radar network: At the leading edge of cloud and precipitation observations. Bull. Amer. Meteor. Soc., 101, E588–E607, https://doi.org/10.1175/BAMS-D-18-0288.1.
Kollias, P., E. Luke, M. Oue, and K. Lamer, 2020b: Agile adaptive radar sampling of fast‐evolving atmospheric phenomena guided by satellite imagery and surface cameras. Geophys. Res. Lett., 47, e2020GL088440, https://doi.org/10.1029/2020GL088440.
Kollias, P., and Coauthors, 2022: Science applications of phased array radars. Bull. Amer. Meteor. Soc., 103, E2370–E2390, https://doi.org/10.1175/BAMS-D-21-0173.1.
Ladino, L. A., A. Korolev, I. Heckman, M. Wolde, A. M. Fridlind, and A. S. Ackerman, 2017: On the role of ice-nucleating aerosol in the formation of ice particles in tropical mesoscale convective systems. Geophys. Res. Lett., 44, 1574–1582, https://doi.org/10.1002/2016GL072455.
Lucas, B. D., and T. Kanade, 1981: An iterative image registration technique with an application to stereo vision. Seventh Int. Joint Conf. on Artificial Intelligence, Vancouver, BC, Canada, IJCAI, 674–679.
Marshall, J. S., and W. M. K. Palmer, 1948: The distribution of raindrops with size. J. Meteor., 5, 165–166, https://doi.org/10.1175/1520-0469(1948)005<0165:TDORWS>2.0.CO;2.
McLaughlin, D., and Coauthors, 2009: Short-wavelength technology and the potential for distributed networks of small radar systems. Bull. Amer. Meteor. Soc., 90, 1797–1818, https://doi.org/10.1175/2009BAMS2507.1.
Miller, M. A., K. Nitschke, T. P. Ackerman, W. R. Ferrell, N. Hickmon, and M. Ivey, 2016: The ARM mobile facilities. The Atmospheric Radiation Measurement (ARM) Program: The First 20 Years, Meteor. Monogr., No. 57, Amer. Meteor. Soc., https://doi.org/10.1175/AMSMONOGRAPHS-D-15-0051.1.
Morrison, H., and Coauthors, 2020: Confronting the challenge of modeling cloud and precipitation microphysics. J. Adv. Model. Earth Syst., 12, e2019MS001689, https://doi.org/10.1029/2019MS001689.
Ryzhkov, A. V., M. R. Kumjian, S. M. Ganson, and P. Zhang, 2013: Polarimetric radar characteristics of melting hail. Part II: Practical implications. J. Appl. Meteor. Climatol., 52, 2871–2886, https://doi.org/10.1175/JAMC-D-13-074.1.
Sanderson, B. M., C. Piani, W. J. Ingram, D. A. Stone, and M. R. Allen, 2008: Towards constraining climate sensitivity by linear analysis of feedback patterns in thousands of perturbed-physics GCM simulations. Climate Dyn., 30, 175–190, https://doi.org/10.1007/s00382-007-0280-7.
Schmit, T. J., Griffith, P., M. Gunshor, J. Daniels, S. Goodman, and W. Lebair, 2017: A closer look at the ABI on the GOES-R series. Bull. Amer. Meteor. Soc., 98, 681–698, https://doi.org/10.1175/BAMS-D-15-00230.1.
Schrom, R. S., and M. R. Kumjian, 2016: Connecting microphysical processes in Colorado winter storms with vertical profiles of radar observations. J. Appl. Meteor. Climatol., 55, 1771–1787, https://doi.org/10.1175/JAMC-D-15-0338.1.
Seeley, J. T., and D. M. Romps, 2015: The effect of global warming on severe thunderstorms in the United States. J. Climate, 28, 2443–2458, https://doi.org/10.1175/JCLI-D-14-00382.1.
Sherwood, S. C., S. Bony, and J.-L. Dufresne, 2014: Spread in model climate sensitivity traced to atmospheric convective mixing. Nature, 505, 37–42, https://doi.org/10.1038/nature12829.
Snyder, J. C., A. V. Ryzhkov, M. R. Kumjian, A. P. Khain, and J. Picca, 2015: A ZDR column detection algorithm to examine convective storm updrafts. Wea. Forecasting, 30, 1819–1844, https://doi.org/10.1175/WAF-D-15-0068.1.
Stein, T. H., R. J. Hogan, P. A. Clark, C. E. Halliwell, K. E. Hanley, H. W. Lean, J. C. Nicol, and R. S. Plant, 2015: The DYMECS project: A statistical approach for the evaluation of convective storms in high-resolution NWP models. Bull. Amer. Meteor. Soc., 96, 939–951, https://doi.org/10.1175/BAMS-D-13-00279.1.
Torres, S. M., and Coauthors, 2016: Adaptive-weather-surveillance and multifunction capabilities of the National Weather Radar Testbed phased array radar. Proc. IEEE, 104, 660–672, https://doi.org/10.1109/JPROC.2015.2484288.
Varble, A. C., and Coauthors, 2021: Utilizing a storm-generating hotspot to study convective cloud transitions: The CACTI experiment. Bull. Amer. Meteor. Soc., 102, E1597–E1620, https://doi.org/10.1175/BAMS-D-20-0030.1.
Zhao, M., 2014: An investigation of the connections among convection, clouds, and climate sensitivity in a global climate model. J. Climate, 27, 1845–1862, https://doi.org/10.1175/JCLI-D-13-00145.1.
