The potential value of small unmanned aircraft systems (UAS) for monitoring the preconvective environment and providing useful information in real time to weather forecasters for evaluation at a National Weather Service (NWS) Forecast Office are addressed. The general goal was to demonstrate whether a combination of fixed-wing and rotary-wing UAS can provide detailed, accurate, and useful measurements of the boundary layer important for determining the potential for convection initiation (CI). Two field operations were held: a validation study in which the UAS data were compared with collocated measurements made by mobile rawinsondes and ground-based remote sensing systems and a real-time experiment held to evaluate the potential value of the UAS observations in an operationally relevant environment. Vertical profile measurements were made by the rotary-wing UAS at two mesonet sites every 30 min up to 763 m (2500 ft) AGL in coordination with fixed-wing UAS transects between the sites. The results showed the ability of the fixed-wing UAS to detect significant spatial gradients in temperature, moisture, and winds. Although neither of two different types of rotary-wing UAS measurements were able to strictly meet the requirements for sensor accuracy, one of the systems came very close to doing so. UAS sensor accuracy, methods for retrieving the winds, and challenges in assessing the representativeness of the observations are highlighted. Interesting mesoscale phenomena relevant to CI forecasting needs are revealed by the UAS. Issues needing to be overcome for UAS to ever become a NOAA operational observing system are discussed.
It has been more than a decade since the National Research Council (2009, 2010) articulated the need for establishing a nationwide mesoscale network to address severe limitations in sampling the atmosphere. Those reports and a follow-on thermodynamic profiling workshop (Hardesty and Hoff 2012) recommended that profiles of wind, temperature, and moisture should extend to 3 km above ground level (AGL), and that for the prediction of convection initiation (CI), a time resolution of 15 min, a vertical resolution of 30 m close to the surface degrading to 100 m at 3 km, a horizontal resolution <10 km, and a bias <5% are needed.
These stringent requirements for forecasting CI exist because potential instability, moisture, vertical wind shear, and mesoscale circulations can display large temporal and spatial variability (Benjamin et al. 2004; Weckwerth and Parsons 2006; Ziegler et al. 2010; Otkin et al. 2011; Hartung et al. 2011; Illingworth et al. 2015; Koch et al. 2016). According to Dabberdt et al. (2005, 961–962), “the full benefit of enhanced forecast model resolution has not been and will not be realized without commensurate improvements in high-resolution meteorological observations (italics added), as well as improvements in data assimilation, model physics, parameterizations, and user-specific analyses and forecast products.”
It is unknown what spatial and temporal scales are important for measurement of the preconvective storm environment (Potvin et al. 2010). Across the southern Great Plains of the United States, a region commonly referred to as Tornado Alley, very strong horizontal gradients in water vapor (>1 g kg−1 km−1) may exist across the oft-present dryline over very small distances (Ziegler et al. 1995; Thompson et al. 2003; Buban et al. 2007). Immediately to the east of the dryline, a very moist boundary layer exists, but it is typically capped by a prominent inversion prior to the development of deep convection, allowing the buildup with diurnal heating of substantial convective available potential energy (CAPE). However, the horizontal variability in the strength and depth of the capping inversion is poorly known, and the destruction of this inversion can happen quickly within a couple of hours, as revealed by ground-based remote sensing systems (Wagner et al. 2008; Koch et al. 2016). This need for very high spatial and temporal resolution also exists for monitoring cross-dryline and cross-frontal vertical circulations often acting as mechanisms for release of the potential instability. These circulations may display spatial scales of 5–50 km and require very-high-vertical-resolution measurements (Atkins et al. 1998; Markowski et al. 1998; Koch and Clark 1999; Weiss and Bluestein 2002; Wakimoto et al. 2006; Buban et al. 2012). Also, supercell storms occur in environments containing substantial vertical wind shear within 1 km of the ground (Rasmussen 2003; Thompson et al. 2003).
The ability of ground-based remote sensing systems to detect significant changes in the 1–2 h prior to CI has been demonstrated (Koch and Clark 1999; Güldner and Spänkuch 2001; Feltz and Mecikalski 2002; Benjamin et al. 2004; Wagner et al. 2008; Knupp et al. 2009; Güldner 2013; Madhulatha et al. 2013; Ratnam et al. 2013; Koch et al. 2016). However, the vertical resolution of these passive remote sensors and lidars degrades rapidly with height above the surface. Aerosol and Doppler wind lidars provide higher resolution and greater accuracy than passive remote sensing systems, but as with infrared sensors, these systems have limited to no ability to profile above optically thick clouds. Moreover, the cost of implementing such remote sensing systems in a large network may be prohibitive (though most of these systems can run unattended with minimal attention for months to years).
It is to these unfilled needs in observing capability that unmanned aircraft systems (UAS) may be directed. We are not aware of any published study that has shown the value of UAS for monitoring the preconvective environment and providing useful information in real time to weather forecasters for evaluation at a National Weather Service (NWS) Forecast Office. The present study attempts to address this need for the first time in a project called Environmental Profiling and Initiation of Convection (EPIC). EPIC’s goal was to demonstrate whether a combination of fixed-wing and rotary-wing UAS can provide detailed and accurate measurements of changes in temperature, moisture, and winds within the atmospheric boundary layer (ABL) important for determining the potential for CI. A number of subsidiary questions were intended to be addressed in EPIC:
What is the maximum altitude and vertical sampling interval needed to capture the strength of important features in the severe storm environment?
What is the required frequency of soundings from the fixed sites using the rotary-wing UAS to capture important temporal variability in these fields, the spatial density of the soundings needing to be collected, and to what extent is supplementary mapping by fixed-wing UAS critical?
How potentially significant would be the warning and forecast improvements realized by having these data available in an NWS Forecast Office?
Methods for deriving high-quality meteorological data from low-altitude, short-endurance (LASE) UAS and laboratory calibration of the specific instruments used in EPIC are discussed in section 2. The experimental design and findings from the calibration/validation (phase 1) and the real-time CI evaluation (phase 2) of EPIC are presented in sections 3 and 4, respectively. Examples illustrating the potential of the UAS to detect important meteorological phenomena for CI forecasting are provided in section 5.
2. Data and methodology
a. Operation of UAS for mesoscale CI studies
UAS are appropriate for “dull, dirty, or dangerous” missions where traditional observing systems are challenged (Houston et al. 2012, p. 42), yet there have been few attempts to observe mesoscale phenomena with UAS. Houston et al. (2012) note that many of the atmospheric science projects utilizing UAS have been conducted over the oceans (e.g., Holland et al. 2001; Cione et al. 2016), in large part because risks resulting from encountering other aircraft and endangering people and property on the surface is nearly nonexistent. A notable exception is flights of fixed-wing UAS across supercell outflow boundaries or cold fronts (Elston et al. 2011, 2015; Frew et al. 2012; Houston et al. 2012; Riganti and Houston 2017). UAS measurements have also been used recently to obtain profiles of sensible heat flux (Lee et al. 2017), of relevance to CI.
Rotary-wing UAS offer certain advantages. The capability for vertical takeoff and landing makes boundary layer profiling possible. Rapid temporal changes of meteorological variables at a fixed location can be obtained with minimal cost and relative ease of setup and operation. However, the accuracy of these measurements is conditional on several factors. Regarding the measurement of temperature, for example, the instrument package needs to be located on the platform such that there is adequate ventilation and proper solar shielding (Jacob et al. 2018). When thermistors are used to measure air temperature, as was the case with the rotary-wing UAS used in EPIC, heat is generated as electric current is run across the thermistor to obtain the resistance of the temperature sensitive resistor. Experiments conducted after the end of EPIC by project participants at the University of Oklahoma revealed that temperatures may be biased by as much as 1.0°C in the absence of ventilation when the sensors are placed close to the tips of the rotors or motors (Greene et al. 2018).
Neither is accurate measurement of winds by rotary-wing UAS a trivial exercise. They move through the air by setting a tilt angle toward the flying direction with the magnitude of the tilt angle roughly proportional to speed and varying to compensate for wind variations during the flight. An onboard inertial measurement unit (IMU) measures accelerations, angular rate, and pitch, roll, and yaw angles using a combination of accelerometers, gyroscopes, and magnetometers, from which the horizontal winds are estimated. In contrast to a fixed-wing aircraft, which is controlled by setting a true airspeed (TAS), rotary-wing UAS fly with a given ground speed, resulting in varying TAS. The ground vector represents the velocity of the rotor-wing UAS’s movement determined by the GPS, while the TAS vector represents the actual velocity toward which the multirotor copter is heading. The deviation of the ground vector from TAS is caused by the wind. Brosy et al. (2017) found the accuracy of wind direction and speed estimation by hovering a typical UAS next to an ultrasonic anemometer to be ±14.5° and 0.7 m s−1, respectively.
The potential of UAS to provide measurements of the atmosphere is promising, but there are big engineering and regulatory hurdles that must first be overcome. Federal Aviation Administration (FAA) guidelines for safe, allowable operation of UAS include the requirement that they be flown within visual line of sight of an operator under the watchful oversight of a trained pilot in command, and at altitudes not exceeding 400 ft (122 m). The operation of UAS as “public aircraft,” which includes those owned and operated by the U.S. state and federal governments, requires obtaining a Certificate of Authorization or Waiver (COA) from the above-stated restrictions for each aircraft and geographic region. COA approval requires an airworthiness statement, operational provisions, and contingency procedures that will be executed for possible equipment malfunctions or emergencies.
b. Project description and design
The calibration of meteorological sensors in controlled chambers or reliance upon manufacturer specs for sensors is not sufficient for determining the accuracy and reliability of atmospheric measurements made by UAS, because they fail to account for the impacts of turbulence, sensor placement on the platform, representativeness errors, etc. Practical accuracy is determined by how well the UAS sensors compare with reference to collocated measurements taken by independent observing systems of generally accepted high quality, such as rawinsonde balloon launches and ultrasonic anemometers. EPIC observation accuracy goals for meteorological measurement validation purposes are shown in Table 1. These goals, which are identical to the Jacob et al. (2018, p. 10) “desired specification goals,” were developed from an informal consensus of atmospheric scientists, National Weather Service, and other subject matter experts in the community. Strictly speaking, they do not represent formal NOAA requirements, yet they are used by the NOAA UAS Program Office as guidelines to follow in evaluation of potential UAS platforms for operational consideration.
EPIC consisted of two field operations: 1) phase 1: a calibration/validation (cal/val) multiobserving system intercomparison study conducted at the Department of Energy (DOE)’s Southern Great Plains (SGP; Sisterson et al. 2016) Central Facility site near Lamont, Oklahoma, on 29–30 October 2016; and 2) phase 2: a field exercise held on 5 days in May 2017 in northern Oklahoma designed to evaluate the potential value of the UAS observations in an operationally relevant real-time environment. Vertical profile measurements were made every 30 min by the UAS up to 400 ft (122 m) AGL in phase 1 and to 2500 ft (763 m) in phase 2 (as approved in the FAA COA process). These rotary-wing UAS profiles were made in coordination with fixed-wing UAS transects between two sites to achieve nearly contemporaneous measurements as the aircraft loitered over the two end points and the mobile vehicles launched rawinsondes.
During phase 1, one of the two sites was at the SGP Central Facility site because of the abundant observing systems available for cross validation of the UAS measurements. The second site was chosen to be at the Oklahoma Mesonet station at Medford (MDF; Fig. 1). The Oklahoma Mesonet comprises 120 automated weather stations located in each of Oklahoma’s 77 counties (Brock et al. 1995; McPherson et al. 2007; Fiebrich et al. 2010). At each site, surface observations of temperature, relative humidity, winds, and other variables are made every 5 min, providing EPIC with valuable background information about mesoscale heterogeneity against which the added information from the UAS measurements could be assessed. During phase 2, two mesonet sites in northern Oklahoma were chosen daily in coordination with National Weather Service-Norman, which also provided weather forecast support for field operations decision-making purposes, as well as evaluation of the impact of the observations on their forecast process. Special observations available for intercomparisons in this study included ground-based remote sensing systems and an instrumented tall tower at the Lamont site, and remote sensing systems and special rawinsonde releases from a mobile platform [Collaborative Lower Atmospheric Mobile Profiling System (CLAMPS)] provided by NSSL.
c. CU TTwistor fixed-wing UAS
The University of Colorado Boulder (CU) TTwistor is a fixed-wing UAS integrated with 2.4-GHz Wi-Fi and 900-MHz communications and control links, a Pixhawk 1.0 autopilot, a Vaisala RS-92 sonde for pressure–temperature–humidity measurements, and an Aeroprobe five-hole probe for airframe-relative wind velocity measurements, converted to an inertial (ground relative) wind by use of a VectorNav VN-200 IMU. Meteorological data collected by the aircraft were transmitted in real time to a ground-based vehicle (tracker) via Wi-Fi. TTwistor routing decisions made by the meteorologist in command were communicated to the tracker containing the ground station, a pilot in command (PIC), and a visual observer (VO). CU obtained an extension of their COA enabling flight of the TTwistor in class E/G airspace up to 2500 ft (763 m) AGL for a region including the SGP and the selected Oklahoma Mesonet sites.
The TTwistor airframe (Fig. 2a) was developed from the earlier Tempest airframe with objectives to increase endurance and improve sensor placements (Elston et al. 2011, 2015; Frew et al. 2012; Houston et al. 2016). The single motor located in the nose of the Tempest required that the RS-92 sonde and five-hole probe be carried on the wing, but the wing is generally not an ideal location for these sensors on a small UAS, particularly in turbulent environments, which can induce relative motions of the sensors from the rolling motions of the airframe and flexing in the wing structure. With no motor occupying the nose of the aircraft, the twin-motor configuration of the TTwistor allows a more favorable location for the RS-92 on the fuselage near the nose and the five-hole probe extending directly from the aircraft nose.
The accuracies of meteorological data shown in Table 2 for the Tempest/TTwistor are based on the sensor placements on the Tempest reported in Houston et al. (2016), in which sensor quality was determined by making intercomparisons between the UAS sensors atop an NSSL mobile mesonet van. The intercomparison showed that differences between the temperature and relative humidity data collected by this UAS and the van were within sensor accuracies. The accuracies of the pressure and relative humidity measurements exceed the EPIC requirements (Table 1), whereas the temperature measurement is slightly less accurate. Wind accuracy shown in Table 2 is based on the wind measured relative to the five-hole probe prior to the transformation of that measurement into the ground-relative frame that employs GPS and aircraft-state data from the IMU. Note that Tempest/TTwistor specifications indicate an endurance of as much as 3 h under ideal flying conditions.
d. OU CopterSonde rotary UAS
The University of Oklahoma (OU) originally planned to adapt a DJI Flame Wheel F550 UAS to the task of accurately profiling the atmosphere in EPIC, but restrictions on the use of the data imposed by the vendor persuaded OU to develop an entirely new UAS for this project—the CopterSonde (Fig. 2b), an octocopter with a weight of 12.7 lb (5.8 kg) and a diameter of 2.3 ft (0.7 m). Typical flight duration achieved was 25 min, limited by battery lifetime, with a maximum ascent rate of 3300 ft min−1 (16.8 m s−1). The CopterSonde utilized a Pixhawk PX4 autopilot with IMU and GPS and an InterMet iMet-XF sensor, which met EPIC’s measurement accuracy specifications. In addition, this “hashtag”-designed UAS was subjected to comparisons with the Oklahoma Mesonet tower and in OU calibration laboratory facilities prior to being utilized in EPIC.
e. Meteomatics Meteodrone rotary UAS
The Meteomatics Meteodrone (Fig. 2c) is a hexacopter with a span of 1.0 ft (0.3 m) and a weight of 1.5 lb (0.7 kg). Typical flight duration was 12 min for an ascent rate of 600 ft min−1 (3.0 m s−1). The Meteodrone used a Mikrokopter autopilot with IMU and GPS. Meteomatics verified that its sensors could provide relative humidity accuracy to ±1.8% with a response time shorter than 4 s and temperature to an accuracy of ±0.1°C. Even though it had been demonstrated in Switzerland that the Meteodrone could provide vertical profiles to 2500 ft (763 m) in 25 min given its nominal climb and descent rate, this needed to be tested in the windy environment of Oklahoma. OU assumed responsibility for the safe operation and liability of this UAS during the project to enable getting a COA for the Meteodrone.
f. Independent observing systems
The calibration values for the various UAS used in this study all needed to be validated in the real atmosphere, where turbulence, solar heating, and other factors might be important. One of the independent observing systems used for UAS measurement validation in EPIC was a mobile remote sensing facility called CLAMPS (Fig. 2d) for field verification of the UAS measurements. CLAMPS (Wagner et al. 2018) consists of an Atmospheric Emitted Radiance Interferometer (AERI; Knuteson et al. 2004) and a microwave radiometer (MWR; Ware et al. 2003) for thermodynamic profiling and a Halo Photonics Stream Line Doppler wind lidar (DWL) for wind profiling in the optically clear lower atmosphere (Pearson et al. 2009). These instruments provided a comprehensive assessment of the evolution of the atmosphere at a temporal resolution of 5 min or better. CLAMPS and a mobile NSSL mesonet vehicle also provided an invaluable rawinsonde launch capability.
AERI instruments were available at the SGP site and on the CLAMPS-2 system. Although AERI is not capable of fully resolving the strength of capping inversions, its 2-min temporal resolution is outstanding for detecting rapid changes in moisture and thermodynamic instability in environments supportive of deep, moist convection (Feltz and Mecikalski 2002; Wagner et al. 2008; Blumberg et al. 2017). AERI observes downwelling infrared radiance emitted by the atmosphere and retrieves profiles of temperature and water vapor and other variables using an optimal estimation–based physical retrieval algorithm from the radiance data (Turner and Löhnert 2014). This algorithm offers the advantage that the observational uncertainty and the sensitivity of the forward model are propagated to provide a full characterization of the uncertainty of the solution. More than 80% of AERI information content is contained in the lowest 2 km of the atmosphere for temperature and water vapor profiles (Turner and Löhnert 2014; Turner and Blumberg 2018). Under clear-sky situations, the mean bias errors with respect to radiosonde profiles are 0.2 K and 0.3 g kg−1 with rms errors of 1.0 K and 0.8 g kg−1 for temperature and water vapor mixing ratio, respectively.
An advantage of microwave radiometers is that they can obtain vertical profiles of temperature and water vapor density under most weather conditions (except heavy rain) in the lower troposphere. CLAMPS and SGP utilize the Radiometrics 35-channel MP-3000 MWR (Güldner and Spänkuch 2001; Ware et al. 2003; Liljegren et al. 2005) to observe brightness temperatures related to atmospheric moisture in 21 frequency bands (22–30 GHz) and temperature in 14 bands (51–59 GHz). Typical errors associated with temperature and humidity retrievals from this radiometer are similar to radiosonde errors: 0.6 K near the surface, increasing to 1.6 K at 7 km, and 0.25 g m−3 error near the surface, increasing to 0.90 g m−3 error at 2 km, respectively. As is the case for the AERI, the effective vertical resolution of the MWR is best near the surface, degrading with height. Koch et al. (2016) discuss the neural network technique used to retrieve temperature and humidity profiles from the MWR.
The DWL systems used by CLAMPS and the SGP site operate in two modes: one mode collects plan position indicator (PPI) scans at a constant elevation periodically, with the second mode collecting 1-s vertical motion data. The PPI data are processed using the velocity–azimuth display (VAD) technique, in which the conical scan creates line-of-sight velocities as a function of azimuth angle, from which it is possible to retrieve all three components of the wind vector. The CLAMPS and SGP DWL systems measured the Doppler shift of the backscattered energy every 2 min with a scanner at 60° elevation. This turned out to be suboptimal, as only two to three range gates were obtainable within the very shallow 400-ft layer near the ground where we could operate.
3. EPIC phase 1: Calibration/validation field study
Contemporaneous UAS and other observing systems were used in phase 1 to cross validate the UAS sensor measurements under real atmospheric conditions, in contrast to calibration work performed under controlled laboratory conditions. The independent observing systems utilized included an instrumented 60-m-tall tower at the SGP Central Facility location, the CLAMPS and mobile mesonet rawinsonde systems, and the remote sensing systems discussed above.
Execution and coordination for this project were challenging, and some problems were encountered, as discussed below, but the lessons learned and the overall excellent quality of the data collected by the various platforms made phase 1 an unqualified success. TTwistor operated under a blanket COA for the EPIC region in class G airspace below 400 ft (122 m) AGL. The waiver request from OU to the FAA to fly to 2500 ft (762 m) was not approved prior to phase 1, so none of the UAS platforms flew above 400 ft (122 m) AGL,
Regarding the rotary-wing UAS, the ascent and descent speeds were the same (~3 m s−1). The measurements from the ascent and descent flights were then combined through averaging of data at the same altitude to cancel out the effects of relative venting differences (Jacob et al. 2018). Profiles of temperature obtained by an NSSL rawinsonde released at the Lamont SGP site and a collocated Meteodrone launch are shown in Fig. 3. The horizontal separation between the UAS and the rawinsonde release point was <200 m and 15 min in an attempt to make the measurements as contemporaneous as possible. The two profiles are quite similar to one another, with differences being smaller than 0.2°C, though the Meteodrone did display a small warm bias with respect to the balloon data.
Example profiles of temperature taken by the SGP AERI located at Medford, the collocated CLAMPS AERI, the Meteodrone, and an NSSL radiosonde launched from Medford are displayed in Fig. 4. This intercomparison exemplifies one of the challenges in using the remotely sensed observations to validate the UAS measurements—the two AERI retrievals differed from one another by an amount comparable to the difference of the Meteodrone temperature values from the SGP values. Fortunately, the differences between the two sets of collocated AERI retrievals mostly fall within the accuracy goal for temperature (±0.2°C; Table 1); similarly, the Meteodrone values are systematically only ~0.2°C warmer than the AERI values of temperature, though they are ~0.6°C warmer than the NSSL radiosonde, suggesting the possibility that the balloon might have been more the outlier than the other observing systems.
A similar intercomparison is shown for relative humidity measurements in Fig. 5. Intercomparisons with the CopterSonde were also made but are not shown here because the comparisons revealed significant problems with this newly developed UAS that led to some system redesigns prior to the initiation of EPIC phase 2. Findings from an intercomparison performed in phase 2 upon a large number of flights (discussed in section 4) more fully characterize the relative accuracy of the measurements from the two rotary-wing UAS.
While the majority of the rawinsonde data collected by NSSL was of high quality, the winds were highly variable, as demonstrated in Fig. 6a. Undamped oscillations of ±2 m s−1 swing about the wind speed profiles obtained from the two Doppler wind lidars and the Meteodrone winds, which generally agree with one another quite well (±0.7 m s−1). The rawinsonde profile would, were a linear fit to the data to be shown, actually fall within the envelope of winds measured by the three other systems. We found that upon substituting the Vaisala RS-92 radiosonde with the new RS-41 system, such high-frequency, noisy wind behavior disappeared (Fig. 6b). Fortunately, we discovered this problem in this early stage of EPIC, so a solution could be found before phase 2 of the project began.
The expectation was that the CopterSonde would be able to fly in winds as high as 50 kt (25.7 m s−1) with an endurance of 25 min. This was subjected to testing on 29 October 2016, when strong southerly winds occurred in the project area within 400 ft (122 m) of the ground. The CopterSonde performed well under such harsh conditions.
One of the questions we sought to answer in EPIC was the required frequency of soundings from the fixed sites using the rotary-wing UAS to capture important temporal variability in these fields. The Meteodrone was flown in an experimental ultrahigh-sampling-frequency (90 s) mode at times. An example shown in Fig. 7 illustrates that under conditions of gusty, windy, cloudless conditions, the temporal variability (resulting from the likely passage of energy-containing eddies) was appreciable—with variability of ±2 m s−1 and ±0.2°C in just a few minutes. However, such intermittent variability is unimportant for capturing phenomena important to short-range weather forecasting. The best approach seemed to be to make as many vertical profiles as possible within a short time window and to average the profiles to obtain the most representative data. Hence, OU and Meteomatics employed 15-min sampling in many of the launches made in phase 2.
Another lesson learned is that the ability to maintain constant visual line of sight (VLOS) to an altitude of 2500 ft (763 m) would be questionable at best for the Meteodrone and CopterSonde, because even at only 400 ft (122 m) altitude, they appeared nearly imperceptible in the sky with the unaided eye. Thus, the two teams agreed to install strobe beacons on the rotary-wing UAS to enhance the VLOS to this high altitude for phase 2.
Flight-level measurements of relative humidity made by the TTwistor fixed-wing aircraft along a transect between Medford and Lamont are shown in Fig. 8. An 8% horizontal gradient in relative humidity measured along this 43-km diagonal transect agrees precisely with concurrent relative humidity differences seen at the adjacent Oklahoma Mesonet stations (48%–56%). A similar comparison of temperature measurements (not shown) revealed excellent agreement in measured spatial gradients of temperature between the TTwistor and Oklahoma Mesonet data.
4. EPIC phase 2: Real-time forecast impact experiment data validation
During this phase of the project in May 2017, the experimental UAS data—from the rotary-wing UAS and the fixed-wing TTwistor aircraft—as well as full-tropospheric rawinsonde data obtained from CLAMPS and the NSSL mobile mesonet van were provided to NWS-Norman Forecast Office in real time for evaluation. The NWS provided daily forecast project support, assisted with decisions concerning data collection and daily waypoint selection, and participated in postanalysis of the UAS data collected to gain insight into its potential operational forecast and warning value. We believe this coordination with an NWS Forecast Office represents a first in the field of UAS applications.
OU successfully obtained a COA waiver from the FAA for phase 2 allowing both the CopterSonde and Meteodrone to fly to 2500 ft (763 m) AGL at the Lamont SGP site and approved Mesonet sites. The Oklahoma Mesonet project manager worked with the landowners of all Oklahoma Mesonet sites selected for possible use (restricted to the area shown by the circle in Fig. 1 because of the effort involved) to obtain their permission allowing the EPIC team to conduct flight exercises over their properties. Data were collected in the prestorm environment multiple times on 5 of the 10 days tentatively set aside for phase 2 (because a forecast of strong convection was a requirement to operate).
The rotary copter data were binned into groups of flights defined by having at least one Meteodrone and one CopterSonde launched within 15 min of an NSSL rawinsonde release. This resulted in the identification of 37 such groups during the 5 days. Then, within each group, curves of best fit were applied to each atmospheric profile to statistically evaluate the measurements relative to the rawinsonde data. The rawinsondes were considered the reference standard, but a couple of the profiles from the two drones appeared more like each other than agreeing with the balloon data. Those cases were identified as ones where the rawinsonde baseline pressure values were erroneous, so they were not used as ground truth. Application of simple curve-fitting procedures produced these tentative results: 1) UAS temperature and relative humidity measurements exhibited differences from the rawinsondes (errors) that mostly fell within the accuracy standards (Table 1); 2) the Meteodrone and CopterSonde exhibited systematic dry biases of ~7% and 4% RH, respectively, and both displayed a slight warm bias of <0.5°C, suggesting a common cause; and 3) unacceptably large wind biases occurred in the CopterSonde measurements.
Rather than show those results here and discuss them in more detail, a more sophisticated statistical procedure was developed using bootstrapping (a random resampling with replacement procedure) and local regression (LOESS) to remove the assumption that the functional fit is invariant with height or time of the data. LOESS is a nonparametric regression method that fits a smooth curve to a set of data points without making any assumptions about the parametric global form of the regression function (Cleveland 1979, 1994; Cleveland and Devlin 1988). Confidence intervals were determined through bootstrap resampling (Efron and Tibshirani 1993) using 5000 replicates and empirical percentile limits. This combined approach, which allows accurate estimation of confidence intervals generated by the statistical resampling (replication), was applied to the entire dataset collected during the experiment. Thus, each day was treated as just another sample drawn from a universal dataset, rather than making direct comparisons with the rawinsonde data on an event basis (i.e., treating each flight group separately as was done with the simple curve-fitting technique). Scatterplot data were fit using a first-order regression every 10 m in height using a span of 0.3; that is, at any given data height, a linear least squares fit included data from 12 levels above and 12 levels below (Δz = 10 m).
The Meteodrone and CopterSonde bias errors as a function of height for temperature, relative humidity, wind speed, and wind direction measurements are shown in Figs. 9–12 , respectively. Bias is defined as the difference of the UAS measurement from that obtained from a collocated rawinsonde, assuming the latter is the standard for accuracy (after having removed the questionable soundings as discussed above). The mean bias error results indicate that neither UAS produced temperature and moisture measurements that exactly met the EPIC accuracy requirement (Table 1), but they came close. The results for CopterSonde and Meteodrone, respectively, are: (i) a warm temperature bias of +0.6° and +0.4°C (compared to the ±0.2°C accuracy requirement); (ii) a relative humidity dry bias of −4% and −7% (vs the ±5% requirement); (iii) a positive wind speed bias of +10.0 and +0.2 m s−1 (vs the ±0.5 m s−1 requirement); and (iv) a clockwise wind direction bias of 24° and 7° (vs the ±5° requirement). Thus, the Meteodrone outperformed the CopterSonde in most respects. Also apparent in these plots is that the CopterSonde exhibited a significant increase of temperature and wind direction bias error with height.
Tukey (1977) box-and-whisker plots of the error distribution of measured variables from the two UAS platforms (Fig. 13) reveal more information about the statistical nature of the intercomparisons, as it shows the various quartile ranges, median values, outliers, and the degree of dispersion (spread) and skewness in the data. Wind speed error distributions show that the CopterSonde measurements were considerably more disperse than the Meteodrone measurements and displayed a strong systematic positive bias. This aspect is best revealed in the histogram plot of errors shown in Fig. 14. Clearly, the Meteodrone produced highly accurate winds, both in speed and direction (nearly matching the accuracy requirement), whereas the CopterSonde was unable to provide acceptably accurate winds.
It is interesting that both the CopterSonde and Meteodrone exhibited slight warm biases in temperature and dry biases in relative humidity. We believe that the warm bias could be attributable to errors introduced by the flow of air over the multiple rotary motors despite attempts to minimize this undesirable effect. Quite intriguing was that the CopterSonde errors increased with height, whereas Meteodrone errors were nearly invariant with height, and that some of the errors were skewed, especially for temperature. This suggested wayward behavior under some conditions. Upon further investigation, it was discovered that the CopterSonde experienced a resonant vibration that impacted the performance of the platform and the quality of the data it produced. This had not been the case in phase 1, or at least it was not readily discernible. When the vertical vector of the CopterSonde oscillated wildly, the data became noisier and the warm bias noticeably greater. The winds measured by the CopterSonde did not agree well with either the Meteodrone or the balloon data. Importantly, the deviant behavior was detected and the problem was fixed by reinforcing the joints and using more robust motor mounts with increased diameter of the rotors to decrease the motor speeds. Unfortunately, the fix did not occur until after the end of the field phase.
5. EPIC phase 2: Detection of important meteorological phenomena
A number of interesting mesoscale features were detected by the rotary-wing UAS, most of which have direct bearing on the forecast of CI. However, there were factors (discussed in section 6) that limited the usefulness of the UAS data in NWS operations.
a. LLJ development in the afternoon ahead of storms: 16 May 2017
On this day, the NWS Storm Prediction Center (SPC) issued an “enhanced risk” of severe storms in the Texas Panhandle and western Oklahoma ahead of a dryline. Unfortunately, no airmass boundaries occurred in the EPIC domain (the reason that TTwistor data are not shown here). NSSL mobile soundings taken at two mesonet stations in northern Oklahoma (Marena and Breckenridge) showed that a strong capping inversion remained intact all day, though it did lift with time. In addition, the hodographs lengthened considerably late in the day as the upper-level jet approached the area (not shown). Meteodrone observations from Marena taken >200 km east of the developing storms along the dryline (Fig. 15) showed the sudden appearance of a stronger wind regime after 2100 UTC (an increase of ~5 m s−1) over a depth of the atmosphere exceeding that of the 763-m layer in which the UAS were permitted to fly. While the longer hodographs and the presence of this developing low-level jet (LLJ; notably before sunset) may not be pertinent to increasing the likelihood of CI, they are quite relevant to increasing the probability that any storms that formed would be severe (Thompson and Edwards 2000; Weisman and Rotunno 2000).
b. Mesoscale moisture plume in the warm sector ahead of storms: 18 May 2017
Anticipation of severe weather in the EPIC area was elevated this day, since SPC issued a high risk for tornadoes and very large hail for a large area inclusive of the EPIC domain. SPC later issued a particularly dangerous situation (PDS) tornado watch, reflecting the highest possible category of tornado risk. Many tornadoes did develop in Oklahoma on this day.
In coordination with NWS-Norman, the EPIC team set up transects with the TTwistor from west of Fairview eastward to Lahoma (Fig. 1). Surprisingly, and with great disappointment, none of the TTwistor tracks showed significant horizontal gradients of wind, temperature, or moisture. However, important phenomena for CI were detected by the rotary-wing UAS and NSSL soundings. Although it might be assumed that diurnal heating, along with the eastward advance of a pronounced dryline in the Texas Panhandle, helped to destabilize the atmosphere and generate explosive convection in western Oklahoma (Fig. 16), the CLAMPS full-tropospheric soundings taken at Fairview (Fig. 17) offered a different explanation: the rapid destruction of the capping inversion and convective inhibition in just 3 h (from 1700 to 2000 UTC) was the result of cooling in the 750–850-hPa layer. Also important is the increasing curvature of the wind hodograph over this span of time, which translates into increasing storm-relative helicity (Droegemeier et al. 1993). This example demonstrates the value of these special kinds of observations.
Though neither of these changes were detected by the rotary-wing UAS, as they occurred well above the 763-m limit, the Meteodrone observations taken at Fairview (Fig. 17) did reveal the sudden appearance of a deep layer of higher moisture content. This same feature was also seen in the observations from Lahoma ~30 min later. This suggests a translational speed for the moist plume of 65 km h−1 (18 m s−1). This feature was immediately followed by pronounced backing of the very-low-level winds, just as seen in the CLAMPS soundings, which resulted in a dramatic increase of storm-relative helicity.
c. Frontal updraft and moisture plume: 19 May 2017
Even though the weather was much less exciting this next day, some interesting mesoscale phenomena related to CI prediction were observed by the Meteodrone. The situation was that of a pronounced cold-frontal passage in the wake of strong convection that produced heavy rainfall in southern Oklahoma (Fig. 18). Cloud cover arising from the storms to the south kept surface temperatures below 80°F (26.5°C) ahead of the cold front, but quite cold air aloft would, according to the CLAMPS sounding taken at 2200 UTC, make CI possible in the absence of any convective inhibition (Fig. 19). An intriguingly narrow plume of greatly enhanced moisture (13 g kg−1) associated with strong convergence in the horizontal winds is implied by the time–height measurements taken by the Meteodrone at 2200 UTC at the Fairview site (Fig. 20). This convergence feature, which was along the leading edge of the cold front that was passing directly overhead at the time, would support a narrow updraft there, thus explaining the narrowness of the moisture plume. Also of great interest is the sudden appearance of a prefrontal low-level jet, a doubling of 6 m s−1 over the background value of ~7 m s−1. The deduced updraft jet, moisture plume, and the prefrontal jet explain the existence of a narrow band of convection along the front in the satellite imagery (Fig. 18). Nonetheless, even with such strong mesoscale forcing, deep convection did not develop at any time along the cold front in Oklahoma. A possible explanation is that air parcels may not have resided long enough within this narrow updraft to reach their level of free convection (LFC) in the presence of very dry air (entrainment) aloft.
6. General conclusions and recommendations
It has been demonstrated that a combination of fixed-wing and rotary-wing UAS can provide detailed, accurate, and very frequent measurements of changes within the boundary layer important for determining the potential for convection initiation. Returning to the list of subsidiary questions that were to be addressed in EPIC, significant progress was made in most respects. Thus, a vertical sampling interval of 5 m for rotary-wing UAS was found to be more than adequate. Rapid temporal variability was at times appreciable, but we found that a rotary-wing UAS sampling rate of 15 min is able to fully resolve sudden changes in preconvective conditions, and a nominal goal of 30-min flights is satisfactory. The time required to ascend and descend to a given height must be factored in when determining the optimal frequency of operations; for example, if future flights were to ever extend to 2500 m (well beyond the 763-m limit allowed in this project), then this would not be possible with a single UAS unless they are constructed to ascend and descend at much higher rates while retaining the integrity of the atmospheric measurements, and battery lifetime is appreciably increased.
Neither of the rotary-wing UAS produced temperature and moisture measurements that exactly met the EPIC observation accuracy goals, but they came close. They both exhibited slight warm and dry biases. The Meteodrone produced highly accurate winds, both in speed and direction, that nearly matched that goal, whereas the CopterSonde did not.
Several important mesoscale phenomena were detected by the rotary-wing UAS despite the limitations in sampling a sufficiently deep part of the atmosphere. In one case, the afternoon rawinsonde observations showed a sudden appearance of a low-level jet east of developing severe storms. In another case, the hourly mobile rawinsondes revealed cooling in the lower troposphere that led to the rapid destruction of a capping inversion, as well as increasing storm-relative helicity. While these important changes occurred above the layer sampled by the UAS, they occurred in concert with UAS observations showing the sudden appearance of a deep layer of higher moisture content just above the surface. Finally, in a third case, a horizontal wind convergence feature associated with a very narrow prefrontal low-level jet was inferred from the Meteodrone measurements taken along the leading edge of a cold front that was passing directly overhead. The implied narrow updraft served as an explanation for the narrowness of a prefrontal moisture plume and the existence of line convection that developed along the front in visible satellite imagery.
The operational usefulness of the EPIC data in real-time operations was rated only as a 3 on a scale of 1–10 by NWS-Norman. The weather forecasters commented that it was hard to determine what was useful and what was not, in part because they place a very heavy reliance on operational observing systems such as satellites and conventional rawinsondes with which they are familiar and that are supported in the operational display and analysis system (AWIPS). The full impact of the LASE-type UAS data will be known only once the data are assimilated into numerical weather prediction models. Observing system simulation experiments (OSSEs) should be used to investigate the optimal observational sampling requirements and strategies. Also, the 763-m maximum allowable altitude was not high enough to detect the “capping inversion,” most of the low-level jet, and other features just mentioned, which have great importance in forecasting severe local storms in the southern Great Plains. The full-tropospheric soundings provided by NSSL were given greater attention by the forecasters than the UAS data because of this height restriction.
Approved UAS flights to at least 2500 m (8200 ft) AGL are recommended for NWS applications, but being given permission to do so means that the FAA would need to approve beyond-visual-line-of-sight (BVLOS) conditions, thus requiring some other means to track the UAS other than visual observations. Were the data to be more routinely available over a greater depth of the atmosphere, in adverse weather conditions, at night, and fully calibrated against operational observing systems, then the UAS data could possibly have a greater role to play in NWS operations. The volume of data may be challenging for forecasters to interrogate in real time, particularly if 15-min profiles are produced from multiple sites, so perhaps UAS data should be presented in an hourly summary format. Another way for rotary-wing UAS data to positively influence NWS operations might be to assimilate the data in real time into rapidly updated high-resolution numerical weather prediction models like the High-Resolution Rapid Refresh model (Benjamin et al. 2016), which can explicitly resolve deep convection and utilize rapidly updated observations. That possibility should be examined in future research.
Finally, it is important to remember that the potential of UAS to provide measurements of the atmosphere is promising, but there are daunting regulatory hurdles that must first be overcome. FAA guidelines for safe, allowable operation of UAS include requirements for line-of-sight (LOS) operations under the watchful eye of a trained pilot in command and at altitudes not exceeding 400 ft (122 m) unless a waiver exception is granted. For UAS to ever become a realizable new observing system for NWS operations, some means for remote, semiautonomous operations of multiple UAS must be developed.
This research was conducted with funding provided by the NOAA UAS Program Office to NSSL, which awarded a contract to Meteomatics under Award WE133R-16-SE-1073, and via NOAA/Office of Oceanic and Atmospheric Research under NOAA–University of Oklahoma Cooperative Agreement NA11OAR4320072, U.S. Department of Commerce. The authors greatly appreciate the assistance provided by Chris Fiebrich, who worked with the landowners of the selected Oklahoma Mesonet sites to successfully gain their permission to conduct our field exercises over their properties, and used Oklahoma Mesonet data as background comparison to the TTwistor transects. Erik Rasmussen developed and tracked vehicle routes to maintain eye contact with the TTwistor at all times and to maintain constant communication among the project participants. The efforts of Robert Huck and Ken Carson to coordinate with the FAA to obtain the COA and waivers and to find students to support the field operations were also invaluable to the success of this project. NSSL scientists Sherman Frederickson and Doug Kennedy contributed vital assistance with the CLAMPS soundings and remote sensing systems. Dave Turner kindly conducted AERI thermodynamic retrievals.