1. Introduction

The utility of a remotely piloted aircraft system (RPAS), also known as an unmanned aerial system (UAS) or unmanned aerial vehicle (UAV), in the airborne collection of measurements is well established (Konrad et al. 1970; Egger et al. 2002; Reuder et al. 2009). The ongoing evolution of the UAS as an atmospheric science measurement tool spans decades, dating back to at least the 1960s (Mayer et al. 2012; Axisa and DeFelice 2016), and includes vehicle fleets varying greatly in size, mass, speed, power, flight duration, navigation methods, and instrument payload. Heavy-payload-bearing, long-endurance UAS vehicles (e.g., the Global Hawk; Brown et al. 2011; Humpage et al. 2014; Li et al. 2016) are typically deployed to high altitudes (>5 km above the surface) with payloads of airborne profilers, microwave sounders, and infrared spectrometers. Vehicles with smaller fuel capacity and limited power and payload space [e.g., the Coyote (Patterson et al. 2014; Cione et al. 2016; Aksoy et al. 2017); the Kahu (Garrett et al. 2011; Cook et al. 2013); the meteorological mini unmanned aerial vehicle (M²AV; Spiess et al. 2007; Martin et al. 2011; Lampert et al. 2016); the ScanEagle (Hodgson et al. 2013; Reineman et al. 2013, 2016); the SMARTSonde (Bonin et al. 2013; Wainwright et al. 2015); and the Small Unmanned Meteorological Observer (SUMO; Reuder et al. 2009; Mayer et al. 2012; Cassano 2014; Båserud et al. 2016)] are often employed at lower altitudes, where meteorological observations may supplement existing near-surface observations and/or vertical profiles with high-frequency, high-resolution measurements over a finite duration and three-dimensional space.

Lower-altitude UAS operations typically involve fixed-wing or rotary-wing vehicles that exchange long endurance and heavy payload capacity for enhanced maneuverability and control, smaller overall mass, lower flight speeds, limited operating range, and reduced cost. These small UAS are also typically easier to operate than their larger, more complicated counterparts. When flown with fast-response sensors and at sufficiently slow ascent/descent rates, the UAS is particularly well suited to sampling atmospheric boundary layer (ABL) transitions and sharp vertical scalar and momentum gradients within and above the ABL (Garrett et al. 2011; Mayer et al. 2012; Bonin et al. 2013; Cassano 2014; Jonassen et al. 2015; Reineman et al. 2013, 2016). Accuracy and precision in measuring vertical gradients within and just above the ABL is critical to identifying the timing and morphology of surface layers, internal boundary layers, well-mixed layers, inversions, and entrainment zones, with broad ramifications for quantifying turbulent eddy strength, electromagnetic signal propagation, and air quality.

Given the demonstrated capability of various UAS to reliably observe very finescale spatial and temporal variability in the lower troposphere, it is of particular interest to explore the utility of UAS meteorological measurements for numerical weather prediction (NWP). Observations from UAS may serve as a source for validation of NWP models (e.g., Mayer et al. 2012) and large-eddy simulation (e.g., Wainwright et al. 2015), as well as a source for model verification (e.g., Jonassen et al. 2012). Observations from UAS may serve additionally as a source for data assimilation (DA) in NWP models (Ágústsson et al. 2014; Jonassen et al. 2012).

Modern DA systems, such as the U.S. Naval Research Laboratory Atmospheric Variational Data Assimilation System (NAVDAS; Daley and Barker 2001) and NAVDAS Accelerated Representer (NAVDAS-AR; Xu et al. 2005), continuously ingest O(10⁵) observations per day from, for example, remote sensing platforms, airborne commercial aircraft, buoys, surface stations, and radiosondes. The impact of individual observations on an NWP model forecast through a DA system is tempered by the specified uncertainties of the preexisting first guess or background state (e.g., a previous model forecast) and the specified uncertainty associated with systematic and random errors in individual sensors and/or platforms (Kalnay 2003). The latter of these uncertainties makes up the diagonal elements of the observation error covariance. If this error term is small (large) for a particular observing sensor or platform, the discrepancy between the measurement and the model field mapped to said measurement (i.e., the innovation) will contribute proportionately more (less) to the analysis.

This study seeks to quantify the impact of UAS observations on NWP through variational data assimilation. To quantify this impact, observations collected from a low-altitude/small payload UAS over a midlatitude maritime environment are assimilated into a three-dimensional variational (3D-Var) DA system (NAVDAS) to yield an analysis for a state-of-the-art coupled model system comprising the Coupled Ocean–Atmosphere Mesoscale Prediction System (COAMPS; Hodur 1997; Doyle et al. 2011),¹ the Naval Research Laboratory Navy Coastal Ocean Model (NCOM; Martin 2000), and WaveWatch III model (WWIII; Tolman 2009). Quantification of the impact of UAS DA is performed through an observing system experiment (OSE; Uppala et al. 1985; Kelly et al. 2004; Zapotocny et al. 2008), which compares the forecast errors of the coupled model with UAS observations assimilated to a control case without UAS observations. The change in performance between these two cases represents the impact of UAS observations and is assessed through a blend of statistical and local comparison.

To the knowledge of the authors, this study represents the first assimilation of UAS observations in a variational analysis as part of a full NWP system. Section 2 outlines the Trident Warrior 2013 (TW-13) field campaign, which provides the source of UAS observations and model verification data used here. Section 3 describes the design of the coupled model system used to evaluate the impact of UAS observations on NWP. Section 4 provides a brief model verification of the control test case, omitting UAS observations. Section 5 diagnoses the impact of UAS observation on NWP through statistical measures. Section 6 outlines conclusions and avenues for future research on UAS DA in NWP. The appendix provides a list of the acronyms and abbreviations used in this paper

2. Trident Warrior 2013 campaign

Trident Warrior is an annual event conducted on alternate U.S. coasts each year as part of Navy Warfare Development Command Fleet Experimentation. Trident Warrior 2013 included experiments across various Navy and government organizations, commercial industry, and academia. The fundamental science question driving the atmospheric experiments of TW-13 was this: Do on-scene UAS observations provide increased skill for electromagnetic (EM) propagation (path loss) prediction, with interests in the S-, C-, and X-bands (~2–12 GHz)? The atmospheric propagation of these signals, such as from a ship’s radar system, is subject to refraction due to environmental conditions, thereby affecting propagation. To address the fundamental question, a field campaign was designed to demonstrate near–real time assimilation of UAS meteorology and oceanography (METOC) data for high-resolution NWP prediction during an intensive operational period (IOP), as well as to assess the impact on model prediction, with a focus on the fields of direct relevance to EM propagation prediction.

The quantity that describes EM signal refraction in the frequencies of interest is modified refractivity M in M-units, given by (1):

View Expanded

where

View Expanded

where r_e is the mean radius of Earth (m), z is height (m) above Earth’s surface, n is the refractive index of air over Earth’s curved surface, e is vapor pressure (hPa), T is temperature (K), p is air pressure (hPa), and N is refractivity expressed in N-units, after Bean and Dutton (1968). Where the local vertical gradient of M is negative in the atmospheric column, a trapping condition occurs where normal EM signal attenuation is diminished, and the signal may propagate farther than normal (Babin et al. 1997). The NWP models provide forecasts of T, p, and e to yield profiles of M as input to numerical models that predict EM path loss, thus providing a pathway for quantifying the impact of UAS observations on EM path loss prediction. The M profile is strongly a function of the water vapor profile (2); thus, the predictive skill of EM signal propagation hinges largely on the predictive skill of the water vapor profile.

The TW-13 IOP was realized by a collaboration of U.S. government and university investigators, including the Naval Postgraduate School (NPS), the U.S. Naval Research Laboratory, the Naval Surface Warfare Center–Dahlgren Division (NSWCDD), The Ohio State University, Oregon State University, Scripps Institution of Oceanography, and the Space and Naval Warfare Systems Command–Systems Center Pacific. The hub of most TW-13 IOP measurements was the Research Vessel (R/V) Knorr, owned by the U.S. Navy and operated by the Woods Hole Oceanographic Institution (WHOI). The research vessel commenced operations at 0600 UTC 13 July 2013 with a science crew of 30 persons and returned to port at 1800 UTC 18 July 2013. The R/V Knorr conducted operations off the U.S. mid-Atlantic coast, with airborne, submersible, and floating instrument packages launched regularly from or alongside the vessel. The METOC measurement platforms included 262 airborne expendable bathy thermographs (AXBTs), a buoy fitted with high-frequency air and water sensors, a ceilometer, a radiometer, 51 radiosondes launched by balloon, multiple radiosondes launched by tethered kite, UAS, four unmanned surface vehicles, eight unmanned underwater vehicles (gliders), and six wave buoys.

The Air–Sea Interaction Research Laboratory (ASIRL) at the Scripps Institution of Oceanography prepared three science payloads for use in Boeing–Insitu ScanEagle UAS vehicles during TW-13. Each UAS flight included one of three payloads: a radiation measurement package (“Radiometric”), a visible and infrared imagery measurement package (“Imaging”), or a METOC measurement package (“Flux”), with launches and flights operated by investigators from NSWCDD on board the R/V Knorr. The Flux payload included a global positioning system (GPS) plus instrumentation for measuring temperature T, absolute humidity, wind speed |u|, and wind direction u_θ. Investigators from ASIRL processed and quality controlled the dataset to yield 1-Hz-frequency observations with precision of order 10⁻² °C, 10⁻³ g m⁻³, 10⁻² m s⁻¹, and 10⁻¹ degrees, respectively, and a latitude and longitude precision of O(10⁻⁹) degrees. The UAS altitude estimate was derived from GPS measurements following the Earth Gravitational Model 1996 (EGM96) geoid and the World Geodetic System 1984 (WGS84) reference ellipsoid (Lemoine et al. 1998). To enable comparison with NWP model fields, a correction was made to the ASIRL dataset for the vertical discrepancy between the surface of the reference geoid sphere and true sea level in TW-13 IOP area (approximately 40 m). The ScanEagle has a maximum takeoff mass of 22.0 kg, of which 8.5 kg may come from payload. It has a length of 1.98 m, a wing span of 3.11 m, and typical mission airspeeds of 28–31 m s⁻¹ (Reineman et al. 2013).

Seven flights of approximately 3–6 h in length were conducted with the Flux payload aboard for a total 34.6 h of flight time (Table 1) spanning a region of approximately 3000 km² (Fig. 1) within the lowest 1550 m above mean sea level (MSL). The flight paths were designed in support of atmosphere, ocean, and EM propagation prediction objectives (e.g., sampling the ocean thermocline slope, radar detection range testing), subject to accommodations for actual conditions as needed. The UAS was navigated to avoid flight through low-visibility conditions, such as fog or low cloud. The flight pattern alternated between horizontal transects just above the ocean surface (for investigation of surface turbulent fluxes, including sensible heat, latent heat, and momentum flux) and helical ascents/descents (to sample vertical profiles). During the near-surface horizontal transects, an “Imaging” payload-bearing UAS was flown concurrently several hundred meters above the “Flux” payload-bearing UAS to monitor flying conditions and the sea surface. Further details on UAS operation during TW-13 can be found in Reineman et al. (2016). All references hereafter to UAS flights refer to the “Flux” payload-bearing ScanEagle UAS flights.

Table 1.

UAS flight launch and recovery times during TW-13 in July 2013. Local time is UTC − 4 h.

View Table

View Full Size

Flight paths of the seven Flux payload UAS flights off the eastern coast of the United States during TW-13. Scale in the upper figure varies; the length of the major axis of the horizontal transects in flight 1 (white) is approximately 20 km.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

The TW-13 IOP synoptic-scale weather was largely governed by a midlevel ridge centered approximately over the U.S. mid-Atlantic region, prompting clear and dry conditions with weak winds overall, with two exceptions. A weak amplitude, low-level inverted trough with an associated stationary front yielded appreciable onshore flow and variable cloud cover in the IOP region on 13 July 2013, with some scattered, weak convective rain showers. Following calm and dry conditions on 14 and 15 July, a slow-moving, low-level trough yielded increased cloud cover in the IOP region, with scattered convective rain showers over adjacent land surfaces. This trough stalled over the region on 16 July, then translated through the region by 17 July, yielding scattered low convective cloud cover and light northerly winds.

Reineman et al. (2016) provide a validation of UAS measurements during the TW-13 IOP by comparison with other campaign observations, including two sets of radiosondes launched for the campaign. A comparison of air temperatures measured during UAS ascent and descent of three helical profiles evaluated against NWP model (described in section 3) temperature spatially interpolated to the observations was made for verification purposes. Results revealed a mean absolute value difference between ascending and descending temperature bias (vs the NWP model) of approximately 0.07 K, with no common directional bias found (the descent was warmer in two profiles, and the ascent was warmer in the other profile). These results corroborate those of Reineman et al. (2016): no substantial systematic bias is present in the ScanEagle UAS temperature measurements.

The NSWCDD balloon-launched radiosondes were launched from the deck of the R/V Knorr and coordinated to operate simultaneously with and adjacent to the UAS flights. The radiosondes were deployed to measure from a position nearly collocated with the helical ascents/descents (Fig. 1). The radiosonde balloons were launched following a controlled-release method designed to limit ascent to approximately 1–2 km. To avoid potential contamination from the ship, observations were retained only during descent. The NPS balloon-launched radiosondes were launched at UTC synoptic times (0000, 0600, 1200, and 1800 UTC) from a smaller adjacent vessel, with observations collected during ascent that proceeded into the lower stratosphere. All radiosondes launched were Vaisala RS92 SGP models. The combined radiosonde dataset was not assimilated by the DA system and serves as the principal independent reference for validation of model predictions with and without UAS data assimilated at the analysis time. Some examples of UAS performance, compared with radiosonde observations for a selection of variables and UAS flights where the two platforms were sufficiently adjacent in space and time, can be found in Fig. 2. A corresponding statistical comparison of the UAS and radiosonde observations can be found in Table 2. Results demonstrate sufficiently strong coherence of spatially and temporally adjacent UAS and radiosonde measurements.

View Full Size

Comparison of fields measured by radiosonde (green crosses) and ScanEagle UAS (red circles) during ScanEagle UAS (left) flight 1 and (right) flight 7 showing (a),(b) potential temperature; (c),(d) water vapor mixing ratio; and (e),(f) horizontal wind speed. All measurements taken in the lowest 2000 m MSL are shown here, applying a 10-s average to both datasets where data are sufficiently dense. The flight 1 comparison evaluates the period 1830–2130 UTC 13 Jul 2013; the flight 7 comparison evaluates the period 2000–2300 UTC 17 Jul 2013. The horizontal separation of the UAS and radiosonde measurements during these two flights varied but was less than 15 km.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

Table 2.

Statistical comparison of ScanEagle UAS and TW-13 campaign radiosonde observations of potential temperature θ, water vapor mixing ratio q, and horizontal wind speed |u| during two UAS flights (flight 1: 1830–2130 UTC 13 Jul 2013; flight 7: 2000–2300 UTC 17 Jul 2013) with multiple radiosonde profiles and UAS flights adjacent in space and time. Quantities were averaged over 5-m bins from 0 to 2000 m MSL, with 10-s temporal averaging. Interplatform comparisons were made using bins containing data from both sources (~30–40 bins during flight 1; 55 bins during flight 7). The average difference is a column average of the difference of the bin means of each platform, computing the difference as ScanEagle UAS minus radiosonde. The MAE is also shown.

View Table

3. Model

The coupled model environment assembled for these investigations includes COAMPS with two-way ocean coupling (NCOM) with a three-dimensional ocean and one-way air–wave coupling (WWIII). COAMPS uses a finite-difference approximation to the fully compressible, nonhydrostatic equations with a terrain-following vertical coordinate transformation and 2nd-order accurate finite difference schemes in time and space. The compressible equations integrate using a time-splitting technique with a semi-implicit formulation for the vertical acoustic modes (Klemp and Wilhelmson 1978). The COAMPS spatial discretization includes four one-way interactive telescopically nested grids (g01, g02, g03, and g04) with horizontal resolutions of 36, 12, 4, and 1.33 km, respectively, centered over the region sampled by the UAS (Fig. 3). The discretized vertical domain consists of 60 terrain-following σ_z layers (Hodur 1997) extending from the surface to σ_z = 50 854, equivalent to 50.854 km MSL. The solution of all model fields, except vertical wind velocity, occurs at the vertical layer midpoints (hereafter referred to as a vertical level); there are 13 levels below 500 m above the terrain, with the lowest level at 10 m above the terrain. Further details on the COAMPS numerical grid and the parameterization’s selected subgrid-scale physics in these investigations can be found in Table 3.

View Full Size

Model domains and nested grids during investigations. The COAMPS grid 1 (g01) is outlined in red, grid 2 (g02) is outlined in yellow, grid 3 (g03) is outlined in brown, and grid 4 (g04) is outlined in pink. The corresponding grid scale is indicated within each outline. The NCOM grid is outlined in a magenta dashed line. The WWIII grid is outlined in a yellow dashed line.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

Table 3.

List of relevant numerical parameters and parameterizations set in COAMPS during investigations.

View Table

The coupled model system invokes the NAVDAS 3D-Var DA system to generate variational analyses for COAMPS. The NAVDAS 3D-Var DA system (hereafter, NAVDAS) assimilates observations from upper-air soundings, surface stations, buoys, ships, commercial aircraft, satellite retrievals, and satellite-derived products (e.g., wind), processing up to 5.5 × 10⁴ individual observations per analysis generated in this study. NAVDAS combines these observations over a 6-h window (±3 h from analysis time) with a background model state to minimize a cost function that yields an optimal analysis (Kalnay 2003). COAMPS derives sea surface temperature (SST) analyses from the operational Navy Coupled Ocean Data Assimilation (NCODA) system using a 3D-Var analysis of satellite and in situ observations (Cummings 2005). The largest COAMPS grid (g01) includes a single nested NCOM grid (401 × 401 grid points, 3-km resolution), covering approximately the same domain as COAMPS grid 2 (g02). The Earth System Modeling Framework (ESMF) infrastructure coordinates feedback between the atmosphere and ocean for surface fluxes, winds, and SST. The largest COAMPS grid (g01) also includes a single nested WWIII grid (261 × 241 grid points; 0.05° resolution), evaluating 25 wavenumbers over 24 directions.

The assimilation of UAS observations into NAVDAS is a new capability developed for this study. A customized observation decoder was written to ingest and process files from UAS platforms into NAVDAS, with portability to other Navy DA systems. From the quantities measured by the UAS (section 2), NAVDAS assimilates the following fields: T, pseudorelative humidity, zonal wind u, and meridional wind υ. Pseudorelative humidity is defined as e/e_{s_bkgd}(T_bkgd), where e is the vapor pressure derived from UAS observations, and e_{s_bkgd} is the (NAVDAS) background state saturation vapor pressure at the (NAVDAS) background state air temperature (T_bkgd). The 1-Hz atmospheric dataset provided by the ScanEagle was averaged using 60-s intervals for level flight segments and 10-s intervals for profile segments to yield a data density similar to other aircraft observations. NAVDAS assimilates the UAS observations in the outermost grid (g01). Quality control measures include filtering bad or suspicious data, validating the flight track, and comparing individual UAS observations against the NAVDAS background state. This customized NAVDAS configuration excludes UAS observations measured at altitudes below 15 m MSL to prevent the use of observations during takeoff or landing. Additional development for this customized UAS configuration includes a temporal interpolation method to map fields in the NAVDAS background state to the observation times in order to compute innovations.

The observation errors specified for the UAS observations are identical to those used with other aircraft data in NAVDAS: 1 K for T, 10% for pseudorelative humidity, and 2.1 m s⁻¹ for u and υ. Observation errors for aircraft-measured temperature and pseudorelative humidity data in NAVDAS gradually increase with altitude, but the UAS did not reach such heights in TW-13. The background error covariance of NAVDAS is uniquely tuned to the COAMPS-provided background field. The effective horizontal decorrelation length scale derived from the background error covariance is 385 km for T, u, and υ, and 185 km for pseudorelative humidity at the surface; these values increase with height. The effective vertical decorrelation length scale is 150 hPa at the surface and increases with height. The impact of these length scales can be seen in Fig. 4, which depicts the analysis increments resulting from a single UAS observation at 950 hPa with a temperature innovation value of +1 K.

View Full Size

Analysis increments at (a)–(d) 950 hPa and (e),(f) longitude 75.81°W, resulting from the assimilation of a single UAS temperature observation at 950 hPa with an innovation value of +1 K. Increments of (b) geopotential height, (c),(f) u-wind components, and (d) υ-wind components are produced in addition to (a),(e) the temperature increments because of the geostrophic and hydrostatic balances in the background error covariance.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

The real-time coupled model simulations were rerun to incorporate post-IOP UAS quality control measures (Reineman et al. 2016). Following a 7-day spinup period, coupled model forecasts out to 24 h ran at each analysis time beginning 1800 UTC 13 July 2013 and repeated at 6-h intervals through 0000 UTC 18 July 2013. A local Cray XE6m supercomputer using 96 processors hosted the model runs. The Navy Global Environmental Model (NAVGEM; Hogan et al. 2014) provides the initial and boundary conditions. Forecast products are available hourly from forecast lead time τ = 0 through τ = 24 h. This study performs an OSE to evaluate the impact of UAS observations on NWP through two coupled model simulations: a control case (hereafter, noUAS) and an experimental case (hereafter, UAS). Both test cases have the same aforementioned model configuration and permit coupled model integration with NAVDAS using all aforementioned conventional observation sources, but UAS additionally assimilates the (ScanEagle) UAS observations as described, while noUAS omits these observations.

4. Control model verification

Coupled model performance in both noUAS and UAS is evaluated for COAMPS fields most relevant to TW-13 objectives and the fields measured by the UAS: potential temperature θ, water vapor mixing ratio q, |u|, and u_θ. Model verification here is restricted to fields in the innermost grid (g04) at 1.33-km horizontal resolution. Evaluation of the control case (noUAS) concentrates on statistical measures comparing model-predicted quantities to radiosonde and UAS observations from TW-13, including the correlation coefficient r, bias, and root-mean-square error (RMSE) (Wilks 2006). A Pearson correlation coefficient (Wilks 2006) is computed for θ, q, and |u|; a circular correlation coefficient (Berens 2009) is computed for u_θ. All model fields are bilinearly interpolated in three-dimensional space to the point of observation. No time interpolation is performed; observations are typically compared against the model forecast hour nearest to the time of observation (1 ≤ τ ≤ 6 h). In a few instances, observations are compared against longer forecast lead times, up to τ = 24 h. The 1-h frequency of model forecasts limits the model-measurement temporal discrepancy to 30 min. The quality controlled 1 Hz UAS measurements were filtered into 1-min-averaged quantities for verification use to mitigate any unexpected signatures resulting from sudden aircraft movement, rotation, or very small turbulent eddies that would be unresolved by COAMPS. Observations from UAS below 15 m MSL were excluded from use in verification for consistency with the NAVDAS quality control techniques.

Figure 5 illustrates the vertical distribution of all observations taken by the ScanEagle UAS and TW-13 radiosondes during the campaign, organized into vertical bins of 25-m depth. ScanEagle UAS observations are skewed high in altitude, with the greatest concentration of observations in the lowest 200 m MSL. The radiosonde observations are comparatively even in distribution, with the NPS synoptic soundings extending into the lower stratosphere (not shown) and NSWCDD asynoptic soundings typically extending to at least 1000 m MSL. Unlike the UAS observations, there was no exclusion of near-surface radiosonde measurements from verification due to the measures applied to control for ship influence (section 2).

View Full Size

Frequency of ScanEagle UAS and TW-13 radiosonde (raob) observations in the lowest 2000 m MSL over the duration of the field campaign. Observations are binned into 10-s time bins and 25-m vertical bins.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

A statistical summary of COAMPS forecast performance during noUAS showing RMSE, r, and bias for the range of forecast lead times tested (1 ≤ τ ≤ 24 h) from all analyses generated (1800 UTC 13 July–0000 UTC 18 July 2013) is shown in Table 4. Verification is limited to below 1550 m MSL for both sets, corresponding to the vertical range of UAS observations taken. The abundance of observations in the vicinity of the top of the ABL (h_ABL) provided by the TW-13 IOP provides a rare opportunity to evaluate thoroughly the COAMPS performance at this critical level. The model defines h_ABL as the lowest level at which the Richardson number (Ri) exceeds a critical value (0.5), a reasonable value for h_ABL estimation (Sugiyama and Nasstrom 1999). The table divides results into three parts: all measurements, measurements below the model-predicted h_ABL, and measurements above the model-predicted h_ABL. Table 4 also distinguishes results using the UAS measurements as verification from those using the TW-13 campaign radiosondes as verification. The temporal bin size of each dataset is set to 10 s for consistency, with the highest frequency used by NAVDAS with the UAS observations, and in an attempt to better capture finescale behavior.

Table 4.

Statistical verification of COAMPS forecasts for noUAS at lead times τ of 1–24 h using (left) ScanEagle UAS observations and (right) TW-13 radiosonde observations for forecast verification. Observations are averaged into 10-s bins. Only observations taken below 1550 m MSL are considered. Statistics include correlation coefficient r, bias, and RMSE, as well as a count of all averaged-observation-model comparisons (nobs). The units of bias and RMSE follow those of the field; the correlation coefficient is dimensionless. Statistics are presented for potential temperature θ, water vapor mixing ratio q, horizontal wind speed |u|, and horizontal wind direction u_θ. Results are partitioned to separate (top) the statistics using all measurements from (middle) those using only those measurements within the model-estimated boundary layer from (bottom) those using only measurements above the model-estimated boundary layer.

View Table

The statistical verification of noUAS reveals a consistent cold bias of approximately −0.3 K above the model-predicted h_ABL, where correlation of model to measurement is relatively strong (r ~ 0.9), and a stronger cold bias below h_ABL (−0.60 to −0.75 K), but with a weaker correlation (r ~ 0.75). This bias is very similar to that found near the surface in COAMPS forecasts off the coast of Chile in the spring season during the VOCALS-Rex campaign (Wang et al. 2011): −0.7 K. The RMSE of θ hovers around 1 K. Comparison of UAS-derived q to modeled q shows effectively no skill below h_ABL but considerably improved performance above h_ABL. The RMSE of q varies between about 2 and 2.5 g kg⁻¹. Both verification datasets reveal a common trend in bias across h_ABL: moist bias below and slight dry bias above. This combined result of a net cold and moist modeled lower troposphere is generally consistent with results found during the CBLAST-Low field campaign several hundred km north of the TW-13 campaign region (Edson et al. 2007). Neither θ nor q shows any substantial change in performance (RMSE, r) for τ = 1–24 h (not shown).

Statistical results for |u| performance are generally more uniform across h_ABL and across datasets, with model-observation correlation strengths hovering around 0.75–0.80. Wind speed forecast bias is variable in sign but small in magnitude, with RMSE scores predictably smaller below h_ABL than above (not shown). Wind speed shows minimal change in RMSE performance with τ out to 24 h, but it does show a gradual reduction in correlation strength with τ (not shown). The analysis of model prediction skill of |u| and u_θ was not expanded because these fields are of indirect relevance to EM path loss.

The top of the ABL is often associated with a sharp inversion of moisture and/or temperature (Stull 1988). Insufficient model resolution or placement of this zone can yield substantial error with respect to observations. There is clear sensitivity to h_ABL in noUAS. The model cold bias identified through statistics is particularly acute in the vicinity of this level (Fig. 6a). The model moist bias (Fig. 6b) also shows strong sensitivity to h_ABL and reveals that the bias at short τ is predominantly concentrated along the model-estimated h_ABL, similar to the θ bias.

View Full Size

Model bias of (a) potential temperature, (b) water vapor mixing ratio, and (c) wind speed in noUAS. Bias is calculated from all available model-measurement comparisons using TW-13radiosonde measurements, organized into vertical bins representing nondimensional geometric height MSL scaled by model-estimated boundary layer depth (h_ABL). The forecast lead times (τ) are merged together into bins of 3 h and are labeled along the abscissa of each subplot.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

Another informative measure of model verification is a vertical profile of the model-measurement differences. Figure 7 compares noUAS fields and UAS measurements with height and with height scaled by h_ABL, as estimated by COAMPS (z × h_ABL⁻¹), using the shortest available forecast lead time (1 ≤ τ ≤ 6 h) examining θ, q, and |u|. The profiles confirm the statistical cold bias across the lower troposphere, as well as the net moist (dry) bias below (above) h_ABL. They also illustrate the considerable variability in model bias over the course of the IOP.

View Full Size

Forecasts from noUAS minus 1-min-averaged observed values from the UAS during all flights. Red (blue) shaded points indicate a positive (negative) model bias. Fields shown are (a),(d) potential temperature; (b),(e) water vapor mixing ratio; and (c),(f) wind speed. (top) The difference over height (m); (bottom) the difference over height scaled by the model-estimated boundary layer height (h_ABL). The orange line represents an average over bins of 50 m (top row) or 0.1 × h_ABL (bottom row). Model forecasts use the nearest available lead time τ to the observations (1 ≤ τ ≤ 6 h).

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

5. Impact of UAS data assimilation: Statistics

The impact of assimilation of ScanEagle UAS data on NWP forecasts is quantified by comparing the results of two coupled model test cases: noUAS and UAS (section 4). The same model fields, model field processing, statistical measures, and temporal and spatial conditions applied to verification of noUAS are applied to UAS. Model verification of the case intercomparison is approached through two methods: 1) grid-scale verification using analyses from the European Centre for Medium-Range Weather Forecasts (ECMWF) and 2) point verification using the TW-13 radiosonde observations (excluded from the DA scheme to ensure data independence).

Grid-scale verification is achieved by comparing model forecast fields of temperature and relative humidity (RH) from noUAS and UAS in the coarsest (g01) and finest grids (g04) to analyses from the control run of the ECMWF Ensemble Prediction System (EPS). The ECMWF EPS control run fields are available through the THORPEX Interactive Grand Global Ensemble (TIGGE) dataset (Bougeault et al. 2010) at 0.5° horizontal resolution on mandatory pressure levels (e.g., 1000 and 925 hPa) at 0000 and 1200 UTC daily from 2006 through the present. The ECMWF EPS analysis fields are linearly interpolated to the COAMPS grid, then mean absolute error (MAE) is computed between the ECMWF source and 6-h forecasts from the 1800 and 0600 UTC analyses of noUAS and UAS at 1000, 925, and 850 hPa (interpolated by COAMPS from the sigma coordinate grid). These mandatory levels correspond to approximately 111, 766, and 1457 m MSL in the U.S. Standard Atmosphere, 1976 (COESA 1976). A brief statistical overview of this grid-scale verification is presented here. Comparison of the MAE from noUAS and UAS reveals limited impact of UAS DA on COAMPS performance in the coarsest grid, whose 36-km horizontal resolution is nearest to that of the verification dataset. The mean of the difference in MAE between noUAS and UAS (ΔMAE) over the TW-13 campaign is less than 0.2 K (0.1%) at all three pressure levels evaluated for temperature (RH). In the finest mesh, one analysis in particular is found to yield relatively large error from UAS DA (1800 UTC 16 July 2013), which is related to an error discussed in further detail below. With the omission of this single analysis, UAS DA yields an average MAE reduction for RH of 0.84% and 0.96% at 1000 and 925 hPa, respectively (within the vertical measurement range of the UAS flights), but an increase of 0.92% at 850 hPa near the top of the measurement range. A decrease of MAE represents model error reduction due to (and thus, a positive impact from) UAS DA. For the temperature field, also omitting the analysis at 1800 UTC 16 July, the ΔMAE is within ±0.2 K, with an error decrease at 925 hPa and an increase at 1000 and 850 hPa, similar to the results in the coarsest grid. While this grid-scale verification provides a metric for UAS DA impact assessment across the COAMPS grids, given the limited spatial and temporal UAS sampling range, additional verification at finer resolutions is needed to more fully quantify the impact of UAS DA.

The schedule of UAS flights during the IOP (Table 1) varied sufficiently to leave several model analysis times devoid of UAS observations (specifically, 0600 UTC 15 July, 0000 UTC 16 July, 0600 UTC 17 July, 1200 UTC 17 July, and 0000 UTC 18 July). Evaluation of the impact of UAS DA included separate assessments that considered 1) all model analysis times and 2) only those analyses with UAS observations assimilated. The statistical change in the impact of UAS DA found between these two evaluation methods was marginal (not shown). In an effort to maximize model-measurement comparisons and better simulate a realistic operational UAS DA process with occasional data stream interruptions, comparisons of noUAS and UAS incorporated all model forecast cycles outlined in section 3.

a. Profiles

Comparison of the RMSE scores of these two model cases (Figs. 8a–d) over the vertical range covered by the UAS reveals a distinct reduction (improvement) signature at short τ (Figs. 8e,f). These plots compare all valid data from campaign radiosondes to model predictions interpolated to the radiosonde profile grouped into vertical bins of 50 m. In the RMSE difference subplots (Figs. 8e,f), blue (red) shading indicates where RMSE of UAS is less (greater) than that of noUAS, thus representing decreased (increased) model error due to UAS DA. The asynoptic sequencing of the NSWCDD soundings, in conjunction with the 6-h interval between analyses, yields a relatively consistent distribution of model-measurement comparisons in forecast lead time, assisted further by grouping lead times into bins of 3 h (Fig. 9a). Both the θ and q model predictions share a coherent reduction in RMSE from roughly 400 to 800 m MSL for τ extending to approximately 6 h. The reduction is approximately 0.5 K for θ RMSE and 1–2 g kg⁻¹ for q RMSE. A weaker signature of RMSE reduction appears to extend to higher altitudes (~1300 m MSL) in both fields over this τ range.

View Full Size

RMSE of (a),(c) potential temperature and (b),(d) water vapor mixing ratio in (a),(b) UAS and (c),(d) noUAS. RMSE is calculated from all available model-measurement comparisons within vertical bins representing geometric height above sea level. Measurements were taken from TW-13 campaign radiosonde observations. The forecast lead times (τ) are merged together into bins of 3 h, labeled along the abscissa of each subplot. The difference in RMSE between UAS and noUAS is shown for (e) potential temperature and (f) water vapor mixing ratio.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Sum of point comparisons (model measurement) made in each vertical bin for each forecast lead time cluster (τ) for (a) the evaluation over geometric height MSL, (b) height scaled by UAS model estimated boundary layer height, and (c) height scaled by noUAS model estimated boundary layer height.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

Closer inspection of individual comparisons between radiosonde profiles and model fields during the entire campaign reveals that the h_ABL level is a focal point for RMSE reduction from UAS DA (Fig. 10). This region corresponds to some of the strongest model biases observed in the noUAS evaluation. The impact appears to be most coherent in the q field. The RMSE reduction in q also extends to the ABL at longer lead times, corresponding with the region of greatest positive bias in the control run (Fig. 6b). The quantity of model-measurement comparisons available in the vertical vicinity of h_ABL in UAS and noUAS (Figs. 9b,c, respectively) varies somewhat in response to differences in h_ABL between the two cases, but otherwise provides a reasonable number of points for field comparison.

View Full Size

As in Fig. 8, but using height scaled by model-estimated boundary layer height as the vertical coordinate.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

The majority of the RMSE increase in the θ field for 3 ≤ τ ≤ 8 h in the lowest 300 m (Fig. 8e) and lower ABL (Fig. 10e) can be traced to two radiosonde profiles with surface observations near 2137 and 2221 UTC 16 July (not shown). These two observed profiles indicate north-northwesterly and northerly low-level winds, respectively. The short-term noUAS forecasts match this well, but corresponding UAS forecasts show westerly wind in the lowest few hundred meters MSL with abundant convective cloud cover and rain. Surface analysis at 2100 UTC 16 July indicates a weak trough axis nearly collocated with the UAS sampling region. Both model cases capture this behavior in general, but UAS strengthens the regional sea level pressure gradient at 2200 UTC 16 July, yielding the local error in low-level wind (Fig. 11). This errant offshore flow contributes to convective cloud development, yielding additional local error in low-level temperature and moisture. This result provides an instructive illustration of the broad spatial impact of UAS-derived innovations in this DA system, which may yield unwarranted influence over mesoscale or synoptic-scale features.

View Full Size

Sea level pressure (hPa) and wind at 100 m above ground/surface level (AGL) over (a),(b) a region of the coarsest model grid (g01) and (c),(d) a region of the finest model grid (g04) from (a),(c) noUAS and (b),(d) UAS using the 4-h forecast fields from the analyses at 1800 UTC 16 Jul 2013. Half (full) barbs indicate wind speed of 5 (10) m s⁻¹. Wind speeds less than 5 m s⁻¹ are illustrated as an unfilled circle.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

The correlations used to construct the background error covariance in the DA system are not specially tuned here for any meteorological phenomenon or measurement platform. Consequently, the three-dimensional footprint of a single UAS observation in the DA system analysis spreads well beyond the point of measurement in three-dimensional space (Fig. 4). The high density of measurements over a limited region serves to amplify the impact of UAS observation on the local analysis state. The decorrelation length scales may be a contributing factor in the relatively strong negative impact from UAS DA seen at longer forecast lead times (Fig. 8f) and above the model-estimated h_ABL (Fig. 10f) in the q field, despite substantial error reduction along and below the model-estimated h_ABL. Some of the increased model error in water vapor mixing ratio prediction found at longer forecast lead times is also attributed to differences in predicted u_θ in the UAS measurement region, due in part to sensitivity to trough placement on 16 July, as described above. Further investigation is suggested on how best to assimilate highly localized observations provided by platforms like the UAS studied here.

Comparison of the bulk statistical results of UAS and noUAS (Table 5) corroborates a net reduction in RMSE of q within the ABL for all τ (≤24 h) and a reduction of q RMSE over the column (≤1550 m MSL) at short τ (<6 h). The latter error reduction serves to mitigate both the preexisting moist bias within the ABL and the dry bias above the ABL (Fig. 6b). Taken together, these results suggest a substantial (on the order of magnitude of the control model bias) and coherent RMSE reduction in q at short τ (≤6 h), corresponding to the approximate vertical region sampled. This is followed by continued RMSE reduction at longer τ in the vicinity of h_ABL and adjacent regions, where control model bias is particularly high. A similar, weaker pattern of RMSE reduction appears in the θ field, but subject to substantial noise, such as from the aforementioned instance of the erroneous trough placement on 16 July. Evaluated over all τ (≤24 h), the |u| and u_θ fields do not demonstrate substantial RMSE change relative to control case values either above or within the ABL. Further investigation in these fields was not pursued, in order to concentrate on fields of direct relevance to M.

Table 5.

Comparison of statistical verification of COAMPS forecasts between UAS and noUAS at lead times τ of (left) 1–6 h and (right) 1–24 h using TW-13 radiosonde observations for forecast verification (10-s average). Only observations taken between 0 and 1550 m MSL are considered. Statistics include correlation coefficient r, bias, and RMSE, as well as a count of all averaged-observation-model comparisons (nobs). There are approximately 600–700 observations within the ABL in the 1 ≤ τ ≤ 24 h range and approximately 160 in the 1 ≤ τ ≤ 6 h range, with the balance falling above the ABL. The units of bias and RMSE follow those of the field; the correlation coefficient is dimensionless. RMSE reduction is highlighted in bold. Statistics are presented for potential temperature θ, water vapor mixing ratio q, horizontal wind speed |u|, and horizontal wind direction u_θ. Results are partitioned to separate (top) the statistics using all measurements from (middle) those using only those measurements within the model-estimated boundary layer from (bottom) those using only measurements above the model-estimated boundary layer.

View Table

The impact on θ and q fields can be seen also through comparisons of model-predicted profiles. Figure 12 illustrates an example showing model profiles verified along the paths of six TW-13 radiosondes launched adjacent to UAS flight 1 over a 6-h time window (1800 UTC 13 July–0000 UTC 14 July) using all radiosonde observations. The control case verification (Figs. 12a,d) reveals a near-surface cold bias (average about 1 K) and a near-surface moist bias (average about 1 g kg⁻¹), coupled with a warm (~1 K) and dry bias (5 g kg⁻¹) centered near 600 m MSL. These biases result from a strong model-predicted inversion in noUAS approximately 200 m thick, centered between 400 and 500 m MSL with a strength of 8.5 g kg⁻¹ (q profile) and 2 K (θ profile) (not shown). The observed profiles show weak evidence of an inversion. The difference of the model-predicted fields between UAS and noUAS is shown in Figs. 12b and 12e. Where the θ (q) curve is negative (positive), this means that UAS was colder (more moist) than noUAS. The difference plots show a strong mitigation of most all control model biases, except the near-surface moist bias. The result is a substantial reduction in average profile biases (Figs. 12c,f). The average of the maximum near-surface cold bias is reduced by about half. The average of the maximum warm bias at 600 m MSL is reduced to nearly zero, with some interprofile variability. The average of the maximum dry bias at 600 m MSL is reduced by about half, also subject to interprofile variability. The UAS case did not remove the inversion signatures, but raised the inversion base height by an average of 125 m and substantially reduced the inversion strength to 4.5 g kg⁻¹ (q) and 1.5 K (θ), conforming better with radiosonde observations.

View Full Size

Verification of COAMPS model performance of (top) potential temperature and (bottom) water vapor mixing ratio during UAS flight 1. (a),(d) The noUAS forecasts minus observed values from campaign radiosondes (raob). (b),(e) UAS forecasts minus noUAS forecasts evaluated at points of observation by campaign radiosondes. (c),(f) UAS forecasts minus observed values from campaign radiosondes. Red (blue) shaded points indicate a positive (negative) model bias. The orange line represents an average over bins of 50 m. Model forecasts use the nearest available lead time τ to the observations (1 ≤ τ ≤ 6 h). Model fields are linearly interpolated to the point of observation in three dimensions.

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

b. Boundary layer inversions

An inversion at the top of the ABL (h_ABL) marks the boundary between the turbulent and well-mixed atmospheric boundary layer below and the free atmosphere above. This interface can be quantified through its morphology (Seidel et al. 2010; Wang and Wang 2014). In this study, a single inversion is sought individually from both the θ and q profiles to account for any differences in inversion signature (Vihma et al. 2011). Each profile is evaluated manually for the presence of a well-mixed layer above the surface layer (Stull 1988; Garratt 1994), recognized by a constant or nearly constant value with altitude. If one exists, its top is defined here as the inversion base height. If the inversion base height is imagined as the global maximum on the profile curve below 2 km MSL (an altitude above the likely maximum h_ABL in the campaign domain), then, moving up the curve from the inversion base height, the inversion top is defined here as the altitude of the profile curve global minimum below 2 km MSL. The inversion thickness is defined here as the altitude difference between its base height and top. The inversion strength is defined here as the change in scalar magnitude (θ, q) between the base height and top. These concepts of inversion base height, top, thickness, and strength are illustrated well in Fig. 4 of Vihma et al. (2011). Multiple inversions may be present in the profile below 2 km MSL. The use of the profile’s global maximum and minimum below 2 km MSL ensures that the inversion greatest in strength is accepted as the profile inversion. If a well-mixed layer is not easily identifiable, the model-predicted h_ABL at the location of the radiosonde launch (NPS radiosondes) or landing (NSWCDD radiosondes) is assessed to help identify inversion presence. If no ABL inversion exists near the model-predicted h_ABL in the profile in question but a clear surface layer inversion signature is present, the latter is accepted as the profile inversion. If no profile inversion can be established following this chain of tests, the profile is declared to be void of inversions.

A principal result of the statistical analysis of the impact of UAS DA on meteorological field prediction was the net reduction in RMSE of model-predicted θ and q in the vicinity of h_ABL. To quantify this impact, each TW-13 radiosonde-measured sounding and corresponding model forecasts (1 ≤ τ ≤ 6 h) interpolated to the radiosonde tracks were evaluated for inversion structure. The TW-13 soundings were clustered according to the nearest-in-time UAS flight and then further divided to group similarly shaped profiles of θ and q, yielding a total of 13 clusters. Of the 13 clusters, only one (toward the middle of UAS flight 3) showed no identifiable inversion in the observed profiles. There were also instances of ambiguous inversion presence in the observed profiles. For the θ profiles, this occurred during UAS flight 2, during the latter part of flight 3, and during all of flight 1. For the q profiles, this occurred in all soundings during flight 3 and most soundings during flight 6. Separate calculations were made considering all soundings with inversion signatures and only soundings with unambiguous inversion structures. For each cluster, the inversion base height, thickness, and strength were quantified for the observed profile, the UAS forecast profile, and the noUAS forecast profile for both θ and q. To create a common metric for evaluation of the UAS DA impact on inversion prediction, a variation on the statistical measure of mean absolute error, the mean normalized absolute error (MNAE), is introduced.

The computation of the MNAE proceeds as follows for a given cluster of observed profiles and corresponding model-predicted profiles. Taking inversion base height as an example morphology component, the MNAE is computed by first normalizing the absolute value difference between the predicted and observed base heights by the standard deviation of base height observed over the campaign. This step is executed for both the θ and q profiles. The two terms are averaged, yielding the MNAE:

View Expanded

where o_i is the observed value of the ith inversion morphology component, y_i is the model forecast value of the ith inversion morphology component, σ_oi is the standard deviation of the observed values of the ith inversion morphology component, and n = 3 (three morphology components evaluated: base height, thickness, and strength). Equation (3) may be expanded as necessary to include additional scalar profiles; for example, weighting coefficients may also be introduced as a way to control influence of individual profiles based on model or observation uncertainty. The MNAE statistical measure was devised because normalization by standard deviation permits combined evaluation of both the temperature and moisture profiles, removes the influence of spread, and mitigates the location problem that can arise when using observed values to normalize.

The MNAE calculations are executed separately using UAS and noUAS as the model forecast source. The difference between the two model cases is computed (ΔMNAE), which represents the impact of UAS DA on inversion morphological component prediction. This method is followed for each component: base height, thickness, and strength. This process is repeated for each of the profile clusters with observed inversion signatures that also have inversion signatures in their corresponding model profiles.

The resulting ΔMNAE expression represents a dimensionless change in error due to UAS DA (UAS vs noUAS as the model forecast value y), scaled by the observed spread of the inversion morphological component. Negative values imply a net improvement in forecast performance due to UAS DA. The normalization permits comparison of relative improvement among the three inversion morphological components examined. Figure 13 illustrates the resulting ΔMNAE (UAS minus noUAS) for each of the profile clusters evaluated with an unambiguous inversion signature present in at least one field (θ or q) in both model-predicted and observed profiles (10 of the 13 available clusters). The profile clusters are labeled according to the concurrent UAS flight, with additional alphabetical suffixes (e.g., a, b, c, …) to indicate multiple clusters per UAS flight as needed.

View Full Size

Change in MNAE of ABL inversion morphology components (base height in blue, thickness in red, and strength in green) for each cluster of measured and model-predicted temperature and moisture profiles evaluated during TW-13, comparing results from UAS vs noUAS. Negative (positive) change indicates a reduction (increase) in error for that morphology component due to UAS data assimilation. Only profile clusters with identifiable inversion signatures in both observed and corresponding modeled profiles are shown here (10 of a possible 13).

Citation: Monthly Weather Review 146, 2; 10.1175/MWR-D-17-0028.1

• Download Figure
• Download figure as PowerPoint slide

Table 6 shows the mean ΔMNAE values by morphological component, averaging across all profile clusters shown in Fig. 13. The results demonstrate net model error reduction across all components given UAS DA, with the greatest improvement seen in inversion strength. To quantify the estimated change in model forecast error due to UAS DA in the units of the morphological components, a product is formed from the ΔMNAE expression and the standard deviation of the observed values of the corresponding morphological component (σ_oi). This strategy provides an approximation of the net change in model prediction error over the IOP due to UAS DA. The results show a similar magnitude of error reduction to both inversion base height and thickness, approximately 30–35 m, with inversion strength error reduction on the order of 0.4 K (θ profile) and 0.7 g kg⁻¹ (q profile). Further investigation of the model-predicted h_ABL, compared to h_ABL, derived from observed scalar profiles reveals that at least some discrepancy between model and measurement derives from the presence of multiple inversions in the profile, and the model-predicted h_ABL may not reflect the strongest inversion of the lower profile.

Table 6.

Change in MNAE of model-forecasted ABL inversion characteristics due to the inclusion of UAS data in the DA scheme. Model forecasts are verified against radiosondes launched during TW-13. Model fields are linearly interpolated to the point of observation in three dimensions.

View Table

6. Conclusions

The Trident Warrior 2013 (TW-13) field campaign sought to investigate the utility of on-scene observations from unmanned systems in providing increased predictive skill to atmospheric fields of relevance to electromagnetic propagation prediction. The campaign launched seven flights of a Boeing–Insitu ScanEagle UAS equipped with a meteorology and oceanography field-observing payload over 5 days off the U.S. East Coast during July, sampling the lowest 1550 m MSL. A coupled mesoscale NWP modeling system (COAMPS–NCOM–WWIII) was configured to simulate the campaign domain and assimilate UAS observations using a variational DA system for the first time.

Experiments incorporating 34.6 h of UAS observations into the 3D-Var DA system demonstrate a consistent, broad reduction of model bias in temperature and moisture at short forecast lead times (τ ≤ 6 h) over the vertical range of measurement due to UAS DA, then subsequently concentrated in the vicinity of the top of the ABL. The local bias reduction is shown at times to exceed 1 K and 3 g kg⁻¹, respectively. Investigations on the impact of UAS DA on ABL inversion prediction reveal a general improvement in vertical position and depth (bias reduction of approximately 30–35 m each) and strength (0.4 K and 0.7 g kg⁻¹ for θ and q profiles, respectively).

Several additional experiments not described here examined the impact of UAS DA relative to other DA data sources, as well as the impact of removing individual environmental sensors from the UAS platform. Initial results suggest that UAS observations assimilated into the coarsest COAMPS model grid could provide a viable analysis to COAMPS (with relatively small increases in RMSE of θ and q) in the absence of the existing comprehensive suite of DA data sources over the same domain. Further investigation is necessary to apply these selective assimilation tests over longer periods.

Results shown here reflect the impact of a single UAS platform over a coastal maritime midlatitude environment during summer. Future experiments in UAS DA should pursue measurements over longer periods to sample a wider range of synoptic conditions. Parallel measurement platforms performing concurrent, separate flight maneuvers (e.g., profiles, transects) of a common environment would help to elucidate the relative impact of different types of UAS sampling patterns. Additional experiments evaluating the impact of measurement frequency and spatial density on model performance would be useful.

Among the principal hindrances to continuous UAS DA in NWP models are the extensive support systems needed to generate continuous observations, including trained human assistance with the launch, navigation, and recovery of the vehicle. Single-use disposable UAS vehicles of the low-altitude class would likely be cost prohibitive at present to justify widespread systematic UAS DA. The “through the sensor” concept of using military vessels and vehicles to obtain environmental observations may also be applied to UAS for METOC observations and offers a promising alternative pathway to increased UAS observations in NWP. Further research is suggested to determine how to overcome the limitations of broad decorrelation length scales when assimilating observations over very limited areas. Results here suggest the decorrelation length scales, which derive from carefully tuned correlations within the DA system, may be amplifying local features with impacts found at the synoptic scale. Future investigations should consider applying UAS observations to 4D-Var DA systems, where asynchronous UAS observations can be weighted appropriately at analysis time and mitigate errors of representation. Future investigations with limited-area models involving impact assessment of UAS deployments that are more extensive in space and/or time than the flights in TW-13, such as what might be anticipated for systematic operational UAS DA, should also consider the impact of UAS DA on the global model providing the boundary conditions.