1. Introduction
Encounters with turbulence by aircraft cruising in the upper troposphere and lower stratosphere (UTLS) are a challenging weather hazard for commercial aviation, and often result in serious injuries for passengers and crews, and flight delays (e.g., Kim et al. 2011; Sharman et al. 2006, 2012b). Turbulence related to strong shear and inertial instabilities, geostrophic adjustment, imbalance, and tropopause folding near upper-level jet and frontal zones is referred to as a clear-air turbulence (CAT) since it normally happens without visible convective activity (e.g., Sharman and Lane 2016). If CAT occurs over mountainous regions, it is often produced by mountain-wave breaking and/or critical-level interactions, and is usually referred to as mountain-wave turbulence (MWT). As the resolution of operational numerical weather prediction (NWP) models has increased, the capability to forecast MWT based on NWP model output has improved (e.g., Elvidge et al. 2017; Kim and Chun 2010, 2011; Lane et al. 2009; Sharman et al. 2012a).
The operational turbulence forecast system of the National Oceanic and Atmospheric Administration/Aviation Weather Center (NOAA/AWC) makes use of predicted vertical velocity from the operational Rapid Refresh (RAP) forecast model (Benjamin et al. 2016) as input into its mountain-wave turbulence prediction algorithm. Recently, unrealistically large areas of the light-or-greater (LOG) intensity turbulence with spurious mountain-wave signals have been reported frequently in wintertime over the mountainous regions in the United States (M. A. Thomas 2016, personal communication). It is expected that the terrain-following sigma vertical coordination in the RAP model is a possible contributor to this overprediction. In particular, the terrain-following vertical layers even at higher levels in the UTLS in the model are distorted over the steep mountain regions, which lead to the spurious horizontal and vertical gradients in the model fields. When it is used in the operational turbulence forecast system like the graphical turbulence guidance (Sharman and Pearson 2017), this contributes to high false alarm rates (or high bias) for the LOG turbulence forecasts in the UTLS.
To resolve this problem, several attempts have been made. One is to use a better smoothing technique to reduce smaller-scale (<6Δx) energy in the model terrain (Park et al. 2016). Another is to use a hybrid vertical coordinate system (Klemp 2011; Park et al. 2019). From an operational perspective, the hybrid sigma–pressure vertical coordinate seemed the better alternative, because it keeps the model terrain as realistic as possible, which gives better performance near the surface as well as in UTLS (Park et al. 2019). Therefore, this has been implemented in the new version of the RAP model (version 4; RAPv4) based on the formulation of Park et al. (2019) and was run in parallel with the conventional terrain-following coordinate system of the RAP model version 3 (RAPv3) by the NOAA/Global Systems Division (GSD) for several months of 2017. This hybrid vertical coordinate system used in new RAPv4 model with some minor upgrades became operational on 12 July 2018 (https://rapidrefresh.noaa.gov).
The aim of this study is to document the reduction of this overprediction of the LOG MWT in the RAPv4. We conducted direct comparisons of the upper-level turbulence forecasts between the new RAPv4 model with the hybrid vertical coordinate system and the older version of the RAPv3 with the traditional sigma coordinate system. The remainder of this paper includes the following sections. In section 2, we briefly introduce the RAP model with the new hybrid sigma–pressure vertical coordinate system. In section 3, we examine a case study to compare the RAPv4 with RAPv3 to show that the new model can reduce the spurious mountain-wave features at cruising altitude. Also, the MWT indices derived from both the RAPv4 and RAPv3 are statistically evaluated using more than 1.2 million in situ aircraft energy dissipation rate (EDR) reports to confirm the improvement of the MWT forecast skill in the new RAPv4 model. The summary and conclusions follow in section 4.
2. NOAA’s operational Rapid Refresh model
The RAP was developed to provide more reliable short-term forecasts by capturing rapidly developing mesoscale weather phenomena. Forecasts are made every hour using the Advanced Research version of the Weather Research and Forecasting (WRF-ARW) Model (Powers et al. 2017; Skamarock et al. 2008) and initialized using all available data, including conventional rawinsonde, surface METAR and mesonet observations, remote sensing data from satellite and radar, and aircraft data, combined with the previous hour’s 1-h forecast via an intermittent data-assimilation system. Lateral boundary conditions are from NOAA’s Global Forecast System (GFS). More details can be found in Benjamin et al. (2016). The domain of the RAP model is shown in Fig. 1, which covers the entire North American continent including the contiguous United States (CONUS), Canada, Mexico, Hawaii, and Alaska to provide better forecast products over the CONUS, North Pacific and Atlantic Oceans, and Alaska regions. The domain has a 13-km horizontal mesh with 51 sigma (eta) levels in the terrain-following vertical coordinate used in the RAPv3 and earlier. This has been updated to use the hybrid pressure–sigma vertical coordinate system in the RAPv4 that is a focus of this study. Forecast model output is available at 15-min intervals out to 21- or 39-h lead time, depending on model initial time. Detailed physical parameterization schemes, data assimilation processes, and recent updates are found in the slides and documents on the NOAA/GSD web page (https://rapidrefresh.noaa.gov) and in Benjamin et al. (2016).
Domain for NOAA’s RAP model with terrain height (shading with 250-m increments).
Citation: Weather and Forecasting 34, 3; 10.1175/WAF-D-18-0187.1
The operational aviation turbulence forecast system is termed the graphical turbulence guidance (GTG; Sharman et al. 2006; Sharman and Pearson 2017), which uses the RAP model outputs as an input to infer aircraft-scale turbulence intensities as a function of the cube root of the EDR (m2/3 s−1) by integrating multiple turbulence diagnostics based on physical downscaling processes for subgrid-scale (i.e., aircraft-scale) turbulence. Formulations of those diagnostics are based on horizontal and vertical gradients of resolved wind, temperature, and other variables from the underlying RAP model outputs. Therefore, its performance is highly dependent on the accuracy of the underlying NWP model. Aviation users have benefited by using this information for their flight planning (https://www.aviationweather.gov/turbulence). For example, they can avoid the forecasted turbulence areas, or they can take an action to turn on the seat belt sign before they encounter the expected turbulence regions. However, recently users have reported that the GTG from the RAPv3 tends to overestimate smooth-to-light turbulence to be LOG intensity especially over the western mountainous region of the United States (M. A. Thomas 2016, personal communication; Park et al. 2016).
Upon investigation, we found that this is partly due to the use of the terrain-following sigma vertical coordination system, which results in artificial mountain-wave-like motions that directly impact the turbulence forecast. To remove this error, Park et al. (2016) implemented RAP-like WRF-ARW model simulations to filter out smaller-scale (<6Δx) energy aloft by applying additional terrain averaging in the WRF Preprocessing System (WPS). This was successful in alleviating the spurious mountain-wave features in the UTLS, but surface features were unrealistically smoothed.
Here, ps is the surface pressure, pt is the pressure at the model top, p0 is 1000 hPa, and B(η) is a relative weighting between terrain-following and pure dry hydrostatic pressure coordinates. Here,
Vertical cross sections of the (left) conventional terrain-following sigma (eta) levels in the RAPv3 and (right) hybrid pressure–sigma levels in the RAPv4 across the Sierra Nevada and Colorado Rocky Mountains in the United States. Steep and complex terrain near 120° and 105°W are the Sierra Nevada and Colorado Rocky Mountains, respectively.
Citation: Weather and Forecasting 34, 3; 10.1175/WAF-D-18-0187.1
3. Results
Figure 3 shows a direct comparison of vertical pressure velocity fields (shading) and upper-level jet stream magnitudes (red contours) from 6-h forecasts of the (Fig. 3a) RAPv4 and (Fig. 3b) RAPv3 models at a typical cruising altitude (35 000 ft) of commercial aircraft valid at 1800 UTC 25 May 2017. At this time an anticyclonically curved flow was present around an upper-level ridge over the eastern Pacific Ocean, with a strong southwesterly polar jet crossing southeast Alaska and then curving anticyclonically back across the Canadian Rockies from the north. A cyclonically curved flow around a downstream upper-level trough occurs over the northwestern United States into the northern plains. A broad subtropical jet is located over southern California and Utah, northern Arizona and New Mexico, and the Colorado Rockies. Both the RAPv4 and RAPv3 models predict similar structures of the large-scale upper-level flows mentioned above, although there are some minor differences in local areas. However, there is a significant discrepancy in the vertical velocity field (color shadings in Fig. 3) between the two models. In particular, small-scale noisy signals in vertical velocity fields over the western mountainous regions are significantly removed in the RAPv4 (Fig. 3a), while those are dominant in the RAPv3 (Fig. 3b). It is also impressive that the RAPv4 still maintains large-amplitude mountain waves over southeast Alaska and the Colorado Rockies (Fig. 3a), indicating the new model with the hybrid vertical coordinate eliminates or damps the artificial vertical motions while retaining the physically meaningful large-amplitude mountain waves. This helps alleviate the high false alarms of LOG-level MWT forecasts over the mountain regions.
Horizontal distributions of the vertical pressure velocity field (shading; Pa s−1) and horizontal wind speed (red contours greater than 30 m s−1 with 10 m s−1 intervals) valid at 1800 UTC 25 May 2017 at 35 000 ft (about z = 10.5 km) from 6-h forecasts of NOAA’s (a) RAPv4 and (b) RAPv3. Plots are zoomed into the western regions in North America in the RAP model domain.
Citation: Weather and Forecasting 34, 3; 10.1175/WAF-D-18-0187.1
To conduct objective evaluations of the improvement of turbulence forecasts based on the RAPv4 model, we have archived more than 1.2 million in situ turbulence reports from commercial aircraft for 4 months (September–December 2017, inclusive). The National Center for Atmospheric Research (NCAR) developed the automated in situ EDR estimation algorithm to estimate the magnitude (i.e., intensity) of atmospheric turbulence that directly affects aircraft (Sharman et al. 2014; Cornman 2016). This is an aircraft-independent turbulence metric and is the official standard for reporting aircraft turbulence by the International Civil Aviation Organization (ICAO 2010). A growing number of commercial aircraft worldwide are equipped to report in situ EDR using the NCAR algorithm. Figure 4 shows the horizontal distribution of the archived in situ EDR data reported at ±1 h around 1800 UTC during the research period, covering most areas of the CONUS including the western mountainous region. These data also cover some of steep mountain regions in southern Greenland under trans-Atlantic flight routes.
Density of automated in situ EDR (m2/3 s−1) scale turbulence reports from commercial aircraft flying at cruising altitudes between 20 000 and 45 000 ft (about z = 6.5–14 km) for 4 months (September–December 2017). Each of the 1 277 844 EDR observations reported during the ±1 h around 1800 UTC are assigned by location to the appropriate grid box of the Rapid Refresh model with 13-km horizontal grid spacing.
Citation: Weather and Forecasting 34, 3; 10.1175/WAF-D-18-0187.1
Here, g is gravitational acceleration,
These two indices are included in the operational GTG MWT forecasts (Sharman and Pearson 2017). Each are then separately converted to EDR by using a lognormal mapping scheme (Sharman and Pearson 2017),
Figure 5 shows the statistical receiver operating characteristic (ROC) curves for the |w|-based EDR forecasts (top) and |w|/Ri-based EDR forecasts (bottom) for the 1 277 844 in situ EDR observations over a 4-month period (September–December 2017). The ROC curves are constructed based on the probability of detection for “yes” forecasts of moderate-or-greater (MOG) turbulence (EDR > 0.22 m2/3 s−1) and for “no” forecasts (i.e., null turbulence; EDR < 0.02 m2/3 s−1) (e.g., Kim et al. 2015, 2018; Sharman et al. 2014; Sharman and Pearson 2017). Among the data shown in Fig. 5, there were 1021 MOG-level observations, and 1 209 539 were null. In the ROC curves of Fig. 5, if the forecast product perfectly discriminates both MOG-level and null turbulence events, the ROC curves move toward the upper-left-hand corner of the diagram and the area under the ROC curve (AUC) approaches unity, while forecasts with no skill follow a diagonal line with the AUC = 0.5 (shown as black diagonal lines in Fig. 5). For both the |w| and |w|/Ri diagnostics, the ROC curves move up to the left significantly for the RAPv4 compared to the RAPv3, giving higher AUC values of 0.69 for |w| and 0.83 for |w|/Ri based on RAPv4 compared to 0.63 for |w| and 0.77 for |w|/Ri based on RAPv3. Thus, the AUC for the RAPv4 with the hybrid vertical coordinate improves by almost 9.5% and 7.8% for the |w|- and |w|/Ri-based EDR turbulence forecasts, respectively.
ROC curves for (top) |w|-based and (bottom) |w|/Ri-based EDR forecasts against observed in situ EDR measurement data from the commercial aircraft shown in Fig. 4. The ROC curves are based on the detection of probabilities for “yes” (PODy) forecasts for MOG turbulence (EDR > 0.22 m2/3 s−1) and for “no” (PODn) forecasts for null turbulence (EDR < 0.02 m2/3 s−1). The results based on RAPv4 and RAPv3 are depicted as blue and black lines, respectively. The max and min ROC curves among 200 additional experiments for randomly selected half-fraction subsets for |w| and |w|/Ri from RAPv3 and RAPv4 are also shown as dashed lines to assess the statistical confidence of the results.
Citation: Weather and Forecasting 34, 3; 10.1175/WAF-D-18-0187.1
For testing the statistical significance of the evaluation results, we set up the 200 additional experiments for constructing the ROC curves, which are based on the 200 subsets of randomly selected half-fraction samples from the total MOG (1021) and null (1 209 539) data. This provides the maximum and minimum limits of the 200 ROC curves with corresponding AUC values, which gives an idea of the statistical robustness of the evaluation results (e.g., Sharman et al. 2006; Kim et al. 2011). Figure 5 includes the maximum and minimum ROC curves and their corresponding AUC scores for both the |w| and |w|/Ri diagnostics from the RAPv3 and RAPv4 outputs. It is found that the maximum and minimum AUC values among the 200 additional experiments fall within ±2%–3% of the computed performance using all data. For example, the maximum and minimum AUC scores for |w| from RAPv4 (RAPv3) are 0.72 (0.65) and 0.68 (0.61), respectively, and those for |w|/Ri from RAPv4 (RAPv3) are 0.84 (0.79) and 0.81 (0.75), respectively. These additional tests confirm that the improvements of the MWT forecasts from the new RAPv4 with the hybrid vertical coordinate system are statistically confident and significant. We also calculated the bias for the MOG and null turbulence forecasts. As expected, it is found that the bias for null forecasts are significantly improved for |w| (0.238) and |w|/Ri (0.127) with RAPv4 compared to those with RAPv3 (0.316 and 0.149) as tabulated in Table 1, because the RAPv4 with the hybrid vertical coordinate alleviates the spurious mountain features and reduces the high bias over the mountainous region (Fig. 2). The bias for MOG turbulence is also improved for |w|-based EDR forecasts, but not for the |w|/Ri-based one.
Statistical evaluation metrics of AUC and bias for MOG turbulence (EDR > 0.22 m2/3 s−1) and null turbulence (EDR < 0.02 m2/3 s−1) for the |w|- and |w|/Ri-based EDR turbulence diagnostics and their scores from the RAPv3 and RAPv4 models against in situ EDR observation data shown in Fig. 4. Bold numbers highlight the significant improvements of the scores in the new RAPv4 model.
4. Summary and conclusions
Aviation users have benefited from the enhanced awareness of expected turbulence areas provided by the NWP-based operational the turbulence forecast product (GTG) over North America provided by the NOAA’s Aviation Weather Center. This product was originally developed by NCAR to integrate multiple turbulence diagnostics to infer aircraft-scale turbulence intensity as a function of EDR from NOAA’s operational RAP model outputs. The performance of the GTG algorithm is therefore highly dependent upon the underlying RAP model. Related to this, users have reported that there are high false alarm rates of light-or-greater (LOG) turbulence forecasts, especially over the western mountainous CONUS. It turned out that this is partly due to the use of conventional terrain-following vertical coordinates that create small-scale artificial wave motions over the steep terrain. To reduce this problem, a hybrid sigma–pressure coordinate system has been implemented in the new model (RAPv4), which replaces the old model (RAPv3) that uses the conventional terrain-following sigma coordinate system.
A comparison of vertical velocity fields from both the RAPv4 and RAPv3 show that smaller-scale artificial waves over mountainous regions are removed or greatly damped in the new RAPv4 model. To conduct objective evaluations, two MWT indices of absolute vertical velocity (|w|) and |w|/Richardson number (|w|/Ri) were tested. These two indices from both the RAPv4 and RAPv3 were converted to an EDR scale, and then those were matched separately with 1.2 million automatically reported in situ EDR estimates from commercial aircraft for 4 months (September–December 2017). Resultant statistics of the area under the receiver operating characteristic (ROC) curves showed that the performance of |w|- and |w|/Ri-based EDR turbulence forecasts from the new RAPv4 are superior to those from the old RAPv3. Bias calculations for moderate-or-greater (MOG) and null turbulence in those products confirm that the RAPv4 with hybrid vertical coordinate system significantly reduces the high bias for null turbulence forecasts. The new RAPv4 system became officially operational on 12 July 2018, providing better guidance for operational turbulence forecasts over North America. This represents a major improvement in an important decision support tool for the aviation community.
Acknowledgments
The authors thank Dr. Joshua W. Scheck at the NOAA/Aviation Weather Center (AWC) for his thorough review on the original manuscript. The authors also thank another anonymous reviewer for his/her comments and suggestions on the paper. This research was funded in part by the Federal Aviation Administration (FAA) Grant DTFACT-17-X-80002. The views expressed are those of the authors and do not necessarily represent the official policy or position of the FAA. Jung-Hoon Kim (JHK) used to work at the NOAA/(AWC) as an affiliated research scientist from the Colorado State University/Cooperative Institute for Research in Atmosphere (CSU/CIRA). He appreciates all of the support from the NOAA/AWC and CSU/CIRA for this work. JHK was supported by the Research Resettlement Fund for the new faculty of Seoul National University. JHK was also supported by the Research and Development for the Korean Meteorological Administration (KMA) Weather, Climate, and Earth System Services. Sang-Hun Park (SHP) was supported by the Yonsei University Future-leading Research Initiative of 2018-22-0021.
REFERENCES
Benjamin, S. G., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The Rapid Refresh. Mon. Wea. Rev., 144, 1669–1694, https://doi.org/10.1175/MWR-D-15-0242.1.
Cornman, L. B., 2016: Airborne in situ measurements of turbulence. Aviation Turbulence: Processes, Detection, Prediction, R. Sharman and T. Lane, Eds., Springer, 97–120, https://doi.org/10.1007/978-3-319-23630-8_5.
Elvidge, A. D., S. B. Vosper, H. Wells, J. C. H. Cheung, S. H. Derbyshire, and D. Turp, 2017: Moving towards a wave-resolved approach to forecasting mountain wave induced clear air turbulence. Meteor. Appl., 24, 540–550, https://doi.org/10.1002/met.1656.
International Civil Aviation Organization, 2010: Meteorological service for international air navigation. Annex 3 to the Convention on International Civil Aviation, 17th ed. ICAO International Standards and Recommended Practices Tech. Annex, 206 pp.
Kim, J.-H., and H.-Y. Chun, 2010: A numerical study of Clear-Air Turbulence (CAT) encounters over South Korea on 2 April 2007. J. Appl. Meteor. Climatol., 49, 2381–2403, https://doi.org/10.1175/2010JAMC2449.1.
Kim, J.-H., and H.-Y. Chun, 2011: Statistics and possible sources of aviation turbulence over South Korea. J. Appl. Meteor. Climatol., 50, 311–324, https://doi.org/10.1175/2010JAMC2492.1.
Kim, J.-H., H.-Y. Chun, R. D. Sharman, and T. L. Keller, 2011: Evaluations of upper-level turbulence diagnostics performance using the Graphical Turbulence Guidance (GTG) system and Pilot Reports (PIREPs) over East Asia. J. Appl. Meteor. Climatol., 50, 1936–1951, https://doi.org/10.1175/JAMC-D-10-05017.1.
Kim, J.-H., W. N. Chan, B. Sridhar, and R. D. Sharman, 2015: Combined winds and turbulence prediction system for automated air-traffic management applications. J. Appl. Meteor. Climatol., 54, 766–784, https://doi.org/10.1175/JAMC-D-14-0216.1.
Kim, J.-H., R. Sharman, M. Strahan, J. Scheck, C. Bartholomew, J. Cheung, P. Buchanan, and N. Gait, 2018: Improvements in nonconvective aviation turbulence prediction for the World Area Forecast System (WAFS). Bull. Amer. Meteor. Soc., 99, 2295–2311, https://doi.org/10.1175/BAMS-D-17-0117.1.
Klemp, J. B., 2011: A terrain-following coordinate with smoothed coordinate surfaces. Mon. Wea. Rev., 139, 2163–2169, https://doi.org/10.1175/MWR-D-10-05046.1.
Lane, T. P., J. D. Doyle, R. Plougonven, M. A. Shapiro, and R. D. Sharman, 2004: Observations and numerical simulations of inertia–gravity waves and shearing instabilities in the vicinity of a jet stream. J. Atmos. Sci., 61, 2692–2706, https://doi.org/10.1175/JAS3305.1.
Lane, T. P., J. D. Doyle, R. D. Sharman, M. A. Shapiro, and C. D. Watson, 2009: Statistics and dynamics of aircraft encounters of turbulence over Greenland. Mon. Wea. Rev., 137, 2687–2702, https://doi.org/10.1175/2009MWR2878.1.
Park, S.-H., J.-H. Kim, R. D. Sharman, and J. B. Klemp, 2016: Update of upper level turbulence forecast by reducing unphysical components of topography in the numerical weather prediction model. Geophys. Res. Lett., 43, 7718–7724, https://doi.org/10.1002/2016GL069446.
Park, S.-H., J. B. Klemp, and J.-H. Kim, 2019: Hybrid mass coordinate in WRF-ARW and its impact on upper-level turbulence forecasting. Mon. Wea. Rev., 147, 971–985, https://doi.org/10.1175/MWR-D-18-0334.1.
Phillips, N. A., 1957: A coordinate system having some special advantages for numerical forecasting. J. Meteor., 14, 184–185, https://doi.org/10.1175/1520-0469(1957)014<0184:ACSHSS>2.0.CO;2.
Powers, J. G., and Coauthors, 2017: The Weather Research and Forecasting Model: Overview, system efforts, and future directions. Bull. Amer. Meteor. Soc., 98, 1717–1737, https://doi.org/10.1175/BAMS-D-15-00308.1.
Sharman, R. D., and T. Lane, Eds., 2016: Aviation Turbulence: Processes, Detection, Prediction. Springer, 523 pp., https://doi.org/10.1007/978-3-319-23630-8.
Sharman, R. D., and J. Pearson, 2017: Prediction of energy dissipation rates for aviation turbulence. Part I: Forecasting nonconvective turbulence. J. Appl. Meteor. Climatol., 56, 317–337, https://doi.org/10.1175/JAMC-D-16-0205.1.
Sharman, R. D., C. Tebaldi, G. Wiener, and J. Wolff, 2006: An integrated approach to mid- and upper-level turbulence forecasting. Wea. Forecasting, 21, 268–287, https://doi.org/10.1175/WAF924.1.
Sharman, R. D., J. D. Doyle, and M. A. Shapiro, 2012a: An investigation of a commercial aircraft encounter with severe clear-air turbulence over western Greenland. J. Appl. Meteor. Climatol., 51, 42–53, https://doi.org/10.1175/JAMC-D-11-044.1.
Sharman, R. D., S. B. Trier, T. P. Lane, and J. D. Doyle, 2012b: Sources and dynamics of turbulence in the upper troposphere and lower stratosphere: A review. Geophys. Res. Lett., 39, L12803, https://doi.org/10.1029/2012GL051996 .
Sharman, R. D., L. B. Cornman, G. Meymaris, J. Pearson, and T. Farrar, 2014: Description and derived climatologies of automated in situ eddy-dissipation-rate reports of atmospheric turbulence. J. Appl. Meteor. Climatol., 53, 1416–1432, https://doi.org/10.1175/JAMC-D-13-0329.1.
Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-475+STR, 113 pp., https://doi.org/10.5065/D68S4MVH.