The Hurricane Weather Research and Forecasting Model (HWRF) is an operational model used to provide numerical guidance in support of tropical cyclone forecasting at the National Hurricane Center. HWRF is a complex multicomponent system, consisting of the Weather Research and Forecasting (WRF) atmospheric model coupled to the Princeton Ocean Model for Tropical Cyclones (POM-TC), a sophisticated initialization package including a data assimilation system and a set of postprocessing and vortex tracking tools. HWRF’s development is centralized at the Environmental Modeling Center of NOAA’s National Weather Service, but it incorporates contributions from a variety of scientists spread out over several governmental laboratories and academic institutions. This distributed development scenario poses significant challenges: a large number of scientists need to learn how to use the model, operational and research codes need to stay synchronized to avoid divergence, and promising new capabilities need to be tested for operational consideration. This article describes how the Developmental Testbed Center has engaged in the HWRF developmental cycle in the last three years and the services it provides to the community in using and developing HWRF.
How the Developmental Testbed Center facilitates the use of HWRF by the community, leading to an improved model for operations.
The Hurricane Weather Research and Forecasting Model (HWRF) is a coupled atmosphere–ocean dynamic forecast model run operationally by the National Oceanographic and Atmospheric Administration (NOAA) National Centers for Environmental Prediction (NCEP) to provide tropical cyclone track and intensity guidance to the National Hurricane Center (NHC) (Tallapragada et al. 2013; Rappaport et al. 2009). The HWRF was implemented operationally in 2007, meeting the need for modernization of numerical models to support the NHC forecasting process.
After the HWRF initial implementation, the NHC required that it be upgraded yearly, and it became clear that, to accelerate the pace of HWRF improvement, it would be necessary to complement the main development occurring at the NOAA Environmental Modeling Center (EMC) with research and development taking place at institutions outside the National Weather Service (NWS). In 2008, a version of the HWRF system (dubbed HWRFX) was adopted and advanced further at the Hurricane Research Division (HRD) of the NOAA/Atlantic Oceanographic and Meteorological Laboratory (AOML) to study the intensity change problem at cloud-resolving scales (X. Zhang et al. 2011; Bao et al. 2012; Gopalakrishnan et al. 2011, 2012, 2013; Yeh et al. 2012). In 2009, the NWS partnered with the Developmental Testbed Center (DTC; Bernardet et al. 2008; Ralph et al. 2013) to facilitate the transition of new research developments onto HWRF operations—a process referred to as R2O. This effort paralleled the establishment of the NOAA Hurricane Forecast Improvement Project (HFIP) in 2008 (Gall et al. 2013)—an initiative geared toward funding tropical cyclone research for accelerating the improvement of operational numerical model guidance to NHC. The hurricane activities in the DTC and HFIP work synergistically to support R2O, with the DTC hurricane efforts being currently fully focused on HWRF.
The DTC’s strategy to make new research results available for operational consideration hinges on three activities: establishing solid code management practices so that all HWRF developers use a single code base, supporting the community in using HWRF and adding innovations to the code, and conducting HWRF testing and evaluation. This paper gives a brief description of the HWRF model and provides more details of the DTC’s activities to stimulate transition of new research to operations.
HWRF is a complex system with multiple components listed below and illustrated in Fig. 1. The system undergoes an operational implementation cycle every year, in which many changes are introduced. The configuration described in this session corresponds to the 2013 operational model (Tallapragada et al. 2013; Bernardet et al. 2013a):
atmospheric model [Weather Research and Forecasting Model (WRF)],
atmospheric preprocessor [WRF Preprocessing System (WPS)],
ocean model [Princeton Ocean Model for Tropical Cyclones (POM-TC)],
atmosphere–ocean–wave coupler (HWRF specific, developed at NCEP),
vortex initialization and libraries (HWRF Utilities),
data assimilation system [Gridpoint Statistical Interpolation (GSI)],
postprocessor [Unified Post Processor (UPP)], and
external vortex tracker [Geophysics Fluid Dynamics Laboratory (GFDL) tracker].
The atmospheric component of HWRF is a configuration of WRF that has been designed to simulate and predict tropical cyclones. It includes the Nonhydrostatic Mesoscale Model (NMM) dynamic core with telescopic, two-way interactive, vortex-following moving nests. The physics suite used operationally in 2013 includes the Simplified Arakawa–Schubert (SAS) cumulus scheme, the GFDL model surface layer and radiation parameterizations, the Global Forecasting System (GFS) boundary layer parameterization, and the tropical Ferrier microphysics scheme. HWRF’s oceanic component, POM-TC, was developed at the University of Rhode Island (URI; Yablonsky et al. 2015).
The atmospheric model is run with three telescopic atmospheric domains. The parent domain covers an 80° × 80° latitude–longitude area with a grid spacing of approximately 27 km, the intermediate nest domain covers 11° × 11° with a 9-km grid spacing, and the innermost nest covers 6.5° × 7.2° with 3-km grid spacing. The nests move to follow the storm.
When a tropical disturbance (i.e., invest) or tropical cyclone is identified by NHC in the North Atlantic or eastern North Pacific basins, HWRF is initialized every 6 h and run out to 126 h. The large-scale atmospheric fields are initialized using the HWRF Data Assimilation System (HDAS) to assimilate conventional observations using the 6-h forecast of the Global Data Assimilation System (GDAS) as a first guess. Both GDAS and HDAS employ a hybrid ensemble–variational configuration of GSI (Wang et al. 2013) that ingests the GFS ensemble to create flow-dependent background error covariances. Important differences between GDAS and HDAS include the types of observations that are assimilated and the fact that HDAS is performed directly on the HWRF domains.
A vortex initialization technique is used to correct the position, intensity, and structure of the vortex, according to observations. The storm-scale initial conditions are further refined with the assimilation of tail Doppler radar data collected by the NOAA P-3 aircraft, when available.
A monthly, three-dimensional temperature and salinity climatology, modified with a feature-based ocean initialization process (in the Atlantic basin only), the daily GFS sea surface temperature (SST), and two phases of ocean model integration, generate initial conditions for the POM-TC oceanic component of HWRF (Yablonsky and Ginis 2008; Yablonsky et al. 2015). HWRF atmospheric and oceanic components then run in parallel and exchange information through the coupler every 9 min: the atmospheric model calculates and sends the heat and momentum fluxes to the ocean, while the ocean model sends the SST to the atmospheric model. The POM-TC uses 18-km horizontal grid spacing.
HWRF postprocessing makes use of the NCEP UPP and of the GFDL vortex tracker, which can extract the tropical cyclone’s location, intensity, and structure from the model output (Tallapragada et al. 2013).
The use of modern software engineering solutions has played a fundamental role in supporting the distributed, fast-paced development of HWRF. Each of the eight HWRF components is kept under software version and revision control using the Subversion software, commonly known as SVN. A ninth code repository centralizes the system and allows checking out the entire HWRF system by linking to each of the eight community repositories as externals.
There are diverse origins for the code repositories of each component, but in all cases they are community repositories set up with the intention of facilitating collaboration among distributed developers. For example, the WRF code repository has existed for over seven years, is housed at the National Center for Atmospheric Research (NCAR), and is administered by the WRF developers in NCAR’s Microscale and Mesoscale Meteorology (MMM) group. Conversely, the GSI code repository has existed for four years, is housed at the NOAA/Earth System Research Laboratory (ESRL), and is administered by the DTC. These community code repositories support general development for many applications. In that sense, HWRF can be considered a configuration of a broader code or a subset that gets chosen through a set of compile- and run-time options. Since the code receives frequent updates for both HWRF and non-HWRF applications, a sophisticated system of developmental branches, synchronization, and consistency checks was established to create safeguards and protect the HWRF operational code against inadvertent changes.
The system described in Fig. 2 applies to each of the eight HWRF components. The HWRF development at EMC is done in branches/HWRF, while individual HWRF developers outside of EMC use project branches, which they keep updated with branches/HWRF. Mature developments are committed to branches/HWRF once acceptance tests are conducted. The HWRF development keeps pace with the larger community through the synchronization between branches/HWRF and the trunk. This mechanism makes capabilities developed for HWRF available to general users and, in turn, makes it possible for HWRF to benefit from developments originating outside its main developers at EMC.
Before bringing contributions from the general community to branches/HWRF, the DTC conducts a consistency check to make sure that changes not intended to alter the HWRF forecast’s answer (e.g., changes in a physical parameterization not employed in HWRF) do not change its results. It is important for the HWRF code to stay synchronized with the community code, even in aspects that do not change the HWRF forecasts, to avoid code divergence, which allows easy transfer of community developments that do impact the HWRF forecasts. Using this rigorous mechanism, the DTC has shown it is safe for an operational code to reside in a community repository outside of NCEP. Since the implementation of the HWRF code management in 2011, the HWRF code has remained unified, with codes for the yearly operational implementation and public release originating from branches/HWRF.
One of the substantial benefits brought about by the unification of the research and operational HWRF codes was the transition to operations of developments for high-resolution forecasting originating from HRD/AOML in the late 2000s. Because of its unique capabilities of procuring TC observations and its considerable expertise in analyzing them, HRD was able to improve both the HWRF physical parameterizations and software infrastructure, leading to an improved model (Gopalakrishnan et al. 2013). The HWRF code unification made it possible for these developments to become available for preimplementation testing, which led to the upgrade of the operational HWRF from a two-domain system (27- and 9-km grid spacing) to a high-resolution triple domain system in 2012.
CONNECTING WITH THE RESEARCH COMMUNITY.
The TC community utilizes a variety of models for research. Work using the Advanced Hurricane WRF [which employs the Advanced Research WRF dynamic core of the WRF model; F. Zhang et al. (2011); Cavallo et al. (2013)] and the University of Wisconsin Nonhydrostatic Model, among others, have been funded by HFIP (Gall et al. 2013) because scientific discoveries and techniques developed using nonoperational models have the potential of benefitting operational numerical weather prediction (NWP) systems. However, when research is conducted directly using operational systems, the process of testing new developments for transition is streamlined, saving time and financial resources. The DTC uses several mechanisms to promote and support the use of HWRF in the research community—both for initial development and for transition of work originally done in other systems. This activity is referred to as transfer of operational capabilities to the research community (O2R).
Code access and user support.
The DTC maintains a WRF for Hurricanes website (www.dtcenter.org/HurrWRF/users), where registered users (currently 520) can obtain the latest HWRF code releases, a users’ guide, scientific documentation, test datasets, benchmarks, and access to a helpdesk. HWRF resident tutorials were taught in Boulder, Colorado, in 2010 and 2011 (Fig. 3), and an online tutorial has been available since 2012. The next in-person tutorials are planned for January 2014 (at the NCEP facilities in College Park, Maryland) and May 2014 (in Taiwan). Releases of well-tested stable code are planned on a yearly schedule, corresponding to the configuration used in operations in that year’s hurricane season. As an example, the HWRF v3.5a released to the public in August 2013 contains all capabilities of the 2013 operational implementation of HWRF.
While access to the HWRF code through the public release mechanism meets the needs of the majority of users, it does not address the requirements of core HWRF developers. For those expert users doing active code development, the DTC provides direct access to the repository. They can obtain the latest versions of the codes (including developmental ones) and use project branches in the code repository to add their contributions, which can be seen by and shared with other developers. The DTC is currently supporting 50 developers in this mode, and the 2013 operational configuration was chosen after a suite of tests was conducted using the code emerging from this multideveloper collaboration.
DTC Visitor Program.
The DTC Visitor Program provides financial and computational resources to selected projects originating from universities and research institutions. Typical areas of interest include adding new capabilities to operational models, conducting tests of new capabilities and NWP technology, and performing diagnostics and verification that can provide guidance for future model development. Three projects focused on HWRF have been funded to date, all of which are currently ongoing:
developing an HWRF diagnostics module to evaluate intensity and structure using synthetic flight paths through tropical cyclones, by J. Vigh of NCAR;
diagnosing tropical cyclone motion forecast errors in HWRF, by T. Galarneau of NCAR; and
improving HWRF track and intensity forecasts via model physics evaluation and tuning, by R. Fovell of University of California, Los Angeles (UCLA).
TESTING AND EVALUATION.
The preimplementation testing for HWRF follows a yearly cycle that is largely dictated by nature and tied to the eastern North Pacific and North Atlantic hurricane seasons. From May through November, the operational model remains frozen to assure consistency in the products sent to NHC. Only emergency bug fixes, particularly those geared toward prevention of runtime failures, are accepted for midseason implementation in the operational machine (the NOAA Weather and Climate Operational Supercomputing System).
During the hurricane season, several real-time experiments are run on NOAA research computers to collect information about the forecast skill and computational performance of promising new capabilities. The machines used for these tests are Zeus, located at the NOAA Environmental Security Computing Center, and the HFIP machine Jet, located at NOAA/ESRL. A subset of the experimental forecasts are sent to NHC following the Stream 1.5 process described by Gall et al. (2013), while the remaining are evaluated solely by HFIP and by the developers themselves. Although several developers contribute new capabilities, real-time HWRF experiments are usually conducted by EMC or by HRD.
As the North Atlantic and eastern North Pacific hurricane seasons approach their end and computer resources become available, selected new capabilities are tested retrospectively on a large number of cases. The number of cases varies depending on the test goals, ranging from selected storms all the way to multiple seasons. New capabilities are tested individually and in bundles in order to assess their potential for improving forecast skill, nonlinear interactions, and computational performance. These retrospective tests are typically conducted by EMC, HRD, and/or DTC. Using the results of the retrospective tests, EMC and NHC determine which new capabilities should be included in the next operational configuration and which new capabilities should be further developed and reconsidered for future implementations.
The tight chronogram of preimplementation testing requires close coordination among developers. Capabilities must be ready and implemented in the centralized code when the timeline demands. The DTC plays two roles in this process: providing code management to assure that all developers are keeping their codes synchronized and performing retrospective testing and evaluation. The next subsection describes an example of how DTC worked with the community to identify a problem, formulate a hypothesis, and conduct a comprehensive retrospective test that led to a change in the operational model.
Experiments for elimination of atmosphere–ocean flux reduction.
The 2012 operational HWRF model displayed a positive intensity bias for the North Atlantic basin (Cangialosi and Franklin 2013). Here intensity bias is defined as the mean error, over the entire hurricane season, of the maximum instantaneous 10-m winds on the model grid, retrieved from the 6-hourly HWRF forecast fields using the GFDL Vortex Tracker. The model initialization had near-zero bias but, over the 5-day forecast period, the bias became larger. The overintensification was not very large for the majority of storms, but could reach 30 kt (1 kt = 0.51 m s−1) in extreme cases, such as for Hurricane Leslie. Storm overintensification can have several causes rooted in model initialization, dynamics, or physics. Since the bias increased in time over the forecast period, with the larger growth occurring in the second and third days of the forecast, model initialization was likely a secondary effect.
A comparison between HWRF SST forecasts and climatological buoy data performed by HRD (Cione and Uhlhorn 2012) suggested that the HWRF ocean undercooled in the wake of tropical cyclones. This result was reviewed by POM-TC developers at URI, who formulated the hypothesis that the HWRF ocean response and, consequently, the forecast SST would improve if a simple change were made in the algorithm controlling the air–sea fluxes. Specifically, the proposed change involved the elimination of a 25% reduction factor that had been applied to HWRF’s air–sea heat and momentum fluxes used to force the POM-TC ocean model since HWRF became operational in 2007. A component of this reduction factor can be considered physically based because, in nature, some of the momentum transferred from the atmosphere to the ocean is transported away and dissipated through waves (Fan et al. 2010)—a process not taken into account in the current operational HWRF. The application of this reduction factor in HWRF, however, was originally implemented primarily as a practical measure to curb hurricane-induced SST overcooling in the tests that preceded HWRF’s initial 2007 implementation. The URI scientists hypothesized that the recent improvements to the HWRF atmosphere, including increased horizontal resolution of the innermost HWRF grid (from 9- to 3-km grid spacing) and physics improvements implemented in 2010–12, had now rendered this flux reduction factor unnecessary. Hence, DTC undertook the testing and evaluation of the HWRF model performance with the removal of the 25% air–sea flux reduction factor.
The design of the retrospective test for the elimination of the reduction factor was decided collectively by DTC, EMC, and URI. The test was conducted by DTC using all North Atlantic and eastern North Pacific storms of the 2012 season. After a control equivalent to the 2012 operational implementation and a variant eliminating the flux reduction were run, the results were analyzed by DTC and HRD (Bernardet et al. 2013b). Figure 4 displays the intensity bias averaged over all 2012 Atlantic storms for the control (termed HD12) and for the elimination of the flux reduction (termed HDFL). The timeliness of the results (described in detail at www.dtcenter.org/eval/hwrf_hdfl_hd12/), which displayed a statistically significant reduction in intensity bias without major changes in other parameters, allowed the adoption of the modified algorithm for the 2013 operational implementation of HWRF.
HWRF is run operationally at NCEP to provide numerical guidance to NHC and, as such, has a need for continuous upgrades. Given the limited resources in the NWS for model development, there is a recognized need for harnessing the research conducted in NOAA laboratories and academic institutions. To facilitate R2O in tropical cyclone NWP, a partnership between NWS and DTC was established in 2009. Efforts have been focused on three fronts: supporting the operational HWRF to the general public (O2R), providing management to keep distributed code development synchronized, and testing innovations with high potential for operational transition using the DTC (R2O). It should be noted that this framework for collaboration is one of a set currently being explored by the DTC—an institution that is involved with the improvement of operational NWP for various applications, such as midlatitude mesoscale weather (Wolff et al. 2012) and ensemble forecasting (Tollerud et al. 2013).
This structure for infusion of new technologies in the NWS has been successful, as seen in the large improvements realized by the HWRF system in the last few years (Tallapragada 2013), and it should be continued in order to meet the ambitious 10-yr goals of forecast improvement established by HFIP (Gall et al. 2013). It is important that the DTC continues to conduct in-depth diagnostics of HWRF to document its strengths and weaknesses and to identify areas in which new research and development is most needed.
Future improvements in HWRF are planned on a variety of fronts. Consideration will be given to transition to a new dynamic core—the NMM in the Arakawa-B grid (NMM-B) in the NOAA Environmental Modeling System (NEMS) framework—in order to enable compatibility with other NOAA operational NWP systems (such as the North American Mesoscale model and the Short Range Ensemble Forecast system), which are transitioning away from WRF-NMM in favor of NMM-B in the NEMS framework. More advanced physical parameterizations, geared toward high-resolution tropical applications, are being tested. Higher-resolution domains and the use of multiple simultaneous moving nests are being developed. Advancements in data assimilation, employing satellite radiances and aircraft in situ observations, are a top priority. Ocean model improvements developed at URI are planned for 2014 HWRF operations, and three-way hurricane–wave–ocean coupling is a strong possibility for the near future. Finally, the coupling of HWRF with downstream applications, such as storm surge and inundation models, is on the horizon. Community involvement is paramount so that measurable advances can be realized in those fronts, and researchers are encouraged to make use of the DTC services to participate in HWRF development.
The DTC is funded by NOAA, the Air Force Weather Agency, the National Science Foundation, and the National Center for Atmospheric Research. This work was partially supported by HFIP.
CURRENT AFFILIATION: Coastal Carolina University, Conway, South Carolina
CURRENT AFFILIATION: Research Computing, University of Colorado, Boulder, Colorado
CURRENT AFFILIATION: AIR Worldwide, Boston, Massachusetts