With a variety of programming languages and data formats available, widespread adoption of computing standards by the atmospheric science community is often difficult to achieve. The Sounding and Hodograph Analysis and Research Program in Python (SHARPpy) is an open-source, cross-platform, upper-air sounding analysis and visualization package. SHARPpy is based on the National Oceanic and Atmospheric Administration/Storm Prediction Center’s (NOAA/SPC) in-house analysis package, SHARP, and is the result of a collaborative effort between forecasters at the SPC and students at the University of Oklahoma’s School of Meteorology. The major aim of SHARPpy is to provide a consistent framework for sounding analysis that is available to all. Nearly all routines are written to be as consistent as possible with the methods researched, tested, and developed in the SPC, which sets this package apart from other sounding analysis tools.
SHARPpy was initially demonstrated and released to the atmospheric community at the American Meteorological Society (AMS) Annual Meeting in 2012, and an updated and greatly expanded version was released at the AMS Annual Meeting in 2015. Since this release, SHARPpy has been adopted by a variety of operational and research meteorologists across the world. In addition, SHARPpy’s open-source nature enables collaborations between other developers, resulting in major additions to the program.
An internationally used open-source sounding analysis software package seeks to promote a community model for software development and help make research more reproducible.
With an ever-increasing diversity of programming languages and data formats available, it becomes challenging to establish computing standards supported by a majority of professional atmospheric scientists. Disturbingly, a survey of sounding analysis software suggests that as a community, there is little guidance on analysis standards for such a widely used, historical data type. In some cases, running similar analysis routines from different programs on the same sounding data produces substantially different results [e.g., convective available potential energy (CAPE) routines]. Attempts to rectify these differences may be hindered by a variety of reasons: programs can be written in different programming languages, are poorly documented, can be proprietary software, or are constrained to run on specific operating systems. These programs also vary in the needs they support: some may enable users to analyze real-time radiosonde data and others may analyze data in bulk to answer research questions. Unfortunately, this level of diversity of sounding analysis programs hinders the atmospheric science community’s ability to reconcile differences and produce generalizable knowledge.
Over the last several years, a unique relationship between atmospheric scientists and meteorology students has produced one solution to this problem. Instead of developing completely new software, sounding analysis code formulated and tested at the National Oceanic and Atmospheric Administration’s (NOAA) Storm Prediction Center (SPC) in Norman, Oklahoma, over the last 25 years was translated into a modern programming language and released through an open-source development framework. This new sounding analysis software, called the Sounding and Hodograph Analysis and Research Program in Python (SHARPpy), is written with a philosophy of flexibility and usability. By being open source and written in Python, SHARPpy offers atmospheric scientists a free program with the benefit of transparent source code that works on most modern operating systems. SHARPpy blends the capability to analyze real-time sounding data through a display and the ability to access libraries of sounding analysis routines (e.g., computing convective indices) through scripts. Through this work, we have attempted to standardize sounding analysis for a wide variety of individuals and organizations within the atmospheric science community. This article outlines the capabilities of SHARPpy and reveals both the story of its student-driven development process and its successful adoption by the larger community. Additionally, the project reveals some guidelines to consider when approaching community-oriented programming.
At the core of SHARPpy is a collection of sounding analysis routines for analyzing thermodynamic and kinematic data. Fundamentally, all analysis routines operate on a single user-supplied profile of temperature, dewpoint, pressure, height, wind speed, and wind direction. Through this framework, data that the program can analyze are not restricted to particular sources (e.g., only radiosondes). Rather, the program can accept and use data from any system that provides vertical thermodynamic and kinematic profiles. After the data are loaded, SHARPpy can calculate a diverse set of quantities relevant to deep, moist convection. Some calculations such as convective inhibition (CINH) require the definition of a parcel’s initial properties [e.g., surface based (SB)]. Other quantities may require the specification of a storm motion vector [e.g., effective storm-relative helicity (ESRH)] or a specific layer [e.g., 0–1-km bulk wind difference (BWD)]. Some parameters are also utilized in conjunction with databases like the Sounding Analog Retrieval System (SARS; Jewell 2010), or within decision trees, such as in the possible hazard type (PHT) box, each of which may provide insight about specific severe thunderstorm hazards. Such routines form the core functionality of SHARPpy and can be accessed depending on how the user wishes to use the program.
The user may choose to interact with the core libraries of the program either by writing scripts that call upon specific routines or by utilizing the program’s graphical user interface (GUI). For example, if the user wishes to perform a bulk analysis of surface-based CAPE (SBCAPE) in a sounding dataset, scripts can be used to bypass the graphical interface and calculate only SBCAPE. By offering this option, users are not required to plot their data to access specific analysis routines. However, if the user wishes to analyze an individual sounding or visualize real-time observed and forecast data [e.g., Global Forecast System (GFS) forecast soundings], the SHARPpy GUI offers a flexible and informative framework.
Figure 1 depicts the GUI output from a sounding released during the historic 3 May 1999 central Oklahoma tornado outbreak. Both the skew T–logp diagram (inset A) and a hodograph plot (inset D) form the core of the display. The skew T–logp image contains vertical profiles of temperature (solid red), virtual temperature (dashed red), wet-bulb temperature (cyan), dewpoint temperature (green), the most unstable (MU) parcel virtual temperature trace (dashed white), the downdraft parcel virtual temperature trace (dashed purple), and wind barbs in knots (kt, where 1 kt = 0.5144 m s-1). If a vertical velocity profile (microbars per second) is included in the data passed to the GUI [e.g., from a numerical weather prediction (NWP) forecast sounding], it is displayed on the left-hand side of inset A (not shown). The hodograph plot is annotated with the Bunkers supercell motion estimates [white left mover (LM) and right mover (RM) markers; Bunkers et al. (2014)], Corfidi mesoscale convective system (MCS) motion estimates [blue up (UP) and down (DN) markers; Corfidi (2003)], and lifting condensation level (LCL)–equilibrium level (EL) mean wind (brown square marker). The layer denoted in the bottom left of inset A [surface (SFC)–1,409 m] and the two thin cyan lines connecting the hodograph to the RM storm motion marker in inset D denote the effective inflow layer top and bottom (Thompson et al. 2007). If the bottom of the effective inflow layer is at the surface, the critical angle parameter and the 0–500-m AGL shear vector (purple line) are displayed in the hodograph (see Esterheld and Giuliano 2008).
Smaller insets supplement the skew T–logp image and hodograph. Such insets include the vertical profile of wind speed (kt; inset B), 100-mb-layer (1 mb = 1 hPa) inferred geostrophic temperature advection (inset C), equivalent potential temperature (θe, in K; inset F), and storm-relative winds (SRW, in kt; inset G). The geostrophic temperature advection values in inset C are calculated by assuming that all the winds in the sounding are in geostrophic balance (see Bluestein 1992, 185–186). They provide a quick method of identifying layers where winds back or veer with height. Insets E and H show the Storm Slinky (see the Storm Slinky sidebar) and the output from the PHT decision tree (see the possible hazard type box sidebar), respectively.
The Storm Slinky is a 2D visualization of the 3D trajectory of a parcel in a convective updraft, viewed from above the updraft tower. The parcel vertical displacement is calculated by first providing a convective parcel a 5 m s−1 initial speed at its LFC, then allowing the parcel to accelerate as a result of buoyancy until it reaches its EL. As the parcel accelerates upward, its horizontal displacement is calculated by advecting the parcel by the storm-relative winds. At the end of the trajectory calculation, the updraft tilt is calculated by determining the angle between the ground and the line between the first and last locations of the parcel.
Visualization of the updraft in this manner can provide insight into what type of storms an environment may produce. Figure SB1 illustrates this through two different Storm Slinky signatures often observed in the spectrum of deep convective environments. For environments supportive of supercells, such as in the 1974 Super Outbreak, deviant storm motions (e.g., Bunkers right-mover motion vector) cause the Storm Slinky rings to be displayed as kidney bean–type shapes (Fig. SB1a), similar to the echoes supercells produce on weather radars. In contrast, environments supportive of single-cell convection may produce signatures where the rings stack on top of one another and display little curvature (Fig. SB1b), suggesting individual cells will be short lived because the storm-relative winds are not sufficient to remove precipitation from their updrafts. While the program defaults to the most unstable parcel and the Bunkers right-mover storm motion vector (Bunkers et al. 2014) to calculate the parcel trajectory, the sensitivity of the Storm Slinky’s shape to permutations of these variables can be considered through the SHARPpy GUI. This is illustrated in Fig. SB1b, where the cloud-layer (LCL–EL) mean wind vector was used instead of the Bunkers storm motion vector, as the environment had deep-layer shear that was unsupportive of supercells.
The PHT decision tree relies on thresholds for multiple indices calculated from the input sounding to determine likely threats supported by the environment. Figure SB2 shows the flow diagram of this algorithm.
Starting with a sequence of sounding-derived indices, the algorithm first considers the possibility of a “particularly dangerous situation” tornado threat (PDS TOR; purple box). The following conditional statements consider tornado, severe thunderstorm, flash flood, winter weather, and excessive heat hazards. When a possible hazard’s conditions are not met, the program moves to the next set of criteria. If no hazards are found, the algorithm returns the value of “none.”
The reasoning for each hazard’s conditional statements is based on a combination of contiguous U.S. (CONUS) climatological datasets, ingredients-based forecasting, and forecaster experience. For the tornado and severe thunderstorm hazards listed, the logic is based on research and experience at the SPC. For example, the logic that controls the PDS TOR category is derived from CONUS supercell proximity sounding studies that describe the environments that support long-lived supercells that produce significant (EF2+) tornadoes (Bunkers et al. 2006; Thompson et al. 2003, 2007, 2012). To expand the list of hazards the program may detect, additional categories were added to SHARPpy. The flash flood category, for example, attempts to identify environments supporting slow-moving MCSs within an anomalously moist troposphere. Anomalously moist tropospheric profiles are detected by determining whether the sounding PW is larger than two standard deviations (2σ) above the mean (μ) climatological value. The estimate of MCS motion is taken to be the magnitude of the Corfidi upshear vector (Corfidi 2003), which must be less than 25 kt to trigger the flash flood category. The blizzard and excessive heat categories attempt to identify conditions that adhere to American Meteorological Society and National Weather Service definitions (American Meteorological Society 2015; Vargo et al. 2015). In the case of the blizzard hazard type, a precipitation-type algorithm based on Bourgouin (2000) is used to detect if the sounding supports snow.
The authors strongly encourage users to exercise healthy skepticism when working with or sharing results from the PHT box. Since its inception, the PHT’s purpose has been to act as an alarm bell that supports forecaster situational awareness. The PHT algorithm has not been extensively verified, and its use without discussion of the proper meteorological and scientific context may lend itself to miscommunication of potential hazards. In this era of widespread availability of model forecast soundings, it is critical to remember that the PHT output can be quite misleading in the case of inaccurate model forecast soundings. Accurate forecasting of any meteorological hazard requires critical analysis by the user, and knowledge of the criteria used in the PHT function may help guide additional questions and data to consider in the forecast process.
SHARPpy uses color as a visual cue. The colored bars in inset C denote the sign of the temperature advection (red, warm; blue, cold), and the lengths of the bars are proportional to the magnitude of the advection. The colors in insets B, D, and E represent different layers of the atmosphere (red, 0–3 km AGL; green, 3–6 km AGL; yellow, 6–9 km AGL; cyan, 9–15 km AGL). The colored vertical bars in inset G represent various layer-averaged storm-relative winds that relate to the supercell type [purple bar, 9–11 km AGL; see Rasmussen and Straka (1998)] and supercell and tornado potential [red bar, 0–2 km AGL; blue bar, 4–6 km AGL; see Thompson (1998)]. Dashed purple lines depict the range of 9–11-km AGL storm-relative winds (40–60 kt) associated with classic supercell structures.
The bottom four insets in Fig. 1 provide an array of calculations for the user. Inset I shows thermodynamic calculations (see Tables 1 and 2 for a full list of quantities), while inset J shows kinematic quantities (see Table 3). Double-clicking the thermodynamic inset (inset I) allows the user to change which parcel types are displayed. The SARS output makes up inset K. SARS provides probabilistic forecasts of significant hail (≥2-in. diameter) and tornadoes by matching a given sounding to past CONUS severe weather proximity soundings (Jewell 2010). A list of the date, time, location, and hazard significance (e.g., hail size or tornado intensity) of the past soundings considered to be the closest matches are also displayed. The last inset, inset L, displays a box-and-whisker plot illustrating how the distribution of the significant tornado parameter using effective-layer quantities (STPE) within the right-moving supercell near-storm environment varies by enhanced Fujita (EF)-scale damage ratings damage ratings as derived from CONUS proximity soundings (Thompson et al. 2012, their Fig. 12). The smaller box within inset L indicates the relative frequency of EF2+ tornado damage with right-moving supercells for STPE and its components [mean-layer CAPE (MLCAPE), mean-layer LCL (MLLCL), etc.], based on Thompson et al. (2012). Insets K and L can each be replaced with one of five additional insets (not shown) by right-clicking on one of the insets. These optional insets give information on winter weather potential, fire weather potential, significant hail potential, and tornado intensity conditional probabilities based on STPE and radar-observed rotational velocity (see Smith et al. 2015) by displaying additional quantities computed from the displayed sounding (e.g., average vertical motion within the dendritic growth zone, expected precipitation type, maximum wind speed in the convective boundary layer).
The insets in Fig. 1 provide the user a summary of the sounding data and aim to offer scientifically supported, redundant output for what meteorological hazards may be associated with the sounding. Although the GUI provides this redundancy both by visual cues (e.g., high-contrast, color-coded text) and different algorithms, its intent is not to replace the practice of considering the raw data. Several insets encourage this step by offering interactivity that is accessible by clicking on the portion of the GUI that the inset lies in. For example, by right-clicking on the skew T–logp, a user can initialize the data readout and lift a custom parcel from the skew T–logp. Left-click dragging the temperature, dewpoint, or hodograph lines allows for the user to modify the sounding data or reset the profile to its original values. Additionally, within the hodograph, the user can modify the storm motion vector used to calculate various indices. Right-clicking within the hodograph provides the ability to center the plot on the various storm motion vectors and provides a tool for displaying the motion of a boundary on the hodograph, which can assist in forecasting the storm mode (Dial et al. 2010). Additional soundings can also be overlaid, and the user may employ the spacebar to change which sounding is in the foreground. Much of the GUI functionality and core routines described within this section have largely been a consequence of the years of development at SPC of SHARP, the parent program of SHARPpy.
SHARPpy DEVELOPMENT TIMELINE.
The story behind the development of SHARPpy begins 25 years ago with the release of SHARP: the first software package written for analyzing sounding data (Hart and Korotky 1991; Doswell 1992). SHARP’s influence is significant: it has facilitated numerous scientific publications and has been used extensively by atmospheric scientists across the world (e.g., Hanstrum et al. 2002; Sioutas et al. 2006). Since SHARP’s release, meteorologists at the SPC continued to develop the program, with a primary focus on enhancing SPC forecast operations. Although occasional releases of SPC-driven updates to this program have been facilitated through the second generation of the Advanced Weather Interactive Processing System (AWIPS-II), many of the modifications the SPC has made to the SHARP code have remained platform dependent and unavailable to the larger scientific community. These issues are not unique to SHARP; many popular sounding analysis programs are also closed source or platform dependent, sometimes both.
In 2011, Patrick Marsh, then a University of Oklahoma (OU) and Cooperative Institute for Mesoscale Meteorological Studies (CIMMS) graduate student, and SPC employees John Hart and Rich Thompson sought to remedy these issues through an open-source release of the SHARP code. Python was the language of choice, as it is a flexible, cross-platform programming language. Additionally, its popularity in the atmospheric sciences had just begun (Lin 2012). A first iteration of the code was released at the 2012 American Meteorological Society (AMS) Annual Meeting (Marsh and Hart 2012). The code contained a demo GUI and parcel lifting routines that could be called either by the GUI or by separate Python scripts and established a foundation for future code development. To spur additional development through community contributions, the code was placed on the Github website (https://github.com/), a venue commonly used by code developers to publicly distribute and track software.
For nearly two years after its initial release, community interest in developing SHARPpy faltered. However, the decision to place this code online eventually led OU undergraduate Kelton Halbert and CIMMS doctoral student Greg Blumberg to revive the project. By forging collaborations with the SPC meteorologists responsible for the in-house development and upgrades to SHARP, they were able to replicate their code and advance the SHARPpy GUI interface and underlying routines. Through these collaborations, they were also able to contribute and facilitate improvements within SHARP. An updated version of SHARPpy was released at the 2015 AMS Annual Meeting (Halbert et al. 2015) to demonstrate the capabilities of the new software. Soon after its 2015 release, Tim Supinie, another OU doctoral student, joined the development team. His contributions to the program further improved the speed of the program and provided a flexible framework for loading various data sources, such as a locally run NWP model. Additionally, his changes enabled both the modification and comparison of profiles within the GUI. These improvements to SHARPpy from 2014 to the present would not have been possible if the original code had not been available online.
Within the first year of the second release of SHARPpy, the atmospheric science community rapidly adopted the program and many have contacted the authors regarding how they have used SHARPpy in their work. In particular, deep-convection forecasting experiments have been able to leverage the tools within the SHARPpy GUI for their projects. For example, during summer 2015, forecasters for the Plains Elevated Convection at Night (PECAN; Geerts et al. 2017) field project used SHARPpy to forecast various nocturnal convective phenomena as well as to analyze data gathered while in the field. Per an anonymous reviewer, the authors have learned that SHARPpy also assisted forecasters during the 2016 Verification of the Origins of Rotation in Tornadoes Experiment-Southeast (VORTEX-SE) project. Outside of the United States, operational meteorologists at Environment and Climate Change Canada added both observed Canadian soundings and forecast output from the Global Environmental Multiscale Model (GEM) to SHARPpy. For them, SHARPpy is an invaluable forecasting tool (K. Ziolkowski 2016, personal communication). Additionally, SHARPpy was used the summer of 2015 during the European Severe Storms Laboratory (ESSL) Testbed to develop severe weather forecasts across Europe (P. Groenemeijer 2015, personal communication). Meteorologists from the ESSL accomplished this by adding their own real-time European radiosonde data stream to the program. Other organizations, such as Pivotal Weather and the College of DuPage, have incorporated SHARPpy into their websites through a point-and-click interface to analyze NWP output. For these organizations and projects, the SHARPpy GUI has provided a useful framework to analyze soundings.
Furthermore, other atmospheric scientists have found SHARPpy’s scripting functions to be an invaluable tool for research. Following the PECAN project, graduate students at various universities (e.g., University of Oklahoma, University of Manitoba, and North Carolina State University) have written scripts to integrate the SHARPpy routines into their research on nighttime convection. Outside of PECAN, the University of Wisconsin Space Science and Engineering Center (SSEC) has worked to incorporate the SHARPpy parcel lifting routines into studies evaluating satellite-based soundings in convective environments (Gartzke et al. 2017). Within the National Severe Storms Laboratory (NSSL), SHARPpy was reconfigured to generate postprocessed grids of CAPE, CINH, and storm-relative helicity (SRH) from real-time runs of the Warn-on-Forecast (WoF; Stensrud et al. 2009) prototype, the NSSL Experimental WoF System for Ensembles (P. Skinner 2016, personal communication; see Wheatley et al. 2015; Jones et al. 2016). On the other side of the globe, meteorologists at the Australian Bureau of Meteorology have used SHARPpy to analyze sea-breeze thunderstorm environments (J. Solderholm 2016, personal communication). This use of the SHARPpy code outside of operational forecasting speaks to its potential success in becoming a standard tool for meteorological applications. The scripting capability of SHARPpy has allowed its many libraries to be used in unique ways and suggests that SHARPpy effectively caters to the diverse needs of the atmospheric science community.
GUIDELINES FOR SIMILAR PROJECTS.
The success and rapid adoption of SHARPpy has suggested three potentially helpful guidelines regarding data types and community-oriented programming that we believe others should consider when pursuing similar projects. First, individuals and organizations should consider publicly sharing their source code and documentation through websites such as Github. As the availability and size of meteorological datasets grows, maintaining strict documentation of code versions and changes becomes essential, especially if the scientific community wishes to combat the reproducibility of computational results crisis (Irving 2016). SHARPpy meets this call by using Github to track code changes and issues, and an added benefit is that the community forum allows for additional creative development of the program by others.
Second, an emphasis should be placed on making the source code well documented and easy to run. A well-documented code base enables others to easily read the code, run it, debug it if necessary, and adapt the code to their needs. Within the SHARPpy source code, we have included information about not only the purpose of each routine, but from where in the scientific literature or atmospheric science community the algorithm was translated or derived. Including this information enables users to trace the science behind the code. SHARPpy fulfills the guideline of being easy to run because of the choice to use a freely available, cross-platform language such as Python.
Finally, the data formats that are used by the program should be easily understood and manipulated by the user. As the size of datasets grows, it becomes critical for both software and data providers to consider methods that make their data and metadata easily understood by the user. Instead of developing our own SHARPpy data format, SHARPpy primarily uses human-readable text and Network Common Data Form (netCDF) formats, which are already widely adopted data formats within the geophysical science community. By doing so, users are not restricted to a single or proprietary data format to run the SHARPpy program. Additionally, use of common data formats gives users flexibility in how they wish to write scripts to manipulate their own data and call SHARPpy routines. We believe that the SHARPpy program has been met with success not only because of the historical roots of the code, but also because the adopted programming and distribution guidelines make the program simple for others to access.
SUMMARY AND NEXT STEPS.
Within this article, we have described the capabilities, development, and adoption of the open-source SHARPpy sounding analysis program. This program contains the capabilities to inspect, modify, and process sounding data either through the program’s GUI interface or externally through the program libraries and scripts. The SHARPpy software package has roots in a program over 25 years old, and is actively developed through a collaborative relationship between OU students and SPC meteorologists. A wide variety of atmospheric scientists across the globe have adopted SHARPpy in their work, and we attribute this success not only to the community’s desire for such a tool, but also the methods we followed in the development of this tool. Given this success, we have outlined guidelines that should prove helpful to others when attempting similar community-oriented programming projects. Importantly, the story behind SHARPpy’s development and adoption serves as an example of the positive impact students can have on the larger community.
Future plans for the SHARPpy program aim to expand the capabilities and usability of the program. First, SHARPpy can serve as a vehicle for introducing new sounding data sources to the larger community. The community-oriented nature of the program has made it easy to collaborate with others to make available experimental datasets not traditionally accessible, such as the experimental runs of the Model for Prediction Across Scales (MPAS) from the NOAA Hazardous Weather Testbed, or the 3-km, 10-member National Center for Atmospheric Research (NCAR) ensemble (Schwartz et al. 2015). SHARPpy may also provide a framework for introducing data from new instruments, such as ground-based thermodynamic sounders and unmanned aerial vehicles to new users, or may provide access to potentially underutilized datasets like satellite-derived soundings (Przybyl et al. 2016). Second, as the use of NWP ensembles grows, efficient methods of displaying and analyzing these data will become even more necessary. Specifically, we intend to investigate and develop additional methods of visualizing vertical profiles from ensembles. Doing so will also require improving the speed of the graphical and numerical routines. A possible avenue is to implement the Davies-Jones (2008) method of calculating pseudoadiabats in parcel lifting routines, which would facilitate both speed and accuracy improvements for all users. Finally, much of the background development of SHARP has focused on diagnosing the potential for deep, moist convection and associated CONUS severe thunderstorm hazards, which is a large part of SPC’s mission. With the help of the community, we hope to expand SHARPpy’s capabilities with respect to other geographical locations and weather types (e.g., winter weather and heavy rainfall forecasting).
SHARPpy is available for download online (https://github.com/sharppy/SHARPpy).
The authors thank the Cooperative Institute for Mesoscale Meteorological Studies (CIMMS) for providing funding for the publication fees. The authors appreciate the contributions by Ryan Jewell (SPC) and Bryan Smith (SPC) that helped support and improve the SHARPpy program. Matthew Bunkers (NWS Rapid City) donated the PW climatology database for each U.S. radiosonde site for use in SHARPpy. Thanks also go to Dave Turner (NOAA/ESRL) for offering suggestions that improved the paper and assistance in image formatting. We also wish to thank the OU School of Meteorology for graciously offering us web server and computer processing space in order to host various sounding data sources for the SHARPpy users. Thanks also go to the three anonymous reviewers, as their comments improved both the clarity and depth of this article.
A supplement to this article is available online (10.1175/BAMS-D-15-00309.2)