The increasing use of mobile phones (MPs) equipped with digital cameras and the ability to post images and information to the Internet in real time has significantly improved the ability to report events almost instantaneously. From the perspective of weather forecasters responsible for issuing severe weather warnings, the old adage holds that a picture is indeed worth a thousand words; a single digital image conveys significantly more information than a simple web-submitted text or phone-relayed report. Timely, quality-controlled, and value-added photography allows the forecaster to ascertain the validity and quality of storm reports. The posting of geolocated, time-stamped storm report photographs utilizing an MP application to U.S. National Weather Service (NWS) Weather Forecast Office (WFO) social media pages has generated recent positive feedback from forecasters. This study establishes the conceptual framework, architectural design, and pathway toward implementation of a formalized photo report (PR) system composed of 1) an MP application, 2) a processing and distribution system, and 3) the Advanced Weather Interactive Processing System II (AWIPS II) data plug-in software. The requirements and anticipated appearance of such a PR system are presented, along with considerations for possible additional features and applications that extend the utility of the system beyond the realm of severe weather applications.
The advent and rapid advancement over the past decade of mobile phone (MP) digital camera technology, coupled with the ability to post images to the Internet and various social media venues (e.g., Twitter, Facebook, etc.) in near–real time, has transformed the way the world receives and processes information. The U.S. National Weather Service (NWS) has recently added social media usage to its operational protocol (NWS 2015) as a means to communicate weather information to the public, as well as monitor weather activity (severe weather in particular) through public posts and images. While severe weather photography contains inherently more information than visual accounts relayed via plain text messaging or phone-in reports, fielding such information from social media can be challenging and time-consuming to forecasters, as they navigate repetitive, irrelevant, or even dubious, conflicting information.
This paper presents a photo report (PR) system concept that bypasses social media completely, allowing registered reporters to submit severe weather PRs through a direct conduit to NWS forecasters. Section 2 discusses recent personal social media interactions with NWS forecasters related to severe weather photos, along with feedback on the newly proposed direct PR system concept. Section 3 describes the PR system concept, including the MP PR application (section 3a), the processing and distribution servers (section 3b), the Advanced Weather Interactive Processing System (AWIPS II; Raytheon 2014) ingest and display software implementation and features (section 3c), performance (section 3d), and benefit considerations (section 3e). The paper concludes with a discussion of additional features and applications (section 4).
2. Background and related efforts
The PR system concept originated through informal social media interactions with NWS Weather Forecast Office (WFO) forecasters while posting personal real-time storm photos during the spring of 2014. The first interaction was with a NWS Goodland, Kansas, forecaster (Scott Lockhart) concerning a tornadic supercell that occurred near Yuma, Colorado, on 7 May 2014. It became clear from this initial interaction (see https://www.facebook.com/scott.longmore and https://twitter.com/scottlongmore) that accurate time and location information should be included with photo posts (Fig. 1a). To demonstrate the added usefulness of geo- and time- reference severe weather photos (Figs. 1b–d), the commercial MP photo application Context Camera (http://cascodelabs.com/#contextcamera), which includes time, latitude, longitude, and view direction information in the image and metadata, was selected and used for all personal NWS WFO social media interactions. Forecaster feedback on WFO-tagged (e.g., @NWSGoodland) severe weather photos (Figs. 1b–d) has been positive, with specific interest in geo- and time-referenced images.
While personally interacting with forecasters, several issues became apparent when using social media feeds to relay severe weather photos. If one’s photo is not tagged with the appropriate NWS WFO tag or feed tag (e.g., #cowx), then the chances of the photo being seen by a forecaster are decreased. Even with the appropriate tag, a potentially helpful PR could be overlooked amidst the numerous posts during a severe weather event. Alternately, many posts are irrelevant, repetitive, and possibly misleading. Acknowledgment of these issues has led to the idea of the direct PR submission concept, designed to circumvent posting to social media altogether. Instead, PRs would be submitted to processing and distribution servers, allowing NWS WFOs to ingest and display them within AWIPS II. The ability of AWIPS II to concurrently ingest and display environmental data for weather forecasting and warning issuance exceeds the capability of social media venues.
The Cooperative Institute for Research in the Atmosphere (CIRA) developed an initial PR system concept white paper that was circulated to WFO forecasters and Science Operation Officers in Denver, Colorado; Cheyenne and Riverton, Wyoming; Goodland, Kansas; Rapid City, Aberdeen, and Sioux Falls, South Dakota; Omaha, Nebraska; and Glasgow, Montana. Feedback ranged from generally positive to enthusiastic with detailed implementation questions such as the following: Will spotter network input be included? How timely will the PRs be? How will PRs time match with radar, satellite, and observations? How will AWIPS display performance be affected during high PR-volume severe weather events? The scope of detailed feedback suggests that there exists a genuine forecaster interest for such a PR system as an operational decision aid.
Previously utilized visual, photographic, and crowdsourced weather information in forecasting, decision-making, and research further supports the development of the PR system concept. Live mobile webcams have proven useful to severe weather forecasting (Pietrycha et al. 2009). During the 22 May 2008 Windsor, Colorado, tornado, visual confirmation was considered important to decision-makers prior to taking action (Schumacher et al. 2010). Storm photogrammetry has been widely used in severe storms research (Wakimoto et al. 2011; Wakimoto and Martner 1992; Zehnder et al. 2007; Rasmussen et al. 2003; Holle 1988). The NWS currently utilizes a web-based text storm report system (NWS 2014). Crowdsourcing weather reports have shown success with the Mobile Precipitation Identification Near the Ground (mPING) project (Elmore et al. 2014). The primary focus of mPING is to validate Weather Surveillance Radar-1988 Doppler (WSR-88D) dual-polarization research algorithms through anonymously submitted crowdsourced text reports detailing the observed precipitation type. Disaster incident photo reporting has recently been added to the Federal Emergency Management Agency’s (FEMA) disaster recovery mobile application to aid emergency managers (Rockwell 2013).
The principal goal of the proposed PR system is to provide near-real-time photo reports during and possibly before a severe weather event (e.g., rotating wall cloud) from registered users to NWS severe weather forecasting operations. The PR system concept also provides forecasters with reasonable discretion to validate the reports and supplements them with conventional severe weather observations (radar, satellite imagery, etc.). Having established the utility of severe weather photos in social media, the interest by NWS forecasters, and the success of visual, photographic, and crowdsourced weather information, the technical implementation and specific features of the PR system concept are now presented.
The PR system concept is composed of three core components: 1) a newly developed MP PR application (Fig. 2, top), 2) the processing and distribution servers (Fig. 2, middle), and 3) the AWIPS II ingest and display clients (Fig. 2, bottom). The PR application allows users to capture and submit photos along with the severe weather type (tornado, hail, etc.), time, location, viewing azimuth, and reporter information. This information is submitted by the PR mobile application to NWS external processing servers, where they are authenticated against the registered users database and subsequently converted into an AWIPS II–compatible data format. The formatted PRs are retrievable by NWS internal servers for distribution to participating WFOs. The WFO AWIPS II servers ingest and store the PRs locally, where they can be requested and displayed by the display clients. The PR system software, processing, and distribution servers would be developed, hosted, and maintained by CIRA in collaboration with the NWS through multiyear funding proposals, and potentially could be transitioned to NWS operations in the future. The implementation and functionality of these components are described in the subsections to follow.
a. Mobile phone photo report application
The intent of the PR application is to allow trained weather spotters, local authorities, and weather enthusiasts to quickly capture and convey to forecasters timely information on rapidly evolving severe weather through a user-friendly MP application interface. Submission of PRs though a standardized interface would create a consistent, quality-controlled platform for relaying severe weather information that is nonintrusive to both NWS forecasters and photo reporters.
b. Processing and distribution
The function of PR processing and distribution servers is to verify PRs through the registered user databases, convert them to an AWIPS II data format, and distribute the reformatted PRs to the appropriate WFOs. Because of NWS security requirements, the PR processing and distribution servers consist of two sets of servers (Fig. 2, middle). The external NWS network processing servers are within a demilitarized zone subnetwork and receive, validate, convert, and stage the AWIPS II–formatted PRs (Fig. 5, left box). The internal distribution servers are behind the NWS firewall; and they continuously query the external servers, retrieve via file transfer protocol secure sockets layer (FTPS), and distribute via the Unidata Local Data Manager (LDM; Unidata 2014) the staged PRs (Fig. 5, right shaded box). Multiple external and internal servers allow for PR high volume during severe weather events, as well as redundancy capabilities in the event of a hardware failure.
c. AWIPS II ingest, storage, and display
AWIPS II is capable of accommodating new datasets by developing Environmental Data Exchange (EDEX), data transport (Common), and visualization (VIZ) software plug-ins for the Display 2 Dimensions (D2D) perspective. Such versatility is ideal for adding PRs to supplement the severe weather forecasting decision-making process.
Under the PR concept, WFO EDEX servers retrieve any County Warning Area (CWA) PRs through the local AWIPS II LDM. These AWIPS II–formatted PRs are ingested and stored in the internal Hierarchal Data Format, version 5 (HDF5) utilizing the developed PR EDEX and Common plug-ins. The optimal PR AWIPS II data format has low NWS network bandwidth size and utilizes commonly ingested EDEX formats and plug-in frameworks. Possible acceptable input formats include Unidata’s Network Common Data Form (netCDF), HDF5, or the Joint Photographic Experts Group (JPEG) with Exchangeable Image File Format. (EXIF).
Once AWIPS II PRs are ingested and stored through the EDEX and Common data plug-ins, they become available for display via the PR VIZ plug-in. A mock-up of the initial PR display concept is shown in Fig. 6. Similar to the AWIPS II surface observations display, PRs are positioned according to their latitude and longitude on the georeferenced display pane. Each PR is indicated with an arrow pointing toward the view direction of the captured photo, along with the report time. The display contains the last hour of PRs, updating every minute as new PRs are received. Individual PRs fade as they age. The past hour’s PRs match in time with each frame of complementary products, such as radar and satellite imagery loops, so that some past continuity is maintained. Hovering over an individual PR displays a thumbnail image in the main pane next to the PR with the location, time, severe weather type, and user information depicted below the thumbnail image.
In addition to the main display pane, a new tab containing a sliding PR viewer tray (Fig. 6, right) opens next to the main pane. Selecting a PR within the main pane scrolls automatically to the corresponding full-resolution photo at the center of the sliding PR viewer tray. Alternatively, time-ordered photos could be viewed by sliding the scroll bar within the tray, which would also highlight the corresponding PR in the main pane. Subsets of PRs in the main pane and photo tray are selected by a pull-down filter and subfilter menus at the top of the PR viewer tray. Possible filters include severe weather type, user name, storm cell identification, a PR proximity range, etc.
d. Performance considerations
To expedite the PR process to an ideal 1–2-min, image size, network bandwidth, server performance, and other possible information bottlenecks must be considered. To avoid saturating the processing and distribution servers, internal NWS network, and WFO AWIPS II systems, initial PR data would be set to a low bandwidth size while still being able to interpret image features within the mobile application for system prototype and testbeds/proving grounds1 testing. Network bandwidth and server processing metrics would be collected and analyzed during these testing exercises to determine the optimal PR size for different scenarios, including varying number of PRs per unit of time, number of processing and distribution servers, etc. Utilizing enterprise cloud server services for the external PR processing would allow for scalability as user registration and PR submission increased. As network bandwidth and server processing capabilities increases, PR size could increase as well through MP application updates and configuration updates in the server processing and distribution software system.
e. Benefit considerations
The PR system concept’s greatest benefit would be delivering near-real-time, time-assured, geolocated, high-quality severe weather and possibly other relevant weather images from in-field registered reporters. These high-quality PRs would supplement and possibly validate non-geolocated or non-time-assured weather photos from other sources while complementing traditional severe weather datasets within AWIPS II. The PR system would also add a framework for incorporating PRs from other possible sources. The system would have the greatest benefit in low-population regions, such as the U.S. high plains, where reports mostly come from storm spotters and enthusiasts. In high-population regions, such as the U.S. East Coast, the PR system would add a high-quality visual dataset within the AWIPS II framework that would supplement the abundant images from other sources (e.g., commercial media, social media, e-mail, or MP-texted images from WFO trusted contacts). The PR system registration requirement may initially limit the number of reporters, but the geolocated, time-assured PRs would be a high-quality control and validation mechanism for weather imagery from these other sources. The adoption of the PR MP application could be facilitated by easing the registration restrictions by transitioning to authorization mechanisms provided by social media companies (e.g., Facebook, Twitter, Google). This would still ensure some accountability of submitted PRs but utilize a more publically familiar authorization mechanism. Another possibility for adoption would be to establish a framework in which commercial mobile applications, as well as other non-MP weather image sources (social media, webcams, etc.), could be transmitted or retrieved by the processing servers.
The initial PR concept implementation and features described above will require experimentation to optimize processing, distribution, ingest, display, and ease of use for forecasters. Here, evaluation and active feedback from forecasters engaged in the process (e.g., via testbed or “proving ground” demonstration program) will be necessary to improve the PR system, to ensure interoperability with other AWIPS II data, and to identify the need for new features. Given the initial feedback from weather forecasters, the fully implemented PR mobile application, processing and distribution, and AWIPS II display system would serve as a valuable supplemental tool for severe weather forecasting.
4. Future work
Once an initial version of the PR system is implemented, tested, approved, and deployed provisionally within the NWS for evaluation, several future capabilities could be added that would encourage severe weather PR submission and enhance severe weather forecasting and research. A PR web server, including an interactive geographic display of real-time and past PRs, is the first expanded feature that could be developed. The PR web server would serve two purposes: i) recognizing reporter photo contributions and ii) creating a searchable, categorized photo dataset for forecasters and researchers. Possible MP application upgrades could include optional concurrent social media posting that would recognize reporter photo contributions. Short video capture could be explored as NWS network and AWIPS II hardware is upgraded. The use of crowdsourcing within a PR system for NWS severe weather forecasting operations could significantly enhance and improve NWS-issued warnings by supplementing existing AWIPS II data with near-real-time PRs of severe weather.
The PR system concept could also be applied to other government agencies and private sector needs. State departments of transportation could utilize the system for road conditions and damage. Federal agencies such as the U.S. Forest Service, the U.S. Geological Survey, and the Environmental Protection Agency could benefit from PRs for forest fires, earthquake, possible pollution violations, etc. Centralized, categorized, and searchable PRs available through an interactive web portal, as in the one described above, could be helpful to private industry, such as insurance adjusters for damage claims and the media for news articles.
The authors thank NWS forecasters Scott Lockhart and Tanja Fransen for their input on the PR system concept and for the inclusion of their social media interactions. The authors would also like to thank the manuscript reviewers for their comments, which improved this paper. The views, opinions, and findings contained in this report are those of the authors and should not be construed as an official National Oceanic and Atmospheric Administration or U.S. government position, policy, or decision.
Testbeds and proving grounds facilitate the orderly transition of research capabilities to operations through development testing in testbeds, and predeployment testing and operational readiness evaluations in operational proving grounds. The NOAA testbeds and proving grounds can be accessed at http://www.testbeds.noaa.gov/.