1. Introduction
Social media platforms (e.g., Facebook, Twitter) provide governmental organizations like the National Weather Service the opportunity to communicate directly with stakeholders. Broadly speaking, such communication can be thought of as taking place in two kinds of contexts: threat periods, when a specific hazard event (e.g., severe storm, flood, etc.) is affecting the population served by the organization; and nonthreat periods, when no specific hazard is active (though recovery, preparedness, or mitigation activities may be taking place). Long-term social media communication that spans such weather cycles is often referred to as engagement, as the goal of this communication is to build trust and create relationships (Lovejoy and Saxton 2012) rather than simply to convey information regarding a specific event. For science-based organizations like the National Weather Service, building trust and creating relationships may be particularly important as these organizations are mandated to warn people of hazardous events and encourage them to take protective action. When a hazardous event does occur, engagement may affect the effectiveness of event-specific communication. For instance, preexisting trust relationships influence how people respond to messages, in particular the extent to which they follow message recommendations (Wachinger et al. 2013). Although much has been written about using social media for stakeholder engagement, especially in the public relations literature (Dhanesh 2017), most research to date has been focused on attempting to catalog and create typologies of messages employed (Olson et al. 2019; Lee and VanDyke 2015; Lovejoy and Saxton 2012). This work has demonstrated that consistent strategies—implicit or explicit—are frequently employed by communicators seeking to engage their audiences, raising the question of how well such strategies work in practice. Evaluating the degree to which engagement strategies result in realized stakeholder engagement using behavioral measures from social media has thus emerged as a new and largely untapped area for research (for a notable exception, see Saxton and Waters 2014).
When individuals interact with messages on social media, individual platforms facilitate a variety of interactions, or impressions, that indicate engagement with a message. Users can typically “like” messages, comment on messages, and pass messages on using platform-specific buttons like “retweeting” on Twitter and “sharing” on Facebook. Of these behavioral actions, message passing has attracted the most scholarly and practical interest (Suh et al. 2010). This behavior not only indicates attention to messages, suggesting that they are salient to the audience, it also spreads messages beyond their original group of recipients, increasing exposure. Message passing has been examined in the context of daily communication by nonprofit organizations (Bail 2016; Saxton and Waters 2014) and in the context of official communication during hazardous events (Sutton et al. 2015; Vos et al. 2018). However, to our knowledge, no research has analyzed how organizational engagement message practices influence message passing for science-based government organizations like the National Weather Service and how that differs over time, particularly between periods of threat and nonthreat.
Although research has established that people turn to social media for information when disaster threatens (Fraustino et al. 2012) and that certain message strategies are more effective for initiating retweeting during the threat period (Sutton et al. 2015; Vos et al. 2018), little research that we know of has examined message passing during nonthreat periods for science-based organizations. In other words, researchers have yet to examine how the engagement strategies that organizations like the National Weather Service use on social media influence message passing during nonthreat periods and whether this communication pattern differs during threat periods. In this paper, we build on existing research that examines message passing in the context of threat (Sutton et al. 2015; Vos et al. 2018) and message passing in the context of everyday communication by nonprofit organizations (Bail 2016; Saxton and Waters 2014) in order to analyze message passing by a science-based organization during threat and nonthreat periods. In particular, we draw on the results of a previous content analysis of media engagement communication on Twitter by National Weather Service forecasting offices (Olson et al. 2019) in order to model how the engagement strategies used by these offices effect message passing during threat and nonthreat periods.
a. Engagement on social media
Although engagement has long been identified as a goal of organizational communication on social media, both scholars and practitioners have struggled to define what engagement on social media means (Dhanesh 2017). Early research focused on the potential for social media to facilitate engagement using two-way or dialogic communication (Taylor and Kent 2014). This research tended to criticize organizations that did not direct messages to individual users or mention users in their messages as these types of dialogic interactions were considered superior to one-way information dissemination (e.g., Lovejoy et al. 2012). However, more recent research conceptualizes social media engagement as a continuum, where interactive communication is but one aspect of engagement (Dhanesh 2017). Engagement is the practice of carrying on long-term communication with an audience with the purpose of building relationships, establishing trust, and facilitating mutual understanding between an organization and its publics. Under this conceptualization, social media engagement strategies are specific approaches to carrying out long-term communication (i.e., communication that extends beyond any single interaction or event) and includes all communicative acts made by an organization, and thus it can be operationalized as the interactive elements facilitated by the platform [i.e., communicating directly to a user, using a hashtag (#), or including a hyperlink] as well as the content of the messages sent. This conceptualization recognizes the sharing of content as an engagement strategy, acknowledging the experimental research that suggests that individuals respond to useful social media content from organizations in a way that builds a relationship (Watkins 2017); for a more in-depth discussion of engagement on social media, see Olson et al. (2019) and Dhanesh (2017).
An examination of social media communication on Twitter by 12 National Weather Service Weather Forecast Offices (WFOs) found that these offices systematically employed both message content and interactive elements to engage stakeholders (Olson et al. 2019). Content posted to Twitter by the WFOs included general information (weather forecasts, current conditions, and weather facts), information related to notable weather events, referred to as “weather products” (e.g., watch, warning, advisory), and community-building information. Notably, only a small proportion of messages included recommended actions (e.g., messages that instructed people to take an action, such as share pictures, attend a spotter class, or take protective action). Interactive features used by the offices included replying directly to users or mentioning them in the text of the tweet (using @username); retweeting messages sent by other users; attaching an image, GIF, or video file to a message; using a keyword hashtag (e.g., #wx, meaning weather) to indicate message topic; including a hyperlink to direct users to additional information; indicating emphasis with an exclamation point; and posing questions.
Results from the study indicated that offices used different practices during threat periods (as indicated by active warning or watch periods) than they did in nonthreat periods (Olson et al. 2019). During nonthreat periods, messages included community building, weather facts, and action content more often. Messages during this time period were also more likely to include hyperlinks, questions, exclamation points, images, and @mentions. During threat periods, the inclusion of nowcast and weather product information increased as did the use of direct replies. However, this study did not examine whether these strategies influenced user engagement with messages, as measured by message passing, and whether user engagement varied between threat and nonthreat periods.
b. Engagement strategies and message passing
Most of the research examining the relationship between messages and message passing has been conducted in the context of hazardous events, when public attention focuses on an issue (e.g., Sutton et al. 2015), likely contributing to increased message passing. Based on the findings from seven case studies, Vos et al. (2018) proposed the Risk Communication on Social Media (RCSM) framework, which draws on theories of diffusion that explain how new information, skills, or production methods, spread across populations over time and how quickly they are adopted (Rogers 2003; Spitzberg 2014). The RCSM identifies several key factors that influence message passing. These factors include the design of the message itself (content and interactive features), the characteristics of the account sending the message, the network of followers associated with the account, and the salience of the message topic at the time the message was sent. Of the four factors that influence message passing, two factors can be easily manipulated by communicators at organizations like the National Weather Service: the characteristics of posted messages, and the time at which messages are posted. For instance, messages that are timed such that they coincide with, or are proximate to, salient events to which they refer (e.g., a severe storm) may receive more attention than those posted at other times, and messages posted during periods of high social media use (e.g., midday, midweek) may be passed on more readily than those posted during periods of low social media use (e.g., very early morning on Sunday in the United States). Likewise, appropriate choices of message style and content can lead to increased salience and greater retransmission probability, while other choices may result in messages that receive less attention.
The results from these previous studies suggest that certain types of content result in more public attention to messages during imminent threat periods, that is, when a threat is urgent and hazardous, as measured by message passing. In particular, messages that contain information about the severity of a threat (i.e., hazard information) and messages that contain action information or information that instructs people about how to respond to a threat (i.e., guidance or advisory messages) were passed on at a higher rate than messages that did not contain this information. In terms of message features that facilitate interaction, described as “microstructure” (Sutton et al. 2014), messages that contain media files are passed on more often (Vos et al. 2018). Using a #keyword hashtag also usually increases message passing, although not always (Vos et al. 2018). However, some microstructure features such as @mentioning another user or @replying directly to message signify a narrowing of the audience for the message, consistently decreasing message passing (Sutton et al. 2015). This effect is an example of the classic observation that information seen as being more broadly relevant to others (e.g., as opposed to being part of a narrower conversation) is more likely to be passed on (Allport and Postman 1947).
The research on message passing under normal conditions by nonprofit organizations using other social media platforms suggest a similar pattern (Bail 2016; Saxton and Waters 2014), in that content and structural elements influence message passing. For example, one study that examined message passing using Facebook messages posted by large nonprofits (Saxton and Waters 2014) found that certain types of content (fundraising and event messages) decreased message passing compared to informational messages. Another study, which examined message passing on Facebook in the context of autism organizations, found that emotional language and unique word choices increased message passing (Bail 2016). Both studies found that including an image or visual (an audiovisual in the autism study and a picture in the large nonprofit study) significantly increased message passing.
This previous research suggests that message microstructure and content choices influence message passing in both threat and nonthreat periods. However, it is unclear how specific engagement strategies used by the National Weather Service influence message passing during normal, nonthreat conditions and whether this pattern changes when bad weather is approaching. In this paper, we seek to answer the following two research questions:
RQ1: How do engagement strategies influence message passing during nonthreat periods?
RQ2: Does the effectiveness of these strategies, as measured by message passing, change during threat periods?
2. Methods
To model how message content and microstructure influence message passing during threat and nonthreat periods, we construct a model from data analyzed as part of a previous study on media engagement on Twitter, described above (Olson et al. 2019). Content analysis was conducted on Twitter messages sent by the accounts of 12 NWS WFOs to identify engagement practices used in threat and nonthreat periods. WFOs were selected using a stratified sample that varied by geographic region, population served (urban or rural), and weather phenomena common to those areas and time of year. While not all examples of extreme weather (e.g., hurricane or blizzard) occurred during our study period, several regularly occurring seasonal threats (including flooding and severe thunderstorms) are included and represent widely experienced weather perturbations. The selected offices were located in Anchorage, Alaska; Boston, Massachusetts; Fort Worth, Texas; Grand Forks, North Dakota; Honolulu, Hawaii; Jackson, Kentucky; Miami, Florida; Norman, Oklahoma; Pueblo, Colorado; Sacramento, California; Seattle, Washington; Charleston, South Carolina.
A total of 10 705 tweets were collected from a three-month period (April–June 2016) using the Twitter Representational State Transfer (REST) API, constituting a census of messages posted by these accounts during the period. For this analysis, we excluded any messages that were retweets or messages that had originated with another account and had been passed on by the account of interest, leaving 9861 messages for analysis. On average, each account produced 821 messages during the study period, with @NWSHonolulu producing the least (62) and @NWSNorman producing the most (2053). Our analysis (as described below) examines the relationship of a variety of message and contextual features to the rate at which these messages are retweeted, while controlling for account-level and other contextual differences.
a. Content analysis
A combination of manual and automated coding strategies were used for the message content analysis (for a more in-depth discussion, see Olson et al. 2019). Messages were first coded for whether they were sent during a threat period. Messages sent during a threat period were conceptualized as those disseminated during a time period when the WFO sending the message was also under an active advisory, watch, or warning. Threat time periods were identified by drawing on the Iowa Environmental Mesonet (IEM) NWS products database (Iowa State University 2019).
1) Manual coding
Messages were manually coded for engagement strategies related to content (see Table 1; Olson et al. 2019). These codes drew on previous research on engagement in the public relations literature (Lovejoy and Saxton 2012; Lovejoy et al. 2012) and research on message passing in the context of threat (Sutton et al. 2015; Vos et al. 2018). The messages were coded by two coders. Intercoder reliability was assessed at the beginning and midpoint of the process, using a subset of the data; reliability was high in both cases (Krippendorff’s alpha ≥ 0.8; Krippendorff 2011). Operational definitions for codes, reliability measures, descriptive information, and examples are provided in Table 1.
Operational definitions, codes, and reliability measures for manual content coding. ICR is intercoder reliability.
2) Automated coding
Computer-automated methods were used to automatically annotate microstructural features of messages that facilitate interaction. Only features that could be automatically and unambiguously identified were coded in this manner. These included the presence of media attachments (pictures, videos, or gifs), hyperlinks, mentions of other users (@username) or direct replies to other users (messages that begin @username), and messages that included a hashtag (#). Sentence structure was also automatically coded for the presence of a question mark (?) or exclamation point (!).
3) Hashtag categorization
Past work in this area has focused on the use of hashtags in official communication during and around the time of specific, high-impact hazard events (Starbird and Palen 2012; Sutton et al. 2015; Vos et al. 2018). In these cases the use of a hashtag in a message, which was found to be one of the largest contributors to message retransmission (Suh et al. 2010) and was motivated by the need to create a virtual “channel” through which members of the public could obtain current information regarding a fast-changing event of substantial general interest. In the present context, however, most communication does not involve response to named, high-impact events, and inspection of the message corpus revealed a much wider diversity of uses for hashtags. To account for the possibility that different types of hashtags have different effects on message retransmission, we conducted additional content analyses of hashtags employing a thematic categorization scheme to determine variation in hashtag use over time. For the hashtag categories, two researchers separately coded the 501 unique hashtags in the dataset for one of seven categories; any disagreements that emerged in this process were resolved through discussion resulting in mutual agreement. The categories included location (#northeast), ordinary/everyday weather descriptors (#rain), campaigns (#floodsafety), organization (#NWS), time (#summer), extreme weather (#hurricane), and named events (#Colin). Hashtags that did not fit into one of these categories were coded as “miscellaneous.” Once unique hashtags were identified, computer-assisted methods were used to automatically identify hashtags that appeared in each message. Hashtag categories are mutually exclusive (i.e., each hashtag was coded for only one category); however, messages could contain more than one hashtag and, thus, could be coded as containing more than one hashtag type.
Operational definitions for hashtag codes, descriptive information, and examples are provided in Table 2.
Hashtag coding features.
b. Data analysis
The outcome variable, message passing, was operationalized as the number of times a message was retweeted or passed on, with the core question of our analyses being what engagement content and microstructure predict message passing. As is typical of Twitter messages, many NWS messages were not retweeted, with most retweeted messages being passed on a very small number of times. However, some messages are retweeted many times, resulting in a highly skewed retweet distribution [mean = 7.6, median = 4, mode = 0 (22.7% of all messages), standard deviation = 16, skewness = 18.9, kurtosis = 824.84]. To account for this, we conducted our analysis using a mixed-effects negative binomial model (Hilbe 2011); preliminary analyses showed that additional zero-inflation was not present (net of what the negative binomial model already produces), and hence we did not employ it here. All analyses were conducted in R with the MASS and glmmADMB packages (Skaug et al. 2010; R Core Team 2016; Venebles and Ripley 2002).
To control for contextual factors unrelated to message content and microstructure, we included the month, the day of the week, and the hour (in local time) a message was sent, using random effects (Venebles and Ripley 2002). We also controlled for the account sending the message using per-account random effects, and the number of followers each account had at the time the message was sent using fixed effects (Venebles and Ripley 2002). (We note that, in particular, these effects control for any account-specific tendency for messages to be retweeted at higher or lower rates, net of other message features.) Fixed effects were used for all other variables. To compare the effect of message content and microstructure on engagement during threat and nonthreat time periods, we included an interaction term for each message-related variable. Variables unrelated to the message itself (e.g., account, time of day) were held constant.
3. Results
Our model (Table 3) indicates that similar message design features influence message passing in both threat and nonthreat period and, in general, they do so in similar ways. In addition, as predicted by the Risk Communication on Social Media framework (Vos et al. 2018), message passing is also influenced by the network of followers attached to the account, the organization sending the message, and the timing of the message. In terms of the message content and interaction variables, the model shows similar results for both threat and nonthreat periods. However, in some cases, the difference between threat and nonthreat periods is statistically significant. To interpret these findings, we provide a frequency interpretation based on the exponentiated beta coefficients (Hilbe 2011).
Negative binomial regression model. Observations: 9861; Akaike information criterion (AIC): 48 378.6; log-likelihood: −24 140.3; dispersion parameter: 2.01; standard error: 0.039; *P < 0.05, **P < 0.01, ***P < 0.001; NS: not significant. Note: This model controls for the account the message originated from, the hour, day of week, and month that the message was sent with random effects variables not listed here.
In our analyses, the main effect for each message feature indicates the impact of that feature on the log of the expected retweet rate in the nonthreat period. Interactions of “threat” with such features indicate the difference in the corresponding effect for threat periods, net of the nonthreat baseline; a significant interaction hence indicates a statistically significant tendency for the influence of the corresponding message feature to be different in threat versus nonthreat periods. In our discussion (below) and in Figs. 1–3, we combine these to show the net effect of each message feature on retweet rates in the threat and nonthreat periods (respectively).
Message content features are shown with their corresponding model estimates, which here represent the expected log count of message passing. Green bars indicate features that positively impact message passing while red bars indicate features that negatively impact message passing. Solid colored bars represent the feature in the nonthreat period while cross-hatched bars represent features during a threat period. Brackets indicate a feature that differs significantly between nonthreat and threat periods, with ***p ≤ 0.001, **p ≤ 0.01, and *p ≤ 0.05.
Citation: Weather, Climate, and Society 11, 4; 10.1175/WCAS-D-19-0021.1
As in Fig. 1, but model estimates for structural features.
Citation: Weather, Climate, and Society 11, 4; 10.1175/WCAS-D-19-0021.1
As in Fig. 1, but model estimates for hashtag features.
Citation: Weather, Climate, and Society 11, 4; 10.1175/WCAS-D-19-0021.1
a. Message content
Two types of engagement content (weather facts and instruction/action) increased message passing in both nonthreat and threat periods (see Fig. 1). Including weather facts in a message significantly increased message passing regardless of whether the message was posted during a threat or nonthreat period (52% and 34%, respectively). Similarly, the inclusion of instruction/action content and information significantly increased message passing (25% in threat periods and 18% in nonthreat periods). In the both cases, the difference between message passing in threat and nonthreat periods was not significant.
Two types of engagement content have significant effects on decreased message passing during nonthreat periods but did not have a significant effect in threat periods. The inclusion of forecast information decreased message passing by 10% in nonthreat periods, holding all other variables constant. Nowcast information or updates on ongoing weather-related events, decreased message passing by 14% in nonthreat periods. In both cases, the difference between message passing in threat and nonthreat periods was not significant.
Two engagement content variables demonstrated significant differences in message passing during threat versus nonthreat periods. Community-building content significantly increased message passing by 21% in nonthreat periods; however, the effect of this variable decreased during threat periods to a nonsignificant effect, and the difference between the two time periods was significant. Similarly, message passing for weather product information significantly decreased in threat periods, compared to the baseline of nonthreat periods. During the nonthreat period, weather product information did not have a significant effect on the message passing; however, during threat periods, weather product information significantly decreased message passing by 32%.
b. Message structure
Structural features influenced message passing in both threat and nonthreat periods. Two of these features (including a visual media element, a microstructural feature of Twitter, and sentence structure, using an exclamation point) increased message passing (see Fig. 2) in both threat and nonthreat periods. Visual media elements like an image or video significantly increased message passing (80% in threat and 65% in nonthreat); the difference between the two periods was not significant. Using an exclamation point (!) also increased message passing in both threat and nonthreat periods (64% and 17%, respectively), with the threat period effect being significantly higher than the effect on messages not posted in threat periods.
Four message structure features (one sentence structure and three microstructure) either decreased message passing or did not significantly influence message passing. Using a question mark (?) decreased message passing by 6% in nonthreat periods; in threat periods, this message strategy did not significantly influence message passing. The difference between the two periods was not significant. Directly responding to another user (@Reply) substantially decreased message passing in both threat (96%) and nonthreat (98%) periods; in addition, the difference between the two periods was significant. Mentioning another user in the text of a message (@Mention) also decreased message passing. During nonthreat periods, this strategy decreased message passing by 31%; in threat periods, this strategy decreased message passing by 53%; the difference between the two periods is statistically significant. Finally, using hyperlinks significantly decreased message passing in nonthreat periods by 33%; however, during threat periods, the variable did not significantly influence message passing. This change in message passing during threat periods was significantly different from nonthreat periods.
c. #Hashtag features
Most types of #hashtags do not significantly influence message passing in threat or nonthreat periods and the difference between periods is minimal (i.e., nonsignificant; see Fig. 3). The clear exception to this observation are hashtags related to named events, like #Bonnie. In both threat and nonthreat periods, named event hashtags increased message passing (110% and 99%, respectively); the difference between the two periods is nonsignificant.
The two hashtag types that show significant differences in their influence in threat and nonthreat time periods are campaign and location hashtags. Campaign hashtags, such as #tornadosafety, do not significantly influence message passing in nonthreat periods but do (negatively) influence message passing during threat periods. During threat periods, using campaign hashtags decreases message passing by 33.5%. Location hashtags do not significantly influence message passing in threat or nonthreat periods. However, the differences in influence between time periods are significant. This is likely because the direction of the nonsignificant findings is negative for nonthreat time periods and positive for threat periods; put another way, our data suggest that the impact of location hashtags is higher in threat versus nonthreat periods, but we lack the statistical power needed, that is, the number of observations necessary to reliably determine whether it is the higher of two positive effects, the less negative of two negative effects, or whether the effects go in different directions (as their point estimates would suggest).
Two hashtag types had a significant influence during one time period, no influence during the other time period, and did not exhibit a significant difference between time periods. During nonthreat time periods, including a time-related hashtag (e.g., #memorialday2016) increased message passing by 21%. However, the influence during threat time periods was nonsignificant, and the difference between the two time periods was nonsignificant. Hashtags related to ordinary weather significantly decreased message passing during threat time periods by 27% but did not have a significant influence during nonthreat time periods. The difference between the two time periods is not significant.
4. Discussion
Organizational engagement on social media can be fostered by the type of information included in the message and the microstructural features that facilitate message interaction. Previous research demonstrated that engagement practices used by WFOs differ based upon the time period during which messages are sent. In this research, we identify which practices lead to the behavioral outcome of message passing in threat and nonthreat periods. We find that some message content types—weather facts and action messages—have consistent effects of increasing message passing in both time periods. The behavioral response of message passing for these two content categories, especially during the threat period, suggests that audiences are drawn to and engage with content that is not only instructive (Sellnow-Richmond et al. 2018) but also helps to normalize or emphasize the threat by comparing it with previous experience (Keller et al. 2006). Importantly, weather facts may provide audiences with points of comparison, anchoring them to memorable events that can help them to assess their current level of risk (Finucane et al. 2000).
We also find that forecast and nowcast messages have consistent effects on decreasing message passing in both time periods. While providing this kind of daily weather information is key to the mission of WFOs, this does not necessarily lead to enhanced retransmission online. One potential explanation is that these messages may be used primarily by readers to satisfy their own curiosity or for personal planning purposes (e.g., deciding what to wear, or whether to bring an umbrella to work) and such individualistic use cases neither require nor motivate information sharing. While perhaps surprising that this would continue to be true during threat periods, scholars and weather forecasters must remember that the “typical” threat is still a fairly quotidian, or ordinary, kind of event with which the affected community is highly experienced (e.g., a thunderstorm). As a result, audience members may have little of the “cognitive unclarity” (Allport and Postman 1947) that would motivate extensive message passing. This situation is quite different from the disaster-based settings that have been the focus of much prior work in this area (Starbird and Palen 2012; Sutton et al. 2015; Vos et al. 2018), where the impacted community typically has substantial uncertainty regarding the evolving threat and where passing information to others has potential social value; determining the threat level at which a community transitions from “individual relevance” of messages to seeing them as “socially relevant” would seem to be an interesting problem for further research.
We also find that, whereas community building increases message passing in nonthreat periods and decreases message passing in threat periods (as might be expected from first principles), weather products that communicate information about hazard likelihood and impact decrease message passing in threat periods. While the RCSM model postulates that content describing threat severity and impact positively affect message passing (Sutton et al. 2015; Vos et al. 2018), recent research on social media engagement practices on Twitter shows that WFOs increase the number of product messages sent during the threat period (Olson et al. 2019). As a result, a saturation or dilution effect may occur, similar to that observed by Caplow (1947) in the context of rumoring during military operations. If message saturation during threat periods indeed reduces the tendency of readers to pass important warning information to others, then WFO practices designed to meet the NWS mission of informing the public about impending weather threats may be able to obtain improved results by incorporating other features that enhance retransmission. For example, of the 1218 messages in our corpus that contain product content, only 99 include action/instruction. Action/instruction is a consistent predictor of message passing (Sutton et al. 2015; Vos et al. 2018). A somewhat larger number, 721, include visual images, which also enhance transmission. Since the inclusion of instructional content has the added benefit of helping message receivers to make decisions on actions that can be taken to protect themselves and others (Frisby et al. 2014, 2013), incorporating it more widely could enhance both message relevance and message reach. More generally, another possibility is that the current weather products, which were designed before the advent of social media, are not ideally suited to social media and could be adjusted to provide better resonance with an online audience.
Message microstructure affects audience engagement through message passing as well. Consistent with prior research, we find that attaching a visual image to a written message increases message passing in both threat and nonthreat periods. In addition, changing the structure of the message itself to include an exclamation point, either for emphasis or to suggest the use of an imperative voice, increases message passing. Some scholars have found, however, that the use of exclamation points can negatively affect perceptions of credibility of the message sender (Castillo et al. 2013) suggesting that sentence syntax should be reserved for times when there is a clear need to communicate the severity of a threat. While neither of these structural changes affect dialogue (two-way communication), they do serve to gain audience attention in a very busy information environment. In direct contrast, message features that are designed to increase two-way interaction (questions, direct @replies, @mentions, and the inclusion of a hyperlink) decrease message passing among audiences in both threat and nonthreat periods. This finding is particularly important for WFOs and other organizations considering their strategic use of communication technologies to note. If the goal of communication is to amplify messages through message passing in order to reach a broader audience, the use of interactive features available on social media may have limited success. However, if the goal is to engage one-on-one with members of an audience, cultivate relationships, or demonstrate direct communication with constituents, interactive features have the potential to facilitate this outcome.
Our findings regarding the use of keyword hashtags also have important implications for message passing in threat and nonthreat periods. Prior research has focused on topical, event-based hashtags during the period of imminent and ongoing threat for events of significant social disruption (Sutton et al. 2015; Vos et al. 2018). In more routine contexts, hashtags may be used in many other ways, and their function of creating an emergent “channel” through which information seekers can learn about a specific event may be less important (Bruns and Burgess 2011). Our findings here bear out the notion that this difference is consequential. As with our prior work, we find in this study that the use of hashtags that characterize or capture a particular event as it is unfolding increases the likelihood of message passing. This phenomenon is especially apparent in threat periods, but it continues in nonthreat periods after the advisory, watch, or warning has been lifted. This suggests an ongoing interest among audience members long past the period of risk and threat into the aftermath of storms and other weather phenomena. However, we also find that there are some hashtags that have a negative effect on message passing. Importantly, we see that campaigns, traditionally used to promote some sort of educational initiative, do not actually appear to increase message passing. This suggests that keyword hashtags are filling a different purpose in this case. As noted, hashtags can function to make it possible for audience members to find content that is relevant to a particular context or time period (Bruns and Burgess 2011). In the case of campaigns, which can be sporadic or ongoing, the use of a consistent hashtag, employed by multiple communicators, can increase visibility of the campaign, or may serve as a marker or reminder for audiences, helping them to recall information later (Bruns et al. 2016). However, such usage of hashtags to facilitate the “push” of messages to readers who may or may not want them—and who may even recognize that they are being “pushed” to by the presence of the hashtag—is very different from the hashtags that help users successfully “pull” information about a flood, wildfire, or other critical hazard that is directly affecting their community. In the latter case, messages successfully located by means of the hashtag may be widely shared, while in the former, the unlooked-for hashtag may at best become a mental marker (or “brand”) for an accidentally encountered campaign. While these hashtags may have other useful effects (as noted above), our data do not support the contention that such hashtags will themselves lead readers to pass such messages to others.
5. Conclusions and future research
Engagement can be thought of as long-term communication on social media among public science-based organizations like the National Weather Service. In different time periods and different contexts, objectives may shift from one-way communication to alert and inform public audiences to two-way communication that facilitates ongoing interaction. While enhancing message passing is only one of many possible objectives, it can nevertheless be important for facilitating widespread dissemination of messages and for generating multiple exposures (ideally from trusted sources) to messages requiring an active response. Our analyses show that many communication practices currently employed by NWS offices do have substantial effects on message passing, and that these can—but do not always—differ in threat versus nonthreat periods. To increase the reach of Twitter messages, our findings suggest that risk communicators should include action-oriented message content, attach visual images, and share weather facts that call to mind historical events. While the use of exclamation marks will help messages to stand out, offices should consider using this strategy sparingly to call attention to the severity of a threat or seriousness of the actions being promoted. There are also many engagement practices that we do not find to increase message passing, such as posting messages that include forecast, nowcast, and weather product content, directing messages to other users, or including a hyperlink. Weather forecasters may have many good reasons to employ messages with such features but doing so may impose a cost with respect to message passing. Where offsetting the retransmission cost is desirable, incorporating factors found to enhance transmission (such as actionable information and visual imagery) may prove effective.
In this paper, we have focused on information amplification and message passing within a diverse group of 12 WFOs, investigating what message content and message structures influence message sharing on Twitter during nonthreat and threat periods. Clearly, additional behavioral responses warrant further investigation, including how engagement practices affect account follower numbers, how two-way communication facilitates trust, and how messages inspire action among those who are at risk. Additional features of messages would also benefit from deeper analysis, such as the coding and quantification of images as objects and what they represent to online audiences. Future research should also consider how images are processed and interpreted by audiences, especially those that contain probabilistic hazard information and accompany weather products. Such lines of inquiry could lead to improved message design and social media strategies that can be implemented across organizations.
Furthermore, while the 12 WFOs represent varied geographical areas and populations exposed to a variety of hazard types that occur seasonally, this particular sample did not include coastal meteorological events, such as hurricane, severe winter weather, such as blizzard, or tornado outbreaks. Nor did we examine how communication strategies may differ between threats that include long-term lead-up in contrast with short-fuse events. This suggests that future research could expand upon the current work to include a wider variety of WFOs, hazard types, and times of year as well as the populations at risk.
Finally, this work demonstrates that the broader context in which messages are posted to social media can have important effects on how audiences engage with those messages. While some features operate similarly from one context to another, others can vary both quantitatively and qualitatively over time. Choosing the right strategy to enhance Twitter message passing—or to achieve any other engagement goal—requires adapting to the circumstances at hand, rather than seeking “one weird trick” that will work in any occasion.
Acknowledgments
This research was supported by the National Science Foundation program on Infrastructure Management for Extreme Events (CMMI-IMEE 1536347 and CMMI-IMEE 1536319). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NSF.
REFERENCES
Allport, G. W., and L. Postman, 1947: The Psychology of Rumor. Henry Holt, 247 pp.
Bail, C. A., 2016: Emotional feedback and the viral spread of social media messages about autism spectrum disorders. Amer. J. Public Health, 06, 1173–1180, https://doi.org/10.2105/AJPH.2016.303181.
Bruns, A., and J. E. Burgess, 2011: The use of Twitter hashtags in the formation of ad hoc publics. Proc. Sixth European Consortium for Political Research General Conf. 2011, Reykjavik, Iceland, ECPR, https://ecpr.eu/Filestore/PaperProposal/90279979-25e8-4090-bd32-2e3cf8ee0c10.pdf.
Bruns, A., B. Moon, A. Paul, and F. Munch, 2016: Towards a typology of hashtag publics: A large-scale comparative study of user engagement across trending topics. Commun. Res. Pract., 2, 20–46, https://doi.org/10.1080/22041451.2016.1155328.
Caplow, T., 1947: Rumors in war. Soc. Forces, 25, 298–302, https://doi.org/10.1093/sf/25.3.298.
Castillo, C., M. Mendoza, and B. Poblete, 2013: Predicting information credibility in time-sensitive social media. Internet Res., 23, 560–588, https://doi.org/10.1108/IntR-05-2012-0095.
Dhanesh, G. S., 2017: Putting engagement in its PRoper place: State of the field, definition and model of engagement in public relations. Public Relat. Rev., 43, 925–933, https://doi.org/10.1016/j.pubrev.2017.04.001.
Finucane, M. L., A. Alhakami, P. Slovic, and S. M. Johnson, 2000: The affect heuristic in judgments of risks and benefits. J. Behav. Decis. Making, 13, 1–17, https://doi.org/10.1002/(SICI)1099-0771(200001/03)13:1<1::AID-BDM333>3.0.CO;2-S.
Fraustino, J. D., B. F. Liu, and Y. Jin, 2012: Social media use during disasters: A review of the knowledge base and gaps. Human Factors/Behavioral Sciences Division, Science and Technology Directorate, 39 pp., https://www.start.umd.edu/sites/default/files/files/publications/START_SocialMediaUseduringDisasters_LitReview.pdf.
Frisby, B. N., D. D. Sellnow, D. R. Lane, S. R. Veil, and T. L. Sellnow, 2013: Instruction in crisis situations: Targeting learning preferences and self-efficacy. Risk Manage., 15, 250–271, https://doi.org/10.1057/rm.2013.7.
Frisby, B. N., S. R. Veil, and T. L. Sellnow, 2014: Instructional messages during health-related crises: Essential content for self-protection. Health Commun., 29, 347–354, https://doi.org/10.1080/10410236.2012.755604.
Hilbe, J. M., 2011: Negative Binomial Regression. Cambridge University Press, 553 pp.
Iowa State University, 2019: Iowa Environmental Mesonet. Accessed July 2017, https://mesonet.agron.iastate.edu.
Keller, C., M. Siegrist, and H. Gutscher, 2006: The role of affect and availability heuristics in risk communication. Risk Anal., 26, 631–639, https://doi.org/10.1111/j.1539-6924.2006.00773.x.
Krippendorff, K., 2011: Computing Krippendorff’s alpha-reliability. University of Pennsylvania, 10 pp., https://repository.upenn.edu/cgi/viewcontent.cgi?article=1043&context=asc_papers.
Lee, N. M., and M. S. VanDyke, 2015: Set it and forget it: The one-way use of social media by government agencies communicating science. Sci. Commun., 37, 533–541, https://doi.org/10.1177/1075547015588600.
Lovejoy, K., and G. D. Saxton, 2012: Information, community, and action: How nonprofit organizations use social media. J. Comput. Mediat. Commun., 17, 337–353, https://doi.org/10.1111/j.1083-6101.2012.01576.x.
Lovejoy, K., R. D. Waters, and G. D. Saxton, 2012: Engaging stakeholders through Twitter: How nonprofit organizations are getting more out of 140 characters or less. Public Relat. Rev., 38, 313–318, https://doi.org/10.1016/j.pubrev.2012.01.005.
Olson, M. K., J. Sutton, S. C. Vos, R. Prestley, S. L. Renshaw, and C. T. Butts, 2019: Build community before the storm: The National Weather Service’s use of social media. J. Contingencies Crisis Manage., https://doi.org/10.1111/1468-5973.12267.
R Core Team, 2016: R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, https://www.R-project.org/.
Rogers, E. M., 2003: Diffusion of Innovations. 5th ed. Free Press, 576 pp.
Saxton, G. D., and R. D. Waters, 2014: What do stakeholders like on Facebook? Examining public reactions to nonprofit organizations’ informational, promotional, and community-building messages. J. Public Relat. Res., 26, 280–299, https://doi.org/10.1080/1062726X.2014.908721.
Sellnow-Richmond, D. D., D. D. Sellnow, and A. M. George, 2018: An IDEA model analysis of instructional risk communication in the time of Ebola. J. Int. Crisis Risk Commun. Res., 1, 135–166, https://doi.org/10.30658/jicrcr.1.1.7.
Skaug, H. J., D. Fournier, A. Nielsen, A. Magnusson, and B. Bolker, 2010: glmmADMB? Generalized linear mixed models using AD Model Builder. R package version 0.6.5.r143, https://rdrr.io/rforge/glmmADMB/.
Spitzberg, B. H., 2014: Toward a model of meme diffusion (M3D). Commun. Theory, 24, 311–339, https://doi.org/10.1111/comt.12042.
Starbird, K., and L. Palen, 2012: (How) will the revolution be retweeted?: Information diffusion and the 2011 Egyptian uprising. Proc. ACM 2012 Conf. on Computer Supported Cooperative Work, Seattle, WA, Association for Computing Machinery, 7–16.
Suh, B., L. Hong, P. Pirolli, and E. H. Chi, 2010: Want to be retweeted? Large scale analytics on factors impacting retweet in twitter network. 2010 IEEE Second Int. Conf. on Social Computing, Minneapolis, MN, IEEE, 177–184, https://doi.org/10.1109/SocialCom.2010.33.
Sutton, J., E. S. Spiro, B. Johnson, S. Fitzhugh, B. Gibson, and C. T. Butts, 2014: Warning tweets: Serial transmission of warning messages during a disaster event. Inf. Commun. Soc., 17, 765–787, https://doi.org/10.1080/1369118X.2013.862561.
Sutton, J., C. B. Gibson, N. E. Phillips, E. S. Spiro, C. League, B. Johnson, S. M. Fitzhugh, and C. T. Butts, 2015: A cross-hazard analysis of Twitter warning messages. Proc. Natl. Acad. Sci. USA, 112, 14 793–14 798, https://doi.org/10.1073/pnas.1508916112.
Taylor, M., and M. L. Kent, 2014: Dialogic engagement: Clarifying foundational concepts. J. Public Relat. Res., 26, 384–398, https://doi.org/10.1080/1062726X.2014.956106.
Venebles, W. N., and R. D. Ripley, 2002: Modern Applied Statistics with S. Springer, 498 pp.
Vos, S. C., J. Sutton, Y. Yu, S. Renshaw, M. K. Olson, C. B. Gibson, and C. T. Butts, 2018: Retweeting risk communication: The role of threat and efficacy. Risk Anal., 38, 2580–2598, https://doi.org/10.1111/risa.13140.
Wachinger, G., O. Renn, C. Begg, and C. Kuhlicke, 2013: The risk perception paradox—Implications for governance and communication of natural hazards. Risk Anal., 33, 1049–1065, https://doi.org/10.1111/j.1539-6924.2012.01942.x.
Watkins, B. A., 2017: Experimenting with dialogue on Twitter: An examination of the influence of the dialogic principles on engagement, interaction, and attitude. Public Relat. Rev., 43, 163–171, https://doi.org/10.1016/j.pubrev.2016.07.002.
