1. Introduction
Risk communication is a vital element in risk management and a promising way to protect public health and safety across a range of domains, including environmental hazards and health (Leiss 1996; Demeritt and Nobert 2014). As a component of risk communication, public risk messages issued by government agencies in the context of natural hazards are important because such messages inform affected populations about hazardous situations and may stimulate protective actions. In recent years, social media have been increasingly used by agencies and organizations to communicate with the public about natural hazards and disasters (Hughes and Palen 2012; Palen and Hughes 2018; Sutton and Kuligowski 2019). Federal, state, and local governments, via emergency management agencies, meteorological departments, and health departments have used social media like Twitter and Facebook to share and collect timely information before, during, and after a variety of hazardous events (Hughes et al. 2014; St. Denis et al. 2014; Li et al. 2018; Scott and Errett 2018).
Message diffusion in the context of natural hazards enables people who are beyond the direct contacts of the initial sender to receive lifesaving messages. Receiving public risk messages enhances the likelihood of taking protective actions (Mileti and Sorensen 1990), although barriers exist between the point of receiving messages and the point of taking actions. Public risk messages disseminated via social media can be retransmitted more easily, to more individuals, and with higher fidelity than via mass media channels such as radio and television (Sutton et al. 2014, 2015). This highlights the need to understand what factors facilitate or suppress retransmission of official risk messages in social media. The present research investigates how an underexamined factor, persuasive message content, influences message diffusion on Twitter in the context of heat hazards. In this study, persuasive message content refers to specific message content that, suggested by theories or empirical studies, has the potential to influence receivers’ attitudes, intentions, or behaviors. This research can benefit public officials especially communication practitioners by identifying evidence-based strategies about risk messaging to increase message diffusion on Twitter. Such strategies also have the potential to motivate people to take protective actions, since these strategies are persuasive message content whose persuasiveness has been suggested by previous studies.
2. Background
a. Message diffusion on social media
Social media sites such as Twitter and Facebook enable message retransmission via functions such as “retweeting” on Twitter and “sharing” on Facebook. Using these functions, people who consume information can also actively promote information to the broader public on social media (Lin et al. 2016b). The number of times the original message was retransmitted is recorded on social media sites, which allows investigation of factors predicting message retransmission with precision unachievable by traditional data sources (Sutton et al. 2015). There is a growing body of research investigating predictors of message retransmission on social media across contexts such as natural hazards (Sutton et al. 2015; Lin et al. 2016a), emerging infectious disease (Vos et al. 2018), software vulnerability (Syed et al. 2018), and marketing (Cvijikj and Michahelles 2013; Walker et al. 2017). Due to limited data availability through other social media platforms (such as Facebook), previous studies have heavily relied on Twitter to investigate retransmission mechanisms. Twitter is a microblogging service, and around one-fifth of U.S. adults (22%) use Twitter (Wojcik and Hughes 2019).
Across research domains, factors related to message retransmission on Twitter can be categorized into two main groups: intrinsic message features and extrinsic factors beyond the messages themselves. For intrinsic message features, previous studies have examined how message retransmission on Twitter is affected by thematic content (Sutton et al. 2014, 2015), message style such as the use of imperative sentence style (Sutton et al. 2015; Vos et al. 2018; Lachlan et al. 2019), message structure such as inclusion of images and URLs (Sutton et al. 2015; Lachlan et al. 2019), and message sentiment (Walker et al. 2017; Yang et al. 2018). Extrinsic message retransmission factors include network features such as the number of followers of the sending account (Vos et al. 2018), authorship of Twitter messages (tweets; Wang et al. 2020), and the created time of tweets (Zhu et al. 2011).
b. A knowledge gap about win-win message strategies
Some of the factors related to message diffusion also influence message persuasion, or the message’s ability to influence recipients’ attitudes, behavioral intentions, and behaviors. For example, images in health communication not only can predict increased message diffusion on Twitter (Vos et al. 2018) but also increase intentions to adopt suggested behaviors (Anderson 1983). Message sources also matter for both message diffusion and message persuasion (Wilson and Sherrell 1993; Wang et al. 2020). Investigating message factors that may influence both message diffusion and message persuasion is important, because it helps identify message strategies that achieve two kinds of message success (persuasion and diffusion). When it comes to message content, limited research attention has been paid to identifying such win-win message content. When investigating message content as a potential factor of message diffusion, researchers across a variety of domains typically inductively categorize message content into thematic content (Sutton et al. 2014; Syed et al. 2018), rather than deductively coding messages into persuasive message content. As a result, much less is known about what persuasive message content enhances message diffusion than what informative themes enhance message diffusion.
Thematic content is usually different from persuasive message content because it is identified based on different considerations. Thematic content is identified based on patterns of meaning within messages, but persuasive message content is identified based on what has been found by previous theories and empirical studies to increase persuasion. Nuanced message content that is persuasive may not be distinguished as separate content themes using an inductive coding method, and thus data-driven thematic content is usually overrepresented relative to concept-driven persuasive message content. For example, hazard information is one type of thematic content that has been positively related to retweet counts across four types of natural hazards (Sutton et al. 2014, 2015). The theme of hazard information includes descriptions about physical characteristics of the hazard itself and/or hazard impacts (Sutton et al. 2015). There is little doubt that risk messages need information about the hazard itself and hazard impacts (Mileti and Sorensen 1990). However, we hesitate to say that the theme of hazard information is persuasive message content. This is because past studies typically disaggregated the hazard information theme into several components and examined the persuasive effects of its components (Morss et al. 2016; Lebel et al. 2018; Potter et al. 2018), instead of examining the persuasive effects of the hazard information theme itself. A possible reason is that studies comparing the presence and absence of the hazard information theme would not provide useful suggestions for risk messaging since risk messages would include hazard information anyway. The hazard information theme may be too broad to be a meaningful unit of persuasive message content. According to previous theoretical and empirical studies about persuasion, what components of the hazard information theme are persuasive message content will be described in the next subsection.
To our knowledge, no study has investigated how persuasive message content influences message diffusion in the context of natural hazards, and the present study is the first study to do so. In the related field of health communication, only one study (Vos et al. 2018) deductively identified specific persuasive message content based on a persuasion theory, the extended parallel process model (Witte 1992). The study found that depicted severity (the depicted magnitude of harm that could happen from the Zika virus) and efficacy (information about protective actions recommended for individuals) enhanced retransmission of official risk messages on Twitter, but no effect was observed regarding depicted susceptibility (who is at risk for negative consequences from the Zika virus) (Vos et al. 2018). The present study was designed in a different context, heat hazards, and used persuasive message content that is suitable to natural hazards.
c. Persuasive message content about natural hazards
Previous studies have suggested some persuasive message content about natural hazards. In recent years, experimental studies disaggregated the theme of hazard information into two components, hazard-based messages and impact-based messages, and compared their persuasive effects (Morss et al. 2016, 2018; Potter et al. 2018). For example, impact-based messages that only contain descriptions about hazard impacts (e.g., potential damage posed to infrastructure) increased risk perceptions of the hazardous event relative to hazard-based messages that only contain descriptions about characteristics of the hazard itself (e.g., wind speed) (Potter et al. 2018). Drawing on fear appeal theories, commonly used in the health communication literature (Witte 1992; Tannenbaum et al. 2015), our prior work (Li et al. 2018) further disaggregated the theme of hazard information into four types of persuasive message content applicable for natural hazards: hazard uncertainty, hazard intensity, health risk susceptibility, and health impact. Our work also identified a fifth type of persuasive message content that was about guidance, termed response instruction (see details in Table 1). We called these five types of persuasive message content persuasive message factors (PMFs) (Li et al. 2018). The present study builds on this prior study and investigates how these PMFs respectively and cumulatively predict the retweet counts of official risk messages about heat hazards.
Definition, coding scheme, and examples for persuasive message factors (adapted from Li et al. 2018).
The persuasive effects of these five PMFs have been suggested by previous studies. With respect to the four PMFs that belong to the broad hazard information theme, meta-analyses of fear appeal studies have found that the independent and joint inclusion of depicted susceptibility (descriptions emphasizing how likely message recipients will be adversely impacted) and depicted severity (descriptions emphasizing negative consequences) in risk messages were persuasive (de Hoog et al. 2007; Tannenbaum et al. 2015). For example, health messages emphasizing the recipient’s personal risk and serious consequences of maladaptation positively influence people’s behavioral intentions and behaviors compared to messages depicting lower susceptibility and lower severity of the negative consequences (Tannenbaum et al. 2015). Li et al. (2018) adapted depicted susceptibility and severity to natural hazards. Hazard uncertainty and health risk susceptibility respectively indicate depicted susceptibility of the hazard itself and depicted susceptibility of hazard impacts, and hazard intensity and health impact respectively indicate depicted severity of the hazard itself and depicted severity of hazard impacts. Definitions of these terms are provided in Table 1. With respect to the PMF of response instruction, meta-analyses of fear appeal studies also suggested the persuasive effects of such efficacy statements (Tannenbaum et al. 2015). Compared to risk messages without efficacy statements, risk messages with efficacy statements improve people’s behavioral intentions and tendency to engage in behaviors through increased perceived self-efficacy (belief in one’s capacity of performing recommended actions) and/or increased perceived response-efficacy (belief that the recommended actions will achieve desirable outcomes) (Floyd et al. 2000; Milne et al. 2000; Witte and Allen 2000; Tannenbaum et al. 2015).
Previous empirical studies in the context of natural hazards also suggested the persuasive effects of some PMFs investigated in the present study. These previous studies may not use the exact terms as we used to describe their manipulation. However, we found these previous studies manipulated a certain PMF described in the present study after comparing their control messages and treatment messages using the definitions of PMFs. These previous studies have found that intentions to take recommended actions can be elevated by each mention of hazard uncertainty (Lebel et al. 2018), hazard intensity (Casteel 2016), impact severity (e.g., negative consequences on health and property; Casteel 2016), and response instructions (Wong-Parodi et al. 2018). In addition, mentions of health risk susceptibility have the potential to address issues that have been identified from previous studies. Failure to personalize heat-health risks has been identified as a main reason why people did not take recommended actions in heat risk messages (Kalkstein and Sheridan 2007; Sheridan 2007; Bassil and Cole 2010). Health risk susceptibility has the potential to avoid the misperception of “it can’t happen to me” by clarifying who and/or which behaviors are at risk for negative impacts from heat events (Li et al. 2018). However, the persuasive effects of health risk susceptibility need future research about natural hazards to provide empirical evidence.
In addition to identifying these five PMFs, our prior work also content-analyzed 904 tweets related to heat hazards issued by a sample of 18 U.S. NWS Weather Forecast Offices (WFOs) in 2016 (Li et al. 2018). We examined the degree to which the five PMFs were mentioned in these official heat risk tweets (Li et al. 2018). The present study expands on this prior study and investigates how four of the five PMFs respectively and cumulatively predict the retweet counts of the official risk messages for heat hazards. The PMF that we removed from the analyses was hazard uncertainty, since heat-related tweets mentioning hazard uncertainty were too rare (only 5 of 904 tweets) to reliably estimate its effects. Our models also controlled for some extrinsic factors of message retransmission such as network features, which will be described in detail in the method section.
d. Different message types
To analyze the respective and cumulative effects of PMFs, this study built models predicting retweet counts for all heat-related tweets. In addition, this study also built separate models for a subset of heat-related tweets that alerted about extreme heat events (heat warning tweets) and for another subset of heat-related tweets that alerted about nonextreme heat events (nonwarning tweets). In this study, extreme and nonextreme heat events were mainly distinguished by whether heat events are accompanied by NWS’s heat watch, warning, and advisory (WWA) products. If a heat-related tweet alerted about a heat event that was accompanied by any of the heat WWAs and also mentioned active heat WWAs in the tweet, this heat-related tweet was categorized as a “heat warning tweet.” If a heat-related tweet alerted about a heat event whose conditions were not hot enough and/or long enough in duration to issue heat WWAs, this tweet was categorized as a “nonwarning tweet.”
Heat hazards pose a serious threat to people in the United States, causing more deaths than floods, hurricanes, and tornadoes combined during 2009 to 2018 (Centers for Disease Control and Prevention 2020). Widespread heat-health impacts affect people across age groups and geographic areas (Hess et al. 2014; Mora et al. 2017). Both heat warning tweets and nonwarning tweets are important to protect the public from negative health impacts from heat. Although local WFOs have highly variable criteria regarding conditions favorable to issue heat WWAs for their forecast areas, conditions that warrant heat WWAs in each WFO indicate that, in general, such conditions are dangerous for the local population within the WFO’s forecast area (Hawkins et al. 2017). Extreme heat events can harm anyone without appropriate actions (Mora et al. 2017), and heat warning tweets communicate such dangerous conditions with the general public in order to motivate protective actions. Nonwarning tweets alert about nonextreme heat events during which negative heat effects are still likely for vulnerable populations such as the elderly, those exercising or working outdoors, and those without adequate hydration (Kovats and Hajat 2008; Mora et al. 2017). Investigating the PMF effects separately for heat warning tweets and nonwarning tweets allows targeted messaging suggestions for risk communicators to create different message types for different heat conditions. Investigating the PMF effects for all heat-related tweets allows description of effects at an aggregate level for all tweets that aim to protect the public from heat-health risks.
We propose two research questions in this study:
How does the inclusion of the persuasive message factors of hazard intensity, health risk susceptibility, health impact, and response instruction influence message retransmission respectively for heat-related tweets, heat warning tweets, and nonwarning tweets posted by U.S. NWS WFOs?
What are the cumulative impacts of the inclusion of the persuasive message factors of hazard intensity, health risk susceptibility, health impact, and response instruction on message retransmission for heat-related tweets, heat warning tweets, and nonwarning tweets posted by U.S. NWS WFOs?
3. Method
a. Data
Official heat-related tweets (N = 904) were collected by our prior work (Li et al. 2018). Using the Twitter search application programming interface (API), tweets and their retweet counts were collected if tweets were posted between 1 June and 31 August 2016 by each official Twitter account of the 18 sampled NWS WFOs. These sampled offices (see Fig. 1) were chosen using theoretical sampling (Singleton and Straits 2010) and these offices demonstrate important variations among the total of 123 U.S. WFOs in terms of local climate and NWS regions. Our prior study (Li et al. 2018) extracted original tweets that contained the English words “hot” or “heat” in the displayed text, and further manually coded the extracted tweets as “heat-related tweets” if the extracted tweets (including the displayed text and text in attached images) indicated that specific heat events either were occurring or upcoming in the forecast areas (intercoder reliability coefficients, Cohen’s kappa = 0.83). This human coding process removed some extracted tweets that, although containing the words “hot” or “heat,” were not heat-related tweets, for example, tweets only stating an expired heat warning. In addition, each of the five PMFs were deductively coded in our prior work (Li et al. 2018). All heat-related tweets (N = 904) were coded based on not only the displayed text but also textual information in attached images. For each heat-related tweet, the five PMFs (hazard uncertainty, hazard intensity, health risk susceptibility, health impact, and response instruction) had its own code (1: presence vs 0: absence). Each tweet could contain one or more PMFs. With respect to intercoder reliability, the Cohen’s kappa values of the five PMFs were all above 0.93 (Li et al. 2018).
b. Operationalization
The dependent variable of retweet counts is the number of times a tweet was retransmitted. The respective effects of the PMFs were operationalized as four variables indicating the presence or absence of each PMF (hazard intensity, health risk susceptibility, health impact, and response instruction). As mentioned earlier, we removed the PMF of hazard uncertainty when modeling the respective and cumulative effects of PMFs because the tweets containing the PMF of hazard uncertainty were rare (only 5 of 904 tweets). The cumulative effect of the PMFs was operationalized as the number of PMFs (hazard intensity, health risk susceptibility, health impact, or response instruction) mentioned in a risk message, which ranged from zero to four.
In additional to heat-related tweets overall (N = 904), the other two message types were two subsets of heat-related tweets: heat warning tweets (N = 223) and nonwarning tweets (N = 436). First, as mentioned earlier, heat warning tweets alerted about current or anticipated extreme heat events that warrant heat WWAs, and nonwarning tweets alerted about current or anticipated nonextreme heat events that did not warrant heat WWAs. For the present study, to be considered a heat warning tweet, a heat-related tweet must 1) be posted within at least one heat WWA’s active period (from issuance time to expiration time) in its respective WFO, and 2) mention at least one heat WWA that has been issued, is currently in effect, or will be in effect in the displayed text or text in attached images. About a quarter of heat-related tweets (N = 223) met the two criteria and were categorized as heat warning tweets. Second, some of the heat-related tweets (N = 245) only met the first criterion, which means they were posted when at least one heat WWA was issued in their respective WFOs but these tweets did not mention the co-occurring heat WWAs. On the one hand, some of these 245 tweets may alert about nonextreme heat events. For example, consider a case in which a heat warning product is issued this morning and indicates that the start time of an extreme heat event is tomorrow. An official tweet may be posted at noon and only mention today’s nonextreme heat situation that does not warrant a watch, warning, or advisory product. On the other hand, some of these 245 tweets may alert about extreme heat events, but they did not mention co-occurring heat WWAs. In this situation, the diffusion mechanism of the tweets may be different from those that met both criteria to be considered heat warning tweets. As a result, we did not identify these 245 heat-related tweets as either heat warning or nonwarning tweets. In other words, although the 245 heat-related tweets were included when we built models using all heat-related tweets, the 245 heat-related tweets were excluded when we built models using the subsets of heat-related tweets: heat warning tweets and nonwarning tweets, because they could not be definitively included in either category. Third, to be considered a nonwarning tweet, a heat-related tweet must have been posted prior to the issuance time of heat WWAs and after the expiration time of heat WWAs in respective WFOs. Data about the issuance/expiration time of archived heat WWAs were collected from the Iowa Environmental Mesonet (2019). About half of heat-related tweets (N = 436) were categorized as nonwarning tweets, and there is no overlap between heat warning tweets and nonwarning tweets.
We also considered control variables (Table 2) to help isolate the relationship between mentions of PMFs and message diffusion. These include the time of day, day of week, and the month the tweet was issued, the sending account and its number of followers, the region of origin, the population of the office’s jurisdiction, and environmental variables (monthly normal temperature and temperature anomaly). The created time of tweets (except created month), network features, and authorship have each been found to have an influence on message retransmission (Zhu et al. 2011; Sutton et al. 2015; Hu et al. 2019; Wang et al. 2020). Seasonality (created month) and environmental variables (monthly normal temperature and monthly temperature anomaly) could influence the sharing behavior of local Twitter users through a mediator, heat risk perception. Early in the warm season, higher mean temperature, and increased temperature anomaly have been associated with higher heat risk perception (Schoessow 2018), and the higher heat risk perception among local Twitter users could motivate more message sharing behaviors regardless of the mention of PMFs among such messages. Aligned with previous studies (Howe et al. 2019), we used mean temperatures (instead of maximum and minimum temperatures) to calculate monthly normal temperatures and temperature anomalies. Mean temperatures were highly correlated with maximum and minimum temperatures in our datasets (Pearson correlation coefficient ranging from 0.88 to 0.97).
Description of control variables.
c. Analytic approach
We modeled the effects of PMFs on message diffusion through a multilevel negative binomial regression model in the R statistical computing environment using the lme4 package (Bates et al. 2015). Respective effects and cumulative effects were modeled separately. For each type of effect, we also modeled each of the three datasets that correspond to heat-related tweets, heat warning tweets, and nonwarning tweets, respectively. The two subsets of heat-related tweets were modeled separately to find out whether the effects of PMFs on message diffusion are different between heat warning tweets and nonwarning tweets. We used negative binomial regression models (Gelman and Hill 2006) because retweet counts in our datasets were overdispersed count data (dispersion parameters ranging from 2.2 to 7.5). Our data were collected with multilevel structures (e.g., tweets within WFOs and WFO regions). Multilevel modeling, compared to classical regression, provided more reasonable estimates because multilevel modeling accounts for group-level variability by including indicators at different levels and also accounts for group-level dependency through partial pooling (Gelman and Hill 2006).
Each of the six multilevel negative binomial models was fit using a combination of individual-level predictors, grouping variables, and group-level predictors. The individual-level predictors were the variables regarding the respective or cumulative effects of the PMFs. These individual-level predictors were treated as fixed effects, which means that their coefficients were estimated using classical maximum likelihood methods (Gelman and Hill 2006). Individual tweets were also grouped according to their created time of day, created day of week, created month, sending WFO, and NWS region. In our study, these grouping variables were treated as random effects and multilevel regression models were restricted to a varying-intercept and constant-slope model. This means that each group within these grouping variables (e.g., each WFO within the grouping variable of sending WFO) could have different intercepts in the multilevel model, and the varying intercepts were estimated using partial pooling (Gelman and Hill 2006). Some of these grouping variables also have group-level predictors: follower counts and population size were two group-level predictors for the group of the sending WFO. Monthly normal temperature and monthly temperature anomaly were group-level predictors across the groups of sending WFO level and created month. These group-level predictors were treated as fixed effects in our models.
The continuous predictors in this study were on different scales. To reduce their impact on parameter estimates, we multiplied the variable of monthly temperature anomaly (°C) by a factor of 10 and transformed the variables of follower counts and population size using the natural log function. For each of the six models, variables treated as fixed effects did not have serious multicollinearity problems, according to the generalized variance-inflation factor (GVIF; Fox and Monette 1992). The highest GVIF among fixed-effect variables in the six models was 2.4. Aligned with GVIF, the highest Pearson correlation between logged follower counts and logged population size was 0.61. All fixed effects were kept in all models regardless of their explanatory effects. For each model, we dropped the random effects that provided little explanatory effect (i.e., with an intraclass correlation coefficient less than 0.0001).
For model diagnostics, we used the plot of Pearson residuals against fitted values on the scale of the linear predictor for our multilevel negative binomial models. This plot is the equivalent of the plot of residuals against fitted values for general linear models (Faraway 2016). For each of the six models, points in the plot of Pearson residuals against fitted values in the scale of the linear predictor were around the horizontal line of zero, with a roughly constant variance, which means that the assumptions of linearity (in the scale of linear predictors) and equal variance of errors (scaling out the variance function) were met for all multilevel negative binomial models.
4. Results
a. Distribution of PMFs
Retweet counts of the heat-related tweets in our dataset ranged from 0 to 217, with a mean of 13.6 (standard deviation SD = 14.9). For the two subsets of heat-related tweets, heat warning tweets had higher retweet counts (mean = 15.5; SD = 13.5) than nonwarning tweets (mean = 10.6; SD = 7.2) without controlling for other variables [t (289.3) = 5; p < 0.001]. Overall, the use of PMFs across message types was quite consistent. Across message types, information about temperature or heat index (the PMF of hazard intensity) was by far the most used PMF and descriptions about the severity of health impacts from heat (the PMF of health impact) was the least frequently mentioned PMF (Fig. 2). About two-thirds of heat warning tweets (N = 158; 70%) mentioned hazard intensity, as did more than four-fifths of heat-related tweets (N = 760; 84%) and nearly 90% nonwarning tweets (N = 392). However, less than one-fifth of tweets mentioned health impact in each category of tweet. The next most used PMF was response instruction across message types, followed by the PMF of health risk susceptibility that describes who, which behavior, or certain places that are at risk from heat.
A majority of tweets used zero or only one PMF in each type of tweet. This was especially the case for nonwarning tweets (N = 314; 72%). For tweets that used one PMF, the percentage of each type of tweet that used the PMF of hazard intensity ranges from 96% to 97%. For tweets that used two PMFs across message types, the percentage of each type of tweet that used the combination of hazard intensity and response instruction ranges from 73% to 85%. Less than 6% tweets used all of the four PMFs in each message type. Descriptive statistics of each type of tweet across grouping variables and group-level predictors can be found in appendix A. Across message types, the number of tweets posted by each sending WFO varied substantially (e.g., heat-related tweets: minimum = 13, maximum = 98, mean = 50, and SD = 30). In contrast, the number of tweets was distributed almost evenly across days of the week. For other grouping variables, more tweets were posted in July but fewer in August. Fewer tweets were posted between 1800 and 0000 local time relative to other times of day. WFOs in the NWS eastern region posted, on average, fewer tweets than WFOs in other regions.
b. Respective and cumulative effects of PMFs
Regarding the respective effect of PMFs, hazard intensity was a consistently positive predictor of retransmission across all types of tweets (Table 3). The other three PMFs, health risk susceptibility, health impact, and response instruction, had statistically significant and positive influence on retweet counts for one or two message types. No PMFs showed negative respective effects on retweet counts. The mention of health risk susceptibility was a statistically significant and positive predictor of retweet counts for nonwarning tweets. The inclusion of health impact had a statistically significant and positive effect on retweet counts in all heat-related tweets and the subset of nonwarning tweets. The mention of response instruction had a statistically significant and positive effect on retweet counts for the heat warning tweets. The effect size of these statistically significant, respective effects was similar, ranging from a 21% increase to a 33% increase in retweets. Given the exploratory nature of this analysis, it is worth noting that, for heat-related tweets, the effect of mentioning health risk susceptibility, incidence rate ratio (IRR) = 1.13 [95% confidence interval (CI): 1.00–1.28], with p = 0.055, and mentioning response instruction, IRR = 1.10 [95% CI: 0.99–1.23], with p = 0.087, approached statistical significance.
Multilevel negative binomial regression predicting the respective effect of PMFs on retweet counts for each type of tweet. Here, b is an unstandardized regression coefficient, SE is standard error, IRR is incidence rate ratio, CI is confidence interval, N is number of groups within a grouping variable, σ2 is the component of variance, ICC is intraclass correlation coefficient, and p values of less than 0.05 are indicated with boldface type.
When compared with the respective effects of individual PMFs, the cumulative effect of PMFs was a more consistent and precise predictor of retweet counts across message types. The number of PMFs was a statistically significant, positive predictor for all types of tweets, and its 95% confidence intervals were consistently narrower than those of the respective effects of separate PMFs (Table 4 and Fig. 3). Every additional type of PMF mentioned in official tweets increased the predicted retweet counts for each type of tweet by a factor of about 1.15, controlling for other variables in the models. Heat-related tweets mentioning four PMFs were estimated to have 48% more retweets than heat-related tweets mentioning one PMF, regardless of the PMF type. For heat warning tweets and nonwarning tweets, tweets containing four PMFs were associated with 53% and 57% more predicted retweets respectively than tweets containing only one PMF. To check whether the effects of the number of PMFs were dependent on a single influential PMF, we conducted 12 additional models (for each PMF and tweet type) dropping tweets mentioning one of the four PMFs from one of three message types. Overall, the effects of the number of PMFs were not driven by a single PMF across message types (see appendix B for details of the statistical analysis). In addition, the cumulative effects of PMFs, as well as the respective effects of each individual PMF, were not statistically significantly different across message types. This is suggested by the overlapped confidence intervals of each predictor for the three datasets (see Fig. 3) and confirmed using a standard method of testing the significance of differences between point estimates (Schenker and Gentleman 2001).
c. Effects of control variables
With respect to the control variables included in the regression models, it is worth noting that population size in the forecast area of WFOs consistently had a positive influence on retweet counts across message types. After controlling for other variables including population size, the follower count of the sending account was not a statistically significant predictor of retweet counts for heat warning tweets and nonwarning tweets, but it had positive effects on retweet counts for heat-related tweets. With respect to the two environmental variables, heat-related tweets posted in places and during months with a higher monthly temperature anomaly predicted slightly increased retweet counts. Heat warning tweets posted in places and during months with higher monthly normal temperature predicted slightly decreased retweet counts. After controlling for other variables in the models, the NWS region, sending WFO, created month of the tweet, and created day of week played varying roles in affecting message diffusion for different message types. The time of day the tweet was posted had only a small influence on message diffusion across message types.
5. Discussion and conclusions
Using official risk messages about heat hazards as a case study, this study investigated the respective and cumulative effects of four types of persuasive message content on message retransmission via social media. We found that official tweets containing more types of PMFs were retweeted more frequently. This finding held true for all heat-related tweets at an aggregate level and was also observed separately among its subsets: heat warning tweets and nonwarning tweets. In respect to the respective effects, the mention of hazard intensity was a positive predictor of retweet counts for heat-related tweets and its two subsets. The mention of health impact was a positive predictor for heat-related tweets and nonwarning tweets. The mention of health risk susceptibility and the mention of response instruction were positive predictors of retweet counts for nonwarning tweets and heat warning tweets, respectively. While some PMFs, as indicated above, showed statistically significant influence for one or two types of tweets and showed statistical insignificance for the other type(s) of tweet(s), each PMF did not show statistically significant differences in its respective effects across three types of tweets.
a. Contributions to theory
Our findings provide insights into how specific message content that is theoretically important to message persuasion influenced message diffusion on social media in the context of natural hazards. To our best knowledge, this is the first study to identify persuasive message content as factors of message retransmission about natural hazards. In the context of health communication, as mentioned earlier, one study about the Zika virus has suggested that depicted severity and efficacy statements were not only persuasive according to a persuasion theory but also effective in terms of message diffusion on Twitter (Vos et al. 2018). In addition, this previous study did not observe the effect of depicted susceptibility on message diffusion, although depicted susceptibility was also persuasive message content (Vos et al. 2018). Our findings about the respective effects of health risk susceptibility, health impact, and response instruction generally align with this previous study, although we did detect a positive effect of health risk susceptibility for tweets alerting nonextreme heat events.
Our research also contributes to understanding the cumulative effects of message content. Previous studies have found that a combined theme of hazard information, which was the equivalent of mentioning at least one of the PMFs among hazard uncertainty, hazard intensity, health risk susceptibility, and health impact, was a positive predictor of message diffusion across four natural hazard events (Sutton et al. 2015). Although this finding sheds some light on the overall effects of persuasive message content, little research attention has been paid specifically to the cumulative effects of message content. The cumulative effects of message content reflect an important message style: specificity. For risk messages, specificity refers to specific information regarding the hazard’s nature and possible consequences, time of impact, location, source, and instructions about protective actions (Mileti and Sorensen 1990). This style of messaging has been found to be persuasive in the context of natural hazards (Mileti and Sorensen 1990; Sutton et al. 2018). Tweets containing a higher number of PMFs are more specific. The positive effects of the number of PMFs detected in the current study suggest that the persuasive message style, specificity, has the potential to enhance message diffusion as well.
In addition to message factors, our study found that audience population size was also a consistent and positive factor of message diffusion, which is in line with one previous study (Hu et al. 2019). A possible explanation of the effect of population size is when a WFO posts a tweet about hazardous weather in its forecast area and if more individuals live in the forecast area, any reader of the tweet would be more likely to have family members, friends, and coworkers living in the affected area, and thus it would be more likely for the reader to think of someone who needs this message and thus retweet it. However, the follower count of sending accounts was not a consistent predictor of message diffusion. Although positive effects of follower counts on message diffusion were found for all heat-related tweets, follower counts did not predict message diffusion for heat warning tweets and nonwarning tweets. Previous studies have also found inconsistent effects of follower counts on message diffusion. Some studies have found positive effects of follower counts on message diffusion (Sutton et al. 2015; Vos et al. 2018; Hu et al. 2019), but some studies have found small negative effects of follower counts on message diffusion (Sutton et al. 2015; Wang et al. 2020). In addition, most previous studies have investigated the effects of follower counts without controlling for the factor of audience population size (Sutton et al. 2015; Vos et al. 2018; Wang et al. 2020). To better understand the effects of follower counts and population size on message diffusion, future research should consider both factors—population size and follower counts—when modeling message diffusion.
b. Contributions to practice
This research informs evidence-based strategies about official risk messaging to enhance message retransmission, thus allowing more people to receive lifesaving messages in the context of natural hazards. When designing official tweets alerting about heat events, no matter whether these events are technically extreme or not, our results about cumulative effects suggest that communicators should use all four PMFs (hazard intensity, health risk susceptibility, health impact, and response instruction) to maximize message diffusion. For official tweets alerting about extreme heat events that are accompanied by heat WWAs, it is especially important to mention the PMFs of hazard intensity and response instruction to enhance message retransmission. Such official tweets should also mention co-occurring heat WWAs in their messages. For official tweets alerting about nonextreme heat events, it is particularly important to mention the PMFs of hazard intensity, health risk susceptibility, and health impact to enhance message diffusion. In addition to contributions on message diffusion, the strategies suggested in our findings also have the potential to promote message persuasion since, in origin, such PMFs were deductively identified based on theoretical and empirical studies about persuasion.
In our datasets, a majority of tweets used zero or only one PMF, and the use of hazard intensity was disproportionately high relative to other PMFs. This fact does not mean that it is infeasible to mention all four types of PMFs in content constrained messages like tweets. In contrast, 280 characters in the displayed text and text in attached images provide ample room to describe each PMF. For example, the hypothetical statement below describes all four PMFs within 140 characters: “Excessive Heat Warning today! Respect the triple-digit heat by drinking enough water and keeping cool! Otherwise everyone is vulnerable to heat-related illnesses.”
c. Limitations and future research
This study had several limitations. First, when predicting the effects of PMFs on message diffusion, we controlled for some extrinsic factors such as network features and authorship of tweets, but our models did not include some intrinsic factors that have been related to message diffusion. For example, we did not consider factors of capitalization of words, inclusion of hashtags, and the imperative sentence style, which have been found to enhance message retransmission in the context of natural hazards (Sutton et al. 2015; Lachlan et al. 2019). These factors—especially the imperative sentence style—may also improve message clarity and message certainty, which are important message styles for risk messages (Mileti and Sorensen 1990; Lachlan et al. 2019). Although our models already explained 44%–57% of the variance in the retweet counts, future research should consider more intrinsic factors to provide a more accurate estimation of the effects of persuasive message content on message diffusion.
Second, our findings about the effects of PMFs were based on data from Twitter. In the United States, Twitter users are younger compared with the general public and users of some other social media sites, such as Facebook (Perrin and Anderson 2019; Wojcik and Hughes 2019). For example, about three-quarters (73%) of Twitter users are less than 50 years old (as compared with 54% of all U.S. adults) (Wojcik and Hughes 2019). Although Twitter users, in themselves, are an important audience of heat-related messages since even younger adults can be at risk of heat-related illnesses and deaths due to maladaptation (Hess et al. 2014; Mora et al. 2017), Twitter users are not representative of the elderly who are at greater risk from heat hazards. To benefit those who are less reachable via Twitter messages, especially the elderly, future research should examine the relationship between message diffusion on Twitter and message diffusion via other communication channels. For example, it is important to understand whether messaging strategies that improve message diffusion on Twitter also improve message diffusion via other channels, such as Facebook and word of mouth. It is also important to understand to what degree those who retweet a message on Twitter further share the information with non-Twitter users via other channels.
Although this study examined the effects of PMFs on message diffusion in the context of heat hazards, the five PMFs were originally designed for natural hazards in general, not limited to heat hazards. To be more applicable to different types of natural hazards beyond those that are primarily health threats, further studies could rename health risk susceptibility and health impact as impact susceptibility and impact severity. These two PMFs could then refer to not only the susceptibility and severity of health-related consequences but also the susceptibility and severity of other aspects of hazard impacts such as infrastructure impacts. Future studies should examine how these five PMFs influence message diffusion for other types of natural hazards such as floods and winter storms. In addition, scholars should continue research to understand the relationship between message persuasion and message diffusion in order to identify win-win communication practices in the context of natural hazards.
A wide variety of natural hazard events will continue to happen due to natural climate variability, with certain hazards like extreme heat being particularly exacerbated by anthropogenic climate changes (IPCC 2012). Effective risk communication about natural hazards is important to stimulate individual protective actions and thus reduce adverse impact on public health and property. To improve official risk messaging, this research empirically tested the influence of persuasive message content on message retransmission on Twitter in the context of heat hazards. We found that official tweets mentioning more types of persuasive message factors and mentioning hazard intensity were respectively associated with higher rates of message retransmission for heat-related tweets and its two subtypes, heat warning tweets and nonwarning tweets. Mentions of health risk susceptibility, health impact, and response instruction respectively demonstrated positive effects on message diffusion for some message types about heat hazards. Our findings could have implications for official risk messages about other types of natural hazards and for those disseminated through other channels such as Facebook and television to maximize message diffusion.
Acknowledgments
Funding for this research was provided in part by the National Science Foundation, Award SES-1459903 “Collaborative Research: Multi-Scale Modeling of Public Perceptions of Heat Wave Risk.”
Data availability statement
Data that support the findings of the paper are available in the Digital Commons at Utah State University (https://digitalcommons.usu.edu/all_datasets/140).
APPENDIX A
Descriptive Statistics of Predictors
Descriptive statistics of each type of tweet across grouping variables and group-level predictors are given in Table A1.
Descriptive statistics of predictors.
APPENDIX B
Checks about the Validity of Cumulative Effects
We checked whether the effects of the number of PMFs were dependent on a single influential PMF by conducting 12 additional models (for each PMF and tweet type) dropping tweets mentioning one of the four PMFs from one of three message types. The effects of the number of PMFs remained statistically significant, positive predictors for eight models, and the other four models were overfitted and not found to have statistically significant, cumulative effects. One of the four models used heat warning tweets removing those containing the PMF of response instruction, in which the cumulative effect approached significance, IRR = 1.25 [95% CI: 0.97–1.60], with p = 0.08. The other three models that did not pass the check used datasets dropping tweets containing the PMF of hazard intensity. Because tweets containing mentions of hazard intensity were disproportionately high in each original dataset, the remaining datasets after removing tweets mentioning hazard intensity did not have enough cases to check the cumulative effects. As an alternative, we modeled the number of PMFs for each original dataset without dropping any tweets and controlled for the variable of hazard intensity in addition to other control variables. For each of the alterative models, the number of PMFs was a statistically significant and positive predictor of retweet counts. Overall, we concluded that the effects of the number of PMFs were not driven by a single PMF across message types.
REFERENCES
Anderson, C. A., 1983: Imagination and expectation: The effect of imagining behavioral scripts on personal influences. J. Pers. Soc. Psychol., 45, 293–305, https://doi.org/10.1037/0022-3514.45.2.293.
Bassil, K. L., and D. C. Cole, 2010: Effectiveness of public health interventions in reducing morbidity and mortality during heat episodes: A structured review. Int. J. Environ. Res. Public Health, 7, 991–1001, https://doi.org/10.3390/ijerph7030991.
Bates, D., M. Mächler, B. Bolker, and S. Walker, 2015: Fitting linear mixed-effects models using lme4. J. Stat. Software, 67, 1–48, https://doi.org/10.18637/jss.v067.i01.
Casteel, M. A., 2016: Communicating increased risk: An empirical investigation of the National Weather Service’s impact-based warnings. Wea. Climate Soc., 8, 219–232, https://doi.org/10.1175/WCAS-D-15-0044.1.
Centers for Disease Control and Prevention, 2020: Underlying cause of death 1999-2018. CDC WONDER online database, accessed 13 May 2020, https://wonder.cdc.gov/ucd-icd10.html.
Cvijikj, I. P., and F. Michahelles, 2013: Online engagement factors on Facebook brand pages. Soc. Network Anal. Min., 3, 843–861, https://doi.org/10.1007/s13278-013-0098-8.
de Hoog, N., W. Stroebe, and J. B. F. De Wit, 2007: The impact of vulnerability to and severity of a health risk on processing and acceptance of fear-arousing communications: A meta-analysis. Rev. Gen. Psychol., 11, 258–285, https://doi.org/10.1037/1089-2680.11.3.258.
Demeritt, D., and S. Nobert, 2014: Models of best practice in flood risk communication and management. Environ. Hazards, 13, 313–328, https://doi.org/10.1080/17477891.2014.924897.
Faraway, J. J., 2016: Generalized linear models. Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models, Chapman and Hall/CRC Press, 151–173.
Floyd, D. L., S. Prentice-Dunn, and R. W. Rogers, 2000: A meta-analysis of research on protection motivation theory. J. Appl. Soc. Psychol., 30, 407–429, https://doi.org/10.1111/j.1559-1816.2000.tb02323.x.
Fox, J., and G. Monette, 1992: Generalized collinearity diagnostics. J. Amer. Stat. Assoc., 87, 178–183, https://doi.org/10.1080/01621459.1992.10475190.
Gelman, A., and J. Hill, 2006: Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press, 648 pp.
Hawkins, M. D., V. Brown, and J. Ferrell, 2017: Assessment of NOAA National Weather Service methods to warn for extreme heat events. Wea. Climate Soc., 9, 5–13, https://doi.org/10.1175/WCAS-D-15-0037.1.
Hess, J. J., S. Saha, and G. Luber, 2014: Summertime acute heat illness in US emergency departments from 2006 through 2010: Analysis of a nationally representative sample. Environ. Health Perspect., 122, 1209–1215, https://doi.org/10.1289/ehp.1306796.
Howe, P. D., J. R. Marlon, X. Wang, and A. Leiserowitz, 2019: Public perceptions of the health risks of extreme heat across US states, counties, and neighborhoods. Proc. Natl. Acad. Sci. USA, 116, 6743–6748, https://doi.org/10.1073/pnas.1813145116.
Hu, X., X. Zhang, and J. Wei, 2019: Public attention to natural hazard warnings on social media in China. Wea. Climate Soc., 11, 183–197, https://doi.org/10.1175/WCAS-D-17-0039.1.
Hughes, A. L., and L. Palen, 2012: The evolving role of the public information officer: An examination of social media in emergency management. J. Homeland Secur. Emerg. Manage., 9, 1547–7355, https://doi.org/10.1515/1547-7355.1976.
Hughes, A. L., L. A. A. St. Denis, L. Palen, and K. M. Anderson, 2014: Online public communications by police & fire services during the 2012 Hurricane Sandy. Proc. SIGCHI Conf. on Human Factors in Computing Systems, New York, NY, ACM, 1505–1514.
Iowa Environmental Mesonet, 2019: Archived NWS Watch/Warnings. Accessed 10 March 2019, https://mesonet.agron.iastate.edu/request/gis/watchwarn.phtml.
IPCC, 2012: Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation. Cambridge University Press, 582 pp., http://www.ipcc.ch/report/srex/.
Kalkstein, A. J., and S. C. Sheridan, 2007: The social impacts of the heat–health watch/warning system in Phoenix, Arizona: Assessing the perceived risk and response of the public. Int. J. Biometeor., 52, 43–55, https://doi.org/10.1007/s00484-006-0073-4.
Kovats, R. S., and S. Hajat, 2008: Heat stress and public health: A critical review. Annu. Rev. Public Health, 29, 41–55, https://doi.org/10.1146/annurev.publhealth.29.020907.090843.
Lachlan, K. A., Z. Xu, E. E. Hutter, R. Adam, and P. R. Spence, 2019: A little goes a long way: Serial transmission of Twitter content associated with Hurricane Irma and implications for crisis communication. J. Strategic Innovation Sustainability, 14, 16–26, https://doi.org/10.33423/jsis.v14i1.984.
Lebel, L., P. Lebel, B. Lebel, A. Uppanunchai, and C. Duangsuwan, 2018: The effects of tactical message inserts on risk communication with fish farmers in northern Thailand. Reg. Environ. Change, 18, 2471–2481, https://doi.org/10.1007/s10113-018-1367-x.
Leiss, W., 1996: Three phases in the evolution of risk communication practice. Ann. Amer. Acad. Pol. Soc. Sci., 545, 85–94, https://doi.org/10.1177/0002716296545001009.
Li, Y., A. L. Hughes, and P. D. Howe, 2018: Communicating crisis with persuasion: examining official Twitter messages on heat hazards. Proc. 15th Int. Conf. on Information Systems for Crisis Response and Management, Rochester, NY, Rochester Institute of Technology, 469–479.
Lin, X., K. A. Lachlan, and P. R. Spence, 2016a: Exploring extreme events on social media: A comparison of user reposting/retweeting behaviors on Twitter and Weibo. Comput. Human Behav., 65, 576–581, https://doi.org/10.1016/j.chb.2016.04.032.
Lin, X., P. R. Spence, T. L. Sellnow, and K. A. Lachlan, 2016b: Crisis communication, learning and responding: Best practices in social media. Comput. Human Behav., 65, 601–605, https://doi.org/10.1016/j.chb.2016.05.080.
Mileti, D. S., and J. H. Sorensen, 1990: Communication of emergency public warnings: A social science perspective and state-of-the-art assessment. Oak Ridge National Laboratory Tech. Rep. ORNL-6009, 160 pp., https://doi.org/10.2172/6137387.
Milne, S., P. Sheeran, and S. Orbell, 2000: Prediction and intervention in health-related behavior: A meta-analytic review of protection motivation theory. J. Appl. Soc. Psychol., 30, 106–143, https://doi.org/10.1111/j.1559-1816.2000.tb02308.x.
Mora, C., C. W. W. Counsell, C. R. Bielecki, and L. V. Louis, 2017: Twenty-seven ways a heat wave can kill you: Deadly heat in the era of climate change. Circ. Cardiovasc. Qual. Outcomes, 10, e004233, https://doi.org/10.1161/CIRCOUTCOMES.117.004233.
Morss, R. E., J. L. Demuth, J. K. Lazo, K. Dickinson, H. Lazrus, and B. H. Morrow, 2016: Understanding public hurricane evacuation decisions and responses to forecast and warning messages. Wea. Forecasting, 31, 395–417, https://doi.org/10.1175/WAF-D-15-0066.1.
Morss, R. E., C. L. Cuite, J. L. Demuth, W. K. Hallman, and R. L. Shwom, 2018: Is storm surge scary? The influence of hazard, impact, and fear-based messages and individual differences on responses to hurricane risks in the USA. Int. J. Disaster Risk Reduct., 30, 44–58, https://doi.org/10.1016/j.ijdrr.2018.01.023.
Palen, L., and A. L. Hughes, 2018: Social media in disaster communication. Handbook of Disaster Research, H. Rodríguez, W. Donner, and J. E. Trainor, Eds., Springer, 497–518.
Perrin, A., and M. Anderson, 2019: Share of U.S. adults using social media, including Facebook, is mostly unchanged since 2018. Pew Research Center, accessed 3 October 2020, https://www.pewresearch.org/fact-tank/2019/04/10/share-of-u-s-adults-using-social-media-including-facebook-is-mostly-unchanged-since-2018/.
Potter, S. H., P. V. Kreft, P. Milojev, C. Noble, B. Montz, A. Dhellemmes, R. J. Woods, and S. Gauden-Ing, 2018: The influence of impact-based severe weather warnings on risk perceptions and intended protective actions. Int. J. Disaster Risk Reduct., 30, 34–43, https://doi.org/10.1016/j.ijdrr.2018.03.031.
PRISM Climate Group, 2018: PRISM Climate Data. Oregon State University, accessed 15 October 2018, http://prism.oregonstate.edu.
Schenker, N., and J. F. Gentleman, 2001: On judging the significance of differences by examining the overlap between confidence intervals. Amer. Stat., 55, 182–186, https://doi.org/10.1198/000313001317097960.
Schoessow, F. S., 2018: Humans as sensors: The influence of extreme heat vulnerability factors on risk perceptions across the contiguous United States. M.S. thesis, Dept. of Environment and Society, Utah State University, 153 pp., https://digitalcommons.usu.edu/etd/7094.
Scott, K. K., and N. A. Errett, 2018: Content, accessibility, and dissemination of disaster information via social media during the 2016 Louisiana floods. J. Public Health Manage. Pract., 24, 370–379, https://doi.org/10.1097/PHH.0000000000000708.
Sheridan, S. C., 2007: A survey of public perception and response to heat warnings across four North American cities: An evaluation of municipal effectiveness. Int. J. Biometeor., 52, 3–15, https://doi.org/10.1007/s00484-006-0052-9.
Singleton, R. A., and B. C. Straits, 2010: Approaches to Social Research. 5th ed. Oxford University Press, 672 pp.
St. Denis, L. A., L. Palen, and K. M. Anderson, 2014: Mastering social media: An analysis of Jefferson County’s communications during the 2013 Colorado floods. Proc. Information Systems for Crisis Response and Management Conf., University Park, PA, ISCRAM, 737–746.
Sutton, J., and E. D. Kuligowski, 2019: Alerts and warnings on short messaging channels: Guidance from an expert panel process. Nat. Hazards Rev., 20, 04019002, https://doi.org/10.1061/(ASCE)NH.1527-6996.0000324.
Sutton, J., E. S. Spiro, B. Johnson, S. Fitzhugh, C. Ben Gibson, and C. T. Butts, 2014: Warning tweets: Serial transmission of messages during the warning phase of a disaster event. Inf. Commun. Soc., 17, 765–787, https://doi.org/10.1080/1369118X.2013.862561.
Sutton, J., C. Ben Gibson, N. E. Phillips, E. S. Spiro, C. League, B. Johnson, S. M. Fitzhugh, and C. T. Butts, 2015: A cross-hazard analysis of terse message retransmission on Twitter. Proc. Natl. Acad. Sci. USA, 112, 14 793–14 798, https://doi.org/10.1073/pnas.1508916112.
Sutton, J., S. C. Vos, M. M. Wood, and M. Turner, 2018: Designing effective tsunami messages: Examining the role of short messages and fear in warning response. Wea. Climate Soc., 10, 75–87, https://doi.org/10.1175/WCAS-D-17-0032.1.
Syed, R., M. Rahafrooz, and J. M. Keisler, 2018: What it takes to get retweeted: An analysis of software vulnerability messages. Comput. Human Behav., 80, 207–215, https://doi.org/10.1016/j.chb.2017.11.024.
Tannenbaum, M. B., J. Hepler, R. S. Zimmerman, L. Saul, S. Jacobs, K. Wilson, and D. Albarracín, 2015: Appealing to fear: A meta-analysis of fear appeal effectiveness and theories. Psychol. Bull., 141, 1178–1204, https://doi.org/10.1037/a0039729.
U.S. Census Bureau, 2017: Annual estimates of the resident population: April 1, 2010 to July 1, 2016. Accessed 15 October 2018, https://www2.census.gov/programs-surveys/popest/tables/2010-2016/counties/totals/.
Vos, S. C., J. Sutton, Y. Yu, S. L. Renshaw, M. K. Olson, C. Ben Gibson, and C. T. Butts, 2018: Retweeting risk communication: The role of threat and efficacy. Risk Anal., 38, 2580–2598, https://doi.org/10.1111/risa.13140.
Walker, L., P. R. Baines, R. Dimitriu, and E. K. Macdonald, 2017: Antecedents of retweeting in a (political) marketing context. Psychol. Mark., 34, 275–293, https://doi.org/10.1002/mar.20988.
Wang, W.-J., T. W. Haase, and C.-H. Yang, 2020: Warning message elements and retweet counts: An analysis of tweets sent during Hurricane Irma. Nat. Hazards Rev., 21, 4019014, https://doi.org/10.1061/(ASCE)NH.1527-6996.0000351.
Wilson, E. J., and D. L. Sherrell, 1993: Source effects in communication and persuasion research: A meta-analysis of effect size. J. Acad. Mark. Sci., 21, 101–112, https://doi.org/10.1007/BF02894421.
Witte, K., 1992: Putting the fear back into fear appeals: The extended parallel process model. Commun. Monogr., 59, 329–349, https://doi.org/10.1080/03637759209376276.
Witte, K., and M. Allen, 2000: A meta-analysis of fear appeals: Implications for effective public health campaigns. Health Educ. Behav., 27, 591–615, https://doi.org/10.1177/109019810002700506.
Wojcik, S., and A. Hughes, 2019: Sizing up Twitter users. Pew Research Center, accessed 2 March 2020, https://www.pewresearch.org/internet/2019/04/24/sizing-up-twitter-users/.
Wong-Parodi, G., B. Fischhoff, and B. Strauss, 2018: Effect of risk and protective decision aids on flood preparation in vulnerable communities. Wea. Climate Soc., 10, 401–417, https://doi.org/10.1175/WCAS-D-17-0069.1.
Yang, Q., C. Tufts, L. Ungar, S. Guntuku, and R. Merchant, 2018: To retweet or not to retweet: Understanding what features of cardiovascular tweets influence their retransmission. J. Health Commun., 23, 1026–1035, https://doi.org/10.1080/10810730.2018.1540671.
Zhu, J., F. Xiong, D. Piao, Y. Liu, and Y. Zhang, 2011: Statistically modeling the effectiveness of disaster information in social media. 2011 IEEE Global Humanitarian Technology Conf., Seattle, WA, IEEE, 431–436.