1. Introduction
There is little doubt that perceptions of daily temperature deviations can influence individuals’ global warming beliefs. When people perceive the day’s local temperature to be warmer than usual, they tend to overestimate the number of warm days throughout the past year, and to report increased belief in and concern about global warming. This local warming effect has been documented with numerous operationalizations, across multiple populations, and at different times (Joireman et al. 2010; Li et al. 2011; Risen and Critcher 2011; Egan and Mullin 2012; Lewandowski et al. 2012; Zaval et al. 2014).
The local warming effect may not always occur, however. For example, Druckman (2015) presents suggestive evidence that the effect may disappear when people receive a reminder to think about temperature patterns over time. Druckman’s results show that prompting people to consider weather fluctuations over time can sever the connection between perceptions of the present day’s temperature deviation and both impressions of the last year’s temperature trends and global warming beliefs. However, Druckman conducted his study on a young sample at a single location, on an uncharacteristically warm day, following a near record-cold winter. Thus, many questions remain. Just how generalizable is this corrective effect? Does the occurrence of the local warming effect vary based on individual differences? Does the impact of a corrective prompt sustain over time?
This paper presents an experimental study that addresses each of these questions. It first presents data that retest the impact of the corrective prompt, with a more heterogeneous sample across multiple locations, and with respect to an additional dependent variable beyond belief in and concern about global warming—specifically, beliefs about the role of humans in causing global warming (see, e.g., Hamilton and Stampone 2013). The expectation is that the prompt will have the same corrective impact on this additional measure. Indeed, the psychological process underlying Druckman’s (2015) findings should also occur here. Without the prompt, individuals tend to substitute readily available direct sensory experience (i.e., perceived daily temperature fluctuations) for more diagnostic but less accessible information (i.e., temperature trends over time)—a pattern of behavior similar to the “end-heuristic” observed by Healy and Lenz (2014). In other words, people tend to engage in attribute substitution (see Kahneman and Frederick 2002). The prompt makes temperature patterns over time more accessible, meaning people do not rely on perceptions of today’s temperature deviation in forming their global warming beliefs. The prediction then is as follows: relative to people who do not receive a prompt to consider temperature patterns over time, people who receive such a prompt will be significantly less likely to base their global warming attitudes on their perceptions of today’s temperature deviation, with all else constant (hypothesis 1).
Second, this paper presents a test of whether the local warming effect varies based on individual differences. Past work suggests that less intelligent or cognitively able individuals are more likely to rely on attribute substitution since they typically lack the knowledge base and motivation to think through longer-term patterns (Stanovich and West 2002; Egan and Mullin 2012): there is “generally a negative correlation between … measures of intelligence and susceptibility to judgment biases” (Kahneman and Frederick 2002, p. 68). The prediction is thus that the local warming effect (which entails using the end-heuristic via reliance on perceptions of today’s temperature deviation) will occur to a significantly greater extent among less cognitively able individuals, with all else constant (hypothesis 2).
Finally, this paper presents results regarding whether the corrective prompt endures over time, continuing to eliminate the local warming effect even without re-exposure. The expectation is that there will be durability over time of the corrective prompt because it should stimulate more elaborative thinking as people search their memories for weather assessments rather than rely on a simple attribution substitution. Such thinking is what minimizes the effects of “more superficial, cue-driven processes” such as the end-heuristic (Visser et al. 2006, p. 5). More generally, “when people [form] elaborated attitudes … their attitudes [are] more likely to persist” (Erber et al. 1995, p. 436). The prediction then is that, relative to those who do not receive a prompt, those who receive a corrective prompt will demonstrate stability in their initial attitudes, and will be significantly less susceptible to the local warming effect (i.e., reliance on perceptions of today’s temperature deviation) a week after receiving the initial prompt, with all else constant (hypothesis 3).
2. Experimental design and procedure
Participants (n = 307) were recruited via Amazon’s Mechanical Turk (MTurk; https://www.mturk.com/mturk/welcome), an online labor market utilized by an increasing number of survey researchers (Buhrmester et al. 2011). MTurk represents an improvement over student-based samples typically available to social scientists in that MTurk samples are fairly heterogeneous and more closely representative of the U.S. population as a whole (Berinsky et al. 2012). Mullinix et al. (2015), in fact, show that the modal social science experiment done on a probability population sample replicates on MTurk. Moreover, MTurk is a noted improvement over Druckman’s (2015) sample that largely consisted of students living in one location (e.g., the respondents here came from a total of 44 different states). It also is the same approach used by Zaval et al.’s (2014) investigation of the local warming effect (for three of their studies). The first surveys described in this paper were conducted on 15 December 2014; each respondent received $0.50 for participating.
Participants were randomly assigned to a control (no prompt) or treatment (prompt) condition. Following Zaval et al. (2014) and Druckman (2015), participants were asked 1) to assess whether the day’s local temperature was warmer or colder than usual for the time of year (TT; with 1 being much colder; 2, somewhat colder; 3, about the same; 4, somewhat warmer; and 5, much warmer); 2) to report what percentage of days over the past year seemed to be warmer than usual compared with the historical average (PDW); 3) to indicate how convinced they are that global warming is happening (GWB; on a four-point scale from not at all convinced to completely convinced); and 4) to indicate how worried they are about global warming (GWW; on a four-point scale from not at all worried to a great deal worried). Extending previous work, participants also were asked about whether they think global warming is naturally occurring or is the result of human activities [GWH; on a seven-point scale, which after an adjustment (see appendix A) ranged from 1 (definitely naturally induced) to 7 (definitely human induced)]. Question wordings are provided in appendix A. In what follows, unless otherwise noted, TT or today’s temperature deviation refers to perceptions of temperature deviations rather than an objective deviation in actual temperatures. The same is true for PDW.
The survey additionally asked each respondent about their demographic characteristics, political ideology, environmental/economic attitudes, and cognitive ability. Specifically, respondents reported their age, income, education, and gender, and located themselves on a seven-point ideology scale ranging from “very liberal” (a score of 1) to “very conservative” (a score of 7). Respondents also reported their environmental/economic attitude in terms of preferences for protecting the environment (a low score of 1) versus maintaining economic growth (a high score of 7). There are not clear directional predictions for all of these control variables; however, prior work suggests that ideology (becoming more conservative) and environmental/economic attitudes (moving toward a preference for economic growth) should have negative effects on all global warming beliefs (e.g., McCright and Dunlap 2011; Marquart-Pyatt et al. 2014; Bolsen et al. 2015).
Respondents’ cognitive ability was assessed using a political knowledge battery that included four items (Cronbach’s alpha = 0.63) (Delli Carpini and Keeter 1996). Others have shown that such a measure can serve as a proxy for intelligence or ability. For example, Rasmussen (2016, p. 7) explains, “Research demonstrates that people who are more intelligent are also more politically knowledgeable[.]” This measure, which may have the advantage of being a domain specific ability proxy, will be used to test the expectation that the local warming effect largely occurs among less cognitively able individuals. A more general cognitive ability measure was not included; future work would benefit from comparing distinct ability measures. Question wordings and scales for these measures also are available in appendix A; this appendix lists all of the questions in the order they were provided to respondents.1
Finally, objective temperature and objective temperature deviations were collected for each respondent’s location; these variables allow for several robustness checks. Appendix B describes how these data were collected and details the robustness check results. These checks rule out the possibility that it is objective conditions that drive the local warming effect, rather than the posited perceptions.
The treatment (prompt) condition (n = 154) differed from the control (no prompt) (n = 153) in only one way. Specifically, as in Druckman (2015), before treatment participants were asked to assess temperature trends over the past year (PDW), they were prompted with the following reminder: “When thinking about temperatures over the last year, remember not only the feeling of today but also how you felt throughout last winter, spring, and summer—when temperatures were different.” Finally, all respondents were contacted 7 days after the initial survey (on 22 December 2014) and asked to participate in another survey that again asked the same series of questions (TT, PDW, GWB, GWW, and GWH).2 In the follow-up, respondents in both experimental groups received the same questions; the prompt was not introduced again for the treatment group. Respondents received $2.00 for completing the follow-up; roughly half of the initial respondents (52% of control respondents and 53% of treatment respondents) accepted the invitation, with 80 control group and 81 treatment group respondents taking part.
To be clear, the experimental approach used here differs from that used by Zaval et al. (2014). Their fourth study (the one most similar to what is presented here) uses an observational approach to explore the existence of the local warming effect. This paper focuses on the impact of the prompt, which means that the key tests entail comparisons across the randomly assigned experimental groups. Thus, even though responses to the measures varied through the sample, given that respondents lived in a host of locations, random assignment to the control (no prompt) or treatment (prompt) condition means that on average the two groups were the same. Consequently, controlling for other variables (including actual rather than perceived temperature deviations; however, see appendix B) is not necessary since the groups should be comparable, on average, other than exposure to the prompt (see Shadish et al. 2002).3 Any differences between experimental groups can be confidently attributed to the prompt.
The control (no prompt) condition should display a similar local warming effect as previous studies (i.e., Zaval et al. 2014; Druckman 2015). That is, among control (no prompt) respondents, perceptions of today’s temperature (TT) should influence perceptions of the percentage of warmer-than-normal days over the past year (PDW), as well as global warming belief (GWB), worry (GWW), and the extent to which respondents believe that global warming is the result of human activities (GWH). In contrast, treated (prompt) respondents should display a significantly smaller or no connection between TT and PDW or the global warming variables (hypothesis 1). The impact of cognitive ability among respondents in the control (the no-prompt condition, where the local warming effect is expected to occur) is explored by assessing whether the effect is significantly larger for those who are less able, as measured by the aforementioned four-item knowledge battery (hypothesis 2). Finally, the impact over time of the prompt is investigated by comparing experimental groups using the follow-up survey data (hypothesis 3).
3. Results and discussion
The first prediction to test is whether the prompt vitiates or severs the connection between TT and the other main variables: PDW and the global warming beliefs. Note that one-tailed tests are used, given the clear directional nature of the hypotheses (Blalock 1979, p. 163). Table 1 presents the results. The first two rows display average scores for the given measures, while the bottom four rows present relevant correlations. The table shows that TT is not significantly different by condition, confirming the success of random assignment.4 On average, respondents in both conditions reported that the present day’s temperature was higher than usual (the midpoint of the scale is 3—so responses above this value indicate warmer than normal temperatures).5 It is not surprising, then, that PDW is significantly higher among control (no prompt) respondents: while both groups perceived the present day to be warmer than usual, only those in the control group made the connection between TT and PDW, leading them to relatively higher estimates of the number of such days over the past year (28.84 vs 24.88).
The impact of the corrective prompt (s.d. = standard deviation; *** = p ≤ 0.01; ** = p ≤ 0.05; * = p ≤ 0.10). Given the directional nature of the hypotheses, all tests of statistical significance are one-tailed. Correlational tests are Pearson’s R. Comparison tests (i.e., in the first two rows) are t tests.
This is further evidenced by the marginally significant correlation (r = 0.15, p = 0.059, one-tailed test) between TT and PDW among control (no prompt) respondents. Consistent with hypothesis 1, TT and PDW are uncorrelated among treated respondents.6 Moreover, as predicted, TT is correlated with each of the global warming measures (GWB, GWW, and GWH) for the control group, but not the treatment group.7 The prompt did not just vitiate the impact of TT; in the case of these data, it eliminated the effect.
Altogether, this is evidence that Druckman’s (2015) prompt generalizes to a broader population at a distinct time. It also extends past work by exploring the local warming effect’s absence or presence when it comes to beliefs about the role of human action in affecting global warming (GWH).
Hypothesis 2 predicts that the local warming effect occurs to a greater extent among less cognitively able individuals. Testing this possibility entails focusing on respondents in the control (no prompt) condition where the local warming effect occurred.8 To do so, each dependent variable is regressed on TT, PDW, demographic and ideological controls (i.e., age, education, gender, income, ideology, and environmental/economic attitudes), cognitive ability, and an interaction of cognitive ability with temperature (TT).9 The results are displayed in Table 2.
Cognitive ability (Cognitive ability TT = cognitive ability and temperature interactions among no prompt respondents; *** p ≤ 0.01; ** p ≤ 0.05; * p ≤ 0.10). These models were estimated via ordinary least squares (OLS) regression with the entries being unstandardized coefficients along with standard errors in parentheses. Given the directional nature of the hypotheses, tests of statistical significance are one-tailed for all variables other than age, education, gender, and income where two-tailed tests are employed.
One thing to note, prior to discussing the results for hypothesis 2, is that the results presented here differ from prior work in one regard. As Table 2 shows, the main TT effect remains significant. Zaval et al. (2014) and Druckman (2015) suggest a mediational argument such that TT works through PDW (i.e., TT affects PDW, which in turn affects the global warming beliefs). The continued significance of TT here suggests that the effects are not entirely mediated through PDW (e.g., Baron and Kenny 1986). The experience of the hot day likely affects other mediators that were unmeasured (e.g., Leiserowitz 2006). For example, TT may influence global warming attitudes via risk assessments (e.g., van der Linden 2015). Alternatively, particularly warm days may cause individuals to worry more about global warming consequences on public health, environmental degradation, or the local ecology, which in turn could generate increased belief in and concern about global warming (see, e.g., Corbett and Durfee 2004; Scannell and Gifford 2013; Weathers and Kendall 2015; Wiest et al. 2015). This paper leaves it to future work to more concretely identify additional mediators.10 For the present purposes, the more important point is that because TT remains significant, the analysis tests hypothesis 2 by interacting cognitive ability with TT (rather than with PDW).
Table 2, consistent with hypothesis 2, shows both a main effect for TT and a significant negative effect of the interaction term across all three models (although it is marginally significant in the GWB model)11—suggesting that the local warming effect is indeed larger for those with lower levels of cognitive ability (see footnote 12 on comparisons with prior work).12 For example, we find that while a one-unit increase in today’s temperature produces a 0.34-unit increase (more than 30% of a standard deviation) in global warming belief (GWB) among the least cognitively able control (no prompt) respondents, the same temperature increase among the most cognitively able yields a change of just −0.007, indistinguishable from zero. A similar pattern appears for the other two dependent variables.13,14
As mentioned, ideology (moving in a conservative direction) and environmental/economic attitudes (moving toward a greater preference for economic growth over environmental protection) were predicted to have significant and negative effects on the global warming variables. This is what was found, using one-tailed tests (given the directional predictions). Fewer a priori expectations existed for the other control variables and thus for those two-tailed tests of significance are used. The results show that age had a significant negative effect on GWH. Cognitive ability also had a significant positive main effect on all variables, although only marginally so for GWB (0.1 level).
The final question to explore is whether the effects of the prompt endure over time. Given the results reviewed above, the prompt seems like a promising means for decoupling the public’s attitudes about global warming from a heuristic (perceived deviation in today’s temperature), especially in light of its robustness to a larger and more diverse population. However, another important matter to consider is the persistence of the treatment effect. If the effects of the prompt quickly dissipate, it may have little value beyond simply improving one-off survey responses. Yet, if the effects of the prompt are more long-lasting, as predicted by hypothesis 3, this would constitute strong evidence of its rhetorical utility for scientists and educators seeking to communicate with the public about global warming.15
As noted above, respondents were recontacted seven days after the initial survey and asked to complete a short follow-up questionnaire.16 Specifically, they were again asked about the present day’s temperature (i.e., on the day of the follow-up) and the percentage of warm days over the past year, in addition to each of the three global warming variables (GWB, GWW, and GWH). For each of the time 2 (T2) models, all political and demographic control variables were excluded because these measures are captured by the dependent variables from time 1 (T1), which are included as controls in the time 2 (T2) regressions. In other words, since the control variables already influenced the initial measures, which are present in these models, there is no need to include them a second time. Tables 3 and 4 present the longevity results.
Persistence of the prompt’s effects over time among no prompt respondents (***p ≤ 0.01; **p ≤ 0.05; *p ≤ 0.10). These models were estimated via OLS regression with the entries being unstandardized coefficients along with standard errors in parentheses. Given the directional nature of the hypotheses, all tests of statistical significance are one-tailed.
Persistence of the prompt’s effects over time among prompted respondents (***p ≤ 0.01; **p ≤ 0.05; *p ≤ 0.10). These models were estimated via OLS regression with the entries being unstandardized coefficients along with standard errors in parentheses. Given the directional nature of the hypotheses, all tests of statistical significance are one-tailed.
Table 3 shows T2 observations for control (no prompt) respondents. Clearly, these results match the findings for control respondents at T1: perceptions of today’s temperature deviation exert a significant influence on all three global warming variables, although significance is marginal for GWH. In short, the local warming effect appears again among control (no prompt) respondents at T2. Additionally, each dependent measure from T1 exerts a sizeable impact on the same measure at T2.
Table 4 shows that treated (prompt) respondents show no signs of the local warming effect at T2—even though the treatment prompt was not readministered. The corrective prompt eliminates the local warming effect at least as much as seven days after the fact, as was predicted with hypothesis 3.17,18 This suggests that even though global warming attitudes may be subject to the influence of such fleeting and arbitrary factors as the present day’s weather, it may be relatively easy to apply an enduring correction to these influences.
As previously intimated, each of the substantive findings described here, as well as the conclusions drawn from them, holds up when accounting for objective temperatures and objective temperature deviations in the models. For a detailed discussion of this, see appendix B.
4. Conclusions
Recent research has shown that perceived short-term local temperature fluctuations can exert undue influence on global warming beliefs. However, one recent study (Druckman 2015) shows a way to successfully counteract it. This paper built on this prior work by demonstrating that the correction is robust in a broader and more diverse sample: the prompt not only neutralizes the local warming effect for students drawn from a single location, but also for adults sampled from across the country. The results also reveal that the prompt’s corrective impact extends to an additional global warming attitude: along with belief in and concern about global warming, the prompt also eliminates the local warming effect with respect to the belief that global warming is human-induced. Moreover, the results indicate that cognitive ability is a possible moderator of the local warming effect. Local daily temperature fluctuations seem to have a significantly greater impact among the less cognitively able. Finally, the results show that the correction can persist for at least one week afterward.
While the findings illuminate the conditions and extent of both the local warming effect and a correction to it, future work might continue to explore other aspects of these phenomena. For instance, in addition to further replication studies across populations and time, scholars might consider whether the correction persists longer than one week, and whether colder-than-normal and warmer-than-normal temperatures are equally “correctable.” Furthermore, researchers have explored the impact of other weather events besides temperature on global warming attitudes and beliefs—for instance, rainfall, coastal erosion, and tropical storms (e.g., Goebbert et al. 2012; Taylor et al. 2014). When other kinds of short-term local weather fluctuations influence global warming beliefs, scholars might ask whether these effects can also be corrected in a similar fashion. Additionally, others might use more general measures of intelligence or ability, rather than the domain-specific political knowledge proxy used here, to assess the role played by cognitive ability in producing the local warming effect.
What are the normative implications? On the one hand, those who are concerned by the local warming effect may be heartened by the finding that a simple admonition to keep in mind less immediate considerations can eliminate the effect. From this perspective, science communicators may want to add the prompt to their rhetorical toolbox when communicating with the public about global warming. Moreover, the prompt may steer people to more of a reliance on PDW, and some suggest that PDW is fairly accurate objectively: “individuals who live in places with rising average temperatures are more likely than others to [correctly] perceive local warming” (Howe et al. 2013, p. 352). On the other hand, priming people to rely on their perceptions of annual weather trends may not be ideal. Such perceptions are rarely entirely accurate and can be skewed by one’s ideology or world view (Goebbert et al. 2012); moreover, any given year can have unusual and unrepresentative weather that could bias opinions. The solution may be to prime people to think about feelings over even longer periods of time, but whether such an approach would work remains unclear. The findings here represent a step forward in understanding the conditionality of the local warming effect. Yet, much remains to be done to further isolate how weather events affect global warming attitudes and to explore the normative significance of such effects.
Acknowledgments
We thank Ethan Busby and Adam Howat for assistance.
APPENDIX A
Wording and Scales for Survey Questions
LocTemp
Is the local temperature today colder or warmer than usual for this time of year?
WarmPercent
Over the past year, what percentage of days seemed to be warmer than usual for that time of year, compared to historical average? [100-point scale anchored by 0%, 50%, and 100%]
GWHappen
How convinced are you that global warming is happening?
GWWorry
How personally worried are you about global warming?
GWHuman
If global warming is happening, to what extent do you think it is caused by human activities, as opposed to natural changes in the environment? (If you believe that global warming is clearly not happening, you can leave this answer blank.) [Authors’ note: We flipped this variable so higher values move toward “definitely human induced.”]
Ideo
Which point on this scale best describes your political views?
EconEnviron
In general, what do you think is more important: protecting the environment, even at the risk of curbing economic growth, OR maintaining a prosperous economy, even if the environment suffers to some extent?
Gender
Are you male or female?
Educate
What is your highest level of education?
Age
What is your age?
Income
What is your estimate of your family’s annual household income (before taxes)?
Cognitive ability (political knowledge) battery:
Many people do not know the answers to the next set of questions, so if you do not know the answer, just leave it blank or check “don’t know.”
Veto
How much of a majority is required for the U.S. Senate and House to override a Presidential veto?
CorrectVeto (1 = 2/3, 0 = anything else)
House
Do you happen to know which party currently has the most members in the House of Representatives in Washington D.C.?
CorrectHouse (1 = Republicans, 0 = anything else)
Constitution
Whose responsibility is it to determine if a law is constitutional?
CorrectConstitution (1 = Supreme Court, 0 = anything else)
Sec State
Who is the current U.S. Secretary of State? Enter your response or write “don’t know” in the space below.
CorrectSecState (1 = John Kerry, 0 = anything else)
APPENDIX B
Objective Temperature Data
Data on objective daily temperatures and objective temperature deviations were collected from the Automated Surface Observing System (ASOS), a climatological observing network maintained by the National Weather Service (NWS), the Federal Aviation Administration, and the Department of Defense. This system consists of hundreds of automated weather stations located primarily at airports around the country. The historical weather archive maintained by Weather Underground, Inc. (https://www.wunderground.com/history/) was used to collect objective temperature data for each respondent. Specifically, participants’ zip codes were used to query historical temperature data for their locations. The Weather Underground system returned temperature data from the NWS ASOS weather station closest to each zip code. Data on the daily high and low temperatures from each zip code for the day that each respondent completed the survey were collected. The historical average high and low temperatures were also collected. All time 1 (T1) surveys were completed on 15 December 2014; the majority of time 2 (T2) surveys were completed one week later on 22 December 2014 (81% of T2 respondents completed the second wave of the survey on 22 December; remaining respondents completed the survey no later than 28 December).
The average temperature during the first wave of the survey (T1) was 47.7°F (standard deviation = 10.2); the mean deviation from the historical average was 5.3°F (standard deviation = 6.8). The average temperature during the second wave of the survey (T2) was 47.3°F (standard deviation = 13.1); the mean deviation from the historical average was 6.5°F (standard deviation = 5.8).
Altogether, the substantive results of the analyses are unchanged when including objective measures of temperature and temperature deviations, and accord with previous findings (e.g., Zaval et al. 2014). These data were used in three ways. First, the correlations among the present day’s objective temperature, objective temperature deviation, perceived temperature deviation (which is referred to in-text as TT), and the perceived number of warmer-than-average days over the past year (PDW) were examined. Both PDW and TT are uncorrelated with objective measures of the temperature at T1 and T2. However, TT is significantly correlated with objective measures of temperature deviations at T1 (r = 0.431, p < 0.001, two-tailed test), and TT measured at T2 (TT2) is significantly correlated with objective temperature deviations at T2 (r = 0.2089, p < 0.01, two-tailed test). The relationship between TT (which asks, “Is the local temperature today colder or warmer than usual for this time of year?”) and objective temperature deviations suggests that respondents did indeed attend to actual weather patterns in formulating their assessments.
Second, all of the models were rerun using today’s actual temperature and then today’s actual temperature deviation instead of perceived temperature deviation. When used in this way, neither actual temperature nor actual temperature deviations consistently influence global warming beliefs either at the time of the initial survey or during the follow-up wave—a result that comports with Zaval et al.’s (2014, 145–146) analysis showing that it is “attention to and perception of today’s temperature, and not actual temperature deviation” that affects recall of past temperature events such as weather patterns over the past year. Note, however, that objective temperature deviations do produce the local warming effect in a single instance—specifically, with respect to worry about global warming (GWW) at time 1 among control (no prompt) respondents—and that the prompt serves to correct this effect, as well.
Third, both variables were added as controls to the regressions. Even when controlling for the objective temperatures and objective deviations, 1) the local warming effect still appears among control (no prompt) respondents, with perceived temperature deviations still significantly influencing global warming beliefs; 2) the same interaction effect between cognitive ability and today’s perceived temperature deviation appears; and 3) the local warming effect still disappears among respondents who received the prompt at both T1 and T2.
Detailed results for each of the analyses described above are available from the authors upon request.
REFERENCES
Baden, C., and Lecheler S. , 2012: Fleeting, fading, or far-reaching? A knowledge-based model of the persistence of framing effects. Commun. Theory, 22, 359–382, doi:10.1111/j.1468-2885.2012.01413.x.
Baron, R. M., and Kenny D. A. , 1986: The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. J. Pers. Soc. Psychol., 51, 1173–1182, doi:10.1037/0022-3514.51.6.1173.
Berinsky, A. J., Huber G. A. , and Lenz G. S. , 2012: Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Polit. Anal., 20, 351–368, doi:10.1093/pan/mpr057.
Blalock, H. M., Jr., 1979: Social Statistics. 2nd ed. McGraw-Hill, 592 pp.
Bolsen, T., Druckman J. N. , and Cook F. L. , 2015: Citizens’, scientists’, and legislators’ beliefs about global climate change. Ann. Amer. Acad. Pol. Soc. Sci., 658, 271–295, doi:10.1177/0002716214558393.
Buhrmester, M., Kwang T. , and Gosling S. D. , 2011: Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspect. Psychol. Sci., 6, 3–5, doi:10.1177/1745691610393980.
Bullock, J. G., and Ha S. E. , 2011: Mediation analysis is harder than it looks. Cambridge Handbook of Experimental Political Science, J. N. Druckman et al., Eds., Cambridge University Press, 508–521.
Corbett, J. B., and Durfee J. L. , 2004: Testing public (un)certainty of science: Media representations of global warming. Sci. Commun., 26, 129–151, doi:10.1177/1075547004270234.
Delli Carpini, M. X., and Keeter S. , 1996: What Americans Know about Politics and Why It Matters. Yale University Press, 416 pp.
Druckman, J. N., 2015: Eliminating the local warming effect. Nat. Climate Change, 5, 176–177, doi:10.1038/nclimate2536.
Egan, P. J., and Mullin M. , 2012: Turning personal experience into political attitudes: The effect of local weather on Americans’ perceptions about global warming. J. Polit., 74, 796–809, doi:10.1017/S0022381612000448.
Erber, M. W., Hodges S. D. , and Wilson T. D. , 1995: Attitude strength, attitude stability, and the effects of analyzing reasons. Attitude Strength: Antecedents and Consequences, R. E. Petty and J. A. Krosnick, Eds., Erlbaum, 433–454.
Goebbert, K., Jenkins-Smith H. C. , Klockow K. , Nowlin M. C. , and Silva C. L. , 2012: Weather, climate, and worldviews: The sources and consequences of public perceptions of changes in local weather patterns. Wea. Climate Soc., 4, 132–144, doi:10.1175/WCAS-D-11-00044.1.
Hamilton, L. C., and Stampone M. D. , 2013: Blowin’ in the wind: Short-term weather and belief in anthropogenic climate change. Wea. Climate Soc., 5, 112–119, doi:10.1175/WCAS-D-12-00048.1.
Healy, A., and Lenz G. S. , 2014: Substituting the end for the whole: Why voters respond primarily to the election-year economy. Amer. J. Pol. Sci., 58, 31–47, doi:10.1111/ajps.12053.
Howe, P. D., Markowitz E. M. , Lee T. M. , Ko C. , and Leiserowitz A. , 2013: Global perceptions of local temperature change. Nat. Climate Change, 3, 352–356, doi:10.1038/nclimate1768.
Joireman, J., Truelove H. B. , and Duell B. , 2010: Effect of outdoor temperature, heat primes and anchoring on belief in global warming. J. Environ. Psychol., 30, 358–367, doi:10.1016/j.jenvp.2010.03.004.
Kahneman, D., and Frederick S. , 2002: Representativeness revisited: Attribute substitution in intuitive judgment. Heuristics and Biases: The Psychology of Intuitive Judgment, A. Gilovich, D. Griffin, and D. Kahneman, Eds., Cambridge University Press, 49–81.
Lecheler, S., and de Vreese C. H. , 2011: Getting real: The duration of framing effects. J. Commun., 61, 959–983, doi:10.1111/j.1460-2466.2011.01580.x.
Leiserowitz, A., 2006: Climate change risk perception and policy preferences: The role of affect, imagery, and values. Climatic Change, 77, 45–72, doi:10.1007/s10584-006-9059-9.
Levay, K. E., Freese J. , and Druckman J. N. , 2016: The demographic and political composition of Mechanical Turk samples. SAGE Open, 6, 1–17, doi:10.1177/2158244016636433.
Lewandowski, G. W., Ciarocco N. J. , and Gately E. L. , 2012: The effect of embodied temperature on perceptions of global warming. Curr. Psychol., 31, 318–324, doi:10.1007/s12144-012-9148-z.
Li, Y., Johnson E. J. , and Zaval L. , 2011: Local warming: Daily temperature change influences belief in global warming. Psychol. Sci., 22, 454–459, doi:10.1177/0956797611400913.
Marquart-Pyatt, S. T., McCright A. M. , Dietz T. , and Dunlap R. E. , 2014: Politics eclipses climate extremes for climate change perceptions. Global Environ. Change, 29, 246–257, doi:10.1016/j.gloenvcha.2014.10.004.
McCright, A. M., and Dunlap R. E. , 2011: Cool dudes: The denial of climate change among conservative white males in the United States. Global Environ. Change, 21, 1163–1172, doi:10.1016/j.gloenvcha.2011.06.003.
Mullinix, K. J., Leeper T. J. , Druckman J. N. , and Freese J. , 2015: The generalizability of survey experiments. J. Exp. Polit. Sci., 2, 109–138, doi:10.1017/XPS.2015.19.
Rasmussen, S. H. R., 2016: Education or personality traits and intelligence as determinants of political knowledge. Polit. Stud., doi:10.1111/1467-9248.12214, in press.
Risen, J. L., and Critcher C. R. , 2011: Visceral fit: While in a visceral state, associated states of the world seem more likely. J. Pers. Soc. Psychol., 100, 777–793, doi:10.1037/a0022460.
Scannell, L., and Gifford R. , 2013: Personally relevant climate change: The role of place attachment and local versus global message framing in engagement. Environ. Behav., 45, 60–85, doi:10.1177/0013916511421196.
Shadish, W. R., Cook T. D. , and Campbell D. T. , 2002: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin, 650 pp.
Stanovich, K. E., and West R. F. , 2002: Individual differences in reasoning: Implications for the rationality debate? Heuristics and Biases: The Psychology of Intuitive Judgment, T. Gilovich, D. Griffin, and D. Kahneman, Eds., Cambridge University Press, 421–440.
Taylor, A., Bruine de Bruin W. , and Dessai S. , 2014: Climate change beliefs and perceptions of weather-related changes in the United Kingdom. Risk Anal., 34, 1995–2004, doi:10.1111/risa.12234.
van der Linden, S., 2015: The social-psychological determinants of climate change risk perceptions: Towards a comprehensive model. J. Environ. Psychol., 41, 112–124, doi:10.1016/j.jenvp.2014.11.012.
Visser, P. S., Bizer G. Y. , and Krosnick J. A. , 2006: Exploring the latent structure of strength-related attitude attributes. Advances in Experimental Social Psychology, M. P. Zanna Ed., Academic Press, 1–67.
Weathers, M. R., and Kendall B. E. , 2015: Developments in the framing of climate change as a public health issue in US newspapers. Environ. Commun., 10, 593–611, doi:10.1080/17524032.2015.1050436.
Wiest, S. L., Raymond L. , and Clawson R. A. , 2015: 2015: Framing, partisan predispositions, and public opinion on climate change. Global Environ. Change, 31, 187–198, doi:10.1016/j.gloenvcha.2014.12.006.
Zaval, L., Keenan E. A. , Johnson E. J. , and Weber E. U. , 2014: How warm days increase belief in global warming. Nat. Climate Change, 4, 143–147, doi:10.1038/nclimate2093.
The demographic breakdown of the sample is as follows. Age was measured as a six-item categorical variable (1 = under 18; 2 = 18–24; 3 = 25–34; 4 = 35–50; 5 = 51–65; 6 = 65+); the mean response was 3.18 with a standard deviation of 0.80. Education was measured as a five-item categorical variable (1 = less than high school; 2 = high school; 3 = some college; 4 = 4-yr college degree; 5 = advanced degree); the mean response was 3.46 with a standard deviation of 0.88. Slightly fewer than half of the respondents were female (46.5%). Income was measured as a five-item categorical variable (1 = < $30,000; 2 = $30,000–$69,999; 3 = $70,000–$99,999; 4 = $100,000–$200,000; 5 = $200,000+); the mean response was 2.07 with a standard deviation of 0.90. As intimated, ideology was measured on a seven-point scale ranging from “very liberal” (a low score of 1) to “very conservative” (a high score of 7); the mean response was 3.26 with a standard deviation of 1.67. Also, as intimated, respondents’ environmental/economic attitudes were assessed by asking them whether they favored “protecting the environment, even at the risk of curbing economic growth” or “maintaining a prosperous economy, even if the environment suffers to some extent?” Respondents’ answers were recorded on a seven-point scale ranging from “definitely protect environment” (a low score of 1) to “definitely maintain prosperous economy” (a high score of 7); the mean response was 3.46 with a standard deviation of 1.83. Finally, the average score on the four-item knowledge battery was 2.99 correct with a standard deviation of 1.15.
At the time of the initial survey, respondents were informed that they would be recontacted to participate in a second brief survey.
The comparison of the two groups on all measured demographics confirmed they matched on average. Specifically, a logistic regression, available from the authors, showed that virtually none of the demographic variables significantly predicted experimental assignment, suggesting balance across conditions.
As mentioned, objective temperature data were collected (see appendix B); those data show that TT is significantly correlated with objective temperature deviations at both T1 (r = 0.43, p < 0.001, two-tailed test) and T2 (r = 0.21, p < 0.01, two-tailed test), suggesting that respondents did indeed attend to actual weather patterns in formulating their responses to this item. PDW, on the other hand, is not correlated with objective temperature at either point, and does marginally differ by condition (i.e., at the 0.1 level of significance). Additionally, neither objective temperatures nor objective temperature deviations differ significantly across conditions either at T1 or T2 (see appendix B).
Objective temperature data show that respondents experienced temperatures on the day of the survey that were a mean of 5.3°F (s.d. = 6.8°F) warmer than the historical average.
Linear regression shows that perceived temperature deviations (TT) influence PDW even when controlling for objective temperatures and objective temperature deviations among control (no prompt) respondents. TT’s effects are insignificant among respondents who received the prompt. Details are available from the authors.
As in Druckman (2015), the correlation between PDW and each dependent variable, while not reported in Table 1, is statistically significant across experimental conditions. There were no predictions about how the relationship among these particular variables might change (or not) based on exposure to the prompt.
It is possible that cognitive ability could moderate an effect in the treatment (prompt) group (e.g., the general null results could hide a small effect among those very low in ability). This possibility was explored, and the result was that cognitive ability has neither a main nor an interactive effect among treated respondents.
When “percent days warmer” (PDW) is left out, all models remain substantively the same.
It is worth noting that recent work on mediation makes clear that the study design used here and by others—which involves the measurement, within a single study, of both the overall effect of the treatment (prompt) and its indirect effect through a potential mediator—makes it impossible to definitively establish mediation (Bullock and Ha 2011). Moreover, another challenge to documenting mediation, if it did occur partially via PDW, is that individuals may not easily translate their perceptions over time into precise estimates.
Similar results obtain using ordered logistic regression instead of ordinary least squares regression (in terms of significance and the direction of coefficients). Also, interacting cognitive ability with PDW instead of TT produces an interaction term that is significant for two of three dependent variables (GWB and GWW).
In part, this replicates the findings of Egan and Mullin (2012), although they proxy for cognitive ability with education, and examine the effect of actual rather than perceived temperatures on attitudes and beliefs. Interestingly, the results here may appear to contradict Zaval et al. (2014), who find that increased knowledge does not eliminate the local warming effect. This may be due to differences in how the respective research designs operationalize “cognitive ability.” Zaval et al. directly manipulate respondents’ knowledge base, presenting treatment-group respondents with information about the relationship between local short-term and broad long-term temperature trends; they find that this information fails to correct the local warming effect. In contrast, the design here controls for respondents’ preexisting levels of cognitive ability using a political knowledge battery. It is possible that less able respondents were unable to integrate the information that Zaval and colleagues presented, allowing the local warming effect to persist in spite of their treatment. The results here show, on the other hand, that for more cognitively able respondents, the local warming effect never appears in the first place.
Analyses using education as a measure of ability rather than the political knowledge battery were also conducted. Those analyses show that education does not work in the same way; that is, it does not moderate the results (cf. Egan and Mullin 2012). This likely reflects the nature of MTurk respondents. While MTurk is better than a student sample, it is still skewed on some variables, including education (see Levay et al. 2016). Indeed, in the sample studied here, variance was limited such that 55% of subjects had at least a 4-yr college degree. Another test run was for an interaction between ideology and cognitive ability as some research has shown significant effects for such an interaction (e.g., Bolsen et al. 2015). The results show no significant interaction, which likely again reflects a lack of variance in the MTurk sample: 58% of respondents were liberal and another 19% were pure independents (see Levay et al. 2016). To be clear, MTurk is generally heterogeneous across variables, but education and ideology are two of the central variables on which it tends to be skewed.
These same analyses were run with the inclusion of objective temperature and objective temperature deviation as control variables. The results remained substantively the same (i.e., perceived temperature deviations influence global warming beliefs among control (no prompt) respondents, and the interaction effects described above remain significant). In other analyses (more directly concerning results presented in Table 1), objective temperature deviation was substituted for perceived temperature deviation and it was found to be significant for only one outcome variable, and, interestingly, in that case, the prompt also eliminated its effect. See appendix B for more discussion; all such results are available from the authors.
Previous studies and reviews of the persistence of treatment effects and attitude change suggest mixed results (e.g., Lecheler and de Vreese 2011; Baden and Lecheler 2012).
One inferential concern is whether the balance achieved through random assignment at time 1 was maintained at time 2, given some response rate attrition. Balance would allow for causal inferences across the experimental groups (e.g., the prompt’s impact is expected to endure and thus the experimental groups should continue to differ when it comes to the effect of perceived temperature deviations at time 2). In results available from the authors, it is shown that balance remains at time 2. The groups likely remain comparable on unobserved measures since there is no reason to expect those receiving the brief and subtle prompt at time 1 would subsequently engage in behaviors distinct from those who do not receive the prompt at time 1.
Substantively, identical effects are obtained via ordered logit: today’s temperature at T2 remains statistically insignificant among those who received the prompt at T1.
While similar in some respects, the consideration of effects over time here differs from that of Egan and Mullin (2012), who find fleeting local warming effects. Egan and Mullin investigate the influence of temperature at T1 on attitudes at T2; in contrast, here the focus is on the effects of a prompt given at T1 and temperature at T2 on attitudes at T2.
