1. Introduction
The use of weather icons to communicate weather forecasts is a long-standing and deeply embedded practice of broadcast meteorologists (BMs). However, there is a lack of peer-reviewed research that addresses actual weather icons currently used by BMs in daily weather forecasts and whether those icons serve any useful purpose as elements in weather messaging. Furthermore, there is an even greater lack of research on the meaning of individual weather icons is and the intent of practitioners who use them or designers who create the icons. Although there are critically important questions about the meaning and intent of weather icons, in designing this study the authors elected to focus instead on both the practice of BMs using weather icons and audience interpretation of weather icons.
Previous studies have illuminated an apparent disconnect between how weather icons are used by the meteorological community and then interpreted by the public. Zabini (2016) and Zabini et al. (2015) found that members of the Italian public frequently interpreted icons showing more precipitation with higher probabilities of precipitation (PoPs), but a majority did not think the icons conveyed any information about precipitation intensity. Therefore, if it is the intent of the BM to communicate precipitation intensity via an icon, it is possible the audience will interpret the icon differently. Furthermore, the two studies found that varying levels of uncertainty were inferred from different icons—even if uncertainty was not intended in the forecast. Joslyn et al. (2009) studied simplified weather icons in comparison with a pie chart to determine which method of communicating a chance of precipitation was most effective. This study found that pie charts showing the chance of precipitation and the chance of no precipitation were preferred over weather icons. Errors in interpretation of the forecast were higher when the chance of having no precipitation was not explicit. A fourth relevant study conducted in Norway found that the public can identify, based on previous experience, nuances in weather icons, such as interpreting the icon of a gray cloud as a chance of rain even if no raindrops are shown (Sivle et al. 2014). Identifying nuances that the public perceives in weather icons is a crucial step in understanding whether their interpretations of icons match the intention of the BMs who choose the icons to include in a forecast.
The preponderance of previous research addressing the messaging of weather through the use of images or graphics focuses on the use of maps and colors (Cappucci 2020; Williams et al. 2020; Bryant et al. 2014), as well as the presentation of radar images (Saunders et al. 2018). Other research focuses on the language used by BMs (Williams et al. 2020), their use of gestures (Sherman-Morris and Lea 2016), and their audience’s ability to comprehend the extended forecast graphic (Reed and Senkbeil 2021, 2020; Kahl and Horwitz 2003), which is increasingly being used in mobile apps (Phan et al. 2018). Phan et al. (2018) examined how college students use daily weather app information on mobile devices. It is the dominant source of information for an entire generation, garnering a shift in how weather information will be consumed in the future. Other generations rely more on traditional broadcast television meteorology (Sherman-Morris 2005). It is therefore important to understand how daily weather icons are used across different information sources to draw parallels between common icons and weather phrases for message consistency.
A recent, but seminal, publication by Williams and Eosco (2021) on message consistency within the weather enterprise emphasized the need to work toward minimizing unnecessary conflicting information within the context of a weather message. Because weather icons used by BMs are commonly presented alongside a qualitative weather phrase and a PoP on graphics (National Research Council 2006), particularly the extended forecast graphic (Reed and Senkbeil 2021, 2020), this study was designed generate data that will allow practitioners, particularly BMs, to critically examine whether the icons they use emphasize, complement, or contradict the weather phrases and PoPs the icons are routinely shown alongside. Generating these data will illuminate for BMs any disparities between how they use and interpret the combination of icons, weather phrases, and PoPs versus how members of the public interpret those same combinations. If it is found that an icon and a particular phrase, or an icon and a PoP value, are in conflict, BMs can avoid using those combinations in their forecasts and therefore avoid disseminating information that may result in confusion.
Our effort is unique in that 1) we use actual and unmodified icons currently used by BMs; 2) we ask participants to respond to the icons in relation to common weather phrases, which is how people are often exposed to the icons; and 3) we ask participants to assign PoPs to icons, so that practitioners can compare how the public associates icons and PoPs with how they currently utilize icons alongside PoPs when presenting forecast information. The following research questions (RQs) guide this study:
-
RQ1) Does the public think weather icons are good illustrators of common weather phrases?
-
RQ2) Does the public associate higher PoPs with icons that show dark clouds, rain, or lightning?
-
RQ3) Does the public discern nuances in details of the weather icons?
RQ1 will generate data that will allow BMs to determine whether there is conflict between the icons and weather phrases that were analyzed. If the data analysis reveals that the icon and phrases contradict each other, then the solution may be as simple as not using the particular icons to illustrate the weather phrases. The data generated by RQ2 and RQ3 can be used by BMs to explore whether the public’s interpretation of the icons differs from their interpretation of the same icon.
2. Methods
a. Survey design and distribution
A 17-question survey was developed in Qualtrics. The survey was designed to show in random order 11 weather icons paired randomly with eight weather phrases. This design reflects the current practice of BMs where icons are routinely paired with other weather information. Items were primarily of the following format: “The [weather icon] pictured above is a good way to illustrate [weather phrase] thunderstorms” (descriptions of the icons and weather phrases used in this study are included later in section 2 within their own subsections). Participants answered these icon illustration questions on a 7-point Likert scale from “strongly agree” to “strongly disagree.” Next, participants were randomly shown one of the 11 weather icons and asked to assign a percent chance of rain from a drop-down list with options ranging from 0% to 100% in intervals of 10. The following demographic information was also collected: 1) state of current residence, 2) U.S. Postal Service Zonal Improvement Plan (ZIP) code, 3) age, 4) gender, 5) ethnicity, 6) race, 7) highest level of education, and 8) estimated annual income. The survey was distributed by broadcast meteorologists to their audiences on Facebook and Twitter. Responses were collected in this manner between 19 August and 15 September 2020.
b. Descriptions of weather icons
Eleven weather icons (The Weather Company 2020) were included in this study (Fig. 1). The 11 icons illustrate daytime weather conditions because daytime icons are more commonly used on weather graphics like the extended forecast graphic (Reed and Senkbeil 2020, 2021). These icons were chosen because the weather conditions illustrated by them represent a variety of common weather conditions, ranging from completely sunny to a dark gray cloud with rain underneath. The remainder of this paragraph serves to describe and name each of the 11 icons from Fig. 1. The first four icons in Fig. 1 do not depict precipitation or lightning. Icon 1 (sun) shows a clear sunny sky. Icon 2 (sun wispy cloud), icon 3 (mostly sunny), and icon 4 (mostly cloudy) show progressively more cloud cover. Icon 5 (sun rain shower), icon 6 (sun thunderstorm), and icon 7 (cloud lightning) depict variable weather conditions with the possibility of a thunderstorm. Icon 7 does not show any precipitation. The remaining icons depict variations in rainfall with and without lightning. Icon 8 (thunderstorm rain) shows a dark gray cloud with lightning bolts and rain underneath. Icon 9 (rain light), icon 10 (rain moderate), and icon 11 (rain heavy) each consist of a gray cloud with different rain intensities underneath. The names of the weather icons were not shown to participants in the survey, and the icons that they were shown were not animated.
c. Descriptions of weather phrases
A group of eight commonly used precipitation terms (National Weather Service 2020a) was paired randomly with the icons described above. The following weather phrases were evaluated: 1) scattered, 2) widely scattered, 3) numerous, 4) periods of, 5) a few, 6) isolated, 7) occasional, and 8) widespread. These phrases are frequently used in forecast discussions and in television broadcasts. In practice, these phrases are sometimes accompanied by “showers and thunderstorms” or “rain and thunderstorms.” The authors chose to use the single term “thunderstorms” 1) to avoid the introduction of a double-barreled statement and 2) because it is reasonable to assume that people’s interpretations of what constitutes a thunderstorm are less variable than how they differentiate showers, rain, drizzle, or mist.
d. Assigning icons a percent chance of rain
In addition, because weather icons are often accompanied by PoPs on weather graphics, it is worthwhile to explore associations between certain weather icons and PoPs. Therefore, we asked participants to assign a PoP between 0% and 100% to a randomly chosen icon from Table 1. Previous research (Reed and Senkbeil 2020; Zabini 2016; Joslyn et al. 2009; Morss et al. 2008) has shown that the public tends to associate icons showing precipitation with higher PoPs, which may conflict with what the meteorologist intends to convey in the forecast. Because the weather icons used in this study show variations in clouds and rainfall, patterns in the participant-assigned PoPs may reveal whether the public perceives nuances in weather conditions shown in the icons.
Descriptive statistics of participant demographics.
e. Data analysis
R statistical software (R Core Team 2020), Minitab (2020), and Microsoft Excel were used to conduct all statistical tests and create figures from the data collected in this study. Participant characteristics were reported descriptively, with participant ZIP codes used to map the spatial distribution of those who took the survey.
Because participants answered the icon illustration questions on a Likert scale, we report the average response for each of the 11 icons to gauge whether people thought the icons were good illustrators of the eight weather phrases. Because our data did not meet the normality or equal variance assumption required to run an analysis of variance (ANOVA), Kruskal–Wallis (KW) tests were conducted. Using a random sample (n = 500) of responses for each weather phrase, the KW test was used to detect whether differences in the illustration score exist between icons. The Nemenyi post hoc test (Sinclair 2019; Pohlert 2014) for multiple comparisons was used to identify significant differences between icons.
A ridgeline plot showing probability distributions of assigned PoPs for each of the 11 weather icons was created. Descriptive statistics were reported on the responses (n = 500) for each icon. A hierarchical agglomerative cluster analysis (DataNovia 2020) was performed to explore whether the icons could be grouped on the basis of similarities in their assigned PoPs.
3. Results
a. Survey demographics
More than a dozen broadcast meteorologists agreed to distribute the online survey by posting a link on their social media pages. This method of distribution resulted in a sample size of 6253. Because participants could skip questions, occasionally the number of responses for a particular question was less than the total sample size. Participant ZIP codes were used to generate a map of their location (Fig. 2). The location of participants is primarily controlled by the location of the meteorologists who posted the survey link. For example, television market footprints in Alabama, Maryland, Massachusetts, Missouri, Oregon, and North Carolina are prominent. The highest rates of responses came from the Birmingham and Huntsville areas in Alabama and the Charlotte, North Carolina, area.
Facebook was the primary social media platform on which the survey was distributed. From Facebook-user research (Perrin and Anderson 2019), men and minority groups are underrepresented in our sample (Table 1). According to Perrin and Anderson (2019), a survey of Facebook users showed that 75% of women used the platform, whereas 63% of men reported using Facebook. Female participants made up 70% of our sample; men made up 29% of our sample. Personal communication with three of the broadcast meteorologists who distributed our survey verifies that their Facebook demographics are split about 75/25 between women and men, respectively. An overwhelming majority (95%) of participants listed White or Caucasian as the race with which they identified most. The Perrin and Anderson (2019) study of social media users showed that use of Facebook among those identifying as White, Black, or Hispanic was 70%, 70%, and 69%, respectively. Because Facebook does not make information on race available to Facebook page owners, we cannot comment on whether the lack of racial diversity present in our sample is also present among people who follow Facebook pages of the BMs who distributed our survey.
b. Icons as illustrators of weather phrases
The next eight subsections are organized according to the weather phrases that were used in our study. Participants were asked whether they felt the weather icon (randomly selected from Fig. 1) was a good illustrator of the indicated weather phrase. Icons will be referred to by their code number (see Fig. 1). Responses were given on a Likert scale from 1 = strongly agree to 7 = strongly disagree. Therefore, lower illustration scores reflect participant sentiment that the icon is a good illustrator of the weather phrase. Table 2 includes the average illustration score for each icon. A KW test was used to examine whether significant differences between any of the icons existed. Because the KW tests revealed significant (p < 0.05) differences for every weather phrase, a Nemenyi post hoc test was conducted to find the icons with significant differences in illustration scores. Table 3 shows coded p values associated with the Nemenyi test. Mean icon illustration scores for each weather phrase are found in parentheses in the following subsections.
Mean illustration scores for each icon by eight different weather phrases, where 7 indicates strong disagreement that the icon is a good illustrator of the weather phrase and 1 indicates strong agreement that the icon is a good illustrator of the weather phrase (n = 500 for each weather phrase; the number in the icon column corresponds to the icon code from Fig. 1).
Results of Nemenyi post hoc test. Asterisks indicate statistically significant p values for differences in mean illustration score between icons (e.g., icon 1 and icon 9), evaluated for each of eight weather phrases. A single asterisk indicates p < 0.05, a double asterisk indicates p < 0.01, and an em dash represents p ≥ 0.05.
1) Scattered thunderstorms
Participants felt that icon 6 was the best illustrator of scattered thunderstorms (Fig. 3a). There was strong disagreement that icons 1 and 2 were good illustrators of scattered thunderstorms. The Nemenyi post hoc comparison test showed statistically significant differences between all icons except 3 and 4; 5 and 6; 5 and 7; 8 and 9; 8 and 10; 8 and 11; 9 and 10; 9 and 11; and 10 and 11 for the scattered thunderstorms weather phrase.
2) Widely scattered thunderstorms
Although icons 6 and 8 had the lowest illustration scores of all icons for this weather phrase (Fig. 3b), the results are not convincing that these two icons are good illustrators of widely scattered thunderstorms. Results of the Nemenyi test show no significant difference between these two icons (p ≥ 0.05). Icon 1 was indicated to be the worst illustrator of the widely scattered thunderstorms phrase.
3) Numerous thunderstorms
For this weather phrase (Fig. 3c), icons 7 and 8 were indicated to the best illustrators of numerous thunderstorms of all 11 icons analyzed. However, their illustration scores are near the middle of the Likert scale, which indicates neither strong agreement nor strong disagreement. Participants rated icon 1 as the poorest illustrator of this weather phrase.
4) Periods of thunderstorms
Icon 6 was again the preferred icon to illustrate periods of thunderstorms (Fig. 3d). There was slight to strong disagreement that all the other icons were good illustrators of this weather phrase; icon 1 was indicated to be the worst illustrator of this weather phrase. There were significant differences among each icon except 3 and 4; 5 and 7; 5 and 8; 6 and 7; 7 and 8; 8 and 9; 9 and 10; 9 and 11; and 10 and 11. This weather phrase was one of only two for which a significant difference (p < 0.05) was seen between icons 5 and 6.
5) A few thunderstorms
None of the icons produced a mean illustration score lower than 4 for this weather phrase (Fig. 4a), however the median response for icon 6 was 3. This indicates slight agreement that icon 6 was a good illustrator of this weather phrase. The Nemenyi post hoc test showed no significant difference between icons 5 and 6 and icons 6 and 7 for this weather phrase. Icons 1 and 2 both received agreement scores above 6, making these the worst illustrators of this weather phrase.
6) Isolated thunderstorms
The icon with the lowest illustration score for this weather phrase (Fig. 4b) was icon 6. The score for icon 6 was near the middle of the Likert scale, where there is neither agreement nor disagreement that this icon was a good illustrator of isolated thunderstorms. Icons 1 and 2 were again the least preferred for this weather phrase with agreement scores higher than six.
7) Occasional thunderstorms
Participants had neither strong agreement nor strong disagreement that icon 6 was a good illustrator of occasional thunderstorms (Fig. 4c). The score for icon 6 was not significantly different from icon 5. Although these icons received the lowest scores of the other nine icons, the results are not convincing that these two icons are good illustrators of this weather phrase.
8) Widespread thunderstorms
Icons 7 and 8 had the two lowest scores for the widespread thunderstorms weather phrase (Fig. 4d). The scores for icons 7 and 8 were not significantly different from one another (p ≥ 0.05). Like several of the other weather phrases, these scores are not low enough to say convincingly that these icons are good illustrators of widespread thunderstorms. There was, however, strong agreement that icons 1, 2, 3, and 4 were not good illustrators of widespread thunderstorms. Each of these icons had an illustration score higher than 6.
c. Assigning PoPs to icons
The final question was designed to explore how PoPs are associated with the weather icons. Participants were shown an icon randomly selected from Table 1 and asked what percent chance of rain best corresponded to that icon. Answers between 0% and 100% were selected from a drop-down menu. Samples of 500 responses were used to create a ridgeline plot of density curves showing the distribution of assigned PoPs per weather icon (Fig. 5). Descriptive statistics on the responses (Table 4) for this question show the most frequent and average response, as well as measures of spread for the assigned PoPs.
Descriptive statistics for assigned probability to each icon. The icon code corresponds to the icon codes given in Fig. 1.
A strong majority (93.2%) of participants selected 0% as the PoP that best corresponded to icon 1. The average PoP for this icon was 1.2%. The most frequently assigned PoP for icon 2 was also 0%; 200 participants chose this value. There was more spread in the results, which lead to this icon having a higher average PoP at 9.12%. The most common answer for icon 3 was also 0%, but the choices ranged from 0% up to 60%. The wide range of responses produced the negative kurtosis (Table 4) for this icon. The average assigned PoP for icon 4 was 32.38%, but the mode was 10%. Similar to the density curve for icon 3 (Fig. 5), the kurtosis for icon 4 was negative, which indicates a large spread in responses.
There was a considerable jump in the mean and mode of assigned PoPs for icons 5–11. Each of these icons showed dark clouds, rain, lightning, or a combination thereof. The mode of the assigned PoPs for icons 5 and 6 was 50%. The mean response for these icons was 46.58% and 49.76%, respectively. The most frequently assigned PoP for icon 7 was 80%, but responses varied from 0% to 100%. While the mean was 52.66%, this icon had the highest standard deviation (σ = 29.16). Furthermore, this icon had the smallest kurtosis (Fig. 5). Icons 8, 9, 10, and 11 each had a mode of 100% for the participant-assigned PoP. The distribution for each of these icons was multimodal, with peaks at 100%, 80%, and 50%. Of these four, icon 10 had the lowest mean PoP at 79.32%. The remaining three had an average assigned PoP of 80% or higher. Icon 11 had the highest average PoP of 88.26%.
d. Cluster analysis
Figure 4 appears to show three distinct groups: icons with assigned PoPs centered near 0%, 50%, and 100%. Furthermore, the mode PoP response (Table 4) supports the idea of having distinct groups. To test the hypothesis that the icons evaluated in this study can be separated into groups (and find the appropriate number of groups), a hierarchical agglomerative cluster (HAC) analysis was conducted (DataNovia 2020; Tullis and Albert 2013). The goal of the HAC is to have the fewest number of clusters while maintaining a reasonably high similarity level. The dendrogram (Fig. 6a) results of the HAC showed that the largest decrease of intragroup similarity occurred between 3 and 2 groups, suggesting that 3 groups should be retained. Figure 6b provides further justification for using 3 clusters. Reading the bar and line graph from left to right shows how the similarity level changes as the HAC algorithm separates the icons into fewer and fewer groups. The large drop in similarity level between 3 clusters and 2 clusters indicates that using 3 groups maximizes similarity level while minimizing the number of clusters.
From the assigned PoPs and HAC results, the icons were objectively partitioned into three groups, and each of these groups was then subjectively named to facilitate discussion. Group 1 was named “sun, no precip” (SNP) and consists of icons 1, 2, and 3. Next, icons 4, 5, 6, and 7 were placed into group 2, named “clouds with rain possible” (CRP). Group 3 consists of icons 8, 9, 10, and 11 and was named “nimbus” (NIM). SNP is associated with PoPs at or near 0%, CRP at or near 50%, and NIM at 100%.
4. Discussion and future work
This section is organized into three paragraphs to discuss the main takeaways from this study: 1) no weather icon was overwhelmingly thought to be a good illustrator of any weather phrase tested, 2) certain icons appeared homogenous to participants despite showing varying levels of cloud cover or intensities of rainfall, and 3) the weather icons evaluated in this study could be grouped into three clusters based on assigned PoP values. Within each following paragraph, recommendations for future work are given.
Many of the results of the items for which people indicated whether they agree that the icons were good illustrators of the weather phrases were not surprising. Participants unambiguously rated icons that showed less rainfall and less cloud cover as poor illustrators of the weather phrases, all of which featured a precipitation term. Specifically, icons 1 and 2 performed the worst for each of the eight weather phrases. Therefore, our results suggest that these two icons should not be used within forecasts that include any of the weather phrases we tested. Because the illustration scores were highest with little ambiguity in interpretation, we can be confident in our assertion that these icons are not good illustrators of any of the weather phrases. Other icons showing precipitation and lightning had lower illustration scores for the weather phrases, but the results reflect more ambiguity of interpretation. It is unclear why these icons did not receive more unanimous support as being good illustrators of the weather phrases. One explanation might be that the representation of lightning in this icon set was not registered by participants. In any case, we cannot convincingly say that icons in this set featuring precipitation and lightning are good illustrators of the weather phrases, because the results point to an ambiguity in interpretation. In the event that the icons are in conflict with the weather phrases, the public may rely more on what the meteorologist verbally conveys (Bergen et al. 2005). Interestingly, as shown in Figs. 3 and 4, there is a convergence of participant interpretation, which may suggest they were more confident in their assessment about which icons were not good illustrators of the weather phrases. However, they were unsure whether any of the icons were good illustrators of the weather phrases. More robust qualitative procedures are needed in future work to determine whether this difference about certainty of response reflects the nature of the differences in how people interpret icons and whether this is exclusive to weather icons. Or perhaps it reflects the inherent uncertainty surrounding predicting the future. Alternatively, this could be a spurious finding of this single study, despite our large sample size. Furthermore, any effects of potential ambiguity in people’s interpretation of the weather phrases are not able to be ascertained in this study.
The second major takeaway from this study was that participants did not appear to discern nuances in many of the weather icons (Sivle et al. 2014), despite those icons showing different weather conditions. The fact that meteorologists along with graphic designers have produced multiple icons to show similar weather conditions supports the idea that in the weather community, these icons have meaningful differences, but our results suggest that belief is not shared with members of the public. Whereas a meteorologist might see a genuine and meaningful difference between the rain light and rain heavy icons, or between the mostly sunny and mostly cloudy icons, in this study, participants do not seem to share those distinctions. This is supported by the results of the Nemenyi post hoc tests presented in Table 3. For example, when comparing icons 3 and 4 (mostly sunny and mostly cloudy), there were no significant differences between these icons’ illustration scores for any of the weather phrases. Therefore, when looking at these icons and deciding whether they are good illustrators of the weather phrases, participants did not perceive these icons as being significantly different. In other words, their differences were imperceptible when deciding if these two icons were good illustrators of each of the eight weather phrases. Similarly, when evaluating whether the rain light, rain moderate, and rain heavy icons were good illustrators of each of the eight weather phrases, people did not perceive these icons as showing different weather conditions—at least not different enough to affect their interpretation of how good these icons were as illustrators of the weather phrases. In these two examples, no significant differences were shown between these icons for all eight weather phrases. The results for the rain light, rain moderate, and rain heavy icons may be because the differences in the icons are harder to pick out in this set. For example, participants are not able to distinguish individual raindrops, the clouds are the same color, and there is no motion to judge the rate at which the drops are moving. But in the case of the mostly sunny and mostly cloudy icons—where participants can count the different number of clouds—there still were no significant differences perceived, so perhaps the public does not pay attention to such details in weather icons. More robust qualitative procedures, such as think-aloud exercises (Sutton and Fischer 2021), may provide a richer understanding of what nuances people discern or do not discern. The other instances in which participants did not perceive the icons as showing different weather conditions are indicated by an em dash in Table 3. It is unknown whether substituting weather icons from a different set would produce different results, but a future effort might consider using this study as a framework to test other icons.
The third major takeaway from this study is that the icons evaluated herein were able to be grouped into three classes (SNP, CRP, and NIM) according to the PoPs that were assigned by participants. This is a significant finding given the concurrent hazard simplification (National Weather Service 2020b) movement within the National Weather Service, which is working to reduce the number of weather alert categories to minimize public confusion and maximize efficiency. A similar effort could be adopted by the weather enterprise to eliminate unnecessary or confusing icons, or combine icons that people view as similar. This would greatly reduce the number of weather icons (in this study from 11 icons to 3) and may help improve the efficiency of weather messaging. Many broadcast meteorologists have modified the weather icons that come standard with their weather systems, or in some cases created new icons to fit their needs. This practice has not been studied and is beyond the scope of this project. However, future projects should explore how these modified icons are interpreted by the public. In addition, it is not the purpose of this study to assign meaning to the weather icons that were evaluated, nor it is the purpose of this study to determine the intent of the BM when choosing icons.
5. Limitations and other considerations
The sample generated by this study is not representative of the U.S. public, and so our ability to generalize our results to other populations is limited. A benefit to using BMs’ social media platforms is the resulting geographic spread and large sample size of participants. Obtaining representative samples through other means can be financially and pragmatically daunting. Given our method of distribution, we are unable to comment on whether the sample we generated is representative of the population that follows each BM that distributed our survey. Furthermore, we cannot comment on the similarities between the social media following of a BM, our sample, and the population that watches television weather broadcasts. Exploring this information should be a priority of a future study, especially given the repeated lack of gender and racial/ethnic minorities in samples generated from social media (Jang and Vorderstrasse 2019).
Although the weather icons we examined in this study are currently in use by BMs, the icons appeared as static images within the survey, whereas when shown on television, the icons are usually in motion (i.e., rain is falling or lightning bolts flash on and off). With the icons being static, it may have been harder for participants to perceive differences in rain intensity or the presence of lightning, which may have affected their responses given that each of the weather phrases was followed by the word “thunderstorms.” It is possible that people indicated no difference between icons 9, 10, and 11 because the weather phrases included “thunderstorms” rather than “rain” or “showers.” Furthermore, it is possible that, despite weather phrase questions appearing in a random order, ordering effects may have affected people’s responses to later questions and especially the PoP question, which appeared last for all participants. The clustering of participants in the Southeast combined with the time of year this survey was distributed also may have affected people’s responses. A future study that compares interpretation of icons or weather phrases at different times of year may reveal whether such influences on perception exist.
6. Conclusions
The study described in this paper sought to gain an understanding of how weather icons are interpreted by the public. Specifically, this study was designed to evaluate if icons currently used by BMs are thought of as good illustrators of common weather phrases. Also, data on associations between weather icons and PoPs was collected by having participants assign PoPs to the icons. We found that none of the 11 weather icons was thought to be a great illustrator of the following weather phrases: scattered thunderstorms, widely scattered thunderstorms, isolated thunderstorms, numerous thunderstorms, a few thunderstorms, periods of thunderstorms, widespread thunderstorms, or occasional thunderstorms. In addition, we found that participants do not seem to discern nuances in some weather icons that attempt to show different weather conditions, such as rain light versus rain heavy. On the basis of how the public associates PoPs with weather icons, the icons tested in this study could be grouped into three classes. Meteorologists and other communicators of weather information can use the results of this study to understand how similar or different people’s interpretations of these weather icons are to their own. If differences in interpretation are present between practitioners and users, there exists an opportunity to improve the communication of weather information and bridge the gap in understanding of meteorological products that has been shown to exist between the public and meteorologists in other studies (Cappucci 2020; Reed and Senkbeil 2020; Williams et al. 2017, 2020; Saunders and Senkbeil 2017; Stewart et al. 2016; Mason and Senkbeil 2015; Radford et al. 2013; Morss et al. 2008).
Acknowledgments.
The authors thank the many broadcast meteorologists who graciously agreed to distribute the online survey to their social media channels. In addition, we thank the three anonymous reviewers who provided generous and thorough feedback that greatly improved this paper.
Data availability statement.
A copy of the survey administered in this study is available in the online supplemental material. Because the data involve human subjects and are under the purview of the University of Alabama’s Institutional Review Board, data cannot be made openly available. Requests for access to the data and code used to complete analysis and produce plots should be made to the corresponding author.
REFERENCES
Bergen, L., T. Grimes, and D. Potter, 2005: How attention partitions itself during simultaneous message presentations. Hum. Commun. Res., 31, 311–336, https://doi.org/10.1111/j.1468-2958.2005.tb00874.x.
Bryant, B., M. Holiner, R. Kroot, K. Sherman-Morris, W. B. Smylie, L. Stryjewski, M. Thomas, and C. I. Williams, 2014: Usage of color scales on radar maps. J. Oper. Meteor., 2, 169–179, https://doi.org/10.15191/nwajom.2014.0214.
Cappucci, M., 2020: The National Weather Service issues highly accurate thunderstorm forecasts. The public doesn’t understand them. Washington Post, accessed 22 February 2021, https://www.washingtonpost.com/weather/2020/06/10/storm-prediction-center-risk-categories/.
DataNovia, 2020: Hierarchical clustering in R: The essentials. Accessed 4 December 2020, https://www.datanovia.com/en/courses/hierarchical-clustering-in-r-the-essentials/.
Jang, M., and A. Vorderstrasse, 2019: Socioeconomic status and racial or ethnic differences in participation: Web-based survey. JMIR Res. Protoc., 8, e11865, https://doi.org/10.2196/11865.
Joslyn, S., L. Nadav-Greenberg, and R. M. Nichols, 2009: Probability of precipitation: Assessment and enhancement of end-user understanding. Bull. Amer. Meteor. Soc., 90, 185–194, https://doi.org/10.1175/2008BAMS2509.1.
Kahl, J. D. W., and K. A. Horwitz, 2003: Daily low or overnight low? Confusion in icon-based forecasts. Bull. Amer. Meteor. Soc., 84, 155–156, https://doi.org/10.1175/1520-0477-84.2.151.
Mason, J. B. and J. C. Senkbeil, 2015: A tornado watch scale to improve public safety. Wea. Climate Soc., 7, 146–158, https://doi.org/10.1175/WCAS-D-14-00035.1.
Minitab, 2020: Minitab 19 Statistical Software. Minitab, Inc., www.minitab.com.
Morss, R. E., J. L. Demuth, and J. K. Lazo, 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974–991, https://doi.org/10.1175/2008WAF2007088.1.
National Research Council, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. The National Academies Press, 124 pp., https://doi.org/10.17226/11699.
National Weather Service, 2020a: Point Forecast Matrices product specifications document. NWS Doc., 7 pp., https://www.weather.gov/media/notification/dir/PFM_Specifications.pdf.
National Weather Service, 2020b: Hazard simplification. Accessed 20 February 2021, https://www.weather.gov/hazardsimplification/.
Perrin, A., and M. Anderson, 2019: Share of U.S. adults using social media, including Facebook, is unchanged since 2018. Pew Research Center, accessed 8 December 2020, https://www.pewresearch.org/fact-tank/2019/04/10/share-of-u-s-adults-using-social-media-including-facebook-is-mostly-unchanged-since-2018/.
Phan, M. D., B. E. Montz, S. Curtis, and T. M. Rickenbach, 2018: Weather on the go: An assessment of smartphone mobile weather application use among college students. Bull. Amer. Meteor. Soc., 99, 2245–2257, https://doi.org/10.1175/BAMS-D-18-0020.1.
Pohlert, T., 2014: The pairwise multiple comparison of mean ranks package (PMCMR). R Package 27, 9 pp., https://cran.microsoft.com/snapshot/2015-03-19/web/packages/PMCMR/vignettes/PMCMR.pdf.
Radford, L., J. C. Senkbeil, and M. Rockman, 2013: Suggestions for alternative tropical cyclone warning graphics in the USA. Disaster Prev. Manage., 22, 192–209, https://doi.org/10.1108/DPM-06-2012-0064.
R Core Team, 2020: R: A language and environment for statistical computing. R Foundation for Statistical Computing, https://r-project.org/.
Reed, J. R., and J. C. Senkbeil, 2020: Perception and comprehension of the extended forecast graphic: A survey of broadcast meteorologists and the public. Bull. Amer. Meteor. Soc., 101, E221–E236, https://doi.org/10.1175/BAMS-D-19-0078.1.
Reed, J. R., and J. C. Senkbeil, 2021: Modifying the extended forecast graphic to improve comprehension. Wea. Climate Soc., 13, 57–66, https://doi.org/10.1175/WCAS-D-20-0086.1.
Saunders, M. E., and J. C. Senkbeil, 2017: Perceptions of hurricane hazards in the mid-Atlantic region. Meteor. Appl., 24, 120–134, https://doi.org/10.1002/met.1611.
Saunders, M. E., K. D. Ash, and J. M. Collins, 2018: Usefulness of United States National Weather Service radar display as rated by website users. Wea. Climate Soc., 10, 673–691, https://doi.org/10.1175/WCAS-D-17-0108.1.
Sherman-Morris, K., 2005: Tornadoes, television and trust—A closer look at the influence of the local weathercaster during severe weather. Global Environ. Change, 6B, 201–210, https://doi.org/10.1016/j.hazards.2006.10.002.
Sherman-Morris, K., and A. M. Lea, 2016: An exploratory study on the influence of severe weather radar broadcasts. J. Oper. Meteor., 4, 108–122, https://doi.org/10.15191/nwajom.2016.0408.
Sinclair, M., 2019: Basic nonparametric tests. Accessed 10 September 2021, http://rstudio-pubs-static.s3.amazonaws.com/515456_e45b1bba99c54a86acccdc8802242d30.html.
Sivle, A. D., S. Dankert Kolstø, P. J. Kirkeby Hansen, and J. Kristiansen, 2014: How do laypeople evaluate the degree of certainty in a weather report? A case study of the use of the web service. Wea. Climate Soc., 6, 399–412, https://doi.org/10.1175/WCAS-D-12-00054.1.
Stewart, A. E., C. A. Williams, M. D. Phan, A. L. Horst, E. D. Knox, and J. A. Knox, 2016: Through the eyes of experts: Meteorologists’ perceptions of the probability of precipitation. Wea. Forecasting, 31, 5–17, https://doi.org/10.1175/WAF-D-15-0058.1.
Sutton, J., and L. M. Fischer, 2021: Understanding visual risk communication messages: An analysis of visual attention allocation and think-aloud responses to tornado graphics. Wea. Climate Soc., 13, 173–188, https://doi.org/10.1175/WCAS-D-20-0042.1.
The Weather Company, 2020: Weather icons set A. IBM, accessed 30 November 2020, https://mediacenter.ibm.com/media/ICON+Set+-+HD-A/0_9blhox1w/86954772.
Tullis, T., and B. Albert, 2013: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Elsevier/Morgan Kaufmann, 301 pp., https://doi.org/10.1016/C2011-0-00016-9.
Williams, C. A., and G. M. Eosco, 2021: Is a consistent message achievable? Defining “message consistency” for weather enterprise researchers and practitioners. Bull. Amer. Meteor. Soc., 102, E279–E295, https://doi.org/10.1175/BAMS-D-18-0250.1.
Williams, C. A., P. W. Miller, A. W. Black, and J. A. Knox, 2017: Throwing caution to the wind: National Weather Service wind products as perceived by a weather-salient sample. J. Oper. Meteor., 5, 103–120, https://doi.org/10.15191/nwajom.2017.0509.
Williams, C. A., A. J. Grundstein, and J. So, 2020: Should severe weather graphics wear uniforms? Understanding the effects of inconsistent convective outlook graphics on members of the public. 15th Symp. on Societal Applications: Policy, Research and Practice, Boston, MA, Amer. Meteor. Soc., 15.1, https://ams.confex.com/ams/2020Annual/webprogram/Paper365011.html.
Zabini, F., 2016: Mobile weather apps or the illusion of certainty. Meteor. Appl., 23, 663–670, https://doi.org/10.1002/met.1589.
Zabini, F., V. Grasso, R. Magno, F. Meneguzzo, and B. Gozzini, 2015: Communication and interpretation of regional weather forecasts: A survey of the Italian public. Meteor. Appl., 22, 495–504, https://doi.org/10.1002/met.1480.