• Abraham, S., R. Bartlett, M. Standage, A. Black, A. Charlton-Perez, and R . McCloy, 2015: Do location-specific forecasts pose a new challenge for communicating uncertainty? Meteor. Appl., 22, 554562, https://doi.org/10.1002/met.1487.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Demuth, J., J. K. Lazo, and R. E. Morss, 2011: Exploring variations in people’s sources, uses, and perceptions of weather forecasts. Wea. Climate Soc., 3, 177192, https://doi.org/10.1175/2011WCAS1061.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joslyn, S., and J. E. LeClerc, 2012: Uncertainty forecasts improve weather-related decisions and attenuate the effects of forecast error. J. Exp. Psychol., 18, 126140, https://doi.org/10.1037/a0025185.

    • Search Google Scholar
    • Export Citation
  • Joslyn, S., L. Nadav-Greenberg, and R. M. Nichols, 2009: Probability of precipitation: Assessment and enhancement of end-user understanding. Bull. Amer. Meteor. Soc., 90, 185194, https://doi.org/10.1175/2008BAMS2509.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kox, T., and A. H. Thieken, 2017: To act or not to act? Factors influencing the general public’s decision about whether to take protective action against severe weather. Wea. Climate Soc., 9, 299315, https://doi.org/10.1175/WCAS-D-15-0078.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lazo, J. K., R. E. Morss, and J. L. Demuth, 2009: 300 billion served. Bull. Amer. Meteor. Soc., 90, 785798, https://doi.org/10.1175/2008BAMS2604.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., J. L. Demuth, and J. K. Lazo, 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974991, https://doi.org/10.1175/2008WAF2007088.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., J. K. Lazo, and J. L. Demuth, 2010: Examining the use of weather forecasts in decision scenarios: Results from a US survey with implications for uncertainty communication. Meteor. Appl., 17, 149162, https://doi.org/10.1002/met.196.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Myers, L., 2019: Understanding the role of weather culture on messaging. 17th Southeast Severe Storms Symp., Starkville, MS, East Mississippi Chapter of the NWA and Amer. Meteor. Soc.

  • Pew Research Center, 2019: For local news, Americans embrace digital but still want strong community connection. Accessed 15 July 2020, https://www.journalism.org/2019/03/26/for-local-news-americans-embrace-digital-but-still-want-strong-community-connection/.

  • Phan, M. D., B. E. Montz, S. Curtis, and T. M. Rickenbach, 2018: Weather on the go: An assessment of smartphone mobile weather application use among college students. Bull. Amer. Meteor. Soc., 99, 22452257, https://doi.org/10.1175/BAMS-D-18-0020.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reed, J. R., 2019: 7-dayzed and confused: Exploring a disconnect between broadcast meteorologists and the public. 44th National Weather Association Meeting, Huntsville, AL, East Mississippi Chapter of the NWA and Amer. Meteor. Soc.

  • Reed, J. R., and J. C. Senkbeil, 2020: Perception and comprehension of the extended forecast graphic: A survey of broadcast meteorologists and the public. Bull. Amer. Meteor. Soc., 101, E221E236, https://doi.org/10.1175/BAMS-D-19-0078.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, A., and M. Anderson, 2018: Appendix A: Detailed table. Social media use in 2018, Pew Research Center, accessed 15 July 2020, https://www.pewresearch.org/internet/2018/03/01/social-media-use-2018-appendix-a-detailed-table/.

  • Statista, 2020: Device usage of Facebook users worldwide through July 2020. Accessed 4 October 2020, https://www.statista.com/statistics/377808/distribution-of-facebook-users-by-device/.

  • Stewart, A. E., C. A. Williams, M. D. Phan, A. L. Horst, E. D. Knox, and J. A. Knox, 2016: Through the eyes of the experts: Meteorologists’ perceptions of the probability of precipitation. Wea. Forecasting, 31, 517, https://doi.org/10.1175/WAF-D-15-0058.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • World Meteorological Organization, 2007: Examples of best practice in communicating weather information. WMO Tech Doc. 1409, 24 pp., https://library.wmo.int/doc_num.php?explnum_id=9368.

  • World Meteorological Organization, 2008: Guidelines on communicating forecast uncertainty. WMO Tech. Doc., 1422, 25 pp., https://library.wmo.int/doc_num.php?explnum_id=4687.

  • Zabini, F., 2016: Mobile weather apps or the illusion of certainty. Meteor. Appl., 23, 663670, https://doi.org/10.1002/met.1589.

  • Zabini, F., V. Grasso, R. Magno, F. Meneguzzo, and B. Gozzini, 2015: Communication and interpretation of regional weather forecasts: A survey of the Italian public. Meteor. Appl., 22, 495504, https://doi.org/10.1002/met.1480.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Four examples of new EFGs tested in this study.

  • View in gallery

    Example of current EFG format used at most television stations across the country.

  • View in gallery

    Footprint of participants in the general public survey posted on social media by three broadcast meteorologists in Alabama (n = 885).

  • View in gallery

    (a) Importance of the EFG to the public. (b) The number of days into the future that people believe forecasters can accurately predict (n = 885 for both questions).

All Time Past Year Past 30 Days
Abstract Views 96 102 0
Full Text Views 10 80 13
PDF Downloads 15 81 15

Modifying the Extended Forecast Graphic to Improve Comprehension

View More View Less
  • 1 Center for Advanced Public Safety, University of Alabama, Tuscaloosa, Alabama
  • | 2 Department of Geography, University of Alabama, Tuscaloosa, Alabama
© Get Permissions
Free access

Abstract

The extended forecast graphic (EFG) is a popular graphic used by meteorologists to convey weather information, but it is poorly understood by the public. Deficiencies in the format, content, and presentation of the EFG contribute to a decrease in the efficacy of this graphic and reduce the comprehension of weather information. The format of the EFG has largely gone unchanged since the graphic first became popular more than four decades ago. The goal of this research was to modify the format of the existing EFG to address current limitations that inhibit understanding and create confusion among the public. Data were gathered from an online survey of the public (n = 885). Four modified versions of the EFG were developed, evaluated, and compared with the existing EFG. Removing probability of precipitation (PoP) information, reducing the number of days shown, and switching to a horizontal layout featuring timing and intensity information resulted in higher percentages for comprehension of weather information and positive comments when compared with the current version. A majority of participants responded that forecasters could accurately predict the weather 3 days out, providing justification for the reduction in number of days shown in the modified EFGs. Results suggest that agencies and members of the meteorological community should continue evaluating and discussing the most effective ways to use graphics to convey weather information to their audiences.

Significance Statement

The extended forecast graphic is used by broadcast and government meteorologists to show forecast data and trends beyond 3 days. Although widely used, the graphic has proven to be confusing to the public. We experiment with various design and content changes in the graphic to reduce confusion caused by existing flaws in the current format of the graphic. We find that reducing the number of days to three and removing probabilities of precipitation to include more specific information improved comprehension. This finding is especially relevant given that existing extended forecast graphics almost always include probabilities of precipitation and increasingly are showing 10 or more days. Future studies should work to uncover other possible flaws in popular weather graphics.

Supplemental information related to this paper is available at the Journals Online website: https://doi.org/10.1175/WCAS-D-20-0086.s1.

© 2020 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Jacob R. Reed, jrreed2@ua.edu

Abstract

The extended forecast graphic (EFG) is a popular graphic used by meteorologists to convey weather information, but it is poorly understood by the public. Deficiencies in the format, content, and presentation of the EFG contribute to a decrease in the efficacy of this graphic and reduce the comprehension of weather information. The format of the EFG has largely gone unchanged since the graphic first became popular more than four decades ago. The goal of this research was to modify the format of the existing EFG to address current limitations that inhibit understanding and create confusion among the public. Data were gathered from an online survey of the public (n = 885). Four modified versions of the EFG were developed, evaluated, and compared with the existing EFG. Removing probability of precipitation (PoP) information, reducing the number of days shown, and switching to a horizontal layout featuring timing and intensity information resulted in higher percentages for comprehension of weather information and positive comments when compared with the current version. A majority of participants responded that forecasters could accurately predict the weather 3 days out, providing justification for the reduction in number of days shown in the modified EFGs. Results suggest that agencies and members of the meteorological community should continue evaluating and discussing the most effective ways to use graphics to convey weather information to their audiences.

Significance Statement

The extended forecast graphic is used by broadcast and government meteorologists to show forecast data and trends beyond 3 days. Although widely used, the graphic has proven to be confusing to the public. We experiment with various design and content changes in the graphic to reduce confusion caused by existing flaws in the current format of the graphic. We find that reducing the number of days to three and removing probabilities of precipitation to include more specific information improved comprehension. This finding is especially relevant given that existing extended forecast graphics almost always include probabilities of precipitation and increasingly are showing 10 or more days. Future studies should work to uncover other possible flaws in popular weather graphics.

Supplemental information related to this paper is available at the Journals Online website: https://doi.org/10.1175/WCAS-D-20-0086.s1.

© 2020 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Jacob R. Reed, jrreed2@ua.edu

1. Introduction

There remains limited published research about how weather forecast graphics primarily produced and disseminated by television stations are understood by the public. The extended forecast graphic (EFG), more commonly referred to as the 5-, 7-, or 10-day (etc.) forecast, is a popular method by which broadcast meteorologists (BMs) convey weather information, but it is poorly understood by the public (Reed and Senkbeil 2020). The EFG is a hallmark of television weather broadcasts and was identified as the most important graphic according to BMs, but there are numerous challenges that limit the public’s ability to comprehend the graphic (Reed and Senkbeil 2020). For example, the current format of the EFG does not allow for the full display of information about hazardous or severe weather. The EFG in many cases also does not contain the information that the general public prioritizes (Reed and Senkbeil 2020), such as precipitation intensity or timing (Demuth et al. 2011; Reed and Senkbeil 2020). Furthermore, there is almost never a display for nighttime weather. In addition, poor understanding of probability-of-precipitation (PoP) information (Kox and Theiken 2017; Stewart et al. 2016; Abraham et al. 2015; Zabini et al. 2015; Morss et al. 2010, 2008) has resulted in the public misinterpreting weather forecasts (Reed and Senkbeil 2020; Zabini et al. 2015; Joslyn et al. 2009). Furthermore, the format of the EFG has largely gone unchanged since the graphic first became popular more than four decades ago (Reed 2019).

Modified versions of the EFG were developed for this study that were based on feedback provided by members of the public and the BM community in surveys described in Reed and Senkbeil (2020). Each of the modified EFGs evaluated in this study addresses one or more of the limitations or flaws mentioned in the previous paragraph. Two major research questions are asked in this study:

  1. Do the modified EFGs result in better comprehension of weather information?

  2. What modifications to the graphics resulted in positive comments?

Numerous explanations are provided as answers to these questions. The methods section outlines the development and strategy behind creating the new EFGs in this study. Each graphic tested is described in its own section, with explanations of the hypothetical forecasts shown in each one. Another section of methods is devoted to discussing how the online survey was created and distributed through BMs on social media. The final section of methods explains the data analysis techniques and statistical procedures performed. The results section is organized into two sections so that it answers the two research questions above. The longest part of the results covers the comparison between each of the modified EFGs and the existing format. Limitations, future research, and conclusions are presented.

2. Methods

a. Modifying the existing EFG

Four modified EFGs were created by the authors and evaluated in this research (Figs. 1a–d). Each of the modified EFGs was created using Baron Lynx weather graphics software, version 1.03 (https://www.baronweather.com/baronlynx). The modified EFGs were designed to each test a limitation or deficiency of the current EFG (Reed and Senkbeil 2020; Fig. 2). Using different weather scenarios provides an avenue for revealing the extent to which different weather content may affect one’s ability to read a graphic. If all weather scenarios in the modified EFGs showed a forecast for snow, for example, and the study participants dislike snow, we might only generate negative comments about the forecast scenario and few or no comments about the format of the graphic. Therefore, to avoid any bias due to a particular forecast scenario, there are differences in the hypothetical forecast data shown and arrangement of forecast variables. The next several sections explain the design of each modified EFG shown in Figs. 1a–d as well as the similarities and differences between each of the new graphics and the existing format.

Fig. 1.
Fig. 1.

Four examples of new EFGs tested in this study.

Citation: Weather, Climate, and Society 13, 1; 10.1175/WCAS-D-20-0086.1

Fig. 2.
Fig. 2.

Example of current EFG format used at most television stations across the country.

Citation: Weather, Climate, and Society 13, 1; 10.1175/WCAS-D-20-0086.1

1) Fig. 1a

Figure 1a differs primarily from the existing format in two ways: it contains fewer days of information and is oriented horizontally rather than vertically. Three days of forecast data are shown in this new version of the EFG. A novelty of this graphic is that instead of organizing the day hourly, there were two boxes of information: one for morning weather and another for afternoon and evening weather. The goal of breaking the day’s forecast into two parts was to be able to illustrate differences in morning versus afternoon or evening weather, which cannot be done as explicitly in the existing EFG format. This utility is especially helpful for atypical weather situations, such as when the warmest part of the day is not during the afternoon. An example of how this kind of forecast is handled by the new EFG is shown on Tuesday’s forecast. The graphic displays that the morning temperature will fall from 68° to 56°, and then temperature will continue to fall from 56° to 44° through the afternoon and evening (In this paper, all temperatures are presented in degrees Fahrenheit). In the current EFG format with a single high and low temperature, there is no indication of when the high or low will occur, and this creates the possibility of someone misinterpreting the forecast in the event of atypical weather. Because it is generally (and correctly) assumed that the warmest part of the day is the afternoon, this graphic is more functional since it should prevent someone from being surprised by or unprepared for the temperature falling during the day.

The other two days’ forecasts show the temperature on Sunday rising from the 40s to the 70s and then falling to the 50s (all in degrees Fahrenheit). The morning forecast panel shows that dry weather is expected, but rain will develop in the afternoon and evening time periods. Monday’s forecast shows a colder scenario, with temperatures rising from 35° to 49°, falling to 21° in the evening. Additional forecast text on Monday’s evening forecast panel informs the public that temperatures will drop below freezing by 2000 local time (LT). One advantage of arranging the EFG horizontally rather than vertically is having the ability to include more descriptive text about impactful weather conditions.

Wind speed and wind direction are included in Fig. 1a. These variables may be of importance to some and are only occasionally included in the current format of the EFG. Here, a dedicated space is provided. Additionally, each day in this new EFG contains a box for a daily rain total, whereas the current format of the EFG does not allow for information about rainfall amounts and as such is almost never shown.

2) Figs. 1b and 1c

Figures 1b and 1c are formatted similarly and thus will both be described in this section. Similar to Fig. 1a, Figs. 1b and 1c are arranged horizontally and are designed to be read across each day and then down to the next day. These EFGs are unique in that they display each day’s forecast weather in hourly segments with multiple temperatures and icons for each time period. Previous research has shown the public has a tendency to misinterpret weather icons (Zabini 2016; Zabini et al. 2015; Joslyn et al. 2009) used on graphics like the EFG, but it is unclear whether or not the misinterpretation is due to the design of the icon itself, or the fact that in most cases a single icon is used to represent an entire day’s worth of weather—or possibly a combination of both. These two graphics also include a space for the meteorologist to include statements about the impacts of the forecast, such as severe or hazardous weather. Graphics in Figs. 1b and 1c share the advantage Fig. 1a has of being able to clearly display atypical weather conditions, such as temperatures falling throughout the day, but in this case with a finer temporal scale.

3) Fig. 1d

Figure 1d is the most unique of the new EFGs evaluated in this paper, and it is also the most different from the current EFG format. The only similarity Fig. 1d shares with the current format is that the graphic is oriented vertically rather than horizontally like Figs. 1a–c. The most striking element of this EFG is that it contains a meteogram to illustrate temperature patterns and bar graphs for anticipated rainfall. While it is common practice to use meteograms and bar graphs in television weather broadcasts, they are almost never a part of the EFG. The use of bar graphs and meteograms on television weather graphics has not been studied, so evaluating them as part of the EFG will provide useful insight as to whether or not these elements are effective at communicating weather forecasts in the broadcast format. The advantage of using a temperature meteogram is that it allows a chance to better visualize temperature changes over time, which is especially critical in the event of atypical weather. An example of this situation is shown on Tuesday’s forecast in Fig. 1d. The meteogram shows the temperature dropping from the middle 70s to the 50s (°F) from the morning to the afternoon. On the current EFG format, the meteorologist is limited in how this situation can be explained or illustrated. Should the highest temperature of the day be put as the forecast high, even if it occurs at a time other than the afternoon? Should the temperature that will occur in the middle of the afternoon be shown as the high temperature? With the new EFG, there is less opportunity for confusion because the decreasing temperature trend throughout the day is more clearly illustrated.

The EFG in Fig. 1d has space between the temperature meteogram at the top of the graphic and the rainfall bar graph at the bottom for the meteorologist to add icons and/or text icons. This allows this graphic to be customized based on what type of weather is expected, with more room than on the current EFG format. The bar graphs at the bottom of the graphic show accumulated rainfall over each of the time periods shown on the EFG. While this is more detail than the current EFG format provides, it is possible this arrangement is not as effective or easy to understand compared to how forecast rainfall is communicated on EFGs in Figs. 1a–c. Even so, it is hoped that feedback gathered from this EFG will aid in creating future graphics that are effective in communicating all aspects of a weather forecast.

b. Survey design and distribution

1) Public survey

A 27-question online survey consisting of demographic, weather graphic opinion, interpretation and comprehension, and open-ended questions was created using Qualtrics. A majority of the questions in this survey were related to each of the new EFGs described in the previous section. One screener question was included to ensure that only individuals who routinely watch television weather broadcasts participated. Of the more than 1300 people who opened the survey, 885 passed the screener question. Therefore, the sample size is 885. Because participants could skip questions or parts of questions, occasionally the sample size for an individual question was lower than 885.

2) Survey distribution

Three prominent Alabama BMs graciously agreed to distribute this survey and solicit responses through their social media platforms (e.g., Facebook and Twitter). The greatest number of responses were generated via Facebook. The spatial distribution of responses mirrored each of the meteorologists’ television market areas (Fig. 3). The highest concentration of participants was in central and northern Alabama, overlapping with locations where two of the BMs work. Another grouping of responses was located closer to the Alabama Gulf Coast and Florida Panhandle, where the third BM works. Other participants were located outside the state of Alabama, with some living as far west as California and others living near the Great Lakes and Northeast.

Fig. 3.
Fig. 3.

Footprint of participants in the general public survey posted on social media by three broadcast meteorologists in Alabama (n = 885).

Citation: Weather, Climate, and Society 13, 1; 10.1175/WCAS-D-20-0086.1

c. Data analysis

A mixture of quantitative and qualitative procedures was used to compare and contrast the comprehension of each of the new EFGs versus the current format. Additionally, feedback was solicited about likes and dislikes for each graphic with the intent of determining which graphic scored high in comprehension and was also liked. These procedures are divided into the following sections to facilitate the organization and rationale for the analysis.

1) Comprehension of the new EFGs

Reed and Senkbeil (2020) found that 80% of people were not able to answer basic questions about the weather forecast after looking at the EFG in its current format. This inability to answer was largely due to the fact that the information being asked about was not displayed on the EFG. The questions included identifying the forecast high and low temperatures, timing of precipitation, intensity of precipitation, and expected amounts of precipitation. Nearly identical questions were asked of participants in this survey with each new EFG, as well as with the current format in order to assess the robustness of the findings in Reed and Senkbeil (2020). Each of these questions was closed response with correct answers and therefore are graded and discussed as percentages of correct or incorrect answers. The numbered list below outlines the questions asked about each of the new EFGs and the current format. The graphic being referred to by each question is listed in the title of each section:

  1. EFG in Fig. 1a

    1. What will the low temperature be Sunday morning?

    2. What will the high temperature be on Tuesday?

  2. EFG in Fig. 1b

    1. How much rain would you expect to fall on Saturday?

    2. Approximately when would you expect rain to begin on Friday?

  3. EFG in Fig. 1c

    1. Approximately when would you expect rain to begin on Thursday?

  4. EFG in Fig. 1d

    1. Approximately when will the high temperature occur on Tuesday?

    2. Approximately how much rain will fall on Tuesday?

  5. EFG in Fig. 2

    1. Approximately how much rain will fall on Wednesday?

    2. Which day of the week will feature the heaviest rain?

The questions listed above are referenced as “comprehension” questions in the results section. It is important to note that the order in which the new EFGs and current format were shown was randomized, as were the answer choices to each question.

2) Collecting feedback on the modified graphics

After looking at each of the modified EFGs, participants were prompted to provide feedback on what they liked and disliked about each graphic. Of the participants that answered, the majority provided one brief phrase or sentence, while a smaller percentage wrote two or three sentences. The authors first independently organized the responses into a number of codes and then met to narrow these codes down to six themes. The authors then independently classified comments for each graphic into one of the six themes. The percentages of responses for each theme were then calculated for each author along with the percentage of responses shared by both authors for each theme. These themes are as follows: 1) positive, 2) positive with specific comments for improvement, 3) neutral/could not determine, 4) negative, 5) negative with specific comments for improvement, and 6) unclassified. Cohen’s kappa tests were used to evaluate the interrater agreement among the authors and the percentages of each theme for each of the four modified EFGs.

3) Revisiting the purpose of the EFG

To gauge the robustness of the notion that the EFG is an important graphic, we asked participants to rank their level of agreement with the following statement: “When I watch a weather broadcast, I pay close attention to the extended forecast.” Values were coded from 1 (strongly disagree) to 4 (strongly agree) after the survey was closed and the mean values are presented in the results section. Participants were also asked how many days into the future they believed forecasters could accurately predict. This question was designed to gauge how many days into the future the general public wanted to see on the EFG based on how far into the future they felt forecasts were accurate.

3. Results

a. Demographics of the public survey

Despite our large sample (n = 885), there was a lack of diversity among participants (Table 1). There was a large spread in the ages of participants, with the greatest number falling between 35 and 64 years old (approximately 68%). More than 85% of participants reported themselves as having at least some college education; approximately 50% had at least a 4-yr college degree. Of the survey participants, 66% were female and 33% were male. A total of 845 of the 885 people who took the survey identified themselves as Caucasian (95.7%). According to data from a Pew Research Center (Smith and Anderson 2018) study of adults in the United States, 62% of men and 74% of women use Facebook. The same study showed that use of Facebook across racial and ethnic groups was more evenly split: 70% of African American adults report using Facebook, 67% of Caucasians use Facebook, and 73% of Hispanics surveyed said they use Facebook. Although beyond the scope of this paper, we believe that the lack of diversity in our sample is an unintended but critical finding that indicates a possible lack of diversity in BMs’ social media reach that will be revisited later in the limitations section.

Table 1.

Demographic data for the general public survey.

Table 1.

b. EFG comprehension: Comparing the current EFG to the new EFGs

Comprehension for the modified EFGs was also analyzed by unique questions about timing or weather impacts for each graphic (Table 2). Some of these questions could have been more difficult to interpret for participants given the weather scenario presented and depending on how participants took the survey (i.e., desktop computer vs phone). These possibilities were taken into consideration for the discussion in this section. Despite these possible inconsistencies in the difficulty of questions between graphics, a pattern emerged that can be combined with the feedback provided by participants to determine which graphic style works best.

Table 2.

Results of the comprehension questions for each new EFG and the current format. The percent correct column is left blank for one question because the correct answer to that question was “I’m not sure,” because the answer was not on the graphic. The final question is presented as number who chose that day and the percent of total respondents to which that number corresponds.

Table 2.

1) Fig. 1a

For Fig. 1a, 98% of participants were able to correctly identify the forecast low temperature for Sunday morning (Table 2). There was, however, some confusion when asked about the high temperature expected on Tuesday. As was described in the methods section, Tuesday’s forecast featured an atypical weather day where the temperature fell throughout the day. The graphic shows the temperature falling from 68° to 56° in the morning and then from 56° to 44° in the afternoon and evening. Of the 31% of incorrect responses to this question, 29% of people chose 56° as the expected high temperature. We believe several participants chose this value because it was the first value in the afternoon weather forecast panel. Because the high temperature usually occurs during the afternoon, participants may have instinctually glanced at only the afternoon forecast panel when looking for the high temperature. In this case, many people overlooked the temperature that was 10° higher in the morning forecast panel. This suggests that people may not understand that daily meteorological highs and lows are assessed for the 24-h period from midnight to midnight.

2) Fig. 1b

The EFG in Fig. 1b had more than 90% correct answers to both questions (Table 2). Participants were able to identify forecast amounts of rainfall on Saturday and timing of when rainfall would begin on Friday with a high level of accuracy. Although it has the same style of graphic as Fig. 1c, the results in the following paragraph sharply contrast with the high percentages for Fig. 1b. It is hypothesized that the dividing bars in Fig. 1b were more clearly placed and consistent with the icons.

3) Fig. 1c

Figure 1c had the highest number of incorrect responses (Table 2) to a question than any other EFG, including the current format. Participants were asked when they thought the rain would begin on Thursday based on the graphic. Only 17% correctly answered 1000 LT; almost 80% of the responses were incorrect. EFG 1C provides an interesting example to explore further. Thursday’s forecast sky conditions and impact are clearly divided between sun and rain icons and no rain and “T-storms: 1–2 in. of rain” halfway between 0900 and 1200 LT. Of the incorrect responses, many people answered “Noon” or “12:30–1 p.m.,” which line up with where the rain icons begin. This suggests that the people looking at the graphic may have been paying more attention to where the icons were located rather than the dividing bar that was located between 0900 and 1200 LT. This finding suggests that future graphics following this or a similar format (even if not in the EFG), should carefully consider where icons are placed in relation to time, as this may influence comprehension of the forecast.

4) Fig. 1d

Almost all participants (95%) correctly answered that Tuesday’s high temperature would occur in the morning (Table 2). This result is encouraging because it suggests the public may not have trouble in reading and interpreting temperature meteograms. If they are understood on this graphic, they may be as equally understood when shown in other graphics or on other platforms. Fewer were sure about interpreting the bar graphs. Incorrect answers made up 33% of responses. While more than half of responses were correct, optimism must be interpreted with caution here because the bar graphs were not labeled in a clear manner.

5) Fig. 2

Results for the comprehension questions for the existing EFG (Fig. 2) are in strong agreement with the findings described in Reed and Senkbeil (2020): in its current format, the EFG is a source of confusion because people cannot accurately answer questions about the forecast after looking at the graphic. In this round of questions, participants were not able to correctly identify how much rain would fall on Wednesday based on the information in the EFG (Table 2). Approximately 80% said they were not sure. The second question related to this EFG required people to identify which day would feature the heaviest rainfall. More than half selected Thursday. We believe this is because Thursday has a high probability of precipitation (60%), and previous research indicates the tendency for people to mistakenly conflate high PoPs with high amounts or high intensity of rainfall (Zabini 2016; Zabini et al. 2015; Joslyn et al. 2009). Interestingly, although Thursday has a higher PoP, the icon for Wednesday seems to indicate heavier precipitation, and still over half selected Thursday as the day with the heaviest rainfall potential. This may suggest that the public places more emphasis on the PoP than the weather icon when evaluating what intensity or amount of rain to expect. Still, in this case it cannot be determined from this graphic whether Thursday will have the heaviest rainfall. Therefore, the conclusion drawn by the public may have been different than what the meteorologist who prepared the graphic intended.

c. EFG feedback: What modifications did people like?

Participants were prompted to provide comments on what they liked and disliked about each graphic following the comprehension questions. An open-ended style question was used to gain a better understanding of what modifications made interpreting the graphic or answering the questions easier or more difficult. A total of 1409 responses were collected from participants for the modified EFGs. Some participants provided comments for more than one graphic. Cohen’s kappa is reported for each graphic as a measure of agreement between the authors’ categorization of participant responses. Feedback for each of the modified EFGs and the corresponding Cohen’s kappa (κ) are presented in the following sections and are summarized in Table 3.

Table 3.

Percentages of comments (n = 180) in each of the six themes by author. Comment themes are coded as follows: 1) positive, 2) positive with specific comments for improvement, 3) neutral/could not determine, 4) negative, 5) negative with specific comments for improvement, and 6) unclassified (double asterisks for Cohen’s kappa represent a p value of less than 0.001).

Table 3.

1) Fig. 1a

There was almost perfect agreement between authors on this EFG (κ = 0.938; p < 0.001). A value of 27.2% of the feedback for Fig. 1a was classified as “positive” with another 8.6% in the “positive with specific comments for improvement” theme (Table 3). Approximately 22% of responses included comments marked as “negative.” Additionally, about 22.5% of participants left comments that were classified as “negative with specific comments for improvement,” producing a net negative reaction to this EFG. Participants generally liked the inclusion of the total rain forecast for the day and the wind speed forecast, however, many did not like the “rising to” and “falling to” wording between temperature ranges and also commented that they were confused about the apparent temperature swing between the evening of Monday and morning of Tuesday. Similar to the current format of the EFG, where atypical weather is difficult to communicate, the large swing in temperature present between Monday night and Tuesday was not easily interpreted by participants.

2) Fig. 1b

Figure 1b produced the highest percentage of positive responses and lowest percentage of negative responses of all of the modified EFGs. The modifications to this graphic included specific timing information and clear language about impacts in a horizontal layout—elements that were prioritized as higher by the public (Reed and Senkbeil 2020). Coupled with the high comprehension results, Fig. 1b is a clear winner for an EFG that is understood and well liked. Approximately 38% of responses were classified as positive, whereas only 10% were negative. When including the specific comment themes, this EFG has the highest net positive rating. Feedback suggested that this graphic was easy to read and understand. Participants liked the red lettering of the severe weather forecast in the first day’s scenario because it was eye-catching and commanded their attention. A common negative theme was that the weather icons were too small and on the third day’s forecast were particularly hard to distinguish. Again, there was near-perfect agreement between the authors’ ratings on Fig. 1b (κ = 0.932; p < 0.001).

3) Fig. 1c

The layout of the graphic in Fig. 1c was identical to that of Fig. 1b, but a different weather scenario was presented. Figure 1c likewise produced frequent positive responses and a net positive response almost as high as Fig. 1b. 32% of participant feedback on this graphic was classified as positive. About 11% of the feedback was strictly negative. Similar to Fig. 1b, there was also near perfect agreement between the ratings of the authors on the comments for this graphic (κ = 0.927; p < 0.001).

4) Fig. 1d

Figure 1d was not popular. This graphic had the highest percentage of purely negative feedback and the lowest percentage of purely positive comments. Figure 1d was unique in that it presented temperatures in the form of a meteogram, where sudden or drastic changes in temperatures can be seen more easily. Several of the positive comments from participants noted this was a good feature of this graphic. However, the presentation of the icons in the center of the graphic and the bar graphs for rain accumulation at the bottom of the graphic drew strong criticism. Participants felt like the impact icons in the center were wasted space and they had trouble determining what the bar graphs meant. Agreement between the authors on classification of comments for Fig. 1d was strong (κ = 0.893; p < 0.001).

d. Number of days shown in an EFG

Our results show strong evidence that the EFG is a graphic the public pays attention to (Fig. 4a). Lazo et al. (2009) found that the public’s confidence in weather forecasts is higher in the short term (days 1–3) than in the long term, where more than half of participants in that study had “very low confidence” in forecasts 7–14 days into the future. Additionally, previous research has shown the public has a tendency to begin paying attention to impending weather events and making decisions about them from two to three days in advance (Myers 2019). These studies in particular revealed a sharp difference in the usefulness of weather forecasts for the short term versus those of the long term. In other words, the general public does not view the short-term forecast in the same manner as the long-term forecast. Perhaps this is why the new EFGs with fewer days into the future of forecast information were preferred over the current format.

Fig. 4.
Fig. 4.

(a) Importance of the EFG to the public. (b) The number of days into the future that people believe forecasters can accurately predict (n = 885 for both questions).

Citation: Weather, Climate, and Society 13, 1; 10.1175/WCAS-D-20-0086.1

We asked participants how many days into the future they felt forecasters could accurately predict, and the results are presented in Fig. 4b; 58% of the public answered three days or less. These results and analyses are not intended to eliminate long-range forecasts or discussions of future trends in weather rather than short-term specifics. Indeed, people should be alerted to potential hazardous weather, such as a severe weather outbreak, landfalling hurricane, or dangerous heat wave, as far in advance as possible. However, the existing EFG format is not equipped to communicate all the hazards and necessary information required to effectively warm people for these kinds of events (Reed and Senkbeil 2020).

4. Limitations and future research

We identify the following limitations associated with the study. Although two of our graphics performed well for comprehension and generated frequent positive comments, there was a large percentage of negative responses for each graphic. Because our survey was distributed via Facebook, a platform most users access primarily through mobile devices (Statista 2020), most participants took our survey on a mobile device, and this may have contributed to some of the negative comments. This is especially seen in the number of comments where a participant noted they were not able to clearly see one or more aspects of a graphic.

Because broadcast meteorologists were used to distribute and solicit responses for the survey on their personal social channels, the spatial distribution and demographic makeup of participants offered insight into what groups of people these broadcast meteorologists are reaching. Our participants were not a representative sample of the general population, and minority groups were underrepresented in the survey sample. Because social media is increasingly becoming a popular method for weather information (Myers 2019; Pew Research Center 2019; Smith and Anderson 2018; Phan et al. 2018), this underrepresentation is concerning. Future research is needed to assess whether this limited reach is a more prevalent issue across the country.

The modified EFGs were designed to each test a limitation or deficiency of the current EFG. For this reason, the decision was made to present varying weather forecast information in each of the experimental EFGs. Furthermore, it would have been impossible given the modifications to each new EFG and the design of the current EFG to create identical weather scenarios across all five graphics. It is worthwhile to hypothesize about how the contents of the weather graphic may affect a person’s ability to comprehend information or their overall perception of a graphic. For example, would a graphic showing a severe weather forecast be less liked because the participant does not like severe weather? Would a participant say a graphic showing monotonous weather helps them plan better simply because of the consistent forecast? These are important questions to consider with limited answers from this research. We note that Figs. 1b and 1c were designed to show differences in weather information presented with an identical style and format. The percentage of positive comments and that both Figs. 1b and 1c were ranked highest, it suggests that in this example, the weather forecast content was not a major influence on their rating. We note that for Fig. 1a, many participants did comment on the weather scenario shown and this possibly had a negative influence on their rating. This information could be very useful to BMs and other members of the weather community that create graphics. This would have been undiscovered had we not chosen that particular scenario. However, a more robust investigation of the extent to which forecast information style and format may affect people’s ability to understand graphics is ripe for future study. For example, should there be a template for communicating high-impact weather that is different from that of routine weather?

One final limitation in this research is the role of uncertainty in communication. Research has shown that the public is receptive to the inclusion of uncertainty information in forecasts. Including uncertainty information can greatly enhance the value of a forecast and promote better decision-making with less potential for confusion (Joslyn and LeClerc 2012; Morss et al. 2008, 2010; World Meteorological Organization 2007, 2008). Our modified graphics, however, were designed primarily to evaluate whether the exclusion of PoP information and a reduction in number of days affected the EFGs likeability. Therefore, other forecast information included on our graphics was deterministic as is typically the case on EFGs currently in use. Making additional graphics to evaluate the role of uncertainty would have potentially been too many to compare and may have been confusing for the public to evaluate in one survey. We suggest that the evaluation of alternative EFGs with a variety of uncertainty information should be a priority for future research.

5. Conclusions

A primary goal of this study was to create new EFGs to better communicate weather forecasts by including more of the information people want, but without sacrificing the likeability or complicating the graphic. Figure 1b emerged as a winner with high comprehension scores and the highest percentages of positive ratings. The modifications to this graphic included specific timing information and clear language about impacts in a horizontal layout. Timing and intensity were the primary elements that people want to see in an EFG and are missing in the current format (Reed and Senkbeil 2020). Despite its popularity, it still had approximately 30% negative, or negative with specific comment responses. This result accentuates the difficulties in creating graphics that are universally understood and well liked, while also emphasizing how poorly the current EFG is performing in comprehension and likeability.

The new EFGs tested in this research seemed to be easier to comprehend, based on the level of accuracy with which people were able to answer basic forecast questions. Some of the new EFGs tested had near 90% accuracy rates, whereas 80%–90% of respondents were unsure of the correct answers after looking at the current EFG format both in this survey and in Reed and Senkbeil (2020). Additionally, our results suggest that people can interpret meteograms with relative ease, although many participants did not like the meteogram in Fig. 1d. This is an important polarized finding given that meteograms are already used in other parts of weather broadcasts on other graphics. Using meteograms on the EFG enables people to better interpret information regarding intensity or duration of a variable like precipitation, or drastic or sudden changes in temperature—neither of which can be easily illustrated given the limitations of the existing EFG format.

The findings presented within this paper have created new research questions that should be answered with future projects. Our results suggest the comprehension of the EFG may be improved by showing fewer days allowing for more specific information to be presented, such as timing and intensity of precipitation. The biggest difference between the current format of the EFG and the new ones put forth in this paper is that the new EFGs display fewer days and have no PoP information. Reducing the number of days shown on the graphics from seven to three allows for inclusion of the information that people prioritize and more specific information for each day. It was found that showing fewer days of forecast information on the EFG and removing PoP information did not appear to lessen the graphic’s appeal.

Any modifications to existing graphics or the creation of new graphics would require careful consideration and discussion by the meteorological community before changes are formally implemented. The purpose of this study is to highlight and offer possible solutions to limitations in existing weather graphics. Furthermore, it is hoped that this study, in conjunction with the findings of Reed and Senkbeil (2020), will inspire more critical evaluation of the efficacy of all weather forecast graphics produced and disseminated by private and government meteorological entities.

Acknowledgments

The authors of the paper thank Alan Sealls, Jason Simpson, and James Spann for soliciting responses to the public survey on their social media channels. We also thank the University of Alabama Graduate School and Department of Geography for providing funding to complete and present this research at regional and national conferences. In addition, we thank the anonymous reviewer of this paper for providing constructive feedback that has improved the quality of this paper.

Data availability statement

A copy of the survey administered in this study is available in the online supplemental material. Because of the data involving human subjects and being under the purview of the University of Alabama’s Institutional Review Board, data cannot be made openly available. Requests for access to the data should be made to the corresponding author.

REFERENCES

  • Abraham, S., R. Bartlett, M. Standage, A. Black, A. Charlton-Perez, and R . McCloy, 2015: Do location-specific forecasts pose a new challenge for communicating uncertainty? Meteor. Appl., 22, 554562, https://doi.org/10.1002/met.1487.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Demuth, J., J. K. Lazo, and R. E. Morss, 2011: Exploring variations in people’s sources, uses, and perceptions of weather forecasts. Wea. Climate Soc., 3, 177192, https://doi.org/10.1175/2011WCAS1061.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joslyn, S., and J. E. LeClerc, 2012: Uncertainty forecasts improve weather-related decisions and attenuate the effects of forecast error. J. Exp. Psychol., 18, 126140, https://doi.org/10.1037/a0025185.

    • Search Google Scholar
    • Export Citation
  • Joslyn, S., L. Nadav-Greenberg, and R. M. Nichols, 2009: Probability of precipitation: Assessment and enhancement of end-user understanding. Bull. Amer. Meteor. Soc., 90, 185194, https://doi.org/10.1175/2008BAMS2509.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kox, T., and A. H. Thieken, 2017: To act or not to act? Factors influencing the general public’s decision about whether to take protective action against severe weather. Wea. Climate Soc., 9, 299315, https://doi.org/10.1175/WCAS-D-15-0078.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lazo, J. K., R. E. Morss, and J. L. Demuth, 2009: 300 billion served. Bull. Amer. Meteor. Soc., 90, 785798, https://doi.org/10.1175/2008BAMS2604.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., J. L. Demuth, and J. K. Lazo, 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974991, https://doi.org/10.1175/2008WAF2007088.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., J. K. Lazo, and J. L. Demuth, 2010: Examining the use of weather forecasts in decision scenarios: Results from a US survey with implications for uncertainty communication. Meteor. Appl., 17, 149162, https://doi.org/10.1002/met.196.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Myers, L., 2019: Understanding the role of weather culture on messaging. 17th Southeast Severe Storms Symp., Starkville, MS, East Mississippi Chapter of the NWA and Amer. Meteor. Soc.

  • Pew Research Center, 2019: For local news, Americans embrace digital but still want strong community connection. Accessed 15 July 2020, https://www.journalism.org/2019/03/26/for-local-news-americans-embrace-digital-but-still-want-strong-community-connection/.

  • Phan, M. D., B. E. Montz, S. Curtis, and T. M. Rickenbach, 2018: Weather on the go: An assessment of smartphone mobile weather application use among college students. Bull. Amer. Meteor. Soc., 99, 22452257, https://doi.org/10.1175/BAMS-D-18-0020.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reed, J. R., 2019: 7-dayzed and confused: Exploring a disconnect between broadcast meteorologists and the public. 44th National Weather Association Meeting, Huntsville, AL, East Mississippi Chapter of the NWA and Amer. Meteor. Soc.

  • Reed, J. R., and J. C. Senkbeil, 2020: Perception and comprehension of the extended forecast graphic: A survey of broadcast meteorologists and the public. Bull. Amer. Meteor. Soc., 101, E221E236, https://doi.org/10.1175/BAMS-D-19-0078.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, A., and M. Anderson, 2018: Appendix A: Detailed table. Social media use in 2018, Pew Research Center, accessed 15 July 2020, https://www.pewresearch.org/internet/2018/03/01/social-media-use-2018-appendix-a-detailed-table/.

  • Statista, 2020: Device usage of Facebook users worldwide through July 2020. Accessed 4 October 2020, https://www.statista.com/statistics/377808/distribution-of-facebook-users-by-device/.

  • Stewart, A. E., C. A. Williams, M. D. Phan, A. L. Horst, E. D. Knox, and J. A. Knox, 2016: Through the eyes of the experts: Meteorologists’ perceptions of the probability of precipitation. Wea. Forecasting, 31, 517, https://doi.org/10.1175/WAF-D-15-0058.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • World Meteorological Organization, 2007: Examples of best practice in communicating weather information. WMO Tech Doc. 1409, 24 pp., https://library.wmo.int/doc_num.php?explnum_id=9368.

  • World Meteorological Organization, 2008: Guidelines on communicating forecast uncertainty. WMO Tech. Doc., 1422, 25 pp., https://library.wmo.int/doc_num.php?explnum_id=4687.

  • Zabini, F., 2016: Mobile weather apps or the illusion of certainty. Meteor. Appl., 23, 663670, https://doi.org/10.1002/met.1589.

  • Zabini, F., V. Grasso, R. Magno, F. Meneguzzo, and B. Gozzini, 2015: Communication and interpretation of regional weather forecasts: A survey of the Italian public. Meteor. Appl., 22, 495504, https://doi.org/10.1002/met.1480.

    • Crossref
    • Search Google Scholar
    • Export Citation

Supplementary Materials

Save