• Abraham, S., R. Bartlett, M. Standage, A. Black, A. Charlton-Perez, and R. McCloy, 2015: Do location specific forecasts pose a new challenge for communicating uncertainty? Meteor. Appl., 22, 554562, https://doi.org/10.1002/met.1487.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bauer, P., A. Thorpe, and G. Brunet, 2015: The quiet revolution of numerical weather prediction. Nature, 525, 4755, https://doi.org/10.1038/nature14956.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Federico, S., 2011: Verification of surface minimum, mean, and maximum temperature forecasts in Calabria for summer 2008. Nat. Hazards Earth Syst. Sci., 11, 487500, https://doi.org/10.5194/nhess-11-487-2011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fleischhut, N., S. M. Herzog, and R. Hertwig, 2020: Weather literacy in times of climate change. Wea. Climate Soc., 12, 435452, https://doi.org/10.1175/WCAS-D-19-0043.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fundel, V. J., N. Fleischhut, S. M. Herzog, M. Göber, and R. Hagedorn, 2019: Promoting the use of probabilistic weather forecasts through a dialogue between scientists, developers and end users. Quart. J. Roy. Meteor. Soc., 145, 210231, https://doi.org/10.1002/qj.3482.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gigerenzer, G., R. Hertwig, E. van den Broek, B. Fasolo, and K. V. Katsikopoulos, 2005: A 30% chance of rain tomorrow: How does the public understand probabilistic weather forecasts? Risk Anal., 25, 623629, https://doi.org/10.1111/j.1539-6924.2005.00608.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., F. Pastor, and M. J. Estrela, 2011: Sensitivity of a mesoscale model to different convective parameterization schemes in a heavy rain event. Nat. Hazards Earth Syst. Sci., 11, 343357, https://doi.org/10.5194/nhess-11-343-2011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., M. J. Estrela, and V. Caselles, 2014a: Operational forecasting of daily summer maximum and minimum temperatures in the Valencia region. Nat. Hazards, 70, 10551076, https://doi.org/10.1007/s11069-013-0861-1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., V. Caselles, and M. J. Estrela, 2014b: Real-time weather forecasting in the western Mediterranean basin: An application of the RAMS model. Atmos. Res., 139, 7189, https://doi.org/10.1016/j.atmosres.2014.01.011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., M. J. Estrela, and V. Caselles, 2015: Verification of the RAMS-based operational weather forecast system in the Valencia region: A seasonal comparison. Nat. Hazards, 75, 19411958, https://doi.org/10.1007/s11069-014-1408-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., V. Caselles, M. J. Estrela, and R. Niclòs, 2016: Impact of initial soil temperature derived from remote sensing and numerical weather prediction datasets on the simulation of extreme heat events. Remote Sens., 8, 589, https://doi.org/10.3390/rs8070589.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., V. Caselles, M. J. Estrela, and J. J. Miró, 2018: Comparative assessment of RAMS and WRF short-term forecasts over eastern Iberian Peninsula using various in-situ observations, remote sensing products and uncoupled land surface model datasets. Atmos. Res., 213, 476491, https://doi.org/10.1016/j.atmosres.2018.06.022.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., R. Niclòs, M. J. Estrela, V. Caselles, and M. J. Barberà, 2019: Simulation of extreme heat events over the Valencia coastal region: Sensitivity to initial conditions and boundary layer parameterizations. Atmos. Res., 218, 315334, https://doi.org/10.1016/j.atmosres.2018.12.016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez Doménech, I., S. Molina Palacios, and J. L. Soler Llorens, 2019: Percepción y valoración de las predicciones meteorológicas: Un estudio exploratorio entre estudiantes del Grado de Ciencias del Mar de la Universidad de Alicante. Investigación e Innovación en la Enseñanza Superior: Nuevos Contextos, Nuevas Ideas, R. Roig-Vila, Ed., Octaedro, 582–591.

  • Handmer, J., and B. Proudley, 2007: Communicating uncertainty via probabilities: The case of weather forecasts. Environ. Hazards, 7, 7987, https://doi.org/10.1016/j.envhaz.2007.05.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Howe, L. C., B. MacInnis, J. A. Krosnick, E. M. Markowitz, and R. Socolow, 2019: Acknowledging uncertainty impacts public acceptance of climate scientists’ predictions. Nat. Climate Change, 9, 863867, https://doi.org/10.1038/s41558-019-0587-5.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joslyn, S., and S. Savelli, 2010: Communicating forecast uncertainty: Public perception of weather forecast uncertainty. Meteor. Appl., 17, 180195, https://doi.org/10.1002/met.190.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joslyn, S., and J. E. LeClerc, 2012: Uncertainty forecasts improve weather-related decisions and attenuate the effects of forecast error. J. Exp. Psychol. Appl., 18, 126140, https://doi.org/10.1037/a0025185.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joslyn, S., L. Nadav-Greenberg, and R. M. Nichols, 2009: Probability of precipitation: Assessment and enhancement of end-user understanding. Bull. Amer. Meteor. Soc., 90, 185194, https://doi.org/10.1175/2008BAMS2509.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Juanchich, M., and M. Sirota, 2019: Not as gloomy as we thought: Reassessing how the public understands probability of precipitation forecasts. J. Cognit. Psychol., 31, 116129, https://doi.org/10.1080/20445911.2018.1553884.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Konold, C., 1989: Informal conceptions of probability. Cognit. Instr., 6, 5998, https://doi.org/10.1207/s1532690xci0601_3.

  • Lazo, J. K., R. E. Morss, and J. L. Demuth, 2009: 300 billion served. Bull. Amer. Meteor. Soc., 90, 785798, https://doi.org/10.1175/2008BAMS2604.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • LeClerc, J., and S. Joslyn, 2012: Odds ratio forecasts increase precautionary action for extreme weather events. Wea. Climate Soc., 4, 263270, https://doi.org/10.1175/WCAS-D-12-00013.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Martínez Fernández, L. C., and J. Olcina Cantos, 2019: La enseñanza escolar del tiempo atmosférico y del clima en España: Currículo educativo y propuestas didácticas. An. Geogr. Univ. Complutense, 39, 125148, https://doi.org/10.5209/aguc.64680.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., J. L. Demuth, and J. K. Lazo, 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974991, https://doi.org/10.1175/2008WAF2007088.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., J. K. Lazo, and J. L. Demuth, 2010: Examining the use of weather forecasts in decision scenarios: Results from a US survey with implications for uncertainty communication. Meteor. Appl., 17, 149162, https://doi.org/10.1002/met.196.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., S. Lichtenstein, B. Fischoff, and R. L. Winkler, 1980: Misinterpretations of precipitation probability forecasts. Bull. Amer. Meteor. Soc., 61, 695701, https://doi.org/10.1175/1520-0477(1980)061<0695:MOPPF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • National Research Council, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. National Academies Press, 124 pp.

  • O’Hanrahan, P. O., and C. Sweeney, 2013: Odds on weather: Probabilities and the public. Weather, 68, 247250, https://doi.org/10.1002/wea.2137.

  • Pastor, F., I. Gómez, and M. J. Estrela, 2010: Numerical study of the October 2007 flash flood in the Valencia region (eastern Spain): The role of orography. Nat. Hazards Earth Syst. Sci., 10, 13311345, https://doi.org/10.5194/nhess-10-1331-2010.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peachey, J. A., D. M. Schultz, R. Morss, P. J. Roebber, and R. Wood, 2013: How forecasts expressing uncertainty are perceived by UK students. Weather, 68, 176181, https://doi.org/10.1002/wea.2094.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Persson, A., 2014: We meteorologists cannot escape probabilities! Tiempo y Clima, 44, 3843.

  • Roulston, M. S., and T. R. Kaplan, 2009: A laboratory-based study of understanding of uncertainty in 5-day site-specific temperature forecasts. Meteor. Appl., 16, 237244, https://doi.org/10.1002/met.113.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roulston, M. S., G. E. Bolton, A. N. Kleit, and A. L. Sears-Collins, 2006: A laboratory study of the benefits of including uncertainty information in weather forecasts. Wea. Forecasting, 21, 116122, https://doi.org/10.1175/WAF887.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sivle, A. D., 2016: Oh no, it’s raining! A study of how information in online weather reports is interpreted, integrated, and used in everyday decision-making by laypeople. Ph.D. thesis, Norwegian Meteorological Institute (MET Norway), 218 pp.

  • Sivle, A. D., and T. Aamodt, 2019: A dialogue-based weather forecast: Adapting language to end-users to improve communication. Weather, 74, 436441, https://doi.org/10.1002/wea.3439.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sivle, A. D., S. D. Kolstø, P. J. Kirkeby Hansen, and J. Kristiansen, 2014: How do laypeople evaluate the degree of certainty in a weather report? A case study of the use of the web service yr.no. Wea. Climate Soc., 6, 399412, https://doi.org/10.1175/WCAS-D-12-00054.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Spiegelhalter, D. J., M. Pearson, and I. Short, 2011: Visualizing uncertainty about the future. Science, 333, 13931400, https://doi.org/10.1126/science.1191181.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stephens, E. M., D. J. Spiegelhalter, K. Mylne, and M. Harrison, 2019: The Met Office Weather Game: Investigating how different methods for presenting probabilistic weather forecasts influence decision-making. Geosci. Commun., 2, 101116, https://doi.org/10.5194/gc-2-101-2019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stewart, A. E., C. A. Williams, M. D. Phan, A. L. Horst, E. D. Knox, and J. A. Knox, 2016: Through the eyes of the experts: Meteorologists’ perceptions of the probability of precipitation. Wea. Forecasting, 31, 517, https://doi.org/10.1175/WAF-D-15-0058.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tiriolo, L., R. C. Torcasio, S. Montesanti, and S. Federico, 2015: Verification of a real time weather forecasting system in southern Italy. Adv. Meteor., 2015, 758250, https://doi.org/10.1155/2015/758250.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Van Bussum, L., Jr., 1999: A composite look at weather surveys: Using several weather surveys to get an estimate of public opinion. Western Region Tech. Attachment 99-20, 11 pp., https://www.weather.gov/media/wrh/online_publications/TAs/ta9920.pdf.

  • Zabini, F., 2016: Mobile weather apps or the illusion of certainty. Meteor. Appl., 23, 663670, https://doi.org/10.1002/met.1589.

  • Zabini, F., V. Grasso, R. Magno, F. Meneguzzo, and B. Gozzini, 2015: Communication and interpretation of regional weather forecasts: A survey of the Italian public. Meteor. Appl., 22, 495504, https://doi.org/10.1002/met.1480.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Students’ expectations for the closed-ended questions: (a) tomorrow’s actual daytime high temperature, given a forecast high temperature of 30°C; (b) tomorrow’s actual nighttime low temperature, given a forecast low temperature of 0°C; (c) tomorrow’s actual wind speed, given a forecast wind speed of 70 km h−1; and (d) tomorrow’s actual accumulated amount of rain in 12 h, given a forecast amount of rain of 150 mm.

  • View in gallery

    Difference expected by students from the deterministic value in the open-ended questions provided as a range or single value, (a) given a daytime high temperature forecast of 40°C, (b) given a nighttime low temperature of −5°C, (c) given a wind speed of 50 km h−1, and (d) given an accumulated amount of rain in 12 h of 100 mm.

  • View in gallery

    Students’ responses for at what (a) forecast probability of daytime high temperature ≥ 40°C, (b) forecast probability of nighttime low temperature ≤ −5°C, (c) forecast probability of wind speed ≥ 70 km h−1, (d) and forecast probability of rain ≥ 150 mm, they would take protective action.

All Time Past Year Past 30 Days
Abstract Views 371 133 0
Full Text Views 64 47 12
PDF Downloads 87 60 16

Perceptions, Uses, and Interpretations of Uncertainty in Current Weather Forecasts by Spanish Undergraduate Students

View More View Less
  • 1 Department of Applied Physics, Faculty of Sciences, University of Alicante, Alicante, Spain
  • | 2 Multidisciplinary Institute for Environmental Studies Ramón Margalef, University of Alicante, Alicante, Spain
  • | 3 Department of Regional Geographic Analysis and Physical Geography, Faculty of Philosophy and Letters, University of Alicante, Alicante, Spain
  • | 4 Department of Physics, Systems Engineering and Signal Theory, University of Alicante, Alicante, Spain
  • | 5 University Institute of Physics Applied to Sciences and Technologies, University of Alicante, Alicante, Spain
© Get Permissions
Free access

Abstract

This quantitative study evaluates how 71 Spanish undergraduate students perceive and interpret the uncertainty inherent to deterministic forecasts. It is based on several questions that asked participants what they expect given a forecast presented under the deterministic paradigm for a specific lead time and a particular weather parameter. In this regard, both normal and extreme weather conditions were studied. Students’ responses to the temperature forecast as it is usually presented in the media expect an uncertainty range of ±1°–2°C. For wind speed, uncertainty shows a deviation of ±5–10 km h−1, and the uncertainty range assigned to the precipitation amount shows a deviation of ±30 mm from the specific value provided in a deterministic format. Participants perceive the minimum night temperatures as the least-biased parameter from the deterministic forecast, while the amount of rain is perceived as the most-biased one. In addition, participants were then asked about their probabilistic threshold for taking appropriate precautionary action under distinct decision-making scenarios of temperature, wind speed, and rain. Results indicate that participants have different probabilistic thresholds for taking protective action and that context and presentation influence forecast use. Participants were also asked about the meaning of the probability-of-precipitation (PoP) forecast. Around 40% of responses reformulated the default options, and around 20% selected the correct answer, following previous studies related to this research topic. As a general result, it has been found that participants infer uncertainty into deterministic forecasts, and they are mostly used to take action in the presence of decision-making scenarios. In contrast, more difficulties were found when interpreting probabilistic forecasts.

© 2020 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Igor Gómez, igor.gomez@ua.es

Abstract

This quantitative study evaluates how 71 Spanish undergraduate students perceive and interpret the uncertainty inherent to deterministic forecasts. It is based on several questions that asked participants what they expect given a forecast presented under the deterministic paradigm for a specific lead time and a particular weather parameter. In this regard, both normal and extreme weather conditions were studied. Students’ responses to the temperature forecast as it is usually presented in the media expect an uncertainty range of ±1°–2°C. For wind speed, uncertainty shows a deviation of ±5–10 km h−1, and the uncertainty range assigned to the precipitation amount shows a deviation of ±30 mm from the specific value provided in a deterministic format. Participants perceive the minimum night temperatures as the least-biased parameter from the deterministic forecast, while the amount of rain is perceived as the most-biased one. In addition, participants were then asked about their probabilistic threshold for taking appropriate precautionary action under distinct decision-making scenarios of temperature, wind speed, and rain. Results indicate that participants have different probabilistic thresholds for taking protective action and that context and presentation influence forecast use. Participants were also asked about the meaning of the probability-of-precipitation (PoP) forecast. Around 40% of responses reformulated the default options, and around 20% selected the correct answer, following previous studies related to this research topic. As a general result, it has been found that participants infer uncertainty into deterministic forecasts, and they are mostly used to take action in the presence of decision-making scenarios. In contrast, more difficulties were found when interpreting probabilistic forecasts.

© 2020 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Igor Gómez, igor.gomez@ua.es

1. Introduction

Evaluating people’s uses, perceptions, and interpretations of uncertainty in current weather forecasts across different contexts is important nowadays, as weather forecasts are extremely useful for a wide range of applications, for instance, in agriculture, energy, transport and tourism, and recreational sectors. These forecasts are based on numerical weather prediction (NWP) models that simulate the state and dynamics of the atmosphere. Therefore, weather forecasts are unavoidably uncertain (Morss et al. 2008). The history of NWP is related to a rapid and continuous development since the turn of the twentieth century. The scientific and technological developments produced over the past 40 years, such as scientific research, in situ, and satellite observational programs, as well as supercomputing and storage facilities have led to improved predictive skills for lead times up to 7 days (Bauer et al. 2015). Therefore, it could seem that it is only a matter of time to achieve the perfect forecast (O’Hanrahan and Sweeney 2013). However, despite the continuous improvement in NWP systems, the uncertainty linked to weather forecasts can never be completely overcome.

Weather forecasts can be consulted in different formats and from a huge variety of sources, producing an enormous volume of information that is of significant usefulness to the general public (Lazo et al. 2009). In recent years, the increasing availability of mobile devices has made weather information within hand’s reach for many users (Abraham et al. 2015; Zabini 2016). However, even though communicating uncertainty is important to avoid false expectations on forecasts, Zabini (2016) pointed out that weather information providers do not consider that uncertainty should be communicated in their developed mobile telephone “apps.” They found that uncertainty was only expressed for probability of precipitation (PoP) in their study sample, and it was generally conveyed in a percentage format. Additionally, although the use of probabilistic weather forecasts has been widely promoted (see, e.g., National Research Council 2006; Sivle 2016; Fundel et al. 2019; Fleischhut et al. 2020), Juanchich and Sirota (2019), based on informal observations, also highlighted that it is currently rare to see probabilistic forecasts on British or American TV channels, or contain at best limited information about uncertainty (Morss et al. 2008). This issue could produce false certainty in forecasts in the target audience of these weather products, not admitting that forecasts may sometimes fail. In this regard, users of weather forecasts are accustomed to receive and access deterministic forecasts, corresponding to the prediction of a single future state, without information of the uncertainty associated with this forecast.

Different previous studies have been conducted with the aim of evaluating how members of the public obtain, perceive, use, and value weather forecasts. For instance, the comprehensive research performed by Morss et al. (2008) evaluated the perception, preferences, and interpretations of the United States public with regard to weather forecast information. They examined, among other questions, to what extent the public infers uncertainty into deterministic forecasts and how interprets commonly available uncertainty information presented in specific forecasts, such as PoP. Their results showed that most respondents inferred uncertainty into the deterministic temperature forecasts. In addition, weather forecasts including uncertainty were preferred by a significant majority of respondents compared to single-valued forecasts. With regard to PoP, Morss et al. (2008) found that the majority of the U.S. public interpreted PoP in a variety of ways, without knowing the meteorological interpretation of PoP forecasts. Lazo et al. (2009) explored the people’s behaviors and attitudes with regard to day-to-day weather forecast information. They investigated four interdependent areas to better understand the public’s needs for weather forecasts. This study highlighted that people obtained weather forecasts from a wide multiplicity of sources, using this information mainly for decision-making activities. Additionally, respondents of their study reported well-formed judgments and understanding about weather forecasts and were confident about the quality, reliability, and accuracy of these forecasts. Morss et al. (2010), examined how members of the U.S. public interpreted and used different types of weather forecasts, but in this case based on people’s responses to decision scenario questions. Their results indicated that people had different probabilistic thresholds for taking protective action. In addition, many respondents in this study inferred uncertainty in deterministic forecasts and were able to interpret specific probabilistic forecasts well enough to use them in the decision questions. Forecasts with uncertainty information are potentially useful to everyday decision-making and are more in line with the current scientific understanding of weather forecast in comparison to deterministic forecasts (Joslyn and Savelli 2010). This previous research highlighted that users of weather forecasts may have fairly advanced uncertainty expectations. In this regard, people anticipate uncertainty in deterministic forecasts and do not expect single value forecasts to verify exactly. This may suggest that the public is prepared to understand explicit uncertainty forecasts. However, Gigerenzer et al. (2005) reported a general misinterpretation of PoP forecasts, following the results already obtained in other previous studies, such as Murphy et al. (1980). Different studies (Gigerenzer et al. 2005; Morss et al. 2008; Joslyn et al. 2009; Morss et al. 2010; O’Hanrahan and Sweeney 2013; Peachey et al. 2013; Abraham et al. 2015; Zabini et al. 2015; Stewart et al. 2016) pointed out that end users are unable to state correctly the interpretation of the PoP forecasts. However, other researchers presented a more positive view on PoP forecasts interpretation (see, e.g., Konold 1989; Roulston et al. 2006; Handmer and Proudley 2007; Roulston and Kaplan 2009; Stephens et al. 2019; Juanchich and Sirota 2019). They showed that a wide range of factors impact how people understand probabilities, but weather forecasters should provide the public a fluent definition of PoP and emphasize the fact that precipitation is not certain.

A full understanding of how weather forecasts users are likely to interpret and use the information of interest is important. Therefore, this line of research has been promoted in the last years, considering different geographical locations as well as distinct target audiences. Morss et al. (2008), Lazo et al. (2009), Morss et al. (2010), Joslyn and Savelli (2010), Stewart et al. (2016), and Juanchich and Sirota (2019) applied their research in the United States. Joslyn and Savelli (2010) reported their results based on university undergraduates’ responses in the Pacific Northwest region of the United States. Peachey et al. (2013) and Abraham et al. (2015) developed their studies with university students and the general public, respectively, in the United Kingdom. Juanchich and Sirota (2019) conducted research with regard to the interpretation of probabilities of precipitation with participants from the United States and the United Kingdom. O’Hanrahan and Sweeney (2013) distributed a survey to weather enthusiasts in the Republic of Ireland in order to evaluate the level of understanding of probabilistic forecasting and their willingness to use these types of forecasts. Sivle et al. (2014) evaluated the perception of the degree of certainty in a weather report by laypeople in Norway, while Zabini et al. (2015) analyzed the communication and interpretation of regional weather forecasts by the Italian public.

In this paper, a survey was issued at the University of Alicante (Spain) during the academic year 2019–20. The current work focuses on perceptions, interpretations and uses of weather forecast information. We try to provide a view of how participants in meteorology- and/or climatology-related subjects taught at the University of Alicante perceive current weather forecasts and related uncertainty. Additionally, we perform a comparison with the previous results mentioned above. These studies have addressed the same issues with regard to the social and educational perception of weather forecasts, and we pursue both different aims as well with the current study. First, we want to explore how participants in the current study perceive, use, and interpret weather forecasts using the previous studies as the common thread and background to conduct our experience. Second, these previous works highlight the need to continue the research in the line of uncertainty perception by people who make use of weather forecasts but are not experts on the subject. Further work is then needed to update findings that are some years old as well as to explore existing findings in diverse contexts. We also investigate possible changes in this regard over the last years, especially for a public that would make a primary use of this information in their future professional career. These kinds of studies have not yet been addressed in Spain. The current study, therefore, would permit us to compare our findings with preceding available information in weather forecasting settings. We base our analysis on both normal conditions as well as on extreme weather conditions for different atmospheric parameters.

The paper is organized as follows. Section 2 presents the method. Section 3 presents results and their discussion. Section 4 includes the summary and conclusions.

2. Method

A total of 71 participants responded to an anonymous survey during the academic year 2019–20, corresponding to students pursuing a degree in marine science or in geography and territory planning. The survey was applied to students enrolled in subjects related to meteorology and/or climatology programs, but with little prior academic experience in these areas of knowledge. Therefore, results presented in the current study are discussed in this context.

The survey questions are extracted from previously published studies performed over different geographical locations and contexts (Morss et al. 2008, 2010) and Joslyn and Savelli (2010). These questions have been adopted with minor modifications, basically using Celsius instead of Fahrenheit, translating the corresponding information to Spanish and using thresholds common of extreme weather hazards over the region of study. Questions can be grouped in three themes. The first one deals with the perception of uncertainty in deterministic forecasts. The second one addresses the uses of forecasts in hypothetical decision-making scenarios. The third theme was selected focusing on the interpretations of PoP forecasts. A total of 17 questions were developed plus an open-ended question intended to gather free comments that participants would consider significant related to the survey. The questionnaire was administered online and accessed with a Google Form link describing it as a study investigating “weather forecasts survey.” Participants were informed that the information collected would remain anonymous.

The atmospheric parameters tested with regard to the first theme above were temperature, wind speed, and precipitation. To evaluate uncertainty expectations, two kinds of questions were designed, and respondents were asked to answer using a closed-ended question and an open-ended format, independently, for each parameter tested.

a. Questions with regard to perceptions of uncertainty in deterministic forecasts

The following questions were included to investigate this topic:

1) Closed-ended questions

  • On a day in August, you notice that the predicted daytime high temperature for the next day is 30°C. What do you think the daytime high temperature will be on the next day? Five distinct options were provided: 30°C, between 29° and 31°C, between 28° and 32°C, between 25° and 35°C, and between 20° and 40°C.

  • On a day in January, you notice that the predicted nighttime low temperature for the next day is 0°C. What do you think the actual low temperature will be? Five distinct options were provided: 0°C, between −1° and 1°C, between −2° and 2°C, between −5° and 5°C, and between −10° and 10°C.

  • On a day in October, you notice that sustained winds of 70 km h−1 are predicted for the next day. What do you think the sustained wind speed will be on the next day? Five distinct options were provided: 70 km h−1, between 65 and 75 km h−1, between 60 and 80 km h−1, between 55 and 88 km h−1, and between 50 and 90 km h−1.

  • On a day in October, you notice that extreme amounts of rainfall of 150 mm in 12 h are predicted for the next day. What do you think the recorded rainfall will be on the next day? Five distinct options were provided: 150 mm, between 120 and 180 mm, between 90 and 210 mm, between 60 and 240 mm, and between 30 and 270 mm.

In all four previous questions, an additional option, “other (please explain),” was introduced in the survey to encourage participants to express their personal interpretation in case they did not agree with the original suggestions.

2) Open-ended questions

  • On a day in August, you notice that the predicted daytime high temperature for the next day is 40°C. What do you think the daytime high temperature will be on the next day?

  • On a day in January, you notice that the predicted nighttime low temperature for the next day is −5°C. What do you think the actual low temperature will be?

  • On a day in October, you notice that sustained winds of 50 km h−1 are predicted for the next day. What do you think the sustained wind speed will be on the next day?

  • On a day in October, you notice that extreme amounts of rainfall of 100 mm in 12 h are predicted for the next day. What do you think the recorded rainfall will be on the next day?

As may be seen in these questions, daytime high and nighttime low temperatures were asked separately. On the one hand, the closed-ended questions included multiple options to be selected ranging from “normal” parameter values in the case of the maximum temperature to extreme weather conditions in the case of wind speed and amount of rainfall. On the other hand, the same atmospheric parameters were asked in an open-ended format. Participants should include their own perception beyond the limited values provided in the closed-ended questions. In this case, extreme values for the daytime high temperature and the nighttime low temperature were used, while adverse weather conditions were still maintained. All participants saw the closed-ended questions first.

b. Questions with regard to uses of forecasts in hypothetical decision-making scenarios

The following questions were included to investigate this topic:

1) Binary decision (yes/no) questions based on a single-value (deterministic) forecast

  • Imagine that it is August and you work with groups vulnerable to the exposure of temperatures of 40°C or higher. You notice that the predicted daytime high temperature for the next day is 40°C. With this information, would you be prepared and take protective action in this scenario?

  • Imagine that it is January and you work with groups vulnerable to the exposure of temperatures of 0°C or lower. You notice that the predicted nighttime low temperature for the next night is 0°C. With this information, would you be prepared and take protective action in this scenario?

  • Imagine that it is October and the weather forecast for tomorrow predicts heave and/or extreme rain of 150 mm in 12 h (possible flooding). With this information, would you be prepared and take protective action in this scenario?

  • Imagine that it is October and the weather forecast for tomorrow predicts sustained winds of 70 km h−1 (possible power outages, downed trees, bridge closures, etc.) for tomorrow. With this information, would you be prepared and take protective action in this scenario?

2) Decision questions based on possible values ranges or percentage chance of exceeding a damage threshold

  • Imagine that it is August and you work with groups vulnerable to the exposure of temperatures of 40°C or higher. For which of the following forecasts would you begin preparation? Participants could choose to not take action, or to select just one option from a list where 10% chance of 40°C to 100% chance of 40°C presented in increments of 10% chance.

  • Imagine that it is January and you work with groups vulnerable to the exposure of temperatures of 0°C or lower. For which of the following forecasts would you begin preparation? Participants could choose to not take action, or to select just one option from a list where 10% chance of 0°C to 100% chance of 0°C presented in increments of 10% chance.

  • Imagine that it is October and the weather forecast for tomorrow predicts heavy and/or extreme rain of 150 mm in 12 h (possible flooding). For which of the following forecasts would you begin preparation? Participants could choose to not take action, or to take among 11 response options: 10% chance of this heavy and/or extreme rain to 100% chance of this heavy and/or extreme rain presented in increments of 10% chance.

  • If you knew that sustained winds of 70 km h−1 were predicted (possible power outages, downed trees, bridge closures, etc.) for tomorrow. For which of the following forecasts would you begin preparation? Participants could choose to not take action, or to select just one option from a list where 10% chance of sustained winds of 70 km h−1 to 100% chance of sustained winds of 70 km h−1, presented in increments of 10% chance.

c. Questions with regard to interpretations of PoP forecasts

To investigate this topic, the next question was included in the survey:

  • Suppose the following text is the forecast for tomorrow: “There is a 60% chance of heavy or extreme rainfall of 150 mm in 12 h tomorrow.” Which of the options listed below do you think best describes what the forecast means? Five distinct options were provided: “This amount of rain will be recorded tomorrow in 60% of the region,” “This amount of rain will be recorded tomorrow for 60% of the time,” “This amount of rain will be recorded on 60% of the days like tomorrow,” “60% of weather forecasters believe that this amount of rain will be recorded tomorrow,” and “I don’t know.” In addition, an “Other (please explain)” option was introduced in the survey to encourage participants to express their personal interpretation in case they did not agree with the original suggestions.

3. Results and discussion

a. Perceptions of uncertainty in deterministic forecasts

1) Evaluation of the closed-ended forecasts responses

Figure 1 presents the participants’ perception and interpretation of the uncertainty associated with deterministic weather forecasts of temperature, wind speed, and amount of rain. Considering the four parameters presented in Fig. 1, a similar distribution is observed, with the exception of the daytime high temperature. In all cases, participants mostly selected the two ranges closest to the deterministic value. Notice that for daytime high temperature (Fig. 1a), 38% of respondents expected a deviation of ±1°C from the deterministic value. In the case of the nighttime low temperature (Fig. 1b), 48% of the participants expected a bias of ±1°C from the deterministic value. Expectation for wind speed shows that most responses (59%) correspond to a deviation of ±5 km h−1 (Fig. 1c). Last, with regard to the amount of rain (Fig. 1d), 72% of participants chose a bias of ±30 mm from the deterministic value. Additionally, one participant included her or his own opinion of the question by means of the open-ended option: “150 mm will be recorded during the 12 hours of heavy and persistence rains.” However, the deterministic value of the forecast was selected here. The two “other” responses are variations of the default rain ranges: “Between 140 and 200 mm” and “Between 140 and 160 mm.” In the first case, a general overestimation is expected, while the second one expected a reduced deviation of just ±10 mm from the deterministic value. Therefore, it seems clear that respondents perceive that there is uncertainty inherent to deterministic weather forecasts for temperature, wind speed and amount of rain. Additionally, the expected deviation from the deterministic forecast value is the lowest one in general (Fig. 1), revealing a positive recognition of the current weather forecasts. The Mann–Whitney–Wilcoxon test applied to the four different parameters shows significant differences between the daytime high temperature and the nighttime low temperature, and amount of rain (p value < 0.05), while no significant differences are obtained between the results of the nighttime low temperature and rain and wind speed, nor between these two latest parameters. Additionally, there is a marginally significant difference when comparing the daytime high temperature with the wind speed (p value = 0.062). Based on mean values computed for the different weather parameters, it is highlighted that the mean amount of rain separating overestimation and underestimation of the deterministic value produces the highest standard deviations, larger than the temperature and wind speed parameters.

Fig. 1.
Fig. 1.

Students’ expectations for the closed-ended questions: (a) tomorrow’s actual daytime high temperature, given a forecast high temperature of 30°C; (b) tomorrow’s actual nighttime low temperature, given a forecast low temperature of 0°C; (c) tomorrow’s actual wind speed, given a forecast wind speed of 70 km h−1; and (d) tomorrow’s actual accumulated amount of rain in 12 h, given a forecast amount of rain of 150 mm.

Citation: Weather, Climate, and Society 13, 1; 10.1175/WCAS-D-20-0048.1

In the study by Morss et al. (2008), fewer than 5% of respondents expected the deterministic forecast value of around 20°C, while more than 65% percent of respondents expected the temperature to be within 1°–2°C of the forecast. Morss et al. (2008) found that more than 20% of participants still expected biases of ±5°C, thus interpreting the forecast as more uncertain. This survey of the U.S. public showed that 95% of participants inferred uncertainty in deterministic high temperature forecasts. Peachey et al. (2013) found that for the deterministic forecast of 20°C for the daytime high temperature, 3% of United Kingdom participant reported exactly this value and the most common value was within the range ±2°C. In addition, comparing the results found by Joslyn and Savelli (2010) and Peachey et al. (2013) with those obtained in the current study in terms of temperature (Figs. 1a,b), there is a common perception among participants that minimum temperature forecasts present, in general, lower biases than maximum temperature forecasts.

2) Evaluation of the open-ended forecasts responses

Two extreme values have been selected for the maximum and minimum temperatures (Fig. 2), 40° and −5°C, respectively, to evaluate participants’ responses by means of open-ended questions. In the first case (Fig. 2a), 34% of participants reported the answer as a range. The mean value of those participants that reported a single value above 40°C (15 participants) is 41.9° ± 1.1°C. In contrast, the mean value of those participants that reported a single value below 40°C (12 participants) is 36° ± 3°C. The global mean value (47 participants) is 40° ± 3°C. In the case of the minimum temperature (Fig. 2b), 22% of participants reported the answer as a range. The mean value of those participants that reported a single value above −5°C (18 participants) is −2.4° ± 1.6°C. In contrast, the mean value of those participants that reported a single value below −5°C (6 participants) is −6.2° ± 0.4°C. The global mean value (55 participants) is −4.3° ± 1.6°C. With regard to extreme temperatures, Joslyn and Savelli (2010) reported a range of −9.16° to 1.23°C forecast daytime high temperature around 38°C, while a mean expectation range between −2.43° and 6.67°C for a forecast nighttime low temperature around −7°C. Besides, Joslyn and Savelli (2010) also found a mean expectation range between −5.72° and 5.43°C for a forecast daytime high temperature around 20°C and a mean expectation range between −3.52° and 4.05°C for a forecast nighttime low temperature of 0°C. Therefore, their results seemed to point toward an underestimation of the daytime high temperature and an overestimation of the nighttime low temperature. In the current study, there is a slightly higher percentage of participants that report a specific value above the deterministic forecast, even larger in the case of the nighttime temperatures. Furthermore, a greater number of respondents reported the forecast value in the case of the nighttime low temperature than for the daytime high temperature: 55 and 20 participants, respectively.

Fig. 2.
Fig. 2.

Difference expected by students from the deterministic value in the open-ended questions provided as a range or single value, (a) given a daytime high temperature forecast of 40°C, (b) given a nighttime low temperature of −5°C, (c) given a wind speed of 50 km h−1, and (d) given an accumulated amount of rain in 12 h of 100 mm.

Citation: Weather, Climate, and Society 13, 1; 10.1175/WCAS-D-20-0048.1

In the case that the wind speed should be freely specified by means of an open-ended question, 38% of responses provided in general a range centered around 50 km h−1 (Fig. 2c), corresponding to the value provided. The other 62% of the total number of responses reported a single wind speed. The mean value of the obtained responses (44 participants) with regard to wind speed is 50 ± 5 km h−1. The mean value of responses above 50 km h−1 (8 participants) shows a deviation of 58 ± 3 km h−1 from the deterministic value. Likewise, the mean value of responses below 50 km h−1 (9 participants) presents a deviation of 42 ± 3 km h−1 from the deterministic value. Joslyn and Savelli (2010) found that participants in their survey expected a wind speed forecast of 65 km h−1 to verify significantly lower than this forecast value. In the current study, the majority of participants reported the deterministic forecast in the case of the open-ended question. Besides, a similar number of participants reported underestimation and overestimation of the predicted value, with the same difference related to the deterministic forecast and the same standard deviation. Likewise, the range 65–70 km h−1 was indicated as “other” by just one participant (1.4% of responses). In this case, the participating student pointed out an underestimation of the wind speed, highlighting that “there could be more intense wind gusts.”

Figure 2d shows the responses for the open-ended rain question, where 100 mm is the value provided; 51% of responses were provided as a range, while 34% selected the deterministic value. Considering responses that provided a specific value (35 responses), the mean value is 100 ± 20 mm. Considering only those responses that provide a rain overestimation (5 participants), the mean value is 140 ± 40 mm, while considering only those responses indicating an underestimation of the forecast rain (5 participants), the mean value is 80 ± 17 mm. With regard to the amount of rainfall, Joslyn and Savelli (2010) found that users expected an underestimation of this parameter.

Comparing the results obtained for the different physical parameters, nighttime low temperature presents the lowest deviation from the single value forecast in Fig. 2 (centered in a difference of 0°C) and at the same time presents the lowest number of responses provided in a range format in the open-ended question. On the other extreme is located the rain forecast. In this case, there is a larger number of responses provided as a range. It seems that participants have a perception that, of all weather forecasts evaluated in the current study, rain forecasts have the largest variability. Additionally, responses appear to highlight the extreme difficulty in forecasting an exact amount of precipitation, even more difficult than for the other parameters. Between the nighttime low temperature forecast and the rain forecast are located the daytime high temperature and the wind speed (Figs. 2a,c), with just a marginally significant difference as mentioned above. Previous studies conducted with regard to the confidence on weather forecasts among undergraduate students in Spain highlighted that temperature and rain were the weather forecast with the highest and lowest confidence, respectively. Wind speed, however, was found to have an in between confidence (Gómez Doménech et al. 2019). Results found in the current study seem to support these previous outcomes. Furthermore, as mentioned above for the temperature field, it seems that presenting the forecast without references to uncertainty, as provided by the open-ended question, leads participants to perceive a more accurate forecast. This result has been observed for the four parameters evaluated in the current study.

b. Uses of forecasts in hypothetical decision-making scenarios

Considering taking action under hypothetical decision-making scenarios, 96% of respondents would be prepared and would take appropriate action in the case in which the prediction of daytime high temperature would be presented as a deterministic value, while 94% would do in the case of the nighttime low temperature. The Mann–Whitney–Wilcoxon test applied to both parameters does not show significant differences (p value = 0.703). In the case of the amount of rainfall and wind speed, 93% would be prepared and would take appropriate action in the case in which the prediction would be presented as a deterministic value. In this case, the percentage of affirmative responses is slightly lower than in the case of temperature. Nevertheless, the application of the Mann–Whitney–Wilcoxon test to the different parameters does not show significant differences (p value > 0.05) between temperature, wind speed, and amount of precipitation.

When the corresponding information is presented as a percentage-chance forecast (Fig. 3), there is a wide range of options that participants may choose under the same hypothetical scenario for temperature, wind speed, and rain. The application of the Mann–Whitney–Wilcoxon test to these parameters does not show significant differences among them (p value > 0.05). This is highlighted for all analyzed parameters, while it is slightly more pronounced for the two temperature parameters. However, there seem to be differences between wind speed and temperature as well as probability of precipitation. In this regard, the vast majority of responses in the case of both daytime high and nighttime low temperatures is located at 60% chance but is slightly higher for wind speed and amount of precipitation (70%). Figure 3d shows that in the case of rain, around 60% of participants would be prepared and would take appropriate action at the percentage-chance range 60%–80%. Surprisingly, it may also result that some participants would take no action for daytime high (9%) and nighttime low (7%) temperatures, wind speed (9%), and precipitation (5%), until the probability of the corresponding forecast reaches 100% chance for the hypothetical scenarios asked in the current work. On the other extreme, some participants would take action at 10% probability of the corresponding forecast for all meteorological parameters. With regard to this lowest (10%) threshold, peaks of responses are found in all cases, but they are more pronounced for the temperature parameters. This result is in agreement with that found by Morss et al. (2010). In their study, they established two scenarios with regard to probabilistic thresholds for protection. One, referred to as the picnic scenario, involved protective action against precipitation. The other one, referred to as the garden scenario, involved protective action against low temperature. They found that respondents selected a lower probabilistic threshold in the garden scenario (53.3%), that is, in the temperature context, than in the picnic scenario (58.3%). Additionally, they suggested that people would tend to respond less consistently in the garden than in the picnic scenario, as the distribution of responses in the garden scenario presented peaks at the 10% and 100% thresholds. A similar result can be seen in the current study (Fig. 3). Temperature and wind probabilistic forecasts are generally not currently publicly available from the Spanish State Meteorological Agency [Agencia Estatal de Meteorología (AEMET)], with the exception of variations in maximum and minimum temperature and probability of wind gust provided by their probabilistic maps. AEMET also offers PoP forecasts for distinct precipitation thresholds. However, even though these forecasts are regularly provided to the Spanish public, we think that this information does not reach the general public and it may be more consulted by professionals or experts in the field. Therefore, it seems that participants are not still very familiar with this information, which may be a plausible reason for the responses found in the current study. This possibility was already pointed out by Morss et al. (2010) in the sense that respondents from the United States answered less consistently in their garden scenario because of their relative lack of familiarity with the forecast format. Differences found in the current study could also be related to more familiarity with the rain probabilistic forecast as compared with a forecast presented in this way for the temperature field.

Fig. 3.
Fig. 3.

Students’ responses for at what (a) forecast probability of daytime high temperature ≥ 40°C, (b) forecast probability of nighttime low temperature ≤ −5°C, (c) forecast probability of wind speed ≥ 70 km h−1, (d) and forecast probability of rain ≥ 150 mm, they would take protective action.

Citation: Weather, Climate, and Society 13, 1; 10.1175/WCAS-D-20-0048.1

Handmer and Proudley (2007) found that respondents from the Australian Bureau of Meteorology Summer 2002 Public User Survey were most likely to consider changing their plans if there is a greater than 51% or above chance of rain. This percentage increased to 60% in the subgroup of Canberra public. They also pointed out the results found by Van Bussum (1999), where 33% Composite Look at Weather Surveys respondents took action at 50% and 34.2% took action at 70%. Peachey et al. (2013) found that the most frequent response for moving the picnic indoors was located at 60% chance of rain, even though 7% of responses would take no protective action in this case. In a scenario using a probabilistic forecast for extreme maximum temperature, the majority of respondents chose to take action at 10%–50% probability. In this case, participants would tend to take action in the 40°C scenario at lower probabilities than for the rain scenario. The results found in the current study are in agreement with these previous studies, as shown in Fig. 3.

c. Interpretations of PoP forecasts

The correct interpretation of the probabilistic precipitation forecast, according to Gigerenzer et al. (2005) and in a general sense, is that this will occur on 60% of days like tomorrow. In the current study, participants were asked about the following scenario: “There is a 60% chance of heavy and persistence rain of 150 mm in 12 hours tomorrow.” Results to this question are included in Tables 1, respectively. The “forecasters voice” and the “other” options are those that had the largest number of responses; 41% of participants expressed the meaning of this concept in their own words. Restatements have also been the most popular response to similar open-ended questions about the interpretation of PoP forecasts (Murphy et al. 1980; Gigerenzer et al. 2005; Morss et al. 2008; Joslyn et al. 2009; Peachey et al. 2013; Abraham et al. 2015, Zabini et al. 2015).

Table 1.

Meaning of the forecast “There is a 60% chance of heavy and persistence rain of 150 mm in 12 hours tomorrow” (N = 71).

Table 1.

In these previous studies, write-in responses may be classified under distinct interpretation type categories. Results obtained in this regard show that this restatement is still related to the probability or relative frequency, according to the classification used by Morss et al. (2008), Peachey et al. (2013) or Abraham et al. (2015). All the restatements fit in the general response: “There is a 60% chance it will rain tomorrow.”

In the study by Zabini et al. (2015), the option “other” represented just a 6% of responses. However, they reported that 47% of respondents believe that the statement “There is a 60% chance of rain tomorrow” means that 60% of weather forecasters believe that it will rain tomorrow. Comparing their results with those found in the current study, we find that 25% of respondents have the same belief with regard to this probabilistic forecast, while 22% of responses selected this option in Morss et al. (2008). The percentage of participants who interpreted PoP as a percentage of time or region is not relevant in the current study (Table 1). This result is more in line with the figures obtained by Zabini et al. (2015), and lower than the percentages found by Morss et al. (2008) and Abraham et al. (2015), where they raise to 15%, approximately.

Responses with regard to area/time references assume some kind of certainty in the corresponding forecast. Previous studies have commonly used percentage of region or time, such as 60% used in Table 1, when asking about PoP interpretations. Considering that forecasts for an exact location and time are usually provided nowadays, maybe these options will become less frequently answered in this kind of questions. This reasoning could a plausible reason for the percentage of respondents shown in Table 1. In this case, there seems to be no awareness of probabilistic information related to the precipitation forecast. In contrast, the interpretation based on percentage provided by participants in the current study reveals some perception of the uncertainty inherent to the precipitation forecast. Nevertheless, restating the forecast communication as a probability or relative frequency does not necessarily mean it is interpreted correctly. There seems to be a rewriting related to what is obvious, “it means exactly what it says,” rather than specifying what users thought the probability refers to. Participants that selected the open-ended question seem to think they have a sense of what PoP means (Murphy et al. 1980; Gigerenzer et al. 2005). According to Morss et al. (2008), “some who offered only a restatement may not require a more detailed interpretation. Others may have had a cognitive difficulty forming a response, or they may have had a more detailed interpretation in mind that they did not (or could not) articulate,” which perfectly seems to be reflected as well in the results found in the current study.

Following this issue, Zabini et al. (2015) used an alternative to the correct interpretation of PoP, based on the probability interpretation language widely accepted among Laboratorio di Monitoraggio e Modellistica Ambientale per lo Sviluppo Sostenibile (LaMMA) forecasters (Italy): “In 6 out of 10 times when meteorologists make this forecast, there will be rain,” different from “It will rain on 60% of the days like tomorrow” or “This amount of rain will fall on 60% of the days like tomorrow” used here. Maybe the latter is more confusing than the statement used in LaMMA, which could explain the differences found in the percentage of responses that chose this option as the correct interpretation of the PoP in contrast with the more extended technically correct interpretation. In this sense, Morss et al. (2008) pointed out that respondents may select the closed meteorological interpretation that sounds best at the time, and many people may not have considered the meaning of PoP prior to being asked for this question. Yet another result with regard to the PoP concept is that there is a natural tendency among participants in the current study to assign centrality to the voice of weather forecasters instead to the event itself. This result agrees with previous studies, such that conducted by Zabini et al. (2015) over Italy. Peachey et al. (2013) showed that only 4% of United Kingdom students provided an interpretation similar to the statement: “If tomorrow happens 10 times, 6 of them would be rainy,” similar to the alternative correct interpretation of PoP used in LaMMA. But they asked students to explain the meaning of the statement “There is a 60% chance of rain” in their own words.

4. Summary and conclusions

The current study investigates students’ perception, use and interpretation of weather forecast uncertainty by analyzing data from a university survey conducted in Spain. The results address three main themes: perceptions of uncertainty in deterministic forecasts, uses of forecasts in hypothetical decision-making scenarios, and interpretations of PoP forecasts by means of a series of questions focused on these issues. To place the results found in the current study within the whole picture of previous research on this topic, we have examined results from these previous works conducted over different and diverse contexts. The findings of this article want to contribute to fundamental understanding of the interpretations of the uncertainty inherent to weather forecasting, based on the answers provided by the participants of the current study. Additionally, we want to evaluate whether perception, use and interpretation of weather forecast uncertainty is different among distinct and diverse contexts considering as well different populations, starting from the review literature used as background for the present study. In doing so, this study assesses the state of the current perception of uncertainty with the aim of considering strategies in the development of user-oriented uncertainty forecast products that easily can reach the general public and students of meteorology and climatology subjects in particular.

The current survey does not only focus on normal climatic conditions of the area of study, but also includes high-impact weather forecast situations, therefore covering a range of contexts. In this regard, questions have been suggested considering vulnerable atmospheric situations for population that raise specific challenges with regard to weather-related risks. Previous studies related to climate and weather hazards using mesoscale models have been conducted over the study area (see, e.g., Pastor et al. 2010; Gómez et al. 2011, 2014b, 2015, 2016, 2018). Traditionally, there has been a difficulty to properly capture the observed minimum temperatures as provided by NWP systems, but with a better simulation of the daytime high temperatures (see, e.g., Federico 2011; Gómez et al. 2014a; Tiriolo et al. 2015; Gómez et al. 2019). However, participants in the current study expected a less degree of uncertainty in the nighttime low temperature than in the daytime high temperature forecasts, as a difference with the scientific evidence.

The results found in the current study with regard to the PoP interpretation show that even today there is a range of interpretations of what a percentage or probability chance refers to. Previous studies, such those conducted by Gigerenzer et al. (2005), Joslyn et al. (2009), Morss et al. (2008), Peachey et al. (2013), Abraham et al. (2015), and Zabini et al. (2015), reported that many laypeople did not know the technically correct definition of PoP forecasts, corroborating earlier work by Murphy et al. (1980). Abraham et al. (2015) showed that increasingly PoP forecasts given in narrow-cast forecasting services, such as smartphone apps, has not yet influenced people’s comprehension of probabilistic information. They found, however, differences between subgroups based on age. The younger group (under 40) as well as people with higher level of educational qualification was more likely to provide the correct response. These authors suggest that the effect of correct interpretation of PoP forecasts among these groups could be related to thinking about uncertainty during their educational career. However, it seems that the probability terminology definition of PoP forecasts has not been consistently and adequately explained, in general, to Spanish students participating in the current study. In this regard, other researchers emphasize the need of promoting the use of probabilistic weather forecasts (see, e.g., Fundel et al. 2019; Fleischhut et al. 2020) by means of a more fluent and simple view on PoP forecasts interpretation that would help weather forecast users to believe in the quality of the interpretation and its utility (Juanchich and Sirota 2019). In this regard, Handmer and Proudley (2007) showed that the participants in their study generally understood the basic probability concept used in weather forecasts. However, they also highlighted that clarity in terms of the forecast event is always needed, even more important when uncertainty is expressed explicitly using numerical probability.

The survey was administered to a group of undergraduate students at the University of Alicante, with 71 respondents. Despite the possibility that the small sample size could be seen as a limitation, the fact is that similar results have been found in previous studies (see, e.g., Morss et al. 2008; Joslyn and Savelli 2010; Morss et al. 2010; Peachey et al. 2013). Peachey et al. (2013) highlighted that administering the same survey to undergraduate students in an introductory weather course given in the United States, yielded similar results to those obtained for undergraduate students in Manchester (United Kingdom). Furthermore, some of the feedback provided by students in the final open-ended question of this study survey were related to their shortage or nonexistence of experience and background in meteorology and climatology, less in weather forecasting. Even though it could be interesting to extend this survey to a wider sample to probe further some of the results presented here, the found similarity of results compared to previous studies yields support for their generality.

Based on the results found in the current work, it is important to correct students’ misinterpretations. In this sense, different strategies should be proposed, maybe related to educational activities about probabilities and the definition of PoP. Nowadays, there are different websites that provide a huge amount of data provided by different NWP models. This information could be used to compare the outputs of distinct atmospheric variables simulated by several models and evaluate the differences and similarities found among them. It could help to visualize the uncertainty associated with deterministic weather forecasts and introduce the concept of ensemble prediction systems to estimate the uncertainty in a weather forecast. In this regard, it becomes fundamental to understand people’s preexisting concepts related to forecast uncertainty in order to properly handle uncertainty information. We think that students’ confidence in forecasts would likely evolve as they gain experience with the basis of models together with fostering activities that bring into play uncertainty with different formats than those presented in the current paper. In this sense, we should put the attention on people’s interpretation of the information as well as their attitudes and behavior toward it (Morss et al. 2008). We should keep in mind as well to promote the main feeling that has arisen from this study in the sense that uncertainty is inherent to weather forecasts. In this regard, despite students know that weather forecasts are not perfect, they also acknowledge their reliability and effectiveness, and the usefulness to anticipate and be prepared in extreme weather situations. Results are in line with acknowledging uncertainty in weather and climate forecasts (Joslyn and LeClerc 2012; Howe et al. 2019). In this case, people seem to recognize that complete certainty in future predictions is not possible but admitting the limitations inherent in these predictions may support their credibility and increase the appropriate use of scientific findings by nonexperts. As such, forecasts with odds ratios can be extremely effective in convincing users to take precautionary action for a wide range of dangerous weather situations in which prompt public response is required (LeClerc and Joslyn 2012). However, LeClerc and Joslyn (2012) also pointed out that even though forecasts with odds ratios can be extremely effective in convincing users to take precautionary action, they should be used with care to avoid false-alarm effects. Supporting these previous results, the current study shows that an indication of the possible uncertainty associated with the forecast in the closed-ended question seems to activate some kind of awareness with regard to forecast uncertainty, whereas this issue is not taken into account, generally speaking, in the open-ended questions, where a free response is suggested. Other choices that could help to normalize and communicate uncertainty could be a more fluent use of uncertainty terminology. In this sense, verbal statements can function equally well as numerical values of probabilities when communicating uncertainty (Persson 2014). For instance, with regard to PoP, a good point to start could be the conceptual understanding, using plain text, such as, “Of 10 days with atmospheric conditions like tomorrow, 6 days would produce this amount of rain,” based more on conceptual understanding than on mathematics of probabilities. We should consider as well that students had not faced this sort of questions before and they even had not thought of this question. Furthermore, reducing the number of possible interpretations of everyday language when used to describe weather or climate should be considered and combined with graphical/numerical information in order to benefit the public (Sivle and Aamodt 2019). In this sense, Spiegelhalter et al. (2011) recommended using multiple formats that combines verbal and numerical information. Even though using multiple information could produce an excess of information, some strategies should tested accordingly to the needs of the audience, objectives of communication and context (Spiegelhalter et al. 2011).

From the previous results, it seems that showing some measure of uncertainty in the forecast statement, as provided by the closed-ended questions, stimulates the perception of some kind of uncertainty in the corresponding forecast. In contrast, providing the deterministic forecast without any reference to uncertainty, as in the open-ended questions, appears to give the impression that an exact forecast is expected. Maybe, the absence of specific ranges of reference could lead participants to perceive a more skillful prediction in terms of the temperature forecasts. Therefore, this issue should be considered when communicating weather forecasts, as it could help participants to be aware of some sense of uncertainty in the corresponding prediction.

The current findings provide a positive view on weather forecasts interpretation, supporting the assumption that participants in the study can make more informed decisions based on this information. Therefore, we should try to educate students on how to correctly interpret probabilities and face weather and climate phenomena in these terms. Students should also be trained in how to correctly interpret the meteorological predictions of the set of atmospheric elements, within the educational curricula of the basic levels of education (Martínez Fernández and Olcina Cantos 2019) for their better understanding at university levels and their greater handling in future research papers. As pointed out by Persson (2014), weather forecasting is a probabilistic matter, whether we “like” it or not. Therefore, as shown in the current work, it is also a matter of acknowledging uncertainty.

Acknowledgments

This work has been funded by the Assistance Programme of University of Alicante “Programa de Redes-I3CE de calidad, innovación e investigación en docencia universitaria. Convocatoria 2019-20. Alicante: Instituto de Ciencias de la Educación (ICE) de la Universidad de Alicante. Ref: [4669].” We are truly grateful to the three anonymous reviewers for their constructive comments, suggestions, and feedback, which have greatly improved this paper.

REFERENCES

  • Abraham, S., R. Bartlett, M. Standage, A. Black, A. Charlton-Perez, and R. McCloy, 2015: Do location specific forecasts pose a new challenge for communicating uncertainty? Meteor. Appl., 22, 554562, https://doi.org/10.1002/met.1487.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bauer, P., A. Thorpe, and G. Brunet, 2015: The quiet revolution of numerical weather prediction. Nature, 525, 4755, https://doi.org/10.1038/nature14956.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Federico, S., 2011: Verification of surface minimum, mean, and maximum temperature forecasts in Calabria for summer 2008. Nat. Hazards Earth Syst. Sci., 11, 487500, https://doi.org/10.5194/nhess-11-487-2011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fleischhut, N., S. M. Herzog, and R. Hertwig, 2020: Weather literacy in times of climate change. Wea. Climate Soc., 12, 435452, https://doi.org/10.1175/WCAS-D-19-0043.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fundel, V. J., N. Fleischhut, S. M. Herzog, M. Göber, and R. Hagedorn, 2019: Promoting the use of probabilistic weather forecasts through a dialogue between scientists, developers and end users. Quart. J. Roy. Meteor. Soc., 145, 210231, https://doi.org/10.1002/qj.3482.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gigerenzer, G., R. Hertwig, E. van den Broek, B. Fasolo, and K. V. Katsikopoulos, 2005: A 30% chance of rain tomorrow: How does the public understand probabilistic weather forecasts? Risk Anal., 25, 623629, https://doi.org/10.1111/j.1539-6924.2005.00608.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., F. Pastor, and M. J. Estrela, 2011: Sensitivity of a mesoscale model to different convective parameterization schemes in a heavy rain event. Nat. Hazards Earth Syst. Sci., 11, 343357, https://doi.org/10.5194/nhess-11-343-2011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., M. J. Estrela, and V. Caselles, 2014a: Operational forecasting of daily summer maximum and minimum temperatures in the Valencia region. Nat. Hazards, 70, 10551076, https://doi.org/10.1007/s11069-013-0861-1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., V. Caselles, and M. J. Estrela, 2014b: Real-time weather forecasting in the western Mediterranean basin: An application of the RAMS model. Atmos. Res., 139, 7189, https://doi.org/10.1016/j.atmosres.2014.01.011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., M. J. Estrela, and V. Caselles, 2015: Verification of the RAMS-based operational weather forecast system in the Valencia region: A seasonal comparison. Nat. Hazards, 75, 19411958, https://doi.org/10.1007/s11069-014-1408-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., V. Caselles, M. J. Estrela, and R. Niclòs, 2016: Impact of initial soil temperature derived from remote sensing and numerical weather prediction datasets on the simulation of extreme heat events. Remote Sens., 8, 589, https://doi.org/10.3390/rs8070589.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., V. Caselles, M. J. Estrela, and J. J. Miró, 2018: Comparative assessment of RAMS and WRF short-term forecasts over eastern Iberian Peninsula using various in-situ observations, remote sensing products and uncoupled land surface model datasets. Atmos. Res., 213, 476491, https://doi.org/10.1016/j.atmosres.2018.06.022.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez, I., R. Niclòs, M. J. Estrela, V. Caselles, and M. J. Barberà, 2019: Simulation of extreme heat events over the Valencia coastal region: Sensitivity to initial conditions and boundary layer parameterizations. Atmos. Res., 218, 315334, https://doi.org/10.1016/j.atmosres.2018.12.016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gómez Doménech, I., S. Molina Palacios, and J. L. Soler Llorens, 2019: Percepción y valoración de las predicciones meteorológicas: Un estudio exploratorio entre estudiantes del Grado de Ciencias del Mar de la Universidad de Alicante. Investigación e Innovación en la Enseñanza Superior: Nuevos Contextos, Nuevas Ideas, R. Roig-Vila, Ed., Octaedro, 582–591.

  • Handmer, J., and B. Proudley, 2007: Communicating uncertainty via probabilities: The case of weather forecasts. Environ. Hazards, 7, 7987, https://doi.org/10.1016/j.envhaz.2007.05.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Howe, L. C., B. MacInnis, J. A. Krosnick, E. M. Markowitz, and R. Socolow, 2019: Acknowledging uncertainty impacts public acceptance of climate scientists’ predictions. Nat. Climate Change, 9, 863867, https://doi.org/10.1038/s41558-019-0587-5.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joslyn, S., and S. Savelli, 2010: Communicating forecast uncertainty: Public perception of weather forecast uncertainty. Meteor. Appl., 17, 180195, https://doi.org/10.1002/met.190.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joslyn, S., and J. E. LeClerc, 2012: Uncertainty forecasts improve weather-related decisions and attenuate the effects of forecast error. J. Exp. Psychol. Appl., 18, 126140, https://doi.org/10.1037/a0025185.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joslyn, S., L. Nadav-Greenberg, and R. M. Nichols, 2009: Probability of precipitation: Assessment and enhancement of end-user understanding. Bull. Amer. Meteor. Soc., 90, 185194, https://doi.org/10.1175/2008BAMS2509.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Juanchich, M., and M. Sirota, 2019: Not as gloomy as we thought: Reassessing how the public understands probability of precipitation forecasts. J. Cognit. Psychol., 31, 116129, https://doi.org/10.1080/20445911.2018.1553884.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Konold, C., 1989: Informal conceptions of probability. Cognit. Instr., 6, 5998, https://doi.org/10.1207/s1532690xci0601_3.

  • Lazo, J. K., R. E. Morss, and J. L. Demuth, 2009: 300 billion served. Bull. Amer. Meteor. Soc., 90, 785798, https://doi.org/10.1175/2008BAMS2604.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • LeClerc, J., and S. Joslyn, 2012: Odds ratio forecasts increase precautionary action for extreme weather events. Wea. Climate Soc., 4, 263270, https://doi.org/10.1175/WCAS-D-12-00013.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Martínez Fernández, L. C., and J. Olcina Cantos, 2019: La enseñanza escolar del tiempo atmosférico y del clima en España: Currículo educativo y propuestas didácticas. An. Geogr. Univ. Complutense, 39, 125148, https://doi.org/10.5209/aguc.64680.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., J. L. Demuth, and J. K. Lazo, 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974991, https://doi.org/10.1175/2008WAF2007088.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., J. K. Lazo, and J. L. Demuth, 2010: Examining the use of weather forecasts in decision scenarios: Results from a US survey with implications for uncertainty communication. Meteor. Appl., 17, 149162, https://doi.org/10.1002/met.196.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., S. Lichtenstein, B. Fischoff, and R. L. Winkler, 1980: Misinterpretations of precipitation probability forecasts. Bull. Amer. Meteor. Soc., 61, 695701, https://doi.org/10.1175/1520-0477(1980)061<0695:MOPPF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • National Research Council, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. National Academies Press, 124 pp.

  • O’Hanrahan, P. O., and C. Sweeney, 2013: Odds on weather: Probabilities and the public. Weather, 68, 247250, https://doi.org/10.1002/wea.2137.

  • Pastor, F., I. Gómez, and M. J. Estrela, 2010: Numerical study of the October 2007 flash flood in the Valencia region (eastern Spain): The role of orography. Nat. Hazards Earth Syst. Sci., 10, 13311345, https://doi.org/10.5194/nhess-10-1331-2010.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peachey, J. A., D. M. Schultz, R. Morss, P. J. Roebber, and R. Wood, 2013: How forecasts expressing uncertainty are perceived by UK students. Weather, 68, 176181, https://doi.org/10.1002/wea.2094.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Persson, A., 2014: We meteorologists cannot escape probabilities! Tiempo y Clima, 44, 3843.

  • Roulston, M. S., and T. R. Kaplan, 2009: A laboratory-based study of understanding of uncertainty in 5-day site-specific temperature forecasts. Meteor. Appl., 16, 237244, https://doi.org/10.1002/met.113.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roulston, M. S., G. E. Bolton, A. N. Kleit, and A. L. Sears-Collins, 2006: A laboratory study of the benefits of including uncertainty information in weather forecasts. Wea. Forecasting, 21, 116122, https://doi.org/10.1175/WAF887.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sivle, A. D., 2016: Oh no, it’s raining! A study of how information in online weather reports is interpreted, integrated, and used in everyday decision-making by laypeople. Ph.D. thesis, Norwegian Meteorological Institute (MET Norway), 218 pp.

  • Sivle, A. D., and T. Aamodt, 2019: A dialogue-based weather forecast: Adapting language to end-users to improve communication. Weather, 74, 436441, https://doi.org/10.1002/wea.3439.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sivle, A. D., S. D. Kolstø, P. J. Kirkeby Hansen, and J. Kristiansen, 2014: How do laypeople evaluate the degree of certainty in a weather report? A case study of the use of the web service yr.no. Wea. Climate Soc., 6, 399412, https://doi.org/10.1175/WCAS-D-12-00054.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Spiegelhalter, D. J., M. Pearson, and I. Short, 2011: Visualizing uncertainty about the future. Science, 333, 13931400, https://doi.org/10.1126/science.1191181.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stephens, E. M., D. J. Spiegelhalter, K. Mylne, and M. Harrison, 2019: The Met Office Weather Game: Investigating how different methods for presenting probabilistic weather forecasts influence decision-making. Geosci. Commun., 2, 101116, https://doi.org/10.5194/gc-2-101-2019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stewart, A. E., C. A. Williams, M. D. Phan, A. L. Horst, E. D. Knox, and J. A. Knox, 2016: Through the eyes of the experts: Meteorologists’ perceptions of the probability of precipitation. Wea. Forecasting, 31, 517, https://doi.org/10.1175/WAF-D-15-0058.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tiriolo, L., R. C. Torcasio, S. Montesanti, and S. Federico, 2015: Verification of a real time weather forecasting system in southern Italy. Adv. Meteor., 2015, 758250, https://doi.org/10.1155/2015/758250.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Van Bussum, L., Jr., 1999: A composite look at weather surveys: Using several weather surveys to get an estimate of public opinion. Western Region Tech. Attachment 99-20, 11 pp., https://www.weather.gov/media/wrh/online_publications/TAs/ta9920.pdf.

  • Zabini, F., 2016: Mobile weather apps or the illusion of certainty. Meteor. Appl., 23, 663670, https://doi.org/10.1002/met.1589.

  • Zabini, F., V. Grasso, R. Magno, F. Meneguzzo, and B. Gozzini, 2015: Communication and interpretation of regional weather forecasts: A survey of the Italian public. Meteor. Appl., 22, 495504, https://doi.org/10.1002/met.1480.

    • Crossref
    • Search Google Scholar
    • Export Citation
Save