Editorial Type:
Article
Article Type:
Research Article

Exploring the Differences in SPC Convective Outlook Interpretation Using Categorical and Numeric Information

Makenzie J. Krocak aInstitute for Public Policy Research and Analysis, University of Oklahoma, Norman, Oklahoma
bCooperative Institute for Severe and High-Impact Weather Research and Operations, National Weather Center, Norman, Oklahoma
cNOAA/Storm Prediction Center, National Weather Center, Norman, Oklahoma

Search for other papers by Makenzie J. Krocak in
Current site
Google Scholar
PubMed Close
https://orcid.org/0000-0002-0943-8725
,
Joseph T. Ripberger aInstitute for Public Policy Research and Analysis, University of Oklahoma, Norman, Oklahoma

Search for other papers by Joseph T. Ripberger in
Current site
Google Scholar
PubMed Close
,
Sean Ernst aInstitute for Public Policy Research and Analysis, University of Oklahoma, Norman, Oklahoma
bCooperative Institute for Severe and High-Impact Weather Research and Operations, National Weather Center, Norman, Oklahoma
cNOAA/Storm Prediction Center, National Weather Center, Norman, Oklahoma

Search for other papers by Sean Ernst in
Current site
Google Scholar
PubMed Close
,
Carol L. Silva aInstitute for Public Policy Research and Analysis, University of Oklahoma, Norman, Oklahoma

Search for other papers by Carol L. Silva in
Current site
Google Scholar
PubMed Close
, and
Hank C. Jenkins-Smith aInstitute for Public Policy Research and Analysis, University of Oklahoma, Norman, Oklahoma

Search for other papers by Hank C. Jenkins-Smith in
Current site
Google Scholar
PubMed Close
Online Publication:
24 Feb 2022
Print Publication:
01 Feb 2022
DOI:
https://doi.org/10.1175/WAF-D-21-0123.1
Page(s):
303–311
Restricted access

Abstract

While previous work has shown that the Storm Prediction Center (SPC) convective outlooks accurately capture meteorological outcomes, evidence suggests stakeholders and the public may misinterpret the categorical words currently used in the product. This work attempts to address this problem by investigating public reactions to alternative information formats that include the following numeric information: 1) numeric risk levels (i.e., “Level 2 of 5”) and 2) numeric probabilities (i.e., “a 5% chance”). In addition, it explores how different combinations of the categorical labels with numeric information may impact public reactions to the product. Survey data comes from the 2020 Severe Weather and Society Survey, a nationally representative survey of U.S. adults. Participants were shown varying combinations of the information formats of interest, and then rated their concern about the weather and the likelihood of changing plans in response to the given information. Results indicate that providing numeric information (in the form of levels or probabilities) increases the likelihood of participants correctly interpreting the convective outlook information relative to categorical labels alone. Including the categorical labels increases misinterpretation, regardless of whether numeric information was included alongside the labels. Finally, findings indicate participants’ numeracy (or their ability to understand and work with numbers) had an impact on correct interpretation of the order of the outlook labels. Although there are many challenges to correctly interpreting the SPC convective outlook, using only numeric labels instead of the current categorical labels may be a relatively straightforward change that could improve public interpretation of the product.

Significance Statement

The SPC convective outlook contains vital information that can help people prepare for a severe weather event. The categorical labels in this product are often ordered incorrectly by members of the public. This work shows using numeric levels or probabilities reduces the tendency for people to order the levels incorrectly.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Makenzie J. Krocak, mjkrocak@ou.edu

Abstract

While previous work has shown that the Storm Prediction Center (SPC) convective outlooks accurately capture meteorological outcomes, evidence suggests stakeholders and the public may misinterpret the categorical words currently used in the product. This work attempts to address this problem by investigating public reactions to alternative information formats that include the following numeric information: 1) numeric risk levels (i.e., “Level 2 of 5”) and 2) numeric probabilities (i.e., “a 5% chance”). In addition, it explores how different combinations of the categorical labels with numeric information may impact public reactions to the product. Survey data comes from the 2020 Severe Weather and Society Survey, a nationally representative survey of U.S. adults. Participants were shown varying combinations of the information formats of interest, and then rated their concern about the weather and the likelihood of changing plans in response to the given information. Results indicate that providing numeric information (in the form of levels or probabilities) increases the likelihood of participants correctly interpreting the convective outlook information relative to categorical labels alone. Including the categorical labels increases misinterpretation, regardless of whether numeric information was included alongside the labels. Finally, findings indicate participants’ numeracy (or their ability to understand and work with numbers) had an impact on correct interpretation of the order of the outlook labels. Although there are many challenges to correctly interpreting the SPC convective outlook, using only numeric labels instead of the current categorical labels may be a relatively straightforward change that could improve public interpretation of the product.

Significance Statement

The SPC convective outlook contains vital information that can help people prepare for a severe weather event. The categorical labels in this product are often ordered incorrectly by members of the public. This work shows using numeric levels or probabilities reduces the tendency for people to order the levels incorrectly.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Makenzie J. Krocak, mjkrocak@ou.edu
Save
Cover Weather and Forecasting
Weather and Forecasting

  • Beyth-Marom, R., 1982: How probable is probable? A numerical translation of verbal probability expressions. J. Forecasting, 1, 257–269, https://doi.org/10.1002/for.3980010305.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Cokely, E. T., M. Galesic, E. Schulz, S. Ghazal, and R. Garcia-Retamero, 2012: Measuring risk literacy: The Berlin Numeracy Test. Judgement Decis. Making, 7, 25–47.

    • Search Google Scholar
    • Export Citation

  • Corfidi, S. F., 1999: The birth and early years of the Storm Prediction Center. Wea. Forecasting, 14, 507–525, https://doi.org/10.1175/1520-0434(1999)014<0507:TBAEYO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • AMS Council, 2008: Enhancing weather information with probability forecasts: An information statement of the American Meteorological Society. Bull. Amer. Meteor. Soc., 89, 1049–1053, https://doi.org/10.1175/1520-0477-89.7.1041.

    • Search Google Scholar
    • Export Citation

  • Edwards, R., and F. Ostby, 2015: Time line of SELS and SPC. Storm Prediction Center. NOAA, accessed 28 February 2020, https://www.spc.noaa.gov/history/timeline.html.

  • Ernst, S., D. LaDue, and A. Gerard, 2018: Understanding emergency manager forecast use in severe weather events. J. Oper. Meteor., 6, 95–105, https://doi.org/10.15191/nwajom.2018.0609.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ernst, S., J. T. Ripberger, M. J. Krocak, H. Jenkins-Smith, and C. Silva, 2021: Colorful language: Investigating public interpretation of the Storm Prediction Center convective outlook. Wea. Forecasting, 36, 1785–1797, https://doi.org/10.1175/WAF-D-21-0001.1.

    • Search Google Scholar
    • Export Citation

  • Fischer, K., and H. Jungermann, 1996: Rarely occurring headaches and rarely occurring blindness: Is rarely = rarely? The meaning of verbal frequentistic labels in specific medical contexts. J. Behav. Decis. Making, 9, 153–172, https://doi.org/10.1002/(SICI)1099-0771(199609)9:3<153::AID-BDM222>3.0.CO;2-W.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Friedman, J. A., and R. Zeckhauser, 2014: Handling and mishandling estimative probability: Likelihood, confidence, and the search for Bin Laden. Intell. Natl. Secur., 30, 77–99, https://doi.org/10.1080/02684527.2014.885202.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Gerst, M. D., and Coauthors, 2020: Using visualization science to improve expert and public understanding of probabilistic temperature and precipitation outlooks. Wea. Climate Soc., 12, 117–133, https://doi.org/10.1175/WCAS-D-18-0094.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Herman, G. R., E. R. Nielsen, and R. S. Schumacher, 2018: Probabilistic verification of Storm Prediction Center convective outlooks. Wea. Forecasting, 33, 161–184, https://doi.org/10.1175/WAF-D-17-0104.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Hitchens, N. M., and H. E. Brooks, 2012: Evaluation of the Storm Prediction Center’s Day 1 convective outlooks. Wea. Forecasting, 27, 1580–1585, https://doi.org/10.1175/WAF-D-12-00061.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Hitchens, N. M., and H. E. Brooks, 2017: Determining criteria for missed events to evaluate significant severe convective outlooks. Wea. Forecasting, 32, 1321–1328, https://doi.org/10.1175/WAF-D-16-0170.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Krocak, M. J., J. T. Ripberger, C. Silva, H. Jenkins-Smith, S. Ernst, A. Bell, and J. Allan, 2020: Measuring change: Public reception, understanding, and responses to severe weather forecasts and warnings in the contiguous United States. Harvard Dataverse, accessed 10 November 2021, https://doi.org/10.7910/DVN/EWOCUA.

    • Crossref
    • Export Citation

  • Krocak, M. J., S. Ernst, J. T. Ripberger, C. Williams, J. Trujillo-Falcón, B. T. Gallo, and P. Marsh, 2021a: The National Weather Service Storm Prediction Center’s convective outlook: Conclusions from past research and recommendations for future development. NOAA, 16 pp., https://www.spc.noaa.gov/publications/krocak/otlk-res.pdf.

  • Krocak, M. J., J. T. Ripberger, C. L. Silva, and B. T. Gallo, 2021b: Enhancing the Storm Prediction Center’s convective outlook with continuous probabilities and conditional intensity forecasts. NOAA Funding Award Project Narrative Award NA21-OAR4590173, 46 pp.

  • Lenhardt, E. D., R. N. Cross, M. J. Krocak, J. T. Ripberger, S. R. Ernst, C. L. Silva, and H. C. Jenkins-Smith, 2020: How likely is that chance of thunderstorms? A study of how National Weather Service Forecast Offices use words of estimative probability and what they mean to the public. J. Oper Meteor., 8, 64–78, https://doi.org/10.15191/nwajom.2020.0805.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Mason, J. B., and J. C. Senkbeil, 2015: A tornado watch scale to improve public response. Wea. Climate Soc., 7, 146–158, https://doi.org/10.1175/WCAS-D-14-00035.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Morss, R. E., J. L. Demuth, and J. K. Lazo, 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974–991, https://doi.org/10.1175/2008WAF2007088.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • National Research Council, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. National Academies Press, 124 pp.

  • NOAA/Storm Prediction Center, 2020: SPC products. NOAA/Storm Prediction Center, accessed 1 December 2020, https://www.spc.noaa.gov/misc/about.html.

  • NOAA/Storm Prediction Center, 2021: National Weather Service instruction. Tech. Rep. 10-512, NOAA/Storm Prediction Center, 73 pp., https://www.nws.noaa.gov/directives/sym/pd01005012curr.pdf.

  • Rosen, Z., M. J. Krocak, J. T. Ripberger, R. Cross, E. Lenhardt, C. L. Silva, and H. C. Jenkins-Smith, 2021: Communicating probability information in hurricane forecasts: Assessing statements that forecasters use on social media and implications for public assessments of reliability. J. Oper. Meteor., 9, 89–101, https://doi.org/10.15191/nwajom.2021.0907.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Schwartz, L. M., S. Woloshin, W. C. Black, and H. G. Welch, 1997: The role of numeracy in understanding the benefit of screening mammography. Ann. Intern. Med., 127, 966–972, https://doi.org/10.7326/0003-4819-127-11-199712010-00003.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Williams, C. A., A. J. Grundstein, and J. So, 2020: Should severe weather graphics wear uniforms? Understanding the effects of inconsistent convective outlook graphics on members of the public. 15th Symp. on Societal Applications: Policy, Research, and Practice, Boston, MA, Amer. Meteor. Soc., 11A.1, https://ams.confex.com/ams/2020Annual/meetingapp.cgi/Paper/365011.

  • Williams, C. A., A. J. Grundstein, and J. So, 2021: What is being enhanced?: An examination of the Storm Prediction Center’s risk category system among members of the public. 16th Symp. on Societal Applications: Policy, Research, and Practice, virtual, Amer. Meteor. Soc., J4.1, https://ams.confex.com/ams/101ANNUAL/meetingapp.cgi/Paper/380671.

  • Wintle, B. C., H. Fraser, B. C. Wills, A. E. Nicholson, and F. Fidler, 2019: Verbal probabilities: Very likely to be somewhat more confusing than numbers. PLOS ONE, 14, e0213522, https://doi.org/10.1371/journal.pone.0213522.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 508 0 0
Full Text Views 2346 1433 131
PDF Downloads 1277 520 48