• APA, 2020: Hazards planning. American Planning Association, accessed 19 June 2020, https://www.planning.org/nationalcenters/hazards/.

  • Argyle, E. M., J. J. Gourley, Z. L. Flamig, T. Hansen, and K. Manross, 2017: Toward a user-centered design of a weather forecasting decision-support tool. Bull. Amer. Meteor. Soc., 98, 373382, https://doi.org/10.1175/BAMS-D-16-0031.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blackstock, K. L., G. J. Kelly, and B. L. Horsey, 2007: Developing and applying a framework to evaluate participatory research for sustainability. Ecol. Econ., 60, 726742, https://doi.org/10.1016/j.ecolecon.2006.05.014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Braun, V., and V. Clarke, 2006: Using thematic analysis in psychology. Qual. Res. Psychol., 3, 77101, https://doi.org/10.1191/1478088706qp063oa.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Briley, L., D. Brown, and S. E. Kalafatis, 2015: Overcoming barriers during the co-production of climate information for decision-making. Climate Risk Manage., 9, 4149, https://doi.org/10.1016/j.crm.2015.04.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cash, D. W., and W. C. Clark, 2001: From science to policy: Assessing the assessment process. Harvard University John F. Kennedy School of Government RWP01-045, 20 pp., https://doi.org/10.2139/ssrn.295570.

    • Crossref
    • Export Citation
  • Cash, D. W., W. C. Clark, F. Alcock, N. M. Dickson, N. Eckley, D. H. Guston, J. Jager, and R. B. Mitchell, 2003: Knowledge systems for sustainable development. Proc. Natl. Acad. Sci. USA, 100, 80868091, https://doi.org/10.1073/pnas.1231332100.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cash, D. W., J. C. Borck, and A. G. Patt, 2006: Countering the loading-dock approach to linking science and decision making: Comparative analysis of the El Niño/Southern Oscillation (ENSO) forecasting systems. Sci. Technol. Hum. Values, 31, 465494, https://doi.org/10.1177/0162243906287547.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dilling, L., and M. C. Lemos, 2011: Creating usable science: Opportunities and constraints for climate knowledge use and their implications for science policy. Global Environ. Change, 21, 680689, https://doi.org/10.1016/j.gloenvcha.2010.11.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ellenburg, W. L., R. T. McNider, J. F. Cruise, and J. R. Christy, 2016: Towards an understanding of the twentieth-century cooling trend in the southeastern United States: Biogeophysical impacts of land-use change. Earth Interact., 20, https://doi.org/10.1175/EI-D-15-0038.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Etz, K. E., and J. A. Arroyo, 2015: Small sample research: Considerations beyond statistical power. Prev. Sci., 16, 10331036, https://doi.org/10.1007/s11121-015-0585-4.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, D. L., and H. M. Ingram, 2009: Making science useful to decision makers: Climate forecasts, water management, and knowledge networks. Wea. Climate Soc., 1, 921, https://doi.org/10.1175/2009WCAS1007.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • FEMA, 2019: Hazard mitigation plan requirement. Accessed 12 December 2019, https://www.fema.gov/hazard-mitigation-plan-requirement.

  • Finucane, M. L., R. Miller, L. K. Corlew, V. W. Keener, M. Burkett, and Z. Grecni, 2013: Understanding the climate-sensitive decisions and information needs of freshwater resource managers in Hawaii. Wea. Climate Soc., 5, 293308, https://doi.org/10.1175/WCAS-D-12-00039.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goodrich, K. A., K. D. Sjostrom, C. Vaughan, L. Nichols, A. Bednarek, and M. Carmen Lemos, 2020: Who are boundary spanners and how can we support them in making knowledge more actionable in sustainability fields? Curr. Opin. Environ. Sustainability, 42, 4551, https://doi.org/10.1016/j.cosust.2020.01.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guido, Z., D. Hill, M. Crimmins, and D. Ferguson, 2013: Informing decisions with a climate synthesis product: Implications for regional climate services. Wea. Climate Soc., 5, 8392, https://doi.org/10.1175/WCAS-D-12-00012.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hartmann, H. C., T. C. Pagano, S. Sorooshian, and R. Bales, 2002: Confidence builders: Evaluating seasonal climate forecasts from user perspectives. Bull. Amer. Meteor. Soc., 83, 683698, https://doi.org/10.1175/1520-0477(2002)083<0683:CBESCF>2.3.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hawkins, M. D., V. Brown, and J. Ferrell, 2017: Assessment of NOAA National Weather Service methods to warn for extreme heat events. Wea. Climate Soc., 9, 513, https://doi.org/10.1175/WCAS-D-15-0037.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hewitson, B., K. Waagsaether, J. Wohland, K. Kloppers, and T. Kara, 2017: Climate information websites: An evolving landscape. Wiley Interdiscip. Rev.: Climate Change, 8, e470, https://doi.org/10.1002/wcc.470.

    • Search Google Scholar
    • Export Citation
  • Hocker, J. E., A. D. Melvin, K. A. Kloesel, C. A. Fiebrich, R. W. Hill, R. D. Smith, and S. F. Piltz, 2018: The evolution and impact of a meteorological outreach program for public safety officials: An update on the Oklahoma Mesonet’s OK-First Program. Bull. Amer. Meteor. Soc., 99, 20092024, https://doi.org/10.1175/BAMS-D-17-0100.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hopkin, C. R., R. H. Hoyle, and N. C. Gottfredson, 2015: Maximizing the yield of small samples in prevention research: A review of general strategies and best practices. Prev. Sci., 16, 950955, https://doi.org/10.1007/s11121-014-0542-7.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • International Standards Organization, 2018: Usability: Definitions and concepts. Part 11, Ergonomics of Human-System Interaction, ISO 9241, https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en.

  • Kirchhoff, C. J., M. C. Lemos, and S. Dessai, 2013: Actionable knowledge for environmental decision making: Broadening the usability of climate science. Annu. Rev. Environ. Resour., 38, 393414, https://doi.org/10.1146/annurev-environ-022112-112828.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Knott, J., and A. Wildavsky, 1980: If dissemination is the solution, what is the problem? Knowl. Creat. Diffus. Util., 1, 421442.

  • Lemos, M. C., and B. J. Morehouse, 2005: The co-production of science and policy in integrated climate assessments. Global Environ. Change, 15, 5768, https://doi.org/10.1016/j.gloenvcha.2004.09.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lemos, M. C., C. J. Kirchhoff, and V. Ramprasad, 2012: Narrowing the climate information usability gap. Nat. Climate Change, 2, 789794, https://doi.org/10.1038/nclimate1614.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lempert, R., J. Arnold, R. Pulwarty, K. Gordon, K. Greig, C. Hawkins Hoffman, D. Sands, and C. Werrell, 2018: Reducing risks through adaptation actions. Impacts, Risks, and Adaptation in the United States: Fourth National Climate Assessment, D. R. Reidmiller et al., Eds., Vol. II, U.S. Global Change Research Program, 13091345, https://doi.org/10.7930/NCA4.2018.CH28.

    • Search Google Scholar
    • Export Citation
  • Lexico, 2020: Definition of suitability in English. Accessed 20 May 2020, https://www.lexico.com/en/definition/suitability.

  • Lourenço, T. C., R. Swart, H. Goosen, and R. Street, 2016: The rise of demand-driven climate services. Nat. Climate Change, 6, 1314, https://doi.org/10.1038/nclimate2836.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Maudlin, L. C., K. S. McNeal, H. Dinon-Aldridge, C. Davis, R. Boyles, and R. M. Atkins, 2020: Website usability differences between males and females: An eye-tracking evaluation of a climate decision support system. Wea. Climate Soc., 12, 183192, https://doi.org/10.1175/WCAS-D-18-0127.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McNie, E. C., 2007: Reconciling the supply of scientific information with user demands: An analysis of the problem and review of the literature. Environ. Sci. Policy, 10, 1738, https://doi.org/10.1016/j.envsci.2006.10.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meadow, A. M., D. B. Ferguson, Z. Guido, A. Horangic, G. Owen, and T. Wall, 2015: Moving toward the deliberate coproduction of climate science knowledge. Wea. Climate Soc., 7, 179191, https://doi.org/10.1175/WCAS-D-14-00050.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morss, R. E., J. K. Lazo, and J. L. Demuth, 2010: Examining the use of weather forecasts in decision scenarios: Results from a US survey with implications for uncertainty communication. Meteor. Appl., 17, 149162, https://doi.org/10.1002/met.196.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Novak, D. R., D. R. Bright, and M. J. Brennan, 2008: Operational forecaster uncertainty needs and future roles. Wea. Forecasting, 23, 10691084, https://doi.org/10.1175/2008WAF2222142.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • NRC, 2009: Informing Decisions in a Changing Climate. The National Academies Press, 200 pp., https://doi.org/10.17226/12626.

  • NRC, 2010: Informing an Effective Response to Climate Change. The National Academies Press, 348 pp.

  • Oakley, N. S., and B. Daudert, 2016: Establishing best practices to improve usefulness and usability of web interfaces providing atmospheric data. Bull. Amer. Meteor. Soc., 97, 263274, https://doi.org/10.1175/BAMS-D-14-00121.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Oh, C. H., 1996: Linking Social Science Information to Policy-Making. Emerald Group Publishing, 201 pp.

  • Overpeck, J. T., G. A. Meehl, S. Bony, and D. R. Easterling, 2011: Climate data challenges in the 21st century. Science, 331, 700702, https://doi.org/10.1126/science.1197869.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Page, R., and L. Dilling, 2019: The critical role of communities of practice and peer learning in scaling hydroclimate information adoption. Wea. Climate Soc., 11, 851862, https://doi.org/10.1175/WCAS-D-18-0130.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Parris, A. S., G. M. Garfin, K. Dow, R. Meter, and S. L. Close, 2016: Climate in Context: Science and Society Partnering for Adaptation. John Wiley and Sons, 312 pp.

    • Search Google Scholar
    • Export Citation
  • Porter, K., C. Scawthorn, N. Dash, and J. Santos, 2017: Natural hazard mitigation saves: 2017 interim report. National Institute of Building Sciences, Multihazard Mitigation Council, 16 pp., https://www.fema.gov/media-library-data/1516812817859-9f866330bd6a1a93f54cdc61088f310a/MS2_2017InterimReport.pdf.

  • Riley, R., R. Edwards, L. Carter, M. Shafer, and M. Boone, 2013: South central U.S. hazard and climate change planning assessment. Southern Climate Impacts Planning Program, 55 pp., http://www.southernclimate.org/publications/Hazard_Planning_Assessment.pdf.

  • Schwab, J. C., 2010: Hazard mitigation: Integrating best practices into planning. American Planning Association Planning Advisory Service Rep. 560, 146 pp., https://www.planning.org/publications/report/9026884/.

  • Shafer, M. A., 2008: Climate literacy and a national climate service. Phys. Geogr., 29, 561574, https://doi.org/10.2747/0272-3646.29.6.561.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • VanderMolen, K., T. U. Wall, and B. Daudert, 2019: A call for the evaluation of web-based climate data and analysis tools. Bull. Amer. Meteor. Soc., 100, 257268, https://doi.org/10.1175/BAMS-D-18-0006.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wall, T. U., T. J. Brown, and N. J. Nausler, 2017a: Spot weather forecasts: Improving utilization, communication, and perceptions of accuracy in sophisticated user groups. Wea. Climate Soc., 9, 215226, https://doi.org/10.1175/WCAS-D-15-0055.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wall, T. U., A. M. Meadow, and A. Horangic, 2017b: Developing evaluation indicators to improve the process of coproducing usable climate science. Wea. Climate Soc., 9, 95107, https://doi.org/10.1175/WCAS-D-16-0008.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 114 114 114
Full Text Views 22 22 22
PDF Downloads 27 27 27

An Evaluation of the Utility of a Decision-Maker-Driven Climate Hazard Assessment Tool

View More View Less
  • 1 Southern Climate Impacts Planning Program, University of Oklahoma, Norman, Oklahoma
© Get Permissions
Restricted access

Abstract

Decision-makers who have little to no formal training in atmospheric science are increasingly accessing and interpreting climate data and information within planning contexts. Many climate decision support tools (DSTs) have been developed to support decision-making across a variety of sectors and scales, but evaluation of such tools has only recently begun to take place. This study conducted a summative evaluation of the utility of a decision-maker-driven climate hazard assessment tool, the Simple Planning Tool (SPT), a climate DST. The SPT was inspired by and codeveloped with emergency managers, planners, and a boundary organization in two south-central U.S. states. The SPT’s target audience was surveyed to assess the tool’s utility, including its saliency, credibility, trustworthiness, and reasons for and impact of information use on decision-making. A high utility was found despite a relatively limited user base at the time of the study. In addition, SPT users represented a range of jurisdictional sizes, geographical scales, and years of experience. Although the small user sample limits generalizability of the study, it is likely a realistic reflection of the number of emergency managers and planners in the two states who are actively and regularly incorporating climate hazards into planning. The data also indicate that climate boundary organizations and climate service providers should work toward utilizing trusted information sources, channels, and procedures within the sectors to which their tool applies to help increase decision-maker awareness and use of their tool.

Significance Statement

A Simple Planning Tool (SPT) for Oklahoma and Arkansas climate hazards was developed in response to needs identified by planners and emergency managers in two states. This study evaluated the usefulness and impact of the SPT about one year after it became available and adds to the limited body of literature on climate decision support tool evaluation. The evaluation showed that that tool was useful and informed plans, but for a limited user base. Climate decision support tool communicators can help raise awareness of their tool(s) by utilizing existing decision-maker trusted information channels. They should also recognize that the full impact of their tool may not be known for several years, especially within the context of long-range planning.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Rachel E. Riley, rriley@ou.edu

Abstract

Decision-makers who have little to no formal training in atmospheric science are increasingly accessing and interpreting climate data and information within planning contexts. Many climate decision support tools (DSTs) have been developed to support decision-making across a variety of sectors and scales, but evaluation of such tools has only recently begun to take place. This study conducted a summative evaluation of the utility of a decision-maker-driven climate hazard assessment tool, the Simple Planning Tool (SPT), a climate DST. The SPT was inspired by and codeveloped with emergency managers, planners, and a boundary organization in two south-central U.S. states. The SPT’s target audience was surveyed to assess the tool’s utility, including its saliency, credibility, trustworthiness, and reasons for and impact of information use on decision-making. A high utility was found despite a relatively limited user base at the time of the study. In addition, SPT users represented a range of jurisdictional sizes, geographical scales, and years of experience. Although the small user sample limits generalizability of the study, it is likely a realistic reflection of the number of emergency managers and planners in the two states who are actively and regularly incorporating climate hazards into planning. The data also indicate that climate boundary organizations and climate service providers should work toward utilizing trusted information sources, channels, and procedures within the sectors to which their tool applies to help increase decision-maker awareness and use of their tool.

Significance Statement

A Simple Planning Tool (SPT) for Oklahoma and Arkansas climate hazards was developed in response to needs identified by planners and emergency managers in two states. This study evaluated the usefulness and impact of the SPT about one year after it became available and adds to the limited body of literature on climate decision support tool evaluation. The evaluation showed that that tool was useful and informed plans, but for a limited user base. Climate decision support tool communicators can help raise awareness of their tool(s) by utilizing existing decision-maker trusted information channels. They should also recognize that the full impact of their tool may not be known for several years, especially within the context of long-range planning.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Rachel E. Riley, rriley@ou.edu
Save