• Advisory Committee on Climate Change and Natural Resource Science, 2015: Report to the Secretary of the Interior. USGS Rep., 86 pp., https://www.sciencebase.gov/catalog/item/5c1d05d3e4b0708288c9bc2a.

  • Archie, K. M., 2014: Mountain communities and climate change adaptation: Barriers to planning and hurdles to implementation in the Southern Rocky Mountain Region of North America. Mitigation Adapt. Strategies Global Change, 19, 569587, https://doi.org/10.1007/s11027-013-9449-z.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Archie, K. M., L. Dilling, J. B. Milford, and F. C. Pampel, 2014: Unpacking the ‘information barrier’: Comparing perspectives on information as a barrier to climate change adaptation in the interior mountain west. J. Environ. Manage., 133, 397410, https://doi.org/10.1016/j.jenvman.2013.12.015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ardener, S., 2005: Ardener’s “muted groups”: The genesis of an idea and its praxis. Women Lang, 28, 5054, 72.

  • Arguez, A., I. Durre, S. Applequist, R. S. Vose, M. F. Squires, X. Yin, R. R. Heim, and T. W. Owen, 2012: NOAA’s 1981–2010 U.S. climate normals: An overview. Bull. Amer. Meteor. Soc., 93, 16871697, https://doi.org/10.1175/BAMS-D-11-00197.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Arnott, J. C., K. J. Mach, and G. Wong-Parodi, 2020a: Editorial overview: The science of actionable knowledge. Curr. Opin. Environ. Sustainability, 42, A1A5, https://doi.org/10.1016/j.cosust.2020.03.007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Arnott, J. C., R. J. Neuenfeldt, and M. C. Lemos, 2020b: Co-producing science for sustainability: Can funding change knowledge use? Global Environ. Change, 60, 101979, https://doi.org/10.1016/j.gloenvcha.2019.101979.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Arnstein, S. R., 1969: A ladder of citizen participation. J. Amer. Inst. Plann., 35, 216224, https://doi.org/10.1080/01944366908977225.

  • Averyt, K., and Coauthors, 2017: Regional climate response collaboratives: Multi-institutional support for climate resilience. Bull. Amer. Meteor. Soc., 99, 891898, https://doi.org/10.1175/BAMS-D-17-0183.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barry, J., and M. Thompson-Fawcett, 2020: Decolonizing the boundaries between the ‘planner’ and the ‘planned’: Implications of Indigenous property development. Plann. Theory Pract., 21, 410425, https://doi.org/10.1080/14649357.2020.1775874.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bartlett, C., M. Marshall, and A. Marshall, 2012: Two-eyed seeing and other lessons learned within a co-learning journey of bringing together Indigenous and mainstream knowledges and ways of knowing. J. Environ. Stud. Sci., 2, 331340, https://doi.org/10.1007/s13412-012-0086-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bathke, D., T. Haigh, T. Bernadt, and N. Wall, 2019: Drought scenario-based exercises. National Drought Mitigation Center Rep., 79 pp., https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1011&context=ndmcpub.

  • Beaury, E. M., E. J. Fusco, M. R. Jackson, B. B. Laginhas, T. L. Morelli, J. M. Allen, V. J. Pasquarella, and B. A. Bradley, 2020: Incorporating climate change into invasive species management: Insights from managers. Biol. Invasions, 22, 233252, https://doi.org/10.1007/s10530-019-02087-6.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bednarek, A. T., and Coauthors, 2018: Boundary spanning at the science–policy interface: The practitioners’ perspectives. Sustainability Sci., 13, 11751183, https://doi.org/10.1007/s11625-018-0550-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Beeton, T. A., S. M. McNeeley, B. W. Miller, and D. S. Ojima, 2019: Grounding simulation models with qualitative case studies: Toward a holistic framework to make climate science usable for US public land management. Climate Risk Manage., 23, 5066, https://doi.org/10.1016/j.crm.2018.09.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Beier, P., L. Hansen, L. Helbrecht, and D. Behar, 2017: A how-to guide for coproduction of actionable science. Conserv. Lett., 10, 288296, https://doi.org/10.1111/conl.12300.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Beierle, T., 2002: The quality of stakeholder-based decisions. Risk Anal., 22, 739749, https://doi.org/10.1111/0272-4332.00065.

  • Bierbaum, R., and Coauthors, 2013: A comprehensive review of climate adaptation in the United States: More than before, but less than needed. Mitigation Adapt. Strategies Global Change, 18, 361406, https://doi.org/10.1007/s11027-012-9423-1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bisbal, G. A., 2019: Practical tips to establish an actionable science portfolio for climate adaptation. Sci. Public Policy, 46, 148153, https://doi.org/10.1093/scipol/scy070.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bojovic, D., A. L. Clair St., I. Christel, M. Terrado, P. Stanzel, P. Gonzalez, and E. J. Palin, 2021: Engagement, involvement and empowerment: Three realms of a coproduction framework for climate services. Global Environ. Change, 68, 102271, https://doi.org/10.1016/j.gloenvcha.2021.102271.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Boy, G. A., 2017: The Handbook of Human-Machine Interaction: A Human-Centered Design Approach. CRC Press, 478 pp.

    • Crossref
    • Export Citation
  • Bracken, L. J., H. A. Bulkeley, and G. Whitman, 2015: Transdisciplinary research: Understanding the stakeholder perspective. J. Environ. Plann. Manage., 58, 12911308, https://doi.org/10.1080/09640568.2014.921596.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bradbury, H., S. Waddell, K. O’ Brien, M. Apgar, B. Teehankee, and I. Fazey, 2019: A call to action research for transformations: The times demand it. Action Res., 17, 310, https://doi.org/10.1177/1476750319829633.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bremer, S., and S. Meisch, 2017: Co-production in climate change research: Reviewing different perspectives. Wiley Interdiscip. Rev.: Climate Change, 8, e482, https://doi.org/10.1002/wcc.482.

    • Crossref
    • Export Citation
  • Briley, L., D. Brown, and S. E. Kalafatis, 2015: Overcoming barriers during the co-production of climate information for decision-making. Climate Risk Manage., 9, 4149, https://doi.org/10.1016/j.crm.2015.04.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brydon-Miller, M., D. Greenwood, and P. Maguire, 2003: Why action research? Action Res., 1, 928, https://doi.org/10.1177/14767503030011002.

  • Bush, V., 1945: Science, the endless frontier: A report to the President. U.S. Government Printing Office Doc., https://www.nsf.gov/about/history/vbush1945.htm.

  • Carter, S. K., and Coauthors, 2020: Bridging the research-management gap: Landscape science in practice on public lands in the western United States. Landscape Ecol., 35, 545560, https://doi.org/10.1007/s10980-020-00970-5.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cash, D., W. C. Clark, F. Alcock, N. M. Dickson, N. Eckley, and J. Jäger, 2003: Salience, credibility, legitimacy and boundaries: Linking research, assessment and decision making. KSG Working Paper RWP02-046, 24 pp.

    • Crossref
    • Export Citation
  • Cash, D., J. C. Borck, and A. G. Patt, 2006: Countering the loading-dock approach to linking science and decision making: Comparative analysis of El Niño/Southern Oscillation (ENSO) forecasting systems. Sci. Technol. Hum. Values, 31, 465494, https://doi.org/10.1177/0162243906287547.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christel, I., D. Hemment, D. Bojovic, F. Cucchietti, L. Calvo, M. Stefaner, and C. Buontempo, 2018: Introducing design in the development of effective climate services. Climate Serv., 9, 111121, https://doi.org/10.1016/j.cliser.2017.06.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clifford, K. R., W. R. Travis, and L. T. Nordgren, 2020: A climate knowledges approach to climate services. Climate Serv., 18, 100155, https://doi.org/10.1016/j.cliser.2020.100155.

    • Search Google Scholar
    • Export Citation
  • Cloyd, E., S. C. Moser, E. Maibach, J. Maldonado, and T. Chen, 2016: Engagement in the Third U.S. National Climate Assessment: Commitment, capacity, and communication for impact. Climatic Change, 135, 3954, https://doi.org/10.1007/s10584-015-1568-y.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Collins, C. S., and C. M. Stockton, 2018: The central role of theory in qualitative research. Int. J. Qual. Methods, 17, 1609406918797475, https://doi.org/10.1177/1609406918797475.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crausbay, S. D., and Coauthors, 2020: Unfamiliar territory: Emerging themes for ecological drought research and management. One Earth, 3, 337353, https://doi.org/10.1016/j.oneear.2020.08.019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cravens, A., and N. Ardoin, 2016: Negotiating credibility and legitimacy in the shadow of an authoritative data source. Ecol. Soc., 21, art30, https://doi.org/10.5751/ES-08849-210430.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Davidson, S., 1998: Spinning the wheel of empowerment. Planning, 1262, 1415.

  • DeCrappeo, N. M., G. A. Bisbal, and A. M. Meadow, 2017: A path to actionable climate science: Perspectives from the field. Environ. Manage., 61, 181187, https://doi.org/10.1007/s00267-017-0960-y.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dilling, L., and M. C. Lemos, 2011: Creating usable science: Opportunities and constraints for climate knowledge use and their implications for science policy. Global Environ. Change, 21, 680689, https://doi.org/10.1016/j.gloenvcha.2010.11.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dilling, L., K. Lackstrom, B. Haywood, K. Dow, M. C. Lemos, J. Berggren, and S. Kalafatis, 2015: What stakeholder needs tell us about enabling adaptive capacity: The intersection of context and information provision across regions in the United States. Wea. Climate Soc., 7, 517, https://doi.org/10.1175/WCAS-D-14-00001.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Diver, S., 2017: Negotiating Indigenous knowledge at the science-policy interface: Insights from the Xáxli’p Community Forest. Environ. Sci. Policy, 73, 111, https://doi.org/10.1016/j.envsci.2017.03.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doemeland, D., and J. Trevino, 2014: Which World Bank reports are widely read? World Bank Working Paper 6851, 32 pp., https://documents1.worldbank.org/curated/en/387501468322733597/pdf/WPS6851.pdf.

    • Crossref
    • Export Citation
  • Doswell, C. A., A. R. Moller, and H. E. Brooks, 1999: Storm spotting and public awareness since the first tornado forecasts of 1948. Wea. Forecasting, 14, 544557, https://doi.org/10.1175/1520-0434(1999)014<0544:SSAPAS>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Enquist, C., and Coauthors, 2017: Foundations of translational ecology. Front. Ecol. Environ., 15, 541550, https://doi.org/10.1002/fee.1733.

  • Eyring, V., S. Bony, G. A. Meehl, C. A. Senior, B. Stevens, R. J. Stouffer, and K. E. Taylor, 2016: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci. Model Dev., 9, 19371958, https://doi.org/10.5194/gmd-9-1937-2016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Few, R., K. Brown, and E. Tompkins, 2007: Public participation and climate change adaptation: Avoiding the illusion of inclusion. Climate Policy, 7, 4659, https://doi.org/10.1080/14693062.2007.9685637.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Filho, W. L., Ed., 2015: Handbook of Climate Change Adaptation. Springer-Verlag, 2198 pp.

  • Fisher, J. R. B., S. A. Wood, M. A. Bradford, and T. R. Kelsey, 2020: Improving scientific impact: How to practice science that influences environmental policy and management. Conserv. Sci. Pract., 2, e210, https://doi.org/10.1111/csp2.210.

    • Search Google Scholar
    • Export Citation
  • Friesen, P., L. Kearns, B. Redman, and A. L. Caplan, 2017: Rethinking the Belmont report? Amer. J. Bioeth., 17, 1521, https://doi.org/10.1080/15265161.2017.1329482.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fyfe, A., K. Coate, S. Curry, S. Lawson, N. Moxham, and C. M. Røstvik, 2017: Untangling academic publishing: A history of the relationship between commercial interests, academic prestige and the circulation of research. U.K. Arts and Humanities Research Council Briefing Paper, 23 pp., https://doi.org/10.5281/zenodo.546100.

    • Crossref
    • Export Citation
  • Gasson, S., 2003: Human-centered vs. user-centered approaches to information system design. J. Inf. Technol. Theory Appl., 5, 29–46.

  • Gerlak, A. K., T. Heikkila, and J. Newig, 2020: Learning in environmental governance: Opportunities for translating theory to practice. J. Environ. Policy Plann., 22, 653666, https://doi.org/10.1080/1523908X.2020.1776100.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gluckman, P., and J. Wilsdon, 2016: From paradox to principles: Where next for scientific advice to governments? Palgrave Commun., 2, 16077, https://doi.org/10.1057/palcomms.2016.77.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goodrich, K. A., K. D. Sjostrom, C. Vaughan, L. Nichols, A. Bednarek, and M. C. Lemos, 2020: Who are boundary spanners and how can we support them in making knowledge more actionable in sustainability fields? Curr. Opin. Environ. Sustainability, 42, 4551, https://doi.org/10.1016/j.cosust.2020.01.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guido, Z., D. Hill, M. Crimmins, and D. Ferguson, 2013: Informing decisions with a climate synthesis product: Implications for regional climate services. Wea. Climate Soc., 5, 8392, https://doi.org/10.1175/WCAS-D-12-00012.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hassol, S. J., 2008: Improving how scientists communicate about climate change. Eos, Trans. Amer. Geophys. Union, 89, 106107, https://doi.org/10.1029/2008EO110002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hegger, D., M. Lamers, A. V. Zeijl-Rozema, and C. Dieperink, 2012: Conceptualising joint knowledge production in regional climate change adaptation projects: Success conditions and levers for action. Environ. Sci. Policy, 18, 5265, https://doi.org/10.1016/j.envsci.2012.01.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hegger, D., A. V. Zeijl-Rozema, and C. Dieperink, 2014: Toward design principles for joint knowledge production projects: Lessons from the deepest polder of The Netherlands. Reg. Environ. Change, 14, 10491062, https://doi.org/10.1007/s10113-012-0382-6.

    • Search Google Scholar
    • Export Citation
  • Human, B. A., and A. Davies, 2010: Stakeholder consultation during the planning phase of scientific programs. Mar. Policy, 34, 645654, https://doi.org/10.1016/j.marpol.2009.12.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • IDEO, 2015: The Field Guide to Human-Centered Design. 1st ed. IDEO.org/Design Kit, 192 pp.

  • Imenda, S., 2014: Is there a conceptual difference between theoretical and conceptual frameworks? J. Soc. Sci., 38, 185195, https://doi.org/10.1080/09718923.2014.11893249.

    • Search Google Scholar
    • Export Citation
  • IPCC, 2018: Summary for policymakers. Global Warming of 1.5°C, V. Masson-Delmotte et al., Eds., Cambridge University Press, 32 pp., https://www.ipcc.ch/site/assets/uploads/sites/2/2019/05/SR15_SPM_version_report_LR.pdf.

  • Jagannathan, K., A. D. Jones, and I. Ray, 2020: The making of a metric: Co-producing decision-relevant climate science. Bull. Amer. Meteor. Soc., 102, E1579E1590, https://doi.org/10.1175/BAMS-D-19-0296.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jansen, T., L. Claassen, I. van Kamp, and D. R. M. Timmermans, 2019: Understanding of the concept of ‘uncertain risk’. A qualitative study among different societal groups. J. Risk Res., 22, 658672, https://doi.org/10.1080/13669877.2018.1503614.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jantarasami, L., J. Lawler, and C. Thomas, 2010: Institutional barriers to climate change adaptation in U.S. National Parks and Forests. Ecol. Soc., 15, art33, https://doi.org/10.5751/ES-03715-150433.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jarvis, R. M., and Coauthors, 2020: Navigating spaces between conservation research and practice: Are we making progress? Ecol. Solutions Evidence, 1, e12028, https://doi.org/10.1002/2688-8319.12028.

    • Search Google Scholar
    • Export Citation
  • Jasanoff, S., Ed., 2004: States of Knowledge: The Co-production of Science and the Social Order. Routledge, 317 pp.

  • Johnson, C. J., and M. P. Gillingham, 2004: Mapping uncertainty: Sensitivity of wildlife habitat ratings to expert opinion. J. Appl. Ecol., 41, 10321041, https://doi.org/10.1111/j.0021-8901.2004.00975.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kalafatis, S. E., M. C. Lemos, Y.-J. Lo, and K. A. Frank, 2015: Increasing information usability for climate adaptation: The role of knowledge networks and communities of practice. Global Environ. Change, 32, 3039, https://doi.org/10.1016/j.gloenvcha.2015.02.007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kemp, K. B., J. J. Blades, P. Z. Klos, T. E. Hall, J. E. Force, P. Morgan, and W. T. Tinkham, 2015: Managing for climate change on federal lands of the western United States: Perceived usefulness of climate science, effectiveness of adaptation strategies, and barriers to implementation. Ecol. Soc., 20, 17, https://doi.org/10.5751/ES-07522-200217.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kimmerer, R. W., 2014: Returning the gift. Minding Nat., 7, 18–24.

  • Kirby, C. K., C. Haruo, K. P. Whyte, J. C. Libarkin, C. Caldwell, and R. Edler, 2019: Ethical collaboration and the need for training: Partnerships between Native American Tribes and climate science organisations. Gateways Int. J. Community Res. Engagement, 12, 5894–5894, https://doi.org/10.5130/ijcre.v12i1.5894.

    • Crossref
    • Export Citation
  • Kirchhoff, C. J., M. C. Lemos, and S. Dessai, 2013: Actionable knowledge for environmental decision making: Broadening the usability of climate science. Annu. Rev. Environ. Resour., 38, 393414, https://doi.org/10.1146/annurev-environ-022112-112828.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kirchhoff, C. J., R. Esselman, and D. Brown, 2015: Boundary organizations to boundary chains: Prospects for advancing climate science application. Climate Risk Manage., 9, 2029, https://doi.org/10.1016/j.crm.2015.04.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Klemm, T., and R. A. McPherson, 2018: Assessing decision timing and seasonal climate forecast needs of winter wheat producers in the south-central United States. J. Appl. Meteor. Climatol., 57, 21292140, https://doi.org/10.1175/JAMC-D-17-0246.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Klenk, N., A. Fiume, K. Meehan, and C. Gibbes, 2017: Local knowledge in climate adaptation research: Moving knowledge frameworks from extraction to co-production. Wiley Interdiscip. Rev.: Climate Change, 8, e475, https://doi.org/10.1002/wcc.475.

    • Crossref
    • Export Citation
  • Kruk, M. C., R. Vose, R. Heim, A. Arguez, J. Enloe, X. Yin, and T. Wallis, 2018: Drought amelioration: An engagement-to-implementation success story. Bull. Amer. Meteor. Soc., 99, 24572462, https://doi.org/10.1175/BAMS-D-17-0177.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • LaChapelle, P. R., S. F. McCool, and M. E. Patterson, 2003: Barriers to effective natural resource planning in a “messy” world. Soc. Nat. Resour., 16, 473490, https://doi.org/10.1080/08941920309151.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Land, M., B. Macura, C. Bernes, and S. Johansson, 2017: A five-step approach for stakeholder engagement in prioritisation and planning of environmental evidence syntheses. Environ. Evid., 6, 25, https://doi.org/10.1186/s13750-017-0104-0.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Latulippe, N., and N. Klenk, 2020: Making room and moving over: Knowledge co-production, Indigenous knowledge sovereignty and the politics of global environmental change decision-making. Curr. Opin. Environ. Sustainability, 42, 714, https://doi.org/10.1016/j.cosust.2019.10.010.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Laursen, S., N. Puniwai, A. S. Genz, S. A. B. Nash, L. K. Canale, and S. Ziegler-Chong, 2018: Collaboration across worldviews: Managers and scientists on Hawaiʻi Island utilize knowledge coproduction to facilitate climate change adaptation. Environ. Manage., 62, 619630, https://doi.org/10.1007/s00267-018-1069-7.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ledingham, J. A., 2003: Explicating relationship management as a general theory of public relations. J. Public Relat. Res., 15, 181198, https://doi.org/10.1207/S1532754XJPRR1502_4.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ledingham, J. A., and S. D. Bruning, 1998: Relationship management in public relations: Dimensions of an organization-public relationship. Public Relat. Rev., 24, 5565, https://doi.org/10.1016/S0363-8111(98)80020-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lee, K. A., J. R. Lee, and P. Bell, 2020: A review of citizen science within the earth sciences: Potential benefits and obstacles. Proc. Geol. Assoc., 131, 605617, https://doi.org/10.1016/j.pgeola.2020.07.010.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leitch, A. M., J. P. Palutikof, D. Rissik, S. L. Boulter, F. N. Tonmoy, S. Webb, A. C. P. Vidaurre, and M. C. Campbell, 2019: Co-development of a climate change decision support framework through engagement with stakeholders. Climatic Change, 153, 587605, https://doi.org/10.1007/s10584-019-02401-0.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lemos, M. C., C. J. Kirchhoff, and V. Ramprasad, 2012: Narrowing the climate information usability gap. Nat. Climate Change, 2, 789794, https://doi.org/10.1038/nclimate1614.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lemos, M. C., and Coauthors, 2018: To co-produce or not to co-produce. Nat. Sustainability, 1, 722724, https://doi.org/10.1038/s41893-018-0191-0.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lemos, M. C., K. S. Wolske, L. V. Rasmussen, J. C. Arnott, M. Kalcic, and C. J. Kirchhoff, 2019: The closer, the better? Untangling scientist–practitioner engagement, interaction, and knowledge use. Wea. Climate Soc., 11, 535548, https://doi.org/10.1175/WCAS-D-18-0075.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lubchenco, J., 2017: Delivering on science’s social contract. Mich. J. Sustainability, 5, 95108, https://doi.org/10.3998/mjs.12333712.0005.106.

    • Search Google Scholar
    • Export Citation
  • Lynam, T., W. de Jong, D. Sheil, T. Kusumanto, and K. Evans, 2007: A review of tools for incorporating community knowledge, preferences, and values into decision making in natural resources management. Ecol. Soc., 12, art5, https://doi.org/10.5751/ES-01987-120105.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lynn, L. E., Jr., 1978: Knowledge and Policy: The Uncertain Connection. National Academies Press, 184 pp.

  • Mach, K. J., and Coauthors, 2020: Actionable knowledge and the art of engagement. Curr. Opin. Environ. Sustainability, 42, 3037, https://doi.org/10.1016/j.cosust.2020.01.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Magliocca, N. R., and Coauthors, 2018: Closing global knowledge gaps: Producing generalized knowledge from case studies of social-ecological systems. Global Environ. Change, 50, 114, https://doi.org/10.1016/j.gloenvcha.2018.03.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Matsui, K., 2015: Problems of defining and validating traditional knowledge: A historical approach. Int. Indig. Policy J., 6, 2, https://doi.org/10.18584/iipj.2015.6.2.2.

    • Search Google Scholar
    • Export Citation
  • Maudlin, L. C., K. S. McNeal, H. Dinon-Aldridge, C. Davis, R. Boyles, and R. M. Atkins, 2020: Website usability differences between males and females: An eye-tracking evaluation of a climate decision support system. Wea. Climate Soc., 12, 183192, https://doi.org/10.1175/WCAS-D-18-0127.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McCarthy, I. P., D. Hannah, L. F. Pitt, and J. M. McCarthy, 2020: Confronting indifference toward truth: Dealing with workplace bullshit. Bus. Horiz., 63, 253263, https://doi.org/10.1016/j.bushor.2020.01.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McKinley, D. C., and Coauthors, 2017: Citizen science can improve conservation science, natural resource management, and environmental protection. Biol. Conserv., 208, 1528, https://doi.org/10.1016/j.biocon.2016.05.015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McNie, E. C., 2007: Reconciling the supply of scientific information with user demands: An analysis of the problem and review of the literature. Environ. Sci. Policy, 10, 1738, https://doi.org/10.1016/j.envsci.2006.10.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McNie, E. C., 2013: Delivering climate services: Organizational strategies and approaches for producing useful climate-science information. Wea. Climate Soc., 5, 1426, https://doi.org/10.1175/WCAS-D-11-00034.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meadow, A. M., D. B. Ferguson, Z. Guido, A. Horangic, G. Owen, and T. Wall, 2015: Moving toward the deliberate coproduction of climate science knowledge. Wea. Climate Soc., 7, 179191, https://doi.org/10.1175/WCAS-D-14-00050.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morisette, J. T., and Coauthors, 2017: Crossing boundaries in a collaborative modeling workspace. Soc. Nat. Resour., 30, 11581167, https://doi.org/10.1080/08941920.2017.1290178.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moser, S. C., and J. A. Ekstrom, 2010: A framework to diagnose barriers to climate change adaptation. Proc. Natl. Acad. Sci. USA, 107, 22 02622 031, https://doi.org/10.1073/pnas.1007887107.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Naskar, M., K. Roy, G. Karnatak, S. K. Nandy, and A. Roy, 2018: Quantifying climate change induced threats to wetland fisheries: A stakeholder-driven approach. Environ. Dev. Sustainability, 20, 28112830, https://doi.org/10.1007/s10668-017-0018-6.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • National Research Council, 2009: Informing Decisions in a Changing Climate. National Academies Press, 200 pp., https://doi.org/10.17226/12626.

    • Crossref
    • Export Citation
  • National Research Council, 2012: Using Science as Evidence in Public Policy. National Academies Press, 122 pp., https://doi.org/10.17226/13460.

    • Crossref
    • Export Citation
  • Naugle, D. E., B. W. Allred, M. O. Jones, D. Twidwell, and J. D. Maestas, 2020: Coproducing science to inform working lands: The next frontier in nature conservation. BioScience, 70, 9096, https://doi.org/10.1093/biosci/biz144.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nguyen, V. M., N. Young, J. W. Brownscombe, and S. J. Cooke, 2019: Collaboration and engagement produce more actionable science: Quantitatively analyzing uptake of fish tracking studies. Ecol. Appl., 29, e01943, https://doi.org/10.1002/eap.1943.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • NIDIS, 2016: The National Integrated Drought Information System implementation plan—2016 update. NOAA Doc., 31 pp., https://www.drought.gov/sites/default/files/2020-06/Implementation-Plan-December-2016-Update.pdf.

  • Norström, A. V., and Coauthors, 2020: Principles for knowledge co-production in sustainability research. Nat. Sustainability, 3, 182190, https://doi.org/10.1038/s41893-019-0448-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Oakley, N. S., and B. Daudert, 2016: Establishing best practices to improve usefulness and usability of web interfaces providing atmospheric data. Bull. Amer. Meteor. Soc., 97, 263274, https://doi.org/10.1175/BAMS-D-14-00121.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • O’Haire, C., and Coauthors, 2011: Appendix G: Strengths and limitations of stakeholder engagement methods. Engaging Stakeholders To Identify and Prioritize Future Research Needs, Methods Future Research Needs Reports, Agency for Healthcare Research and Quality, G-1–G-3.

  • Oliver, K., A. Kothari, and N. Mays, 2019: The dark side of coproduction: Do the costs outweigh the benefits for health research? Health Res. Policy Syst., 17, 33, https://doi.org/10.1186/s12961-019-0432-3.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Orbe, M. P., 1998: From the standpoint(s) of traditionally muted groups: Explicating a co-cultural communication theoretical model. Commun. Theory, 8, 126, https://doi.org/10.1111/j.1468-2885.1998.tb00209.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Phillipson, J., P. Lowe, A. Proctor, and E. Ruto, 2012: Stakeholder engagement and knowledge exchange in environmental research. J. Environ. Manage., 95, 5665, https://doi.org/10.1016/j.jenvman.2011.10.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pielke, R. A., Jr., 2007: The Honest Broker: Making Sense of Science in Policy and Politics. Cambridge University Press, 198 pp.

    • Crossref
    • Export Citation
  • Plattner, H., C. Meinel, and L. Leifer, Eds., 2011: Design Thinking: Understand–Improve–Apply. Springer-Verlag, 260 pp.

    • Crossref
    • Export Citation
  • Porter, K., and Coauthors, 2011: Overview of the ARkStorm scenario. U.S. Geological Survey Open File Rep. 2010-1312, 201 pp., https://pubs.usgs.gov/of/2010/1312/of2010-1312_text.pdf.

  • Prior, J., 2013: Community engagement to resolve climate adaptation conflicts. Climate Adaptation Futures. John Wiley and Sons, 167–176.

    • Crossref
    • Export Citation
  • Pulver, S., N. Ulibarri, K. Sobocinski, S. Alexander, M. Johnson, P. McCord, and J. Dell’Angelo, 2018: Frontiers in socio-environmental research: Components, connections, scale, and context. Ecol. Soc., 23, art23, https://doi.org/10.5751/ES-10280-230323.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Purdy, A. J., and Coauthors, 2019: Designing drought indicators. Bull. Amer. Meteor. Soc., 100, 23272341, https://doi.org/10.1175/BAMS-D-18-0146.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Richards, R., M. Sanó, A. Roiko, R. W. Carter, M. Bussey, J. Matthews, and T. F. Smith, 2013: Bayesian belief modeling of climate change impacts for informing regional adaptation options. Environ. Modell. Software, 44, 113121, https://doi.org/10.1016/j.envsoft.2012.07.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rittel, H. W. J., and M. M. Webber, 1973: Dilemmas in a general theory of planning. Policy Sci., 4, 155169, https://doi.org/10.1007/BF01405730.

  • Rose, D. C., and Coauthors, 2018: The major barriers to evidence-informed conservation policy and possible solutions. Conserv. Lett., 11, e12564, https://doi.org/10.1111/conl.12564.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rosendahl, D., R. McPherson, A. Wootten, E. Mullens, J. Blackband, and A. Bryan, 2019: Making sense of local climate projections. Eos, 100, https://doi.org/10.1029/2019EO136493.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ross, S. L., and Coauthors, 2013: The SAFRR (Science Application for Risk Reduction) Tsunami Scenario—Executive summary and introduction. California Geological Survey Special Rep. 229, 17 pp., https://pubs.usgs.gov/of/2013/1170/a/pdf/of2013-1170a.pdf.

  • Rouse, W., 2007: People and Organizations: Explorations of Human-Centered Design. John Wiley and Sons, 446 pp.

    • Crossref
    • Export Citation
  • Rowe, G., and L. J. Frewer, 2005: A typology of public engagement mechanisms. Sci. Technol. Hum. Values, 30, 251290, https://doi.org/10.1177/0162243904271724.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rumore, D., T. Schenk, and L. Susskind, 2016: Role-play simulations for climate change adaptation education and engagement. Nat. Climate Change, 6, 745750, https://doi.org/10.1038/nclimate3084.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rundstrom, R., 2009: Counter-mapping. International Encyclopedia of Human Geography, R. Kitchin and N. Thrift, Eds., Elsevier, 314–318.

    • Crossref
    • Export Citation
  • Saarman, E., and Coauthors, 2013: The role of science in supporting marine protected area network planning and design in California. Ocean Coastal Manage., 74, 4556, https://doi.org/10.1016/j.ocecoaman.2012.08.021.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Samson, E., P. E. Hirsch, S. C. F. Palmer, J. W. Behrens, T. Brodin, and J. M. J. Travis, 2017: Early engagement of stakeholders with individual-based modeling can inform research for improving invasive species management: The round goby as a case study. Front. Ecol. Evol., 5, 149, https://doi.org/10.3389/fevo.2017.00149.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sarewitz, D., and R. A. Pielke Jr., 2007: The neglected heart of science policy: Reconciling supply of and demand for science. Environ. Sci. Policy, 10, 516, https://doi.org/10.1016/j.envsci.2006.10.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sarkki, S., H. I. Heikkinen, and A. Löf, 2021: Reindeer herders as stakeholders or rights-holders? Introducing a social equity-based conceptualization relevant for Indigenous and local communities. Nordic Perspectives on the Responsible Development of the Arctic: Pathways to Action, D. C. Nord, Ed., Springer Polar Sciences Series, Springer International Publishing, 271–292.

    • Crossref
    • Export Citation
  • Scheufele, D. A., and S. Iyengar, 2017: The state of framing research: A call for new directions. The Oxford Handbook of Political Communication, Oxford University Press, https://doi.org/10.1093/oxfordhb/9780199793471.013.47.

    • Crossref
    • Export Citation
  • Schlesinger, W. H., 2010: Translational ecology. Science, 329, 609609, https://doi.org/10.1126/science.1195624.

  • Sharfstein, J. M., 2016: Banishing “stakeholders.” Milbank Quart., 94, 476479, https://doi.org/10.1111/1468-0009.12208.

  • Simonsen, J., and T. Robertson, Eds., 2013: Routledge International Handbook of Participatory Design. 1st ed. Routledge, 320 pp.

    • Crossref
    • Export Citation
  • Stedman, R. C., N. A. Connelly, T. A. Heberlein, D. J. Decker, and S. B. Allred, 2019: The end of the (research) world as we know it? Understanding and coping with declining response rates to mail surveys. Soc. Nat. Resour., 32, 11391154, https://doi.org/10.1080/08941920.2019.1587127.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Symstad, A. J., N. A. Fisichelli, B. W. Miller, E. Rowland, and G. W. Schuurman, 2017: Multiple methods for multiple futures: Integrating qualitative scenario planning and quantitative simulation modeling for natural resource decision making. Climate Risk Manage., 17, 7891, https://doi.org/10.1016/j.crm.2017.07.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Taylor, K. E., R. J. Stouffer, and G. A. Meehl, 2012: An overview of CMIP5 and the experiment design. Bull. Amer. Meteor. Soc., 93, 485498, https://doi.org/10.1175/BAMS-D-11-00094.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Turnhout, E., T. Metze, C. Wyborn, N. Klenk, and E. Louder, 2020: The politics of co-production: Participation, power, and transformation. Curr. Opin. Environ. Sustainability, 42, 1521, https://doi.org/10.1016/j.cosust.2019.11.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • U.S. Government, 1971: Alaska Native Claims Settlement. 43 U.S. Code, https://uscode.house.gov/view.xhtml?path=/prelim@title43/chapter33&edition=prelim.

  • Wall, T., A. M. Meadow, and A. Horangic, 2017: Developing evaluation indicators to improve the process of coproducing usable climate science. Wea. Climate Soc., 9, 95107, https://doi.org/10.1175/WCAS-D-16-0008.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • West, S., L. J. Haider, S. Stålhammar, and S. Woroniecki, 2020: A relational turn for sustainability science? Relational thinking, leverage points and transformations. Ecosyst. People, 16, 304325, https://doi.org/10.1080/26395916.2020.1814417.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Western Governors’ Association, 2004: Creating a drought early warning system for the 21st century. National Integrated Drought Information System Rep., 16 pp., https://www.drought.gov/sites/default/files/2020-06/200406_WGA_NIDIS_Report.pdf.

  • White, D. D., A. Wutich, K. L. Larson, P. Gober, T. Lant, and C. Senneville, 2010: Credibility, salience, and legitimacy of boundary objects: Water managers’ assessment of a simulation model in an immersive decision theater. Sci. Public Policy, 37, 219232, https://doi.org/10.3152/030234210X497726.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Whyte, K. P., 2014: Justice forward: Tribes, climate adaptation and responsibility. Climate Change and Indigenous Peoples in the United States: Impacts, Experiences and Actions, J. K. Maldonado, B. Colombi, and R. Pandya, Eds., Springer International Publishing, 9–22.

    • Crossref
    • Export Citation
  • Whyte, K. P., 2020: Too late for Indigenous climate justice: Ecological and relational tipping points. Wiley Interdiscip. Rev.: Climate Change, 11, e603, https://doi.org/10.1002/wcc.603.

    • Search Google Scholar
    • Export Citation
  • Wilkins, E. J., and Coauthors, 2019: Rural-urban differences in hunting and birdwatching attitudes and participation intent. Hum. Dimens. Wildl., 24, 530547, https://doi.org/10.1080/10871209.2019.1661046.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilson, M. A., and R. B. Howarth, 2002: Discourse-based valuation of ecosystem services: Establishing fair outcomes through group deliberation. Ecol. Econ., 41, 431443, https://doi.org/10.1016/S0921-8009(02)00092-7.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wong-Parodi, G., K. J. Mach, K. Jagannathan, and K. D. Sjostrom, 2020: Insights for developing effective decision support tools for environmental sustainability. Curr. Opin. Environ. Sustainability, 42, 5259, https://doi.org/10.1016/j.cosust.2020.01.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • World Meteorological Organization, 2014: Implementation plan of the Global Framework for Climate Services. WMO Doc., 81 pp., https://gfcs.wmo.int/sites/default/files/implementation-plan//GFCS-IMPLEMENTATION-PLAN-FINAL-14211_en.pdf.

  • Wright, P., and J. McCarthy, 2010: Experience-centered design: Designers, users, and communities in dialogue. Synthesis Lectures on Human-Centered Informatics, Vol. 3, Morgan and Claypool, 123 pp., https://doi.org/10.2200/S00229ED1V01Y201003HCI009.

    • Crossref
    • Export Citation
  • Wyborn, C., A. Datta, J. Montana, M. Ryan, P. Leith, B. Chaffin, C. Miller, and L. van Kerkhoff, 2019: Co-producing sustainability: Reordering the governance of science, policy, and practice. Annu. Rev. Environ. Resour., 44, 319–346, https://doi.org/10.1146/annurev-environ-101718-033103.

    • Crossref
    • Export Citation
  • View in gallery
    Fig. 1.

    Adapted wheel of participation: outer text organizes strategies for the creation of actionable science described in this article by their corresponding approach from the wheel of participation; inner text is from Davidson (1998) and demonstrates the continuous spectrum of community engagement across each approach. Inform is characterized by minimal interaction with stakeholders, focused on one-way communication of quality information. Consult includes collecting targeted input from stakeholders, whereas participate is distinguished by active partnership with stakeholders that allows them limited decision-making capabilities. The empower segment seeks to delegate or entrust stakeholders with significant decision-making power such that they become coequal team members.

All Time Past Year Past 30 Days
Abstract Views 1738 1116 0
Full Text Views 670 534 114
PDF Downloads 549 416 77

Engaging with Stakeholders to Produce Actionable Science: A Framework and Guidance

Aparna Bamzai-DodsonaU.S. Geological Survey North Central Climate Adaptation Science Center, Fort Collins, Colorado
bUniversity of Oklahoma Department of Geography and Environmental Sustainability, Norman, Oklahoma

Search for other papers by Aparna Bamzai-Dodson in
Current site
Google Scholar
PubMed
Close
,
Amanda E. CravenscU.S. Geological Survey Fort Collins Science Center, Fort Collins, Colorado

Search for other papers by Amanda E. Cravens in
Current site
Google Scholar
PubMed
Close
,
Alisa A. WadeaU.S. Geological Survey North Central Climate Adaptation Science Center, Fort Collins, Colorado

Search for other papers by Alisa A. Wade in
Current site
Google Scholar
PubMed
Close
, and
Renee A. McPhersonaU.S. Geological Survey North Central Climate Adaptation Science Center, Fort Collins, Colorado
dU.S. Geological Survey South Central Climate Adaptation Science Center, Norman, Oklahoma

Search for other papers by Renee A. McPherson in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

Natural and cultural resource managers are increasingly working with the scientific community to create information on how best to adapt to the current and projected impacts of climate change. Engaging with these managers is a strategy that researchers can use to ensure that scientific outputs and findings are actionable (or useful and usable). In this article, the authors adapt Davidson’s wheel of participation to characterize and describe common stakeholder engagement strategies across the spectrum of inform, consult, participate, and empower. This adapted framework provides researchers with a standardized vocabulary for describing their engagement approach, guidance on how to select an approach, methods for implementing engagement, and potential barriers to overcome. While there is often no one “best” approach to engaging with stakeholders, researchers can use the objectives of their project and the decision context in which their stakeholders operate to guide their selection. Researchers can also revisit this framework over time as their project objectives shift and their stakeholder relationships evolve.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Aparna Bamzai-Dodson, abamzai@usgs.gov

Abstract

Natural and cultural resource managers are increasingly working with the scientific community to create information on how best to adapt to the current and projected impacts of climate change. Engaging with these managers is a strategy that researchers can use to ensure that scientific outputs and findings are actionable (or useful and usable). In this article, the authors adapt Davidson’s wheel of participation to characterize and describe common stakeholder engagement strategies across the spectrum of inform, consult, participate, and empower. This adapted framework provides researchers with a standardized vocabulary for describing their engagement approach, guidance on how to select an approach, methods for implementing engagement, and potential barriers to overcome. While there is often no one “best” approach to engaging with stakeholders, researchers can use the objectives of their project and the decision context in which their stakeholders operate to guide their selection. Researchers can also revisit this framework over time as their project objectives shift and their stakeholder relationships evolve.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Aparna Bamzai-Dodson, abamzai@usgs.gov

1. Introduction

Climate change is impacting natural and cultural resources throughout the United States and globally (IPCC 2018). Resource managers and decision-makers are more frequently in need of science-informed tools and guidance on how best to adapt to current and projected future conditions (Filho 2015). However, adaptation to climate change is a wicked problem (Rittel and Webber 1973) with many complex tradeoffs and potential barriers (Bierbaum et al. 2013). The creation of climate adaptation science cannot be separated from the application of such science, and engagement between researchers and science end users is critical to the successful integration of science into actions and policy (Sarewitz and Pielke 2007; McNie 2007; Lubchenco 2017). This engagement also needs to include the integration of local communities and knowledge because people and places have unique characteristics, and scientific products must respond to those unique qualities to be perceived as credible and to be accepted (Laursen et al. 2018).

The term stakeholder is often used to mean an ambiguous and amorphous group of all interested parties (Sharfstein 2016). In this article, we define stakeholders as the end users of the outputs and findings of a scientific process. Specifically, we focus on resource managers (e.g., state wildlife agency), decision-makers (e.g., conservation land trust), and members of the public use project outputs to take action (e.g., private landowners). End users may also include Indigenous peoples and communities, whose rights to sovereignty and self-governance are governed by a complex landscape of federal, state, and local treaties and laws (e.g., U.S. Government 1971). We recognize the colonial legacy of the term stakeholder (Barry and Thompson-Fawcett 2020) and, following Sarkki et al. (2021), suggest the term rights holder in these contexts instead. For brevity, we continue to use stakeholder throughout this article as an umbrella term for all end-user communities (rights holders and nonrights holders). Engaging these various stakeholders is one important means to ensuring that scientific outputs and findings are actionable—defined generally as useful and usable—with a growing body of literature on how to do this effectively (Arnott et al. 2020a).

At its core, information and data must be seen as salient (relevant to the decision choices), credible (scientifically plausible and technically accurate), and legitimate (created through a fair and unbiased process) to be accepted as actionable (Cash et al. 2003). Although scientists may perceive their products as authoritative, stakeholders tend to approach new information and tools from a skeptical point of view, resulting in a need for salience, credibility, and legitimacy to be negotiated through extended dialogue and trust building (Cravens and Ardoin 2016; White et al. 2010). For example, the effective provision of climate information to stakeholders has been shown to require an understanding of the decision context and information usability (Dilling et al. 2015), an incorporation of managers’ perceptions of risk (Kirchhoff et al. 2013), and an assessment of research impact tied to understanding the audience and their need for evidence (Fisher et al. 2020). Engaging stakeholders in science creation can result in products tailored to appropriate spatial scales with focus on variables relevant to their decision contexts, a matter of key importance to resource managers. Increases in globally averaged temperature do not begin to scratch the surface of the climate change story, and managers instead seek information specific to the resources under their charge (Clifford et al. 2020). Additionally, the successful creation of usable science needs institutional buy-in and support for interactions on behalf of science users, producers, and analysts (Dilling and Lemos 2011).

Although there is a long history of calls for actionable science in other fields of public policy (e.g., Bush 1945; Lynn 1978; National Research Council 2012), similar calls relevant to climate adaptation have increased only recently (e.g., National Research Council 2009; Lemos et al. 2012). In a Federal Advisory Committee report on natural resource and climate adaptation science submitted to the U.S. Secretary of the Interior in 2015, a recommendation emphasized the need for continued provision of research that “provides data, analyses, projections, or tools that can support decisions regarding the management of the risks and impacts of climate change” (Advisory Committee on Climate Change and Natural Resource Science 2015; Beier et al. 2017). Implementation of this recommendation has included a renewed commitment to the use of regional climate collaboratives working to support management action (Averyt et al. 2017), the development of specific informal partnerships to support the creation of actionable science (Bisbal 2019), and the use of formal stakeholder advisory committees and science advisory panels (DeCrappeo et al. 2017).

Specific calls for more landscape-scale science for the management of federal and working lands also emphasize the need for engagement with managers. One oft-cited approach to achieve actionability is the use of coproduction, which generally refers to researchers and stakeholders working on the same team to produce science useful for decision-making (Carter et al. 2020; Naugle et al. 2020). Coproduction is an imprecise term, however, used in the literature to describe a range of different approaches to working with stakeholders (Mach et al. 2020; Bremer and Meisch 2017; Norström et al. 2020; Wyborn et al. 2019). Lack of clarity around the language used by researchers and practitioners obscures the full range of approaches available for stakeholder engagement and limits our ability to intercompare implementation of engagement methods and to identify lessons learned, effective practices, and pitfalls to avoid.

Conceptual frameworks can be a useful way to organize complex concepts with a high degree of explanatory power, provide a mental model of how a diverse set of literature works together (Collins and Stockton 2018; Imenda 2014), and be used as a mechanism to distill theoretical concepts for practical application (Magliocca et al. 2018; Pulver et al. 2018). In this article, we adapt an existing conceptual framework to better characterize common stakeholder engagement approaches used to conduct actionable climate adaptation research. The framework presents a broader spectrum of engagement, clarifying the position of coproduction and other ambiguous terminology. The foundational basis for the chosen framework relies on Davidson (1998), selected because it is grounded in stakeholder engagement for community planning and prioritizes important concepts for climate adaptation research (e.g., role of stakeholder). While Davidson (1998) is a continuous spectrum of engagement with distinct endpoints, its nonhierarchical wheel structure promotes a deliberate, thoughtful choice of an approach to engage stakeholders instead of simply prioritizing the highest intensity of engagement (unlike linear hierarchical structures, e.g., a ladder in Arnstein 1969). Other similar frameworks exist but are not highlighted here because they prioritize concepts that are less relevant for scientific research or beyond the scope of this article; for example, citizen participation and empowerment (Arnstein 1969), the quality of resulting stakeholder decisions (Beierle 2002), or the direction and type of flows of information between researcher and stakeholder (Rowe and Frewer 2005). Additionally, while other frameworks describe engagement from an academic viewpoint, they often do not prioritize the practical application for researchers charged with conducting engagement or are focused on a narrow form of engagement (e.g., communications methods in Bojovic et al. 2021).

Our adapted framework (Fig. 1) helps researchers navigate and select the approach that best fits their objectives for a given project, thereby providing a standardized vocabulary for describing stakeholder engagement and filling a critical gap in the literature about practical application. The framework consists of four segments: inform, consult, participate, and empower. The inform segment is characterized by minimal interaction with stakeholders, focused on one-way communication of quality information, such as press releases or newsletters, from researcher to stakeholder. The consult segment includes collecting targeted input from stakeholders through avenues such as surveys or public meetings, while the participate segment is distinguished by active partnership with stakeholders that allows them limited decision-making capabilities (e.g., through a formal or informal advisory committee). The empower segment seeks to delegate or entrust stakeholders with significant decision-making power such that they become coequal team members. The most important part of utilizing our adapted framework is understanding the context of a specific project and ensuring that investments are made in the most appropriate kind of engagement to meet the stakeholders’ science needs (Lemos et al. 2018).

Fig. 1.
Fig. 1.

Adapted wheel of participation: outer text organizes strategies for the creation of actionable science described in this article by their corresponding approach from the wheel of participation; inner text is from Davidson (1998) and demonstrates the continuous spectrum of community engagement across each approach. Inform is characterized by minimal interaction with stakeholders, focused on one-way communication of quality information. Consult includes collecting targeted input from stakeholders, whereas participate is distinguished by active partnership with stakeholders that allows them limited decision-making capabilities. The empower segment seeks to delegate or entrust stakeholders with significant decision-making power such that they become coequal team members.

Citation: Weather, Climate, and Society 13, 4; 10.1175/WCAS-D-21-0046.1

In this article, we outline our adapted framework, its four approaches, and examples of their appropriate use in section 2. In section 3, we provide guidance to researchers, in particular, on how they can select the most appropriate way to work with stakeholders based on their goals. We conclude in section 4 by urging scientists to work with end users of information and products to ensure current knowledge is implemented, from national to local decisions.

2. Engagement for actionable science in the real world

As climate adaptation researchers have aimed to engage stakeholders, time and financial stresses tend to pressure researchers into adopting previously used engagement strategies without deep thought about what strategy might be best suited to a particular outcome. Yet, building a habit of assessing which strategy to use on the front end can result in better outcomes. For each of the four approaches in this framework, we discuss associated science objectives, engagement methods, and barriers to implementation. Each engagement method has strengths, weaknesses, and resource constraints that influence what might be the best fit for a project. While many of these methods might span across approaches, we have tried to seat each method within the approach in which those activities commonly occur. We also include a few examples to illustrate successful uses of each approach. Table 1 provides a summary of this information.

Table 1.

Summary description of each approach from the adapted wheel (inform, consult, participate, and empower) by intended project objective(s), potential engagement methods, common barriers, and case examples.

Table 1.

A researcher’s choice of approach might be dictated by resource constraints or other barriers (Rose et al. 2018) and require balancing the transaction costs of greater stakeholder interaction against expected gains in the quality and quantity of outputs and outcomes (Lemos et al. 2019). Although the level of technical complexity or degree of interdisciplinarity are important elements of the project context, they do not tell the whole story and may in fact prove to be misleading if they are the only elements considered. For example, a project focused on running a complex physical model may need extensive stakeholder engagement to determine the appropriate parameterization for key variables (e.g., Morisette et al. 2017). On the other hand, for a different type of complex study, such as a social–ecological systems study, limited stakeholder engagement might be possible when the end-user community and decision context can be clearly and narrowly defined (e.g., Beeton et al. 2019). Additionally, a researcher may need to revisit this framework over the lifetime of a project as existing stakeholder relationships are strengthened or new audiences are engaged (Klenk et al. 2017).

a. Inform

As outlined in Fig. 1, inform is characterized by the one-way communication of decision-relevant project results. It differs from consult in the direction of information exchange; inform transfers information from researcher to stakeholder, whereas consult moves it from stakeholder to researcher. Inform is the most traditional and common approach of providing scientific information from researchers to users. Researchers in the basic science community often select inform as their approach by default because there has been a tendency to consider any interactions with the users of the research as a potential source of bias.

Some have criticized this approach as insufficient, terming it as the “loading dock approach”; Cash et al. (2006) describe it as “you take it out there, and you leave it on the loading dock and you say, there it is. And then you walk away and go back inside.” However, we argue that inform can be a viable and proper approach for creating actionable science when chosen strategically. Appropriate uses of the inform approach intend to transfer knowledge rapidly and result in some science-based action. These uses either create and disseminate datasets applicable across a broad geographic scale (Arguez et al. 2012), summarize a large body of complex, novel, or synthetic science (Guido et al. 2013), or require scientific legitimacy that stands above the political or policy context (Pielke 2007). The International Network for Government Science Advice, an affiliated body of the International Science Council, studies issues of credibility and legitimacy of evidence used to inform policy conversations around topics that are highly contested, such as climate change mitigation, genetically modified crops, and biodiversity conservation, and has found that such research must be seen as unimpeachable and not unduly influenced by any one side of a policy debate (Gluckman and Wilsdon 2016).

An effective use of inform is exemplified by “tornado politics,” as described by Pielke (2007), whereby an information provider does not have time to iterate on scientific products prior to one-way delivery to an audience (e.g., meteorologists issuing a tornado warning to the public) to meet a desired outcome (e.g., saving lives). In some cases, inform may be selected as a training approach when there is a need for clear one-way information transfer to participants, such as teaching storm spotters how to recognize tornado development (Doswell et al. 1999). Another appropriate use of inform is when the potential user base is so large and diverse that stakeholder interactions would be prohibitively complex, costly, and time-consuming. Examples of actionable scientific outputs for broad audiences include the data and graphics published for researchers, managers, and the public at large for the Coupled Model Intercomparison Project (CMIP; Eyring et al. 2016; Taylor et al. 2012) or the synthesis and interpretation of climate information in La Niña Drought Tracker during the drought of 2010/11 in Arizona and New Mexico (Guido et al. 2013). In these cases, the initial knowledge delivery by the scientific entity was followed with engagement with other scientists (e.g., broadcast meteorologists) using other approaches to increase saliency, yet the original transfer of science for public action was a one-way knowledge transfer.

Examples of methods for the inform approach include scientific publications, as well as webinars, seminars, presentations, white papers, briefings, brochures, and fact sheets. In the research community, inform is dominated by peer-reviewed journal articles, which energize their scholarly audience (i.e., their stakeholders) to consider new theories, data, models, or methods in their own research (Fyfe et al. 2017). Science published in peer-reviewed journals or other jargon-rich or highly technical formats is not intended to be and, as a result, is often not necessarily useful for most decision-makers (Hassol 2008); hence, many government agencies and independent organizations summarize specific subject matter using language appropriate for broader audiences. In all of these inform examples, information that is intended to be actionable is translated without extensive stakeholder engagement; the information is intended or assumed to be useful without the need for two-way engagement.

Barriers to informing stakeholders without consultation, participation, or empowerment tend to preclude the effective use of inform except in specific circumstances. For example, scientific articles that appear in professional journals, magazines, or blogs may be hidden behind paywalls, limiting their audience (e.g., Archie et al. 2014). Even with open-access publications or other more targeted information, such as fact sheets, stakeholders may not know where to find the information or have limited time to read it. For example, Doemeland and Trevino (2014) found that less than 15% of the World Bank’s policy reports were downloaded at least 250 times, yet over 2 times that many were never downloaded and over 85% were never cited. In addition, without intervention in the process of delivering science to stakeholders, there generally is a mismatch between the climate information product and the user’s decision content that requires some negotiation and engagement (e.g., Briley et al. 2015).

b. Consult

As opposed to inform, consult reverses the flow of one-way communication—from stakeholder to researcher. Consult seeks to access stakeholder input, on preidentified options, at discrete waypoints in the research process when it is not necessary to deeply understand the decision-making context of the science nor experiences of those being consulted, such as required for participate. Consult is appropriate for engagement when input is only needed on certain specifics of the broader research project, when addressing a topic already identified as a priority by stakeholders, or to ensure transparency and input when stakeholder numbers are large and individual stakeholder engagement is not tractable. Consulting with stakeholders often is conducted during one of three distinct phases of a research project: needs assessment, design and implementation, and products and outputs. Traditional climate services rely on these multiple, discrete stakeholder engagements to identify priority topics and delivery mechanisms for climate information provision in response to user needs (World Meteorological Organization 2014).

Needs assessments seek to identify key science or support that would most benefit the end user. There is evidence that organizations are more apt to use information that they requested (McNie 2013). This phase may be the most open-ended for consultation, whereby stakeholders may inform practice-relevant topics for research (e.g., Land et al. 2017). In this regard, consult can blur into participate, as a diverse stakeholder group can fundamentally shift the research agenda. Many examples of needs assessments focus on asking stakeholders to rank a preexisting list of “knowledge gaps” assembled from the literature (e.g., Human and Davies 2010). However, often these knowledge gaps themselves may have been identified through prior engagement efforts. For example, Crausbay et al. (2020) incorporated stakeholder input in a modified horizon-scanning exercise to identify gaps in the state of knowledge about ecological drought that could improve management response to events if filled. Related to a needs assessment, but more specific, is stakeholder engagement for problem framing. Phillipson et al. (2012) found that this type of engagement resulted in the greatest shift in research relevance to the stakeholders themselves.

Consult can be applied during the research design and implementation phase to guide specific decisions on research approaches and methods or to provide specific information inputs. Data collection or provision is a form of consult conducted early in research implementation. For example, crowdsourcing data (a subset of citizen science) can promote topical understanding and speed the dissemination and management uptake of findings, while also providing relatively inexpensive data (Lee et al. 2020; McKinley et al. 2017). Stakeholders also can provide information in the form of their own expertise. For example, in science syntheses, stakeholders may contribute to or review sections, such as in the National Climate Assessment (Cloyd et al. 2016). Expert elicitation processes effectively serve as opportunities to consult stakeholders when gathering data, such as the Delphi method (Naskar et al. 2018). Empirical social science also treats stakeholders as data sources, to better understand current preferences or inform how research findings can best inform management options (e.g., Wilkins et al. 2019; Klemm and McPherson 2018). Another common form of consult is relying on stakeholder expertise to parameterize models, such as the use of Bayesian belief networks (Richards et al. 2013) or other elicitation methods (e.g., see engagement for individual-based models in Samson et al. 2017).

At the end of a research project, consult can use stakeholder input to validate a project or estimate uncertainty in the findings (e.g., Johnson and Gillingham 2004). Alternatively, engagement can identify which outputs are most useful and how to disseminate them. For example, decision-support tool development can greatly benefit from consult to test the usability of the tool (Oakley and Daudert 2016; Wong-Parodi et al. 2020). End-of-project consults can help ensure that research products are useful, such as helping to identify clear visualizations or review accessible outputs such as fact sheets and online story maps. A recent project used eye-tracking data from website visitors to better understand how natural resource managers look for information and how layouts can make research findings most accessible (Maudlin et al. 2020).

Consult includes a number of approaches for discrete interactions. Examples that seek breadth of input include surveys and opinion polls or voting on preestablished options. For more in-depth elicitation, consult applies focus groups, town halls, interviews, or expert elicitation. Workshops or “colliders,” where researchers meet intensively with resource managers about a given modeling effort, can also be considered. Typically, citizen science projects (or other means of engaging stakeholders in data provision) define the frequency, thresholds, or occurrences for individuals to gather and send data to researchers. O’Haire et al. (2011) provide a useful summary of the strengths and limitations of a number of these engagement methods.

A key barrier to the consult process follows from the disconnect between a stakeholder’s input and the research results. Consult is founded on an assumed “promise” that stakeholder input will be used, and, given the often one-off nature of consult, stakeholders may feel exploited if there is no clear pathway between their time and input and the culmination of—and self-benefit from—the research project (Friesen et al. 2017). Researchers must resist what Arnstein (1969) calls “the empty ritual of participation,” where input is collected and claimed to be considered but the status quo persists, particularly in cases where power imbalances exist across groups of stakeholders. This problem may be compounded as stakeholder engagement requests increase, with a concomitant decline in survey responses in natural resources social sciences (Stedman et al. 2019) and growing reports of “stakeholder fatigue” (Bracken et al. 2015). To resolve this issue, it is imperative that researchers follow up with stakeholders, reporting back in detail how their input was used.

Another common barrier is a mismatch in the timing of the stakeholder engagement and the research input need. Consult elicits input that represents a single moment in time and that may not match either the researcher’s process nor the immediacy of the stakeholder’s need for the information and/or outputs produced by the project. Further, in the case of needs assessments, it may be difficult to elicit useful information from busy stakeholders, who often are too burdened with projects to think long-term and proactively about their needs for research outputs that would not be available for multiple years. Seeking to elicit upcoming key management decisions or problems rather than explicitly asking “what science would benefit your work?” may assist researchers in overcoming this obstacle (Beier et al. 2017).

A shortcoming to consult is that stakeholder input does not necessarily result in clear direction for the researcher. Seeking input from a large array of diverse stakeholders is recommended, but these differences can result in divergent input. Certain approaches, such as the Delphi method, seek to bring a smaller group to consensus, but other forms of engagement, such as surveys, offer no such promise. Further, the input obtained can vary greatly depending on who or what is asked. For example, Archie et al. (2014) found that the level of knowledge about climate change greatly affected survey respondents’ reporting of climate adaptation efforts. In general, there is substantial evidence that how an issue is framed (e.g., presented in a positive or negative light) significantly affects how stakeholders respond (Scheufele and Iyengar 2017), although there is also research suggesting ways forward to better elicit input in response to complex issues (e.g., Jansen et al. 2019).

c. Participate

Participate encompasses a range of engagement methods whereby scientists and stakeholders share decision-making. Unlike inform and consult, these approaches represent sustained two-way interaction (although the length of time and type of interaction varies). Stakeholders do not play the leadership role of coequal project partner or team member, however, in the manner of empower. Participate supports science projects that seek to provide place-based, contextualized, or customized information. Participate generates information needed to support planning, policy making, or other kinds of agency decision-making where there is a need to understand stakeholders’ perceptions or experiences in greater depth than might be captured by a consult approach. Another common objective for participate is when developing a product such as a computerized decision support tool or a guidance document that needs sustained input from stakeholders.

Participate encompasses approaches that engage deeply enough with diverse stakeholders to gain significant understanding of their point of view, while not fully integrating those stakeholders into the decision-making process. One way to enact participate approaches is through iterative engagement with (either the same or different groups of) stakeholders at defined points in a project’s life cycle. Particularly common in the development of tangible products such as decision support tools, this approach plans multiple touch points during a project to engage with stakeholders to ensure that concerns and aspirations are mutually understood. For instance, users might provide input in the scoping phase, then provide input to a series of prototype versions as the tool is developed, have an opportunity to test the final design to improve the usability of a tool, and finally engage during training sessions to learn how to use the final product (Leitch et al. 2019). Despite the substantial sustained engagement over what might be a period of months or years, the stakeholders are not fully integrated into the project team and do not participate in the work that happens between these discrete touch points. For instance, they are not included in project management or planning, the evaluation of alternatives, identifying preferred options, or the technical aspects of tool building.

This iterative process of engaging with users multiple times over the course of the project described in Leitch et al. (2019) generally follows a framework variously called human- or user-centered design (Boy 2017; Gasson 2003; Rouse 2007; Wright and McCarthy 2010) or design thinking (Plattner et al. 2011). Such a design framework represents a structured method for developing products or experiences and focuses on developing a deep understanding of the intended user’s needs and point of view from the start of a project. Christel et al. (2018) described a climate services project that used a design-based approach to create a product to visualize seasonal wind speed predictions for decision-makers in wind energy. The team found that a human-centered design framework helped sustain cross-disciplinary collaboration between the intended users in the wind energy industry, the research team, and the visualization experts. Others have similarly argued that user-centered design improved the development of drought indicators by providing a systematic method to understand and incorporate the needs of specific types of drought-information users (Purdy et al. 2019).

Other participate approaches emphasize the value of incorporating diverse partnerships and points of view into a project. Translational ecology is an emerging strategy arising from a recognition among ecologists of the need for actionable ecological knowledge (Schlesinger 2010). It is defined as “intentional processes in which ecologists, stakeholders, and decision-makers work collaboratively to develop ecological research via joint consideration of the sociological, ecological, and political contexts of an environmental problem that ideally results in improved environment-related decision making” (Enquist et al. 2017, p. 542). Beaury et al. (2020) used translational ecology as their strategy for engaging with invasive species managers to understand their needs and barriers and for developing an agenda for continued interactions between managers and researchers. This iterative, two-way engagement strategy will inform future research directions and ensure that scientific findings and outputs are designed to be directly incorporated into on-the-ground invasive species management.

Boundary spanning is another broad participatory strategy that recognizes that many partners might be needed to address issues across the science–policy interface and that provides guidance for the deliberate engagement of individuals and organizations (Bednarek et al. 2018; Goodrich et al. 2020). One “boundary organization” can work as a translator with both science producers and users, or two or more boundary organizations can join together in a “boundary chain” to collaborate, share costs, and pool resources (Kalafatis et al. 2015; Kirchhoff et al. 2015); different types of goals or projects will be easier for organizations at certain points in the chain to accomplish. We have placed boundary spanning within participate as the place where this activity is most likely to occur but recognize that boundary individuals and organizations often utilize approaches and methods across the entire spectrum of the wheel.

An example of a boundary organization is the National Integrated Drought Information System (NIDIS). In response to a call from the Western Governors’ Association for integrated physical, hydrological, environmental, and socioeconomic decision-ready drought data and tools (Western Governors’ Association 2004), the National Oceanic and Atmospheric Administration (NOAA) created NIDIS. Recognizing the existing expertise and breadth of ongoing drought science, NIDIS brings together federal, state, local, and Tribal agencies and taps into existing data and information networks to identify management needs, synthesize science, and create decision-support tools. NIDIS also leverages partnerships with other boundary organizations, such as the National Drought Mitigation Center and the NOAA Regional Integrated Sciences and Assessments Teams, to facilitate and synthesize dialogue from national and regional working groups at a variety of scales (NIDIS 2016). Spanning a broad pool of science producers, users, and other boundary individuals and organizations allows NIDIS to efficiently and dynamically call on expertise, capabilities, and resources to address questions and challenges as they arise.

Engagement methods for participate overlap with consult and empower but are distinguished from the first by the depth of the engagement and from the second by the way that the research team retains control of decision-making throughout the engagement process. Methods include surveys and semistructured interviews, though with more interaction than consult. For instance, the same survey might be administered at multiple points in time, or a research team might do a set of interviews and then have a follow-up workshop to share results and get feedback about how the team is interpreting the findings. Other engagement methods include repeated focus groups or listening sessions, whose potential for social learning can support the need for diverse perspectives of participate methods (Gerlak et al. 2020), or advisory boards, which provide a formal means for stakeholders to continually provide input to a project. For example, during the State of California’s process of siting new marine protected areas under the auspices of the Marine Life Protected Act Initiative, the Science Advisory Team provided ongoing guidance about how to evaluate the scientific merit of proposals (Saarman et al. 2013). Projects using human-centered design approaches also might draw inspiration for engagement methods from the field of design; for instance, designers make extensive use of participant observation (IDEO 2015).

Simulations, scenarios, and role-playing games can educate stakeholders to understand how to use scientific data and information, build collaborative decision-making capacity around wicked problems, and create local or regional communities of practice for sustained social learning (Rosendahl et al. 2019; Rumore et al. 2016). For instance, drought simulations have aided local, state, and national drought preparedness and planning efforts (Bathke et al. 2019). The U.S. Geological Survey’s Science Application for Risk Reduction (SAFRR) Project runs large-scale scenarios (i.e., 200+ person) to support communities in preparing for natural disasters (Porter et al. 2011; Ross et al. 2013). Demonstrating the complex consequences of hazards, such as earthquakes, SAFRR scenarios allow stakeholders to participate in hazard response and recovery exercises designed to connect scientific information and community needs around natural disasters.

There are three common barriers to using participate approaches. One, this approach requires the time, resources, and expertise for researchers to run such an engagement effort because it relies on deep stakeholder engagement sustained over the course of a project (Kemp et al. 2015). Two, it requires that the chosen stakeholders similarly have the time, capacity, and motivation to (or expected benefit from) participation (Bracken et al. 2015). Three, at the beginning of a project, it may not be possible to precisely define when, how, or with whom engagement will happen; hence, sustained engagement, especially across multiple points in time, requires flexibility and adaptability (LaChapelle et al. 2003).

d. Empower

As in participate, approaches within empower require sustained, two-way interaction between stakeholders and researchers. In empower approaches, however, stakeholders are included as coequal team members through every step of the project, including but not limited to defining the problem statement, designing research questions, selecting methods, collecting and interpreting data, and developing output products. Stakeholders are entrusted to make project decisions; not just to participate in or be consulted on project decisions. To build and sustain the partnerships needed for empower approaches, researchers and stakeholders must dedicate significant resources (e.g., time and money) within the project toward engagement. When viewed from outside of the project team, participate approaches with highly equitable stakeholder inclusion may appear to be indistinguishable from empower approaches.

Empower approaches can be particularly effective when projects take place within contexts where the voices of particular stakeholders or rights holders have been muted or disenfranchised in decision-making processes (Ardener 2005; Orbe 1998). Empowerment of stakeholders can result in fundamental shifts in power and governance by placing equal importance on information from diverse knowledge systems (Wyborn et al. 2019) and by allowing for holistic representations of human–nature connectedness informed by the ethics of care (West et al. 2020). As a result, empower can support the creation of place-based science where local knowledge is critical and the implementation of inclusive science-informed plans, decisions, or actions.

Coproduction is the term often used to describe the process of engaging with and empowering stakeholders in the process of creating actionable science (Meadow et al. 2015). Hegger et al. (2012, 2014) instead suggest calling this approach joint knowledge production, as the term coproduction is used by science and technology studies scholars to refer to mutually shaping interaction between the production of knowledge and the production of social order (Jasanoff 2004). The foundational principles of joint knowledge production include that the scientific objectives must be grounded in the decision context, that iterative discussions between all members throughout the project are essential, that management risk and scientific uncertainty must both be clearly defined and communicated, and that the process of engagement is just as important as the outputs and must be evaluated (Beier et al. 2017). Scientists and nonscientists work together to include and give equal weight to local knowledge and qualitative information in the sum of the evidence considered by the project team (e.g., Diver 2017). Joint knowledge production that ignores power differentials can reinforce power imbalances and impede wider societal progress (Turnhout et al. 2020), and the removal of local knowledge from the context of its production can result in outputs and outcomes that are ineffective for local governance and practices (Klenk et al. 2017). We have situated joint knowledge production within empower but recognize that, in practice, researchers often utilize joint knowledge production principles in forms that span other segments of the wheel [e.g., contractual, consultative, and collaborative in Meadow et al. (2015)].

Approaches similar to joint knowledge production include consensus building, action research, and participatory design. Consensus building entails iterative, usually facilitated, conversations with stakeholders to determine outcomes that are at least acceptable to all involved. Consensus building not only considers the range of stakeholder views on an issue but explicitly recognizes power differentials between stakeholders and focuses on distributing power in a just and equitable manner (Prior 2013). Action research acknowledges that blending theory and practice to create solutions to real-world problems requires a deep understanding of the values of the individuals and communities involved. Action research also recognizes that researchers and stakeholders are continually impacted by their interactions and can learn and evolve over the course of the project (Brydon-Miller et al. 2003; Bradbury et al. 2019). Participatory design emerged from the production of informational technologies as a process by which users and designers can experience mutual learning and codesign products (Simonsen and Robertson 2013). Users and designers participate in frequent, iterative conversations such that redesign elements can be continually and critically examined for usefulness and usability (Kruk et al. 2018). Important to all these approaches is active stakeholder participation characterized by inclusion and empowerment (Few et al. 2007).

Specific tools support empowering stakeholder voices by shifting the balance of power and purposefully including end-user knowledge and perspectives in the production of scientific research. Countermapping or participatory mapping allows local stakeholders to develop visual representations of the spatial relationships between physical locations and boundaries and can be effective for establishing Indigenous rights and sovereignty (Rundstrom 2009). Discourse-based valuation diverges from conventional valuation of public goods by examining convergent values across a group of stakeholders through a well-ordered deliberative forum as opposed to aggregating individual preferences. This method provides a nonmarket based valuation for public goods that can enhance social equity (Wilson and Howarth 2002). Scenario planning provides a structured format through which researchers and managers can explore quantitative and qualitative “what if” scenarios for the future together. Participants are able to consider multiple, layered dimensions of uncertainty and risk and are encouraged to consider innovative or outside-the-box resource management next steps (Symstad et al. 2017).

In addition to this sample of high engagement methods and techniques, empower projects can utilize those from the inform, consult, and participate approaches at various points in the engagement process, in combination with each other, and at increased frequency. Projects have flexibility in selecting which of these tools to use in which circumstances and thus need to carefully consider their individual strengths and weaknesses. While there is often no single right engagement method to use, selecting the wrong method can widen the gulf between researchers and stakeholders (Lynam et al. 2007).

Barriers for empower approaches include many of those described for inform, consult, and participate. Additionally, empower can be particularly hampered by the unique cultural and institutional barriers faced by both stakeholders and researchers (Jarvis et al. 2020). For example, Archie (2014) found that elected officials were less likely than career officials to consider climate change a priority for planning in Rocky Mountain communities. Without effective leadership, shifts in political will or agency priorities can redirect the allocation of resources and leave stakeholders with limited scales of influence (Moser and Ekstrom 2010). Formal implementation of environmental legislation also can impede adaptation planning if it is perceived as prescriptive (e.g., Endangered Species Act, which can prevent action) and not process oriented (e.g., National Environmental Policy Act, which informs but does not prevent action; Jantarasami et al. 2010).

Researchers also face a number of barriers to engaging in empower approaches. Unpublished research by the second author of this article (A. Cravens) noted that funding models, data access, job descriptions that omitted stakeholder engagement, and similar constraints prevented producers of drought information from engaging with stakeholders in ways they wished, even though the majority recognized the importance of stakeholder engagement to effective tool development. Academic faculty, especially those who are pretenure, may not be evaluated favorably unless they produce peer-reviewed publications. In such cases, researchers also must consider the costs of investing in relationship building and stakeholder engagement with no guarantee of scientific outputs or even management impact (Oliver et al. 2019). In these situations, the process of empower itself can sometimes be considered as an important outcome. Interactions that are part of a project whose goals are not fully realized can be essential to strengthening relationships and provide a foundation for future success (Jagannathan et al. 2020).

3. Guidance for actionable science engagement approach selection

A project team may elect to inform stakeholders when providing rapid response information in a crisis situation, working with a homogenous or well-studied community, or responding to needs that already have been clearly articulated. Consulting stakeholders may be appropriate when the stakeholder community is large and diverse, whereas using participatory approaches may work best when the stakeholders and their needs are able to be well defined. Empowering the stakeholders may be an option when a project is addressing a complex, non-time-sensitive issue or in decision contexts where a shift in the balance of power between groups of stakeholders or between end users and researchers needs to occur. The wheel of participation represents a continuous spectrum, such that individual engagement approaches may span across segment boundaries depending on implementation.

It is important to note that there is no individual “best” approach to engagement and that the use of multiple approaches may be needed to engage with different stakeholders on the same project. We suggest some guiding thought questions in Table 2 that researchers can use when selecting an approach. We provide sample end point answers to the inform, consult, participate, empower spectrum that researchers can use to gauge where their answer to each thought question falls. Multiple answers clustered under a single approach might provide an indicator of what kind of engagement a researcher can start exploring for their project. Conversely, answers that span approaches might indicate the need to use a range of engagement approaches or methods to achieve project objectives.

Table 2.

Guiding thought questions to inform the selection of an engagement approach across the inform, consult, participate, and empower spectrum. Each question provides example end points, which researchers can use to situate themselves along the spectrum. A clear answer may emerge if multiple responses cluster under a single approach.

Table 2.

We strongly encourage researchers to keep in mind cultural elements when selecting an approach and to default to the use of empower approaches when working with muted groups. For example, when engaging with Indigenous communities, it is important to start with a justice-forward mindset, moving past only addressing historical wrongdoing to actively leading the removal of institutional and bureaucratic obstructions in order for partner communities to flourish (Whyte 2014). While there is a demonstrated need for training to inform how non-Indigenous researchers ethically engage with Indigenous communities without causing additional harm (Kirby et al. 2019), all researchers can keep in mind that there are tipping points around the areas of consent, trust, accountability, and responsibility that may result in permanent and irreversible relational damage if crossed (Whyte 2020). Demands for stakeholder input and participation may also be particularly burdensome to these communities when projects do not provide them with adequate support for the capacity and resources necessary to respond. Although the research community has taken some steps to move from an extractive model of Indigenous knowledge (Matsui 2015) to an integrative model of bringing together Indigenous and Western ways of knowing [e.g., two-eyed seeing in Bartlett et al. (2012)], we must now encourage research leadership that is foundationally grounded in Indigenous scholarship and that empowers Indigenous ways of knowing, sovereignty, and self-determination (Latulippe and Klenk 2020; Kimmerer 2014).

In addition, a lack of good communication between scientists and decision-makers is a major barrier to evidence-informed conservation policy (Rose et al. 2018), and we thus encourage researchers to explicitly address communications when selecting an engagement approach. Social processes that aid in knowledge exchange within a group include cultivating an atmosphere of psychological safety and trust from the beginning, using laughter and humor throughout the conversation to regulate the collective mood, and regularly reflecting group dynamics back to the group to promote communal ownership of process and outcomes (Morisette et al. 2017). Fundamental to this exchange are good communications practices for improving information relevance and uptake: encouraging critical thinking, valuing multiple types and lines of evidence over opinions, avoiding excessive jargon, and eliminating pointless meetings (McCarthy et al. 2020). To sustain strong relationships over the long-term, good communication must be augmented by supportive organizational behavior centered on trust, commitment, and openness (Ledingham 2003; Ledingham and Bruning 1998).

While the focus of this article is to provide the research community with a framework and guidance for engaging with stakeholders, such information may also be valuable to practitioners working at the nexus of knowledge creation, synthesis, translation, and application. Individuals and organizations in boundary spanning roles can be integral initiators and facilitators of the dialogue between researchers and stakeholders, often investing in long-term relationship building across diverse communities. Such boundary spanners can use this framework to help researchers and stakeholders negotiate an appropriate role within a project that best suits their application needs and available resources. Program administrators and funders of actionable science projects also have a role to play; by asking researchers to articulate how stakeholder engagement strategies support a given project’s scientific objectives, administrators and funders can encourage thoughtful approach selection. Resource managers, or any stakeholders, can benefit from this framework by engaging in projects that have a well-considered articulation of how their role and the selected engagement approach is likely to result in outputs and outcomes that meet their decision or planning needs.

4. Conclusions

In this article, we have adapted Davidson’s wheel of participation to create a standardized vocabulary for actionable science researchers and the wider scientific community to use when determining their stakeholder engagement strategy. Our adapted framework puts different approaches to stakeholder engagement on a continuous spectrum (inform, consult, participate, empower) and demonstrates that there are multiple paths to the same end goal of actionable science. We emphasize the need to find the right fit for each specific project and stakeholder community, and we provide high-level direction on how selection of an approach might be influenced by both the characteristics of the research context (e.g., science objectives; Table 1) and the decision context (e.g., power and stakeholder dynamics; Table 2). Our suggested guiding thought questions give researchers factors that they might want to consider in designing their stakeholder engagement strategy.

Researchers can also use our adapted framework when describing their projects to funding agencies, senior leadership, or evaluators. Public science funders are increasingly requiring stakeholder engagement as part of their funding criteria to increase the relevance of research outputs to society (Arnott et al. 2020b). These requirements likely will expand over time, as evidence demonstrates that well-designed stakeholder engagement can lead to scientific uptake in the policy process (e.g., Nguyen et al. 2019). By providing researchers with the language and context for selecting a particular approach, this framework can help provide justification for why a project might be required to invest substantial time and resources into engagement and why institutions need to consider this work as an integral part of the scientific process (i.e., a research and creative activity rather than as professional service or outreach). A constraint of the framework, however, is that it does not provide definitions for what successful engagement and actionability look like and how those characteristics might be measured and evaluated. Wall et al. (2017) can serve as a starting point for researchers interested in defining and evaluating success, but deeper consideration was outside the scope of this article.

We have presented our framework for stakeholder engagement using the example of climate change, which shares many qualities in common with other wicked problems in environmental and sustainability science. These challenges are ill-defined and multilayered, have no clear single optimal solution, require value judgments to resolve, and may continue on in the form of a new wicked problem once action has been taken (Rittel and Webber 1973). Science to address wicked problems in general requires an actionable approach and at least some degree of engagement with users of the science. Thus, our framework could potentially be generalized for use by researchers across these fields and is not necessarily constrained solely to the topic of actionable climate adaptation science.

Acknowledgments

This work was funded by the U.S. Geological Survey South Central and North Central Climate Adaptation Science Centers. Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. government. We thank author Bamzai-Dodson’s dissertation committee members and the members of the Climate Adaptation Science Center Evaluation Working Group for insightful discussions around what constitutes actionable science, what success might look like, and how it could be measured. We thank our peer reviewers for their thoughtful feedback, which honed and improved the concepts in this article. Special thanks are given to Kristen Donahue for her design work on Fig. 1. We hope that our research supports future continued conservation and protection efforts.

REFERENCES

  • Advisory Committee on Climate Change and Natural Resource Science, 2015: Report to the Secretary of the Interior. USGS Rep., 86 pp., https://www.sciencebase.gov/catalog/item/5c1d05d3e4b0708288c9bc2a.

  • Archie, K. M., 2014: Mountain communities and climate change adaptation: Barriers to planning and hurdles to implementation in the Southern Rocky Mountain Region of North America. Mitigation Adapt. Strategies Global Change, 19, 569587, https://doi.org/10.1007/s11027-013-9449-z.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Archie, K. M., L. Dilling, J. B. Milford, and F. C. Pampel, 2014: Unpacking the ‘information barrier’: Comparing perspectives on information as a barrier to climate change adaptation in the interior mountain west. J. Environ. Manage., 133, 397410, https://doi.org/10.1016/j.jenvman.2013.12.015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ardener, S., 2005: Ardener’s “muted groups”: The genesis of an idea and its praxis. Women Lang, 28, 5054, 72.

  • Arguez, A., I. Durre, S. Applequist, R. S. Vose, M. F. Squires, X. Yin, R. R. Heim, and T. W. Owen, 2012: NOAA’s 1981–2010 U.S. climate normals: An overview. Bull. Amer. Meteor. Soc., 93, 16871697, https://doi.org/10.1175/BAMS-D-11-00197.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Arnott, J. C., K. J. Mach, and G. Wong-Parodi, 2020a: Editorial overview: The science of actionable knowledge. Curr. Opin. Environ. Sustainability, 42, A1A5, https://doi.org/10.1016/j.cosust.2020.03.007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Arnott, J. C., R. J. Neuenfeldt, and M. C. Lemos, 2020b: Co-producing science for sustainability: Can funding change knowledge use? Global Environ. Change, 60, 101979, https://doi.org/10.1016/j.gloenvcha.2019.101979.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Arnstein, S. R., 1969: A ladder of citizen participation. J. Amer. Inst. Plann., 35, 216224, https://doi.org/10.1080/01944366908977225.

  • Averyt, K., and Coauthors, 2017: Regional climate response collaboratives: Multi-institutional support for climate resilience. Bull. Amer. Meteor. Soc., 99, 891898, https://doi.org/10.1175/BAMS-D-17-0183.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barry, J., and M. Thompson-Fawcett, 2020: Decolonizing the boundaries between the ‘planner’ and the ‘planned’: Implications of Indigenous property development. Plann. Theory Pract., 21, 410425, https://doi.org/10.1080/14649357.2020.1775874.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bartlett, C., M. Marshall, and A. Marshall, 2012: Two-eyed seeing and other lessons learned within a co-learning journey of bringing together Indigenous and mainstream knowledges and ways of knowing. J. Environ. Stud. Sci., 2, 331340, https://doi.org/10.1007/s13412-012-0086-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bathke, D., T. Haigh, T. Bernadt, and N. Wall, 2019: Drought scenario-based exercises. National Drought Mitigation Center Rep., 79 pp., https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1011&context=ndmcpub.

  • Beaury, E. M., E. J. Fusco, M. R. Jackson, B. B. Laginhas, T. L. Morelli, J. M. Allen, V. J. Pasquarella, and B. A. Bradley, 2020: Incorporating climate change into invasive species management: Insights from managers. Biol. Invasions, 22, 233252, https://doi.org/10.1007/s10530-019-02087-6.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bednarek, A. T., and Coauthors, 2018: Boundary spanning at the science–policy interface: The practitioners’ perspectives. Sustainability Sci., 13, 11751183, https://doi.org/10.1007/s11625-018-0550-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Beeton, T. A., S. M. McNeeley, B. W. Miller, and D. S. Ojima, 2019: Grounding simulation models with qualitative case studies: Toward a holistic framework to make climate science usable for US public land management. Climate Risk Manage., 23, 5066, https://doi.org/10.1016/j.crm.2018.09.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Beier, P., L. Hansen, L. Helbrecht, and D. Behar, 2017: A how-to guide for coproduction of actionable science. Conserv. Lett., 10, 288296, https://doi.org/10.1111/conl.12300.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Beierle, T., 2002: The quality of stakeholder-based decisions. Risk Anal., 22, 739749, https://doi.org/10.1111/0272-4332.00065.

  • Bierbaum, R., and Coauthors, 2013: A comprehensive review of climate adaptation in the United States: More than before, but less than needed. Mitigation Adapt. Strategies Global Change, 18, 361406, https://doi.org/10.1007/s11027-012-9423-1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bisbal, G. A., 2019: Practical tips to establish an actionable science portfolio for climate adaptation. Sci. Public Policy, 46, 148153, https://doi.org/10.1093/scipol/scy070.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bojovic, D., A. L. Clair St., I. Christel, M. Terrado, P. Stanzel, P. Gonzalez, and E. J. Palin, 2021: Engagement, involvement and empowerment: Three realms of a coproduction framework for climate services. Global Environ. Change, 68, 102271, https://doi.org/10.1016/j.gloenvcha.2021.102271.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Boy, G. A., 2017: The Handbook of Human-Machine Interaction: A Human-Centered Design Approach. CRC Press, 478 pp.

    • Crossref
    • Export Citation
  • Bracken, L. J., H. A. Bulkeley, and G. Whitman, 2015: Transdisciplinary research: Understanding the stakeholder perspective. J. Environ. Plann. Manage., 58, 12911308, https://doi.org/10.1080/09640568.2014.921596.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bradbury, H., S. Waddell, K. O’ Brien, M. Apgar, B. Teehankee, and I. Fazey, 2019: A call to action research for transformations: The times demand it. Action Res., 17, 310, https://doi.org/10.1177/1476750319829633.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bremer, S., and S. Meisch, 2017: Co-production in climate change research: Reviewing different perspectives. Wiley Interdiscip. Rev.: Climate Change, 8, e482, https://doi.org/10.1002/wcc.482.

    • Crossref
    • Export Citation
  • Briley, L., D. Brown, and S. E. Kalafatis, 2015: Overcoming barriers during the co-production of climate information for decision-making. Climate Risk Manage., 9, 4149, https://doi.org/10.1016/j.crm.2015.04.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brydon-Miller, M., D. Greenwood, and P. Maguire, 2003: Why action research? Action Res., 1, 928, https://doi.org/10.1177/14767503030011002.

  • Bush, V., 1945: Science, the endless frontier: A report to the President. U.S. Government Printing Office Doc., https://www.nsf.gov/about/history/vbush1945.htm.

  • Carter, S. K., and Coauthors, 2020: Bridging the research-management gap: Landscape science in practice on public lands in the western United States. Landscape Ecol., 35, 545560, https://doi.org/10.1007/s10980-020-00970-5.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cash, D., W. C. Clark, F. Alcock, N. M. Dickson, N. Eckley, and J. Jäger, 2003: Salience, credibility, legitimacy and boundaries: Linking research, assessment and decision making. KSG Working Paper RWP02-046, 24 pp.

    • Crossref
    • Export Citation
  • Cash, D., J. C. Borck, and A. G. Patt, 2006: Countering the loading-dock approach to linking science and decision making: Comparative analysis of El Niño/Southern Oscillation (ENSO) forecasting systems. Sci. Technol. Hum. Values, 31, 465494, https://doi.org/10.1177/0162243906287547.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christel, I., D. Hemment, D. Bojovic, F. Cucchietti, L. Calvo, M. Stefaner, and C. Buontempo, 2018: Introducing design in the development of effective climate services. Climate Serv., 9, 111121, https://doi.org/10.1016/j.cliser.2017.06.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clifford, K. R., W. R. Travis, and L. T. Nordgren, 2020: A climate knowledges approach to climate services. Climate Serv., 18, 100155, https://doi.org/10.1016/j.cliser.2020.100155.

    • Search Google Scholar
    • Export Citation
  • Cloyd, E., S. C. Moser, E. Maibach, J. Maldonado, and T. Chen, 2016: Engagement in the Third U.S. National Climate Assessment: Commitment, capacity, and communication for impact. Climatic Change, 135, 3954, https://doi.org/10.1007/s10584-015-1568-y.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Collins, C. S., and C. M. Stockton, 2018: The central role of theory in qualitative research. Int. J. Qual. Methods, 17, 1609406918797475, https://doi.org/10.1177/1609406918797475.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crausbay, S. D., and Coauthors, 2020: Unfamiliar territory: Emerging themes for ecological drought research and management. One Earth, 3, 337353, https://doi.org/10.1016/j.oneear.2020.08.019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cravens, A., and N. Ardoin, 2016: Negotiating credibility and legitimacy in the shadow of an authoritative data source. Ecol. Soc., 21, art30, https://doi.org/10.5751/ES-08849-210430.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Davidson, S., 1998: Spinning the wheel of empowerment. Planning, 1262, 1415.

  • DeCrappeo, N. M., G. A. Bisbal, and A. M. Meadow, 2017: A path to actionable climate science: Perspectives from the field. Environ. Manage., 61, 181187, https://doi.org/10.1007/s00267-017-0960-y.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dilling, L., and M. C. Lemos, 2011: Creating usable science: Opportunities and constraints for climate knowledge use and their implications for science policy. Global Environ. Change, 21, 680689, https://doi.org/10.1016/j.gloenvcha.2010.11.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dilling, L., K. Lackstrom, B. Haywood, K. Dow, M. C. Lemos, J. Berggren, and S. Kalafatis, 2015: What stakeholder needs tell us about enabling adaptive capacity: The intersection of context and information provision across regions in the United States. Wea. Climate Soc., 7, 517, https://doi.org/10.1175/WCAS-D-14-00001.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Diver, S., 2017: Negotiating Indigenous knowledge at the science-policy interface: Insights from the Xáxli’p Community Forest. Environ. Sci. Policy, 73, 111, https://doi.org/10.1016/j.envsci.2017.03.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doemeland, D., and J. Trevino, 2014: Which World Bank reports are widely read? World Bank Working Paper 6851, 32 pp., https://documents1.worldbank.org/curated/en/387501468322733597/pdf/WPS6851.pdf.

    • Crossref
    • Export Citation
  • Doswell, C. A., A. R. Moller, and H. E. Brooks, 1999: Storm spotting and public awareness since the first tornado forecasts of 1948. Wea. Forecasting, 14, 544557, https://doi.org/10.1175/1520-0434(1999)014<0544:SSAPAS>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Enquist, C., and Coauthors, 2017: Foundations of translational ecology. Front. Ecol. Environ., 15, 541550, https://doi.org/10.1002/fee.1733.

  • Eyring, V., S. Bony, G. A. Meehl, C. A. Senior, B. Stevens, R. J. Stouffer, and K. E. Taylor, 2016: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci. Model Dev., 9, 19371958, https://doi.org/10.5194/gmd-9-1937-2016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Few, R., K. Brown, and E. Tompkins, 2007: Public participation and climate change adaptation: Avoiding the illusion of inclusion. Climate Policy, 7, 4659, https://doi.org/10.1080/14693062.2007.9685637.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Filho, W. L., Ed., 2015: Handbook of Climate Change Adaptation. Springer-Verlag, 2198 pp.

  • Fisher, J. R. B., S. A. Wood, M. A. Bradford, and T. R. Kelsey, 2020: Improving scientific impact: How to practice science that influences environmental policy and management. Conserv. Sci. Pract., 2, e210, https://doi.org/10.1111/csp2.210.

    • Search Google Scholar
    • Export Citation
  • Friesen, P., L. Kearns, B. Redman, and A. L. Caplan, 2017: Rethinking the Belmont report? Amer. J. Bioeth., 17, 1521, https://doi.org/10.1080/15265161.2017.1329482.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fyfe, A., K. Coate, S. Curry, S. Lawson, N. Moxham, and C. M. Røstvik, 2017: Untangling academic publishing: A history of the relationship between commercial interests, academic prestige and the circulation of research. U.K. Arts and Humanities Research Council Briefing Paper, 23 pp., https://doi.org/10.5281/zenodo.546100.

    • Crossref
    • Export Citation
  • Gasson, S., 2003: Human-centered vs. user-centered approaches to information system design. J. Inf. Technol. Theory Appl., 5, 29–46.

  • Gerlak, A. K., T. Heikkila, and J. Newig, 2020: Learning in environmental governance: Opportunities for translating theory to practice. J. Environ. Policy Plann., 22, 653666, https://doi.org/10.1080/1523908X.2020.1776100.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gluckman, P., and J. Wilsdon, 2016: From paradox to principles: Where next for scientific advice to governments? Palgrave Commun., 2, 16077, https://doi.org/10.1057/palcomms.2016.77.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goodrich, K. A., K. D. Sjostrom, C. Vaughan, L. Nichols, A. Bednarek, and M. C. Lemos, 2020: Who are boundary spanners and how can we support them in making knowledge more actionable in sustainability fields? Curr. Opin. Environ. Sustainability, 42, 4551, https://doi.org/10.1016/j.cosust.2020.01.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guido, Z., D. Hill, M. Crimmins, and D. Ferguson, 2013: Informing decisions with a climate synthesis product: Implications for regional climate services. Wea. Climate Soc., 5, 8392, https://doi.org/10.1175/WCAS-D-12-00012.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hassol, S. J., 2008: Improving how scientists communicate about climate change. Eos, Trans. Amer. Geophys. Union, 89, 106107, https://doi.org/10.1029/2008EO110002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hegger, D., M. Lamers, A. V. Zeijl-Rozema, and C. Dieperink, 2012: Conceptualising joint knowledge production in regional climate change adaptation projects: Success conditions and levers for action. Environ. Sci. Policy, 18, 5265, https://doi.org/10.1016/j.envsci.2012.01.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hegger, D., A. V. Zeijl-Rozema, and C. Dieperink, 2014: Toward design principles for joint knowledge production projects: Lessons from the deepest polder of The Netherlands. Reg. Environ. Change, 14, 10491062, https://doi.org/10.1007/s10113-012-0382-6.

    • Search Google Scholar
    • Export Citation
  • Human, B. A., and A. Davies, 2010: Stakeholder consultation during the planning phase of scientific programs. Mar. Policy, 34, 645654, https://doi.org/10.1016/j.marpol.2009.12.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • IDEO, 2015: The Field Guide to Human-Centered Design. 1st ed. IDEO.org/Design Kit, 192 pp.

  • Imenda, S., 2014: Is there a conceptual difference between theoretical and conceptual frameworks? J. Soc. Sci., 38, 185195, https://doi.org/10.1080/09718923.2014.11893249.

    • Search Google Scholar
    • Export Citation
  • IPCC, 2018: Summary for policymakers. Global Warming of 1.5°C, V. Masson-Delmotte et al., Eds., Cambridge University Press, 32 pp., https://www.ipcc.ch/site/assets/uploads/sites/2/2019/05/SR15_SPM_version_report_LR.pdf.

  • Jagannathan, K., A. D. Jones, and I. Ray, 2020: The making of a metric: Co-producing decision-relevant climate science. Bull. Amer. Meteor. Soc., 102, E1579E1590, https://doi.org/10.1175/BAMS-D-19-0296.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jansen, T., L. Claassen, I. van Kamp, and D. R. M. Timmermans, 2019: Understanding of the concept of ‘uncertain risk’. A qualitative study among different societal groups. J. Risk Res., 22, 658672, https://doi.org/10.1080/13669877.2018.1503614.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jantarasami, L., J. Lawler, and C. Thomas, 2010: Institutional barriers to climate change adaptation in U.S. National Parks and Forests. Ecol. Soc., 15, art33, https://doi.org/10.5751/ES-03715-150433.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jarvis, R. M., and Coauthors, 2020: Navigating spaces between conservation research and practice: Are we making progress? Ecol. Solutions Evidence, 1, e12028, https://doi.org/10.1002/2688-8319.12028.

    • Search Google Scholar
    • Export Citation
  • Jasanoff, S., Ed., 2004: States of Knowledge: The Co-production of Science and the Social Order. Routledge, 317 pp.

  • Johnson, C. J., and M. P. Gillingham, 2004: Mapping uncertainty: Sensitivity of wildlife habitat ratings to expert opinion. J. Appl. Ecol., 41, 10321041, https://doi.org/10.1111/j.0021-8901.2004.00975.x.

    • Crossref