1. Introduction
A simple web search for the phrase “climate information” yields a myriad of results, from detailed climate analysis provided by the National Oceanic and Atmospheric Administration (NOAA), the National Aeronautics and Space Administration (NASA), and even a tutorial explaining the differences between weather and climate. Taking this search one step further introduces social media platforms, such as Twitter or Facebook, and suddenly society’s capacity to learn and share climate information grows with each person (Ripberger et al. 2014). For the climate scientist, the suite of information available online is abundant, clear, and adheres to the rigor imposed by the scientific community. However, for the water resources manager seeking information on whether his reservoir will go dry in the next 3–6 months or for a coastal city port authority seeking information on unusually high tides, the amount of climate information online (and in print) is truly staggering. Finding the material is the simple step; screening, interpreting, and distilling it to actionable information are the challenges (Clark et al. 2016).
In these examples, both the water manager and the port authority are users. For the purposes of this article and following the terminology defined by Freeman (1984), a user is defined as someone who directly uses an organization’s products and services to inform a decision that may lead to action. What this example scenario lacks is a bridge between the science and the user—someone to translate the scientific information from a 3-month seasonal climate outlook to the user for their decision-making purposes. Similarly, the user may be in need of a 3-month seasonal outlook but perhaps in a location-specific tabular form versus a graphical interface. Yet, the scientist is currently only producing the graphical version and is unaware the existing visual is not meeting user needs. User engagement is a mechanism that can bridge this gap. This process of user engagement allows users across multiple sectors to interact to gather requirements, discuss uses and applications of an organization’s products, and engage in continuous dialogue to improve the products to meet customer needs (Gibbons 2000). In the end, this continuous dialogue between the scientific and user communities aims to ensure that the user is getting the information they need in a format that enables them to take action, while the scientist is using the latest technologies and methods to generate information based on an understanding of user requirements.
It is our contention that user engagement of this nature remains a highly underutilized tactic for assessing the performance, applicability, and usefulness of existing, and future, climate information products and services. Perhaps one fundamental reason for the underutilization stems from the long adhered to model for how science operates. In 1945, V. Bush authored a manuscript that argued in favor of the separation of science and the user community in order to “maintain objectivity and credibility…of the science” (Bush 1945). Furthermore, this model relies on knowledge produced in university settings and ignores other sources of knowledge and disciplinary perspectives (Gibbons et al. 1994). As a result, the development of science, information, and knowledge has been dependent on this model for the last 70+ years and is entrenched into day-to-day science activities and research. Consequently, operating on a new model, one that works to engage with society and the community, requires substantial reversal of current procedures and adaptation to a new regime. To encourage the scientific research community to break the chains of this practice thus requires a full paradigm shift away from the university setting and out toward the community, where different perspectives, knowledge, and disciplines exist. The literature suggests that this is best accomplished through the participation of and input from a multitude of users and stakeholders in a way that enables them to help shape and refine the content and format of products and services (e.g., Jacobs 2002). However, successful user engagement requires time, resources, effort, and the ability to build relationships (Rickinson et al. 2011).
Demonstrating actual use cases in support of the coproduction of knowledge addresses an existing knowledge gap in this area as stated by Reed et al. (2014) and Meadow et al. (2015). To that end, this article showcases the successful application of, and output from, the coproduction of knowledge user engagement model through two uniquely successful activities led by the NOAA National Centers for Environmental Information (NCEI) and in partnership with the NOAA Office of Coastal Management (OCM) Coral Reef Conservation Program (CRCP). The first activity is a series of drought amelioration and termination workshops coordinated by NOAA NCEI and the National Integrated Drought Information System (NIDIS). The second is a series of in-country dialogues held in the islands of the Pacific during much of 2014 by NOAA NCEI and NOAA CRCP. Each of these engagement activities exemplified the benefits of a robust and sustained relationship between the scientific and user communities. The final section of the article highlights lessons learned and recommended best practices for continued successful user engagement.
2. Understanding the coproduction of knowledge
Following Reed et al. (2014), there exist five principles for effective knowledge exchange. Applying those five principles to the case of climate scientists offers the following:
Design—Does a communications strategy exist? Does the project allow for flexibility to stakeholder feedback?
Represent—Has the project leader identified likely users and embedded them within the research?
Engage—Is there an understanding of what motivates the user group to participate? Are there opportunities for informal interaction and learning between the users and the scientists?
Impact—When is the best time to conduct user engagement for each project?
Reflect and sustain—How often should the scientist revisit the user group to assess effectiveness? Is there an understanding of what user needs persist after the research has ceased?
The application of these five principles outlined by Reed et al. (2014) provides a concrete pathway along which successful knowledge exchange can be done. For more information on knowledge exchange in interdisciplinary and multistakeholder research areas, the reader is referred to Fazey et al. (2014).
There is a variety of existing user engagement strategies to choose from when addressing the questions posed by Reed et al. (2014). Presentations at workshops and conferences with a question and answer period afterward are one very common approach in the weather and climate sciences. According to Rickinson et al. (2011), this approach brings the scientific knowledge to the user and offers an immediate check of the validity of the science while collecting additional data from the audience-driven feedback. Another strategy often used is needs assessments, which include interviews either in person or by phone to have users answer questions about what products and information they need to make decisions. While these models offer a big picture perspective for user engagement, users in these models are typically brought in only as a means to “serve a predetermined research agenda” (Rickinson et al. 2011, p. 33).
In light of the differences between these user engagement approaches, there is one model we find to be the most effective at drawing out requirements, referred to as the coproduction of knowledge (Studd 2002; Chilvers 2008; Pohl 2007; Meadow et al. 2015). In 2009, Carney et al. found that during engagement activities featuring coproduction of knowledge, the engagement often encouraged a wider array of input from various communities (than otherwise might have been considered by the researcher alone) and led to discussions of uncertainties in climate change research, especially in the stakeholder region. In addition, the coproduction of knowledge model tends to encourage buy in and support from stakeholders and fosters mutual learning from both the stakeholder and the scientist (Carney et al. 2009). Coproduction of knowledge is an effective way to advance collaboration between scientists and decision-makers and exhibits three key elements: 1) it establishes long-term relationships between scientists and stakeholders, 2) it facilitates an open and iterative process of two-way communication, and 3) it focuses on usable science (Lemos and Morehouse 2005; Dilling and Lemos 2011; Meadow et al. 2015).
Collectively, the literature emphasizes the importance of establishing, building, and maintaining project–stakeholder relationships as part of the coproduction of knowledge. These relationships can be sustained by building reciprocal trust through “improving communication skills, behaving reliably, showing commitment, being sincere, and working towards reaching project milestones” (Karlsen et al. 2008, p. 1). As such, the participation from the community requires interactions in the research process, including defining the problem and formulation of research questions, all the way to analyzing the results and developing usable information (Gibbons 2000; Kirchhoff et al. 2013). This creates a positive feedback loop (also called iterativity; Lemos and Morehouse 2005) where users and stakeholders are able exchange information thereby creating “more relevant and usable information for solving problems and improving the fit between what users want and what science can offer” (Kirchhoff et al. 2013, p. 11). In turn, this generated more “buy in” from stakeholders, improved trust between scientists and the users, and improved communication. Such interactions allow the users to build a portfolio of information to manage the uncertainty in their specific decision (Kirchhoff et al. 2013).
The principle of iterativity describes the capacity to “sustain ongoing flows of information and participation between science and decision makers” (Lemos and Morehouse 2005, p. 61). In that article, this principle is used to describe the interaction between science and policy, and in 2011 Dilling and Lemos discussed modes of science production, revisiting the importance of iterativity in stakeholder engagement where there is a push of data and information from scientists and a pull of information from potential users. In this model, scientists iteratively work with users to refine their climate products and services, while users offer feedback on the utility and applicability of said products and services. This is the embodiment of the coproduction of knowledge, the output from which often better aligns with user needs than that which might have been produced by scientists alone (Dilling and Lemos 2011). The result is a product that is truly actionable, providing information that helps address a societal problem through the demonstration of mutual learning and the iterative, back-and-forth nature to the product refinement and thus the term coproduction of knowledge.
Reed et al. (2014) continue this discussion and strongly suggest that successful user engagement and knowledge exchange should be designed as part of the research process (rather than an afterthought), inclusive of inputs from both the research users and stakeholders. As part of this process, the intended outcomes, Reed et al. (2014) argue, should be made clear with reasoning and explanation as to how and why those activities will deliver outcomes. In the end, they found that this process was essential to foster a sense of trust and create a sense of shared ownership in the research questions and findings. A study by Beierle (2002, p. 747), who examined the value of user engagement on the quality of environmental decisions, found that “most of the evidence points toward quality decision making from stakeholder processes.” That study goes on to say that the more intense the interaction or engagement with stakeholders is, the more likely the project will result in higher-quality decisions. This is further supported by Meadow et al. (2015, p. 179), who demonstrated that “when knowledge is coproduced it is more likely to be accepted and used by decision makers.”
The following two sections focus on the application of the coproduction of knowledge to two very different geographies, cultures, and informational needs. These case studies exemplify not only the time commitment needed for successful coproduction of knowledge, but also the iterative, back-and-forth nature to the dialogue and its outputs.
3. Coproduction of knowledge and drought information
In 2014, Werner et al. prepared a service assessment of NOAA’s drought information services to stakeholders impacted by the historic California drought (Werner et al. 2014). The goals of the assessment were to identify gaps in services and opportunities for improvement to support decision-makers impacted by the drought. Previous service assessments generally focused entirely within the NOAA National Weather Service (NWS) and on weather events such as tornados, hurricanes, and floods. Service assessments had generally not focused on long-duration events such as drought or more broadly across NOAA. An assessment team of 10 professionals across NOAA conducted interviews with over 100 stakeholders to gather insight into their drought information needs. The team focused on stakeholders in water resources, agriculture, and fisheries. Interviews focused on which decisions were being made and what went into making those decisions. This tactic is consistent with other requirement-gathering efforts across the discipline, including work by Finucane et al. (2013), who targeted freshwater resource managers. In both instances, users were asked about the politics, infrastructure, and influences in their decision-making, and, in the case of the California drought, the extent to which NOAA’s services helped decision-makers make better decisions. Importantly, these needs assessments started a conversation that could be leveraged for other projects to continue to assess the community’s needs for rainfall and drought information. One significant takeaway from this drought service assessment was that users wanted information on drought termination. They asked questions like “How much rain will it take to end the current drought?”. Recognizing this as a user requirement in 2014, NIDIS spearheaded the first iteration of this effort and commissioned a research project led by NCEI to determine the usefulness of existing drought termination tools and what changes were needed to the existing suite of products.
To that end, a drought amelioration user engagement workshop (effectively the second iteration of this effort) was conducted in Orange County, California, in September 2015 with the media, general public, political leadership, and other key stakeholders to identify sector-specific requirements aimed at enhancing the drought amelioration web resource product suite.
The morning session of the 1-day workshop featured a facilitated discussion during which the participants went online and worked through the NCEI drought termination and amelioration tool.1 Feedback gathered throughout the session determined that the website was difficult to navigate and understand. Following the back-and-forth nature of the coproduction of knowledge model, workshop participants were given an opportunity to vote on a variety of their identified suggested changes for the website, relying on their experiences and sector-specific needs for drought information. This was important because dozens of requirements were identified during the session. Rather than the scientists deciding the priority of each requirement, the workshop attendees decided among themselves, through this voting exercise. The changes that garnered the most votes included moving toward a high-resolution gridded dataset instead of (long used) climate divisions, improving the graphical presentation of the maps (adding zoom-in capabilities and improving the legends), adding more identifying information to the online data files, and streamlining the information in the introduction to the tool.
In the next session, another component of the coproduction of knowledge model was used: storytelling, which has been shown to facilitate the building of relationships between users and researchers (Bickmore and Cassell 2001). Here, workshop participants shared their knowledge of drought, their favorite go-to tools, and actions taken during a drought within their specific discipline. In addition, breakout groups were identified by sector; some represented local media (news station reporters, on-air personnel) and some were from local water municipalities, while others represented academia. During a report-out session in plenary following the group activity, nearly all of the workshop participants agreed that the information presented in the NCEI tool was not relevant to drought recovery in California, while others found it to be useful for assessing drought recovery in the Midwest. This same discussion revealed that the drought recovery tool could be further improved if it incorporated additional indices and variables, including snowpack, precipitation/streamflow needed to restore reservoir levels and groundwater levels, and precipitation needed to restore deficit loss since beginning of the drought. Some asked if a new drought recovery index could be created that incorporates existing drought and water resources’ indices.
By the end of the workshop, it was clear that the coproduction of knowledge model applied in this user engagement workshop was successful. We were provided invaluable insights that included revelations on some of the misunderstandings and semantics around the maps, plots, and descriptions on the NCEI drought amelioration tool. We also learned about differences in what constitutes a drought from the perspective of each sector and how drought information is relayed to their constituents. From the user perspective, users gained knowledge about new and upcoming regional drought and climate products from other NOAA offices, including the Western Regional Climate Center and the National Weather Service River Forecast Centers. In addition, a whole session was devoted to informing them on the impacts from what was an expected El Niño during the 2015–16 California water year. Together, the discussions illuminated new concepts, including the strong desire for a drought index that takes into account the water resources infrastructure. Furthermore, the discussions highlighted the perceptions in drought termination versus amelioration, where wealthier counties viewed drought termination as more important than drought amelioration, while in less affluent counties, drought amelioration products were favored over drought termination products. Quite simply, neither we nor the workshop participants would have learned these things had there not been this back-and-forth nature to the discussions. Based on this feedback from the workshop participants, the NCEI drought amelioration and termination website has been modified for cosmetic and documentary changes. The text is more streamlined, map legends have been revised, and the raw data are now available in more than just plain text file format. To address the initial user-identified concern in the variation in drought intensity and spatial extent among the NOAA drought termination products, a gateway web page2 was developed by NCEI climate scientists that discuss the differences between the various NOAA drought termination products and their appropriate uses, especially between other NOAA monthly drought termination products.
Finally, the third iteration of this effort featured a follow-on drought amelioration workshop that was held on 2 June 2017 in Lake Tahoe, Nevada, titled “Do Floods Terminate a Drought? A Workshop on Drought Recovery Tools, Perspectives, and Situational Awareness.” The purpose of this workshop was to reengage with participants and a new suite of users to determine if the changes made to the NCEI drought amelioration and termination tool improved any actual decision-making in the real world. There were two dozen persons in attendance representing a variety of disciplines (Tribal Leaders, Bloomberg News, California State Climate Office, the NOAA National Weather Service, the California Bureau of Reclamation, the Desert Research Institute, etc.). The morning of the workshop provided the users with hands-on experience with the revised NCEI drought termination and amelioration tool (demonstrating the flow of information from user to scientist). Participants stated that the revised tool would indeed help inform their decision-making, especially with respect to the “projected drought” section of the website, where NOAA seasonal forecasts are used to compute a projected value of the drought indices on the website. The knowledge exchange was reversed in the afternoon session as the scientists provided details, statistics, historical perspective, and the scientific background behind the anomalously wet 2016–17 water year and addressed the question posed in the title of the workshop. Prior to the workshop, there was some question whether a drought recovery workshop was even necessary or relevant given the abundant rain and snow that hit California and Nevada during the 2016–17 water year. However, as it turned out, many of the participants were already preparing for the next drought. In this regard the discussion of the drought amelioration tool was exceptionally timely. The enthusiasm from the participants was epitomized when, at the end of the workshop, they specifically requested a fourth iteration of this effort by holding another drought amelioration workshop in Boise, Idaho, in 2018, where different sectors could be targeted, including farmers, water managers, and other users of the Snake River.
The iterative nature to this work is central to its success and helps 1) continue to improve the application of climate science to decision-making and 2) drive science- and data-focused tool improvement in ways that are both scientifically important and ultimately applicable to decision-making.
4. Coproduction of knowledge in Pacific small island developing states
Pacific leaders, such as the president of the Republic of the Marshall Islands and the president of Kiribati, continue to call for assistance as they strive to understand, predict, and adapt to a changing climate [see the 2017 conference proceedings of the Pacific Risk Management Ohana (PRiMO) at https://coast.noaa.gov/primo/]. The development and delivery of actionable information about climate patterns and trends—and their impacts on communities, businesses, and ecosystems—is essential to many aspects of policy, planning, and decision-making. NOAA, in cooperation with the Department of State and the U.S. Agency for International Development (USAID), undertook a program from 2012 to 2014 to support climate change adaptation in the Pacific small island developing states by conducting a series of activities to enhance scientific and technical capacity. The ultimate goal was to improve the delivery of climate information while establishing a mechanism for sustained communication, engagement, and partnership in the Pacific Islands. This was accomplished by identifying needs at a regional scale, subregionally refining the ideas, and then engaging in in-country dialogues (Fig. 1). Those dialogues set into motion country-specific programs and activities, which feed back to the regional level through mechanisms such as the Pacific Islands Climate (PICs) panel under the Pacific Meteorological Council (PMC) and Global Framework for Climate Services (GFCS) processes. In the case of the example that follows, this iterative process has continued with follow-up work and dialogues in the countries, through small grants, after the initial project and funding ended.
Process of sustained engagement in the Pacific Islands.
Citation: Weather, Climate, and Society 9, 4; 10.1175/WCAS-D-16-0127.1
The project initiated with the Pacific Islands Climate Services Forum in Suva, Fiji, in 2013. The goal of the forum was to raise awareness and engage in a dialogue about climate services and to strengthen and build new relationships between producers and users of climate information to address issues of critical importance in the region. Over 200 participants from more than 30 countries, including representatives from over 25 institutions, regional organizations, and donors involved in climate services in the region attended. Key recommendations from the forum included the following:
To focus on the transformation of information by placing it in the local context so that it was easily accessible and understood. This included aggregating multiple sources of information into one product or in one place.
To ground products and services development and delivery in the iterative coproduction of knowledge, highlighting not only the importance of combining local and traditional knowledge with the latest climate science and information but working side by side to develop those products and services.
To align and coordinate activities with climate service providers in the Pacific region.
Based on the outcomes of the regional forum, the approach was refined subregionally through facilitated interactions and three focus areas were identified: 1) freshwater resources and drought, 2) community resilience and sea level rise/coastal inundation, and 3) sustaining coral reef ecosystems. The need for in-country dialogues was confirmed; these were planned and facilitated in partnership with key agencies on the ground and brought together generators of climate data and information, with those from local agencies that could translate that information and with those that transmit the information to the public.
The in-country dialogue process evolved during the project. Basically, 2 to 3 months of working with the local planning team culminated in 4 to 5 days on the ground to share and learn from one another. An important aspect of the process was developing an understanding of what impacts were being felt, how decisions were made, how information flowed, and what information was missing. Working through this information resulted in the identification of programs and activities to strengthen climate services in the country. Over the course of 6 months in 2014, dialogues were held in five countries, spanning the three focus areas with the involvement of representatives from seven additional countries:
Republic of the Marshall Islands in April 2014 (with additional participation from Micronesia and Palau);
Vanuatu in June 2014 (with additional participants from the Solomon Islands, Papua New Guinea, and the Republic of the Marshall Islands);
American Samoa in July 2014 (with additional participants from Samoa and Tonga);
Samoa in August 2014 (with additional participants from American Samoa); and
the Cook Islands in September 2014 (with additional participants from Kiribati).
Dialogue planning began as a conversation with the national Meteorological Offices, who determined topic and issues of focus and identified key agencies (e.g., fisheries, agriculture, water resource, natural resource, and coastal zone management) to be part of the team to design the dialogue. A key step was to then identify the transmitters—those trusted voices that can engage with the community whether it be the local minister, a local nongovernmental organization (NGO), theater group, village chief or elder, fishers, local marine management area, or park managers—and ask them to take part as well. Finally representatives from other regional organizations and donors who were doing related work in the country were asked to participate. The wide variety of backgrounds represented at these dialogues helped foster the coproduction of knowledge, not just between the climate scientists and local meteorologists who hosted the meetings, but also between the local participants as well.
The dialogue itself is composed of four steps designed to maximize user input: 1) a predialogue technical exchange, 2) sharing climate knowledge, 3) diagnosing climate services, and 4) building climate stories (climate services dialogue process; Fig. 2). Documentation of the process and materials needed to carry out this user engagement process can be found in the Pacific Islands Climate Storybook (NOAA 2015). Each step is explored below in the context of the work in Vanuatu with the Vanuatu Meteorological and Geo-Hazards Department (VMGD).
Method of the climate services dialogue process used and refined in the Pacific region during the 2014 in-country dialogs.
Citation: Weather, Climate, and Society 9, 4; 10.1175/WCAS-D-16-0127.1
An initial dialogue was held in June 2014 in Vanuatu. This resulted in a plan of engagement to strengthen the Vanuatu Climate Early Warning System with a focus on coral reef ecosystems and fisheries given their importance to coastal protection and food security. Specifically, VMGD identified the need to hold a follow-up dialogue with the Vanuatu Fisheries Service on climate impacts. They wanted to update their ENSO Handbook for the Marine Sector with possible El Niño or La Niña impacts and actions to reduce the effects of those impacts on the community. Finally, there was need for a technical transfer between NOAA and VMGD information technology (IT) and product development staff to assist them in integrating NOAA data into their Climate Early Warning System. Additional funding to continue the iterative process in 2015 was secured.
The second dialogue was scheduled for 2015 but was postponed until May 2016 because of the impacts of Tropical Cyclone Pam. The first step in this round of dialogue was a predialogue technical exchange with the planning team. The goal of this interaction is to work with a small subset of local technical partners to describe and diagnose the state of existing climate services delivery and to define and refine the objectives for the dialogue before the full suite of participants join. In the case of Vanuatu, we worked primarily with the VMGD Climate Services Division to understand what satellite and other data and information streams were being pulled in, what their IT infrastructure and programming capacity was, and what the goals for strengthening their marine climate early warning system were.
The second step, sharing climate knowledge, kicked off the full dialogue with participants from the fisheries department, local business owners, fishers, community members, as well as scientific and technical partners. Participants shared experiential- and science-based knowledge on climate-related events and impacts from their past experience. This information is the basis to inform future products and explore best practices in effective delivery of information to inform decisions that can reduce impacts to communities during events.
In the third step, participants diagnose climate services by describing existing climate services and information and assessing the capacity of those services to support early warning for specific events. This was done by using a specific recent climate-related event to review existing decision-making processes to identify actions and information to strengthen early warning and response; in this case, Vanuatu focused on coral bleaching and the impacts of Tropical Cyclone Pam.
Finally, in the fourth step, building climate stories, participants explored techniques in climate communication; built their draft stories (case studies) to communicate key messages, best practices, and lessons learned; and finalized steps to improve and support climate services. The story development process was initiated at the dialogue where stories are selected and drafted (Fig. 3). The stories are finalized postdialogue after a literature review is completed, and links to relevant key messages are highlighted in partnership with the local author. Each dialogue developed and refined specific key messages and best practices that helped align the science with on-the-ground actions. Importantly, these key messages were a fundamental outcome of the dialogues and were embedded throughout the story process.
Flow diagram for building a complete climate story as used in the Pacific small island developing states.
Citation: Weather, Climate, and Society 9, 4; 10.1175/WCAS-D-16-0127.1
During the second dialogue in Vanuatu, four outcomes were accomplished, and the next iteration of activities was identified. Technical experts from NOAA and the University of Hawaii worked with VMGD technical staff to find and retrieve remote sensing data and build the code to develop a product based on a set of indicators and thresholds that provide indication of areas of coral reef that could be at risk for bleaching. Work with the Vanuatu Fisheries Department, local fishers, and dive tour operators, and so on, resulted in a strategy to build out a nearshore pelagic fisheries and safety monitor within the climate early warning system to help community fishers safely and efficiently access tuna. This included an exploration of key indicators for the early warning system such as chlorophyll and sea surface temperature thresholds based on the scientific literature associated with those indicators and other oceanographic data that would be necessary. A draft of the marine sector impacts and actions for their ENSO handbook was completed. Finally, an inaugural National Statement of the Impacts of El Niño and Tropical Cyclone Pam on the Marine Sector and a Climate Marine Bulletin and Outlook for the Fisheries Department were completed. Both of these products already had corollaries for the agricultural sector in Vanuatu. Since this engagement, Vanuatu has continued to mainstream the outcomes of the work we have done. The outcomes of the technical exchange and dialogue have been integrated into the recently released Framework for Climate Services [Secretariat of the Pacific Regional Environment Programme (SPREP) 2016], one of the first of such plans in the Pacific Islands. They continue to work to incorporate NOAA and other data into their climate early warning system and website.
The work in Vanuatu is a prime example of how an iterative process of user engagement strengthens climate services. Three key outputs emerged from the dialogues, as follows:
a better understanding of the state of national climate early warning systems and steps identified to strengthen these services through an iterative process;
a significant increase in regional coordination and collaboration among programs and partnerships across the Pacific, including national governmental counterparts, regional organizations and networks, and stakeholders in multiple sectors; and
follow-up on the recommendations of past dialogues and continuation of the iterative process to generate feedback on existing climate products produced by NOAA NCEI.
Feedback on existing climate products included a request from users related to indicators of climate change and variability (heavy rain events, extreme temperatures, unusually windy periods, etc.). They asked for an easy-to-digest format so that someone without an advanced degree in the science of weather and climate can readily identify what is happening locally and potentially act upon this information. As a result, climate scientists produced new graphical displays and then returned to the users to ensure it met their requirements for consumption by a nontechnical audience. Much like the case for the previously discussed drought workshop, the iterative nature to the Pacific Islands work also facilitated a continued need for engagement with users and helped climate scientists understand and implement periodic improvements to existing climate information, products, and services to the Pacific region both from NOAA’s NCEI and OCM.
5. Synthesis and conclusions
User engagement is designed to connect the needs of the users with the capabilities of the science. While a variety of user engagement strategies exist for gathering requirements, the literature advises that interactions should foster a dialogue among the scientific and user communities such that the user is getting the information they need in a format that enables them to take action, while the scientist is using the latest technologies and methods to generate information that is based on an understanding of user requirements. This approach is most closely aligned with the coproduction of knowledge model of user engagement. We selected and refined this back-and-forth user engagement model as evidenced through the NIDIS drought amelioration workshop and the climate services dialogues held in the Pacific Islands. As a result of these interactive and iterative user engagement experiences, climate services users became better informed about the current state of knowledge about climate variability and its impacts; more skilled in understanding, translating, and applying the science behind and consequences of a changing climate; better able to make use of the technical capabilities at their disposal to assess adaptation options and strategies; and, as a result, able to make better decisions as they set priorities and allocate resources. Climate services providers (i.e., the hosts of the dialogs) became much more informed about what needs and questions were most relevant and, consequently, are now better able to match products and services to user requirements.
With an emphasis on engagement and consultation between service providers and users—to include those trusted agents who can deliver information to the public—activities in each of these two cases included repackaging and dissemination of existing climate-related products and services; development of new or enhanced products and services; and the advancement of subregional and in-country training and core capacity building. Throughout our engagement activities, we worked to improve the process to capture the most information possible while ensuring maximum participation from the user audience.
The following are some recommended best practices that were a result of these interactions. While these recommendations are consistent with those in the existing literature (e.g., Beierle 2002; Jacobs 2002; Reed et al. 2014), they reinforce the message of the importance of and techniques for maximizing stakeholder interactions:
Interactions with decision-makers should be made more deliberate to improve understanding of decision points and what climate data or services might be applicable.
“Stakeholder participation processes should be tailored to fit the unique needs and opportunities of each context…[recognizing] the social, political, economic dimensions affecting various stakeholder groups” (NOAA 2004, p. 19). Stakeholder participation is an essential component of any project, as “stakeholders view their involvement as meaningful and as making a difference” (Pirk 2002).
Workshops and dialogues should be less focused on the presentation of science, instead engaging with participants in relating major climate and weather events to local-scale impacts. This interaction can be used to inform the user community on existing climate information and products that may be applied to future analogous weather and climate events. In addition, this encourages a dialogue where users illustrate how and where these products can improve decision-making.
Maximizing interaction and learning both ways is essential, whether through facilitated discussions or other facilitator tools such as small group work, voting, use of colored dots, and so on. This signals that the workshop hosts are eager to learn from the attendees, which in turn can encourage attendees to be more outspoken and contributory to the overall objectives of the workshop.
Importantly, this iterative process for user engagement does not negate the need for scientific innovation, but rather user engagement often leads to innovation and even helps refine it such that the iterative process can begin again. Furthermore, the iterative nature of such dialogues can help generators of climate information to advance useful, locally relevant, and regionally applicable tools and insights. Applying the coproduction of the knowledge model for user engagement activities also fosters trust between the primary scientific organization and the stakeholder community. Given the demonstrated success of the coproduction of the knowledge model of user engagement described herein and the ready-to-use nature of the existing guidance (e.g., NOAA 2015), we strongly encourage future user engagement activities to utilize the coproduction of knowledge model, especially in the context of climate information.
Acknowledgments
The authors thank the National Integrated Drought Information System (NIDIS), the U.S. Agency for International Development, and the NOAA Coral Reef Conservation Program for supporting these efforts. The work in the Pacific Islands could not have been done without the remarkable input and effort from Kitty Courtney, Karl Fellenius, Scott Heron, Michael Lameier, James Potemra, and William Sweet. We would also like to thank the following key partners: Akapo Akapo, Utu Abe Malae, Arona Ngari, Ausetalia Mulipola Titimaea, Reginald White, Australian Bureau of Meteorology, College of the Marshall Islands, Commonwealth Scientific and Industrial Research Organization, Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), National Institute of Water and Atmospheric Research, National Marine Sanctuary of American Samoa, Secretariat of the Pacific Regional Environment Programme, University of South Pacific, Vanuatu Meteorology and Geo-Hazards Department, the Pacific Island Regional Meteorological Service Directors, and our many other regional and in-country partners. Special thanks to Anthony Arguez for his support and edits of this manuscript as well as the insightful discussion on the topic of user engagement. Additionally, the authors would like to acknowledge and thank the reviewers of this manuscript, whose comments and suggestions strengthened the discussion.
REFERENCES
Beierle, T. C., 2002: The quality of stakeholder-based decisions. Risk Anal., 22, 739–749, doi:10.1111/0272-4332.00065.
Bickmore, T., and J. Cassell, 2001: Relational agents: A model and implementation of building user trust. Proc. SIGCHI Conf. on Human Factors in Computing Systems, Seattle, WA, Association for Computing Machinery, 396–403, doi:10.1145/365024.365304.
Bush, V., 1945: Science the Endless Frontier. U.S. Government Printing Office, 184 pp.
Carney, S., L. Whitmarsh, S. A. Nicholson-Cole, and S. Shackley, 2009: A dynamic typology of stakeholder engagement within climate change research. Tyndall Centre for Climate Change Research Working Paper 128, 47 pp., http://www.tyndall.ac.uk/sites/default/files/publications/twp128.pdf.
Chilvers, J., 2008: Deliberative competence: Theoretical and practitioner perspectives on effective participatory appraisal practice. Sci. Technol. Hum. Values, 33, 155–185, doi:10.1177/0162243907307594.
Clark, W. C., L. van Kerkhoff, L. Lebel, and G. C. Gallopin, 2016: Crafting usable knowledge for sustainable development. Proc. Natl. Acad. Sci. USA, 113, 4570–4578, doi:10.1073/pnas.1601266113.
Dilling, L., and M. C. Lemos, 2011: Creating usable science: Opportunities and constraints for climate knowledge use and their implications for science policy. Global Environ. Change, 21, 680–689, doi:10.1016/j.gloenvcha.2010.11.006.
Fazey, I., and Coauthors, 2014: Evaluating knowledge exchange in interdisciplinary and multi-stakeholder research. Global Environ. Change, 25, 204–220, doi:10.1016/j.gloenvcha.2013.12.012.
Finucane, M. L., R. Miller, L. K. Corlew, and V. W. Keener, 2013: Understanding the climate-sensitive decisions and information needs of freshwater resource managers in Hawaii. Wea. Climate Soc., 5, 293–308, doi:10.1175/WCAS-D-12-00039.1.
Freeman, R. E., 1984: Strategic Management: A Stakeholder Approach. Pitman, 276 pp.
Gibbons, M., 2000: Mode 2 society and the emergence of context-sensitive science. Sci. Public Policy, 27, 159–163, doi:10.3152/147154300781782011.
Gibbons, M., C. Limoges, H. Nowotny, S. Schartzman, P. Scott, and M. Trow, 1994: The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies. Sage Press, 179 pp.
Jacobs, K., 2002: Connecting science, policy, and decision-making: A handbook for researchers and science agencies. NOAA Office of Global Programs, 30 pp.
Karlsen, J. T., K. Graee, and M. J. Massaoud, 2008: Building trust in project-stakeholder relationships. Balt. J. Manage., 3, 7–22, doi:10.1108/17465260810844239.
Kirchhoff, C. J., M. C. Lemos, and S. Dessai, 2013: Actionable knowledge for environmental decision-making: Broadening the usability of climate science. Annu. Rev. Environ. Resour., 38, 393–414, doi:10.1146/annurev-environ-022112-112828.
Lemos, M. C., and B. J. Morehouse, 2005: The co-production of science and policy in integrated climate assessments. Global Environ. Change, 15, 57–68, doi:10.1016/j.gloenvcha.2004.09.004.
Meadow, A. M., D. B. Ferguson, Z. Guido, A. Horangic, G. Owen, and T. Wall, 2015: Moving toward the deliberate coproduction of climate science knowledge. Wea. Climate Soc., 7, 179–191, doi:10.1175/WCAS-D-14-00050.1.
NOAA, 2004: Stakeholder participation: A synthesis of current literature. NOAA/National Marine Protected Areas Center Rep., 26 pp., http://marineprotectedareas.noaa.gov/pdf/publications/Stakeholder_Synthesis.pdf.
NOAA, 2015: Pacific Islands climate storybook. NOAA/USAID Rep., 77 pp., http://pacificislandsclimate.org/storybook/Pacific_Islands_Climate_Storybook.pdf.
Pirk, S., 2002: Expanding public participation in environmental justice: Methods, legislation, litigation and beyond. J. Environ. Law Litigation, 17, 207–240.
Pohl, C., 2007: From science to policy through transdisciplinary research. Environ. Sci. Policy, 11, 46–53, doi:10.1016/j.envsci.2007.06.001.
Reed, M. S., L. C. Stringer, I. Fazey, A. C. Evely, and J. H. J. Kruijsen, 2014: Five principles for the practice of knowledge exchange in environmental management. J. Environ. Manage., 146, 337–345, doi:10.1016/j.jenvman.2014.07.021.
Rickinson, M., J. Sebba, and A. Edwards, 2011: Improving Research through User Engagement. Routledge, 143 pp.
Ripberger, J. T., H. C. Jenkins-Smith, C. L. Silva, D. E. Carlson, and M. Henderson, 2014: Social media and severe weather: Do tweets provide a valid indicator of public attention to severe weather risk communication? Wea. Climate Soc., 6, 520–530, doi:10.1175/WCAS-D-13-00028.1.
SPREP, 2016: Vanuatu framework for climate services. National Institute of Water and Atmospheric Research/Vanuatu Meteorological and Geo-Hazards Department, Government of the Republic of Vanuatu, NIWA Client Rep. 2016022WN, 67 pp.
Studd, K., 2002: An introduction to deliberative methods of stakeholder and public participation. English Nature Rep. 474, 65 pp.
Werner, K., and Coauthors, 2014: California drought 2014 service assessment. U.S. Department of Commerce/National Oceanic and Atmospheric Administration Rep., 72 pp., http://www.nws.noaa.gov/om/assessments/pdfs/drought_ca14.pdf.
