1. Introduction
As we come to grips with the impacts of climate change on our natural and cultural resources, our cities and towns, and our personal health and well-being, the production of “usable” climate knowledge—information that can help inform management, planning, and governance—has become a goal for many scientists, agencies, and governments. One promising way to develop usable climate knowledge is to coproduce it. Coproduction of knowledge is the process of producing usable, or actionable, science through collaboration between scientists and those who use science to make policy and management decisions. Coproduction involves collaborations between scientists and decision makers to frame research questions, decide how to answer the questions, and analyze the findings (Lemos and Morehouse 2005). Research on the outcomes of collaborations between scientists and decision makers has shown that when knowledge is coproduced it is more likely to be accepted and used by decision makers. By participating in its production, the information becomes more transparent to end users (Jasanoff and Wynne 1998); the process by which the information is produced is perceived to be more legitimate (Cash et al. 2006); the information is more likely to be at spatial and temporal scales useful to decision makers (Dilling and Lemos 2011); the knowledge is easier to integrate with existing information because it fits into the decision framework of the agency or organization (Carbone and Dow 2005; Lemos et al. 2012); and the end users gain a greater sense of ownership over the final product because they have contributed to it (Robinson and Tansey 2006). Because coproduction of knowledge takes time and resources to do well and is a process that is not well understood there are currently a limited numbers of scientists who undertake it (Cvitanovic et al. 2015; Shanley and López 2009), contributing to a gap between the number of people producing usable climate science and the demand from users for that information (Dilling and Lemos 2011; Lemos et al. 2014).
This paper aims to help close that gap by presenting several examples of modes of engagement and collaborative research approaches that can be used to structure a process of coproduction of climate science. Mode of engagement refers to the basic character of the interactions between scientists and decision makers. Research approaches mean a set of guidelines and activities designed to guide collaborative processes and data collection methods in order to achieve the overall research goals. We hope to lower the barriers to coproduction of knowledge by framing these approaches—action research (AR), transdisciplinarity, rapid assessment process (RAP), participatory integrated assessment (PIA), and boundary organizations—as tools to help guide and support researchers undertaking this challenging, yet rewarding, research. Several of these approaches have been used in climate science knowledge production (transdisciplinarity, PIA, and boundary organizations) and two (AR and RAP) are more generally associated with social science or development work. All of these approaches have been tested over time and in various contexts and been shown to be effective in engaging community members and decision makers in research processes.
The key elements in a successful coproduction process have been identified generally as building ongoing relationships between scientists and stakeholders, ensuring two-way communication between the groups, and maintaining a focus on the production of usable science (Dilling and Lemos 2011; Lemos and Morehouse 2005; National Research Council 2009). The factors required to support these activities are typically ensuring that the science team has the technical and disciplinary capabilities to answer the question; ensuring that both groups have the ability to facilitate the relationship; and ensuring that the science team has the resources (money, time, people) to complete the work in a timely and effective manner (Cvitanovic et al. 2015; Shanley and López 2009). However, confusion remains about exactly what should occur in a coproduction process to yield actionable science—what coproduction actually “looks like”—and why seemingly actionable science is not always used by decision makers. Although many factors may influence the use, or lack of use, of climate science in decision making, we propose that one factor inhibiting its use is that the knowledge is not genuinely being coproduced. Rather, researchers and decision makers may be interacting to some degree, but that interaction may be fairly superficial and may not be sufficient to result in coproduction of knowledge (Pregernig 2006). In other words, there has been too little attention given to planning for and execution of intensive and effective collaborative research activities that can lead to the coproduction of usable climate science.
Research on public participation in policy making has demonstrated that the ways in which participatory processes occur matter to the outcomes. More extensive engagement, such as through negotiation and mediation activities, tends to lead to higher-quality policy decisions, while cursory public or management input at meetings does not (Beierle 2002; Rowe and Frewer 2005). However, inexperience among researchers, insufficient resources, or a lack of clear guidance on best practices in collaborative knowledge production—or a combination of these and other factors—may hinder efforts to coproduce usable climate science. We argue that one way to improve the process of coproduction is to follow an established collaboration protocol, grounded in participatory research literature, because it can provide guidance on how to plan and manage collaborative activities, frameworks in which to examine stakeholders’ decision contexts and concerns, and guidance on resources required to undertake collaborative research; it can also create better opportunities to evaluate the effectiveness of coproduced knowledge through comparative analysis. Using an established approach can help ensure that we “get the right participation and get the participation right” (Stern and Fireberg 1996). We present a small sample of approaches that can be applied to collaborative climate research. This is not an exhaustive list, but the approaches have been selected to present a range of possible ways to structure a coproduction process, depending on the research question, strengths of the research team, available resources, and the needs of the stakeholders.
2. An evolution in thinking about the dialogue between science and policy: How did we get to the idea of coproduction?
In the decades following World War II, U.S. science policy was heavily influenced by Vannevar Bush’s report titled “Science, the Endless Frontier” (Bush 1960). Bush articulated a vision for the contribution of scientific knowledge to society wherein “basic” research generated new knowledge and “applied” research found practical applications for that knowledge. This reasoning resulted in a linear model of science policy through which knowledge was generated in one domain (science) and then handed off to a recipient domain (society). The two sectors were intentionally isolated (Byerly and Pielke 1995; Pielke 1997; Stokes 1997) in order to insulate science from the value-laden world of applications.
By the early 1970s this linear model of science was critiqued as insufficient for dealing with complex, “wicked” problems that require scientific knowledge but also “rely upon elusive political judgment for resolution” (Rittel and Webber 1973, p. 160). Environmental issues, including climate change, are often cited as the epitome of wicked problems because they involve differing values that result in conflicts that cannot be solved by the simple application of scientific knowledge. As Ludwig (2001, p. 763) noted, “[t]here are no experts on these problems, nor can there be.” Instead, the new reasoning goes, we should establish and maintain a dialogue among the various interested parties, creating a process “in which scientific expertise takes its place at the table with local and environmental concerns” in order to achieve creative solutions to complicated problems (Funtowicz and Ravetz 1993).1 When the need for more inclusive science production processes was recognized, the door was opened for a more integrated approach to addressing complex problems: intentionally bringing together science and other knowledge systems (Cornell et al. 2013).
Application of coproduction to climate science knowledge
In an attempt to understand the roles of science in society and society in science, Jasanoff and Wynne (1998) examined several developments in science and technology to demonstrate the ways in which those developments were the product of an “interplay of scientific discovery and description with other political, economic, and social forces” (p. 4). They noted that this process, which they called “co-production” of knowledge, did not represent a tainting of pure scientific discovery by external influences, but rather was a more accurate representation of the ways in which knowledge (particularly, knowledge useful for policy action) is simultaneously constructed and influenced by the society and culture in which it is developed. Lövbrand (2011) labeled this “descriptive co-production” because it described an existing phenomenon. Jasanoff and Wynne (1998) suggested that more generally accepted scientific explanations about the world, in particular about climate change, would emerge “through inclusion rather than exclusion, through participation rather than mystification, and through transparency rather than black boxing” (p. 77).
The descriptive framework created by Jasanoff and Wynne was reframed as a model for improved science and policy development by, among others, Lemos and Morehouse (2005), Dilling and Lemos (2011), and Lemos et al. (2012). Lövbrand (2011) named this new model “prescriptive co-production,” calling it “a normative framework for improved science–society relations” (p. 226). An early example of this new model of coproduction was articulated by Lemos and Morehouse (2005) who identify iterativity in the scientist–stakeholder partnership as the key component in successful coproduction of climate knowledge. Iterativity depends on three components: 1) repeated interaction with stakeholders, including during problem definition, research, analysis, and testing results; 2) production of usable science, including making the science understandable, available, and accessible to users; and 3) interdisciplinarity, ensuring that the research integrates all the necessary disciplinary knowledge.
Later, Lemos et al. (2012) refined this prescriptive coproduction model to more narrowly focus on the issue of information usability. They noted that the usability of science depends on users’ perception of their information need, how well new knowledge interplays with existing knowledge within the user group, and the level of interaction between knowledge producers and knowledge users. Other factors identified by Lemos et al. (2012) that improve the usability of climate science are two-way communication between the groups and establishment of an ongoing relationship between the groups, both of which increase the information users’ perception of information salience, credibility, and legitimacy, and can address users’ concerns about scientific uncertainty.
While the newer prescriptive models outline basic goals or tenets of how to conduct collaborative and usable research, the actual processes by which these activities are undertaken is not well documented. The ways in which collaboration is conducted, decision makers are identified, questions are articulated, and iterativity is achieved are important to the ultimate goal of the production of usable knowledge. Research on public participation in policy and decision making has demonstrated that the structure and implementation of participatory activities impacts the outcome of the collaboration (Beierle 2002; Rowe and Frewer 2005; Stern and Fireberg 1996). Good integration of decision makers’ knowledge into science and the scientists’ knowledge into policy or management requires a strong process, designed around specific collaborative goals, that is executed effectively. We describe this process as deliberate coproduction, which involves explicitly planning coproduction into research processes and applying the best practices in collaborative research to achieve usable science.
3. Modes and approaches to deliberately coproduce climate science knowledge
We present four overarching modes of engagement, as defined by Biggs (1989), that outline types of relationships between researchers and stakeholders: contractual, consultative, collaborative, and collegial. Different modes can accomplish different research objectives and each has different resource and project management requirements, according to Biggs (1989) (Table 1). Understanding first how different types of engagement can support different research objectives (i.e., which research questions require end-user perspectives to find a solution and which can rest on a linear-science model) and second how to plan the required engagement are both critical to the goal of developing usable climate science. Biggs’ (1989) modes of engagement are somewhat oversimplified characterizations of the ways in which scientists and stakeholders work together. In reality, the lines between the modes are fuzzy, which allows for engagement activities and outcomes to apply to more than one mode.2 Nonetheless, the simplified form helps distinguish some general principles for engagement.
We also present five approaches to collaborative research—AR, transdisciplinarity, RAP, PIA, and boundary organizations—that can help researchers and stakeholders work together to achieve the tenets of coproduction and produce the knowledge needed by the stakeholders. The context in which a collaborative effort takes place is critical in the selection of an approach. The type of research question determines the general mode of engagement required; then, the people involved, the resources available, the capacities of the scientists and stakeholders to engage in the process, and the political context in which the work takes place all influence the specific research approach best suited to the inquiry. These factors can change during a research project, and flexibility and willingness to correct course along the way is essential to the process of coproduction of knowledge (McNie 2007).
a. Modes of engagement
Mode of engagement refers to the basic character of the interactions between scientists and decision makers: Is the engagement egalitarian? Is the communication two-way? In which aspects of the research are the stakeholders involved? Who will make the final decisions about research methods and/or policy outcomes? Although Biggs (1989) wrote in the context of agricultural research, his modes of engagement are more broadly applicable because of the general principles he highlighted. He stressed that the modes are distinguished by “differences in objectives and the organizational and managerial arrangements they require for implementation” (Biggs 1989, p. 3), not by their ability to solve problems. Each can solve problems effectively when the mode is appropriate to the particular question, context, and resources available.
In the contractual mode, the research emphasis is on testing or verifying technology. Biggs’ (1989) term “contractual” refers to contracts between scientists and farmers for the use of land, services, and resources to test experimental technology under real-world conditions. It does not refer to situations in which stakeholders contract with scientists to answer stakeholder-driven questions. We liken Biggs’ contractual mode to standard academic research wholly conducted by scientists, albeit with the intension of developing real-world applications.
The consultative mode involves “diagnosis, design, technology development, testing, verification, and diffusion” in order to solve a problem pertinent to the community (Biggs 1989, p. 6). In this mode there is interaction between the scientists and stakeholders at specific stages of the research, such as initial problem definition, verification of results, and diffusion of findings. However, the interaction is not necessarily ongoing throughout the process. Stakeholder input may be facilitated or filtered through a social scientist or other research team member who may act as a science translator, somewhat reducing the opportunity for direct interaction and mutual learning between the science team and the stakeholders.
The collaborative mode involves continuous interaction between scientists and stakeholders, who are seen as partners in the research process (Biggs 1989). This mode focuses on questions that require stakeholder input, such as their local knowledge related to resource use, to answer the broader scientific question. Stakeholders are directly involved in the research and, unlike consultative mode, are more likely to speak for themselves in the process. In this mode, the stakeholders are brought into Western science processes, perhaps even receiving formal training as part of their involvement.
Biggs’ fourth mode is “collegial,” which he defined as the formal research system actively strengthening the informal (stakeholder driven) research and knowledge development system. In other words, not only are researchers pursuing a standard scientific research project, they are also helping to increase the stakeholders’ capacity to design and conduct their own research and solve problems. The collegial mode recognizes that the knowledge gained through local epistemologies is valuable and can support scientist-driven research.
While any of these modes can be used to effectively answer a stakeholder-driven question, we note that the engagement required to call a process coproduction of knowledge—to provide enough engagement for stakeholders to feel that the process has been legitimate [using Cash et al.’s (2006) definition]—is more likely to come from collaborative or collegial modes because these modes include the kind of long-term, two-way relationships that lead to coproduction of knowledge.
Once a researcher and stakeholder have determined the research question and general mode of engagement most suited to the question and context, the next step is to identify a specific research approach that will help them understand each other’s concerns, languages, and collaboratively develop usable knowledge. For each general mode of engagement, there are a number of specific research approaches that can help to achieve these goals.
b. Approaches to collaboration
While Biggs’ modes of engagement provide general guidance on how different levels of engagement support different research objectives, the approaches discussed below provide more detail about how to accomplish the necessary level of engagement. Each approach lays out specific activities and actions that researchers can take to reach both the research and collaboration goals of a given project (Table 2). The importance of process also leads us to stress the importance of interdisciplinary research teams, as suggested by Lemos and Morehouse (2005). In addition to all the scientific disciplines involved in producing climate knowledge, social scientists on the team can be instrumental in framing the collaborative approaches, interviewing stakeholders, elucidating the perspectives of stakeholders (Cvitanovic et al. 2014), and encouraging scientists to challenge their own assumptions and biases as they interact with stakeholders and the knowledge the stakeholders bring to the table.
Approaches to collaboration categorized by the mode(s) of engagement they fulfill.
1) Action Research
Action research (AR) is the approach that laid the foundation for collaborative research in the social sciences. As defined by Lewin (1946), AR is a qualitative research approach designed to both solve practical problems and further our generalizable knowledge of societal structures and processes. Lewin directed the method toward communities facing challenging social and economic situations for which no immediate solution was apparent. Lewin recognized that solutions must be meaningful within the context of the community and developed the AR approach to collaborate with community members to frame the inquiry, undertake the research, analyze the findings, and take action. “Together, the professional researcher and the stakeholders define the problems to be examined, cogenerate knowledge about them, learn and execute social research techniques, take actions, and interpret the results of actions based on what they have learned” (Greenwood and Levin 2007, p. 3). The transparency of the AR approach is intended to ensure that stakeholders view the process and outcomes as legitimate and beneficial. While much AR focuses on social issues, some foundational work has occurred in the context of organizational studies (Greenwood et al. 1993; Whyte and Whyte 1991) (see case study below), which may be more pertinent to those working in the context of management agencies and policy making. The role of the academic researcher in AR may be better described as facilitator and teacher, providing technical guidance to community members while allowing for full community control of the information and resulting actions (Greenwood and Levin 2007).
A key tenet of AR is that once the problem has been diagnosed, action must be taken to change the situation and alleviate the problem (Greenwood and Levin 2007) and those actions should be assessed to determine their effectiveness. In the context of coproduction, this can mean taking policy or management action based on research findings then monitoring the outcomes. The interplay between action and reflection defines AR. The roots of AR are as a tool in effecting social change and, as such, it has been called “openly ideological research” (Lather 1986), which some researchers may find problematic because it implies a lack of objectivity. However, AR has been modified over the years for use in less political contexts although the ultimate goal of AR remains that stakeholders take action to address a problem. Researchers can modify their role to support research, reflection, and analysis necessary to this problem solving. Because AR requires that stakeholders drive the entire process from the framing of the problem to research, analysis, and decision making, it fits only the collegial mode of engagement.
Action research case study
In 1980, Xerox and its union workers launched an innovative experiment in participatory action research. The experiment grew out of recognition at Xerox that the market and manufacturing practices were shifting rapidly and that, to remain competitive, they would need to update their practices. Union and management representatives, as well as other employees, were trained as “problem solving teams” (PSTs) that identified problems within the organization and experimented with solutions. When Xerox considered outsourcing the manufacturing of wire harnesses found in some of their products, which was projected to save more than US$3 million dollars per year but would cost 150–180 union jobs, the union asked management for an opportunity to save their jobs by studying the wire harness assembly operation to identify potential cost savings. The employee PST was able to identify surplus costs and make recommendations about cutting them. In the end, the jobs stayed at Xerox and the company saved more than US$4.2 million per year (Pace and Argona 1991). Both management and the union saw this transparent process as legitimate and the outcomes as mutually beneficial.
2) Transdisciplinarity
Transdisciplinarity is a research approach that integrates multidisciplinary academic and practitioner knowledge through specific processes to produce a unified product (Jahn et al. 2012). The term has also been used as a broad theoretical concept to explain how knowledge can be produced and how to make science more interdisciplinary and democratic (Jahn et al. 2012). The goals of transdisciplinary research are to address complex, socially relevant problems (Hirsch Hadorn et al. 2006), reconcile social demand for and academic production of knowledge (Hoffmann-Riem et al. 2008), and build upon and use disciplinary knowledge (Klein 2004) while integrating disciplinary and “extra-scientific” knowledge (Jahn et al. 2012).
Jahn et al. (2012) proposed a conceptual model of transdisciplinary research, identifying three phases. Phase 1 is problem transformation, during which the societal problem is framed then related to scientific knowledge. The social and scientific problems are then linked to form a boundary object3 and finally transformed from a boundary object into epistemic objects, or research questions. In phase 2, interdisciplinary integration, the disciplinary science teams interact with each other in several stages to produce new knowledge related to the research questions. Transdisciplinary integration occurs in phase 3 when the results of the knowledge production are assessed and products are assembled for both science and society. Mauser et al. (2013) developed a similar framework for the use of a transdisciplinary approach in sustainability research identifying the phases as codesign of the research (phase 1), coproduction of knowledge (phase 2), and codissemination of the results (phase 3). In both models, knowledge production is integrated and researchers and practitioners are engaged in each phase. The commitment to integrating science and other forms of knowledge in the transdisciplinary approach, and the sustained interactions it requires, makes transdisciplinarity an example of a research approach in the collegial mode of engagement. However, it differs from AR in that it brings the various participants together to accomplish specific tasks, while AR allows for a more immersive experience in which researchers interact with stakeholders within the stakeholders’ social context, which may allow researchers to develop a deeper understanding of stakeholder needs and knowledge systems.
Transdisciplinarity case study
A transdisciplinary project was undertaken to address the issue of the level of active ingredients in pharmaceuticals for human use (active pharmaceutical ingredient, API) in water in Germany (Jahn et al. 2012). The project began by asking a group of stakeholders including medical and pharmaceutical professionals, public health professionals, and water managers to frame the issue as a societal problem. Next, subject matter experts provided a scientific framing of the issue and decisions about the focus of the project, which was to identify strategies to reduce API in waters but which were also sensitive to the conflicting values inherent in the issue. The project team worked with scientists and stakeholders to create a boundary object, which became the following statement: The occurrence of APIs in communal water cycles is an undesirable side effect of the normal mode of operation in the health care system. Research questions related to risk governance, risk perception, and risk communication were then developed based on the statement. After designing a process by which interdisciplinary integration could occur three project subgroups each developed a strategy to reduce APIs, which were then compiled into a formal document. Project outcomes included adoption of one of the reduction measures by a municipality. The strategic combination of scientists and industry stakeholders resulted in specific strategies to address an immediate environmental harm.
3) Rapid Assessment Process
Rapid assessment process, or RAP (Beebe 2001), is a structured approach to the use of qualitative research methods to identify the “most important elements of the local situation from the perspective of the local participants” (Beebe 2001, p. xvii; emphasis added) and the key terms and categories used by the participants so that problems can be solved in ways that fit within local knowledge frameworks. For example, Cvitanovic et al. (2014) found that natural resource managers do not necessarily consider scientific information to be more important than other knowledge, highlighting the importance of understanding how those managers frame the issues and which knowledge systems they rely on before attempting to develop usable knowledge for them. RAP was designed for use when there is an urgent need for intervention and/or when the resources (time, money, and people) are not available for long-term ethnographic research, in contrast to AR, which relies on long-term immersion in stakeholder communities.
RAP requires multiperson, multidisciplinary teams to enact its two main tenets: triangulation of data and iterative analysis. Teams should also include members of the local community whenever possible. Research teams should draw information from two groups of participants: a convenience sample to gain a broad overview of the issue and key informants selected for their particular knowledge of the situation. Multiperson data collection teams help researchers gain multiple perspectives on the situation and help them avoid missing key details during interviews and observations. Iterative data analysis requires the research team to spend significant amounts of time discussing among themselves what they heard and observed during fieldwork. Beebe (2001) stressed that at least one member of the research team should have training in social science research methods to ensure that inquiries are structured appropriately.
Using a RAP approach can be helpful for climate science research teams seeking to understand the management context in which climate science will be applied. By better understanding organizational functions, information flows within the agency, how decisions are made, and previous experiences with climate information, scientists may be better able to produce climate knowledge more readily usable by resource managers. RAP best represents either the consultative or collaborative modes because while it integrates stakeholder knowledge into the research process, the research team most often performs the analysis and interventions; the local community members or stakeholders are not necessarily part of these tasks. Researchers may find that they still need an approach with more opportunities for in-depth engagement in order to support knowledge coproduction. However, RAP can help lay a strong foundation for a relationship to be built between scientists and stakeholders.
RAP case study
Westphal and Hirsch (2010) used a RAP approach to better understand the attitudes and behaviors of Chicago residents toward climate change as part of the city’s climate action planning process. They found that by deploying a team of researchers to work in a number of Chicago communities they were able to collect a large amount of data from residents in a short amount of time. They paid community members a stipend to help with research activities such as connecting researchers with key community organizations, facilitating focus groups, informing study design, and data analysis. The research team (including community members) used interviews, focus groups, participant observation, and other more novel methods such as drawing exercises to gather information from community members about climate change concerns. Westphal and Hirsch (2010) reported a key outcome of the RAP approach was that neighborhood concerns were placed at the center of discussions so that actions resulting from this project could balance local concerns and broader climate change concerns, reinforcing the sense that local voices had been heard.
4) Participatory Integrated Assessment
Participatory integrated assessment (PIA) is a multidisciplinary approach that seeks to develop policy- or decision-relevant knowledge about environmental problems through the integration of stakeholder knowledge into modeling and scenario-planning efforts (Salter et al. 2010; Toth and Hizsnyik 1998). PIA facilitates the integration of stakeholder knowledge and values into models and scenarios of climate change that can then be used to inform decision-making processes (Salter et al. 2010; van Asselt Marjolein and Rijkens-Klomp 2002). Stakeholders in PIA are can range from policy makers to the affected general public.
PIA frameworks rely on a set of primary disciplinary elements and primary integration tools (Toth and Hizsnyik 1998). Primary disciplinary elements are methods, theories, and models that address the issue of interest such as general circulation models, demographic models, opinion surveys, and participatory models (Toth and Hizsnyik 1998; van Asselt Marjolein and Rijkens-Klomp 2002). Primary integration tools can range from simple flow diagrams to complex network charts, or from plain checklists to impact matrices (Toth and Hizsnyik 1998). Participation mechanisms vary depending on the context and questions but can include workshops for larger, more public groups or focus groups for smaller, more targeted groups of stakeholders. PIA’s focus on integrating a variety of forms of stakeholder knowledge with more standard scientific knowledge and its flexible approach in selecting participants means that it has the potential to be used in consultative, collaborative, or collegial modes. However, there are some limits on the ways in which stakeholder knowledge is likely to be used, due to the focus on technical models and scenarios, which could constrain the ways some stakeholders are able to participate.
Participatory integrated assessment case study
Climate Options for the Long Term (COOL) was a PIA project that was part of the development of long-term climate policy at the Dutch, European, and global scales (Berk et al. 2002). The project was designed as a series of workshops with the objectives of 1) exploring long-term targets for stabilizing greenhouse gas emissions; 2) exploring the most promising options for long-term international climate policy and their implications for the medium term; 3) enhancing the understanding between countries with different positions and interests in climate change; 4) broadening the understanding of scientific aspects of climate issues; and 5) developing common frameworks for analyzing and evaluating policy options (Berk et al. 2002). Utilizing a back-casting methodology, participants developed a potential future scenario based both on models and stakeholder input and reasoned backward to the present to identify policy goals consistent with achieving the future scenario (Salter et al. 2010). Workshop participants included policy makers from both developed and developing countries, stakeholders involved in international climate change policy negotiations, and climate scientists. Stakeholders were mainly involved in option assessment, goal setting, and strategy formulation (Kloprogge and van der Sluijs 2006). The project resulted in the development of strategies and policy goals for technological adaptations to meet an 80% reduction in greenhouse gas emissions for the Netherlands (Salter et al. 2010). Targeted stakeholder input helped the participants agree on a future scenario goal and created buy-in on strategies to achieve the mutual goal.
5) Boundary Organizations
A boundary organization is a group or institution that takes on the challenging tasks of both working at and managing the science–policy boundary (Guston 2001). The role of a boundary organization is to facilitate the process of coproduction by allowing scientists and decision makers to maintain their independence and objectivity while also creating some permeability of the boundary to allow for coproduction of knowledge (Clark et al. 2011). To be successful in managing the boundary, these organizations take on four key functions (Cash et al. 2006):
Convening—the process of bringing parties together for face-to-face contact; this forms the foundation for relationships of trust and mutual respect.
Translation—either literally, as from one language to another, or figuratively, as from one side of the boundary to the other.
Collaboration—bringing the actors together in an effort to coproduce knowledge.
Mediation—representing and evaluating different interests so that mutual gains can be created and the process is perceived as fair and just.
These functions can appear in different mixes in different organizations (Cash et al. 2006). Boundary organizations act as “an intermediary between the users and the scientists, and [are] fluent in both worlds” (Dilling and Lemos 2011, p. 685). Knowledge brokers are individuals who fulfill many of the same functions as boundary organizations, acting as intermediaries between researchers and decision makers (Meyer 2010; Michaels 2009). They may work within boundary organizations or become members of the research team with the specific task of mediating the science–policy boundary.
There are several ways to approach creating or using a boundary organization in the process of knowledge coproduction. A research or policy team can create a new boundary organization to suit a particular project or purpose, which can ensure that information is customized for the intended user (Dilling and Lemos 2011). See the case study (below) for an example of a boundary organization created to facilitate one specific project. A second approach is to use an existing boundary organization to mediate a coproduction process. For example, the NOAA Regional Integrated Sciences and Assessments (RISA) program, established in 1995 with one organization to address a specific regional problem (Pulwarty et al. 2009), now consists of 10 programs, which have the capacity to work with new stakeholders and scientists on a range of projects. Science shops, a European model in which universities support small research groups whose goal is to democratize science by making scientists available to answer community groups’ research questions either free of charge or at reduced rates (Fischer et al. 2004; Gnaiger and Martin 2001), are another type of existing boundary organization. Finally, an existing organization can take on the role of a boundary organization, although Dilling and Lemos (2011) caution that this can require large-scale mission change and is, therefore, not always practical.
Boundary organizations or knowledge brokers do not all work in the same way nor do individual boundary organizations work the same way on each project (Michaels 2009); they have training and experience that helps them select appropriate modes and approaches based on the specific questions and contexts of each individual project. Using a boundary organization, particularly one that is established, allows researchers to connect with experts who can help guide the collaborations and who may be able to use their existing connections to lay the foundation for a new collaboration between researchers and decision makers. Because the goal of a boundary organization is to facilitate collaboration between scientists and stakeholders, they regularly work within the consultative, collaborative, and collegial modes.
Boundary organizations case study
As part of a project to address the combined risks of sea level rise, population growth, and development of economic assets along the Dutch coast, the Dutch government appointed a committee on sustainable coastal development, which functioned as a boundary organization (Boezeman et al. 2013). The committee was made up of both scientists and politicians—members from both sides of the boundary. Within each of these domains, a number of disciplines were represented such as climate science, water engineering, agriculture, politicians, and business representatives, which kept either domain from being “overhomogenised” (Boezeman et al. 2013). The committee members and staff were also well connected outside of the group, which helped them to act as boundary agents with the broader Dutch community. The committee routinely sought opinions and ideas from regional stakeholders, which enabled them to gain the trust of the stakeholders and to refine their recommendations based on stakeholder experiences. Finally, the committee created a boundary object—in this case a report—which was used as a formative tool to vet and debate scientific and other policy-relevant information as well as translate technical information to reach multiple audiences. By working at the intersection of several boundaries (science/policy and general public/policy makers), the committee was able to craft recommendations for a “worst-case” sea level projection that went beyond the then-current IPCC sea level projections, reaching beyond the current scientific consensus to address, through policy, a potentially much more significant threat to the Dutch people.
4. Evaluating coproduction of climate science knowledge
A consistent refrain in the literature on coproduction of knowledge within the climate sciences is the need to assess the impact of the science as well as to understand why and under what conditions the science is or is not used as expected (Bellamy et al. 2001; Fazey et al. 2014). The complexities of evaluating impacts on natural resource management or attributing outcomes directly to any one particular action make it tempting to rely on more easily tracked metrics, such as number of peer-reviewed articles or other research outputs (Bell et al. 2011; Roux et al. 2010). While the scientific credibility afforded by peer review is important to ensure the quality of the science developed through coproduction, usability, which is the intended outcome of coproduction, must be evaluated in new ways more suitable to its unique role in both advancing science and societal outcomes (Bell et al. 2011; Fazey et al. 2014). As Fazey et al. (2014) noted, different types of knowledge exchange (modes of engagement in our terms) require different evaluative approaches. Since engagement and coproduction are processes, we note the importance of evaluation approaches that address process as well as outcomes. To evaluate process, one must understand how and why a particular collaborative approach and mode of engagement are intended to work, which is made easier by using an existing and tested collaborative research approach.
There have been some preliminary steps taken toward evaluating coproduction as a process as well as the desired outcome of that process: usable science. The National Research Council (NRC) developed a set of metrics to evaluate usable science and the processes used to produce it in the U.S. Climate Change Science Program (CCSP) (National Research Council 2005). The NRC metrics consist of process metrics, which include variables such as leadership, priority setting, and promotion of partnerships; input metrics such as sufficient intellectual and technological foundation to support the research and sufficient resources to complete the program; output metrics, which include peer-reviewed results that are also broadly accessible to users; outcome metrics such as improved scientific understanding and operational use of the results; and impact metrics, which measure long-term impacts such as an increase in the public understanding of climate issues. While the NRC metrics can be helpful in framing the kinds of questions that are necessary to assess the success of a coproduction of knowledge process, they fall short in terms of closely examining the process by which new knowledge is produced or coproduced. The process metrics focus on the presence or absence of various resources or activities (a leader with sufficient authority, development of a multiyear plan, a strategy for setting priorities and allocating resources, for example) but do not address how those resources are used or activities are undertaken.
Dilling and Lemos (2011) provided more detailed suggestions such as focusing on outcomes like the scientist–stakeholder relationship, the accessibility of the science knowledge produced, and progress on specific societal outcomes. As discussed above, Lemos et al. (2012) posited that the three key variables in the successful production of usable science are users’ perception of the information’s fit to their needs, how well the new information fits within their existing knowledge frameworks (interplay), and the level and quality of interaction between science producers and science users. Kirchhoff et al. (2013) identified two-way communication and long-term relationships as keys to successful coproduction because they allow for trust building and accountability, which increases users’ perceptions of information salience, credibility, and legitimacy.
Reed et al. (2014) distilled a set of five principles for effective knowledge exchange from a broad review of literature and expert interviews. They found that effective knowledge exchange requires that the process be designed into the research project, that stakeholders should be systematically selected to ensure accurate representation, that long-term relationships should be built on two-way communication and cogeneration of knowledge, that the focus should be on tangible, timely results, and that researchers should reflect on their work and refine their practice.
One example of an effort to apply these kinds of evaluation measures to a specific boundary organization comes from the Pacific RISA (Table 3). The program has identified indicators and metrics similar to those suggested in the literature above. They track partnerships and collaborations to gauge both the reach of their partnerships by counting how many stakeholders they work with and who they are missing in their collaborations as well as tracking the level and quality of those relations through qualitative descriptions of stakeholder roles and involvement.
Example of metrics developed to assess scientist–stakeholder collaboration, adapted from Ferguson et al. (2015).
5. Conclusions
The research on coproduction of knowledge has found that greater engagement between scientists and stakeholders tends to produce more usable science because engagement engenders trust in both the science and the science producers (Dilling and Lemos 2011; Lemos et al. 2012). The crucial next step in making coproduction of knowledge a more widely accepted and used approach to creating usable (and used) science is to refine our understanding, through empirical study, of what specific actions and activities most effectively produce the trusting, long-term relationships necessary to the coproduction of usable science. In other words, if we are more deliberate in how we coproduce knowledge and in how we assess the processes and outcomes involved, we can speed the process of learning and be more effective in coproducing climate science. We believe that by using established approaches, such as those described above, we stand a better chance of creating processes in which we can effectively establish working partnerships between scientists and stakeholders. Using and evaluating existing approaches may also help us develop new approaches, through iterative testing, that prove particularly effective in the climate science community.
The approaches discussed here provide frameworks to help both scientists and decision makers better understand the needs of and challenges facing their partners. It is crucial, however, that attention be given to how the approaches are undertaken. Researchers interested in coproduction of knowledge or other forms of collaborative research should reflect upon the questions being raised by decision makers, the context in which those questions arise, and the resources available to answer the questions. The answers to these questions will determine which mode of engagement and research approach will be most effective in any given project.
In much the same way that descriptive coproduction notes the interplay between science and society, deliberate coproduction should be an interplay between social science and physical or natural science. Social science can help structure and guide the ways in which the physical or natural science is deployed in search of policy or resource management answers. The social science practice of researcher reflection can also be considered an integral part of coproduction of knowledge, encouraging researchers to reflect upon their experiences, their challenges, and their successes. Lessons learned from one project can then be consciously applied to another coproduction process.
More research focused on the outcomes of collaborative knowledge production can also help move the field forward. Case studies describing how particular projects were structured, detailing both challenges and successes, and describing whether or how new climate science has been integrated into management decision making can help future researchers and stakeholders better understand the dynamics of collaboration and set reasonable goals for the use of new knowledge. Broader investigations of collaborative approaches using common evaluative frameworks will allow us to be rigorous in the ways we identify the specific elements that contribute most directly to coproduction of knowledge and usable science.
The context in which scientists and stakeholder collaborate, the questions being asked, the approach taken to build the partnership, and the specific actions and activities used to further the collaboration will all impact the outcome of the production of usable science. By being deliberate about our approaches to collaboration and reflecting upon on our practices we can advance the practice of knowledge coproduction, better integrate science and decision making, and address some of the most urgent environmental challenges of our time.
Acknowledgments
The authors thank three anonymous reviewers who provided excellent feedback and suggestions on this paper. This work was supported by the Department of the Interior Southwest Climate Science Center Award G13AC00326 and the National Oceanic and Atmospheric Administration’s Climate Program Office through Grant NA12OAR4310124 with the Climate Assessment for the Southwest program at the University of Arizona and Grant NA11OAR4310150 with the California Nevada Applications Program at the Desert Research Institute.
REFERENCES
Beebe, J., 2001: Rapid Assessment Process. Altamira Press, 224 pp.
Beierle, T. C., 2002: The quality of stakeholder-based decisions. Risk Anal., 22, 739–749, doi:10.1111/0272-4332.00065.
Bell, S., Shaw B. , and Boaz A. , 2011: Real-world approaches to assessing the impact of environmental research on policy. Res. Eval., 20, 227–237, doi:10.3152/095820211X13118583635792.
Bellamy, J. A., Walker D. H. , McDonald G. T. , and Syme G. J. , 2001: A systems approach to the evaluation of natural resource management initiatives. J. Environ. Manage., 63, 407–423, doi:10.1006/jema.2001.0493.
Berk, M. M., van Minnen J. G. , Metz B. , Moomaw W. , den Elzen M. G. J. , van Vuuren D. P. , and Gupta J. , 2002: Climate Options for the Long term (COOL) Global Dialogue - Synthesis Report. RIVM Rep. 490200003, 57 pp. [Available online at http://pblweb10.prolocation.net/sites/default/files/cms/publicaties/490200003.pdf.]
Biggs, S. D., 1989: Resource-poor farmer participation in research: A synthesis of experiences from nine national agricultural research systems. International Service for National Agricultural Research, 37 pp.
Boezeman, D., Vink M. , and Leroy P. , 2013: The Dutch Delta Committee as a boundary organization. Environ. Sci. Policy, 27, 162–171, doi:10.1016/j.envsci.2012.12.016.
Bush, V., 1960: Science, the endless frontier: A report to the President on a program for postwar scientific research. 220 pp. [Available online at https://www.nsf.gov/od/lpa/nsf50/vbush1945.htm.]
Byerly, R., and Pielke R. A. , 1995: The changing ecology of United States science. Science, 269, 1531–1532, doi:10.1126/science.269.5230.1531.
Carbone, G. J., and Dow K. , 2005: Water resource management and drought forecasts in South Carolina. J. Amer. Water Resour. Assoc., 41, 145–155, doi:10.1111/j.1752-1688.2005.tb03724.x.
Cash, D. W., Borck J. C. , and Patt A. G. , 2006: Countering the loading-dock approach to linking science and decision making—Comparative analysis of El Niño/Southern Oscillation (ENSO) forecasting systems. Sci. Technol. Hum. Values, 31, 465–494, doi:10.1177/0162243906287547.
Clark, W. C., Tomich T. P. , van Noordwijk M. , Guston D. , Catacutan D. , Dickson N. M. , and McNie E. , 2011: Boundary work for sustainable development: Natural resource management at the Consultative Group on International Agricultural Research (CGIAR). Proc. Natl. Acad. Sci. USA, doi:10.1073/pnas.0900231108.
Cornell, S., and Coauthors, 2013: Opening up knowledge systems for better responses to global environmental change. Environ. Sci. Policy, 28, 60–70, doi:10.1016/j.envsci.2012.11.008.
Cvitanovic, C., Marshall N. A. , Wilson S. K. , Dobbs K. , and Hobday A. J. , 2014: Perceptions of Australian marine protected area managers regarding the role, importance, and achievability of adaptation for managing the risks of climate change. Ecol. Soc., 19, 33, doi:10.5751/ES-07019-190433.
Cvitanovic, C., Hobday A. J. , van Kerkhoff L. , and Marshall N. A. , 2015: Overcoming barriers to knowledge exchange for adaptive resource management; the perspectives of Australian marine scientists. Mar. Policy, 52, 38–44, doi:10.1016/j.marpol.2014.10.026.
Dilling, L., and Lemos M. C. , 2011: Creating usable science: Opportunities and constraints for climate knowledge use and their implications for science policy. Global Environ. Change, 21, 680–689, doi:10.1016/j.gloenvcha.2010.11.006.
Fazey, I., and Coauthors, 2014: Evaluating knowledge exchange in interdisciplinary and multi-stakeholder research. Global Environ. Change, 25, 204–220, doi:10.1016/j.gloenvcha.2013.12.012.
Ferguson, D. B., Finucane M. L. , Keener V. W. , and Owen G. , 2015: Evaluation to advance science policy: Lessons from Pacific RISA and CLIMAS. Climate in Context, A. Parris, and G. Garfin, Eds., Wiley-Blackwell, in press.
Fischer, C., Leydesdorff L. , and Schophaus M. , 2004: Science shops in Europe: The public as stakeholder. Sci. Public Policy, 31, 199–211, doi:10.3152/147154304781780028.
Funtowicz, S. O., and Ravetz J. R. , 1993: Science for the post-normal age. Futures, 25, 739–755, doi:10.1016/0016-3287(93)90022-L.
Gnaiger, A., and Martin E. , 2001: Science shops: Operational options. SCIPAS Rep. 1, 125 pp. [Available online at http://www.livingknowledge.org/livingknowledge/science-shops/projects/scipas.]
Greenwood, D. J., and Levin M. , 2007: Introduction to Action Research: Social Research for Social Change. 2nd ed. Sage Publications, 299 pp.
Greenwood, D. J., Whyte W. F. , and Harkavy I. , 1993: Participatory action research as a process and as a goal. Hum. Relat., 46, 175–192, doi:10.1177/001872679304600203.
Guston, D. H., 2001: Boundary organizations in environmental policy and science: An introduction. Sci. Technol. Hum. Values, 26, 399–408, doi:10.1177/016224390102600401.
Hirsch Hadorn, G., Bradley D. , Pohl C. , Rist S. , and Wiesmann U. , 2006: Implications of transdisciplinarity for sustainability research. Ecol. Econ., 60, 119–128, doi:10.1016/j.ecolecon.2005.12.002.
Hoffmann-Riem, H., and Coauthors, 2008: Ideas of the handbook. Handbook of Transdisciplinary Research, G. Hirsch Hadorn, et al., Eds., Springer, 3–17.
Jahn, T., Bergmann M. , and Keil F. , 2012: Transdisciplinarity: Between mainstreaming and marginalization. Ecol. Econ., 79, 1–10, doi:10.1016/j.ecolecon.2012.04.017.
Jasanoff, S., and Wynne B. , 1998: Science and decisionmaking. Human Choice and Climate Change, S. Rayner and E. L. Malone, Eds., Battelle Press, 1–87.
Kirchhoff, C. J., Lemos M. C. , and Dessai S. , 2013: Actionable knowledge for environmental decision maker: Broadening the usability of climate science. Annu. Rev. Environ. Res., 38, 393–414, doi:10.1146/annurev-environ-022112-112828.
Klein, J. T., 2004: Prospects for transdisciplinarity. Futures, 36, 515–526, doi:10.1016/j.futures.2003.10.007.
Kloprogge, P., and van der Sluijs J. P. , 2006: The inclusion of stakeholder knowledge and perspectives in integrated assessment of climate change. Climatic Change, 75, 359–389, doi:10.1007/s10584-006-0362-2.
Lather, P., 1986: Issues of validity in openly ideological research: Between a rock and a soft place. Interchange, 17, 63–84, doi:10.1007/BF01807017.
Lemos, M. C., and Morehouse B. J. , 2005: The co-production of science and policy in integrated climate assessments. Global Environ. Change, 15, 57–68, doi:10.1016/j.gloenvcha.2004.09.004.
Lemos, M. C., Kirchhoff C. J. , and Ramprasad V. , 2012: Narrowing the climate information usability gap. Nat. Climate Change, 2, 789–793, doi:10.1038/nclimate1614.
Lemos, M. C., Kirchhoff C. J. , Kalafatis S. E. , Scavia D. , and Rood R. B. , 2014: Moving climate information off the shelf: Boundary chains and the role of RISAs as adaptive organizations. Bull. Amer. Meteor. Soc., 6, 273–285, doi:10.1175/WCAS-D-13-00044.1.
Lewin, K., 1946: Action research and minority problems. J. Soc. Issues, 2, 34–46, doi:10.1111/j.1540-4560.1946.tb02295.x.
Lövbrand, E., 2011: Co-producing European climate science and policy: A cautionary note on the making of useful knowledge. Sci. Public Policy, 38, 225–236, doi:10.3152/030234211X12924093660516.
Ludwig, D., 2001: The era of management is over. Ecosystems, 4, 758–764, doi:10.1007/s10021-001-0044-x.
Mauser, W., Klepper G. , Rice M. , Schmalzbauer B. S. , Hackmann H. , Leemans R. , and Moore H. , 2013: Transdisciplinary global change research: The co-creation of knowledge for sustainability. Curr. Opin. Environ. Sustainability, 5, 420–431, doi:10.1016/j.cosust.2013.07.001.
McNie, E. C., 2007: Reconciling the supply of scientific information with user demands: An analysis of the problem and review of the literature. Environ. Sci. Policy, 10, 17–38, doi:10.1016/j.envsci.2006.10.004.
Meyer, M., 2010: The rise of the knowledge broker. Sci. Commun., 32, 118–127, doi:10.1177/1075547009359797.
Michaels, S., 2009: Matching knowledge brokering strategies to environmental policy problems and settings. Environ. Sci. Policy, 12, 994–1011, doi:10.1016/j.envsci.2009.05.002.
National Research Council, 2005: Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. National Academies Press, 162 pp.
National Research Council, 2009: Informing Decisions in a Changing Climate. National Academies Press, 200 pp.
Pace, L. A., and Argona D. R. , 1991: Participatory action research: A view from Xerox. Participatory Action Research, W. F. Whyte, Ed., Sage, 56–69.
Pielke, R. A., 1997: Asking the right questions: Atmospheric sciences research and societal needs. Bull. Amer. Meteor. Soc., 78, 255–264, doi:10.1175/1520-0477(1997)078<0255:ATRQAS>2.0.CO;2.
Pregernig, M., 2006: Transdisciplinarity viewed from afar: science-policy assessments as forums for the creation of transdisciplinary knowledge. Sci. Public Policy, 33, 445–455, doi:10.3152/147154306781778867.
Pulwarty, R. S., Simpson C. , and Nierenberg C. R. , 2009: The Regional Integrated Sciences and Assessments (RISA) Program: Crafting effective assessments for the long haul. Integrated Regional Assessment of Global Climate Change, C. G. Knight, and J. Jager, Eds., Cambridge University Press, 367–393.
Reed, M. S., Stringer L. C. , Fazey I. , Evely A. C. , and Kruijsen J. H. J. , 2014: Five principles for the practice of knowledge exchange in environmental management. J. Environ. Manage., 146, 337–345, doi:10.1016/j.jenvman.2014.07.021.
Rittel, H. W. J., and Webber M. M. , 1973: Dilemmas in a general theory of planning. Policy Sci., 4, 155–169, doi:10.1007/BF01405730.
Robinson, J., and Tansey J. , 2006: Co-production, emergent properties and strong interactive social research: The Georgia Basin Futures Project. Sci. Public Policy, 33, 151–160, doi:10.3152/147154306781779064.
Roux, D. J., Stirzaker R. J. , Breen C. M. , Lefroy E. C. , and Cresswell H. P. , 2010: Framework for participative reflection on the accomplishment of transdisciplinary research programs. Environ. Sci. Policy, 13, 733–741, doi:10.1016/j.envsci.2010.08.002.
Rowe, G., and Frewer L. J. , 2005: A typology of public engagement mechanisms. Sci. Technol. Hum. Values, 30, 251–290, doi:10.1177/0162243904271724.
Salter, J., Robinson J. , and Wiek A. , 2010: Participatory methods of integrated assessment—A review. Wiley Interdiscip. Rev.: Climate Change, 1, 697–717, doi:10.1002/wcc.73.
Shanley, P., and López C. , 2009: Out of the loop: Why research rarely reaches policy makers and the public and what can be done. Biotropica, 41, 535–544, doi:10.1111/j.1744-7429.2009.00561.x.
Star, S. L., and Griesemer J. R. , 1989: Institutional ecology, ‘translations’ and boundary objects: Amateurs and professionals in Berkeley's Museum of Vertebrate Zoology, 1907–39. Soc. Stud. Sci., 19, 387–420, doi:10.1177/030631289019003001.
Stern, P. C., and Fireberg H. V. , Eds., 1996: Understanding Risk: Informing Decisions in a Democratic Society. National Academies Press, 249 pp.
Stokes, D. E., 1997: Pasteur’s Quadrant: Basic Science and Technological Innovation. Brookings Institution Press, 180 pp.
Toth, F., and Hizsnyik E. , 1998: Integrated environmental assessment methods: Evolution and applications. Environ. Model. Assess., 3, 193–207, doi:10.1023/A:1019071008074.
van Asselt Marjolein, B. A., and Rijkens-Klomp N. , 2002: A look in the mirror: Reflection on participation in Integrated Assessment from a methodological perspective. Global Environ. Change, 12, 167–184, doi:10.1016/S0959-3780(02)00012-2.
Westphal, L. M., and Hirsch J. , 2010: Engaging Chicago residents in climate change action: Results from rapid ethnographic inquiry. Cities Environ.,3, 13.
Whyte, W. F., and Whyte K. K. , 1991: Making Mondragon: The Growth and Dynamics of the Worker Cooperative Complex. 2nd ed. ILR Press, 333 pp.
The perception that scientific knowledge is being pushed aside in coproduction processes may also reduce some scientists’ willingness to participate in these efforts. However, as Jasanoff and Wynne (1998) pointed out, the science/society dichotomy is false; the two have always intermingled and the more they do, the greater the opportunities to produce usable science.
We also recognize that the terms used by Biggs are not necessarily clear in the context in which we apply them. For example, Biggs applies the term “collaborative mode” to a specific type of engagement, while we use the term “collaborative” to mean engaged science research more broadly. However, we present Biggs’ terms to provide the reader with the direct link to the history of research on stakeholder-driven research.
Boundary objects are defined by Star and Greisemer (1989) as scientific objects or other materials that are meaningful to and can be understood by the various participants in a transdisciplinary (or other collaborative) research process.
