1. Introduction
Climate assessment reports are large-scale endeavors that evaluate the state of the science to inform the public and policy makers and address specific information or decision needs (National Research Council 2007). Early assessments sought to provide authoritative biogeophysical science on questions surrounding whether climate change was a serious physical phenomenon, its causes, and consequences. Social and behavioral sciences are essential to understanding climate change drivers, impacts, and responses (Agrawal et al. 2012; Fischhoff 2020; Jorgenson et al. 2019; Thomas et al. 2019). Scientific questions and public information needs about climate change have evolved over time. Assessments have changed in response to these needs. They now examine the science available to inform decisions at different administrative scales (e.g., federal, state, municipal), for example. However, the biogeophysical sciences continue to predominate. Assessments have “approached their task beginning with the climate science and connecting through resource impacts to the rest of society” (Liverman 2016). Many in the research community have articulated the need to increase integration of the social sciences into global change assessments (Agrawal et al. 2012; Hackmann et al. 2014; Moser et al. 2016; National Academies of Sciences, Engineering, and Medicine 2016a; National Research Council 2007; Reid et al. 2009; Weaver et al. 2014; Yearley 2009). Social science inclusion is necessary for more comprehensive scientific assessment of how climate, ecological systems, the built environment, infrastructure, and human social systems (i.e., institutions, social norms, culture, markets, language, governance, relationships, property rights, etc.) intersect; it also facilitates the production of more actionable research (Agrawal et al. 2012; Dilling and Lemos 2011; Hackmann et al. 2014; Lemos and Morehouse 2005; Palsson et al. 2013; Weaver et al. 2014). Here, we use the term “social sciences” as a catchall for social, behavioral, and economic disciplines (e.g., geography, sociology, anthropology, economics), including applied fields (e.g., communications, public administration). Different types of assessments and chapters within assessments have distinct topical foci and orientation toward decisions, with implications for which disciplinary specialties and types of data are most relevant.
This article provides a fresh perspective on the evolving role of the social sciences in climate assessments by using qualitative research methods to analyze author experiences with writing the United States’ Fourth National Climate Assessment (NCA4). It shifts the conversation from “why” to “how” to better integrate the social sciences into assessments. We adopt Weaver et al.’s (2014) language of striving for “coequal intellectual partnerships” between the biogeophysical and social sciences in climate research. We use the term coequality deliberately to provoke reflection on how assessments are structured in ways that privilege the physical over the social (Oreskes 2015). Researchers from the social sciences note that their disciplines often play a secondary, narrow role in climate research as a whole (Bulkeley 2019; Castree 2015; Hackmann et al. 2014; Norgaard 2018; Victor 2015). They are expected to fit into natural science paradigms and data, rather than the reverse (Crate 2011). Instead of working in tandem to analyze drivers, impacts, and responses, the social sciences are primarily expected to help assess options for and overcome barriers to climate solutions (Lahsen and Turnhout 2021). Their scope often is limited to “providing the knowledge required to enable society” to take action (Bulkeley 2019). Social scientists have begun to push back against this secondary scope, contemplating what a more evenly balanced partnership in climate research would look like and how to achieve it (Bulkeley 2019; Castree 2015; Hackmann et al. 2014; Norgaard 2018).
The literature to date addresses how epistemological differences between climate scientists and social scientists present obstacles to coequal partnerships (Goldman et al. 2018; Nightingale et al. 2020; Obermeister 2017). By epistemology, we mean the worldviews, assumptions, and practices that underlie how scientists generate knowledge and frame issues. Climate scientists rely on models and measurements to characterize the causes of climate change as anthropogenic forcing; social scientists use sociological and historical evidence to ascribe causation to settler colonialism and industrialization. They also can come to contrasting conclusions about solutions and how to implement them. In addition to epistemology, we argue, attention also must be paid to the institutional context and writing processes specific to assessments that affect social science inclusion. Post hoc evaluations of previous National Climate Assessments (NCAs) note categorical differences in how social and economic topics were treated and make recommendations for improving coverage (Liverman 2016; Morgan et al. 2005). What makes this project distinct is its use of qualitative methods to analyze authors’ lived experiences with writing NCA4. In so doing, it uncovers the role that cultural worldviews and knowledge governance play in shaping how the social sciences are used as they are in the NCA. These cultural worldviews are not limited to epistemologies of how to investigate climate change. They involve intertwined beliefs that those involved with producing NCAs have about the role of social sciences in assessments and the role of assessments in society. Knowledge governance involves the “formal and informal mechanisms that deal with the selection, coverage, regulation, and integration of various forms of knowledge” (Obermeister 2017). It is the means by which cultural worldviews are woven into the institutional apparatus, architecture, and content of assessment reports.
Our analysis sheds light on barriers and opportunities to further integrate the social sciences in future assessments. This project was conducted under the auspices of the Social Sciences Coordinating Committee, an interagency body of the U.S. Global Change Research Program (USGCRP). The committee provides advice to USGCRP on including social science methods, findings, and disciplinary perspectives in its activities. The article proposes potential pathways toward coequal intellectual partnerships in NCAs in ways that consider knowledge governance and worldviews: by expanding the diversity of author expertise, enhancing connections between authors and society, reconsidering report framing, and bridging knowledge systems. It discusses implications for outreach and engagement to foster the report’s impact in society, which was important to authors and part of how they conceived what social science is and does in assessments. The conclusion notes ways in which this project has contributed to changes in the Fifth National Climate Assessment’s (NCA5) writing process.
2. NCA4 institutional context and writing processes
NCAs are quadrennial reports mandated by the Global Change Research Act of 1990 to analyze “the effects of global change on the natural environment, agriculture, energy production and use, land and water resources, transportation, human health and welfare, human social systems, and biological diversity.” USGCRP, which consists of 13 federal agencies, develops the report and submits it to Congress. While Congress is the primary recipient, NCAs are also intended to be used by federal agencies, state and local decision-makers, educators, researchers, journalists, and the private sector.
NCA4 is divided into two volumes: Volume I, the Climate Change Special Report (Wuebbles et al. 2017); and Volume II, Impacts, Risks, and Adaptation in the United States (Reidmiller et al. 2018). This article focuses on Volume II (herein, “NCA4”), which contains 1 overview, 16 sectoral, 10 regional, and 2 response chapters. NCA4 was developed by approximately 300 contributors, about half from federal agencies and half nonfederal experts from academia, state or local government, nonprofits, and the private sector. Chapter teams consisted of federal coordinating lead authors, chapter leads, chapter authors, technical contributors, and review editors. USGCRP appoints federal coordinating lead authors who select chapter leads, who in turn choose chapter authors and technical contributors. Review editors are appointed later in the process by USGCRP.
Each chapter has three primary sections: a background or state of the sector/region, key messages with accompanying narrative text, and traceable accounts. The report uses risk-based framing (RBF) to orient key messages around risks that a changing climate poses to a sector/region and how that risk is managed (National Academies of Sciences, Engineering, and Medicine 2016b; Weaver et al. 2017). Traceable accounts describe the evidence base and assign confidence and likelihood statements to each key message. Chapter teams prepared five official drafts (first through fifth order). Drafts underwent reviews including from the public, USGCRP member agencies, and the National Academies of Sciences, Engineering, and Medicine.
Chapter teams operated largely remotely. Many held in-person or virtual stakeholder engagement workshops to share initial chapter plans and listen to constituencies’ priorities and needs. A three-day, in-person, all-author meeting was attended by 221 authors to finalize revisions to the fourth-order draft after public and National Academies comments. The report was published online in November of 2018. USGCRP and chapter authors shared NCA4 findings at scientific conferences and other venues.
3. Methods
This article presents findings of the Social Sciences Coordinating Committee’s evaluation of social science inclusion in NCA4. Qualitative data collection and analysis enabled us to examine the NCA4 writing processes. We convened two focus groups of NCA4 authors to discuss their experiences with the writing process and perspectives on social science inclusion. The sampling frame was a list shared by USGCRP of NCA4 contributors. We used stratified random sampling to achieve coverage of chapter topics, author roles, and disciplinary expertise, oversampling social scientists to ensure that their perspectives were captured in the data. We categorized authors’ disciplinary expertise using publicly available information. We recruited participants via email and scheduled one session with sectoral chapter authors and one with regional chapter authors. Sessions were held via Zoom for 90 min. The groups had 7 and 12 participants, with about half being federal employees; in total, there were 7 social scientists, 7 biophysical scientists, and 5 with interdisciplinary or practitioner backgrounds. A prepared moderator’s guide ensured consistency between sessions and provided flexibility to probe further on topics that arose in the course of conversation. We recorded sessions, checked transcripts for accuracy, anonymized transcripts, whiteboards, and chats, and coded these data using NVivo and Excel. We developed a codebook using deductive and inductive approaches, adding and revising themes through a modified axial coding process to identify categories and subcategories (Strauss and Corbin 1990). We ensured intercoder reliability by holding team discussions and comparing coding samples. A convergence of themes between the focus groups indicated we had reached data saturation and did not need further sessions.
Qualitative research methods such as focus groups yield insights into how individuals perceive and construct meaning of social phenomena. The research team’s position as NCA4 knowledge producers and evaluators afforded them unique access to participants, ability to generate productive lines of questioning, and interpretive context. It also required that we take measures to avoid partiality (e.g., not exclude potentially unfavorable data). Qualitative methods require the researcher to be close to the data in ways that might introduce bias. We minimized this risk and ensured data quality by practicing reflexivity; being transparent about decisions made and assumptions held; taking a systematic approach to designing, conducting, and analyzing the study; and documenting the research process (Reynolds et al. 2011). Being trained in the practice of research reflexivity, we are aware of our own standpoint about social science and NCAs. What we heard in focus groups surprised us in many cases. We have strived to present findings that stay true to participant sentiments. Sharing results with USGCRP staff and interagency committees has helped us clarify our arguments and provided further data triangulation. Our results are in line with literature on social science integration, lending support for their validity.
In addition to focus groups, we analyzed social science content in NCA4, building on an initial review the Social Sciences Coordinating Committee conducted on a near-final draft. That review evaluated how chapters covered relevant topics and used social science to analyze “human social systems” as per the Global Change Research Act. Our team revisited published chapters and conducted a qualitative thematic analysis with the same lens. Journal articles cited in the assessment were categorized and counted by discipline using the Global Change Information System.
4. Findings
In this section, we discuss focus group findings. Four themes emerged around how and why the social sciences were included as they were in NCA4: authors’ cultural worldviews, author expertise, report framing, and knowledge systems. In addition, we discuss author perspectives on outreach and engagement in the writing process. Stakeholder engagement had a limited effect on social science inclusion but is part of these cultural worldviews about what social science is and does in assessments.
a. Authors’ cultural worldviews
Focus group participants conceptualized what “social sciences” are in relation to how they enhance the report’s impact in society. Participants generally spoke positively about social science, feeling it allowed them to incorporate new literature and topics, explain community context, and include a strong human element in the report. They were enthusiastic when talking about what they saw as social science done well: when chapters addressed social equity and vulnerability; contained key messages that speak to societal values; and included forms of evidence such as Indigenous knowledge (IK), local knowledge (LK), socioeconomic data, and perspectives of underrepresented groups.
They associated social sciences’ positive contribution with the ability to engage readers. Engaging readers emerged as a central concern for participants and the predominant way they understood the social sciences, in relation to communicating about climate change and spurring action. When asked what “social science” and “society and climate” meant in the context of the report, one participant responded, “Interactions and motivations. So, how do people interact with each other, how do they interact with the natural world? And then why do people take the actions that they do and what are the kinds of things that are actually going to motivate their actions in the future?” Another said, “Interpretations, because of the subject we’re dealing with and the difficulty and need to understand who you’re talking to.” The responses center on report audiences’ values, behaviors, motivations, and communication, not on disciplinary approaches to analyzing drivers, impacts, and responses.
Participants wanted NCA4 to represent what people value, describe why it is at risk, and communicate what can be done about it. As one stated, “understanding the pros and cons, the puts and takes, so risk communication for me is a really important component in how social science can actually be meaningful to us.” They cared a great deal about the assessment’s ability to make an impact in society by advancing science, education, and decision-making. They wanted it to be a “one-stop shop of what’s new with climate change” (focus group participant); “a marker and a semi-consistent way of renewing the challenges that we are and will be facing in the near and long term” (focus group participant); and a shaper of research agendas: “I’m kind of hoping that this NCA process might actually motivate people to start to do some of those types of studies [on adaptation]” (focus group participant).
Participants’ principal orientation toward the social sciences was as a means of achieving what they saw as the public function of the assessment above and beyond statutory requirements: to educate, spur action, and inform decisions. They viewed the social sciences as helping them do so by including a broad evidence base, connecting report content to societal values, and providing information about response actions. This finding is in line with other researchers’ observations that the social sciences have a limited scope in climate research (Bulkeley 2019; Castree 2015). The social sciences were also, for some participants, a means of achieving a more transformative role for NCAs. They aspired for it to ask larger questions and change the dialogue, for example, prompting reflection on “what it means to be coincident with the coast as a people” (focus group participant) under high sea level rise scenarios. These authors felt that social science has a key role to play in helping assessments spur societal thinking and decision-making to be more future oriented and transformative.
b. Author expertise
Our analysis of author disciplines found that there were more biogeophysical than social scientists on chapter teams; economists were the most representative of the social sciences (Figs. 1a,b). This breakdown mirrors that of Intergovernmental Panel on Climate Change (IPCC) Working Groups (Obermeister 2017; Victor 2015). NCA4 chapter leadership was also unevenly divided among disciplines, with social scientists composing 2 of 32 chapter leads and 2 of 37 coordinating lead authors.
(a) NCA4 authors by discipline. (b) NCA4 social scientist authors by discipline, as categorized by the authors of this study. In (a), “other” refers to instances in which authors had interdisciplinary or practitioner backgrounds, e.g., sustainability or environmental policy. In (b), “other” refers to other social science disciplines outside those identified in the chart.
Citation: Weather, Climate, and Society 14, 4; 10.1175/WCAS-D-21-0157.1
Focus group participants saw positive benefits of having social scientists on chapter teams. One participant said, “It really enriched how we looked at the literature review, what was important, together. It felt integrated.” They recognized that social scientist numbers and disciplinary diversity were limited: “we tended to go to more of an economic space, and that probably also reflects that the social scientist on our team was an economist” (focus group participant). Author expertise influenced chapter teams’ comfort with and ability to integrate a broad range of literature: “I’m an economist, and that’s what I think of first in social systems. I’m aware of lots of other disciplines that I know very little about” (focus group participant). Participants found it helpful when team members had experience working across disciplines: “We had two social scientists in our group. They were extremely helpful, in part because they already had a lot of experience dealing with engineers and scientists, physical scientists. I have experience with other reports, where just making communication efficient between social scientists and physical scientists and engineers was extremely difficult” (focus group participant).
c. Report framing
NCA4 chapters are oriented around the risks that a changing climate poses to a sector/region and how that risk is managed. The intention behind using RBF in assessments is to incorporate societal dimensions and concerns into the analysis and create products that support decision-making for managing risks (National Academies of Sciences, Engineering, and Medicine 2016b; Weaver et al. 2017). RBF has no single definition or standard protocol. Its implementation is shaped by knowledge governance of the assessment, including USGCRP guidance, chapter team determinations of what the most relevant climate risks are to their sector/region, and authors’ decisions about what constitutes usable information for decision-making. USGCRP guidance on NCA4 key message formulation recommends using RBF to focus on issues of high importance and aid communication. NCA4 defines risk as “threats to life, health and safety, the environment, economic well-being, and other things of value to society . . . In some cases, risks are described in quantitative terms: estimates of how likely a given threat is to occur (probability) and the damages that would result if it did happen (consequences)” (Reidmiller et al. 2018).
RBF facilitated inclusion of evidence from the social sciences in NCA4. One participant described thinking about “why my mom in Nebraska should care about what’s going on in the oceans.” As a result, the team added information about economic costs to fisheries from climate change as a way to connect distant ocean ecosystems with values about a strong economy. However, both sectoral and regional participants expressed that RBF restricts social science inclusion in other ways. They worried it does not readily capture important societal concerns such as equity or risks to nature’s inherent values. They felt the writing process approached risk within a primarily probabilistic framework, which constrained inclusion of nonquantitative studies on “power structures and governance and why things happen the way they do amongst people” (focus group participant), topics that are critical to understanding the uneven distribution of climate-related risks and impacts.
Focus group participants found that RBF was a challenge to implement given the enormous geographic scope, diversity of topics, and limited page numbers they had. Said one author: “You can’t make generalizations about American Indian tribes in [state] because there’s such a range in terms of wealth, infrastructure, and resources.” Chapter teams had to navigate sensitive issues when deciding which risks, sectoral/regional assets, and values to include. While they felt their chapters addressed important topics, “you still wonder if there are things that you missed that should have been covered” (focus group participant).
d. Knowledge systems
Climate assessments tend to draw more heavily on published literature in the biogeophysical than the social sciences. Only 12% of journal references in the IPCC’s Third Assessment Report are from the social sciences; within those, economics predominate (Bjurström and Polk 2011). In NCA4, approximately 20% of cited journal articles come from the social sciences; the remaining 80% come from the biogeophysical literature. The preference for published, peer-reviewed, quantitative literature in assessments leads to gaps in the inclusion of the social sciences and other forms of evidence (e.g., IK, LK).
One reason why is that research gaps in climate social science on drivers, impacts, and responses have persisted across assessments (Callaghan et al. 2020; Maldonado et al. 2015; Roesch-McNally et al. 2020; Victor 2015). For example, focus group participants reported that chapter teams struggled to find research on what adaptation actions do or do not work, and why. In the absence of scientific analyses, they used case studies drawn largely from gray literature to show how adaptation materializes: “the limitation for us was not seeing the kind of studies we were hoping to [ . . . ] to try to identify some examples where people are doing things that have a positive impact, that had a measurable effect” (focus group participant).
Another reason involves the ways that knowledge systems come to inform and be informed by knowledge governance of assessments. In the traceable accounts section of chapters, authors assign confidence statements to key messages. They need to support confidence statements with “high quality” evidence. Authors viewed these requirements as impeding their incorporation of evidence from the social sciences, IK, and LK. Regional chapters could document climate change impacts in particular communities from case study research but found it difficult to assign confidence levels to the entire region. Other topics and literatures did not lend themselves to being rated in this manner: “It was a little difficult for me to conceptualize likelihood or confidence rating in terms of ‘here’s how people conceptualize their changes in well-being in relation to climate change’” (focus group participant). Focus group participants felt that including IK was critical for describing environmental observations and histories of a changing climate; impacts on indigenous livelihoods, culture, and economy; and the many adaptation and mitigation activities that tribal governments and Indigenous peoples have undertaken. They stressed that IK and LK enhanced the overall quality of the report, as also reported in other publications on NCAs (Maldonado et al. 2015; Roesch-McNally et al. 2020). IK is not the same as social science but there are similarities in being qualitative and place-based.
Focus group participants reported difficulties with incorporating IK. One obstacle was the report writing timeline and structure. One regional chapter author described how they had reached out to tribes to get consent for highlighting their climate adaptation work in a case study, a process that took time. We also heard that the 2000 Information Quality Act was a barrier. This act requires federal agencies to verify the utility, objectivity, transparency and traceability, and integrity and security of information they disseminate. It is not that the act specifically prohibits IK and LK, and chapter teams did make efforts to include these, but its presence in the knowledge governance of the report presented an obstacle in the minds of authors. They felt it impeded the report’s ability to engage systematically with important forms of evidence. Said one author: “how we really are able to include as evidence, as knowledge, these kinds of observations that come from communities or people within communities that are situated within landscapes and seascapes that they have deep awareness and observations of . . . I think we need to capture that, and I don’t think we’re good at capturing it yet.” This concern about information quality, we assert, reflects more than administrative challenges in navigating federal statutes. It speaks to how entrenched scientific and institutional authority can unintentionally limit the study of environmental topics in ways could lead to more transformative societal change (Lahsen and Turnhout 2021), the very type of change that some report authors would like to see NCAs spark.
e. Outreach and engagement
A final theme that emerged in the focus groups was participants’ emphasis on the importance of outreach and engagement during the writing process. This emphasis reflects their conceptualization of social science as a means of fostering actionable science and strengthening the assessment’s societal impact. We heard that stakeholder engagement workshops enabled chapter teams to access local and practitioner knowledge, identify case studies, craft key messages, understand values and risks in their region or sector, and learn about climate response actions. Participants described engagement workshops as critical parts of the writing process that enabled authors to shape and test key messages framed in terms of risk to what people value. These social interactions were a means of enacting RBF: “We started out with a focused set of what we thought our key messages were initially and we went into the outreach and the stakeholder gathering . . . and changed completely the direction we were going based on that feedback” (focus group participant).
Workshops did have some effect on structuring the inclusion of social science topics through their influence on key message development. Before their workshop, one regional chapter team envisioned their key messages as being quite discrete: water, energy, agriculture, then “social science/decision making/community outreach” (focus group participant). After the workshop, the team reorganized key messages into new, people-centric categories such “food, energy, and water resources,” and “human health.” Public engagement helped chapters expand discussion of climate response in key messages: “We changed that dialogue and approached it from a solutions perspective—to really talk about what are those solutions and how are we adapting and how do we mitigate” (focus group participant). The importance of engagement workshops to developing key messages is reflected in other analyses of NCAs (Maldonado et al. 2015; Roesch-McNally et al. 2020).
Engagement workshops fostered social connections between chapter authors and their constituencies. The nature of these exchanges was distinct from formal draft reviews: “it was really valuable because there was so much information that came from that willingness to share and have that open discussion” (focus group participant). Participants lauded workshops as an opportunity to reach out beyond the usual scientific community. As one said, “I found . . . [it] to be a really good experience because we were reaching out to people that weren’t in our normal cadre of professionals.” These connections were meaningful to report authors and reinforced their idea that the report has an important role to play in society by providing information about climate change risks in ways that are relevant to different social and cultural groups.
5. Discussion: Toward coequal intellectual partnerships in NCAs
a. What might it look like
Despite notable progress on integrating the social sciences in recent NCAs, NCA4 chapter topics were largely still defined in biogeophysical terms and assessed by reviewing primarily biogeophysical literature. Our analysis of focus groups revealed cultural worldviews wherein “social science” is tantamount to fostering actionable science and increasing the impact of NCAs in society. In this view, the principal roles of the social sciences are to communicate climate risks in ways that resonate with societal values, provide evidence of climate impacts on key sectoral or regional interests, and build connections with report constituencies to develop key messages and share findings. In contrast, a more coequal intellectual partnership would be rooted in a worldview that associates “social science” with analyzing topics and literature in equal measure to fostering actionable science. In this view, the social sciences would have expanded roles in assessments: to identify key science questions, topics, and framings; incorporate a broader evidence base on drivers, impacts, and responses; and offer methods to identify societal values, conduct engagement, and provide ongoing evaluation of whether the assessment is meeting its goals.
A coequal intellectual partnership in assessments is not about establishing quotas for authors, word counts, or references. As one focus group participant said, you cannot achieve it by “shaking the salt shaker and adding social scientists.” Rather, it is about how knowledge governance can support social and biogeophysical sciences in making parallel contributions throughout the writing process. Table 1 lists elements of an NCA produced through coequal intellectual partnerships, along with hypothetical examples. These elements might be distributed unevenly across the entirety of a report, but each chapter would have at least some.
Elements of coequal climate assessments, with hypothetical examples.
The literature on operationalizing coequal intellectual partnerships in global change research calls for action on multiple fronts: scientific tools and methods (e.g., scenarios, models), researcher skills, problem formulation, building trust, publishing, institutional arrangements, organizational structures, scientific committee composition, research investments, researcher diversity, decision support processes, and interdisciplinary team dynamics (Brown et al. 2015; Fischer et al. 2011; Freeth and Caniglia 2020; Freudenburg and Gramling 2002; Harris and Lyon 2013; Lemos and Morehouse 2005; Maxwell 2014; Maxwell et al. 2019; Palmer et al. 2016; Palsson et al. 2013; Rounsevell et al. 2014; Sovacool et al. 2015; Stokols et al. 2008; Strang 2009; Weaver et al. 2014). The challenges in achieving coequal partnerships are plentiful: impenetrable institutional processes, skepticism on the part of social and biogeophysical scientists, team power dynamics, constrained organizational structures, funding limitations, intra and interdisciplinary hierarchies, conflicting epistemologies and jargon, incompatible research timelines and data, and policy oversight (Bulkeley 2019; Crate 2011; Gardner 2013; Maxwell 2014; Maxwell et al. 2019; Nightingale et al. 2020; Norgaard 2018; Strang 2009; Victor 2015; Viseu 2015). Even defining and interpreting key terms is a challenge. Concepts such as “vulnerability” carry different meanings and implications within and across disciplines, for example (Adger 2006; Marino and Faas 2020; O’Brien et al. 2004).
Global, national, and state-level assessments have different mandates, principal audiences, and organization (Table 2). Their knowledge governance and writing processes differ as well, with implications for social science integration. In IPCC reports, it might entail more nuanced social and behavioral variables in models and scenarios or more qualitative evidence of how people are experiencing a changing climate around the globe. In the California climate assessment, it might entail reorganizing summary reports to target specific types of decision-makers in the state. Qualitative research on author experiences with specific assessments could help discern how to most effectively integrate the social sciences in distinct institutional contexts.
Comparison of IPCC Sixth Assessment Report (AR6), NCA4, and Fourth California Climate Assessment’s (CCA4) mandate, audience, and organization.
b. Multiple roles for social science in worldviews and knowledge governance
The cultural worldview expressed in our focus groups conceptualizes social science in terms of how it fulfills the report’s potential to have a positive impact on society and fulfil its public function by crafting key messages in socially relevant terms. A coequal intellectual partnership places comparable emphasis on the role of social science in framing the assessment, analyzing literature, and contributing to knowledge of drivers, impacts, and responses. Achieving such a partnership entails specific attention to expanding worldviews of USGCRP, member agencies, and authors. Guidance and trainings for authors and staff can have long-term positive effects by institutionalizing changes in knowledge governance. It can help distinguish between integration of social science research and experts into report writing from application of social science theory and methods to design outreach and engagement (Eisenhauer et al. 2021). Integration and application are equally important and can overlap in practice but need to be treated separately in knowledge governance.
c. Diversity of author expertise
The disciplinary expertise of NCA authors is critically important for shaping how information is included and presented. It affects chapter teams’ capacities to assess the latest science, access information sources, and socialize the report in professional networks. Recruitment and selection for NCA chapter teams begins with a Federal Register Notice. USGCRP, member agencies, disciplinary societies, professional associations, and others can work to make sure authors from a variety of disciplines and backgrounds are aware of notice, see a potential role for themselves, and are motivated to apply. Authors with interdisciplinary, practitioner, and IK or LK experience can be force multipliers for bridging knowledge systems on chapter teams. In addition, ensuring that social scientists are well represented among chapter leadership is important for ensuring that social sciences play a meaningful role in the areas we have discussed such as chapter framing, establishing an evidence base, and use of diverse knowledge systems. Once chapter teams are selected, advice from the literature on interdisciplinary integration and team science can be used to support chapter leads in nurturing the writing process (Fischer et al. 2011; Freeth and Caniglia 2020; Harris and Lyon 2013; Ledford 2015; Stokols et al. 2008).
Creating a sense of community, we posit, is an essential component of building and sustaining more diverse author teams. Focus group participants lauded the in-person all-author meeting as valuable, in part because it provided productive writing time, but more importantly, because it fostered a sense of community among NCA4 authors. Sharing stories and experiences among current authors and with prospective authors is a way of cultivating such connections. For example, the Society for Applied Anthropology’s 2021 annual meeting convened a roundtable on anthropological engagement with global change assessments.
d. Connecting authors and societal constituencies
Focus group participants felt that stakeholder engagement workshops were a critical part of the writing process. The workshops built connections between authors and their sectoral or regional constituencies (e.g., professional associations, academic researchers, nonprofits, tribal members, and government agencies). They afforded opportunities for building community and consensus across geographies and disciplines, helped teams stay abreast of emerging issues, provided a forum to engage underrepresented voices, and brought local stories to a national stage. For focus group participants, they were essential to developing key messages in ways that would increase the report’s impact in society.
Long-lasting engagement strategies are essential for bridging climate assessments and action (Moss et al. 2019). Focus group participants thought NCAs could do better at more iterative and broad-reaching engagement, tapping into existing networks and conversations. Doing so, they mused, could help NCAs play a more transformative role in mitigation, adaptation, and resilience. Their suggestion, echoed by Roesch-McNally et al. (2020), offers several prospective benefits. It could sustain learning and evaluation between assessment cycles, strengthen bidirectional communication between authors and constituencies, and feed into efforts to tailor sustained assessment products for specific applications. It also is a means of securing buy-in and forestalling criticism of NCA findings (Moser et al. 2016). It brings about greater openness and transparency in scientific decision-making, which is necessary for more transformative assessments that engage deeply with social, economic, and political questions about global change and human well-being (Lahsen and Turnhout 2021).
Social science methods and techniques can be applied to craft outreach and engagement in ways that build long-term relationships, enable meaningful public participation, pay attention to power dynamics, and are beneficial and not burdensome to participants. For NCAs, doing so might entail holding workshops at multiple points in the writing process and after publication, leveraging existing meetings and networks, and building stronger ties with science boundary organizations. For example, the National Oceans and Atmospheric Administration’s Climate Adaptation Partnerships program, U.S. Geological Survey’s Climate Adaptation Science Centers, and U.S. Department of Agriculture’s Climate Hubs have established mechanisms to engage their networks. Connecting via these boundary organizations could help NCAs meet the needs of different audiences.
e. Reconsidering report framing
The use of RBF in NCA4 enabled author teams to emphasize how a changing climate affects things people value. It fostered inclusion of some social science topics and evidence. The intention of RBF is to improve the connection between science and decision-making (National Academies of Sciences, Engineering, and Medicine 2016b). The way it plays out in practice, though, indicates that RBF is not in and of itself a solution to coequal intellectual partnerships in assessments. It is interpreted and implemented in ways that can curtail analysis of human social systems. A risk framing limits scientific inquiry of societally important topics in assessments (e.g., justice and equity) and constrains discussion of climate solutions to primarily technical realms (Dunlap and Brulee 2015; Nightingale et al. 2020). It means that assessments are not asking “questions that social scientists can answer,” making it harder for researchers from these disciplines to participate (Victor 2015). It reinforces disconnects between assessments and bodies of work such as anthropological and geographic research on the production and dissemination of climate science itself (Barnes et al. 2013; Bulkeley 2019; Lahsen 2013). RBF perpetuates worldviews that climate change is an external force that acts upon society, that natural and social systems are distinct, and that communicating in a way that speaks to societal values to is equivalent to social science. In contrast, many sociologists, anthropologists, and geographers approach climate change as a condition of society (Barnes et al. 2013; Dunlap and Brulee 2015; Jorgenson et al. 2019; Nightingale et al. 2020; Norgaard 2018). For example, they investigate how societal forces both drive climate change and create socioeconomic inequalities in distributing climate risks. Reconsidering report framing is about more than correcting missed opportunities. It is a chance to level out disciplinary hierarchies that, as Oreskes (2015) argues, have been reified in the architecture of assessments.
The limitations of RBF in practice point to a need for reflection on and potential revision of report framing. Focus group participants suggested problem-based framing as one alternative. Chapters could begin with an update on climate-related problems their sector or region is experiencing; discuss why these problems arose, considering both physical and social contexts; and examine responses, including gaps, evidence of effectiveness, and obstacles to success. This approach would allow for discussion of risks and risk management but would also include questions such as, “How do we know what is working in adaptation at the local level, for whom, and why?” Other suggestions for reframing include: a collaborative, multidisciplinary assessment structured around solutions; and sustained assessment products designed for adaptation and mitigation implementation (Kowarsch et al. 2017; Moss et al. 2019). A broader interpretation of RBF, one that is not probabilistic and is more inclusive of both societal and climate underpinnings of risk, might better accommodate the social sciences but remains to be demonstrated in practice.
In the short and medium terms, NCAs could operate under RBF but draw on the social sciences to improve its implementation and correct potential distortions. New content could be included: chapters might discuss how and why human social systems distribute risk unevenly, analyze climate-related risks as part of cumulative impacts on health and well-being, examine the risk perceptions of different cultural and social groups (including scientists), and incorporate qualitative evidence of how people around the United States experience a changing climate. Application of social science methods could help chapter teams more systematically identify the values their constituencies have when crafting key messages. In the longer term, it could be constructive to reconsider report framing in a deliberative process that includes social scientists to reflect on the core questions being asked of the assessment.
f. Bridging knowledge systems
Bridging knowledge systems in NCAs to foster inclusion of different forms of knowledge and types of evidence could enhance its scientific content, increase its utility to decision-making, foster culturally appropriate scientific practices, and address power imbalances in knowledge production (Dilling and Lemos 2011; Kirchhoff et al. 2013; Moreno-Cely et al. 2021; Moss et al. 2019; Weichselgartner and Arheimer 2019). One short-term step would be for USGCRP to marshal literature for technical inputs into NCA5 to aid chapter teams in covering relevant socioeconomic aspects of their sector or region (e.g., energy behavior, coastal livelihoods, and environmental justice and air quality). Agrawal et al.’s (2012) call for social scientists to become more engaged in climate analysis and action reminds us that there are steps that disciplinary professional societies, individual researchers, and journal editors could all take to build bridges, such as sharing calls for authors, review editors, and public comments in future NCAs.
In the medium term, USGCRP could work with tribal and other experts to develop additional guidance and training on using qualitative evidence, IK, and LK in traceable accounts. The recent Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES) Global Assessment Report on Biodiversity and Ecosystem Services serves as one model for how assessments can incorporate IK and LK (McElwee et al. 2020). In the longer term, federal agencies and other research funders could design requests for proposals for social scientific analyses of climate change, responses, and actionable science to fill research gaps (Arnott et al. 2020; Callaghan et al. 2020; Roesch-McNally et al. 2020).
6. Conclusions
By using qualitative research methods, this study reveals insights into the evolving role of social sciences in NCAs. Cultural worldviews and knowledge governance affect how the social sciences were integrated in NCA4. The ways authors conceptualize “social science” center on making the assessment meaningful to report constituents, rather than a disciplinary orientation. It is tied to their desire to have the report make a difference in climate mitigation, adaptation, and resilience in the United States. These worldviews and desires are woven through knowledge governance of the NCA including RBF guidance, traceable account requirements, and stakeholder engagement workshops.
This article contributes to understanding of the opportunities and barriers for coequal intellectual partnerships that could augment NCA’s scientific content and its impact in society. A pathway to coequal intellectual partnerships includes short to long-term steps such as recruiting chapter teams with greater diversity of expertise, providing additional training and guidance, fostering a sense of community among authors, crafting additional connections between authors and external constituencies, reconsidering report framing, and filling gaps in climate social science research. We present key areas for improvement and examples of what a well-integrated report would look like. The pathway expounded upon here focuses primarily on production of the report and integration of social science content. There are additional areas of opportunity for applying the social sciences throughout the NCA life cycle. One is to borrow methods for doing outreach and engagement in ways that help the assessment serve the more transformative role that authors desire. Another is to evaluate how well engagement is going and whether report chapters serve the needs of their intended audience (e.g., via bibliometric analysis, usability testing, reader commentaries, and postengagement workshop feedback).
The items mentioned above are specific to NCA’s institutional and social context. The overall approaches are adaptable to other global change assessments but likely would play out differently. Global assessments might invite participation of more human geographers to analyze regional vulnerabilities, whereas state assessments might include more public health experts to examine how climate and social stressors interact in particular locales. On a deeper level, coequal partnerships necessitate attention to how knowledge governance of assessments privileges certain knowledge systems, worldviews, and disciplinary hierarchies (Barnes et al. 2013; Lahsen and Turnhout 2021; Obermeister 2017; Oreskes 2015). The physical sciences have begun to subsume the social in climate models and projections, a trend that has gone largely unnoticed (Oreskes 2015). In contrast, our use of the term “coequal intellectual partnerships” has gotten unexpected (to us) pushback throughout this project. We have received questions about what it means. We have been challenged to prove why social scientists should be at the climate assessment table in ways that oceanographers, ecologists, and atmospheric scientists are not.
Global change knowledge production occasionally encounters windows of opportunity for making evolutionary leaps forward in progress (Lahsen and Turnhout 2021). This project has found such a window—and helped to open it. The very act of conducting this evaluation and sharing our findings with USGCRP and its member agencies has prompted action. NCA5, currently underway, is already more representative of social science and social scientists than NCA4. USGCRP made specific mention of social science and economics in the Federal Register Notice call for authors. It held a webinar for social scientists on how they could participate in different capacities, reaching 150 participants. As a result of these efforts, there is a greater percentage of chapter leadership and authors from social science backgrounds than in previous NCAs. Social scientists compose 20% of chapter authors; there are 9 chapter leads and 3 coordinating lead authors with social science backgrounds. Moreover, two new chapters are dedicated to economics and human social systems, respectively. Author guidance on the social sciences has been more robust and public engagement has expanded. USGCRP offered a public comment period for the zero-order draft and held engagement workshops for almost all report chapters. Authors initiated and led two additional workshops, one specifically for youth and another dedicated to issues of environmental justice. Experts on IK are expanding previous author guidance in collaboration with Information Quality Act experts. The impact of these changes and opportunities for continued advancement are topics for continued exploration. It also speaks to the need for ongoing evaluation and monitoring to help understand how these changes are working in practice and what effects they have on the report’s scientific content and its impact in society.
Acknowledgments.
Thanks are given to the Social Sciences Coordinating Committee and to USGCRP for supporting this project. Sarah Pinter’s aid with the focus groups was immensely helpful, as was Amrutha Elamparuthy’s analysis of NCA4 citations. Many thanks are given to all of our focus group participants. The reviewers helped to improve the drafts. Any remaining errors are ours. Authors Keely Maxwell and Emily Eisenhauer have no declared financial interests; author Allyza Lustig is a contracted employee of ICF to the National Coordination Office of USGCRP. Nonfinancial interests: Authors Maxwell and Eisenhauer are members of the Social Sciences Coordinating Committee. Author Maxwell was a chapter lead for NCA4. The three authors are working on NCA5 in various capacities. This publication has been reviewed in accordance with U.S. Environmental Protection Agency policy and approved for publication. The views expressed in this article are those of the authors and do not necessarily represent the views or the policies of the agency.
Data availability statement.
Because of privacy concerns, the data cannot be made available.
REFERENCES
Adger, W. N., 2006: Vulnerability. Global Environ. Change, 16, 268–281, https://doi.org/10.1016/j.gloenvcha.2006.02.006.
Agrawal, A., M. C. Lemos, B. Orlove, and J. Ribot, 2012: Cool heads for a hot world – Social sciences under a changing sky. Global Environ. Change, 22, 329–331, https://doi.org/10.1016/j.gloenvcha.2012.02.003.
Arnott, J. C., C. J. Kirchhoff, R. M. Meyer, A. M. Meadow, and A. T. Bednarek, 2020: Sponsoring actionable science: What public science funders can do to advance sustainability and the social contract for science. Curr. Opin. Environ. Sustainability, 42, 38–44, https://doi.org/10.1016/j.cosust.2020.01.006.
Barnes, J., and Coauthors, 2013: Contribution of anthropology to the study of climate change. Nat. Climate Change, 3, 541–544, https://doi.org/10.1038/nclimate1775.
Bjurström, A., and M. Polk, 2011: Physical and economic bias in climate change research: A scientometric study of IPCC Third Assessment Report. Climatic Change, 108 (1–2), 1–22, https://doi.org/10.1007/s10584-011-0018-8.
Brown, R., A. Deletic, and T. H. F. Wong, 2015: How to catalyse collaboration. Nature, 525, 315–317, https://doi.org/10.1038/525315a.
Bulkeley, H., 2019: Navigating climate’s human geographies: Exploring the whereabouts of climate politics. Dialogues Hum. Geogr., 9, 3–17, https://doi.org/10.1177/2043820619829920.
Callaghan, M. W., J. C. Minx, and P. M. Forster, 2020: A topography of climate change research. Nat. Climate Change, 10, 118–123, https://doi.org/10.1038/s41558-019-0684-5.
Castree, N., 2015: Geographers and the discourse of an earth transformed: Influencing the intellectual weather or changing the intellectual climate? Geogr. Res., 53, 244–254, https://doi.org/10.1111/1745-5871.12125.
Crate, S. A., 2011: Climate and culture: Anthropology in the era of contemporary climate change. Annu. Rev. Anthropol., 40, 175–194, https://doi.org/10.1146/annurev.anthro.012809.104925.
Dilling, L., and M. C. Lemos, 2011: Creating usable science: Opportunities and constraints for climate knowledge use and their implications for science policy. Global Environ. Change, 21, 680–689, https://doi.org/10.1016/j.gloenvcha.2010.11.006.
Dunlap, R. E., and R. J. Brulee, Eds., 2015: Climate Change and Society: Sociological Perspectives. Oxford University Press, 394 pp.
Eisenhauer, E., K. C. Williams, K. Margeson, S. Paczuski, M. C. Hano, and K. Mulvaney, 2021: Advancing translational research in environmental science: The role and impact of social sciences. Environ. Sci. Policy, 120, 165–172, https://doi.org/10.1016/j.envsci.2021.03.010.
Fischer, A. R. H., H. Tobi, and A. Ronteltap, 2011: When natural met social: A review of collaboration between the natural and social sciences. Interdiscip. Sci. Rev., 36, 341–358, https://doi.org/10.1179/030801811X13160755918688.
Fischhoff, B., 2020: Making behavioral science integral to climate science and action. Behav. Public Policy, 5, 439–453, https://doi.org/10.1017/bpp.2020.38.
Freeth, R., and G. Caniglia, 2020: Learning to collaborate while collaborating: Advancing interdisciplinary sustainability research. Sustainability Sci., 15, 247–261, https://doi.org/10.1007/s11625-019-00701-z.
Freudenburg, W. R., and R. Gramling, 2002: Scientific expertise and natural resource decisions: Social science participation on interdisciplinary scientific committees. Soc. Sci. Quart., 83, 119–136, https://doi.org/10.1111/1540-6237.00074.
Gardner, S. K., 2013: Paradigmatic differences, power, and status: A qualitative investigation of faculty in one interdisciplinary research collaboration on sustainability science. Sustainability Sci., 8, 241–252, https://doi.org/10.1007/s11625-012-0182-4.
Goldman, M. J., M. D. Turner, and M. Daly, 2018: A critical political ecology of human dimensions of climate change: Epistemology, ontology, and ethics. Wiley Interdiscip. Rev.: Climate Change, 9, e526, https://doi.org/10.1002/wcc.526.
Hackmann, H., S. C. Moser, and A. L. St. Clair, 2014: The social heart of global environmental change. Nat. Climate Change, 4, 653–655, https://doi.org/10.1038/nclimate2320.
Harris, F., and F. Lyon, 2013: Transdisciplinary environmental research: Building trust across professional cultures. Environ. Sci. Policy, 31, 109–119, https://doi.org/10.1016/j.envsci.2013.02.006.
Jorgenson, A. K., and Coauthors, 2019: Social science perspectives on drivers of and responses to global climate change. Wiley Interdiscip. Rev.: Climate Change, 10, e554, https://doi.org/10.1002/wcc.554.
Kirchhoff, C. J., M. C. Lemos, and S. Dessai, 2013: Actionable knowledge for environmental decision making: Broadening the usability of climate science. Annu. Rev. Environ. Resour., 38, 393–414, https://doi.org/10.1146/annurev-environ-022112-112828.
Kowarsch, M., and Coauthors, 2017: A road map for global environmental assessments. Nat. Climate Change, 7, 379–382, https://doi.org/10.1038/nclimate3307.
Lahsen, M., 2013: Anatomy of dissent: A cultural analysis of climate skepticism. Amer. Behav. Sci., 57, 732–753, https://doi.org/10.1177/0002764212469799.
Lahsen, M., and E. Turnhout, 2021: How norms, needs, and power in science obstruct transformations towards sustainability. Environ. Res. Lett., 16, 025008, https://doi.org/10.1088/1748-9326/abdcf0.
Ledford, H., 2015: Team science. Nature, 525, 308–311, https://doi.org/10.1038/525308a.
Lemos, M., and B. J. Morehouse, 2005: The co-production of science and policy in integrated climate assessments. Global Environ. Change, 15, 57–68, https://doi.org/10.1016/j.gloenvcha.2004.09.004.
Liverman, D., 2016: U.S. National Climate Assessment gaps and research needs: Overview, the economy and the international context. Climatic Change, 135, 173–186, https://doi.org/10.1007/s10584-015-1464-5.
Maldonado, J. , and Coauthors, 2015: Engagement with indigenous peoples and honoring traditional knowledge systems. Climatic Change, 135, 111–126, https://doi.org/10.1007/s10584-015-1535-7.
Marino, E. K., and A. J. Faas, 2020: Is vulnerability an outdated concept? After subjects and spaces. Ann. Anthropol. Pract., 44, 33–46, https://doi.org/10.1111/napa.12132.
Maxwell, K., 2014: Getting there from here. Nat. Climate Change, 4, 936–937, https://doi.org/10.1038/nclimate2413.
Maxwell, K., B. Hubbell, and E. Eisenhauer, 2019: Institutional insights on integrating social and environmental science for solutions-driven research. Environ. Sci. Policy, 101, 97–105, https://doi.org/10.1016/j.envsci.2019.08.003.
McElwee, P., and Coauthors, 2020: Working with Indigenous and local knowledge (ILK) in large‐scale ecological assessments: Reviewing the experience of the IPBES Global Assessment. J. Appl. Ecol., 57, 1666–1676, https://doi.org/10.1111/1365-2664.13705.
Moreno-Cely, A., D. Cuajera-Nahui, C. G. Escobar-Vasquez, T. Vanwing, and N. Tapia-Ponce, 2021: Breaking monologues in collaborative research: Bridging knowledge systems through a listening-based dialogue of wisdom approach. Sustainability Sci., 16, 919–931, https://doi.org/10.1007/s11625-021-00937-8.
Morgan, M. G., and Coauthors, 2005: Learning from the U.S. National Assessment of Climate Change Impacts. Environ. Sci. Technol., 39, 9023–9032, https://doi.org/10.1021/es050865i.
Moser, S. C., J. M. Melillo, K. L. Jacobs, R. H. Moss, and J. L. Buizer, 2016: Aspirations and common tensions: Larger lessons from the third US national climate assessment. Climatic Change, 135, 187–201, https://doi.org/10.1007/s10584-015-1530-z.
Moss, R. H., and Coauthors, 2019: Evaluating knowledge to support climate action: A framework for sustained assessment. Report of an independent advisory committee on applied climate assessment. Wea. Climate Soc., 11, 465–487, https://doi.org/10.1175/WCAS-D-18-0134.1.
National Academies of Sciences, Engineering, and Medicine, 2016a: Review of the U.S. Global Change Research Program’s Update to the Strategic Plan Document. National Academies Press, 92 pp., https://doi.org/10.17226/23396.
National Academies of Sciences, Engineering, and Medicine, 2016b: Characterizing Risk in Climate Change Assessments: Proceedings of a Workshop. National Academies Press, 100 pp., https://doi.org/10.17226/23569.
National Research Council, 2007: Analysis of Global Change Assessments: Lessons Learned. National Academies Press, 196 pp., https://doi.org/10.17226/11868.
Nightingale, A. J., and Coauthors, 2020: Beyond technical fixes: Climate solutions and the great derangement. Climate Dev., 12, 343–352, https://doi.org/10.1080/17565529.2019.1624495.
Norgaard, K. M., 2018: The sociological imagination in a time of climate change. Global Planet. Change, 163, 171–176, https://doi.org/10.1016/j.gloplacha.2017.09.018.
Obermeister, N., 2017: From dichotomy to duality: Addressing interdisciplinary epistemological barriers to inclusive knowledge governance in global environmental assessments. Environ. Sci. Policy, 68, 80–86, https://doi.org/10.1016/j.envsci.2016.11.010.
O’Brien, K., S. Eriksen, L. Nygaard, and A. Schjolden, 2004: Why different interpretations of vulnerability matter in climate change discourses. Climate Policy, 7, 73–88, https://doi.org/10.1080/14693062.2007.9685639.
Oreskes, N., 2015: How earth science has become a social science. Hist. Soc. Res., 40, 246–270.
Palmer, M. A., J. G. Kramer, J. Boyd, and D. Hawthorne, 2016: Practices for facilitating interdisciplinary synthetic research: The National Socio-Environmental Synthesis Center (SESYNC). Curr. Opin. Environ. Sustainability, 19, 111–122, https://doi.org/10.1016/j.cosust.2016.01.002.
Palsson, G., and Coauthors, 2013: Reconceptualizing the ‘Anthropos’ in the Anthropocene: Integrating the social sciences and humanities in global environmental change research. Environ. Sci. Policy, 28, 3–13, https://doi.org/10.1016/j.envsci.2012.11.004.
Reid, W. V., C. Bréchignac, and Y. T. Lee, 2009: Earth system research priorities. Science, 325, 245–245, https://doi.org/10.1126/science.1178591.
Reidmiller, D. R., and Coauthors, 2018: Impacts, Risks, and Adaptation in the United States: Fourth National Climate Assessment. Vol. II, U.S. Global Change Research Program, 186 pp., https://doi.org/10.7930/NCA4.2018.RiB.
Reynolds, J., J. Kizito, N. Ezumah, P. Mangesho, E. Allen, and C. Chandler, 2011: Quality assurance of qualitative research: A review of the discourse. Health Res. Policy Syst., 9, 43–53, https://doi.org/10.1186/1478-4505-9-43.
Roesch-McNally, G., and Coauthors, 2020: Beyond climate impacts: Knowledge gaps and process-based reflection on preparing a regional chapter for the Fourth National Climate Assessment. Wea. Climate Soc., 12, 337–350, https://doi.org/10.1175/WCAS-D-19-0060.1.
Rounsevell, M. D. A., and Coauthors, 2014: Towards decision-based global land use models for improved understanding of the Earth system. Earth Syst. Dyn., 5, 117–137, https://doi.org/10.5194/esd-5-117-2014.
Sovacool, B., and Coauthors, 2015: Integrating social science in energy research. Energy Res. Soc. Sci., 6, 95–99, https://doi.org/10.1016/j.erss.2014.12.005.
Stokols, D., S. Misra, R. P. Moser, K. L. Hall, and B. K. Taylor, 2008: The ecology of team science. Amer. J. Prev. Med., 35 (2), S96–S115, https://doi.org/10.1016/j.amepre.2008.05.003.
Strang, V., 2009: Integrating the social and natural sciences in environmental research: A discussion paper. Environ. Dev. Sustainability, 11, 1–18, https://doi.org/10.1007/s10668-007-9095-2.
Strauss, A., and J. M. Corbin, 1990: Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Sage Publishing, 457 pp.
Thomas, K., and Coauthors, 2019: Explaining differential vulnerability to climate change: A social science review. Wiley Interdiscip. Rev.: Climate Change, 10, e565, https://doi.org/10.1002/wcc.565.
Victor, D., 2015: Climate change: Embed the social sciences in climate policy. Nature, 520, 27–29, https://doi.org/10.1038/520027a.
Viseu, A., 2015: Integration of social science into research is crucial. Nature, 525, 291, https://doi.org/10.1038/525291a.
Weaver, C. P., and Coauthors, 2014: From global change science to action with social sciences. Nat. Climate Change, 4, 656–659, https://doi.org/10.1038/nclimate2319.
Weaver, C. P., R. H. Moss, K. L. Ebi, P. H. Gleick, P. C. Stern, C. Tebaldi, R. S. Wilson, and J.L. Arvai, 2017: Reframing climate change assessments around risk: Recommendations for the US National Climate Assessment. Environ. Res. Lett., 12, 12, https://doi.org/10.1088/1748-9326/aa7494.
Weichselgartner, J., and B. Arheimer, 2019: Evolving climate services into knowledge–action systems. Wea. Climate Soc., 11, 385–399, https://doi.org/10.1175/WCAS-D-18-0087.1.
Wuebbles, D. J., and Coauthors, 2017: Climate Science Special Report: Fourth National Climate Assessment. Vol. I, U.S. Global Change Research Program, 470 pp., https://doi.org/10.7930/J0J964J6.
Yearley, S., 2009: Sociology and climate change after Kyoto: What roles for social science in understanding climate change? Curr. Sociol., 57, 389–405, https://doi.org/10.1177/0011392108101589.
