Workshop on “Research Knowledge Transitions & Compound Hazards: Generating Ideas for Adapting Social Science to Your Workspace”
What: | This 2-day workshop brought together experts from NOAA, emergency management, and broadcast meteorology to discuss the processes and activities of transitioning social science research into their respective workplaces. The agenda for day 1 focused on eliciting participant experiences with new knowledge applications and the challenges of these processes and activities. The goal was to listen and let participants shape the conversation through small and large group discussions. Outcomes of this conversation included a better understanding of the top-down and bottom-up complexities of research transitions and the collaborative knowledge production activities that are a common motivator. Day 2 focused on sharing examples of successful and ongoing social science research that has or will be transitioned into various operational spaces. Outcomes highlighted participants’ common motivations for applying new knowledge, primarily collaboration via engagement with stakeholders, partners, and the public. |
When: | 6–7 June 2023 |
Where: | Online |
Two reports spanning a history of 50 years summarize some of today’s most relevant challenges and opportunities of the applicability and integration of social and behavioral sciences (SBS) within the Weather Enterprise. The first, “The Warning System: A Social Science Perspective” (McLuckie 1973), was an early attempt to describe how effective response and dissemination of warning messaging posed an operational problem to the National Weather Service (NWS). This analysis noted a disciplinary concern still echoed in the present: “There is relatively little understanding (…) of how to translate relevant existing knowledge within social science into practical applications” (McLuckie 1973). The second report, published by the National Academies of Sciences, Engineering, and Mathematics (NASEM), made a case for SBS’s role in the Weather Enterprise and characterized the biggest obstacles to advancing practical applications in weather readiness, hazard monitoring, emergency management, and forecasting processes. NASEM suggested that the full integration of SBS in weather operations would be a reality only when “the relatively small scale, intermittency, and inconsistency of investment in these kinds of efforts” (NASEM 2018) was overcome.
The path of making SBS actionable in the Weather Enterprise reflects a common feature of the social sciences core disciplines (e.g., sociology, anthropology, psychology, political sciences, and economics) in the United States. During the first half of the twentieth century, these disciplines built their methodological foundations and scientific character, but in the second half, scholars committed to action-oriented research (Given 2008). In this transition, the classic lines demarcating SBS questions about society and people’s behavior, as well as the divide between quantitative and qualitative research methods, were blurred (Symonds and Gorard 2010). As a result, current SBS is composed of varied disciplines and subfields (i.e., criminology, women’s studies, nursing, public health, and English), guided by specialized disciplinary frameworks and paradigms, and the dynamically evolving methodological and theoretical demands of sectors, governments, and social contexts, like the ones witnessed by the weather community (Demuth et al. 2007).
Background on research transitions for SBS in NOAA
As a science-based services and stewardship organization, the National Oceanic and Atmospheric Administration (NOAA) is committed to fulfilling its mission by continuously enhancing its products and services (NOAA 2016). While research findings have broad benefits, research value comes from real-world application, specifically when practitioners turn knowledge into action. Practitioners, however, have reported challenges in locating, accessing, and translating research outcomes, often citing constraints in time, resources, and expertise (Fothergill 2000; Williams 2021). In response, NOAA and other federal agencies developed frameworks and processes designed to bridge the gap between research and practice (National Research Council 2003). Research-to-applications (R2X) allows NOAA to identify, coordinate, and facilitate the transition of cutting-edge research and development to operations (R2O), commercialization (R2C), and other applications and uses (R2X). At its core, R2X revolves around people, highlighting the critical role of coordination, collaboration, and the early establishment of trusted relationships between researchers and practitioners. A tangible outcome of this collaboration is the codevelopment of a transition plan, which are living documents that act as roadmaps, encouraging researchers and practitioners to envision potential applications from the start and anticipate operational challenges that may hinder the integration of research findings into operations and other applications (NOAA 2022).
NOAA’s Policy on Research and Development was intentionally designed to accommodate a wide range of research and development outputs, including knowledge (NOAA 2016). Measuring knowledge transfer is more elusive and complex in comparison to the tangible outputs of technology, which often results in an overemphasis on technology transfers. Yet, the transfer of knowledge is a critical research output, particularly for SBS. Insights from SBS projects encompass a variety of outcomes, ranging from user feedback on product or service changes to recommendations for improved risk communication. Knowledge transfer, in particular, was a focal point at the 2019 SBS R2O Workshop hosted by NOAA’s Weather Program Office (WPO). Throughout the workshop, participants repeatedly emphasized the importance of knowledge transfer and its broader benefits, which extend beyond the 24/7 warning environment, to guide the people, processes, and organizational policies surrounding the R2X process (NOAA 2020).
After receiving input from the workshop, WPO’s Social Science Program (SSP) focused on optimizing the R2X transition process. Using a human-centered approach, the SSP improved their R2X processes by prioritizing knowledge as a critical research output and fostering a more collaborative, iterative, and agile transition process to reduce burden on researchers and practitioners (Eosco and Williams 2022). After receiving positive feedback, the SSP is now exploring effective methods to share funded social science research knowledge with external partners. Yet, several questions still linger about the most effective ways to reach, present, and monitor the use of NOAA-funded SBS research findings by practitioners, and any subsequent benefits they experience. This is why the SSP was excited to participate in this new research transitions workshop, described below, as it offered a valuable opportunity to gather feedback on their current approaches for integrating SBS research into their workplaces.
Research transitions workshop: Background and program
To identify how National Weather Service (NWS) partners, such as emergency managers (EMs) and broadcast meteorologists (BMETs), think about and practice transitioning social science knowledge into their professional spaces, a research team of Texas Tech University and Texas A&M University hosted the virtual workshop “Research Knowledge Transitions and Compound Hazards: Generating Ideas for Adapting Social Science to Your Workspace” on 6 and 7 June 2023. The virtual workshop was part of the NOAA grant, “Improving Knowledge about NWS Forecaster Core Partner Needs for Reducing Vulnerability to Compound Threats in Landfalling Tropical Cyclones Amid Covid-19.” It aims to help NWS forecasters enhance warning practices and decision support for EMs and BMETs in co-occurring wind and water threats (Nielsen et al. 2015; Henderson et al. 2020).
The workshop was attended by 17 participants from NOAA/NWS, emergency management, and broadcast meteorology, all of whom are critical in the communication cycle of weather threat messaging. Participation was open only to individuals who had previously collaborated in developing grant outcomes at different stages, including recommending topics and methodological instruments, facilitating communication with individuals from their communities, or participating in interviews. During workshop planning, NOAA’s SSP collaboratively designed the agenda and participants shared materials that they considered essential to the conversation. This coproduction of knowledge strategy (Schneider et al. 2021) was initiated in May 2023 with the release of an online workshop site dedicated to communicating and disseminating open information about research and operations (https://www.redlaboratory.org/r20) and a working document to which participants contributed. Since then, the open-access site has served as a resource repository (https://www.redlaboratory.org/general-5) for partners and researchers across the weather community.
Over the 2-day workshop, conversations focused on the concepts of social science, research transitions, and examples of successful R2X case studies of social science outcomes related to the magnified risks of overlapping tornado and flash flood warnings, or TORFFs (Nielsen et al. 2015), which can give conflicting safety advice (see also Henderson et al. 2020). On the first day of the workshop, the research group set the tone through interactive presentations about the foundations of SBS and the processes for coproduction of knowledge, as well as an introduction to some initial grant findings from the qualitative analysis of interviews. A general discussion was focused on how participants transition new social science research into different aspects of their workspaces and practices. On the second day, presentations and discussion focused on three topics: 1) a conceptual approach to operationalizing knowledge through transition (Castle Williamsberg and Gina Eosco); 2) forecaster training based on previous TORFF research in the social sciences (Katy Christian); and 3) current opportunities for TORFF research transition through a warning database (Erik Nielsen). After each presentation, participants discussed how R2X could be made meaningful to their communities of practice and which actions could be taken further to enhance and disseminate social science research. Table 1 presents a summary of each activity.
Description of workshop sessions.
After the workshop, authors Porter and Checkoway reviewed workshop materials to synthesize insights, which are offered below. This summative description of workshop discussions was completed through an iterative analysis and a collaborative review of this report with workshop participants to ensure completeness and accuracy.
Workshop results
Participants raised issues related to two broad topics: knowledge production and transfer, which highlight how participant experiences challenged normative ideas about the process of knowledge transfer as unidirectional and linear; and engagement through collaboration, which illustrates the barriers to, complexities of, and opportunities for sharing knowledge with different constituents. Group discussions suggested there are some similarities across public and private sectors when transitioning new knowledge within participants’ respective workplace cultures. We briefly summarize the results below.
Knowledge production and transfer.
Participant discussion of research transition activities suggests that within the Weather Enterprise, the processes of knowledge transfer often involve ad hoc decisions and complex or time-consuming negotiations of social, political, and organizational contexts in their respective workplaces. This can include, for example, limitations of one’s position in the organizational hierarchy or developing training procedures for changes in operations. These conversations highlight the messy process of integrating new knowledge, complicating more normative models of research transition (e.g., R2O). Such transitions resist a framework of linear knowledge production in theory but employ assessments (e.g., readiness levels) that evoke linearity in practice (NOAA 2017). In the social sciences, knowledge is situated (e.g., occurs in social contexts) and negotiated across epistemic communities and different actors. It is widely studied as a complicated and evolving set of frameworks that reflect relationships between various entities and the infrastructures that have emerged to facilitate scientific and economic exchange, such as the modes of knowledge production (Gibbons et al. 2005) and the triple helix of innovation (Etzkowitz and Leydesdorff 1996). In this workshop, participants shared experiences that illustrate these highly dynamic processes and the rich ways that transitioning knowledge is not just contextual but varied in its goals and definitions (see Table 2).
Participant responses to the question: What does transitioning knowledge into your workspace mean to you?
On day 1 of the workshop, participants were asked to describe how new knowledge and practices are transitioned to their spaces and what these processes might look like (Fig. 1). Figure 1 shows responses given by participants who have previously transitioned knowledge into their workspace, along with solutions and challenges with transitioning knowledge. Participants gave many examples of how the Weather Enterprise can promote collaboration, including conferences and professional associations, multilevel educational outreach programs, social science outcomes that promote workplace changes, stakeholder engagement, and communication with core partners at a regional level. Fostering experimental spaces and testing novel applications of new ideas and practices were noted as ways to generate positive impacts on internal processes and promote innovation.
Jamboard example from group discussions on day 1. Each color represents responses from individuals within different small group discussions. Examples were synthesized into one Jamboard to illustrate ideas across groups.
Citation: Bulletin of the American Meteorological Society 105, 4; 10.1175/BAMS-D-23-0310.1
In both the Jamboard and subsequent discussions, participants reported wide variations in how their operational spaces collect, produce, and implement new knowledge and methodologies, specifically those from social science research. Participants described the ways they and their organizations incorporate new knowledge through two primary forms: a bottom-up approach and a top-down approach. Their experiences enhance our understanding of 1) how accessible social science information and insights are and 2) whether enough is being done to make social science transferable to new sectors and applications.
For participants, the bottom-up approach to the knowledge transfer process is typically more individualistic in nature, especially when someone proposes new ideas or protocols for their work environment. However, the experimental nature of their suggestions comes with additional time and resource costs, which can hinder implementation. For example, one participant described seeking information from various sources (e.g., conferences, social media, literature, collaborators, etc.) to improve upon their agency’s existing protocols. Within their agency, the participant noted that some decision-making is primarily informed by a superior’s “gut feelings,” which creates processes for transitions that are ad hoc and subjective. They explained other common challenges, such as buy-in from leadership and insufficient time to carry out these suggested changes (e.g., literature reviews, workshop attendance, and collaboration with others). Other participants acknowledged that for those largely unfamiliar with social sciences and the benefits of its inclusion within practices, it is even more difficult to navigate the chain of command and receive approvals.
Top-down approaches to knowledge transfer were more frequently reported by participants who were 1) affiliated with organizations that have established research transition processes, 2) occupied roles with an emphasis on knowledge development, or 3) held senior positions and so had more latitude to test and implement new knowledge. Although still guided by largely subjective processes, these positions afforded some the ability to streamline how and when new research insights could be transferred and integrated. For example, NOAA participants shared that a common way they learn about new practices informed by social science research is through online training modules. Despite the value of the content, however, modules make it difficult to measure the impact and utility of this knowledge for employers. As one participant said, “Is this [knowledge] being used, and if so, how?” This suggests that although research insights and social science practices may be transitioning to new applications within the Weather Enterprise, there can be challenges with measuring the success of new programming over the long-term, which also impedes the ability to optimize its utility for the organization. In sum, both bottom-up and top-down processes have different challenges related to access to insights and the transferability of knowledge to new sectors and applications.
Engagement through collaboration.
Workshop participants identified collaboration as the key expression of their engagement and central to knowledge transfer. Collaboration was described as the interaction and cooperation among different disciplines and sectors (i.e., private, public, academic) to foster an environment of work carried out toward common goals. It also included sharing information and knowledge with partners and end users, such as different publics, and understanding how these users interpret and act on these contributions. In the context of transitioning new knowledge to their respective workplaces, the desire to improve engagement emerged as a key motivation. This reflects similar commitments from NOAA and other public service organizations that have the responsibility to serve society. For example, in a 2008 NOAA report titled “Engaging NOAA’s Constituents,” the authors highlight the importance of engagement to this government agency (Kudrna et al. 2008). They write, “Engagement implies a commitment of service to society through a partnership based on reciprocity and sharing of goals, objectives, and resources between NOAA and society. Implicit to engagement is a respect for each partner that involves listening, dialogue, understanding, and mutual support” (Kudrna et al. 2008, p. 7). While participants acknowledged respect for one another and feel they share values that are important to engagement, in practice, these are insufficient to realize tangible outcomes. Hence, the topic of engagement illustrated additional obstacles to knowledge transition (see Fig. 2).
Jamboard example from small group discussions on day 2. These responses are from one of the small groups and the colors represent different participants.
Citation: Bulletin of the American Meteorological Society 105, 4; 10.1175/BAMS-D-23-0310.1
Participants reported that collaboration within the Weather Enterprise involves knowledge and practices being shared both within and outside of organizations. General agreement was that a wide network of actors generates momentum for change and can often assist in bypassing internal hurdles for integrating new knowledge, such as “red tape” (see Fig. 2). Likewise, such networks can promote the sharing, testing, and integration of social science informed ideas and practices across the sectors (see Fig. 1). Still, some participants acknowledged it can be unclear when social science findings are actionable and how best to operationalize them. Participants noted, for example, that there appear to be few guidelines for how to use social science outcomes gleaned from others in their communication with the public.
Collaboration occurs for participants in both informal and formal ways. Informal collaboration is achieved through emails, in-person discussions, and the processes of catalyzing ideas into action. For example, participants noted that workshops and conference panels provide brief opportunities for actors throughout the Weather Enterprise to contribute ideas and take new knowledge back to their respective workplaces. However, workplaces themselves can have policies that dampen these efforts. Although all three sectors represented in the workshop report supporting and participating in such collaborative spaces, those in the private sector shared that sometimes the collaborative process can be hindered by limited resources or a company’s interest in protecting its intellectual property. Formal collaboration, on the other hand, is achieved by building a network of longstanding relationships and communication channels both within and outside an organization (e.g., via local news stations, NWS forecast offices, public officials, and social media). Participants shared that established collaboration networks can drive collective success in communicating with different publics, whether through educational outreach opportunities or more widely disseminating risk information. Participants’ experiences highlight the importance of fostering relationships with others to affect change.
Research gaps and next steps
In addition to highlighting the processual and sociopolitical challenges of transitioning knowledge to different workplaces, participants noted some missing pieces of the research transition puzzle. First is an understanding of the lack of clarity for research transitions. Many reported that the process of knowledge transfer and implementation can be abstract (e.g., poorly defined), underdocumented, and highly dynamic over time. Participants would like to see the concept of research transitions (e.g., R2O, R2X) more clearly described within their respective organizations, especially as it relates to social science knowledge integration. Second, participants emphasized that measuring the effectiveness of outcomes from research that has been transitioned would aid in, and perhaps justify, the implementation of future transition practices in their respective workplaces. Finally, many participants noted that there are other contextual challenges related to transitioning knowledge that fail to get considered in these complex and time-consuming efforts. Mental health and critical incident stress due to burnout, for example, emerged as an important aspect of this labor, which is often unsupported or underaccounted for in research transitions. Ideally, intentional design and consideration of cross-sector knowledge contexts and its transfer would prioritize the labor intensiveness, messiness, and accessibility of information and promote collaboration within a wide variety of weather-sector contexts and applications. Next steps for the team include offering more collaborative workshops to understand better the frameworks, activities, and opportunities for social science research transitions.
Acknowledgments.
We wish to thank workshop participants for their time, insights, and assistance with reviewing this workshop summary. Participants included Tom Bedard, Erica Grow Cei, Katy Christian, Christina Crowe, Gina Eosco, Jessica Fieux, Justin Gibbs, Jill Hardy, Doug Hilderbrand, Paul Kirkwood, Ji Sun Lee, Maureen McCann, Matt Moreland, Ashley Morris, Jessica Schauer, Alan Sealls, Cassandra Shivers-Williams, and Castle Williamsberg. This workshop was conducted as part of NOAA Award NA21OAR4590214.
References
Demuth, J. L., E. Gruntfest, R. E. Morss, S. Drobot, and J. K. Lazo, 2007: WAS*IS: Building a community for integrating meteorology and social science. Bull. Amer. Meteor. Soc., 88, 1729–1738, https://doi.org/10.1175/BAMS-88-11-1729.
Eosco, G., and C. Williams, 2022: Readiness levels, transition plans, and knowledge transfers, oh my! NOAA Central Library, 28 June 2022, YouTube, https://www.youtube.com/watch?v=4MFkV88qlyA.
Etzkowitz, H., and L. Leydesdorff, 1996: The future location of research: A Triple Helix of university-industry-government relations, II. EASST Rev., 15, 20–25.
Fothergill, A., 2000: Knowledge transfer between researchers and practitioners. Nat. Hazards Rev., 1, 91–98, https://doi.org/10.1061/(ASCE)1527-6988(2000)1:2(91).
Gibbons, M., C. Limoges, and H. Nowotny, 2005: Evolution of knowledge production. Knowledge: Critical Concepts, Vol. 2, Routledge, 274–301.
Given, L., Ed., 2008: The SAGE Encyclopedia of Qualitative Research Methods. Sage, 1072 pp.
Henderson, J., E. R. Nielsen, G. R. Herman, and R. S. Schumacher, 2020: A hazard multiple: Overlapping tornado and flash flood warnings in a National Weather Service Forecast Office in the southeastern United States. Wea. Forecasting, 35, 1459–1481, https://doi.org/10.1175/WAF-D-19-0216.1.
Kudrna, F., G. Wheeler, and J. V. Byrne, 2008: Engaging NOAA’s constituents: A report to the NOAA science advisory board. NOAA, 46 pp., https://repository.library.noaa.gov/view/noaa/36452.
McLuckie, B., 1973: The warning system: A social science perspective. National Weather Service, 66 pp.
NASEM, 2018: Integrating Social and Behavioral Sciences within the Weather Enterprise. National Academies Press, 160 pp.
National Research Council, 2003: Satellite Observations of the Earth’s Environment: Accelerating the Transition of Research to Operations. The National Academies Press, 163 pp., https://doi.org/10.17226/10658.
Nielsen, E. R., G. R. Herman, R. C. Tournay, J. M. Peters, and R. S. Schumacher, 2015: Double impact: When both tornadoes and flash floods threaten the same place at the same time. Wea. Forecasting, 30, 1673–1693, https://doi.org/10.1175/WAF-D-15-0084.1.
NOAA, 2016: NAO 216-105B: Policy on research and development transitions. 11 pp., https://www.noaa.gov/organization/administration/nao-216-105b-policy-on-research-and-development-transitions.
NOAA, 2017: Policy on research and development transitions. Procedural Handbook, 24 pp., https://www.noaa.gov/sites/default/files/legacy/document/2020/Mar/Handbook_NAO216-105B_03-21-17.pdf.
NOAA, 2020: Social & behavioral science research to operations workshop. Workshop Rep., 64 pp., https://repository.library.noaa.gov/view/noaa/27398.
NOAA, 2022: Transition plans. NOAA Office of Research Transition and Application, https://orta.research.noaa.gov/plans/.
Schneider, F., and Coauthors, 2021: Co-production of knowledge and sustainability transformations: A strategic compass for global research networks. Curr. Opin. Environ. Sustainability, 49, 127–142, https://doi.org/10.1016/j.cosust.2021.04.007.
Symonds, J., and S. Gorard, 2010: Death of mixed methods? Or the rebirth of research as a craft. Eval. Res. Educ., 23, 121–136, https://doi.org/10.1080/09500790.2010.483514.
Williams, B. D., 2021: Understanding where emergency management gets the knowledge to solve the problems they face: Where are we more than 20 years after the IJMED special edition calls on closing the gap? Int. J. Mass Emerg. Disasters, 39, 417–433, https://doi.org/10.1177/028072702103900305.