• Aksenov, Y., E. E. Popova, A. Yool, A. G. Nurser, T. D. Williams, L. Bertino, and J. Bergh, 2017: On the future navigability of Arctic sea routes: High-resolution projections of the Arctic Ocean and sea ice. Mar. Policy, 75, 300317, https://doi.org/10.1016/j.marpol.2015.12.027.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Alexander, M., and S. Dessai, 2019: What can climate services learn from the broader services literature? Climatic Change, 157, 133149, https://doi.org/10.1007/s10584-019-02388-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Arctic Council, 2009: Arctic Marine Shipping Assessment 2009 Report. NOAA/Pacific Marine Environmental Laboratory, 194 pp., https://www.pmel.noaa.gov/arctic-zone/detect/documents/AMSA_2009_Report_2nd_print.pdf.

  • Arhippainen, L., and M. Tähti, 2003: Empirical evaluation of user experience in two adaptive mobile application prototypes. Proc. Second Int. Conf. on Mobile and Ubiquitous Multimedia, Linköping, Sweden, ACM, 2734, https://ep.liu.se/ecp/011/007/ecp011007.pdf.

    • Search Google Scholar
    • Export Citation
  • Arnal, L., M.-H. Ramos, E. C. de Perez, H. L. Cloke, E. Stephens, F. Wetterhall, S. J. van Andel, and F. Pappenberger, 2016: Willingness-to-pay for a probabilistic flood forecast: A risk-based decision-making game. Hydrol. Earth Syst. Sci., 20, 31093128, https://doi.org/10.5194/hess-20-3109-2016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bainbridge, L., 1983: Ironies of automation. Analysis, Design and Evaluation of Man–Machine Systems, Elsevier, 129–135.

    • Crossref
    • Export Citation
  • Becu, N., 2020: Les courants d’influence et la pratique de la simulation participative: Contours, design et contributions aux changements sociétaux et organisationnels dans les territoires. M.S. thesis, Dept. of Geography, La Rochelle University, 258 pp., https://hal.archives-ouvertes.fr/tel-02515352/document.

  • Becu, N., and Coauthors, 2017: Participatory simulation to foster social learning on coastal flooding prevention. Environ. Modell. Software, 98, 111, https://doi.org/10.1016/j.envsoft.2017.09.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berle, Ø., J. B. Rice Jr., and B. E. Asbjørnslett, 2011: Failure modes in the maritime transportation system: A functional approach to throughput vulnerability. Marit. Policy Manage., 38, 605632, https://doi.org/10.1080/03088839.2011.615870.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bibby, K., F. Margulies, J. Rijnsdorp, R. Withers, and I. Makarov, 1975: Man’s role in control systems. IFAC Proc., 8, 664683, https://doi.org/10.1016/S1474-6670(17)67612-2.

    • Search Google Scholar
    • Export Citation
  • Blair, B., and M. Müller-Stoffels, 2019: Maritime futures 2035: The Arctic region. Wageningen University and Research Workshop Rep. and Tech. Doc., 92 pp., http://salienseas.com/wp-content/uploads/2019/06/Workshop_Report.pdf.

  • Blair, B., O. A. Lee, and M. Lamers, 2020: Four paradoxes of the user–provider interface: A responsible innovation framework for sea ice services. Sustainability, 12, 448, https://doi.org/10.3390/su12020448.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bommel, P., 2020: Participatory modelling and interactive simulation to support the management of the commons. HDR thesis, Université Montpellier II, 89 pp., https://hal.archives-ouvertes.fr/tel-02568654/document.

  • Boyd, E., B. Nykvist, S. Borgström, and I. A. Stacewicz, 2015: Anticipatory governance for social-ecological resilience. Ambio, 44, 149161, https://doi.org/10.1007/s13280-014-0604-x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brandsen, T., and M. Honingh, 2016: Distinguishing different types of coproduction: A conceptual analysis based on the classical definitions. Public Adm. Rev., 76, 427435, https://doi.org/10.1111/puar.12465.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bremer, S., A. Wardekker, S. Dessai, S. Sobolowski, R. Slaattelid, and J. van der Sluijs, 2019: Toward a multi-faceted conception of co-production of climate services. Climate Serv., 13, 4250, https://doi.org/10.1016/j.cliser.2019.01.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brudney, J. L., and R. E. England, 1983: Toward a definition of the coproduction concept. Public Adm. Rev., 43, 5965, https://doi.org/10.2307/975300.

  • Brunet, G., and Coauthors, 2010: Collaboration of the weather and climate communities to advance subseasonal-to-seasonal prediction. Bull. Amer. Meteor. Soc., 91, 13971406, https://doi.org/10.1175/2010BAMS3013.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chilvers, J., and M. Kearnes, 2020: Remaking participation in science and democracy. Sci. Technol. Hum. Values, 45, 347380, https://doi.org/10.1177/0162243919850885.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crochemore, L., M.-H. Ramos, F. Pappenberger, S. J. Andel, and A. W. Wood, 2016: An experiment on risk-based decision-making in water management using monthly probabilistic forecasts. Bull. Amer. Meteor. Soc., 97, 541551, https://doi.org/10.1175/BAMS-D-14-00270.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crookall, D., 2010: Serious games, debriefing, and simulation/gaming as a discipline. Simul. Gaming, 41, 898920, https://doi.org/10.1177/1046878110390784.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crookall, D., 2014: Engaging (in) gameplay and (in) debriefing. Simul. Gaming, 45, 416427, https://doi.org/10.1177/1046878114559879.

  • Crookall, D., and N. Becu, 2020: Companion modelling and participatory simulation: A glimpse. EGU General Assembly, EGU, Abstract EGU2020-8968, https://doi.org/10.5194/egusphere-egu2020-8968.

    • Crossref
    • Export Citation
  • DeMott, C., Á. G. Muñoz, C. D. Roberts, C. M. Spillman, and F. Vitart, 2021: The benefits of better ocean weather forecasting. Eos, 102, https://doi.org/10.1029/2021EO210601.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dilling, L., and M. C. Lemos, 2011: Creating usable science: Opportunities and constraints for climate knowledge use and their implications for science policy. Global Environ. Change, 21, 680689, https://doi.org/10.1016/j.gloenvcha.2010.11.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doddema, M., 2019: Employing a role playing game and debriefing approach to validate practices and identify variations in response dynamics. MethodsX, 6, 143149, https://doi.org/10.1016/j.mex.2018.12.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fleming, K., and Coauthors, 2020: The use of serious games in engaging stakeholders for disaster risk reduction, management and climate change adaption information elicitation. Int. J. Disaster Risk Reduct., 49, 101669, https://doi.org/10.1016/j.ijdrr.2020.101669.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Flood, S., N. A. Cradock-Henry, P. Blackett, and P. Edwards, 2018: Adaptive and interactive climate futures: Systematic review of ‘serious games’ for engagement and decision-making. Environ. Res. Lett., 13, 063005, https://doi.org/10.1088/1748-9326/aac1c6.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gascard, J.-C., K. Riemann-Campe, R. Gerdes, H. Schyberg, R. Randriamampianina, M. Karcher, J. Zhang, and M. Rafizadeh, 2017: Future sea ice conditions and weather forecasts in the Arctic: Implications for Arctic shipping. Ambio, 46, 355367, https://doi.org/10.1007/s13280-017-0951-5.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gausemeier, J., A. Fink, and O. Schlake, 1998: Scenario management: An approach to develop future potentials. Technol. Forecasting Soc. Change, 59, 111130, https://doi.org/10.1016/S0040-1625(97)00166-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Giorgetta, M. A., and Coauthors, 2013: Climate and carbon cycle changes from 1850 to 2100 in MPI-ESM simulations for the Coupled Model Intercomparison Project phase 5. J. Adv. Model. Earth Syst., 5, 572597, https://doi.org/10.1002/jame.20038.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goessling, H. F., and T. Jung, 2018: A probabilistic verification score for contours: Methodology and application to Arctic ice-edge forecasts. Quart. J. Roy. Meteor. Soc., 144, 735743, https://doi.org/10.1002/qj.3242.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hovelsrud, G. K., B. Poppel, B. Van Oort, and J. D. Reist, 2011: Arctic societies, cultures, and peoples in a changing cryosphere. Ambio, 40, 100110, https://doi.org/10.1007/s13280-011-0219-4.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • IPCC, 2019: Summary for policymakers. IPCC Special Report on the Ocean and Cryosphere in a Changing Climate, H.-O. Pörtner et al., Eds., Cambridge University Press, 36 pp., https://www.ipcc.ch/site/assets/uploads/sites/3/2019/11/03_SROCC_SPM_FINAL.pdf.

  • Jeuring, J., and M. Knol-Kauffman, 2019: Mapping weather, water, ice and climate knowledge & information needs for maritime activities in the Arctic. Survey Rep., 44 pp., https://salienseas.com/wp-content/uploads/2019/09/SALIENSEAS-Mapping-WWIC-Knowledge-and-Information-Needs-for-Maritime-Activities-in-the-Arctic_FINAL_20190830.pdf.

  • Jeuring, J., M. Knol-Kauffman, and A. Sivle, 2020: Toward valuable weather and sea-ice services for the marine Arctic: Exploring user–producer interfaces of the Norwegian Meteorological Institute. Polar Geogr., 43, 139159, https://doi.org/10.1080/1088937X.2019.1679270.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Johnson, S. J., and Coauthors, 2019: SEAS5: The new ECMWF seasonal forecast system. Geosci. Model Dev., 12, 10871117, https://doi.org/10.5194/gmd-12-1087-2019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Juliani, A. and Coauthors 2018. Unity: A general platform for intelligent agents. arXiv 1809.02627, 28 pp., https://arxiv.org/pdf/1809.02627.pdf.

    • Search Google Scholar
    • Export Citation
  • Kapp, K. M., 2012: The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education. John Wiley and Sons, 336 pp.

    • Crossref
    • Export Citation
  • Kern, S., A. Rösel, L. T. Pedersen, N. Ivanova, R. Saldo, and R. T. Tonboe, 2016: The impact of melt ponds on summertime microwave brightness temperatures and sea-ice concentrations. Cryosphere, 10, 22172239, https://doi.org/10.5194/tc-10-2217-2016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kern, S., T. Lavergne, D. Notz, L. T. Pedersen, and R. Tonboe, 2020: Satellite passive microwave sea-ice concentration data set inter-comparison for Arctic summer conditions. Cryosphere, 14, 24692493, https://doi.org/10.5194/tc-14-2469-2020.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Knol, M., P. Arbo, P. Duske, S. Gerland, M. Lamers, O. Pavlova, A. D. Sivle, and S. Tronstad, 2018: Making the Arctic predictable: The changing information infrastructure of Arctic weather and sea ice services. Polar Geogr., 41, 279293, https://doi.org/10.1080/1088937X.2018.1522382.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lamers, M., P. Duske, and L. van Bets, 2018a: Understanding user needs: A practice-based approach to exploring the role of weather and sea ice services in European Arctic expedition cruising. Polar Geogr., 41, 262278, https://doi.org/10.1080/1088937X.2018.1513959.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lamers, M., M. Knol, M. Müller, B. Blair, J. Jeuring, T. Rasmussen, and A. Sivle, 2018b: Enhancing the Saliency of Climate Services for Marine Mobility Sectors in European Arctic Seas (SALIENSEAS). Wageningen University and Research Social Sciences Group Stakeholder Advisory Group Workshop Rep., 28 pp.

  • Le Page, C., and Coauthors, 2013: Agent-based modelling and simulation applied to environmental management. Simulating Social Complexity, Springer, 499–540.

    • Crossref
    • Export Citation
  • Litinski, V., 2013: Games for health: Building the case. Int. J. Gaming Comput.-Mediated Simulations, 5, 108115, https://doi.org/10.4018/jgcms.2013070108.

    • Search Google Scholar
    • Export Citation
  • Lovecraft, A. L., N. Fresco, D. Cost, and B. Blair, 2017: Northern Alaska Scenarios Project report: Creating healthy, sustainable communities in Arctic Alaska. Communities in Arctic Alaska, 84 pp., https://uaf-snap.org/wp-content/uploads/2020/08/NASP-Report-2017_small.pdf.

  • Lovreglio, R., V. Gonzalez, Z. Feng, R. Amor, M. Spearpoint, J. Thomas, M. Trotter, and R. Sacks, 2018: Prototyping virtual reality serious games for building earthquake preparedness: The Auckland City Hospital case study. Adv. Eng. Inform., 38, 670682, https://doi.org/10.1016/j.aei.2018.08.018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McNie, E. C., 2012: Delivering climate services: Organizational strategies and approaches for producing useful climate-science information. Wea. Climate Soc., 5, 1426, https://doi.org/10.1175/WCAS-D-11-00034.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meadow, A. M., D. B. Ferguson, Z. Guido, A. Horangic, G. Owen, and T. Wall, 2015: Moving toward the deliberate coproduction of climate science knowledge. Wea. Climate Soc., 7, 179191, https://doi.org/10.1175/WCAS-D-14-00050.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Miller, C. A., and C. Wyborn, 2018: Co-production in global sustainability: Histories and theories. Environ. Sci. Policy, 113, 88–95, https://doi.org/10.1016/j.envsci.2018.01.016.

    • Search Google Scholar
    • Export Citation
  • Mulder, K. J., M. Lickiss, A. Black, A. J. Charlton‐Perez, R. McCloy, and J. S. Young, 2020: Designing environmental uncertainty information for experts and non-experts: Does data presentation affect users’ decisions and interpretations? Meteor. Appl., 27, e1821, https://doi.org/10.1002/met.1821.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • O’Brien, H. L., and E. G. Toms, 2008: What is user engagement? A conceptual framework for defining user engagement with technology. J. Amer. Soc. Inf. Sci. Technol., 59, 938955, https://doi.org/10.1002/asi.20801.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ostrom, E., 1996: Crossing the great divide: Coproduction, synergy, and development. World Dev., 24, 10731087, https://doi.org/10.1016/0305-750X(96)00023-X.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palerme, C., and M. Müller, 2021: Calibration of sea ice drift forecasts using random forest algorithms. Cryosphere, 15, 39894004, https://doi.org/10.5194/tc-15-3989-2021.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palerme, C., M. Müller, and A. Melsom, 2019: An intercomparison of verification scores for evaluating the sea ice edge position in seasonal forecasts. Geophys. Res. Lett., 46, 47574763, https://doi.org/10.1029/2019GL082482.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palerme, C., P. Kountouris, A. Stocker, M. Knol-Kauffman, P. Cochrane, M. Müller, and L. Rabenstein, 2021: Delivering high quality sea-ice information around the Svalbard archipelago to marine end-users. J. Oper. Oceanogr., 14 (Suppl. 1), 109–113, https://doi.org/10.1080/1755876X.2021.1946240.

    • Crossref
    • Export Citation
  • Palma, D., and Coauthors, 2019: Cruising the marginal ice zone: Climate change and Arctic tourism. Polar Geogr., 42, 215235, https://doi.org/10.1080/1088937X.2019.1648585.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Parker, W. S., and G. Lusk, 2019: Incorporating user values into climate services. Bull. Amer. Meteor. Soc., 100, 16431650, https://doi.org/10.1175/BAMS-D-17-0325.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rainville, L., and Coauthors, 2020: Improving situational awareness in the Arctic Ocean. Front. Mar. Sci., 7, 581139, https://doi.org/10.3389/fmars.2020.581139.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ricker, R., S. Hendricks, L. Kaleschke, X. Tian-Kunze, J. King, and C. Haas, 2017: A weekly Arctic sea-ice thickness data record from merged CryoSat-2 and SMOS satellite data. Cryosphere, 11, 16071623, https://doi.org/10.5194/tc-11-1607-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sardar, Z., 2010: The namesake: Futures; futures studies; futurology; futuristic; foresight—What’s in a name? Futures, 42, 177184, https://doi.org/10.1016/j.futures.2009.11.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sarewitz, D., and R. A. Pielke Jr., 2007: The neglected heart of science policy: Reconciling supply of and demand for science. Environ. Sci. Policy, 10, 5–16, https://doi.org/10.1016/j.envsci.2006.10.001.

    • Crossref
    • Export Citation
  • Saritas, O., 2013: Systemic foresight methodology. Science, Technology and Innovation Policy for the Future: Potentials and Limits of Foresight Studies, D. Meissner, L. Gokhberg, and A. Sokolov, Eds., Springer, 83–117.

    • Crossref
    • Export Citation
  • Scott, D. J., C. J. Lemieux, and L. Malone, 2011: Climate services to support sustainable tourism and adaptation to climate change. Climate Res., 47, 111122, https://doi.org/10.3354/cr00952.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sheldon, L., 2020: The Multiplayer Classroom: Designing Coursework as a Game. CRC Press, 338 pp.

    • Crossref
    • Export Citation
  • Sheppard, S. R., A. Shawa, D. Flanders, S. Burch, A. Wiek, J. Carmichael, J. Robinson, and S. Cohen, 2011: Future visioning of local climate change: A framework for community engagement and planning with scenarios and visualisation. Futures, 43, 400412, https://doi.org/10.1016/j.futures.2011.01.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Skinner, C., 2020: Flash Flood!: A SeriousGeoGames activity combining science festivals, video games, and virtual reality with research data for communicating flood risk and geomorphology. Geosci. Commun., 3, 117, https://doi.org/10.5194/gc-3-1-2020.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stilgoe, J., R. Owen, and P. Macnaghten, 2013: Developing a framework for responsible innovation. Res. Policy, 42, 15681580, https://doi.org/10.1016/j.respol.2013.05.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stocker, A. N., A. H. Renner, and M. Knol-Kauffman, 2020: Sea ice variability and maritime activity around Svalbard in the period 2012–2019. Sci. Rep., 10, 17043, https://doi.org/10.1038/s41598-020-74064-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Student, J., M. Lamers, and B. Amelung, 2020: A dynamic vulnerability approach for tourism destinations. J. Sustainable Tourism, 28, 475496, https://doi.org/10.1080/09669582.2019.1682593.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tall, A., J. Hansen, A. Jay, B. M. Campbell, J. Kinyangi, P. K. Aggarwal, and R. Zougmoré, et al.2014: Scaling up climate services for farmers: Mission Possible. Learning from good practice in Africa and South Asia. CCAFS Rep. 13, 44 pp., https://hdl.handle.net/10568/42445.

  • Terrado, M., and Coauthors, 2019: The Weather Roulette: A game to communicate the usefulness of probabilistic climate predictions. Bull. Amer. Meteor. Soc., 100, 19091921, https://doi.org/10.1175/BAMS-D-18-0214.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tietsche, S., and Coauthors, 2020: Recommendations for more user-relevant sea-ice forecasts. European Centre for Medium-Range Weather Forecasts KEPLER Deliverable Rep. D4.3, 30 pp., https://drive.google.com/file/d/1SGCW4TCBYI7jEEWbziZI0aOuoCnAtiHD/view.

  • Turnhout, E., T. Metze, C. Wyborn, N. Klenk, and E. Louder, 2020: The politics of co-production: Participation, power, and transformation. Curr. Opin. Environ. Sustainability, 42, 1521, https://doi.org/10.1016/j.cosust.2019.11.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van den Hoogen, J., J. Lo, and S. Meijer, 2016: Debriefing research games: Context, substance and method. Simul. Gaming, 47, 368388, https://doi.org/10.1177/1046878116651023.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaughan, C., S. Dessai, and C. Hewitt, 2018: Surveying climate services: What can we learn from a bird’s-eye view? Wea. Climate Soc., 10, 373395, https://doi.org/10.1175/WCAS-D-17-0030.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vincent, K., M. Daly, C. Scannell, and B. Leathes, 2018: What can climate services learn from theory and practice of co-production? Climate Serv., 12, 4858, https://doi.org/10.1016/j.cliser.2018.11.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Voinov, A., and F. Bousquet, 2010: Modelling with stakeholders. Environ. Modell. Software, 25, 12681281, https://doi.org/10.1016/j.envsoft.2010.03.007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Voinov, A., and Coauthors, 2018: Tools and methods in participatory modeling: Selecting the right tool for the job. Environ. Modell. Software, 109, 232255, https://doi.org/10.1016/j.envsoft.2018.08.028.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wagner, P. M., O. J. Hegelund, N. Hughes, K. Qvistgaard, J. Seppänen, P. Eriksson, A. Kangas, and I. Grönfeldt, 2019: Stakeholder needs: Maritime sector needs. Norwegian Meteorological Institute KEPLER Deliverable Rep. D1.1, 68 pp., https://drive.google.com/file/d/15C9_YGk63MxjhR6BtNMiWCjZ8hU4PbIy/view.

  • Wagner, P. M., and Coauthors, 2020: Sea-ice information and forecast needs for industry maritime stakeholders. Polar Geogr., 43, 160187, https://doi.org/10.1080/1088937X.2020.1766592.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, M., and J. E. Overland, 2012: A sea ice free summer Arctic within 30 years: An update from CMIP5 models. Geophys. Res. Lett., 39, L18501, https://doi.org/10.1029/2012GL052868.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weichselgartner, J., and B. Arheimer, 2019: Evolving climate services into knowledge–action systems. Wea. Climate Soc., 11, 385399, https://doi.org/10.1175/WCAS-D-18-0087.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • White, C. J., and Coauthors, 2017: Potential applications of subseasonal‐to‐seasonal (S2S) predictions. Meteor. Appl., 24, 315325, https://doi.org/10.1002/met.1654.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Whitton, N., 2011: Game engagement theory and adult learning. Simul. Gaming, 42, 596609, https://doi.org/10.1177/1046878110378587.

  • Whitton, N., and A. Moseley, 2014: Deconstructing engagement: Rethinking involvement in learning. Simul. Gaming, 45, 433449, https://doi.org/10.1177/1046878114554755.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    The sequence of project activities and stakeholder engagements.

  • View in gallery
    Fig. 2.

    A screenshot of the S2S sea ice forecast service depicting (sub)seasonal sea ice probabilities.

  • View in gallery
    Fig. 3.

    (a) Probability in weeks during which the S2S sea ice forecasts outperform climatology (assessed from the forecasts starting between 1999 and 2014). The climatology is defined as the mean observed sea ice conditions during the 10 years preceding the forecasts. The three probability columns represent the duration in weeks during which the probability that the forecasts are better than the climatology is 75%, 50%, and 25%. The reliabilities of the 2019 period of the simulation were based on these estimates. (b) The simulated 2035 reliabilities used only for gaming purposes to approximate the skill of a future S2S sea ice forecast. The linked 2035 projection in the game was based on the global coupled ocean–atmosphere–land MPI-ESM-LR for the period 2030–39.

  • View in gallery
    Fig. 4.

    Twelve illustrated future projections from the 2035 most robust 2035 scenario bundle (Blair and Muller-Stoffels 2019). Panel captions follow the format “key factor | future projection”: (a) accessibility of Arctic sea routes | easy access; (b) user-centric information infrastructures and data | few specialized actors; (c) predictability of sea ice variability | gradual improvements in predictive models; (d) regulations and policies affecting Arctic operations | economic and commercial uses dominate; (e) demand for Arctic resources | seafood first; (f) global economic trends | Arctic rush; (g) geopolitical stability | status quo (occasional bullying); (h) major incidents and critical events | status quo; (i) sustainable and resilient Arctic communities | expatriate haven; (j) trajectory of development in marine technologies | techno-utopia for some, stormy seas for others; (k) fluctuating energy prices | northern push; and (l) China’s strategic plan | Chinese finger cuffs. The illustrations are by B. Köhler.

  • View in gallery
    Fig. 5.

    (a) The game console showing an event card and wild card narrative in play based on a roll-the-dice game mechanism. The console shows the current year, available bank, and reputation points (pink bar). (b) The itinerary view informing players about the voyage duration, planned activities, main ports of call, and vessel specifications. Itineraries are based on project partner Hurtigruten’s cruise portfolio and are used with permission.

  • View in gallery
    Fig. 6.

    The simulation flow.

  • View in gallery
    Fig. 7.

    Date choices for each itinerary. Months show the number of players who chose a date in each month. Red denotes the highest frequency.

  • View in gallery
    Fig. 8.

    Results from eight evaluative statements during individual debriefing between rounds, showing how many of the six mariners placed marks on the smiley-face side of the scale at the beginning (round 3, in 2019) and at the end (round 12, in 2035) of the simulation.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1277 1268 37
PDF Downloads 616 609 37

Coproducing Sea Ice Predictions with Stakeholders Using Simulation

Berill BlairaWageningen University and Research, Wageningen, Netherlands

Search for other papers by Berill Blair in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-3298-9816
,
Malte MüllerbNorwegian Meteorological Institute, Oslo, Norway
cUniversity of Oslo, Oslo, Norway

Search for other papers by Malte Müller in
Current site
Google Scholar
PubMed
Close
,
Cyril PalermebNorwegian Meteorological Institute, Oslo, Norway

Search for other papers by Cyril Palerme in
Current site
Google Scholar
PubMed
Close
,
Rayne BlairdEszterházy Károly University, Eger, Hungary
eUtrecht University, Utrecht, Netherlands

Search for other papers by Rayne Blair in
Current site
Google Scholar
PubMed
Close
,
David CrookallfUniversité International de la Mer, Cagnes-sur-Mer, France
gUniversité Côte d’Azur, Nice, France

Search for other papers by David Crookall in
Current site
Google Scholar
PubMed
Close
,
Maaike Knol-KauffmanhUiT The Arctic University of Norway, Tromsø, Norway

Search for other papers by Maaike Knol-Kauffman in
Current site
Google Scholar
PubMed
Close
, and
Machiel LamersaWageningen University and Research, Wageningen, Netherlands

Search for other papers by Machiel Lamers in
Current site
Google Scholar
PubMed
Close
Open access

Abstract

Forecasts of sea ice evolution in the Arctic region for several months ahead can be of considerable socioeconomic value for a diverse range of marine sectors and for local community supply logistics. However, subseasonal-to-seasonal (S2S) forecasts represent a significant technical challenge, and translating user needs into scientifically manageable procedures and robust user confidence requires collaboration among a range of stakeholders. We developed and tested a novel, transdisciplinary coproduction approach that combined socioeconomic scenarios and participatory, research-driven simulation gaming to test a new S2S sea ice forecast system with experienced mariners in the cruise tourism sector. Our custom-developed computerized simulation game known as “ICEWISE” integrated sea ice parameters, forecast technology, and human factors as a participatory environment for stakeholder engagement. We explored the value of applications-relevant S2S sea ice prediction and linked uncertainty information. Results suggest that the usefulness of S2S services is currently most evident in schedule-dependent sectors but is expected to increase as a result of anticipated changes in the physical environment and continued growth in Arctic operations. Reliable communication of uncertainty information in sea ice forecasts must be demonstrated and trialed before users gain confidence in emerging services and technologies. Mariners’ own intuition, experience, and familiarity with forecast service provider reputation impact the extent to which sea ice information may reduce uncertainties and risks for Arctic mariners. Our insights into the performance of the combined foresight/simulation coproduction model in brokering knowledge across a range of domains demonstrates promise. We conclude with an overview of the potential contributions from S2S sea ice predictions and from experiential coproduction models to the development of decision-driven and science-informed climate services.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Berill Blair, berill.blair@wur.nl

Abstract

Forecasts of sea ice evolution in the Arctic region for several months ahead can be of considerable socioeconomic value for a diverse range of marine sectors and for local community supply logistics. However, subseasonal-to-seasonal (S2S) forecasts represent a significant technical challenge, and translating user needs into scientifically manageable procedures and robust user confidence requires collaboration among a range of stakeholders. We developed and tested a novel, transdisciplinary coproduction approach that combined socioeconomic scenarios and participatory, research-driven simulation gaming to test a new S2S sea ice forecast system with experienced mariners in the cruise tourism sector. Our custom-developed computerized simulation game known as “ICEWISE” integrated sea ice parameters, forecast technology, and human factors as a participatory environment for stakeholder engagement. We explored the value of applications-relevant S2S sea ice prediction and linked uncertainty information. Results suggest that the usefulness of S2S services is currently most evident in schedule-dependent sectors but is expected to increase as a result of anticipated changes in the physical environment and continued growth in Arctic operations. Reliable communication of uncertainty information in sea ice forecasts must be demonstrated and trialed before users gain confidence in emerging services and technologies. Mariners’ own intuition, experience, and familiarity with forecast service provider reputation impact the extent to which sea ice information may reduce uncertainties and risks for Arctic mariners. Our insights into the performance of the combined foresight/simulation coproduction model in brokering knowledge across a range of domains demonstrates promise. We conclude with an overview of the potential contributions from S2S sea ice predictions and from experiential coproduction models to the development of decision-driven and science-informed climate services.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Berill Blair, berill.blair@wur.nl

1. Introduction

A recent IPCC report has yet again highlighted the dramatic effects of climate change in the Arctic (IPCC 2019). The region is warming twice as fast as the rest of the planet and dramatic impacts from rapid warming are evident on land and at sea alike. Over the past 40 years, rapid warming has pushed sea ice into a melting and thinning trend, and Arctic sea ice extent has decreased for all months of the year. As sea ice becomes thinner and more fragmented, the influence of surface waves increases, causing significant shifts in the dynamic and thermodynamic properties of sea ice (Aksenov et al. 2017).

These environmental changes are increasing the dynamics and complexities of the decision context for Arctic mariners (Gascard et al. 2017) and have profound socioeconomic consequences (Hovelsrud et al. 2011). Safe maritime waterways play an important role both in Arctic regional economies and at the global scale, with 80% of global trade in goods being transported by ship (Berle et al. 2011). Climatic changes in the Arctic are propelling growth in shipping, tourism, and fisheries (Arctic Council 2009). For example, researchers already observe shifts in the Svalbard, Norway, cruising season, stretching seasons to earlier in the spring and later in the autumn (Stocker et al. 2020). This increases the demand for salient Arctic weather and climate predictions for a variety of time scales, which in turn places expectations on our current global and regional forecasting systems. However, the transferability of information services in remote Arctic Ocean settings is more restricted when compared with more temperate zones because of limited connectivity to communication satellites (Lamers et al. 2018a,b). Moreover, the field of sea ice information provision is relatively new, resulting in a growing, but still limited, familiarity between users and producers about information needs, requirements, and technical possibilities (Wagner et al. 2019). One of the key developments in meteorological and physical oceanographic (metocean) information delivery is the ability to move beyond observation of ice conditions in the form of sea ice charts toward reliable sea ice forecasts days, weeks and even months ahead.

In other words, while improving access to and quality of climate-relevant information is particularly pertinent to those living or operating in remote and dynamic polar-marine environments, it is often unclear at what time scale environmental information is needed. Questions include whether longer-range metocean forecasts would be of use, and what constitutes real salience for stakeholders. In this regard, research efforts have focused on considering the context of Arctic marine activities for better provision and dissemination of credible products and services to increase the safety and resilience of marine operations (e.g., Rainville et al. 2020; Tietsche et al. 2020; Wagner et al. 2019, 2020).

Under the project Enhancing the Saliency of Climate Services for Marine Mobility Sectors in European Arctic Seas (SALIENSEAS), a multinational consortium of scientists has coproduced with users, improved services for subseasonal-to-seasonal (S2S) sea ice forecasts (defined here as time scales from 2 weeks to 3 months). S2S forecasts fill the gap between weather- and climate-scale predictions. In contrast with other, more user-centric models of innovation such as the “climate services” approach (White et al. 2017), which is driven by bottom-up coexploration with users to increase societal value, S2S prediction is typically categorized as applied research (scientific discovery driving innovation); and for which demand is increasing (e.g., Scott et al. 2011; Vaughan et al. 2018; Parker and Lusk 2019). Our transdisciplinary consortium integrated an experiential, forward-looking, user-focused approach into the development of an S2S prediction of sea ice probabilities, in an effort to extend Arctic marine actors’ time horizon for planning.

We report on a novel approach in which we engaged end users to test a new S2S forecast of sea ice probabilities. Due to our project’s focus on the European Arctic, our main stakeholders were representatives of marine sectors active around Svalbard and Greenland. Our approach served the following objectives:

  1. Explore the usability of the product under both current and under future conditions in light of the expected dynamic changes of the next 15 years (objective 1).

  2. Test the product’s potential to reduce uncertainties in the users’ decision environment and to explore users’ levels of confidence in S2S forecasting (objective 2).

  3. Gain insight into the use of participatory computerized simulation as a method for researching complex interrelations among a nexus of three domains: natural phenomena (sea ice variation), forecast technologies (assessment of seasonal sea ice risk), and human/social factors (perceptions and levels of trust and confidence; socioeconomic scenarios), especially with regard to achieving the first two objectives (objective 3).

Individual, social, and cultural factors and contexts within which a product is used, as well as the specific attributes of the product itself, all influence user engagement (Arhippainen and Tähti 2003; ’O’Brien and Toms 2008; Lamers et al. 2018a). For this reason, we were particularly interested in socioeconomic scenarios (i.e., anticipated social, economic, political, cultural, and technological changes) because, as sea ice conditions and dynamics in the Arctic Ocean are projected to change drastically by 2035 (Wang and Overland 2012), the specifics of the technologies used (ships and other equipment), the demand for Arctic marine transport, and the intensity and patterns of traffic are also likely to change. Our first objective aimed at exploring the extent to which products and services anticipate, and are resilient to, upcoming changes in the physical environment and in the users’ operational contexts. In addition, the first objective emphasizes that we explore usability. Here, we take into account that a product becomes usable when actors can relate to it and perceive it as useful and credible, that it answers to users’ needs, and that operators have sufficient capacity to use the service (e.g., technical, skills, and financial). Usability can thus only be achieved in a process of coproduction (see e.g., Dilling and Lemos 2011). Our second objective probed how users experience and gain confidence in a forecast’s reliability estimate. For example, does a threshold exist that is low enough to render predictions irrelevant? Our third objective arose in our search for a research method that would allow us to achieve the first two objectives. We aimed to develop a method that would be cost effective and yet provide sufficiently reliable results to be useful to service providers and end users.

Direct questioning about perceived risks and uncertainties during operations do not always lend themselves well to research via traditional inquiries (interviews and self-report surveys) into a complex nexus of factors, especially those arising from three different domains. Stakeholders can and do experience difficulty accurately recalling and rating past perceptions and connecting them to varying environmental conditions. As an alternative, experiential research approaches such as participatory and computerized simulation are able to provide a rich and reliable environment that also facilitates three basic goals of research: experimentation, replication, and learning. Simulation games have already been employed in some (mostly hydrometeorological) forecast service experiments (e.g., Tall et al. 2014; Arnal et al. 2016; Crochemore et al. 2016; Terrado et al. 2019) and in communicating about the risk of flooding (e.g., Skinner 2020).

Here we propose a novel framework that expands on previous research by integrating into the simulation key social–economic–technological scenarios that represent plausible technological, socioeconomic, and policy futures and also physical parameters, such as projections of sea ice conditions. We combine anticipatory methods, such as scenario development, with participatory simulation and a custom-developed computerized game. We consider the implications of our approach for brokering effective partnerships in the development of decision-driven and science-informed climate services.

2. Conceptual framework

Coproduction refers to the voluntary exchange of ideas, collaboration across organizations and disciplines, through which input from individuals and groups is transformed into goods and services (see, e.g., Brudney and England 1983; Ostrom 1996; Meadow et al. 2015; Alexander and Dessai 2019). Coproduction does not refer to a single approach to collaboration (e.g., Brandsen and Honingh 2016; Miller and Wyborn 2018) and different frameworks have been explored in the context of diverse disciplines and rationales (Turnhout et al. 2020). Coproduction in climate services should be inclusive, collaborative, and flexible (Vincent et al. 2018), so as to improve mutual understanding among actors (Bremer et al. 2019) and result in improved products that are useful, useable, and used (McNie 2012; Vaughan et al. 2018). While the rationale for coproduction varies from citizen empowerment to the depoliticization of the science–policy interface, typically all coproduction projects aim to align the production of information with their demand (e.g., Sarewitz and Pielke 2007).

Numerous monikers have been coined for the study of alternative futures and each distinct name has its own history in literature (Sardar 2010). In this paper we use foresight to denote anticipatory activities. Foresight aims to increase our understanding of systems (social, ecological, industrial/sectoral) and complex interactions and feedback to explore emergent properties and probable future system states (Saritas 2013; Boyd et al. 2015). Scenario building is a widely used foresight tool, often used to explore with stakeholders the important dynamics between present actions and future outcomes in the context of climate change (Sheppard et al. 2011; Lovecraft et al. 2017). The foresight component of our research relied on participatory scenario development and robustness analysis to produce plausible and consistent future scenarios [see section 3b(1)].

Simulation games are a category of games that have a purpose beyond entertainment, which is usually educational or instructional (e.g., Fleming et al. 2020). Game mechanics can make learning and instruction more engaging or immersive (e.g., Whitton 2011; Whitton and Moseley 2014), while providing a place for experimentation, feedback, and a sense of accomplishment (Kapp 2012). The value of simulation-gaming methods for learning, training, and instruction have been well established in the literature (e.g., Voinov and Bousquet 2010; Le Page et al. 2013; Litinski 2013; Voinov et al. 2018; Sheldon 2020), and games have been deployed, for example, to improve disaster preparedness (e.g., Lovreglio et al. 2018). Participatory simulation can also help organizations to solve complex problems or guide multistakeholder decision-making (Becu et al. 2017; Becu 2020; Bommel 2020). For example, simulations can help stakeholders to collectively explore complexities, clarify new ideas and strategies, and challenge beliefs (Flood et al. 2018; Crookall and Becu 2020; Student et al. 2020).

Debriefing is the key to the generation of learning that can result from an immersive, experiential activity such as simulation and gaming and participatory simulation (Crookall 2010; van den Hoogen et al. 2016; Doddema 2019). The simulation gaming facilitates player engagement, which results in emerging emotions, understandings, decisions, and perceptions, while debriefing provides a platform for the fruitful discussion of those experiences (Crookall 2014). The debriefing session is also often where the most valuable data are collected, which was the case in our research. Debriefing must be planned in advance and provide a structured environment.

3. Methods

The sequence of project activities and stakeholder engagements are depicted in Fig. 1.

Fig. 1.
Fig. 1.

The sequence of project activities and stakeholder engagements.

Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0048.1

a. S2S sea ice forecast: Demonstration service development

Early stages of planning for a new S2S sea ice forecast used scoping workshops (Lamers et al. 2018b), document analysis, one-on-one interviews, and surveys (Jeuring and Knol-Kauffman 2019) to collect information from key stakeholders about the spatial and temporal parameters of the problem space.

Based on stakeholder input, a website showing S2S forecasts of sea ice concentration was then developed by the Norwegian Meteorological Institute (MET Norway; Fig. 2). The forecasts are produced by the European Centre for Medium-Range Weather Forecasts seasonal prediction system 5 (SEAS5; Johnson et al. 2019) and provide probabilistic information based on 51 different simulations. The sea ice map shows probabilities for sea ice concentrations greater than 15% for 6 months into the future, with the outlook initialized starting from the date of visit to the website. The reliability of the forecast is provided for users (Fig. 3) and depends on its range (how far out it is viewed) and the season. The performances of the S2S forecasts and a climatological reference (defined as the 10 years preceding the forecast start date) were evaluated using passive microwave observations of sea ice concentration and a metric called the spatial probability score (Goessling and Jung 2018). Then, the duration during which the S2S forecasts significantly outperform the climatological reference was assessed using weekly means of the spatial probability score for each forecast during the period 1999–2014. The distribution of these durations within the 16 years of this analysis is reported in Fig. 3, and further details about this evaluation are described in Palerme et al. (2019).

Fig. 2.
Fig. 2.

A screenshot of the S2S sea ice forecast service depicting (sub)seasonal sea ice probabilities.

Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0048.1

Fig. 3.
Fig. 3.

(a) Probability in weeks during which the S2S sea ice forecasts outperform climatology (assessed from the forecasts starting between 1999 and 2014). The climatology is defined as the mean observed sea ice conditions during the 10 years preceding the forecasts. The three probability columns represent the duration in weeks during which the probability that the forecasts are better than the climatology is 75%, 50%, and 25%. The reliabilities of the 2019 period of the simulation were based on these estimates. (b) The simulated 2035 reliabilities used only for gaming purposes to approximate the skill of a future S2S sea ice forecast. The linked 2035 projection in the game was based on the global coupled ocean–atmosphere–land MPI-ESM-LR for the period 2030–39.

Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0048.1

To help stakeholders participate in the simulation-gaming exercise where the product would be trialed, a dedicated gaming version of the sea ice forecast was developed separately. Its functions were designed 1) to provide players with the possibility of initializing a forecast in any month of the year, and 2) to enable a 2035 “forecast” or long-range projection of sea ice probabilities for the 2035 period of the game. The 2035 projection was based on the global coupled ocean–atmosphere–land Max Planck Institute Earth System Model, low resolution (MPI-ESM-LR; Giorgetta et al. 2013) for the period 2030–39. Using the 2019 forecast’s reliability estimates as a baseline, we approximated probabilities for the 2035 projections to simulate expected future improvements in skillfulness. These future reliability estimates, used only for gaming purposes, simulated a 1–18-week increase in outlook (depending on the month and reliability level) relative to 2019 estimates.

The S2S sea ice forecast was pilot tested by two key stakeholders who had experience in navigating ice-infested waters and in vessel scheduling. One session was held remotely (March 2019 via Skype), the other was held in person (June 2019 in Tromsø, Norway). The sessions were semistructured to elicit spontaneous and calculated feedback. From the perspective of our coproduction process, these sessions facilitated an initial reflection about the product’s usability—how easily users grasp the forecast and the reliability estimates—in support of subsequent activities at the simulation workshop.

b. Simulation with future scenarios

We developed a participatory, computerized simulation-gaming environment called “ICEWISE”1 for use with stakeholders in the cruise tourism sector. Cruise tourism is one of the key sectors developing rapidly in the region. It is known to rely more on tactical and subseasonal forecasts than other maritime sectors, because they sell trips long in advance (up to two years) and will already have their areas of interest established prior to their trip, with considerable flexibility (depending on the vessel type) on where they will travel within a given area and timing (see also Lamers et al. 2018b). ICEWISE allowed us to facilitate a participatory simulation in which potential users of the new S2S sea ice forecast make voyage plans.

ICEWISE was designed to simulate certain aspects of users’ decision environment to gather user feedback about the S2S sea ice forecast, while also giving participants room to discuss how their needs will evolve in the coming years, thereby providing producers with strategic insights for planning upcoming developments. Socioeconomic scenarios help producers consider and respond to the requirements of the product (user needs) and the process itself, such as monitoring ways in which development can be further enhanced to meet the expected challenges of rapidly changing decision environments (Blair et al. 2020). The simulation was not intended to be entirely isomorphic with real life, but rather a playful, semirealistic platform to stimulate in-depth discussions in the debriefing sessions [see section 3b(3) below].

1) Scenarios

To facilitate a simulation game in which players can both engage with the forecast service under current conditions, and also experiment “in the future,” we adopted a simulation time flow during which players are exposed to projected future sea ice conditions for 2035, as well as plausible sociopolitical shifts for that time horizon. The 2035 scenarios were developed in a workshop held in November 2018 at the Danish Meteorological Institute, in collaboration with 23 experts in the fields of Arctic maritime sectors, navigational safety, community resilience, economy, policy, climate services, and climate change. Participants produced environmental, social, economic and technical factors that drive decisions around marine operations. These key factors and future projections produced by participants were used in a robustness analysis (Gausemeier et al. 1998) to produce three scenario outcomes with a unique emphasis on either consistency or plausibility, or a combination of both (robustness) (Blair and Müller-Stoffels 2019). The robust scenario bundle was then illustrated by an artist (Fig. 4) and used in the development of event cards and narratives for the simulation ( Table S1 in the online supplemental material).

Fig. 4.
Fig. 4.

Twelve illustrated future projections from the 2035 most robust 2035 scenario bundle (Blair and Muller-Stoffels 2019). Panel captions follow the format “key factor | future projection”: (a) accessibility of Arctic sea routes | easy access; (b) user-centric information infrastructures and data | few specialized actors; (c) predictability of sea ice variability | gradual improvements in predictive models; (d) regulations and policies affecting Arctic operations | economic and commercial uses dominate; (e) demand for Arctic resources | seafood first; (f) global economic trends | Arctic rush; (g) geopolitical stability | status quo (occasional bullying); (h) major incidents and critical events | status quo; (i) sustainable and resilient Arctic communities | expatriate haven; (j) trajectory of development in marine technologies | techno-utopia for some, stormy seas for others; (k) fluctuating energy prices | northern push; and (l) China’s strategic plan | Chinese finger cuffs. The illustrations are by B. Köhler.

Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0048.1

2) Simulation game

ICEWISE was developed using Unity Engine and its 2D graphical user interface (GUI) utilities. We chose Unity due to its streamlined development interface and multiple documented (Juliani et al. 2018) advantages: ease-of-use, cross-platform compatibility, and a graphical user interface designed to efficiently streamline the iteration of complex environments or novel tasks. This allowed for easy iteration of the game’s systems and user interface, allowing time to make necessary changes and tests throughout development.

The simulation was designed to simulate multiple scenarios in which end users would use the forecasting system. The simulation was designed with both near-horizon activities (1–2-week outlook; e.g., navigational planning) and longer-term activities (up to a 16-week outlook; e.g., certain fleet and itinerary planning) and decision-makers in mind. Users are asked to assume the role of an itinerary planner for a fictional Arctic cruise company. The simulation begins by emphasizing to participants that they are to assume a planner’s role, which may be different from their normal routines (e.g., in the case of mariners who participate). Facilitators discuss with participants the contrasting situations about 1) what is real, in which several metocean factors are important to safe operations in real life; and 2) what is simulated and experimental (ICEWISE), in which we isolated sea ice forecasts. Each participant works individually. A total of 12 rounds of play are divided into two periods—six rounds in 2019 and six in 2035. Before the 2019 period a short slideshow summarizes important events and developments impacting Arctic cruising at the time. Similarly, before the 2035 period begins the simulation shows a brief narrative based on key factors developed at our scenario workshop.

Participants are told that the reward system underlying the game calculates money accumulated and reputation points that simulate public perception of the participant’s cruise company. The participant is then asked to view an itinerary; in some rounds in Greenland and in other rounds in Svalbard (Fig. S1 in the online supplemental material). Next, the participant rolls a virtual dice and in return receives an event card describing a specific incident or development that either increases or decreases their bank and reputation points. The event cards serve to simulate potential social, political, economic, and environmental developments that can impact the player’s business environment and in turn their strategies and voyage planning activities. Figure 5 depicts screenshots of the event card and itinerary displays.

Fig. 5.
Fig. 5.

(a) The game console showing an event card and wild card narrative in play based on a roll-the-dice game mechanism. The console shows the current year, available bank, and reputation points (pink bar). (b) The itinerary view informing players about the voyage duration, planned activities, main ports of call, and vessel specifications. Itineraries are based on project partner Hurtigruten’s cruise portfolio and are used with permission.

Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0048.1

In one-half of the rounds the participant is instructed to select a start date for the cruise season as early in the spring as safely possible, whereas in other rounds they have to select the season’s final voyage date late in the autumn. After date selection, the participants invest money in the voyage from an available bank of money and then indicate their own sense of certainty in the success of the voyage between 0 (complete uncertainty) and 100 (complete certainty). Success is defined as a voyage without major disruptions or adverse events. Unsuccessful rounds translate to loss of financial investment and loss of reputation points due to customer dissatisfaction or diminished safety record. Next, the participant examines a sea ice forecast, in each round with a different lead time (alternating between the ranges of 1–4, 4–8, and 8–12 weeks) and with varying degrees of reliability. The reliability of each prediction was referenced (as in Fig. 3) and was announced by a dedicated moderator seated next to each player. At this point the participant is able to update their chosen date, investment amount, and self-assessed certainty score. Participants are made aware that the more risky their selected date is (the earlier in the spring, or the later in the autumn), the higher the potential return on their investment due to an extended cruise season is but also the greater the risk of failure is (loss of money and/or reputation points).

The player finishes each round by clicking on the ship’s “throttle” to launch the voyage and receive feedback2 on its success and change in bank and reputation points. The return on investment is adjusted in an inverse relationship with the risks faced. Figure 6 is an overview of the simulation’s steps.

Fig. 6.
Fig. 6.

The simulation flow.

Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0048.1

We tested the beta version of the game (October 2019) with five mariners of an expedition cruise company who had ample experience navigating ice-infested waters. Feedback from testers facilitated the planning of activities (e.g., how much introduction of the forecast and simulation is needed for participants to grasp the basics) and the format of the moderation at the upcoming participatory simulation workshop.

3) Debriefing

A structured debriefing session was conducted after the simulation session, relying on a predetermined set of reflective questions. These questions surveyed players’ emotions during the simulation, how their thoughts about voyage planning changed after the simulation, as well as any changes in players’ self-confidence as decision-makers. We made plans for individual feedback in between rounds and at the end, as well as for group feedback during breakout group discussions. Individual feedback was recorded during the simulation at the end of rounds 3, 6, 9, 12. The feedback at this stage was designed to be a quick snapshot of the player’s journey through the simulation. In-simulation questions included eight evaluative statements (see section 4b), each with a 4-point scale using emoji icons from happy (☺) to sad (☹). The emoticons can be considered as approximate indicators of participants’ progress through the simulation.

4. Results

Six mariners from the cruise and ice pilotage sectors and three researchers from MET Norway participated in the simulation exercise at a workshop in January 2020 in Tromsø, Norway. Two marine pilots, two captains, one chief officer, and one navigation officer participated. Except for the navigation officer, who had 8 years of experience, all mariners had over 20 years of experience in marine operations. The level of experience navigating in Arctic waters with seasonal ice cover ranged between 8 and 15 years (3 participants) and one to two years (2 participants). One participant had extensive (15+ years) subarctic experience with sporadic engagement with sea ice. The two marine pilots also had substantial operation planning and management experience in the Arctic.

Mixing users and providers enlivened the debriefing session, by allowing the researchers to put themselves in the day-to-day experience of users, enabling them to ask targeted questions. Five workshop facilitators were present to assist with game play, to take notes and to moderate the debriefing session.

a. Simulation results

Participants moved through the 12 rounds in less than 1.5 h. The data input by participants were saved to a database. A cursory overview of player data at the workshop suggested that an increase in forecast skill did not necessarily result in greater reported player confidence and higher investments. These first impressions were used during debriefing to reflect on the choices made by participants in an exploratory capacity to stimulate discussion.

For 2035, participants generally chose the first and last voyage dates earlier in the spring and later in the autumn than they did for 2019 (Fig. 7). For the earliest voyage to circumnavigate Svalbard in 2019 most players selected a date in July, and for 2035 most selected a date in June. Similarly, for 2019 most participants scheduled the last date of the season for September, and for 2035 they scheduled it for October. The northwest Svalbard itineraries were similarly stretched into earlier and later adjacent months for 2035 in comparison with 2019. The Disko Bay, west Greenland, itinerary was the only exception, where the preferred shoulder-season dates were similar for 2035 and 2019.

Fig. 7.
Fig. 7.

Date choices for each itinerary. Months show the number of players who chose a date in each month. Red denotes the highest frequency.

Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0048.1

Following the workshop, we checked each participant’s average self-reported confidence levels in rounds that took place for 2019 and 2035. While there was no significant self-reported confidence change in the group during rounds played for 2035, money investments told a different story. Five players invested on average more money for 2035 than they did for 2019 (group average = EUR 128 000 investment increase in 2035), and only one participant invested less (EUR 33 000 decrease). Although statistical significance could not be confirmed via our small cohort, these preliminary findings suggest two potential factors to be explored further:

  1. the extent to which willingness to invest and self-reported uncertainty may paint entirely different pictures about participants’ feelings about their decision environment and

  2. the extent to which better forecast reliability versus more-favorable sea ice conditions may increase participants’ confidence in operational planning in the 2035 period.

b. Debriefing results

In-simulation feedback is shown in Fig. 8 from the first debriefing checkpoint (round 3) to the last (round 12). Round-3 feedback indicated a sense of satisfaction (higher number of happy emoticons) among the group for some statements such as complexity, issues, confidence in forecasts, random events, and outcomes, but voyage planning and self-confidence were rated as low. At the same time, the group exhibited neutral feelings about forecast reliability. By the end of round 12, the scores for all propositions had risen. Statements 3 and 5 about ease of planning and self-confidence, both show a significant increase in satisfaction, congruent with the noted increase in 2035 investment trends relative to 2019 (see section 4a). The group responded with higher satisfaction levels to all propositions except the proposition about wildcard events.

Fig. 8.
Fig. 8.

Results from eight evaluative statements during individual debriefing between rounds, showing how many of the six mariners placed marks on the smiley-face side of the scale at the beginning (round 3, in 2019) and at the end (round 12, in 2035) of the simulation.

Citation: Weather, Climate, and Society 14, 2; 10.1175/WCAS-D-21-0048.1

After the simulation, a 2.5-h group debriefing session was led by an experienced facilitator. It is noteworthy that the debriefing took one hour or roughly 70% longer than the simulation. Participant feedback about the forecast product, factors that impact its use and its potential impacts on navigation included the following themes.

1) Forecast-guided decisions

Participants explained that, in choosing their dates, even when a forecast had high reliability, the ratio to which the forecast vs. their own experience factored into their final decision was roughly 40/60 (the forecast had a roughly 40% influence over the final decision). When given an itinerary that included unfamiliar areas, this ratio changed to 60% reliance on the forecast and 40% on their own experience and intuitions. Participants agreed that when they use services with which they are familiar and that they trust explicitly, they rely 90% on the forecast, and only 10% on own experience. A participant noted, “When I am in doubt, my own knowledge and experience wins every time in terms of decisions I make.”

2) Probabilistic forecasts of sea ice

The concept of predicting sea ice is not yet trusted. In real life, sea ice is not homogenous in any one area, and depending on the forecast’s resolution, this can have an impact on forecast reliability. Offering users the option to display probabilities for various sea ice concentrations (10% or 20%) would be a useful feature. Some participants felt it was somewhat difficult to fully operationalize the two layers of probabilities embedded in the product: the probabilities of the sea ice concentration in combination with the forecast’s reliability estimate.

3) Usability

Participants agreed that the full potential of S2S sea ice predictions materializes in route and capacity planning, but useful applications for tactical and navigational decisions exist in certain sectors (e.g. schedule-dependent cargo shipping). Based on the participant’s feedback in the Northwest Passage where the possibility of reaching Cambridge Bay varies year to year, a longer (3–4 week) sea ice forecast outlook would be useful between Nuuk and Cambridge Bay. However, S2S sea ice forecast is not as relevant for the west Greenland itinerary as it is for those in Svalbard and east Greenland because in west Greenland icebergs coming from outlet glaciers is the most relevant parameter for sailing. For mariners who work on much shorter (1–7 days) tactical time scales, usefulness is limited through the requirement of subkilometer resolution products (e.g., Tietsche et al. 2020). Current model systems for short-term predictions have coarser resolutions, and only a few can resolve some important characteristics such as sea ice leads, ridges, and fast ice. In addition, sea ice observations that can be used to constrain the models have a number of limitations, such as the coarse spatial resolution of passive microwave satellites, and the limited spatiotemporal sampling of synthetic aperture radars. Furthermore, surface melting significantly affects the quality of sea ice observations during the melting season (Kern et al. 2016, 2020; Ricker et al. 2017), and therefore the accuracy of sea ice forecasts that can be used by mariners for operational purposes. However, new interactive visualization tools combined with machine learning methods for forecast calibration are showing promising results in making state-of-the-art forecasting models products useful for tactical decision-making (Palerme and Müller 2021; Palerme et al. 2021).

4) Trust

Trust in a service is crucial to its usefulness, and develops over time, although the length of time may be mitigated by familiarity with the service provider (trusted providers’ products are trusted faster). Mariners discover through experience which climate and weather services are the most reliable.

5) The future of safe, sustainable Arctic navigation

It forms a risk for operators when policy makers base decisions on data without checking the practices and routines of stakeholders. Policy makers may be motivated to refer to an S2S sea ice forecast to regulate where companies can go and when Arctic routes see an increase in traffic. A participant wondered about such regulatory impacts in the future and noted that a recent publication of 15–20-yr historical sea ice data and risk assessment index showed supposed un-navigable areas at certain times when their company definitely operated there without issues. He emphasized that the cruise sector is quite flexible with adaptable itineraries unlike cargo ships that must adhere to set schedules.

Participants have observed shifting socioeconomic trends for the Arctic cruise sector, and the industry around Greenland and Svalbard is expected to continue to grow. An increasing number of cruise companies are interested in Greenland’s tourism potential. It is expected that once waters become too busy around other tourism hot spots such as Iceland, more and more cruise companies will relocate activities farther north. In the near future, the expansion of the Arctic cruise will likely result in companies increasing the number of ships with which they operate, as opposed to expanding the season. In the long run, stretching the shoulder season into the spring makes sense, but for cruise tourism late fall means little or no wildlife sightings and no interest from customers. These trends will grow the demand for, and investments in, salient metocean services for planning purposes. As companies invest more into Arctic operations, they also increase the financial risk potential of major disruptions or adverse events, driving the demand for accurate, decision-relevant information.

Changes in the physical environment are also swift. Participants shared that the 2035 sea ice forecasts in the simulation are realistic, and they are already observing these trends. Participants have encountered more drift ice (bigger concentrations of broken multiyear ice) during operations in the past 10 years. When they consider operating in such conditions, the weather must be good and the ship must be able to navigate openings in the ice. All the factors involved in that scenario have high variability, and mariners need effective and sustainable decision support tools. Changing, dynamic environments diminish the navigator’s ability to rely on past trends and experience. If a changing climate results in the generation of new sailing patterns, this will also increase the role of forecasts in mariners’ decision-making. Participants proposed that a future long-range sea ice forecast, optimized for navigators, should include layers of different sea ice concentrations and combine drift and wind information on the map. For navigators, such a service would be most relevant at the medium-range time scale, making available data of the 10 previous days, and showing predictions 10 days out.

Although the primary objective of the workshop was to gather feedback about an S2S sea ice forecast developed by researchers from MET Norway, participants also provided feedback about the simulation experience:

6) Learning

Participants mentioned learning as a positive impact of the simulation. Some reported more awareness about how the reliability of the sea ice forecast is calculated and greater ease of use in reading the sea ice probabilities on the forecast. For mariners, the game highlighted the different motivations and working contexts of vessel-based crew and onshore personnel or “office guys” as they put it, concluding that “safety and money need to work together.” Safety may be first, but the robustness of the business goes hand in hand with safety and sustainability. For this, collaboration between crew and onshore personnel is needed, even if they have different ways of making decisions.

7) Realism

The simulation was limited in complexity in comparison with real life, because many more factors besides ice must be considered when navigating Arctic waters. In terms of long-term planning however, the simulation was deemed to be fairly isomorphic with real life.

8) Enjoyment

Most participants considered the simulation to be enjoyable and felt positive about experimenting. They agreed that the gambling element (increased rate of return for increased push into shoulder season) and the roll-the-dice event cards were difficult for them to handle as for mariners, safety is first, and they do not routinely engage with wider social, economic, and political contexts of marine operations. They agreed, however, that for long-term planners this element of the game would be more familiar as itinerary development can have an element of gambling in it.

5. Discussion

a. S2S prediction of sea ice probabilities: Potential applicability

The extended‐range lead time of S2S predictions is where decisions can be supported in a range of sectors, but this new frontier is still in development for both its operational and application-focused capabilities (White et al. 2017; DeMott et al. 2021). Our project confirmed several present and future socioeconomic applications of sea ice probabilities at this time scale among representatives of Arctic marine sectors, and also highlighted several constraints that impact usability and uptake (objective 1). Participants agreed that the concept of predicting sea ice beyond the tactical (1–2 days) window will take time to trust, even if forecast skillfulness is transparently communicated to them. They also agreed that expected changes in the physical environment and simultaneous developments in Arctic routes will increase the applicability of, and reliance on, extended range outlooks. Presently, only schedule-dependent sectors would find use for an S2S sea ice service for planning decisions. For example, in the cruise sector the usefulness is limited to specific locations or to route and capacity planners who make use of lower spatial and temporal resolution forecasts (though navigators too may find a practical use for it in combination with other services).

Concern arose about future policy implications, mainly that regulators would use probabilistic predictions to control traffic, without sufficient understanding of operators’ practices and the diverse margins of safety across sectors. User feedback, such as those obtained during the simulation game, can help determine location- and sector-specific constraints that govern buy-in and the social benefit that may be derived as a result. Teasing out the diversity of needs and potential benefits is where social science research and innovative, participatory methods can contribute to a seamless prediction system by identifying channels for generating and communicating decision-relevant information, assessing the use and value of this information, and transferring knowledge and experiences to other regions (Brunet et al. 2010).

Rapid Arctic changes bring about new opportunities by expanding the scope of marine operations and linked markets, but they also pose risks and reveal vulnerabilities. In response, the portfolio and quality of services are expected to grow in the coming years, as providers strategize about how to optimize development in an efficient and responsible manner. However, the skill and reliability of S2S sea ice predictions in reducing uncertainties and risks for Arctic mariners has to be demonstrated and trialed, before users have confidence in and trust emerging services and technologies. S2S forecast skill and reliability is gradually improving but the middle ground between what is required and what is possible needs to be explored further (White et al. 2017). More work is also needed to develop transparent, easy-to-understand communication of forecast skill based upon the type of audience receiving the information (Mulder et al. 2020). Some users are more risk-averse than others as a function of the specific requirements of their operational environment, while others are less so. For example, some sectors depend heavily on interactions with the Arctic Marginal Ice Zone (expedition cruise tourism, research, and some fishing vessels), while others prefer to operate on the outskirts (shipping), in continuous ice cover (icebreaking), or avoid ice completely (resource extraction and infrastructure development activities) (Palma et al. 2019; Wagner et al. 2020). While accurate predictions are vital to all users, the margin of safety is different across user groups, driving diverse information and forecast skill assessment needs. Coproduction can highlight context and user-specific appropriate mechanisms for generating and communicating decision-relevant measures of forecast skill.

Results from our simulation-gaming workshop could not confirm a low threshold for reliability estimates that renders predictions irrelevant, nor did a universal high threshold emerge that reduced uncertainties for users (objective 2). Users’ own experience, familiarity with a location and with the historical range of local conditions, as well as trust in a forecast (or its provider) greatly mitigate the extent to which any single forecast reduces uncertainties in planning and tactical decisions. For example, a captain who is experienced in navigating sea ice can find their practical knowledge lacking on new routes where glacial ice is the most prominent challenge. Although our participants explained that when ambiguities arise about a forecast’s skill, it is their experience that will be most prominent in driving decisions; they also admitted that new products from well-trusted service providers would be quickly trusted. Yet end users tend to consult multiple types and sources of data depending on the temporal and spatial availability of, and their access to, services and then use them in combination to enable best decisions.

Mariners around Greenland and Svalbard rely on sea ice monitoring products from the National Ice Services, as well as weather forecasts from various platforms [e.g., Global Maritime Distress and Safety System (GMDSS), yr.no, windy.com]. To our knowledge, sea ice forecast information is not yet taken into account for tactical decision-making on the bridge. Users from large-scale operations typically have detailed and advanced information services at their disposal, tailored to their needs, which often require a fee and are sometimes even more advanced than what public meteorological services deliver (Knol et al. 2018). Smaller operations typically have access to some publicly available services and some historical data, but they tend also to rely heavily on field experience. For some users, predictions about the probability of high-impact events are more relevant than most probable future mean states (Brunet et al. 2010). In one such example, a participant noted that the area between Cape Farewell and Iceland tends to have extremely bad weather from September to May. For traffic in those areas, weather warnings are most important. Awareness of such operational priorities are an important consideration for service providers and for policy makers as well.

b. Simulation and foresight in climate services coproduction

The socioeconomic benefits (e.g., protection of life and property, sustainability of the environment) of predictive systems that pursue a seamless process, are enhanced when they incorporate social science with users’ knowledge and experience (Brunet et al. 2010). Participatory, experiential research approaches engage diverse user groups for mutual learning. Our simulation- and foresight-based framework has been particularly successful in helping participants to learn about the goals, needs and perspectives of other user and provider groups, and to think about present and future strategies in support (objective 3). We found that participants had a stimulating effect on each other following simulation gaming; when ideas and perspectives resonated among participants it often propelled spontaneous, insightful discussions. For example, in one discussion during the debriefing, reflection about the simulation turned into a sharing of enriching anecdotes between two captains. The simulation and debriefing provided an opportunity for in-depth reflection on routines and on a wealth of ideas (Crookall 2010). Participants also reported increased awareness in general about the science behind sea ice reliability measures, and greater ease of use of the product. Our mariner participants also grew their understanding of the different working contexts of vessel-based crew and onshore personnel, which, according to them, is vital for the harmonization of the operational safety and the long-term economic viability of the business.

The simulation and debriefing also revealed that, even when predictions make explicit underlying uncertainties, and even when reliability estimates are high, experienced mariners can be cautious to adopt a new service into their routine. This is important for service providers and policy makers because it entails a lag in the uptake of the services that are in development now, or those that are planned for the near future. This lag translates to a delay in the implementation of solutions that are designed to mediate risks inherent in the safety and sustainability of Arctic social and ecological systems. Participatory simulation-gaming and foresight methods allow participants to think in advance about future needs, and about how to adapt when changes and new demands arise, what services can best support adaptation and how to build trust. Future-facing climate service coproduction approaches have an important role to play in the resilience of communities and industries in all rapidly changing regions.

The expectation is that coproduction results in services that are both decision-driven and science-informed, with experts having the capacity to confidently lead the production of such marketable, salient products. However, it has been argued that the coproduction of services entails several paradoxical relations (Blair et al. 2020) that create conflictual conditions and significant tensions for providers of sea ice predictions: Scientists have to balance simultaneously expertise- and user-driven innovation without losing grasp of production-oriented, high-impact research; they have to assess and make transparent to users their own limitations; and they also must communicate the uncertainties and skill underlying their products, all the while confidently meeting ever-evolving user needs. In addition, one can argue that the automation paradox (Bibby et al. 1975; Bainbridge 1983) will become increasingly pervasive in sea ice prediction, as services evolve from mainly manual processes to increasing automation. Ironically, the automation of forecasting services will likely require ever-increasing levels of human input in order to become usable information (Jeuring et al. 2020).

In navigating these multiple, simultaneous, and conflicting requirements, it is more important than ever that the forecasting innovation model is product relevant (how will risks and benefits be distributed), process relevant (who, when, and how should be involved), and purpose relevant (who will benefit and what are the alternatives) (Stilgoe et al. 2013; Weichselgartner and Arheimer 2019; Blair et al. 2020). Coproduction can lead to more enlightened decisions by facilitating learning and aligning the mindset of policy makers and scientists about investments into the future of forecasting with the practical needs of end users. Because coproduction approaches and resulting services are always selective in whose needs they serve, and may come with unintended consequences, social science also plays an important role in creating awareness of these potential double effects (Chilvers and Kearnes 2020).

While coproduction is a resource-intensive process for service users and providers alike; it is not always a necessary or appropriate mode of engagement. A practical step in further developing S2S predictions is determining the sociomaterial environments of user groups and the type of information gaps that exist. Where demand and potential for use of S2S forecasting is high, and where knowledge uncertainty is also high (i.e., complex, high-stakes problems that push the boundaries of existing technological and scientific capacities), user-driven product development is necessary to pool the material, social and cognitive capacities necessary for innovation and to ensure the eventual uptake of the product. Where knowledge uncertainties and demand are both low, more traditional, science-driven design approaches and ad hoc stakeholder engagement, may be sufficient.

6. Conclusions

Considerable advancements have been made in the development of S2S predictions, but much work remains to explore the applicability and social benefits of S2S services. Products and services should aim to align the supply of information with user demand, should be tailored to specific user interests and routines, and should be ready to integrate in the decision-making practices of users.

This study demonstrated a forward-looking, user-centric, climate-service approach to designing an S2S forecast of sea ice probabilities, generating decision-relevant feedback on the current and future applicability of S2S predictions and user-tailored uncertainty and confidence measures. The results suggest that it will take time and experimentation before users will trust sea ice prediction beyond the tactical window and that transparently communicated forecast uncertainties do not eliminate the need for a trial period for users to fully adopt the product. In addition to testing the reliability of new forecast services, the reputation of service providers as producers of high-quality products also facilitates user uptake of services. In the end, we could not confirm a direct relationship between the reliability of the forecast, and uncertainty inferred from users. Our results suggested that mariners make decisions with high levels of confidence and that the skill of informational products plays a minor role in their uncertainty calculus, relative to operational experience and intuition.

Expected changes in the physical environment and simultaneous developments in Arctic routes will increase the applicability of, and demand for, extended range outlooks for tactical purposes as well as capacity planning. For example, companies that increasingly invest in Arctic operations may minimize their financial risk from potential disruptions or adverse events if reliably informed by long-range forecasts. However, many challenges remain in the optimization of S2S sea ice predictions to provide the most reliable information on relevant scales for all users. Immersive, experiential coproduction methods are an underutilized resource that can help to manage expectations, and to prioritize the most useful product features and thresholds in order to generate benefits from innovation. In our simulation-gaming activities participants reported having increased their awareness about the science behind sea ice reliability measures, their understanding of the different working contexts of vessel-based crew and onshore personnel, and factors that promote the long-term economic viability of the business.

Risk management and decision-making across sectors will continue to require such effective dialogues between service providers and users, as well as tools that balance mutual influence between actors. Foresight-based participatory simulation that includes a range of user-relevant parameters that impact stakeholders’ decisions and interactions, such as biophysical, social, economic, and political drivers of change, can play an important role in mitigating long-term risks, supporting sustainable adaptation, and enhancing sectoral and societal resilience in fast-changing environments.

1

A modified, online version of the ICEWISE game can be found online (http://salienseas.com/?page_id=3070).

2

Winning or losing each round was based on a probabilistic algorithm using the forecast’s reliability estimates.

Acknowledgments.

Authors B. Blair and M. Lamers acknowledge research funding for the SALIENSEAS project, part of the ERA4CS program, as cofunded by the Netherlands Organisation for Scientific Research and by the Horizon 2020 Framework Programme of the European Union (Grant 690462). Authors M. Knol-Kauffman, M. Müller, and C. Palerme acknowledge SALIENSEAS research funding from the Norwegian Research Council Contract 276223. The SALIENSEAS project is endorsed by the Year of Polar Prediction (YOPP), a flagship activity of the Polar Prediction Project (PPP), initiated by the World Weather Research Programme (WWRP) of the World Meteorological Organisation (WMO). We are grateful for all those participants taking part in SALIENSEAS workshops.

Data availability statement.

All original contributions presented in the study are included in the article and online supplemental materials.

REFERENCES

  • Aksenov, Y., E. E. Popova, A. Yool, A. G. Nurser, T. D. Williams, L. Bertino, and J. Bergh, 2017: On the future navigability of Arctic sea routes: High-resolution projections of the Arctic Ocean and sea ice. Mar. Policy, 75, 300317, https://doi.org/10.1016/j.marpol.2015.12.027.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Alexander, M., and S. Dessai, 2019: What can climate services learn from the broader services literature? Climatic Change, 157, 133149, https://doi.org/10.1007/s10584-019-02388-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Arctic Council, 2009: Arctic Marine Shipping Assessment 2009 Report. NOAA/Pacific Marine Environmental Laboratory, 194 pp., https://www.pmel.noaa.gov/arctic-zone/detect/documents/AMSA_2009_Report_2nd_print.pdf.

  • Arhippainen, L., and M. Tähti, 2003: Empirical evaluation of user experience in two adaptive mobile application prototypes. Proc. Second Int. Conf. on Mobile and Ubiquitous Multimedia, Linköping, Sweden, ACM, 2734, https://ep.liu.se/ecp/011/007/ecp011007.pdf.

    • Search Google Scholar
    • Export Citation
  • Arnal, L., M.-H. Ramos, E. C. de Perez, H. L. Cloke, E. Stephens, F. Wetterhall, S. J. van Andel, and F. Pappenberger, 2016: Willingness-to-pay for a probabilistic flood forecast: A risk-based decision-making game. Hydrol. Earth Syst. Sci., 20, 31093128, https://doi.org/10.5194/hess-20-3109-2016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bainbridge, L., 1983: Ironies of automation. Analysis, Design and Evaluation of Man–Machine Systems, Elsevier, 129–135.

    • Crossref
    • Export Citation
  • Becu, N., 2020: Les courants d’influence et la pratique de la simulation participative: Contours, design et contributions aux changements sociétaux et organisationnels dans les territoires. M.S. thesis, Dept. of Geography, La Rochelle University, 258 pp., https://hal.archives-ouvertes.fr/tel-02515352/document.

  • Becu, N., and Coauthors, 2017: Participatory simulation to foster social learning on coastal flooding prevention. Environ. Modell. Software, 98, 111, https://doi.org/10.1016/j.envsoft.2017.09.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berle, Ø., J. B. Rice Jr., and B. E. Asbjørnslett, 2011: Failure modes in the maritime transportation system: A functional approach to throughput vulnerability. Marit. Policy Manage., 38, 605632, https://doi.org/10.1080/03088839.2011.615870.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bibby, K., F. Margulies, J. Rijnsdorp, R. Withers, and I. Makarov, 1975: Man’s role in control systems. IFAC Proc., 8, 664683, https://doi.org/10.1016/S1474-6670(17)67612-2.

    • Search Google Scholar
    • Export Citation
  • Blair, B., and M. Müller-Stoffels, 2019: Maritime futures 2035: The Arctic region. Wageningen University and Research Workshop Rep. and Tech. Doc., 92 pp., http://salienseas.com/wp-content/uploads/2019/06/Workshop_Report.pdf.

  • Blair, B., O. A. Lee, and M. Lamers, 2020: Four paradoxes of the user–provider interface: A responsible innovation framework for sea ice services. Sustainability, 12, 448, https://doi.org/10.3390/su12020448.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bommel, P., 2020: Participatory modelling and interactive simulation to support the management of the commons. HDR thesis, Université Montpellier II, 89 pp., https://hal.archives-ouvertes.fr/tel-02568654/document.

  • Boyd, E., B. Nykvist, S. Borgström, and I. A. Stacewicz, 2015: Anticipatory governance for social-ecological resilience. Ambio, 44, 149161, https://doi.org/10.1007/s13280-014-0604-x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brandsen, T., and M. Honingh, 2016: Distinguishing different types of coproduction: A conceptual analysis based on the classical definitions. Public Adm. Rev., 76, 427435, https://doi.org/10.1111/puar.12465.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bremer, S., A. Wardekker, S. Dessai, S. Sobolowski, R. Slaattelid, and J. van der Sluijs, 2019: Toward a multi-faceted conception of co-production of climate services. Climate Serv., 13, 4250, https://doi.org/10.1016/j.cliser.2019.01.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brudney, J. L., and R. E. England, 1983: Toward a definition of the coproduction concept. Public Adm. Rev., 43, 5965, https://doi.org/10.2307/975300.

  • Brunet, G., and Coauthors, 2010: Collaboration of the weather and climate communities to advance subseasonal-to-seasonal prediction. Bull. Amer. Meteor. Soc., 91, 13971406, https://doi.org/10.1175/2010BAMS3013.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chilvers, J., and M. Kearnes, 2020: Remaking participation in science and democracy. Sci. Technol. Hum. Values, 45, 347380, https://doi.org/10.1177/0162243919850885.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crochemore, L., M.-H. Ramos, F. Pappenberger, S. J. Andel, and A. W. Wood, 2016: An experiment on risk-based decision-making in water management using monthly probabilistic forecasts. Bull. Amer. Meteor. Soc., 97, 541551, https://doi.org/10.1175/BAMS-D-14-00270.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crookall, D., 2010: Serious games, debriefing, and simulation/gaming as a discipline. Simul. Gaming, 41, 898920, https://doi.org/10.1177/1046878110390784.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crookall, D., 2014: Engaging (in) gameplay and (in) debriefing. Simul. Gaming, 45, 416427, https://doi.org/10.1177/1046878114559879.

  • Crookall, D., and N. Becu, 2020: Companion modelling and participatory simulation: A glimpse. EGU General Assembly, EGU, Abstract EGU2020-8968, https://doi.org/10.5194/egusphere-egu2020-8968.

    • Crossref
    • Export Citation
  • DeMott, C., Á. G. Muñoz, C. D. Roberts, C. M. Spillman, and F. Vitart, 2021: The benefits of better ocean weather forecasting. Eos, 102, https://doi.org/10.1029/2021EO210601.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dilling, L., and M. C. Lemos, 2011: Creating usable science: Opportunities and constraints for climate knowledge use and their implications for science policy. Global Environ. Change, 21, 680689, https://doi.org/10.1016/j.gloenvcha.2010.11.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doddema, M., 2019: Employing a role playing game and debriefing approach to validate practices and identify variations in response dynamics. MethodsX, 6, 143149, https://doi.org/10.1016/j.mex.2018.12.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fleming, K., and Coauthors, 2020: The use of serious games in engaging stakeholders for disaster risk reduction, management and climate change adaption information elicitation. Int. J. Disaster Risk Reduct., 49, 101669, https://doi.org/10.1016/j.ijdrr.2020.101669.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Flood, S., N. A. Cradock-Henry, P. Blackett, and P. Edwards, 2018: Adaptive and interactive climate futures: Systematic review of ‘serious games’ for engagement and decision-making. Environ. Res. Lett., 13, 063005, https://doi.org/10.1088/1748-9326/aac1c6.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gascard, J.-C., K. Riemann-Campe, R. Gerdes, H. Schyberg, R. Randriamampianina, M. Karcher, J. Zhang, and M. Rafizadeh, 2017: Future sea ice conditions and weather forecasts in the Arctic: Implications for Arctic shipping. Ambio, 46, 355367, https://doi.org/10.1007/s13280-017-0951-5.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gausemeier, J., A. Fink, and O. Schlake, 1998: Scenario management: An approach to develop future potentials. Technol. Forecasting Soc. Change, 59, 111130, https://doi.org/10.1016/S0040-1625(97)00166-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Giorgetta, M. A., and Coauthors, 2013: Climate and carbon cycle changes from 1850 to 2100 in MPI-ESM simulations for the Coupled Model Intercomparison Project phase 5. J. Adv. Model. Earth Syst., 5, 572597, https://doi.org/10.1002/jame.20038.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goessling, H. F., and T. Jung, 2018: A probabilistic verification score for contours: Methodology and application to Arctic ice-edge forecasts. Quart. J. Roy. Meteor. Soc., 144, 735743, https://doi.org/10.1002/qj.3242.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hovelsrud, G. K., B. Poppel, B. Van Oort, and J. D. Reist, 2011: Arctic societies, cultures, and peoples in a changing cryosphere. Ambio, 40, 100110, https://doi.org/10.1007/s13280-011-0219-4.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • IPCC, 2019: Summary for policymakers. IPCC Special Report on the Ocean and Cryosphere in a Changing Climate, H.-O. Pörtner et al., Eds., Cambridge University Press, 36 pp., https://www.ipcc.ch/site/assets/uploads/sites/3/2019/11/03_SROCC_SPM_FINAL.pdf.

  • Jeuring, J., and M. Knol-Kauffman, 2019: Mapping weather, water, ice and climate knowledge & information needs for maritime activities in the Arctic. Survey Rep., 44 pp., https://salienseas.com/wp-content/uploads/2019/09/SALIENSEAS-Mapping-WWIC-Knowledge-and-Information-Needs-for-Maritime-Activities-in-the-Arctic_FINAL_20190830.pdf.

  • Jeuring, J., M. Knol-Kauffman, and A. Sivle, 2020: Toward valuable weather and sea-ice services for the marine Arctic: Exploring user–producer interfaces of the Norwegian Meteorological Institute. Polar Geogr., 43, 139159, https://doi.org/10.1080/1088937X.2019.1679270.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Johnson, S. J., and Coauthors, 2019: SEAS5: The new ECMWF seasonal forecast system. Geosci. Model Dev., 12, 10871117, https://doi.org/10.5194/gmd-12-1087-2019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Juliani, A. and Coauthors 2018. Unity: A general platform for intelligent agents. arXiv 1809.02627, 28 pp., https://arxiv.org/pdf/1809.02627.pdf.

    • Search Google Scholar
    • Export Citation
  • Kapp, K. M., 2012: The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education. John Wiley and Sons, 336 pp.

    • Crossref
    • Export Citation
  • Kern, S., A. Rösel, L. T. Pedersen, N. Ivanova, R. Saldo, and R. T. Tonboe, 2016: The impact of melt ponds on summertime microwave brightness temperatures and sea-ice concentrations. Cryosphere, 10, 22172239, https://doi.org/10.5194/tc-10-2217-2016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kern, S., T. Lavergne, D. Notz, L. T. Pedersen, and R. Tonboe, 2020: Satellite passive microwave sea-ice concentration data set inter-comparison for Arctic summer conditions. Cryosphere, 14, 24692493, https://doi.org/10.5194/tc-14-2469-2020.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Knol, M., P. Arbo, P. Duske, S. Gerland, M. Lamers, O. Pavlova, A. D. Sivle, and S. Tronstad, 2018: Making the Arctic predictable: The changing information infrastructure of Arctic weather and sea ice services. Polar Geogr., 41, 279293, https://doi.org/10.1080/1088937X.2018.1522382.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lamers, M., P. Duske, and L. van Bets, 2018a: Understanding user needs: A practice-based approach to exploring the role of weather and sea ice services in European Arctic expedition cruising. Polar Geogr., 41, 262278, https://doi.org/10.1080/1088937X.2018.1513959.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lamers, M., M. Knol, M. Müller, B. Blair, J. Jeuring, T. Rasmussen, and A. Sivle, 2018b: Enhancing the Saliency of Climate Services for Marine Mobility Sectors in European Arctic Seas (SALIENSEAS). Wageningen University and Research Social Sciences Group Stakeholder Advisory Group Workshop Rep., 28 pp.

  • Le Page, C., and Coauthors, 2013: Agent-based modelling and simulation applied to environmental management. Simulating Social Complexity, Springer, 499–540.

    • Crossref
    • Export Citation
  • Litinski, V., 2013: Games for health: Building the case. Int. J. Gaming Comput.-Mediated Simulations, 5, 108115, https://doi.org/10.4018/jgcms.2013070108.

    • Search Google Scholar
    • Export Citation
  • Lovecraft, A. L., N. Fresco, D. Cost, and B. Blair, 2017: Northern Alaska Scenarios Project report: Creating healthy, sustainable communities in Arctic Alaska. Communities in Arctic Alaska, 84 pp., https://uaf-snap.org/wp-content/uploads/2020/08/NASP-Report-2017_small.pdf.

  • Lovreglio, R., V. Gonzalez, Z. Feng, R. Amor, M. Spearpoint, J. Thomas, M. Trotter, and R. Sacks, 2018: Prototyping virtual reality serious games for building earthquake preparedness: The Auckland City Hospital case study. Adv. Eng. Inform., 38, 670682, https://doi.org/10.1016/j.aei.2018.08.018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McNie, E. C., 2012: Delivering climate services: Organizational strategies and approaches for producing useful climate-science information. Wea. Climate Soc., 5, 1426, https://doi.org/10.1175/WCAS-D-11-00034.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meadow, A. M., D. B. Ferguson, Z. Guido, A. Horangic, G. Owen, and T. Wall, 2015: Moving toward the deliberate coproduction of climate science knowledge. Wea. Climate Soc., 7, 179191, https://doi.org/10.1175/WCAS-D-14-00050.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Miller, C. A., and C. Wyborn, 2018: Co-production in global sustainability: Histories and theories. Environ. Sci. Policy, 113, 88–95, https://doi.org/10.1016/j.envsci.2018.01.016.

    • Search Google Scholar
    • Export Citation
  • Mulder, K. J., M. Lickiss, A. Black, A. J. Charlton‐Perez, R. McCloy, and J. S. Young, 2020: Designing environmental uncertainty information for experts and non-experts: Does data presentation affect users’ decisions and interpretations? Meteor. Appl., 27, e1821, https://doi.org/10.1002/met.1821.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • O’Brien, H. L., and E. G. Toms, 2008: What is user engagement? A conceptual framework for defining user engagement with technology. J. Amer. Soc. Inf. Sci. Technol., 59, 938955, https://doi.org/10.1002/asi.20801.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ostrom, E., 1996: Crossing the great divide: Coproduction, synergy, and development. World Dev., 24, 10731087, https://doi.org/10.1016/0305-750X(96)00023-X.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palerme, C., and M. Müller, 2021: Calibration of sea ice drift forecasts using random forest algorithms. Cryosphere, 15, 39894004, https://doi.org/10.5194/tc-15-3989-2021.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palerme, C., M. Müller, and A. Melsom, 2019: An intercomparison of verification scores for evaluating the sea ice edge position in seasonal forecasts. Geophys. Res. Lett., 46, 47574763, https://doi.org/10.1029/2019GL082482.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palerme, C., P. Kountouris, A. Stocker, M. Knol-Kauffman, P. Cochrane, M. Müller, and L. Rabenstein, 2021: Delivering high quality sea-ice information around the Svalbard archipelago to marine end-users. J. Oper. Oceanogr., 14 (Suppl. 1), 109–113, https://doi.org/10.1080/1755876X.2021.1946240.

    • Crossref
    • Export Citation
  • Palma, D., and Coauthors, 2019: Cruising the marginal ice zone: Climate change and Arctic tourism. Polar Geogr., 42, 215235, https://doi.org/10.1080/1088937X.2019.1648585.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Parker, W. S., and G. Lusk, 2019: Incorporating user values into climate services. Bull. Amer. Meteor. Soc., 100, 16431650, https://doi.org/10.1175/BAMS-D-17-0325.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rainville, L., and Coauthors, 2020: Improving situational awareness in the Arctic Ocean. Front. Mar. Sci., 7, 581139, https://doi.org/10.3389/fmars.2020.581139.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ricker, R., S. Hendricks, L. Kaleschke, X. Tian-Kunze, J. King, and C. Haas, 2017: A weekly Arctic sea-ice thickness data record from merged CryoSat-2 and SMOS satellite data. Cryosphere, 11, 16071623, https://doi.org/10.5194/tc-11-1607-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sardar, Z., 2010: The namesake: Futures; futures studies; futurology; futuristic; foresight—What’s in a name? Futures, 42, 177184, https://doi.org/10.1016/j.futures.2009.11.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sarewitz, D., and R. A. Pielke Jr., 2007: The neglected heart of science policy: Reconciling supply of and demand for science. Environ. Sci. Policy, 10, 5–16, https://doi.org/10.1016/j.envsci.2006.10.001.

    • Crossref
    • Export Citation
  • Saritas, O., 2013: Systemic foresight methodology. Science, Technology and Innovation Policy for the Future: Potentials and Limits of Foresight Studies, D. Meissner, L. Gokhberg, and A. Sokolov, Eds., Springer, 83–117.

    • Crossref
    • Export Citation
  • Scott, D. J., C. J. Lemieux, and L. Malone, 2011: Climate services to support sustainable tourism and adaptation to climate change. Climate Res., 47, 111122, https://doi.org/10.3354/cr00952.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sheldon, L., 2020: The Multiplayer Classroom: Designing Coursework as a Game. CRC Press, 338 pp.

    • Crossref
    • Export Citation
  • Sheppard, S. R., A. Shawa, D. Flanders, S. Burch, A. Wiek, J. Carmichael, J. Robinson, and S. Cohen, 2011: Future visioning of local climate change: A framework for community engagement and planning with scenarios and visualisation. Futures, 43, 400412, https://doi.org/10.1016/j.futures.2011.01.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Skinner, C., 2020: Flash Flood!: A SeriousGeoGames activity combining science festivals, video games, and virtual reality with research data for communicating flood risk and geomorphology. Geosci. Commun., 3, 117, https://doi.org/10.5194/gc-3-1-2020.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stilgoe, J., R. Owen, and P. Macnaghten, 2013: Developing a framework for responsible innovation. Res. Policy, 42, 15681580, https://doi.org/10.1016/j.respol.2013.05.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stocker, A. N., A. H. Renner, and M. Knol-Kauffman, 2020: Sea ice variability and maritime activity around Svalbard in the period 2012–2019. Sci. Rep., 10, 17043, https://doi.org/10.1038/s41598-020-74064-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Student, J., M. Lamers, and B. Amelung, 2020: A dynamic vulnerability approach for tourism destinations. J. Sustainable Tourism, 28, 475496, https://doi.org/10.1080/09669582.2019.1682593.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tall, A., J. Hansen, A. Jay, B. M. Campbell, J. Kinyangi, P. K. Aggarwal, and R. Zougmoré, et al.2014: Scaling up climate services for farmers: Mission Possible. Learning from good practice in Africa and South Asia. CCAFS Rep. 13, 44 pp., https://hdl.handle.net/10568/42445.

  • Terrado, M., and Coauthors, 2019: The Weather Roulette: A game to communicate the usefulness of probabilistic climate predictions. Bull. Amer. Meteor. Soc., 100, 19091921, https://doi.org/10.1175/BAMS-D-18-0214.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tietsche, S., and Coauthors, 2020: Recommendations for more user-relevant sea-ice forecasts. European Centre for Medium-Range Weather Forecasts KEPLER Deliverable Rep. D4.3, 30 pp., https://drive.google.com/file/d/1SGCW4TCBYI7jEEWbziZI0aOuoCnAtiHD/view.

  • Turnhout, E., T. Metze, C. Wyborn, N. Klenk, and E. Louder, 2020: The politics of co-production: Participation, power, and transformation. Curr. Opin. Environ. Sustainability, 42, 1521, https://doi.org/10.1016/j.cosust.2019.11.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van den Hoogen, J., J. Lo, and S. Meijer, 2016: Debriefing research games: Context, substance and method. Simul. Gaming, 47, 368388, https://doi.org/10.1177/1046878116651023.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaughan, C., S. Dessai, and C. Hewitt, 2018: Surveying climate services: What can we learn from a bird’s-eye view? Wea. Climate Soc., 10, 373395, https://doi.org/10.1175/WCAS-D-17-0030.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vincent, K., M. Daly, C. Scannell, and B. Leathes, 2018: What can climate services learn from theory and practice of co-production? Climate Serv., 12, 4858, https://doi.org/10.1016/j.cliser.2018.11.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Voinov, A., and F. Bousquet, 2010: Modelling with stakeholders. Environ. Modell. Software, 25, 12681281, https://doi.org/10.1016/j.envsoft.2010.03.007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Voinov, A., and Coauthors, 2018: Tools and methods in participatory modeling: Selecting the right tool for the job. Environ. Modell. Software, 109, 232255, https://doi.org/10.1016/j.envsoft.2018.08.028.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wagner, P. M., O. J. Hegelund, N. Hughes, K. Qvistgaard, J. Seppänen, P. Eriksson, A. Kangas, and I. Grönfeldt, 2019: Stakeholder needs: Maritime sector needs. Norwegian Meteorological Institute KEPLER Deliverable Rep. D1.1, 68 pp., https://drive.google.com/file/d/15C9_YGk63MxjhR6BtNMiWCjZ8hU4PbIy/view.

  • Wagner, P. M., and Coauthors, 2020: Sea-ice information and forecast needs for industry maritime stakeholders. Polar Geogr., 43, 160187, https://doi.org/10.1080/1088937X.2020.1766592.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, M., and J. E. Overland, 2012: A sea ice free summer Arctic within 30 years: An update from CMIP5 models. Geophys. Res. Lett., 39, L18501, https://doi.org/10.1029/2012GL052868.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weichselgartner, J., and B. Arheimer, 2019: Evolving climate services into knowledge–action systems. Wea. Climate Soc., 11, 385399, https://doi.org/10.1175/WCAS-D-18-0087.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • White, C. J., and Coauthors, 2017: Potential applications of subseasonal‐to‐seasonal (S2S) predictions. Meteor. Appl., 24, 315325, https://doi.org/10.1002/met.1654.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Whitton, N., 2011: Game engagement theory and adult learning. Simul. Gaming, 42, 596609, https://doi.org/10.1177/1046878110378587.

  • Whitton, N., and A. Moseley, 2014: Deconstructing engagement: Rethinking involvement in learning. Simul. Gaming, 45, 433449, https://doi.org/10.1177/1046878114554755.

    • Crossref
    • Search Google Scholar
    • Export Citation

Supplementary Materials

Save