The interagency National Earth System Prediction Capability (ESPC) collaboration convened a 2.5- day workshop in early March 2018 in response to guidance from its Executive Steering Group (ESG), the Weather Research and Forecast Innovation Act of 2017 (Public Law 115-25, 131 Stat. 91, www.gpo.gov/fdsys/pkg/PLAW-115publ25/pdf/PLAW-115publ25.pdf), commonly known as the Weather Act, and the recent report of the National Academy of Sciences (NAS) Committee on Developing a U.S. Research Agenda to Advance Subseasonal to Seasonal Forecasting. The Committee’s 3-year study (National Academies of Sciences, Engineering, and Medicine 2016) describes a strategy to increase the nation’s capacity for subseasonal to seasonal (S2S) forecasting, and to develop a 10-year scientific research agenda to accelerate progress. The panel concluded that “S2S forecasts will be as widely used a decade from now as weather forecasts are today.” However, that aggressive vision needs sustained and dedicated effort to accomplish. The report also recommended development of an S2S cyberinfrastructure supporting the vision—a national plan with support for workforce development and career-path encouragement for S2S.

In addition, the workshop, which included attendees from the U.S. Departments of Defense, Energy, the Interior, and State; the Environmental Protection Agency (EPA); the Federal Emergency Management Agency (FEMA); the National Aeronautics and Space Administration (NASA); the National Science Foundation (NSF); the National Oceanic and Atmospheric Administration (NOAA); the U.S. Department of Agriculture (USDA); and the California Department of Water Resources, as well as from academia and the private sector, provided opportunity for community engagement and commentary on a projected NOAA report to Congress required by the signing of the Weather Act. That report on seasonal and subseasonal forecast capability will go to the Committee on Commerce, Science, and Transportation of the Senate and the Committee on Science, Space, and Technology of the House of Representatives. The NOAA report is scheduled for submission to Congress by 18 October 2018.

The “Metrics, Post-Processing and Products for S2S” workshop was intended as an initial meeting to enable broad discussion of user needs, agency capabilities and products, gaps between needs and capabilities, and potential operational and technological solutions to address those gaps, especially in defining and refining postprocessing solutions, developer metrics, and reliability metrics. Continuing community engagement and consensus building on the development of viable sets of measures, applicable to various situations, purposes, and goals, was an important expected outcome of the meeting. The workshop was an exploration in gaining better understanding of these myriad elements comprising the challenges of S2S forecasting. Discussions emphasized identifying and understanding community and user needs, current community ability to meet those needs, developing appropriate measures of progress, and advancing community ability to better meet those needs.

Inclement weather on 2 March in the Washington, D.C., area forced the cancellation of the final half day of the meeting; nevertheless, much was accomplished in identifying current agency capabilities for S2S prediction and how agencies evaluate them (current metrics); gaps between current capabilities and needs; potential operational solutions to gaps; potential technological solutions to gaps; partial discussion of usability, reliability, and improved metrics (developer metrics and reliability metrics); and partial discussion of required additional research.

Areas of discussion that were touched on in the meeting, but ultimately limited by the final day cancellation, included robust identification of user data (parameters, frequency, availability, reliability) and product needs; more discussion on the topics of usability, reliability, and improved metrics, both for development and reliability; and more in-depth discussion of required additional research.

Initial framing of meeting goals included setting context and background for both the NAS report on S2S and for the congressional report required by the Weather Act, followed by multiple briefs with question-and-answer periods. Early whole group discussions centered on concerns related to prediction skill and the limits of predictability; strong encouragement to organize effectively as an S2S community; recommendations to focus down on a “do-able few” priorities for S2S forecasting; prioritization of products and tools; and application of postprocessing contributions to improve prediction skill for S2S. Subsequent meeting sessions focused on multiple agency capabilities in forecasting, modeling, and prediction; identifying user needs; and identifying the gaps in capabilities versus needs. The development of effective measures supporting S2S prediction will be an ongoing target and goal. Since S2S prediction relies so heavily on coupling with other model domains (ocean, land, cryosphere, biosphere), developing and emphasizing metrics to assess model fidelity within these other domains will be essential—analogous to the tools developed for the atmosphere. Critical issues to resolve include optimal ensemble size, ensemble generation techniques, member/component resolution, reforecast period, initialization frequency, multimodel ensemble construction (purposeful vs ad hoc), model weighting, forecasts of opportunity, data assimilation, observing systems, initialization, model tuning, model improvement, model complexity (component coupling), and research-to-operations-and-operations-to-research (R2O2R) collaboration. Additional questions include how many models, and of what types, in such an ensemble; how many members of each model; how much coupling between components; and with what resolutions for the separate components.

CHALLENGES AND RECOMMENDATIONS.

Successful S2S prediction faces a number of serious challenges; foremost among these are the diversity of users and needs, the degree or intermittency of predictability at S2S time scales, and the need for greater resources to observe, process, store, and analyze.

The workshop resulted in a number of recommendations to address these challenges.

1. Participants felt that building a strong S2S community including scientists, operational centers, social scientists, and users, similar to the weather and climate communities, would greatly advance S2S prediction. Facilitating this community would involve appropriate training for users and forecasters; building community software and data libraries for users and developers; including research community access to operational models; and a coordinating group to advocate for resources for science and operations to include significant investment in data storage hardware, software, and high performance computing (HPC). There was also support for the notion of end-to-end thinking, in the sense of developing start-to-finish goals for implementing understanding of user needs into S2S community practice.

2. To enhance S2S prediction skill and better manage resources, several participants raised the need for carefully designing an operational configuration that robustly meets user needs and is also accessible to the research community. This would involve careful experimentation to determine model diversity, ensemble size and resolution, ensemble generation and initialization frequency, reforecast characteristics, and required process resolution, as well as carefully crafted postprocessing.

3. Participants voiced a strong need for observations to address S2S, including in underobserved regions such as the Arctic and oceans; to develop new types of observations to address critical parameters such as ice and deep soil moisture; and to enhance coupled data assimilation to better utilize existing observations.

4. Because of the low limits of predictability and unique user needs, participants felt there is a need to focus verification and validation more on user needs: probabilistic prediction, object-oriented verification that could incorporate both temporal and spatial variations, user value or return-on-investment, and more user engagement in the development of tailored products.

5. There was a call for improving public/private policy, and especially for improving the federal funding model and empowering a federal coordinating authority for the S2S enterprise. If S2S is to be a community-based endeavor, improving communication across agencies is critical: “Each agency appropriation being done separately does not lend itself toward community-wide objectives.” The National ESPC Executive Steering Group and the Interagency Weather Research Coordination Committee (IWRCC) both operate within the organizational structure administered by the Office of the Federal Coordinator for Meteorology. The IWRCC concerns itself with interagency research in S2S, among other topics, and could complement ESPC efforts in operational advancements. Both groups are addressing Weather Act tasking in this area. Establishing an effective linkage between these two groups may advance both research and operations in S2S and enable a more effective coordination mechanism.

In summary, the workshop participants greatly appreciated the opportunity to discuss the issues, encouraging all effort toward ongoing coordination, communication, and collaboration for enhancing the national capacity for effective seasonal to subseasonal forecasting and prediction skill. Establishing mechanisms for continuing conversation and community engagement toward more robust metrics identification and solution-seeking, based on the recommendations offered, is a critical first step. This workshop served as a foundation for future collaborative efforts; one of the first includes upcoming AMS sessions on assessing and defining predictability on the S2S time scale, and observations and coupled data assimilation techniques for S2S. By continuing to leverage public, private, and academic expertise, a path toward increasingly actionable information for end users in all sectors is attainable.

Further information on the workshop, including agenda, individual presentations, and final report, are provided on the website https://cpaess.ucar.edu/meetings/2018/metrics-and-postprocessing-workshop.