Waves to Weather Exploring the Limits of Predictability of Weather

,

AFFILIATIONS: Craig, Janjić, Laurian, Mayer, Redl, and Tempest-Ludwigs-Maximilians-Universität, Munich, Germany; Fink, Hoose, Knippertz, and Lerch-Karlsruhe Institute of Technology, Karlsruhe, Germany; Miltenberger, Riemer, and Wirth-Johannes-Gutenberg-Universität, Mainz, Germany W aves to Weather (W2W) is a collaborative research center formed to investigate the limits of predictability of weather.Our ability to make forecasts up to a week or more in advance contributes to a strong economy and protects human life and property.Although we commonly speak of "weather," particularly high-impact weather events take many different forms that result from different physical processes.The most destructive weather disasters in recent years have been associated with floods and extreme winds, severe winter weather, and heat waves, as well as wildfires (Jones and Golding 2014).National meteorological and hydrological services play a central role in observing and forecasting these events and the value of their predictions is widely recognized.
The quality of weather predictions has improved dramatically over the last 50 years, but it is not easy to identify a particular breakthrough that led to the improvement.Bauer et al. (2015) described this process as a "quiet revolution" that resulted from a steady accumulation of improvements in observational coverage, data assimilation methods, and numerical models with higher resolution and more detailed representations of physical processes.These developments were driven in turn by increased computing power and improved physical understanding.A more visible part of the quiet revolution is the use of ensemble forecasts to provide probabilistic predictions that add value through information about uncertainties in the forecasts.
From a practical perspective, weather forecasts have been improving at a rate of about 1 day of lead time every 10 years, a trend that has been maintained for at least 50 years (Stern and Davidson 2015).This is in some ways comparable to Moore's law, the observation that microprocessor speeds doubled approximately every 18 months (Moore 1965).This "law" is little more than an empirical trend that resulted from the collective impact of a wide variety of individual process improvements.As with weather prediction, there is no underlying theory of why these improvements should lead to such a steady increase over many years, nor any reason to be confident that the trend would continue.
It has been known for decades that improvements in the skill of weather forecasts cannot continue forever.The predictability of weather is fundamentally limited by the chaotic nature of the underlying dynamical equations and the multiscale nature of atmospheric dynamics (Lorenz 1963(Lorenz , 1969)).Interestingly, the sensitive dependence on initial conditions exhibited for example by the Lorenz (1963) model is not enough, in itself, to prevent forecasts of any desired accuracy from being obtained, if the initial conditions and model are accurate enough.Instead, Lorenz (1969) explains that the limit on predictability arises from the increasingly rapid growth of errors as the spatial scale of the errors gets smaller.If the error growth rate increases fast enough with decreasing scale, small-scale errors will soon grow to saturation no matter how accurately the initial-condition errors are measured.These small-scale errors will then contaminate successively larger scales of motion until all predictability is lost.Lorenz speculates that energetic events such as cumulus clouds can provide the environment for rapid growth of small-scale errors, so that even the flap of a butterfly's wings can change the world's weather. 1ow will we know when we have reached the intrinsic limit of predictability due to rapid growth of small-scale errors?In many cases, predictability will be limited by growth on synoptic scales and the intrinsic limit due to upscale growth is not relevant (Durran and Gingrich 2014;Selz 2019).But the rate of error growth on different scales depends strongly on the state of the atmospheric flow (Lorenz 1996;Palmer et al. 2014;Selz 2019).Most small-scale disturbances in the atmosphere will grow slowly or even decay, and the system is relatively predictable, except in certain sensitive situations.Only a butterfly that is in the right place at the right time can hope to influence the weather over the entire planet.
A key motivation for the establishment of W2W is the possibility that in certain cases the accuracy of practical weather forecasts may be constrained by the intrinsic limit of predictability.As an example, Fig. 1 shows the skill of the European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble forecasts for a region over the eastern Atlantic, for 2 months centered on the time of the North Atlantic Waveguide and Downstream impacts Experiment (NAWDEX; Schäfler et al. 2018).For this measure of skill, it can be seen that the model forecasts are often close to the perfect value of 100%, but there are times when the skill drops dramatically.Such forecast "busts" have been studied by Rodwell et al. (2016), who suggested that they might be associated with particular weather events upstream, such as large continental convective outbreaks.These may be examples of the sensitive conditions that intrinsically limit predictability.Indeed, in numerical experiments where the initial conditions were perturbed by small amplitude noise, perturbations initially grow rapidly only where precipitation is occurring (Selz and Craig 2015).
Understanding how the intrinsic limits of predictability appear in the atmosphere and how they can be addressed in practical forecasting systems is the challenge that motivates W2W.The name Waves to Weather is intended to evoke the interplay between large-scale, more predictable phenomena such as Rossby or Kelvin waves and local weather events like wind storms or heat waves.The choice of name emphasizes that we approach the challenge of predictability by trying to understand its physical basis.In this article, we will outline who we are, highlight some early scientific results that have changed the way we understand the problem, and present how we hope to provide benefits for the scientific community and society at large.

The W2W Consortium
W2W is structured as a Collaborative Research Center (CRC), funded by the national research agency, the Deutsche Forschungsgemeinschaft (DFG; German Research Foundation).A CRC is the largest "bottom-up" funding instrument of the DFG, where researchers are free to propose projects on any topic that they believe to be scientifically important.Funding is for a period of 12 years, subject to review every 4 years.W2W was formed in 2015, and is currently in the second funding phase (phase 2).Full information is available on our website (www.wavestoweather.de),but in brief, phase 2 of W2W includes 24 research projects, involving a total of 30 principal investigators, and employs 36 doctoral and postdoctoral researchers, along with 5 staff in scientific computing and administration.The proposal was conceived jointly by the Ludwig-Maximilians-Universität (LMU) in Munich, the Johannes-Gutenberg-Universität (JGU) in Mainz, and the Karlsruhe Institute of Technology (KIT).So-called satellite projects have also been included to bring in particular scientific expertise.These are located at the Technical University in Munich, Ruprecht-Karls-Universität in Heidelberg, the University of Hamburg, and the Institute for Atmospheric Physics of the German Aerospace Center (DLR).
From the start of W2W, it was clear that interdisciplinary collaboration would be essential.The consortium had its origin in a smaller research group, the DFG Priority Program "Predictability and Dynamics of Weather Systems in the Atlantic-European Sector" (PANDOWAE; DFG 2016), which focused on the dynamical processes underlying weather prediction.W2W retains this foundation in atmospheric dynamics and includes a wide range of additional expertise.Knowledge of clouds and other physical processes is essential to understanding the fundamental behavior of error growth.Understanding how physics and dynamics act and interact in numerical forecasting systems requires contributions from mathematics and numerical analysis, in particular inverse methods and uncertainty quantification.Statistics and machine learning provide the basis for the creation and evaluation of probabilistic forecasts based on ensemble output, and visualization provides new techniques for exploring large datasets and analyzing the uncertainty in the information they contain.The long duration of a CRC allows close collaborations to develop over time, as individual scientists come to understand the language and priorities of other research fields, and to identify new problems where their methods and expertise can be applied.
Engagement with the international weather research community is a priority for W2W.Just as PANDOWAE was conceived as a German contribution to The Observing System Research and Predictability Experiment (THORPEX) program (Parsons et al. 2017) of the World Weather Research Programme (WWRP), W2W will contribute to a wide range of WWRP actions, and has been recognized as its first joint project.A prime example of our strong support of international research was the organization of the German contribution to the major international NAWDEX field campaign, where we participated with two aircraft and led the operations center.We also support the organization of international conferences, including hosting the Conference on Predictability and Multi-Scale Prediction of High Impact Weather (Laurian 2017), andthe 19th Cyclone Workshop (McTaggart-Cowan 2019).We are developing close ties to the weather services in our region, including the German Weather Service (DWD) and ECMWF, with the goal of creating knowledge transfer projects to exploit new research results.The first such project has already been funded, jointly with the Italian regional meteorological service Agenzia Regionale Prevenzione e Ambiente dell'Emilia Romagna Servizio Idro-Meteo-Clima (ARPAE-SIMC).In this project we aim to take results on how Rossby wave packets influence the predictability of extreme precipitation events in northern Italy and present them in ways that are useful to forecasters (Grazzini et al. 2019(Grazzini et al. , 2020(Grazzini et al. , 2021)).Finally, an international scientific advisory board with representatives from academic research and operational meteorological services provides guidance and feedback on the research and transfer strategies.

Research strategy and progress
A forecast of a high-impact weather event more than a few days in advance is influenced by the state of the atmosphere all over the globe.To determine what aspect is limiting the predictability, three fundamental questions must be addressed.First is the role of "butterflies"-How and how fast do errors grow upscale from arbitrarily small scales?While this will eventually limit the predictability of the atmosphere, in practice the imperfections of the current observing and forecasting systems usually lead to errors on larger scales that grow fast enough that butterflies are irrelevant (Durran and Gingrich 2014;Selz 2019;Zhang et al. 2019;Žagar and Szunyogh 2020).Answering this question will be reached can only be answered with a better understanding of the mechanisms responsible for error growth across the range of scales.The second question is, What is the impact of clouds and other diabatic processes on predictability?Rapid error growth is generally associated with regions where clouds and precipitation are active (Selz and Craig 2015;Baumgart et al. 2019), processes that are not well represented in current prediction models.An essential goal is to quantify the contributions of these imperfections.Finally, although a small error on the other side of the world may ruin a weather forecast, a good prediction of the large-scale weather pattern does not guarantee a good prediction of a local weather event.Local conditions also play an important role, and we ask, How can the impacts of different error sources be combined in the form of a probabilistic forecast?Each of these three questions motivates a group of research projects in W2W, defining a research area.Phase 1 of W2W has applied a broad range of concepts and tools to these questions, and has produced some significant results in each area.
Upscale error growth.Our research into upscale error growth is guided by the hypothesis that diabatic processes, most importantly latent heat release in regions of precipitation, are the prime mechanism for the growth of small initial errors.To model this source of uncertainty, we developed a stochastic boundary layer scheme (Kober and Craig 2016;Rasp et al. 2018;Hirt et al. 2019).Interestingly, while the scheme proved significant for reducing biases in numerical forecasts, the rate and amplitude of upscale error growth seemed relatively insensitive to the details of the small-scale perturbations, provided that sufficient variability was present.
Upscale error growth occurs when small variations in diabatic heating lead to strong modifications of the geostrophically balanced synoptic flow.An obvious path for this modification would be the production of diabatic potential vorticity (PV) anomalies, for example reduced PV in the outflow of a warm conveyor belt, which would then be amplified by baroclinic instability in an intensifying cyclone.Surprisingly, neither diabatic PV modification nor baroclinic instability turn out to be of primary importance.Instead, analysis of the evolution of PV errors shows that the changes to the upper-level flow produced by diabatic heating are mainly associated with strong divergent winds, which lead to geostrophic adjustment and creation of PV anomalies by advection.These perturbations are amplified by nonlinear interactions at tropopause level along the Rossby waveguide, rather than by baroclinic interactions (Baumgart et al. 2018(Baumgart et al. , 2019;;Baumgart and Riemer 2019;Bierdel et al. 2017Bierdel et al. , 2018)).This sequence of mechanisms is shown in Fig. 2 for an ensemble of simulations that differ only in small-scale variability introduced by a stochastic convective parameterization (Baumgart et al. 2019).
Beyond the synoptic scale, our experiments indicate a previously unrecognized stage of error growth to even larger scales (Baumgart et al. 2019).This stage appears to be associated with changes to the envelope of Rossby wave packets, and the appearance and disappearance of blocking situations.Central to the analysis of these large-scale processes is an improved diagnostic framework, based on local finite-amplitude wave activity, to identify large-scale circulation features and associated errors (Ghinassi et al. 2018(Ghinassi et al. , 2020)).
Complementing the theoretical research on error growth, W2W played a leading role in the coordination of NAWDEX.Aircraft observations from this campaign show significant errors in the operational analyses near the tropopause, where divergent circulations are strong (Schäfler et al. 2018).The campaign period featured several significant forecast busts (Fig. 1), and preliminary analysis suggests that the poor forecasts are associated with interactions among families of cyclones, leading to atmospheric regime changes such as the establishment of a blocking pattern over Scandinavia (Schäfler et al. 2018).The observed behavior is consistent with our new understanding of error growth mechanisms, but also indicates that the contribution of dynamics on scales larger than that of single cyclones is essential to understanding predictability at longer forecast ranges.Both results show the importance of moving beyond established paradigms for error growth.
Cloud-scale uncertainties.Rapid error growth is typically associated with diabatic processes in and around clouds.However, the complexity of cloud processes across a range of scales from individual droplets to synoptic-scale cyclones means that they can only be crudely represented in numerical models, and are a key source of uncertainty in weather and climate predictions.Improving the treatment of cloud processes in models has been a research priority for many decades and is unlikely to have a quick solution.In W2W, we focus on a different question, and seek to quantify the uncertainty that our lack of knowledge of cloud processes creates, and to evaluate its contribution to limiting the overall predictability of the atmosphere.
One type of error is structural uncertainty.These are uncertainties associated with errors in the formulation of the models, often resulting from oversimplified or incorrect representation of the physical processes.One example of a common oversimplification is the use of saturation adjustment schemes, which assume that any supersaturation with respect to water vapor is immediately removed by condensation.The resulting excess condensation gives additional diabatic heating that feeds back on the cloud dynamics and leads to further errors (e.g., Grabowski and Morrison 2017).Barrett et al. (2019) demonstrated that precipitation forecasts are sensitive to the treatment of supersaturation even within a single model time step.Progress in the direction to resolve the supersaturation issue has been made by the development of a new prototype cloud microphysical scheme by Porz et al. (2018).Finally, the impact of structural differences in cloud microphysical schemes was demonstrated by a systematic mathematical analysis by Rosemeier et al. (2018).Using asymptotic methods, they demonstrated that the schemes have a similar long-time behavior under specific supersaturation conditions, although their nontrivial equilibrium points differ.An additional example of dramatic simplification is the common approximation that radiative transfer acts only in the vertical direction.This has been shown to result in a stronger destabilization of cloud layers and reduced cloud organization compared to solutions from 3D radiative transfer (Klinger et al. 2017;Črnivec andMayer 2019, 2020).
A second source of uncertainty lies in the numerical constants used in parameterizations of physical processes.The impact of these parameters is often studied using sensitivity experiments, in which the parameter space is sampled by running simulations for different combinations of values.Since the parameter space is enormous, such strategies are rarely comprehensive or quantitatively reliable.We have addressed this using an emulator approach.The parameter space was efficiently sampled with a limited number of numerical simulations and a statistical surrogate model was created, allowing results to be obtained for other parameter combinations without additional simulations (Wellmann et al. 2018(Wellmann et al. , 2020)).In parallel, we have developed a visual data analysis workflow to explore the large datasets from the set of simulations (Kumpf et al. 2019).Regions of the parameter space with similar behavior can be detected efficiently, and the associated spatial and temporal regions can be identified.Another promising approach is parameter estimation as part of the data assimilation process, which not only provides an optimized estimate of the value, but also information about the degree to which the observations constrain the outcome.Ruckstuhl andJanjić (2018, 2020) estimated surface roughness lengths using an ensemble Kalman filter, and found that the improved values provided a clear benefit for short-term precipitation forecasts.
With so many sources of cloud-scale uncertainties in principle, an additional question of great practical importance emerges: which model uncertainties contribute most significantly to limiting the predictability of particular weather events?We have started to address this by exploring the joint sensitivities to various process parameterizations and environmental conditions (Baur et al. 2018;Schneider et al. 2019;Barthlott and Hoose 2018;Keil et al. 2019).Figure 3 shows an example of the spread of ensemble forecasts of precipitation over Germany that result from perturbations to soil moisture, boundary layer, and cloud microphysical parameterizations (Keil et al. 2019).Two additional forecasts are provided for reference, an ensemble perturbed by small amplitude white noise, and the limited area ensemble prediction system of the German Weather Service.The noise perturbations give an indication of the spread that results from the intrinsic growth of small-scale errors, while the operational ensemble spread has been roughly tuned to capture the uncertainty of the forecast system in practice.The figure suggests that the intrinsic uncertainty is much smaller than the practical error in this case, but that a combination of perturbations to different physical processes could account for the difference.Although improving the representation of cloud-scale processes in weather prediction systems is an important goal of atmospheric science, seeing the question in the context of the limited predictability of weather requires a change of emphasis.Better knowledge is required regarding the uncertainty of cloud-scale processes, requiring the application of new mathematical and numerical techniques.
Predictability of local weather.The third research area deals with the practical predictability in weather forecasts of local highimpact weather events, which is influenced by large-scale dynamical features such as Rossby wave packets or equatorial waves in the tropics, but also by local processes such as land surface interactions.Our initial approach to the problem attempted to assess the relative roles of these two influences for a variety of weather systems with different characteristics, including surface wind gusts associated with midlatitude winter cyclones, summertime heat waves, storms that show characteristics of tropical development, and convective precipitation over tropical Africa.
An important corroboration of the initial hypothesis of large-scale inf luence was found for the understanding of heat waves.Fragkoulidis et al. (2018) and Fragkoulidis and Wirth (2020) demonstrated that local waviness related to Rossby waves in the upper troposphere is much more important for local temperature extremes in the lower troposphere and near the surface than circumglobal waves.Using a backward trajectory analysis for summer heat waves that occurred in five European regions during the period 1979-2016, Zschenderlein et al. (2019) found that large-scale subsidence and adiabatic warming is a very important contributor to extreme surface temperatures.Another example of an observed influence of larger scales is local rainfall over tropical West Africa. Schlueter et al. (2019a,b) revealed how rainfall intensities are modified in the wet and dry phases of different tropical wave types, including the influence of wave interference.However, further research showed that this knowledge is not yet reflected in forecast skill.An evaluation of ensembles of forecast models found that current operational systems are inferior even to climatological predictions over parts of the tropics, especially in tropical Africa (Vogel et al. 2018(Vogel et al. , 2020)).
Given the limited practical predictability of many high-impact weather events, we have moved on to explore how numerical forecasts can be combined with climatological and other information to produce improved probabilistic forecasts.For wind gusts in winter storms, Pantillon et al. (2018) showed that classical regression-based statistical postprocessing generally improves raw ensemble forecasts from a convection-permitting model.However, postprocessing fails to reduce forecast errors for some winter storms with uncharacteristic forecast errors, and even increase them for a rare sting-jet cyclone affecting western Europe in October 2013.For 2-m temperature forecasts for Germany, Rasp and Lerch (2018) proposed a novel postprocessing approach based on neural networks that can incorporate nonlinear relationships between arbitrary predictors and forecast distribution parameters, learned in an automated data-driven way. Figure 4 shows that the neural network approach supersedes state-of-the-art techniques at most locations, and furthermore that it can provide insight into the relative importance of different physical mechanisms.In this example, the relative importance of each input parameter is measured by the decrease in forecast skill when its values are permuted randomly.As might be hoped, the model prediction of 2-m temperature is a very important input, but the figure also shows that information about the location of the station is very important, suggesting a role for small-scale orographic detail that is not well represented at the model resolution.
These studies show that while our ability to understand and predict local weather events is based on knowledge and simulation of the underlying physical and dynamical mechanisms, this knowledge does not represent all of the information available to us.Statistical postprocessing and machine learning enable the application of additional climatological and geographical data.So far, such methods are generally applied to the output of the numerical forecasting system.However, the information about mechanisms, revealed for example by the neural network model in the previous example, suggests that even more can be gained by hybrid methods that use dynamical and statistical knowledge to produce a forecast together.
New challenges and interdisciplinary approaches.Over the course of a 12-yr program new tools and methods will emerge-and even new scientific problems.Here we provide a few examples of these developments, focusing on how interdisciplinary collaboration helps us address them.
A scientific area where W2W can potentially make an important contribution is the emerging field of subseasonal prediction.While phase 1 of W2W was focused more on the classical numerical weather prediction scales of 1-10 days, new projects in phase 2 are addressing aspects of subseasonal prediction (10-30 days).On the one hand, this builds on previous research by extending the analysis of error growth to even larger scales, quantifying the impact of diabatic processes, and identifying methods for producing probabilistic forecasting when predictability is low.However, new expertise was also required, in topics including cyclone families (Dacre and Pinto 2020), persistent weather regimes (Hauser et al. 2020), and new physical mechanisms such as interactions with the stratosphere (Kautz et al. 2020).These new projects will also open new frontiers for collaboration with the seasonal and climate prediction communities, including valuable resources such as the new Subseasonal-to-Seasonal Prediction Project (S2S; WMO 2020) and Subseasonal Experiment (SubX; NOAA 2020) databases.
In addition, W2W can benefit from recent developments in other disciplines regarding the representation of uncertainty in numerical models.For initial-condition uncertainty, data assimilation provides an efficient framework to combine the influences of observation and model uncertainty (Zeng et al. 2017(Zeng et al. , 2020;;Janjić et al. 2018).On the other hand, the representation of forecast errors by ensembles is very costly, and methods that provide uncertainty information without computing large numbers of numerical forecasts would be very desirable.An example of a promising approach is uncertainty quantification with the stochastic Galerkin method, which integrates uncertainties forward in time using a spectral approximation in the stochastic space.As a result, only a few model simulations are required to determine the impact of uncertain parameters in complex atmospheric flows.A preliminary study showed the effectiveness of this method in describing the impact of parameter uncertainty in a cloud microphysical model (Chertock et al. 2019), and work is continuing to models that fully couple the microphysics and dynamics.
Two further areas where interdisciplinary work already has an impact on W2W are statistical postprocessing of ensemble forecasts (Vannitsem et al. 2020) and visual analytics (Rautenhaus et al. 2018).Studies of statistical postprocessing in W2W have focused on comparing different methods, in some cases including machine learning techniques such as deep neural networks (Baran and Lerch 2018;Rasp and Lerch 2018;Lang et al. 2020).Methods from visual analytics have been employed to identify physically interesting structures in large datasets, for example jet stream cores in ensemble forecasts (Kern et al. 2018).Finally, convolutional neural network models have been successfully applied to downscaling of wind forecasts (Höhlein et al. 2020).Each of these methods has its own strengths and weaknesses, and a long-term goal of W2W is to develop new hybrid algorithms that could be used for the Unauthenticated | Downloaded 12/29/23 09:12 PM UTC investigations of error growth, regime transitions, and for exploring new directions in postprocessing of ensemble forecasts.

The research environment
The success of an ambitious research program depends on a community of capable and committed scientists and much of the planning and coordination within W2W is dedicated to creating an optimal working environment, where people have the training and resources to achieve their goals.This section describes some of those initiatives.
Early-career scientists.W2W currently employs more than 36 early-career scientists (ECS; doctoral students and postdoctoral researchers).Their performance and productivity are crucial for the consortium's success, and their knowledge and ideas make an essential contribution to the ongoing strategy of W2W.W2W, in turn, has a responsibility to provide an optimal environment, in which the ECS have the potential to develop their skills and prosper as independent researchers.The guiding principle in developing this environment is that the ECS know best what they require.The CRC has a dedicated budget for ECS activities, which is delegated to an elected ECS committee, under the oversight of the W2W Steering Group.A representative of this committee joins the regular meetings of the steering group, allowing for an efficient exchange of information.
The ECS committee is responsible for the organization of meetings to enhance scientific and general skills, to foster connections within the W2W community, and to generate and encourage research collaborations.A recent example is a Machine Learning Workshop, where basic and advanced aspects were taught through a variety of lectures and hands-on coding activities.This was led by a former Ph.D. student from phase 1 of W2W, and illustrates how ECS are able to take the initiative in meeting their scientific needs and interests.ECS meetings also play a key role in building a sense of community, not just at the individual institutions, but across W2W.Other community building activities include biweekly virtual coffee breaks, an active "language tandem," and a "writing club" where scientific texts are exchanged for feedback.A particularly important initiative of the ECS committee was the establishment of a program in which each ECS has the opportunity for an extended visit to an international research institution.Funding is also available to invite international ECS to visit Germany.
Equal opportunities.W2W strives to offer an open and inclusive environment for persons of all gender identities and sexual orientation, of diverse cultural backgrounds, age, and other dimensions of diversity.Activities to support this are led by the W2W Equal Opportunity Committee (EOC), consisting of six elected members, three women and three men, on career levels from Ph.D. student to professor.
A special focus of the funded activities within W2W is on family friendliness and the advancement of women in academia.This includes childcare options at project meetings, home office equipment and student assistants for parents with young children, and offers of individual coaching for female ECS.In addition, the EOC is available for informal and confidential requests from all W2W members regarding topics related to equal opportunities such as parental leave or career development.
To raise awareness about prevailing biases in the scientific community and in academia, the topic is addressed prominently at the W2W annual meetings (with workshops and invited presentations) and in the W2W newsletter.Outreach activities targeting the next generation of students [in particular, a comic book featuring female and male role models (Laurian 2020) and contributions to open-day activities for schoolgirls] aim to convey a diverse picture of scientists in the W2W research fields and to improve the gender balance and diversity in future.
Scientific computing.Expensive computations and very big datasets are fundamental components of basic research on weather predictability.Ensemble sizes of up to a thousand members and petabytes of data pose great challenges for our scientists, including aspects such as cross-site collaboration and reproducibility of scientific results.To meet these needs, W2W includes a scientific computing subproject, pursuing a two-pronged approach of service and research.
In the service component, an infrastructure for storage and exchange of data has been a main focus for phase 1.Scientists from all sites have a central storage system at their disposal for collaboration, which is connected to a cluster for interactive analysis and remote visualization.The investment in this open-source powered system (VirtualGL 2020; Jupyter 2020) has paid off especially during the pandemic conditions of 2020/21.Scientists using this system were able to transfer their activities to home office with a minimum of disruption, and continue their interactions with colleagues from their own institutions using tools that were introduced to promote collaboration with remote partners.In phase 2, this system is being extended by a data management platform (iRODS 2020), which connects the partner universities and allows us to follow principles for Findability, Accessibility, Interoperability, and Reuse (FAIR) of data and code.Our scientific programmers are not only working in support of project members, but also toward the open-source publication of ensemble-related tools developed in W2W.
To enable us to meet future requirements, our scientific computing activities include a research component, where we seek to apply new methods developed in other research fields.One of these is algorithmic differentiation, which will potentially allow us to replace some expensive sensitivity studies (Baumgartner et al. 2019).A further topic is lossy compression, which, combined with more parallelization, has the potential to overcome limitations imposed by the vast amount of data we expect to handle.
Dissemination and outreach.Despite its focus on basic research, the W2W program has instigated substantial dissemination and outreach activities, guided by a strategic plan and coordinated by a team of junior and senior scientists.The web page of W2W (www.wavestoweather.de)contains information on research, people, publications, meetings, seminars, scientific guests, etc., but it is only the starting point for a range of activities under the twin headings of dissemination and outreach.
For dissemination, i.e., the communication and transfer of scientific output into other research settings, we mainly target university scientists and research departments of weather services.The visibility of our research is enhanced by special collections in prominent journals (Laurian andCraig 2016, 2019), and by hosting scientific meetings (see the "The W2W Consortium" section).In addition, we reach out to the community through (i) overview and review articles on key research topics in W2W [e.g., Wirth et al. (2018) on Rossby wave packets, Rautenhaus et al. (2018) on visualization, andVannitsem et al. (2020) on postprocessing], (ii) dedicated workshops for enhanced knowledge exchange (the first of these is planned in association with the annual meeting of the European Meteorological Society in 2022), and (iii) (interactive) web resources for data, software and forecast products developed within W2W.Transfer projects (see the "The W2W Consortium" section) also offer a powerful way to integrate research into operations.
For outreach, i.e., the communication of the broader impacts of research results, we mainly target the general public, the media, and school children to train the next generation of weather enthusiasts.A key goal is to improve the understanding of probabilistic weather forecasts illustrated through examples of current weather events and significant (often extreme) events from the past.W2W scientists contribute to this goal through public presentations to lay audiences, contributions to museum exhibitions and open days, press releases and media interviews, workshops, and the development of educational material.
Sustainability.The 12-yr funding of CRCs such as W2W represents a substantial investment from the research council and from the partner universities.This raises expectations that the program will have a lasting impact, not only on the relevant scientific field, but also on the structure and capabilities of the participating institutions.
A key requirement for a sustainable impact is ensuring that a critical mass of expertise is maintained over the long term.At each of the three universities, new permanent positions have been filled at different levels, including new professors, senior scientists, and research group leaders, as well as scientific computing staff.W2W topics have also gained an increasing coverage in the curricula of the participating universities, for example through updated or newly designed courses in numerical weather prediction, data assimilation, synoptic meteorology, statistics, and data analysis, at both undergraduate and graduate levels.
W2W is effective in fostering interdisciplinary collaboration.For example, KIT intensified interactions with the Heidelberg Institute for Theoretical Studies (HITS), while at JGU connections between meteorology and applied mathematics and computational sciences were strengthened.At LMU the collaboration with the DWD was expanded and placed on a sustainable basis within the Hans-Ertel-Zentrum (HErZ; Simmer et al. 2016) for weather research.
As a result of all these initiatives, W2W enhances the visibility and appreciation of atmospheric sciences at the three universities and opens new possibilities for recruitment and interdisciplinary collaboration, strengthening it in the competition with other subject areas.
Looking to the future This article has attempted to summarize the W2W program to investigate the limits of predictability of weather, and to highlight some of the scientific results that have been obtained so far.Although the focus has been on W2W, we are part of a global community of researchers.We anticipate that our specific efforts will make a contribution to the scientific progress of the field as it advances in different ways: • by moving beyond old paradigms, for example, on mechanisms for error growth, • by changing the research questions, including a stronger focus on quantifying uncertainty rather than hoping to eliminate all errors, and • by interdisciplinary transfer of knowledge and creating new hybrid forecasting methods.
A major focus of our efforts has been devoted to creating a positive and productive environment for the scientists participating in the program, to give us a sustainable foundation for progress in the exciting field of weather prediction research.

Fig. 1 .
Fig. 1.Skill of forecasts from the ECMWF Integrated Forecasting System (IFS), shown by a time series of anomaly correlation coefficient (ACC) for geopotential height at 500 hPa over an area 35°-75°N, 60°-0°W for a forecast lead time of +120 h: IFS deterministic forecast (black line), ensemble mean (red line), 50% of the ensemble members (orange area), and all members (yellow area).Gray shading shows the time period of the NAWDEX campaign, with the periods influenced by three low-predictability events highlighted in darker gray.

Fig. 2 .
Fig. 2. Sources of potential vorticity (PV) error show the physical mechanisms responsible for upscale error growth.(a),(b) PV differences (color) on an isentrope (325 K) intersecting the tropopause (solid and dashed contours) in an experiment in which the only initial difference is in the seeding of the stochastic Plant-Craig convective scheme.(a) After 5 days localized, mesoscale errors have developed, which (b) grow to errors on the scale of individual troughs and ridges after day 15.(c) Processes that govern multistage error growth.The initial rapid growth of small-scale differences is governed by differences in nonconservative processes, mainly diabatic heating (green curve, first 12 h), followed by mesoscale differences in the displacement of the tropopause by upper-tropospheric divergent outflow (red curve).The differences in the nonlinear tropopause dynamics later govern the further amplification and upscale growth of the PV differences (blue curve).For details, see Baumgart et al. (2019).

Fig. 3 .
Fig. 3. Impact on forecast uncertainty of different aspects of model error in convection-permitting ensemble forecasts for Germany.Plots show time series of ensemble spread (S n ) of the domain-averaged precipitation for cases with (top) strong and (bottom) weak large-scale forcing.Ensemble subsets (colored lines) include prescribed soil moisture heterogeneities (Soil), boundary layer perturbations (PSP), various cloud condensation nuclei concentrations (CCN), and white noise perturbations (WNoise).For reference, the operational high-resolution ensemble (EPS) is shown (from Keil et al. 2019).

Fig. 4 .
Fig. 4. (a) Observing station locations color-coded by the best-performing model for 2-m temperature forecasts over Germany.Blue dots indicate stations where the neural network-based postprocessing was superior to state-of-the-art approaches.(b) Feature importance (defined in terms of decrease in mean continuous ranked probability score when randomly permuting input parameters) for the 10 most important predictors on a logarithmic scale.Green bars indicate ECMWF mean ensemble predictions of meteorological variables; red bars are station-specific information (after Rasp and Lerch 2018).