Studying changes made to initial conditions or other model aspects can yield valuable insights into dynamics and predictability but are associated with an unrealistic phenomenon called chaos seeding that can cause misinterpretations of results.
Since chaos was discovered by Ed Lorenz in the 1960s (Lorenz 1963), there has been substantial interest in how differences in model initial conditions evolve to determine the predictability and sensitivity of the atmospheric state within a nonlinear system of governing equations. Some recent studies have resulted in new insights based on initial-condition error. These include the importance of storm-scale errors that emerge from downscaled meso- or synoptic-scale error [relative to the role of upscale growth of initially smaller errors (Durran and Weyn 2016)] or the distinction between intrinsic [“the extent to which prediction is possible if an optimum procedure is used” (Melhauser and Zhang 2012, 3350–3351)] and practical [“the ability to predict based on the procedures currently available” (Melhauser and Zhang 2012, p. 3350)] predictability. Many of these principles are applied to today’s numerical weather prediction (NWP) models and state-of-the-art data assimilation systems to mitigate initial-condition error toward producing the most skillful weather forecasts through both the routine observational network (e.g., Langland and Baker 2004) and adaptive observing strategies (Langland 2005; Majumdar 2016). Collectively, the large body of work regarding the chaotic evolution of initial-condition errors has provided valuable guidance to the development of operational forecasting/data assimilation systems that aim to decrease detrimental weather-related impacts on safety and the economy. Improved computational efficiency now allows current efforts aimed at understanding predictability and atmospheric sensitivity to initial conditions to directly address severe convection (Clark et al. 2012), a major threat to society.
In addition to initial conditions, there are other aspects of NWP models that play a role in the evolution of the atmospheric state and its predictability. Lateral boundary conditions used by limited-area mesoscale models, typically a requirement for regional convection-allowing simulations, directly affect the evolving state. Model physics parameterizations that handle subgrid processes such as turbulence in the planetary boundary layer, cloud processes, and radiation are not perfect, and their accuracy certainly plays a role in the skill of model forecasts. Further, the wealth of studies regarding the skill of different model parameterizations suggests there is no “magic bullet,” or combination of parameterizations, that is best for many times, locations, scales, and atmospheric flow situations. This may explain why multiple parameterizations for different types of physics (i.e., cloud physics or boundary layer turbulence) are available to both operational and research NWP models [e.g., the Weather Research and Forecasting (WRF) Model (Skamarock et al. 2008)]. The accuracy of numerical schemes, ad hoc techniques such as numerical diffusion (Knievel et al. 2007) or the use of model-top damping (Klemp et al. 2008), and model resolution also influence predictability.
One common and widely used strategy to better understand predictability or sensitivities in the atmospheric system is to use perturbation experiments to assess the evolution of differences within model simulations. These differences may involve initial conditions, boundary conditions, model physics, or other aspects (e.g., model resolution). These types of experiments have led to numerous conclusions regarding the effects of these differences in model simulations, and the simplicity and value of perturbation experiments means they are likely to be continued in the future. However, we have discovered a significant issue in the use of perturbation experiments that can bring into question the validity of results. It has been found that any perturbation made to model prognostic variables can produce an unavoidable, rapid, and unrealistic propagation of numerical noise that can seed chaos and lead to misinterpretations of the evolution of the prescribed perturbation. “Chaos seeding” is thus defined here for the purposes of this study as the rapid and unrealistic creation of perturbations throughout the entire model state during the solution of the governing equations. Characterizing and demonstrating chaos seeding and bringing its implications to the attention of those who employ perturbation experiments and to those who are guided by results from such experiments in any numerical model, atmospheric or not, is the purpose of this study. Additionally, we present some techniques that can be useful in discriminating realistic effects from those associated with chaos seeding. Given the breadth of numerical perturbation experiments in the field of atmospheric science alone, and the subsequent results that those experiments have provided, it is hoped this study can be used both to enhance the foundation perturbation experimental results have provided to the academic and operational communities in the past and to help avoid any issues chaos seeding may create in the future.
BACKGROUND.
The phenomenon of chaos seeding has barely been noticed in previous studies, and the seriousness of its effects has been largely overlooked. In characterizing the predictability and error growth dynamics within cloud-resolving NWP models, Hohenegger and Schar (2007) observed rapid propagation of perturbations throughout a roughly 900 km by 650 km domain at 2.2-km grid spacing. They attributed this rapid propagation of perturbations to acoustic and gravity waves, which were rapidly amplified in areas of convection. The study mentions the possibility of numerical noise playing a role in the unrealistically rapid propagation of perturbations but states that the more realistic, slower modes ultimately overpower the signal, which may be a consequence of the relatively small domain size. Leoncini et al. (2010) attributed similar effects within a slightly larger model domain with 4-km grid spacing entirely to acoustic, Lamb, or gravity waves. While these studies clearly demonstrate how areas of convection far from the source of a change made to model initial conditions can quickly be associated with amplifying perturbations, they stop short of discussing whether the effects are realistic or not and, in fact, attribute them primarily to known physical processes. Distinguishing if the effects are realistic or not, however, represents a key contribution to the predictability base of knowledge, a fact supported by Hoheneggar and Schar (2007) in their discussion of how rapidly propagating perturbations limit predictability (and can also complicate the design of high-resolution data assimilation and observation targeting systems). Making this distinction between realistic and unrealistic seeding of subsequent rapid perturbation growth and discussing its implications are primary goals of our study here.
One of the only studies to the authors’ knowledge to approach the generalization of the effects of chaos seeding, with the blame placed squarely on unrealistic processes, was Hodyss and Majumdar (2007, hereafter HM2007), a study that also clearly states the significance of unrealistic processes on the interpretation of predictability. They used perturbation experiments to examine the forecast effects on vertically integrated total energy of assimilating particular observations in a global spectral model. HM2007 discuss the discovery of the “contamination” of the state in dynamically unrelated regions that “mysteriously appeared,” presumably a result of truncation errors in either the data assimilation or in the model’s spectral representation of the spatial field. The rapidly amplifying perturbations they find are attributed to unrealistic processes, “muddling” the dynamic interpretation of the assimilated observations in the first place. Interestingly, HM2007 found that this contamination existed initially in remote locations from the perturbation source, in contrast to the other studies discussed above and the study presented here, which finds that perturbation rapidly propagates. Nonetheless, whatever the cause of the initial and very small perturbations, they are able to rapidly amplify through nonlinear processes, with the smallest perturbations growing the fastest in areas of convection (Zhang et al. 2003; Luo and Zhang 2011). In particular, Zhang et al. (2003) show that for domainwide sinusoidal initial temperature perturbations with varying magnitudes, different growth rates of difference total energy [defined through the sum of the squares of wind and temperature perturbations as detailed in Eq. (1) of Zhang et al. (2003)] between the runs are found, with much faster relative growth rates for much smaller perturbations (Fig. 1). Here, we aim to generalize these adverse effects and show, at least within one very common atmospheric modeling system, their unavoidable nature and much greater prevalence than was previously realized.
Difference total energy growth rates for different sinusoidal initial temperature perturbation magnitudes over a 36-h simulation window (from Zhang et al. 2003).
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
PLANTING THE SEEDS—THE PROPAGATION OF NUMERICAL NOISE.
Here, we use the WRF Model to characterize the unrealistic rapid propagation of perturbations that ultimately seed chaotic growth. Figure 2 shows a 12-km modeling domain with a nested domain at 4-km grid spacing (both with 38 vertical levels) and the source of a perturbation made to soil moisture [effectively 1 in. (1 in. = 2.54 cm) of water added at a single model grid point] that will be examined here. The WRF Model parameterizations used are the Yonsei University (YSU) planetary boundary layer scheme (Hu et al. 2013), the Kain–Fritsch cumulus parameterization (Kain and Fritsch 1990, 1993), the NOAA/NCEP–Oregon State University–Air Force Research Laboratory–NOAA/Office of Hydrology land surface model (Noah; Chen and Dudhia 2001), Thompson microphysics (Thompson et al. 2004), the Rapid Radiative Transfer Model (RRTM) longwave radiation scheme (Mlawer et al. 1997), and the Dudhia shortwave radiation scheme (Dudhia 1989). Two model simulations on the 12-km grid are first executed with a 60-s time step to 60 h, with the only difference being the soil moisture perturbation.
The 12- and 4-km domains and soil moisture perturbation location used for various perturbation experiments in this study.
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
Table 1 shows the resulting propagation of the perturbation at 500-km intervals every 10 min for the first hour of the simulation with respect to lowest eta level (∼40 m from the ground) potential temperature (a WRF prognostic variable). Relatively large perturbations are observed in the first hour above the soil moisture perturbation, with a rapid propagation of significantly smaller magnitudes throughout the domain. This rapid propagation occurs at speeds of about 3,600 km h−1, well above the propagation speed of any known physical process, including that of acoustic modes. While potential temperature is shown in Table 1, these very small perturbation magnitudes propagate into every prognostic variable and travel three-dimensionally. This process thus perturbs all prognostic variables with very small magnitudes over even the largest mesoscale domains within an hour or two. To ensure the effect is not rooted in the aspects of the specific computing architecture, this test was performed on different computing platforms, and the perturbation was also made to a diagnostic variable. No change was found on different computing platforms, and perturbations made to diagnostic variables had no effect on the atmospheric state—the phenomenon is rooted in the numerical solution of the prognostic variables.
Lowest-eta-level potential temperature perturbations (K) at 0-, 10-, 20-, 30-, and 40-min simulation times at locations every 500 km between 0 and 2,000 km to the east of the perturbation location shown in Fig. 2 on the 12-km domain using a WRF single-precision configuration.
Further, the tiny and speedy perturbations are unavoidable. The use of digital filter initialization (DFI; Peckham et al. 2016), sixth-order numerical diffusion (Knievel et al. 2007), or an upper boundary damping layer (Klemp et al. 2008) does nothing to change the rapid propagation of the perturbations within the simulation. Ultimately, the rapidly propagating perturbations shown in Table 1 are a consequence of an expanding domain of influence involved with the solution of the partial differential equations that govern the system as described in Durran (2010, section 3.2.3). Figure 3 shows how spatial finite-difference schemes used in gridpoint models can rapidly communicate perturbations spatially since the solution at future times depends on the state at the previous time at both that grid point and a number of points surrounding that grid point. For example, fourth-order finite-difference approximations to the spatial derivatives that appear in atmospheric governing equations incorporate information from a two-gridpoint neighborhood to use with time-stepping schemes to solve for the state at the next time. As shown in Fig. 3, this allows a propagation of a perturbation at a rate of two grid points per time step.
A one-dimensional example of how a spatial finite-difference scheme propagates information from a perturbed variable across the domain. Large blue dots represent grid points where information from the perturbation, initially at point x6 at time to, is communicated. The red arrows show which points are used at a prior time to calculate the solution for the fourth-order center-difference scheme shown here.
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
In practice, grid staggering is used, and thus, the fifth-order finite-difference approximation used in the WRF experiments here uses an effective neighborhood of 2.5 points in each direction but is inherently diffusive (Skamarock et al. 2008; Knievel et al. 2007) such that a five-point neighborhood determines the solution at the next time. This would predict a propagation speed of 5∆x per time step (60 s), or 3,600 km h−1, which essentially matches the observed propagation speed in the WRF Model. This strongly suggests that chaos seeding results from the expanding numerical domain of influence determined by the specific numerics in a gridpoint model, with the rate of expansion substantially faster than any real dynamical process. Further, this characteristic means the propagation of perturbations is somewhat resolution independent since finer grid spacing requires smaller time steps to avoid Courant–Friedrichs–Lewy (CFL) errors, which would maintain the rapid propagation.
Now that the chaos-seeding mechanism has been established, the key question becomes, given small unrealistic perturbations throughout the domain shortly after the start of a simulation, can these perturbations grow large enough to adversely affect perturbation experiments? As Fig. 1 demonstrates, the relative growth of the smallest perturbations can be expected to be most rapid, and the following section attempts to provide an answer to this question for different types of perturbation strategies.
THE GROWTH OF NUMERICAL NOISE AND ITS CONSEQUENCES.
Here, we provide several examples of different types of perturbation experiments that can all be affected adversely by the chaotic growth of seeded numerical noise.
Initial-condition perturbations.
Perturbing initial conditions is a popular way to study how changes to atmospheric variables at the beginning of a forecast evolve to influence the state later in time. Data assimilation observation impact experiments such as those conducted in HM2007 fall under this type of perturbation strategy and are widely used to make conclusions about how certain assimilated observations add skill to forecasts. Chaos seeding through the rapid propagation of numerical noise, however, can substantially cloud the interpretation of these types of experiments. Figures 4 and 5 show an example of these adverse effects for a simulation begun at 0000 UTC 19 May 2013. The 4-km domain (shown in Fig. 2) was used to simulate heavy precipitation in the southeastern United States. This simulation used boundary conditions from the 12-km parent domain, and both domains were initialized by Global Forecast System (GFS) initial conditions (GFS forecasts are also used for boundary conditions for the 12-km grid). With the exception of no cumulus parameterization and a 20-s time step, the 4-km simulation utilizes the same physics as the 12-km grid. Figure 4 shows a subset of the 4-km control simulation over the southeastern United States, depicting 6-h precipitation valid at both 6- and 30-h simulation times. Localized areas of 6-h precipitation up to roughly 25 mm appear by 6-h forecast time, and by 30-h forecast time, mesoscale areas of precipitation with finer-scale local maxima reaching well over 40 mm appear in Georgia and the Carolinas.
Control 6-h accumulated precipitation over a portion of the 4-km domain valid at (left) 6- and (right) 30-h simulation time.
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
Accumulated 6-h precipitation differences (left) between the control simulation and the simulation with soil moisture perturbed in the Carolinas (realistic) and (right) between the control simulation and the simulation with soil moisture perturbed in western NV at the location shown in Fig. 2 (unrealistic) at 6-, 12-, and 30-h simulation times.
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
Figure 5 depicts the differences in 6-h precipitation at 6-, 12-, and 30-h forecast times from two independent sets of perturbation experiments. The left column shows differences in precipitation resulting from an initial-condition soil moisture perturbation of 1 in. of added water (to simulate irrigation) made under the area where precipitation differences begin to appear in the control run (indicated by the pale yellow box in the top-left panel)—this perturbation was designed in a way that one might expect precipitation could be directly influenced by soil moisture and is referred to as the “realistic” experiment. The right column represents precipitation differences resulting from a perturbation of the same magnitude at a single grid point in western Nevada (shown in Fig. 2)—these differences represent the rapid growth of numerical noise that quickly spreads from the perturbation location in Nevada across the entire domain and are referred to here as “unrealistic.” Figure 5 shows that by 12-h forecast time, the nature of the differences for both perturbation strategies are very similar, and by 30-h forecast time, both experiments show larger-scale 6-h precipitation differences of around 200 mm. While some differences in detail are evident in these difference fields, the differences exhibit similar characteristics among the two experiments in terms of magnitude and spatial nature. In fact, several other unrealistic experiments, performed by simply moving the perturbation location in western Nevada by a grid point or two, produce differences very similar to those between the unrealistic and realistic experiments shown in Fig. 5.
These results reveal the heart of the problem—while the precipitation differences that evolve in the realistic experiment could be expected to result from a local soil moisture perturbation, these differences have likely evolved from the rapid growth of numerical noise. This is because the same type of evolution occurs whether the perturbation is local or so far away that only unrealistic processes could have caused it. This is not to say some realistic effect is not present from the local soil moisture perturbation in the left column of Fig. 5, but that effect can simply not be diagnosed in a meaningful way given the rapid growth of noise that is also occurring. More generally, these results suggest that interpretations of the evolution of model variables stemming from any change to initial conditions could be severely and adversely plagued by the phenomenon of chaos seeding.
Figure 6 presents a second example of the adverse effects of chaos seeding involving a particularly important predictability problem—forecasting severe convection. Figure 6 shows two forecasts: a control 24-h forecast initialized at 0000 UTC 19 May 2013 and a forecast that is perturbed with the same 1-in. soil moisture perturbation at the single grid point in western Nevada. Maximum hourly 2–5-km updraft helicity is plotted in colors, revealing the rotating storm tracks that typically reveal supercells that can produce severe weather including large hail and tornadoes. Key differences in rotating storm tracks occur, most notably in the highlighted area of southeastern Kansas where a long rotating storm track with updraft helicity exceeding 100 m2 s−2 is found in the perturbed run but absent in the control simulation. This difference does not reflect a change in timing—no updraft helicity track was produced in this area in the control run within hours of the 24-h forecast time shown. The change is due to chaos seeding by numerical noise and is completely unrealistic. From the perspective of a researcher unaware of chaos seeding who is attempting to understand how differences in model initial conditions might affect severe convective initiation and maintenance, the result shown in Fig. 6 is particularly dire. This is because any perturbation to initial conditions would likely elicit a similar response to severe convection in that simulation, leading the researcher to believe the result is physically relevant. More generally, erroneous conclusions made in the presence of chaos seeding have the potential to produce substantial setbacks in the physical understanding of atmospheric sensitivities to initial conditions.
Maximum 2–5-km updraft helicity over the last hour at 24-h forecast time for both (left) the control simulation and (right) the perturbed simulation using a soil moisture perturbation in western NV.
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
Physics parameterizations.
The use of different physics parameterizations in otherwise similar model simulations is a common method used to understand how different physics schemes affect simulations of the atmosphere. For example, different microphysical schemes that employ different techniques to evolve various microphysical species can influence the evolving atmosphere in a number of ways, including the distribution of radiation and latent heat processes. Other types of parameterizations, such as the wind farm parameterization implemented in WRF (Fitch et al. 2012), are intended to represent atmospheric processes that otherwise are not accounted for within a modeling system. In this way, the use of such parameterizations can reveal the effects on the atmosphere from human activities such as the creation of a wind farm, allowing the study of inadvertent weather modification and yielding results that may inform important policy decisions. These studies, like those that vary the initial conditions, can also be contaminated by chaos seeding. Figure 7 shows the difference in surface pressure from two simulations—one without any wind farms and one with a simulated aggregate of wind farms in northwestern Texas—valid at 1-h simulation time. In contrast to the more realistic dipole of surface pressure differences surrounding the prescribed wind farm aggregate, the areas highlighted by the circles in Fig. 7 are far-field localized surface pressure differences associated with areas of precipitation. These differences arise from chaos seeding and do not represent realistic processes. While it is obvious at 1-h simulation time which differences are real and which are not, perturbations simulated over longer time periods quickly become interspersed, and even grow upscale and span much of the domain, so that it is essentially impossible to diagnose whether the wind farm or chaos seeding causes any given difference. This issue is not limited to the wind farm parameterization—a modeling domain using any two runs with different parameterizations quickly becomes saturated with numerical noise, which can grow rapidly to reveal large perturbations that have little to do with differences in the parameterizations themselves.
Surface pressure differences at 1-h simulation time between a control simulation with no wind farm and a perturbed simulation with a wind farm aggregate simulated over the box in northwestern TX.
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
Variable boundary conditions.
The use of different lateral boundary conditions is another technique that can be used to assess a source of variability with regard to atmospheric evolution, particularly for limited-area mesoscale modeling systems. One important question these types of studies may try to answer is, How fast do different boundary conditions infiltrate the modeling domain to cause changes on the grid? This question is important because it can guide the design of finescale grids, helping to size the grid appropriately such that locations in question are affected more by the initial conditions on that grid rather than incoming boundary conditions (usually from a coarser model). To understand whether chaos seeding from numerical noise can affect this type of experiment, two simulations were conducted on two different domains. Two 7-h simulations on two different domains (domains shown in top row of Fig. 8) were initialized and run with GFS initial and lateral boundary conditions from the forecast valid at 1200 UTC 20 May 2013 (0–7-h forecast), and then two additional simulations were run on the two domains but with GFS lateral boundary conditions from the 6-h-old GFS forecast (initialized at 0600 UTC 20 May 2013, supplying boundary conditions from forecast hours 6–13). Very similar differences from the use of different boundary conditions occur on both domains by 7-h simulation time. By examining the differences between runs on both domains within the WRF output files, it is clear that numerical noise has been created across the entire grid by 1-h forecast time solely as a result of the unrealistic propagation of numerical noise from the lateral boundaries. While it would be tempting to explain the 7-h forecast differences in precipitation in Oklahoma shown in Fig. 8 as a result of real physical processes migrating into the interior from different boundary conditions (particularly on the smaller grid), these differences are actually completely unrealistic, caused by the rapid chaotic growth of numerical noise seeded well within the interior of the grid.
(top) Differences in 6-h accumulated precipitation at 7-h simulation time for two different 12-km domain sizes representing the effects of perturbed boundary conditions. (bottom) Zoomed-in portions over OK.
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
STRATEGIES TO MITIGATE THE SEEDING OF CHAOS IN PERTURBATION EXPERIMENTS.
The primary purpose of this study is to create an awareness of the issues involved with chaos seeding by numerical noise propagation within perturbation experiments. However, through experiments conducted by the authors, certain experimental and analysis procedures have proven to be helpful in mitigating the misinterpretations that might be created through chaos seeding and could be useful in future work. The following presents three such techniques.
Ensemble sensitivity analysis.
One important observation of the numerical noise is that once it has propagated to seed the entire domain, it appears to have produced small positive and negative values seemingly randomly distributed and uncorrelated with the initial perturbation magnitude. In contrast, actual physical processes are typically correlated with the initial perturbation. One effective technique to reveal a realistic physical dependence on an initial-condition perturbation in the presence of chaos seeding is an ensemble sensitivity approach (ESA; Hakim and Torn 2008; Ancell and Hakim 2007). This approach requires an ensemble of simulations, each run with differences to the initial perturbation (e.g., magnitudes of the soil moisture perturbation). A linear regression of a chosen metric valid later in the forecast window, such as accumulated precipitation, is performed onto the perturbed initial-condition variable (e.g., soil moisture). Numerous ESA studies have shown significant and useful linear relationships between initial-condition variables and forecast variables, even when the chosen metric involves storm-scale variables and substantial nonlinearity (Bednarczyk and Ancell 2015; Hill et al. 2016).
Figure 9 shows an example of how ESA can discriminate real dynamical processes from unrealistic ones created through chaos seeding. The left column of Fig. 9 shows 12-h precipitation differences at 36-h forecast time from two different perturbed 12-km model simulations and a control run initialized at 1200 UTC 18 May 2013. Perturbations to soil moisture representing 1 in. of irrigation were made over the red box in both perturbed runs. Here, we focus on precipitation over the black box in Nebraska and South Dakota as an area that after 36 h of simulation time could potentially be affected by irrigation in the panhandle of Texas. Significant differences to precipitation occur in Nebraska and South Dakota because of soil moisture perturbations in both Texas and New York. From the inspection of this pair of precipitation difference plots, it is impossible to infer a dynamical link between the quantity of precipitation in Nebraska and South Dakota to soil moisture in the Texas Panhandle since we know perturbing moisture there is also seeding chaos and may erroneously be reflecting the same response as that from perturbing soil moisture in New York (for which we can expect no dynamical link).
(left) Differences in 12-h accumulated precipitation at 36-h simulation time for soil moisture perturbations over the red box in both (top) TX and (bottom) NY. (right) The linear regressions of the average precipitation metric valid within the black box in the images in the left column onto irrigation magnitudes for both perturbation experiments.
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
In an attempt to mitigate this issue, an ESA approach is employed, and an ensemble of a range of soil moisture perturbations (0–2.5 in. of irrigated water) is performed at both perturbation locations. The linear regressions of the average precipitation in the black box onto initial soil moisture magnitudes are shown in the right column of Fig. 9. The slope of the linear regression for the Texas perturbations relative to that in New York (about −0.65 vs −0.16 mm in.−1), the p value that indicates the confidence level at which the relationship is significantly different than zero (0.01 vs 0.08), and the magnitude of the correlation coefficient (0.562 vs 0.126) all suggest a significantly stronger relationship between precipitation and soil moisture in Texas relative to the unrealistic processes involved with the New York perturbations. Further, the unrealistic New York perturbations are unable to elicit a precipitation response larger than 0.6 mm, a value well exceeded by perturbations in Texas. Comparing ensemble statistics between a clearly unrealistic set of responses and a range that might be expected to be physically relevant can thus reveal realistic dynamical links between initial-condition variables and chosen forecast metrics. Here, average precipitation over a fixed location was chosen as the forecast metric for illustrative purposes, but in general, any forecast metric could be chosen to determine whether the application of ESA shows any significance over a benchmark ensemble that exhibits only the unrealistic behavior of chaos seeding.
Empirical orthogonal analysis.
Another method that can reveal the realistic effects of initial-condition perturbations in the presence of chaos seeding is empirical orthogonal function (EOF) analysis (Wilks 2011). Figure 10 shows a modeling domain with 12-km grid spacing used with EOF analysis to reveal realistic effects over those from chaos seeding. Two areas of 1-in. soil moisture perturbations were made—one in Nebraska and one in the northeastern United States (perturbation locations shown in green in Fig. 10). A control run initialized by the GFS, as well as the two perturbed runs, were executed over a 96-h simulation window. EOF analysis was then applied to the 3-h precipitation time series from all three simulations, and the sum of significant EOFs [those leading EOFs that describe the majority of the variability and can be distinguished from the remaining EOFs through Kaiser’s rule (Wilks 2011)] from the control run was differenced from that in each perturbed run (shown in color shading in Fig. 10).
Differences between the sum of significant EOFs for a control simulation and a simulation with soil moisture perturbed (in an area shown by the green box) in (top left) NE (realistic EOFs) and (top right) the northeastern United States (unrealistic EOFs). (bottom) The control 96-h accumulated precipitation.
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
The left panel of Fig. 10 clearly exhibits larger-scale EOF difference patterns, while the right panel is limited to small-scale features (these also appear in the left panel). In this way, the larger-scale features in the difference field in the left panel reveal modes of variability introduced to the precipitation field because of irrigation in Nebraska that do not occur because of irrigation in the northeastern United States. Since both perturbations are associated with rapid chaos seeding throughout the domain, the larger-scale EOF difference patterns suggest these changes in precipitation are a result of realistic processes stemming from moister soil in Nebraska. Comparing the EOF signal to the raw 96-h accumulated precipitation field (also shown in Fig. 10), the new and realistic variability added by irrigation in Nebraska clearly correlates to the areas of heavy precipitation over the simulation window. These dynamical insights are simply not possible by examining the single difference field between a control and perturbed run such as that created by the soil moisture differences made in Nebraska—any differences could be merely a result of chaos seeding, which cannot be ruled out without comparison against benchmark simulations that are certain to contain these unrealistic effects.
The use of double precision.
The use of double-precision variables in WRF is another possible mitigation technique to the chaos-seeding problem. This is because Table 1, which reflects single-precision WRF integrations, suggests round-off error is the cause of the noise that is spread rapidly throughout the model domain. If double-precision WRF simulations were used, it may be the case that substantially smaller magnitudes of noise would propagate throughout the domain, forcing perturbation growth to be achieved over many more orders of magnitude to become significant. To first test whether chaos seeding produces substantially smaller magnitudes at double precision, WRF simulations were performed in the same manner as that which produced the magnitudes of noise in Table 1 but were done at double precision. Table 2 shows that indeed the perturbation noise that spreads rapidly throughout the domain is orders of magnitude smaller than that at single precision. This suggests that the subsequent growth of unrealistic noise may be limited if using a double-precision configuration.
Figure 11 shows the differences in 6-h precipitation at both 30- and 36-h forecast times for both WRF single- and double-precision simulations on the 12-km domain shown in Fig. 2 (both using the same perturbation to soil moisture in western Nevada). As expected, the WRF single-precision runs show perturbations of larger magnitude, reaching over 200 mm. Maximum differences with double precision are about half as big, although RMS differences are nearly the same at 30 h and are about two-thirds the size of the single-precision value at 36 h. Spatially, the signals are similar as both experiments reveal nearly the same locations and spatial scale of the differences, although at 30-h forecast time, significant differences at single precision appear in North Carolina but are absent at double precision. These results indicate that while the effects of chaos seeding can be reduced somewhat with double precision, they cannot be eliminated and likely will still lead to problems of discerning real signals in perturbation experiments. Given the substantial computing cost of double-precision simulations, the use of double precision does not seem to offer a practical solution to chaos seeding. This dependence of the extent of the effects of chaos seeding on precision may also have implications for the use of imprecise processing to improve accuracy in atmospheric simulations, a technique suggested in Düben et al. (2013). Since chaos seeding leads to larger perturbations between simulations at lower precision as shown in Fig. 11, the potential for chaos seeding to affect otherwise realistic processes within simulations is also enhanced at lower precision. Whether the detrimental influences of chaos seeding at lower precision outweigh the benefits allowed through higher resolution or additional ensemble members as discussed in Düben et al. (2013) is unclear but should be examined if atmospheric modeling with less precise processing is used to perform the types of experiments described here.
Accumulated 6-h precipitation differences between the control simulation and the perturbed simulation (soil moisture perturbed in western NV) at (left) 30- and (right) 36-h simulation times for both (top) single- and (bottom) double-precision experiments.
Citation: Bulletin of the American Meteorological Society 99, 3; 10.1175/BAMS-D-17-0129.1
SUMMARY AND DISCUSSION.
Perturbation experiments are a common technique used to investigate how differing various aspects such as initial conditions, physics parameterizations, or boundary conditions can affect numerical model simulations. These experiments can yield important results that improve our fundamental understanding of atmospheric processes and predictability. We have discovered, however, that these types of experiments can be significantly contaminated by the seeding of chaos through the rapid propagation of numerical noise within a gridpoint model. This phenomenon begins with the rapid communication by spatial discretization schemes of a perturbation signal and its associated round-off error at speeds well over those of any known physical process. This quickly results in tiny differences throughout the model domain with respect to the entire atmospheric state, supplying the seeds for rapid chaotic growth. While areas dominated by dry dynamics and their slower perturbation growth rates appear to be relatively immune to these tiny perturbations, areas of precipitation and moist convection experience rapid chaotic growth, becoming significant on the order of hours or even minutes. The nonlinear nature of perturbation growth in these areas results in the smallest perturbations growing the fastest, and although begun from a tiny perturbation that was created in a completely unrealistic way, these perturbations (which then grow through real physical processes) quickly resemble any growing differences begun from perturbations that were physically possible.
It might be possible to dismiss rapid chaotic growth that occurs far enough from a perturbation source to be obviously unrealistic through chaos seeding by numerical noise propagation. However, examples were shown here that confirm the same unrealistic processes occur locally, making it very difficult to distinguish perturbation growth caused by chaos seeding from actual physical processes. From another perspective, without the knowledge of chaos seeding, one would have no reason to not interpret perturbation growth within model sensitivity experiments as a real physical process. This suggests a high likelihood that a number of conclusions based on perturbation experiments might be influenced, and contaminated, by chaos seeding and its subsequent unrealistic effects.
Here, the phenomenon of chaos seeding was discovered within the WRF gridpoint model, although other studies with different gridpoint modeling systems suggest the issue is more generalized since common numerical schemes are the cause. Spectral models likely suffer the same issue of chaos seeding given their inherent instantaneous communication of perturbations across the entire modeling domain. Thus, chaos seeding within perturbation experiments appears to be a universal modeling problem. In turn, our hope with this study is to bring awareness to this relatively unknown issue to the field of atmospheric sciences and other fields where chaos seeding may plague perturbation experiments, such that attempts can be made by researchers to remove potential misinterpretations from their work. From a predictability perspective, chaos seeding presents an intrinsic limit on the predictability of certain features since, even if nearly all sources of error can be removed in a numerical weather forecast, any tiny error in any limited part of the domain will rapidly seed the entire model grid with other tiny errors, which will subsequently evolve wherever the atmosphere supports rapid perturbation growth. Ensemble sensitivity and EOF analysis were two techniques presented here that have the potential to mitigate chaos seeding in perturbation experiments toward distinguishing realistic processes, and we hope that these and other new techniques can be used to ensure that chaos seeding does not harm the integrity of modeling experiments in a variety of scientific disciplines.
ACKNOWLEDGMENTS
The authors wish to thank Cliff Mass, Sharan Majumdar, Dan Holdaway, Steven Fletcher, and many others for engaging in deep conversations on the issue of chaos seeding that resulted in an improved study. Three reviewers including Dale Durran, Ron McTaggart-Cowan, and Gretchen Mullendore provided extremely helpful comments and suggestions. This work has been supported by National Science Foundation CAREER Grant AGS-1151627.
REFERENCES
Ancell, B. C., and G. J. Hakim, 2007: Comparing ensemble and adjoint sensitivity analysis with applications to observation targeting. Mon. Wea. Rev., 135, 4117–4134, https://doi.org/10.1175/2007MWR1904.1.
Bednarczyk, C. N., and B. C. Ancell, 2015: Ensemble sensitivity analysis applied to a southern plains convective event. Mon. Wea. Rev., 143, 230–249, https://doi.org/10.1175/MWR-D-13-00321.1.
Chen, F., and J. Dudhia, 2001: Coupling an advanced land surface–hydrology model with the Penn State–NCAR MM5 modeling system. Part I: Model description and implementation. Mon. Wea. Rev., 129, 569–585, https://doi.org/10.1175/1520-0493(2001)129<0569:CAALSH>2.0.CO;2.
Clark, A. J., and Coauthors, 2012: An overview of the 2010 Hazardous Weather Testbed Experimental Forecast Program Spring Experiment. Bull. Amer. Meteor. Soc., 93, 55–74, https://doi.org/10.1175/BAMS-D-11-00040.1.
Düben, P. D., H. McNamara, and T. N. Palmer, 2014: The use of imprecise processing to improve accuracy in weather & climate prediction. J. Comput. Phys., 271, 2–18.
Dudhia, J., 1989: Numerical study of convection observed during the Winter Monsoon Experiment using a mesoscale two-dimensional model. J. Atmos. Sci., 46, 3077–3107, https://doi.org/10.1175/1520-0469(1989)046<3077:NSOCOD>2.0.CO;2.
Durran, D. R., 2010: Numerical Methods for Fluid Dynamics with Applications to Geophysics. Springer, 516 pp.
Durran, D. R., and J. A. Weyn, 2016: Thunderstorms do not get butterflies. Bull. Amer. Meteor. Soc., 97, 237–243, https://doi.org/10.1175/BAMS-D-15-00070.1.
Fitch, A. C., J. B. Olson, J. K. Lundquist, J. Dudhia, A. K. Gupta, J. Michalakes, and I. Barstad, 2012: Local and mesoscale impacts of wind farms as parameterized in a mesoscale NWP model. Mon. Wea. Rev., 140, 3017–3038, https://doi.org/10.1175/MWR-D-11-00352.1.
Hakim, G. J., and R. D. Torn, 2008: Ensemble synoptic analysis. Synoptic–Dynamic Meteorology and Weather Analysis and Forecasting: A Tribute to Fred Sanders, Meteor. Monogr., No. 55, Amer. Meteor. Soc., 147–161.
Hill, J. A., C. C. Weiss, and B. C. Ancell, 2016: Ensemble sensitivity analysis for mesoscale forecasts of dryline convection initiation. Mon. Wea. Rev., 144, 4161–4182, https://doi.org/10.1175/MWR-D-15-0338.1.
Hodyss, D., and S. J. Majumdar, 2007: The contamination of ‘data impact’ in global models by rapidly growing mesoscale instabilities. Quart. J. Roy. Meteor. Soc., 133, 1865–1875, https://doi.org/10.1002/qj.157.
Hohenegger, C., and C. Schar, 2007: Atmospheric predictability at synoptic versus cloud-resolving scales. Bull. Amer. Meteor. Soc., 88, 1783–1793, https://doi.org/10.1175/BAMS-88-11-1783.
Hu, X.-M., P. M. Klein, and M. Xue, 2013: Evaluation of the updated YSU planetary boundary layer scheme within WRF for wind resource and air quality assessments. J. Geophys. Res. Atmos., 118, 10 490–10 505, https://doi.org/10.1002/jgrd.50823.
Kain, J. S., and J. M. Fritsch, 1990: A one-dimensional entraining/detraining plume model and its application in convective parameterization. J. Atmos. Sci., 47, 2784–2802, https://doi.org/10.1175/1520-0469(1990)047<2784:AODEPM>2.0.CO;2.
Kain, J. S., and J. M. Fritsch, 1993: Convective parameterization for mesoscale models: The Kain–Fritsch scheme. The Representation of Cumulus Convection in Numerical Models, Meteor. Monogr., No. 46, Amer. Meteor. Soc., 165–170.
Klemp, J. B., J. Dudhia, and A. D. Hassiotis, 2008: An upper gravity-wave absorbing layer for NWP applications. Mon. Wea. Rev., 136, 3987–4004, https://doi.org/10.1175/2008MWR2596.1.
Knievel, J. C., G. H. Bryan, and J. P. Hacker, 2007: Explicit numerical diffusion in the WRF Model. Mon. Wea. Rev., 135, 3808–3824, https://doi.org/10.1175/2007MWR2100.1.
Langland, R. H., 2005: Issues in targeted observing. Quart. J. Roy. Meteor. Soc., 131, 3409–3425, https://doi.org/10.1256/qj.05.130.
Langland, R. H., and N. L. Baker, 2004: Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus, 56A, 189–201, https://doi.org/10.3402/tellusa.v56i3.14413.
Leoncini, G., R. S. Plant, S. L. Gray, and P. A. Clark, 2010: Perturbation growth at the convective scale for CSIP IOP18. Quart. J. Roy. Meteor. Soc., 136, 653–670, https://doi.org/10.1002/qj.587.
Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130–141, https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.
Luo, Y., and L. Zhang, 2011: A case study of the error growth and predictability of a meiyu frontal heavy precipitation event. Acta Meteor. Sin., 25, 430–440, https://doi.org/10.1007/s13351-011-0404-1.
Majumdar, S. J., 2016: A review of targeted observations. Bull. Amer. Meteor. Soc., 97, 2287–2303, https://doi.org/10.1175/BAMS-D-14-00259.1.
Melhauser, C., and F. Zhang, 2012: Practical and intrinsic predictability of severe and convective weather at the mesoscales. J. Atmos. Sci., 69, 3350–3371, https://doi.org/10.1175/JAS-D-11-0315.1.
Mlawer, E. J., S. J. Taubman, P. D. Brown, M. J. Iacono, and S. A. Clough, 1997: Radiative transfer for inhomogeneous atmosphere: RRTM, a validated correlated-k model for the longwave. J. Geophys. Res., 102, 16 663–16 682, https://doi.org/10.1029/97JD00237.
Peckham, S. E., T. G. Smirnova, S. G. Benjamin, J. M. Brown, and J. S. Kenyon, 2016: Implementation of a digital filter initialization in the WRF Model and its application in the Rapid Refresh. Mon. Wea. Rev., 144, 99–106, https://doi.org/10.1175/MWR-D-15-0219.1.
Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-475+STR, 113 pp., https://doi.org/10.5065/D68S4MVH.
Thompson, G., R. M. Rasmussen, and K. Manning, 2004: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part I: Description and sensitivity analysis. Mon. Wea. Rev., 132, 519–542, https://doi.org/10.1175/1520-0493(2004)132<0519:EFOWPU>2.0.CO;2.
Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Elsevier, 676 pp.
Zhang, F., C. Snyder, and R. Rotunno, 2003: Effects of moist convection on mesoscale predictability. J. Atmos. Sci., 60, 1173–1185, https://doi.org/10.1175/1520-0469(2003)060<1173:EOMCOM>2.0.CO;2.
