1. Introduction

Physical and chemical processes happening in Earth’s atmosphere span many orders of magnitude in terms of their spatial and temporal scales, which presents great challenges to numerical modeling. For example, in general circulation models, motions or phenomena that cannot be resolved in space or time but have significant impact on the large-scale flow motions need to be accounted for using parameterizations (see, e.g., McFarlane 2011).

In recent years, stochastic parameterizations have become an active area of research [see review by Berner et al. (2017) and Leutbecher et al. (2017)]. The fundamental principle behind the stochastic formulation is that the state of the unresolved processes at any instant is not entirely determined by the state of the resolved processes. Thus, an element of randomness needs to be introduced to account for this indeterminacy. This randomness can act as a source of roughness in the temporal evolution of the governing equations’ right-hand-side terms as well as in the evolution of the solution. Deterministic time integration schemes used in numerical weather prediction and climate projection models, however, typically assume temporal smoothness of the underlying solutions. When such schemes are applied naively to stochastic parameterizations, the conditions for solution convergence might no longer be satisfied.

There is a large body of work dedicated to the development of stochastic numerical schemes and stochastic versions of deterministic numerical schemes, some especially targeted for weather and climate applications (see e.g., Rüemelin 1982; Sura and Penland 2002; Sardeshmukh et al. 2003; Ewald et al. 2004; Ewald and Temam 2005; Hansen and Penland 2006). The purpose of our work is more modest. We want to study the use of a correction term that can help improve the solution accuracy of deterministic schemes when part of the variables of the model is replaced by noise and in particular colored noise.

As shown in Hodyss et al. (2013), the convergence issue of deterministic numerical schemes when applied to stochastic parameterizations can be investigated through the use of tools from stochastic analysis [see e.g., section 3.3 in Øksendal (2003) and section 4.9 in Kloeden and Platen (1992)]. In particular, for the cases when an unresolved process is replaced by a rough random process (e.g., white noise), it is not difficult to construct examples for which popular deterministic numerical schemes (e.g., Euler forward and backward, Adams–Bashforth) will no longer converge to the physically relevant solution except for special cases [e.g., the second-order Runge–Kutta scheme analyzed by Hodyss et al. (2013)]. Multiple examples relevant for atmospheric modeling can be found in Fig. 3 in the paper of Hodyss et al. (2013). Here, “physically relevant solution” refers to the one corresponding to ordinary calculus (see discussion below). In the study by Hodyss et al. (2014), ensemble simulations of Hurricane Isaac in the year 2012 were conducted using the Navy Operational Global Atmospheric Prediction System; it was shown that the choice of numerical scheme for the stochastic term can lead to failure in predicting the correct ensemble mean of hurricane intensity.

The mathematical reason for the lack of convergence is that when we replace an unresolved process with white noise, the equations describing the phenomena under investigation make sense only in integral form (not in the usual differential form). The integral form of the equations contains a temporal integral of an expression involving the white noise process. If we try to estimate such an integral through a limiting process involving progressively refined subintervals, different answers will be obtained depending on the manner we choose to discretize the interval of integration (see e.g., section 4.9 in Kloeden and Platen 1992). The two most-studied discretization methods in stochastic analysis are: (i) using the left endpoint of each subinterval (which leads to the Itô integral or Itô interpretation) and (ii) using the middle point of each subinterval (which leads to the Stratonovich integral or Stratonovich interpretation). The Stratonovich interpretation leads to ordinary calculus while the Itô interpretation does not (see e.g., sections 3.9 and 4 in Øksendal 2003). Because the feasibility of weather forecasting and climate projection depends fundamentally on the predictable (deterministic) components of the weather and climate systems, one can argue that the Stratonovich calculus is more relevant for the primary targets of these applications. However, there are also certain aspects of weather and climate dynamics (e.g., the impact of individual cloud/aerosol particles on the evolution of cloud systems and the impact of tropical convection on the global circulation), which contain characteristics that can be usefully described by Itô calculus. The present study is targeted at the first type of applications, and hence aim at ensuring the discrete numerical solutions of the model equations are consistent with the Stratonovich interpretation. For a more general discussion about the Itô–Stratonovich dilemma, see for example, chapter 7 in Kloeden and Platen (1992), the review paper Mannella and McClintock (2012), and Moon and Wettlaufer (2014).

It is important to note that many popular time integration schemes designed for deterministic problems will converge to the Itô solution when applied to stochastic problems driven by white noise (Kloeden and Platen 1992). In other words, naively describing an unresolved process by white noise and solving the stochastic equation with a deterministic numerical scheme can lead to erroneous results even in the limit of infinite temporal resolution. Fortunately, the Itô and Stratonovich interpretations are related, and this relationship can help recover, at least to some extent, the convergence of deterministic numerical schemes to the physically relevant Stratonovich solution. The connection between the two interpretations comes in the form of a correction term called the Itô correction. When the Itô correction is added to the equation, the numerical solution under the Itô interpretation converges to the Stratonovich solution.

While the abovementioned Itô–Stratonovich correspondence is a basic concept in stochastic analysis, the widely known form of the Itô correction applies only to the case of white noise. A key feature of white noise is that it has zero autocorrelation (and hence no memory). Given the typical time step size of seconds to an hour in weather and climate models, some parameterized processes (e.g., turbulence and cumulus convection) can have characteristic time scales equivalent to multiple time steps. Therefore, colored noise, which has nonzero autocorrelation length, can provide a better description of such processes. In fact, the state of the art in accounting for model uncertainties of Earth systems points to the need of stochastic processes with spatiotemporal correlations (see e.g., sections 3 and 5 in Leutbecher et al. 2017), which makes our construction more relevant for applications.

A fundamental difference between colored noise and white noise is that colored noise is in principle resolvable while white noise is not. In other words, if one could use small enough step sizes, there would be no distinction between the Itô and Stratonovich interpretations for the case of colored noise. All deterministic numerical schemes will eventually converge to the Stratonovich solution. But even in simple examples, let alone the very complex and expensive systems encountered in weather and climate prediction, the critical time step that recovers convergence to the Stratonovich solution can be prohibitively small. As a result, for realistically affordable time step sizes, the dichotomy between Itô and Stratonovich interpretations practically exists and needs to be addressed also for the cases of colored noise. In other words, the long time steps in practical applications motivate us to find time-stepping methods with higher accuracy.

Another point worth mentioning is that, as Hodyss et al. (2013) and Hodyss et al. (2014) have pointed out, certain deterministic numerical schemes (e.g., the second-order Runge–Kutta scheme) have (an unaveraged version of) the Itô correction already “built-in” and hence perform better for stochastic problems. Such schemes are typically multistage schemes that require multiple evaluations of the right-hand side of the governing equations, making them very expensive for weather and climate models. The Itô correction, in contrast, allows for the use of single-stage schemes (e.g., the Euler forward scheme) to be complemented by a correction constructed only for the stochastic term, and hence can be cost effective.

For these reasons, we present in this paper a generalization of the Itô correction that is valid for noises of any color. We use an advection-diffusion equation with constant or spatially varying advection velocity to demonstrate that, for both white and colored noises, the generalized Itô correction can accelerate convergence to the Stratonovich solution when added to the Euler forward scheme. We demonstrate that improved convergence means higher accuracy for the same step size or, alternatively, larger step size (and, hence, lower computational cost) for the same target accuracy. These results from the simple but relevant test problem provide a proof of concept, which motivates further exploration of the generalized Itô correction for the purpose of helping improve the solution accuracy and efficiency in atmospheric models of various complexity, including general circulation models using stochastic parameterizations.

The remainder of the paper is organized as follows: section 2 presents the derivation of the generalized Itô correction. Section 3 contains a presentation of the test problem, the advection–diffusion equation with constant and spatially varying advection velocity, along with analytical results (supplemental details can be found in the appendix). Section 4 contains numerical results. Finally, section 5 contains a discussion of our results as well as suggestions for future work.

2. The generalized Itô correction

We focus on how to numerically solve Eq. (4) after its form has been derived; how to construct a good P_s(u) to approximate the original P(u) is a separate topic that is outside the scope of the current work.

3. Test problem

Before we proceed, we want to make an important remark about the formula for I. For the case of white noise (α = 0), we obtain ${\lim }_{N_f\to \infty }\left(1/N_f\right)\left\{\left[C{\left({\omega }_0\right)}^2/2\right]+{\displaystyle {\sum }_{m=1}^{N_f}C{\left({\omega }_m\right)}^2}\right\}=1,$ and we recover the usual Itô correction expression. However, for the case of colored noise (α ≠ 0) with exponentially decaying spectrum, we have ${\lim }_{N_f\to \infty }\left(1/N_f\right)\left\{\left[C{\left({\omega }_0\right)}^2/2\right]+{\displaystyle {\sum }_{m=1}^{N_f}C{\left({\omega }_m\right)}^2}\right\}=0.$ This result is not surprising. As we have explained also in the introduction, in the case of colored noise, the distinction between the Itô and Stratonovich interpretations disappears in the limit of infinite temporal resolution. The reason is that in the limit of infinite temporal resolution a nonwhite colored noise is resolved and thus all discretizations of the integral of the stochastic term give the same answer. However, any numerical experiment that one conducts always has finite temporal resolution. In this case, the generalized Itô correction for the case of colored noise is no longer zero. Moreover, as we show with our numerical results, it can play a significant role in restoring or accelerating convergence to the Stratonovich solution. This remark is particularly pertinent for weather and climate applications where due to computational limitations we are always forced to use larger timesteps than the shortest time scales present in the solution.

4. Numerical results

In this section we use numerical results to show how noise n(t) of different color (roughness in time) can affect the solution convergence. We also demonstrate how the inclusion of the generalized Itô correction can help in restoring and/or accelerating convergence. As explained in section 2c, the validity of our generalized Itô correction is not tied to any specific time-stepping method. For simplicity, we use in our numerical experiments the forward Euler scheme as an illustrating example.

b. Case with constant advection velocity (ε = 0)

For the case with ε = 0, the discretization of Eq. (26) using forward Euler with the Itô correction included is given by

${\displaystyle \frac{{\widehat{F}}_k\left(t_{j+1}\right)-{\widehat{F}}_k\left(t_j\right)}{\mathrm{\Delta }t}}=-ikc{\widehat{F}}_k\left(t_j\right)-\mu k^2{\widehat{F}}_k\left(t_j\right)+ik\rho {\widehat{F}}_k\left(t_j\right)n\left(t_j\right)+I_k\left(t_j\right),$(35)

where n(t_j) is the colored noise at t = t_j and I_k(t_j) is the Itô correction for F_k at t_j:

$I_k\left(t_j\right)=-{\displaystyle \frac{1}{2}}\hspace{0.17em}{\rho }^2{k}^2{\widehat{F}}_k\left(t_j\right){\displaystyle \frac{1}{N_f}}\left[{\displaystyle \frac{C{\left({\omega }_0\right)}^2}{2}}+{\displaystyle \sum _{m=1}^{N_f}C{\left({\omega }_m\right)}^2}\right].$(36)

We note that although the expression for the Itô correction in Eq. (23) involves a limiting process, the limit is not present in the expression in Eq. (36) because we have discretized the equation and thus have picked a finite time step.

Figure 1a [which appears also in Hodyss et al. (2013)] shows the effect of different noises on the convergence of the Euler scheme without the Itô correction. The thick dots are the l₂ error of the numerical solution averaged over 100 realizations of the noise process; the error bars denote the standard deviation around the average. Using the terminology of stochastic analysis, this plot (and the rest of them in the paper) shows the strong convergence of the numerical solution.¹

View Full Size

Error in the numerical solution of the 1D advection–diffusion equation with constant advection velocity [ε = 0 in Eq. (20)] and the dependency on time step size (x axis) and characteristics of the noise term (α = 0, 10⁻⁶, 10⁻⁵, 10⁻⁴, or 1, shown in different colors). Results obtained using the forward Euler scheme (left) without and (right) with the generalized Itô correction, respectively, are shown. Simulations were performed for 100 realizations of the noise process and the l₂ solution error was calculated separately for each realization using Eq. (34). The thick dots are the mean error of the 100 realizations; the vertical bars denote the standard deviation around the mean. The two dashed lines are reference lines indicating convergence rates of 0.5 (upper) and 1.0 (lower), respectively.

Citation: Monthly Weather Review 148, 6; 10.1175/MWR-D-19-0178.1

• Download Figure
• Download figure as PowerPoint slide

We make two observations. First, for the case of white noise (α = 0, purple line), the Euler scheme without the Itô correction fails to converge to the analytical solution no matter how small the step size is [it converges to the Itô solution, cf. section 10.2 in Kloeden and Platen (1992)]. Second, for the case of colored noise (α ≠ 0, blue, green, orange, and red lines), the Euler scheme without Itô correction will start converging to the analytical solution with order 1 [as predicted by deterministic numerical analysis, see e.g., chapter I.7 in Hairer et al. (1992)] when the step size becomes smaller than some critical step size that depends on the color of the noise (value of α). The more red the noise is (larger α), the larger is the critical step size [see also Hodyss et al. (2013) for a discussion and estimation of the critical stepsize].

Figure 1b shows the effect of including the Itô correction. We want to make again two observations. First, for the case of white noise (α = 0, purple line), the Euler scheme with the Itô correction does converge to the analytical solution with order 1/2 (see Hu et al. (2016) for an explanation of this convergence rate). We note that this numerical result was mentioned in Hodyss et al. (2013) although not illustrated by any graphic there. Second, for the case of colored noise (α ≠ 0, blue, green, orange and red lines), the Euler scheme with the generalized Itô correction starts converging to the analytical solution with order 1 for larger step sizes than the Euler scheme without the Itô correction. Thus, the addition of the Itô correction can help restore and/or accelerate convergence of the forward Euler scheme.

c. Case with spatially varying advection velocity (ε ≠ 0)

We continue with the case of a spatially dependent advection velocity with ε = 10⁻³ A small value was chosen for ε because the forward Euler scheme is explicit and only first order. As such, it needs a very large number of steps in order to reach the asymptotic convergence regime for larger values of ε due to the need to resolve steepening gradients associated with the oscillatory nature of the spatial perturbation of the advection velocity. Moreover, the cost of evaluation of the noise n(t), which depends quadratically on the number of time steps, becomes very large when ε is large. For practical purposes (computational cost), we chose a small ε for the demonstration here.

We discretized Eq. (32) using the Euler scheme with the generalized Itô correction:

${\displaystyle \frac{\widehat{\mathbf{F}}\left(t_{j+1}\right)-\widehat{\mathbf{F}}\left(t_j\right)}{\mathrm{\Delta }t}}=\mathsf{D}\widehat{\mathbf{F}}\left(t_j\right)+\rho n\left(t_j\right)\mathsf{H}\widehat{\mathbf{F}}\left(t_j\right)+\mathbf{I}\left(t_j\right),$(37)

where the Itô correction reads

$\mathbf{I}\left(t_j\right)={\displaystyle \frac{{\rho }^2}{2}}{\displaystyle \frac{1}{N_f}}\left[{\displaystyle \frac{C{\left({\omega }_0\right)}^2}{2}}+{\displaystyle \sum _{m=1}^{N_f}C{\left({\omega }_m\right)}^2}\right]\mathsf{G}\widehat{\mathbf{F}}\left(t_j\right),$(38)

with the matrix $\mathsf{G}$ being

$\mathsf{G}=\text{Diag}\{-{\left(-N_x\right)}^2,-{\left(-N_x+1\right)}^2,\dots ,-{\left(N_x-1\right)}^2,-N_x^2\}.$(39)

The truncation wavenumber N_x was chosen empirically: a test simulation was conducted using Eq. (37) with a large N_x; an inspection of the magnitude of the resulting ${\widehat{F}}_k$ revealed N_x = 5 was sufficient to retain all modes with $\left|{\widehat{F}}_k\right|>{10}^{-4}$. Hence, N_x = 5 was then used to obtain the results shown in Fig. 2.

View Full Size

As in Fig. 1, but for the case of ε = 10⁻³ in Eq. (20).

Citation: Monthly Weather Review 148, 6; 10.1175/MWR-D-19-0178.1

• Download Figure
• Download figure as PowerPoint slide

Figure 2a shows that for the case of white noise (α = 0, purple line), the forward Euler scheme without the Itô correction fails to converge to the analytical solution as expected. Figure 2b shows how the inclusion of the Itô correction can restore convergence with order 1/2. We note that the standard deviation bars around the mean appear larger than in the case with ε = 0 because of the logarithmic scale of the plot. This figure demonstrates that for the case of colored noise (α ≠ 0), the use of the generalized Itô correction again accelerates the establishment of the order 1 convergence regime predicted by deterministic numerical analysis.

5. Conclusions

Stochastic parameterizations are increasing in popularity in numerical weather prediction and climate modeling as a way to improve the statistical representation of the studied phenomena. Naive implementation of such parameterizations with deterministic numerical time integration schemes can cause serious convergence issues. Such issues can be alleviated by the addition of certain correction terms (known as the Itô correction in stochastic analysis) to the deterministic numerical schemes. However, the Itô correction was originally derived only for the special case when the stochastic process is represented by white noise. For numerical weather prediction and climate modeling it will be useful to have the option to properly handle colored noise.

We have derived a generalized Itô correction for the case of colored noise and applied it to a test problem of an advection-diffusion equation driven by noise of different colors. Our results indicate that the generalized Ito correction can substantially reduce the time discretization error, significantly improve the convergence rate of the numerical solutions and therefor allow for the use of significantly larger step sizes.

While our derivation started from a stochastic differential equation, the fact that colored noise is in principle resolvable by sufficiently small step sizes implies that the generalized Itô correction can also be useful for deterministic problems for the purpose of improving solution convergence, accuracy, and efficiency. Compared to higher-order schemes like the Runge–Kutta family, the Itô correction is less costly in terms of computing time; compared to implicit methods that may provide better stability, the Itô correction is less intrusive in terms of the code modification it requires.

In the future, we plan to apply the current framework to more realistic atmospheric modeling problems (e.g., simplified versions of the atmospheric general circulation models or their parameterizations). We acknowledge the fact that the parameterizations will likely not be given directly in the multiplicative form required by our formulation. However, there is hope that new approaches [e.g., training a neural network to represent the function g(u) in Eq. (2)] will allow us to still use our construction for more involved parameterizations.

The generalized Itô correction for the case of colored noise still assumes that the subgrid phenomena evolve on a significantly shorter time scale than the phenomena we resolve explicitly. However, as was mentioned in section 2b, the Itô correction can be interpreted as a memory term in model reduction formalisms. This opens the possibility of exploiting more sophisticated types of memory terms that correspond to more nuanced and realistic noise processes.