Quantifying the Impact of Vertical Resolution on the Representation of Marine Boundary Layer Physics for Global-Scale Models

Mark A. Smalley aJoint Institute for Regional Earth System Science and Engineering, University of California, Los Angeles, Los Angeles, California
bJet Propulsion Laboratory, California Institute of Technology, Pasadena, California

Search for other papers by Mark A. Smalley in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-0731-6759
,
Matthew D. Lebsock bJet Propulsion Laboratory, California Institute of Technology, Pasadena, California

Search for other papers by Matthew D. Lebsock in
Current site
Google Scholar
PubMed
Close
, and
Joao Teixeira aJoint Institute for Regional Earth System Science and Engineering, University of California, Los Angeles, Los Angeles, California
bJet Propulsion Laboratory, California Institute of Technology, Pasadena, California

Search for other papers by Joao Teixeira in
Current site
Google Scholar
PubMed
Close
Open access

Abstract

While GCM horizontal resolution has received the majority of scale improvements in recent years, ample evidence suggests that a model’s vertical resolution exerts a strong control on its ability to accurately simulate the physics of the marine boundary layer. Here we show that, regardless of parameter tuning, the ability of a single-column model (SCM) to simulate the subtropical marine boundary layer improves when its vertical resolution is improved. We introduce a novel objective tuning technique to optimize the parameters of an SCM against profiles of temperature and moisture and their turbulent fluxes, horizontal winds, cloud water, and rainwater from large-eddy simulations (LES). We use this method to identify optimal parameters for simulating marine stratocumulus and shallow cumulus. The novel tuning method utilizes an objective performance metric that accounts for the uncertainty in the LES output, including the covariability between model variables. Optimization is performed independently for different vertical grid spacings and value of time step, ranging from coarse scales often used in current global models (120 m, 180 s) to fine scales often used in parameterization development and large-eddy simulations (10 m, 15 s). Uncertainty-weighted disagreement between the SCM and LES decreases by a factor of ∼5 when vertical grid spacing is improved from 120 to 10 m, with time step reductions being of secondary importance. Model performance is shown to converge at a vertical grid spacing of 20 m, with further refinements to 10 m leading to little further improvement.

Significance Statement

In successive generations of computer models that simulate Earth’s atmosphere, improvements have been mainly accomplished by reducing the horizontal sizes of discretized grid boxes, while the vertical grid spacing has seen comparatively lesser refinements. Here we advocate for additional attention to be paid to the number of vertical layers in these models, especially in the model layers closest to Earth’s surface where climatologically important marine stratocumulus and shallow cumulus clouds reside. Our experiments show that the ability of a one-dimensional model to represent physical processes important to these clouds is strongly dependent on the model’s vertical grid spacing.

© 2023 American Meteorological Society. This published article is licensed under the terms of a Creative Commons Attribution 4.0 International (CC BY 4.0) License .

Corresponding author: Mark A. Smalley, mark.a.smalley@jpl.nasa.gov

Abstract

While GCM horizontal resolution has received the majority of scale improvements in recent years, ample evidence suggests that a model’s vertical resolution exerts a strong control on its ability to accurately simulate the physics of the marine boundary layer. Here we show that, regardless of parameter tuning, the ability of a single-column model (SCM) to simulate the subtropical marine boundary layer improves when its vertical resolution is improved. We introduce a novel objective tuning technique to optimize the parameters of an SCM against profiles of temperature and moisture and their turbulent fluxes, horizontal winds, cloud water, and rainwater from large-eddy simulations (LES). We use this method to identify optimal parameters for simulating marine stratocumulus and shallow cumulus. The novel tuning method utilizes an objective performance metric that accounts for the uncertainty in the LES output, including the covariability between model variables. Optimization is performed independently for different vertical grid spacings and value of time step, ranging from coarse scales often used in current global models (120 m, 180 s) to fine scales often used in parameterization development and large-eddy simulations (10 m, 15 s). Uncertainty-weighted disagreement between the SCM and LES decreases by a factor of ∼5 when vertical grid spacing is improved from 120 to 10 m, with time step reductions being of secondary importance. Model performance is shown to converge at a vertical grid spacing of 20 m, with further refinements to 10 m leading to little further improvement.

Significance Statement

In successive generations of computer models that simulate Earth’s atmosphere, improvements have been mainly accomplished by reducing the horizontal sizes of discretized grid boxes, while the vertical grid spacing has seen comparatively lesser refinements. Here we advocate for additional attention to be paid to the number of vertical layers in these models, especially in the model layers closest to Earth’s surface where climatologically important marine stratocumulus and shallow cumulus clouds reside. Our experiments show that the ability of a one-dimensional model to represent physical processes important to these clouds is strongly dependent on the model’s vertical grid spacing.

© 2023 American Meteorological Society. This published article is licensed under the terms of a Creative Commons Attribution 4.0 International (CC BY 4.0) License .

Corresponding author: Mark A. Smalley, mark.a.smalley@jpl.nasa.gov

1. Introduction

Shallow marine clouds have important feedbacks in the climate system. Their reflective nature sends much of the downwelling solar radiation back to space, reducing the amount of solar energy imparted into the Earth system (Ockert-Bell and Hartmann 1992). The character of the low cloud regime has a significant influence on their cooling effect with stratocumulus providing significantly more cooling than cumulus regimes (L’Ecuyer et al. 2019). Models have a long-standing tendency to underestimate subtropical cloud cover while simultaneously overestimating cloud brightness (Nam et al. 2012). Simulating the transitions between subtropical cloud regimes is a well-known challenge for global models (Teixeira et al. 2011). Resolving marine boundary layers is challenged by the presence of frequent thin cloud layers capped by thin inversion layers. The modeling community has known for decades that improved simulation of subtropical marine boundary layer clouds requires fine vertical resolution, manifested in the model by reducing the vertical grid spacing Δz (Bushell and Martin 1999; Teixeira 1999).

A key limitation when designing and operating a global model is the considerable computational expense incurred by long-term climate simulations, requiring difficult compromises between horizontal grid spacing, vertical grid spacing, and the complexity of other model physical processes. Of course, the number of grid points strongly regulates this computational expense, with increases in computational cost expected to scale approximately linearly with the total number of model grid points. Because there are two horizontal dimensions, the number of horizontal grid points scales with the square of the horizontal resolution, which is often reported as a scalar (e.g., “0.5° simulations”).

Historically, the vertical grid spacing of global models has received less attention than horizontal grid spacing. Figure 1 shows the number of grid points and vertical layers in models contributing to long-term climate simulations in each of the six Intergovernmental Panel on Climate Change (IPCC) reports (Cubasch and Cess 1990; Gates et al. 1995; McAvaney et al. 2001; Randall et al. 2007; Flato et al. 2013; Gutiérrez et al. 2021). In the first IPCC report, no models had more than 20 layers. Eventually models began adding vertical layers, but in the recent decade many of these layers were added to the stratospheric upper portions of the domains instead of adding layers to the boundary layer. Models with layers above 60 km are designated in Fig. 1b by gray lines. The black distributions show that, for most models contributing to the standard set of IPCC experiments, the median number of layers has increased by a factor of 3.5 (from 9 to 32) over the last 30 years, whereas the median number of horizontal grid points has increased by a factor of 24 (from ∼1700 to ∼40 800). Furthermore, many of the newer models have domain tops higher than 60 km, demonstrating the more recent focus on resolving processes near the tropopause and in the stratosphere, which is the cause of a significant amount of the increase in the number of vertical layers.

Fig. 1.
Fig. 1.

Evolution of two of the primary model properties that contribute to the computational cost of global models: (a) the number of horizontal grid points and (b) the number of vertical layers in global models that contributed to the six IPCC reports. Later IPCC experiments included more detailed formulations of Earth’s atmosphere above the troposphere, so a distinction is made for models with tops higher than 60 km, as many of the layers in those models are above the troposphere and, therefore, irrelevant to boundary layer processes examined in this work. Circles indicate the median and the line ends indicate the 25th to 75th percentiles. The horizontal grid spacing of spectral models is estimated using the “L2” relation found provided by Laprise (1992), who explains that the values may be an underestimate of the true grid spacing of spectral models. As time passed, more models converted to fixed latitude–longitude grids so the fractional increase in horizontal grid points visible in (a) can be interpreted as a lower bound on the relative improvements to horizontal grid spacing in global models.

Citation: Monthly Weather Review 151, 11; 10.1175/MWR-D-23-0078.1

This is not to say that model Δz has not received attention from the modeling community. Considerable improvements to global models have been recently reported in terms of the simulation of low clouds (Bogenschutz et al. 2021) and upper-tropospheric structure (Wang et al. 2019). To address the issue of Δz in an efficient and adaptive manner, Yamaguchi et al. (2017) introduced a method of refining the model’s vertical grid spacing named Framework for Improvement by Vertical Enhancement (FIVE), first in a single-column model (SCM) and more recently in the Energy Exascale Earth System Model (E3SM; Lee et al. 2021). The FIVE routine works by effectively placing layers at specific heights that would benefit most from decreased Δz, thus acting as an additional subgrid parameterization to more accurately resolve sharp gradients and small-scale processes in and between the boundary layer and free troposphere while reducing the extra computational burden of those extra layers. Lee et al. (2022) extended the FIVE framework to multiple stratocumulus basins, finding that refinements in model horizontal grid spacing from 1° to 0.25° add to the benefits of FIVE’s refinements to vertical grid spacing in the marine boundary layer. Bogenschutz et al. (2023) show that FIVE retains those improvements over the southeast Pacific Ocean while maintaining a computational cost advantage over control simulations with small Δz.

In this work we provide further evidence that reducing Δz in global models is a key pathway to more accurate simulations of the marine boundary layer by running a simplified SCM containing turbulence, cloud, and warm rain parameterizations for two well-known field campaign cases and comparing our model output to the results of multimodel large-eddy simulation (LES) ensembles. To do so fairly, we employ a novel method of tuning our model objectively for each value of Δz. Our results show that simply reducing Δz results in vastly improved simulations of the marine boundary layer thermodynamic structure, fluxes, clouds, and precipitation, often times irrespective of the tuning of the parameters. Results also show that a nearly fivefold reduction in disagreement from the LES benchmarks occurs when vertical grid spacing improves from values corresponding to current-day Earth System Models (ESMs; ∼120 m) to 10 m vertical grid spacing. A central takeaway is that the efficacy of tuning efforts for model parameters is fundamentally limited by coarsely spaced vertical layers.

This manuscript is organized as follows. Section 2 provides background information about the model, simulated cases, and validation framework. Section 3 introduces the objective parameter estimation used to tune the model separately at each Δz. Section 4 presents the performance of the model at each Δz. Finally, section 5 provides a discussion of results and outlook for modeling at coarse Δz.

2. Model and simulation cases

a. Jet Propulsion Laboratory’s eddy-diffusivity/mass-flux parameterization

The Jet Propulsion Laboratory’s eddy-diffusivity/mass-flux (JPL-EDMF; Sušelj et al. 2012, 2013, 2022) is a unified turbulence, cloud, and microphysics parameterization. Various versions have been published for different science goals including dry boundary layers (Witek et al. 2011), shallow nonprecipitating convection (Sušelj et al. 2012, 2021), and deep convection (Sušelj et al. 2019). The version utilized here was recently coupled to a new warm rain microphysics formulation in Sušelj et al. (2022) and solves prognostic equations for grid-scale values of liquid water potential temperature, total water mixing ratio, horizontal winds, turbulent kinetic energy (TKE), and the variance of excess saturation, which is used to diagnose cloud properties and rain formation. The JPL-EDMF represents subgrid processes by splitting the subgrid domain into a convective portion and a nonconvective portion of the grid box.

Convective turbulent transport is modeled with 20 steady-state plumes initialized at the surface layer from the right tail of an assumed normal vertical-velocity distribution. Plumes are modeled as noninteracting, horizontally uniform convective updrafts. Each plume experiences a number of stochastically determined entrainment events, which dilute its thermodynamic properties. A convective plume rises until its vertical velocity is no longer positive. A warm phase condensation and a single-moment cloud microphysical scheme is coupled to thermodynamic properties of the updrafts. An iterative method is used to obtain consistent structure of thermodynamic, cloud and rain properties.

Locally driven mixing is accomplished in the nonconvective environment by a downgradient eddy-diffusivity approach that is a function of prognostically solved turbulent kinetic energy (TKE) equation and a mixing length scale that follows Teixeira and Cheinet (2004). TKE dissipation follows Mellor and Yamada (1974). For this nonconvective environment, the parameterization assumes a joint normal probability distribution function between subgrid thermodynamic and kinematic state variables, which are needed to diagnose the nonconvective cloud and rain properties.

JPL-EDMF uses the same cloud and rain microphysical parameterization for both updrafts and nonconvective environments, solved consistently with assumptions of the uniform distribution of thermodynamic properties for the former and joint-normal distribution for the latter. Cloud water and cloud fraction are diagnosed assuming that saturation adjustment happens quickly so that any portion of the grid that has saturation excess greater than zero contains cloud water. Microphysical source and sink terms follow the formulations of Khairoutdinov and Kogan (2000) and include autoconversion of cloud to rain droplets, accretion of rain droplets by cloud and evaporation of rain droplets. The precipitation drop size distribution is prescribed and follows an exponential distribution with the intercept parameter described by a power-law dependance on the slope parameter as in Abel and Boutle (2012). A portion of the plume-generated rainwater is detrained from plumes into the nonconvective environment, where it may evaporate or fall to the surface, depending on conditions in the nonconvective environment. The portion of detrained water depends on the vertical wind shear experienced by the plumes.

The JPL-EDMF of course contains fixed parameters that could potentially have different optimal values for different Δz. In this manuscript we examine 18 of those parameters, which are explained in detail in Sušelj et al. (2022) and outlined in Table 1. Smalley et al. (2022) provides a detailed sensitivity study of the JPL-EDMF to those parameters but at a fixed Δz = 20 m, which the JPL-EDMF developers typically use during validation studies. Here we expand the parameters’ valid ranges in order to accommodate the different values of Δz. We note that the JPL-EDMF used here requires a fixed Δz throughout the vertical domain, while operational forecasting models and global circulation models have variable Δz that increase with height to reduce the computational expense in layers that usually lack sharp gradients.

Table 1.

JPL-EDMF constants to be investigated here.

Table 1.

b. Model and simulation cases

Here, the JPL-EDMF is implemented as an SCM, taking initial conditions and forcings from their corresponding values used in LES intercomparisons of two well-known field campaigns. The DYCOMS2rf2 case is based on the second research flight of the second Dynamics and Chemistry of Marine Stratocumulus (DYCOMS2rf2; Ackerman et al. 2009) and is included to represent subtropical marine stratocumulus with some drizzle reaching the surface. The DYCOMS2rf2 simulations are run for a total of 6 h and we select the 14 drizzling cases with no cloud droplet sedimentation to match the analogous processes in the JPL-EDMF. The second simulation case is based on conditions observed during the Rain in Cumulus over the Ocean (RICO; Rauber et al. 2007) campaign near Barbados and Antigua. The RICO case represents the more convective end of the shallow boundary layer spectrum and includes a slowly deepening boundary layer that is simulated for 24 h. The LES intercomparison is introduced in van Zanten et al. (2011) and consists of 13 LES models with precipitation and no cloud droplet sedimentation. Together, these cases are intended to represent differing manifestations of the shallow marine boundary layer and associated clouds and precipitation, challenging the JPL-EDMF to produce the correct boundary layer structure across different regimes within a unified parameterization framework.

We define several Quantities of Interest (QI), which are vertical profiles of variables against which the SCM is tuned. The list of QIs is shown in Table 2. The QIs range from moist-conserved and prognostic variables (θl, qt, U, and V) to diagnostic cloud and precipitation properties (CF, ql, and qr), to fluxes of temperature and moisture [F(θl) and F(qt)]. To perform a fair comparison across the different values of Δz, QIs from the SCM and the LES are each averaged to 120 m, which is the coarsest Δz examined here. It is worth emphasizing here that this approach does not provide an advantage bias to the fine-Δz simulations simply for an ability to resolve fine-scale features of the QIs. Instead, the fine-Δz simulations must outperform the coarse-Δz simulations by producing a more accurate coarse-Δz profile of the QIs.

Table 2.

List of model output Quantities of Interest (QI), their symbols, and their multiplicative factors applied to the model output to improve numerical stability.

Table 2.

c. Performance metric χ2

To objectively tune the parameters of the SCM, we define a single scalar metric that describes the overall agreement of the SCM with the LES benchmarks. In so doing, we take into account that there is uncertainty in the LES QIs and that there is significant covariance between both the model layers and the individual QIs. We employ χ2 as a metric for the performance of the SCM simulations compared to the LES intercomparison benchmarks. χ2 can be interpreted as the sum of the uncertainty-weighted deviations from the LES benchmarks and is defined in Eq. (1):
χi2=[YiFi(x)]TSY,i1[YiFi(x)].
Here Y and F are column vectors of Quantities of Interest (QIs) from the LES intercomparisons and SCM, respectively; x is a vector of SCM parameters; and SY is the covariance matrix of LES intercomparison QIs. The subscript (i) refers to either the DYCOMS2rf2 case (i = 1) or RICO case (i = 2). Here we take Yi to be the mean of each QI across the set of intercomparison LES simulations. The vector Yi is composed of the vertical profiles of the nine QIs listed in Table 2. For example, Y1 is constructed by stacking the LES intercomparison DYCOMS2rf2 QI profiles into a single column vector (profile of θl then the profile of qt, etc.). Vector F2 is constructed in the same manner but with the SCM output QIs for the RICO simulation. In this work we perform simulations at a number of different vertical grid spacings. To make χ2 comparable across simulations that utilize different Δz, it is evaluated at the lowest vertical grid spacing (Δz = 120 m) for all simulations. For DYCOMS2rf2 this results in 11 layers and vector of length 99, whereas for RICO this results in 25 layers and a vector of length 225.

An important element of this work is the definition of the covariance matrix SY. To do so we take advantage of the variance in LES output across the intercomparison of LES simulations. SY,i is constructed by computing the covariance of the LES model outputs, which have been averaged to the Δz = 120 m layers. LES QIs often covary strongly between adjacent layers and with other QIs, necessitating these off-diagonal elements in the calculation of χi2. Equation (1) requires an inversion of SY,i, which can lead to numerical instability when the columns of SY are not linearly independent. This numerical instability can be estimated via the “condition number for inversion.” To mitigate these issues by reducing the condition number for inversion of SY,i, we apply constant factors to each QI (Table 2) and add a value of 10−5 to each value along the diagonal of SY,i. The stabilized covariance matrices are presented in Figs. S1 and S2 in the online supplemental material, which illustrate the large covariances in some neighboring layers and between some QIs for both cases.

Because the RICO domain is deeper (thus requiring more 120 m layers) and we want the DYCOMS2rf2 simulations to count equally as the RICO simulation, we need to weight the results from each case based by the respective number of layers (N1 and N2). The resulting χ2 in Eq. (2) is the scalar that describes the performance of the model compared to LES benchmarks, accounting for the uncertainty in the LES intercomparisons:
χ2=(N2χ12+N1χ22)(N1+N2).

3. Objective parameter identification

The JPL-EDMF has previously been manually tuned as an SCM at Δz = 20 m (Sušelj et al. 2022). We would like to understand the performance of the model at different grid spacings (Δz = 10, 20, 40, 60, and 120 m), but simply running the 20-m-tuned model at different Δz does not result in a fair comparison so we first need to tune the model separately and objectively at each Δz examined in this work. From previous work (Smalley et al. 2022) we have identified 18 parameters that are normally held constant in the model but may have different ideal values for different values of Δz. Of course, we are unable to adequately sample the 18-dimensional parameter space to test each parameter combination so we must seek computationally efficient methods of objectively tuning the model at each Δz.

To objectively tune the SCM, we seek to minimize the χ2 in Eq. (2). As an initial attempt we implemented a least squares minimization using Gauss–Newton iteration, which assumes that the model can be locally approximated as linear. However, we found that this algorithm was frequently unable to find the correct χ2 minimum due to nonlinearities in the model which resulted in inefficient minimization and often unstable parameter values. Other candidate methods of objective parameter searches include adaptive Markov chain Monte Carlo (Järvinen et al. 2010), surrogate models (Bellprat et al. 2012; Langhans et al. 2019; Boukabara et al. 2021; Couvreux et al. 2021), and simulated annealing (Ingber 1989; Jackson et al. 2003; Yang et al. 2012).

To efficiently identify the optimal set of parameter values, we employ a novel objective method called Cycling Two At a Time (CTAT), in which parameters are optimized in pairs (Fig. 2). Before CTAT begins for a given Δz, χ2 is computed using the default parameter settings and that Δz. This χ2 serves as the starting threshold χ2 for the first parameter pair. Next, the 18 parameters introduced in Table 2 are randomly grouped into nine pairs. The first 2 parameters are then optimized, while the other 16 are held at their default values. The CTAT algorithm creates a linear-spaced 9 × 9 mesh of values for the two parameters, extending the full breadth of each parameter’s valid range (Table 2). Both DYCOMS2rf2 and RICO are then simulated and χ2 is computed for each element of the two-parameter mesh. If any of the 81 values of χ2 are less than the threshold χ2, the parameter values that resulted in that minimum χ2 are applied to the model and the threshold χ2 is updated to the lowest value. Otherwise, the previous parameter settings and χ2 are retained. This step is repeated for each remaining parameter pair. Once all nine parameter pairs have been completed (thus completing a single CTAT round), the process is repeated using the parameter settings found from the first round of CTAT but with randomly reordered parameter pairings for a total of four CTAT rounds. This work also shows that there is negligible improvement to the model simulations after the fourth round of CTAT iterations.

Fig. 2.
Fig. 2.

CTAT algorithm flowchart.

Citation: Monthly Weather Review 151, 11; 10.1175/MWR-D-23-0078.1

It is possible that, through a single round of CTAT, the algorithm may work itself into a path that does not allow it to reach the overall best solution. We therefore complete 25 separate replicates of the CTAT algorithm for each Δz. Implementing CTAT 25 times for each value of Δz is expected to produce reliable results describing the dependence of the model performance as a function of Δz.

In this work we describe the results of three separate experiments that utilize the CTAT methodology. Experiment 1 explores the behavior of the model as a function of Δz while simultaneously reducing the time step. For the vertical grid spacings Δz = 10, 20, 40, 60, and 120 m, the time steps are set to 15, 30, 60, 90, and 180 s, respectively. CTATs are performed for 4 rounds of two-parameter pairings. Experiment 2 holds the vertical grid spacing constant at Δz = 120 m and reduces the time step to the values implemented in Experiment 1 to demonstrate that the simulation improvements found in Experiment 1 are mostly due to reductions in Δz and not reductions in time step. Experiment 2 also employs four CTAT rounds. Experiment 3 extends the number of CTAT cycles from 4 to 20 rounds to show that the conclusions from Experiment 1 to Experiment 2 are not sensitive to the number of times each parameter is tuned in the tuning procedure.

4. Objective tuning results

The top row of Fig. 3 reveals the main results of Experiment 1, which alters both Δz and time step. The χ2 from each CTAT replicate is reduced as further parameter pairs are examined and that the final values of χ2 are strongly dependent on Δz. The values of χ2 tend to converge to a representative value for a given Δz before about nine iterations (when each parameter has been examined once), although there is still some spread in χ2, especially when Δz = 120 m. Moderate additional reductions are observed in the two to four CTAT cycles, which often come in discrete jumps. Figure 3b presents the distribution of χ2 at the end of all 25 replicates for each value of Δz. There is a clear dependence of the final χ2 on vertical grid spacing, with Δz = 10 and 20 m producing similar agreement with LES, while the Δz = 40, 60 m, and 120 m χ2 produce increasing disagreement with LES. Figure 3b shows that the SCM performance converges around Δz = 20 m, demonstrating that further refinements to Δz = 10 m do not produce meaningful improvements to the model performance in the DYCOMS2rf2 and RICO cases. Until the vertical grid spacing reaches 10 m, there is a tendency for the worst performing simulations for a given Δz to outperform nearly all of the simulations with coarser Δz. The best-performing Δz = 120 m CTAT replicate produces a parameter configuration that results in ∼4.5 times the χ2 as the best-performing Δz = 10 m CTAT replicate.

Fig. 3.
Fig. 3.

Evolution of χ2 for all 25 CTAT replicates for (a) Experiment 1, (c) Experiment 2, and (e) Experiment 3 and normalized histograms of the final values of χ2 at the end of (b) Experiment 1, (d) Experiment 2, and (f) Experiment 3.

Citation: Monthly Weather Review 151, 11; 10.1175/MWR-D-23-0078.1

The results of Experiment 1 clearly demonstrate the limitations of tuning a model with coarse Δz as a means to improve simulation fidelity. However, the changes to vertical grid spacing in Experiment 1 accompanied changes in time step so it is not immediately clear if those improvements were due to reductions in vertical grid spacing or time step. The middle row of Fig. 3 reveals the results of Experiment 2, in which the vertical grid spacing is set to Δz = 120 m and the time step is varied at the values used in Experiment 1. The coarse-Δz (120 m, 180 s; solid red line) results in Fig. 3b are repeated in Fig. 3d. Simulations with reduced time step tend to result in better agreement with the LES benchmarks, with the average χ2 at Δt = 180 s reducing from 6.0 × 106 to 4.6 × 106 when the time step is reduced to 15 s. Figure 3d and its comparison to Fig. 3b clearly demonstrate that reducing the model time step results in improvements to the model but those improvements are not large enough to overcome the deficiencies that result from its coarse vertical grid spacing of Δz = 120 m.

The bottom row of Fig. 3 reveals the results of Experiment 3, in which the CTAT optimization is extended from 4 to 20 rounds. The χ2 value improves very little after the fourth round of parameter combinations (Iteration > 36 in Fig. 3e) and the final χ2 values (Fig. 3f) are comparable to the Δz = 120 m distribution in Fig. 3b.

Figure 3 reveals that the model’s agreement with LES is strongly sensitive to Δz, time step plays a minor role in the improvements, and that the results are insensitive to the number of tuning rounds (when the number of rounds is greater than 4). Therefore, further analyses will focus on the results of Experiment 1 in order to explore how the tuned parameter values change as a function of Δz, how the best-performing simulations at each Δz result in different QI profiles, and how the efficacy of tuning is different as a function of Δz.

The final retrieved parameter values from each CTAT replicate are presented in Fig. 4. Some parameters tend to be optimized with consistent values for a given Δz (ατ and adiff) or even across different Δz (ϕ, bw, and adiss). In some cases, there is no firm agreement across CTAT replicates for a given Δz. This could indicate that the model is insensitive to that parameter relative to other parameters. In previous work determining the sensitivity of this model to its parameters at Δz = 20 m, Smalley et al. (2022) reported that the model is most sensitive to the values of ϕ and bw, both of which govern the plume lateral entrainment rates and vertical velocity, respectively. Both of these parameters demonstrate consensus values across different Δz in this work (Fig. 4). On the other hand, Smalley et al. (2022) find little sensitivity to the size distribution parameters (x1 and x2), which also exhibit little consensus in Fig. 4. These results reiterate the conclusion of Smalley et al. (2022) that the SCM results are most sensitive to the parameterization of the dynamics, especially the plume dynamics, as opposed to the representation of the microphysics.

Fig. 4.
Fig. 4.

Retrieved parameter values from CTAT, presented as histograms with nine evenly spaced bins spanning each parameter’s valid range from Table 2.

Citation: Monthly Weather Review 151, 11; 10.1175/MWR-D-23-0078.1

The χ2 is useful for this tuning exercise in that it aggregates the comparison of multiple height-resolved quantities into a single scalar metric. However, in doing so it obscures the details of the model performance, leaving the reasons for a particular simulation’s poor performance a mystery. Here we elucidate which QIs are driving the variability in χ2. Figures 58 provide visual assurance that the simulated DYCOMS2rf2 and RICO QI profiles at 20 m are, indeed, closer to the LES benchmark than the profiles at 120 m. In Figs. 5 and 7, the best simulation at each Δz is compared to the LES mean and ±1 standard deviation. The profiles are presented as weighted averages at the 120 m CTAT evaluation layers, so these are the exact data being compared to the LES benchmarks in computing χ2 within the CTAT algorithm (along with the RICO profiles revealed later). Figures 6 and 8 presents the QI profiles shown in Figs. 5 and 7 as the relative difference from the LES models according to Eq. (3), where the relative difference Dr is a function of F, Y, and the standard deviation of LES values σLES:
Dr=abs(FY)/σLES.
The Dr values may be interpreted as simplified contributions to the total χ2, where larger values tend to contribute more toward the total χ2 than smaller values. Dr is similar to the contribution of a single QI’s layer value to the total χ2, except the off-diagonal elements of SY are ignored in the calculation of Dr.
Fig. 5.
Fig. 5.

DYCOMS2rf2 QI profiles for the CTAT replicates that retrieved the lowest χ2. Gray-shaded regions exhibit the ±1 sigma range of the LES ensemble.

Citation: Monthly Weather Review 151, 11; 10.1175/MWR-D-23-0078.1

Fig. 6.
Fig. 6.

Relative difference Dr in SCM DYCOMS2rf2 QIs found using CTAT from the LES benchmarks.

Citation: Monthly Weather Review 151, 11; 10.1175/MWR-D-23-0078.1

Fig. 7.
Fig. 7.

As in Fig. 5, but for the RICO QI profiles.

Citation: Monthly Weather Review 151, 11; 10.1175/MWR-D-23-0078.1

Fig. 8.
Fig. 8.

As in Fig. 6, but for the relative difference Dr in RICO QIs.

Citation: Monthly Weather Review 151, 11; 10.1175/MWR-D-23-0078.1

For the DYCOMS2rf2 case (Fig. 5), the Δz = 120 m model does not produce the correct profile of temperature and moisture, placing the boundary layer top too high. There is too little cloudiness and the fluxes of temperature and moisture are too low and too high, respectively. These errors result in very large values of Dr above the boundary layer in layers where the LES models agree. In addition, the temperature inversion at the top of the boundary layer is not as sharp at Δz = 120 and 60 m (Fig. 5a), leading to a low-bias in cloud fraction (CF) and cloud water mixing ratio (ql) in layers where the LES benchmark contains cloud water. The LES models are very consistent in placing close to 100% cloudiness in these layers so the 120 m model is punished severely for its CF of only about 0.6 (Fig. 6c). The too-high boundary layer and dearth of CF at the top of the cloud layer in result in the greatest Dr of all QIs of both DYCOMS2rf2 and RICO cases (Fig. 6c). In particular, we note that the monotonic relationship between Δz and model performance can be seen layer-by-layer in many of the DYCOMS2rf2 QIs.

We note that the qr does not compare favorably at any value of Δz and that qr also showed disagreement from LES in previous work (Sušelj et al. 2022). It may be that the combination of power-law autoconversion and accretion similar to Khairoutdinov and Kogan (2000) coupled with the Abel and Boutle (2012) raindrop size distribution is unable to capture the LES vertical profile of qr. The fluxes F(θl) have a similar shape for all values of Δz, with disagreement from LES progressively increasing with coarser Δz. There are only slight changes in the winds as a function of Δz but we note that the model fails to reproduce the vertical structure of horizontal winds within the boundary layer at all of these values of Δz.

Figures 7 and 8 show SCM profiles of QI and Dr for the RICO case. Overall, the performance of the model at all grid spacings is generally better for the RICO case than the DYCOMS2rf2 case, likely due to the challenges of simulating the sharp changes in temperature and moisture at the top of the boundary layer in DYCOMS2rf2. The RICO temperature and moisture profiles for different Δz show a separation near the top of the cloud layer, between about 1800 and 2400 m. The small Δz simulations tend to be too cool and too moist, while the opposite is true for the Δz = 40 and 60 m. Curiously, the Δz = 120 m simulation matches the LES temperature and moisture profiles most closely but is still unable to match the condensed water variables (Figs. 8c–e). The Δz = 120 m model is able to produce close to the correct amount of cloud fraction near cloud base but does not produce enough detrained water near the cloud top to produce the stratiform bump at about 2000 m, possibly due to the too-shallow plume top heights (Fig. 8c). The Δz = 10 and 20 m results also struggle to produce the stratiform bump at 2000 m but more closely match the full profile of CF, ql, and qr. The temperature and moisture fluxes are clearly best-simulated by the Δz = 10 and 20 m simulations. The model struggles to simulate the U and V profiles regardless of Δz, reproducing the general shapes and magnitudes of the LES profiles but not the changes with height.

It is worth noting that Figs. 58 analyzed the performance of just a single (best performing) simulation at each Δz. We also want to emphasize that the model appears to be easier to tune at small Δz than at large Δz. Specifically, Figs. S3 and S4 show this by plotting the full range of QI profiles at the end of each CTAT replicate. Ranges are most often largest for the Δz = 120 m models, suggesting that model performance is more sensitive to parameter tuning at coarse Δz. Also evident from Figs. S3 and S4 is that the reasons for the Δz = 120 m models to err compared to the LES are not always consistent. For example, the best-performing Δz = 120 m model simulates the most-negative temperature flux at 600 m but at least one other Δz = 120 m simulation has the highest temperature flux (Fig. S3f). On the other hand, all the Δz = 120 m models underestimate cloud fraction both compared to LES and SCM simulations with smaller Δz (Fig. S3c). Equation (4) presents a way to summarize this QI variability. For each model layer and case, Sz,i represents the uncertainty in QI values from 25 CTAT replicates σCTAT,z,i relative to the LES uncertainty σLES,z,i:
Sz,i=σCTAT,z,iσLES,z,i.

Figure 9 presents histograms of Sz,i for the layer-weighted combination of cases (Fig. 9a), DYCOMS2rf2 (Fig. 9b), and RICO (Fig. 9c). In both cases, the Δz = 120 m model exits the CTAT algorithm with a wider range of model states, as estimated by the QIs. The relationship is near-monotonic, but with the Δz = 60 m models showing slightly less variability than the Δz = 40 m models. The dependence of Sz,i on Δz is clear in both the DYCOMS2rf2 and RICO cases; however, the dependence is most evident for the stratocumulus case.

Fig. 9.
Fig. 9.

Normalized histograms of the relative spread Sz,i for the (a) combination of cases, (b) DYCOMS2rf2, and (c) RICO. Histograms have 14 log-spaced bins between 0.01 and 13.5. The dashes at the left side of each panel indicate the proportion of Sz,i that fall below 0.01. Dots on the abscissa indicate the average Sz,i for that Δz. In (a), Sz,i from DYCOMS2rf2 and RICO have been added in such a way to weight the two cases equally.

Citation: Monthly Weather Review 151, 11; 10.1175/MWR-D-23-0078.1

5. Discussion and conclusions

This work demonstrates the strong influence of vertical resolution on the performance of the JPL-EDMF run as a SCM for two simulation cases that are commonly used in parameterization development and validation. To do so, we employed a novel objective tuning method that accounts for uncertainty in LES model output, including LES output covariances, to define an objective metric (χ2) against which the SCM can be tuned. We then identify the best parameter tunings at each vertical grid spacing. The tuning method stabilizes the parameter optimization by restricting the optimization to two parameters at a time. Once the best-performing combination of those two parameters is identified, those values are fixed within the model and the algorithm continues to the next parameter pair. The algorithm performs several rounds of these iterations, with each round containing randomized parameter pairings.

We present the results of three experiments, which demonstrated improvements to model performance as vertical grid spacing and time step were reduced, showed that the improvements to the model were mostly due to reductions in vertical grid spacing instead of reductions in time step, and showed that the tuning method is insensitive to the number of times each parameter is considered in the tuning process beyond four cycles. Our findings can be summarized as follows.

  • Reducing the model’s vertical grid spacing consistently improves model performance in simulations of marine shallow boundary layers. These improvements are mostly due to reductions in vertical grid spacing and not the reductions in model time step.

  • The model’s improvement as vertical grid spacing is reduced exists regardless of parameter tuning. For example, the worst performing model tuned at Δz = 60 m performs better than the best performing model tuned at Δz = 120 m.

  • The SCM performance converges near Δz = 20 m with further refinements at Δz = 10 m leading to diminishing returns in terms of agreement with LES benchmarks.

  • Model results are more sensitive to parameter tuning at large Δz than at small Δz. As a result, tuning should be “easier” when Δz is small.

We acknowledge that the tuning methodology introduced here is not necessarily ideal for larger-scale tuning efforts, like for an Earth System Model with a coupling to the circulation and more interactive physical processes. Other methods have been used in objective model tuning efforts, like adaptive Markov chain Monte Carlo (Järvinen et al. 2010), surrogate models (Bellprat et al. 2012; Langhans et al. 2019; Boukabara et al. 2021; Couvreux et al. 2021), and simulated annealing (Ingber 1989; Jackson et al. 2003; Yang et al. 2012). These established alternatives are potentially more computationally efficient and more widely utilized by the community. However, for the experiments needed for this work, the novel tuning algorithm provided robust answers to the question of how vertical grid spacing affects model performance in a single column model framework. We also acknowledge the possibility of local χ2 minima in the 18-dimensional parameter space interrogated by the CTAT algorithm being smaller than the parameter mesh (9 × 9 here) used in the tuning algorithm, leading to imperfect journeys toward the best possible agreement with the LES benchmarks. However, the performance of other techniques would also suffer under these difficult conditions. Only a complete exploration of the parameter space can be assured to find the global minimum.

Although it can be expected that CPU time increases when the vertical grid is refined, we note that the improvements to the horizontal grid result in a N2 factor of computational increase, while increases in the vertical dimension scale with N. Uncertainty in boundary layer clouds are at the heart of climate model uncertainty in projections of global temperature change (Bony and Dufresne 2005; Zelinka et al. 2013). Given the utmost importance of accurately representing these clouds in climate simulations, the particular sensitivity of simulated marine boundary layer clouds to model vertical grid spacing, and the reduced computational burden of reducing vertical grid spacing compared to horizontal grid spacing, we advocate for a renewed emphasis on reducing vertical grid spacing in global models to improve the efficacy of critical long term climate simulations. We also note that, while the vertical grid spacing in these experiments is constant throughout the vertical domain, there have been recent efforts to reduce the vertical grid spacing specifically in layers that would likely most benefit from those extra model layers. In particular, the Framework for Improvement by Vertical Enhancement is a computationally efficient method to add model layers where it is most beneficial (Yamaguchi et al. 2017; Chen et al. 2021; Lee et al. 2021, 2022; Bogenschutz et al. 2023). Increasing the number of vertical layers only where most beneficial minimizes the added computational expense while providing improved performance in simulating the clouds and circulations in these crucial weather regimes.

Acknowledgments.

The authors recognize the improvements to this manuscript that were a result of the thoughtful questions and suggestions made by the three anonymous reviewers and the editor. The authors thank Dr. Derek Posselt and Nimrod Carmon of the Jet Propulsion Laboratory for their thoughtful guidance in navigating the shortcomings of optimal estimation for our application and in overcoming the challenges of achieving numerical stability in the inversion of the SY covariance matrix. The authors also thank Dr. Kay Suselj for his thoughtful comments regarding the description of the JPL-EDMF. The research described in this paper was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration (80NM0018D0004). This research was supported by the NASA MAP Program, the U.S. Department of Energy, Office of Biological and Environmental Research, Earth System Modeling (DE-SC0019242), the ONR Marine Meteorology Program (N000141812432), the National Science Foundation Climate Process Team (CPT) Program (AGS 1916619), and the NASA CloudSat/CALIPSO science team (WBS/RTOP:105357/967701.02.01.02.69). The High Performance Computing resources used in this investigation were provided by funding from the JPL Information and Technology Solutions Directorate.

Data availability statement.

The raw experimental results for each simulation are large and were not saved, but the QI results for each end-state CTAT optimization, the LES benchmarks, and the optimization layer heights are available from zenodo at https://zenodo.org/records/10045653.

REFERENCES

  • Abel, S. J., and I. A. Boutle, 2012: An improved representation of the raindrop size distribution for single-moment microphysics schemes. Quart. J. Roy. Meteor. Soc., 138, 21512162, https://doi.org/10.1002/qj.1949.

    • Search Google Scholar
    • Export Citation
  • Ackerman, A. S., and Coauthors, 2009: Large-eddy simulations of a drizzling, stratocumulus-topped marine boundary layer. Mon. Wea. Rev., 137, 10831110, https://doi.org/10.1175/2008MWR2582.1.

    • Search Google Scholar
    • Export Citation
  • Bellprat, O., S. Kotlarski, D. Lüthi, and C. Schär, 2012: Objective calibration of regional climate models. J. Geophys. Res., 117, D23115, https://doi.org/10.1029/2012JD018262.

    • Search Google Scholar
    • Export Citation
  • Bogenschutz, P. A., T. Yamaguchi, and H.-H. Lee, 2021: The energy exascale Earth system model simulations with high vertical resolution in the lower troposphere. J. Adv. Model. Earth Syst., 13, e2020MS002239, https://doi.org/10.1029/2020MS002239.

    • Search Google Scholar
    • Export Citation
  • Bogenschutz, P. A., H.-H. Lee, Q. Tang, and T. Yamaguchi, 2023: Combining regional mesh refinement with vertically enhanced physics to target marine stratocumulus biases as demonstrated in the energy exascale Earth system model version 1. Geosci. Model Dev., 16, 335352, https://doi.org/10.5194/gmd-16-335-2023.

    • Search Google Scholar
    • Export Citation
  • Bony, S., and J.-L. Dufresne, 2005: Marine boundary layer clouds at the heart of tropical cloud feedback uncertainties in climate models. Geophys. Res. Lett., 32, L20806, https://doi.org/10.1029/2005GL023851.

    • Search Google Scholar
    • Export Citation
  • Boukabara, S.-A., and Coauthors, 2021: Outlook for exploiting artificial intelligence in the Earth and environmental sciences. Bull. Amer. Meteor. Soc., 102, E1016E1032, https://doi.org/10.1175/BAMS-D-20-0031.1.

    • Search Google Scholar
    • Export Citation
  • Bushell, A. C., and G. M. Martin, 1999: The impact of vertical resolution upon GCM simulations of marine stratocumulus. Climate Dyn., 15, 293318, https://doi.org/10.1007/s003820050283.

    • Search Google Scholar
    • Export Citation
  • Chen, Y.-S., T. Yamaguchi, P. A. Bogenschutz, and G. Feingold, 2021: Model evaluation and intercomparison of marine warm low cloud fractions with neural network ensembles. J. Adv. Model. Earth Syst., 13, e2021MS002625, https://doi.org/10.1029/2021MS002625.

    • Search Google Scholar
    • Export Citation
  • Couvreux, F., and Coauthors, 2021: Process-based climate model development harnessing machine learning: I. A calibration tool for parameterization improvement. J. Adv. Model. Earth Syst., 13, e2020MS002217, https://doi.org/10.1029/2020MS002217.

    • Search Google Scholar
    • Export Citation
  • Cubasch, U., and R. D. Cess, 1990: Processes and modelling. Climate Change 1990: The IPCC Scientific Assessment, J. T. Houghton, G. J. Jenkins and J. J. Ephraums, Eds., Cambridge University Press, 69–92.

  • Flato, G., and Coauthors, 2013: Evaluation of climate models. Climate Change 2013: The Physical Science Basis, T. F. Stocker et al., Eds., Cambridge University Press, 741–866.

  • Gates, W. L., and Coauthors, 1995: Climate models—Evaluation. Climate Change 1995: The Science of Climate Change, J. T. Houghton et al., Eds., Cambridge University Press, 229–284.

  • Gutiérrez, J. M., and Coauthors, 2021: Annex II: Models. Climate Change 2021: The Physical Science Basis, V. Masson-Delmotte et al., Eds., Cambridge University Press, 2087–2138, https://doi.org/10.1017/9781009157896.016.

  • Ingber, L., 1989: Very fast simulated re-annealing. Math. Comput. Modell., 12, 967973, https://doi.org/10.1016/0895-7177(89)90202-1.

  • Jackson, C., Y. Xia, M. K. Sen, and P. L. Stoffa, 2003: Optimal parameter and uncertainty estimation of a land surface model: A case study using data from Cabauw, Netherlands. J. Geophys. Res., 108, 4583, https://doi.org/10.1029/2002JD002991.

    • Search Google Scholar
    • Export Citation
  • Järvinen, H., P. Räisänen, M. Laine, J. Tamminen, A. Ilin, E. Oja, A. Solonen, and H. Haario, 2010: Estimation of ECHAM5 climate model closure parameters with adaptive MCMC. Atmos. Chem. Phys., 10, 999310 002, https://doi.org/10.5194/acp-10-9993-2010.

    • Search Google Scholar
    • Export Citation
  • Khairoutdinov, M., and Y. Kogan, 2000: A new cloud physics parameterization in a large-eddy simulation model of marine stratocumulus. Mon. Wea. Rev., 128, 229243, https://doi.org/10.1175/1520-0493(2000)128<0229:ANCPPI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Langhans, W., J. Mueller, and W. D. Collins, 2019: Optimization of the eddy-diffusivity/mass-flux shallow cumulus and boundary-layer parameterization using surrogate models. J. Adv. Model. Earth Syst., 11, 402416, https://doi.org/10.1029/2018MS001449.

    • Search Google Scholar
    • Export Citation
  • Laprise, R., 1992: The resolution of global spectral models. Bull. Amer. Meteor. Soc., 73, 14531455, https://doi.org/10.1175/1520-0477-73.9.1453.

    • Search Google Scholar
    • Export Citation
  • L’Ecuyer, T. S., Y. Hang, A. V. Matus, and Z. Wang, 2019: Reassessing the effect of cloud type on Earth’s energy balance in the age of active spaceborne observations. Part I: Top of atmosphere and surface. J. Climate, 32, 61976217, https://doi.org/10.1175/JCLI-D-18-0753.1.

    • Search Google Scholar
    • Export Citation
  • Lee, H.-H., P. Bogenschutz, and T. Yamaguchi, 2021: The implementation of framework for improvement by vertical enhancement into energy exascale Earth system model. J. Adv. Model. Earth Syst., 13, e2020MS002240, https://doi.org/10.1029/2020MS002240.

    • Search Google Scholar
    • Export Citation
  • Lee, H.-H., P. Bogenschutz, and T. Yamaguchi, 2022: Resolving away stratocumulus biases in modern global climate models. Geophys. Res. Lett., 49, e2022GL099422, https://doi.org/10.1029/2022GL099422.

    • Search Google Scholar
    • Export Citation
  • McAvaney, B. J., and Coauthors, 2001: Model evaluation. Climate Change 2001: The Scientific Basis, J. T. Houghton et al., Eds., Cambridge University Press, 471–524.

  • Mellor, G. L., and T. Yamada, 1974: A hierarchy of turbulence closure models for planetary boundary layers. J. Atmos. Sci., 31, 17911806, https://doi.org/10.1175/1520-0469(1974)031<1791:AHOTCM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Nam, C., S. Bony, J.-L. Dufresne, and H. Chepfer, 2012: The ‘too few, too bright’ tropical low-cloud problem in CMIP5 models. Geophys. Res. Lett., 39, L21801, https://doi.org/10.1029/2012GL053421.

    • Search Google Scholar
    • Export Citation
  • Ockert-Bell, M. E., and D. L. Hartmann, 1992: The effect of cloud type on Earth’s energy balance: Results for selected regions. J. Climate, 5, 11571171, https://doi.org/10.1175/1520-0442(1992)005<1157:TEOCTO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Randall, D. A., and Coauthors, 2007: Climate models and their evaluation. Climate Change 2007: The Physical Science Basis, S. Solomon et al., Eds., Cambridge University Press, 589–662.

  • Rauber, R. M., and Coauthors, 2007: Rain in shallow cumulus over the ocean: The RICO campaign. Bull. Amer. Meteor. Soc., 88, 19121928, https://doi.org/10.1175/BAMS-88-12-1912.

    • Search Google Scholar
    • Export Citation
  • Smalley, M. A., K. Suselj, M. D. Lebsock, and M. K. Witte, 2022: Coupling warm rain with an eddy diffusivity/mass flux parameterization: 2. Sensitivities and comparison to observations. J. Adv. Model. Earth Syst., 14, e2021MS002729, https://doi.org/10.1029/2021MS002729.

    • Search Google Scholar
    • Export Citation
  • Sušelj, K., J. Teixeira, and G. Matheou, 2012: Eddy diffusivity/mass flux and shallow cumulus boundary layer: An updraft PDF multiple mass flux scheme. J. Atmos. Sci., 69, 15131533, https://doi.org/10.1175/JAS-D-11-090.1.

    • Search Google Scholar
    • Export Citation
  • Sušelj, K., J. Teixeira, and D. Chung, 2013: A unified model for moist convective boundary layers based on a stochastic eddy-diffusivity/mass-flux parameterization. J. Atmos. Sci., 70, 19291953, https://doi.org/10.1175/JAS-D-12-0106.1.

    • Search Google Scholar
    • Export Citation
  • Sušelj, K., M. J. Kurowski, and J. Teixeira, 2019: A unified eddy-diffusivity/mass-flux approach for modeling atmospheric convection. J. Atmos. Sci., 76, 25052537, https://doi.org/10.1175/JAS-D-18-0239.1.

    • Search Google Scholar
    • Export Citation
  • Sušelj, K., J. Teixeira, M. J. Kurowski, and A. Molod, 2021: Improving the representation of subtropical boundary layer clouds in the NASA GEOS model with the eddy-diffusivity/mass-flux parameterization. Mon. Wea. Rev., 149, 793809, https://doi.org/10.1175/MWR-D-20-0183.1.

    • Search Google Scholar
    • Export Citation
  • Suselj, K., M. Smalley, M. D. Lebsock, M. J. Kurowski, M. K. Witte, and J. Teixeira, 2022: Coupling warm rain with an eddy diffusivity/mass flux parameterization: 1. Model description and validation. J. Adv. Model. Earth Syst., 14, e2021MS002736, https://doi.org/10.1029/2021MS002736.

    • Search Google Scholar
    • Export Citation
  • Teixeira, J., 1999: The impact of increased boundary layer vertical resolution on the ECMWF forecast system. ECMWF Tech. Memo. 268, 55 pp., https://www.ecmwf.int/en/elibrary/76615-impact-increased-boundary-layer-vertical-resolution-ecmwf-forecast-system.

  • Teixeira, J., and S. Cheinet, 2004: A simple mixing length formulation for the eddy-diffusivity parameterization of dry convection. Bound.-Layer Meteor., 110, 435453, https://doi.org/10.1023/B:BOUN.0000007230.96303.0d.

    • Search Google Scholar
    • Export Citation
  • Teixeira, J., and Coauthors, 2011: Tropical and subtropical cloud transitions in weather and climate prediction models: The GCSS/WGNE Pacific cross-section intercomparison (GPCI). J. Climate, 24, 52235256, https://doi.org/10.1175/2011JCLI3672.1.

    • Search Google Scholar
    • Export Citation
  • van Zanten, M. C., and Coauthors, 2011: Controls on precipitation and cloudiness in simulations of trade-wind cumulus as observed during RICO. J. Adv. Model. Earth Syst., 3(2), https://doi.org/10.1029/2011MS000056.

    • Search Google Scholar
    • Export Citation
  • Wang, W., M. Shangguan, W. Tian, T. Schmidt, and A. Ding, 2019: Large uncertainties in estimation of tropical tropopause temperature variabilities due to model vertical resolution. Geophys. Res. Lett., 46, 10 04310 052, https://doi.org/10.1029/2019GL084112.

    • Search Google Scholar
    • Export Citation
  • Witek, M. L., J. Teixeira, and G. Matheou, 2011: An integrated TKE-based eddy diffusivity/mass flux boundary layer closure for the dry convective boundary layer. J. Atmos. Sci., 68, 15261540, https://doi.org/10.1175/2011JAS3548.1.

    • Search Google Scholar
    • Export Citation
  • Yamaguchi, T., G. Feingold, and V. E. Larson, 2017: Framework for improvement by vertical enhancement: A simple approach to improve representation of low and high-level clouds in large-scale models. J. Adv. Model. Earth Syst., 9, 627646, https://doi.org/10.1002/2016MS000815.

    • Search Google Scholar
    • Export Citation
  • Yang, B., Y. Qian, G. Lin, R. Leung, and Y. Zhang, 2012: Some issues in uncertainty quantification and parameter tuning: A case study of convective parameterization scheme in the WRF regional climate model. Atmos. Chem. Phys., 12, 24092427, https://doi.org/10.5194/acp-12-2409-2012.

    • Search Google Scholar
    • Export Citation
  • Zelinka, M. D., S. A. Klein, K. E. Taylor, T. Andrews, M. J. Webb, J. M. Gregory, and P. M. Forster, 2013: Contributions of different cloud types to feedbacks and rapid adjustments in CMIP5. J. Climate, 26, 50075027, https://doi.org/10.1175/JCLI-D-12-00555.1.

    • Search Google Scholar
    • Export Citation

Supplementary Materials

Save
  • Abel, S. J., and I. A. Boutle, 2012: An improved representation of the raindrop size distribution for single-moment microphysics schemes. Quart. J. Roy. Meteor. Soc., 138, 21512162, https://doi.org/10.1002/qj.1949.

    • Search Google Scholar
    • Export Citation
  • Ackerman, A. S., and Coauthors, 2009: Large-eddy simulations of a drizzling, stratocumulus-topped marine boundary layer. Mon. Wea. Rev., 137, 10831110, https://doi.org/10.1175/2008MWR2582.1.

    • Search Google Scholar
    • Export Citation
  • Bellprat, O., S. Kotlarski, D. Lüthi, and C. Schär, 2012: Objective calibration of regional climate models. J. Geophys. Res., 117, D23115, https://doi.org/10.1029/2012JD018262.

    • Search Google Scholar
    • Export Citation
  • Bogenschutz, P. A., T. Yamaguchi, and H.-H. Lee, 2021: The energy exascale Earth system model simulations with high vertical resolution in the lower troposphere. J. Adv. Model. Earth Syst., 13, e2020MS002239, https://doi.org/10.1029/2020MS002239.

    • Search Google Scholar
    • Export Citation
  • Bogenschutz, P. A., H.-H. Lee, Q. Tang, and T. Yamaguchi, 2023: Combining regional mesh refinement with vertically enhanced physics to target marine stratocumulus biases as demonstrated in the energy exascale Earth system model version 1. Geosci. Model Dev., 16, 335352, https://doi.org/10.5194/gmd-16-335-2023.

    • Search Google Scholar
    • Export Citation
  • Bony, S., and J.-L. Dufresne, 2005: Marine boundary layer clouds at the heart of tropical cloud feedback uncertainties in climate models. Geophys. Res. Lett., 32, L20806, https://doi.org/10.1029/2005GL023851.

    • Search Google Scholar
    • Export Citation
  • Boukabara, S.-A., and Coauthors, 2021: Outlook for exploiting artificial intelligence in the Earth and environmental sciences. Bull. Amer. Meteor. Soc., 102, E1016E1032, https://doi.org/10.1175/BAMS-D-20-0031.1.

    • Search Google Scholar
    • Export Citation
  • Bushell, A. C., and G. M. Martin, 1999: The impact of vertical resolution upon GCM simulations of marine stratocumulus. Climate Dyn., 15, 293318, https://doi.org/10.1007/s003820050283.

    • Search Google Scholar
    • Export Citation
  • Chen, Y.-S., T. Yamaguchi, P. A. Bogenschutz, and G. Feingold, 2021: Model evaluation and intercomparison of marine warm low cloud fractions with neural network ensembles. J. Adv. Model. Earth Syst., 13, e2021MS002625, https://doi.org/10.1029/2021MS002625.

    • Search Google Scholar
    • Export Citation
  • Couvreux, F., and Coauthors, 2021: Process-based climate model development harnessing machine learning: I. A calibration tool for parameterization improvement. J. Adv. Model. Earth Syst., 13, e2020MS002217, https://doi.org/10.1029/2020MS002217.

    • Search Google Scholar
    • Export Citation
  • Cubasch, U., and R. D. Cess, 1990: Processes and modelling. Climate Change 1990: The IPCC Scientific Assessment, J. T. Houghton, G. J. Jenkins and J. J. Ephraums, Eds., Cambridge University Press, 69–92.

  • Flato, G., and Coauthors, 2013: Evaluation of climate models. Climate Change 2013: The Physical Science Basis, T. F. Stocker et al., Eds., Cambridge University Press, 741–866.

  • Gates, W. L., and Coauthors, 1995: Climate models—Evaluation. Climate Change 1995: The Science of Climate Change, J. T. Houghton et al., Eds., Cambridge University Press, 229–284.

  • Gutiérrez, J. M., and Coauthors, 2021: Annex II: Models. Climate Change 2021: The Physical Science Basis, V. Masson-Delmotte et al., Eds., Cambridge University Press, 2087–2138, https://doi.org/10.1017/9781009157896.016.

  • Ingber, L., 1989: Very fast simulated re-annealing. Math. Comput. Modell., 12, 967973, https://doi.org/10.1016/0895-7177(89)90202-1.

  • Jackson, C., Y. Xia, M. K. Sen, and P. L. Stoffa, 2003: Optimal parameter and uncertainty estimation of a land surface model: A case study using data from Cabauw, Netherlands. J. Geophys. Res., 108, 4583, https://doi.org/10.1029/2002JD002991.

    • Search Google Scholar
    • Export Citation
  • Järvinen, H., P. Räisänen, M. Laine, J. Tamminen, A. Ilin, E. Oja, A. Solonen, and H. Haario, 2010: Estimation of ECHAM5 climate model closure parameters with adaptive MCMC. Atmos. Chem. Phys., 10, 999310 002, https://doi.org/10.5194/acp-10-9993-2010.

    • Search Google Scholar
    • Export Citation
  • Khairoutdinov, M., and Y. Kogan, 2000: A new cloud physics parameterization in a large-eddy simulation model of marine stratocumulus. Mon. Wea. Rev., 128, 229243, https://doi.org/10.1175/1520-0493(2000)128<0229:ANCPPI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Langhans, W., J. Mueller, and W. D. Collins, 2019: Optimization of the eddy-diffusivity/mass-flux shallow cumulus and boundary-layer parameterization using surrogate models. J. Adv. Model. Earth Syst., 11, 402416, https://doi.org/10.1029/2018MS001449.

    • Search Google Scholar
    • Export Citation
  • Laprise, R., 1992: The resolution of global spectral models. Bull. Amer. Meteor. Soc., 73, 14531455, https://doi.org/10.1175/1520-0477-73.9.1453.

    • Search Google Scholar
    • Export Citation
  • L’Ecuyer, T. S., Y. Hang, A. V. Matus, and Z. Wang, 2019: Reassessing the effect of cloud type on Earth’s energy balance in the age of active spaceborne observations. Part I: Top of atmosphere and surface. J. Climate, 32, 61976217, https://doi.org/10.1175/JCLI-D-18-0753.1.

    • Search Google Scholar
    • Export Citation
  • Lee, H.-H., P. Bogenschutz, and T. Yamaguchi, 2021: The implementation of framework for improvement by vertical enhancement into energy exascale Earth system model. J. Adv. Model. Earth Syst., 13, e2020MS002240, https://doi.org/10.1029/2020MS002240.

    • Search Google Scholar
    • Export Citation
  • Lee, H.-H., P. Bogenschutz, and T. Yamaguchi, 2022: Resolving away stratocumulus biases in modern global climate models. Geophys. Res. Lett., 49, e2022GL099422, https://doi.org/10.1029/2022GL099422.

    • Search Google Scholar
    • Export Citation
  • McAvaney, B. J., and Coauthors, 2001: Model evaluation. Climate Change 2001: The Scientific Basis, J. T. Houghton et al., Eds., Cambridge University Press, 471–524.

  • Mellor, G. L., and T. Yamada, 1974: A hierarchy of turbulence closure models for planetary boundary layers. J. Atmos. Sci., 31, 17911806, https://doi.org/10.1175/1520-0469(1974)031<1791:AHOTCM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Nam, C., S. Bony, J.-L. Dufresne, and H. Chepfer, 2012: The ‘too few, too bright’ tropical low-cloud problem in CMIP5 models. Geophys. Res. Lett., 39, L21801, https://doi.org/10.1029/2012GL053421.

    • Search Google Scholar
    • Export Citation
  • Ockert-Bell, M. E., and D. L. Hartmann, 1992: The effect of cloud type on Earth’s energy balance: Results for selected regions. J. Climate, 5, 11571171, https://doi.org/10.1175/1520-0442(1992)005<1157:TEOCTO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Randall, D. A., and Coauthors, 2007: Climate models and their evaluation. Climate Change 2007: The Physical Science Basis, S. Solomon et al., Eds., Cambridge University Press, 589–662.

  • Rauber, R. M., and Coauthors, 2007: Rain in shallow cumulus over the ocean: The RICO campaign. Bull. Amer. Meteor. Soc., 88, 19121928, https://doi.org/10.1175/BAMS-88-12-1912.

    • Search Google Scholar
    • Export Citation
  • Smalley, M. A., K. Suselj, M. D. Lebsock, and M. K. Witte, 2022: Coupling warm rain with an eddy diffusivity/mass flux parameterization: 2. Sensitivities and comparison to observations. J. Adv. Model. Earth Syst., 14, e2021MS002729, https://doi.org/10.1029/2021MS002729.

    • Search Google Scholar
    • Export Citation
  • Sušelj, K., J. Teixeira, and G. Matheou, 2012: Eddy diffusivity/mass flux and shallow cumulus boundary layer: An updraft PDF multiple mass flux scheme. J. Atmos. Sci., 69, 15131533, https://doi.org/10.1175/JAS-D-11-090.1.

    • Search Google Scholar
    • Export Citation
  • Sušelj, K., J. Teixeira, and D. Chung, 2013: A unified model for moist convective boundary layers based on a stochastic eddy-diffusivity/mass-flux parameterization. J. Atmos. Sci., 70, 19291953, https://doi.org/10.1175/JAS-D-12-0106.1.

    • Search Google Scholar
    • Export Citation
  • Sušelj, K., M. J. Kurowski, and J. Teixeira, 2019: A unified eddy-diffusivity/mass-flux approach for modeling atmospheric convection. J. Atmos. Sci., 76, 25052537, https://doi.org/10.1175/JAS-D-18-0239.1.

    • Search Google Scholar
    • Export Citation
  • Sušelj, K., J. Teixeira, M. J. Kurowski, and A. Molod, 2021: Improving the representation of subtropical boundary layer clouds in the NASA GEOS model with the eddy-diffusivity/mass-flux parameterization. Mon. Wea. Rev., 149, 793809, https://doi.org/10.1175/MWR-D-20-0183.1.

    • Search Google Scholar
    • Export Citation
  • Suselj, K., M. Smalley, M. D. Lebsock, M. J. Kurowski, M. K. Witte, and J. Teixeira, 2022: Coupling warm rain with an eddy diffusivity/mass flux parameterization: 1. Model description and validation. J. Adv. Model. Earth Syst., 14, e2021MS002736, https://doi.org/10.1029/2021MS002736.

    • Search Google Scholar
    • Export Citation
  • Teixeira, J., 1999: The impact of increased boundary layer vertical resolution on the ECMWF forecast system. ECMWF Tech. Memo. 268, 55 pp., https://www.ecmwf.int/en/elibrary/76615-impact-increased-boundary-layer-vertical-resolution-ecmwf-forecast-system.

  • Teixeira, J., and S. Cheinet, 2004: A simple mixing length formulation for the eddy-diffusivity parameterization of dry convection. Bound.-Layer Meteor., 110, 435453, https://doi.org/10.1023/B:BOUN.0000007230.96303.0d.

    • Search Google Scholar
    • Export Citation
  • Teixeira, J., and Coauthors, 2011: Tropical and subtropical cloud transitions in weather and climate prediction models: The GCSS/WGNE Pacific cross-section intercomparison (GPCI). J. Climate, 24, 52235256, https://doi.org/10.1175/2011JCLI3672.1.

    • Search Google Scholar
    • Export Citation
  • van Zanten, M. C., and Coauthors, 2011: Controls on precipitation and cloudiness in simulations of trade-wind cumulus as observed during RICO. J. Adv. Model. Earth Syst., 3(2), https://doi.org/10.1029/2011MS000056.

    • Search Google Scholar
    • Export Citation
  • Wang, W., M. Shangguan, W. Tian, T. Schmidt, and A. Ding, 2019: Large uncertainties in estimation of tropical tropopause temperature variabilities due to model vertical resolution. Geophys. Res. Lett., 46, 10 04310 052, https://doi.org/10.1029/2019GL084112.

    • Search Google Scholar
    • Export Citation
  • Witek, M. L., J. Teixeira, and G. Matheou, 2011: An integrated TKE-based eddy diffusivity/mass flux boundary layer closure for the dry convective boundary layer. J. Atmos. Sci., 68, 15261540, https://doi.org/10.1175/2011JAS3548.1.

    • Search Google Scholar
    • Export Citation
  • Yamaguchi, T., G. Feingold, and V. E. Larson, 2017: Framework for improvement by vertical enhancement: A simple approach to improve representation of low and high-level clouds in large-scale models. J. Adv. Model. Earth Syst., 9, 627646, https://doi.org/10.1002/2016MS000815.

    • Search Google Scholar
    • Export Citation
  • Yang, B., Y. Qian, G. Lin, R. Leung, and Y. Zhang, 2012: Some issues in uncertainty quantification and parameter tuning: A case study of convective parameterization scheme in the WRF regional climate model. Atmos. Chem. Phys., 12, 24092427, https://doi.org/10.5194/acp-12-2409-2012.

    • Search Google Scholar
    • Export Citation
  • Zelinka, M. D., S. A. Klein, K. E. Taylor, T. Andrews, M. J. Webb, J. M. Gregory, and P. M. Forster, 2013: Contributions of different cloud types to feedbacks and rapid adjustments in CMIP5. J. Climate, 26, 50075027, https://doi.org/10.1175/JCLI-D-12-00555.1.

    • Search Google Scholar
    • Export Citation
  • Fig. 1.

    Evolution of two of the primary model properties that contribute to the computational cost of global models: (a) the number of horizontal grid points and (b) the number of vertical layers in global models that contributed to the six IPCC reports. Later IPCC experiments included more detailed formulations of Earth’s atmosphere above the troposphere, so a distinction is made for models with tops higher than 60 km, as many of the layers in those models are above the troposphere and, therefore, irrelevant to boundary layer processes examined in this work. Circles indicate the median and the line ends indicate the 25th to 75th percentiles. The horizontal grid spacing of spectral models is estimated using the “L2” relation found provided by Laprise (1992), who explains that the values may be an underestimate of the true grid spacing of spectral models. As time passed, more models converted to fixed latitude–longitude grids so the fractional increase in horizontal grid points visible in (a) can be interpreted as a lower bound on the relative improvements to horizontal grid spacing in global models.

  • Fig. 2.

    CTAT algorithm flowchart.

  • Fig. 3.

    Evolution of χ2 for all 25 CTAT replicates for (a) Experiment 1, (c) Experiment 2, and (e) Experiment 3 and normalized histograms of the final values of χ2 at the end of (b) Experiment 1, (d) Experiment 2, and (f) Experiment 3.

  • Fig. 4.

    Retrieved parameter values from CTAT, presented as histograms with nine evenly spaced bins spanning each parameter’s valid range from Table 2.

  • Fig. 5.

    DYCOMS2rf2 QI profiles for the CTAT replicates that retrieved the lowest χ2. Gray-shaded regions exhibit the ±1 sigma range of the LES ensemble.

  • Fig. 6.

    Relative difference Dr in SCM DYCOMS2rf2 QIs found using CTAT from the LES benchmarks.

  • Fig. 7.

    As in Fig. 5, but for the RICO QI profiles.

  • Fig. 8.

    As in Fig. 6, but for the relative difference Dr in RICO QIs.

  • Fig. 9.

    Normalized histograms of the relative spread Sz,i for the (a) combination of cases, (b) DYCOMS2rf2, and (c) RICO. Histograms have 14 log-spaced bins between 0.01 and 13.5. The dashes at the left side of each panel indicate the proportion of Sz,i that fall below 0.01. Dots on the abscissa indicate the average Sz,i for that Δz. In (a), Sz,i from DYCOMS2rf2 and RICO have been added in such a way to weight the two cases equally.

All Time Past Year Past 30 Days
Abstract Views 180 180 0
Full Text Views 417 417 13
PDF Downloads 279 279 12