1. Introduction
The parameterization of cloud and precipitation microphysics is beset by a number of fundamental difficulties. Because there is a limited theoretical understanding of many microphysical processes, development of model parameterizations must rely heavily on cloud and precipitation observations. Unfortunately, these observations generally do not provide direct information on the microphysical process rates themselves, and so model developers must consider measurements of bulk quantities such as cloud water content that evolve through the net effects of several interacting microphysical and dynamical processes. As argued in Morrison et al. (2020, hereafter Part I), the incorporation of such observational information in a microphysics scheme should be posed as a probabilistic problem, with Bayes’s theorem defining the solution. Use of Bayesian parameter estimation has the important advantage of quantifying uncertainty within the microphysics scheme in question.
As opposed to Bayesian approaches to state estimation employed in, for example, data assimilation for numerical weather forecasts, parameter estimation and related uncertainty quantification problems are often characterized by strong nonlinearities and relatively low problem dimensionality (order <100 unknowns to estimate)—for these problems Markov chain Monte Carlo (MCMC) samplers are well suited to the task of estimating the parameter probability density function (PDF) owing to their generality and lack of assumptions on the underlying form of the PDF or linearity of model sensitivity to parameter perturbation.
Our ultimate goal is to use observations to inform BOSS probabilistically, determining the nature and complexity of its physical sensitivities using the wealth of microphysical information present in, for example, polarimetric radar observations, together with the limited a priori microphysical insights that exist. As a proof of concept, we here show that this is feasible within an idealized modeling framework. This should be considered a minimum requirement for a scheme like BOSS, as use of real observations to constrain BOSS will unavoidably introduce other uncertainties (observational noise/calibration, atmospheric-state uncertainty, forward simulator uncertainty, etc.) These difficulties, and approaches to mitigate them, will be left to future work.
There are numerous potential applications for a full account of structural and parametric uncertainty in microphysics, in particular within the context of data assimilation and probabilistic forecasts. For example, multimodel and multiphysics ensembles are commonly employed in these contexts in an attempt to span the structural uncertainties associated with discrete microphysics parameterization choices (Berner et al. 2011, 2015; Jankov et al. 2017). These multimodel ensembles help produce forecasts with sufficient spread to characterize model forecast error, but have no clearly defined probabilistic meaning. An ensemble of BOSS simulations should be capable of spanning structural and parametric uncertainty in a unified, consistent way that is informed by observations and also probabilistically reflects both lack of observational constraints on microphysical knowledge and the limited physical insights that do exist. It is also possible that BOSS may be relevant for development of stochastic microphysics schemes, but we do not explore such issues here.
A crucial goal of BOSS is that process-level understanding is gained via observational constraint—the statistical–physical approach we have chosen renders all BOSS parameters physically interpretable, and in the ideal limit of a perfect model and comprehensive observations, we would expect that the representation of microphysical processes within BOSS would approach the “true” process rates of nature. In this sense, constraint of BOSS can be viewed in the context of microphysical “fingerprinting” (Kumjian and Prat 2014; Moiseev et al. 2015, 2017) or microphysical process retrievals (Williams 2016; Tridon et al. 2017), with the additional benefit of robust estimation of uncertainties arising from a priori assumptions and observational uncertainties.
In the current paper, Part II of this series, we consider three configurations of BOSS with varying degrees of complexity and explore synthetic constraint of the scheme within one-dimensional idealized rainshaft simulations. Our goals are to show that BOSS parameters can be accurately constrained within such a framework using MCMC, and to compare configurations of BOSS with varying degrees of structural complexity. Further, we analyze the structure of uncertainty in BOSS, including correlation between parameters, uncertainty in modeled microphysical processes, and forecast uncertainty that results from microphysics uncertainty.
Idealized rainshaft simulations are used here because we limit our initial development of BOSS to rain microphysical processes (evaporation, collision–coalescence, breakup, and sedimentation) owing to the additional complexities of ice microphysics, including well-known uncertainties in scattering characteristics of ice- and mixed-phase hydrometeors (Schrom and Kumjian 2018). We also for now ignore nonprecipitating cloud microphysics (drop activation, autoconversion, etc.)—these processes will be addressed in subsequent work. A one-dimensional steady-state rainshaft is chosen so that microphysical sensitivities are isolated from dynamical and thermodynamical feedbacks that can obscure their signal (Morrison and Grabowski 2007; Posselt and Vukicevic 2010; Shipway and Hill 2012). Throughout this paper, we refer to the one-dimensional rainshaft simulation as the “model,” the microphysics scheme used by the model as the “scheme,” and the specific setup for the BOSS scheme (e.g., number and choice of prognostic moments) as the “configuration.”
The primary purpose of this paper is demonstrating how BOSS can be constrained using observations and an MCMC sampler. Of particular interest are the following questions:
Can idealized observations of prognostic variables be used to unambiguously retrieve the true value of model parameters when the true values are known?
Do BOSS parameter PDFs display non-Gaussianity (indicative of nonlinear model response to parameter perturbations), or correlations in uncertainty between parameters (which suggest compensating errors)?
Can BOSS match the performance of a traditional bulk parameterization scheme, especially with respect to the representation of microphysical processes? How does this vary with the number of prognostic moments?
How does varying BOSS complexity (e.g., the number of terms in process rate power expressions) affect estimation of BOSS parameters and model forecast fidelity?
Question (i) will be answered by performing “known truth” experiments where observations are generated using the same model, scheme, and configuration that are used for parameter estimation. Questions (ii) and (iii) will be addressed by generating synthetic observations using a traditional three-moment bulk microphysics scheme (MORR; see section 3a in Part I for a description), and performing Bayesian parameter estimations with various configurations of BOSS. Question (iv) will be addressed by considering a known-truth abstraction of the BOSS parameterization framework.
The paper is organized as follows. Section 2 gives a description of the methodology, including an overview of BOSS and the MCMC sampling. Section 3 details the experimental design of each test using MCMC and synthetic observations to constrain BOSS. Section 4 presents results and brief discussion. A summary and conclusions are provided in section 5.
2. Methods
a. Overview of BOSS
The focus of this work is on rain microphysics. As such, the following microphysical processes are included: condensation/evaporation, collision–coalescence, breakup, and sedimentation. BOSS is flexible and can utilize any combination of prognostic moments and any number of terms in the process rate power expressions; these choices can be tailored depending on the observational constraint and Bayesian methodology used. Herein we utilize a two-moment configuration of BOSS prognosing M0 and M3 (M0-M3-B) or a three-moment configuration prognosing M0, M3, and M6 (M0-M3-M6-B), similar to the tests in Part I (here Mk is the kth moment of the DSD). We also test the impacts of including either one or two terms for the process rate power expressions for evaporation in two-moment BOSS (M0-M3-B2). As described in the introduction, this allows systematic testing of the effects of the number of parameters and scheme complexity.
We constrain BOSS with synthetic observations of M0 and M3 (and M6 additionally for M0-M3-M6-B) at 15 vertical levels for an ensemble of rainshaft simulations, as well as with synthetic observations of moment fluxes at the surface (M3 flux is proportional to rain rate) as might be obtained from a disdrometer. The synthetic observations are produced using a steady-state rainshaft model employing a three-moment configuration of Morrison et al. (2009) (MORR; see Part I for details). We note that MORR is an appealing choice of “truth” for this first test of BOSS constraint, as the functional form of its process rate formulations are in some cases power laws consistent with BOSS (e.g., for evaporation and sedimentation), and in other cases different functions (e.g., exponential for coalescence and breakup; see Part I appendix for more detail). Three-moment MORR is chosen instead of two moment so as to investigate the case where the model (e.g., M0-M3-B) has less structural complexity compared with the “true” model.
A practical consideration for using more realistic (e.g., radar) observations is that, owing to the lack of a specified DSD functional form in BOSS, many observational quantities cannot be easily forward simulated from BOSS prognostic DSD moments. Instead, to perform such forward simulation, observable variables would have to be related to prognostic moments in some other way (e.g., statistically). The recently developed moment-based operator of Kumjian et al. (2019), for example, uses a database of hundreds of millions of observed and bin-simulated DSDs to draw statistical relationships between DSD moments and polarimetric radar variables. This approach has the additional benefit of producing estimates of forward-simulator uncertainty. Testing of such methods with BOSS will be addressed in subsequent research and is outside of the scope of the current work, and we instead limit ourselves in this initial proof-of-concept work to the idealized case where model prognostic DSD moments are directly observed—a condition seldom met with real observations.
Both the BOSS simulations and synthetic observations are based on a steady-state rainshaft model with a vertical depth of 2 km and 80 equally spaced levels. The model is initialized with a constant thermodynamic profile, and a fixed DSD at column top (e.g., fixed values of M0, M3, M6)—see Part I for more details. Relative humidity (RH) conditions are drawn randomly from an even sampling of raining and nonraining days from in situ measurements by the U.S. Climate Reference Network (Diamond et al. 2013; Leeper et al. 2015). Column-top DSDs are drawn randomly from the database described in Morrison et al. (2019): a distribution of M0, M3, and M6 values created by an equal mixture of bin model and disdrometer DSD datasets, with the additional constraint that the reflectivity distribution of this dataset is uniform between 10 and 45 dBZ. The choice of boundary condition distributions affects the constraint of BOSS; for example, tests using a climatological distribution of rain rates (not shown) resulted in weaker constraint of coalescence/breakup parameters owing to the reduced effect of these processes in the predominantly lower-rain-rate cases. The DSD database and the choices made in its construction are described in Morrison et al. (2019).
Figure 1 illustrates schematically how observational constraint of BOSS is performed. Initial conditions are drawn from climatology, disdrometers, and bin model simulations (blue); these initialize both the model using MORR that generates the synthetic observations, as well as the model using BOSS (green). The MCMC algorithm uses the synthetic observations to constrain sampling of BOSS parameters such that the distribution of sampled points are drawn from the posterior parameter PDF
Schematic of the approach to estimate BOSS parameters in this study.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Schematic of the approach to estimate BOSS parameters in this study.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Schematic of the approach to estimate BOSS parameters in this study.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
b. MCMC
We employ the Adaptive Metropolis (AM; Haario et al. 2001) sampler, an MCMC-like algorithm, to estimate the multivariate posterior parameter PDF, which also provides estimates of maximum a posteriori (MAP; i.e., the most probable) parameter values and attendant uncertainty. MCMC methods employ a modified random walk (chain) through the multidimensional parameter space, with proposal steps in the random walk either accepted or rejected depending on the ratio of posterior probabilities of the previous parameter values and the proposed parameter values. The behavior of AM is similar to that of the basic Metropolis sampler (Metropolis et al. 1953), described in detail elsewhere (Mosegaard and Tarantola 1995; MacKay 2005), except that the sampler’s proposal covariance is continuously updated using the covariance of samples in the chain. This process ensures that the proposal covariance (which controls the random walk step size) is optimal for the underlying probability density under consideration. Because of this adaptive characteristic, the sampler’s proposal covariance depends on all previous steps, and is not, strictly speaking, a Markov chain. In practice, AM has all other pertinent qualities of the Metropolis MCMC sampler, and so we refer to it, slightly inaccurately, as an MCMC method.
The prior for all parameters is a bounded uniform distribution with bounds chosen to enclose a majority of the probability mass, except in cases where further parameter perturbation has little or no influence on BOSS behavior (e.g., if the posterior distribution has constant probability over the range of a parameter). The bounded uniform distribution is defined over the space of log10(a) and linear β and δ parameters; a parameters are sampled in log space because a power-law P = aMb becomes a simple linear function with parameters log(a) and b upon application of the log transform.
The likelihood defines all sources of uncertainty not present in the prior, and should ideally include
observational uncertainties (e.g., instrument noise or calibration uncertainties),
uncertainties in forward simulation of observable quantities from the model simulation, and
structural uncertainties in the model or forward simulator formulations.
Typically only (i) is quantified, and to the extent that (ii) and (iii) remain unquantified, posterior distributions of forward observations
3. Experimental design
As briefly discussed in the introduction, we present three different experiment types to serve as proof of concept of the probabilistic constraint of BOSS using observations within a Bayesian inference framework. By using idealized models, and a simplified testing framework, we for now ignore the difficulties in using real observations to inform a microphysics scheme [such as model initial condition, boundary condition, and representativeness errors, e.g., as described in Hodyss and Nichols (2015)]. We instead look to highlight challenges in estimation of BOSS parameters in the ideal case where these issues do not exist, or have been adequately addressed. The evaluation of a new approach like BOSS benefits from a modest set of first tests via well-controlled experiments, and we therefore do not seek to address all aspects of the problem of estimating microphysics from observations here.
We expect that a single rainshaft case will be insufficient to constrain BOSS, as not all microphysical processes are likely to exert a dominant and observable influence on evolution of the DSD in a single case (Kumjian and Prat 2014). To determine the number of cases sufficient to constrain BOSS, we use the criterion that independent choices of column model boundary and thermodynamic conditions should produce equivalent estimates of BOSS parameters. Simply put, we posit that best-estimate BOSS parameter spread from independent choices of boundary and thermodynamic conditions should not be greater than the spread described by the posterior
Figure 2 shows these statistics for the default number of cases (40) and default choice of observational error
Statistics of convergence of BOSS parameter PDFs under variation of cases selected for three choices of observational error standard deviation and number of cases used to constrain BOSS. Lighter bars show standard deviation of MAP and parameter PDF average values, respectively, for different independent choices of case initial conditions. Dark bars show average standard deviation of parameter PDFs for each independent choice of case initial conditions. Shown are statistics (top) for the default values of observational error standard deviation and number of constraining cases, (middle) with reduced observational error, and (bottom) reduced standard deviation.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Statistics of convergence of BOSS parameter PDFs under variation of cases selected for three choices of observational error standard deviation and number of cases used to constrain BOSS. Lighter bars show standard deviation of MAP and parameter PDF average values, respectively, for different independent choices of case initial conditions. Dark bars show average standard deviation of parameter PDFs for each independent choice of case initial conditions. Shown are statistics (top) for the default values of observational error standard deviation and number of constraining cases, (middle) with reduced observational error, and (bottom) reduced standard deviation.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Statistics of convergence of BOSS parameter PDFs under variation of cases selected for three choices of observational error standard deviation and number of cases used to constrain BOSS. Lighter bars show standard deviation of MAP and parameter PDF average values, respectively, for different independent choices of case initial conditions. Dark bars show average standard deviation of parameter PDFs for each independent choice of case initial conditions. Shown are statistics (top) for the default values of observational error standard deviation and number of constraining cases, (middle) with reduced observational error, and (bottom) reduced standard deviation.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
a. Known-truth experiment
We first test BOSS for a case where there is no structural uncertainty—where the constraining observations are themselves generated from a BOSS model realization. This is deemed a “known-truth” experiment because the true sensitivities of observed microphysical behavior should in principle be reproduced with correct estimation of parameter values. In this case MCMC should accurately find the true parameter values—that is, MAP parameter values should match true parameter values—and uncertainty should only be a product of the nonlinear inverse mapping of observational uncertainty onto the parameter space (see, e.g., Vukicevic and Posselt 2008). For these experiments, observational uncertainty is set arbitrarily low (half of default values) to minimize effects of marginal integration in interpretation of the parameter PDFs. We test the simplest implementation of our model, the one-term two-moment BOSS, with M0 and M3 treated as prognostic (M0-M3-B). The truth parameters used to generate the observations are set to MAP values from constraint by MORR described in the experiment below; in other words, they optimally emulate MORR behavior.
b. BOSS model, MORR truth
In the second set of experiments, we consider cases where there is some degree of structural error owing to a mismatch between the “truth” and model microphysics. The performance of configurations of BOSS with various degrees of structural complexity is explored, with all parameters constrained using synthetic observations drawn from rain-shaft simulations using MORR. We compare the performance of three configurations of BOSS: one-term two-moment BOSS with prognostic M0 and M3 (M0-M3-B), two-term two-moment BOSS with prognostic M0 and M3 (M0-M3-B2), and one-term three-moment BOSS with prognostic M0, M3, and M6 (M0-M3-M6-B). In each case, observations of prognostic microphysical variables (M0 and M3 for M0-M3-B and M0-M3-B2, M6 additionally for M0-M3-M6-B) are used at 15 levels in the simulated rain-shaft. The two-term configuration of BOSS (M0-M3-B2) has two terms only for evaporation processes to reduce the number of free parameters, and to reflect the fact that evaporation is described in MORR via two distinct terms, with the second describing ventilation effects on evaporation (see the appendix in Part I).
After the BOSS posterior parameter PDFs have been estimated using MCMC, one thousand parameter samples are chosen from these PDFs, and BOSS and MORR are run again for 10 cases whose initial conditions have been drawn from the climatological database of M0, M3, M6, and RH. This is done so that BOSS can be evaluated for cases that are independent of those that were used to constrain BOSS parameters. BOSS prognostic moments at all levels are output for comparison to MORR, and profiles of microphysical process rates are also output from both BOSS and MORR. As our goal is for BOSS to “learn” correct microphysical process rates from observational constraint, this comparison provides information on the feasibility of such constraint. Finally, these one thousand parameter samples are run for 100 cases to generate more thorough summary statistics of BOSS process rate error relative to MORR.
c. Two-term known truth
Ideally, if the constraining observations are generated via (8) with L = 2, then estimation should be improved when the model being fit—also using (8)—has L = 2. We test this by using MCMC to estimate parameters for various choices of “true” parameter values (i.e., the ones used to generate observations), and various choices of the model being fit [i.e., varying L in (8)]. We consider three cases where the true model is two term: one where the functional response P is positive definite and monotonic over the values of M considered, another where P is positive definite and nonmonotonic, and a third where the functional response is not positive definite or negative definite, but is monotonic. In each case we consider underfitting (model L = 1), perfect fit (model L = 2) and overfitting (model L = 3). We show posterior parameter PDFs and values of the modeled and true functional response P (see section 4c).
4. Results
a. Known-truth experiments
Parameter PDFs of the known-truth experiments (M0-M3-B constrained by observations of M0-M3-B) are shown in Fig. 3. In the case of all parameter values, the MAP solution (in blue) is nearly identical to the true parameter values (in red). Further, the peak of the marginal distribution is also nearly identical to the true and MAP parameter values. These results indicate that MCMC is able to unambiguously recover the true parameter values when the structure of the constrained model matches the true model.
Posterior parameter PDFs from the known-truth experiment with two-moment (M0, M3), one-term BOSS. “True” parameter values are shown in red, and MAP parameter values from MCMC are shown in blue (where only blue is visible, crosshairs overlap). The parameter PDFs are joint 2D marginals for the parameters as indicated, and 1D marginals along the diagonal.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Posterior parameter PDFs from the known-truth experiment with two-moment (M0, M3), one-term BOSS. “True” parameter values are shown in red, and MAP parameter values from MCMC are shown in blue (where only blue is visible, crosshairs overlap). The parameter PDFs are joint 2D marginals for the parameters as indicated, and 1D marginals along the diagonal.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Posterior parameter PDFs from the known-truth experiment with two-moment (M0, M3), one-term BOSS. “True” parameter values are shown in red, and MAP parameter values from MCMC are shown in blue (where only blue is visible, crosshairs overlap). The parameter PDFs are joint 2D marginals for the parameters as indicated, and 1D marginals along the diagonal.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Correlated uncertainty between parameters occurs when there is some possibility for compensating errors in parameter values. This is evident in Fig. 3 between matching pairs of log(a) and β or δ parameters—likely owing to the linear relationship between log(a) and β or δ parameters in the BOSS process rate equations. Other correlations can be explained by physically plausible mechanisms for compensation of errors. For example, the weak positive correlations between evaporation parameters ]most noticeably log(aM0,evap) and βM0,evap] and fall speed parameters [especially log(aM0,fall) and βM0,fall] can be explained by noting that a weaker evaporation rate coupled with a slower fall speed can produce similar vertical profiles of DSD moments as stronger evaporation and faster fall speed. Also noticeable is a negative correlation between coalescence and breakup parameters. These processes have opposing effects, with coalescence decreasing M0 and breakup increasing M0, and thus an underestimation of coalescence could potentially be offset by an underestimation of breakup. However, the correlation between respective a parameters is negative and is, thus, unexpected. It is possible that the correlation is a manifestation of coalescence and breakup being jointly correlated with a third process [analogous to the trivariate relationships explored in van Lier-Walqui et al. (2014, Fig. 4)], and in this context we note that breakup and coalescence parameters show some weak negative and positive correlations with log(aM0,fall) and βM0,evap. Correlations are also important because they indicate where reduction in uncertainty in one parameter or process is tied to reduction of uncertainty in other parameters.
b. BOSS model, MORR truth
As mentioned previously, results showed virtually identical performance between M0-M3-B and M0-M3-B2, despite the latter’s increase in structural complexity with two terms in the power expression for evaporation. In many cases, M0-M3-B2 performed notably worse than M0-M3-B from the perspective of MAP log likelihood. For this reason, results are not shown for M0-M3-B2, and a detailed exploration of the reasons for this behavior are discussed in section 4c. Here we focus solely on comparison between M0-M3-B and M0-M3-M6-B.
A basic measure of BOSS performance is shown in Fig. 4: the posterior distribution of −[f(x) − y]T
Posterior distribution of nonnormalized log-likelihood {−[f(x) − y]T
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Posterior distribution of nonnormalized log-likelihood {−[f(x) − y]T
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Posterior distribution of nonnormalized log-likelihood {−[f(x) − y]T
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
The parameter PDFs, shown in Fig. 5, are an MCMC estimate of the probability
Posterior parameter PDFs for M0-M3-B, constrained by MORR. MAP parameter values are indicated by blue crosshairs. The parameter PDFs are joint 2D marginals for the parameters as indicated, and 1D marginals along the diagonal.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Posterior parameter PDFs for M0-M3-B, constrained by MORR. MAP parameter values are indicated by blue crosshairs. The parameter PDFs are joint 2D marginals for the parameters as indicated, and 1D marginals along the diagonal.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Posterior parameter PDFs for M0-M3-B, constrained by MORR. MAP parameter values are indicated by blue crosshairs. The parameter PDFs are joint 2D marginals for the parameters as indicated, and 1D marginals along the diagonal.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
One thousand samples of parameter values are then randomly drawn from the posterior parameter PDFs to run 10 independent cases of 1000 simulations each—essentially these are a set of “evaluation” cases that are independent of the 40 “training” cases. Profiles of prognostic moment output for these cases are shown in Figs. 6 and 7 for M0-M3-B and M0-M3-M6-B (black lines), respectively, together with rain-rate histograms. These figures also show comparable profiles using MORR (red lines), and profiles using the MAP BOSS parameter values (blue lines). With a few exceptions, results indicate that M0-M3-M6-B has similar prediction of M0 compared with M0-M3-B. Some notable exceptions such as case 7 show the M0-M3-B ensemble more centered on the MORR profile than the M0-M3-M6-B ensemble, but with a poorer MAP match to MORR. In many instances, such as cases 1, 3, 7, and 9, there is more M0 uncertainty in M0-M3-M6-B profiles than for M0-M3-B. Overall, Fig. 4 suggests generally better performance for M0-M3-M6-B, but the greater scheme degrees of freedom compared with M0-M3-B may be the cause of increased uncertainty in certain moments for certain cases.
Profiles of prognostic moments (M0, M3) and rain-rate histograms, for 10 cases where the model is run using BOSS with 1000 parameter values sampled from the M0-M3-B PDF (black), using BOSS with the M0-M3-B MAP parameter values (blue), and using MORR (red). Specified upper boundary conditions sampled for each case are shown by text in the rain-rate histogram plots.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Profiles of prognostic moments (M0, M3) and rain-rate histograms, for 10 cases where the model is run using BOSS with 1000 parameter values sampled from the M0-M3-B PDF (black), using BOSS with the M0-M3-B MAP parameter values (blue), and using MORR (red). Specified upper boundary conditions sampled for each case are shown by text in the rain-rate histogram plots.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Profiles of prognostic moments (M0, M3) and rain-rate histograms, for 10 cases where the model is run using BOSS with 1000 parameter values sampled from the M0-M3-B PDF (black), using BOSS with the M0-M3-B MAP parameter values (blue), and using MORR (red). Specified upper boundary conditions sampled for each case are shown by text in the rain-rate histogram plots.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 6, but for M0, M3, and M6 using the three-moment M0-M3-M6-B. The y axes are identical to Fig. 6, but the x-axis ranges vary.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 6, but for M0, M3, and M6 using the three-moment M0-M3-M6-B. The y axes are identical to Fig. 6, but the x-axis ranges vary.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 6, but for M0, M3, and M6 using the three-moment M0-M3-M6-B. The y axes are identical to Fig. 6, but the x-axis ranges vary.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Regarding M3 profiles, there is some evidence, such as in cases 2, 4, and 6, of M0-M3-M6-B outperforming M0-M3-B, with the case 2 profile a particularly notable example where the entire M0-M3-B ensemble and its MAP value underestimates the MORR M3 profile. This is also clearly visible in the underestimation of rain rate for case 4 for M0-M3-B; M0-M3-M6-B has a relatively unbiased estimate of rain rate for this case. For M0-M3-M6-B, MAP profiles of M6 are generally good estimates of MORR M6, although in many cases (e.g., cases 1, 3, and 9) the ensemble displays considerable bias in its mean relative to MORR. This bias in ensemble mean indicates that BOSS parameter uncertainty tends to produce consistently smaller M6 values for these simulations. However, the fact that the MAP profile is a good approximation of MORR indicates that BOSS has the flexibility to reproduce well MORR’s evolution of DSD moment profiles. More generally, multidimensional distributions that are non-Gaussian may have marginal distributions with modes that differ from those of the PDF in its full “native” dimensions—see Vukicevic and Posselt (2008) for illustrative examples of this phenomenon.
Figures 8 and 9 are generated similarly to Figs. 6 and 7, and show profiles of microphysical process rates from the 10 validation cases using M0-M3-B and M0-M3-M6-B, respectively. All microphysical process rates are output at every level in the model column from both BOSS and MORR. The processes of collision–coalescence and collisional breakup are combined (coalescence–breakup) to match how these processes are treated within MORR. However, negative and positive values of coalescence–breakup are shown separately to distinguish between breakup- and coalescence-dominated cases, respectively. This also allows for displaying the rates in dB, which more adequately shows the full range of values simulated (which can span multiple orders of magnitude between column top and column bottom). Values of coalescence and breakup rates for M6 are multiplied by −1 so that positive values of coalescence–breakup for M6 match with positive values of coalescence–breakup for M0. Moment fall speeds are presented in meters per second.
As in Fig. 6, but for process rate output from each respective simulation.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 6, but for process rate output from each respective simulation.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 6, but for process rate output from each respective simulation.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 8, but for process rate profile output from each respective simulation. The y axes are identical to Fig. 8, but the x-axis ranges vary.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 8, but for process rate profile output from each respective simulation. The y axes are identical to Fig. 8, but the x-axis ranges vary.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 8, but for process rate profile output from each respective simulation. The y axes are identical to Fig. 8, but the x-axis ranges vary.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
In effect, Figs. 8 and 9 demonstrate the degree to which BOSS has “learned” microphysical process rates via Bayesian constraint from bulk rain observations. We note that for both M0-M3-B and M0-M3-M6-B, BOSS process rates are generally constrained to within 1–2 dB of MORR. Furthermore, the shape of the MORR process rates profiles are well captured by BOSS process rates—where process rates increase/decrease with height in MORR, they tend to do so in BOSS as well. The transition between breakup- and coalescence-dominated regions of the profile, shown in the balance between profiles with positive and negative coalescence–breakup, is also well represented in BOSS when it exists for MORR (e.g., cases 3, 4, and 9), with some uncertainty in the height at which this transition occurs. For example, for M0-M3-B case 4 this transition occurs about 400 m too low in the BOSS MAP profile, but is much better estimated in case 3. We note that in cases where process rates approach 0, strongly underestimated values may still be insignificant, because the scale shown is in log units (i.e., a 100-fold underestimation of an insignificantly small number is still an insignificantly small number). This, in part, explains the increase in BOSS spread for large negative dB values of M0coal shown in Fig. 8 for cases 3, 4, and 9.
Comparison of the results from M0-M3-B, shown in Fig. 8, and from M0-M3-M6-B, shown in Fig. 9, displays tradeoffs in how each configuration of BOSS gains process-level information from the observational constraint. For example, while M0-M3-M6-B better estimates MORR profiles of VM3, most readily visible in cases 1, 4, 5, 6, and 10, M0-M3-B better characterizes M0coal and M0break, especially for cases in which the rainshaft transitions from coalescence-dominated evolution aloft to breakup dominated near the surface (cases 3, 4, and 9).
Figure 10 shows summary histograms of BOSS process rate error (dB) for all levels and all simulations. Here to produce more robust statistics we run 100 cases initialized from the same database as before. In virtually all cases, the PDFs of error are nearly symmetric about 0, indicating that estimation of process rate is, in the mean, mostly unbiased. Furthermore, with the exception of M6coal–M6break and M0coal–M0break, process rate error is mostly less than 2 dB for all processes. Comparing results from M0-M3-B and M0-M3-M6-B, we find virtually identical results for M3evap, a slight advantage for M0-M3-B in M0evap, VM0, and VM3, and no clearly better choice for M0coal–M0break; these process error distributions have generally equal probability of 0-dB error, but differ in mean error and error standard deviation between M0-M3-B and M0-M3-M6-B. We note that this error distribution is effectively the sum of two errors (from coalescence and breakup separately) and thus its variance is the sum of each individual variance.
Process rate error (dB) of BOSS simulations relative to MORR, aggregated for all levels in the column for 100 simulations, which include the 10 simulations shown in Figs. 8 and 9, and 90 additional, independently generated simulations. M0-M3-B and M0-M3-M6-B process rate error histograms are shown in red and gray, respectively. The y axes are consistent across rows.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Process rate error (dB) of BOSS simulations relative to MORR, aggregated for all levels in the column for 100 simulations, which include the 10 simulations shown in Figs. 8 and 9, and 90 additional, independently generated simulations. M0-M3-B and M0-M3-M6-B process rate error histograms are shown in red and gray, respectively. The y axes are consistent across rows.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Process rate error (dB) of BOSS simulations relative to MORR, aggregated for all levels in the column for 100 simulations, which include the 10 simulations shown in Figs. 8 and 9, and 90 additional, independently generated simulations. M0-M3-B and M0-M3-M6-B process rate error histograms are shown in red and gray, respectively. The y axes are consistent across rows.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
c. Two-term known truth
As discussed previously, tests using M0-M3-B2 generally showed no improvement over M0-M3-B, despite the increased degrees of freedom with which to fit BOSS behavior toward MORR. Furthermore, owing to strong non-Gaussianity of the M0-M3-B2 parameter PDF, the performance of the MCMC sampler was considerably reduced, which often resulted in worse performance of M0-M3-B2 relative to M0-M3-B. This was true even when M0-M3-B2 was used to generate the constraining observations—that is to say, even when the truth was generated using a two-term power law for evaporation, a one-term model often provided a better fit (results not shown here for brevity). Upon further investigation, this issue was found to be emblematic of a certain class of problems where parameters of generalized power expressions are estimated. This is illustrated by examining the model response P as a function of independent variable M following the idealization of the BOSS process rate power expressions in (8). We show one case that is representative of the problems encountered in estimating M0-M3-B2 parameters, as well as two contrasting cases where unambiguous estimation of parameters for a two-term power expression is straightforward.
Results for the three cases are shown in Figs. 11–13. The first, Fig. 11, shows the case where the true solution is monotonic and positive definite. The top plot shows the true solution (green) as well as the contribution of each term in the two-term power law (orange and blue). The three rows below show the results of Bayesian parameter estimation using one-term (underfit), two-term (perfect fit), and three-term (overfit) models. For this case, the one-term model shows an approximately Gaussian bivariate PDF for the parameters, and a very good match between the solution of this model and truth. The two- and three-term models also produce good matches to the true solution. However, their respective parameter PDFs are strongly non-Gaussian. In this case, a one-term solution provides a very good approximation of two-term truth while maintaining a well-behaved (e.g., Gaussian) posterior distribution. Strong non-Gaussianity observed for the two- and three-term solutions could result in difficulty in estimating those parameters. This case is similar to that found exploring two terms in BOSS—a one-term solution performed adequately and two-term parameter PDFs were found to be strongly non-Gaussian.
Tests of estimation of parameters of two- and one-term power-law models for the case in which the true power law is a monotonically increasing two-term power law. (top) In the full two-term “truth,” the blue and red lines representing each power-law term comprising the full solution shown by the green line. Values of the “truth” parameters are to the left of this plot. (bottom-right column) Results using the one-term, two-term, and three-term models for 100 forward calculations using parameters sampled from the posterior PDFs (blue lines) and the “truth” (red lines). (bottom-left column) Posterior parameter PDFs for the one-term, two-term, and three-term models.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Tests of estimation of parameters of two- and one-term power-law models for the case in which the true power law is a monotonically increasing two-term power law. (top) In the full two-term “truth,” the blue and red lines representing each power-law term comprising the full solution shown by the green line. Values of the “truth” parameters are to the left of this plot. (bottom-right column) Results using the one-term, two-term, and three-term models for 100 forward calculations using parameters sampled from the posterior PDFs (blue lines) and the “truth” (red lines). (bottom-left column) Posterior parameter PDFs for the one-term, two-term, and three-term models.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Tests of estimation of parameters of two- and one-term power-law models for the case in which the true power law is a monotonically increasing two-term power law. (top) In the full two-term “truth,” the blue and red lines representing each power-law term comprising the full solution shown by the green line. Values of the “truth” parameters are to the left of this plot. (bottom-right column) Results using the one-term, two-term, and three-term models for 100 forward calculations using parameters sampled from the posterior PDFs (blue lines) and the “truth” (red lines). (bottom-left column) Posterior parameter PDFs for the one-term, two-term, and three-term models.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 11, but for the case that the true power law is a nonmonotonic two-term power law.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 11, but for the case that the true power law is a nonmonotonic two-term power law.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Fig. 11, but for the case that the true power law is a nonmonotonic two-term power law.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Figs. 11 and 12, but for the case that the true power law is a monotonic but not positive-definite two-term power law.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Figs. 11 and 12, but for the case that the true power law is a monotonic but not positive-definite two-term power law.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
As in Figs. 11 and 12, but for the case that the true power law is a monotonic but not positive-definite two-term power law.
Citation: Journal of the Atmospheric Sciences 77, 3; 10.1175/JAS-D-19-0071.1
Truth in the second case (Fig. 12) is marked by a nonmonotonic relationship between the response P and changes in the independent variable M. Again, for this case, the one-term model’s parameter PDF is nearly Gaussian, but its solution is a poor approximation of truth—a single power law inherently cannot reproduce this nonmonotonic function. The parameter PDF of the two-term model now displays only mild nonlinearities, with nearly Gaussian one-dimensional marginal distributions; this two-term model produces solutions that closely match truth. The three-term model again displays strongly non-Gaussian parameter PDFs, but its solutions closely approximate truth. In this case, where the two-term model was necessary to approximate the two-term truth, the posterior PDF was closer to Gaussianity than in the first case.
Truth in the third case (Fig. 13) is a monotonic two-term function, but not positive definite (nor negative definite, i.e., it changes sign with M). Again, the one-term model has a Gaussian parameter PDF and solutions that poorly approximate truth, as it inherently cannot reproduce solutions that change sign. The parameter PDF of the two-term model is again moderately non-Gaussian, but generally well behaved (e.g., unimodal), and its solutions are a good approximation of truth. The three-term model again has non-Gaussian parameter PDFs, and solutions that well approximate truth. We note, but do not show here, that this case can also be well described by estimating a1, b1, and a2 with b2 set to 0, thus reducing the number of free parameters from four to three. To summarize, for this third class of two-term power laws, a model with at least two terms (and at least three or four free parameters) is necessary to reasonably approximate truth.
The first case (Fig. 11, monotonic and positive definite) is quite similar to most microphysical process rates, which we typically would expect to vary monotonically with moment value. For example, an increase in the bulk droplet concentration while maintaining the same DSD shape will increase the value of all moments (i.e., all moment orders), and all else equal bulk process rates will be larger because they are operating on more drops. We therefore expect that for many of these processes (though not all, particularly for higher-order moments), a single power-law term may be sufficient to capture natural behavior. As noted in Part I, the final case shown here (monotonic, not positive definite or negative definite) can be thought of as analogous to the combined processes of collision–coalescence and breakup within BOSS; each process is governed by a separate power law, but has opposite effects on the evolution of DSD moments. Effectively, the combined coalescence–breakup process is not positive definite or negative definite, and thus must be represented by at least two terms, because it is negative in coalescence-dominated regimes and positive in breakup-dominated regimes for the change in Mk when k < 3 (vice versa for k > 3).
5. Discussion and conclusions
A novel rain microphysics scheme, the Bayesian Observationally Constrained Statistical–Physical Scheme (BOSS; introduced in Part I), was constrained within a one-dimensional idealized rain-shaft model using synthetic observations that were generated by a “traditional” three-moment bulk microphysics scheme. Different configurations of BOSS with varying degrees and types of structural complexity were compared to ascertain the possible challenges and potential of developing a microphysics scheme informed almost entirely by observations. All constraint was performed using an MCMC-like Bayesian parameter estimation algorithm that also produced robust estimates of parameter uncertainty.
Results showed that, in general, observations were capable of informing BOSS parameters without significant nonuniqueness in the solutions. Correlated uncertainty between BOSS parameters indicated the possibility for compensating errors in process rates, but generally these effects were weak compared with correlations between log(a) and β and δ parameters. Correlation can suggest additional observations that could be particularly useful for constraining BOSS. For example, correlation between uncertainty in evaporation and sedimentation parameters indicates that estimation of both sedimentation and evaporation parameters would benefit by better constraint of the sedimentation rate, for example, via direct measurements of Doppler fall speed by vertically pointing radars.
It was also noted that collision–coalescence and collisional breakup parameters were somewhat worse constrained than sedimentation and evaporation parameters. We noted that the error variance is effectively the sum of the individual variances of coalescence and breakup. However, this only partly explains the greater uncertainty in coalescence/breakup versus evaporation. An explanation for this increased uncertainty is that, in the case of M0-M3-B, observations of M3 inform only two processes, evaporation of M3 and sedimentation of M3, whereas observations of M0 informed four: evaporation, sedimentation, collision–coalescence, and breakup. Collision–coalescence and breakup were therefore more poorly constrained than, say, evaporation of M3. This underscores the need for strong observational information relating to the evolution of moments of interest, in particular those that are evolved by more parameterized physical processes. These observational needs suggest that M6 might provide a better companion to M3 in a two-moment scheme, as radar reflectivity and reflectivity-weighted fall speed are routinely observed by scanning and profiling radar systems, respectively. Another reason for the poorer constraint of collision–coalescence and breakup parameters is the fact that most cases used for observationally constraining BOSS were more strongly controlled by evaporation—Fig. 8 showed that the combined rates of coalescence/breakup were much lower than that of evaporation for at least six of the cases considered (cf. M0evap and M0coal for cases 1, 2, 5, 6, 8, and 10). As such, model evolution was relatively less sensitive to perturbations in coalescence/breakup parameters, resulting in weaker constraint. Hypothetically, this uncertainty could be reduced by observationally constraining using more cases where coalescence and/or breakup rates are at least as large as evaporation rates. Future work will explore the diversity of realistic observations needed to inform each process, something beyond the scope of the current research.
One of the key features of BOSS relative to other schemes is its flexibility in structural complexity; both the number of prognostic moments, choice of prognostic moments, and the number of terms in the generalized power expressions for the process rates can be systematically altered to match observed microphysical complexity. Here, we considered two increases in complexity beyond that of the two-moment BOSS with single-term power-law expressions for the process rates (M0-M3-B): the addition of another prognostic moment (M6, in M0-M3-M6-B), and separately, the addition of another term in process rate expressions (M0-M3-B2). The addition of M6 as a prognostic moment improved overall scheme fidelity, in particular improving prediction of M3 and rain rate. However, estimation of microphysical process rates (for M0 and M3) was not significantly improved compared with M0-M3-B. This was likely a consequence of additional degrees of freedom without a commensurate increase in observational information; the number of free parameters more than doubled from 12 to 30, whereas the observation vector length increased by only 50%. In tests with an appropriately increased number of observations for M0-M3-M6-B (not shown), process errors for M0-M3-M6-B were, for all processes, found to be lower than for both M0-M3-B and M0-M3-M6-B with the default number of observations.
Results from tests adding structural complexity to BOSS by using two power-law terms to model evaporation had little to no positive impact on results, with some MAP estimates noticeably worse than using one power-law term, despite the fact that the true process rate in MORR was a two-term function of the prognostic moments. These results can be interpreted within the context of idealized tests using simplified power laws (section 4). For certain two-term power laws, which are monotonic and positive definite, a single power-law term was sufficient to adequately model the true function response, and the addition of further terms showed symptoms of overfitting (e.g., strongly non-Gaussian parameter PDFs). Other two-term power-law functions, namely, those that are either nonmonotonic or monotonic but not positive definite or negative definite, required a two-term model for adequate fit, and only showed symptoms of overfitting for three-term models. These three classes of two-term power laws are neither exhaustive nor necessarily stereotypes for the true functional form of microphysical process rates. Instead they give insight into where problems can arise with fitting power expression models, and yield some limited heuristics to guide interpretation of BOSS parameter estimation results. First, if a two-term model does not show obvious improvement over a one-term model, and the two-term model parameter PDFs are strongly non-Gaussian, then it is likely that the two-term model is overfitting to the truth even if truth is multiterm. If the true functional response is nonmonotonic or nonpositive (or nonnegative) definite, it is highly likely that at least two terms will be needed to simulate the truth, as is the case for the combined coalescence–breakup process. We leave open the possibility that there exist monotonic, positive- or negative-definite functions that require multiple power laws for adequate modeling of microphysical processes.
Future progress and development of BOSS will ultimately depend on the ability to formulate appropriate parameter inference problems. This involves three interlocking components: the observations used to constrain the model, the model that simulates the observed weather (of which BOSS is one component), and the Bayesian parameter estimation framework. There are crucial considerations over what sort of real observations may be necessary to provide adequate constraint on BOSS process representation. Additionally, the ability to retrieve microphysical information from observations will depend closely on the choice of model used to simulate weather. Here we used a steady-state rain-shaft model, and while there may be some weather phenomena amenable to such treatment (broad, spatially and temporally homogeneous stratiform regions of mesoscale convective systems, etc.), most convective rain will be poorly simulated by a steady-state model. Furthermore, observations from these convective systems may be necessary to constrain microphysical processes such as collision–coalescence and collisional breakup, which tend to be more active at higher rain rates. On the other hand, freely evolving (i.e., nonsteady state) simulations will suffer from error growth associated with both physical uncertainties as well as initial condition uncertainty, with strong coupling between the freely evolving microphysical, dynamical, and thermodynamical states. As such, these cases will likely require simultaneous state and parameter estimation in order to prevent state errors from rendering microphysical errors uninterpretable. No widely used system exists for such simultaneous parameter and state estimation, though candidates have been proposed (e.g., Laine et al. 2012). Finally, we note that observations should be capable of informing both the most probable parameter values, as well as choice of most probable model (or BOSS configuration); that is, they should be capable of informing not only parameter estimation experiments, but also model selection experiments. Bayesian techniques for model selection are commonly applied to hydrological (Höge et al. 2018) and seismological (Sambridge et al. 2013) modeling, but have not yet gained traction within the atmospheric sciences.
Although these issues will require concerted efforts to resolve, BOSS is uniquely suited to providing the microphysical modeling framework for these tasks, owing to the flexibility of its structure while retaining a physical framework. Thus, observationally driven development of BOSS also holds promise to yield meaningful physical insights into microphysics on the process level because BOSS retains a physically interpretable structure. In these respects, BOSS has advantages over both traditional microphysics schemes, as well as “black box” approaches to data-driven parameterization development.
Acknowledgments
This work was funded by U.S. DOE Atmospheric System Research Grant DE-SC0016579. The National Center for Atmospheric Research is sponsored by the National Science Foundation. MM and SL acknowledge support by the National Science Foundation under Grant DMS-1619630. MM gratefully acknowledges support by the Office of Naval Research (Grant N00173-17-2-C003).
REFERENCES
Berner, J., S.-Y. Ha, J. P. Hacker, A. Fournier, and C. Sn, 2011: Uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 1972–1995, https://doi.org/10.1175/2010MWR3595.1.
Berner, J., K. R. Fossell, S.-Y. Ha, J. P. Hacker, and C. Snyder, 2015: Increasing the skill of probabilistic forecasts: Understanding performance improvements from model-error representations. Mon. Wea. Rev., 143, 1295–1320, https://doi.org/10.1175/MWR-D-14-00091.1.
Diamond, H. J., and Coauthors, 2013: U.S. Climate Reference Network after one decade of operations: Status and assessment. Bull. Amer. Meteor. Soc., 94, 485–498, https://doi.org/10.1175/BAMS-D-12-00170.1.
Haario, H., E. Saksman, and J. Tamminen, 2001: An adaptive Metropolis algorithm. Bernoulli, 7, 223–242, https://doi.org/10.2307/3318737.
Hodyss, D., and N. Nichols, 2015: The error of representation: Basic understanding. Tellus, 67A, 24822, https://doi.org/10.3402/tellusa.v67.24822.
Höge, M., T. Wöhling, and W. Nowak, 2018: A primer for model selection: The decisive role of model complexity. Water Resour. Res., 54, 1688–1715, https://doi.org/10.1002/2017WR021902.
Jankov, I., and Coauthors, 2017: A performance comparison between multiphysics and stochastic approaches within a North American RAP ensemble. Mon. Wea. Rev., 145, 1161–1179, https://doi.org/10.1175/MWR-D-16-0160.1.
Kumjian, M. R., and O. P. Prat, 2014: The impact of raindrop collisional processes on the polarimetric radar variables. J. Atmos. Sci., 71, 3052–3067, https://doi.org/10.1175/JAS-D-13-0357.1.
Kumjian, M. R., C. P. Martinkus, O. P. Prat, S. Collis, M. van Lier-Walqui, and H. C. Morrison, 2019: A moment-based polarimetric radar forward operator for rain microphysics. J. Appl. Meteor. Climatol., 58, 113–130, https://doi.org/10.1175/JAMC-D-18-0121.1.
Laine, M., A. Solonen, H. Haario, and H. Järvinen, 2012: Ensemble prediction and parameter estimation system: The method. Quart. J. Roy. Meteor. Soc., 138, 289–297, https://doi.org/10.1002/QJ.922.
Leeper, R. D., J. Rennie, and M. A. Palecki, 2015: Observational perspectives from U.S. Climate Reference Network (USCRN) and Cooperative Observer Program (COOP) network: Temperature and precipitation comparison. J. Atmos. Oceanic Technol., 32, 703–721, https://doi.org/10.1175/JTECH-D-14-00172.1.
MacKay, D. J. C., 2005: Information Theory, Inference, and Learning Algorithms. 7th ed. Cambridge University Press, 628 pp.
Metropolis, N., and Coauthors, 1953: Equation of state calculations by fast computing machines. J. Chem. Phys., 21, 1087–1092, https://doi.org/10.1063/1.1699114.
Moiseev, D. N., S. Lautaportti, J. Tyynela, and S. Lim, 2015: Dual-polarization radar signatures in snowstorms: Role of snowflake aggregation. J. Geophys. Res. Atmos., 120, 12 644–12 655, https://doi.org/10.1002/2015JD023884.
Moiseev, D. N., A. von Lerber, and J. Tiira, 2017: Quantifying the effect of riming on snowfall using ground-based observations. J. Geophys. Res. Atmos., 122, 4019–4037, https://doi.org/10.1002/2016JD026272.
Morrison, H., and W. W. Grabowski, 2007: Comparison of bulk and bin warm-rain microphysics models using a kinematic framework. J. Atmos. Sci., 64, 2839–2861, https://doi.org/10.1175/JAS3980.
Morrison, H., G. Thompson, and V. Tatarskii, 2009: Impact of cloud microphysics on the development of trailing stratiform precipitation in a simulated squall line: Comparison of one- and two-moment schemes. Mon. Wea. Rev., 137, 991–1007, https://doi.org/10.1175/2008MWR2556.1.
Morrison, H., M. R. Kumjian, C. P. Martinkus, O. P. Prat, and M. van Lier-Walqui, 2019: A general N-moment normalization method for deriving raindrop size distribution scaling relationships. J. Appl. Meteor. Climatol., 58, 247–267, https://doi.org/10.1175/JAMC-D-18-0060.1.
Morrison, H., M. van Lier-Walqui, M. R. Kumjian, and O. P. Prat, 2020: A Bayesian approach for statistical–physical bulk parameterization of rain microphysics. Part I: Scheme description. J. Atmos. Sci., 77, 1019–1041, https://doi.org/10.1175/JAS-D-19-0070.1.
Mosegaard, K., and A. Tarantola, 1995: Monte Carlo sampling of solutions to inverse problems. J. Geophys. Res., 100, 12 431–12 447, https://doi.org/10.1029/94JB03097.
Posselt, D. J., and T. Vukicevic, 2010: Robust characterization of model physics uncertainty for simulations of deep moist convection. Mon. Wea. Rev., 138, 1513–1535, https://doi.org/10.1175/2009MWR3094.1.
Sambridge, M., T. Bodin, K. Gallagher, and H. Tkalčić, 2013: Transdimensional inference in the geosciences. Philos. Trans. Roy. Soc., 371A, 20110547, https://doi.org/10.1098/rsta.2011.0547.
Schrom, R. S., and M. R. Kumjian, 2018: Bulk-density representations of branched planar ice crystals: Errors in the polarimetric radar variables. J. Appl. Meteor. Climatol., 57, 333–346, https://doi.org/10.1175/JAMC-D-17-0114.1.
Shipway, B. J., and A. A. Hill, 2012: Diagnosis of systematic differences between multiple parametrizations of warm rain microphysics using a kinematic framework. Quart. J. Roy. Meteor. Soc., 138, 2196–2211, https://doi.org/10.1002/qj.1913.
Tridon, F., A. Battaglia, and D. Watters, 2017: Evaporation in action sensed by multiwavelength Doppler radars. J. Geophys. Res. Atmos., 122, 9379–9390, https://doi.org/10.1002/2016JD025998.
van Lier-Walqui, M., T. Vukicevic, and D. J. Posselt, 2014: Linearization of microphysical parameterization uncertainty using multiplicative process perturbation parameters. Mon. Wea. Rev., 142, 401–413, https://doi.org/10.1175/MWR-D-13-00076.1.
Vukicevic, T., and D. Posselt, 2008: Analysis of the impact of model nonlinearities in inverse problem solving. J. Atmos. Sci., 65, 2803–2823, https://doi.org/10.1175/2008JAS2534.1.
Williams, C. R., 2016: Reflectivity and liquid water content vertical decomposition diagrams to diagnose vertical evolution of raindrop size distributions. J. Atmos. Oceanic Technol., 33, 579–595, https://doi.org/10.1175/JTECH-D-15-0208.1.
