## 1. Introduction

Climate change is a defining problem of our time. It is hard to plan for future warming without knowing its magnitude, but our ±1*σ* “likely” confidence range for equilibrium climate sensitivity (ECS; the global-average surface warming due to doubling CO_{2} and letting the climate re-equilibrate) is currently 1.5–4.5 K (IPCC 2013)—which is disturbingly large. This uncertainty has persisted for decades despite large advances in our understanding of the climate system (Knutti et al. 2017).

Emergent constraints offer a possible path to narrowing this spread. An emergent constraint is a current-climate quantity that has skill at predicting future changes in climate. Such predictors may be valuable shortcuts to the complex and uncertain process of directly simulating climate change in a general circulation model (GCM) or inferring it from imperfect observational records. Because the physical processes governing climate change are generally the same ones that control present-day seasonal, weather-scale, and diurnal variations, it is likely that real emergent constraints exist. Hall and Qu (2006) was one of the first papers to identify such a constraint. They found that the seasonal cycle of snow albedo over Northern Hemisphere land is tightly correlated with snow albedo feedback over this region in 17 model simulations from phase 3 of the Coupled Model Intercomparison Project (CMIP3). This emergent constraint has an obvious motivation: surface warming reduces snow cover irrespective of whether that warming is due to seasonal changes in insolation or CO_{2}-induced climate change. Nearly 40 other emergent constraints have been proposed since 2006 (Hall et al. 2019), although few have had such a satisfying physical explanation.

Several limitations and assumptions apply to the use of emergent constraints to predict ECS. First, the ECS simulated by a climate model is generally estimated from an integration of finite length (customarily 150 years in CMIP5) that is not fully equilibrated and keeps certain physics fixed (e.g., vegetation type and land ice). The response of that model (or the real climate system) to a time-varying radiative forcing is not determined purely by the ECS, but also depends upon natural and forced changes in the pattern of surface warming (e.g., Armour et al. 2013; Gregory and Andrews 2016). ECS is also a problematic target for emergent constraints because it arises from interaction between many processes. As a result, it is questionable whether any single current-climate variable would explain a large fraction of ECS variability. This is why Klein and Hall (2015) suggest that emergent constraints should be targeted toward a single climate feedback mechanism (e.g., snow cover) whenever possible. Nevertheless, many studies (including this one) focus on emergent constraints for global climate sensitivity proxies such as ECS or the climate feedback parameter *λ* (Cess et al. 1989) because of their importance. Last, emergent constraints in general derive from a blend of scientific reasoning and a posteriori optimization to maximize their correlation with ECS or *λ* over a modest set of GCMs, and only the most promising constraints are likely to be published. This suggests a risk of constraints being “overconfident”—that is, better correlated with ECS or *λ* over the GCMs on which they were first tested and optimized than in another independent set of GCMs. A statistical analysis by Caldwell et al. (2014) showed that a posteriori data mining of the limited set of GCMs in the CMIP5 archive could easily yield apparent constraints with a strong sample correlation to ECS despite there being no true correlation.

Emergent constraints have already been noted to predict larger climate sensitivity than expected from other lines of evidence (Tian 2015; Klein and Hall 2015). If agreement between constraints gives us confidence in their predictions, this is an alarming finding. A goal of this paper is to develop an approach for combining emergent constraints to provide a confidence range for ECS, or any other climate sensitivity proxy, while accounting for the issues just raised.

A key idealization that we make is that the climate proxy and the constraints have a multivariate Gaussian distribution. This idealization is not perfect. Both ECS and several of the emergent constraints that we use marginally fail statistical tests of normality, as we will describe in section 2b. This should make us wary about the tails of any estimated probability density function (PDF), which are most sensitive to the Gaussian assumption. We will use our theory to estimate Gaussian PDFs for ECS and also for its approximate inverse *λ*, the climate sensitivity parameter. Those PDFs cannot be fully consistent with each other because if *λ* is Gaussian then ECS is not. This is most problematic if these PDFs encompass a broad range of climate sensitivities, which ours do not. We have chosen to highlight ECS because it has been a popular target for emergent constraints. It might be more plausible to assume that *λ*, which can be composed as the sum of physical feedbacks, has a Gaussian distribution; that will yield a PDF of ECS with a longer positive tail. Readers should be aware of the inevitable trade-off between the simplicity of our approach and the limitations encompassed by the assumptions that go into it.

In mathematical terms, our approach is a straightforward application of the theory of multivariate Gaussian distributions developed by statisticians many decades ago. This theory may be unfamiliar to most climate scientists, but our method ultimately reduces to an application of multiple linear regression.

The emergent constraints used and the relevant data are described in section 2, and terminology is described in section 3. In section 4, we derive and apply our method to individual emergent constraints with observational uncertainty. Section 5 discusses correlations between the constraints. Section 6 uses Method C (described therein) to derive a Gaussian PDF of the climate proxy given multiple correlated constraints and shows its connection to multiple linear regression. It then applies Method C to sets of 4 and 11 constraints, including the use of robust regression to reduce artifacts in the 11-constraint case arising from strong sample correlations between constraints that must be estimated using an overly small set of GCMs. Section 7 presents and applies Method U, a special case of Method C that neglects any correlations between constraints outside their mutual correlation with the climate proxy. Method U can be more easily analyzed and interpreted, and it is used to explain why the full set of constraints confidently predicts a higher climate sensitivity than almost any single constraint. It is also more robust to our small sample of GCMs. Section 8 presents and applies an overconfidence adjustment to account for overfitting. Section 9 presents conclusions.

## 2. Data

### a. Choice of emergent constraints

For this study, we rely on 11 emergent constraints evaluated in Caldwell et al. (2018, hereinafter CZK18). These include the four constraints CZK18 judged to be “credible” (significantly correlated with ECS and supported by a physical mechanism that correctly identifies dominant physical processes and geographical regions that create this correlation), and seven constraints they judged to be “uncertain” or “unclear” (significantly correlated with ECS but not amenable to the above assessment of credibility). We will call these constraints “possible.” Two other “unclear” Klein constraints from CZK18 had to be excluded from our analysis for technical reasons described in section 2b. We also excluded six constraints assessed not to be credible in Table 4 of CZK18. Short explanations of each constraint that we used along with original citations and evaluations from CZK18 are provided in Table 1.

Short description of each emergent constraint tested in this paper along with original citation and evaluation from CZK18. Possible constraints are classified as untestable (lack a physical explanation or do not have an explanation that can be decomposed into feedback and forcing terms) or unclear (ambiguous when evaluated using the CZK18 criteria).

### b. Do the constraints have Gaussian distributions across the GCMs?

With our small subset of GCMs, we cannot fully test the plausibility of a multivariate Gaussian distribution for describing the joint PDF of a climate proxy such as ECS and a large set of emergent constraints. However, we can test the Gaussianity of the univariate marginal distributions of ECS and the individual constraints using a test of Shapiro and Wilk (1965). The ECS and 4 of the 11 constraints that we will use (1, 2, 8, and 11) fail this test at the 95% significance level (*p* < 0.05), but none of these fail at the 99% significance level. Since the GCMs are not fully independent, these significance levels may be overstated. Overall, we conclude that the Gaussianity assumption is marginally acceptable for our dataset.

### c. Observational estimates of constraints

CZK18 focused on the evaluation of emergent constraints using model data, while this study aims to use the observed values of those constraints to make climate sensitivity predictions. This requires observational estimates (including uncertainty) for the constraints. It would be ideal to obtain these estimates directly from the original data sources, but this is impractical given the number and diversity of constraints we use. Thus, we rely almost exclusively on values communicated by the papers originally proposing each constraint. These studies employed a variety of approaches and levels of detail in describing observational uncertainty. As a result, we are forced to make approximations to achieve uniformity of observed uncertainty estimates across constraints.

For simplicity, our analysis assumes observational uncertainty is normally distributed. The PDF of observed values for constraint *i* is specified by its mean *μ*_{o,i} and standard deviation *σ*_{o,i}. Although convenient, this assumption is not appropriate for two “Klein” constraints discussed by CZK18, which are based on positive semidefinite measures of model skill. Hence these two constraints were excluded from our analysis.

Our observed values and the information used to construct them are summarized in Table 2. Studies that provide mean and some multiple of the standard deviation were trivial to process. For studies which provide bounds for a given confidence level, we compute the number of standard deviations for that confidence level for a normal distribution, and we rescale the quoted range to estimate *σ*_{o,i}. Where several estimates of *μ*_{o,j} and *σ*_{o,j} were provided, we average the estimated means, and we increase *σ*_{o,j} such that *μ*_{o,i} ± 1*σ*_{o,i} just encompasses all of the individual estimated ±1*σ* ranges. For constraints that provide only minimum and maximum credible values (often taken from a pair of observations), we take the average of these values as the mean and 1/2 the distance between these values as the standard deviation. Because two samples provide a very poor sense of uncertainty, we occasionally use extra information from papers and/or expert judgement to modify these values, as noted in Table 2.

Observed values for all constraints used in this study and explanation of how they were obtained. See the text for details.

## 3. Terminology and covariance estimation

Our mathematical nomenclature is as follows. Capitalized Latin letters denote random variables, and lowercase versions of the same letter indicate particular values of these variables. Vectors are boldfaced. We define *n* emergent constraints will be labeled *U* is *p*(*u*), and similarly for multivariate distributions.

The main mathematical formula that we use is the joint PDF of the components of a column vector **U** of *m* zero-mean Gaussian random variables that are known to have an *m* × *m* covariance matrix

### Calculating correlation and covariance from GCM samples

A key input to our analysis is the (*n* + 1) × (*n* + 1) covariance matrix *n* constraints

The approach that we settled on is to build a GCM covariance matrix based on the best possible estimates of the correlation coefficients. We compute the correlation coefficient *i*, *j*) of constraints using all GCMs for which both constraints are available. We use a similar approach for GCM-based correlation coefficients *j*th constraint, as well as for calculating the standard deviation of each constraint

Note that in general, a different set of GCMs is used for computing each of the three terms on the right-hand side.

Here and in the rest of the paper, rows and columns of the covariance matrix are indexed starting at 0, index 0 corresponds to the climate proxy, and indices 1 through *n* correspond to the *n* constraints.

## 4. Climate sensitivity PDF from a single constraint

In this section we describe our approach for computing a PDF of the climate sensitivity proxy

An emergent constraint is based on a GCM-based relationship between

Philosophically, this frames our mathematical representation of emergent constraints. Unlike prior studies, we do not start by performing a GCM-based linear regression to determine

If we could exactly observe that

First, we cannot exactly observe the true value of *σ*_{o}. This observational error is assumed to be independent of

where

Second, the process of formulation and selection of emergent constraints may result in overconfidence, that is, constraints that are more highly correlated with the climate sensitivity proxy than would be obtained from a different independent random sample of GCMs, if such existed. We counteract overconfidence by artificially reducing the covariance between *X* and

It is convenient to work with standardized variables with a mean of 0 and a standard deviation of 1:

The normalization of the standardized *X* accounts for both its variance across GCMs and its observational uncertainty. Overlines indicate averages over the GCM ensemble.

The covariance matrix of *Y* and *X*, derived from the sample of GCMs and adjusted for observational constraint uncertainty, is

where *Y* and *X*, or equivalently between *r* is smaller in magnitude than the GCM-estimated correlation coefficient

We translate the observational estimate of the constraint,

We condition the joint PDF of *Y* and *X* on this known value of *X* to obtain a Gaussian posterior for *Y*:

where, substituting Eq. (7) for

From this formula, we can read off the mean *y*^{(1)} and standard deviation *σ*^{(1)} of the Gaussian posterior distribution of *Y*:

The superscript (1) denotes that this is a one-constraint estimate. We will define the estimated posterior “range” of *Y* as lying within 2 standard deviations of the mean, that is, *y*^{(1)} ± 2*σ*^{(1)}. Later, we will use a subscript *i* to denote an estimate that is based on constraint *i*.

Past studies of emergent constraints have typically used linear regression to quantify the relationship between the constraint and the climate sensitivity proxy. If we ignore observational uncertainty we could obtain the above result by regressing the climate sensitivity proxy *Y* on the constraint *X*. In this case, *r* would be the GCM-based correlation coefficient between *y* = *rx* matches the posterior mean *y*^{(1)} when evaluated at *x*_{o}. The residual in *Y* around that fit has a standard error (1 − *r*^{2})^{1/2} that matches *σ*^{(1)}.

Our approach naturally generalizes this regression method. It incorporates observational uncertainty in the constraint. For a single constraint, this was also done by Bowman et al. (2018) under similar assumptions but using a different mathematical approach. It is reassuring that, after accounting for our different notation and normalization, our formulas in Eqs. (10) and (11) are isomorphic to Eqs. (18) and (23) of Bowman et al. (2018). Unlike earlier work our approach extends naturally to many constraints, for which it becomes a form of multiple linear regression.

### Single-constraint results for ECS

Table 3 gives the correlation coefficients *r*_{i} between each constraint *i* and two choices of climate proxy *Y* (ECS and climate feedback parameter *λ*). To simplify the ensuing discussion, we henceforth “sign correct” all constraints so that *r*_{i} > 0, by flipping the sign of those constraints that are negatively correlated with *Y* (indicated by a minus sign in the “Sign” column). For ECS, we also tabulate the GCM-based correlation coefficient *r*_{i} since the observational uncertainty is assumed to be uncorrelated with ECS. This table also gives normalized constraint values *x*_{o,i} (in units of standard deviation) for the sign-corrected constraints, such that *x*_{o,i} > 0 favors *y* > 0 (climate proxy larger than the GCM mean). Eight of the 11 constraints have positive *x*_{o,i}, with values up to 2.4 for constraint 1 (Sherwood D). Constraints 2 (Brient Shal), 9 (Lipat), and 11 (Cox) have modestly negative *x*_{o,i} in the range from −0.4 to −0.8.

For each constraint *i*: the sign of the correlation with ECS, standardized best-guess value *x*_{o,i}, ratio of observational uncertainty *σ*_{o,i} to GCM-based standard deviation *r*_{i} with the two climate proxies, after adjusting for observational uncertainty as described in the text.

Last, Table 3 gives *Y*. For the remaining 6 constraints, the ratio is larger than 0.5. For these constraints, observational uncertainty cannot be neglected—it substantially reduces *r*_{i}, broadens the posterior range, and moves *Y* toward the GCM mean). This is most pronounced for Constraint 6 (Qu), with a ratio of 1.7; the ratio lies between 0.64 and 1.01 for the remaining five constraints.

The formulas in the previous section were nondimensional. To redimensionalize a constraint value *x*, we scale with its standard deviation *σ*_{o} from Table 2 and *λ*.

Estimated ±2*σ* climate proxy ranges.

Our approach is illustrated in Fig. 1, using the Su constraint as an example. For clarity, the figure is presented in terms of original rather than standardized variables. This constraint is constructed by computing the climatological-average latitude–height cross section of zonal average relative humidity (a proxy for the Hadley circulation) from model and observations, then computing for each model the slope of model versus observed values using each (latitude, height) value as a separate data point. Perfect agreement with observations occurs when the (nondimensional) slope is 1. Observations are inferred from satellite remote sensing; their uncertainties impart to the constraint a relatively large observational standard error of 0.25, as indicated by the PDF along the *y* axis. The cyan ellipse shows a contour of joint probability density between ECS

The posterior PDFs of ECS given each constraint separately are shown in Fig. 2, along with the PDF of ECS from CMIP3 + CMIP5 models, shown both as a histogram and a Gaussian fit. Credible constraints are shown in Fig. 2a and possible constraints are shown in Fig. 2b. Their means vary from 3.0 K (constraint 9 = Lipat) to 4.0 K (constraint 1 = Sherwood D), with a standard deviation of 0.5 K (constraint 3 = Zhai) to 0.7 K (constraint 6 = Qu). Eight of the 11 constraints have PDFs peaked at an ECS greater than the GCM mean. Constraint credibility has no systematic effect on peak probabilities or distribution widths.

Posterior PDFs for ECS based on individual (a) credible and (b) possible constraints using the method of section 3d.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

Posterior PDFs for ECS based on individual (a) credible and (b) possible constraints using the method of section 3d.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

Posterior PDFs for ECS based on individual (a) credible and (b) possible constraints using the method of section 3d.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

These Gaussian PDFs are derived from the formulas for nondimensional posterior mean [Eq. (10)] and standard deviation [Eq. (11)] using the information in Table 3. These are redimensionalized as discussed above.

These PDFs do not account for possible overconfidence, which would further broaden their width. Even so, all of the constraints have substantial overlap with each other. This is reassuring because if two constraints had disjoint PDFs we would be forced to conclude that at least one of them must be wrong (which would be an disturbing finding worthy of further study). Results for the climate sensitivity parameter *λ* (not shown) are similar.

## 5. Dependence between constraints

Combining constraints is only useful if those constraints provide some independent information. Interdependence of constraints was investigated in CZK18 by computing correlations between each pair of constraints. CZK18 found that constraints were more correlated than expected by chance but that identifying pairs of significantly correlated constraints on the basis of related physical explanations was generally unsuccessful. They noted that some pairwise correlation is expected because all constraints are by construction correlated with ECS.

For our 11 constraints, there are 55 pairs of constraints. If we flip the sign of constraints that are negatively correlated with ECS, then 52 of the 55 constraint pairs are positively correlated across the GCMs, which is strong evidence for such mutual dependence; 14 of these have a positive correlation coefficient exceeding 0.4, and none have a correlation coefficient more negative than −0.25.

To correct for the mutual dependence between constraints that is associated with ECS, we compute the partial correlation coefficient (https://www.encyclopediaofmath.org/index.php/Partial_correlation_coefficient) between constraints *X*_{i} and *X*_{j} given ECS *Y* (denoted hereinafter by a subscript 0):

This is the correlation coefficient between the residuals after *X*_{i} and *X*_{j} have been linearly regressed on *Y*. Here *r*_{ij} is the sample correlation coefficient between *X*_{i} and *Y*, and similarly for *r*_{i}, while *r*_{ij} is the sample correlation coefficient between *X*_{i} and *X*_{j}. We call two constraints with a partial correlation of zero “conditionally uncorrelated”; for the multivariate Gaussian distributions assumed in this paper, this is equivalent to conditional independence. The partial correlation coefficient is the correlation coefficient can vary between −1 and 1. Positive sign-corrected partial correlation indicates constraints that covary in the same sense as we would expect based on their correlation with *Y*.

Correcting for the mutual dependence with ECS removes some but not all of the covariation between our 11 constraints. Figure 3 shows the sign-corrected pairwise partial correlations. Well over one-half (38 of 55) are positive. This is suggestive, but are these partial correlations statistically significant? As in CZK18, choosing an appropriate number of degrees of freedom is difficult because there are complicated structural dependences between models (Masson and Knutti 2011; Knutti et al. 2013; Sanderson et al. 2015). Recall that some constraints could not be computed using outputs available from all GCMs, so each *r*_{ij0} was computed on the basis of an (*i*, *j*)-dependent subset of the GCMs. Following CZK18, we handle this issue by using a fairly lax 90% two-sided test and by assuming each GCM that goes into the calculation of a particular partial correlation coefficient *r*_{ij0} is independent. Both assumptions favor false positives, but partial correlations deemed not to be significant would almost certainly also be deemed insignificant with other reasonable assumptions. For a typical number of contributing GCMs (20), a partial correlation of magnitude 0.38 or larger is significant by this standard, based on a *t* test.

Partial correlations between pairs of emergent constraints over all available models, removing their joint correlation with ECS. The number of models used for each constraint pair is listed in parentheses in the bottom of each box. Boxes with orange or purple hue have positive or negative partial correlations, respectively, that are statistically significant at 90% using a two-tailed *t* test. White or gray boxes indicate weaker partial correlations that are statistically insignificant. In both cases, intensified shading indicates stronger partial correlation magnitudes.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

Partial correlations between pairs of emergent constraints over all available models, removing their joint correlation with ECS. The number of models used for each constraint pair is listed in parentheses in the bottom of each box. Boxes with orange or purple hue have positive or negative partial correlations, respectively, that are statistically significant at 90% using a two-tailed *t* test. White or gray boxes indicate weaker partial correlations that are statistically insignificant. In both cases, intensified shading indicates stronger partial correlation magnitudes.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

Partial correlations between pairs of emergent constraints over all available models, removing their joint correlation with ECS. The number of models used for each constraint pair is listed in parentheses in the bottom of each box. Boxes with orange or purple hue have positive or negative partial correlations, respectively, that are statistically significant at 90% using a two-tailed *t* test. White or gray boxes indicate weaker partial correlations that are statistically insignificant. In both cases, intensified shading indicates stronger partial correlation magnitudes.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

We found that only 6 of 55 constraint pairs have a significantly positive partial correlation; these are shown by orange shading in Fig. 3. Conversely, 1 of 55 constraint pairs has a significantly negative partial correlation, shown by purple shading in Fig. 3. Some expected relationships between constraints are borne out, like tight correlation between Siler and Volodin. Other constraints are significantly correlated even though their explanations seem to be unrelated, like Brient Shal and Brient Alb. This motivates our Method C (for “correlated”), which accounts for partial correlation between constraints. However, other constraint pairs, like Zhai and Brient Alb, are weakly correlated even though they share a physical explanation, and only 11% of the constraint pairs have a partial correlation above the 90% significance threshold. Thus, it is also a reasonable overall assumption to neglect partial correlations among our set of constraints, motivating the simpler Method U (for “uncorrelated”) discussed in section 7.

## 6. Method C: Climate sensitivity PDF from multiple correlated constraints

### a. Theory

The method introduced in section 4 generalizes transparently to the case of multiple constraints *σ*_{0,1}, …, *σ*_{0,n}. We form a column vector **U** of length *n* + 1 whose components are the climate sensitivity proxy *n* + 1) × (*n* + 1) covariance matrix modified for observational uncertainty are

Here *δ*_{ij} is 1 if *i* = *j* and 0 otherwise. The matrix

We standardize *Y* and the standardized constraints *X*_{i} is a correlation matrix with components

We also derive standardized values *x*_{o,i} for each observational estimate

We need the conditional distribution of *Y* given known values *x*_{o,i} for all the random variables *X*_{i}. We will show that this is actually an application of multiple linear regression of the climate proxy *Y* on the constraints *X*_{i}, which is in turn a natural generalization of the one-constraint case.

Eaton (1983) provides convenient formulas for the means of the conditional distribution of a multivariate Gaussian distribution given known values of some of the random variables, which we specialize to the case at hand. To use Eaton’s formulas, we must break

Here, *C*_{YY} = 1 is 1 × 1; **C**_{YX} = **r** is the column vector of *r*_{i}s, *i* = 1, …, *n*; and *n* × *n* covariance matrix of the standardized constraints *X*_{i}, which is *Y*. Since *Y* is standardized, these are *μ*_{Y} = 0 and var(*Y*) = 1. Last, we need the mean *μ*_{X} = 0 of the column *n*-vector of standardized constraints.

Then Eaton’s formula for the mean of *Y*, given the observed estimates *x*_{o,i} of the *X*_{i}s, is

where, since

The corresponding formula for the conditional variance of *Y* is

These formulas show that Method C is equivalent to multiple linear regression of *Y* on the *X*_{i}s using the GCMs [with no constant term in Eq. (17), since the predictors and predictand have mean of 0 by construction]. In particular, Eq. (18) is precisely the “normal equation” of multiple regression. Equation (20) is the expected mean squared residual of this prediction. A subtlety is that, unlike in standard multiple linear regression,

### b. Sampling uncertainty in the covariance matrix

In theory, Method C solves our problem for multiple correlated constraints, because it specifies the Gaussian posterior PDF of the climate proxy *Y* in terms of known quantities. In practice, if the covariance matrix is derived from a finite sample of GCMs it is sensitive to sampling uncertainty, especially if the number of constraints is comparable to the number of GCMs. For instance, if we have 10 constraints and 20 GCMs, we are using 20 samples to estimate an 11-dimensional correlation matrix with 11 × 12 ÷ 2 = 66 independent entries, which is a highly underconstrained problem. Thus, we anticipate that Method C may fail or give spurious results, especially if partial correlations between the constraints are important (multicollinearity). In section 6c, we apply Method C to ECS estimation, and we consider a penalized regression approach to improve its robustness.

### c. Applying method C to ECS

#### 1) Four credible constraints

The red curve in Fig. 4a shows the posterior PDF for ECS estimated using Method C based on the four sign-corrected credible constraints. It is compared with other PDFs, including the Gaussian CMIP prior (black dashed) and the average of the single-constraint PDFs (solid black). Methods U and U3 will be discussed in the following section.

Posterior PDFs for ECS based on (a) 4 credible constraints and (c) all 11 constraints, using Methods C, U, and U3 and, for (c), Method C regularized via ridge regression (see the text). The black dashed lines in (a) and (c) show the CMIP Gaussian prior for comparison. (b),(d) Method C results, but each PDF is calculated with one GCM removed to test for robustness.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

Posterior PDFs for ECS based on (a) 4 credible constraints and (c) all 11 constraints, using Methods C, U, and U3 and, for (c), Method C regularized via ridge regression (see the text). The black dashed lines in (a) and (c) show the CMIP Gaussian prior for comparison. (b),(d) Method C results, but each PDF is calculated with one GCM removed to test for robustness.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

Posterior PDFs for ECS based on (a) 4 credible constraints and (c) all 11 constraints, using Methods C, U, and U3 and, for (c), Method C regularized via ridge regression (see the text). The black dashed lines in (a) and (c) show the CMIP Gaussian prior for comparison. (b),(d) Method C results, but each PDF is calculated with one GCM removed to test for robustness.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

Method C gives a strikingly higher and narrower PDF than the average of the single-constraint PDFs, which in turn is shifted slightly higher than the CMIP mean PDF. This may seem surprising but can be viewed as the result of strong evidence (3 of 4 constraints) agreeing that ECS is likely more positive than the GCM mean. Mathematically stated, of the four observed *x*_{o,i}, only *x*_{o,2} < 0.

Using Eqs. (17) and (18), Method C predicts that the nondimensional most likely ECS is

The nondimensional ECS standard deviation, derived from Eq. (20), is *σ* ECS range of 4.14 ± 0.82 K.

Constraint 3 (Zhai) has the strongest weight *a*_{3} = 0.48 on the Method C ECS mean. Constraint 4 (Brient Alb) has only 60% of the weight of constraint 3 despite having a similar correlation *r* ≈ 0.7 with ECS; this is due to its covariances with the other constraints. All four constraints have positive weights *a*_{i} > 0. This positivity property is physically reasonable and desirable. For instance, if constraint *i* has observed value *x*_{o,i} > 0, its correlation with ECS is *r*_{i} > 0, and all other constraints have observed values of zero, then we expect *y* > 0; this is only possible if *a*_{i} > 0. However, weight positivity is not guaranteed by Method C.

Figure 4b assesses the sensitivity to the GCM sampling by redoing the 4-constraint analysis with each of the GCMs omitted in turn when computing the needed correlation matrix. The posterior ECS PDF is robust to this test. Removing any two GCMs provides very similar results, with the most likely value of ECS always being 3.9–4.2 K (not shown).

#### 2) All constraints

Method C can also be applied to all 11 constraints, since the 12 × 12 estimated correlation matrix has all positive eigenvalues. It gives a ± 2*σ* ECS range 4.28 ± 0.68 K, shown as the dashed red curve in Fig. 4c. This PDF has a slightly higher mean and a narrower distribution of ECS than the 4-constraint estimate. Eight of the 11 weights *a*_{i} are positive. Two are marginally negative, and one, *a*_{10} = −0.47, is strongly negative. This arises because of multicollinearity; constraint 10 (Siler) has very strong partial correlations with some of the other constraints, exceeding 0.5 with constraints 2 (Brient Shal) and 5 (Volodin). These partial correlations are also quite uncertain due to the small number of GCMs. Because the observed value *x*_{o,10} = 0.35 is small, the large negative weight does not have a strong direct impact on the most probable ECS. Nevertheless, this suggests the Method C ECS PDF may be less robust to sampling uncertainty in the correlation matrix with 11 constraints than with 4 constraints. This defeats the point of using more constraints.

Figure 4d assesses this by recomputing posterior PDFs after removing a single GCM, as in Fig. 4b. The posterior ECS PDFs are mostly robust to this test, but less so than with the four constraints, since there are now three clear outliers among the 11 ranges. The mean ECS ranges from 3.7 to 4.6 K across these cases and from 3.6 to 5 K when two GCMs are removed (not shown).

#### 3) Ridge regression to improve robustness

Ridge regression (Hoerl 1962) is a technique for penalizing large constraint weights *a*_{i} (specifically, the sum of squares of the weights) that can result from multicollinearity and a poorly conditioned covariance matrix. Because samples from all GCMs are not available for every predictor (constraint) and predictand (climate proxy), we can only use regression implementations based on a modified covariance matrix. This precludes most popular robust regression methods, e.g., “LASSO” (Santosa and Symes 1986), which are not trivial to adapt to this problem formulation. However, ridge regression can be cast as multiple regression with an additional diagonal term in the covariance matrix, that is,

A larger value of the user-chosen regularization parameter *γ* > 0 corresponds to stronger weight penalization. This comes at the expense of potential bias in the estimated predictand toward its a priori mean, in this case the GCM mean of the climate proxy.

For the 11-constraint problem, all the sign-corrected constraint weights became nonnegative for *γ* ≥ *γ*_{c} = 0.64. This is strong regularization; the sum of the squared weights is less than 20% as large as the unregularized solution. The zero weight at *γ*_{c} corresponds to constraint 10 (Siler), which had a strongly negative sign-corrected weight in the unregularized regression. The PDF for *γ*_{c} is shown as the solid red line in Fig. 4c and is labeled “C, RidgeRegr.” It has a smaller mean and broader width than the unregularized Method C but is qualitatively similar.

The condition number of the unregularized *γ* = *γ*_{c} desirably reduces this down to 6. For the 4-constraint problem, the condition number of the unregularized

Ultimately, the choice of *γ* is subjective. A smaller value *γ* ≈ 0.25 reduces the ratio of the smallest negative weight to the largest positive weight by a factor of 3 relative to the unregularized solution, with much less reduction in the mean and increase in the width of the PDF relative to *γ*_{c}. This might provide a better trade-off between robustness and bias.

## 7. Method U: Estimation of *Y* from conditionally uncorrelated constraints

Overall, Fig. 3 suggests that most of the partial correlations between the constraint pairs are not that large and are statistically insignificant. Thus it is reasonable to consider the special case that all the constraints are mutually uncorrelated when conditioned on a given value of the climate sensitivity proxy *Y*. In that case, Method C reduces to a simpler Method U (for “uncorrelated”) that requires only estimates of the correlation coefficients *r*_{i} between *Y* and the individual constraints (given in Table 3). These can be computed fairly reliably with 10 or more GCMs, eliminating the need for regularization approaches like ridge regression, even when there are many constraints. For a single constraint, Method U and Method C both reduce to the method already presented in section 4.

The Method U mean and variance of the climate proxy *Y* can be derived from the Method C Eqs. (17)–(20) using linear algebra. However, the same results are easier and more illuminating to derive directly from the joint multivariate Gaussian PDF

Since the constraints are assumed to have Gaussian PDFs, and they are uncorrelated when conditioned on the Gaussian variable *Y*, they are independent when conditioned on *Y*. Hence the joint PDF of the standardized constraints conditioned on *Y* can be written as a product:

The conditional PDF for constraint *X*_{i} is given by Eqs. (10) and (11) with *x* and *y* swapped:

Setting the constraints equal to their standardized observed values, we obtain the posterior PDF

where, with a slight rewrite of Eq. (25),

Here

and

That is, the desired posterior is a product of Gaussian PDFs with means *y*_{o,i} and standard deviation *r*_{i} with the climate proxy, the weight *w*_{i} is large and the standard deviation is small. The standardized GCM prior *p*(*y*) has been included in this notation as an *i* = 0 Gaussian PDF with mean 0 and standard deviation 1.

Equations (26) and (27) are the keys to understanding how multiple constraints combine to determine the posterior PDF of the climate proxy. The posterior will maximize for those *y* that are most consistent with all of the *y*_{o,i}s within their individual uncertainties

For each constraint, the “proxy estimate” *y*_{o,i} is the predicted value of the climate proxy we would obtain by linear regression applied to the single-constraint problem with the dependent and independent variables swapped, that is, regressing *X*_{i} on *Y* and using the linear fit to determine *Y* for the observed value *X*_{i} = *x*_{o,i}. In that regression problem, since both *X*_{i} and *Y* are standard normal, the slope of the regression line is their correlation coefficient *r*_{i} and the residual standard deviation of *X*_{i} is *Y* from this constraint are a factor 1/*r*_{i} larger than for *X*_{i}. For constraints that are poorly correlated with *Y*, the proxy estimate *y*_{o,i} can be quite large even if the observed constraint value *x*_{o,i} is not large, but the uncertainty of this proxy estimate is even larger.

Continuing, we can write

Equation (30) has the form of weighted least squares estimation of the mean *y* of a Gaussian random variable *Y* based on samples *y*_{o,i}, *i* = 0, …, *n*, with standard errors *Y* is a Gaussian with mean and standard deviation

where

The normalized weights *w*_{i}/*w*_{s} sum to 1, so the posterior mean is a weighted average of estimates *y*_{o,i} derived from individual constraints.

If several constraints all suggest large anomalies *y*_{o,i} of the same sign, they can overwhelm the anchoring contribution of the GCM prior and combine to create a large *Y* that is farther from the GCM mean than most or all of the single-constraint estimates, which are more strongly anchored to the GCM prior.

To connect to Method C, the Method U posterior mean of *Y* can also be written as a linear combination of the constraint values:

with constraint weights

The Method U constraint weights *a*_{i} are appealingly interpretable. They are positive, and they increase with the correlation coefficients *r*_{i} of the constraints with the climate proxy.

More conditionally uncorrelated constraints always increase *w*_{s}, decrease *Y*, especially those having a high correlation with *Y*.

### a. Consistency with single-constraint posterior PDF

When applied to only one constraint, Methods C and U both reduce to the single-constraint posterior. Indeed, the Method U formulas (after dropping the index 1 to match the single-constraint notation)

match our single-constraint formulas in Eqs. (10) and (11). However, Method U provides an interpretation that generalizes to multiple constraints, namely that the posterior mean of *Y* is a weighted average of its GCM prior (which is zero), and the proxy estimate *y*_{o} = *x*_{0}/*r* from the constraint, multiplied by a normalized weight *r*^{2} < 1. In this interpretation, the contribution of the GCM prior “dilutes” the effect of the constraint by a factor of *r*^{2}.

### b. Method U results for ECS

The blue curve in Fig. 4a shows the Method U estimate of the ECS posterior given the four credible constraints. This Gaussian PDF has a ±2*σ* range of 3.3–4.9 K, similar to the 4-constraint estimate from Method C. That is, partial correlation among the four credible constraints has almost no impact on the posterior PDF. Indeed, according to Method U, the most likely ECS is

The coefficients are very similar to those in the analogous formula in Eq. (21) derived from Method C. The blue curve in Fig. 4c shows the Method U ECS posterior for all 11 constraints, which has a ±2*σ* range of 3.4–4.6 K. This is similar but slightly narrower than the 4-constraint posterior.

Method U is consistent with Method C, especially for the 4-constraint case, and has the attraction of interpretability. Figure 5a shows one way to visualize its results. Each constraint is represented by a colored vertical bar with height equal to the weight *w*_{i}, located at an ECS redimensionalized from *y*_{o,i} using the GCM mean and standard deviation. For any set of constraints, the posterior mean ECS is at the horizontal center of gravity (weighted average) of their bars, including the GCM prior (constraint 0). The 11-constraint posterior mean and uncertainty range are shown for comparison.

Method U all-constraint interpretive results: (a) Weights plotted as vertical bars at the observational estimate of ECS from each constraint, calculated using swapped regression. Each bar is labeled by its constraint number *i*. The blue dashed bar (*i* = 0) is the GCM prior. The other thickened bars are the four credible constraints. The circle and the horizontal dotted black line show the all-constraint ±2*σ* posterior range of ECS. (b) Construction of Method U posterior of ECS as a product of PDFs from individual constraints, shown as an accumulation of log-likelihood. Color shading shows the log-likelihood contributed by the GCM prior and each of the four credible constraints. The horizontal dotted black line shows the resulting 4-constraint ±2*σ* posterior range of ECS.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

Method U all-constraint interpretive results: (a) Weights plotted as vertical bars at the observational estimate of ECS from each constraint, calculated using swapped regression. Each bar is labeled by its constraint number *i*. The blue dashed bar (*i* = 0) is the GCM prior. The other thickened bars are the four credible constraints. The circle and the horizontal dotted black line show the all-constraint ±2*σ* posterior range of ECS. (b) Construction of Method U posterior of ECS as a product of PDFs from individual constraints, shown as an accumulation of log-likelihood. Color shading shows the log-likelihood contributed by the GCM prior and each of the four credible constraints. The horizontal dotted black line shows the resulting 4-constraint ±2*σ* posterior range of ECS.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

Method U all-constraint interpretive results: (a) Weights plotted as vertical bars at the observational estimate of ECS from each constraint, calculated using swapped regression. Each bar is labeled by its constraint number *i*. The blue dashed bar (*i* = 0) is the GCM prior. The other thickened bars are the four credible constraints. The circle and the horizontal dotted black line show the all-constraint ±2*σ* posterior range of ECS. (b) Construction of Method U posterior of ECS as a product of PDFs from individual constraints, shown as an accumulation of log-likelihood. Color shading shows the log-likelihood contributed by the GCM prior and each of the four credible constraints. The horizontal dotted black line shows the resulting 4-constraint ±2*σ* posterior range of ECS.

Citation: Journal of Climate 33, 17; 10.1175/JCLI-D-19-0911.1

Recall that for each constraint, the weight *w*_{i} is a strongly increasing function of the correlation coefficients *r*_{i} with ECS. Most of the constraints have correlation coefficients with ECS of between 0.4 and 0.6 and modest weights of 0.1–0.5. Constraints 3 (Zhai) and 4 (Brient Alb) have larger weights near 1 because of their stronger correlation with ECS, so they are particularly influential to the ECS posterior range. “Constraint 0,” the GCM prior, has a weight of 1. With a single constraint, the GCM prior substantially moderates how far the posterior mean is likely to deviate from the GCM mean ECS, as we saw in Fig. 2. In combination, the four credible constraints significantly outweigh the GCM prior, reducing its influence. With 11 constraints, the GCM range of ECS only plays a modest role in shaping the posterior.

The observational proxy estimates *y*_{o,i} span a wide range, but 8 of 11 exceed the GCM prior of 3.2 K and their 11-constraint median is nearly 4.5 K, so it is not surprising that the Method U all-constraint mean ECS estimate is 4 K. This is larger than the average of the single-constraint estimates, because those are more strongly anchored to the GCM prior. Constraint 1 (Sherwood D) has an anomalously large ECS estimate of nearly 8 K because its observational estimate is *x*_{o,1} = 2.4 standard deviations above the GCM mean, and because it has a relatively small correlation *r*_{1} ≈ 0.4 with ECS. However, that small *r*_{1} also causes it to have a small proxy weight *w*_{1}.

Figure 5b shows the construction of the Method U posterior PDF of ECS as a product of independent Gaussian PDFs from individual constraints. This is easiest to visualize logarithmically as a summation of log-likelihood (the exponent of the Gaussian), which is a parabolic function of ECS for each constraint. The color shades show log-likelihood contributed by the GCM prior and each of the four credible constraints (adding the other seven constraints is trivial but would clutter the plot). Each constraint favors a different ECS range centered around its observational ECS estimate plotted in Fig. 5a, over which its log-likelihood is not too negative. The GCM prior weights the ECS toward the GCM-mean ECS of 3.2 K, but constraint 1 (Sherwood D) favors very high ECS, while constraint 2 (Brient Shal) favors low ECS. The resulting 4-constraint posterior log-likelihood is the black parabola bounding the lowest shaded region. For both this and the 11-constraint ECS posterior, no single constraint is responsible for the mean exceeding 4 K; that is a collective result of several constraints, lending more credence to the result.

## 8. Adjusting for overconfidence

We present a method to adjust Methods C and U for overconfidence by artificially reducing the correlation coefficients *r*_{i} between the constraints and *Y*, while leaving the partial correlation coefficients between the constraints unchanged.

An obvious way to do this would be to scale all the correlation coefficients *r*_{i} by the same factor. We use a related but slightly different approach. A constraint that is nearly perfectly correlated with ECS over 20+ GCMs should also be extremely highly correlated with ECS over a different set of GCMs. Thus we treat overconfidence as leading to a systematic underestimate by a user-specified factor *α*^{2} ≤ 1 of the variance in each of the constraints that is unexplained by regression onto *Y*. This amounts to scaling the ratio *α*^{2}:

for which the correspondingly reduced correlation coefficients are

For small values of *r*_{i}, the reduced correlation coefficient is a factor of *α* as large as the original coefficient; for large *r*_{i}, the fractional reduction is less.

To use Method C, we could adjust the correlation matrix to preserve the partial correlations *r*_{ij0} between constraints while reducing the *r*_{i}s. However, if we are highly uncertain about *r*_{i}, then our empirical estimates of the partial correlations are at least as uncertain and potentially meaningless.

Some overconfidence correction (*α* < 1) seems merited given our earlier arguments about a priori selection and optimization of constraints. One compelling basis for such a judgement is testing based on an independent set of GCMs. Some previous examples include some of the discredited constraints in CZK18, and the minimal correlation of the “credible” Sherwood D constraint with ECS across a perturbed parameter ensemble (Wagman and Jackson 2018). This latter work suggests a large uncertainty enhancement for constraint overconfidence is needed even for physically appealing emergent constraints. Further research on this issue is needed.

As a sensitivity test, we apply an extreme overconfidence adjustment *α* = ⅓ (a ninefold inflation of unexplained variance) to Method U. The green curves U3 in Figs. 4a and 4c show the resulting ECS PDFs for the 4-constraint and the 11-constraint cases. In both cases, the overconfidence correction widens the posterior range (Table 4) and moves the posterior mean ECS somewhat toward the GCM mean. The wider range is expected, because constraints weakly correlated with ECS are less informative.

## 9. Conclusions

We derive a new “Method C” of combining correlated emergent constraints to estimate a Gaussian PDF for ECS or any other global climate sensitivity proxy from a finite sample of GCMs. It accounts for observational uncertainty of the constraints and (optionally) for overconfidence in the estimated correlation between the constraints and the proxy. The method is based on approximating the joint PDF of the proxy and the constraints as multivariate Gaussian, and it can also be framed in terms of multiple linear regression. With our limited sample of 40 CMIP GCMs, most of which provided inadequate outputs to compute some constraints, this PDF is inadequately sampled, and therefore the method may not give robust results as the number of constraints becomes comparable to the number of GCMs. Ridge regression is shown to improve the robustness of the combined prediction but somewhat biases the estimated PDF of the climate proxy toward the GCM mean. We also present “Method U,” which neglects any partial correlations between emergent constraints that are not related to their joint correlation with the climate proxy; this is more robust when applied to a small sample of GCMs and is more interpretable.

We apply these methods to a set of four credible and seven other “possible” constraints from CZK18 and compare them with PDFs derived from single constraints. Taken singly, the 11 constraints imply ECS PDFs with ±2*σ* ranges having means between 3 and 4 K and ±2*σ* widths of ±0.8–1.2 K. Reassuringly, all of these constraints give overlapping PDFs for ECS, with eight of the 11 favoring ECS higher than the GCM mean. The 4-constraint ±2*σ* ECS range estimated by both Methods C and U is close to 4 ± 0.8 K, which lies exclusively in the upper half of the GCM range.

Using all 11 constraints, the Method C ECS range showed some sensitivity to omission of individual GCMs from our dataset but universally favored ECS well above the GCM mean. With a larger sample of GCMs, the robustness of Method C should further improve. With 11 constraints, Method U produced a similar but narrower ECS range when compared with 4 constraints. Relative to the single-constraint analysis, the Method U multiconstraint analysis generates PDFs that are less like the GCM prior. Since a large majority of the constraints individually favor ECS values above the GCM mean, combining them more strongly favors that result. This is an important and interesting result of this analysis.

We propose a user-chosen adjustment factor *α* to account for constraint overconfidence. This factor reduces the correlation coefficients of all the constraints with the climate sensitivity proxy while leaving their conditional correlations with each other unaltered. With a strong overconfidence adjustment *α* = ⅓, the ±2*σ* 11-constraint estimated ECS range of Method U doubles in width to 2.7–5.3 K, but its mean remains close to 4 K.

The same hierarchy of approaches was applied to the climate sensitivity parameter *λ* = −3.7 W m^{−2}/ECS, with similar results shown in Table 4. Arguably, it is more plausible to fit *λ* with a Gaussian distribution than ECS. A Gaussian *λ* implies a skewed distribution for ECS. Indeed, the ±2*σ* range of *λ* for Method U using 11 constraints implies an ECS range of 3.4–6.1 K, that is, a substantially longer positive tail and a slightly truncated negative tail relative to the alternate assumption of Gaussian ECS.

We conclude that climate sensitivity estimated from combining the most reasonable current emergent constraints is very likely above the CMIP3/5 GCM mean of 3.2 K and has roughly even odds of exceeding 4 K. To better interpret and bolster this provocative result, we should continue to search for more physically motivated emergent constraints aimed at regime-specific cloud feedbacks (e.g., Qu et al. 2013). We will also examine how the 11 considered emergent constraints fare in predicting ECS and other measures of climate sensitivity across the GCMs in the ongoing CMIP6 intercomparison, several of which have ECS well in excess of 4 K.

## Acknowledgments

We acknowledge the modeling groups, the Program for Climate Model Diagnosis and Intercomparison (PCMDI), and the World Climate Research Program’s Working Group on Coupled Modeling for making available the CMIP3 and CMIP5 multimodel datasets, supported by the U.S. Department of Energy (DOE) Office of Science. This work was supported by the Office of Science (BER) at Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344 and BER’s Regional and Global Climate Modeling (RGCM) Program. Steve Klein instigated this study and heavily influenced early drafts. Yuying Zhang provided estimates of the Volodin constraint from multiple satellites.

## REFERENCES

Armour, K. C., C. M. Bitz, and G. H. Roe, 2013: Time-varying climate sensitivity from regional feedbacks.

, 26, 4518–4534, https://doi.org/10.1175/JCLI-D-12-00544.1.*J. Climate*Bowman, K. W., N. Cressie, X. Qu, and A. Hall, 2018: A hierarchical statistical framework for emergent constraints: Application to snow-albedo feedback.

, 45, 13 050–13 059, https://doi.org/10.1029/2018GL080082.*Geophys. Res. Lett.*Brient, F., 2020: Reducing uncertainties in climate projections with emergent constraints: Concepts, examples and prospects.

, 37, 1–15, https://doi.org/10.1007/s00376-019-9140-8.*Adv. Atmos. Sci.*Brient, F., and T. Schneider, 2016: Constraints on climate sensitivity from space-based measurements of low-cloud reflection.

, 29, 5821–5835, https://doi.org/10.1175/JCLI-D-15-0897.1.*J. Climate*Brient, F., T. Schneider, Z. Tan, S. Bony, X. Qu, and A. Hall, 2016: Shallowness of tropical low clouds as a predictor of climate models’ response to warming.

, 47, 433–449, https://doi.org/10.1007/s00382-015-2846-0.*Climate Dyn.*Caldwell, P. M., C. S. Bretherton, M. D. Zelinka, S. A. Klein, B. D. Santer, and B. M. Sanderson, 2014: Statistical significance of climate sensitivity predictors obtained by data mining.

, 41, 1803–1808, https://doi.org/10.1002/2014GL059205.*Geophys. Res. Lett.*Caldwell, P. M., M. D. Zelinka, and S. A. Klein, 2018: Evaluating emergent constraints on equilibrium climate sensitivity.

, 31, 3921–3942, https://doi.org/10.1175/JCLI-D-17-0631.1.*J. Climate*Cess, R. D., and Coauthors, 1989: Interpretation of cloud-climate feedback as produced by 14 atmospheric general circulation models.

, 245, 513–516, https://doi.org/10.1126/science.245.4917.513.*Science*Cox, P. M., C. Huntingford, and M. S. Williamson, 2018: Emergent constraint on equilibrium climate sensitivity from global temperature variability.

, 553, 319–322, https://doi.org/10.1038/nature25450.*Nature*Eaton, M. L., 1983:

*Multivariate Statistics: A Vector Space Approach*. Institute of Mathematical Statistics, 512 pp.Gregory, J. M., and T. Andrews, 2016: Variation in climate sensitivity and feedback parameters during the historical period.

, 43, 3911–3920, https://doi.org/10.1002/2016GL068406.*Geophys. Res. Lett.*Hall, A., and X. Qu, 2006: Using the current seasonal cycle to constrain snow albedo feedback in future climate change.

, 33, L03502, https://doi.org/10.1029/2005GL025127.*Geophy. Res. Lett.*Hall, A., P. Cox, C. Huntingford, and S. Klein, 2019: Progressing emergent constraints on future climate change.

, 9, 269–278, https://doi.org/10.1038/s41558-019-0436-6.*Nat. Climate Change*Hoerl, A. E., 1962: Application of ridge analysis to regression problems.

, 58, 54–59.*Chem. Eng. Prog.*IPCC, 2013: Summary for policymakers.

*Climate Change 2013: The Physical Science Basis*, T. F. Stocker et al., Eds., Cambridge University Press, 3–29, http://www.climatechange2013.org/images/uploads/WGI_AR5_SPM_brochure.pdf.Klein, S. A., and A. Hall, 2015: Emergent constraints for cloud feedbacks.

, 1, 276–287, https://doi.org/10.1007/s40641-015-0027-1.*Curr. Climate Change Rep.*Knutti, R., D. Masson, and A. Gettelman, 2013: Climate model genealogy: Generation CMIP5 and how we got there.

, 40, 1194–1199, https://doi.org/10.1002/grl.50256.*Geophys. Res. Lett.*Knutti, R., M. A. A. Rugenstein, and G. C. Hegerl, 2017: Beyond equilibrium climate sensitivity.

, 10, 727–736, https://doi.org/10.1038/ngeo3017.*Nat. Geosci.*Lipat, B. R., G. Tselioudis, K. M. Grise, and L. M. Polvani, 2017: CMIP5 models’ shortwave cloud radiative response and climate sensitivity linked to the climatological Hadley cell extent.

, 44, 5739–5748, https://doi.org/10.1002/2017GL073151.*Geophys. Res. Lett.*Masson, D., and R. Knutti, 2011: Climate model genealogy.

, 38, L08703, https://doi.org/10.1029/2011GL046864.*Geophys. Res. Lett.*Qu, X., A. Hall, S. A. Klein, and P. M. Caldwell, 2013: On the spread of changes in marine low cloud cover in climate model simulations of the 21st century.

, 42, 2603–2626, https://doi.org/10.1007/s00382-013-1945-z.*Climate Dyn.*Read, W. G., and Coauthors, 2007: Aura Microwave Limb Sounder upper tropospheric and lower stratospheric H2O and relative humidity with respect to ice validation.

, 112, D24S35, https://doi.org/10.1029/2007JD008752.*J. Geophys. Res.*Reynolds, R. W., N. A. Rayner, T. M. Smith, D. C. Stokes, and W. Wang, 2002: An improved in situ and satellite SST analysis for climate.

, 15, 1609–1625, https://doi.org/10.1175/1520-0442(2002)015<1609:AIISAS>2.0.CO;2.*J. Climate*Rossow, W., and R. A. Schiffer, 1991: ISCCP cloud data products.

, 72, 2–20, https://doi.org/10.1175/1520-0477(1991)072<0002:ICDP>2.0.CO;2.*Bull. Amer. Meteor. Soc.*Sanderson, B. M., R. Knutti, and P. Caldwell, 2015: A representative democracy to reduce interdependency in a multimodel ensemble.

, 28, 5171–5194, https://doi.org/10.1175/JCLI-D-14-00362.1.*J. Climate*Santosa, F., and W. W. Symes, 1986: Linear inversion of band-limited reflection seismograms.

, 7, 1307–1330, https://doi.org/10.1137/0907087.*SIAM J. Sci. Statist. Comput.*Shapiro, S. S., and M. B. Wilk, 1965: An analysis of variance test for normality (complete samples).

, 52, 591–611, https://doi.org/10.1093/biomet/52.3-4.591.*Biometrika*Sherwood, S. C., S. Bony, and J.-L. Dufresne, 2014: Spread in model climate sensitivity traced to atmospheric convective mixing.

, 505, 37–42, https://doi.org/10.1038/nature12829.*Nature*Siler, N., S. Po-Chedley, and C. S. Bretherton, 2018: Variability in modeled cloud feedback tied to differences in the climatological spatial pattern of clouds.

, 50, 1209–1220, https://doi.org/10.1007/s00382-017-3673-2.*Climate Dyn.*Strutz, T., 2016:

*Data Fitting and Uncertainty: A Practical Introduction to Weighted Least Squares and Beyond*. Springer, 244 pp.Su, H., J. H. Jiang, C. Zhai, T. J. Shen, J. D. Neelin, G. L. Stephens, and Y. L. Yung, 2014: Weakening and strengthening structures in the Hadley circulation change under global warming and implications for cloud response and climate sensitivity.

, 119, 5787–5805, https://doi.org/10.1002/2014JD021642.*J. Geophys. Res. Atmos.*Tian, B., 2015: Spread of model climate sensitivity linked to double-intertropical convergence zone bias.

, 42, 4133–4141, https://doi.org/10.1002/2015GL064119.*Geophys. Res. Lett.*Volodin, E. M., 2008: Relation between temperature sensitivity to doubled carbon dioxide and the distribution of clouds in current climate models.

, 44, 288–299, https://doi.org/10.1134/S0001433808030043.*Izv. Atmos. Ocean. Phys.*Wagman, B. M., and C. S. Jackson, 2018: A test of emergent constraints on cloud feedback and climate sensitivity using a calibrated single-model ensemble.

, 31, 7515–7532, https://doi.org/10.1175/JCLI-D-17-0682.1.*J. Climate*Williamson, D. B., and P. G. Sansom, 2019: How are emergent constraints quantifying uncertainty and what do they leave behind?

, 100, 2571–2588, https://doi.org/10.1175/BAMS-D-19-0131.1.*Bull. Amer. Meteor. Soc.*Zhai, C., J. H. Jiang, and H. Su, 2015: Long-term cloud change imprinted in seasonal cloud variation: More evidence of high climate sensitivity.

, 42, 8729–8737, https://doi.org/10.1002/2015GL065911.*Geophys. Res. Lett.*