A Bayesian Algorithm for Reconstructing Climate Anomalies in Space and Time. Part II: Comparison with the Regularized Expectation–Maximization Algorithm

Martin P. Tingley Statistical and Applied Mathematical Sciences Institute, Research Triangle Park, North Carolina, and Department of Earth and Planetary Sciences, Harvard University, Cambridge, Massachusetts

Search for other papers by Martin P. Tingley in
Current site
Google Scholar
PubMed
Close
and
Peter Huybers Department of Earth and Planetary Sciences, Harvard University, Cambridge, Massachusetts

Search for other papers by Peter Huybers in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

presented a Bayesian algorithm for reconstructing climate anomalies in space and time (BARCAST). This method involves specifying simple parametric forms for the spatial covariance and temporal evolution of the climate field as well as “observation equations” describing the relationships between the data types and the corresponding true values of the climate field. As this Bayesian approach to reconstructing climate fields is new and different, it is worthwhile to compare it in detail to the more established regularized expectation–maximization (RegEM) algorithm, which is based on an empirical estimate of the joint data covariance matrix and a multivariate regression of the instrumental time series onto the proxy time series. The differing assumptions made by BARCAST and RegEM are detailed, and the impacts of these differences on the analysis are discussed. Key distinctions between BARCAST and RegEM include their treatment of spatial and temporal covariance, the prior information that enters into each analysis, the quantities they seek to impute, the end product of each analysis, the temporal variance of the reconstructed field, and the treatment of uncertainty in both the imputed values and functions of these imputations. Differences between BARCAST and RegEM are illustrated by applying the two approaches to various surrogate datasets. If the assumptions inherent to BARCAST are not strongly violated, then in scenarios comparable to practical applications BARCAST results in reconstructions of both the field and the spatial mean that are more skillful than those produced by RegEM, as measured by the coefficient of efficiency. In addition, the uncertainty intervals produced by BARCAST are narrower than those estimated using RegEM and contain the true values with higher probability.

Corresponding author address: Martin P. Tingley, Department of Earth and Planetary Sciences, Harvard University, 20 Oxford St., Cambridge, MA 02138. Email: tingley@fas.harvard.edu

Abstract

presented a Bayesian algorithm for reconstructing climate anomalies in space and time (BARCAST). This method involves specifying simple parametric forms for the spatial covariance and temporal evolution of the climate field as well as “observation equations” describing the relationships between the data types and the corresponding true values of the climate field. As this Bayesian approach to reconstructing climate fields is new and different, it is worthwhile to compare it in detail to the more established regularized expectation–maximization (RegEM) algorithm, which is based on an empirical estimate of the joint data covariance matrix and a multivariate regression of the instrumental time series onto the proxy time series. The differing assumptions made by BARCAST and RegEM are detailed, and the impacts of these differences on the analysis are discussed. Key distinctions between BARCAST and RegEM include their treatment of spatial and temporal covariance, the prior information that enters into each analysis, the quantities they seek to impute, the end product of each analysis, the temporal variance of the reconstructed field, and the treatment of uncertainty in both the imputed values and functions of these imputations. Differences between BARCAST and RegEM are illustrated by applying the two approaches to various surrogate datasets. If the assumptions inherent to BARCAST are not strongly violated, then in scenarios comparable to practical applications BARCAST results in reconstructions of both the field and the spatial mean that are more skillful than those produced by RegEM, as measured by the coefficient of efficiency. In addition, the uncertainty intervals produced by BARCAST are narrower than those estimated using RegEM and contain the true values with higher probability.

Corresponding author address: Martin P. Tingley, Department of Earth and Planetary Sciences, Harvard University, 20 Oxford St., Cambridge, MA 02138. Email: tingley@fas.harvard.edu

Save
  • Anderson, T., 2003: An Introduction to Multivariate Statistical Analysis. 3rd ed. Wiley, 752 pp.

  • Banerjee, S., B. P. Carlin, and A. E. Gelfand, 2004: Hierarchical Modeling and Analysis for Spatial Data. Chapman & Hall, 472 pp.

  • Casella, G., and R. Berger, 2002: Statistical Inference. Thomson Learning Pacific Grove, 700 pp.

  • Cook, E., K. Briffa, and P. Jones, 1994: Spatial regression methods in dendroclimatology: A review and comparison of two techniques. Int. J. Climatol., 14 , 379402.

    • Search Google Scholar
    • Export Citation
  • Cook, E., D. Meko, D. Stahle, and M. Cleaveland, 1999: Drought reconstructions for the continental United States. J. Climate, 12 , 11451162.

    • Search Google Scholar
    • Export Citation
  • Dempster, A. P., N. M. Laird, and D. B. Rubin, 1977: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc., 39 , 138.

    • Search Google Scholar
    • Export Citation
  • Devore, J., 1995: Probability and Statistics for Engineering and the Sciences. 4th ed. Brooks/Cole, 750 pp.

  • Fierro, R., G. Golub, P. Hansen, and D. O’Leary, 1997: Regularization by truncated total least squares. SIAM J. Sci. Comput., 18 , 12231241.

    • Search Google Scholar
    • Export Citation
  • Frost, C., and S. Thompson, 2000: Correcting for regression dilution bias: Comparison of methods for a single predictor variable. J. Roy. Stat. Soc., 163 , 173189.

    • Search Google Scholar
    • Export Citation
  • Gelman, A., J. B. Carlin, H. S. Stern, and D. B. Rubin, 2003: Bayesian Data Analysis. 2nd ed. Chapman & Hall, 696 pp.

  • Golub, G., and C. Van Loan, 1980: An analysis of the total least squares problem. SIAM J. Numer. Anal., 17 , 883893.

  • Golub, G., M. Heath, and G. Wahba, 1979: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 21 , 215223.

    • Search Google Scholar
    • Export Citation
  • Hegerl, G., and Coauthors, 2007: Understanding and attributing climate change. Climate Change 2007: The Physical Science Basis. S. Solomon et al., Eds., Cambridge University Press, 663–745.

    • Search Google Scholar
    • Export Citation
  • Hoerl, A., and R. Kennard, 1970: Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12 , 5567.

  • Jansen, E., and Coauthors, 2007: Palaeoclimate. Climate Change 2007: The Physical Science Basis, S. Solomon et al., Eds., Cambridge University Press, 433–498.

    • Search Google Scholar
    • Export Citation
  • Jones, P., and Coauthors, 2009: High-resolution palaeoclimatology of the last millennium: A review of current status and future prospects. Holocene, 19 , 349. doi:10.1177/0959683608098952.

    • Search Google Scholar
    • Export Citation
  • Krakauer, N., T. Schneider, J. Randerson, and S. Olsen, 2004a: Linear inversion methods and generalized cross-validation. Online supplement to “Using generalized cross-validation to select parameters in inversions for regional carbon fluxes,” 7 pp. [Available online at http://ce193.engr.ccny.cuny.edu/papers/GCV_appendix.pdf].

    • Search Google Scholar
    • Export Citation
  • Krakauer, N., T. Schneider, J. Randerson, and S. Olsen, 2004b: Using generalized cross-validation to select parameters in inversions for regional carbon fluxes. Geophys. Res. Lett., 31 , L19108. doi:10.1029/2004GL020323.

    • Search Google Scholar
    • Export Citation
  • Lee, T., F. Zwiers, and M. Tsao, 2008: Evaluation of proxy-based millennial reconstruction methods. Climate Dyn., 31 , 263281.

  • Luterbacher, J., D. Dietrich, E. Xoplaki, M. Grosjean, and H. Wanner, 2004: European seasonal and annual temperature variability, trends, and extremes since 1500. Science, 303 , 14991503.

    • Search Google Scholar
    • Export Citation
  • Mann, M., R. Bradley, and M. Hughes, 1998: Global-scale temperature patterns and climate forcing over the past six centuries. Nature, 392 , 779787.

    • Search Google Scholar
    • Export Citation
  • Mann, M., S. Rutherford, E. Wahl, and C. Ammann, 2007a: Reply. J. Climate, 20 , 56715674.

  • Mann, M., S. Rutherford, E. Wahl, and C. Ammann, 2007b: Robustness of proxy-based climate field reconstruction methods. J. Geophys. Res., 112 , D12109. doi:10.1029/2006JD008272.

    • Search Google Scholar
    • Export Citation
  • Mann, M., Z. Zhang, M. Hughes, R. Bradley, S. Miller, S. Rutherford, and F. Ni, 2008: Proxy-based reconstructions of hemispheric and global surface temperature variations over the past two millennia. Proc. Natl. Acad. Sci. USA, 105 , 1325213257.

    • Search Google Scholar
    • Export Citation
  • NRC, 2006: Surface Temperature Reconstructions for the Last 2000 Years. The National Academies Press, 160 pp.

  • Osborn, T. J., and K. R. Briffa, 2006: The spatial extent of 20th-century warmth in the context of the past 1200 years. Science, 311 , 841844.

    • Search Google Scholar
    • Export Citation
  • Rubin, D., 1976: Inference and missing data. Biometrika, 63 , 581592.

  • Rutherford, S., M. Mann, T. Delworth, and R. Stouffer, 2003: Climate field reconstruction under stationary and nonstationary forcing. J. Climate, 16 , 462479.

    • Search Google Scholar
    • Export Citation
  • Rutherford, S., M. Mann, T. Osborn, R. Bradley, K. Briffa, M. Hughes, and P. Jones, 2005: Proxy-based Northern Hemisphere surface temperature reconstructions: Sensitivity to method, predictor network, target season, and target domain. J. Climate, 18 , 23082329.

    • Search Google Scholar
    • Export Citation
  • Schneider, T., 2001: Analysis of incomplete climate data: Estimation of mean values and covariance matrices and imputation of missing values. J. Climate, 14 , 853871.

    • Search Google Scholar
    • Export Citation
  • Smerdon, J., and A. Kaplan, 2007: Comments on “Testing the fidelity of methods used in proxy-based reconstructions of past climate: The role of the standardization interval”. J. Climate, 20 , 56665670.

    • Search Google Scholar
    • Export Citation
  • Smerdon, J., A. Kaplan, and D. Chang, 2008: On the origin of the standardization sensitivity in RegEM climate field reconstructions. J. Climate, 21 , 67106723.

    • Search Google Scholar
    • Export Citation
  • Steig, E., D. Schneider, S. Rutherford, M. Mann, J. Comiso, and D. Shindell, 2009: Warming of the Antarctic ice-sheet surface since the 1957 International Geophysical Year. Nature, 457 , 459462.

    • Search Google Scholar
    • Export Citation
  • Tikhonov, A., and V. Arsenin, 1977: Solutions of Ill-Posed Problems. Winston, 258 pp.

  • Tingley, M., 2009: A Bayesian approach to reconstructing space–time climate fields from proxy and instrumental time series, applied to 600 years of Northern Hemisphere surface temperature data. Ph.D. dissertation, Harvard University, 230 pp.

  • Tingley, M., and P. Huybers, 2010: A Bayesian algorithm for reconstructing climate anomalies in space and time. Part I: Development and applications to paleoclimate reconstruction problems. J. Climate, 23 , 27592781.

    • Search Google Scholar
    • Export Citation
  • van Huffel, S., and J. Vandewalle, 1991: The Total Least Squares Problem: Computational Aspects and Analysis. Society for Industrial Mathematics, 314 pp.

    • Search Google Scholar
    • Export Citation
  • von Storch, H., E. Zorita, and F. González-Rouco, 2009: Assessment of three temperature reconstruction methods in the virtual reality of a climate simulation. Int. J. Earth Sci., 98 , 6782.

    • Search Google Scholar
    • Export Citation
  • Zar, J. H., 1999: Biostatistical Analysis. 4th ed. Pearson Education, 929 pp.

  • Zhang, Z., M. Mann, and E. Cook, 2004: Alternative methods of proxy-based climate field reconstruction: Application to summer drought over the conterminous United States back to AD 1700 from tree-ring data. Holocene, 14 , 502516.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 773 336 10
PDF Downloads 341 121 7