Homogenization of Daily Temperature Data

Anuradha P. Hewaarachchi Department of Statistics and Computer Science, University of Kelaniya, Kelaniya, Sri Lanka

Search for other papers by Anuradha P. Hewaarachchi in
Current site
Google Scholar
PubMed
Close
,
Yingbo Li Department of Mathematical Sciences, Clemson University, Clemson, South Carolina, and Department of Statistical Science, Southern Methodist University, Dallas, Texas

Search for other papers by Yingbo Li in
Current site
Google Scholar
PubMed
Close
,
Robert Lund Department of Mathematical Sciences, Clemson University, Clemson, South Carolina

Search for other papers by Robert Lund in
Current site
Google Scholar
PubMed
Close
, and
Jared Rennie Cooperative Institute for Climate and Satellites—North Carolina, North Carolina State University, Raleigh, and NOAA/National Center for Environmental Information, Asheville, North Carolina

Search for other papers by Jared Rennie in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

This paper develops a method for homogenizing daily temperature series. While daily temperatures are statistically more complex than annual or monthly temperatures, techniques and computational methods have been accumulating that can now model and analyze all salient statistical characteristics of daily temperature series. The goal here is to combine these techniques in an efficient manner for multiple changepoint identification in daily series; computational speed is critical as a century of daily data has over 36 500 data points. The method developed here takes into account 1) metadata, 2) reference series, 3) seasonal cycles, and 4) autocorrelation. Autocorrelation is especially important: ignoring it can degrade changepoint techniques, and sample autocorrelations of day-to-day temperature anomalies are often as large as 0.7. While daily homogenization is not conducted as commonly as monthly or annual homogenization, daily analyses provide greater detection precision as they are roughly 30 times as long as monthly records. For example, it is relatively easy to detect two changepoints less than two years apart with daily data, but virtually impossible to flag these in corresponding annually averaged data. The developed methods are shown to work in simulation studies and applied in the analysis of 46 years of daily temperatures from South Haven, Michigan.

Corresponding author e-mail: Robert Lund, lund@clemson.edu

Abstract

This paper develops a method for homogenizing daily temperature series. While daily temperatures are statistically more complex than annual or monthly temperatures, techniques and computational methods have been accumulating that can now model and analyze all salient statistical characteristics of daily temperature series. The goal here is to combine these techniques in an efficient manner for multiple changepoint identification in daily series; computational speed is critical as a century of daily data has over 36 500 data points. The method developed here takes into account 1) metadata, 2) reference series, 3) seasonal cycles, and 4) autocorrelation. Autocorrelation is especially important: ignoring it can degrade changepoint techniques, and sample autocorrelations of day-to-day temperature anomalies are often as large as 0.7. While daily homogenization is not conducted as commonly as monthly or annual homogenization, daily analyses provide greater detection precision as they are roughly 30 times as long as monthly records. For example, it is relatively easy to detect two changepoints less than two years apart with daily data, but virtually impossible to flag these in corresponding annually averaged data. The developed methods are shown to work in simulation studies and applied in the analysis of 46 years of daily temperatures from South Haven, Michigan.

Corresponding author e-mail: Robert Lund, lund@clemson.edu
Save
  • Beasley, D., D. R. Bull, and R. R. Martin, 1993: An overview of genetic algorithms: Part 1, fundamentals. Univ. Comput., 15, 58–69.

    • Search Google Scholar
    • Export Citation
  • Caussinus, H., and O. Mestre, 2004: Detection and correction of artificial shifts in climate series. J. Roy. Stat. Soc., 53C, 405–425, doi:10.1111/j.1467-9876.2004.05155.x.

    • Search Google Scholar
    • Export Citation
  • Cerf, R., 1998: Asymptotic convergence of genetic algorithms. Adv. Appl. Probab., 30, 521–550, doi:10.1017/S0001867800047418.

  • Chan, N. H., C. Y. Yau, and R.-M. Zhang, 2014: Group LASSO for structural break time series. J. Amer. Stat. Assoc., 109, 590–599, doi:10.1080/01621459.2013.866566.

    • Search Google Scholar
    • Export Citation
  • Davis, R. A., T. C. M. Lee, and G. A. Rodrigues-Yam, 2006: Structural break estimation for nonstationary time series models. J. Amer. Stat. Assoc., 101, 223–239, doi:10.1198/016214505000000745.

    • Search Google Scholar
    • Export Citation
  • Della-Marta, P. M., and H. Wanner, 2006: A method of homogenizing the extremes and mean of daily temperature measurements. J. Climate, 19, 4179–4197, doi:10.1175/JCLI3855.1.

    • Search Google Scholar
    • Export Citation
  • Fryzlewicz, P., 2014: Wild binary segmentation for multiple change-point detection. Ann. Stat., 42, 2243–2281, doi:10.1214/14-AOS1245.

    • Search Google Scholar
    • Export Citation
  • Gallagher, C., R. Lund, and M. Robbins, 2012: Changepoint detection in daily precipitation series. Environmetrics, 23, 407–419, doi:10.1002/env.2146.

    • Search Google Scholar
    • Export Citation
  • Goldberg, D. E., and J. H. Holland, 1988: Genetic algorithms and machine learning. Mach. Learn., 3, 95–99, doi:10.1023/A:1022602019183.

    • Search Google Scholar
    • Export Citation
  • Grünwald, P. D., I. J. Myung, and M. A. Pitt, 2005: Advances in Minimum Description Length: Theory and Applications. MIT Press, 444 pp.

  • Hansen, M. H., and B. Yu, 2001: Model selection and the principle of minimum description lengths. J. Amer. Stat. Assoc., 96, 746–774, doi:10.1198/016214501753168398.

    • Search Google Scholar
    • Export Citation
  • Kuglitsch, F. G., A. Toreti, E. Xoplaki, P. M. Della-Marta, J. Luterbacher, and H. Wanner, 2009: Homogenization of daily maximum temperature series in the Mediterranean. J. Geophys. Res., 114, D15108, doi:10.1029/2008JD011606.

  • Li, S., and R. Lund, 2012: Multiple changepoint detection via genetic algorithms. J. Climate, 25, 674–686, doi:10.1175/2011JCLI4055.1.

    • Search Google Scholar
    • Export Citation
  • Li, Y., and R. Lund, 2015: Multiple changepoint detection using metadata. J. Climate, 28, 4199–4216, doi:10.1175/JCLI-D-14-00442.1.

    • Search Google Scholar
    • Export Citation
  • Li, Y., R. Lund, and H. A. Priyadarshani, 2016: Bayesian minimal description lengths for multiple changepoint detection. [Available online at https://arxiv.org/abs/1511.07238.]

  • Liu, G., Q. Shao, R. Lund, and J. Woody, 2016: Testing for seasonal means in time series data. Environmetrics, 27, 198–211, doi:10.1002/env.2383.

    • Search Google Scholar
    • Export Citation
  • Lu, Q., and R. Lund, 2007: Simple linear regression with multiple level shifts. Can. J. Stat., 35, 447–458, doi:10.1002/cjs.5550350308.

    • Search Google Scholar
    • Export Citation
  • Lu, Q., R. Lund, and T. Lee, 2010: An MDL approach to the climate segmentation problem. Ann. Appl. Stat., 4, 299–319, doi:10.1214/09-AOAS289.

    • Search Google Scholar
    • Export Citation
  • Lund, R., and J. Reeves, 2002: Detection of undocumented changepoints: A revision of the two-phase regression model. J. Climate, 15, 2547–2554, doi:10.1175/1520-0442(2002)015<2547:DOUCAR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Lund, R., H. Hurd, P. Bloomfield, and R. Smith, 1995: Climatological time series with periodic correlation. J. Climate, 8, 2787–2809, doi:10.1175/1520-0442(1995)008<2787:CTSWPC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Lund, R., L. Seymour, and K. Kafadar, 2001: Temperature trends in the United States. Environmetrics, 12, 673–690, doi:10.1002/env.468.

    • Search Google Scholar
    • Export Citation
  • Menne, M. J., and C. N. Williams Jr., 2005: Detection of undocumented changepoints using multiple test statistics and composite reference series. J. Climate, 18, 4271–4286, doi:10.1175/JCLI3524.1.

    • Search Google Scholar
    • Export Citation
  • Menne, M. J., and C. N. Williams Jr., 2009: Homogenization of temperature series via pairwise comparisons. J. Climate, 22, 1700–1717, doi:10.1175/2008JCLI2263.1.

    • Search Google Scholar
    • Export Citation
  • Mitchell, J. M., 1953: On the causes of instrumentally observed secular temperature trends. J. Meteor., 10, 244–261, doi:10.1175/1520-0469(1953)010<0244:OTCOIO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Potter, K. W., 1981: Illustration of a new test for detecting a shift in mean in precipitation series. Mon. Wea. Rev., 109, 2040–2045, doi:10.1175/1520-0493(1981)109<2040:IOANTF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Reeves, J., J. Chen, X. Wang, R. Lund, and Q. Q. Lu, 2007: A review and comparison of changepoint detection techniques for climate data. J. Appl. Meteor. Climatol., 46, 900–915, doi:10.1175/JAM2493.1.

    • Search Google Scholar
    • Export Citation
  • Rissanen, J., 1989: Stochastic Complexity in Statistical Inquiry. World Scientific Publishing, 188 pp.

  • Toreti, A., F. G. Kuglitsch, E. Xoplaki, and J. Luterbacher, 2012: A novel approach for the detection of inhomogeneities affecting climate time series. J. Appl. Meteor. Climatol., 51, 317–326, doi:10.1175/JAMC-D-10-05033.1.

    • Search Google Scholar
    • Export Citation
  • Trewin, B., 2013: A daily homogenized temperature data set for Australia. Int. J. Climatol., 33, 1510–1529, doi:10.1002/joc.3530.

    • Search Google Scholar
    • Export Citation
  • Venema, V., and Coauthors, 2012: Benchmarking homogenization algorithms for monthly data. Climate Past, 8, 89–115, doi:10.5194/cp-8-89-2012.

    • Search Google Scholar
    • Export Citation
  • Vincent, L. A., 1998: A technique for the identification of inhomogeneities in Canadian temperature series. J. Climate, 11, 1094–1104, doi:10.1175/1520-0442(1998)011<1094:ATFTIO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Vincent, L. A., and X. Zhang, 2002: Homogenization of daily temperatures over Canada. J. Climate, 15, 1322–1334, doi:10.1175/1520-0442(2002)015<1322:HODTOC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Wang, X. L., H. Chen, Y. Wu, Y. Feng, and Q. Pu, 2010: New techniques for the detection and adjustment of shifts in daily precipitation data series. J. Appl. Meteor. Climatol., 49, 2416–2436, doi:10.1175/2010JAMC2376.1.

    • Search Google Scholar
    • Export Citation
  • Wang, X. L., Y. Feng, and L. A. Vincent, 2014: Observed changes in one-in-20 year extremes of Canadian surface air temperatures. Atmos.-Ocean, 52, 222–231, doi:10.1080/07055900.2013.818526.

    • Search Google Scholar
    • Export Citation
  • Xu, W., Q. Li, X. L. Wang, S. Yang, L. Cao, and Y. Feng, 2013: Homogenization of Chinese daily surface air temperatures and analysis of trends in the extreme temperature indices. J. Geophys. Res., 118, 9708–9720, doi:10.1002/jgrd.50791.

    • Search Google Scholar
    • Export Citation
  • Yau, C. Y., and Z. Zhao, 2016: Inference for multiple change points in time series via likelihood ratio scan statistics. J. Roy. Stat. Soc., 78B, 895–916, doi:10.1111/rssb.12139.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 917 364 36
PDF Downloads 449 128 4