Quality Control of Weather Data during Extreme Events

Jinsheng You High Plains Regional Climate Center, University of Nebraska at Lincoln, Lincoln, Nebraska

Search for other papers by Jinsheng You in
Current site
Google Scholar
PubMed
Close
and
Kenneth G. Hubbard High Plains Regional Climate Center, University of Nebraska at Lincoln, Lincoln, Nebraska

Search for other papers by Kenneth G. Hubbard in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Quality assurance (QA) procedures have been automated to reduce the time and labor necessary to discover outliers in weather data. Measurements from neighboring stations are used in this study in a spatial regression test to provide preliminary estimates of the measured data points. The new method does not assign the largest weight to the nearest estimate but, instead, assigns the weights according to the standard error of estimate. In this paper, the spatial test was employed to study patterns in flagged data in the following extreme events: the 1993 Midwest floods, the 2002 drought, Hurricane Andrew (1992), and a series of cold fronts during October 1990. The location of flagged records and the influence zones for such events relative to QA were compared. The behavior of the spatial test in these events provides important information on the probability of making a type I error in the assignment of the quality control flag. Simple pattern recognition tools that identify zones wherein frequent flagging occurs are illustrated. These tools serve as a means of resetting QA flags to minimize the number of type I errors as demonstrated for the extreme events included here.

Corresponding author address: Jinsheng You, High Plains Regional Climate Center, University of Nebraska at Lincoln, Lincoln, NE 68583-0728. Email: jyou2@unl.edu

Abstract

Quality assurance (QA) procedures have been automated to reduce the time and labor necessary to discover outliers in weather data. Measurements from neighboring stations are used in this study in a spatial regression test to provide preliminary estimates of the measured data points. The new method does not assign the largest weight to the nearest estimate but, instead, assigns the weights according to the standard error of estimate. In this paper, the spatial test was employed to study patterns in flagged data in the following extreme events: the 1993 Midwest floods, the 2002 drought, Hurricane Andrew (1992), and a series of cold fronts during October 1990. The location of flagged records and the influence zones for such events relative to QA were compared. The behavior of the spatial test in these events provides important information on the probability of making a type I error in the assignment of the quality control flag. Simple pattern recognition tools that identify zones wherein frequent flagging occurs are illustrated. These tools serve as a means of resetting QA flags to minimize the number of type I errors as demonstrated for the extreme events included here.

Corresponding author address: Jinsheng You, High Plains Regional Climate Center, University of Nebraska at Lincoln, Lincoln, NE 68583-0728. Email: jyou2@unl.edu

Save
  • Allen, R. G., 1996: Assessing integrity of weather data for reference evapotranspiration estimation. J. Irrig. Drainage Eng, 122 , 97106.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Eischeid, J. K., Baker C. B. , Karl T. , and Diaz H. F. , 1995: The quality control of long-term climatological data using objective data analysis. J. Appl. Meteor, 34 , 27872795.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Eischeid, J. K., Pasteris P. A. , Diaz H. F. , Plantico M. S. , and Lott N. J. , 2000: Creating a serially complete, national daily time series of temperature and precipitation for the western United States. J. Appl. Meteor, 39 , 15801591.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feng, S., Hu Q. , and Wang W. , 2004: Quality control of daily meteorological data in China 1951–2000: A new dataset. Int. J. Climatol, 24 , 853870.

  • Gandin, L. S., 1988: Complex quality control of meteorological observations. Mon. Wea. Rev, 116 , 11371156.

  • Geiger, M., Diabate L. , Menard L. , and Wald L. , 2002: A Web service for controlling the quality of measurements of global solar irradiation. Sol. Energy, 73 , 475480.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guttman, N. V., and Quayle R. G. , 1990: A review of cooperative temperature data validation. J. Atmos. Oceanic Technol, 7 , 334339.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guttman, N. V., Karl C. , Reek T. , and Shuler V. , 1988: Measuring the performance of data validators. Bull. Amer. Meteor. Soc, 69 , 14481452.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hu, Q., Feng S. , and Schaefer G. , 2002: Quality control for USDA NRCS SM-ST network soil temperatures: A method and dataset. J. Appl. Meteor, 41 , 607619.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hubbard, K. G., 2001: Automated Weather Stations for Applications in Agriculture and Water Resources Management. AGM-3 WMO/TD No. 1074, 248 pp.

  • Hubbard, K. G., and You J. , 2005: Sensitivity analysis of quality assurance using a spatial regression approach—A case study of the maximum/minimum air temperature. J. Atmos. Oceanic Technol, 22 , 15201530.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hubbard, K. G., DeGaetano A. T. , and Robbins K. D. , 2004: Announcing a modern Applied Climatic Information System (ACIS). Bull. Amer. Meteor. Soc, 85 , 811812.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hubbard, K. G., Goddard S. , Sorensen W. D. , Wells N. , and Osugi T. T. , 2005: Performance of quality assurance procedures for an Applied Climate Information System. J. Atmos. Oceanic Technol, 22 , 105112.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jain, A. K., Duin R. R. W. , and Mao J. , 2000: Statistical pattern recognition: A review. IEEE Trans. Pattern Anal. Mach. Intell, 22 , 437.

  • Lott, J. N., 1994: The U.S. summer of 1993: A sharp contrast in weather extremes. Weather, 49 , 11. 370383.

  • Meek, D. W., and Hatfield J. L. , 1994: Data quality checking for single station meteorological databases. Agric. For. Meteor, 69 , 85109.

  • NCDC, cited. 2005: Extreme weather and climate events. [Available online at http://www.ncdc.noaa.gov/oa/climate/severeweather/extremes.html.].

  • Wade, C. G., 1987: A quality control program for surface mesometeorological data. J. Atmos. Oceanic Technol, 4 , 435453.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 930 290 21
PDF Downloads 638 137 5