Editorial Type:
Article
Article Type:
Research Article

Assessing the Short-Term Forecast Capability of Nonstandardized Surface Observations Using the National Digital Forecast Database (NDFD)

Joby L. Hilliker Department of Geology and Astronomy, West Chester University, West Chester, Pennsylvania

Search for other papers by Joby L. Hilliker in
Current site
Google Scholar
PubMed Close
,
Girish Akasapu Department of Computer Science, West Chester University, West Chester, Pennsylvania

Search for other papers by Girish Akasapu in
Current site
Google Scholar
PubMed Close
, and
George S. Young Department of Meteorology, The Pennsylvania State University, University Park, Pennsylvania

Search for other papers by George S. Young in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Jul 2010
DOI:
https://doi.org/10.1175/2010JAMC2137.1
Page(s):
1397–1411
Restricted access

Abstract

The number of surface observations from nonstandardized networks across the United States has appreciably increased the last several years. Automated Weather Services, Inc. (AWS), maintains one example of this type of network offering nonstandardized observations for ∼8000 sites. The present study assesses the utility of such a network to improve short-term (i.e., lead times <12 h) National Digital Forecast Database (NDFD) forecasts for three parameters most relevant to the energy industry—temperature, dewpoint, and wind speed. A 1-yr sample of 13 AWS sites is chosen to evaluate the magnitude of forecast improvement (skill) and influence of physical location (siting) on such improvements. Hourly predictions are generated using generalized additive modeling (GAM)—a nonlinear statistical equation incorporating a predetermined set of the most significant AWS and NDFD predictors. Two references are used for comparison: (i) persistence climatology (PC) forecasts and (ii) NDFD forecasts calibrated to the AWS sites (CNDFD). The skill, measured via the percent improvement (reduction) in the mean absolute error (MAE), of forecasts generated by the study’s technique (CNDFD+) is comparable (<5%) to PC for lead times of 1–3 h for dewpoint and wind speed. Skill relative to PC slowly increases with lead time, with temperature exhibiting the greatest relative-to-PC skill (∼30% at 12 h). When compared to baseline CNDFD forecasts, the MAE of the generated CNDFD+ forecasts is reduced 65% for temperature and dewpoint at the 1-h lead time. An exponential drop in improvement occurs for longer lead times. Wind speed improvements are notably less, with little skill (<5%) demonstrated for forecasts beyond 4 h. Overall, CNDFD+ forecasts have the greatest accuracy relative to CNDFD and PC for the middle (3–7 h) lead times tested in the study. Variations in CNDFD+ skill exist with respect to AWS location. Tested stations located in complex terrain generally exhibit greater skill relative to CNDFD than the 13-station average for temperature (and, to a lesser degree, dewpoint). Relative to PC, however, the same subset of stations exhibits skill below the 13-station average. No conclusive relationship can be made between CNDFD+ skill and the sample stations located near water.

Corresponding author address: Joby L. Hilliker, Department of Geology and Astronomy, West Chester University, 223 Merion Science Building, West Chester, PA 19383. Email: jhilliker@wcupa.edu

Abstract

The number of surface observations from nonstandardized networks across the United States has appreciably increased the last several years. Automated Weather Services, Inc. (AWS), maintains one example of this type of network offering nonstandardized observations for ∼8000 sites. The present study assesses the utility of such a network to improve short-term (i.e., lead times <12 h) National Digital Forecast Database (NDFD) forecasts for three parameters most relevant to the energy industry—temperature, dewpoint, and wind speed. A 1-yr sample of 13 AWS sites is chosen to evaluate the magnitude of forecast improvement (skill) and influence of physical location (siting) on such improvements. Hourly predictions are generated using generalized additive modeling (GAM)—a nonlinear statistical equation incorporating a predetermined set of the most significant AWS and NDFD predictors. Two references are used for comparison: (i) persistence climatology (PC) forecasts and (ii) NDFD forecasts calibrated to the AWS sites (CNDFD). The skill, measured via the percent improvement (reduction) in the mean absolute error (MAE), of forecasts generated by the study’s technique (CNDFD+) is comparable (<5%) to PC for lead times of 1–3 h for dewpoint and wind speed. Skill relative to PC slowly increases with lead time, with temperature exhibiting the greatest relative-to-PC skill (∼30% at 12 h). When compared to baseline CNDFD forecasts, the MAE of the generated CNDFD+ forecasts is reduced 65% for temperature and dewpoint at the 1-h lead time. An exponential drop in improvement occurs for longer lead times. Wind speed improvements are notably less, with little skill (<5%) demonstrated for forecasts beyond 4 h. Overall, CNDFD+ forecasts have the greatest accuracy relative to CNDFD and PC for the middle (3–7 h) lead times tested in the study. Variations in CNDFD+ skill exist with respect to AWS location. Tested stations located in complex terrain generally exhibit greater skill relative to CNDFD than the 13-station average for temperature (and, to a lesser degree, dewpoint). Relative to PC, however, the same subset of stations exhibits skill below the 13-station average. No conclusive relationship can be made between CNDFD+ skill and the sample stations located near water.

Corresponding author address: Joby L. Hilliker, Department of Geology and Astronomy, West Chester University, 223 Merion Science Building, West Chester, PA 19383. Email: jhilliker@wcupa.edu

Save
Cover Journal of Applied Meteorology and Climatology
Journal of Applied Meteorology and Climatology

  • AWS, 2007: WeatherBug Weather Station Surface Observing System: User’s guide. WeatherBug Professional, 66 pp. [Available from AWS Convergence Technologies, Inc., 12410 Milestone Center Drive, Suite 300, Germantown, MD 20876].

    • Search Google Scholar
    • Export Citation

  • AWS, cited. 2010: About us: WeatherBug overview. [Available online at http://weather.weatherbug.com/about-us/about-us.html].

  • Baggaley, D. G., and J. M. Hanesiak, 2005: An empirical blowing snow forecast technique for the Canadian Arctic and the Prairie Provinces. Wea. Forecasting, 20 , 51–62.

    • Search Google Scholar
    • Export Citation

  • Benjamin, S. G., B. E. Schwartz, and R. E. Cole, 1999: Accuracy of ACARS wind and temperature observations determined by collocation. Wea. Forecasting, 14 , 1032–1038.

    • Search Google Scholar
    • Export Citation

  • Benjamin, S. G., and Coauthors, 2004: An hourly assimilation–forecast cycle: The RUC. Mon. Wea. Rev., 132 , 495–518.

  • Bolzern, P., G. Fronza, and G. Brusasca, 1982: Temperature effects on the winter daily electric load. J. Appl. Meteor., 21 , 241–243.

  • Bondarenko, V., T. Ochotta, D. Saupe, and W. Wergen, 2007: The interaction between model resolution, observation resolution and observation density in data assimilation: A two-dimensional study. Preprints, 11th Symp. on Integrated Observing and Assimilation Systems for the Atmosphere, Oceans, and Land Surface (IOAS-AOLS), San Antonio, TX, Amer. Meteor. Soc., 5.19. [Available online at http://ams.confex.com/ams/pdfpapers/117655.pdf].

    • Search Google Scholar
    • Export Citation

  • Buell, C. E., 1958: Meaning of combined climate and persistence forecasts. J. Meteor., 15 , 564–565.

  • Cosgrove, R. L., and B. Sfanos, 2004: Producing MOS snowfall amount forecasts from cooperative observer reports. Preprints, 20th Conf. on Weather Analysis and Forecasting, Seattle, WA, Amer. Meteor. Soc., 6.3. [Available online at http://ams.confex.com/ams/pdfpapers/69445.pdf].

    • Search Google Scholar
    • Export Citation

  • Dagostaro, V. J., W. A. Shaffer, M. J. Schenk, J. L. Gorline, and A. A. Taylor, 2004: A prototype verification system for examining NDFD forecasts. Preprints, 17th Conf. on Probability and Statistics in the Atmospheric Sciences, Seattle, WA, Amer. Meteor. Soc., 2.7. [Available online at http://ams.confex.com/ams/pdfpapers/73589.pdf].

    • Search Google Scholar
    • Export Citation

  • Daley, R., 1993: Estimating observation error statistics for atmospheric data assimilation. Ann. Geophys., 11 , 634–647.

  • Dallavalle, J. P., and B. Glahn, 2005: Toward a gridded MOS system. Preprints, 21st Conf. on Weather Analysis and Forecasting, Washington, DC, Amer. Meteor. Soc., 13B.2. [Available online at http://ams.confex.com/ams/pdfpapers/94998.pdf].

    • Search Google Scholar
    • Export Citation

  • Dallavalle, J. P., M. C. Erickson, and J. C. Maloney III, 2004: Model Output Statistics (MOS) guidance for short-range projections. Preprints, 20th Conf. on Weather Analysis and Forecasting, Seattle, WA, Amer. Meteor. Soc., 6.1. [Available online at http://ams.confex.com/ams/pdfpapers/73764.pdf].

    • Search Google Scholar
    • Export Citation

  • De Boor, C., 1978: A Practical Guide to Splines. Springer-Verlag, 392 pp.

  • DiMego, G., 2006: WRF-NMM & GSI analysis to replace Eta model & 3DVar in NAM decision brief. National Centers for Environmental Prediction, 115 pp. [Available online at http://www.emc.ncep.noaa.gov/WRFinNAM/Update_WRF-NMM_replacing_Eta_in_NAM2.pdf].

    • Search Google Scholar
    • Export Citation

  • Du, J., and Coauthors, 2009: NCEP short-range ensemble forecast (SREF) system upgrade in 2009. Preprints, 19th Conf. on Numerical Weather Prediction and 23rd Conf. on Weather Analysis and Forecasting, Omaha, NE, Amer. Meteor. Soc., 4A.4.

    • Search Google Scholar
    • Export Citation

  • Efroymson, M. A., 1960: In Mathematical Methods for Digital Computers. John Wiley & Sons, 191 pp.

  • Fabbian, D., R. de Dear, and S. Lellyett, 2007: Application of artificial neural network forecasts to predict fog at Canberra International Airport. Wea. Forecasting, 22 , 372–381.

    • Search Google Scholar
    • Export Citation

  • Geerts, B., 2008: Dryline characteristics near Lubbock, Texas, based on radar and West Texas Mesonet data for May 2005 and May 2006. Wea. Forecasting, 23 , 392–406.

    • Search Google Scholar
    • Export Citation

  • Ghirardelli, J. E., 2005: An overview of the redeveloped Localized Aviation MOS Program (LAMP) for short-range forecasting. Preprints, 21st Conf. on Weather Analysis and Forecasting, Washington, DC, Amer. Meteor. Soc., 13B.5. [Available online at http://ams.confex.com/ams/pdfpapers/95038.pdf].

    • Search Google Scholar
    • Export Citation

  • Ghirardelli, J. E., and B. Glahn, 2010: The Meteorological Development Laboratory’s aviation weather prediction system. Wea. Forecasting, 25 , 1027–1051.

    • Search Google Scholar
    • Export Citation

  • Glahn, B., and D. P. Ruth, 2003: The new digital forecast database of the National Weather Service. Bull. Amer. Meteor. Soc., 84 , 195–201.

    • Search Google Scholar
    • Export Citation

  • Glahn, B., K. K. Gilbert, R. Cosgrove, D. Ruth, and K. Sheets, 2008: Gridded MOS guidance in the national digital guidance database. Preprints, 19th Conf. on Probability and Statistics, New Orleans, LA, Amer. Meteor. Soc., 11.3. [Available online at http://ams.confex.com/ams/pdfpapers/132526.pdf].

    • Search Google Scholar
    • Export Citation

  • Glahn, B., K. K. Gilbert, R. Cosgrove, D. Ruth, and K. Sheets, 2009: The gridding of MOS. Wea. Forecasting, 24 , 520–529.

  • Glahn, H. R., and D. A. Lowry, 1972: The use of Model Output Statistics (MOS) in objective weather forecasting. J. Appl. Meteor., 11 , 1203–1211.

    • Search Google Scholar
    • Export Citation

  • Grover-Kopec, E. K., and J. M. Fritsch, 2003: The impact of radar data on short-term forecasts of surface temperature, dewpoint depression, and wind speed. Wea. Forecasting, 18 , 1230–1241.

    • Search Google Scholar
    • Export Citation

  • Hart, K. A., W. J. Steenburgh, D. J. Onton, and A. J. Siffert, 2004: An evaluation of mesoscale-model-based Model Output Statistics (MOS) during the 2002 Olympic and Paralympic Winter Games. Wea. Forecasting, 19 , 200–218.

    • Search Google Scholar
    • Export Citation

  • Hastie, T. J., and R. J. Tibshirani, 1990: Generalized Additive Models. Chapman and Hall, 335 pp.

  • Helms, D., cited. 2005: Citizen Weather Observer Program (CWOP): Weather station siting, performance, and data quality guide. CWOP Doc., 88 pp. [Available online at http://home.comcast.net/~dshelms/CWOP_Guide.pdf].

    • Search Google Scholar
    • Export Citation

  • Hilliker, J. L., R. Vislocky, and J. M. Fritsch, 1999: An observations-based statistical system for warm-season hourly probabilistic forecasts of low ceiling at the San Francisco International Airport. J. Appl. Meteor., 38 , 1692–1705.

    • Search Google Scholar
    • Export Citation

  • Hilliker, J. L., G. S. Young, and J. M. Fritsch, 2007: An objective, statistical forecast system for short-term probabilistic forecasts of convection. Natl. Wea. Dig., 31 (1) 9–23.

    • Search Google Scholar
    • Export Citation

  • Horel, J., and Coauthors, 2002: Mesowest: Cooperative mesonets in the western United States. Bull. Amer. Meteor. Soc., 83 , 211–225.

  • Hughes, K. K., 2001: Development of MOS thunderstorm and severe thunderstorm forecast equations with multiple data sources. Preprints, 18th Conf. on Weather Analysis and Forecasting, Fort Lauderdale, FL, Amer. Meteor. Soc., 191–195.

    • Search Google Scholar
    • Export Citation

  • Illton, B. G., J. B. Basara, D. K. Fisher, R. Elliott, C. A. Fiebrich, K. C. Crawford, K. Humes, and E. Hunt, 2008: Mesoscale monitoring of soil moisture across a statewide network. J. Atmos. Oceanic Technol., 25 , 167–181.

    • Search Google Scholar
    • Export Citation

  • Janjić, T., and S. E. Cohn, 2006: Treatment of observation error due to unresolved scales in atmospheric data assimilation. Mon. Wea. Rev., 134 , 2900–2915.

    • Search Google Scholar
    • Export Citation

  • Leyton, S. M., and J. M. Fritsch, 2003: Short-term probabilistic forecasts of ceiling and visibility utilizing high-density surface weather observations. Wea. Forecasting, 18 , 891–902.

    • Search Google Scholar
    • Export Citation

  • Liu, Z., and F. Rabier, 2002: The interaction between model resolution, observation resolution and observation density in data assimilation: A one-dimensional study. Quart. J. Roy. Meteor. Soc., 128 , 1367–1386.

    • Search Google Scholar
    • Export Citation

  • Ludwig, F. L., J. Horel, and C. D. Whiteman, 2004: Using EOF analysis to identify surface wind patterns in mountain valleys. J. Appl. Meteor., 43 , 969–983.

    • Search Google Scholar
    • Export Citation

  • Martin, F. L., J. R. Borsting, F. J. Steckbeck, and A. H. Manhard Jr., 1963: Statistical prediction methods for North American anticyclones. J. Appl. Meteor., 2 , 508–516.

    • Search Google Scholar
    • Export Citation

  • Miller, P. A., M. Barth, L. Benjamin, D. Helms, M. Campbell, J. Facundo, and J. O’Sullivan, 2005: The Meteorological Assimilation and Data Ingest System (MADIS)—Providing value-added observations to the meteorological community. Preprints, 21st Conf. on Weather Analysis and Forecasting, Washington, DC, Amer. Meteor. Soc., P1.95.

    • Search Google Scholar
    • Export Citation

  • Mollner, M., 2005: Notes on the National Digital Forecast Database verification. Western Region Tech. Attachment 05-02, 8 pp. [Available online at http://www.wrh.noaa.gov/wrh/05TAs/ta0502.pdf].

    • Search Google Scholar
    • Export Citation

  • Murphy, A., 1992: Climatology, persistence, and their linear combination as standards of reference in skill scores. Wea. Forecasting, 7 , 692–698.

    • Search Google Scholar
    • Export Citation

  • Myrick, D. T., and J. D. Horel, 2006: Verification of surface temperature forecasts from the National Digital Forecast Database over the western United States. Wea. Forecasting, 21 , 869–892.

    • Search Google Scholar
    • Export Citation

  • Myrick, D. T., and J. D. Horel, 2008: Sensitivity of surface analyses over the western United States to RAWS observations. Wea. Forecasting, 23 , 145–158.

    • Search Google Scholar
    • Export Citation

  • Nadolski, V. L., 1998: Automated Surface Observing System. ASOS User’s Guide, National Oceanic and Atmospheric Administration, Department of Commerce, 56 pp. [Available online at http://www.nws.noaa.gov/asos/pdfs/aum-toc.pdf].

    • Search Google Scholar
    • Export Citation

  • National Research Council, 2008: Observing Weather and Climate from the Ground Up: A Nationwide Network of Networks. National Academy Press, 250 pp.

    • Search Google Scholar
    • Export Citation

  • Neter, J., M. H. Kutner, C. J. Nachtsheim, and W. Wasserman, 1996: Applied Linear Statistical Models. 4th ed. Irwin Professional Publishing, 1408 pp.

    • Search Google Scholar
    • Export Citation

  • NWS, 1993: Quality control incoming data. Technique Specification Package 88-21-R1 for AWIPS-90, National Oceanic and Atmospheric Administration, Department of Commerce.

    • Search Google Scholar
    • Export Citation

  • NWS, cited. 2006: National Weather Service ASOS sites. [Available online at http://www.nws.noaa.gov/mirs/public/prods/maps/map_images/national-maps/asos_us.jpg].

    • Search Google Scholar
    • Export Citation

  • Quixote Transportation Technologies, Inc., 2010: RWIS: Road/Runway Weather Information Systems, 2 pp. [Available online at http://www.qttinc.com/pdf/lit-center/RWIS%20Platforms_8-27-09.pdf].

    • Search Google Scholar
    • Export Citation

  • Robinson, P. J., 1997: Modeling utility load and temperature relationships for use with long-lead forecasts. J. Appl. Meteor., 36 , 591–598.

    • Search Google Scholar
    • Export Citation

  • Rothe, J. P., A. K. Wadhwani, and S. Wadhwani, 2009: Short term load forecasting using multi parameter regression. Int. J. Comput. Sci. Inf. Secur., 6 (2) 303–306.

    • Search Google Scholar
    • Export Citation

  • Ruth, D. P., B. Glahn, V. Dagostaro, and K. Gilbert, 2009: The performance of MOS in the digital age. Wea. Forecasting, 24 , 504–519.

  • Schmeits, M. J., K. J. Kok, D. H. P. Vogelezang, and R. M. van Westrhenen, 2008: Probabilistic forecasts of (severe) thunderstorms for the purpose of issuing a weather alarm in the Netherlands. Wea. Forecasting, 23 , 1253–1267.

    • Search Google Scholar
    • Export Citation

  • Serreze, M., M. P. Clark, R. L. Armstrong, D. A. McGinnis, and R. S. Pulwarty, 1999: Characteristics of the western United States snowpack from snowpack telemetry (SNOTEL) data. Water Resour. Res., 35 , 2145–2160.

    • Search Google Scholar
    • Export Citation

  • S-PLUS, 2001: S-PLUS 6 for Windows: User’s guide. Insightful Corporation, 688 pp.

  • Teisberg, T. J., R. F. Weiher, and A. Khotanzad, 2005: The economic value of temperature forecasts in electricity generation. Bull. Amer. Meteor. Soc., 86 , 1765–1771.

    • Search Google Scholar
    • Export Citation

  • Valor, E., V. Meneu, and V. Caselles, 2001: Daily air temperature and electricity load in Spain. J. Appl. Meteor., 40 , 1413–1421.

  • Vislocky, R. L., and J. M. Fritsch, 1995: Generalized additive models versus linear regression in generating probabilistic MOS forecasts of aviation weather parameters. Wea. Forecasting, 10 , 669–680.

    • Search Google Scholar
    • Export Citation

  • Vislocky, R. L., and J. M. Fritsch, 1997: An automated, observations-based system for short-term prediction of ceiling and visibility. Wea. Forecasting, 12 , 31–43.

    • Search Google Scholar
    • Export Citation

  • Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. Academic Press, 627 pp.

  • Zachariassen, J., K. Zeller, N. Nikolov, and T. McClelland, 2003: A review of the Forest Service Remote Automated Weather Station (RAWS) network. General Tech. Rep. RMRS-GTR-119, Rocky Mountain Research Station, U.S. Forest Service, Fort Collins, CO, 153 pp. [Available online at http://www.fs.fed.us/rm/pubs/rmrs_gtr119.pdf].

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 867 611 35
PDF Downloads 147 36 4