Probabilistic Verification of Storm Prediction Center Convective Outlooks

Gregory R. Herman Department of Atmospheric Science, Colorado State University, Fort Collins, Colorado

Search for other papers by Gregory R. Herman in
Current site
Google Scholar
PubMed
Close
,
Erik R. Nielsen Department of Atmospheric Science, Colorado State University, Fort Collins, Colorado

Search for other papers by Erik R. Nielsen in
Current site
Google Scholar
PubMed
Close
, and
Russ S. Schumacher Department of Atmospheric Science, Colorado State University, Fort Collins, Colorado

Search for other papers by Russ S. Schumacher in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Eight years’ worth of day 1 and 4.5 years’ worth of day 2–3 probabilistic convective outlooks from the Storm Prediction Center (SPC) are converted to probability grids spanning the continental United States (CONUS). These results are then evaluated using standard probabilistic forecast metrics including the Brier skill score and reliability diagrams. Forecasts are gridded in two different ways: one with a high-resolution grid and interpolation between probability contours and another on an 80-km-spaced grid without interpolation. Overall, the highest skill is found for severe wind forecasts and the lowest skill is observed for tornadoes; for significant severe criteria, the opposite discrepancy is observed, with highest forecast skill for significant tornadoes and approximately no overall forecast skill for significant severe winds. Highest climatology-relative skill is generally observed over the central and northern Great Plains and Midwest, with the lowest—and often negative—skill seen in the West, southern Texas, and the Atlantic Southeast. No discernible year-to-year trend in skill was identified; seasonally, forecasts verified the best in the spring and late autumn and worst in the summer and early autumn. Forecasts are also evaluated in CAPE-versus-shear parameter space; forecasts struggle most in very low shear but also in high-shear, low-CAPE environments. In aggregate, forecasts for all variables verified more skillfully using interpolated probability grids, suggesting utility in interpreting forecasts as a continuous field. Forecast reliability results depend substantially on the interpretation of the forecast fields, but day 1 and day 2–3 tornado outlooks consistently exhibit an underforecast bias.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Gregory R. Herman, gherman@atmos.colostate.edu

Abstract

Eight years’ worth of day 1 and 4.5 years’ worth of day 2–3 probabilistic convective outlooks from the Storm Prediction Center (SPC) are converted to probability grids spanning the continental United States (CONUS). These results are then evaluated using standard probabilistic forecast metrics including the Brier skill score and reliability diagrams. Forecasts are gridded in two different ways: one with a high-resolution grid and interpolation between probability contours and another on an 80-km-spaced grid without interpolation. Overall, the highest skill is found for severe wind forecasts and the lowest skill is observed for tornadoes; for significant severe criteria, the opposite discrepancy is observed, with highest forecast skill for significant tornadoes and approximately no overall forecast skill for significant severe winds. Highest climatology-relative skill is generally observed over the central and northern Great Plains and Midwest, with the lowest—and often negative—skill seen in the West, southern Texas, and the Atlantic Southeast. No discernible year-to-year trend in skill was identified; seasonally, forecasts verified the best in the spring and late autumn and worst in the summer and early autumn. Forecasts are also evaluated in CAPE-versus-shear parameter space; forecasts struggle most in very low shear but also in high-shear, low-CAPE environments. In aggregate, forecasts for all variables verified more skillfully using interpolated probability grids, suggesting utility in interpreting forecasts as a continuous field. Forecast reliability results depend substantially on the interpretation of the forecast fields, but day 1 and day 2–3 tornado outlooks consistently exhibit an underforecast bias.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Gregory R. Herman, gherman@atmos.colostate.edu
Save
  • Agee, E., and S. Childs, 2014: Adjustments in tornado counts, F-scale intensity, and path width for assessing significant tornado destruction. J. Appl. Meteor. Climatol., 53, 14941505, https://doi.org/10.1175/JAMC-D-13-0235.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Agresti, A., and B. A. Coull, 1998: Approximate is better than “exact” for interval estimation of binomial proportions. Amer. Stat., 52, 119126, https://doi.org/10.1080/00031305.1998.10480550.

    • Search Google Scholar
    • Export Citation
  • Anderson, C. J., C. K. Wikle, Q. Zhou, and J. A. Royle, 2007: Population influences on tornado reports in the United States. Wea. Forecasting, 22, 571579, https://doi.org/10.1175/WAF997.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anderson-Frey, A. K., Y. P. Richardson, A. R. Dean, R. L. Thompson, and B. T. Smith, 2016: Investigation of near-storm environments for tornado events and warnings. Wea. Forecasting, 31, 17711790, https://doi.org/10.1175/WAF-D-16-0046.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anthony, R. W., and P. W. Leftwich Jr., 1992: Trends in severe local storm watch verification at the National Severe Storms Forecast Center. Wea. Forecasting, 7, 613622, https://doi.org/10.1175/1520-0434(1992)007<0613:TISLSW>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baldwin, M. E., and J. S. Kain, 2006: Sensitivity of several performance measures to displacement error, bias, and event frequency. Wea. Forecasting, 21, 636648, https://doi.org/10.1175/WAF933.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barnes, L. R., E. C. Gruntfest, M. H. Hayden, D. M. Schultz, and C. Benight, 2007: False alarms and close calls: A conceptual model of warning accuracy. Wea. Forecasting, 22, 11401147, https://doi.org/10.1175/WAF1031.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bieringer, P., and P. S. Ray, 1996: A comparison of tornado warning lead times with and without NEXRAD Doppler radar. Wea. Forecasting, 11, 4752, https://doi.org/10.1175/1520-0434(1996)011<0047:ACOTWL>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bothwell, P., J. Hart, and R. Thompson, 2002: An integrated three-dimensional objective analysis scheme in use at the Storm Prediction Center. 21st Conf. on Severe Local Storms/19th Conf. on Weather Analysis and Forecasting/15th Conf. on Numerical Weather Prediction, San Antonio, TX, Amer. Meteor. Soc., JP3.1, https://ams.confex.com/ams/pdfpapers/47482.pdf.

  • Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78, 13, https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brimelow, J. C., G. W. Reuter, R. Goodson, and T. W. Krauss, 2006: Spatial forecasts of maximum hail size using prognostic model soundings and HAILCAST. Wea. Forecasting, 21, 206219, https://doi.org/10.1175/WAF915.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bröcker, J., and L. A. Smith, 2007: Increasing the reliability of reliability diagrams. Wea. Forecasting, 22, 651661, https://doi.org/10.1175/WAF993.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brotzge, J., S. Erickson, and H. Brooks, 2011: A 5-yr climatology of tornado false alarms. Wea. Forecasting, 26, 534544, https://doi.org/10.1175/WAF-D-10-05004.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Childs, C., 2004: Interpolating surfaces in ArcGIS spatial analyst. ArcUser, ESRI, Redlands, CA, http://www.esri.com/news/arcuser/0704/files/interpolating.pdf.

  • Davis, J. M., and M. D. Parker, 2014: Radar climatology of tornadic and nontornadic vortices in high-shear, low-CAPE environments in the mid-Atlantic and southeastern United States. Wea. Forecasting, 29, 828853, https://doi.org/10.1175/WAF-D-13-00127.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dean, A. R., R. S. Schneider, R. L. Thompson, J. Hart, and P. D. Bothwell, 2009: The conditional risk of severe convection estimated from archived NWS/Storm Prediction Center mesoscale objective analyses: Potential uses in support of forecast operations and verification. 23rd Conf. on Weather Analysis and Forecasting/19th Conf. on Numerical Weather Prediction, Omaha, NE, Amer. Meteor. Soc., 6A.5, https://ams.confex.com/ams/pdfpapers/154304.pdf.

  • Doswell, C. A., III, 2007: Small sample size and data quality issues illustrated using tornado occurrence data. Electron. J. Severe Storms Meteor., 2 (5), http://www.ejssm.org/ojs/index.php/ejssm/article/viewArticle/26/27.

    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., III, R. Davies-Jones, and D. L. Keller, 1990: On summary measures of skill in rare event forecasting based on contingency tables. Wea. Forecasting, 5, 576585, https://doi.org/10.1175/1520-0434(1990)005<0576:OSMOSI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., III, S. J. Weiss, and R. H. Johns, 1993: Tornado forecasting: A review. The Tornado: Its Structure, Dynamics, Prediction, and Hazards, Geophys. Monogr., Vol. 79, Amer. Geophys. Union, 557–571.

    • Crossref
    • Export Citation
  • Drobot, S., and D. J. Parker, 2007: Advances and challenges in flash flood warnings. Environ. Hazards, 7, 173178, https://doi.org/10.1016/j.envhaz.2007.09.001.

    • Search Google Scholar
    • Export Citation
  • Edwards, R., and G. W. Carbin, 2016: Estimated convective winds: Reliability and effects on severe-storm climatology. 28th Conf. on Severe Local Storms, Portland, OR, Amer. Meteor. Soc., 14B.6, https://ams.confex.com/ams/28SLS/webprogram/Paper300279.html.

  • Edwards, R., G. W. Carbin, and S. F. Corfidi, 2015: Overview of the Storm Prediction Center. 13th History Symp., Phoenix, AZ, Amer. Meteor. Soc., 1.1, https://ams.confex.com/ams/95Annual/webprogram/Paper266329.html.

  • Evans, J. S., and C. A. Doswell III, 2001: Examination of derecho environments using proximity soundings. Wea. Forecasting, 16, 329342, https://doi.org/10.1175/1520-0434(2001)016<0329:EODEUP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ferree, J., 2009: National change of the hail criteria for severe storms from 3/4 inch to 1 inch beginning January 5, 2010. National Weather Service, 8 pp., http://www.nws.noaa.gov/oneinchhail/docs/One_Inch_Hail.pdf.

  • Gagne, D. J., A. McGovern, S. E. Haupt, R. A. Sobash, J. K. Williams, and M. Xue, 2017: Storm-based probabilistic hail forecasting with machine learning applied to convection-allowing ensembles. Wea. Forecasting, 32, 18191840, https://doi.org/10.1175/WAF-D-17-0010.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gallus, W. A., Jr., N. A. Snook, and E. V. Johnson, 2008: Spring and summer severe weather reports over the Midwest as a function of convective mode: A preliminary study. Wea. Forecasting, 23, 101113, https://doi.org/10.1175/2007WAF2006120.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gensini, V. A., and W. S. Ashley, 2011: Climatology of potentially severe convective environments from the North American Regional Reanalysis. Electron. J. Severe Storms Meteor., 6 (8), http://www.ejssm.org/ojs/index.php/ejssm/article/viewArticle/85.

    • Search Google Scholar
    • Export Citation
  • Gensini, V. A., T. L. Mote, and H. E. Brooks, 2014: Severe-thunderstorm reanalysis environments and collocated radiosonde observations. J. Appl. Meteor. Climatol., 53, 742751, https://doi.org/10.1175/JAMC-D-13-0263.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gourley, J. J., and Coauthors, 2013: A unified flash flood database across the United States. Bull. Amer. Meteor. Soc., 94, 799805, https://doi.org/10.1175/BAMS-D-12-00198.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hales, J., Jr., 1988: Improving the watch/warning program through use of significant event data. Preprints, 15th Conf. on Severe Local Storms, Baltimore, MD, Amer. Meteor. Soc., 165–168.

  • Hamill, T. M., and J. Juras, 2006: Measuring forecast skill: Is it real skill or is it the varying climatology? Quart. J. Roy. Meteor. Soc., 132, 29052924, https://doi.org/10.1256/qj.06.25.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hart, J. A., and A. E. Cohen, 2016: The challenge of forecasting significant tornadoes from June to October using convective parameters. Wea. Forecasting, 31, 20752084, https://doi.org/10.1175/WAF-D-16-0005.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Herman, G. R., and R. S. Schumacher, 2016: Extreme precipitation in models: An evaluation. Wea. Forecasting, 31, 18531879, https://doi.org/10.1175/WAF-D-16-0093.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hitchens, N. M., and H. E. Brooks, 2012: Evaluation of the Storm Prediction Center’s day 1 convective outlooks. Wea. Forecasting, 27, 15801585, https://doi.org/10.1175/WAF-D-12-00061.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hitchens, N. M., and H. E. Brooks, 2014: Evaluation of the Storm Prediction Center’s convective outlooks from day 3 through day 1. Wea. Forecasting, 29, 11341142, https://doi.org/10.1175/WAF-D-13-00132.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hitchens, N. M., and H. E. Brooks, 2017: Determining criteria for missed events to evaluate significant severe convective outlooks. Wea. Forecasting, 32, 13211328, https://doi.org/10.1175/WAF-D-16-0170.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jacks, E., 2014: Service change notice 14-42. Fire and Public Weather Services Branch, National Weather Service, http://www.nws.noaa.gov/os/notification/scn14-42day1-3outlooks_cca.htm.

  • Kain, J. S., S. J. Weiss, J. J. Levit, M. E. Baldwin, and D. R. Bright, 2006: Examination of convection-allowing configurations of the WRF Model for the prediction of severe convective weather: The SPC/NSSL Spring Program 2004. Wea. Forecasting, 21, 167181, https://doi.org/10.1175/WAF906.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kay, M. P., and H. E. Brooks, 2000: Verification of probabilistic severe storm forecasts at the SPC. Preprints, 20th Conf. on Severe Local Storms, Orlando, FL, Amer. Meteor. Soc, 9.3.

  • Lackmann, G., 2011: Midlatitude Synoptic Meteorology. Amer. Meteor. Soc., 360 pp.

    • Crossref
    • Export Citation
  • Markowski, P., and Y. Richardson, 2010: Mesoscale Meteorology in Midlatitudes. John Wiley and Sons, 424 pp.

    • Crossref
    • Export Citation
  • McGovern, A., D. J. Gagne, J. K. Williams, R. A. Brown, and J. B. Basara, 2014: Enhancing understanding and improving prediction of severe weather through spatiotemporal relational learning. Mach. Learn., 95, 2750, https://doi.org/10.1007/s10994-013-5343-x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McGovern, A., K. L. Elmore, D. J. Gagne, S. E. Haupt, C. D. Karstens, R. Lagerquist, T. Smith, and J. K. Williams, 2017: Using artificial intelligence to improve real-time decision-making for high-impact weather. Bull. Amer. Meteor. Soc., 98, 20732090, https://doi.org/10.1175/BAMS-D-16-0123.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mesinger, F., and Coauthors, 2006: North American Regional Reanalysis. Bull. Amer. Meteor. Soc., 87, 343360, https://doi.org/10.1175/BAMS-87-3-343.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and R. L. Winkler, 1977: Reliability of subjective probability forecasts of precipitation and temperature. Appl. Stat., 26, 4147, https://doi.org/10.2307/2346866.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nielsen, E. R., and R. S. Schumacher, 2016: Using convection-allowing ensembles to understand the predictability of an extreme rainfall event. Mon. Wea. Rev., 144, 36513676, https://doi.org/10.1175/MWR-D-16-0083.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nielsen, E. R., G. R. Herman, R. C. Tournay, J. M. Peters, and R. S. Schumacher, 2015: Double impact: When both tornadoes and flash floods threaten the same place at the same time. Wea. Forecasting, 30, 16731693, https://doi.org/10.1175/WAF-D-15-0084.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • NWS, 2012: Technical implementation notice 11-53. National Weather Service Headquarters, http://www.nws.noaa.gov/os/notification/tin11-53ruc_rapaae.htm.

  • NWS, 2017a: Weather fatalities 2016. Office of Climate, Weather, and Water Services, National Weather Service, http://www.nws.noaa.gov/om/hazstats.shtml.

  • NWS, 2017b: Service change notice 17-100. National Centers for Environmental Prediction, Weather Prediction Center, http://www.nws.noaa.gov/os/notification/scn17-100wpc_excessive_rainfall.htm.

  • Polger, P. D., B. S. Goldsmith, R. C. Przywarty, and J. R. Bocchieri, 1994: National Weather Service warning performance based on the WSR-88D. Bull. Amer. Meteor. Soc., 75, 203214, https://doi.org/10.1175/1520-0477(1994)075<0203:NWSWPB>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schneider, R. S., and A. R. Dean, 2008: A comprehensive 5-year severe storm environment climatology for the continental United States. 24th Conf. on Severe Local Storms, Savannah, GA, Amer. Meteor. Soc., 16A.4, https://ams.confex.com/ams/24SLS/techprogram/paper_141748.htm.

  • Schroeder, A. J., and Coauthors, 2016: The development of a flash flood severity index. J. Hydrol., 541, 523532, https://doi.org/10.1016/j.jhydrol.2016.04.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sherburn, K. D., and M. D. Parker, 2014: Climatology and ingredients of significant severe convection in high-shear, low-CAPE environments. Wea. Forecasting, 29, 854877, https://doi.org/10.1175/WAF-D-13-00041.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sherburn, K. D., M. D. Parker, J. R. King, and G. M. Lackmann, 2016: Composite environments of severe and nonsevere high-shear, low-CAPE convective events. Wea. Forecasting, 31, 18991927, https://doi.org/10.1175/WAF-D-16-0086.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Simmons, K. M., and D. Sutter, 2005: WSR-88D radar, tornado warnings, and tornado casualties. Wea. Forecasting, 20, 301310, https://doi.org/10.1175/WAF857.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Simmons, K. M., and D. Sutter, 2008: Tornado warnings, lead times, and tornado casualties: An empirical investigation. Wea. Forecasting, 23, 246258, https://doi.org/10.1175/2007WAF2006027.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sobash, R. A., J. S. Kain, D. R. Bright, A. R. Dean, M. C. Coniglio, and S. J. Weiss, 2011: Probabilistic forecast guidance for severe thunderstorms based on the identification of extreme phenomena in convection-allowing model forecasts. Wea. Forecasting, 26, 714728, https://doi.org/10.1175/WAF-D-10-05046.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sobash, R. A., G. S. Romine, C. S. Schwartz, D. J. Gagne, and M. L. Weisman, 2016a: Explicit forecasts of low-level rotation from convection-allowing models for next-day tornado prediction. Wea. Forecasting, 31, 15911614, https://doi.org/10.1175/WAF-D-16-0073.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sobash, R. A., C. S. Schwartz, G. S. Romine, K. R. Fossell, and M. L. Weisman, 2016b: Severe weather prediction using storm surrogates from an ensemble forecasting system. Wea. Forecasting, 31, 255271, https://doi.org/10.1175/WAF-D-15-0138.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • SPC, 2017a: SPC convective outlooks. Storm Prediction Center, http://www.spc.noaa.gov/cgi-bin-spc/getacrange.pl.

  • SPC, 2017b: SVRGIS (updated: 15 May 2017). Storm Prediction Center, http://www.spc.noaa.gov/gis/svrgis/.

  • SPC, 2017c: Severe weather climatology (1982–2011). Storm Prediction Center, http://www.spc.noaa.gov/new/SVRclimo/climo.php?parm=anySvr.

  • Stephenson, D., B. Casati, C. Ferro, and C. Wilson, 2008: The extreme dependency score: A non-vanishing measure for forecasts of rare events. Meteor. Appl., 15, 4150, https://doi.org/10.1002/met.53.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stough, S., E. Leitman, J. Peters, and J. Correia Jr., 2010: The role of Storm Prediction Center products in decision making leading up to severe weather events. Storm Prediction Center, 14 pp., http://www.spc.noaa.gov/publications/leitman/stough.pdf.

  • Surcel, M., I. Zawadzki, and M. Yau, 2016: The case-to-case variability of the predictability of precipitation by a storm-scale ensemble forecasting system. Mon. Wea. Rev., 144, 193212, https://doi.org/10.1175/MWR-D-15-0232.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Trapp, R. J., D. M. Wheatley, N. T. Atkins, R. W. Przybylinski, and R. Wolf, 2006: Buyer beware: Some words of caution on the use of severe wind reports in postevent assessment and research. Wea. Forecasting, 21, 408415, https://doi.org/10.1175/WAF925.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaughan, M. T., B. H. Tang, and L. F. Bosart, 2017: Climatology and analysis of high-impact, low predictive skill severe weather events in the northeast United States. Wea. Forecasting, 32, 19031919, https://doi.org/10.1175/WAF-D-17-0044.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Verbout, S. M., H. E. Brooks, L. M. Leslie, and D. M. Schultz, 2006: Evolution of the U.S. tornado database: 1954–2003. Wea. Forecasting, 21, 8693, https://doi.org/10.1175/WAF910.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vescio, M. D., and R. L. Thompson, 2001: Subjective tornado probability forecasts in severe weather watches. Wea. Forecasting, 16, 192195, https://doi.org/10.1175/1520-0434(2001)016<0192:FSFSTP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Elsevier, 676 pp.

    • Crossref
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1903 430 62
PDF Downloads 904 247 47