The mission of the global weather enterprise is to continually improve the quality of weather information products. Verification—the act of comparing a prediction against a truth—is an essential activity in all parts of this value chain, whether it is for delivering an observation-based product, a numerical weather prediction (NWP) product, or a postprocessed product tailored for a specific end-user. Verification helps to ensure that new products are of higher quality than its predecessors and is a prerequisite for adoption by the community. Furthermore, verification results give users a better idea of a product’s strengths, weaknesses, and uncertainty.
Verification also plays a major role in the day-to-day work of product developers. Verification is used for diagnosing problems with existing products, understanding the behavior of algorithms being developed, and for making comparisons between multiple candidate algorithms. This highly iterative process requires extensive fine-tuning of algorithms to ensure that the product performs well under a wide range of weather situations and supports its end-users’ needs.
Verif is a simple yet flexible verification tool designed for effective product development. Verif was developed at the University of British Columbia in a research environment that delivers customized weather forecasts for government agencies and private companies. It has been further extended at the Norwegian Meteorological Institute (MET Norway), where it is used to refine operational weather forecasts for the popular weather app Yr (www.yr.no) and for hydrological and climatological products available on SeNorge (www.senorge.no). This development has resulted in several official releases, the latest of which is version 1.3 and which is presented here. Verif is suitable for a wide variety of deterministic and probabilistic applications, and is easy to install. Full documentation of the tool is available on the Verif website (https://github.com/WFRT/verif) or see the sidebar for how to get started.
We thank three anonymous reviewers for their helpful comments that improved this article. This work has been partly funded by the Canadian Natural Sciences and Engineering Research Council (NSERC) and British Columbia Hydro and Power Authority (BC Hydro). We thank former and current members of the UBC Weather Forecast Research Team for extensive testing and suggestions leading to improvements to Verif. We also acknowledge the helpful comments and suggestions we have received from Verif’s community of users.
Brady, R. X., and A. Spring, 2021: climpred: Verification of weather and climate forecasts. J. Open Source Software, 6, 2781, https://doi.org/10.21105/joss.02781.
Brown, B., and Coauthors, 2021: The Model Evaluation Tools (MET): More than a decade of community-supported forecast verification. Bull. Amer. Meteor. Soc., 102, E782–E807, https://doi.org/10.1175/BAMS-D-19-0093.1.
Ebert, E., and Coauthors, 2013: Progress and challenges in forecast verification. Meteor. Appl., 20, 130–139, https://doi.org/10.1002/met.1392.
Gilleland, E., 2022: Comparing spatial fields with SpatialVx: Spatial forecast verification in R. NCAR, 69 pp., https://doi.org/10.5065/4px3-5a05.
Gneiting, T., F. Balabdaoui, and A. E. Raftery, 2007: Probabilistic forecasts, calibration and sharpness. J. Roy. Stat. Soc., 69B, 243–268, https://doi.org/10.1111/j.1467-9868.2007.00587.x.
Hamill, T. M., and J. Juras, 2006: Measuring forecast skill: Is it real skill or is it the varying climatology? Quart. J. Roy. Meteor. Soc., 132, 2905–2923, https://doi.org/10.1256/qj.06.25.
Jolliffe, I. T., and D. B. Stephenson, Eds., 2012: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. 2nd ed. John Wiley and Sons, 296 pp.
Manubens, N., and Coauthors, 2018: An R package for climate forecast verification. Environ. Modell. Software, 103, 29–42, https://doi.org/10.1016/j.envsoft.2018.01.018.
Murphy, A. H., 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8, 281–293, https://doi.org/10.1175/1520-0434(1993)008<0281:WIAGFA>2.0.CO;2.
NCAR Research Application Program, 2015: Verification: Weather forecast verification utilities, version 1.42. R package, https://cran.r-project.org/package=verification.
Nipen, T. N., I. A. Seierstad, C. Lussana, J. Kristiansen, and Ø. Hov, 2020: Adopting citizen observations in operational weather prediction. Bull. Amer. Meteor. Soc., 101, E43–E57, https://doi.org/10.1175/BAMS-D-18-0237.1.
Taylor, K. E., 2001: Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res., 106, 7183–7192, https://doi.org/10.1029/2000JD900719.
Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. International Geophysics Series, Vol. 100, Academic Press, 676 pp.