The code was also subject to this error,2 and hence the original Table 5 and Fig. 7 are recalculated using the correct formula. Corrected versions are displayed here in Table 5 and Fig. 7. The conclusions drawn in Schulz and Lerch (2022) still hold, but the numbers stated in one paragraph need to be updated. The third paragraph of section 4d should now read as follows:
Ratio of lead time–station combinations (%) where pairwise DM tests indicate statistically significant CRPS differences after applying a Benjamini–Hochberg procedure to account for multiple testing for a nominal level of α = 0.05 of the corresponding one-sided tests. The (i, j) entry in the ith row and jth column indicates the ratio of cases where the null hypothesis of equal predictive performance of the corresponding one-sided DM test is rejected in favor of the model in the ith row when compared with the model in the jth column. The remainder of the sum of (i, j) and (j, i) entry to 100% is the ratio of cases for which the score differences are not significant.
Best method at each station in terms of the CRPS, averaged over all lead times. The point sizes indicate the level of statistical significance of the observed CRPS differences relative to the methods only from the other groups of methods for all lead times. Three different point sizes are possible, with the smallest size indicating statistically significant differences for at most 90% of the performed tests, the middle size is for up to 99%, and the largest is to 100%, meaning all differences are statistically significant.
Citation: Monthly Weather Review 151, 5; 10.1175/MWR-D-23-0010.1
We find that the observed score differences are statistically significant for a high ratio of stations and lead times; see Table 5. In particular, DRN and BQN significantly outperform the basic models at more than 94%, and even significantly outperform QRF and EMOS-GB at more than 50% of all combinations of stations and lead times. Among the locally estimated methods, QRF performs best but only provides significant improvements over the NN-based methods for around 1% of the cases.
That is, the three percentages have been reduced from 97%, 80%, and 5% to 94%, 50%, and 1%, respectively.
In general, the ratio of significant tests in Table 5 decreases. Especially the comparison of the network methods DRN, BQN, and HEN are affected, where DRN and BQN now perform significantly better than the other method at less than 2% instead of at ∼44%–46%. For HEN, QRF, and EMOS-GB, the ratio of significant tests for the superiority of DRN and BQN also decreases by ∼25–40 percentage points.
In comparing the original Fig. 7 in Schulz and Lerch (2022) with the revised Fig. 7, it is seen that the size of symbols decreases because of the decreased ratio of significant tests. Note that the best-performing method at each location is not affected and does not change.
We thank Ron McTaggart-Cowan, Michael Scheuerer, and Alexander Jordan for constructive discussions on the different variants of the DM test.
In addition to this Corrigendum, we updated the code on the corresponding GitHub repository (https://github.com/benediktschulz/paper_pp_wind_gusts).
REFERENCES
Diebold, F. X., and R. S. Mariano, 1995: Comparing predictive accuracy. J. Bus. Econ. Stat., 13, 253–263, https://doi.org/10.1080/07350015.1995.10524599.
Gneiting, T., and M. Katzfuss, 2014: Probabilistic forecasting. Annu. Rev. Stat. Appl., 1, 125–151, https://doi.org/10.1146/annurev-statistics-062713-085831.
Jordan, A., 2016: Facets of forecast evaluation. Ph.D. dissertation, Karlsruhe Institute of Technology, 112 pp., https://doi.org/10.5445/IR/1000063629.
Schulz, B., and S. Lerch, 2022: Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison. Mon. Wea. Rev., 150, 235–257, https://doi.org/10.1175/MWR-D-21-0150.1.
