• Casati, B., 2010: New developments of the intensity-scale technique within the Spatial Verification Methods Intercomparison Project. Wea. Forecasting, 25, 113143, doi:10.1175/2009WAF2222257.1.

    • Search Google Scholar
    • Export Citation
  • Casati, B., G. Ross, and D. Stephenson, 2004: A new intensity-scale approach for the verification of spatial precipitation forecasts. Meteor. Appl., 11, 141154, doi:10.1017/S1350482704001239.

    • Search Google Scholar
    • Export Citation
  • Casati, B., and Coauthors, 2008: Forecast verification: Current status and future directions. Meteor. Appl., 15, 318, doi:10.1002/met.52.

    • Search Google Scholar
    • Export Citation
  • Crewell, S., and Coauthors, 2008: The general observation period 2007 within the priority program on quantitative precipitation forecasting: Concept and first results. Meteor. Z., 17, 849866, doi:10.1127/0941-2948/2008/0336.

    • Search Google Scholar
    • Export Citation
  • Crocker, R., and M. Mittermaier, 2013: Exploratory use of a satellite cloud mask to verify NWP models. Meteor. Appl., 20, 197205, doi:10.1002/met.1384.

    • Search Google Scholar
    • Export Citation
  • Davis, C., B. Brown, and R. Bullock, 2006a: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134, 17721784, doi:10.1175/MWR3145.1.

    • Search Google Scholar
    • Export Citation
  • Davis, C., B. Brown, and R. Bullock, 2006b: Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Wea. Rev., 134, 17851795, doi:10.1175/MWR3146.1.

    • Search Google Scholar
    • Export Citation
  • Derrien, M., and H. Le Gléau, 2005: MSG/SEVIRI cloud mask and type from SAFNWC. Int. J. Remote Sens., 26, 47074732, doi:10.1080/01431160500166128.

    • Search Google Scholar
    • Export Citation
  • Derrien, M., and H. Le Gléau, 2010: Improvement of cloud detection near sunrise and sunset by temporal-differencing and region-growing techniques with real-time SEVIRI. Int. J. Remote Sens., 31, 17651780, doi:10.1080/01431160902926632.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., 2008: Fuzzy verification of high-resolution gridded forecasts: a review and proposed framework. Meteor. Appl., 15, 5164, doi:10.1002/met.25.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and J. McBride, 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239, 179202, doi:10.1016/S0022-1694(00)00343-7.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and W. A. Gallus Jr., 2009: Toward better understanding of the contiguous rain area (CRA) method for spatial forecast verification. Wea. Forecasting, 24, 14011415, doi:10.1175/2009WAF2222252.1.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and Coauthors, 2013: Progress and challenges in forecast verification. Meteor. Appl., 20, 130139, doi:10.1002/met.1392.

    • Search Google Scholar
    • Export Citation
  • Eggert, B., P. Berg, J. Haerter, D. Jacob, and C. Moseley, 2015: Temporal and spatial scaling impacts on extreme precipitation. Atmos. Chem. Phys., 15, 59575971, doi:10.5194/acp-15-5957-2015.

    • Search Google Scholar
    • Export Citation
  • EUMETSAT, 2012a: Effective radiances and brightness temperature relation tables for Meteosat Second Generation. EUM/OPS-MSG/TEN/08/0024, 630 pp. [Available online at http://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=PDF_TEN_080024_RAD_BRIGHT_TEMP&RevisionSelectionMethod=LatestReleased&Rendition=Web.]

  • EUMETSAT, 2012b: The conversion from effective radiances to equivalent brightness temperatures. EUM/MET/TEN/11/0569, 49 pp. [Available online at https://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=PDF_EFFECT_RAD_TO_BRIGHTNESS&RevisionSelectionMethod=LatestReleased&Rendition=Web.]

  • Evaristo, R., X. Xie, S. Troemel, M. Diederich, J. Simon, and C. Simmer, 2014: A macrophysical life cycle description for precipitating systems. Geophys. Res. Abstr., 16, Abstract EGU2014-10322. [Available online at http://meetingorganizer.copernicus.org/EGU2014/EGU2014-10322.pdf.]

    • Search Google Scholar
    • Export Citation
  • Früh, B., J. Bendix, T. Nauss, M. Paulat, A. Pfeiffer, J. W. Schipper, B. Thies, and H. Wernli, 2007: Verification of precipitation from regional climate simulations and remote-sensing observations with respect to ground-based observations in the upper Danube catchment. Meteor. Z., 16, 275293, doi:10.1127/0941-2948/2007/0210.

    • Search Google Scholar
    • Export Citation
  • Gilleland, E., 2014: SpatialVx: Spatial forecast verification, version 0.2-0 R package. [Available online at http://CRAN.R-project.org/package=SpatialVx.]

  • Gilleland, E., D. Ahijevych, B. G. Brown, B. Casati, and E. E. Ebert, 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24, 14161430, doi:10.1175/2009waf2222269.1.

    • Search Google Scholar
    • Export Citation
  • Good, P., 2000: Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses. Springer, 271 pp.

  • Hammann, E., A. Behrendt, F. Le Mounier, and V. Wulfmeyer, 2015: Temperature profiling of the atmospheric boundary layer with rotational Raman lidar during the HD(CP)2 Observational Prototype Experiment. Atmos. Chem. Phys., 15, 28672881, doi:10.5194/acp-15-2867-2015.

    • Search Google Scholar
    • Export Citation
  • Keil, C., A. Tafferner, and T. Reinhardt, 2006: Synthetic satellite imagery in the Lokal-Modell. Atmos. Res., 82, 1925, doi:10.1016/j.atmosres.2005.01.008.

    • Search Google Scholar
    • Export Citation
  • Kolmogorov, A. N., 1933: Sulla determinazione empirica di una legge di distribuzione (On the empirical determination of a law of distribution). G. Ist. Ital. Attuari, 4, 8391.

    • Search Google Scholar
    • Export Citation
  • Leoncini, G., R. Plant, S. Gray, and P. Clark, 2013: Ensemble forecasts of a flood-producing storm: Comparison of the influence of model-state perturbations and parameter modifications. Quart. J. Roy. Meteor. Soc., 139, 198211, doi:10.1002/qj.1951.

    • Search Google Scholar
    • Export Citation
  • Li, J., and A. D. Heap, 2008: A Review of Spatial Interpolation Methods for Environmental Scientists. Australia Geoscience, 137 pp.

  • Mason, D. M., and J. H. Schuenemeyer, 1983: A modified Kolmogorov–Smirnov test sensitive to tail alternatives. Ann. Stat., 11, 933946, doi:10.1214/aos/1176346259.

    • Search Google Scholar
    • Export Citation
  • Nachamkin, J. E., 2009: Application of the composite method to the spatial forecast verification methods intercomparison dataset. Wea. Forecasting, 24, 13901400, doi:10.1175/2009WAF2222225.1.

    • Search Google Scholar
    • Export Citation
  • Nam, C. C. W., J. Quaas, R. Neggers, C. Siegenthaler-Le Drian, and F. Isotta, 2014: Evaluation of boundary layer cloud parameterizations in the ECHAM5 general circulation model using CALIPSO and CloudSat satellite data. J. Adv. Model. Earth Syst., 6, 300314, doi:10.1002/2013MS000277.

    • Search Google Scholar
    • Export Citation
  • Reuter, M., W. Thomas, P. Albert, M. Lockhoff, R. Weber, K. Karlsson, and J. Fischer, 2009: The CM-SAF and FUB cloud detection schemes for SEVIRI: Validation with synoptic data and initial comparison with MODIS and CALIPSO. J. Appl. Meteor. Climatol., 48, 301316, doi:10.1175/2008JAMC1982.1.

    • Search Google Scholar
    • Export Citation
  • Roberts, N. M., and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 7897, doi:10.1175/2007MWR2123.1.

    • Search Google Scholar
    • Export Citation
  • Schättler, U., G. Doms, and C. Schraf, 2013: A description of the nonhydrostatic regional COSMO-Model. Part VII: User’s guide. Consortium for Small-Scale Modelling, 200 pp. [Available online at http://www2.cosmo-model.org/content/model/documentation/core/cosmoUserGuide.pdf.]

  • Shi, X., J. Liu, Y. Li, H. Tian, and X. Liu, 2014: Improved SAL method and its application to verifying regional soil moisture forecasting. Sci. China Earth Sci., 57, 26572670, doi:10.1007/s11430-014-4901-9.

    • Search Google Scholar
    • Export Citation
  • Smirnov, N. V., 1939: On the estimation of the discrepancy between empirical curves of distribution for two independent samples. Bull. Math. Univ. Moscou, 2, 314.

    • Search Google Scholar
    • Export Citation
  • Sommeria, G., and J. Deardorff, 1977: Subgrid-scale condensation in models of nonprecipitating clouds. J. Atmos. Sci., 34, 344355, doi:10.1175/1520-0469(1977)034<0344:SSCIMO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Steinke, S., S. Eikenberg, U. Löhnert, G. Dick, D. Klocke, P. Di Girolamo, and S. Crewell, 2015: Assessment of small-scale integrated water vapour variability during HOPE. Atmos. Chem. Phys., 15, 26752692, doi:10.5194/acp-15-2675-2015.

    • Search Google Scholar
    • Export Citation
  • Wernli, H., M. Paulat, M. Hagen, and C. Frei, 2008: SAL—A novel quality measure for the verification of quantitative precipitation forecasts. Mon. Wea. Rev., 136, 44704487, doi:10.1175/2008MWR2415.1.

    • Search Google Scholar
    • Export Citation
  • Wernli, H., C. Hofmann, and M. Zimmer, 2009: Spatial forecast verification methods intercomparison project: Application of the SAL technique. Wea. Forecasting, 24, 14721484, doi:10.1175/2009WAF2222271.1.

    • Search Google Scholar
    • Export Citation
  • Zacharov, P., D. Rezacova, and R. Brozkova, 2013: Evaluation of the QPF of convective flash flood rainfalls over the Czech territory in 2009. Atmos. Res., 131, 95107, doi:10.1016/j.atmosres.2013.03.007.

    • Search Google Scholar
    • Export Citation
  • Zimmer, M., H. Wernli, C. Frei, and M. Hagen, 2009: Feature-based verification of deterministic precipitation forecasts with SAL during COPS. Proc. MAP D-PHASE Scientific Meeting, Bologna, Italy, Institute of Atmospheric Sciences and Climate and ARPA-SIM, 116–121. [Available online at http://www.smr.arpa.emr.it/dphase-cost/master_proceeding_final.pdf.]

  • Zimmer, M., G. Craig, C. Keil, and H. Wernli, 2011: Classification of precipitation events with a convective response timescale and their forecasting characteristics. Geophys. Res. Lett., 38, L05802, doi:10.1029/2010GL046199.

    • Search Google Scholar
    • Export Citation
  • Zinner, T., L. Bugliaro, and B. Mayer, 2005: Remote sensing of inhomogeneous clouds with MSG/SEVIRI. Proc. EUMETSAT Meteorological Satellite Conf., Dubrovnik, Croatia, EUMETSAT. [Available online at http://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=PDF_CONF_P46_S6_01_ZINNER_V&RevisionSelectionMethod=LatestReleased&Rendition=Web.]

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 200 140 6
PDF Downloads 98 77 4

Using the SAL Technique for Spatial Verification of Cloud Processes: A Sensitivity Analysis

View More View Less
  • 1 Meteorological Institute, University of Bonn, Bonn, Germany
© Get Permissions Rent on DeepDyve
Restricted access

Abstract

The feature-based spatial verification method named for its three score components: structure, amplitude, and location (SAL) is applied to cloud data, that is, two-dimensional spatial fields of total cloud cover and spectral radiance. Model output is obtained from the German-focused Consortium for Small-Scale Modeling (COSMO-DE) forward operator Synthetic Satellite Simulator (SynSat) and compared with SEVIRI satellite data. The aim of this study is twofold: first, to assess the applicability of SAL to this kind of data and, second, to analyze the role of external object identification algorithms (OIA) and the effects of observational uncertainties on the resulting scores. A comparison of three different OIA shows that the threshold level, which is a fundamental part of all studied algorithms, induces high sensitivity and unstable behavior of object-dependent SAL scores (i.e., even very small changes in parameter values lead to large changes in the resulting scores). An in-depth statistical analysis reveals significant effects on distributional quantities commonly used in the interpretation of SAL, for example, median and interquartile distance. Two sensitivity indicators that are based on the univariate cumulative distribution functions are derived. They make it possible to assess the sensitivity of the SAL scores to threshold-level changes without computationally expensive iterative calculations of SAL for various thresholds. The mathematical structure of these indicators connects the sensitivity of the SAL scores to parameter changes with the effect of observational uncertainties. Last, the discriminating power of SAL is studied. It is shown that—for large-scale cloud data—changes in the parameters may have larger effects on the object-dependent SAL scores (i.e., the S and L2 scores) than does a complete loss of temporal collocation.

Current affiliation: Meteorological Institute, University of Bonn, Germany.

Corresponding author address: Michael Weniger, Meteorological Institute, University of Bonn, Auf dem Hügel 20, 53121 Bonn, Germany. E-mail: mweniger@uni-bonn.de

Abstract

The feature-based spatial verification method named for its three score components: structure, amplitude, and location (SAL) is applied to cloud data, that is, two-dimensional spatial fields of total cloud cover and spectral radiance. Model output is obtained from the German-focused Consortium for Small-Scale Modeling (COSMO-DE) forward operator Synthetic Satellite Simulator (SynSat) and compared with SEVIRI satellite data. The aim of this study is twofold: first, to assess the applicability of SAL to this kind of data and, second, to analyze the role of external object identification algorithms (OIA) and the effects of observational uncertainties on the resulting scores. A comparison of three different OIA shows that the threshold level, which is a fundamental part of all studied algorithms, induces high sensitivity and unstable behavior of object-dependent SAL scores (i.e., even very small changes in parameter values lead to large changes in the resulting scores). An in-depth statistical analysis reveals significant effects on distributional quantities commonly used in the interpretation of SAL, for example, median and interquartile distance. Two sensitivity indicators that are based on the univariate cumulative distribution functions are derived. They make it possible to assess the sensitivity of the SAL scores to threshold-level changes without computationally expensive iterative calculations of SAL for various thresholds. The mathematical structure of these indicators connects the sensitivity of the SAL scores to parameter changes with the effect of observational uncertainties. Last, the discriminating power of SAL is studied. It is shown that—for large-scale cloud data—changes in the parameters may have larger effects on the object-dependent SAL scores (i.e., the S and L2 scores) than does a complete loss of temporal collocation.

Current affiliation: Meteorological Institute, University of Bonn, Germany.

Corresponding author address: Michael Weniger, Meteorological Institute, University of Bonn, Auf dem Hügel 20, 53121 Bonn, Germany. E-mail: mweniger@uni-bonn.de
Save