• Ben Bouallègue, Z. B., , and S. E. Theis, 2014: Spatial techniques applied to precipitation ensemble forecasts: From verification results to probabilistic products. Meteor. Appl., 21, 922929, doi:10.1002/met.1435.

    • Search Google Scholar
    • Export Citation
  • Casati, B., , G. Ross, , and D. B. Stephenson, 2004: A new intensity-scale approach for the verification of spatial precipitation forecasts. Meteor. Appl., 11, 141154, doi:10.1017/S1350482704001239.

    • Search Google Scholar
    • Export Citation
  • Chen, S., and et al. , 2003: COAMPS version 3 model description—General theory and equations. NRL Tech Note NRL/PU/7500-03-448, 148 pp. [Available online at http://www.nrlmry.navy.mil/coamps_docs/base/docs/COAMPS_2003.pdf.]

  • Daley, R., , and E. Barker, 2001: NAVDAS: Formulation and diagnostics. Mon. Wea. Rev., 129, 869883, doi:10.1175/1520-0493(2001)129<0869:NFAD>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Davies, H. C., 1976: A lateral boundary formulation for multi-level prediction models. Quart. J. Roy. Meteor. Soc., 102, 405418, doi:10.1002/qj.49710243210.

    • Search Google Scholar
    • Export Citation
  • Davis, C., , B. Brown, , and R. Bullock, 2006: Object-based verification of precipitation forecasts. Part I: Methods and application to mesoscale rain areas. Mon. Wea. Rev., 134, 17721784, doi:10.1175/MWR3145.1.

    • Search Google Scholar
    • Export Citation
  • Duc, L., , K. Saito, , and H. Seko, 2013: Spatial-temporal fractions verification for high-resolution ensemble forecasts. Tellus, 65A, 18171, doi:10.3402/tellusa.v65i0.18171.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., 2008: Fuzzy verification of high resolution gridded forecasts: A review and proposed framework. Meteor. Appl., 15, 5164, doi:10.1002/met.25.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., , and J. L. McBride, 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239, 179202, doi:10.1016/S0022-1694(00)00343-7.

    • Search Google Scholar
    • Export Citation
  • Hodur, R. M., 1997: The Naval Research Laboratory’s Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS). Mon. Wea. Rev., 125, 14141430, doi:10.1175/1520-0493(1997)125<1414:TNRLSC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hogan, T. F., , M. S. Peng, , J. A. Ridout, , and W. M. Clune, 2002: A description of the impact of changes to NOGAPS convection parameterization and the increase in resolution to T239L30. Naval Research Laboratory Memo. Rep. NRL/MR/7530-02-52, 10 pp.

  • Hsu, W.-R., , and A. H. Murphy, 1986: The attributes diagram: A geometrical framework for assessing the quality of probability forecasts. Int. J. Forecasting, 2, 285293, doi:10.1016/0169-2070(86)90048-8.

    • Search Google Scholar
    • Export Citation
  • Lockhoff, M., , O. Zolina, , C. Simmer, , and J. Schulz, 2014: Evaluation of satellite-retrieved extreme precipitation over europe using gauge observations. J. Climate, 27, 607623, doi:10.1175/JCLI-D-13-00194.1.

    • Search Google Scholar
    • Export Citation
  • Mason, S. J., 2004: On using “climatology” as a reference strategy in the brier and ranked probability skill scores. Mon. Wea. Rev., 132, 18911895, doi:10.1175/1520-0493(2004)132<1891:OUCAAR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Mittermaier, M., , and N. Roberts, 2010: Intercomparison of spatial forecast verification methods: Identifying skillful spatial scales using the fractions skill score. Wea. Forecasting, 25, 343354, doi:10.1175/2009WAF2222260.1.

    • Search Google Scholar
    • Export Citation
  • Mittermaier, M., , N. Roberts, , and S. A. Thompson, 2013: A long-term assessment of precipitation forecast skill using the Fractions Skill Score. Meteor. Appl., 20, 176186, doi:10.1002/met.296.

    • Search Google Scholar
    • Export Citation
  • Nachamkin, J. E., 2004: Mesoscale verification using meteorological composites. Mon. Wea. Rev., 132, 941955, doi:10.1175/1520-0493(2004)132<0941:MVUMC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Nachamkin, J. E., , J. Schmidt, , and C. Mitrescu, 2009: Verification of cloud forecasts over the eastern Pacific using passive satellite retrievals. Mon. Wea. Rev., 137, 34853500, doi:10.1175/2009MWR2853.1.

    • Search Google Scholar
    • Export Citation
  • Roberts, N. M., 2005: An investigation of the ability of a storm scale configuration of the Met Office NWP model to predict flood producing rainfall. Met Office Tech. Rep. 455, Joint Centre for Mesoscale Meteorology Rep. 150, 80 pp. [Available online at http://research.metoffice.gov.uk/research/nwp/publications/papers/technical_reports/2005/FRTR455/FRTR455.pdf.]

  • Roberts, N. M., , and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 7897, doi:10.1175/2007MWR2123.1.

    • Search Google Scholar
    • Export Citation
  • Romine, G. S., , C. S. Schwartz, , C. Snyder, , J. L. Anderson, , and M. L. Weisman, 2013: Model bias in a continuously cycled assimilation system and its influence on convection-permitting forecasts. Mon. Wea. Rev., 141, 12631284, doi:10.1175/MWR-D-12-00112.1.

    • Search Google Scholar
    • Export Citation
  • Rutledge, S. A., , and P. V. Hobbs, 1983: The mesoscale and microscale structure and organization of clouds and precipitation in midlatitude cyclones. VIII: A model for the “seeder-feeder” process in warm-frontal rainbands. J. Atmos. Sci., 40, 11851206, doi:10.1175/1520-0469(1983)040<1185:TMAMSA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Rutledge, S. A., , and P. V. Hobbs, 1984: The mesoscale and microscale structure and organization of clouds and precipitation in midlatitude cyclones. XII: A diagnostic modeling study of precipitation development in narrow cold-frontal rainbands. J. Atmos. Sci., 41, 29492972, doi:10.1175/1520-0469(1984)041<2949:TMAMSA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Schmidt, J. M., 2001: Moist physics development for the Naval Research Laboratory’s Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS). BACIMO, CD-ROM.

  • Schwartz, C. S., and et al. , 2010: Toward improved convection-allowing ensembles: Model physics sensitivities and optimizing probabilistic guidance with small ensemble membership. Wea. Forecasting, 25, 263280, doi:10.1175/2009WAF2222267.1.

    • Search Google Scholar
    • Export Citation
  • Söhne, N., , J.-P. Chaboureau, , and F. Guichard, 2008: Verification of cloud cover forecast with satellite observation over West Africa. Mon. Wea. Rev., 136, 44214434, doi:10.1175/2008MWR2432.1.

    • Search Google Scholar
    • Export Citation
  • Stratman, D. R., , M. C. Coniglio, , S. E. Koch, , and M. Xue, 2013: Use of multiple verification methods to evaluate forecasts of convection from hot- and cold-start convection-allowing models. Wea. Forecasting, 28, 119138, doi:10.1175/WAF-D-12-00022.1.

    • Search Google Scholar
    • Export Citation
  • Theis, S. E., , A. Hense, , and U. Damrath, 2005: Probabilistic precipitation forecasts from a deterministic model: A pragmatic approach. Meteor. Appl., 12, 257268, doi:10.1017/S1350482705001763.

    • Search Google Scholar
    • Export Citation
  • Weusthoff, T., , F. Ament, , M. Arpagaus, , and M. W. Rotach, 2010: Assessing the benefits of convection-permitting models by neighborhood verification: Examples from MAP D-PHASE. Mon. Wea. Rev., 138, 34183433, doi:10.1175/2010MWR3380.1.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Elsevier, 627 pp.

  • Zacharov, P., , and D. Rezacova, 2009: Using the fractions skill score to assess the relationship between an ensemble QPF spread and skill. Atmos. Res., 94, 684693, doi:10.1016/j.atmosres.2009.03.004.

    • Search Google Scholar
    • Export Citation
  • Zepeda-Arce, J., , E. Foufoula-Georgiou, , and K. K. Droegemeier, 2000: Space-time rainfall organization and its role in validating quantitative precipitation forecasts. J. Geophys. Res., 105, 10 12910 146, doi:10.1029/1999JD901087.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 84 84 6
PDF Downloads 64 64 6

Applying a Neighborhood Fractions Sampling Approach as a Diagnostic Tool

View More View Less
  • 1 Naval Research Laboratory, Monterey, California
© Get Permissions
Restricted access

Abstract

The fractions skill score (FSS) belongs to a class of spatial neighborhood techniques that measures forecast skill from samples of gridded forecasts and observations at increasing spatial scales. Each sample contains the fraction of the predicted and observed quantities that exist above a threshold value. Skill is gauged by the rate that the observed and predicted fractions converge with increasing scale. In this study, neighborhood sampling is applied to diagnose the performance of high-resolution (1.67 km) precipitation forecasts over central Florida. Reliability diagrams derived from the spatial fractions indicate that the FSS can be influenced by small, low-predictability events. Further tests indicate the FSS is subtly affected by samples from points on and near the grid boundaries. Inclusion of these points tends to reduce the magnitude and sensitivity of the FSS, especially at large scales. An attempt to mine data from the set of neighborhood fractions was moderately successful at obtaining descriptive information about the precipitation fields. The width of the distribution of the fractions at each scale provided information concerning forecast resolution and sharpness. The rate at which the distribution of the fractions converged toward the domain mean with increasing scale was found to be sensitive to the uniformity of coverage of precipitation through the domain. Generally, the 6-h forecasts possessed greater spatial skill than those at 12 h. High-FSS values at 12 h were mostly associated with evenly distributed precipitation patterns, while the 6-h forecasts also performed well for several nonuniform cases.

Denotes Open Access content.

Corresponding author address: Jason E. Nachamkin, Naval Research Laboratory, 7 Grace Hopper Ave., Monterey, CA 93943. E-mail: jason.nachamkin@nrlmry.navy.mil

Abstract

The fractions skill score (FSS) belongs to a class of spatial neighborhood techniques that measures forecast skill from samples of gridded forecasts and observations at increasing spatial scales. Each sample contains the fraction of the predicted and observed quantities that exist above a threshold value. Skill is gauged by the rate that the observed and predicted fractions converge with increasing scale. In this study, neighborhood sampling is applied to diagnose the performance of high-resolution (1.67 km) precipitation forecasts over central Florida. Reliability diagrams derived from the spatial fractions indicate that the FSS can be influenced by small, low-predictability events. Further tests indicate the FSS is subtly affected by samples from points on and near the grid boundaries. Inclusion of these points tends to reduce the magnitude and sensitivity of the FSS, especially at large scales. An attempt to mine data from the set of neighborhood fractions was moderately successful at obtaining descriptive information about the precipitation fields. The width of the distribution of the fractions at each scale provided information concerning forecast resolution and sharpness. The rate at which the distribution of the fractions converged toward the domain mean with increasing scale was found to be sensitive to the uniformity of coverage of precipitation through the domain. Generally, the 6-h forecasts possessed greater spatial skill than those at 12 h. High-FSS values at 12 h were mostly associated with evenly distributed precipitation patterns, while the 6-h forecasts also performed well for several nonuniform cases.

Denotes Open Access content.

Corresponding author address: Jason E. Nachamkin, Naval Research Laboratory, 7 Grace Hopper Ave., Monterey, CA 93943. E-mail: jason.nachamkin@nrlmry.navy.mil
Save