Search Results

You are looking at 1 - 10 of 10 items for :

  • Model performance/evaluation x
  • The 1st NOAA Workshop on Leveraging AI in the Exploitation of Satellite Earth Observations & Numerical Weather Prediction x
  • Refine by Access: Content accessible to me x
Clear All
Kyle A. Hilburn, Imme Ebert-Uphoff, and Steven D. Miller

values. Performance is evaluated using metrics including the mean-square error (MSE), coefficient of determination R 2 , categorical metrics (probability of detection, false-alarm rate, critical success index, and categorical bias) at various output threshold levels, and evaluation of the root-mean-square difference (RMSD) binned over the range of true output values. A potential disadvantage of ML is that it is statistically based, making it harder to interpret. So, besides producing a trained and

Open access
Hanoi Medina, Di Tian, Fabio R. Marin, and Giovanni B. Chirico

global NWP models ( Bauer et al. 2015 ). The representation of these processes is especially challenging over continental areas from the Southern Hemisphere where the abundant vegetation and the sparse observations for evaluation and data assimilation have limited the models’ accuracy. Recent progress in forecasting tropical convection ( Bechtold et al. 2014 ; Subramanian et al. 2017 ) and the increasing quantity and quality of global information encourage the use of NWP for tropical precipitation

Full access
Ryan Lagerquist, Amy McGovern, Cameron R. Homeyer, David John Gagne II, and Travis Smith

are 1D with lower spatial resolution), which would present a major difficulty for non-ML-based postprocessing methods such as SSPF. The rest of this paper is organized as follows. Section 2 briefly describes the inner workings of CNNs [a more thorough description is provided in Lagerquist et al. (2019) , hereafter L19 ], section 3 describes the input data and preprocessing, section 4 describes experiments used to find the best CNNs, section 5 evaluates performance of the best CNNs, and

Free access
Imme Ebert-Uphoff and Kyle Hilburn

, by trying different sets of hyperparameters, training a complete model for each set, evaluating the resulting model, and then deciding which hyperparameter set results in best performance. Algorithms range from simple exhaustive grid search (as illustrated in the “Using performance measures for NN tuning” section) to sophisticated algorithms ( Kasim et al. 2020 ; Hertel et al. 2020 ). Sample application: Image-to-image translation from GOES to MRMS. We demonstrate many of the concepts in this

Full access
Anthony Wimmers, Christopher Velden, and Joshua H. Cossuth

. However, it is difficult to generalize this difference because of the small sample size for category 5. Overall, the improvement is enough to justify limiting the remaining model evaluation to only the two-channel version of DeepMicroNet going forward. Fig . 5. (a) Intensity error (RMSE) according to best track MSW for the three model versions labeled in the legend, and (b) average standard deviation of the PDFs according to best track MSW. b. Model performance The following describes a two

Full access
Ricardo Martins Campos, Vladimir Krasnopolsky, Jose-Henrique G. M. Alves, and Stephen G. Penny

total of 65 input variables x compose the n inputs for the ANN model. The outputs y consist of three variables only (Hs, Tp, and U10; m = 3 ) from the NDBC buoys, targeted by the model. Each ANN addresses one forecast time, with the focus of Campos et al. (2017) on the fifth day, which is approximately the time when ensemble forecasts start to have better performance than deterministic forecasts, according to Alves et al. (2013) . All variables were normalized to the interval between −1

Full access
Eric D. Loken, Adam J. Clark, Amy McGovern, Montgomery Flora, and Kent Knopfmeier

), probability of false detection (POFD), success ratio (SR), bias, and critical success index (CSI) can then be obtained [e.g., see Eqs. (3)–(7) in Loken et al. 2017 ]. These metrics form the basis of other forecast evaluation tools used herein, such as the ROC curve ( Mason 1982 ) and performance diagram ( Roebber 2009 ). ROC curves plot POD against POFD at multiple forecast probability thresholds (here, 1%, 2%, and 5%–95% in intervals of 5%). Area under the ROC curve (AUC) provides a measure of forecast

Full access
Sid-Ahmed Boukabara, Vladimir Krasnopolsky, Stephen G. Penny, Jebb Q. Stewart, Amy McGovern, David Hall, John E. Ten Hoeve, Jason Hickey, Hung-Lung Allen Huang, John K. Williams, Kayo Ide, Philippe Tissot, Sue Ellen Haupt, Kenneth S. Casey, Nikunj Oza, Alan J. Geer, Eric S. Maddy, and Ross N. Hoffman

( Krasnopolsky 2013 ), the most important being to achieve high performance within the host NWP model. Fast emulations of existing model physics parameterizations are usually developed for complex parameterizations that are computational bottlenecks, such as atmospheric radiation parameterizations and the planetary boundary layer (e.g., Wang et al. 2019 ). Krasnopolsky (2019) demonstrated that a 0.1 K day −1 RMS accuracy can be obtained for varied individual instantaneous profiles with shallow NN

Open access
Sid-Ahmed Boukabara, Vladimir Krasnopolsky, Jebb Q. Stewart, Eric S. Maddy, Narges Shahroudi, and Ross N. Hoffman

. Statistics for estimating the accuracy of the heating rate calculations (in K day −1 ) and the computational performance (speedup) of the ANN emulation vs the original parameterization for the NCAR CAM (T42L26) ( Collins et al. 2002 ) and for the NCEP CFS (T126L64) longwave radiation (LWR) and shortwave radiation (SWR) parameterizations. RRTMG is the Rapid Radiative Transfer Model for GCMs ( Clough et al. 2005 ). Here, the speedup shows an averaged (over an independent global dataset) ratio of the timing

Free access
Amy McGovern, Ryan Lagerquist, David John Gagne II, G. Eli Jergensen, Kimberly L. Elmore, Cameron R. Homeyer, and Travis Smith

soundings from a mesoscale numerical model to predict winter precipitation type. At the storm scale, we use DL to predict the probability that a storm will produce a tornado within the next hour and traditional ML methods to classify a storm’s convective mode. Lagerquist et al. (2018 , 2019a) , Gagne et al. (2019) , McGovern et al. (2018), and Jergensen et al. (2019) focus on the training and evaluation of these models, while we focus on MIV. MACHINE LEARNING. We briefly review ML as needed for the

Full access